2024-04-21T08:28:54.365 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-21T08:28:54.365 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi --owner scheduled_leonidus@teuthology --verbose --description fs/workload/{0-ubuntu_22.04 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} --archive /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526 -- /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526/orig.config.yaml 2024-04-21T08:28:54.427 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-21T08:28:54.531 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526 branch: wip-lusov-quiescer description: fs/workload/{0-ubuntu_22.04 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: null first_in_suite: false job_id: '7666526' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 4f6bc5273643aaf5660909e85400647a8951c10b owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7599 sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e sleep_before_teardown: 0 subset: 111/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa suite_relpath: qa suite_repo: https://github.com/leonid-s-usov/ceph-ceph suite_sha1: 4f6bc5273643aaf5660909e85400647a8951c10b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCpZN/cVekf/8bChBUe7YM06GyrOTAkrKgTpxP870P/PGnZY5GZVM43heKzMP6yZroQpndv8oCerPEUCSuDaMqU= smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKfndWoCu9JAh2dDQweXrGZUyUspjAOLuZmnJ2dw6UQXmkAokkcP+U9f4I6USO/+i9x6Fy9D1tn3JbnDwZWJ4a4= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLlnjZa4dD40YjtP5Xwt4EoGQpoz546iQ37BU1BCF3FLyZ22Wn56euhAG87DiXfrt5ZpJrmZgcosN6A8ZRUw1hg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-21_08:12:55 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-21T08:28:54.531 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa; will attempt to use it 2024-04-21T08:28:54.532 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks 2024-04-21T08:28:54.532 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-21T08:28:54.533 INFO:teuthology.task.internal:Checking packages... 2024-04-21T08:28:54.559 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a630765c1f93c6e61d9b40c4b76437611d5ef29e' 2024-04-21T08:28:54.559 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-21T08:28:54.559 INFO:teuthology.packaging:ref: None 2024-04-21T08:28:54.559 INFO:teuthology.packaging:tag: None 2024-04-21T08:28:54.560 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-21T08:28:54.560 INFO:teuthology.packaging:sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:28:54.560 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-21T08:28:54.792 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3154.ga630765c 2024-04-21T08:28:54.793 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-21T08:28:54.800 INFO:teuthology.task.internal:no buildpackages task found 2024-04-21T08:28:54.800 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-21T08:28:54.807 INFO:teuthology.task.internal:Saving configuration 2024-04-21T08:28:54.829 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-21T08:28:54.835 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T08:28:54.862 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:23:30.286312', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCpZN/cVekf/8bChBUe7YM06GyrOTAkrKgTpxP870P/PGnZY5GZVM43heKzMP6yZroQpndv8oCerPEUCSuDaMqU='} 2024-04-21T08:28:54.884 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:23:30.288418', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKfndWoCu9JAh2dDQweXrGZUyUspjAOLuZmnJ2dw6UQXmkAokkcP+U9f4I6USO/+i9x6Fy9D1tn3JbnDwZWJ4a4='} 2024-04-21T08:28:54.907 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:23:30.287465', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLlnjZa4dD40YjtP5Xwt4EoGQpoz546iQ37BU1BCF3FLyZ22Wn56euhAG87DiXfrt5ZpJrmZgcosN6A8ZRUw1hg='} 2024-04-21T08:28:54.907 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-21T08:28:54.914 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-21T08:28:54.914 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-21T08:28:54.914 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-21T08:28:54.914 INFO:teuthology.run_tasks:Running task console_log... 2024-04-21T08:28:55.023 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f02d407d0d0>, signals=[15]) 2024-04-21T08:28:55.023 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-21T08:28:55.029 INFO:teuthology.task.internal:Opening connections... 2024-04-21T08:28:55.029 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-04-21T08:28:55.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:28:55.107 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2024-04-21T08:28:55.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:28:55.181 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-04-21T08:28:55.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:28:55.277 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-21T08:28:55.284 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-04-21T08:28:55.302 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-04-21T08:28:55.303 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-04-21T08:28:55.360 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2024-04-21T08:28:55.360 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="9" 2024-04-21T08:28:55.360 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2024-04-21T08:28:55.360 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2024-04-21T08:28:55.360 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="9" 2024-04-21T08:28:55.360 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el9" 2024-04-21T08:28:55.360 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T08:28:55.360 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-04-21T08:28:55.361 INFO:teuthology.orchestra.run.smithi082.stdout:LOGO="fedora-logo-icon" 2024-04-21T08:28:55.361 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T08:28:55.361 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2024-04-21T08:28:55.361 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T08:28:55.361 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T08:28:55.361 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T08:28:55.361 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-04-21T08:28:55.384 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2024-04-21T08:28:55.403 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2024-04-21T08:28:55.403 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-04-21T08:28:55.459 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Stream" 2024-04-21T08:28:55.460 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="9" 2024-04-21T08:28:55.460 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2024-04-21T08:28:55.460 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2024-04-21T08:28:55.460 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="9" 2024-04-21T08:28:55.460 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el9" 2024-04-21T08:28:55.460 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T08:28:55.460 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2024-04-21T08:28:55.460 INFO:teuthology.orchestra.run.smithi106.stdout:LOGO="fedora-logo-icon" 2024-04-21T08:28:55.460 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T08:28:55.460 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/" 2024-04-21T08:28:55.460 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T08:28:55.461 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T08:28:55.461 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T08:28:55.461 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-04-21T08:28:55.484 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-04-21T08:28:55.501 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-04-21T08:28:55.502 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-04-21T08:28:55.559 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-04-21T08:28:55.559 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-04-21T08:28:55.559 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-04-21T08:28:55.559 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-04-21T08:28:55.560 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-04-21T08:28:55.560 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-04-21T08:28:55.560 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T08:28:55.560 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-04-21T08:28:55.560 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-04-21T08:28:55.560 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T08:28:55.560 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-04-21T08:28:55.560 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T08:28:55.560 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T08:28:55.560 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T08:28:55.560 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-04-21T08:28:55.583 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-21T08:28:55.593 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-21T08:28:55.599 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-21T08:28:55.599 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-04-21T08:28:55.602 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2024-04-21T08:28:55.604 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-04-21T08:28:55.620 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-21T08:28:55.626 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-21T08:28:55.626 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-04-21T08:28:55.659 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2024-04-21T08:28:55.663 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-04-21T08:28:55.765 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-21T08:28:55.841 INFO:teuthology.run_tasks:Running task kernel... 2024-04-21T08:28:55.855 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-21T08:28:55.855 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-21T08:28:55.856 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-21T08:28:55.856 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T08:28:55.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:28:55.873 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-04-21T08:28:55.929 INFO:teuthology.orchestra.run.smithi082.stdout:5.14.0-437.el9.x86_64 2024-04-21T08:28:55.929 INFO:teuthology.task.kernel:Running kernel on smithi082: 5.14.0-437.el9.x86_64 2024-04-21T08:28:55.929 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum install -y kernel 2024-04-21T08:28:58.440 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-21T08:29:03.148 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 19 MB 00:03 2024-04-21T08:29:07.382 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-21T08:29:09.074 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-04-21T08:29:10.353 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-21T08:29:15.408 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-21T08:29:17.314 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-21T08:29:17.314 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-21T08:29:17.395 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-04-21T08:29:17.396 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2024-04-21T08:29:17.396 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-04-21T08:29:17.492 DEBUG:teuthology.orchestra.run.smithi082:> echo no | sudo yum reinstall kernel || true 2024-04-21T08:29:17.988 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:02 ago on Sun 21 Apr 2024 08:29:15 AM UTC. 2024-04-21T08:29:18.125 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-04-21T08:29:18.136 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-04-21T08:29:18.137 INFO:teuthology.orchestra.run.smithi082.stdout: Package Architecture Version Repository Size 2024-04-21T08:29:18.137 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-04-21T08:29:18.137 INFO:teuthology.orchestra.run.smithi082.stdout:Reinstalling: 2024-04-21T08:29:18.137 INFO:teuthology.orchestra.run.smithi082.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-21T08:29:18.137 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:29:18.137 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-04-21T08:29:18.137 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-04-21T08:29:18.137 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:29:18.137 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 255 k 2024-04-21T08:29:18.137 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 0 2024-04-21T08:29:18.138 INFO:teuthology.orchestra.run.smithi082.stderr:Operation aborted. 2024-04-21T08:29:18.198 INFO:teuthology.orchestra.run.smithi082.stdout:Is this ok [y/N]: 2024-04-21T08:29:18.199 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum reinstall -y kernel || true 2024-04-21T08:29:18.692 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:03 ago on Sun 21 Apr 2024 08:29:15 AM UTC. 2024-04-21T08:29:18.809 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-04-21T08:29:18.810 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-04-21T08:29:18.810 INFO:teuthology.orchestra.run.smithi082.stdout: Package Architecture Version Repository Size 2024-04-21T08:29:18.810 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-04-21T08:29:18.810 INFO:teuthology.orchestra.run.smithi082.stdout:Reinstalling: 2024-04-21T08:29:18.810 INFO:teuthology.orchestra.run.smithi082.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-21T08:29:18.810 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:29:18.810 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-04-21T08:29:18.810 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-04-21T08:29:18.810 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:29:18.811 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 255 k 2024-04-21T08:29:18.811 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 0 2024-04-21T08:29:18.811 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-04-21T08:29:19.103 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.4 MB/s | 255 kB 00:00 2024-04-21T08:29:19.103 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:29:19.103 INFO:teuthology.orchestra.run.smithi082.stdout:Total 876 kB/s | 255 kB 00:00 2024-04-21T08:29:19.121 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-04-21T08:29:19.296 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-04-21T08:29:19.296 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-04-21T08:29:19.301 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-04-21T08:29:19.302 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-04-21T08:29:19.430 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-04-21T08:29:19.494 INFO:teuthology.orchestra.run.smithi082.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-21T08:29:19.494 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T08:29:20.174 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T08:29:20.174 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-21T08:29:20.860 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T08:29:20.861 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:29:20.861 INFO:teuthology.orchestra.run.smithi082.stdout:Reinstalled: 2024-04-21T08:29:20.861 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-21T08:29:20.861 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:29:20.861 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-04-21T08:29:21.066 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2024-04-21T08:29:21.112 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-21T08:29:21.113 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-21T08:29:21.113 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-21T08:29:21.114 DEBUG:teuthology.parallel:result is None 2024-04-21T08:29:21.114 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-21T08:29:21.121 INFO:teuthology.task.internal:Creating test directory... 2024-04-21T08:29:21.121 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T08:29:21.125 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T08:29:21.127 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T08:29:21.148 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-21T08:29:21.155 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-21T08:29:21.162 INFO:teuthology.task.internal:Creating archive directory... 2024-04-21T08:29:21.163 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T08:29:21.180 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T08:29:21.185 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T08:29:21.284 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-21T08:29:21.290 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-21T08:29:21.291 DEBUG:teuthology.orchestra.run.smithi106:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T08:29:21.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:29:21.307 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T08:29:21.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:29:21.325 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T08:29:21.329 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T08:29:21.352 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T08:29:21.364 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:29:21.387 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:29:21.390 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:29:21.397 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:29:21.408 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:29:21.428 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:29:21.430 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-21T08:29:21.440 INFO:teuthology.task.internal:Configuring sudo... 2024-04-21T08:29:21.441 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T08:29:21.447 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T08:29:21.452 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T08:29:21.496 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-21T08:29:21.510 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-21T08:29:21.510 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T08:29:21.513 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T08:29:21.518 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T08:29:21.552 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:29:21.607 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:29:21.651 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:29:21.733 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:29:21.772 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:29:21.772 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T08:29:21.835 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:29:21.879 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:29:21.925 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:29:22.008 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:29:22.050 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-21T08:29:22.050 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T08:29:22.116 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:29:22.159 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:29:22.217 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:29:22.302 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:29:22.342 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-21T08:29:22.342 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T08:29:22.409 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-04-21T08:29:22.414 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2024-04-21T08:29:22.417 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-04-21T08:29:22.445 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:29:22.446 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:29:22.495 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:29:22.913 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-21T08:29:22.921 INFO:teuthology.task.internal:Starting timer... 2024-04-21T08:29:22.921 INFO:teuthology.run_tasks:Running task pcp... 2024-04-21T08:29:22.931 INFO:teuthology.run_tasks:Running task selinux... 2024-04-21T08:29:22.940 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-21T08:29:22.940 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2024-04-21T08:29:22.994 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2024-04-21T08:29:22.995 DEBUG:teuthology.orchestra.run.smithi106:> sudo service auditd rotate 2024-04-21T08:29:23.067 INFO:teuthology.orchestra.run.smithi106.stdout:Rotating logs: 2024-04-21T08:29:23.069 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-04-21T08:29:23.139 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-04-21T08:29:23.141 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-21T08:29:23.141 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2024-04-21T08:29:23.172 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2024-04-21T08:29:23.172 DEBUG:teuthology.orchestra.run.smithi106:> /usr/sbin/getenforce 2024-04-21T08:29:23.216 INFO:teuthology.orchestra.run.smithi106.stdout:Permissive 2024-04-21T08:29:23.216 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-04-21T08:29:23.240 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-04-21T08:29:23.240 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi082.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi106.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2024-04-21T08:29:23.241 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:29:23.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:29:23.266 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:29:23.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:29:23.295 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:29:23.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:29:23.321 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-21T08:29:23.321 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2024-04-21T08:29:23.344 DEBUG:teuthology.orchestra.run.smithi106:> sudo /usr/sbin/setenforce permissive 2024-04-21T08:29:23.372 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-04-21T08:29:23.397 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-21T08:29:23.411 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-21T08:29:23.432 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-21T08:29:23.446 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi082.front.sepia.ceph.com,smithi106.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-21T08:33:09.743 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi082.front.sepia.ceph.com'), Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-04-21T08:33:09.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-21T08:33:09.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:33:09.821 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-04-21T08:33:09.901 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-21T08:33:09.901 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-21T08:33:09.901 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:33:09.975 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-04-21T08:33:10.054 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-21T08:33:10.054 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-21T08:33:10.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:33:10.130 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-04-21T08:33:10.205 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-21T08:33:10.206 INFO:teuthology.run_tasks:Running task clock... 2024-04-21T08:33:10.218 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-21T08:33:10.218 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T08:33:10.219 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:33:10.221 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T08:33:10.221 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:33:10.224 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T08:33:10.224 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:33:10.247 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T08:33:10.251 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T08:33:10.260 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T08:33:10.265 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T08:33:10.274 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T08:33:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2024-04-21T08:33:10.288 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T08:33:10.288 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: ntpd: command not found 2024-04-21T08:33:10.292 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2024-04-21T08:33:10.298 INFO:teuthology.orchestra.run.smithi106.stdout:506 Cannot talk to daemon 2024-04-21T08:33:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T08:33:10.310 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-04-21T08:33:10.311 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T08:33:10.318 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T08:33:10.320 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-04-21T08:33:10.324 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T08:33:10.333 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T08:33:10.346 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T08:33:10.373 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:33:10.375 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:33:10.376 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-04-21T08:33:10.376 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:33:10.376 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:33:10.376 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:33:10.376 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:33:10.376 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:33:10.379 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:33:10.379 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-21T08:33:10.380 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:33:10.380 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:33:10.380 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:33:10.380 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:33:10.402 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:33:10.405 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:33:10.405 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-04-21T08:33:10.406 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:33:10.406 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:33:10.406 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:33:10.406 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:33:10.407 INFO:teuthology.run_tasks:Running task install... 2024-04-21T08:33:10.416 DEBUG:teuthology.task.install:project ceph 2024-04-21T08:33:10.416 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e'}} 2024-04-21T08:33:10.416 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e'} 2024-04-21T08:33:10.416 INFO:teuthology.task.install:Using flavor: default 2024-04-21T08:33:10.422 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-21T08:33:10.422 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-21T08:33:10.423 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T08:33:10.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:33:10.426 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T08:33:10.426 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:33:10.427 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T08:33:10.427 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:33:10.569 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T08:33:10.569 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T08:33:10.575 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T08:33:10.575 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T08:33:10.637 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T08:33:10.638 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T08:33:10.678 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T08:33:10.679 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:33:10.679 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T08:33:10.703 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T08:33:10.703 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T08:33:10.743 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T08:33:10.743 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-21T08:33:10.743 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T08:33:10.745 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T08:33:10.745 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-21T08:33:10.745 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T08:33:10.769 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T08:33:10.769 DEBUG:teuthology.orchestra.run.smithi106:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T08:33:10.771 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T08:33:10.771 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T08:33:10.771 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T08:33:10.833 DEBUG:teuthology.orchestra.run.smithi106:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T08:33:10.836 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T08:33:10.843 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T08:33:10.906 DEBUG:teuthology.orchestra.run.smithi106:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T08:33:10.907 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2024-04-21T08:33:10.908 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2024-04-21T08:33:10.909 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T08:33:10.968 INFO:teuthology.orchestra.run.smithi106.stdout:check_obsoletes = 1 2024-04-21T08:33:10.969 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum clean all 2024-04-21T08:33:10.972 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-04-21T08:33:10.974 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-04-21T08:33:11.133 INFO:teuthology.orchestra.run.smithi082.stdout:48 files removed 2024-04-21T08:33:11.160 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T08:33:11.194 INFO:teuthology.orchestra.run.smithi106.stdout:48 files removed 2024-04-21T08:33:11.198 INFO:teuthology.orchestra.run.smithi134.stdout:48 files removed 2024-04-21T08:33:11.221 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T08:33:11.226 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T08:33:11.845 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 196 kB/s | 83 kB 00:00 2024-04-21T08:33:11.893 INFO:teuthology.orchestra.run.smithi106.stdout:ceph packages for x86_64 201 kB/s | 83 kB 00:00 2024-04-21T08:33:11.962 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 175 kB/s | 83 kB 00:00 2024-04-21T08:33:12.246 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-21T08:33:12.268 INFO:teuthology.orchestra.run.smithi106.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-04-21T08:33:12.395 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-21T08:33:12.660 INFO:teuthology.orchestra.run.smithi106.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-04-21T08:33:12.694 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-21T08:33:12.754 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-04-21T08:33:13.219 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-21T08:33:13.297 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-21T08:33:15.021 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-21T08:33:15.129 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 3.4 MB/s | 8.0 MB 00:02 2024-04-21T08:33:17.033 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-21T08:33:18.980 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-21T08:33:19.247 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 19 MB 00:04 2024-04-21T08:33:20.372 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-04-21T08:33:21.351 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-21T08:33:23.688 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.2 MB 00:00 2024-04-21T08:33:25.506 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-21T08:33:26.373 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-21T08:33:26.527 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-04-21T08:33:28.541 INFO:teuthology.orchestra.run.smithi106.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:33:28.542 INFO:teuthology.orchestra.run.smithi106.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:33:28.552 INFO:teuthology.orchestra.run.smithi106.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T08:33:28.552 INFO:teuthology.orchestra.run.smithi106.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T08:33:28.553 INFO:teuthology.orchestra.run.smithi106.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T08:33:28.554 INFO:teuthology.orchestra.run.smithi106.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T08:33:28.556 INFO:teuthology.orchestra.run.smithi106.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T08:33:28.558 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T08:33:28.559 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T08:33:28.617 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-21T08:33:28.632 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-21T08:33:28.632 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2024-04-21T08:33:28.632 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T08:33:28.633 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T08:33:28.634 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T08:33:28.635 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T08:33:28.635 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T08:33:28.635 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T08:33:28.635 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T08:33:28.635 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T08:33:28.635 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T08:33:28.635 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T08:33:28.635 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:33:28.635 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T08:33:28.635 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T08:33:28.635 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrading: 2024-04-21T08:33:28.635 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T08:33:28.636 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:33:28.636 INFO:teuthology.orchestra.run.smithi106.stdout:Installing dependencies: 2024-04-21T08:33:28.636 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T08:33:28.636 INFO:teuthology.orchestra.run.smithi106.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T08:33:28.636 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T08:33:28.636 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T08:33:28.636 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T08:33:28.636 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T08:33:28.636 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T08:33:28.636 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T08:33:28.636 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T08:33:28.636 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T08:33:28.637 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T08:33:28.638 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T08:33:28.639 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T08:33:28.640 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T08:33:28.641 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T08:33:28.642 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T08:33:28.643 INFO:teuthology.orchestra.run.smithi106.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T08:33:28.644 INFO:teuthology.orchestra.run.smithi106.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T08:33:28.644 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T08:33:28.644 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T08:33:28.644 INFO:teuthology.orchestra.run.smithi106.stdout:Installing weak dependencies: 2024-04-21T08:33:28.644 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T08:33:28.644 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:33:28.644 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-04-21T08:33:28.645 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-21T08:33:28.645 INFO:teuthology.orchestra.run.smithi106.stdout:Install 148 Packages 2024-04-21T08:33:28.645 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrade 2 Packages 2024-04-21T08:33:28.645 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:33:28.645 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 218 M 2024-04-21T08:33:28.646 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-04-21T08:33:28.697 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 781 kB/s | 6.2 MB 00:08 2024-04-21T08:33:29.494 INFO:teuthology.orchestra.run.smithi106.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 35 kB/s | 6.6 kB 00:00 2024-04-21T08:33:30.052 INFO:teuthology.orchestra.run.smithi106.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 1.6 MB/s | 890 kB 00:00 2024-04-21T08:33:30.131 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-21T08:33:30.478 INFO:teuthology.orchestra.run.smithi106.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 341 kB/s | 145 kB 00:00 2024-04-21T08:33:31.590 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-04-21T08:33:31.669 INFO:teuthology.orchestra.run.smithi106.stdout:(4/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-04-21T08:33:31.803 INFO:teuthology.orchestra.run.smithi106.stdout:(5/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 1.8 MB/s | 2.4 MB 00:01 2024-04-21T08:33:32.195 INFO:teuthology.orchestra.run.smithi106.stdout:(6/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-04-21T08:33:32.916 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 21 MB 00:02 2024-04-21T08:33:33.750 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:33:33.751 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:33:33.761 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T08:33:33.762 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T08:33:33.763 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T08:33:33.764 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T08:33:33.765 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T08:33:33.768 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T08:33:33.769 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T08:33:33.795 INFO:teuthology.orchestra.run.smithi106.stdout:(7/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 2.4 MB/s | 4.7 MB 00:01 2024-04-21T08:33:33.827 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-04-21T08:33:33.836 INFO:teuthology.orchestra.run.smithi082.stdout:======================================================================================= 2024-04-21T08:33:33.836 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout:======================================================================================= 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T08:33:33.837 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T08:33:33.838 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T08:33:33.839 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T08:33:33.840 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T08:33:33.841 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T08:33:33.842 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T08:33:33.843 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T08:33:33.844 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T08:33:33.845 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T08:33:33.846 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T08:33:33.847 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T08:33:33.848 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T08:33:33.848 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T08:33:33.848 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T08:33:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:Installing weak dependencies: 2024-04-21T08:33:33.848 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T08:33:33.848 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:33:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-04-21T08:33:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:======================================================================================= 2024-04-21T08:33:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:Install 148 Packages 2024-04-21T08:33:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 2 Packages 2024-04-21T08:33:33.848 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:33:33.849 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 218 M 2024-04-21T08:33:33.849 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-04-21T08:33:34.412 INFO:teuthology.orchestra.run.smithi106.stdout:(8/150): ceph-common-19.0.0-3154.ga630765c.el9. 4.1 MB/s | 21 MB 00:05 2024-04-21T08:33:34.495 INFO:teuthology.orchestra.run.smithi106.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 303 kB/s | 25 kB 00:00 2024-04-21T08:33:34.716 INFO:teuthology.orchestra.run.smithi082.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-21T08:33:35.850 INFO:teuthology.orchestra.run.smithi082.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 786 kB/s | 890 kB 00:01 2024-04-21T08:33:36.025 INFO:teuthology.orchestra.run.smithi082.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 829 kB/s | 145 kB 00:00 2024-04-21T08:33:37.342 INFO:teuthology.orchestra.run.smithi082.stdout:(4/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 1.8 MB/s | 2.4 MB 00:01 2024-04-21T08:33:37.629 INFO:teuthology.orchestra.run.smithi106.stdout:(10/150): ceph-osd-19.0.0-3154.ga630765c.el9.x8 3.1 MB/s | 17 MB 00:05 2024-04-21T08:33:37.787 INFO:teuthology.orchestra.run.smithi106.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 1.4 MB/s | 225 kB 00:00 2024-04-21T08:33:37.842 INFO:teuthology.orchestra.run.smithi082.stdout:(5/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 3.0 MB/s | 1.5 MB 00:00 2024-04-21T08:33:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 325 kB/s | 27 kB 00:00 2024-04-21T08:33:37.971 INFO:teuthology.orchestra.run.smithi106.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 318 kB/s | 32 kB 00:00 2024-04-21T08:33:38.040 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-21T08:33:38.305 INFO:teuthology.orchestra.run.smithi106.stdout:(14/150): ceph-radosgw-19.0.0-3154.ga630765c.el 2.3 MB/s | 11 MB 00:04 2024-04-21T08:33:38.338 INFO:teuthology.orchestra.run.smithi106.stdout:(15/150): libcephfs2-19.0.0-3154.ga630765c.el9. 2.0 MB/s | 746 kB 00:00 2024-04-21T08:33:38.438 INFO:teuthology.orchestra.run.smithi106.stdout:(16/150): librados-devel-19.0.0-3154.ga630765c. 1.2 MB/s | 127 kB 00:00 2024-04-21T08:33:38.464 INFO:teuthology.orchestra.run.smithi106.stdout:(17/150): libcephsqlite-19.0.0-3154.ga630765c.e 1.0 MB/s | 163 kB 00:00 2024-04-21T08:33:38.780 INFO:teuthology.orchestra.run.smithi106.stdout:(18/150): libradosstriper1-19.0.0-3154.ga630765 1.4 MB/s | 501 kB 00:00 2024-04-21T08:33:38.873 INFO:teuthology.orchestra.run.smithi106.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 492 kB/s | 45 kB 00:00 2024-04-21T08:33:38.884 INFO:teuthology.orchestra.run.smithi082.stdout:(6/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 4.5 MB/s | 4.7 MB 00:01 2024-04-21T08:33:38.940 INFO:teuthology.orchestra.run.smithi106.stdout:(20/150): python3-ceph-common-19.0.0-3154.ga630 2.0 MB/s | 135 kB 00:00 2024-04-21T08:33:38.950 INFO:teuthology.orchestra.run.smithi082.stdout:(7/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 1.2 MB/s | 5.4 MB 00:04 2024-04-21T08:33:39.057 INFO:teuthology.orchestra.run.smithi106.stdout:(21/150): python3-cephfs-19.0.0-3154.ga630765c. 1.4 MB/s | 162 kB 00:00 2024-04-21T08:33:39.207 INFO:teuthology.orchestra.run.smithi106.stdout:(22/150): python3-rados-19.0.0-3154.ga630765c.e 2.1 MB/s | 321 kB 00:00 2024-04-21T08:33:39.349 INFO:teuthology.orchestra.run.smithi106.stdout:(23/150): python3-rbd-19.0.0-3154.ga630765c.el9 2.1 MB/s | 298 kB 00:00 2024-04-21T08:33:39.433 INFO:teuthology.orchestra.run.smithi106.stdout:(24/150): python3-rgw-19.0.0-3154.ga630765c.el9 1.2 MB/s | 100 kB 00:00 2024-04-21T08:33:39.508 INFO:teuthology.orchestra.run.smithi106.stdout:(25/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-21T08:33:40.200 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:33:40.201 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:33:40.211 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T08:33:40.212 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T08:33:40.213 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T08:33:40.214 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T08:33:40.216 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T08:33:40.218 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T08:33:40.219 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T08:33:40.278 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-04-21T08:33:40.288 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-21T08:33:40.288 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T08:33:40.289 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T08:33:40.290 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T08:33:40.291 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T08:33:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T08:33:40.293 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T08:33:40.294 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T08:33:40.295 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T08:33:40.296 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T08:33:40.296 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T08:33:40.296 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T08:33:40.296 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T08:33:40.296 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T08:33:40.296 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T08:33:40.296 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T08:33:40.296 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T08:33:40.296 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T08:33:40.296 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T08:33:40.296 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T08:33:40.296 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T08:33:40.297 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T08:33:40.297 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T08:33:40.297 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T08:33:40.297 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T08:33:40.297 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T08:33:40.297 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T08:33:40.297 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T08:33:40.297 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T08:33:40.297 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T08:33:40.297 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T08:33:40.297 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T08:33:40.297 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T08:33:40.298 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T08:33:40.299 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout:Install 148 Packages 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 2 Packages 2024-04-21T08:33:40.300 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:33:40.301 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 218 M 2024-04-21T08:33:40.301 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-04-21T08:33:40.650 INFO:teuthology.orchestra.run.smithi106.stdout:(26/150): librgw2-19.0.0-3154.ga630765c.el9.x86 2.5 MB/s | 5.4 MB 00:02 2024-04-21T08:33:40.700 INFO:teuthology.orchestra.run.smithi106.stdout:(27/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 2.6 MB/s | 3.1 MB 00:01 2024-04-21T08:33:40.734 INFO:teuthology.orchestra.run.smithi106.stdout:(28/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-21T08:33:40.759 INFO:teuthology.orchestra.run.smithi106.stdout:(29/150): ceph-grafana-dashboards-19.0.0-3154.g 582 kB/s | 34 kB 00:00 2024-04-21T08:33:40.785 INFO:teuthology.orchestra.run.smithi106.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 2.8 MB/s | 142 kB 00:00 2024-04-21T08:33:40.850 INFO:teuthology.orchestra.run.smithi082.stdout:(8/150): ceph-osd-19.0.0-3154.ga630765c.el9.x86 8.6 MB/s | 17 MB 00:01 2024-04-21T08:33:40.950 INFO:teuthology.orchestra.run.smithi082.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 252 kB/s | 25 kB 00:00 2024-04-21T08:33:41.683 INFO:teuthology.orchestra.run.smithi134.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-04-21T08:33:42.060 INFO:teuthology.orchestra.run.smithi106.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 2.7 MB/s | 3.5 MB 00:01 2024-04-21T08:33:42.227 INFO:teuthology.orchestra.run.smithi106.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 1.5 MB/s | 249 kB 00:00 2024-04-21T08:33:42.319 INFO:teuthology.orchestra.run.smithi106.stdout:(33/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 539 kB/s | 49 kB 00:00 2024-04-21T08:33:42.435 INFO:teuthology.orchestra.run.smithi106.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3154.ga 141 kB/s | 16 kB 00:00 2024-04-21T08:33:42.611 INFO:teuthology.orchestra.run.smithi106.stdout:(35/150): ceph-volume-19.0.0-3154.ga630765c.el9 1.5 MB/s | 273 kB 00:00 2024-04-21T08:33:42.625 INFO:teuthology.orchestra.run.smithi134.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 945 kB/s | 890 kB 00:00 2024-04-21T08:33:42.792 INFO:teuthology.orchestra.run.smithi134.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 872 kB/s | 145 kB 00:00 2024-04-21T08:33:42.994 INFO:teuthology.orchestra.run.smithi106.stdout:(36/150): cephadm-19.0.0-3154.ga630765c.el9.noa 1.9 MB/s | 762 kB 00:00 2024-04-21T08:33:43.095 INFO:teuthology.orchestra.run.smithi106.stdout:(37/150): cephfs-top-19.0.0-3154.ga630765c.el9. 183 kB/s | 18 kB 00:00 2024-04-21T08:33:43.195 INFO:teuthology.orchestra.run.smithi106.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 806 kB/s | 80 kB 00:00 2024-04-21T08:33:43.295 INFO:teuthology.orchestra.run.smithi106.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 7.9 MB/s | 803 kB 00:00 2024-04-21T08:33:43.370 INFO:teuthology.orchestra.run.smithi106.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-04-21T08:33:43.446 INFO:teuthology.orchestra.run.smithi106.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-21T08:33:43.571 INFO:teuthology.orchestra.run.smithi106.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-04-21T08:33:43.696 INFO:teuthology.orchestra.run.smithi106.stdout:(43/150): ceph-mgr-diskprediction-local-19.0.0- 2.5 MB/s | 7.4 MB 00:02 2024-04-21T08:33:43.722 INFO:teuthology.orchestra.run.smithi106.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 709 kB/s | 106 kB 00:00 2024-04-21T08:33:43.780 INFO:teuthology.orchestra.run.smithi106.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-21T08:33:43.805 INFO:teuthology.orchestra.run.smithi106.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-21T08:33:43.872 INFO:teuthology.orchestra.run.smithi106.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-21T08:33:44.014 INFO:teuthology.orchestra.run.smithi106.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-04-21T08:33:44.065 INFO:teuthology.orchestra.run.smithi106.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-21T08:33:44.115 INFO:teuthology.orchestra.run.smithi106.stdout:(50/150): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-04-21T08:33:44.140 INFO:teuthology.orchestra.run.smithi106.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 286 kB/s | 21 kB 00:00 2024-04-21T08:33:44.158 INFO:teuthology.orchestra.run.smithi134.stdout:(4/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-04-21T08:33:44.165 INFO:teuthology.orchestra.run.smithi106.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-21T08:33:44.191 INFO:teuthology.orchestra.run.smithi106.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-04-21T08:33:44.233 INFO:teuthology.orchestra.run.smithi106.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 440 kB/s | 18 kB 00:00 2024-04-21T08:33:44.291 INFO:teuthology.orchestra.run.smithi106.stdout:(55/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-21T08:33:44.309 INFO:teuthology.orchestra.run.smithi106.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 147 kB/s | 11 kB 00:00 2024-04-21T08:33:44.342 INFO:teuthology.orchestra.run.smithi106.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-21T08:33:44.359 INFO:teuthology.orchestra.run.smithi106.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-04-21T08:33:44.401 INFO:teuthology.orchestra.run.smithi106.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-21T08:33:44.435 INFO:teuthology.orchestra.run.smithi106.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-21T08:33:44.460 INFO:teuthology.orchestra.run.smithi106.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-04-21T08:33:44.485 INFO:teuthology.orchestra.run.smithi106.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-21T08:33:44.511 INFO:teuthology.orchestra.run.smithi106.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-21T08:33:44.536 INFO:teuthology.orchestra.run.smithi106.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-21T08:33:44.561 INFO:teuthology.orchestra.run.smithi106.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-04-21T08:33:44.611 INFO:teuthology.orchestra.run.smithi134.stdout:(5/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-04-21T08:33:44.687 INFO:teuthology.orchestra.run.smithi106.stdout:(66/150): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-21T08:33:44.754 INFO:teuthology.orchestra.run.smithi106.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 22 MB/s | 4.1 MB 00:00 2024-04-21T08:33:44.795 INFO:teuthology.orchestra.run.smithi106.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-21T08:33:44.829 INFO:teuthology.orchestra.run.smithi106.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-21T08:33:44.854 INFO:teuthology.orchestra.run.smithi106.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-21T08:33:44.921 INFO:teuthology.orchestra.run.smithi106.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-21T08:33:44.921 INFO:teuthology.orchestra.run.smithi134.stdout:(6/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 2.0 MB/s | 1.5 MB 00:00 2024-04-21T08:33:44.971 INFO:teuthology.orchestra.run.smithi106.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-21T08:33:45.046 INFO:teuthology.orchestra.run.smithi106.stdout:(73/150): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-04-21T08:33:45.097 INFO:teuthology.orchestra.run.smithi106.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-21T08:33:45.172 INFO:teuthology.orchestra.run.smithi106.stdout:(75/150): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-04-21T08:33:45.197 INFO:teuthology.orchestra.run.smithi106.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-21T08:33:45.223 INFO:teuthology.orchestra.run.smithi106.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-04-21T08:33:45.248 INFO:teuthology.orchestra.run.smithi106.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-21T08:33:45.273 INFO:teuthology.orchestra.run.smithi106.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-21T08:33:45.298 INFO:teuthology.orchestra.run.smithi106.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-04-21T08:33:45.357 INFO:teuthology.orchestra.run.smithi106.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-21T08:33:45.424 INFO:teuthology.orchestra.run.smithi106.stdout:(82/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-04-21T08:33:45.449 INFO:teuthology.orchestra.run.smithi106.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-21T08:33:45.475 INFO:teuthology.orchestra.run.smithi106.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-21T08:33:45.491 INFO:teuthology.orchestra.run.smithi106.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-21T08:33:45.534 INFO:teuthology.orchestra.run.smithi106.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-21T08:33:45.559 INFO:teuthology.orchestra.run.smithi106.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-21T08:33:45.584 INFO:teuthology.orchestra.run.smithi106.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-21T08:33:45.610 INFO:teuthology.orchestra.run.smithi106.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-21T08:33:45.635 INFO:teuthology.orchestra.run.smithi106.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-21T08:33:45.685 INFO:teuthology.orchestra.run.smithi106.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-04-21T08:33:45.735 INFO:teuthology.orchestra.run.smithi106.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-04-21T08:33:45.786 INFO:teuthology.orchestra.run.smithi106.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-21T08:33:45.828 INFO:teuthology.orchestra.run.smithi106.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-04-21T08:33:45.870 INFO:teuthology.orchestra.run.smithi106.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-21T08:33:45.920 INFO:teuthology.orchestra.run.smithi106.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-21T08:33:46.104 INFO:teuthology.orchestra.run.smithi106.stdout:(97/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-04-21T08:33:46.121 INFO:teuthology.orchestra.run.smithi106.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 288 kB/s | 58 kB 00:00 2024-04-21T08:33:46.154 INFO:teuthology.orchestra.run.smithi106.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-21T08:33:46.171 INFO:teuthology.orchestra.run.smithi106.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-21T08:33:46.197 INFO:teuthology.orchestra.run.smithi106.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-04-21T08:33:46.247 INFO:teuthology.orchestra.run.smithi106.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-04-21T08:33:46.311 INFO:teuthology.orchestra.run.smithi134.stdout:(7/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 2.8 MB/s | 4.7 MB 00:01 2024-04-21T08:33:46.597 INFO:teuthology.orchestra.run.smithi106.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-04-21T08:33:46.931 INFO:teuthology.orchestra.run.smithi106.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 801 kB/s | 548 kB 00:00 2024-04-21T08:33:47.031 INFO:teuthology.orchestra.run.smithi106.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-04-21T08:33:47.215 INFO:teuthology.orchestra.run.smithi106.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 500 kB/s | 309 kB 00:00 2024-04-21T08:33:47.715 INFO:teuthology.orchestra.run.smithi106.stdout:(107/150): ceph-test-19.0.0-3154.ga630765c.el9. 3.7 MB/s | 49 MB 00:13 2024-04-21T08:33:47.740 INFO:teuthology.orchestra.run.smithi106.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 49 kB/s | 26 kB 00:00 2024-04-21T08:33:47.815 INFO:teuthology.orchestra.run.smithi106.stdout:(109/150): libarrow-9.0.0-10.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-04-21T08:33:47.866 INFO:teuthology.orchestra.run.smithi106.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2024-04-21T08:33:47.949 INFO:teuthology.orchestra.run.smithi106.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-21T08:33:48.041 INFO:teuthology.orchestra.run.smithi082.stdout:(10/150): ceph-test-19.0.0-3154.ga630765c.el9.x 6.9 MB/s | 49 MB 00:07 2024-04-21T08:33:48.058 INFO:teuthology.orchestra.run.smithi106.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-04-21T08:33:48.108 INFO:teuthology.orchestra.run.smithi106.stdout:(113/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-04-21T08:33:48.141 INFO:teuthology.orchestra.run.smithi082.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 2.2 MB/s | 225 kB 00:00 2024-04-21T08:33:48.159 INFO:teuthology.orchestra.run.smithi106.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 433 kB/s | 43 kB 00:00 2024-04-21T08:33:48.184 INFO:teuthology.orchestra.run.smithi106.stdout:(115/150): liboath-2.6.7-2.el9.x86_64.rpm 104 kB/s | 49 kB 00:00 2024-04-21T08:33:48.209 INFO:teuthology.orchestra.run.smithi106.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 319 kB/s | 32 kB 00:00 2024-04-21T08:33:48.225 INFO:teuthology.orchestra.run.smithi082.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 325 kB/s | 27 kB 00:00 2024-04-21T08:33:48.259 INFO:teuthology.orchestra.run.smithi106.stdout:(117/150): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-21T08:33:48.317 INFO:teuthology.orchestra.run.smithi082.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 346 kB/s | 32 kB 00:00 2024-04-21T08:33:48.326 INFO:teuthology.orchestra.run.smithi106.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-21T08:33:48.360 INFO:teuthology.orchestra.run.smithi106.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-21T08:33:48.419 INFO:teuthology.orchestra.run.smithi106.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-04-21T08:33:48.444 INFO:teuthology.orchestra.run.smithi106.stdout:(121/150): python3-cheroot-8.6.0-4.el9.noarch.r 662 kB/s | 172 kB 00:00 2024-04-21T08:33:48.478 INFO:teuthology.orchestra.run.smithi106.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-04-21T08:33:48.517 INFO:teuthology.orchestra.run.smithi082.stdout:(14/150): libcephfs2-19.0.0-3154.ga630765c.el9. 3.6 MB/s | 746 kB 00:00 2024-04-21T08:33:48.570 INFO:teuthology.orchestra.run.smithi106.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-04-21T08:33:48.594 INFO:teuthology.orchestra.run.smithi106.stdout:(124/150): python3-grpcio-tools-1.46.7-10.el9.x 961 kB/s | 144 kB 00:00 2024-04-21T08:33:48.617 INFO:teuthology.orchestra.run.smithi082.stdout:(15/150): libcephsqlite-19.0.0-3154.ga630765c.e 1.6 MB/s | 163 kB 00:00 2024-04-21T08:33:48.653 INFO:teuthology.orchestra.run.smithi106.stdout:(125/150): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-04-21T08:33:48.670 INFO:teuthology.orchestra.run.smithi106.stdout:(126/150): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-21T08:33:48.695 INFO:teuthology.orchestra.run.smithi106.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-21T08:33:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:(16/150): librados-devel-19.0.0-3154.ga630765c. 1.2 MB/s | 127 kB 00:00 2024-04-21T08:33:48.754 INFO:teuthology.orchestra.run.smithi106.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-21T08:33:48.779 INFO:teuthology.orchestra.run.smithi106.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-21T08:33:48.855 INFO:teuthology.orchestra.run.smithi106.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-04-21T08:33:48.880 INFO:teuthology.orchestra.run.smithi106.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-04-21T08:33:48.893 INFO:teuthology.orchestra.run.smithi082.stdout:(17/150): libradosstriper1-19.0.0-3154.ga630765 2.8 MB/s | 501 kB 00:00 2024-04-21T08:33:48.955 INFO:teuthology.orchestra.run.smithi106.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 577 kB/s | 58 kB 00:00 2024-04-21T08:33:48.980 INFO:teuthology.orchestra.run.smithi106.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-21T08:33:49.010 INFO:teuthology.orchestra.run.smithi082.stdout:(18/150): ceph-radosgw-19.0.0-3154.ga630765c.el 1.1 MB/s | 11 MB 00:10 2024-04-21T08:33:49.014 INFO:teuthology.orchestra.run.smithi106.stdout:(134/150): python3-kubernetes-26.1.0-2.el9.noar 3.2 MB/s | 1.0 MB 00:00 2024-04-21T08:33:49.048 INFO:teuthology.orchestra.run.smithi106.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-04-21T08:33:49.081 INFO:teuthology.orchestra.run.smithi106.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-04-21T08:33:49.102 INFO:teuthology.orchestra.run.smithi082.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 493 kB/s | 45 kB 00:00 2024-04-21T08:33:49.115 INFO:teuthology.orchestra.run.smithi106.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-04-21T08:33:49.148 INFO:teuthology.orchestra.run.smithi106.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-21T08:33:49.182 INFO:teuthology.orchestra.run.smithi106.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-21T08:33:49.216 INFO:teuthology.orchestra.run.smithi106.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-04-21T08:33:49.249 INFO:teuthology.orchestra.run.smithi106.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-04-21T08:33:49.269 INFO:teuthology.orchestra.run.smithi082.stdout:(20/150): python3-ceph-common-19.0.0-3154.ga630 809 kB/s | 135 kB 00:00 2024-04-21T08:33:49.283 INFO:teuthology.orchestra.run.smithi106.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-21T08:33:49.325 INFO:teuthology.orchestra.run.smithi106.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-21T08:33:49.350 INFO:teuthology.orchestra.run.smithi106.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-04-21T08:33:49.369 INFO:teuthology.orchestra.run.smithi134.stdout:(8/150): ceph-radosgw-19.0.0-3154.ga630765c.el9 3.5 MB/s | 11 MB 00:03 2024-04-21T08:33:49.409 INFO:teuthology.orchestra.run.smithi106.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-21T08:33:49.426 INFO:teuthology.orchestra.run.smithi106.stdout:(146/150): python3-zc-lockfile-2.0-10.el9.noarc 198 kB/s | 20 kB 00:00 2024-04-21T08:33:49.427 INFO:teuthology.orchestra.run.smithi082.stdout:(21/150): python3-cephfs-19.0.0-3154.ga630765c. 1.0 MB/s | 162 kB 00:00 2024-04-21T08:33:49.453 INFO:teuthology.orchestra.run.smithi134.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 303 kB/s | 25 kB 00:00 2024-04-21T08:33:49.459 INFO:teuthology.orchestra.run.smithi106.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-21T08:33:49.543 INFO:teuthology.orchestra.run.smithi106.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-21T08:33:49.770 INFO:teuthology.orchestra.run.smithi082.stdout:(22/150): python3-rados-19.0.0-3154.ga630765c.e 938 kB/s | 321 kB 00:00 2024-04-21T08:33:49.862 INFO:teuthology.orchestra.run.smithi082.stdout:(23/150): librgw2-19.0.0-3154.ga630765c.el9.x86 5.5 MB/s | 5.4 MB 00:00 2024-04-21T08:33:49.981 INFO:teuthology.orchestra.run.smithi082.stdout:(24/150): python3-rgw-19.0.0-3154.ga630765c.el9 840 kB/s | 100 kB 00:00 2024-04-21T08:33:50.056 INFO:teuthology.orchestra.run.smithi082.stdout:(25/150): python3-rbd-19.0.0-3154.ga630765c.el9 1.0 MB/s | 298 kB 00:00 2024-04-21T08:33:50.089 INFO:teuthology.orchestra.run.smithi082.stdout:(26/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 786 kB/s | 85 kB 00:00 2024-04-21T08:33:50.207 INFO:teuthology.orchestra.run.smithi082.stdout:(27/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 1.4 MB/s | 171 kB 00:00 2024-04-21T08:33:50.323 INFO:teuthology.orchestra.run.smithi082.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3154.g 293 kB/s | 34 kB 00:00 2024-04-21T08:33:50.432 INFO:teuthology.orchestra.run.smithi082.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 1.3 MB/s | 142 kB 00:00 2024-04-21T08:33:50.886 INFO:teuthology.orchestra.run.smithi134.stdout:(10/150): ceph-osd-19.0.0-3154.ga630765c.el9.x8 2.8 MB/s | 17 MB 00:05 2024-04-21T08:33:50.916 INFO:teuthology.orchestra.run.smithi082.stdout:(30/150): ceph-common-19.0.0-3154.ga630765c.el9 1.3 MB/s | 21 MB 00:16 2024-04-21T08:33:51.002 INFO:teuthology.orchestra.run.smithi106.stdout:(149/150): librados2-19.0.0-3154.ga630765c.el9. 2.2 MB/s | 3.4 MB 00:01 2024-04-21T08:33:51.103 INFO:teuthology.orchestra.run.smithi134.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 1.0 MB/s | 225 kB 00:00 2024-04-21T08:33:51.126 INFO:teuthology.orchestra.run.smithi106.stdout:(150/150): librbd1-19.0.0-3154.ga630765c.el9.x8 1.9 MB/s | 3.1 MB 00:01 2024-04-21T08:33:51.134 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:33:51.134 INFO:teuthology.orchestra.run.smithi106.stdout:Total 9.7 MB/s | 218 MB 00:22 2024-04-21T08:33:51.187 INFO:teuthology.orchestra.run.smithi134.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 325 kB/s | 27 kB 00:00 2024-04-21T08:33:51.262 INFO:teuthology.orchestra.run.smithi134.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 423 kB/s | 32 kB 00:00 2024-04-21T08:33:51.274 INFO:teuthology.orchestra.run.smithi082.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 4.2 MB/s | 3.5 MB 00:00 2024-04-21T08:33:51.374 INFO:teuthology.orchestra.run.smithi082.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 2.4 MB/s | 249 kB 00:00 2024-04-21T08:33:51.466 INFO:teuthology.orchestra.run.smithi082.stdout:(33/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 538 kB/s | 49 kB 00:00 2024-04-21T08:33:51.550 INFO:teuthology.orchestra.run.smithi082.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3154.ga 197 kB/s | 16 kB 00:00 2024-04-21T08:33:51.579 INFO:teuthology.orchestra.run.smithi134.stdout:(14/150): libcephfs2-19.0.0-3154.ga630765c.el9. 2.3 MB/s | 746 kB 00:00 2024-04-21T08:33:51.650 INFO:teuthology.orchestra.run.smithi082.stdout:(35/150): ceph-volume-19.0.0-3154.ga630765c.el9 2.7 MB/s | 273 kB 00:00 2024-04-21T08:33:51.804 INFO:teuthology.orchestra.run.smithi134.stdout:(15/150): libcephsqlite-19.0.0-3154.ga630765c.e 726 kB/s | 163 kB 00:00 2024-04-21T08:33:51.880 INFO:teuthology.orchestra.run.smithi134.stdout:(16/150): librados-devel-19.0.0-3154.ga630765c. 1.7 MB/s | 127 kB 00:00 2024-04-21T08:33:51.892 INFO:teuthology.orchestra.run.smithi082.stdout:(36/150): cephadm-19.0.0-3154.ga630765c.el9.noa 3.1 MB/s | 762 kB 00:00 2024-04-21T08:33:51.992 INFO:teuthology.orchestra.run.smithi082.stdout:(37/150): cephfs-top-19.0.0-3154.ga630765c.el9. 182 kB/s | 18 kB 00:00 2024-04-21T08:33:52.138 INFO:teuthology.orchestra.run.smithi134.stdout:(17/150): libradosstriper1-19.0.0-3154.ga630765 1.9 MB/s | 501 kB 00:00 2024-04-21T08:33:52.164 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-04-21T08:33:52.184 INFO:teuthology.orchestra.run.smithi082.stdout:(38/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 1.5 MB/s | 3.1 MB 00:02 2024-04-21T08:33:52.287 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-04-21T08:33:52.287 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-04-21T08:33:52.326 INFO:teuthology.orchestra.run.smithi082.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 241 kB/s | 80 kB 00:00 2024-04-21T08:33:52.510 INFO:teuthology.orchestra.run.smithi082.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-21T08:33:52.702 INFO:teuthology.orchestra.run.smithi082.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-04-21T08:33:53.060 INFO:teuthology.orchestra.run.smithi082.stdout:(42/150): libgfortran-11.4.1-3.el9.x86_64.rpm 917 kB/s | 803 kB 00:00 2024-04-21T08:33:53.143 INFO:teuthology.orchestra.run.smithi082.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-04-21T08:33:53.169 INFO:teuthology.orchestra.run.smithi082.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 981 kB/s | 106 kB 00:00 2024-04-21T08:33:53.269 INFO:teuthology.orchestra.run.smithi082.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-04-21T08:33:53.294 INFO:teuthology.orchestra.run.smithi082.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 999 kB/s | 125 kB 00:00 2024-04-21T08:33:53.378 INFO:teuthology.orchestra.run.smithi082.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-21T08:33:53.992 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-04-21T08:33:53.993 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-04-21T08:33:54.405 INFO:teuthology.orchestra.run.smithi134.stdout:(18/150): librgw2-19.0.0-3154.ga630765c.el9.x86 2.4 MB/s | 5.4 MB 00:02 2024-04-21T08:33:54.505 INFO:teuthology.orchestra.run.smithi134.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 452 kB/s | 45 kB 00:00 2024-04-21T08:33:54.597 INFO:teuthology.orchestra.run.smithi134.stdout:(20/150): python3-ceph-common-19.0.0-3154.ga630 1.4 MB/s | 135 kB 00:00 2024-04-21T08:33:54.664 INFO:teuthology.orchestra.run.smithi134.stdout:(21/150): python3-cephfs-19.0.0-3154.ga630765c. 2.4 MB/s | 162 kB 00:00 2024-04-21T08:33:54.798 INFO:teuthology.orchestra.run.smithi134.stdout:(22/150): python3-rados-19.0.0-3154.ga630765c.e 2.4 MB/s | 321 kB 00:00 2024-04-21T08:33:54.853 INFO:teuthology.orchestra.run.smithi082.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 442 kB/s | 689 kB 00:01 2024-04-21T08:33:54.965 INFO:teuthology.orchestra.run.smithi134.stdout:(23/150): python3-rbd-19.0.0-3154.ga630765c.el9 1.7 MB/s | 298 kB 00:00 2024-04-21T08:33:55.065 INFO:teuthology.orchestra.run.smithi134.stdout:(24/150): python3-rgw-19.0.0-3154.ga630765c.el9 997 kB/s | 100 kB 00:00 2024-04-21T08:33:55.166 INFO:teuthology.orchestra.run.smithi134.stdout:(25/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 855 kB/s | 85 kB 00:00 2024-04-21T08:33:55.220 INFO:teuthology.orchestra.run.smithi082.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 512 kB/s | 943 kB 00:01 2024-04-21T08:33:55.374 INFO:teuthology.orchestra.run.smithi134.stdout:(26/150): ceph-common-19.0.0-3154.ga630765c.el9 1.5 MB/s | 21 MB 00:14 2024-04-21T08:33:55.599 INFO:teuthology.orchestra.run.smithi134.stdout:(27/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 763 kB/s | 171 kB 00:00 2024-04-21T08:33:55.603 INFO:teuthology.orchestra.run.smithi082.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 142 kB/s | 107 kB 00:00 2024-04-21T08:33:55.725 INFO:teuthology.orchestra.run.smithi134.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3154.g 273 kB/s | 34 kB 00:00 2024-04-21T08:33:55.796 INFO:teuthology.orchestra.run.smithi082.stdout:(51/150): ceph-mgr-diskprediction-local-19.0.0- 1.5 MB/s | 7.4 MB 00:04 2024-04-21T08:33:55.821 INFO:teuthology.orchestra.run.smithi082.stdout:(52/150): elfutils-libelf-devel-0.190-2.el9.x86 36 kB/s | 21 kB 00:00 2024-04-21T08:33:55.908 INFO:teuthology.orchestra.run.smithi134.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 776 kB/s | 142 kB 00:00 2024-04-21T08:33:56.430 INFO:teuthology.orchestra.run.smithi082.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-04-21T08:33:56.442 INFO:teuthology.orchestra.run.smithi134.stdout:(30/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 2.4 MB/s | 3.1 MB 00:01 2024-04-21T08:33:56.655 INFO:teuthology.orchestra.run.smithi082.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 298 kB/s | 313 kB 00:01 2024-04-21T08:33:56.955 INFO:teuthology.orchestra.run.smithi082.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-04-21T08:33:57.197 INFO:teuthology.orchestra.run.smithi082.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-04-21T08:33:57.830 INFO:teuthology.orchestra.run.smithi082.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 183 kB/s | 160 kB 00:00 2024-04-21T08:33:57.856 INFO:teuthology.orchestra.run.smithi082.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-04-21T08:33:58.100 INFO:teuthology.orchestra.run.smithi134.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 1.6 MB/s | 3.5 MB 00:02 2024-04-21T08:33:58.359 INFO:teuthology.orchestra.run.smithi134.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 964 kB/s | 249 kB 00:00 2024-04-21T08:33:58.467 INFO:teuthology.orchestra.run.smithi134.stdout:(33/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 454 kB/s | 49 kB 00:00 2024-04-21T08:33:58.568 INFO:teuthology.orchestra.run.smithi134.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3154.ga 165 kB/s | 16 kB 00:00 2024-04-21T08:33:58.835 INFO:teuthology.orchestra.run.smithi134.stdout:(35/150): ceph-volume-19.0.0-3154.ga630765c.el9 1.0 MB/s | 273 kB 00:00 2024-04-21T08:33:59.247 INFO:teuthology.orchestra.run.smithi082.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 418 kB/s | 582 kB 00:01 2024-04-21T08:33:59.314 INFO:teuthology.orchestra.run.smithi082.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 447 kB/s | 662 kB 00:01 2024-04-21T08:33:59.452 INFO:teuthology.orchestra.run.smithi134.stdout:(36/150): ceph-mgr-diskprediction-local-19.0.0- 2.5 MB/s | 7.4 MB 00:03 2024-04-21T08:33:59.485 INFO:teuthology.orchestra.run.smithi134.stdout:(37/150): cephadm-19.0.0-3154.ga630765c.el9.noa 1.1 MB/s | 762 kB 00:00 2024-04-21T08:33:59.544 INFO:teuthology.orchestra.run.smithi134.stdout:(38/150): cephfs-top-19.0.0-3154.ga630765c.el9. 199 kB/s | 18 kB 00:00 2024-04-21T08:33:59.611 INFO:teuthology.orchestra.run.smithi134.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 642 kB/s | 80 kB 00:00 2024-04-21T08:33:59.644 INFO:teuthology.orchestra.run.smithi134.stdout:(40/150): libgfortran-11.4.1-3.el9.x86_64.rpm 7.9 MB/s | 803 kB 00:00 2024-04-21T08:33:59.669 INFO:teuthology.orchestra.run.smithi134.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-21T08:33:59.695 INFO:teuthology.orchestra.run.smithi134.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-21T08:33:59.736 INFO:teuthology.orchestra.run.smithi134.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-21T08:33:59.756 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-04-21T08:33:59.762 INFO:teuthology.orchestra.run.smithi134.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-21T08:33:59.779 INFO:teuthology.orchestra.run.smithi134.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-21T08:33:59.804 INFO:teuthology.orchestra.run.smithi134.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-21T08:33:59.823 INFO:teuthology.orchestra.run.smithi082.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 44 kB/s | 25 kB 00:00 2024-04-21T08:33:59.829 INFO:teuthology.orchestra.run.smithi134.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-21T08:33:59.830 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T08:33:59.913 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T08:33:59.956 INFO:teuthology.orchestra.run.smithi082.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 74 kB/s | 47 kB 00:00 2024-04-21T08:34:00.117 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T08:34:00.390 INFO:teuthology.orchestra.run.smithi082.stdout:(63/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 670 kB/s | 3.0 MB 00:04 2024-04-21T08:34:00.444 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T08:34:00.555 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:34:00.676 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:34:00.706 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:34:00.797 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:34:00.815 INFO:teuthology.orchestra.run.smithi082.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 295 kB/s | 292 kB 00:00 2024-04-21T08:34:00.840 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:34:00.849 INFO:teuthology.orchestra.run.smithi082.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 211 kB/s | 188 kB 00:00 2024-04-21T08:34:00.869 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T08:34:00.895 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T08:34:01.015 INFO:teuthology.orchestra.run.smithi082.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 56 kB/s | 35 kB 00:00 2024-04-21T08:34:01.046 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T08:34:01.092 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T08:34:01.120 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:34:01.223 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:34:01.246 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:34:01.308 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:34:01.330 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:34:01.351 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:34:01.380 INFO:teuthology.orchestra.run.smithi134.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 438 kB/s | 689 kB 00:01 2024-04-21T08:34:01.546 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:34:01.580 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:34:01.663 INFO:teuthology.orchestra.run.smithi134.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 514 kB/s | 943 kB 00:01 2024-04-21T08:34:01.687 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T08:34:01.784 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T08:34:01.812 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T08:34:01.847 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T08:34:01.918 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T08:34:01.964 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T08:34:02.007 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T08:34:02.039 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T08:34:02.068 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T08:34:02.113 INFO:teuthology.orchestra.run.smithi134.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 146 kB/s | 107 kB 00:00 2024-04-21T08:34:02.139 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T08:34:02.191 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T08:34:02.221 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T08:34:02.230 INFO:teuthology.orchestra.run.smithi134.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 38 kB/s | 21 kB 00:00 2024-04-21T08:34:02.265 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T08:34:02.359 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T08:34:02.392 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T08:34:02.435 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T08:34:02.464 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T08:34:02.536 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T08:34:02.579 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T08:34:02.657 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T08:34:02.830 INFO:teuthology.orchestra.run.smithi134.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 54 kB/s | 33 kB 00:00 2024-04-21T08:34:02.949 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T08:34:02.998 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T08:34:03.147 INFO:teuthology.orchestra.run.smithi134.stdout:(53/150): flex-2.6.4-9.el9.x86_64.rpm 303 kB/s | 313 kB 00:01 2024-04-21T08:34:03.537 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T08:34:03.627 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T08:34:03.655 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T08:34:03.689 INFO:teuthology.orchestra.run.smithi134.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 34 kB/s | 18 kB 00:00 2024-04-21T08:34:03.698 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T08:34:03.732 INFO:teuthology.orchestra.run.smithi082.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 599 kB/s | 1.6 MB 00:02 2024-04-21T08:34:03.736 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T08:34:03.769 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T08:34:03.824 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T08:34:03.897 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T08:34:03.929 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T08:34:03.964 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T08:34:04.011 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T08:34:04.047 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T08:34:04.090 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T08:34:04.132 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T08:34:04.198 INFO:teuthology.orchestra.run.smithi134.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-04-21T08:34:04.279 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T08:34:04.346 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T08:34:04.385 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T08:34:04.426 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T08:34:04.521 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T08:34:04.560 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T08:34:04.600 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T08:34:04.626 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T08:34:04.631 INFO:teuthology.orchestra.run.smithi134.stdout:(56/150): ceph-test-19.0.0-3154.ga630765c.el9.x 3.2 MB/s | 49 MB 00:15 2024-04-21T08:34:04.650 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T08:34:04.724 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T08:34:04.852 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T08:34:04.891 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T08:34:04.919 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T08:34:04.953 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T08:34:04.997 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T08:34:05.035 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T08:34:05.066 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T08:34:05.096 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T08:34:05.131 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T08:34:05.164 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T08:34:05.206 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T08:34:05.232 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T08:34:05.265 INFO:teuthology.orchestra.run.smithi134.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-04-21T08:34:05.319 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T08:34:05.348 INFO:teuthology.orchestra.run.smithi134.stdout:(58/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 139 kB/s | 160 kB 00:01 2024-04-21T08:34:05.362 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T08:34:05.393 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T08:34:05.440 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T08:34:05.524 INFO:teuthology.orchestra.run.smithi082.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 509 kB/s | 911 kB 00:01 2024-04-21T08:34:05.539 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T08:34:05.586 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T08:34:06.307 INFO:teuthology.orchestra.run.smithi082.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 170 kB/s | 133 kB 00:00 2024-04-21T08:34:06.325 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T08:34:06.367 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T08:34:06.401 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T08:34:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T08:34:06.559 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T08:34:06.648 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T08:34:06.648 INFO:teuthology.orchestra.run.smithi134.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 448 kB/s | 582 kB 00:01 2024-04-21T08:34:06.674 INFO:teuthology.orchestra.run.smithi134.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 470 kB/s | 662 kB 00:01 2024-04-21T08:34:06.686 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T08:34:06.715 INFO:teuthology.orchestra.run.smithi082.stdout:(70/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 721 kB/s | 4.1 MB 00:05 2024-04-21T08:34:06.863 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T08:34:06.892 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T08:34:06.969 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T08:34:07.249 INFO:teuthology.orchestra.run.smithi134.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 42 kB/s | 25 kB 00:00 2024-04-21T08:34:07.265 INFO:teuthology.orchestra.run.smithi082.stdout:(71/150): openblas-openmp-0.3.21-2.el9.x86_64.r 752 kB/s | 4.7 MB 00:06 2024-04-21T08:34:07.308 INFO:teuthology.orchestra.run.smithi134.stdout:(62/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 684 kB/s | 3.0 MB 00:04 2024-04-21T08:34:07.333 INFO:teuthology.orchestra.run.smithi134.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 72 kB/s | 47 kB 00:00 2024-04-21T08:34:07.629 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T08:34:07.750 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T08:34:07.974 INFO:teuthology.orchestra.run.smithi082.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 124 kB/s | 87 kB 00:00 2024-04-21T08:34:07.975 INFO:teuthology.orchestra.run.smithi134.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 55 kB/s | 35 kB 00:00 2024-04-21T08:34:08.158 INFO:teuthology.orchestra.run.smithi134.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 222 kB/s | 188 kB 00:00 2024-04-21T08:34:08.242 INFO:teuthology.orchestra.run.smithi134.stdout:(66/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 295 kB/s | 292 kB 00:00 2024-04-21T08:34:08.632 INFO:teuthology.orchestra.run.smithi082.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 539 kB/s | 1.0 MB 00:01 2024-04-21T08:34:09.002 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T08:34:09.088 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T08:34:09.116 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T08:34:09.149 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T08:34:09.466 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T08:34:09.504 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:34:09.593 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:34:09.624 INFO:teuthology.orchestra.run.smithi082.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 246 kB/s | 244 kB 00:00 2024-04-21T08:34:09.625 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T08:34:09.661 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T08:34:10.141 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T08:34:10.173 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:34:10.244 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:34:10.274 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T08:34:10.558 INFO:teuthology.orchestra.run.smithi082.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 266 kB/s | 248 kB 00:00 2024-04-21T08:34:10.942 INFO:teuthology.orchestra.run.smithi134.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 608 kB/s | 1.6 MB 00:02 2024-04-21T08:34:11.149 INFO:teuthology.orchestra.run.smithi082.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-04-21T08:34:11.691 INFO:teuthology.orchestra.run.smithi082.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 35 kB/s | 19 kB 00:00 2024-04-21T08:34:12.525 INFO:teuthology.orchestra.run.smithi082.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 207 kB/s | 172 kB 00:00 2024-04-21T08:34:12.573 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:34:12.613 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:34:12.693 INFO:teuthology.orchestra.run.smithi134.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 521 kB/s | 911 kB 00:01 2024-04-21T08:34:13.142 INFO:teuthology.orchestra.run.smithi082.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-04-21T08:34:13.158 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:34:13.199 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:34:13.288 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T08:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T08:34:13.289 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:13.317 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:34:13.468 INFO:teuthology.orchestra.run.smithi134.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 172 kB/s | 133 kB 00:00 2024-04-21T08:34:13.910 INFO:teuthology.orchestra.run.smithi134.stdout:(70/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 736 kB/s | 4.1 MB 00:05 2024-04-21T08:34:14.675 INFO:teuthology.orchestra.run.smithi082.stdout:(80/150): postgresql-server-13.14-1.el9.x86_64. 707 kB/s | 5.8 MB 00:08 2024-04-21T08:34:14.685 INFO:teuthology.orchestra.run.smithi134.stdout:(71/150): openblas-openmp-0.3.21-2.el9.x86_64.r 723 kB/s | 4.7 MB 00:06 2024-04-21T08:34:15.402 INFO:teuthology.orchestra.run.smithi134.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 122 kB/s | 87 kB 00:00 2024-04-21T08:34:15.625 INFO:teuthology.orchestra.run.smithi082.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 264 kB/s | 250 kB 00:00 2024-04-21T08:34:15.785 INFO:teuthology.orchestra.run.smithi134.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 551 kB/s | 1.0 MB 00:01 2024-04-21T08:34:16.059 INFO:teuthology.orchestra.run.smithi082.stdout:(82/150): python3-babel-2.9.1-2.el9.noarch.rpm 755 kB/s | 6.0 MB 00:08 2024-04-21T08:34:16.526 INFO:teuthology.orchestra.run.smithi082.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 247 kB/s | 222 kB 00:00 2024-04-21T08:34:16.719 INFO:teuthology.orchestra.run.smithi134.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 262 kB/s | 244 kB 00:00 2024-04-21T08:34:16.751 INFO:teuthology.orchestra.run.smithi082.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 112 kB/s | 77 kB 00:00 2024-04-21T08:34:17.151 INFO:teuthology.orchestra.run.smithi082.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 68 kB/s | 42 kB 00:00 2024-04-21T08:34:17.644 INFO:teuthology.orchestra.run.smithi134.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 268 kB/s | 248 kB 00:00 2024-04-21T08:34:17.734 INFO:teuthology.orchestra.run.smithi082.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 282 kB/s | 269 kB 00:00 2024-04-21T08:34:17.927 INFO:teuthology.orchestra.run.smithi082.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 205 kB/s | 159 kB 00:00 2024-04-21T08:34:18.236 INFO:teuthology.orchestra.run.smithi134.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-04-21T08:34:18.577 INFO:teuthology.orchestra.run.smithi082.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 78 kB/s | 51 kB 00:00 2024-04-21T08:34:18.710 INFO:teuthology.orchestra.run.smithi082.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 286 kB/s | 279 kB 00:00 2024-04-21T08:34:18.761 INFO:teuthology.orchestra.run.smithi134.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 36 kB/s | 19 kB 00:00 2024-04-21T08:34:19.227 INFO:teuthology.orchestra.run.smithi082.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 82 kB/s | 54 kB 00:00 2024-04-21T08:34:19.594 INFO:teuthology.orchestra.run.smithi134.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 207 kB/s | 172 kB 00:00 2024-04-21T08:34:19.844 INFO:teuthology.orchestra.run.smithi082.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-04-21T08:34:20.103 INFO:teuthology.orchestra.run.smithi082.stdout:(92/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 750 kB/s | 5.1 MB 00:06 2024-04-21T08:34:20.220 INFO:teuthology.orchestra.run.smithi134.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-04-21T08:34:20.511 INFO:teuthology.orchestra.run.smithi082.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 66 kB/s | 44 kB 00:00 2024-04-21T08:34:21.019 INFO:teuthology.orchestra.run.smithi082.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 248 kB/s | 221 kB 00:00 2024-04-21T08:34:21.111 INFO:teuthology.orchestra.run.smithi082.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 60 kB/s | 36 kB 00:00 2024-04-21T08:34:21.178 INFO:teuthology.orchestra.run.smithi134.stdout:(80/150): postgresql-server-13.14-1.el9.x86_64. 768 kB/s | 5.8 MB 00:07 2024-04-21T08:34:21.670 INFO:teuthology.orchestra.run.smithi082.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 73 kB/s | 47 kB 00:00 2024-04-21T08:34:22.070 INFO:teuthology.orchestra.run.smithi082.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 318 kB/s | 305 kB 00:00 2024-04-21T08:34:22.112 INFO:teuthology.orchestra.run.smithi134.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 269 kB/s | 250 kB 00:00 2024-04-21T08:34:22.337 INFO:teuthology.orchestra.run.smithi082.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 87 kB/s | 58 kB 00:00 2024-04-21T08:34:22.829 INFO:teuthology.orchestra.run.smithi082.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 124 kB/s | 94 kB 00:00 2024-04-21T08:34:22.996 INFO:teuthology.orchestra.run.smithi082.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 97 kB/s | 64 kB 00:00 2024-04-21T08:34:23.005 INFO:teuthology.orchestra.run.smithi134.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 249 kB/s | 222 kB 00:00 2024-04-21T08:34:23.104 INFO:teuthology.orchestra.run.smithi082.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-04-21T08:34:23.196 INFO:teuthology.orchestra.run.smithi082.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-04-21T08:34:23.337 INFO:teuthology.orchestra.run.smithi134.stdout:(83/150): python3-babel-2.9.1-2.el9.noarch.rpm 770 kB/s | 6.0 MB 00:07 2024-04-21T08:34:23.446 INFO:teuthology.orchestra.run.smithi082.stdout:(103/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-04-21T08:34:23.587 INFO:teuthology.orchestra.run.smithi134.stdout:(84/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.5 MB/s | 5.1 MB 00:03 2024-04-21T08:34:23.738 INFO:teuthology.orchestra.run.smithi134.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 105 kB/s | 77 kB 00:00 2024-04-21T08:34:23.822 INFO:teuthology.orchestra.run.smithi082.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 877 kB/s | 548 kB 00:00 2024-04-21T08:34:23.914 INFO:teuthology.orchestra.run.smithi082.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-04-21T08:34:23.989 INFO:teuthology.orchestra.run.smithi082.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 570 kB/s | 309 kB 00:00 2024-04-21T08:34:24.021 INFO:teuthology.orchestra.run.smithi134.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 62 kB/s | 42 kB 00:00 2024-04-21T08:34:24.089 INFO:teuthology.orchestra.run.smithi082.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-21T08:34:24.197 INFO:teuthology.orchestra.run.smithi082.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-04-21T08:34:24.290 INFO:teuthology.orchestra.run.smithi082.stdout:(109/150): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-21T08:34:24.317 INFO:teuthology.orchestra.run.smithi082.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 576 kB/s | 67 kB 00:00 2024-04-21T08:34:24.390 INFO:teuthology.orchestra.run.smithi082.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-21T08:34:24.502 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:34:24.502 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /sys 2024-04-21T08:34:24.502 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /proc 2024-04-21T08:34:24.502 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /mnt 2024-04-21T08:34:24.502 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /var/tmp 2024-04-21T08:34:24.502 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /home 2024-04-21T08:34:24.502 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /root 2024-04-21T08:34:24.502 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /tmp 2024-04-21T08:34:24.502 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:24.549 INFO:teuthology.orchestra.run.smithi082.stdout:(112/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.8 MB/s | 16 MB 00:05 2024-04-21T08:34:24.554 INFO:teuthology.orchestra.run.smithi134.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 286 kB/s | 269 kB 00:00 2024-04-21T08:34:24.580 INFO:teuthology.orchestra.run.smithi134.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 188 kB/s | 159 kB 00:00 2024-04-21T08:34:24.583 INFO:teuthology.orchestra.run.smithi082.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-21T08:34:24.616 INFO:teuthology.orchestra.run.smithi082.stdout:(114/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-04-21T08:34:24.675 INFO:teuthology.orchestra.run.smithi082.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-04-21T08:34:24.699 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:34:24.708 INFO:teuthology.orchestra.run.smithi082.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-21T08:34:24.714 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:34:24.715 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:34:24.715 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T08:34:24.715 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:24.775 INFO:teuthology.orchestra.run.smithi082.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-21T08:34:24.825 INFO:teuthology.orchestra.run.smithi082.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-21T08:34:24.859 INFO:teuthology.orchestra.run.smithi082.stdout:(119/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 141 kB/s | 43 kB 00:00 2024-04-21T08:34:24.884 INFO:teuthology.orchestra.run.smithi082.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-21T08:34:24.918 INFO:teuthology.orchestra.run.smithi082.stdout:(121/150): python3-colorama-0.4.4-12.el9.noarch 383 kB/s | 35 kB 00:00 2024-04-21T08:34:25.026 INFO:teuthology.orchestra.run.smithi082.stdout:(122/150): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-21T08:34:25.038 INFO:teuthology.orchestra.run.smithi134.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 274 kB/s | 279 kB 00:01 2024-04-21T08:34:25.052 INFO:teuthology.orchestra.run.smithi082.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-21T08:34:25.119 INFO:teuthology.orchestra.run.smithi082.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-21T08:34:25.144 INFO:teuthology.orchestra.run.smithi082.stdout:(125/150): python3-google-auth-2.28.2-1.el9.noa 712 kB/s | 203 kB 00:00 2024-04-21T08:34:25.169 INFO:teuthology.orchestra.run.smithi082.stdout:(126/150): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-21T08:34:25.205 INFO:teuthology.orchestra.run.smithi134.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 78 kB/s | 51 kB 00:00 2024-04-21T08:34:25.211 INFO:teuthology.orchestra.run.smithi082.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-04-21T08:34:25.231 INFO:teuthology.orchestra.run.smithi134.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 82 kB/s | 54 kB 00:00 2024-04-21T08:34:25.236 INFO:teuthology.orchestra.run.smithi082.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-21T08:34:25.253 INFO:teuthology.orchestra.run.smithi082.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 234 kB/s | 20 kB 00:00 2024-04-21T08:34:25.320 INFO:teuthology.orchestra.run.smithi082.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-21T08:34:25.338 INFO:teuthology.orchestra.run.smithi082.stdout:(131/150): python3-logutils-0.3.5-21.el9.noarch 460 kB/s | 46 kB 00:00 2024-04-21T08:34:25.362 INFO:teuthology.orchestra.run.smithi082.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 725 kB/s | 79 kB 00:00 2024-04-21T08:34:25.404 INFO:teuthology.orchestra.run.smithi082.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 688 kB/s | 58 kB 00:00 2024-04-21T08:34:25.446 INFO:teuthology.orchestra.run.smithi082.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-04-21T08:34:25.496 INFO:teuthology.orchestra.run.smithi082.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 980 kB/s | 90 kB 00:00 2024-04-21T08:34:25.522 INFO:teuthology.orchestra.run.smithi082.stdout:(136/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-04-21T08:34:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-21T08:34:25.589 INFO:teuthology.orchestra.run.smithi082.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-04-21T08:34:25.639 INFO:teuthology.orchestra.run.smithi082.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-21T08:34:25.664 INFO:teuthology.orchestra.run.smithi082.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 504 kB/s | 59 kB 00:00 2024-04-21T08:34:25.690 INFO:teuthology.orchestra.run.smithi082.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-04-21T08:34:25.740 INFO:teuthology.orchestra.run.smithi082.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 506 kB/s | 51 kB 00:00 2024-04-21T08:34:25.774 INFO:teuthology.orchestra.run.smithi082.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-21T08:34:25.791 INFO:teuthology.orchestra.run.smithi082.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 891 kB/s | 90 kB 00:00 2024-04-21T08:34:25.849 INFO:teuthology.orchestra.run.smithi134.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 65 kB/s | 42 kB 00:00 2024-04-21T08:34:25.866 INFO:teuthology.orchestra.run.smithi082.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-04-21T08:34:25.892 INFO:teuthology.orchestra.run.smithi082.stdout:(146/150): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-04-21T08:34:25.898 INFO:teuthology.orchestra.run.smithi134.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 66 kB/s | 44 kB 00:00 2024-04-21T08:34:25.917 INFO:teuthology.orchestra.run.smithi082.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-04-21T08:34:26.084 INFO:teuthology.orchestra.run.smithi082.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-21T08:34:26.208 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:34:26.233 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:34:26.233 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:34:26.233 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T08:34:26.234 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:34:26.234 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:34:26.234 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:26.450 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:34:26.472 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:34:26.472 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:34:26.472 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T08:34:26.472 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:34:26.473 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:34:26.473 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:26.506 INFO:teuthology.orchestra.run.smithi134.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 59 kB/s | 36 kB 00:00 2024-04-21T08:34:26.525 INFO:teuthology.orchestra.run.smithi082.stdout:(149/150): librados2-19.0.0-3154.ga630765c.el9. 5.4 MB/s | 3.4 MB 00:00 2024-04-21T08:34:26.790 INFO:teuthology.orchestra.run.smithi134.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 234 kB/s | 221 kB 00:00 2024-04-21T08:34:26.918 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:34:26.940 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:34:26.940 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:34:26.941 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T08:34:26.941 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:34:26.941 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:34:26.941 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:27.042 INFO:teuthology.orchestra.run.smithi082.stdout:(150/150): librbd1-19.0.0-3154.ga630765c.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-21T08:34:27.045 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:34:27.045 INFO:teuthology.orchestra.run.smithi082.stdout:Total 4.1 MB/s | 218 MB 00:53 2024-04-21T08:34:27.048 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T08:34:27.104 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T08:34:27.140 INFO:teuthology.orchestra.run.smithi134.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 75 kB/s | 47 kB 00:00 2024-04-21T08:34:27.160 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T08:34:27.345 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T08:34:27.393 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T08:34:27.471 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T08:34:27.559 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T08:34:27.699 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T08:34:27.735 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T08:34:27.775 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T08:34:27.803 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T08:34:27.807 INFO:teuthology.orchestra.run.smithi134.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 87 kB/s | 58 kB 00:00 2024-04-21T08:34:27.840 INFO:teuthology.orchestra.run.smithi134.stdout:(98/150): socat-1.7.4.1-5.el9.x86_64.rpm 290 kB/s | 305 kB 00:01 2024-04-21T08:34:28.091 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-04-21T08:34:28.218 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-04-21T08:34:28.218 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-04-21T08:34:28.247 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:34:28.277 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:34:28.309 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T08:34:28.370 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T08:34:28.409 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:34:28.507 INFO:teuthology.orchestra.run.smithi134.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 134 kB/s | 94 kB 00:00 2024-04-21T08:34:28.533 INFO:teuthology.orchestra.run.smithi134.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 92 kB/s | 64 kB 00:00 2024-04-21T08:34:28.791 INFO:teuthology.orchestra.run.smithi134.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-04-21T08:34:29.150 INFO:teuthology.orchestra.run.smithi134.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-04-21T08:34:29.367 INFO:teuthology.orchestra.run.smithi134.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 2.5 MB/s | 548 kB 00:00 2024-04-21T08:34:29.439 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:34:29.442 INFO:teuthology.orchestra.run.smithi134.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-04-21T08:34:29.476 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:34:29.500 INFO:teuthology.orchestra.run.smithi134.stdout:(105/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-04-21T08:34:29.517 INFO:teuthology.orchestra.run.smithi134.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-04-21T08:34:29.568 INFO:teuthology.orchestra.run.smithi134.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-04-21T08:34:29.618 INFO:teuthology.orchestra.run.smithi134.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 974 kB/s | 49 kB 00:00 2024-04-21T08:34:29.676 INFO:teuthology.orchestra.run.smithi134.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-21T08:34:29.727 INFO:teuthology.orchestra.run.smithi134.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-21T08:34:29.852 INFO:teuthology.orchestra.run.smithi134.stdout:(111/150): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-21T08:34:29.894 INFO:teuthology.orchestra.run.smithi134.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-21T08:34:29.938 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-04-21T08:34:29.938 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-04-21T08:34:29.953 INFO:teuthology.orchestra.run.smithi134.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-04-21T08:34:29.978 INFO:teuthology.orchestra.run.smithi134.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 519 kB/s | 43 kB 00:00 2024-04-21T08:34:30.003 INFO:teuthology.orchestra.run.smithi134.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-04-21T08:34:30.028 INFO:teuthology.orchestra.run.smithi134.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-04-21T08:34:30.054 INFO:teuthology.orchestra.run.smithi134.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-21T08:34:30.079 INFO:teuthology.orchestra.run.smithi134.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-21T08:34:30.121 INFO:teuthology.orchestra.run.smithi134.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-04-21T08:34:30.147 INFO:teuthology.orchestra.run.smithi134.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 526 kB/s | 35 kB 00:00 2024-04-21T08:34:30.172 INFO:teuthology.orchestra.run.smithi134.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 3.9 MB/s | 203 kB 00:00 2024-04-21T08:34:30.255 INFO:teuthology.orchestra.run.smithi134.stdout:(122/150): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-04-21T08:34:30.280 INFO:teuthology.orchestra.run.smithi134.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-21T08:34:30.306 INFO:teuthology.orchestra.run.smithi134.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-21T08:34:30.331 INFO:teuthology.orchestra.run.smithi134.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-21T08:34:30.356 INFO:teuthology.orchestra.run.smithi134.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-04-21T08:34:30.382 INFO:teuthology.orchestra.run.smithi134.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-04-21T08:34:30.407 INFO:teuthology.orchestra.run.smithi134.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-04-21T08:34:30.439 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:34:30.457 INFO:teuthology.orchestra.run.smithi134.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-21T08:34:30.477 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:34:30.482 INFO:teuthology.orchestra.run.smithi134.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 614 kB/s | 46 kB 00:00 2024-04-21T08:34:30.508 INFO:teuthology.orchestra.run.smithi134.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-21T08:34:30.533 INFO:teuthology.orchestra.run.smithi134.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-21T08:34:30.558 INFO:teuthology.orchestra.run.smithi134.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-21T08:34:30.576 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:34:30.584 INFO:teuthology.orchestra.run.smithi134.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-21T08:34:30.609 INFO:teuthology.orchestra.run.smithi134.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-04-21T08:34:30.634 INFO:teuthology.orchestra.run.smithi134.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-04-21T08:34:30.659 INFO:teuthology.orchestra.run.smithi134.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-04-21T08:34:30.693 INFO:teuthology.orchestra.run.smithi134.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-21T08:34:30.718 INFO:teuthology.orchestra.run.smithi134.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-21T08:34:30.741 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T08:34:30.752 INFO:teuthology.orchestra.run.smithi134.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-04-21T08:34:30.770 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:34:30.777 INFO:teuthology.orchestra.run.smithi134.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 870 kB/s | 51 kB 00:00 2024-04-21T08:34:30.793 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:34:30.793 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:34:30.793 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T08:34:30.793 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:34:30.793 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:34:30.793 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:30.802 INFO:teuthology.orchestra.run.smithi134.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-21T08:34:30.828 INFO:teuthology.orchestra.run.smithi134.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-21T08:34:30.838 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:34:30.849 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:34:30.853 INFO:teuthology.orchestra.run.smithi134.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-04-21T08:34:30.878 INFO:teuthology.orchestra.run.smithi134.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-04-21T08:34:30.920 INFO:teuthology.orchestra.run.smithi134.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-21T08:34:30.946 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T08:34:30.979 INFO:teuthology.orchestra.run.smithi134.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-21T08:34:30.994 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T08:34:31.354 INFO:teuthology.orchestra.run.smithi134.stdout:(148/150): librbd1-19.0.0-3154.ga630765c.el9.x8 8.4 MB/s | 3.1 MB 00:00 2024-04-21T08:34:31.471 INFO:teuthology.orchestra.run.smithi134.stdout:(149/150): librados2-19.0.0-3154.ga630765c.el9. 6.2 MB/s | 3.4 MB 00:00 2024-04-21T08:34:31.921 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T08:34:31.965 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:34:31.988 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:34:31.988 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:34:31.988 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T08:34:31.988 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:34:31.988 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:34:31.988 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:32.033 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:34:32.054 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:34:32.055 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:34:32.055 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T08:34:32.055 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:32.100 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:34:32.122 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:34:32.122 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:34:32.122 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T08:34:32.123 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:34:32.123 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:34:32.123 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:32.426 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:34:32.448 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:34:32.448 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:34:32.448 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T08:34:32.448 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:34:32.448 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:34:32.448 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:35.729 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-04-21T08:34:35.810 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T08:34:35.835 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T08:34:35.878 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T08:34:36.207 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T08:34:36.320 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:34:36.441 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:34:36.470 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:34:36.562 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:34:36.612 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:34:36.642 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T08:34:36.667 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T08:34:36.826 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T08:34:36.881 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T08:34:36.909 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:34:36.996 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:34:37.018 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:34:37.046 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T08:34:37.073 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:34:37.088 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T08:34:37.094 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:34:37.120 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:34:37.127 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T08:34:37.160 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T08:34:37.187 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T08:34:37.311 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:34:37.344 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:34:37.368 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T08:34:37.426 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T08:34:37.452 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T08:34:37.455 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T08:34:37.542 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T08:34:37.585 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T08:34:37.620 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T08:34:37.694 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T08:34:37.736 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T08:34:37.779 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T08:34:37.811 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T08:34:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T08:34:37.910 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T08:34:37.963 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T08:34:37.992 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T08:34:38.026 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:34:38.029 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T08:34:38.070 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:34:38.090 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:34:38.105 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T08:34:38.138 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T08:34:38.182 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T08:34:38.344 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T08:34:38.475 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T08:34:38.517 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T08:34:38.594 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T08:34:38.731 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T08:34:38.787 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T08:34:38.863 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T08:34:38.949 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T08:34:38.990 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T08:34:39.074 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T08:34:39.103 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T08:34:39.103 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:34:39.187 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:34:39.187 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:34:39.328 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T08:34:39.417 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T08:34:39.452 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T08:34:39.495 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T08:34:39.533 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T08:34:39.567 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T08:34:39.622 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T08:34:39.696 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T08:34:39.733 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T08:34:39.770 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T08:34:39.818 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T08:34:39.853 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T08:34:39.897 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T08:34:39.939 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T08:34:40.078 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T08:34:40.128 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T08:34:40.158 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T08:34:40.199 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T08:34:40.295 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T08:34:40.340 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T08:34:40.372 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T08:34:40.398 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T08:34:40.422 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T08:34:40.498 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T08:34:40.627 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T08:34:40.663 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T08:34:40.693 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T08:34:40.725 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T08:34:40.770 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T08:34:40.807 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T08:34:40.847 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T08:34:40.877 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T08:34:40.903 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T08:34:40.936 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T08:34:40.978 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T08:34:41.013 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T08:34:41.102 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T08:34:41.160 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T08:34:41.191 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T08:34:41.239 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T08:34:41.348 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T08:34:41.410 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T08:34:42.159 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T08:34:42.207 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T08:34:42.240 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T08:34:42.342 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T08:34:42.390 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T08:34:42.479 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T08:34:42.525 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T08:34:42.702 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T08:34:42.731 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T08:34:42.809 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T08:34:43.474 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T08:34:43.590 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T08:34:44.858 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T08:34:44.943 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T08:34:44.972 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T08:34:45.004 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T08:34:45.056 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:34:45.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T08:34:45.056 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T08:34:45.057 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T08:34:45.058 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T08:34:45.059 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T08:34:45.081 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T08:34:45.081 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T08:34:45.081 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T08:34:45.081 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T08:34:45.082 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T08:34:45.082 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T08:34:45.082 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T08:34:45.082 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T08:34:45.082 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T08:34:45.082 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T08:34:45.083 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T08:34:45.083 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T08:34:45.083 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T08:34:45.083 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T08:34:45.083 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T08:34:45.084 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T08:34:45.084 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T08:34:45.084 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T08:34:45.084 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T08:34:45.084 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T08:34:45.084 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T08:34:45.085 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T08:34:45.085 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T08:34:45.085 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T08:34:45.085 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T08:34:45.085 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T08:34:45.085 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T08:34:45.086 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T08:34:45.086 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T08:34:45.086 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T08:34:45.086 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T08:34:45.086 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T08:34:45.087 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T08:34:45.087 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T08:34:45.087 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T08:34:45.087 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T08:34:45.087 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T08:34:45.088 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T08:34:45.088 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T08:34:45.088 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T08:34:45.088 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T08:34:45.088 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T08:34:45.089 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T08:34:45.089 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T08:34:45.089 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T08:34:45.089 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T08:34:45.089 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T08:34:45.090 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T08:34:45.090 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T08:34:45.090 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T08:34:45.090 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T08:34:45.091 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T08:34:45.091 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T08:34:45.091 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T08:34:45.091 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T08:34:45.091 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T08:34:45.092 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T08:34:45.092 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T08:34:45.092 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T08:34:45.092 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T08:34:45.092 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T08:34:45.093 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T08:34:45.093 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T08:34:45.093 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T08:34:45.093 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T08:34:45.093 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T08:34:45.094 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T08:34:45.094 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T08:34:45.094 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T08:34:45.094 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T08:34:45.094 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T08:34:45.095 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T08:34:45.095 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T08:34:45.095 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T08:34:45.095 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T08:34:45.095 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T08:34:45.096 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T08:34:45.096 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T08:34:45.096 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T08:34:45.096 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T08:34:45.096 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T08:34:45.097 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T08:34:45.097 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T08:34:45.097 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T08:34:45.097 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T08:34:45.097 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T08:34:45.097 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T08:34:45.098 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T08:34:45.098 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T08:34:45.098 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T08:34:45.098 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T08:34:45.098 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T08:34:45.099 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T08:34:45.099 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T08:34:45.099 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T08:34:45.099 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T08:34:45.099 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T08:34:45.099 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T08:34:45.100 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T08:34:45.100 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T08:34:45.100 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T08:34:45.100 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T08:34:45.100 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T08:34:45.100 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T08:34:45.101 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T08:34:45.101 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T08:34:45.101 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T08:34:45.101 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T08:34:45.101 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T08:34:45.101 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T08:34:45.102 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T08:34:45.102 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T08:34:45.102 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T08:34:45.102 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T08:34:45.102 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T08:34:45.102 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T08:34:45.103 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T08:34:45.103 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T08:34:45.103 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T08:34:45.103 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T08:34:45.103 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T08:34:45.103 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T08:34:45.316 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T08:34:45.350 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:34:45.433 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:34:45.464 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T08:34:45.500 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T08:34:45.770 INFO:teuthology.orchestra.run.smithi134.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 803 kB/s | 16 MB 00:20 2024-04-21T08:34:45.775 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:34:45.776 INFO:teuthology.orchestra.run.smithi134.stdout:Total 3.3 MB/s | 218 MB 01:05 2024-04-21T08:34:45.980 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T08:34:46.012 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:34:46.083 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:34:46.113 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T08:34:46.780 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-04-21T08:34:46.904 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-04-21T08:34:46.904 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-04-21T08:34:47.093 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:34:47.094 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:47.094 INFO:teuthology.orchestra.run.smithi106.stdout:Upgraded: 2024-04-21T08:34:47.094 INFO:teuthology.orchestra.run.smithi106.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.094 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.094 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2024-04-21T08:34:47.094 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T08:34:47.094 INFO:teuthology.orchestra.run.smithi106.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T08:34:47.094 INFO:teuthology.orchestra.run.smithi106.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T08:34:47.094 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T08:34:47.094 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.094 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.095 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T08:34:47.096 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T08:34:47.097 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T08:34:47.098 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T08:34:47.099 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T08:34:47.099 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T08:34:47.099 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T08:34:47.099 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T08:34:47.099 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T08:34:47.099 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T08:34:47.099 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T08:34:47.099 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T08:34:47.099 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T08:34:47.099 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T08:34:47.099 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T08:34:47.099 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T08:34:47.100 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T08:34:47.101 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T08:34:47.101 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T08:34:47.101 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T08:34:47.101 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T08:34:47.101 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T08:34:47.101 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T08:34:47.101 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T08:34:47.101 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T08:34:47.101 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T08:34:47.101 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T08:34:47.101 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T08:34:47.101 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T08:34:47.102 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T08:34:47.102 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T08:34:47.102 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T08:34:47.102 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T08:34:47.102 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T08:34:47.102 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T08:34:47.102 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T08:34:47.102 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T08:34:47.102 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T08:34:47.102 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T08:34:47.102 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T08:34:47.102 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T08:34:47.103 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T08:34:47.104 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T08:34:47.105 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T08:34:47.105 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.105 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.105 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:34:47.105 INFO:teuthology.orchestra.run.smithi106.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T08:34:47.105 INFO:teuthology.orchestra.run.smithi106.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T08:34:47.105 INFO:teuthology.orchestra.run.smithi106.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T08:34:47.105 INFO:teuthology.orchestra.run.smithi106.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T08:34:47.105 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T08:34:47.105 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T08:34:47.106 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T08:34:47.106 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-21T08:34:47.106 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-04-21T08:34:47.478 DEBUG:teuthology.parallel:result is None 2024-04-21T08:34:48.397 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:34:48.436 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:34:48.607 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-04-21T08:34:48.608 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-04-21T08:34:48.976 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:34:49.023 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:34:49.105 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:34:49.106 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T08:34:49.106 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T08:34:49.106 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:34:49.132 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:34:54.446 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-04-21T08:34:54.521 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T08:34:54.555 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T08:34:54.597 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T08:34:54.928 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T08:34:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:34:55.169 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:34:55.199 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:34:55.290 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:34:55.332 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:34:55.361 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T08:34:55.387 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T08:34:55.538 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T08:34:55.592 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T08:34:55.620 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:34:55.724 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:34:55.746 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:34:55.792 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:34:55.814 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:34:55.833 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:34:56.030 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:34:56.063 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:34:56.171 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T08:34:56.267 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T08:34:56.304 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T08:34:56.339 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T08:34:56.413 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T08:34:56.454 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T08:34:56.498 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T08:34:56.530 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T08:34:56.559 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T08:34:56.627 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T08:34:56.690 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T08:34:56.719 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T08:34:56.755 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T08:34:56.832 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T08:34:56.891 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T08:34:56.934 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T08:34:56.963 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T08:34:57.033 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T08:34:57.086 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T08:34:57.160 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T08:34:57.304 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T08:34:57.355 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T08:34:57.886 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T08:34:57.968 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T08:34:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T08:34:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T08:34:58.077 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T08:34:58.110 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T08:34:58.165 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T08:34:58.238 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T08:34:58.269 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T08:34:58.306 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T08:34:58.353 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T08:34:58.389 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T08:34:58.432 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T08:34:58.474 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T08:34:58.612 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T08:34:58.654 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T08:34:58.684 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T08:34:58.726 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T08:34:58.821 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T08:34:58.868 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T08:34:58.910 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T08:34:58.933 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T08:34:58.958 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T08:34:59.032 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T08:34:59.160 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T08:34:59.198 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T08:34:59.226 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T08:34:59.260 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T08:34:59.305 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T08:34:59.343 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T08:34:59.382 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T08:34:59.413 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T08:34:59.446 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T08:34:59.480 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T08:34:59.524 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T08:34:59.555 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T08:34:59.636 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T08:34:59.686 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T08:34:59.717 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T08:34:59.765 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T08:34:59.863 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T08:34:59.910 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T08:35:00.552 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:35:00.552 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2024-04-21T08:35:00.552 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2024-04-21T08:35:00.552 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2024-04-21T08:35:00.552 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2024-04-21T08:35:00.552 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2024-04-21T08:35:00.552 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /root 2024-04-21T08:35:00.552 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2024-04-21T08:35:00.552 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:35:00.654 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T08:35:00.699 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T08:35:00.726 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:35:00.741 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:35:00.741 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:00.741 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T08:35:00.741 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:35:00.758 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T08:35:00.858 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T08:35:00.908 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T08:35:00.995 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T08:35:01.052 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T08:35:01.229 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T08:35:01.276 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T08:35:01.361 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T08:35:02.016 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T08:35:02.124 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T08:35:02.220 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:35:02.245 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:35:02.245 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:02.246 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T08:35:02.246 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:35:02.246 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:35:02.246 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:35:02.462 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:35:02.484 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:35:02.485 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:02.485 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T08:35:02.485 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:35:02.485 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:35:02.485 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:35:02.938 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:35:02.960 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:35:02.960 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:02.960 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T08:35:02.961 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:35:02.961 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:35:02.961 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:35:03.068 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T08:35:03.126 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T08:35:03.181 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T08:35:03.368 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T08:35:03.377 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T08:35:03.414 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T08:35:03.469 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T08:35:03.492 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T08:35:03.497 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T08:35:03.530 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T08:35:03.582 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T08:35:03.739 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T08:35:03.781 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T08:35:03.812 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T08:35:03.836 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T08:35:03.840 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T08:35:03.877 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:35:03.968 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:35:04.006 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T08:35:04.043 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T08:35:04.292 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:35:04.347 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:35:04.379 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T08:35:04.441 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T08:35:04.470 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:35:04.524 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T08:35:04.555 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:35:04.617 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:35:04.640 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T08:35:05.508 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:35:05.547 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:35:06.507 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:35:06.538 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:35:06.642 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:35:06.811 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T08:35:06.840 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:35:06.864 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:35:06.864 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:06.864 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T08:35:06.864 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:35:06.864 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:35:06.864 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:35:06.908 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:35:06.919 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:35:06.942 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:35:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:35:07.033 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T08:35:07.082 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T08:35:07.527 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:35:07.572 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T08:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T08:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:07.698 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:35:08.008 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T08:35:08.035 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:35:08.058 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:35:08.058 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:08.058 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T08:35:08.058 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:35:08.058 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:35:08.058 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:35:08.102 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:35:08.124 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:35:08.124 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:08.125 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T08:35:08.125 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:35:08.178 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:35:08.201 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:35:08.201 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:08.201 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T08:35:08.201 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:35:08.201 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:35:08.201 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:35:08.504 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:35:08.526 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:35:08.526 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:08.526 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T08:35:08.526 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:35:08.526 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:35:08.527 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:35:13.115 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T08:35:13.156 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T08:35:13.198 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T08:35:13.232 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T08:35:13.257 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T08:35:13.447 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T08:35:13.495 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T08:35:13.525 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T08:35:14.146 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:35:14.180 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:35:14.200 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:35:14.981 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T08:35:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T08:35:15.109 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T08:35:15.191 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T08:35:15.239 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T08:35:15.239 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:35:15.314 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:35:15.314 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:35:19.142 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:35:19.142 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-04-21T08:35:19.142 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-04-21T08:35:19.143 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-04-21T08:35:19.143 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-04-21T08:35:19.143 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-04-21T08:35:19.143 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-04-21T08:35:19.143 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-04-21T08:35:19.143 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:19.334 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:35:19.349 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:35:19.349 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:19.349 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T08:35:19.349 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:20.821 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:35:20.846 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:35:20.846 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:20.847 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T08:35:20.847 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:35:20.847 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:35:20.847 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:21.071 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:35:21.075 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:35:21.075 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T08:35:21.075 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T08:35:21.075 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T08:35:21.075 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T08:35:21.075 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T08:35:21.075 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:35:21.075 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:35:21.075 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T08:35:21.075 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T08:35:21.075 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T08:35:21.075 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T08:35:21.076 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T08:35:21.077 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T08:35:21.077 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T08:35:21.077 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T08:35:21.077 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T08:35:21.077 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T08:35:21.077 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T08:35:21.088 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T08:35:21.089 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T08:35:21.089 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T08:35:21.089 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T08:35:21.089 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T08:35:21.089 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T08:35:21.089 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T08:35:21.089 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T08:35:21.089 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T08:35:21.089 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T08:35:21.089 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T08:35:21.089 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T08:35:21.090 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T08:35:21.090 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T08:35:21.090 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T08:35:21.090 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T08:35:21.090 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T08:35:21.090 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T08:35:21.090 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T08:35:21.090 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T08:35:21.090 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T08:35:21.090 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T08:35:21.090 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T08:35:21.090 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T08:35:21.091 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T08:35:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T08:35:21.093 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T08:35:21.094 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T08:35:21.095 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T08:35:21.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T08:35:21.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T08:35:21.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T08:35:21.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T08:35:21.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T08:35:21.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T08:35:21.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T08:35:21.098 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:35:21.098 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:21.098 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T08:35:21.098 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:35:21.099 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:35:21.099 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:21.538 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:35:21.561 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:35:21.561 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:21.561 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T08:35:21.561 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:35:21.561 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:35:21.561 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:21.677 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T08:35:21.734 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T08:35:21.789 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T08:35:21.972 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T08:35:22.023 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T08:35:22.108 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T08:35:22.204 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T08:35:22.343 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T08:35:22.381 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T08:35:22.412 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T08:35:22.450 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T08:35:22.900 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:35:22.931 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:35:22.963 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T08:35:23.024 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T08:35:23.054 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:35:23.233 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:35:23.233 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:35:23.233 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-04-21T08:35:23.233 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.233 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.233 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-04-21T08:35:23.233 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T08:35:23.234 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T08:35:23.234 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T08:35:23.234 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T08:35:23.234 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.234 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.234 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.234 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.234 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:23.234 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.234 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.234 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.234 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:23.235 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.236 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.236 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:23.236 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T08:35:23.236 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T08:35:23.236 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T08:35:23.236 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T08:35:23.236 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T08:35:23.236 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T08:35:23.236 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T08:35:23.236 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T08:35:23.236 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T08:35:23.236 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T08:35:23.237 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T08:35:23.237 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.237 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.237 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.237 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T08:35:23.237 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T08:35:23.237 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T08:35:23.237 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T08:35:23.237 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T08:35:23.237 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.237 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.237 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T08:35:23.238 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.238 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T08:35:23.238 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T08:35:23.238 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T08:35:23.238 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T08:35:23.238 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T08:35:23.238 INFO:teuthology.orchestra.run.smithi082.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T08:35:23.238 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T08:35:23.238 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T08:35:23.238 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T08:35:23.238 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T08:35:23.238 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.239 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T08:35:23.240 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T08:35:23.241 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T08:35:23.242 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T08:35:23.243 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T08:35:23.244 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T08:35:23.245 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T08:35:23.245 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T08:35:23.245 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T08:35:23.245 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T08:35:23.245 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T08:35:23.245 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:35:23.245 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-04-21T08:35:23.604 DEBUG:teuthology.parallel:result is None 2024-04-21T08:35:24.093 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:35:24.130 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:35:25.100 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:35:25.148 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:35:25.248 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:35:25.410 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T08:35:25.441 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:35:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:35:25.464 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:25.464 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T08:35:25.464 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:35:25.464 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:35:25.464 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:25.509 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:35:25.519 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:35:25.617 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T08:35:25.665 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T08:35:26.601 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T08:35:26.636 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:35:26.658 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:35:26.659 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:26.659 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T08:35:26.659 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:35:26.659 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:35:26.659 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:26.704 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:35:26.725 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:35:26.725 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:26.725 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T08:35:26.725 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:26.779 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:35:26.801 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:35:26.801 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:26.801 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T08:35:26.801 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:35:26.801 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:35:26.801 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:27.105 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:35:27.127 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:35:27.127 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:35:27.127 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T08:35:27.127 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:35:27.127 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:35:27.127 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:31.733 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T08:35:31.774 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T08:35:31.815 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T08:35:31.847 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T08:35:31.875 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T08:35:32.056 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T08:35:32.104 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T08:35:32.134 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T08:35:32.768 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:35:32.807 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:35:32.826 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:35:33.598 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T08:35:33.685 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T08:35:33.727 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T08:35:33.807 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T08:35:33.840 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T08:35:33.840 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:35:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:35:33.924 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:35:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:35:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T08:35:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T08:35:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T08:35:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T08:35:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T08:35:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T08:35:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T08:35:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T08:35:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T08:35:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T08:35:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T08:35:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T08:35:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T08:35:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T08:35:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T08:35:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T08:35:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T08:35:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T08:35:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T08:35:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T08:35:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T08:35:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T08:35:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T08:35:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T08:35:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T08:35:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T08:35:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T08:35:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T08:35:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T08:35:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T08:35:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T08:35:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T08:35:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T08:35:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T08:35:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T08:35:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T08:35:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T08:35:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T08:35:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T08:35:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T08:35:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T08:35:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T08:35:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T08:35:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T08:35:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T08:35:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T08:35:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T08:35:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T08:35:42.162 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:35:42.162 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:42.162 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-04-21T08:35:42.162 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.163 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:42.164 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T08:35:42.165 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.166 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.166 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.166 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T08:35:42.166 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T08:35:42.166 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T08:35:42.166 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T08:35:42.166 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T08:35:42.166 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.166 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.166 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T08:35:42.166 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.166 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T08:35:42.167 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T08:35:42.168 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T08:35:42.169 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T08:35:42.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T08:35:42.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T08:35:42.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T08:35:42.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T08:35:42.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T08:35:42.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T08:35:42.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T08:35:42.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T08:35:42.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T08:35:42.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T08:35:42.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T08:35:42.170 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T08:35:42.171 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T08:35:42.172 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T08:35:42.173 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T08:35:42.174 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T08:35:42.174 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T08:35:42.174 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T08:35:42.174 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:35:42.174 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-04-21T08:35:42.559 DEBUG:teuthology.parallel:result is None 2024-04-21T08:35:42.560 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:35:42.797 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T08:35:42.823 INFO:teuthology.orchestra.run.smithi082.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T08:35:42.823 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T08:35:42.823 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T08:35:42.825 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:35:42.976 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T08:35:43.001 INFO:teuthology.orchestra.run.smithi106.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T08:35:43.001 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T08:35:43.001 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T08:35:43.003 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:35:43.145 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T08:35:43.166 INFO:teuthology.orchestra.run.smithi134.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T08:35:43.166 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T08:35:43.167 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T08:35:43.168 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-21T08:35:43.168 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:35:43.168 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T08:35:43.199 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-21T08:35:43.200 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T08:35:43.230 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-21T08:35:43.230 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T08:35:43.255 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-21T08:35:43.256 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:35:43.256 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T08:35:43.283 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T08:35:43.350 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-21T08:35:43.350 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T08:35:43.378 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T08:35:43.445 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-21T08:35:43.445 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T08:35:43.468 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T08:35:43.532 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-21T08:35:43.532 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:35:43.532 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T08:35:43.559 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T08:35:43.625 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-21T08:35:43.625 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T08:35:43.655 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T08:35:43.720 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-21T08:35:43.720 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T08:35:43.743 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T08:35:43.805 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-21T08:35:43.806 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:35:43.806 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T08:35:43.833 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T08:35:43.899 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-21T08:35:43.899 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T08:35:43.922 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T08:35:43.986 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-21T08:35:43.986 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T08:35:44.010 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T08:35:44.073 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-21T08:35:44.169 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-21T08:35:44.169 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:35:44.170 INFO:tasks.cephadm:Cluster fsid is 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:35:44.170 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-21T08:35:44.170 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.82', 'mon.b': '172.21.15.106', 'mon.c': '172.21.15.134'} 2024-04-21T08:35:44.170 INFO:tasks.cephadm:First mon is mon.a on smithi082 2024-04-21T08:35:44.170 INFO:tasks.cephadm:First mgr is x 2024-04-21T08:35:44.170 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-21T08:35:44.170 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname $(hostname -s) 2024-04-21T08:35:44.199 DEBUG:teuthology.orchestra.run.smithi106:> sudo hostname $(hostname -s) 2024-04-21T08:35:44.222 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2024-04-21T08:35:44.246 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-21T08:35:44.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:35:44.463 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78740/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3154-ga630765c', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3154.ga630765c'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-19 11:13:53.331234', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-21T08:35:44.609 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 a630765c1f93c6e61d9b40c4b76437611d5ef29e from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:35:44.611 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm 2024-04-21T08:35:44.611 DEBUG:teuthology.orchestra.run.smithi082:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T08:35:45.067 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 08:35 /home/ubuntu/cephtest/cephadm 2024-04-21T08:35:45.067 DEBUG:teuthology.orchestra.run.smithi106:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T08:35:45.688 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 08:35 /home/ubuntu/cephtest/cephadm 2024-04-21T08:35:45.688 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T08:35:46.157 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 08:35 /home/ubuntu/cephtest/cephadm 2024-04-21T08:35:46.157 DEBUG:teuthology.orchestra.run.smithi082:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T08:35:46.175 DEBUG:teuthology.orchestra.run.smithi106:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T08:35:46.194 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T08:35:46.229 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e on all hosts... 2024-04-21T08:35:46.229 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T08:35:46.231 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T08:35:46.236 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T08:35:47.748 INFO:teuthology.orchestra.run.smithi106.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:35:47.862 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:35:47.934 INFO:teuthology.orchestra.run.smithi082.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:36:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:36:15.505 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:36:15.506 INFO:teuthology.orchestra.run.smithi082.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T08:36:15.506 INFO:teuthology.orchestra.run.smithi082.stdout: "repo_digests": [ 2024-04-21T08:36:15.506 INFO:teuthology.orchestra.run.smithi082.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T08:36:15.506 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-21T08:36:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:36:15.558 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-21T08:36:15.558 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:36:15.558 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T08:36:15.558 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2024-04-21T08:36:15.558 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T08:36:15.559 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-04-21T08:36:15.559 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-21T08:36:15.833 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-21T08:36:15.834 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:36:15.834 INFO:teuthology.orchestra.run.smithi106.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T08:36:15.834 INFO:teuthology.orchestra.run.smithi106.stdout: "repo_digests": [ 2024-04-21T08:36:15.834 INFO:teuthology.orchestra.run.smithi106.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T08:36:15.834 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-21T08:36:15.834 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-21T08:36:15.860 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph 2024-04-21T08:36:15.892 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph 2024-04-21T08:36:15.919 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2024-04-21T08:36:15.945 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /etc/ceph 2024-04-21T08:36:15.972 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /etc/ceph 2024-04-21T08:36:15.996 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2024-04-21T08:36:16.019 INFO:tasks.cephadm:Writing seed config... 2024-04-21T08:36:16.020 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-21T08:36:16.020 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-21T08:36:16.020 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-21T08:36:16.021 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-21T08:36:16.022 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-21T08:36:16.023 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-21T08:36:16.024 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:36:16.024 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-21T08:36:16.041 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 24d05dee-ffba-11ee-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-21T08:36:16.042 DEBUG:teuthology.orchestra.run.smithi082:mon.a> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.a.service 2024-04-21T08:36:16.085 DEBUG:teuthology.orchestra.run.smithi082:mgr.x> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.x.service 2024-04-21T08:36:16.128 INFO:tasks.cephadm:Bootstrapping... 2024-04-21T08:36:16.128 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e -v bootstrap --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.82 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:36:16.324 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:36:16.324 INFO:teuthology.orchestra.run.smithi082.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e', '-v', 'bootstrap', '--fsid', '24d05dee-ffba-11ee-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.82', '--skip-admin-label'] 2024-04-21T08:36:16.325 INFO:teuthology.orchestra.run.smithi082.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-21T08:36:16.325 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying podman|docker is present... 2024-04-21T08:36:16.351 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-21T08:36:16.352 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying lvm2 is present... 2024-04-21T08:36:16.352 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying time synchronization is in place... 2024-04-21T08:36:16.359 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-21T08:36:16.359 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-21T08:36:16.365 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-21T08:36:16.365 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-04-21T08:36:16.372 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout enabled 2024-04-21T08:36:16.378 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout active 2024-04-21T08:36:16.378 INFO:teuthology.orchestra.run.smithi082.stdout:Unit chronyd.service is enabled and running 2024-04-21T08:36:16.378 INFO:teuthology.orchestra.run.smithi082.stdout:Repeating the final host check... 2024-04-21T08:36:16.406 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-21T08:36:16.406 INFO:teuthology.orchestra.run.smithi082.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-21T08:36:16.407 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl is present 2024-04-21T08:36:16.407 INFO:teuthology.orchestra.run.smithi082.stdout:lvcreate is present 2024-04-21T08:36:16.412 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-21T08:36:16.412 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-21T08:36:16.418 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-21T08:36:16.418 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-04-21T08:36:16.425 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout enabled 2024-04-21T08:36:16.432 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout active 2024-04-21T08:36:16.432 INFO:teuthology.orchestra.run.smithi082.stdout:Unit chronyd.service is enabled and running 2024-04-21T08:36:16.432 INFO:teuthology.orchestra.run.smithi082.stdout:Host looks OK 2024-04-21T08:36:16.432 INFO:teuthology.orchestra.run.smithi082.stdout:Cluster fsid: 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:36:16.433 INFO:teuthology.orchestra.run.smithi082.stdout:Acquiring lock 140132498820256 on /run/cephadm/24d05dee-ffba-11ee-bc93-c7b262605968.lock 2024-04-21T08:36:16.433 INFO:teuthology.orchestra.run.smithi082.stdout:Lock 140132498820256 acquired on /run/cephadm/24d05dee-ffba-11ee-bc93-c7b262605968.lock 2024-04-21T08:36:16.433 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying IP 172.21.15.82 port 3300 ... 2024-04-21T08:36:16.433 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying IP 172.21.15.82 port 6789 ... 2024-04-21T08:36:16.433 INFO:teuthology.orchestra.run.smithi082.stdout:Base mon IP(s) is [172.21.15.82:3300, 172.21.15.82:6789], mon addrv is [v2:172.21.15.82:3300,v1:172.21.15.82:6789] 2024-04-21T08:36:16.438 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.82 metric 100 2024-04-21T08:36:16.438 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.82 metric 100 2024-04-21T08:36:16.441 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-21T08:36:16.441 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-21T08:36:16.441 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-21T08:36:16.446 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-21T08:36:16.447 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-21T08:36:16.447 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-21T08:36:16.447 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-21T08:36:16.447 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout inet6 fe80::8fde:e834:f54e:2fd3/64 scope link noprefixroute 2024-04-21T08:36:16.447 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-21T08:36:16.447 INFO:teuthology.orchestra.run.smithi082.stdout:Mon IP `172.21.15.82` is in CIDR network `172.21.0.0/20` 2024-04-21T08:36:16.447 INFO:teuthology.orchestra.run.smithi082.stdout:Mon IP `172.21.15.82` is in CIDR network `172.21.0.0/20` 2024-04-21T08:36:16.447 INFO:teuthology.orchestra.run.smithi082.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-21T08:36:16.447 INFO:teuthology.orchestra.run.smithi082.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-21T08:36:16.448 INFO:teuthology.orchestra.run.smithi082.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:36:16.939 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 2024-04-21T08:36:16.940 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:36:16.940 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Getting image source signatures 2024-04-21T08:36:16.940 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying blob sha256:7eab8a2da10d06ac88aea1fd8998a2559b4de9d07c585e3b1e6ee6c4d35fc119 2024-04-21T08:36:16.940 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-21T08:36:16.940 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying config sha256:b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 2024-04-21T08:36:16.940 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-21T08:36:17.839 INFO:teuthology.orchestra.run.smithi082.stdout:ceph: stdout ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T08:36:17.839 INFO:teuthology.orchestra.run.smithi082.stdout:Ceph version: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T08:36:17.839 INFO:teuthology.orchestra.run.smithi082.stdout:Extracting ceph user uid/gid from container image... 2024-04-21T08:36:18.859 INFO:teuthology.orchestra.run.smithi082.stdout:stat: stdout 167 167 2024-04-21T08:36:18.860 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial keys... 2024-04-21T08:36:19.851 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQAD0CRmUcYnDhAAwiYjw7IXzo4sislIaB1NNQ== 2024-04-21T08:36:20.742 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQAE0CRmHnhMCRAAFoa99cwOh7Ccg7pMVNILDA== 2024-04-21T08:36:21.699 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQAF0CRmIVklBhAAMmqndWfnzO9XjiXw6qp6EA== 2024-04-21T08:36:21.699 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial monmap... 2024-04-21T08:36:22.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-21T08:36:22.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-21T08:36:22.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:36:22.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-21T08:36:22.635 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool for a [v2:172.21.15.82:3300,v1:172.21.15.82:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-21T08:36:22.635 INFO:teuthology.orchestra.run.smithi082.stdout:setting min_mon_release = quincy 2024-04-21T08:36:22.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: set fsid to 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:36:22.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-21T08:36:22.635 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:36:22.635 INFO:teuthology.orchestra.run.smithi082.stdout:Creating mon... 2024-04-21T08:36:23.608 INFO:teuthology.orchestra.run.smithi082.stdout:create mon.a on 2024-04-21T08:36:23.823 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-21T08:36:23.961 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-21T08:36:24.142 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-24d05dee-ffba-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-24d05dee-ffba-11ee-bc93-c7b262605968.target. 2024-04-21T08:36:24.142 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-24d05dee-ffba-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-24d05dee-ffba-11ee-bc93-c7b262605968.target. 2024-04-21T08:36:24.383 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.a 2024-04-21T08:36:24.383 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to reset failed state of unit ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.a.service: Unit ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.a.service not loaded. 2024-04-21T08:36:24.540 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-24d05dee-ffba-11ee-bc93-c7b262605968.target.wants/ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-24d05dee-ffba-11ee-bc93-c7b262605968@.service. 2024-04-21T08:36:25.092 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:36:25.092 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-04-21T08:36:25.098 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:36:25.099 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-04-21T08:36:25.099 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-04-21T08:36:25.099 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to enable service . firewalld.service is not available 2024-04-21T08:36:25.099 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mon to start... 2024-04-21T08:36:25.099 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mon... 2024-04-21T08:36:26.162 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:26 smithi082 ceph-mon[27598]: from='client.? 172.21.15.82:0/517016242' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-21T08:36:26.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout cluster: 2024-04-21T08:36:26.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout id: 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:36:26.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-21T08:36:26.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:26.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout services: 2024-04-21T08:36:26.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.560258s) 2024-04-21T08:36:26.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-21T08:36:26.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-21T08:36:26.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:26.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout data: 2024-04-21T08:36:26.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-21T08:36:26.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-21T08:36:26.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout pgs: 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.528+0000 7f091bcd1700 1 Processor -- start 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.529+0000 7f091bcd1700 1 -- start start 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.529+0000 7f091bcd1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 0x7f0914106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.529+0000 7f091bcd1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0914107050 con 0x7f0914106660 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.529+0000 7f0919a6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 0x7f0914106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.529+0000 7f0919a6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 0x7f0914106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51972/0 (socket says 172.21.15.82:51972) 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.529+0000 7f0919a6d700 1 -- 172.21.15.82:0/2443161352 learned_addr learned my addr 172.21.15.82:0/2443161352 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.529+0000 7f0919a6d700 1 -- 172.21.15.82:0/2443161352 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09141095f0 con 0x7f0914106660 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.530+0000 7f0919a6d700 1 --2- 172.21.15.82:0/2443161352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 0x7f0914106a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f09040098a0 tx=0x7f0904009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a4c838f21a7a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.530+0000 7f0918a6b700 1 -- 172.21.15.82:0/2443161352 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0904004160 con 0x7f0914106660 2024-04-21T08:36:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.530+0000 7f0918a6b700 1 -- 172.21.15.82:0/2443161352 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0904004300 con 0x7f0914106660 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.530+0000 7f0918a6b700 1 -- 172.21.15.82:0/2443161352 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09040036a0 con 0x7f0914106660 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.530+0000 7f091bcd1700 1 -- 172.21.15.82:0/2443161352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 msgr2=0x7f0914106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.530+0000 7f091bcd1700 1 --2- 172.21.15.82:0/2443161352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 0x7f0914106a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f09040098a0 tx=0x7f0904009bb0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.530+0000 7f091bcd1700 1 -- 172.21.15.82:0/2443161352 shutdown_connections 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.530+0000 7f091bcd1700 1 --2- 172.21.15.82:0/2443161352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 0x7f0914106a80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.530+0000 7f091bcd1700 1 -- 172.21.15.82:0/2443161352 >> 172.21.15.82:0/2443161352 conn(0x7f0914101fb0 msgr2=0x7f09141043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.530+0000 7f091bcd1700 1 -- 172.21.15.82:0/2443161352 shutdown_connections 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.530+0000 7f091bcd1700 1 -- 172.21.15.82:0/2443161352 wait complete. 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.531+0000 7f091bcd1700 1 Processor -- start 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.531+0000 7f091bcd1700 1 -- start start 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.531+0000 7f091bcd1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 0x7f091419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.531+0000 7f091bcd1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0914109b20 con 0x7f0914106660 2024-04-21T08:36:26.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.531+0000 7f0919a6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 0x7f091419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.531+0000 7f0919a6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 0x7f091419b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51978/0 (socket says 172.21.15.82:51978) 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.531+0000 7f0919a6d700 1 -- 172.21.15.82:0/517016242 learned_addr learned my addr 172.21.15.82:0/517016242 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.531+0000 7f0919a6d700 1 -- 172.21.15.82:0/517016242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091419bec0 con 0x7f0914106660 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.532+0000 7f0919a6d700 1 --2- 172.21.15.82:0/517016242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 0x7f091419b980 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0904000c00 tx=0x7f0904004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.532+0000 7f090affd700 1 -- 172.21.15.82:0/517016242 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0904003db0 con 0x7f0914106660 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.532+0000 7f090affd700 1 -- 172.21.15.82:0/517016242 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0904011630 con 0x7f0914106660 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.532+0000 7f091bcd1700 1 -- 172.21.15.82:0/517016242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091419c150 con 0x7f0914106660 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.532+0000 7f090affd700 1 -- 172.21.15.82:0/517016242 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0904011950 con 0x7f0914106660 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.532+0000 7f091bcd1700 1 -- 172.21.15.82:0/517016242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091419ee40 con 0x7f0914106660 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.532+0000 7f090affd700 1 -- 172.21.15.82:0/517016242 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0904011af0 con 0x7f0914106660 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.532+0000 7f090affd700 1 -- 172.21.15.82:0/517016242 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f090401c450 con 0x7f0914106660 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.533+0000 7f091bcd1700 1 -- 172.21.15.82:0/517016242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09140640a0 con 0x7f0914106660 2024-04-21T08:36:26.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.535+0000 7f090affd700 1 -- 172.21.15.82:0/517016242 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0904032020 con 0x7f0914106660 2024-04-21T08:36:26.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.605+0000 7f091bcd1700 1 -- 172.21.15.82:0/517016242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f091402e100 con 0x7f0914106660 2024-04-21T08:36:26.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.605+0000 7f090affd700 1 -- 172.21.15.82:0/517016242 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f0904004260 con 0x7f0914106660 2024-04-21T08:36:26.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.606+0000 7f091bcd1700 1 -- 172.21.15.82:0/517016242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 msgr2=0x7f091419b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.606+0000 7f091bcd1700 1 --2- 172.21.15.82:0/517016242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 0x7f091419b980 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0904000c00 tx=0x7f0904004750 comp rx=0 tx=0).stop 2024-04-21T08:36:26.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.606+0000 7f091bcd1700 1 -- 172.21.15.82:0/517016242 shutdown_connections 2024-04-21T08:36:26.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.606+0000 7f091bcd1700 1 --2- 172.21.15.82:0/517016242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0914106660 0x7f091419b980 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.606+0000 7f091bcd1700 1 -- 172.21.15.82:0/517016242 >> 172.21.15.82:0/517016242 conn(0x7f0914101fb0 msgr2=0x7f0914192660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:26.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.606+0000 7f091bcd1700 1 -- 172.21.15.82:0/517016242 shutdown_connections 2024-04-21T08:36:26.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:25.606+0000 7f091bcd1700 1 -- 172.21.15.82:0/517016242 wait complete. 2024-04-21T08:36:26.174 INFO:teuthology.orchestra.run.smithi082.stdout:mon is available 2024-04-21T08:36:26.174 INFO:teuthology.orchestra.run.smithi082.stdout:Assimilating anything we can from ceph.conf... 2024-04-21T08:36:27.271 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:27 smithi082 ceph-mon[27598]: from='client.? 172.21.15.82:0/2315263826' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-21T08:36:27.271 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:27 smithi082 ceph-mon[27598]: from='client.? 172.21.15.82:0/2315263826' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-21T08:36:27.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:27.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [global] 2024-04-21T08:36:27.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout fsid = 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:36:27.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.82:3300,v1:172.21.15.82:6789] 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [osd] 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-21T08:36:27.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.604+0000 7f5111ff0700 1 Processor -- start 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.604+0000 7f5111ff0700 1 -- start start 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.604+0000 7f5111ff0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 0x7f510c10b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.604+0000 7f5111ff0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f510c07bdc0 con 0x7f510c108f50 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.604+0000 7f510b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 0x7f510c10b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.604+0000 7f510b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 0x7f510c10b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51984/0 (socket says 172.21.15.82:51984) 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.604+0000 7f510b7fe700 1 -- 172.21.15.82:0/1427033170 learned_addr learned my addr 172.21.15.82:0/1427033170 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.605+0000 7f510b7fe700 1 -- 172.21.15.82:0/1427033170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c10b880 con 0x7f510c108f50 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.605+0000 7f510b7fe700 1 --2- 172.21.15.82:0/1427033170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 0x7f510c10b340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0098a0 tx=0x7f50fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aabfeff19368c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.605+0000 7f510a7fc700 1 -- 172.21.15.82:0/1427033170 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50fc004160 con 0x7f510c108f50 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.605+0000 7f510a7fc700 1 -- 172.21.15.82:0/1427033170 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50fc004300 con 0x7f510c108f50 2024-04-21T08:36:27.281 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.605+0000 7f510a7fc700 1 -- 172.21.15.82:0/1427033170 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50fc0036a0 con 0x7f510c108f50 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.605+0000 7f5111ff0700 1 -- 172.21.15.82:0/1427033170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 msgr2=0x7f510c10b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.605+0000 7f5111ff0700 1 --2- 172.21.15.82:0/1427033170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 0x7f510c10b340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0098a0 tx=0x7f50fc009bb0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.606+0000 7f5111ff0700 1 -- 172.21.15.82:0/1427033170 shutdown_connections 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.606+0000 7f5111ff0700 1 --2- 172.21.15.82:0/1427033170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 0x7f510c10b340 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.606+0000 7f5111ff0700 1 -- 172.21.15.82:0/1427033170 >> 172.21.15.82:0/1427033170 conn(0x7f510c102920 msgr2=0x7f510c104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.606+0000 7f5111ff0700 1 -- 172.21.15.82:0/1427033170 shutdown_connections 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.606+0000 7f5111ff0700 1 -- 172.21.15.82:0/1427033170 wait complete. 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.606+0000 7f5111ff0700 1 Processor -- start 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.606+0000 7f5111ff0700 1 -- start start 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f5111ff0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 0x7f510c19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f5111ff0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f510c10bc90 con 0x7f510c108f50 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f510b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 0x7f510c19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.282 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f510b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 0x7f510c19b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51996/0 (socket says 172.21.15.82:51996) 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f510b7fe700 1 -- 172.21.15.82:0/2315263826 learned_addr learned my addr 172.21.15.82:0/2315263826 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f510b7fe700 1 -- 172.21.15.82:0/2315263826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c19bf30 con 0x7f510c108f50 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f510b7fe700 1 --2- 172.21.15.82:0/2315263826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 0x7f510c19b9f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f50fc000c00 tx=0x7f50fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f5108ff9700 1 -- 172.21.15.82:0/2315263826 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50fc003db0 con 0x7f510c108f50 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f5108ff9700 1 -- 172.21.15.82:0/2315263826 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50fc011630 con 0x7f510c108f50 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f5111ff0700 1 -- 172.21.15.82:0/2315263826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c19c1c0 con 0x7f510c108f50 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f5108ff9700 1 -- 172.21.15.82:0/2315263826 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50fc011950 con 0x7f510c108f50 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f5111ff0700 1 -- 172.21.15.82:0/2315263826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c19ee60 con 0x7f510c108f50 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f5108ff9700 1 -- 172.21.15.82:0/2315263826 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f50fc01a460 con 0x7f510c108f50 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.607+0000 7f5108ff9700 1 -- 172.21.15.82:0/2315263826 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f50fc01b780 con 0x7f510c108f50 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.608+0000 7f5111ff0700 1 -- 172.21.15.82:0/2315263826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c0640a0 con 0x7f510c108f50 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.610+0000 7f5108ff9700 1 -- 172.21.15.82:0/2315263826 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f50fc031020 con 0x7f510c108f50 2024-04-21T08:36:27.283 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.679+0000 7f5111ff0700 1 -- 172.21.15.82:0/2315263826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f510c02e0d0 con 0x7f510c108f50 2024-04-21T08:36:27.284 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.682+0000 7f5108ff9700 1 -- 172.21.15.82:0/2315263826 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f50fc01b980 con 0x7f510c108f50 2024-04-21T08:36:27.284 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.682+0000 7f5108ff9700 1 -- 172.21.15.82:0/2315263826 <== mon.0 v2:172.21.15.82:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f50fc01a890 con 0x7f510c108f50 2024-04-21T08:36:27.284 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.683+0000 7f5111ff0700 1 -- 172.21.15.82:0/2315263826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 msgr2=0x7f510c19b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.284 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.683+0000 7f5111ff0700 1 --2- 172.21.15.82:0/2315263826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 0x7f510c19b9f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f50fc000c00 tx=0x7f50fc004750 comp rx=0 tx=0).stop 2024-04-21T08:36:27.284 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.683+0000 7f5111ff0700 1 -- 172.21.15.82:0/2315263826 shutdown_connections 2024-04-21T08:36:27.284 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.683+0000 7f5111ff0700 1 --2- 172.21.15.82:0/2315263826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c108f50 0x7f510c19b9f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.284 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.683+0000 7f5111ff0700 1 -- 172.21.15.82:0/2315263826 >> 172.21.15.82:0/2315263826 conn(0x7f510c102920 msgr2=0x7f510c1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.284 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.683+0000 7f5111ff0700 1 -- 172.21.15.82:0/2315263826 shutdown_connections 2024-04-21T08:36:27.284 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:26.683+0000 7f5111ff0700 1 -- 172.21.15.82:0/2315263826 wait complete. 2024-04-21T08:36:27.284 INFO:teuthology.orchestra.run.smithi082.stdout:Generating new minimal ceph.conf... 2024-04-21T08:36:28.329 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:28 smithi082 ceph-mon[27598]: from='client.? 172.21.15.82:0/1314368868' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:28.332 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.658+0000 7efea3f31700 1 Processor -- start 2024-04-21T08:36:28.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.658+0000 7efea3f31700 1 -- start start 2024-04-21T08:36:28.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.658+0000 7efea3f31700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 0x7efe9c108820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:28.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.658+0000 7efea3f31700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c108df0 con 0x7efe9c108400 2024-04-21T08:36:28.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.659+0000 7efea1ccd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 0x7efe9c108820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:28.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.659+0000 7efea1ccd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 0x7efe9c108820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51998/0 (socket says 172.21.15.82:51998) 2024-04-21T08:36:28.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.659+0000 7efea1ccd700 1 -- 172.21.15.82:0/400099601 learned_addr learned my addr 172.21.15.82:0/400099601 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:28.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.659+0000 7efea1ccd700 1 -- 172.21.15.82:0/400099601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9c109560 con 0x7efe9c108400 2024-04-21T08:36:28.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.659+0000 7efea1ccd700 1 --2- 172.21.15.82:0/400099601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 0x7efe9c108820 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe94009e80 tx=0x7efe9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e8ae74f489a669 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:28.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.659+0000 7efea0ccb700 1 -- 172.21.15.82:0/400099601 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe94017070 con 0x7efe9c108400 2024-04-21T08:36:28.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.659+0000 7efea0ccb700 1 -- 172.21.15.82:0/400099601 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efe9400b8c0 con 0x7efe9c108400 2024-04-21T08:36:28.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.659+0000 7efea0ccb700 1 -- 172.21.15.82:0/400099601 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe94004490 con 0x7efe9c108400 2024-04-21T08:36:28.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.660+0000 7efea3f31700 1 -- 172.21.15.82:0/400099601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 msgr2=0x7efe9c108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:28.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.660+0000 7efea3f31700 1 --2- 172.21.15.82:0/400099601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 0x7efe9c108820 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe94009e80 tx=0x7efe9400b330 comp rx=0 tx=0).stop 2024-04-21T08:36:28.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.660+0000 7efea3f31700 1 -- 172.21.15.82:0/400099601 shutdown_connections 2024-04-21T08:36:28.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.660+0000 7efea3f31700 1 --2- 172.21.15.82:0/400099601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 0x7efe9c108820 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:28.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.660+0000 7efea3f31700 1 -- 172.21.15.82:0/400099601 >> 172.21.15.82:0/400099601 conn(0x7efe9c1039c0 msgr2=0x7efe9c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:28.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.660+0000 7efea3f31700 1 -- 172.21.15.82:0/400099601 shutdown_connections 2024-04-21T08:36:28.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.660+0000 7efea3f31700 1 -- 172.21.15.82:0/400099601 wait complete. 2024-04-21T08:36:28.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efea3f31700 1 Processor -- start 2024-04-21T08:36:28.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efea3f31700 1 -- start start 2024-04-21T08:36:28.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efea3f31700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 0x7efe9c19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:28.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efea3f31700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c10c910 con 0x7efe9c108400 2024-04-21T08:36:28.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efea1ccd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 0x7efe9c19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:28.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efea1ccd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 0x7efe9c19c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52002/0 (socket says 172.21.15.82:52002) 2024-04-21T08:36:28.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efea1ccd700 1 -- 172.21.15.82:0/1314368868 learned_addr learned my addr 172.21.15.82:0/1314368868 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:28.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efea1ccd700 1 -- 172.21.15.82:0/1314368868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9c19c6e0 con 0x7efe9c108400 2024-04-21T08:36:28.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efea1ccd700 1 --2- 172.21.15.82:0/1314368868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 0x7efe9c19c1a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efe94004a60 tx=0x7efe9400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:28.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efe92ffd700 1 -- 172.21.15.82:0/1314368868 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe94020020 con 0x7efe9c108400 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efe92ffd700 1 -- 172.21.15.82:0/1314368868 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efe94003c60 con 0x7efe9c108400 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efe92ffd700 1 -- 172.21.15.82:0/1314368868 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe94017040 con 0x7efe9c108400 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.661+0000 7efea3f31700 1 -- 172.21.15.82:0/1314368868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9c19c970 con 0x7efe9c108400 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.662+0000 7efea3f31700 1 -- 172.21.15.82:0/1314368868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe9c19f660 con 0x7efe9c108400 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.662+0000 7efe92ffd700 1 -- 172.21.15.82:0/1314368868 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efe94023020 con 0x7efe9c108400 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.662+0000 7efe92ffd700 1 -- 172.21.15.82:0/1314368868 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efe9400f050 con 0x7efe9c108400 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.663+0000 7efea3f31700 1 -- 172.21.15.82:0/1314368868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe9c0640a0 con 0x7efe9c108400 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.665+0000 7efe92ffd700 1 -- 172.21.15.82:0/1314368868 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7efe9402a020 con 0x7efe9c108400 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.732+0000 7efea3f31700 1 -- 172.21.15.82:0/1314368868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efe9c19f830 con 0x7efe9c108400 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.733+0000 7efe92ffd700 1 -- 172.21.15.82:0/1314368868 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7efe9400b860 con 0x7efe9c108400 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.733+0000 7efea3f31700 1 -- 172.21.15.82:0/1314368868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 msgr2=0x7efe9c19c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.733+0000 7efea3f31700 1 --2- 172.21.15.82:0/1314368868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 0x7efe9c19c1a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efe94004a60 tx=0x7efe9400bd50 comp rx=0 tx=0).stop 2024-04-21T08:36:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.734+0000 7efea3f31700 1 -- 172.21.15.82:0/1314368868 shutdown_connections 2024-04-21T08:36:28.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.734+0000 7efea3f31700 1 --2- 172.21.15.82:0/1314368868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe9c108400 0x7efe9c19c1a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:28.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.734+0000 7efea3f31700 1 -- 172.21.15.82:0/1314368868 >> 172.21.15.82:0/1314368868 conn(0x7efe9c1039c0 msgr2=0x7efe9c1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:28.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.734+0000 7efea3f31700 1 -- 172.21.15.82:0/1314368868 shutdown_connections 2024-04-21T08:36:28.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:27.734+0000 7efea3f31700 1 -- 172.21.15.82:0/1314368868 wait complete. 2024-04-21T08:36:28.341 INFO:teuthology.orchestra.run.smithi082.stdout:Restarting the monitor... 2024-04-21T08:36:28.600 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:28 smithi082 systemd[1]: Stopping Ceph mon.a for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:36:28.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:28 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a[27594]: 2024-04-21T08:36:28.600+0000 7f2188f3e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:36:28.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:28 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a[27594]: 2024-04-21T08:36:28.600+0000 7f2188f3e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-21T08:36:28.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:28 smithi082 podman[27839]: 2024-04-21 08:36:28.743685577 +0000 UTC m=+0.248169876 container died 7d530032e8d1e6b8fc0d2f4b88fedff7dd04c69482ec46ff21b30c3f2ff95d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-21T08:36:29.316 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 podman[27839]: 2024-04-21 08:36:29.10068454 +0000 UTC m=+0.605168838 container cleanup 7d530032e8d1e6b8fc0d2f4b88fedff7dd04c69482ec46ff21b30c3f2ff95d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-21T08:36:29.317 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 bash[27839]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a 2024-04-21T08:36:29.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 podman[27853]: 2024-04-21 08:36:29.318263391 +0000 UTC m=+0.578022826 container remove 7d530032e8d1e6b8fc0d2f4b88fedff7dd04c69482ec46ff21b30c3f2ff95d7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-21T08:36:29.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 systemd[1]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-21T08:36:29.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 systemd[1]: Stopped Ceph mon.a for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:36:29.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 systemd[1]: Starting Ceph mon.a for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:36:29.929 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 podman[27948]: 2024-04-21 08:36:29.559313792 +0000 UTC m=+0.017720787 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:36:29.929 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 podman[27948]: 2024-04-21 08:36:29.661176456 +0000 UTC m=+0.119583430 container create 1d8a7d05453aed00536ee3ba7a01cd7d65328a94ed433c52b1590f938055607b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:36:29.929 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 podman[27948]: 2024-04-21 08:36:29.846600342 +0000 UTC m=+0.305007322 container init 1d8a7d05453aed00536ee3ba7a01cd7d65328a94ed433c52b1590f938055607b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS) 2024-04-21T08:36:29.929 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 podman[27948]: 2024-04-21 08:36:29.84954309 +0000 UTC m=+0.307950061 container start 1d8a7d05453aed00536ee3ba7a01cd7d65328a94ed433c52b1590f938055607b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:36:29.929 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: set uid:gid to 167:167 (ceph:ceph) 2024-04-21T08:36:29.929 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev), process ceph-mon, pid 2 2024-04-21T08:36:29.929 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: pidfile_write: ignore empty --pid-file 2024-04-21T08:36:29.929 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: load: jerasure load: lrc 2024-04-21T08:36:29.930 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: RocksDB version: 7.9.2 2024-04-21T08:36:29.930 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Git sha 0 2024-04-21T08:36:29.930 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Compile date 2024-04-19 14:19:22 2024-04-21T08:36:29.930 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: DB SUMMARY 2024-04-21T08:36:29.930 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: DB Session ID: QGVFIO0H8KZX85MA3ZN2 2024-04-21T08:36:29.930 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: CURRENT file: CURRENT 2024-04-21T08:36:29.930 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: IDENTITY file: IDENTITY 2024-04-21T08:36:29.930 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-21T08:36:29.930 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-21T08:36:29.930 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104554 ; 2024-04-21T08:36:29.930 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.error_if_exists: 0 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.create_if_missing: 0 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.paranoid_checks: 1 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.env: 0x55890970acc0 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.fs: PosixFileSystem 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.info_log: 0x55890aa73be0 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_file_opening_threads: 16 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.statistics: (nil) 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.use_fsync: 0 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_log_file_size: 0 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-21T08:36:29.931 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.keep_log_file_num: 1000 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.recycle_log_file_num: 0 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.allow_fallocate: 1 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.allow_mmap_reads: 0 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.allow_mmap_writes: 0 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.use_direct_reads: 0 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.create_missing_column_families: 0 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.db_log_dir: 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.wal_dir: 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-21T08:36:29.932 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-21T08:36:29.933 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-21T08:36:29.933 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.advise_random_on_open: 1 2024-04-21T08:36:29.933 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.db_write_buffer_size: 0 2024-04-21T08:36:29.933 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.write_buffer_manager: 0x55890aa070e0 2024-04-21T08:36:29.933 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-21T08:36:29.933 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-21T08:36:29.933 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-21T08:36:29.933 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.rate_limiter: (nil) 2024-04-21T08:36:29.933 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-21T08:36:29.935 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.wal_recovery_mode: 2 2024-04-21T08:36:29.935 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.enable_thread_tracking: 0 2024-04-21T08:36:29.935 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.enable_pipelined_write: 0 2024-04-21T08:36:29.935 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.unordered_write: 0 2024-04-21T08:36:29.935 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-21T08:36:29.935 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-21T08:36:29.935 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-21T08:36:29.935 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.row_cache: None 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.wal_filter: None 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.allow_ingest_behind: 0 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.two_write_queues: 0 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.manual_wal_flush: 0 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.wal_compression: 0 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.atomic_flush: 0 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.log_readahead_size: 0 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.best_efforts_recovery: 0 2024-04-21T08:36:29.936 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.allow_data_in_errors: 0 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.db_host_id: __hostname__ 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_background_jobs: 2 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_background_compactions: -1 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_subcompactions: 1 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_total_wal_size: 0 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-21T08:36:29.937 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_open_files: -1 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bytes_per_sync: 0 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_readahead_size: 0 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_background_flushes: -1 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Compression algorithms supported: 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: kZSTD supported: 0 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: kXpressCompression supported: 0 2024-04-21T08:36:29.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: kLZ4HCCompression supported: 1 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: kZlibCompression supported: 1 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: kSnappyCompression supported: 1 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: kLZ4Compression supported: 1 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: kBZip2Compression supported: 0 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.merge_operator: 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_filter: None 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_filter_factory: None 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.sst_partitioner_factory: None 2024-04-21T08:36:29.939 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55890aa73dc0) 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: cache_index_and_filter_blocks: 1 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: pin_top_level_index_and_filter: 1 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_type: 0 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: data_block_index_type: 0 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_shortening: 1 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: checksum: 4 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: no_block_cache: 0 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache: 0x55890aa391f0 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_name: BinnedLRUCache 2024-04-21T08:36:29.940 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_options: 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: capacity : 536870912 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: num_shard_bits : 4 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: strict_capacity_limit : 0 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: high_pri_pool_ratio: 0.000 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_compressed: (nil) 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: persistent_cache: (nil) 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_size: 4096 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_size_deviation: 10 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_restart_interval: 16 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_block_restart_interval: 1 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: metadata_block_size: 4096 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: partition_filters: 0 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: use_delta_encoding: 1 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: filter_policy: bloomfilter 2024-04-21T08:36:29.941 INFO:journalctl@ceph.mon.a.smithi082.stdout: whole_key_filtering: 1 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout: verify_compression: 0 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout: read_amp_bytes_per_bit: 0 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout: format_version: 5 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout: enable_index_compression: 1 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_align: 0 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout: max_auto_readahead_size: 262144 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout: prepopulate_block_cache: 0 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout: initial_auto_readahead_size: 8192 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout: num_file_reads_for_auto_readahead: 2 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.write_buffer_size: 33554432 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_write_buffer_number: 2 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compression: NoCompression 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bottommost_compression: Disabled 2024-04-21T08:36:29.942 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.prefix_extractor: nullptr 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.num_levels: 7 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-21T08:36:29.943 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compression_opts.level: 32767 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compression_opts.strategy: 0 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compression_opts.enabled: false 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.target_file_size_base: 67108864 2024-04-21T08:36:29.944 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.arena_block_size: 1048576 2024-04-21T08:36:29.945 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.disable_auto_compactions: 0 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-21T08:36:29.946 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.inplace_update_support: 0 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.bloom_locality: 0 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.max_successive_merges: 0 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.paranoid_file_checks: 0 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.force_consistency_checks: 1 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.report_bg_io_stats: 0 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.ttl: 2592000 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-21T08:36:29.947 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.enable_blob_files: false 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.min_blob_size: 0 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.blob_file_size: 268435456 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.blob_file_starting_level: 0 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 066b9a0c-e179-493f-8a9b-b4190978c071 2024-04-21T08:36:29.948 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688589877804, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688589878832, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99783, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97665, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14038, "raw_average_key_size": 49, "raw_value_size": 90011, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713688589, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "066b9a0c-e179-493f-8a9b-b4190978c071", "db_session_id": "QGVFIO0H8KZX85MA3ZN2", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688589878914, "job": 1, "event": "recovery_finished"} 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55890ab4e000 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: DB pointer 0x55890ab38000 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** DB Stats ** 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T08:36:29.949 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** Compaction Stats [default] ** 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: L0 2/0 99.30 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: Sum 2/0 99.30 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** Compaction Stats [default] ** 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T08:36:29.950 INFO:journalctl@ceph.mon.a.smithi082.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 145.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative compaction: 0.00 GB write, 32.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval compaction: 0.00 GB write, 32.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-21T08:36:29.951 INFO:journalctl@ceph.mon.a.smithi082.stdout: Block cache BinnedLRUCache@0x55890aa391f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: starting mon.a rank 0 at public addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] at bind addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: mon.a@-1(???) e1 preinit fsid 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: mon.a@-1(???).mds e1 new map 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: mon.a@-1(???).mds e1 print_map 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout: e1 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout: legacy client fscid: -1 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout: No filesystems configured 2024-04-21T08:36:29.952 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-21T08:36:29.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:36:29.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:36:29.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:36:29.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-21T08:36:29.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 bash[27948]: 1d8a7d05453aed00536ee3ba7a01cd7d65328a94ed433c52b1590f938055607b 2024-04-21T08:36:29.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 systemd[1]: Started Ceph mon.a for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:36:29.953 INFO:teuthology.orchestra.run.smithi082.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-21T08:36:30.256 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-21T08:36:30.256 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: monmap epoch 1 2024-04-21T08:36:30.256 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: fsid 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:36:30.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: last_changed 2024-04-21T08:36:22.015155+0000 2024-04-21T08:36:30.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: created 2024-04-21T08:36:22.015155+0000 2024-04-21T08:36:30.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: min_mon_release 19 (squid) 2024-04-21T08:36:30.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: election_strategy: 1 2024-04-21T08:36:30.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-21T08:36:30.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: fsmap 2024-04-21T08:36:30.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: osdmap e1: 0 total, 0 up, 0 in 2024-04-21T08:36:30.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:29 smithi082 ceph-mon[27967]: mgrmap e1: no daemons active 2024-04-21T08:36:30.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.358+0000 7f77d74e1700 1 Processor -- start 2024-04-21T08:36:30.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.358+0000 7f77d74e1700 1 -- start start 2024-04-21T08:36:30.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.358+0000 7f77d74e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 0x7f77d010b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.358+0000 7f77d74e1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f77d007be60 con 0x7f77d0108eb0 2024-04-21T08:36:30.958 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.359+0000 7f77d527d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 0x7f77d010b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.359+0000 7f77d527d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 0x7f77d010b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52008/0 (socket says 172.21.15.82:52008) 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.359+0000 7f77d527d700 1 -- 172.21.15.82:0/727787310 learned_addr learned my addr 172.21.15.82:0/727787310 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.359+0000 7f77d527d700 1 -- 172.21.15.82:0/727787310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d010b7e0 con 0x7f77d0108eb0 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.359+0000 7f77d527d700 1 --2- 172.21.15.82:0/727787310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 0x7f77d010b2a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0098a0 tx=0x7f77cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba032d25651bc85 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.359+0000 7f77c7fff700 1 -- 172.21.15.82:0/727787310 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77cc004160 con 0x7f77d0108eb0 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.359+0000 7f77c7fff700 1 -- 172.21.15.82:0/727787310 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f77cc004300 con 0x7f77d0108eb0 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.359+0000 7f77c7fff700 1 -- 172.21.15.82:0/727787310 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77cc011af0 con 0x7f77d0108eb0 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.360+0000 7f77d74e1700 1 -- 172.21.15.82:0/727787310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 msgr2=0x7f77d010b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.360+0000 7f77d74e1700 1 --2- 172.21.15.82:0/727787310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 0x7f77d010b2a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0098a0 tx=0x7f77cc009bb0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.360+0000 7f77d74e1700 1 -- 172.21.15.82:0/727787310 shutdown_connections 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.360+0000 7f77d74e1700 1 --2- 172.21.15.82:0/727787310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 0x7f77d010b2a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.360+0000 7f77d74e1700 1 -- 172.21.15.82:0/727787310 >> 172.21.15.82:0/727787310 conn(0x7f77d0102920 msgr2=0x7f77d0104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.360+0000 7f77d74e1700 1 -- 172.21.15.82:0/727787310 shutdown_connections 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.360+0000 7f77d74e1700 1 -- 172.21.15.82:0/727787310 wait complete. 2024-04-21T08:36:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.361+0000 7f77d74e1700 1 Processor -- start 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.361+0000 7f77d74e1700 1 -- start start 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.361+0000 7f77d74e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 0x7f77d019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.361+0000 7f77d74e1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f77d010c730 con 0x7f77d0108eb0 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.361+0000 7f77d527d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 0x7f77d019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.361+0000 7f77d527d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 0x7f77d019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52024/0 (socket says 172.21.15.82:52024) 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.361+0000 7f77d527d700 1 -- 172.21.15.82:0/2954358920 learned_addr learned my addr 172.21.15.82:0/2954358920 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.361+0000 7f77d527d700 1 -- 172.21.15.82:0/2954358920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d019c7e0 con 0x7f77d0108eb0 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.362+0000 7f77d527d700 1 --2- 172.21.15.82:0/2954358920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 0x7f77d019c2a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f77cc011ac0 tx=0x7f77cc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.362+0000 7f77c67fc700 1 -- 172.21.15.82:0/2954358920 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77cc003970 con 0x7f77d0108eb0 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.362+0000 7f77c67fc700 1 -- 172.21.15.82:0/2954358920 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f77cc003b10 con 0x7f77d0108eb0 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.362+0000 7f77d74e1700 1 -- 172.21.15.82:0/2954358920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d019ca70 con 0x7f77d0108eb0 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.362+0000 7f77c67fc700 1 -- 172.21.15.82:0/2954358920 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77cc024610 con 0x7f77d0108eb0 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.362+0000 7f77d74e1700 1 -- 172.21.15.82:0/2954358920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d019f760 con 0x7f77d0108eb0 2024-04-21T08:36:30.960 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.362+0000 7f77c67fc700 1 -- 172.21.15.82:0/2954358920 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f77cc024bd0 con 0x7f77d0108eb0 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.362+0000 7f77c67fc700 1 -- 172.21.15.82:0/2954358920 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f77cc019700 con 0x7f77d0108eb0 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.363+0000 7f77d74e1700 1 -- 172.21.15.82:0/2954358920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d00640a0 con 0x7f77d0108eb0 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.365+0000 7f77c67fc700 1 -- 172.21.15.82:0/2954358920 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f77cc02ddf0 con 0x7f77d0108eb0 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.433+0000 7f77d74e1700 1 -- 172.21.15.82:0/2954358920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f77d019f930 con 0x7f77d0108eb0 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.435+0000 7f77c67fc700 1 -- 172.21.15.82:0/2954358920 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f77cc01a9e0 con 0x7f77d0108eb0 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.435+0000 7f77d74e1700 1 -- 172.21.15.82:0/2954358920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 msgr2=0x7f77d019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.436+0000 7f77d74e1700 1 --2- 172.21.15.82:0/2954358920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 0x7f77d019c2a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f77cc011ac0 tx=0x7f77cc011ba0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.436+0000 7f77d74e1700 1 -- 172.21.15.82:0/2954358920 shutdown_connections 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.436+0000 7f77d74e1700 1 --2- 172.21.15.82:0/2954358920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77d0108eb0 0x7f77d019c2a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.436+0000 7f77d74e1700 1 -- 172.21.15.82:0/2954358920 >> 172.21.15.82:0/2954358920 conn(0x7f77d0102920 msgr2=0x7f77d0103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.436+0000 7f77d74e1700 1 -- 172.21.15.82:0/2954358920 shutdown_connections 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:30.436+0000 7f77d74e1700 1 -- 172.21.15.82:0/2954358920 wait complete. 2024-04-21T08:36:30.961 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-21T08:36:30.984 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:36:30.984 INFO:teuthology.orchestra.run.smithi082.stdout:Creating mgr... 2024-04-21T08:36:30.985 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-21T08:36:30.985 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-21T08:36:31.237 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.x 2024-04-21T08:36:31.237 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to reset failed state of unit ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.x.service: Unit ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-21T08:36:31.425 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-24d05dee-ffba-11ee-bc93-c7b262605968.target.wants/ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-24d05dee-ffba-11ee-bc93-c7b262605968@.service. 2024-04-21T08:36:31.779 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:31 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2954358920' entity='client.admin' 2024-04-21T08:36:31.977 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:36:31.978 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-04-21T08:36:31.984 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:36:31.984 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-04-21T08:36:31.984 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-04-21T08:36:31.984 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to enable service . firewalld.service is not available 2024-04-21T08:36:31.992 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:36:31.993 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-04-21T08:36:32.000 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:36:32.000 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-04-21T08:36:32.000 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-04-21T08:36:32.000 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-21T08:36:32.000 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr to start... 2024-04-21T08:36:32.000 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr... 2024-04-21T08:36:32.066 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:31 smithi082 systemd[1]: Started Ceph mgr.x for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:36:32.342 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:32 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:32.066+0000 7ff083b1f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:36:32.342 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:32 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:32.254+0000 7ff083b1f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:36:32.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:32 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3442060894' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:36:32.609 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:32 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:32.360+0000 7ff083b1f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:36:32.971 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:32 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:32.681+0000 7ff083b1f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:36:32.971 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:32 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:32.766+0000 7ff083b1f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:36:33.244 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:33.244 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:36:33.244 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:36:33.244 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:36:33.244 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:36:33.244 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:36:33.244 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:36:33.244 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:33.244 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:36:33.244 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:36:33.245 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:36:33.246 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:36:33.246 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:36:33.246 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:36:33.246 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:36:33.246 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:33.246 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:36:33.246 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:36:33.246 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:36:33.253 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:36:33.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:36:25.047649+0000", 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:33.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.443+0000 7f6b84f42700 1 Processor -- start 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.443+0000 7f6b84f42700 1 -- start start 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.443+0000 7f6b84f42700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 0x7f6b8010b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.443+0000 7f6b84f42700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8007bdc0 con 0x7f6b80108ea0 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.444+0000 7f6b7e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 0x7f6b8010b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.444+0000 7f6b7e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 0x7f6b8010b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52044/0 (socket says 172.21.15.82:52044) 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.444+0000 7f6b7e59c700 1 -- 172.21.15.82:0/1854724453 learned_addr learned my addr 172.21.15.82:0/1854724453 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.444+0000 7f6b7e59c700 1 -- 172.21.15.82:0/1854724453 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8010b7d0 con 0x7f6b80108ea0 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.444+0000 7f6b7e59c700 1 --2- 172.21.15.82:0/1854724453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 0x7f6b8010b290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6b740098a0 tx=0x7f6b74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eeab17468a84751 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.444+0000 7f6b7d59a700 1 -- 172.21.15.82:0/1854724453 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b74004160 con 0x7f6b80108ea0 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.444+0000 7f6b7d59a700 1 -- 172.21.15.82:0/1854724453 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b74004300 con 0x7f6b80108ea0 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.444+0000 7f6b7d59a700 1 -- 172.21.15.82:0/1854724453 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b74011af0 con 0x7f6b80108ea0 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.445+0000 7f6b84f42700 1 -- 172.21.15.82:0/1854724453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 msgr2=0x7f6b8010b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.445+0000 7f6b84f42700 1 --2- 172.21.15.82:0/1854724453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 0x7f6b8010b290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6b740098a0 tx=0x7f6b74009bb0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.445+0000 7f6b84f42700 1 -- 172.21.15.82:0/1854724453 shutdown_connections 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.445+0000 7f6b84f42700 1 --2- 172.21.15.82:0/1854724453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 0x7f6b8010b290 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.445+0000 7f6b84f42700 1 -- 172.21.15.82:0/1854724453 >> 172.21.15.82:0/1854724453 conn(0x7f6b801027f0 msgr2=0x7f6b80104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.445+0000 7f6b84f42700 1 -- 172.21.15.82:0/1854724453 shutdown_connections 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.445+0000 7f6b84f42700 1 -- 172.21.15.82:0/1854724453 wait complete. 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.446+0000 7f6b84f42700 1 Processor -- start 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.446+0000 7f6b84f42700 1 -- start start 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.446+0000 7f6b84f42700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 0x7f6b8019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.446+0000 7f6b84f42700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8010c6d0 con 0x7f6b80108ea0 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.446+0000 7f6b7e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 0x7f6b8019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.446+0000 7f6b7e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 0x7f6b8019e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52050/0 (socket says 172.21.15.82:52050) 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.446+0000 7f6b7e59c700 1 -- 172.21.15.82:0/3442060894 learned_addr learned my addr 172.21.15.82:0/3442060894 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:33.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.447+0000 7f6b7e59c700 1 -- 172.21.15.82:0/3442060894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8019e930 con 0x7f6b80108ea0 2024-04-21T08:36:33.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.447+0000 7f6b7e59c700 1 --2- 172.21.15.82:0/3442060894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 0x7f6b8019e3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6b74003f40 tx=0x7f6b74011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:33.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.447+0000 7f6b6f7fe700 1 -- 172.21.15.82:0/3442060894 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b740038b0 con 0x7f6b80108ea0 2024-04-21T08:36:33.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.447+0000 7f6b6f7fe700 1 -- 172.21.15.82:0/3442060894 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b74003a50 con 0x7f6b80108ea0 2024-04-21T08:36:33.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.447+0000 7f6b84f42700 1 -- 172.21.15.82:0/3442060894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8019ebc0 con 0x7f6b80108ea0 2024-04-21T08:36:33.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.447+0000 7f6b6f7fe700 1 -- 172.21.15.82:0/3442060894 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b7401a5c0 con 0x7f6b80108ea0 2024-04-21T08:36:33.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.447+0000 7f6b84f42700 1 -- 172.21.15.82:0/3442060894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b801a18b0 con 0x7f6b80108ea0 2024-04-21T08:36:33.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.447+0000 7f6b6f7fe700 1 -- 172.21.15.82:0/3442060894 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6b7401ac50 con 0x7f6b80108ea0 2024-04-21T08:36:33.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.447+0000 7f6b6f7fe700 1 -- 172.21.15.82:0/3442060894 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6b74019700 con 0x7f6b80108ea0 2024-04-21T08:36:33.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.448+0000 7f6b84f42700 1 -- 172.21.15.82:0/3442060894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b800640a0 con 0x7f6b80108ea0 2024-04-21T08:36:33.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.452+0000 7f6b6f7fe700 1 -- 172.21.15.82:0/3442060894 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6b740199a0 con 0x7f6b80108ea0 2024-04-21T08:36:33.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.524+0000 7f6b84f42700 1 -- 172.21.15.82:0/3442060894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6b801a1df0 con 0x7f6b80108ea0 2024-04-21T08:36:33.258 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.524+0000 7f6b6f7fe700 1 -- 172.21.15.82:0/3442060894 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6b74019b80 con 0x7f6b80108ea0 2024-04-21T08:36:33.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.525+0000 7f6b84f42700 1 -- 172.21.15.82:0/3442060894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 msgr2=0x7f6b8019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.525+0000 7f6b84f42700 1 --2- 172.21.15.82:0/3442060894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 0x7f6b8019e3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6b74003f40 tx=0x7f6b74011890 comp rx=0 tx=0).stop 2024-04-21T08:36:33.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.525+0000 7f6b84f42700 1 -- 172.21.15.82:0/3442060894 shutdown_connections 2024-04-21T08:36:33.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.525+0000 7f6b84f42700 1 --2- 172.21.15.82:0/3442060894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b80108ea0 0x7f6b8019e3f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.525+0000 7f6b84f42700 1 -- 172.21.15.82:0/3442060894 >> 172.21.15.82:0/3442060894 conn(0x7f6b801027f0 msgr2=0x7f6b80103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:33.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.525+0000 7f6b84f42700 1 -- 172.21.15.82:0/3442060894 shutdown_connections 2024-04-21T08:36:33.259 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:32.525+0000 7f6b84f42700 1 -- 172.21.15.82:0/3442060894 wait complete. 2024-04-21T08:36:33.259 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (1/15)... 2024-04-21T08:36:34.222 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:33 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:33.961+0000 7ff083b1f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:36:34.222 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:34 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:34.128+0000 7ff083b1f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:36:34.590 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:34 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:34.306+0000 7ff083b1f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:36:34.590 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:34 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:34.590+0000 7ff083b1f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:36:34.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:34 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:34.681+0000 7ff083b1f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:36:35.357 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:34 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:34.882+0000 7ff083b1f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:36:35.357 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:35 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:35.071+0000 7ff083b1f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:36:36.029 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:35 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/435576833' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:36:36.029 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:35 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:35.677+0000 7ff083b1f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:36:36.322 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:36 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:36.114+0000 7ff083b1f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:36:36.322 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:36 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:36.210+0000 7ff083b1f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:36:36.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:36.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:36:36.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:36:36.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:36:36.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:36:36.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:36:36.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:36:36.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:36.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:36:36.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:36:36.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:36:36.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:36:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:36:36.336 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:36:36.336 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:36.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:36:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:36:25.047649+0000", 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.645+0000 7f5fd82c8700 1 Processor -- start 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.645+0000 7f5fd82c8700 1 -- start start 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.645+0000 7f5fd82c8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 0x7f5fd010b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.645+0000 7f5fd82c8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd007be60 con 0x7f5fd0108eb0 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.645+0000 7f5fd6064700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 0x7f5fd010b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.646+0000 7f5fd6064700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 0x7f5fd010b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39938/0 (socket says 172.21.15.82:39938) 2024-04-21T08:36:36.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.646+0000 7f5fd6064700 1 -- 172.21.15.82:0/1836323194 learned_addr learned my addr 172.21.15.82:0/1836323194 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.646+0000 7f5fd6064700 1 -- 172.21.15.82:0/1836323194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd010b7e0 con 0x7f5fd0108eb0 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.646+0000 7f5fd6064700 1 --2- 172.21.15.82:0/1836323194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 0x7f5fd010b2a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00098a0 tx=0x7f5fc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe2a54d121d94fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.646+0000 7f5fd5062700 1 -- 172.21.15.82:0/1836323194 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fc0004160 con 0x7f5fd0108eb0 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.646+0000 7f5fd5062700 1 -- 172.21.15.82:0/1836323194 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5fc0004300 con 0x7f5fd0108eb0 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.646+0000 7f5fd5062700 1 -- 172.21.15.82:0/1836323194 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fc0011af0 con 0x7f5fd0108eb0 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.647+0000 7f5fd82c8700 1 -- 172.21.15.82:0/1836323194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 msgr2=0x7f5fd010b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.647+0000 7f5fd82c8700 1 --2- 172.21.15.82:0/1836323194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 0x7f5fd010b2a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00098a0 tx=0x7f5fc0009bb0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.647+0000 7f5fd82c8700 1 -- 172.21.15.82:0/1836323194 shutdown_connections 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.647+0000 7f5fd82c8700 1 --2- 172.21.15.82:0/1836323194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 0x7f5fd010b2a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.647+0000 7f5fd82c8700 1 -- 172.21.15.82:0/1836323194 >> 172.21.15.82:0/1836323194 conn(0x7f5fd0102920 msgr2=0x7f5fd0104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.647+0000 7f5fd82c8700 1 -- 172.21.15.82:0/1836323194 shutdown_connections 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.647+0000 7f5fd82c8700 1 -- 172.21.15.82:0/1836323194 wait complete. 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.648+0000 7f5fd82c8700 1 Processor -- start 2024-04-21T08:36:36.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.648+0000 7f5fd82c8700 1 -- start start 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.648+0000 7f5fd82c8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 0x7f5fd019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.648+0000 7f5fd82c8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd010c730 con 0x7f5fd0108eb0 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.648+0000 7f5fd6064700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 0x7f5fd019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.648+0000 7f5fd6064700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 0x7f5fd019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39946/0 (socket says 172.21.15.82:39946) 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.648+0000 7f5fd6064700 1 -- 172.21.15.82:0/435576833 learned_addr learned my addr 172.21.15.82:0/435576833 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.648+0000 7f5fd6064700 1 -- 172.21.15.82:0/435576833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd019c7e0 con 0x7f5fd0108eb0 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.648+0000 7f5fd6064700 1 --2- 172.21.15.82:0/435576833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 0x7f5fd019c2a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009fb0 tx=0x7f5fc0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.648+0000 7f5fc77fe700 1 -- 172.21.15.82:0/435576833 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fc00036a0 con 0x7f5fd0108eb0 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.648+0000 7f5fc77fe700 1 -- 172.21.15.82:0/435576833 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5fc001a430 con 0x7f5fd0108eb0 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.649+0000 7f5fd82c8700 1 -- 172.21.15.82:0/435576833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd019ca70 con 0x7f5fd0108eb0 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.649+0000 7f5fc77fe700 1 -- 172.21.15.82:0/435576833 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fc0019550 con 0x7f5fd0108eb0 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.649+0000 7f5fd82c8700 1 -- 172.21.15.82:0/435576833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd019f760 con 0x7f5fd0108eb0 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.649+0000 7f5fc77fe700 1 -- 172.21.15.82:0/435576833 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5fc0019be0 con 0x7f5fd0108eb0 2024-04-21T08:36:36.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.649+0000 7f5fc77fe700 1 -- 172.21.15.82:0/435576833 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5fc0024730 con 0x7f5fd0108eb0 2024-04-21T08:36:36.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.649+0000 7f5fd82c8700 1 -- 172.21.15.82:0/435576833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd00640a0 con 0x7f5fd0108eb0 2024-04-21T08:36:36.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.652+0000 7f5fc77fe700 1 -- 172.21.15.82:0/435576833 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5fc001a5e0 con 0x7f5fd0108eb0 2024-04-21T08:36:36.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.724+0000 7f5fd82c8700 1 -- 172.21.15.82:0/435576833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5fd019fca0 con 0x7f5fd0108eb0 2024-04-21T08:36:36.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.724+0000 7f5fc77fe700 1 -- 172.21.15.82:0/435576833 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5fc001a8e0 con 0x7f5fd0108eb0 2024-04-21T08:36:36.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.725+0000 7f5fd82c8700 1 -- 172.21.15.82:0/435576833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 msgr2=0x7f5fd019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.725+0000 7f5fd82c8700 1 --2- 172.21.15.82:0/435576833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 0x7f5fd019c2a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009fb0 tx=0x7f5fc0011940 comp rx=0 tx=0).stop 2024-04-21T08:36:36.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.725+0000 7f5fd82c8700 1 -- 172.21.15.82:0/435576833 shutdown_connections 2024-04-21T08:36:36.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.725+0000 7f5fd82c8700 1 --2- 172.21.15.82:0/435576833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd0108eb0 0x7f5fd019c2a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.725+0000 7f5fd82c8700 1 -- 172.21.15.82:0/435576833 >> 172.21.15.82:0/435576833 conn(0x7f5fd0102920 msgr2=0x7f5fd0103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:36.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.725+0000 7f5fd82c8700 1 -- 172.21.15.82:0/435576833 shutdown_connections 2024-04-21T08:36:36.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:35.725+0000 7f5fd82c8700 1 -- 172.21.15.82:0/435576833 wait complete. 2024-04-21T08:36:36.342 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (2/15)... 2024-04-21T08:36:36.608 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:36 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:36.414+0000 7ff083b1f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:36:37.268 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:36 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:36.938+0000 7ff083b1f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:36:37.268 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:37 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:37.030+0000 7ff083b1f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:36:37.608 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:37 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:37.268+0000 7ff083b1f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:36:37.608 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:37 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:37.400+0000 7ff083b1f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:36:37.608 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:37 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:37.495+0000 7ff083b1f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:36:37.935 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:37 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:37.611+0000 7ff083b1f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:36:37.935 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:37 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:37.935+0000 7ff083b1f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:36:38.758 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:38.538+0000 7ff083b1f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:36:38.758 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:38.758+0000 7ff083b1f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:36:39.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: Activating manager daemon x 2024-04-21T08:36:39.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: mgrmap e2: x(active, starting, since 0.00338015s) 2024-04-21T08:36:39.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: from='mgr.14100 172.21.15.82:0/2341510590' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:36:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: from='mgr.14100 172.21.15.82:0/2341510590' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:36:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: from='mgr.14100 172.21.15.82:0/2341510590' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:36:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: from='mgr.14100 172.21.15.82:0/2341510590' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:36:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: from='mgr.14100 172.21.15.82:0/2341510590' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:36:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: Manager daemon x is now available 2024-04-21T08:36:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: from='mgr.14100 172.21.15.82:0/2341510590' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:36:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: from='mgr.14100 172.21.15.82:0/2341510590' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:36:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: from='mgr.14100 172.21.15.82:0/2341510590' entity='mgr.x' 2024-04-21T08:36:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: from='mgr.14100 172.21.15.82:0/2341510590' entity='mgr.x' 2024-04-21T08:36:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:38 smithi082 ceph-mon[27967]: from='mgr.14100 172.21.15.82:0/2341510590' entity='mgr.x' 2024-04-21T08:36:39.494 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:39.494 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:36:39.494 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:36:39.494 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:36:39.494 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:36:39.494 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:39.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:36:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:36:25.047649+0000", 2024-04-21T08:36:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.783+0000 7fdbd5a4b700 1 Processor -- start 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.783+0000 7fdbd5a4b700 1 -- start start 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.783+0000 7fdbd5a4b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 0x7fdbd01076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.783+0000 7fdbd5a4b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0107c90 con 0x7fdbd01072a0 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.783+0000 7fdbceffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 0x7fdbd01076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.783+0000 7fdbceffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 0x7fdbd01076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40006/0 (socket says 172.21.15.82:40006) 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.783+0000 7fdbceffd700 1 -- 172.21.15.82:0/76606386 learned_addr learned my addr 172.21.15.82:0/76606386 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.784+0000 7fdbceffd700 1 -- 172.21.15.82:0/76606386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd01084a0 con 0x7fdbd01072a0 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.784+0000 7fdbceffd700 1 --2- 172.21.15.82:0/76606386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 0x7fdbd01076c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00098a0 tx=0x7fdbc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52c9c4a72e2b830 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:39.499 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.784+0000 7fdbcdffb700 1 -- 172.21.15.82:0/76606386 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc0004160 con 0x7fdbd01072a0 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.784+0000 7fdbcdffb700 1 -- 172.21.15.82:0/76606386 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdbc0004300 con 0x7fdbd01072a0 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.784+0000 7fdbcdffb700 1 -- 172.21.15.82:0/76606386 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc0011af0 con 0x7fdbd01072a0 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.785+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/76606386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 msgr2=0x7fdbd01076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.785+0000 7fdbd5a4b700 1 --2- 172.21.15.82:0/76606386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 0x7fdbd01076c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00098a0 tx=0x7fdbc0009bb0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.785+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/76606386 shutdown_connections 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.785+0000 7fdbd5a4b700 1 --2- 172.21.15.82:0/76606386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 0x7fdbd01076c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.785+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/76606386 >> 172.21.15.82:0/76606386 conn(0x7fdbd01028e0 msgr2=0x7fdbd0104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.785+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/76606386 shutdown_connections 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.785+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/76606386 wait complete. 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.786+0000 7fdbd5a4b700 1 Processor -- start 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.786+0000 7fdbd5a4b700 1 -- start start 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.786+0000 7fdbd5a4b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 0x7fdbd019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.786+0000 7fdbd5a4b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd007afc0 con 0x7fdbd01072a0 2024-04-21T08:36:39.500 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.787+0000 7fdbceffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 0x7fdbd019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.787+0000 7fdbceffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 0x7fdbd019a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40026/0 (socket says 172.21.15.82:40026) 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.787+0000 7fdbceffd700 1 -- 172.21.15.82:0/3000577063 learned_addr learned my addr 172.21.15.82:0/3000577063 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.787+0000 7fdbceffd700 1 -- 172.21.15.82:0/3000577063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd019a540 con 0x7fdbd01072a0 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.787+0000 7fdbceffd700 1 --2- 172.21.15.82:0/3000577063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 0x7fdbd019a000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0003f40 tx=0x7fdbc0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.787+0000 7fdbd4a49700 1 -- 172.21.15.82:0/3000577063 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc00038b0 con 0x7fdbd01072a0 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.787+0000 7fdbd4a49700 1 -- 172.21.15.82:0/3000577063 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdbc0003a50 con 0x7fdbd01072a0 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.787+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/3000577063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd019a7d0 con 0x7fdbd01072a0 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.787+0000 7fdbd4a49700 1 -- 172.21.15.82:0/3000577063 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc001a5c0 con 0x7fdbd01072a0 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.787+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/3000577063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd019abf0 con 0x7fdbd01072a0 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.788+0000 7fdbd4a49700 1 -- 172.21.15.82:0/3000577063 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 2) ==== 46314+0+0 (secure 0 0 0) 0x7fdbc001a870 con 0x7fdbd01072a0 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.789+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/3000577063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd00640a0 con 0x7fdbd01072a0 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.789+0000 7fdbd4a49700 1 -- 172.21.15.82:0/3000577063 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdbc004d650 con 0x7fdbd01072a0 2024-04-21T08:36:39.501 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.791+0000 7fdbd4a49700 1 -- 172.21.15.82:0/3000577063 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdbc0019c70 con 0x7fdbd01072a0 2024-04-21T08:36:39.502 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.864+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/3000577063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdbd019d6b0 con 0x7fdbd01072a0 2024-04-21T08:36:39.502 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.864+0000 7fdbd4a49700 1 -- 172.21.15.82:0/3000577063 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdbd019d6b0 con 0x7fdbd01072a0 2024-04-21T08:36:39.502 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.865+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/3000577063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 msgr2=0x7fdbd019a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:39.502 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.865+0000 7fdbd5a4b700 1 --2- 172.21.15.82:0/3000577063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 0x7fdbd019a000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0003f40 tx=0x7fdbc0011890 comp rx=0 tx=0).stop 2024-04-21T08:36:39.502 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.865+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/3000577063 shutdown_connections 2024-04-21T08:36:39.502 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.865+0000 7fdbd5a4b700 1 --2- 172.21.15.82:0/3000577063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbd01072a0 0x7fdbd019a000 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:39.502 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.865+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/3000577063 >> 172.21.15.82:0/3000577063 conn(0x7fdbd01028e0 msgr2=0x7fdbd0190620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:39.502 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.865+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/3000577063 shutdown_connections 2024-04-21T08:36:39.502 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:38.865+0000 7fdbd5a4b700 1 -- 172.21.15.82:0/3000577063 wait complete. 2024-04-21T08:36:39.502 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (3/15)... 2024-04-21T08:36:40.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:39 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3000577063' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:36:40.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:39 smithi082 ceph-mon[27967]: mgrmap e3: x(active, since 1.00736s) 2024-04-21T08:36:42.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:41 smithi082 ceph-mon[27967]: mgrmap e4: x(active, since 2s) 2024-04-21T08:36:45.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:44 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/734256757' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:36:50.832 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:50.832 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:36:50.832 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:36:50.832 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:36:50.832 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:50.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:36:50.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:50.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:36:25.047649+0000", 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:36:50.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.641+0000 7fd66de7c700 1 Processor -- start 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.641+0000 7fd66de7c700 1 -- start start 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.641+0000 7fd66de7c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 0x7fd6681076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.641+0000 7fd66de7c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd668107c90 con 0x7fd6681072a0 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.641+0000 7fd6677fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 0x7fd6681076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.641+0000 7fd6677fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 0x7fd6681076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54784/0 (socket says 172.21.15.82:54784) 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.641+0000 7fd6677fe700 1 -- 172.21.15.82:0/2004960860 learned_addr learned my addr 172.21.15.82:0/2004960860 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.641+0000 7fd6677fe700 1 -- 172.21.15.82:0/2004960860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6681084a0 con 0x7fd6681072a0 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.642+0000 7fd6677fe700 1 --2- 172.21.15.82:0/2004960860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 0x7fd6681076c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6580098a0 tx=0x7fd658009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c61a5c5cc0eac69 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.642+0000 7fd6667fc700 1 -- 172.21.15.82:0/2004960860 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd658004160 con 0x7fd6681072a0 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.642+0000 7fd6667fc700 1 -- 172.21.15.82:0/2004960860 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd658004300 con 0x7fd6681072a0 2024-04-21T08:36:50.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.642+0000 7fd6667fc700 1 -- 172.21.15.82:0/2004960860 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd658011af0 con 0x7fd6681072a0 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.642+0000 7fd66de7c700 1 -- 172.21.15.82:0/2004960860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 msgr2=0x7fd6681076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.642+0000 7fd66de7c700 1 --2- 172.21.15.82:0/2004960860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 0x7fd6681076c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6580098a0 tx=0x7fd658009bb0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.642+0000 7fd66de7c700 1 -- 172.21.15.82:0/2004960860 shutdown_connections 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.642+0000 7fd66de7c700 1 --2- 172.21.15.82:0/2004960860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 0x7fd6681076c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.642+0000 7fd66de7c700 1 -- 172.21.15.82:0/2004960860 >> 172.21.15.82:0/2004960860 conn(0x7fd6681028e0 msgr2=0x7fd668104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.642+0000 7fd66de7c700 1 -- 172.21.15.82:0/2004960860 shutdown_connections 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.642+0000 7fd66de7c700 1 -- 172.21.15.82:0/2004960860 wait complete. 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.643+0000 7fd66de7c700 1 Processor -- start 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.643+0000 7fd66de7c700 1 -- start start 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.643+0000 7fd66de7c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 0x7fd66819c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.643+0000 7fd66de7c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd66810b6d0 con 0x7fd6681072a0 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.643+0000 7fd6677fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 0x7fd66819c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:50.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.643+0000 7fd6677fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 0x7fd66819c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54798/0 (socket says 172.21.15.82:54798) 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.643+0000 7fd6677fe700 1 -- 172.21.15.82:0/734256757 learned_addr learned my addr 172.21.15.82:0/734256757 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.644+0000 7fd6677fe700 1 -- 172.21.15.82:0/734256757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66819c740 con 0x7fd6681072a0 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.644+0000 7fd6677fe700 1 --2- 172.21.15.82:0/734256757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 0x7fd66819c200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd658003f40 tx=0x7fd658011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.644+0000 7fd664ff9700 1 -- 172.21.15.82:0/734256757 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6580038b0 con 0x7fd6681072a0 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.644+0000 7fd66de7c700 1 -- 172.21.15.82:0/734256757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66819c9d0 con 0x7fd6681072a0 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.644+0000 7fd664ff9700 1 -- 172.21.15.82:0/734256757 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd658003a50 con 0x7fd6681072a0 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.644+0000 7fd664ff9700 1 -- 172.21.15.82:0/734256757 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd65801a5c0 con 0x7fd6681072a0 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.644+0000 7fd66de7c700 1 -- 172.21.15.82:0/734256757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66819f6c0 con 0x7fd6681072a0 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.645+0000 7fd664ff9700 1 -- 172.21.15.82:0/734256757 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fd65801a8c0 con 0x7fd6681072a0 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.645+0000 7fd66de7c700 1 -- 172.21.15.82:0/734256757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66810b850 con 0x7fd6681072a0 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.645+0000 7fd664ff9700 1 --2- 172.21.15.82:0/734256757 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fd6500395d0 0x7fd65003ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.645+0000 7fd664ff9700 1 -- 172.21.15.82:0/734256757 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd65804db80 con 0x7fd6681072a0 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.645+0000 7fd666ffd700 1 --2- 172.21.15.82:0/734256757 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fd6500395d0 0x7fd65003ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.646+0000 7fd666ffd700 1 --2- 172.21.15.82:0/734256757 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fd6500395d0 0x7fd65003ba90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd65c006fd0 tx=0x7fd65c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.651+0000 7fd664ff9700 1 -- 172.21.15.82:0/734256757 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd658019c70 con 0x7fd6681072a0 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.930+0000 7fd66de7c700 1 -- 172.21.15.82:0/734256757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd66802e360 con 0x7fd6681072a0 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.931+0000 7fd664ff9700 1 -- 172.21.15.82:0/734256757 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd65804d5a0 con 0x7fd6681072a0 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.934+0000 7fd66de7c700 1 -- 172.21.15.82:0/734256757 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fd6500395d0 msgr2=0x7fd65003ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.934+0000 7fd66de7c700 1 --2- 172.21.15.82:0/734256757 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fd6500395d0 0x7fd65003ba90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd65c006fd0 tx=0x7fd65c006e40 comp rx=0 tx=0).stop 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.934+0000 7fd66de7c700 1 -- 172.21.15.82:0/734256757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 msgr2=0x7fd66819c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.934+0000 7fd66de7c700 1 --2- 172.21.15.82:0/734256757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 0x7fd66819c200 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd658003f40 tx=0x7fd658011890 comp rx=0 tx=0).stop 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.934+0000 7fd66de7c700 1 -- 172.21.15.82:0/734256757 shutdown_connections 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.934+0000 7fd66de7c700 1 --2- 172.21.15.82:0/734256757 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fd6500395d0 0x7fd65003ba90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.934+0000 7fd66de7c700 1 --2- 172.21.15.82:0/734256757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6681072a0 0x7fd66819c200 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.934+0000 7fd66de7c700 1 -- 172.21.15.82:0/734256757 >> 172.21.15.82:0/734256757 conn(0x7fd6681028e0 msgr2=0x7fd6681036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.934+0000 7fd66de7c700 1 -- 172.21.15.82:0/734256757 shutdown_connections 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:44.934+0000 7fd66de7c700 1 -- 172.21.15.82:0/734256757 wait complete. 2024-04-21T08:36:50.840 INFO:teuthology.orchestra.run.smithi082.stdout:mgr is available 2024-04-21T08:36:51.781 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:51 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1734338042' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-21T08:36:52.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:52.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [global] 2024-04-21T08:36:52.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout fsid = 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:36:52.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-21T08:36:52.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-21T08:36:52.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-21T08:36:52.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-21T08:36:52.033 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-21T08:36:52.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:52.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-21T08:36:52.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-21T08:36:52.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-21T08:36:52.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [osd] 2024-04-21T08:36:52.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-21T08:36:52.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-21T08:36:52.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-21T08:36:52.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-21T08:36:52.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-21T08:36:52.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.268+0000 7f3602f16700 1 Processor -- start 2024-04-21T08:36:52.034 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.268+0000 7f3602f16700 1 -- start start 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.268+0000 7f3602f16700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 0x7f35fc107730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.268+0000 7f3602f16700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc107d00 con 0x7f35fc107310 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.269+0000 7f3600cb2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 0x7f35fc107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.269+0000 7f3600cb2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 0x7f35fc107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54806/0 (socket says 172.21.15.82:54806) 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.269+0000 7f3600cb2700 1 -- 172.21.15.82:0/3137133830 learned_addr learned my addr 172.21.15.82:0/3137133830 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.269+0000 7f3600cb2700 1 -- 172.21.15.82:0/3137133830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35fc108510 con 0x7f35fc107310 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.269+0000 7f3600cb2700 1 --2- 172.21.15.82:0/3137133830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 0x7f35fc107730 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f35f00098a0 tx=0x7f35f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a1972abf126f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.269+0000 7f35fb7fe700 1 -- 172.21.15.82:0/3137133830 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35f0004160 con 0x7f35fc107310 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.269+0000 7f35fb7fe700 1 -- 172.21.15.82:0/3137133830 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f35f0004300 con 0x7f35fc107310 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.269+0000 7f35fb7fe700 1 -- 172.21.15.82:0/3137133830 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35f0011af0 con 0x7f35fc107310 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.270+0000 7f3602f16700 1 -- 172.21.15.82:0/3137133830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 msgr2=0x7f35fc107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.270+0000 7f3602f16700 1 --2- 172.21.15.82:0/3137133830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 0x7f35fc107730 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f35f00098a0 tx=0x7f35f0009bb0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.035 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.270+0000 7f3602f16700 1 -- 172.21.15.82:0/3137133830 shutdown_connections 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.270+0000 7f3602f16700 1 --2- 172.21.15.82:0/3137133830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 0x7f35fc107730 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.270+0000 7f3602f16700 1 -- 172.21.15.82:0/3137133830 >> 172.21.15.82:0/3137133830 conn(0x7f35fc1028b0 msgr2=0x7f35fc104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.270+0000 7f3602f16700 1 -- 172.21.15.82:0/3137133830 shutdown_connections 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.270+0000 7f3602f16700 1 -- 172.21.15.82:0/3137133830 wait complete. 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f3602f16700 1 Processor -- start 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f3602f16700 1 -- start start 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f3602f16700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 0x7f35fc07ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f3602f16700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc10b740 con 0x7f35fc107310 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f3600cb2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 0x7f35fc07ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f3600cb2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 0x7f35fc07ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54810/0 (socket says 172.21.15.82:54810) 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f3600cb2700 1 -- 172.21.15.82:0/1734338042 learned_addr learned my addr 172.21.15.82:0/1734338042 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f3600cb2700 1 -- 172.21.15.82:0/1734338042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35fc19efe0 con 0x7f35fc107310 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f3600cb2700 1 --2- 172.21.15.82:0/1734338042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 0x7f35fc07ba40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f35f00117b0 tx=0x7f35f0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:52.036 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f35f9ffb700 1 -- 172.21.15.82:0/1734338042 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35f0003af0 con 0x7f35fc107310 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f35f9ffb700 1 -- 172.21.15.82:0/1734338042 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f35f0020070 con 0x7f35fc107310 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f3602f16700 1 -- 172.21.15.82:0/1734338042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35fc19f1b0 con 0x7f35fc107310 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.271+0000 7f35f9ffb700 1 -- 172.21.15.82:0/1734338042 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35f0019630 con 0x7f35fc107310 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.272+0000 7f3602f16700 1 -- 172.21.15.82:0/1734338042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35fc19f550 con 0x7f35fc107310 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.272+0000 7f35f9ffb700 1 -- 172.21.15.82:0/1734338042 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f35f00198a0 con 0x7f35fc107310 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.273+0000 7f3602f16700 1 -- 172.21.15.82:0/1734338042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35fc10b8c0 con 0x7f35fc107310 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.273+0000 7f35f9ffb700 1 --2- 172.21.15.82:0/1734338042 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f35ec039670 0x7f35ec03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.273+0000 7f35f9ffb700 1 -- 172.21.15.82:0/1734338042 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f35f004dae0 con 0x7f35fc107310 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.273+0000 7f35fbfff700 1 --2- 172.21.15.82:0/1734338042 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f35ec039670 0x7f35ec03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.273+0000 7f35fbfff700 1 --2- 172.21.15.82:0/1734338042 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f35ec039670 0x7f35ec03bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f35e8006fd0 tx=0x7f35e8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.278+0000 7f35f9ffb700 1 -- 172.21.15.82:0/1734338042 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35f004d500 con 0x7f35fc107310 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.470+0000 7f3602f16700 1 -- 172.21.15.82:0/1734338042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f35fc0640a0 con 0x7f35fc107310 2024-04-21T08:36:52.037 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.471+0000 7f35f9ffb700 1 -- 172.21.15.82:0/1734338042 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f35f00042a0 con 0x7f35fc107310 2024-04-21T08:36:52.038 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.474+0000 7f3602f16700 1 -- 172.21.15.82:0/1734338042 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f35ec039670 msgr2=0x7f35ec03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.038 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.474+0000 7f3602f16700 1 --2- 172.21.15.82:0/1734338042 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f35ec039670 0x7f35ec03bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f35e8006fd0 tx=0x7f35e8006e40 comp rx=0 tx=0).stop 2024-04-21T08:36:52.038 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.474+0000 7f3602f16700 1 -- 172.21.15.82:0/1734338042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 msgr2=0x7f35fc07ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:52.038 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.474+0000 7f3602f16700 1 --2- 172.21.15.82:0/1734338042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 0x7f35fc07ba40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f35f00117b0 tx=0x7f35f0011b70 comp rx=0 tx=0).stop 2024-04-21T08:36:52.038 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.474+0000 7f3602f16700 1 -- 172.21.15.82:0/1734338042 shutdown_connections 2024-04-21T08:36:52.038 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.474+0000 7f3602f16700 1 --2- 172.21.15.82:0/1734338042 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f35ec039670 0x7f35ec03bb30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.038 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.474+0000 7f3602f16700 1 --2- 172.21.15.82:0/1734338042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35fc107310 0x7f35fc07ba40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:52.038 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.474+0000 7f3602f16700 1 -- 172.21.15.82:0/1734338042 >> 172.21.15.82:0/1734338042 conn(0x7f35fc1028b0 msgr2=0x7f35fc1033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:52.038 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.474+0000 7f3602f16700 1 -- 172.21.15.82:0/1734338042 shutdown_connections 2024-04-21T08:36:52.038 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:51.474+0000 7f3602f16700 1 -- 172.21.15.82:0/1734338042 wait complete. 2024-04-21T08:36:52.038 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling cephadm module... 2024-04-21T08:36:53.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:52 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4112960383' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-21T08:36:53.108 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:52 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: ignoring --setuser ceph since I am not root 2024-04-21T08:36:53.108 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:52 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: ignoring --setgroup ceph since I am not root 2024-04-21T08:36:53.108 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:52 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:52.948+0000 7fb9d33f9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:36:53.536 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:53.131+0000 7fb9d33f9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:36:53.537 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:53.234+0000 7fb9d33f9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:36:53.537 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:53.537+0000 7fb9d33f9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:36:53.787 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:53.622+0000 7fb9d33f9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:36:53.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.469+0000 7f211cf80700 1 Processor -- start 2024-04-21T08:36:53.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.469+0000 7f211cf80700 1 -- start start 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.469+0000 7f211cf80700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 0x7f21181088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.469+0000 7f211cf80700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2118108ea0 con 0x7f21181084b0 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.469+0000 7f211659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 0x7f21181088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.469+0000 7f211659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 0x7f21181088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54822/0 (socket says 172.21.15.82:54822) 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.469+0000 7f211659c700 1 -- 172.21.15.82:0/2398264852 learned_addr learned my addr 172.21.15.82:0/2398264852 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.470+0000 7f211659c700 1 -- 172.21.15.82:0/2398264852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2118109660 con 0x7f21181084b0 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.470+0000 7f211659c700 1 --2- 172.21.15.82:0/2398264852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 0x7f21181088d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2110009e80 tx=0x7f211000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f0e430b0676360 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.470+0000 7f211559a700 1 -- 172.21.15.82:0/2398264852 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2110017070 con 0x7f21181084b0 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.470+0000 7f211559a700 1 -- 172.21.15.82:0/2398264852 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f211000b8c0 con 0x7f21181084b0 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.470+0000 7f211559a700 1 -- 172.21.15.82:0/2398264852 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2110004490 con 0x7f21181084b0 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.471+0000 7f211cf80700 1 -- 172.21.15.82:0/2398264852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 msgr2=0x7f21181088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.471+0000 7f211cf80700 1 --2- 172.21.15.82:0/2398264852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 0x7f21181088d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2110009e80 tx=0x7f211000b330 comp rx=0 tx=0).stop 2024-04-21T08:36:53.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.471+0000 7f211cf80700 1 -- 172.21.15.82:0/2398264852 shutdown_connections 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.471+0000 7f211cf80700 1 --2- 172.21.15.82:0/2398264852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 0x7f21181088d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.471+0000 7f211cf80700 1 -- 172.21.15.82:0/2398264852 >> 172.21.15.82:0/2398264852 conn(0x7f2118103af0 msgr2=0x7f2118105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.471+0000 7f211cf80700 1 -- 172.21.15.82:0/2398264852 shutdown_connections 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.471+0000 7f211cf80700 1 -- 172.21.15.82:0/2398264852 wait complete. 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.471+0000 7f211cf80700 1 Processor -- start 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.471+0000 7f211cf80700 1 -- start start 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.472+0000 7f211cf80700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 0x7f211819c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.472+0000 7f211cf80700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f211810c890 con 0x7f21181084b0 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.472+0000 7f211659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 0x7f211819c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.472+0000 7f211659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 0x7f211819c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54826/0 (socket says 172.21.15.82:54826) 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.472+0000 7f211659c700 1 -- 172.21.15.82:0/4112960383 learned_addr learned my addr 172.21.15.82:0/4112960383 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.472+0000 7f211659c700 1 -- 172.21.15.82:0/4112960383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211819c6c0 con 0x7f21181084b0 2024-04-21T08:36:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.472+0000 7f211659c700 1 --2- 172.21.15.82:0/4112960383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 0x7f211819c180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2110004a60 tx=0x7f211000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.472+0000 7f21077fe700 1 -- 172.21.15.82:0/4112960383 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2110020020 con 0x7f21181084b0 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.472+0000 7f21077fe700 1 -- 172.21.15.82:0/4112960383 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2110003c60 con 0x7f21181084b0 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.472+0000 7f21077fe700 1 -- 172.21.15.82:0/4112960383 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2110017040 con 0x7f21181084b0 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.472+0000 7f211cf80700 1 -- 172.21.15.82:0/4112960383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211819c950 con 0x7f21181084b0 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.472+0000 7f211cf80700 1 -- 172.21.15.82:0/4112960383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f211819f640 con 0x7f21181084b0 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.473+0000 7f21077fe700 1 -- 172.21.15.82:0/4112960383 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f2110023020 con 0x7f21181084b0 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.473+0000 7f211cf80700 1 -- 172.21.15.82:0/4112960383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21180640a0 con 0x7f21181084b0 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.473+0000 7f21077fe700 1 --2- 172.21.15.82:0/4112960383 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f2100039670 0x7f210003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.473+0000 7f21077fe700 1 -- 172.21.15.82:0/4112960383 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f211000f050 con 0x7f21181084b0 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.473+0000 7f2115d9b700 1 --2- 172.21.15.82:0/4112960383 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f2100039670 0x7f210003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.474+0000 7f2115d9b700 1 --2- 172.21.15.82:0/4112960383 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f2100039670 0x7f210003bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f210c006fd0 tx=0x7f210c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.478+0000 7f21077fe700 1 -- 172.21.15.82:0/4112960383 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f211002a020 con 0x7f21181084b0 2024-04-21T08:36:53.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.731+0000 7f211cf80700 1 -- 172.21.15.82:0/4112960383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f211819fd30 con 0x7f21181084b0 2024-04-21T08:36:53.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.783+0000 7f21077fe700 1 -- 172.21.15.82:0/4112960383 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f2110003e80 con 0x7f21181084b0 2024-04-21T08:36:53.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.784+0000 7f21077fe700 1 -- 172.21.15.82:0/4112960383 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f2110025400 con 0x7f21181084b0 2024-04-21T08:36:53.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.785+0000 7f211cf80700 1 -- 172.21.15.82:0/4112960383 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f2100039670 msgr2=0x7f210003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.785+0000 7f211cf80700 1 --2- 172.21.15.82:0/4112960383 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f2100039670 0x7f210003bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f210c006fd0 tx=0x7f210c006e40 comp rx=0 tx=0).stop 2024-04-21T08:36:53.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.785+0000 7f211cf80700 1 -- 172.21.15.82:0/4112960383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 msgr2=0x7f211819c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:53.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.785+0000 7f211cf80700 1 --2- 172.21.15.82:0/4112960383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 0x7f211819c180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2110004a60 tx=0x7f211000bd50 comp rx=0 tx=0).stop 2024-04-21T08:36:53.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.785+0000 7f211cf80700 1 -- 172.21.15.82:0/4112960383 shutdown_connections 2024-04-21T08:36:53.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.785+0000 7f211cf80700 1 --2- 172.21.15.82:0/4112960383 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f2100039670 0x7f210003bb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.786+0000 7f211cf80700 1 --2- 172.21.15.82:0/4112960383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21181084b0 0x7f211819c180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:53.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.786+0000 7f211cf80700 1 -- 172.21.15.82:0/4112960383 >> 172.21.15.82:0/4112960383 conn(0x7f2118103af0 msgr2=0x7f21181044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:53.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.786+0000 7f211cf80700 1 -- 172.21.15.82:0/4112960383 shutdown_connections 2024-04-21T08:36:53.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:52.786+0000 7f211cf80700 1 -- 172.21.15.82:0/4112960383 wait complete. 2024-04-21T08:36:54.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:53 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4112960383' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-21T08:36:54.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:53 smithi082 ceph-mon[27967]: mgrmap e5: x(active, since 14s) 2024-04-21T08:36:54.999 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:54 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3752485150' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-21T08:36:54.999 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:54 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:54.828+0000 7fb9d33f9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:36:54.999 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:54.999+0000 7fb9d33f9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:36:55.358 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:55.170+0000 7fb9d33f9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:36:55.378 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:36:55.378 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-21T08:36:55.379 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T08:36:55.379 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-21T08:36:55.379 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-21T08:36:55.379 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:36:55.379 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.457+0000 7f036b5a6700 1 Processor -- start 2024-04-21T08:36:55.380 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.457+0000 7f036b5a6700 1 -- start start 2024-04-21T08:36:55.380 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.457+0000 7f036b5a6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 0x7f0364108c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:55.380 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.457+0000 7f036b5a6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0364109250 con 0x7f0364108860 2024-04-21T08:36:55.380 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.457+0000 7f0369342700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 0x7f0364108c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:55.380 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.457+0000 7f0369342700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 0x7f0364108c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38566/0 (socket says 172.21.15.82:38566) 2024-04-21T08:36:55.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.457+0000 7f0369342700 1 -- 172.21.15.82:0/3192107410 learned_addr learned my addr 172.21.15.82:0/3192107410 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:55.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.458+0000 7f0369342700 1 -- 172.21.15.82:0/3192107410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03641093d0 con 0x7f0364108860 2024-04-21T08:36:55.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.458+0000 7f0369342700 1 --2- 172.21.15.82:0/3192107410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 0x7f0364108c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03600098a0 tx=0x7f0360009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f84a9225ef95a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:55.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.458+0000 7f035bfff700 1 -- 172.21.15.82:0/3192107410 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0360004160 con 0x7f0364108860 2024-04-21T08:36:55.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.458+0000 7f035bfff700 1 -- 172.21.15.82:0/3192107410 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0360004300 con 0x7f0364108860 2024-04-21T08:36:55.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.458+0000 7f035bfff700 1 -- 172.21.15.82:0/3192107410 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0360011af0 con 0x7f0364108860 2024-04-21T08:36:55.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.459+0000 7f036b5a6700 1 -- 172.21.15.82:0/3192107410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 msgr2=0x7f0364108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.459+0000 7f036b5a6700 1 --2- 172.21.15.82:0/3192107410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 0x7f0364108c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03600098a0 tx=0x7f0360009bb0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.459+0000 7f036b5a6700 1 -- 172.21.15.82:0/3192107410 shutdown_connections 2024-04-21T08:36:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.459+0000 7f036b5a6700 1 --2- 172.21.15.82:0/3192107410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 0x7f0364108c80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.459+0000 7f036b5a6700 1 -- 172.21.15.82:0/3192107410 >> 172.21.15.82:0/3192107410 conn(0x7f0364101f70 msgr2=0x7f03641043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.459+0000 7f036b5a6700 1 -- 172.21.15.82:0/3192107410 shutdown_connections 2024-04-21T08:36:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.459+0000 7f036b5a6700 1 -- 172.21.15.82:0/3192107410 wait complete. 2024-04-21T08:36:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.459+0000 7f036b5a6700 1 Processor -- start 2024-04-21T08:36:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f036b5a6700 1 -- start start 2024-04-21T08:36:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f036b5a6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 0x7f036419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f036b5a6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0364107100 con 0x7f0364108860 2024-04-21T08:36:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f0369342700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 0x7f036419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f0369342700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 0x7f036419e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38568/0 (socket says 172.21.15.82:38568) 2024-04-21T08:36:55.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f0369342700 1 -- 172.21.15.82:0/3752485150 learned_addr learned my addr 172.21.15.82:0/3752485150 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:36:55.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f0369342700 1 -- 172.21.15.82:0/3752485150 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036419e8e0 con 0x7f0364108860 2024-04-21T08:36:55.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f0369342700 1 --2- 172.21.15.82:0/3752485150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 0x7f036419e3a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0360003f40 tx=0x7f0360011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:55.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f035a7fc700 1 -- 172.21.15.82:0/3752485150 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03600038b0 con 0x7f0364108860 2024-04-21T08:36:55.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f036b5a6700 1 -- 172.21.15.82:0/3752485150 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036419ebd0 con 0x7f0364108860 2024-04-21T08:36:55.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f035a7fc700 1 -- 172.21.15.82:0/3752485150 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0360003a50 con 0x7f0364108860 2024-04-21T08:36:55.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f035a7fc700 1 -- 172.21.15.82:0/3752485150 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f036001a5c0 con 0x7f0364108860 2024-04-21T08:36:55.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.460+0000 7f036b5a6700 1 -- 172.21.15.82:0/3752485150 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03641a1860 con 0x7f0364108860 2024-04-21T08:36:55.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.461+0000 7f035a7fc700 1 -- 172.21.15.82:0/3752485150 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f036001a8c0 con 0x7f0364108860 2024-04-21T08:36:55.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.461+0000 7f035a7fc700 1 --2- 172.21.15.82:0/3752485150 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f0350039620 0x7f035003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:55.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.461+0000 7f035a7fc700 1 -- 172.21.15.82:0/3752485150 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0360024430 con 0x7f0364108860 2024-04-21T08:36:55.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.461+0000 7f0368b41700 1 -- 172.21.15.82:0/3752485150 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f0350039620 msgr2=0x7f035003bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2710226711 2024-04-21T08:36:55.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.461+0000 7f0368b41700 1 --2- 172.21.15.82:0/3752485150 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f0350039620 0x7f035003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:36:55.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.461+0000 7f036b5a6700 1 -- 172.21.15.82:0/3752485150 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03640640a0 con 0x7f0364108860 2024-04-21T08:36:55.385 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.467+0000 7f035a7fc700 1 -- 172.21.15.82:0/3752485150 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f036000f2e0 con 0x7f0364108860 2024-04-21T08:36:55.385 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.663+0000 7f0368b41700 1 -- 172.21.15.82:0/3752485150 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f0350039620 msgr2=0x7f035003bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2710226711 2024-04-21T08:36:55.385 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.663+0000 7f0368b41700 1 --2- 172.21.15.82:0/3752485150 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f0350039620 0x7f035003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:36:55.385 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.730+0000 7f036b5a6700 1 -- 172.21.15.82:0/3752485150 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f03641a1a30 con 0x7f0364108860 2024-04-21T08:36:55.385 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.730+0000 7f035a7fc700 1 -- 172.21.15.82:0/3752485150 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f03641a1a30 con 0x7f0364108860 2024-04-21T08:36:55.385 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.733+0000 7f036b5a6700 1 -- 172.21.15.82:0/3752485150 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f0350039620 msgr2=0x7f035003bae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:36:55.386 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.733+0000 7f036b5a6700 1 --2- 172.21.15.82:0/3752485150 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f0350039620 0x7f035003bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.386 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.733+0000 7f036b5a6700 1 -- 172.21.15.82:0/3752485150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 msgr2=0x7f036419e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:55.386 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.733+0000 7f036b5a6700 1 --2- 172.21.15.82:0/3752485150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 0x7f036419e3a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0360003f40 tx=0x7f0360011890 comp rx=0 tx=0).stop 2024-04-21T08:36:55.386 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.733+0000 7f036b5a6700 1 -- 172.21.15.82:0/3752485150 shutdown_connections 2024-04-21T08:36:55.386 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.733+0000 7f036b5a6700 1 --2- 172.21.15.82:0/3752485150 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7f0350039620 0x7f035003bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.386 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.733+0000 7f036b5a6700 1 --2- 172.21.15.82:0/3752485150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0364108860 0x7f036419e3a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.386 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.733+0000 7f036b5a6700 1 -- 172.21.15.82:0/3752485150 >> 172.21.15.82:0/3752485150 conn(0x7f0364101f70 msgr2=0x7f03641949e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:55.386 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.733+0000 7f036b5a6700 1 -- 172.21.15.82:0/3752485150 shutdown_connections 2024-04-21T08:36:55.386 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:54.733+0000 7f036b5a6700 1 -- 172.21.15.82:0/3752485150 wait complete. 2024-04-21T08:36:55.386 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for the mgr to restart... 2024-04-21T08:36:55.387 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr epoch 5... 2024-04-21T08:36:55.680 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:55.465+0000 7fb9d33f9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:36:55.680 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:55.559+0000 7fb9d33f9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:36:55.963 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:55.769+0000 7fb9d33f9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:36:56.358 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:55.963+0000 7fb9d33f9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:36:56.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:56 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:56.556+0000 7fb9d33f9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:36:57.268 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:56 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:56.974+0000 7fb9d33f9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:36:57.268 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:57 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:57.068+0000 7fb9d33f9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:36:57.268 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:57 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:57.268+0000 7fb9d33f9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:36:58.108 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:57 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:57.785+0000 7fb9d33f9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:36:58.108 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:57 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:57.879+0000 7fb9d33f9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:36:58.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:58.123+0000 7fb9d33f9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:36:58.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:58.255+0000 7fb9d33f9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:36:58.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:58.349+0000 7fb9d33f9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:36:58.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:58.464+0000 7fb9d33f9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:36:59.059 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:58.761+0000 7fb9d33f9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:36:59.626 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:59.361+0000 7fb9d33f9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:36:59.626 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:36:59.576+0000 7fb9d33f9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:37:00.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: Active manager daemon x restarted 2024-04-21T08:37:00.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: Activating manager daemon x 2024-04-21T08:37:00.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: osdmap e2: 0 total, 0 up, 0 in 2024-04-21T08:37:00.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: mgrmap e6: x(active, starting, since 0.0035824s) 2024-04-21T08:37:00.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:37:00.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:37:00.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:37:00.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:37:00.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:37:00.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: Manager daemon x is now available 2024-04-21T08:37:00.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:00.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:00.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:00.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:36:59 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:00.841 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:00 smithi082 ceph-mon[27967]: Found migration_current of "None". Setting to last migration. 2024-04-21T08:37:00.841 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:00 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:37:00.841 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:00 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:37:00.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:00 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:00.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:00 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:00.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:00 smithi082 ceph-mon[27967]: mgrmap e7: x(active, since 1.00528s) 2024-04-21T08:37:01.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:37:01.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.782+0000 7fefc792c700 1 Processor -- start 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.782+0000 7fefc792c700 1 -- start start 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.782+0000 7fefc792c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 0x7fefc007c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.782+0000 7fefc792c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fefc007c8e0 con 0x7fefc007de60 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.782+0000 7fefc56c8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 0x7fefc007c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.782+0000 7fefc56c8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 0x7fefc007c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38582/0 (socket says 172.21.15.82:38582) 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.782+0000 7fefc56c8700 1 -- 172.21.15.82:0/4292073881 learned_addr learned my addr 172.21.15.82:0/4292073881 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.783+0000 7fefc56c8700 1 -- 172.21.15.82:0/4292073881 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc007ca60 con 0x7fefc007de60 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.783+0000 7fefc56c8700 1 --2- 172.21.15.82:0/4292073881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 0x7fefc007c310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0098a0 tx=0x7fefbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c5d6b1100df46f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.783+0000 7fefb7fff700 1 -- 172.21.15.82:0/4292073881 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefbc004160 con 0x7fefc007de60 2024-04-21T08:37:01.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.783+0000 7fefb7fff700 1 -- 172.21.15.82:0/4292073881 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fefbc004300 con 0x7fefc007de60 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.783+0000 7fefb7fff700 1 -- 172.21.15.82:0/4292073881 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefbc011af0 con 0x7fefc007de60 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.784+0000 7fefc792c700 1 -- 172.21.15.82:0/4292073881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 msgr2=0x7fefc007c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.784+0000 7fefc792c700 1 --2- 172.21.15.82:0/4292073881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 0x7fefc007c310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0098a0 tx=0x7fefbc009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.784+0000 7fefc792c700 1 -- 172.21.15.82:0/4292073881 shutdown_connections 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.784+0000 7fefc792c700 1 --2- 172.21.15.82:0/4292073881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 0x7fefc007c310 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.784+0000 7fefc792c700 1 -- 172.21.15.82:0/4292073881 >> 172.21.15.82:0/4292073881 conn(0x7fefc01039c0 msgr2=0x7fefc0105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.784+0000 7fefc792c700 1 -- 172.21.15.82:0/4292073881 shutdown_connections 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.784+0000 7fefc792c700 1 -- 172.21.15.82:0/4292073881 wait complete. 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.784+0000 7fefc792c700 1 Processor -- start 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.784+0000 7fefc792c700 1 -- start start 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.784+0000 7fefc792c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 0x7fefc019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.784+0000 7fefc792c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fefc010e8d0 con 0x7fefc007de60 2024-04-21T08:37:01.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.785+0000 7fefc56c8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 0x7fefc019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.785+0000 7fefc56c8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 0x7fefc019e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38586/0 (socket says 172.21.15.82:38586) 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.785+0000 7fefc56c8700 1 -- 172.21.15.82:0/3587645966 learned_addr learned my addr 172.21.15.82:0/3587645966 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.785+0000 7fefc56c8700 1 -- 172.21.15.82:0/3587645966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc019e900 con 0x7fefc007de60 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.785+0000 7fefc56c8700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 0x7fefc019e3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fefbc003f40 tx=0x7fefbc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.785+0000 7fefb67fc700 1 -- 172.21.15.82:0/3587645966 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefbc0038b0 con 0x7fefc007de60 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.785+0000 7fefb67fc700 1 -- 172.21.15.82:0/3587645966 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fefbc003a50 con 0x7fefc007de60 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.785+0000 7fefc792c700 1 -- 172.21.15.82:0/3587645966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc019eb90 con 0x7fefc007de60 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.785+0000 7fefb67fc700 1 -- 172.21.15.82:0/3587645966 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefbc01a5c0 con 0x7fefc007de60 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.786+0000 7fefc792c700 1 -- 172.21.15.82:0/3587645966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc01a1880 con 0x7fefc007de60 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.786+0000 7fefb67fc700 1 -- 172.21.15.82:0/3587645966 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fefbc01a8c0 con 0x7fefc007de60 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.786+0000 7fefb67fc700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 0x7fefac03bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.786+0000 7fefc792c700 1 -- 172.21.15.82:0/3587645966 --> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fefc0069b50 con 0x7fefac039620 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.786+0000 7fefb67fc700 1 -- 172.21.15.82:0/3587645966 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fefbc04db30 con 0x7fefc007de60 2024-04-21T08:37:01.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.786+0000 7fefc4ec7700 1 -- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 msgr2=0x7fefac03bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2710226711 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.786+0000 7fefc4ec7700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 0x7fefac03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.987+0000 7fefc4ec7700 1 -- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 msgr2=0x7fefac03bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2710226711 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:55.987+0000 7fefc4ec7700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 0x7fefac03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:56.388+0000 7fefc4ec7700 1 -- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 msgr2=0x7fefac03bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2710226711 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:56.388+0000 7fefc4ec7700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 0x7fefac03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:57.189+0000 7fefc4ec7700 1 -- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 msgr2=0x7fefac03bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2710226711 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:57.189+0000 7fefc4ec7700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 0x7fefac03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:58.790+0000 7fefc4ec7700 1 -- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 msgr2=0x7fefac03bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2710226711 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:58.790+0000 7fefc4ec7700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 0x7fefac03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:59.579+0000 7fefb67fc700 1 -- 172.21.15.82:0/3587645966 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fefbc02d910 con 0x7fefc007de60 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:59.580+0000 7fefb67fc700 1 -- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 msgr2=0x7fefac03bae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:36:59.580+0000 7fefb67fc700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 0x7fefac03bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.581+0000 7fefb67fc700 1 -- 172.21.15.82:0/3587645966 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fefbc017d70 con 0x7fefc007de60 2024-04-21T08:37:01.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.581+0000 7fefb67fc700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7fefac03cd20 0x7fefac03f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.581+0000 7fefb67fc700 1 -- 172.21.15.82:0/3587645966 --> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fefbc007da0 con 0x7fefac03cd20 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.581+0000 7fefc4ec7700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7fefac03cd20 0x7fefac03f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.582+0000 7fefc4ec7700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7fefac03cd20 0x7fefac03f110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fefb0003a10 tx=0x7fefb0009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.582+0000 7fefb67fc700 1 -- 172.21.15.82:0/3587645966 <== mgr.14120 v2:172.21.15.82:6800/880382736 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fefbc007da0 con 0x7fefac03cd20 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefc792c700 1 -- 172.21.15.82:0/3587645966 --> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fefc0104b30 con 0x7fefac03cd20 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefb67fc700 1 -- 172.21.15.82:0/3587645966 <== mgr.14120 v2:172.21.15.82:6800/880382736 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fefc0104b30 con 0x7fefac03cd20 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefc792c700 1 -- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7fefac03cd20 msgr2=0x7fefac03f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefc792c700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7fefac03cd20 0x7fefac03f110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fefb0003a10 tx=0x7fefb0009290 comp rx=0 tx=0).stop 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefc792c700 1 -- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 msgr2=0x7fefc019e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefc792c700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 0x7fefc019e3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fefbc003f40 tx=0x7fefbc011890 comp rx=0 tx=0).stop 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefc792c700 1 -- 172.21.15.82:0/3587645966 shutdown_connections 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefc792c700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7fefac03cd20 0x7fefac03f110 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:01.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefc792c700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefc007de60 0x7fefc019e3c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:01.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefc792c700 1 --2- 172.21.15.82:0/3587645966 >> [v2:172.21.15.82:6800/2710226711,v1:172.21.15.82:6801/2710226711] conn(0x7fefac039620 0x7fefac03bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:01.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefc792c700 1 -- 172.21.15.82:0/3587645966 >> 172.21.15.82:0/3587645966 conn(0x7fefc01039c0 msgr2=0x7fefc0104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:01.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefc792c700 1 -- 172.21.15.82:0/3587645966 shutdown_connections 2024-04-21T08:37:01.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:00.589+0000 7fefc792c700 1 -- 172.21.15.82:0/3587645966 wait complete. 2024-04-21T08:37:01.174 INFO:teuthology.orchestra.run.smithi082.stdout:mgr epoch 5 is available 2024-04-21T08:37:01.174 INFO:teuthology.orchestra.run.smithi082.stdout:Setting orchestrator backend to cephadm... 2024-04-21T08:37:01.789 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:01 smithi082 ceph-mon[27967]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-21T08:37:01.789 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:01 smithi082 ceph-mon[27967]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-21T08:37:01.789 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:01 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:02.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.561+0000 7f6a297a4700 1 Processor -- start 2024-04-21T08:37:02.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.561+0000 7f6a297a4700 1 -- start start 2024-04-21T08:37:02.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.562+0000 7f6a297a4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 0x7f6a24108840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:02.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.562+0000 7f6a297a4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24108e10 con 0x7f6a24108420 2024-04-21T08:37:02.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.562+0000 7f6a22ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 0x7f6a24108840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:02.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.562+0000 7f6a22ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 0x7f6a24108840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38672/0 (socket says 172.21.15.82:38672) 2024-04-21T08:37:02.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.562+0000 7f6a22ffd700 1 -- 172.21.15.82:0/3226260860 learned_addr learned my addr 172.21.15.82:0/3226260860 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:02.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.562+0000 7f6a22ffd700 1 -- 172.21.15.82:0/3226260860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a241095d0 con 0x7f6a24108420 2024-04-21T08:37:02.381 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.562+0000 7f6a22ffd700 1 --2- 172.21.15.82:0/3226260860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 0x7f6a24108840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6a140098a0 tx=0x7f6a14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed564be9c3d3d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.562+0000 7f6a227fc700 1 -- 172.21.15.82:0/3226260860 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a14004160 con 0x7f6a24108420 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.562+0000 7f6a227fc700 1 -- 172.21.15.82:0/3226260860 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6a14004300 con 0x7f6a24108420 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.562+0000 7f6a227fc700 1 -- 172.21.15.82:0/3226260860 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a14011af0 con 0x7f6a24108420 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.563+0000 7f6a297a4700 1 -- 172.21.15.82:0/3226260860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 msgr2=0x7f6a24108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.563+0000 7f6a297a4700 1 --2- 172.21.15.82:0/3226260860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 0x7f6a24108840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6a140098a0 tx=0x7f6a14009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.563+0000 7f6a297a4700 1 -- 172.21.15.82:0/3226260860 shutdown_connections 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.563+0000 7f6a297a4700 1 --2- 172.21.15.82:0/3226260860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 0x7f6a24108840 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.563+0000 7f6a297a4700 1 -- 172.21.15.82:0/3226260860 >> 172.21.15.82:0/3226260860 conn(0x7f6a241039e0 msgr2=0x7f6a24105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.563+0000 7f6a297a4700 1 -- 172.21.15.82:0/3226260860 shutdown_connections 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.563+0000 7f6a297a4700 1 -- 172.21.15.82:0/3226260860 wait complete. 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a297a4700 1 Processor -- start 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a297a4700 1 -- start start 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a297a4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 0x7f6a2419c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:02.382 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a297a4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410c800 con 0x7f6a24108420 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a22ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 0x7f6a2419c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a22ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 0x7f6a2419c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38680/0 (socket says 172.21.15.82:38680) 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a22ffd700 1 -- 172.21.15.82:0/2874855732 learned_addr learned my addr 172.21.15.82:0/2874855732 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a22ffd700 1 -- 172.21.15.82:0/2874855732 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2419c700 con 0x7f6a24108420 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a22ffd700 1 --2- 172.21.15.82:0/2874855732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 0x7f6a2419c1c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6a14003f40 tx=0x7f6a14011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a20ff9700 1 -- 172.21.15.82:0/2874855732 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a140038b0 con 0x7f6a24108420 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a20ff9700 1 -- 172.21.15.82:0/2874855732 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6a14003a50 con 0x7f6a24108420 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a297a4700 1 -- 172.21.15.82:0/2874855732 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2419c990 con 0x7f6a24108420 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a20ff9700 1 -- 172.21.15.82:0/2874855732 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a1401a5c0 con 0x7f6a24108420 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.564+0000 7f6a297a4700 1 -- 172.21.15.82:0/2874855732 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2419f680 con 0x7f6a24108420 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.565+0000 7f6a20ff9700 1 -- 172.21.15.82:0/2874855732 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f6a1401a8c0 con 0x7f6a24108420 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.565+0000 7f6a20ff9700 1 --2- 172.21.15.82:0/2874855732 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f6a10039510 0x7f6a1003b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.565+0000 7f6a20ff9700 1 -- 172.21.15.82:0/2874855732 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6a1404d970 con 0x7f6a24108420 2024-04-21T08:37:02.383 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.565+0000 7f6a297a4700 1 -- 172.21.15.82:0/2874855732 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2410c980 con 0x7f6a24108420 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.566+0000 7f6a1bfff700 1 --2- 172.21.15.82:0/2874855732 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f6a10039510 0x7f6a1003b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.566+0000 7f6a1bfff700 1 --2- 172.21.15.82:0/2874855732 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f6a10039510 0x7f6a1003b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c006fd0 tx=0x7f6a0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.571+0000 7f6a20ff9700 1 -- 172.21.15.82:0/2874855732 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a14019c70 con 0x7f6a24108420 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.632+0000 7f6a20ff9700 1 -- 172.21.15.82:0/2874855732 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6a1402d910 con 0x7f6a24108420 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.789+0000 7f6a297a4700 1 -- 172.21.15.82:0/2874855732 --> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f6a24104af0 con 0x7f6a10039510 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.794+0000 7f6a20ff9700 1 -- 172.21.15.82:0/2874855732 <== mgr.14120 v2:172.21.15.82:6800/880382736 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a24104af0 con 0x7f6a10039510 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.796+0000 7f6a297a4700 1 -- 172.21.15.82:0/2874855732 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f6a10039510 msgr2=0x7f6a1003b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.796+0000 7f6a297a4700 1 --2- 172.21.15.82:0/2874855732 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f6a10039510 0x7f6a1003b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c006fd0 tx=0x7f6a0c006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.796+0000 7f6a297a4700 1 -- 172.21.15.82:0/2874855732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 msgr2=0x7f6a2419c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.797+0000 7f6a297a4700 1 --2- 172.21.15.82:0/2874855732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 0x7f6a2419c1c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6a14003f40 tx=0x7f6a14011890 comp rx=0 tx=0).stop 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.797+0000 7f6a297a4700 1 -- 172.21.15.82:0/2874855732 shutdown_connections 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.797+0000 7f6a297a4700 1 --2- 172.21.15.82:0/2874855732 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f6a10039510 0x7f6a1003b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.384 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.797+0000 7f6a297a4700 1 --2- 172.21.15.82:0/2874855732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24108420 0x7f6a2419c1c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:02.385 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.797+0000 7f6a297a4700 1 -- 172.21.15.82:0/2874855732 >> 172.21.15.82:0/2874855732 conn(0x7f6a241039e0 msgr2=0x7f6a241043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:02.385 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.797+0000 7f6a297a4700 1 -- 172.21.15.82:0/2874855732 shutdown_connections 2024-04-21T08:37:02.385 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:01.797+0000 7f6a297a4700 1 -- 172.21.15.82:0/2874855732 wait complete. 2024-04-21T08:37:03.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:02 smithi082 ceph-mon[27967]: [21/Apr/2024:08:37:01] ENGINE Bus STARTING 2024-04-21T08:37:03.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:02 smithi082 ceph-mon[27967]: [21/Apr/2024:08:37:01] ENGINE Serving on http://172.21.15.82:8765 2024-04-21T08:37:03.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:02 smithi082 ceph-mon[27967]: [21/Apr/2024:08:37:01] ENGINE Serving on https://172.21.15.82:7150 2024-04-21T08:37:03.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:02 smithi082 ceph-mon[27967]: [21/Apr/2024:08:37:01] ENGINE Bus STARTED 2024-04-21T08:37:03.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:02 smithi082 ceph-mon[27967]: mgrmap e8: x(active, since 2s) 2024-04-21T08:37:03.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:02 smithi082 ceph-mon[27967]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:37:03.054 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:02 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:03.054 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:02 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-21T08:37:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.847+0000 7f06774f6700 1 Processor -- start 2024-04-21T08:37:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.847+0000 7f06774f6700 1 -- start start 2024-04-21T08:37:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.847+0000 7f06774f6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 0x7f067010b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.848+0000 7f06774f6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f067007be60 con 0x7f0670108ea0 2024-04-21T08:37:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.848+0000 7f0675292700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 0x7f067010b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.848+0000 7f0675292700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 0x7f067010b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44570/0 (socket says 172.21.15.82:44570) 2024-04-21T08:37:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.848+0000 7f0675292700 1 -- 172.21.15.82:0/1008214850 learned_addr learned my addr 172.21.15.82:0/1008214850 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.848+0000 7f0675292700 1 -- 172.21.15.82:0/1008214850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067010b7d0 con 0x7f0670108ea0 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.848+0000 7f0675292700 1 --2- 172.21.15.82:0/1008214850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 0x7f067010b290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f06600098a0 tx=0x7f0660009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f195cd5efd68c467 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.848+0000 7f0667fff700 1 -- 172.21.15.82:0/1008214850 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0660004160 con 0x7f0670108ea0 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.848+0000 7f0667fff700 1 -- 172.21.15.82:0/1008214850 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0660004300 con 0x7f0670108ea0 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.848+0000 7f0667fff700 1 -- 172.21.15.82:0/1008214850 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0660011af0 con 0x7f0670108ea0 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.849+0000 7f06774f6700 1 -- 172.21.15.82:0/1008214850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 msgr2=0x7f067010b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.849+0000 7f06774f6700 1 --2- 172.21.15.82:0/1008214850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 0x7f067010b290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f06600098a0 tx=0x7f0660009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.849+0000 7f06774f6700 1 -- 172.21.15.82:0/1008214850 shutdown_connections 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.849+0000 7f06774f6700 1 --2- 172.21.15.82:0/1008214850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 0x7f067010b290 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.849+0000 7f06774f6700 1 -- 172.21.15.82:0/1008214850 >> 172.21.15.82:0/1008214850 conn(0x7f0670102890 msgr2=0x7f0670104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.849+0000 7f06774f6700 1 -- 172.21.15.82:0/1008214850 shutdown_connections 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.849+0000 7f06774f6700 1 -- 172.21.15.82:0/1008214850 wait complete. 2024-04-21T08:37:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.849+0000 7f06774f6700 1 Processor -- start 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f06774f6700 1 -- start start 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f06774f6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 0x7f067019c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f06774f6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f067010c720 con 0x7f0670108ea0 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f0675292700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 0x7f067019c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f0675292700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 0x7f067019c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44584/0 (socket says 172.21.15.82:44584) 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f0675292700 1 -- 172.21.15.82:0/615962402 learned_addr learned my addr 172.21.15.82:0/615962402 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f0675292700 1 -- 172.21.15.82:0/615962402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067019c720 con 0x7f0670108ea0 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f0675292700 1 --2- 172.21.15.82:0/615962402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 0x7f067019c1e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0660011ac0 tx=0x7f0660011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f06667fc700 1 -- 172.21.15.82:0/615962402 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0660003b20 con 0x7f0670108ea0 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f06667fc700 1 -- 172.21.15.82:0/615962402 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f066001a8c0 con 0x7f0670108ea0 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f06774f6700 1 -- 172.21.15.82:0/615962402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067019c9b0 con 0x7f0670108ea0 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f06667fc700 1 -- 172.21.15.82:0/615962402 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06600248d0 con 0x7f0670108ea0 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.850+0000 7f06774f6700 1 -- 172.21.15.82:0/615962402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067019f6a0 con 0x7f0670108ea0 2024-04-21T08:37:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.851+0000 7f06667fc700 1 -- 172.21.15.82:0/615962402 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0660022020 con 0x7f0670108ea0 2024-04-21T08:37:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.851+0000 7f06774f6700 1 -- 172.21.15.82:0/615962402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0670069b50 con 0x7f0670108ea0 2024-04-21T08:37:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.852+0000 7f06667fc700 1 --2- 172.21.15.82:0/615962402 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f065c0396c0 0x7f065c03bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.852+0000 7f06667fc700 1 -- 172.21.15.82:0/615962402 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f066004d810 con 0x7f0670108ea0 2024-04-21T08:37:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.852+0000 7f0674a91700 1 --2- 172.21.15.82:0/615962402 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f065c0396c0 0x7f065c03bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.852+0000 7f0674a91700 1 --2- 172.21.15.82:0/615962402 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f065c0396c0 0x7f065c03bb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f066c006fd0 tx=0x7f066c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:02.857+0000 7f06667fc700 1 -- 172.21.15.82:0/615962402 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f066001ab20 con 0x7f0670108ea0 2024-04-21T08:37:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:03.053+0000 7f06774f6700 1 -- 172.21.15.82:0/615962402 --> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f0670103a70 con 0x7f065c0396c0 2024-04-21T08:37:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:03.054+0000 7f06667fc700 1 -- 172.21.15.82:0/615962402 <== mgr.14120 v2:172.21.15.82:6800/880382736 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f0670103a70 con 0x7f065c0396c0 2024-04-21T08:37:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:03.056+0000 7f06774f6700 1 -- 172.21.15.82:0/615962402 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f065c0396c0 msgr2=0x7f065c03bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:03.056+0000 7f06774f6700 1 --2- 172.21.15.82:0/615962402 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f065c0396c0 0x7f065c03bb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f066c006fd0 tx=0x7f066c006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:03.056+0000 7f06774f6700 1 -- 172.21.15.82:0/615962402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 msgr2=0x7f067019c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:03.056+0000 7f06774f6700 1 --2- 172.21.15.82:0/615962402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 0x7f067019c1e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0660011ac0 tx=0x7f0660011ba0 comp rx=0 tx=0).stop 2024-04-21T08:37:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:03.056+0000 7f06774f6700 1 -- 172.21.15.82:0/615962402 shutdown_connections 2024-04-21T08:37:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:03.056+0000 7f06774f6700 1 --2- 172.21.15.82:0/615962402 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f065c0396c0 0x7f065c03bb80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:03.056+0000 7f06774f6700 1 --2- 172.21.15.82:0/615962402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0670108ea0 0x7f067019c1e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:03.056+0000 7f06774f6700 1 -- 172.21.15.82:0/615962402 >> 172.21.15.82:0/615962402 conn(0x7f0670102890 msgr2=0x7f0670103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:03.056+0000 7f06774f6700 1 -- 172.21.15.82:0/615962402 shutdown_connections 2024-04-21T08:37:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:03.056+0000 7f06774f6700 1 -- 172.21.15.82:0/615962402 wait complete. 2024-04-21T08:37:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:Generating ssh key... 2024-04-21T08:37:04.016 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:03 smithi082 ceph-mon[27967]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:37:04.736 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: Generating public/private rsa key pair. 2024-04-21T08:37:04.736 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: Your identification has been saved in /tmp/tmpfqnbe99j/key. 2024-04-21T08:37:04.736 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: Your public key has been saved in /tmp/tmpfqnbe99j/key.pub. 2024-04-21T08:37:04.736 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: The key fingerprint is: 2024-04-21T08:37:04.736 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: SHA256:Wh/2dvUW5vwqUke/rZGcds8UVD2b399MzttHKxwidPM ceph-24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:37:04.736 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: The key's randomart image is: 2024-04-21T08:37:04.736 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: +---[RSA 3072]----+ 2024-04-21T08:37:04.736 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: | o| 2024-04-21T08:37:04.736 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: | .o| 2024-04-21T08:37:04.737 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: | .+| 2024-04-21T08:37:04.737 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: | . o oo | 2024-04-21T08:37:04.737 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: | S + o. =+| 2024-04-21T08:37:04.737 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: | o + +.E=+X| 2024-04-21T08:37:04.737 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: | . o.=.OO@| 2024-04-21T08:37:04.737 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: | ...= O%| 2024-04-21T08:37:04.737 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: | . .++B| 2024-04-21T08:37:04.737 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: +----[SHA256]-----+ 2024-04-21T08:37:05.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.042+0000 7f9f3f0d6700 1 Processor -- start 2024-04-21T08:37:05.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.042+0000 7f9f3f0d6700 1 -- start start 2024-04-21T08:37:05.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.042+0000 7f9f3f0d6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 0x7f9f38108840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:05.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.042+0000 7f9f3f0d6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38108e10 con 0x7f9f38108420 2024-04-21T08:37:05.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.042+0000 7f9f3ce72700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 0x7f9f38108840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:05.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.042+0000 7f9f3ce72700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 0x7f9f38108840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44594/0 (socket says 172.21.15.82:44594) 2024-04-21T08:37:05.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.042+0000 7f9f3ce72700 1 -- 172.21.15.82:0/3844106745 learned_addr learned my addr 172.21.15.82:0/3844106745 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:05.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.042+0000 7f9f3ce72700 1 -- 172.21.15.82:0/3844106745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f381095d0 con 0x7f9f38108420 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.042+0000 7f9f3ce72700 1 --2- 172.21.15.82:0/3844106745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 0x7f9f38108840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0098a0 tx=0x7f9f2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865187c58c0da4e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.043+0000 7f9f377fe700 1 -- 172.21.15.82:0/3844106745 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c004160 con 0x7f9f38108420 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.043+0000 7f9f377fe700 1 -- 172.21.15.82:0/3844106745 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9f2c004300 con 0x7f9f38108420 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.043+0000 7f9f377fe700 1 -- 172.21.15.82:0/3844106745 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c011af0 con 0x7f9f38108420 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.043+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/3844106745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 msgr2=0x7f9f38108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.043+0000 7f9f3f0d6700 1 --2- 172.21.15.82:0/3844106745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 0x7f9f38108840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0098a0 tx=0x7f9f2c009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.043+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/3844106745 shutdown_connections 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.043+0000 7f9f3f0d6700 1 --2- 172.21.15.82:0/3844106745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 0x7f9f38108840 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.043+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/3844106745 >> 172.21.15.82:0/3844106745 conn(0x7f9f381039e0 msgr2=0x7f9f38105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.043+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/3844106745 shutdown_connections 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.043+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/3844106745 wait complete. 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.044+0000 7f9f3f0d6700 1 Processor -- start 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.044+0000 7f9f3f0d6700 1 -- start start 2024-04-21T08:37:05.011 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.044+0000 7f9f3f0d6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 0x7f9f3819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.044+0000 7f9f3f0d6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3810c800 con 0x7f9f38108420 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.044+0000 7f9f3ce72700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 0x7f9f3819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.044+0000 7f9f3ce72700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 0x7f9f3819c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44608/0 (socket says 172.21.15.82:44608) 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.044+0000 7f9f3ce72700 1 -- 172.21.15.82:0/2735347860 learned_addr learned my addr 172.21.15.82:0/2735347860 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.044+0000 7f9f3ce72700 1 -- 172.21.15.82:0/2735347860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3819c700 con 0x7f9f38108420 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.044+0000 7f9f3ce72700 1 --2- 172.21.15.82:0/2735347860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 0x7f9f3819c1c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c000c00 tx=0x7f9f2c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.044+0000 7f9f35ffb700 1 -- 172.21.15.82:0/2735347860 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c003b20 con 0x7f9f38108420 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.045+0000 7f9f35ffb700 1 -- 172.21.15.82:0/2735347860 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9f2c01a8c0 con 0x7f9f38108420 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.045+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/2735347860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3819c990 con 0x7f9f38108420 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.045+0000 7f9f35ffb700 1 -- 172.21.15.82:0/2735347860 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f2c024940 con 0x7f9f38108420 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.045+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/2735347860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3819f680 con 0x7f9f38108420 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.046+0000 7f9f35ffb700 1 -- 172.21.15.82:0/2735347860 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9f2c022020 con 0x7f9f38108420 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.046+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/2735347860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3810c980 con 0x7f9f38108420 2024-04-21T08:37:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.046+0000 7f9f35ffb700 1 --2- 172.21.15.82:0/2735347860 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f9f280396c0 0x7f9f2803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.046+0000 7f9f35ffb700 1 -- 172.21.15.82:0/2735347860 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9f2c04d7d0 con 0x7f9f38108420 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.046+0000 7f9f37fff700 1 --2- 172.21.15.82:0/2735347860 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f9f280396c0 0x7f9f2803bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.046+0000 7f9f37fff700 1 --2- 172.21.15.82:0/2735347860 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f9f280396c0 0x7f9f2803bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9f24006fd0 tx=0x7f9f24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.051+0000 7f9f35ffb700 1 -- 172.21.15.82:0/2735347860 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f2c021d80 con 0x7f9f38108420 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.247+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/2735347860 --> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f9f38104af0 con 0x7f9f280396c0 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.397+0000 7f9f35ffb700 1 -- 172.21.15.82:0/2735347860 <== mgr.14120 v2:172.21.15.82:6800/880382736 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9f38104af0 con 0x7f9f280396c0 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.399+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/2735347860 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f9f280396c0 msgr2=0x7f9f2803bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.399+0000 7f9f3f0d6700 1 --2- 172.21.15.82:0/2735347860 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f9f280396c0 0x7f9f2803bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9f24006fd0 tx=0x7f9f24006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.399+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/2735347860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 msgr2=0x7f9f3819c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.399+0000 7f9f3f0d6700 1 --2- 172.21.15.82:0/2735347860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 0x7f9f3819c1c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c000c00 tx=0x7f9f2c011b70 comp rx=0 tx=0).stop 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.399+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/2735347860 shutdown_connections 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.399+0000 7f9f3f0d6700 1 --2- 172.21.15.82:0/2735347860 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f9f280396c0 0x7f9f2803bb80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.399+0000 7f9f3f0d6700 1 --2- 172.21.15.82:0/2735347860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f38108420 0x7f9f3819c1c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:05.013 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.400+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/2735347860 >> 172.21.15.82:0/2735347860 conn(0x7f9f381039e0 msgr2=0x7f9f381043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:05.014 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.400+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/2735347860 shutdown_connections 2024-04-21T08:37:05.014 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:04.400+0000 7f9f3f0d6700 1 -- 172.21.15.82:0/2735347860 wait complete. 2024-04-21T08:37:05.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:05 smithi082 ceph-mon[27967]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:37:05.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:05 smithi082 ceph-mon[27967]: Generating ssh key... 2024-04-21T08:37:05.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:05 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:05.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:05 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:06.267 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkNmgVHbETqv/6Sm6ZaE235c5uRROeXx5CPqkKQw3LqoV/VPtZMLDNXzKEvNnlcszIi4Nu8DcTHTC6VkIgPq7E5Rs+wKjT2zgQfuYuDNi+SzfUaUuMesHcxZ103rDXlLrMSkHTIgrOemEGvcK0g019L8MGiY+ZCMI/KP/LXBAprKR+0M1f7Q1XjkA4ThhXViqo4n7TqJHi5GX9lIQuXShEblFb5Q2M3UT5VDJ4S6+C75HKCeJKhSnoqwZy/3onqLWdviPBee338x7Mz1SOpmyo/sHbUNrThTzwx8oscfaZBqqfvjWWJxHzB2Kt9bCfYrEUdpPywF1IczhY31BFtB2PT70FiRc0aIBMmYI56yeeo92lsVVOFzaPw3EPuPt52SWqpTkCH6FGdCQzxMLzsZ1nhtdQ0FDklX89HWv2c0qhWK9MIgZMfOs67B8e3vtV2Eu+4AVQBIhqf1m8h8afe5MfHc+p2G79Th9vIFAc5V1LO41lDxYhGjLc1uguacErId0= ceph-24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:37:06.267 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.454+0000 7f2909579700 1 Processor -- start 2024-04-21T08:37:06.267 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.455+0000 7f2909579700 1 -- start start 2024-04-21T08:37:06.267 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.455+0000 7f2909579700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 0x7f2904107730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:06.267 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.455+0000 7f2909579700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2904107d00 con 0x7f2904107310 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.455+0000 7f2902ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 0x7f2904107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.455+0000 7f2902ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 0x7f2904107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44620/0 (socket says 172.21.15.82:44620) 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.455+0000 7f2902ffd700 1 -- 172.21.15.82:0/4294254851 learned_addr learned my addr 172.21.15.82:0/4294254851 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.455+0000 7f2902ffd700 1 -- 172.21.15.82:0/4294254851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2904108510 con 0x7f2904107310 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.455+0000 7f2902ffd700 1 --2- 172.21.15.82:0/4294254851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 0x7f2904107730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009e80 tx=0x7f28fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d9204e844c389f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.455+0000 7f2901ffb700 1 -- 172.21.15.82:0/4294254851 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28fc017070 con 0x7f2904107310 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.455+0000 7f2901ffb700 1 -- 172.21.15.82:0/4294254851 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f28fc00b8c0 con 0x7f2904107310 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.455+0000 7f2901ffb700 1 -- 172.21.15.82:0/4294254851 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28fc004490 con 0x7f2904107310 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.456+0000 7f2909579700 1 -- 172.21.15.82:0/4294254851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 msgr2=0x7f2904107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.456+0000 7f2909579700 1 --2- 172.21.15.82:0/4294254851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 0x7f2904107730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009e80 tx=0x7f28fc00b330 comp rx=0 tx=0).stop 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.456+0000 7f2909579700 1 -- 172.21.15.82:0/4294254851 shutdown_connections 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.456+0000 7f2909579700 1 --2- 172.21.15.82:0/4294254851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 0x7f2904107730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.456+0000 7f2909579700 1 -- 172.21.15.82:0/4294254851 >> 172.21.15.82:0/4294254851 conn(0x7f29041028b0 msgr2=0x7f2904104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.456+0000 7f2909579700 1 -- 172.21.15.82:0/4294254851 shutdown_connections 2024-04-21T08:37:06.268 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.456+0000 7f2909579700 1 -- 172.21.15.82:0/4294254851 wait complete. 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.457+0000 7f2909579700 1 Processor -- start 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.457+0000 7f2909579700 1 -- start start 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.457+0000 7f2909579700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 0x7f290419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.457+0000 7f2909579700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f290410b740 con 0x7f2904107310 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.457+0000 7f2902ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 0x7f290419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.457+0000 7f2902ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 0x7f290419c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44622/0 (socket says 172.21.15.82:44622) 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.457+0000 7f2902ffd700 1 -- 172.21.15.82:0/22813660 learned_addr learned my addr 172.21.15.82:0/22813660 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.457+0000 7f2902ffd700 1 -- 172.21.15.82:0/22813660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290419c7c0 con 0x7f2904107310 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.457+0000 7f2902ffd700 1 --2- 172.21.15.82:0/22813660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 0x7f290419c280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f28fc004a60 tx=0x7f28fc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.458+0000 7f28ebfff700 1 -- 172.21.15.82:0/22813660 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28fc020020 con 0x7f2904107310 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.458+0000 7f28ebfff700 1 -- 172.21.15.82:0/22813660 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f28fc012430 con 0x7f2904107310 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.458+0000 7f2909579700 1 -- 172.21.15.82:0/22813660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290419ca50 con 0x7f2904107310 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.458+0000 7f28ebfff700 1 -- 172.21.15.82:0/22813660 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28fc017040 con 0x7f2904107310 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.458+0000 7f2909579700 1 -- 172.21.15.82:0/22813660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f290419f740 con 0x7f2904107310 2024-04-21T08:37:06.269 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.459+0000 7f28ebfff700 1 -- 172.21.15.82:0/22813660 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f28fc023020 con 0x7f2904107310 2024-04-21T08:37:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.459+0000 7f2909579700 1 -- 172.21.15.82:0/22813660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f290410b8c0 con 0x7f2904107310 2024-04-21T08:37:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.459+0000 7f28ebfff700 1 --2- 172.21.15.82:0/22813660 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28ec039670 0x7f28ec03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.459+0000 7f28ebfff700 1 -- 172.21.15.82:0/22813660 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f28fc00f050 con 0x7f2904107310 2024-04-21T08:37:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.459+0000 7f29027fc700 1 --2- 172.21.15.82:0/22813660 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28ec039670 0x7f28ec03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.459+0000 7f29027fc700 1 --2- 172.21.15.82:0/22813660 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28ec039670 0x7f28ec03bb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f28f8006fd0 tx=0x7f28f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.464+0000 7f28ebfff700 1 -- 172.21.15.82:0/22813660 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28fc02a020 con 0x7f2904107310 2024-04-21T08:37:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.663+0000 7f2909579700 1 -- 172.21.15.82:0/22813660 --> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f290402e3a0 con 0x7f28ec039670 2024-04-21T08:37:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.663+0000 7f28ebfff700 1 -- 172.21.15.82:0/22813660 <== mgr.14120 v2:172.21.15.82:6800/880382736 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f290402e3a0 con 0x7f28ec039670 2024-04-21T08:37:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.665+0000 7f2909579700 1 -- 172.21.15.82:0/22813660 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28ec039670 msgr2=0x7f28ec03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.665+0000 7f2909579700 1 --2- 172.21.15.82:0/22813660 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28ec039670 0x7f28ec03bb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f28f8006fd0 tx=0x7f28f8006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.665+0000 7f2909579700 1 -- 172.21.15.82:0/22813660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 msgr2=0x7f290419c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.665+0000 7f2909579700 1 --2- 172.21.15.82:0/22813660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 0x7f290419c280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f28fc004a60 tx=0x7f28fc00bd50 comp rx=0 tx=0).stop 2024-04-21T08:37:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.665+0000 7f2909579700 1 -- 172.21.15.82:0/22813660 shutdown_connections 2024-04-21T08:37:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.665+0000 7f2909579700 1 --2- 172.21.15.82:0/22813660 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28ec039670 0x7f28ec03bb30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.665+0000 7f2909579700 1 --2- 172.21.15.82:0/22813660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2904107310 0x7f290419c280 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.665+0000 7f2909579700 1 -- 172.21.15.82:0/22813660 >> 172.21.15.82:0/22813660 conn(0x7f29041028b0 msgr2=0x7f2904103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.666+0000 7f2909579700 1 -- 172.21.15.82:0/22813660 shutdown_connections 2024-04-21T08:37:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:05.666+0000 7f2909579700 1 -- 172.21.15.82:0/22813660 wait complete. 2024-04-21T08:37:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:37:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:Adding key to root@localhost authorized_keys... 2024-04-21T08:37:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:Adding host smithi082... 2024-04-21T08:37:06.686 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:06 smithi082 ceph-mon[27967]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:37:08.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:07 smithi082 ceph-mon[27967]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi082", "addr": "172.21.15.82", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:37:08.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:07 smithi082 ceph-mon[27967]: Deploying cephadm binary to smithi082 2024-04-21T08:37:11.566 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Added host 'smithi082' with addr '172.21.15.82' 2024-04-21T08:37:11.566 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.703+0000 7ff7df630700 1 Processor -- start 2024-04-21T08:37:11.566 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.704+0000 7ff7df630700 1 -- start start 2024-04-21T08:37:11.566 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.704+0000 7ff7df630700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 0x7ff7d8106b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.704+0000 7ff7df630700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81070f0 con 0x7ff7d8106700 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.704+0000 7ff7dd3cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 0x7ff7d8106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.704+0000 7ff7dd3cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 0x7ff7d8106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44626/0 (socket says 172.21.15.82:44626) 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.704+0000 7ff7dd3cc700 1 -- 172.21.15.82:0/1667177195 learned_addr learned my addr 172.21.15.82:0/1667177195 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.704+0000 7ff7dd3cc700 1 -- 172.21.15.82:0/1667177195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d8109690 con 0x7ff7d8106700 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.704+0000 7ff7dd3cc700 1 --2- 172.21.15.82:0/1667177195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 0x7ff7d8106b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40098a0 tx=0x7ff7d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8312494e3f99c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.705+0000 7ff7cffff700 1 -- 172.21.15.82:0/1667177195 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7d4004160 con 0x7ff7d8106700 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.705+0000 7ff7cffff700 1 -- 172.21.15.82:0/1667177195 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff7d4004300 con 0x7ff7d8106700 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.705+0000 7ff7cffff700 1 -- 172.21.15.82:0/1667177195 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7d4011af0 con 0x7ff7d8106700 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.705+0000 7ff7df630700 1 -- 172.21.15.82:0/1667177195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 msgr2=0x7ff7d8106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.705+0000 7ff7df630700 1 --2- 172.21.15.82:0/1667177195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 0x7ff7d8106b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40098a0 tx=0x7ff7d4009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.705+0000 7ff7df630700 1 -- 172.21.15.82:0/1667177195 shutdown_connections 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.705+0000 7ff7df630700 1 --2- 172.21.15.82:0/1667177195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 0x7ff7d8106b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.705+0000 7ff7df630700 1 -- 172.21.15.82:0/1667177195 >> 172.21.15.82:0/1667177195 conn(0x7ff7d81020d0 msgr2=0x7ff7d81044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:11.567 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.705+0000 7ff7df630700 1 -- 172.21.15.82:0/1667177195 shutdown_connections 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.705+0000 7ff7df630700 1 -- 172.21.15.82:0/1667177195 wait complete. 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.706+0000 7ff7df630700 1 Processor -- start 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.706+0000 7ff7df630700 1 -- start start 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.706+0000 7ff7df630700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 0x7ff7d819c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.706+0000 7ff7df630700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d810c8c0 con 0x7ff7d8106700 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.706+0000 7ff7dd3cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 0x7ff7d819c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.706+0000 7ff7dd3cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 0x7ff7d819c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44636/0 (socket says 172.21.15.82:44636) 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.706+0000 7ff7dd3cc700 1 -- 172.21.15.82:0/60416269 learned_addr learned my addr 172.21.15.82:0/60416269 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.707+0000 7ff7dd3cc700 1 -- 172.21.15.82:0/60416269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d819c730 con 0x7ff7d8106700 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.707+0000 7ff7dd3cc700 1 --2- 172.21.15.82:0/60416269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 0x7ff7d819c1f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4000c00 tx=0x7ff7d4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.707+0000 7ff7ce7fc700 1 -- 172.21.15.82:0/60416269 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7d4003b20 con 0x7ff7d8106700 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.707+0000 7ff7ce7fc700 1 -- 172.21.15.82:0/60416269 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff7d401a8c0 con 0x7ff7d8106700 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.707+0000 7ff7ce7fc700 1 -- 172.21.15.82:0/60416269 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7d4024940 con 0x7ff7d8106700 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.707+0000 7ff7df630700 1 -- 172.21.15.82:0/60416269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d819c9c0 con 0x7ff7d8106700 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.707+0000 7ff7df630700 1 -- 172.21.15.82:0/60416269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d819f6b0 con 0x7ff7d8106700 2024-04-21T08:37:11.568 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.708+0000 7ff7ce7fc700 1 -- 172.21.15.82:0/60416269 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff7d4022020 con 0x7ff7d8106700 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.708+0000 7ff7df630700 1 -- 172.21.15.82:0/60416269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d80640a0 con 0x7ff7d8106700 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.708+0000 7ff7ce7fc700 1 --2- 172.21.15.82:0/60416269 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7ff7c4039670 0x7ff7c403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.708+0000 7ff7ce7fc700 1 -- 172.21.15.82:0/60416269 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff7d404d860 con 0x7ff7d8106700 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.708+0000 7ff7dcbcb700 1 --2- 172.21.15.82:0/60416269 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7ff7c4039670 0x7ff7c403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.708+0000 7ff7dcbcb700 1 --2- 172.21.15.82:0/60416269 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7ff7c4039670 0x7ff7c403bb30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8006fd0 tx=0x7ff7c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.713+0000 7ff7ce7fc700 1 -- 172.21.15.82:0/60416269 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7d402c3a0 con 0x7ff7d8106700 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:06.914+0000 7ff7df630700 1 -- 172.21.15.82:0/60416269 --> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi082", "addr": "172.21.15.82", "target": ["mon-mgr", ""]}) -- 0x7ff7d8103170 con 0x7ff7c4039670 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:10.956+0000 7ff7ce7fc700 1 -- 172.21.15.82:0/60416269 <== mgr.14120 v2:172.21.15.82:6800/880382736 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ff7d8103170 con 0x7ff7c4039670 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:10.959+0000 7ff7df630700 1 -- 172.21.15.82:0/60416269 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7ff7c4039670 msgr2=0x7ff7c403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:10.959+0000 7ff7df630700 1 --2- 172.21.15.82:0/60416269 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7ff7c4039670 0x7ff7c403bb30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8006fd0 tx=0x7ff7c8006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:10.959+0000 7ff7df630700 1 -- 172.21.15.82:0/60416269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 msgr2=0x7ff7d819c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:10.959+0000 7ff7df630700 1 --2- 172.21.15.82:0/60416269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 0x7ff7d819c1f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4000c00 tx=0x7ff7d4011b70 comp rx=0 tx=0).stop 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:10.959+0000 7ff7df630700 1 -- 172.21.15.82:0/60416269 shutdown_connections 2024-04-21T08:37:11.569 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:10.959+0000 7ff7df630700 1 --2- 172.21.15.82:0/60416269 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7ff7c4039670 0x7ff7c403bb30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:11.570 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:10.959+0000 7ff7df630700 1 --2- 172.21.15.82:0/60416269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d8106700 0x7ff7d819c1f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:11.570 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:10.959+0000 7ff7df630700 1 -- 172.21.15.82:0/60416269 >> 172.21.15.82:0/60416269 conn(0x7ff7d81020d0 msgr2=0x7ff7d8102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:11.570 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:10.959+0000 7ff7df630700 1 -- 172.21.15.82:0/60416269 shutdown_connections 2024-04-21T08:37:11.570 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:10.959+0000 7ff7df630700 1 -- 172.21.15.82:0/60416269 wait complete. 2024-04-21T08:37:11.570 INFO:teuthology.orchestra.run.smithi082.stdout:Deploying unmanaged mon service... 2024-04-21T08:37:12.078 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:11 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:12.078 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:11 smithi082 ceph-mon[27967]: Added host smithi082 2024-04-21T08:37:12.078 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:11 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:12.910 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-21T08:37:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.073+0000 7f24bc0ce700 1 Processor -- start 2024-04-21T08:37:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.073+0000 7f24bc0ce700 1 -- start start 2024-04-21T08:37:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.074+0000 7f24bc0ce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 0x7f24b41076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.074+0000 7f24bc0ce700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f24b4107ca0 con 0x7f24b41072b0 2024-04-21T08:37:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.074+0000 7f24b9e6a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 0x7f24b41076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.074+0000 7f24b9e6a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 0x7f24b41076d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44646/0 (socket says 172.21.15.82:44646) 2024-04-21T08:37:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.074+0000 7f24b9e6a700 1 -- 172.21.15.82:0/805214183 learned_addr learned my addr 172.21.15.82:0/805214183 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.074+0000 7f24b9e6a700 1 -- 172.21.15.82:0/805214183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b41084b0 con 0x7f24b41072b0 2024-04-21T08:37:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.074+0000 7f24b9e6a700 1 --2- 172.21.15.82:0/805214183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 0x7f24b41076d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009e80 tx=0x7f24ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edcfd3dd17811af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.074+0000 7f24b8e68700 1 -- 172.21.15.82:0/805214183 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24ac017070 con 0x7f24b41072b0 2024-04-21T08:37:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.074+0000 7f24b8e68700 1 -- 172.21.15.82:0/805214183 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f24ac00b8c0 con 0x7f24b41072b0 2024-04-21T08:37:12.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.074+0000 7f24b8e68700 1 -- 172.21.15.82:0/805214183 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24ac004490 con 0x7f24b41072b0 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.075+0000 7f24bc0ce700 1 -- 172.21.15.82:0/805214183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 msgr2=0x7f24b41076d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.075+0000 7f24bc0ce700 1 --2- 172.21.15.82:0/805214183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 0x7f24b41076d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009e80 tx=0x7f24ac00b330 comp rx=0 tx=0).stop 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.075+0000 7f24bc0ce700 1 -- 172.21.15.82:0/805214183 shutdown_connections 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.075+0000 7f24bc0ce700 1 --2- 172.21.15.82:0/805214183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 0x7f24b41076d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.075+0000 7f24bc0ce700 1 -- 172.21.15.82:0/805214183 >> 172.21.15.82:0/805214183 conn(0x7f24b4102890 msgr2=0x7f24b4104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.075+0000 7f24bc0ce700 1 -- 172.21.15.82:0/805214183 shutdown_connections 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.075+0000 7f24bc0ce700 1 -- 172.21.15.82:0/805214183 wait complete. 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24bc0ce700 1 Processor -- start 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24bc0ce700 1 -- start start 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24bc0ce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 0x7f24b419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24bc0ce700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f24b410b6e0 con 0x7f24b41072b0 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24b9e6a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 0x7f24b419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24b9e6a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 0x7f24b419b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44652/0 (socket says 172.21.15.82:44652) 2024-04-21T08:37:12.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24b9e6a700 1 -- 172.21.15.82:0/1343620873 learned_addr learned my addr 172.21.15.82:0/1343620873 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24b9e6a700 1 -- 172.21.15.82:0/1343620873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b419b630 con 0x7f24b41072b0 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24b9e6a700 1 --2- 172.21.15.82:0/1343620873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 0x7f24b419b0f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f24ac004a60 tx=0x7f24ac00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24b2ffd700 1 -- 172.21.15.82:0/1343620873 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24ac020020 con 0x7f24b41072b0 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24b2ffd700 1 -- 172.21.15.82:0/1343620873 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f24ac003c60 con 0x7f24b41072b0 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24bc0ce700 1 -- 172.21.15.82:0/1343620873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b419b8c0 con 0x7f24b41072b0 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24b2ffd700 1 -- 172.21.15.82:0/1343620873 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24ac017040 con 0x7f24b41072b0 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.076+0000 7f24bc0ce700 1 -- 172.21.15.82:0/1343620873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b419e5b0 con 0x7f24b41072b0 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.077+0000 7f24b2ffd700 1 -- 172.21.15.82:0/1343620873 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f24ac023020 con 0x7f24b41072b0 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.077+0000 7f24bc0ce700 1 -- 172.21.15.82:0/1343620873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b410b860 con 0x7f24b41072b0 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.078+0000 7f24b2ffd700 1 --2- 172.21.15.82:0/1343620873 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f249c039670 0x7f249c03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.078+0000 7f24b2ffd700 1 -- 172.21.15.82:0/1343620873 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f24ac00f050 con 0x7f24b41072b0 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.078+0000 7f24b9669700 1 --2- 172.21.15.82:0/1343620873 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f249c039670 0x7f249c03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:12.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.078+0000 7f24b9669700 1 --2- 172.21.15.82:0/1343620873 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f249c039670 0x7f249c03bb30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f24a8006fd0 tx=0x7f24a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.083+0000 7f24b2ffd700 1 -- 172.21.15.82:0/1343620873 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24ac02a020 con 0x7f24b41072b0 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.300+0000 7f24bc0ce700 1 -- 172.21.15.82:0/1343620873 --> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f24b41039d0 con 0x7f249c039670 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.304+0000 7f24b2ffd700 1 -- 172.21.15.82:0/1343620873 <== mgr.14120 v2:172.21.15.82:6800/880382736 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f24b41039d0 con 0x7f249c039670 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.306+0000 7f24bc0ce700 1 -- 172.21.15.82:0/1343620873 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f249c039670 msgr2=0x7f249c03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.306+0000 7f24bc0ce700 1 --2- 172.21.15.82:0/1343620873 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f249c039670 0x7f249c03bb30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f24a8006fd0 tx=0x7f24a8006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.306+0000 7f24bc0ce700 1 -- 172.21.15.82:0/1343620873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 msgr2=0x7f24b419b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.306+0000 7f24bc0ce700 1 --2- 172.21.15.82:0/1343620873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 0x7f24b419b0f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f24ac004a60 tx=0x7f24ac00bd50 comp rx=0 tx=0).stop 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.306+0000 7f24bc0ce700 1 -- 172.21.15.82:0/1343620873 shutdown_connections 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.306+0000 7f24bc0ce700 1 --2- 172.21.15.82:0/1343620873 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f249c039670 0x7f249c03bb30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.306+0000 7f24bc0ce700 1 --2- 172.21.15.82:0/1343620873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24b41072b0 0x7f24b419b0f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.306+0000 7f24bc0ce700 1 -- 172.21.15.82:0/1343620873 >> 172.21.15.82:0/1343620873 conn(0x7f24b4102890 msgr2=0x7f24b41032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.307+0000 7f24bc0ce700 1 -- 172.21.15.82:0/1343620873 shutdown_connections 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:12.307+0000 7f24bc0ce700 1 -- 172.21.15.82:0/1343620873 wait complete. 2024-04-21T08:37:12.914 INFO:teuthology.orchestra.run.smithi082.stdout:Deploying unmanaged mgr service... 2024-04-21T08:37:13.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:13 smithi082 ceph-mon[27967]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:37:13.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:13 smithi082 ceph-mon[27967]: Saving service mon spec with placement count:5 2024-04-21T08:37:13.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:13 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:14.830 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:14 smithi082 ceph-mon[27967]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:37:14.830 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:14 smithi082 ceph-mon[27967]: Saving service mgr spec with placement count:2 2024-04-21T08:37:14.830 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:14 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:14.830 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:14 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:14.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-21T08:37:14.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.351+0000 7f0c4a1dd700 1 Processor -- start 2024-04-21T08:37:14.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.351+0000 7f0c4a1dd700 1 -- start start 2024-04-21T08:37:14.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.351+0000 7f0c4a1dd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 0x7f0c441088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:14.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.351+0000 7f0c4a1dd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0c44108ea0 con 0x7f0c441084b0 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.351+0000 7f0c437fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 0x7f0c441088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.351+0000 7f0c437fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 0x7f0c441088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57834/0 (socket says 172.21.15.82:57834) 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.351+0000 7f0c437fe700 1 -- 172.21.15.82:0/1486927549 learned_addr learned my addr 172.21.15.82:0/1486927549 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.351+0000 7f0c437fe700 1 -- 172.21.15.82:0/1486927549 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c44109660 con 0x7f0c441084b0 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.352+0000 7f0c437fe700 1 --2- 172.21.15.82:0/1486927549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 0x7f0c441088d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0c340098a0 tx=0x7f0c34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10860ccfdf4e76ba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.352+0000 7f0c427fc700 1 -- 172.21.15.82:0/1486927549 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c34004160 con 0x7f0c441084b0 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.352+0000 7f0c427fc700 1 -- 172.21.15.82:0/1486927549 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0c34004300 con 0x7f0c441084b0 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.352+0000 7f0c427fc700 1 -- 172.21.15.82:0/1486927549 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c34011af0 con 0x7f0c441084b0 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.352+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/1486927549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 msgr2=0x7f0c441088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.352+0000 7f0c4a1dd700 1 --2- 172.21.15.82:0/1486927549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 0x7f0c441088d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0c340098a0 tx=0x7f0c34009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.352+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/1486927549 shutdown_connections 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.352+0000 7f0c4a1dd700 1 --2- 172.21.15.82:0/1486927549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 0x7f0c441088d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:14.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.352+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/1486927549 >> 172.21.15.82:0/1486927549 conn(0x7f0c44103af0 msgr2=0x7f0c44105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.352+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/1486927549 shutdown_connections 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.352+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/1486927549 wait complete. 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.353+0000 7f0c4a1dd700 1 Processor -- start 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.353+0000 7f0c4a1dd700 1 -- start start 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.353+0000 7f0c4a1dd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 0x7f0c4419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.353+0000 7f0c4a1dd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4410c890 con 0x7f0c441084b0 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.353+0000 7f0c437fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 0x7f0c4419c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.353+0000 7f0c437fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 0x7f0c4419c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57842/0 (socket says 172.21.15.82:57842) 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.353+0000 7f0c437fe700 1 -- 172.21.15.82:0/2596703141 learned_addr learned my addr 172.21.15.82:0/2596703141 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.354+0000 7f0c437fe700 1 -- 172.21.15.82:0/2596703141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4419c740 con 0x7f0c441084b0 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.354+0000 7f0c437fe700 1 --2- 172.21.15.82:0/2596703141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 0x7f0c4419c200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0c34003f40 tx=0x7f0c34011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.354+0000 7f0c40ff9700 1 -- 172.21.15.82:0/2596703141 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c340038b0 con 0x7f0c441084b0 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.354+0000 7f0c40ff9700 1 -- 172.21.15.82:0/2596703141 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0c34003a50 con 0x7f0c441084b0 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.354+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/2596703141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4419c9d0 con 0x7f0c441084b0 2024-04-21T08:37:14.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.354+0000 7f0c40ff9700 1 -- 172.21.15.82:0/2596703141 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3401a5c0 con 0x7f0c441084b0 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.354+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/2596703141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4419f6c0 con 0x7f0c441084b0 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.355+0000 7f0c40ff9700 1 -- 172.21.15.82:0/2596703141 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0c3401a8c0 con 0x7f0c441084b0 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.355+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/2596703141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c440640a0 con 0x7f0c441084b0 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.355+0000 7f0c40ff9700 1 --2- 172.21.15.82:0/2596703141 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f0c2c039620 0x7f0c2c03bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.355+0000 7f0c40ff9700 1 -- 172.21.15.82:0/2596703141 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0c3404dd20 con 0x7f0c441084b0 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.355+0000 7f0c42ffd700 1 --2- 172.21.15.82:0/2596703141 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f0c2c039620 0x7f0c2c03bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.356+0000 7f0c42ffd700 1 --2- 172.21.15.82:0/2596703141 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f0c2c039620 0x7f0c2c03bae0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c38006fd0 tx=0x7f0c38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.360+0000 7f0c40ff9700 1 -- 172.21.15.82:0/2596703141 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c34019c70 con 0x7f0c441084b0 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.573+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/2596703141 --> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0c44104c50 con 0x7f0c2c039620 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.581+0000 7f0c40ff9700 1 -- 172.21.15.82:0/2596703141 <== mgr.14120 v2:172.21.15.82:6800/880382736 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0c44104c50 con 0x7f0c2c039620 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.583+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/2596703141 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f0c2c039620 msgr2=0x7f0c2c03bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.583+0000 7f0c4a1dd700 1 --2- 172.21.15.82:0/2596703141 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f0c2c039620 0x7f0c2c03bae0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c38006fd0 tx=0x7f0c38006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.583+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/2596703141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 msgr2=0x7f0c4419c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:14.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.583+0000 7f0c4a1dd700 1 --2- 172.21.15.82:0/2596703141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 0x7f0c4419c200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0c34003f40 tx=0x7f0c34011890 comp rx=0 tx=0).stop 2024-04-21T08:37:14.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.583+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/2596703141 shutdown_connections 2024-04-21T08:37:14.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.583+0000 7f0c4a1dd700 1 --2- 172.21.15.82:0/2596703141 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f0c2c039620 0x7f0c2c03bae0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:14.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.583+0000 7f0c4a1dd700 1 --2- 172.21.15.82:0/2596703141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c441084b0 0x7f0c4419c200 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:14.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.583+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/2596703141 >> 172.21.15.82:0/2596703141 conn(0x7f0c44103af0 msgr2=0x7f0c44104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:14.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.584+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/2596703141 shutdown_connections 2024-04-21T08:37:14.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:13.584+0000 7f0c4a1dd700 1 -- 172.21.15.82:0/2596703141 wait complete. 2024-04-21T08:37:16.355 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:15 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:16.356 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:15 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/705852862' entity='client.admin' 2024-04-21T08:37:16.605 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.304+0000 7fe176eba700 1 Processor -- start 2024-04-21T08:37:16.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.304+0000 7fe176eba700 1 -- start start 2024-04-21T08:37:16.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.304+0000 7fe176eba700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe170108e50 0x7fe17010b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:16.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.304+0000 7fe176eba700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe17007be60 con 0x7fe170108e50 2024-04-21T08:37:16.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.304+0000 7fe174c56700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe170108e50 0x7fe17010b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:16.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.304+0000 7fe174c56700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe170108e50 0x7fe17010b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57846/0 (socket says 172.21.15.82:57846) 2024-04-21T08:37:16.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.304+0000 7fe174c56700 1 -- 172.21.15.82:0/2949369241 learned_addr learned my addr 172.21.15.82:0/2949369241 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:16.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.305+0000 7fe174c56700 1 -- 172.21.15.82:0/2949369241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17010b780 con 0x7fe170108e50 2024-04-21T08:37:16.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.305+0000 7fe174c56700 1 --2- 172.21.15.82:0/2949369241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe170108e50 0x7fe17010b240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe1640098a0 tx=0x7fe164009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d2b39574ce2646 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:16.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.305+0000 7fe16f7fe700 1 -- 172.21.15.82:0/2949369241 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe164004160 con 0x7fe170108e50 2024-04-21T08:37:16.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.305+0000 7fe16f7fe700 1 -- 172.21.15.82:0/2949369241 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe164004300 con 0x7fe170108e50 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.305+0000 7fe16f7fe700 1 -- 172.21.15.82:0/2949369241 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe164011a20 con 0x7fe170108e50 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.306+0000 7fe176eba700 1 -- 172.21.15.82:0/2949369241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe170108e50 msgr2=0x7fe17010b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.306+0000 7fe176eba700 1 --2- 172.21.15.82:0/2949369241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe170108e50 0x7fe17010b240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe1640098a0 tx=0x7fe164009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.306+0000 7fe176eba700 1 -- 172.21.15.82:0/2949369241 shutdown_connections 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.306+0000 7fe176eba700 1 --2- 172.21.15.82:0/2949369241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe170108e50 0x7fe17010b240 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.306+0000 7fe176eba700 1 -- 172.21.15.82:0/2949369241 >> 172.21.15.82:0/2949369241 conn(0x7fe1701028e0 msgr2=0x7fe170104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.306+0000 7fe176eba700 1 -- 172.21.15.82:0/2949369241 shutdown_connections 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.306+0000 7fe176eba700 1 -- 172.21.15.82:0/2949369241 wait complete. 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.307+0000 7fe176eba700 1 Processor -- start 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.307+0000 7fe176eba700 1 -- start start 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.307+0000 7fe176eba700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe17019c330 0x7fe17019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.307+0000 7fe176eba700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe17010c780 con 0x7fe17019c330 2024-04-21T08:37:16.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.307+0000 7fe174c56700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe17019c330 0x7fe17019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.307+0000 7fe174c56700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe17019c330 0x7fe17019c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57850/0 (socket says 172.21.15.82:57850) 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.307+0000 7fe174c56700 1 -- 172.21.15.82:0/705852862 learned_addr learned my addr 172.21.15.82:0/705852862 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.307+0000 7fe174c56700 1 -- 172.21.15.82:0/705852862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17019cc90 con 0x7fe17019c330 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.307+0000 7fe174c56700 1 --2- 172.21.15.82:0/705852862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe17019c330 0x7fe17019c750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe164006af0 tx=0x7fe164012fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.308+0000 7fe16dffb700 1 -- 172.21.15.82:0/705852862 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1640036a0 con 0x7fe17019c330 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.308+0000 7fe16dffb700 1 -- 172.21.15.82:0/705852862 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe164020070 con 0x7fe17019c330 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.308+0000 7fe176eba700 1 -- 172.21.15.82:0/705852862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17019cf20 con 0x7fe17019c330 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.308+0000 7fe16dffb700 1 -- 172.21.15.82:0/705852862 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe164025e20 con 0x7fe17019c330 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.308+0000 7fe176eba700 1 -- 172.21.15.82:0/705852862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe17019fa00 con 0x7fe17019c330 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.309+0000 7fe16dffb700 1 -- 172.21.15.82:0/705852862 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe16401b470 con 0x7fe17019c330 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.309+0000 7fe16dffb700 1 --2- 172.21.15.82:0/705852862 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7fe1600396c0 0x7fe16003bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.309+0000 7fe176eba700 1 -- 172.21.15.82:0/705852862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1700640a0 con 0x7fe17019c330 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.309+0000 7fe16dffb700 1 -- 172.21.15.82:0/705852862 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe16404dbe0 con 0x7fe17019c330 2024-04-21T08:37:16.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.309+0000 7fe16ffff700 1 --2- 172.21.15.82:0/705852862 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7fe1600396c0 0x7fe16003bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.310+0000 7fe16ffff700 1 --2- 172.21.15.82:0/705852862 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7fe1600396c0 0x7fe16003bb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe15c006fd0 tx=0x7fe15c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.314+0000 7fe16dffb700 1 -- 172.21.15.82:0/705852862 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe16402c300 con 0x7fe17019c330 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.564+0000 7fe176eba700 1 -- 172.21.15.82:0/705852862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fe1701a0140 con 0x7fe17019c330 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.566+0000 7fe16dffb700 1 -- 172.21.15.82:0/705852862 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fe16404d940 con 0x7fe17019c330 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.569+0000 7fe176eba700 1 -- 172.21.15.82:0/705852862 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7fe1600396c0 msgr2=0x7fe16003bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.569+0000 7fe176eba700 1 --2- 172.21.15.82:0/705852862 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7fe1600396c0 0x7fe16003bb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe15c006fd0 tx=0x7fe15c006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.569+0000 7fe176eba700 1 -- 172.21.15.82:0/705852862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe17019c330 msgr2=0x7fe17019c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.569+0000 7fe176eba700 1 --2- 172.21.15.82:0/705852862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe17019c330 0x7fe17019c750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe164006af0 tx=0x7fe164012fc0 comp rx=0 tx=0).stop 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.569+0000 7fe176eba700 1 -- 172.21.15.82:0/705852862 shutdown_connections 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.569+0000 7fe176eba700 1 --2- 172.21.15.82:0/705852862 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7fe1600396c0 0x7fe16003bb80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.569+0000 7fe176eba700 1 --2- 172.21.15.82:0/705852862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe17019c330 0x7fe17019c750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.569+0000 7fe176eba700 1 -- 172.21.15.82:0/705852862 >> 172.21.15.82:0/705852862 conn(0x7fe1701028e0 msgr2=0x7fe170103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:16.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.569+0000 7fe176eba700 1 -- 172.21.15.82:0/705852862 shutdown_connections 2024-04-21T08:37:16.610 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:15.569+0000 7fe176eba700 1 -- 172.21.15.82:0/705852862 wait complete. 2024-04-21T08:37:17.314 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:17 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:17.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.097+0000 7f63097cf700 1 Processor -- start 2024-04-21T08:37:17.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.097+0000 7f63097cf700 1 -- start start 2024-04-21T08:37:17.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.097+0000 7f63097cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 0x7f630410b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:17.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.097+0000 7f63097cf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f630407be60 con 0x7f6304108e30 2024-04-21T08:37:17.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.097+0000 7f6302ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 0x7f630410b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:17.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.097+0000 7f6302ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 0x7f630410b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57860/0 (socket says 172.21.15.82:57860) 2024-04-21T08:37:17.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.097+0000 7f6302ffd700 1 -- 172.21.15.82:0/3527352845 learned_addr learned my addr 172.21.15.82:0/3527352845 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:17.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.098+0000 7f6302ffd700 1 -- 172.21.15.82:0/3527352845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630410b760 con 0x7f6304108e30 2024-04-21T08:37:17.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.098+0000 7f6302ffd700 1 --2- 172.21.15.82:0/3527352845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 0x7f630410b220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f62f40098a0 tx=0x7f62f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1d4f9e403aa200 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:17.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.098+0000 7f6301ffb700 1 -- 172.21.15.82:0/3527352845 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62f4004160 con 0x7f6304108e30 2024-04-21T08:37:17.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.098+0000 7f6301ffb700 1 -- 172.21.15.82:0/3527352845 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62f4004300 con 0x7f6304108e30 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.098+0000 7f6301ffb700 1 -- 172.21.15.82:0/3527352845 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62f4011a20 con 0x7f6304108e30 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.099+0000 7f63097cf700 1 -- 172.21.15.82:0/3527352845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 msgr2=0x7f630410b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.099+0000 7f63097cf700 1 --2- 172.21.15.82:0/3527352845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 0x7f630410b220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f62f40098a0 tx=0x7f62f4009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.099+0000 7f63097cf700 1 -- 172.21.15.82:0/3527352845 shutdown_connections 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.099+0000 7f63097cf700 1 --2- 172.21.15.82:0/3527352845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 0x7f630410b220 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.099+0000 7f63097cf700 1 -- 172.21.15.82:0/3527352845 >> 172.21.15.82:0/3527352845 conn(0x7f6304102840 msgr2=0x7f6304104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.099+0000 7f63097cf700 1 -- 172.21.15.82:0/3527352845 shutdown_connections 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.099+0000 7f63097cf700 1 -- 172.21.15.82:0/3527352845 wait complete. 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.100+0000 7f63097cf700 1 Processor -- start 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.100+0000 7f63097cf700 1 -- start start 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.100+0000 7f63097cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 0x7f630419c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.100+0000 7f63097cf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f630410c760 con 0x7f6304108e30 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.100+0000 7f6302ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 0x7f630419c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:17.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.100+0000 7f6302ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 0x7f630419c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57872/0 (socket says 172.21.15.82:57872) 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.100+0000 7f6302ffd700 1 -- 172.21.15.82:0/1974333786 learned_addr learned my addr 172.21.15.82:0/1974333786 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.100+0000 7f6302ffd700 1 -- 172.21.15.82:0/1974333786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630419c680 con 0x7f6304108e30 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.101+0000 7f6302ffd700 1 --2- 172.21.15.82:0/1974333786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 0x7f630419c140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62f4000c00 tx=0x7f62f4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.101+0000 7f62ebfff700 1 -- 172.21.15.82:0/1974333786 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62f4003aa0 con 0x7f6304108e30 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.101+0000 7f62ebfff700 1 -- 172.21.15.82:0/1974333786 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62f401b8f0 con 0x7f6304108e30 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.101+0000 7f63097cf700 1 -- 172.21.15.82:0/1974333786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630419c910 con 0x7f6304108e30 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.101+0000 7f62ebfff700 1 -- 172.21.15.82:0/1974333786 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62f4023870 con 0x7f6304108e30 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.101+0000 7f63097cf700 1 -- 172.21.15.82:0/1974333786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630419f600 con 0x7f6304108e30 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.102+0000 7f62ebfff700 1 -- 172.21.15.82:0/1974333786 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f62f4023b70 con 0x7f6304108e30 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.102+0000 7f62ebfff700 1 --2- 172.21.15.82:0/1974333786 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62ec039670 0x7f62ec03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.102+0000 7f62ebfff700 1 -- 172.21.15.82:0/1974333786 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f62f404df10 con 0x7f6304108e30 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.102+0000 7f63027fc700 1 --2- 172.21.15.82:0/1974333786 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62ec039670 0x7f62ec03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:17.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.102+0000 7f63097cf700 1 -- 172.21.15.82:0/1974333786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63040640a0 con 0x7f6304108e30 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.103+0000 7f63027fc700 1 --2- 172.21.15.82:0/1974333786 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62ec039670 0x7f62ec03bb30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f62f8006fd0 tx=0x7f62f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.107+0000 7f62ebfff700 1 -- 172.21.15.82:0/1974333786 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62f401f030 con 0x7f6304108e30 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.314+0000 7f63097cf700 1 -- 172.21.15.82:0/1974333786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f630419f7d0 con 0x7f6304108e30 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.319+0000 7f62ebfff700 1 -- 172.21.15.82:0/1974333786 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f62f4016260 con 0x7f6304108e30 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.322+0000 7f63097cf700 1 -- 172.21.15.82:0/1974333786 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62ec039670 msgr2=0x7f62ec03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.322+0000 7f63097cf700 1 --2- 172.21.15.82:0/1974333786 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62ec039670 0x7f62ec03bb30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f62f8006fd0 tx=0x7f62f8006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.322+0000 7f63097cf700 1 -- 172.21.15.82:0/1974333786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 msgr2=0x7f630419c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.322+0000 7f63097cf700 1 --2- 172.21.15.82:0/1974333786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 0x7f630419c140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62f4000c00 tx=0x7f62f4011aa0 comp rx=0 tx=0).stop 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.322+0000 7f63097cf700 1 -- 172.21.15.82:0/1974333786 shutdown_connections 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.322+0000 7f63097cf700 1 --2- 172.21.15.82:0/1974333786 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62ec039670 0x7f62ec03bb30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.322+0000 7f63097cf700 1 --2- 172.21.15.82:0/1974333786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6304108e30 0x7f630419c140 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.322+0000 7f63097cf700 1 -- 172.21.15.82:0/1974333786 >> 172.21.15.82:0/1974333786 conn(0x7f6304102840 msgr2=0x7f63041031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.322+0000 7f63097cf700 1 -- 172.21.15.82:0/1974333786 shutdown_connections 2024-04-21T08:37:17.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:17.322+0000 7f63097cf700 1 -- 172.21.15.82:0/1974333786 wait complete. 2024-04-21T08:37:17.916 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling the dashboard module... 2024-04-21T08:37:18.563 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:18 smithi082 ceph-mon[27967]: from='mgr.14120 172.21.15.82:0/1219915205' entity='mgr.x' 2024-04-21T08:37:18.563 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:18 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1974333786' entity='client.admin' 2024-04-21T08:37:19.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:19 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2958685518' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-21T08:37:19.608 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:19 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: ignoring --setuser ceph since I am not root 2024-04-21T08:37:19.608 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:19 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: ignoring --setgroup ceph since I am not root 2024-04-21T08:37:19.608 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:19 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:19.503+0000 7fce901d1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:37:20.108 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:19 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:19.696+0000 7fce901d1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:37:20.108 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:19 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:19.806+0000 7fce901d1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:37:20.125 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.337+0000 7f40dd39d700 1 Processor -- start 2024-04-21T08:37:20.125 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.337+0000 7f40dd39d700 1 -- start start 2024-04-21T08:37:20.125 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.337+0000 7f40dd39d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 0x7f40d81069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:20.125 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.337+0000 7f40dd39d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f40d8068660 con 0x7f40d80681b0 2024-04-21T08:37:20.125 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.337+0000 7f40d6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 0x7f40d81069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:20.125 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.337+0000 7f40d6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 0x7f40d81069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57874/0 (socket says 172.21.15.82:57874) 2024-04-21T08:37:20.125 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.337+0000 7f40d6ffd700 1 -- 172.21.15.82:0/2063476470 learned_addr learned my addr 172.21.15.82:0/2063476470 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.338+0000 7f40d6ffd700 1 -- 172.21.15.82:0/2063476470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d80687e0 con 0x7f40d80681b0 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.338+0000 7f40d6ffd700 1 --2- 172.21.15.82:0/2063476470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 0x7f40d81069d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f40c80098a0 tx=0x7f40c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7759b3fe670b05bc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.338+0000 7f40d5ffb700 1 -- 172.21.15.82:0/2063476470 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c8004160 con 0x7f40d80681b0 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.338+0000 7f40d5ffb700 1 -- 172.21.15.82:0/2063476470 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c8004300 con 0x7f40d80681b0 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.338+0000 7f40d5ffb700 1 -- 172.21.15.82:0/2063476470 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c8011a20 con 0x7f40d80681b0 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.339+0000 7f40dd39d700 1 -- 172.21.15.82:0/2063476470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 msgr2=0x7f40d81069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.339+0000 7f40dd39d700 1 --2- 172.21.15.82:0/2063476470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 0x7f40d81069d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f40c80098a0 tx=0x7f40c8009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.339+0000 7f40dd39d700 1 -- 172.21.15.82:0/2063476470 shutdown_connections 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.339+0000 7f40dd39d700 1 --2- 172.21.15.82:0/2063476470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 0x7f40d81069d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.339+0000 7f40dd39d700 1 -- 172.21.15.82:0/2063476470 >> 172.21.15.82:0/2063476470 conn(0x7f40d8101f50 msgr2=0x7f40d81043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.339+0000 7f40dd39d700 1 -- 172.21.15.82:0/2063476470 shutdown_connections 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.339+0000 7f40dd39d700 1 -- 172.21.15.82:0/2063476470 wait complete. 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.339+0000 7f40dd39d700 1 Processor -- start 2024-04-21T08:37:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.340+0000 7f40dd39d700 1 -- start start 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.340+0000 7f40dd39d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 0x7f40d819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.340+0000 7f40dd39d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f40d8108350 con 0x7f40d80681b0 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.340+0000 7f40d6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 0x7f40d819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.340+0000 7f40d6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 0x7f40d819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57890/0 (socket says 172.21.15.82:57890) 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.340+0000 7f40d6ffd700 1 -- 172.21.15.82:0/2958685518 learned_addr learned my addr 172.21.15.82:0/2958685518 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.340+0000 7f40d6ffd700 1 -- 172.21.15.82:0/2958685518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d819e940 con 0x7f40d80681b0 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.340+0000 7f40d6ffd700 1 --2- 172.21.15.82:0/2958685518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 0x7f40d819e400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009520 tx=0x7f40c8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.341+0000 7f40bffff700 1 -- 172.21.15.82:0/2958685518 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c8003af0 con 0x7f40d80681b0 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.341+0000 7f40bffff700 1 -- 172.21.15.82:0/2958685518 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c801a8e0 con 0x7f40d80681b0 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.341+0000 7f40dd39d700 1 -- 172.21.15.82:0/2958685518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d819ebd0 con 0x7f40d80681b0 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.341+0000 7f40bffff700 1 -- 172.21.15.82:0/2958685518 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c80247a0 con 0x7f40d80681b0 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.341+0000 7f40dd39d700 1 -- 172.21.15.82:0/2958685518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d81a18c0 con 0x7f40d80681b0 2024-04-21T08:37:20.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.342+0000 7f40dd39d700 1 -- 172.21.15.82:0/2958685518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d80640a0 con 0x7f40d80681b0 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.342+0000 7f40bffff700 1 -- 172.21.15.82:0/2958685518 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f40c8022020 con 0x7f40d80681b0 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.342+0000 7f40bffff700 1 --2- 172.21.15.82:0/2958685518 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f40c0039850 0x7f40c003bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.342+0000 7f40bffff700 1 -- 172.21.15.82:0/2958685518 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f40c800b7e0 con 0x7f40d80681b0 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.342+0000 7f40d67fc700 1 --2- 172.21.15.82:0/2958685518 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f40c0039850 0x7f40c003bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.343+0000 7f40d67fc700 1 --2- 172.21.15.82:0/2958685518 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f40c0039850 0x7f40c003bd10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f40cc006fd0 tx=0x7f40cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.348+0000 7f40bffff700 1 -- 172.21.15.82:0/2958685518 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40c8018710 con 0x7f40d80681b0 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:18.614+0000 7f40dd39d700 1 -- 172.21.15.82:0/2958685518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f40d81a1a90 con 0x7f40d80681b0 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:19.320+0000 7f40bffff700 1 -- 172.21.15.82:0/2958685518 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f40c8032020 con 0x7f40d80681b0 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:19.321+0000 7f40bffff700 1 -- 172.21.15.82:0/2958685518 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f40c8022020 con 0x7f40d80681b0 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:19.323+0000 7f40dd39d700 1 -- 172.21.15.82:0/2958685518 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f40c0039850 msgr2=0x7f40c003bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:19.323+0000 7f40dd39d700 1 --2- 172.21.15.82:0/2958685518 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f40c0039850 0x7f40c003bd10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f40cc006fd0 tx=0x7f40cc006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:19.323+0000 7f40dd39d700 1 -- 172.21.15.82:0/2958685518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 msgr2=0x7f40d819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:19.323+0000 7f40dd39d700 1 --2- 172.21.15.82:0/2958685518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 0x7f40d819e400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009520 tx=0x7f40c8011aa0 comp rx=0 tx=0).stop 2024-04-21T08:37:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:19.323+0000 7f40dd39d700 1 -- 172.21.15.82:0/2958685518 shutdown_connections 2024-04-21T08:37:20.129 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:19.323+0000 7f40dd39d700 1 --2- 172.21.15.82:0/2958685518 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f40c0039850 0x7f40c003bd10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:20.129 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:19.323+0000 7f40dd39d700 1 --2- 172.21.15.82:0/2958685518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40d80681b0 0x7f40d819e400 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:20.129 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:19.323+0000 7f40dd39d700 1 -- 172.21.15.82:0/2958685518 >> 172.21.15.82:0/2958685518 conn(0x7f40d8101f50 msgr2=0x7f40d810bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:20.129 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:19.324+0000 7f40dd39d700 1 -- 172.21.15.82:0/2958685518 shutdown_connections 2024-04-21T08:37:20.129 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:19.324+0000 7f40dd39d700 1 -- 172.21.15.82:0/2958685518 wait complete. 2024-04-21T08:37:20.385 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:20 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2958685518' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-21T08:37:20.386 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:20 smithi082 ceph-mon[27967]: mgrmap e9: x(active, since 19s) 2024-04-21T08:37:20.386 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:20 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:20.130+0000 7fce901d1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:37:20.386 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:20 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:20.226+0000 7fce901d1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:37:21.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:37:21.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-21T08:37:21.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T08:37:21.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-21T08:37:21.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-21T08:37:21.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:37:21.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.644+0000 7f62ca5c6700 1 Processor -- start 2024-04-21T08:37:21.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.644+0000 7f62ca5c6700 1 -- start start 2024-04-21T08:37:21.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.645+0000 7f62ca5c6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 0x7f62bc005db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.645+0000 7f62ca5c6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc006380 con 0x7f62bc005990 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.645+0000 7f62c95c4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 0x7f62bc005db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.645+0000 7f62c95c4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 0x7f62bc005db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57918/0 (socket says 172.21.15.82:57918) 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.645+0000 7f62c95c4700 1 -- 172.21.15.82:0/3158861394 learned_addr learned my addr 172.21.15.82:0/3158861394 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.647+0000 7f62c95c4700 1 -- 172.21.15.82:0/3158861394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62bc006be0 con 0x7f62bc005990 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.648+0000 7f62c95c4700 1 --2- 172.21.15.82:0/3158861394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 0x7f62bc005db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f62c406abb0 tx=0x7f62c40773b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd57c3252451696 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.648+0000 7f62bbfff700 1 -- 172.21.15.82:0/3158861394 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62c4068ea0 con 0x7f62bc005990 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.648+0000 7f62bbfff700 1 -- 172.21.15.82:0/3158861394 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c4084070 con 0x7f62bc005990 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.649+0000 7f62ca5c6700 1 -- 172.21.15.82:0/3158861394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 msgr2=0x7f62bc005db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.649+0000 7f62ca5c6700 1 --2- 172.21.15.82:0/3158861394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 0x7f62bc005db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f62c406abb0 tx=0x7f62c40773b0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.649+0000 7f62ca5c6700 1 -- 172.21.15.82:0/3158861394 shutdown_connections 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.649+0000 7f62ca5c6700 1 --2- 172.21.15.82:0/3158861394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 0x7f62bc005db0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.649+0000 7f62ca5c6700 1 -- 172.21.15.82:0/3158861394 >> 172.21.15.82:0/3158861394 conn(0x7f62bc09fd10 msgr2=0x7f62bc0a2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:21.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.650+0000 7f62ca5c6700 1 -- 172.21.15.82:0/3158861394 shutdown_connections 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.650+0000 7f62ca5c6700 1 -- 172.21.15.82:0/3158861394 wait complete. 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.650+0000 7f62ca5c6700 1 Processor -- start 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.654+0000 7f62ca5c6700 1 -- start start 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.654+0000 7f62ca5c6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 0x7f62bc014bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.654+0000 7f62ca5c6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc009fb0 con 0x7f62bc005990 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.658+0000 7f62c95c4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 0x7f62bc014bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.658+0000 7f62c95c4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 0x7f62bc014bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57930/0 (socket says 172.21.15.82:57930) 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.658+0000 7f62c95c4700 1 -- 172.21.15.82:0/2348454940 learned_addr learned my addr 172.21.15.82:0/2348454940 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.668+0000 7f62c95c4700 1 -- 172.21.15.82:0/2348454940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62bc0150f0 con 0x7f62bc005990 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.676+0000 7f62c95c4700 1 --2- 172.21.15.82:0/2348454940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 0x7f62bc014bb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f62c406bac0 tx=0x7f62c4051840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.681+0000 7f62ba7fc700 1 -- 172.21.15.82:0/2348454940 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62c40698e0 con 0x7f62bc005990 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.681+0000 7f62ba7fc700 1 -- 172.21.15.82:0/2348454940 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c4084070 con 0x7f62bc005990 2024-04-21T08:37:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.681+0000 7f62ca5c6700 1 -- 172.21.15.82:0/2348454940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62bc015380 con 0x7f62bc005990 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.681+0000 7f62ca5c6700 1 -- 172.21.15.82:0/2348454940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62bc015860 con 0x7f62bc005990 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.682+0000 7f62ba7fc700 1 -- 172.21.15.82:0/2348454940 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62c4068630 con 0x7f62bc005990 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.682+0000 7f62ba7fc700 1 -- 172.21.15.82:0/2348454940 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f62c4068850 con 0x7f62bc005990 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.682+0000 7f62ca5c6700 1 -- 172.21.15.82:0/2348454940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62bc0048c0 con 0x7f62bc005990 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.682+0000 7f62ba7fc700 1 --2- 172.21.15.82:0/2348454940 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62b00396a0 0x7f62b003bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.682+0000 7f62ba7fc700 1 -- 172.21.15.82:0/2348454940 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f62c40bb870 con 0x7f62bc005990 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.682+0000 7f62c8dc3700 1 -- 172.21.15.82:0/2348454940 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62b00396a0 msgr2=0x7f62b003bb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/880382736 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.682+0000 7f62c8dc3700 1 --2- 172.21.15.82:0/2348454940 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62b00396a0 0x7f62b003bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.690+0000 7f62ba7fc700 1 -- 172.21.15.82:0/2348454940 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62c4098070 con 0x7f62bc005990 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.883+0000 7f62c8dc3700 1 -- 172.21.15.82:0/2348454940 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62b00396a0 msgr2=0x7f62b003bb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/880382736 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.883+0000 7f62c8dc3700 1 --2- 172.21.15.82:0/2348454940 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62b00396a0 0x7f62b003bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.954+0000 7f62ca5c6700 1 -- 172.21.15.82:0/2348454940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f62bc14d210 con 0x7f62bc005990 2024-04-21T08:37:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.955+0000 7f62ba7fc700 1 -- 172.21.15.82:0/2348454940 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f62c4088e60 con 0x7f62bc005990 2024-04-21T08:37:21.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.957+0000 7f62ca5c6700 1 -- 172.21.15.82:0/2348454940 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62b00396a0 msgr2=0x7f62b003bb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:37:21.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.957+0000 7f62ca5c6700 1 --2- 172.21.15.82:0/2348454940 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62b00396a0 0x7f62b003bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.957+0000 7f62ca5c6700 1 -- 172.21.15.82:0/2348454940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 msgr2=0x7f62bc014bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:21.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.957+0000 7f62ca5c6700 1 --2- 172.21.15.82:0/2348454940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 0x7f62bc014bb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f62c406bac0 tx=0x7f62c4051840 comp rx=0 tx=0).stop 2024-04-21T08:37:21.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.957+0000 7f62ca5c6700 1 -- 172.21.15.82:0/2348454940 shutdown_connections 2024-04-21T08:37:21.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.957+0000 7f62ca5c6700 1 --2- 172.21.15.82:0/2348454940 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f62b00396a0 0x7f62b003bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.957+0000 7f62ca5c6700 1 --2- 172.21.15.82:0/2348454940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62bc005990 0x7f62bc014bb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:21.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.957+0000 7f62ca5c6700 1 -- 172.21.15.82:0/2348454940 >> 172.21.15.82:0/2348454940 conn(0x7f62bc09fd10 msgr2=0x7f62bc0a0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:21.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.957+0000 7f62ca5c6700 1 -- 172.21.15.82:0/2348454940 shutdown_connections 2024-04-21T08:37:21.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:20.957+0000 7f62ca5c6700 1 -- 172.21.15.82:0/2348454940 wait complete. 2024-04-21T08:37:21.452 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for the mgr to restart... 2024-04-21T08:37:21.452 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr epoch 9... 2024-04-21T08:37:21.573 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:21 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2348454940' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-21T08:37:21.574 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:21 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:21.447+0000 7fce901d1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:37:21.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:21 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:21.623+0000 7fce901d1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:37:21.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:21 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:21.799+0000 7fce901d1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:37:22.186 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:22 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:22.093+0000 7fce901d1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:37:22.186 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:22 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:22.186+0000 7fce901d1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:37:22.499 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:22 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:22.391+0000 7fce901d1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:37:22.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:22 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:22.584+0000 7fce901d1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:37:23.599 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:23 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:23.179+0000 7fce901d1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:37:23.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:23 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:23.600+0000 7fce901d1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:37:23.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:23 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:23.695+0000 7fce901d1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:37:24.358 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:23 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:23.895+0000 7fce901d1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:37:24.750 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:24.413+0000 7fce901d1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:37:24.750 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:24.505+0000 7fce901d1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:37:25.097 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:24.750+0000 7fce901d1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:37:25.097 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:24.883+0000 7fce901d1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:37:25.098 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:24.979+0000 7fce901d1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:37:25.098 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:25 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:25.098+0000 7fce901d1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:37:25.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:25 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:25.409+0000 7fce901d1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:37:26.278 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:26.008+0000 7fce901d1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:37:26.278 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:37:26.225+0000 7fce901d1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:37:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-mon[27967]: Active manager daemon x restarted 2024-04-21T08:37:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-mon[27967]: Activating manager daemon x 2024-04-21T08:37:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-mon[27967]: osdmap e3: 0 total, 0 up, 0 in 2024-04-21T08:37:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-mon[27967]: mgrmap e10: x(active, starting, since 0.00355108s) 2024-04-21T08:37:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:37:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:37:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:37:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:37:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:37:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-mon[27967]: Manager daemon x is now available 2024-04-21T08:37:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:37:27.560 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:37:27.560 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:27 smithi082 ceph-mon[27967]: mgrmap e11: x(active, since 1.00605s) 2024-04-21T08:37:27.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:37:27.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-21T08:37:27.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-21T08:37:27.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:37:27.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.893+0000 7f2917b0a700 1 Processor -- start 2024-04-21T08:37:27.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.894+0000 7f2917b0a700 1 -- start start 2024-04-21T08:37:27.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.894+0000 7f2917b0a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 0x7f291010b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.894+0000 7f2917b0a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f291007be40 con 0x7f2910108ec0 2024-04-21T08:37:27.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.894+0000 7f29158a6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 0x7f291010b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.894+0000 7f29158a6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 0x7f291010b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57934/0 (socket says 172.21.15.82:57934) 2024-04-21T08:37:27.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.894+0000 7f29158a6700 1 -- 172.21.15.82:0/2160406102 learned_addr learned my addr 172.21.15.82:0/2160406102 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:27.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.894+0000 7f29158a6700 1 -- 172.21.15.82:0/2160406102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291010b7f0 con 0x7f2910108ec0 2024-04-21T08:37:27.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.894+0000 7f29158a6700 1 --2- 172.21.15.82:0/2160406102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 0x7f291010b2b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f29000098a0 tx=0x7f2900009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391bd756413e4062 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.895+0000 7f29148a4700 1 -- 172.21.15.82:0/2160406102 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2900004160 con 0x7f2910108ec0 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.895+0000 7f29148a4700 1 -- 172.21.15.82:0/2160406102 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2900004300 con 0x7f2910108ec0 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.895+0000 7f29148a4700 1 -- 172.21.15.82:0/2160406102 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2900011a20 con 0x7f2910108ec0 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.895+0000 7f2917b0a700 1 -- 172.21.15.82:0/2160406102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 msgr2=0x7f291010b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.895+0000 7f2917b0a700 1 --2- 172.21.15.82:0/2160406102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 0x7f291010b2b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f29000098a0 tx=0x7f2900009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.895+0000 7f2917b0a700 1 -- 172.21.15.82:0/2160406102 shutdown_connections 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.895+0000 7f2917b0a700 1 --2- 172.21.15.82:0/2160406102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 0x7f291010b2b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.895+0000 7f2917b0a700 1 -- 172.21.15.82:0/2160406102 >> 172.21.15.82:0/2160406102 conn(0x7f2910102890 msgr2=0x7f2910104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.896+0000 7f2917b0a700 1 -- 172.21.15.82:0/2160406102 shutdown_connections 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.896+0000 7f2917b0a700 1 -- 172.21.15.82:0/2160406102 wait complete. 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.896+0000 7f2917b0a700 1 Processor -- start 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.896+0000 7f2917b0a700 1 -- start start 2024-04-21T08:37:27.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.896+0000 7f2917b0a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 0x7f291019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.896+0000 7f2917b0a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f291010c7a0 con 0x7f2910108ec0 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.896+0000 7f29158a6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 0x7f291019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.896+0000 7f29158a6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 0x7f291019c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57950/0 (socket says 172.21.15.82:57950) 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.896+0000 7f29158a6700 1 -- 172.21.15.82:0/3139980322 learned_addr learned my addr 172.21.15.82:0/3139980322 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.897+0000 7f29158a6700 1 -- 172.21.15.82:0/3139980322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291019c6d0 con 0x7f2910108ec0 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.897+0000 7f29158a6700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 0x7f291019c190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f29000116e0 tx=0x7f2900011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.897+0000 7f2906ffd700 1 -- 172.21.15.82:0/3139980322 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2900003a60 con 0x7f2910108ec0 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.897+0000 7f2917b0a700 1 -- 172.21.15.82:0/3139980322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291019c960 con 0x7f2910108ec0 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.897+0000 7f2906ffd700 1 -- 172.21.15.82:0/3139980322 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290001b8f0 con 0x7f2910108ec0 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.897+0000 7f2906ffd700 1 -- 172.21.15.82:0/3139980322 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2900019860 con 0x7f2910108ec0 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.897+0000 7f2917b0a700 1 -- 172.21.15.82:0/3139980322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291019f650 con 0x7f2910108ec0 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.898+0000 7f2906ffd700 1 -- 172.21.15.82:0/3139980322 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f2900019b60 con 0x7f2910108ec0 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.898+0000 7f2906ffd700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 0x7f28fc03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.898+0000 7f2917b0a700 1 -- 172.21.15.82:0/3139980322 --> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2910069b50 con 0x7f28fc039670 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.898+0000 7f2906ffd700 1 -- 172.21.15.82:0/3139980322 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f290004ecc0 con 0x7f2910108ec0 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.898+0000 7f29150a5700 1 -- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 msgr2=0x7f28fc03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/880382736 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:21.898+0000 7f29150a5700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 0x7f28fc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:22.099+0000 7f29150a5700 1 -- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 msgr2=0x7f28fc03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/880382736 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:22.099+0000 7f29150a5700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 0x7f28fc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:22.500+0000 7f29150a5700 1 -- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 msgr2=0x7f28fc03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/880382736 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:22.500+0000 7f29150a5700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 0x7f28fc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:23.301+0000 7f29150a5700 1 -- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 msgr2=0x7f28fc03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/880382736 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:23.301+0000 7f29150a5700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 0x7f28fc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:24.902+0000 7f29150a5700 1 -- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 msgr2=0x7f28fc03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/880382736 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:24.902+0000 7f29150a5700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 0x7f28fc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:26.229+0000 7f2906ffd700 1 -- 172.21.15.82:0/3139980322 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f290001a8f0 con 0x7f2910108ec0 2024-04-21T08:37:27.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:26.229+0000 7f2906ffd700 1 -- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 msgr2=0x7f28fc03bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:37:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:26.229+0000 7f2906ffd700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 0x7f28fc03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.230+0000 7f2906ffd700 1 -- 172.21.15.82:0/3139980322 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f290001a3d0 con 0x7f2910108ec0 2024-04-21T08:37:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.230+0000 7f2906ffd700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f28fc03cf00 0x7f28fc03f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.230+0000 7f2906ffd700 1 -- 172.21.15.82:0/3139980322 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2910069b50 con 0x7f28fc03cf00 2024-04-21T08:37:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.231+0000 7f29150a5700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f28fc03cf00 0x7f28fc03f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.231+0000 7f29150a5700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f28fc03cf00 0x7f28fc03f2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f290c003a10 tx=0x7f290c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.232+0000 7f2906ffd700 1 -- 172.21.15.82:0/3139980322 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f2910069b50 con 0x7f28fc03cf00 2024-04-21T08:37:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.238+0000 7f2917b0a700 1 -- 172.21.15.82:0/3139980322 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f29101039f0 con 0x7f28fc03cf00 2024-04-21T08:37:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.239+0000 7f2906ffd700 1 -- 172.21.15.82:0/3139980322 <== mgr.14152 v2:172.21.15.82:6800/2138749385 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f29101039f0 con 0x7f28fc03cf00 2024-04-21T08:37:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.239+0000 7f2917b0a700 1 -- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f28fc03cf00 msgr2=0x7f28fc03f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.239+0000 7f2917b0a700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f28fc03cf00 0x7f28fc03f2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f290c003a10 tx=0x7f290c0092b0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.239+0000 7f2917b0a700 1 -- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 msgr2=0x7f291019c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.239+0000 7f2917b0a700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 0x7f291019c190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f29000116e0 tx=0x7f2900011aa0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.239+0000 7f2917b0a700 1 -- 172.21.15.82:0/3139980322 shutdown_connections 2024-04-21T08:37:27.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.239+0000 7f2917b0a700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f28fc03cf00 0x7f28fc03f2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.239+0000 7f2917b0a700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910108ec0 0x7f291019c190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.239+0000 7f2917b0a700 1 --2- 172.21.15.82:0/3139980322 >> [v2:172.21.15.82:6800/880382736,v1:172.21.15.82:6801/880382736] conn(0x7f28fc039670 0x7f28fc03bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.239+0000 7f2917b0a700 1 -- 172.21.15.82:0/3139980322 >> 172.21.15.82:0/3139980322 conn(0x7f2910102890 msgr2=0x7f29101032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:27.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.239+0000 7f2917b0a700 1 -- 172.21.15.82:0/3139980322 shutdown_connections 2024-04-21T08:37:27.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:27.239+0000 7f2917b0a700 1 -- 172.21.15.82:0/3139980322 wait complete. 2024-04-21T08:37:27.846 INFO:teuthology.orchestra.run.smithi082.stdout:mgr epoch 9 is available 2024-04-21T08:37:27.846 INFO:teuthology.orchestra.run.smithi082.stdout:Generating a dashboard self-signed certificate... 2024-04-21T08:37:28.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:28 smithi082 ceph-mon[27967]: [21/Apr/2024:08:37:26] ENGINE Bus STARTING 2024-04-21T08:37:28.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:28 smithi082 ceph-mon[27967]: [21/Apr/2024:08:37:26] ENGINE Serving on https://172.21.15.82:7150 2024-04-21T08:37:28.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:28 smithi082 ceph-mon[27967]: [21/Apr/2024:08:37:26] ENGINE Serving on http://172.21.15.82:8765 2024-04-21T08:37:28.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:28 smithi082 ceph-mon[27967]: [21/Apr/2024:08:37:26] ENGINE Bus STARTED 2024-04-21T08:37:28.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:28 smithi082 ceph-mon[27967]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-21T08:37:28.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:28 smithi082 ceph-mon[27967]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-21T08:37:29.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:29 smithi082 ceph-mon[27967]: mgrmap e12: x(active, since 2s) 2024-04-21T08:37:29.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:29.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:29.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.257+0000 7f332a0c5700 1 Processor -- start 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.257+0000 7f332a0c5700 1 -- start start 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.257+0000 7f332a0c5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 0x7f3324107730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.257+0000 7f332a0c5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3324107d00 con 0x7f3324107310 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.257+0000 7f33237fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 0x7f3324107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.257+0000 7f33237fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 0x7f3324107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60120/0 (socket says 172.21.15.82:60120) 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.257+0000 7f33237fe700 1 -- 172.21.15.82:0/3405456204 learned_addr learned my addr 172.21.15.82:0/3405456204 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.258+0000 7f33237fe700 1 -- 172.21.15.82:0/3405456204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3324108510 con 0x7f3324107310 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.258+0000 7f33237fe700 1 --2- 172.21.15.82:0/3405456204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 0x7f3324107730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f33140098a0 tx=0x7f3314009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d59eefac162e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.258+0000 7f3322ffd700 1 -- 172.21.15.82:0/3405456204 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3314004160 con 0x7f3324107310 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.258+0000 7f3322ffd700 1 -- 172.21.15.82:0/3405456204 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3314004300 con 0x7f3324107310 2024-04-21T08:37:29.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.258+0000 7f3322ffd700 1 -- 172.21.15.82:0/3405456204 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3314011a20 con 0x7f3324107310 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.258+0000 7f332a0c5700 1 -- 172.21.15.82:0/3405456204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 msgr2=0x7f3324107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.258+0000 7f332a0c5700 1 --2- 172.21.15.82:0/3405456204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 0x7f3324107730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f33140098a0 tx=0x7f3314009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.259+0000 7f332a0c5700 1 -- 172.21.15.82:0/3405456204 shutdown_connections 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.259+0000 7f332a0c5700 1 --2- 172.21.15.82:0/3405456204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 0x7f3324107730 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.259+0000 7f332a0c5700 1 -- 172.21.15.82:0/3405456204 >> 172.21.15.82:0/3405456204 conn(0x7f33241028b0 msgr2=0x7f3324104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.259+0000 7f332a0c5700 1 -- 172.21.15.82:0/3405456204 shutdown_connections 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.259+0000 7f332a0c5700 1 -- 172.21.15.82:0/3405456204 wait complete. 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.259+0000 7f332a0c5700 1 Processor -- start 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.259+0000 7f332a0c5700 1 -- start start 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.259+0000 7f332a0c5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 0x7f332419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.260+0000 7f332a0c5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f332410b8c0 con 0x7f3324107310 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.260+0000 7f33237fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 0x7f332419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.260+0000 7f33237fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 0x7f332419c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60126/0 (socket says 172.21.15.82:60126) 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.260+0000 7f33237fe700 1 -- 172.21.15.82:0/3166402912 learned_addr learned my addr 172.21.15.82:0/3166402912 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:29.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.260+0000 7f33237fe700 1 -- 172.21.15.82:0/3166402912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332419c800 con 0x7f3324107310 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.260+0000 7f33237fe700 1 --2- 172.21.15.82:0/3166402912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 0x7f332419c2c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3314004660 tx=0x7f3314011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.260+0000 7f33217fa700 1 -- 172.21.15.82:0/3166402912 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3314003a60 con 0x7f3324107310 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.260+0000 7f33217fa700 1 -- 172.21.15.82:0/3166402912 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f331401b8f0 con 0x7f3324107310 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.260+0000 7f332a0c5700 1 -- 172.21.15.82:0/3166402912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332419ca90 con 0x7f3324107310 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.260+0000 7f33217fa700 1 -- 172.21.15.82:0/3166402912 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3314023890 con 0x7f3324107310 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.260+0000 7f332a0c5700 1 -- 172.21.15.82:0/3166402912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f332419f780 con 0x7f3324107310 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.261+0000 7f33217fa700 1 -- 172.21.15.82:0/3166402912 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f331402c3f0 con 0x7f3324107310 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.261+0000 7f332a0c5700 1 -- 172.21.15.82:0/3166402912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f332410ba40 con 0x7f3324107310 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.261+0000 7f33217fa700 1 --2- 172.21.15.82:0/3166402912 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f33100395b0 0x7f331003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.261+0000 7f33217fa700 1 -- 172.21.15.82:0/3166402912 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f331404e080 con 0x7f3324107310 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.262+0000 7f331affd700 1 --2- 172.21.15.82:0/3166402912 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f33100395b0 0x7f331003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.262+0000 7f331affd700 1 --2- 172.21.15.82:0/3166402912 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f33100395b0 0x7f331003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f330c006fd0 tx=0x7f330c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.266+0000 7f33217fa700 1 -- 172.21.15.82:0/3166402912 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f331401f030 con 0x7f3324107310 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.291+0000 7f33217fa700 1 -- 172.21.15.82:0/3166402912 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f33140193d0 con 0x7f3324107310 2024-04-21T08:37:29.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.475+0000 7f332a0c5700 1 -- 172.21.15.82:0/3166402912 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f332402e230 con 0x7f33100395b0 2024-04-21T08:37:29.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.752+0000 7f33217fa700 1 -- 172.21.15.82:0/3166402912 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f332402e230 con 0x7f33100395b0 2024-04-21T08:37:29.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.754+0000 7f332a0c5700 1 -- 172.21.15.82:0/3166402912 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f33100395b0 msgr2=0x7f331003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.754+0000 7f332a0c5700 1 --2- 172.21.15.82:0/3166402912 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f33100395b0 0x7f331003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f330c006fd0 tx=0x7f330c006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:29.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.754+0000 7f332a0c5700 1 -- 172.21.15.82:0/3166402912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 msgr2=0x7f332419c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.754+0000 7f332a0c5700 1 --2- 172.21.15.82:0/3166402912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 0x7f332419c2c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3314004660 tx=0x7f3314011aa0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.754+0000 7f332a0c5700 1 -- 172.21.15.82:0/3166402912 shutdown_connections 2024-04-21T08:37:29.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.754+0000 7f332a0c5700 1 --2- 172.21.15.82:0/3166402912 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f33100395b0 0x7f331003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.754+0000 7f332a0c5700 1 --2- 172.21.15.82:0/3166402912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3324107310 0x7f332419c2c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.754+0000 7f332a0c5700 1 -- 172.21.15.82:0/3166402912 >> 172.21.15.82:0/3166402912 conn(0x7f33241028b0 msgr2=0x7f3324192e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:29.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.754+0000 7f332a0c5700 1 -- 172.21.15.82:0/3166402912 shutdown_connections 2024-04-21T08:37:29.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:28.754+0000 7f332a0c5700 1 -- 172.21.15.82:0/3166402912 wait complete. 2024-04-21T08:37:29.342 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial admin user... 2024-04-21T08:37:30.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:30 smithi082 ceph-mon[27967]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:37:30.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:30 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:30.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$EszCfoKxcawT2plrso0T9u/MQE7lusnezDiU4DmIQVNcdeZX27rSy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713688650, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-21T08:37:30.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.822+0000 7fdde01b0700 1 Processor -- start 2024-04-21T08:37:30.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.823+0000 7fdde01b0700 1 -- start start 2024-04-21T08:37:30.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.823+0000 7fdde01b0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 0x7fddd8108820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:30.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.823+0000 7fdde01b0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8108df0 con 0x7fddd8108400 2024-04-21T08:37:30.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.823+0000 7fddddf4c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 0x7fddd8108820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:30.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.823+0000 7fddddf4c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 0x7fddd8108820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60142/0 (socket says 172.21.15.82:60142) 2024-04-21T08:37:30.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.823+0000 7fddddf4c700 1 -- 172.21.15.82:0/3975646452 learned_addr learned my addr 172.21.15.82:0/3975646452 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:30.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.823+0000 7fddddf4c700 1 -- 172.21.15.82:0/3975646452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd81095b0 con 0x7fddd8108400 2024-04-21T08:37:30.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.823+0000 7fddddf4c700 1 --2- 172.21.15.82:0/3975646452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 0x7fddd8108820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fddd40098a0 tx=0x7fddd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461634549f33ac6a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.824+0000 7fdddcf4a700 1 -- 172.21.15.82:0/3975646452 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddd4004160 con 0x7fddd8108400 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.824+0000 7fdddcf4a700 1 -- 172.21.15.82:0/3975646452 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd4004300 con 0x7fddd8108400 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.824+0000 7fdddcf4a700 1 -- 172.21.15.82:0/3975646452 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddd4011a20 con 0x7fddd8108400 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.824+0000 7fdde01b0700 1 -- 172.21.15.82:0/3975646452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 msgr2=0x7fddd8108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.824+0000 7fdde01b0700 1 --2- 172.21.15.82:0/3975646452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 0x7fddd8108820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fddd40098a0 tx=0x7fddd4009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.824+0000 7fdde01b0700 1 -- 172.21.15.82:0/3975646452 shutdown_connections 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.824+0000 7fdde01b0700 1 --2- 172.21.15.82:0/3975646452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 0x7fddd8108820 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.824+0000 7fdde01b0700 1 -- 172.21.15.82:0/3975646452 >> 172.21.15.82:0/3975646452 conn(0x7fddd81039c0 msgr2=0x7fddd8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.824+0000 7fdde01b0700 1 -- 172.21.15.82:0/3975646452 shutdown_connections 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.824+0000 7fdde01b0700 1 -- 172.21.15.82:0/3975646452 wait complete. 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.825+0000 7fdde01b0700 1 Processor -- start 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.825+0000 7fdde01b0700 1 -- start start 2024-04-21T08:37:30.855 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.825+0000 7fdde01b0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 0x7fddd819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:30.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.825+0000 7fdde01b0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fddd810c960 con 0x7fddd8108400 2024-04-21T08:37:30.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.825+0000 7fddddf4c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 0x7fddd819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:30.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.825+0000 7fddddf4c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 0x7fddd819c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60154/0 (socket says 172.21.15.82:60154) 2024-04-21T08:37:30.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.825+0000 7fddddf4c700 1 -- 172.21.15.82:0/3511686439 learned_addr learned my addr 172.21.15.82:0/3511686439 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:30.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.825+0000 7fddddf4c700 1 -- 172.21.15.82:0/3511686439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd819c6d0 con 0x7fddd8108400 2024-04-21T08:37:30.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.826+0000 7fddddf4c700 1 --2- 172.21.15.82:0/3511686439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 0x7fddd819c190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fddd40116e0 tx=0x7fddd4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:30.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.826+0000 7fddceffd700 1 -- 172.21.15.82:0/3511686439 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddd40039e0 con 0x7fddd8108400 2024-04-21T08:37:30.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.826+0000 7fddceffd700 1 -- 172.21.15.82:0/3511686439 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd401b8f0 con 0x7fddd8108400 2024-04-21T08:37:30.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.826+0000 7fdde01b0700 1 -- 172.21.15.82:0/3511686439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd819c960 con 0x7fddd8108400 2024-04-21T08:37:30.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.826+0000 7fddceffd700 1 -- 172.21.15.82:0/3511686439 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddd4023760 con 0x7fddd8108400 2024-04-21T08:37:30.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.826+0000 7fdde01b0700 1 -- 172.21.15.82:0/3511686439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd819f650 con 0x7fddd8108400 2024-04-21T08:37:30.856 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.827+0000 7fddceffd700 1 -- 172.21.15.82:0/3511686439 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fddd40239d0 con 0x7fddd8108400 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.827+0000 7fddceffd700 1 --2- 172.21.15.82:0/3511686439 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fddc4039670 0x7fddc403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.827+0000 7fddceffd700 1 -- 172.21.15.82:0/3511686439 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fddd404e040 con 0x7fddd8108400 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.827+0000 7fdddd74b700 1 --2- 172.21.15.82:0/3511686439 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fddc4039670 0x7fddc403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.827+0000 7fdde01b0700 1 -- 172.21.15.82:0/3511686439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd810cae0 con 0x7fddd8108400 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.827+0000 7fdddd74b700 1 --2- 172.21.15.82:0/3511686439 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fddc4039670 0x7fddc403bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fddc8006fd0 tx=0x7fddc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:29.832+0000 7fddceffd700 1 -- 172.21.15.82:0/3511686439 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fddd401f030 con 0x7fddd8108400 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:30.068+0000 7fdde01b0700 1 -- 172.21.15.82:0/3511686439 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fddd819f820 con 0x7fddc4039670 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:30.297+0000 7fddceffd700 1 -- 172.21.15.82:0/3511686439 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fddd819f820 con 0x7fddc4039670 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:30.299+0000 7fdde01b0700 1 -- 172.21.15.82:0/3511686439 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fddc4039670 msgr2=0x7fddc403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:30.299+0000 7fdde01b0700 1 --2- 172.21.15.82:0/3511686439 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fddc4039670 0x7fddc403bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fddc8006fd0 tx=0x7fddc8006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:30.299+0000 7fdde01b0700 1 -- 172.21.15.82:0/3511686439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 msgr2=0x7fddd819c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:30.299+0000 7fdde01b0700 1 --2- 172.21.15.82:0/3511686439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 0x7fddd819c190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fddd40116e0 tx=0x7fddd4011aa0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:30.299+0000 7fdde01b0700 1 -- 172.21.15.82:0/3511686439 shutdown_connections 2024-04-21T08:37:30.857 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:30.299+0000 7fdde01b0700 1 --2- 172.21.15.82:0/3511686439 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fddc4039670 0x7fddc403bb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:30.299+0000 7fdde01b0700 1 --2- 172.21.15.82:0/3511686439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd8108400 0x7fddd819c190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:30.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:30.299+0000 7fdde01b0700 1 -- 172.21.15.82:0/3511686439 >> 172.21.15.82:0/3511686439 conn(0x7fddd81039c0 msgr2=0x7fddd8104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:30.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:30.299+0000 7fdde01b0700 1 -- 172.21.15.82:0/3511686439 shutdown_connections 2024-04-21T08:37:30.858 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:30.299+0000 7fdde01b0700 1 -- 172.21.15.82:0/3511686439 wait complete. 2024-04-21T08:37:30.858 INFO:teuthology.orchestra.run.smithi082.stdout:Fetching dashboard port number... 2024-04-21T08:37:31.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:31 smithi082 ceph-mon[27967]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:37:31.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:32.565 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:32 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4240737446' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-21T08:37:32.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 8443 2024-04-21T08:37:32.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.299+0000 7f7e3f11c700 1 Processor -- start 2024-04-21T08:37:32.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.299+0000 7f7e3f11c700 1 -- start start 2024-04-21T08:37:32.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.300+0000 7f7e3f11c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 0x7f7e3810b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:32.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.300+0000 7f7e3f11c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3807be60 con 0x7f7e38108e60 2024-04-21T08:37:32.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.300+0000 7f7e3ceb8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 0x7f7e3810b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.300+0000 7f7e3ceb8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 0x7f7e3810b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60170/0 (socket says 172.21.15.82:60170) 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.300+0000 7f7e3ceb8700 1 -- 172.21.15.82:0/1338404803 learned_addr learned my addr 172.21.15.82:0/1338404803 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.300+0000 7f7e3ceb8700 1 -- 172.21.15.82:0/1338404803 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3810b790 con 0x7f7e38108e60 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.300+0000 7f7e3ceb8700 1 --2- 172.21.15.82:0/1338404803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 0x7f7e3810b250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0098a0 tx=0x7f7e2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9858d1597e98b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.300+0000 7f7e377fe700 1 -- 172.21.15.82:0/1338404803 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e2c004160 con 0x7f7e38108e60 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.300+0000 7f7e377fe700 1 -- 172.21.15.82:0/1338404803 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e2c004300 con 0x7f7e38108e60 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.300+0000 7f7e377fe700 1 -- 172.21.15.82:0/1338404803 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e2c011a20 con 0x7f7e38108e60 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.301+0000 7f7e3f11c700 1 -- 172.21.15.82:0/1338404803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 msgr2=0x7f7e3810b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.301+0000 7f7e3f11c700 1 --2- 172.21.15.82:0/1338404803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 0x7f7e3810b250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0098a0 tx=0x7f7e2c009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.301+0000 7f7e3f11c700 1 -- 172.21.15.82:0/1338404803 shutdown_connections 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.301+0000 7f7e3f11c700 1 --2- 172.21.15.82:0/1338404803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 0x7f7e3810b250 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.301+0000 7f7e3f11c700 1 -- 172.21.15.82:0/1338404803 >> 172.21.15.82:0/1338404803 conn(0x7f7e38102890 msgr2=0x7f7e38104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:32.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.301+0000 7f7e3f11c700 1 -- 172.21.15.82:0/1338404803 shutdown_connections 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.301+0000 7f7e3f11c700 1 -- 172.21.15.82:0/1338404803 wait complete. 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.302+0000 7f7e3f11c700 1 Processor -- start 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.302+0000 7f7e3f11c700 1 -- start start 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.302+0000 7f7e3f11c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 0x7f7e381086e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.302+0000 7f7e3f11c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3810c790 con 0x7f7e38108e60 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.302+0000 7f7e3ceb8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 0x7f7e381086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.302+0000 7f7e3ceb8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 0x7f7e381086e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60186/0 (socket says 172.21.15.82:60186) 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.302+0000 7f7e3ceb8700 1 -- 172.21.15.82:0/4240737446 learned_addr learned my addr 172.21.15.82:0/4240737446 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.302+0000 7f7e3ceb8700 1 -- 172.21.15.82:0/4240737446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e38108c20 con 0x7f7e38108e60 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.302+0000 7f7e3ceb8700 1 --2- 172.21.15.82:0/4240737446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 0x7f7e381086e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c000c00 tx=0x7f7e2c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.302+0000 7f7e35ffb700 1 -- 172.21.15.82:0/4240737446 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e2c003aa0 con 0x7f7e38108e60 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.303+0000 7f7e3f11c700 1 -- 172.21.15.82:0/4240737446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e38106e10 con 0x7f7e38108e60 2024-04-21T08:37:32.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.303+0000 7f7e35ffb700 1 -- 172.21.15.82:0/4240737446 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e2c01b8f0 con 0x7f7e38108e60 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.303+0000 7f7e35ffb700 1 -- 172.21.15.82:0/4240737446 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e2c023870 con 0x7f7e38108e60 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.303+0000 7f7e3f11c700 1 -- 172.21.15.82:0/4240737446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e38107350 con 0x7f7e38108e60 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.304+0000 7f7e35ffb700 1 -- 172.21.15.82:0/4240737446 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f7e2c023b20 con 0x7f7e38108e60 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.304+0000 7f7e3f11c700 1 -- 172.21.15.82:0/4240737446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e380640a0 con 0x7f7e38108e60 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.304+0000 7f7e35ffb700 1 --2- 172.21.15.82:0/4240737446 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7e28039720 0x7f7e2803bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.304+0000 7f7e35ffb700 1 -- 172.21.15.82:0/4240737446 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7e2c04df50 con 0x7f7e38108e60 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.304+0000 7f7e37fff700 1 --2- 172.21.15.82:0/4240737446 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7e28039720 0x7f7e2803bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.304+0000 7f7e37fff700 1 --2- 172.21.15.82:0/4240737446 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7e28039720 0x7f7e2803bbe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7e24006fd0 tx=0x7f7e24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.309+0000 7f7e35ffb700 1 -- 172.21.15.82:0/4240737446 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e2c01f030 con 0x7f7e38108e60 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.514+0000 7f7e3f11c700 1 -- 172.21.15.82:0/4240737446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f7e381075d0 con 0x7f7e38108e60 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.515+0000 7f7e35ffb700 1 -- 172.21.15.82:0/4240737446 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f7e2c048020 con 0x7f7e38108e60 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.517+0000 7f7e3f11c700 1 -- 172.21.15.82:0/4240737446 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7e28039720 msgr2=0x7f7e2803bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.517+0000 7f7e3f11c700 1 --2- 172.21.15.82:0/4240737446 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7e28039720 0x7f7e2803bbe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7e24006fd0 tx=0x7f7e24006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.517+0000 7f7e3f11c700 1 -- 172.21.15.82:0/4240737446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 msgr2=0x7f7e381086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:32.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.517+0000 7f7e3f11c700 1 --2- 172.21.15.82:0/4240737446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 0x7f7e381086e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c000c00 tx=0x7f7e2c011aa0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.517+0000 7f7e3f11c700 1 -- 172.21.15.82:0/4240737446 shutdown_connections 2024-04-21T08:37:32.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.517+0000 7f7e3f11c700 1 --2- 172.21.15.82:0/4240737446 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7e28039720 0x7f7e2803bbe0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.517+0000 7f7e3f11c700 1 --2- 172.21.15.82:0/4240737446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e38108e60 0x7f7e381086e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:32.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.517+0000 7f7e3f11c700 1 -- 172.21.15.82:0/4240737446 >> 172.21.15.82:0/4240737446 conn(0x7f7e38102890 msgr2=0x7f7e38103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:32.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.517+0000 7f7e3f11c700 1 -- 172.21.15.82:0/4240737446 shutdown_connections 2024-04-21T08:37:32.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:31.517+0000 7f7e3f11c700 1 -- 172.21.15.82:0/4240737446 wait complete. 2024-04-21T08:37:32.841 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:37:32.841 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-04-21T08:37:32.849 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:37:32.849 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-04-21T08:37:32.849 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-04-21T08:37:32.849 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-21T08:37:32.851 INFO:teuthology.orchestra.run.smithi082.stdout:Ceph Dashboard is now available at: 2024-04-21T08:37:32.851 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:37:32.851 INFO:teuthology.orchestra.run.smithi082.stdout: URL: https://smithi082:8443/ 2024-04-21T08:37:32.851 INFO:teuthology.orchestra.run.smithi082.stdout: User: admin 2024-04-21T08:37:32.851 INFO:teuthology.orchestra.run.smithi082.stdout: Password: d47kvvvm4s 2024-04-21T08:37:32.851 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:37:32.852 INFO:teuthology.orchestra.run.smithi082.stdout:Saving cluster configuration to /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config directory 2024-04-21T08:37:32.852 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling autotune for osd_memory_target 2024-04-21T08:37:33.471 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:33 smithi082 ceph-mon[27967]: mgrmap e13: x(active, since 6s) 2024-04-21T08:37:34.063 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.266+0000 7fe2c64bd700 1 Processor -- start 2024-04-21T08:37:34.063 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.266+0000 7fe2c64bd700 1 -- start start 2024-04-21T08:37:34.063 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.266+0000 7fe2c64bd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 0x7fe2c0106e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.266+0000 7fe2c64bd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c01073f0 con 0x7fe2c0106a00 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.266+0000 7fe2bffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 0x7fe2c0106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.267+0000 7fe2bffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 0x7fe2c0106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34868/0 (socket says 172.21.15.82:34868) 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.267+0000 7fe2bffff700 1 -- 172.21.15.82:0/1947472616 learned_addr learned my addr 172.21.15.82:0/1947472616 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.267+0000 7fe2bffff700 1 -- 172.21.15.82:0/1947472616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c0107c00 con 0x7fe2c0106a00 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.267+0000 7fe2bffff700 1 --2- 172.21.15.82:0/1947472616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 0x7fe2c0106e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000aa40 tx=0x7fe2b000ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aadae78167e828a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.267+0000 7fe2beffd700 1 -- 172.21.15.82:0/1947472616 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b000b040 con 0x7fe2c0106a00 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.267+0000 7fe2beffd700 1 -- 172.21.15.82:0/1947472616 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b000c8f0 con 0x7fe2c0106a00 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.267+0000 7fe2beffd700 1 -- 172.21.15.82:0/1947472616 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b0003ae0 con 0x7fe2c0106a00 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.268+0000 7fe2c64bd700 1 -- 172.21.15.82:0/1947472616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 msgr2=0x7fe2c0106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.268+0000 7fe2c64bd700 1 --2- 172.21.15.82:0/1947472616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 0x7fe2c0106e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000aa40 tx=0x7fe2b000ad50 comp rx=0 tx=0).stop 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.268+0000 7fe2c64bd700 1 -- 172.21.15.82:0/1947472616 shutdown_connections 2024-04-21T08:37:34.064 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.268+0000 7fe2c64bd700 1 --2- 172.21.15.82:0/1947472616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 0x7fe2c0106e20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.268+0000 7fe2c64bd700 1 -- 172.21.15.82:0/1947472616 >> 172.21.15.82:0/1947472616 conn(0x7fe2c0102040 msgr2=0x7fe2c0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.268+0000 7fe2c64bd700 1 -- 172.21.15.82:0/1947472616 shutdown_connections 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.268+0000 7fe2c64bd700 1 -- 172.21.15.82:0/1947472616 wait complete. 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.268+0000 7fe2c64bd700 1 Processor -- start 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.268+0000 7fe2c64bd700 1 -- start start 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.269+0000 7fe2c64bd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 0x7fe2c018f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.269+0000 7fe2c64bd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c0108370 con 0x7fe2c0106a00 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.269+0000 7fe2bffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 0x7fe2c018f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.269+0000 7fe2bffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 0x7fe2c018f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34876/0 (socket says 172.21.15.82:34876) 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.269+0000 7fe2bffff700 1 -- 172.21.15.82:0/2458019222 learned_addr learned my addr 172.21.15.82:0/2458019222 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.269+0000 7fe2bffff700 1 -- 172.21.15.82:0/2458019222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c018fb00 con 0x7fe2c0106a00 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.269+0000 7fe2bffff700 1 --2- 172.21.15.82:0/2458019222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 0x7fe2c018f5c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0003820 tx=0x7fe2b00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:34.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.269+0000 7fe2bd7fa700 1 -- 172.21.15.82:0/2458019222 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b0021410 con 0x7fe2c0106a00 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.269+0000 7fe2bd7fa700 1 -- 172.21.15.82:0/2458019222 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b00108a0 con 0x7fe2c0106a00 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.269+0000 7fe2c64bd700 1 -- 172.21.15.82:0/2458019222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c018dcf0 con 0x7fe2c0106a00 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.269+0000 7fe2bd7fa700 1 -- 172.21.15.82:0/2458019222 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b0018740 con 0x7fe2c0106a00 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.269+0000 7fe2c64bd700 1 -- 172.21.15.82:0/2458019222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c018e230 con 0x7fe2c0106a00 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.270+0000 7fe2bd7fa700 1 -- 172.21.15.82:0/2458019222 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe2b001f070 con 0x7fe2c0106a00 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.270+0000 7fe2c64bd700 1 -- 172.21.15.82:0/2458019222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c00640a0 con 0x7fe2c0106a00 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.270+0000 7fe2bd7fa700 1 --2- 172.21.15.82:0/2458019222 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe2a8039710 0x7fe2a803bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.270+0000 7fe2bd7fa700 1 -- 172.21.15.82:0/2458019222 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe2b004dec0 con 0x7fe2c0106a00 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.270+0000 7fe2bf7fe700 1 --2- 172.21.15.82:0/2458019222 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe2a8039710 0x7fe2a803bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.271+0000 7fe2bf7fe700 1 --2- 172.21.15.82:0/2458019222 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe2a8039710 0x7fe2a803bbd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4006fd0 tx=0x7fe2b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.275+0000 7fe2bd7fa700 1 -- 172.21.15.82:0/2458019222 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe2b0014070 con 0x7fe2c0106a00 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.471+0000 7fe2c64bd700 1 -- 172.21.15.82:0/2458019222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fe2c0195ba0 con 0x7fe2c0106a00 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.472+0000 7fe2bd7fa700 1 -- 172.21.15.82:0/2458019222 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fe2b0010d50 con 0x7fe2c0106a00 2024-04-21T08:37:34.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.474+0000 7fe2c64bd700 1 -- 172.21.15.82:0/2458019222 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe2a8039710 msgr2=0x7fe2a803bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:34.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.474+0000 7fe2c64bd700 1 --2- 172.21.15.82:0/2458019222 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe2a8039710 0x7fe2a803bbd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4006fd0 tx=0x7fe2b4006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:34.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.474+0000 7fe2c64bd700 1 -- 172.21.15.82:0/2458019222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 msgr2=0x7fe2c018f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:34.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.474+0000 7fe2c64bd700 1 --2- 172.21.15.82:0/2458019222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 0x7fe2c018f5c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0003820 tx=0x7fe2b00040c0 comp rx=0 tx=0).stop 2024-04-21T08:37:34.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.474+0000 7fe2c64bd700 1 -- 172.21.15.82:0/2458019222 shutdown_connections 2024-04-21T08:37:34.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.474+0000 7fe2c64bd700 1 --2- 172.21.15.82:0/2458019222 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe2a8039710 0x7fe2a803bbd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:34.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.474+0000 7fe2c64bd700 1 --2- 172.21.15.82:0/2458019222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c0106a00 0x7fe2c018f5c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:34.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.474+0000 7fe2c64bd700 1 -- 172.21.15.82:0/2458019222 >> 172.21.15.82:0/2458019222 conn(0x7fe2c0102040 msgr2=0x7fe2c0192e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:34.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.474+0000 7fe2c64bd700 1 -- 172.21.15.82:0/2458019222 shutdown_connections 2024-04-21T08:37:34.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:33.474+0000 7fe2c64bd700 1 -- 172.21.15.82:0/2458019222 wait complete. 2024-04-21T08:37:35.413 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.523+0000 7f4093800700 1 Processor -- start 2024-04-21T08:37:35.414 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.523+0000 7f4093800700 1 -- start start 2024-04-21T08:37:35.414 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.523+0000 7f4093800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 0x7f408c106df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:35.414 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.524+0000 7f4093800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f408c1073c0 con 0x7f408c1069d0 2024-04-21T08:37:35.414 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.524+0000 7f409159c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 0x7f408c106df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:35.414 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.524+0000 7f409159c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 0x7f408c106df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34890/0 (socket says 172.21.15.82:34890) 2024-04-21T08:37:35.414 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.524+0000 7f409159c700 1 -- 172.21.15.82:0/483699105 learned_addr learned my addr 172.21.15.82:0/483699105 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:35.414 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.524+0000 7f409159c700 1 -- 172.21.15.82:0/483699105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c107bd0 con 0x7f408c1069d0 2024-04-21T08:37:35.414 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.524+0000 7f409159c700 1 --2- 172.21.15.82:0/483699105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 0x7f408c106df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f40880098a0 tx=0x7f4088009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526d955f671e6741 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:35.414 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.524+0000 7f4083fff700 1 -- 172.21.15.82:0/483699105 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4088004160 con 0x7f408c1069d0 2024-04-21T08:37:35.414 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.524+0000 7f4083fff700 1 -- 172.21.15.82:0/483699105 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4088004300 con 0x7f408c1069d0 2024-04-21T08:37:35.414 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.524+0000 7f4083fff700 1 -- 172.21.15.82:0/483699105 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4088011a20 con 0x7f408c1069d0 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.525+0000 7f4093800700 1 -- 172.21.15.82:0/483699105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 msgr2=0x7f408c106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.525+0000 7f4093800700 1 --2- 172.21.15.82:0/483699105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 0x7f408c106df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f40880098a0 tx=0x7f4088009bb0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.525+0000 7f4093800700 1 -- 172.21.15.82:0/483699105 shutdown_connections 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.525+0000 7f4093800700 1 --2- 172.21.15.82:0/483699105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 0x7f408c106df0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.525+0000 7f4093800700 1 -- 172.21.15.82:0/483699105 >> 172.21.15.82:0/483699105 conn(0x7f408c101f50 msgr2=0x7f408c1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.525+0000 7f4093800700 1 -- 172.21.15.82:0/483699105 shutdown_connections 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.525+0000 7f4093800700 1 -- 172.21.15.82:0/483699105 wait complete. 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.526+0000 7f4093800700 1 Processor -- start 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.526+0000 7f4093800700 1 -- start start 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.526+0000 7f4093800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 0x7f408c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.526+0000 7f4093800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f408c108340 con 0x7f408c1069d0 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.526+0000 7f409159c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 0x7f408c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:35.415 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.526+0000 7f409159c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 0x7f408c199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34904/0 (socket says 172.21.15.82:34904) 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.526+0000 7f409159c700 1 -- 172.21.15.82:0/3795281490 learned_addr learned my addr 172.21.15.82:0/3795281490 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.526+0000 7f409159c700 1 -- 172.21.15.82:0/3795281490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c19a500 con 0x7f408c1069d0 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.527+0000 7f409159c700 1 --2- 172.21.15.82:0/3795281490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 0x7f408c199fc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40880116e0 tx=0x7f4088011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.527+0000 7f40827fc700 1 -- 172.21.15.82:0/3795281490 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40880039e0 con 0x7f408c1069d0 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.527+0000 7f40827fc700 1 -- 172.21.15.82:0/3795281490 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f408801b8f0 con 0x7f408c1069d0 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.527+0000 7f4093800700 1 -- 172.21.15.82:0/3795281490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408c19a790 con 0x7f408c1069d0 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.527+0000 7f40827fc700 1 -- 172.21.15.82:0/3795281490 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4088023760 con 0x7f408c1069d0 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.527+0000 7f4093800700 1 -- 172.21.15.82:0/3795281490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408c19d480 con 0x7f408c1069d0 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.528+0000 7f40827fc700 1 -- 172.21.15.82:0/3795281490 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4088023940 con 0x7f408c1069d0 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.528+0000 7f4093800700 1 -- 172.21.15.82:0/3795281490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f408c0640a0 con 0x7f408c1069d0 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.528+0000 7f40827fc700 1 --2- 172.21.15.82:0/3795281490 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f407803a890 0x7f407803cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.528+0000 7f40827fc700 1 -- 172.21.15.82:0/3795281490 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f408804e0a0 con 0x7f408c1069d0 2024-04-21T08:37:35.416 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.528+0000 7f4090d9b700 1 --2- 172.21.15.82:0/3795281490 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f407803a890 0x7f407803cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.528+0000 7f4090d9b700 1 --2- 172.21.15.82:0/3795281490 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f407803a890 0x7f407803cd50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f407c006fd0 tx=0x7f407c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.533+0000 7f40827fc700 1 -- 172.21.15.82:0/3795281490 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f408801f030 con 0x7f408c1069d0 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.841+0000 7f4093800700 1 -- 172.21.15.82:0/3795281490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f408c02e1d0 con 0x7f408c1069d0 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.844+0000 7f40827fc700 1 -- 172.21.15.82:0/3795281490 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f408800b3e0 con 0x7f408c1069d0 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.846+0000 7f4093800700 1 -- 172.21.15.82:0/3795281490 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f407803a890 msgr2=0x7f407803cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.846+0000 7f4093800700 1 --2- 172.21.15.82:0/3795281490 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f407803a890 0x7f407803cd50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f407c006fd0 tx=0x7f407c006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.846+0000 7f4093800700 1 -- 172.21.15.82:0/3795281490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 msgr2=0x7f408c199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.846+0000 7f4093800700 1 --2- 172.21.15.82:0/3795281490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 0x7f408c199fc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40880116e0 tx=0x7f4088011aa0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.846+0000 7f4093800700 1 -- 172.21.15.82:0/3795281490 shutdown_connections 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.846+0000 7f4093800700 1 --2- 172.21.15.82:0/3795281490 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f407803a890 0x7f407803cd50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.846+0000 7f4093800700 1 --2- 172.21.15.82:0/3795281490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f408c1069d0 0x7f408c199fc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.846+0000 7f4093800700 1 -- 172.21.15.82:0/3795281490 >> 172.21.15.82:0/3795281490 conn(0x7f408c101f50 msgr2=0x7f408c190be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.846+0000 7f4093800700 1 -- 172.21.15.82:0/3795281490 shutdown_connections 2024-04-21T08:37:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-21T08:37:34.846+0000 7f4093800700 1 -- 172.21.15.82:0/3795281490 wait complete. 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout:Or, if you are only running a single cluster on this host: 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout: ceph telemetry on 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:37:35.418 INFO:teuthology.orchestra.run.smithi082.stdout:For more information see: 2024-04-21T08:37:35.419 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:37:35.419 INFO:teuthology.orchestra.run.smithi082.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-21T08:37:35.419 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:37:35.419 INFO:teuthology.orchestra.run.smithi082.stdout:Bootstrap complete. 2024-04-21T08:37:35.450 INFO:tasks.cephadm:Fetching config... 2024-04-21T08:37:35.451 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:37:35.451 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-21T08:37:35.466 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-21T08:37:35.466 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:37:35.466 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-21T08:37:35.521 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-21T08:37:35.521 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:37:35.521 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-21T08:37:35.586 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-21T08:37:35.587 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:37:35.587 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-21T08:37:35.642 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-21T08:37:35.642 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkNmgVHbETqv/6Sm6ZaE235c5uRROeXx5CPqkKQw3LqoV/VPtZMLDNXzKEvNnlcszIi4Nu8DcTHTC6VkIgPq7E5Rs+wKjT2zgQfuYuDNi+SzfUaUuMesHcxZ103rDXlLrMSkHTIgrOemEGvcK0g019L8MGiY+ZCMI/KP/LXBAprKR+0M1f7Q1XjkA4ThhXViqo4n7TqJHi5GX9lIQuXShEblFb5Q2M3UT5VDJ4S6+C75HKCeJKhSnoqwZy/3onqLWdviPBee338x7Mz1SOpmyo/sHbUNrThTzwx8oscfaZBqqfvjWWJxHzB2Kt9bCfYrEUdpPywF1IczhY31BFtB2PT70FiRc0aIBMmYI56yeeo92lsVVOFzaPw3EPuPt52SWqpTkCH6FGdCQzxMLzsZ1nhtdQ0FDklX89HWv2c0qhWK9MIgZMfOs67B8e3vtV2Eu+4AVQBIhqf1m8h8afe5MfHc+p2G79Th9vIFAc5V1LO41lDxYhGjLc1uguacErId0= ceph-24d05dee-ffba-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T08:37:35.735 INFO:teuthology.orchestra.run.smithi082.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkNmgVHbETqv/6Sm6ZaE235c5uRROeXx5CPqkKQw3LqoV/VPtZMLDNXzKEvNnlcszIi4Nu8DcTHTC6VkIgPq7E5Rs+wKjT2zgQfuYuDNi+SzfUaUuMesHcxZ103rDXlLrMSkHTIgrOemEGvcK0g019L8MGiY+ZCMI/KP/LXBAprKR+0M1f7Q1XjkA4ThhXViqo4n7TqJHi5GX9lIQuXShEblFb5Q2M3UT5VDJ4S6+C75HKCeJKhSnoqwZy/3onqLWdviPBee338x7Mz1SOpmyo/sHbUNrThTzwx8oscfaZBqqfvjWWJxHzB2Kt9bCfYrEUdpPywF1IczhY31BFtB2PT70FiRc0aIBMmYI56yeeo92lsVVOFzaPw3EPuPt52SWqpTkCH6FGdCQzxMLzsZ1nhtdQ0FDklX89HWv2c0qhWK9MIgZMfOs67B8e3vtV2Eu+4AVQBIhqf1m8h8afe5MfHc+p2G79Th9vIFAc5V1LO41lDxYhGjLc1uguacErId0= ceph-24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:37:35.746 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkNmgVHbETqv/6Sm6ZaE235c5uRROeXx5CPqkKQw3LqoV/VPtZMLDNXzKEvNnlcszIi4Nu8DcTHTC6VkIgPq7E5Rs+wKjT2zgQfuYuDNi+SzfUaUuMesHcxZ103rDXlLrMSkHTIgrOemEGvcK0g019L8MGiY+ZCMI/KP/LXBAprKR+0M1f7Q1XjkA4ThhXViqo4n7TqJHi5GX9lIQuXShEblFb5Q2M3UT5VDJ4S6+C75HKCeJKhSnoqwZy/3onqLWdviPBee338x7Mz1SOpmyo/sHbUNrThTzwx8oscfaZBqqfvjWWJxHzB2Kt9bCfYrEUdpPywF1IczhY31BFtB2PT70FiRc0aIBMmYI56yeeo92lsVVOFzaPw3EPuPt52SWqpTkCH6FGdCQzxMLzsZ1nhtdQ0FDklX89HWv2c0qhWK9MIgZMfOs67B8e3vtV2Eu+4AVQBIhqf1m8h8afe5MfHc+p2G79Th9vIFAc5V1LO41lDxYhGjLc1uguacErId0= ceph-24d05dee-ffba-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T08:37:35.785 INFO:teuthology.orchestra.run.smithi106.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkNmgVHbETqv/6Sm6ZaE235c5uRROeXx5CPqkKQw3LqoV/VPtZMLDNXzKEvNnlcszIi4Nu8DcTHTC6VkIgPq7E5Rs+wKjT2zgQfuYuDNi+SzfUaUuMesHcxZ103rDXlLrMSkHTIgrOemEGvcK0g019L8MGiY+ZCMI/KP/LXBAprKR+0M1f7Q1XjkA4ThhXViqo4n7TqJHi5GX9lIQuXShEblFb5Q2M3UT5VDJ4S6+C75HKCeJKhSnoqwZy/3onqLWdviPBee338x7Mz1SOpmyo/sHbUNrThTzwx8oscfaZBqqfvjWWJxHzB2Kt9bCfYrEUdpPywF1IczhY31BFtB2PT70FiRc0aIBMmYI56yeeo92lsVVOFzaPw3EPuPt52SWqpTkCH6FGdCQzxMLzsZ1nhtdQ0FDklX89HWv2c0qhWK9MIgZMfOs67B8e3vtV2Eu+4AVQBIhqf1m8h8afe5MfHc+p2G79Th9vIFAc5V1LO41lDxYhGjLc1uguacErId0= ceph-24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:37:35.797 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkNmgVHbETqv/6Sm6ZaE235c5uRROeXx5CPqkKQw3LqoV/VPtZMLDNXzKEvNnlcszIi4Nu8DcTHTC6VkIgPq7E5Rs+wKjT2zgQfuYuDNi+SzfUaUuMesHcxZ103rDXlLrMSkHTIgrOemEGvcK0g019L8MGiY+ZCMI/KP/LXBAprKR+0M1f7Q1XjkA4ThhXViqo4n7TqJHi5GX9lIQuXShEblFb5Q2M3UT5VDJ4S6+C75HKCeJKhSnoqwZy/3onqLWdviPBee338x7Mz1SOpmyo/sHbUNrThTzwx8oscfaZBqqfvjWWJxHzB2Kt9bCfYrEUdpPywF1IczhY31BFtB2PT70FiRc0aIBMmYI56yeeo92lsVVOFzaPw3EPuPt52SWqpTkCH6FGdCQzxMLzsZ1nhtdQ0FDklX89HWv2c0qhWK9MIgZMfOs67B8e3vtV2Eu+4AVQBIhqf1m8h8afe5MfHc+p2G79Th9vIFAc5V1LO41lDxYhGjLc1uguacErId0= ceph-24d05dee-ffba-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T08:37:35.835 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkNmgVHbETqv/6Sm6ZaE235c5uRROeXx5CPqkKQw3LqoV/VPtZMLDNXzKEvNnlcszIi4Nu8DcTHTC6VkIgPq7E5Rs+wKjT2zgQfuYuDNi+SzfUaUuMesHcxZ103rDXlLrMSkHTIgrOemEGvcK0g019L8MGiY+ZCMI/KP/LXBAprKR+0M1f7Q1XjkA4ThhXViqo4n7TqJHi5GX9lIQuXShEblFb5Q2M3UT5VDJ4S6+C75HKCeJKhSnoqwZy/3onqLWdviPBee338x7Mz1SOpmyo/sHbUNrThTzwx8oscfaZBqqfvjWWJxHzB2Kt9bCfYrEUdpPywF1IczhY31BFtB2PT70FiRc0aIBMmYI56yeeo92lsVVOFzaPw3EPuPt52SWqpTkCH6FGdCQzxMLzsZ1nhtdQ0FDklX89HWv2c0qhWK9MIgZMfOs67B8e3vtV2Eu+4AVQBIhqf1m8h8afe5MfHc+p2G79Th9vIFAc5V1LO41lDxYhGjLc1uguacErId0= ceph-24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:37:35.847 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-21T08:37:36.081 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:36.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:35 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3795281490' entity='client.admin' 2024-04-21T08:37:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.845+0000 7f178874b700 1 -- 172.21.15.82:0/3928248112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780100770 msgr2=0x7f1780100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.845+0000 7f178874b700 1 --2- 172.21.15.82:0/3928248112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780100770 0x7f1780100b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f177c009910 tx=0x7f177c009c20 comp rx=0 tx=0).stop 2024-04-21T08:37:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.846+0000 7f178874b700 1 -- 172.21.15.82:0/3928248112 shutdown_connections 2024-04-21T08:37:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.846+0000 7f178874b700 1 --2- 172.21.15.82:0/3928248112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780100770 0x7f1780100b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.846+0000 7f178874b700 1 -- 172.21.15.82:0/3928248112 >> 172.21.15.82:0/3928248112 conn(0x7f17800fc0f0 msgr2=0x7f17800fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.846+0000 7f178874b700 1 -- 172.21.15.82:0/3928248112 shutdown_connections 2024-04-21T08:37:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.846+0000 7f178874b700 1 -- 172.21.15.82:0/3928248112 wait complete. 2024-04-21T08:37:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.846+0000 7f178874b700 1 Processor -- start 2024-04-21T08:37:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.846+0000 7f178874b700 1 -- start start 2024-04-21T08:37:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.847+0000 7f178874b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780100770 0x7f1780197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.847+0000 7f178874b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f17801083f0 con 0x7f1780100770 2024-04-21T08:37:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.847+0000 7f17864e7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780100770 0x7f1780197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.847+0000 7f17864e7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780100770 0x7f1780197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34922/0 (socket says 172.21.15.82:34922) 2024-04-21T08:37:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.847+0000 7f17864e7700 1 -- 172.21.15.82:0/2234572102 learned_addr learned my addr 172.21.15.82:0/2234572102 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.847+0000 7f17864e7700 1 -- 172.21.15.82:0/2234572102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1780198040 con 0x7f1780100770 2024-04-21T08:37:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.847+0000 7f17864e7700 1 --2- 172.21.15.82:0/2234572102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780100770 0x7f1780197b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f177c005800 tx=0x7f177c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.847+0000 7f17777fe700 1 -- 172.21.15.82:0/2234572102 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177c027070 con 0x7f1780100770 2024-04-21T08:37:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.847+0000 7f178874b700 1 -- 172.21.15.82:0/2234572102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17801982d0 con 0x7f1780100770 2024-04-21T08:37:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.847+0000 7f17777fe700 1 -- 172.21.15.82:0/2234572102 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f177c00be10 con 0x7f1780100770 2024-04-21T08:37:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.847+0000 7f17777fe700 1 -- 172.21.15.82:0/2234572102 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f177c00f460 con 0x7f1780100770 2024-04-21T08:37:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.847+0000 7f178874b700 1 -- 172.21.15.82:0/2234572102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f178019afc0 con 0x7f1780100770 2024-04-21T08:37:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.848+0000 7f17777fe700 1 -- 172.21.15.82:0/2234572102 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f177c02a070 con 0x7f1780100770 2024-04-21T08:37:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.848+0000 7f17777fe700 1 --2- 172.21.15.82:0/2234572102 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f176c0396b0 0x7f176c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.848+0000 7f17777fe700 1 -- 172.21.15.82:0/2234572102 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f177c01e030 con 0x7f1780100770 2024-04-21T08:37:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.849+0000 7f178874b700 1 -- 172.21.15.82:0/2234572102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1780108b70 con 0x7f1780100770 2024-04-21T08:37:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.849+0000 7f1785ce6700 1 --2- 172.21.15.82:0/2234572102 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f176c0396b0 0x7f176c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.849+0000 7f1785ce6700 1 --2- 172.21.15.82:0/2234572102 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f176c0396b0 0x7f176c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1770006fd0 tx=0x7f1770006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:38.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:38.854+0000 7f17777fe700 1 -- 172.21.15.82:0/2234572102 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f177c031070 con 0x7f1780100770 2024-04-21T08:37:39.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:39.082+0000 7f178874b700 1 -- 172.21.15.82:0/2234572102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f178002e020 con 0x7f1780100770 2024-04-21T08:37:39.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:39.084+0000 7f17777fe700 1 -- 172.21.15.82:0/2234572102 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f177c013070 con 0x7f1780100770 2024-04-21T08:37:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:39.087+0000 7f178874b700 1 -- 172.21.15.82:0/2234572102 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f176c0396b0 msgr2=0x7f176c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:39.087+0000 7f178874b700 1 --2- 172.21.15.82:0/2234572102 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f176c0396b0 0x7f176c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1770006fd0 tx=0x7f1770006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:39.087+0000 7f178874b700 1 -- 172.21.15.82:0/2234572102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780100770 msgr2=0x7f1780197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:39.087+0000 7f178874b700 1 --2- 172.21.15.82:0/2234572102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780100770 0x7f1780197b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f177c005800 tx=0x7f177c009f60 comp rx=0 tx=0).stop 2024-04-21T08:37:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:39.087+0000 7f178874b700 1 -- 172.21.15.82:0/2234572102 shutdown_connections 2024-04-21T08:37:39.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:39.087+0000 7f178874b700 1 --2- 172.21.15.82:0/2234572102 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f176c0396b0 0x7f176c03bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:39.087+0000 7f178874b700 1 --2- 172.21.15.82:0/2234572102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1780100770 0x7f1780197b00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:39.087+0000 7f178874b700 1 -- 172.21.15.82:0/2234572102 >> 172.21.15.82:0/2234572102 conn(0x7f17800fc0f0 msgr2=0x7f17800fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:39.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:39.087+0000 7f178874b700 1 -- 172.21.15.82:0/2234572102 shutdown_connections 2024-04-21T08:37:39.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:39.087+0000 7f178874b700 1 -- 172.21.15.82:0/2234572102 wait complete. 2024-04-21T08:37:40.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:40 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2234572102' entity='client.admin' 2024-04-21T08:37:40.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:40.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:40.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:37:40.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:40.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:40.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:40.290 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-21T08:37:40.290 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-21T08:37:40.523 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:42.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.107+0000 7f95aad40700 1 -- 172.21.15.82:0/3414975898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95a4103bb0 msgr2=0x7f95a4103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:42.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.107+0000 7f95aad40700 1 --2- 172.21.15.82:0/3414975898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95a4103bb0 0x7f95a4103fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9594009910 tx=0x7f9594009c20 comp rx=0 tx=0).stop 2024-04-21T08:37:42.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.107+0000 7f95aad40700 1 -- 172.21.15.82:0/3414975898 shutdown_connections 2024-04-21T08:37:42.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.107+0000 7f95aad40700 1 --2- 172.21.15.82:0/3414975898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95a4103bb0 0x7f95a4103fd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.107+0000 7f95aad40700 1 -- 172.21.15.82:0/3414975898 >> 172.21.15.82:0/3414975898 conn(0x7f95a40fda90 msgr2=0x7f95a40ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:42.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.107+0000 7f95aad40700 1 -- 172.21.15.82:0/3414975898 shutdown_connections 2024-04-21T08:37:42.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.107+0000 7f95aad40700 1 -- 172.21.15.82:0/3414975898 wait complete. 2024-04-21T08:37:42.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.107+0000 7f95aad40700 1 Processor -- start 2024-04-21T08:37:42.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.108+0000 7f95aad40700 1 -- start start 2024-04-21T08:37:42.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.108+0000 7f95aad40700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95a4103bb0 0x7f95a4197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:42.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.108+0000 7f95aad40700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4108520 con 0x7f95a4103bb0 2024-04-21T08:37:42.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.108+0000 7f95a8adc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95a4103bb0 0x7f95a4197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:42.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.108+0000 7f95a8adc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95a4103bb0 0x7f95a4197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34944/0 (socket says 172.21.15.82:34944) 2024-04-21T08:37:42.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.108+0000 7f95a8adc700 1 -- 172.21.15.82:0/826575879 learned_addr learned my addr 172.21.15.82:0/826575879 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:42.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.108+0000 7f95a8adc700 1 -- 172.21.15.82:0/826575879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a4198150 con 0x7f95a4103bb0 2024-04-21T08:37:42.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.108+0000 7f95a8adc700 1 --2- 172.21.15.82:0/826575879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95a4103bb0 0x7f95a4197c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9594005800 tx=0x7f9594009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:42.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.108+0000 7f95a1ffb700 1 -- 172.21.15.82:0/826575879 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9594027070 con 0x7f95a4103bb0 2024-04-21T08:37:42.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.108+0000 7f95a1ffb700 1 -- 172.21.15.82:0/826575879 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959400be10 con 0x7f95a4103bb0 2024-04-21T08:37:42.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.108+0000 7f95a1ffb700 1 -- 172.21.15.82:0/826575879 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f959400f460 con 0x7f95a4103bb0 2024-04-21T08:37:42.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.108+0000 7f95aad40700 1 -- 172.21.15.82:0/826575879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a41983e0 con 0x7f95a4103bb0 2024-04-21T08:37:42.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.109+0000 7f95aad40700 1 -- 172.21.15.82:0/826575879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a419b0d0 con 0x7f95a4103bb0 2024-04-21T08:37:42.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.109+0000 7f95a1ffb700 1 -- 172.21.15.82:0/826575879 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f959402a070 con 0x7f95a4103bb0 2024-04-21T08:37:42.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.110+0000 7f95a1ffb700 1 --2- 172.21.15.82:0/826575879 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f958c0396b0 0x7f958c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:42.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.110+0000 7f95aad40700 1 -- 172.21.15.82:0/826575879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a4108ca0 con 0x7f95a4103bb0 2024-04-21T08:37:42.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.110+0000 7f95a1ffb700 1 -- 172.21.15.82:0/826575879 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f959401e030 con 0x7f95a4103bb0 2024-04-21T08:37:42.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.110+0000 7f95a3fff700 1 --2- 172.21.15.82:0/826575879 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f958c0396b0 0x7f958c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:42.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.110+0000 7f95a3fff700 1 --2- 172.21.15.82:0/826575879 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f958c0396b0 0x7f958c03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9598006fd0 tx=0x7f9598006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:42.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.115+0000 7f95a1ffb700 1 -- 172.21.15.82:0/826575879 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9594013070 con 0x7f95a4103bb0 2024-04-21T08:37:42.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.317+0000 7f95aad40700 1 -- 172.21.15.82:0/826575879 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f95a4062e80 con 0x7f958c0396b0 2024-04-21T08:37:42.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.321+0000 7f95a1ffb700 1 -- 172.21.15.82:0/826575879 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f95a4062e80 con 0x7f958c0396b0 2024-04-21T08:37:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.324+0000 7f95aad40700 1 -- 172.21.15.82:0/826575879 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f958c0396b0 msgr2=0x7f958c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.324+0000 7f95aad40700 1 --2- 172.21.15.82:0/826575879 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f958c0396b0 0x7f958c03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9598006fd0 tx=0x7f9598006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.324+0000 7f95aad40700 1 -- 172.21.15.82:0/826575879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95a4103bb0 msgr2=0x7f95a4197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.324+0000 7f95aad40700 1 --2- 172.21.15.82:0/826575879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95a4103bb0 0x7f95a4197c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9594005800 tx=0x7f9594009f60 comp rx=0 tx=0).stop 2024-04-21T08:37:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.324+0000 7f95aad40700 1 -- 172.21.15.82:0/826575879 shutdown_connections 2024-04-21T08:37:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.324+0000 7f95aad40700 1 --2- 172.21.15.82:0/826575879 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f958c0396b0 0x7f958c03bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.324+0000 7f95aad40700 1 --2- 172.21.15.82:0/826575879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95a4103bb0 0x7f95a4197c10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.324+0000 7f95aad40700 1 -- 172.21.15.82:0/826575879 >> 172.21.15.82:0/826575879 conn(0x7f95a40fda90 msgr2=0x7f95a40fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.324+0000 7f95aad40700 1 -- 172.21.15.82:0/826575879 shutdown_connections 2024-04-21T08:37:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:42.324+0000 7f95aad40700 1 -- 172.21.15.82:0/826575879 wait complete. 2024-04-21T08:37:43.139 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi106 2024-04-21T08:37:43.139 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-21T08:37:43.139 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/etc/ceph/ceph.conf 2024-04-21T08:37:43.158 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-21T08:37:43.158 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:37:43.216 INFO:tasks.cephadm:Adding host smithi106 to orchestrator... 2024-04-21T08:37:43.216 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch host add smithi106 2024-04-21T08:37:43.450 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:43.572 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:43 smithi082 ceph-mon[27967]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:37:43.572 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:43 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:43.573 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:43 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:43.573 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:43 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:37:43.573 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:43 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:37:43.573 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:43 smithi082 ceph-mon[27967]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-21T08:37:43.573 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:43 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:43.573 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:43 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:43.573 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:43 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:44 smithi082 ceph-mon[27967]: Updating smithi082:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:37:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:44 smithi082 ceph-mon[27967]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:37:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:44 smithi082 ceph-mon[27967]: Updating smithi082:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:37:45.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.068+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/3027694602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d41015b0 msgr2=0x7fb6d4101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.068+0000 7fb6d9b7a700 1 --2- 172.21.15.82:0/3027694602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d41015b0 0x7fb6d4101990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400b0a0 tx=0x7fb6c400b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.068+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/3027694602 shutdown_connections 2024-04-21T08:37:45.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.068+0000 7fb6d9b7a700 1 --2- 172.21.15.82:0/3027694602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d41015b0 0x7fb6d4101990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.068+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/3027694602 >> 172.21.15.82:0/3027694602 conn(0x7fb6d40fce40 msgr2=0x7fb6d40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:45.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.068+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/3027694602 shutdown_connections 2024-04-21T08:37:45.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.068+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/3027694602 wait complete. 2024-04-21T08:37:45.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.069+0000 7fb6d9b7a700 1 Processor -- start 2024-04-21T08:37:45.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.069+0000 7fb6d9b7a700 1 -- start start 2024-04-21T08:37:45.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.069+0000 7fb6d9b7a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d41015b0 0x7fb6d4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.069+0000 7fb6d9b7a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d410b830 con 0x7fb6d41015b0 2024-04-21T08:37:45.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.069+0000 7fb6d37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d41015b0 0x7fb6d4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.069+0000 7fb6d37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d41015b0 0x7fb6d4198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57220/0 (socket says 172.21.15.82:57220) 2024-04-21T08:37:45.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.069+0000 7fb6d37fe700 1 -- 172.21.15.82:0/4263951038 learned_addr learned my addr 172.21.15.82:0/4263951038 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:45.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.070+0000 7fb6d37fe700 1 -- 172.21.15.82:0/4263951038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d41991a0 con 0x7fb6d41015b0 2024-04-21T08:37:45.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.070+0000 7fb6d37fe700 1 --2- 172.21.15.82:0/4263951038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d41015b0 0x7fb6d4198c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4006130 tx=0x7fb6c40097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:45.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.070+0000 7fb6d0ff9700 1 -- 172.21.15.82:0/4263951038 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6c4021020 con 0x7fb6d41015b0 2024-04-21T08:37:45.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.070+0000 7fb6d0ff9700 1 -- 172.21.15.82:0/4263951038 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6c40079b0 con 0x7fb6d41015b0 2024-04-21T08:37:45.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.070+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/4263951038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d4199430 con 0x7fb6d41015b0 2024-04-21T08:37:45.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.070+0000 7fb6d0ff9700 1 -- 172.21.15.82:0/4263951038 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6c4017040 con 0x7fb6d41015b0 2024-04-21T08:37:45.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.070+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/4263951038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d41998f0 con 0x7fb6d41015b0 2024-04-21T08:37:45.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.071+0000 7fb6d0ff9700 1 -- 172.21.15.82:0/4263951038 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb6c4024020 con 0x7fb6d41015b0 2024-04-21T08:37:45.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.071+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/4263951038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d4102a90 con 0x7fb6d41015b0 2024-04-21T08:37:45.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.071+0000 7fb6d0ff9700 1 --2- 172.21.15.82:0/4263951038 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb6bc0396b0 0x7fb6bc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.071+0000 7fb6d0ff9700 1 -- 172.21.15.82:0/4263951038 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb6c400f050 con 0x7fb6d41015b0 2024-04-21T08:37:45.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.071+0000 7fb6d2ffd700 1 --2- 172.21.15.82:0/4263951038 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb6bc0396b0 0x7fb6bc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.071+0000 7fb6d2ffd700 1 --2- 172.21.15.82:0/4263951038 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb6bc0396b0 0x7fb6bc03bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8006fd0 tx=0x7fb6c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:45.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.076+0000 7fb6d0ff9700 1 -- 172.21.15.82:0/4263951038 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6c402b020 con 0x7fb6d41015b0 2024-04-21T08:37:45.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:45.277+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/4263951038 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}) -- 0x7fb6d40640a0 con 0x7fb6bc0396b0 2024-04-21T08:37:45.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:45 smithi082 ceph-mon[27967]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:37:47.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:47 smithi082 ceph-mon[27967]: Deploying cephadm binary to smithi106 2024-04-21T08:37:47.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:47 smithi082 ceph-mon[27967]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:37:49.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:49.365+0000 7fb6d0ff9700 1 -- 172.21.15.82:0/4263951038 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb6d40640a0 con 0x7fb6bc0396b0 2024-04-21T08:37:49.365 INFO:teuthology.orchestra.run.smithi082.stdout:Added host 'smithi106' with addr '172.21.15.106' 2024-04-21T08:37:49.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:49.369+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/4263951038 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb6bc0396b0 msgr2=0x7fb6bc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:49.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:49.369+0000 7fb6d9b7a700 1 --2- 172.21.15.82:0/4263951038 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb6bc0396b0 0x7fb6bc03bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8006fd0 tx=0x7fb6c8006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:49.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:49.369+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/4263951038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d41015b0 msgr2=0x7fb6d4198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:49.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:49.369+0000 7fb6d9b7a700 1 --2- 172.21.15.82:0/4263951038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d41015b0 0x7fb6d4198c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4006130 tx=0x7fb6c40097d0 comp rx=0 tx=0).stop 2024-04-21T08:37:49.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:49.369+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/4263951038 shutdown_connections 2024-04-21T08:37:49.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:49.369+0000 7fb6d9b7a700 1 --2- 172.21.15.82:0/4263951038 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb6bc0396b0 0x7fb6bc03bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:49.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:49.369+0000 7fb6d9b7a700 1 --2- 172.21.15.82:0/4263951038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d41015b0 0x7fb6d4198c60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:49.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:49.369+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/4263951038 >> 172.21.15.82:0/4263951038 conn(0x7fb6d40fce40 msgr2=0x7fb6d4102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:49.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:49.369+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/4263951038 shutdown_connections 2024-04-21T08:37:49.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:49.369+0000 7fb6d9b7a700 1 -- 172.21.15.82:0/4263951038 wait complete. 2024-04-21T08:37:49.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:49 smithi082 ceph-mon[27967]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:37:49.966 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-21T08:37:50.203 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:50.591 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:50.591 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:50 smithi082 ceph-mon[27967]: Added host smithi106 2024-04-21T08:37:50.591 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:50.591 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:51.573 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:51 smithi082 ceph-mon[27967]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:37:51.573 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:51 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:51.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.800+0000 7eff5542a700 1 -- 172.21.15.82:0/800491234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff501015f0 msgr2=0x7eff501019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:51.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.800+0000 7eff5542a700 1 --2- 172.21.15.82:0/800491234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff501015f0 0x7eff501019d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff4000b0a0 tx=0x7eff4000b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.800+0000 7eff5542a700 1 -- 172.21.15.82:0/800491234 shutdown_connections 2024-04-21T08:37:51.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.800+0000 7eff5542a700 1 --2- 172.21.15.82:0/800491234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff501015f0 0x7eff501019d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.800+0000 7eff5542a700 1 -- 172.21.15.82:0/800491234 >> 172.21.15.82:0/800491234 conn(0x7eff50076ea0 msgr2=0x7eff500772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:51.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.800+0000 7eff5542a700 1 -- 172.21.15.82:0/800491234 shutdown_connections 2024-04-21T08:37:51.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.800+0000 7eff5542a700 1 -- 172.21.15.82:0/800491234 wait complete. 2024-04-21T08:37:51.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.801+0000 7eff5542a700 1 Processor -- start 2024-04-21T08:37:51.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.801+0000 7eff5542a700 1 -- start start 2024-04-21T08:37:51.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.801+0000 7eff5542a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff501015f0 0x7eff50197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:51.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.801+0000 7eff5542a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7eff5010b820 con 0x7eff501015f0 2024-04-21T08:37:51.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.801+0000 7eff4effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff501015f0 0x7eff50197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:51.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.801+0000 7eff4effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff501015f0 0x7eff50197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57250/0 (socket says 172.21.15.82:57250) 2024-04-21T08:37:51.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.801+0000 7eff4effd700 1 -- 172.21.15.82:0/1103922076 learned_addr learned my addr 172.21.15.82:0/1103922076 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:51.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.801+0000 7eff4effd700 1 -- 172.21.15.82:0/1103922076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff50198340 con 0x7eff501015f0 2024-04-21T08:37:51.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.801+0000 7eff4effd700 1 --2- 172.21.15.82:0/1103922076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff501015f0 0x7eff50197e00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff40006130 tx=0x7eff400097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:51.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.802+0000 7eff47fff700 1 -- 172.21.15.82:0/1103922076 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff40021020 con 0x7eff501015f0 2024-04-21T08:37:51.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.802+0000 7eff47fff700 1 -- 172.21.15.82:0/1103922076 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff400079b0 con 0x7eff501015f0 2024-04-21T08:37:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.802+0000 7eff5542a700 1 -- 172.21.15.82:0/1103922076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff501985d0 con 0x7eff501015f0 2024-04-21T08:37:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.802+0000 7eff5542a700 1 -- 172.21.15.82:0/1103922076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5019c6d0 con 0x7eff501015f0 2024-04-21T08:37:51.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.802+0000 7eff47fff700 1 -- 172.21.15.82:0/1103922076 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff40017040 con 0x7eff501015f0 2024-04-21T08:37:51.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.803+0000 7eff47fff700 1 -- 172.21.15.82:0/1103922076 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7eff40024020 con 0x7eff501015f0 2024-04-21T08:37:51.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.803+0000 7eff5542a700 1 -- 172.21.15.82:0/1103922076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff50102a80 con 0x7eff501015f0 2024-04-21T08:37:51.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.803+0000 7eff47fff700 1 --2- 172.21.15.82:0/1103922076 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7eff3c039650 0x7eff3c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:51.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.803+0000 7eff47fff700 1 -- 172.21.15.82:0/1103922076 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7eff4000f050 con 0x7eff501015f0 2024-04-21T08:37:51.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.803+0000 7eff4e7fc700 1 --2- 172.21.15.82:0/1103922076 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7eff3c039650 0x7eff3c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:51.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.803+0000 7eff4e7fc700 1 --2- 172.21.15.82:0/1103922076 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7eff3c039650 0x7eff3c03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7eff38006fd0 tx=0x7eff38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:51.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:51.808+0000 7eff47fff700 1 -- 172.21.15.82:0/1103922076 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff4002b020 con 0x7eff501015f0 2024-04-21T08:37:52.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:52.014+0000 7eff5542a700 1 -- 172.21.15.82:0/1103922076 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff50199160 con 0x7eff3c039650 2024-04-21T08:37:52.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:52.015+0000 7eff47fff700 1 -- 172.21.15.82:0/1103922076 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7eff50199160 con 0x7eff3c039650 2024-04-21T08:37:52.015 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:37:52.015 INFO:teuthology.orchestra.run.smithi082.stdout:[{"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}] 2024-04-21T08:37:52.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:52.017+0000 7eff5542a700 1 -- 172.21.15.82:0/1103922076 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7eff3c039650 msgr2=0x7eff3c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:52.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:52.017+0000 7eff5542a700 1 --2- 172.21.15.82:0/1103922076 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7eff3c039650 0x7eff3c03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7eff38006fd0 tx=0x7eff38006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:52.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:52.017+0000 7eff5542a700 1 -- 172.21.15.82:0/1103922076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff501015f0 msgr2=0x7eff50197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:52.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:52.017+0000 7eff5542a700 1 --2- 172.21.15.82:0/1103922076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff501015f0 0x7eff50197e00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff40006130 tx=0x7eff400097d0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:52.017+0000 7eff5542a700 1 -- 172.21.15.82:0/1103922076 shutdown_connections 2024-04-21T08:37:52.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:52.017+0000 7eff5542a700 1 --2- 172.21.15.82:0/1103922076 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7eff3c039650 0x7eff3c03bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:52.017+0000 7eff5542a700 1 --2- 172.21.15.82:0/1103922076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff501015f0 0x7eff50197e00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:52.017+0000 7eff5542a700 1 -- 172.21.15.82:0/1103922076 >> 172.21.15.82:0/1103922076 conn(0x7eff50076ea0 msgr2=0x7eff50069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:52.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:52.018+0000 7eff5542a700 1 -- 172.21.15.82:0/1103922076 shutdown_connections 2024-04-21T08:37:52.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:52.018+0000 7eff5542a700 1 -- 172.21.15.82:0/1103922076 wait complete. 2024-04-21T08:37:52.619 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2024-04-21T08:37:52.619 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-21T08:37:52.619 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2024-04-21T08:37:52.637 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-21T08:37:52.638 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:37:52.696 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2024-04-21T08:37:52.696 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch host add smithi134 2024-04-21T08:37:52.929 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:53.286 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:53.286 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:52 smithi082 ceph-mon[27967]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:37:53.286 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:52 smithi082 ceph-mon[27967]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:37:54.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.449+0000 7f5ced394700 1 -- 172.21.15.82:0/3779858629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce81015b0 msgr2=0x7f5ce8101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:54.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.449+0000 7f5ced394700 1 --2- 172.21.15.82:0/3779858629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce81015b0 0x7f5ce8101990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009910 tx=0x7f5cdc009c20 comp rx=0 tx=0).stop 2024-04-21T08:37:54.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.449+0000 7f5ced394700 1 -- 172.21.15.82:0/3779858629 shutdown_connections 2024-04-21T08:37:54.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.449+0000 7f5ced394700 1 --2- 172.21.15.82:0/3779858629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce81015b0 0x7f5ce8101990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.449+0000 7f5ced394700 1 -- 172.21.15.82:0/3779858629 >> 172.21.15.82:0/3779858629 conn(0x7f5ce80fce40 msgr2=0x7f5ce80ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.449+0000 7f5ced394700 1 -- 172.21.15.82:0/3779858629 shutdown_connections 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.449+0000 7f5ced394700 1 -- 172.21.15.82:0/3779858629 wait complete. 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.450+0000 7f5ced394700 1 Processor -- start 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.450+0000 7f5ced394700 1 -- start start 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.450+0000 7f5ced394700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce81015b0 0x7f5ce8198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.450+0000 7f5ced394700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce810b830 con 0x7f5ce81015b0 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.450+0000 7f5ce6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce81015b0 0x7f5ce8198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.450+0000 7f5ce6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce81015b0 0x7f5ce8198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51186/0 (socket says 172.21.15.82:51186) 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.450+0000 7f5ce6ffd700 1 -- 172.21.15.82:0/4140314783 learned_addr learned my addr 172.21.15.82:0/4140314783 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.450+0000 7f5ce6ffd700 1 -- 172.21.15.82:0/4140314783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce8199210 con 0x7f5ce81015b0 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.450+0000 7f5ce6ffd700 1 --2- 172.21.15.82:0/4140314783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce81015b0 0x7f5ce8198cd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0056d0 tx=0x7f5cdc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:54.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.451+0000 7f5ccffff700 1 -- 172.21.15.82:0/4140314783 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cdc026070 con 0x7f5ce81015b0 2024-04-21T08:37:54.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.451+0000 7f5ccffff700 1 -- 172.21.15.82:0/4140314783 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc00bba0 con 0x7f5ce81015b0 2024-04-21T08:37:54.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.451+0000 7f5ccffff700 1 -- 172.21.15.82:0/4140314783 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cdc003920 con 0x7f5ce81015b0 2024-04-21T08:37:54.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.451+0000 7f5ced394700 1 -- 172.21.15.82:0/4140314783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce8199500 con 0x7f5ce81015b0 2024-04-21T08:37:54.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.451+0000 7f5ced394700 1 -- 172.21.15.82:0/4140314783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce81999c0 con 0x7f5ce81015b0 2024-04-21T08:37:54.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.452+0000 7f5ccffff700 1 -- 172.21.15.82:0/4140314783 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5cdc029070 con 0x7f5ce81015b0 2024-04-21T08:37:54.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.452+0000 7f5ced394700 1 -- 172.21.15.82:0/4140314783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce8102a90 con 0x7f5ce81015b0 2024-04-21T08:37:54.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.452+0000 7f5ccffff700 1 --2- 172.21.15.82:0/4140314783 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5cd00396b0 0x7f5cd003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:54.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.452+0000 7f5ccffff700 1 -- 172.21.15.82:0/4140314783 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5cdc01e070 con 0x7f5ce81015b0 2024-04-21T08:37:54.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.452+0000 7f5ce67fc700 1 --2- 172.21.15.82:0/4140314783 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5cd00396b0 0x7f5cd003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:54.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.452+0000 7f5ce67fc700 1 --2- 172.21.15.82:0/4140314783 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5cd00396b0 0x7f5cd003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8006fd0 tx=0x7f5cd8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.457+0000 7f5ccffff700 1 -- 172.21.15.82:0/4140314783 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5cdc013070 con 0x7f5ce81015b0 2024-04-21T08:37:54.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:54.657+0000 7f5ced394700 1 -- 172.21.15.82:0/4140314783 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) -- 0x7f5ce806ae60 con 0x7f5cd00396b0 2024-04-21T08:37:55.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:55 smithi082 ceph-mon[27967]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:37:55.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:55 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:55.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:55 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:55.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:55 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:55.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:55 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:55.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:55 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi106", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:37:55.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:55 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:37:55.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:55 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:37:56.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:56 smithi082 ceph-mon[27967]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:37:56.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:56 smithi082 ceph-mon[27967]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-21T08:37:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:56 smithi082 ceph-mon[27967]: Deploying cephadm binary to smithi134 2024-04-21T08:37:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:56 smithi082 ceph-mon[27967]: Updating smithi106:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:37:57.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:57 smithi082 ceph-mon[27967]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:37:57.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:57 smithi082 ceph-mon[27967]: Updating smithi106:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:37:57.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:57 smithi082 ceph-mon[27967]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:37:58.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:58.765+0000 7f5ccffff700 1 -- 172.21.15.82:0/4140314783 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5ce806ae60 con 0x7f5cd00396b0 2024-04-21T08:37:58.765 INFO:teuthology.orchestra.run.smithi082.stdout:Added host 'smithi134' with addr '172.21.15.134' 2024-04-21T08:37:58.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:58.767+0000 7f5ced394700 1 -- 172.21.15.82:0/4140314783 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5cd00396b0 msgr2=0x7f5cd003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:58.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:58.767+0000 7f5ced394700 1 --2- 172.21.15.82:0/4140314783 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5cd00396b0 0x7f5cd003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8006fd0 tx=0x7f5cd8006e40 comp rx=0 tx=0).stop 2024-04-21T08:37:58.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:58.767+0000 7f5ced394700 1 -- 172.21.15.82:0/4140314783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce81015b0 msgr2=0x7f5ce8198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:58.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:58.767+0000 7f5ced394700 1 --2- 172.21.15.82:0/4140314783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce81015b0 0x7f5ce8198cd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc0056d0 tx=0x7f5cdc0057b0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:58.767+0000 7f5ced394700 1 -- 172.21.15.82:0/4140314783 shutdown_connections 2024-04-21T08:37:58.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:58.767+0000 7f5ced394700 1 --2- 172.21.15.82:0/4140314783 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5cd00396b0 0x7f5cd003bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:58.767+0000 7f5ced394700 1 --2- 172.21.15.82:0/4140314783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ce81015b0 0x7f5ce8198cd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:58.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:58.767+0000 7f5ced394700 1 -- 172.21.15.82:0/4140314783 >> 172.21.15.82:0/4140314783 conn(0x7f5ce80fce40 msgr2=0x7f5ce8102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:58.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:58.768+0000 7f5ced394700 1 -- 172.21.15.82:0/4140314783 shutdown_connections 2024-04-21T08:37:58.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:37:58.768+0000 7f5ced394700 1 -- 172.21.15.82:0/4140314783 wait complete. 2024-04-21T08:37:59.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:59 smithi082 ceph-mon[27967]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:37:59.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:59 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:59.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:59 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:59.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:59 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:59.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:59 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:37:59.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:37:59 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:59.406 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-21T08:37:59.635 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:00 smithi082 ceph-mon[27967]: Added host smithi134 2024-04-21T08:38:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:00 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:00 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.146+0000 7f88e20b7700 1 -- 172.21.15.82:0/2967447429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88dc1084f0 msgr2=0x7f88dc1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.146+0000 7f88e20b7700 1 --2- 172.21.15.82:0/2967447429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88dc1084f0 0x7f88dc1088d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009910 tx=0x7f88cc009c20 comp rx=0 tx=0).stop 2024-04-21T08:38:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.146+0000 7f88e20b7700 1 -- 172.21.15.82:0/2967447429 shutdown_connections 2024-04-21T08:38:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.146+0000 7f88e20b7700 1 --2- 172.21.15.82:0/2967447429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88dc1084f0 0x7f88dc1088d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.146+0000 7f88e20b7700 1 -- 172.21.15.82:0/2967447429 >> 172.21.15.82:0/2967447429 conn(0x7f88dc0fdf70 msgr2=0x7f88dc100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.146+0000 7f88e20b7700 1 -- 172.21.15.82:0/2967447429 shutdown_connections 2024-04-21T08:38:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.146+0000 7f88e20b7700 1 -- 172.21.15.82:0/2967447429 wait complete. 2024-04-21T08:38:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.146+0000 7f88e20b7700 1 Processor -- start 2024-04-21T08:38:01.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.146+0000 7f88e20b7700 1 -- start start 2024-04-21T08:38:01.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.147+0000 7f88e20b7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88dc1084f0 0x7f88dc197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:01.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.147+0000 7f88e20b7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc10c950 con 0x7f88dc1084f0 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.147+0000 7f88db7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88dc1084f0 0x7f88dc197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.147+0000 7f88db7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88dc1084f0 0x7f88dc197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51206/0 (socket says 172.21.15.82:51206) 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.147+0000 7f88db7fe700 1 -- 172.21.15.82:0/978034585 learned_addr learned my addr 172.21.15.82:0/978034585 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.147+0000 7f88db7fe700 1 -- 172.21.15.82:0/978034585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc1983c0 con 0x7f88dc1084f0 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.147+0000 7f88db7fe700 1 --2- 172.21.15.82:0/978034585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88dc1084f0 0x7f88dc197e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0056d0 tx=0x7f88cc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.147+0000 7f88d97fa700 1 -- 172.21.15.82:0/978034585 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88cc026070 con 0x7f88dc1084f0 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.148+0000 7f88d97fa700 1 -- 172.21.15.82:0/978034585 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88cc00bba0 con 0x7f88dc1084f0 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.148+0000 7f88e20b7700 1 -- 172.21.15.82:0/978034585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc1986b0 con 0x7f88dc1084f0 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.148+0000 7f88d97fa700 1 -- 172.21.15.82:0/978034585 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88cc003920 con 0x7f88dc1084f0 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.148+0000 7f88e20b7700 1 -- 172.21.15.82:0/978034585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc19c750 con 0x7f88dc1084f0 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.149+0000 7f88d97fa700 1 -- 172.21.15.82:0/978034585 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f88cc029070 con 0x7f88dc1084f0 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.149+0000 7f88e20b7700 1 -- 172.21.15.82:0/978034585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88dc103ba0 con 0x7f88dc1084f0 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.149+0000 7f88d97fa700 1 --2- 172.21.15.82:0/978034585 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f88c8039650 0x7f88c803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:01.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.149+0000 7f88d97fa700 1 -- 172.21.15.82:0/978034585 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f88cc01e070 con 0x7f88dc1084f0 2024-04-21T08:38:01.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.149+0000 7f88d2ffd700 1 --2- 172.21.15.82:0/978034585 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f88c8039650 0x7f88c803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:01.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.149+0000 7f88d2ffd700 1 --2- 172.21.15.82:0/978034585 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f88c8039650 0x7f88c803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f88c4006fd0 tx=0x7f88c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:01.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.154+0000 7f88d97fa700 1 -- 172.21.15.82:0/978034585 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88cc013070 con 0x7f88dc1084f0 2024-04-21T08:38:01.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.358+0000 7f88e20b7700 1 -- 172.21.15.82:0/978034585 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f88dc1991e0 con 0x7f88c8039650 2024-04-21T08:38:01.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.360+0000 7f88d97fa700 1 -- 172.21.15.82:0/978034585 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f88dc1991e0 con 0x7f88c8039650 2024-04-21T08:38:01.359 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:38:01.360 INFO:teuthology.orchestra.run.smithi082.stdout:[{"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2024-04-21T08:38:01.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.362+0000 7f88e20b7700 1 -- 172.21.15.82:0/978034585 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f88c8039650 msgr2=0x7f88c803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.362+0000 7f88e20b7700 1 --2- 172.21.15.82:0/978034585 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f88c8039650 0x7f88c803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f88c4006fd0 tx=0x7f88c4006e40 comp rx=0 tx=0).stop 2024-04-21T08:38:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.362+0000 7f88e20b7700 1 -- 172.21.15.82:0/978034585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88dc1084f0 msgr2=0x7f88dc197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.362+0000 7f88e20b7700 1 --2- 172.21.15.82:0/978034585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88dc1084f0 0x7f88dc197e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0056d0 tx=0x7f88cc0057b0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.362+0000 7f88e20b7700 1 -- 172.21.15.82:0/978034585 shutdown_connections 2024-04-21T08:38:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.362+0000 7f88e20b7700 1 --2- 172.21.15.82:0/978034585 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f88c8039650 0x7f88c803bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.362+0000 7f88e20b7700 1 --2- 172.21.15.82:0/978034585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88dc1084f0 0x7f88dc197e80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.362+0000 7f88e20b7700 1 -- 172.21.15.82:0/978034585 >> 172.21.15.82:0/978034585 conn(0x7f88dc0fdf70 msgr2=0x7f88dc10a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.362+0000 7f88e20b7700 1 -- 172.21.15.82:0/978034585 shutdown_connections 2024-04-21T08:38:01.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:01.362+0000 7f88e20b7700 1 -- 172.21.15.82:0/978034585 wait complete. 2024-04-21T08:38:01.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:01 smithi082 ceph-mon[27967]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:01.933 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-21T08:38:01.933 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-21T08:38:02.162 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:02.559 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:02 smithi082 ceph-mon[27967]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:02.559 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:02 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:03.449 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:03 smithi082 ceph-mon[27967]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:03.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.712+0000 7f2c823fe700 1 -- 172.21.15.82:0/3612938541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c7c1085c0 msgr2=0x7f2c7c1089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.712+0000 7f2c823fe700 1 --2- 172.21.15.82:0/3612938541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c7c1085c0 0x7f2c7c1089a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009910 tx=0x7f2c6c009c20 comp rx=0 tx=0).stop 2024-04-21T08:38:03.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.713+0000 7f2c823fe700 1 -- 172.21.15.82:0/3612938541 shutdown_connections 2024-04-21T08:38:03.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.713+0000 7f2c823fe700 1 --2- 172.21.15.82:0/3612938541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c7c1085c0 0x7f2c7c1089a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.713+0000 7f2c823fe700 1 -- 172.21.15.82:0/3612938541 >> 172.21.15.82:0/3612938541 conn(0x7f2c7c0fc5f0 msgr2=0x7f2c7c0fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:03.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.713+0000 7f2c823fe700 1 -- 172.21.15.82:0/3612938541 shutdown_connections 2024-04-21T08:38:03.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.713+0000 7f2c823fe700 1 -- 172.21.15.82:0/3612938541 wait complete. 2024-04-21T08:38:03.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.713+0000 7f2c823fe700 1 Processor -- start 2024-04-21T08:38:03.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.713+0000 7f2c823fe700 1 -- start start 2024-04-21T08:38:03.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.714+0000 7f2c823fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c7c1085c0 0x7f2c7c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.714+0000 7f2c823fe700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c10c9a0 con 0x7f2c7c1085c0 2024-04-21T08:38:03.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.714+0000 7f2c7bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c7c1085c0 0x7f2c7c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.714+0000 7f2c7bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c7c1085c0 0x7f2c7c197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59124/0 (socket says 172.21.15.82:59124) 2024-04-21T08:38:03.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.714+0000 7f2c7bfff700 1 -- 172.21.15.82:0/592478716 learned_addr learned my addr 172.21.15.82:0/592478716 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:38:03.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.714+0000 7f2c7bfff700 1 -- 172.21.15.82:0/592478716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c1983c0 con 0x7f2c7c1085c0 2024-04-21T08:38:03.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.714+0000 7f2c7bfff700 1 --2- 172.21.15.82:0/592478716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c7c1085c0 0x7f2c7c197e80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0056d0 tx=0x7f2c6c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:03.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.714+0000 7f2c797fa700 1 -- 172.21.15.82:0/592478716 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c6c027070 con 0x7f2c7c1085c0 2024-04-21T08:38:03.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.714+0000 7f2c797fa700 1 -- 172.21.15.82:0/592478716 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c6c00be10 con 0x7f2c7c1085c0 2024-04-21T08:38:03.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.714+0000 7f2c823fe700 1 -- 172.21.15.82:0/592478716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7c198650 con 0x7f2c7c1085c0 2024-04-21T08:38:03.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.714+0000 7f2c797fa700 1 -- 172.21.15.82:0/592478716 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c6c022d20 con 0x7f2c7c1085c0 2024-04-21T08:38:03.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.714+0000 7f2c823fe700 1 -- 172.21.15.82:0/592478716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7c19c750 con 0x7f2c7c1085c0 2024-04-21T08:38:03.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.715+0000 7f2c797fa700 1 -- 172.21.15.82:0/592478716 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2c6c02a070 con 0x7f2c7c1085c0 2024-04-21T08:38:03.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.715+0000 7f2c797fa700 1 --2- 172.21.15.82:0/592478716 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2c64039650 0x7f2c6403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.715+0000 7f2c797fa700 1 -- 172.21.15.82:0/592478716 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2c6c01e030 con 0x7f2c7c1085c0 2024-04-21T08:38:03.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.715+0000 7f2c7b7fe700 1 --2- 172.21.15.82:0/592478716 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2c64039650 0x7f2c6403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.715+0000 7f2c823fe700 1 -- 172.21.15.82:0/592478716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7c103bd0 con 0x7f2c7c1085c0 2024-04-21T08:38:03.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.716+0000 7f2c7b7fe700 1 --2- 172.21.15.82:0/592478716 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2c64039650 0x7f2c6403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2c70006fd0 tx=0x7f2c70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:03.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.721+0000 7f2c797fa700 1 -- 172.21.15.82:0/592478716 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c6c031070 con 0x7f2c7c1085c0 2024-04-21T08:38:03.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:03.920+0000 7f2c823fe700 1 -- 172.21.15.82:0/592478716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f2c7c0640d0 con 0x7f2c7c1085c0 2024-04-21T08:38:04.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:04.387+0000 7f2c797fa700 1 -- 172.21.15.82:0/592478716 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f2c7c0640d0 con 0x7f2c7c1085c0 2024-04-21T08:38:04.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:04.391+0000 7f2c823fe700 1 -- 172.21.15.82:0/592478716 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2c64039650 msgr2=0x7f2c6403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:04.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:04.391+0000 7f2c823fe700 1 --2- 172.21.15.82:0/592478716 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2c64039650 0x7f2c6403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2c70006fd0 tx=0x7f2c70006e40 comp rx=0 tx=0).stop 2024-04-21T08:38:04.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:04.391+0000 7f2c823fe700 1 -- 172.21.15.82:0/592478716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c7c1085c0 msgr2=0x7f2c7c197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:04.391+0000 7f2c823fe700 1 --2- 172.21.15.82:0/592478716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c7c1085c0 0x7f2c7c197e80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0056d0 tx=0x7f2c6c0057b0 comp rx=0 tx=0).stop 2024-04-21T08:38:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:04.391+0000 7f2c823fe700 1 -- 172.21.15.82:0/592478716 shutdown_connections 2024-04-21T08:38:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:04.391+0000 7f2c823fe700 1 --2- 172.21.15.82:0/592478716 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2c64039650 0x7f2c6403bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:04.391+0000 7f2c823fe700 1 --2- 172.21.15.82:0/592478716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c7c1085c0 0x7f2c7c197e80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:04.391+0000 7f2c823fe700 1 -- 172.21.15.82:0/592478716 >> 172.21.15.82:0/592478716 conn(0x7f2c7c0fc5f0 msgr2=0x7f2c7c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:04.391+0000 7f2c823fe700 1 -- 172.21.15.82:0/592478716 shutdown_connections 2024-04-21T08:38:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:04.391+0000 7f2c823fe700 1 -- 172.21.15.82:0/592478716 wait complete. 2024-04-21T08:38:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:adjusted tunables profile to default 2024-04-21T08:38:04.709 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:04 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/592478716' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-21T08:38:05.022 INFO:tasks.cephadm:Adding mon.a on smithi082 2024-04-21T08:38:05.022 INFO:tasks.cephadm:Adding mon.b on smithi106 2024-04-21T08:38:05.022 INFO:tasks.cephadm:Adding mon.c on smithi134 2024-04-21T08:38:05.023 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch apply mon '3;smithi082:172.21.15.82=a;smithi106:172.21.15.106=b;smithi134:172.21.15.134=c' 2024-04-21T08:38:05.259 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:05.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:05.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/592478716' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-21T08:38:05.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:38:05.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:05.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:06.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.761+0000 7f94f4d81700 1 -- 172.21.15.134:0/4150446215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0101590 msgr2=0x7f94f0101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:06.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.761+0000 7f94f4d81700 1 --2- 172.21.15.134:0/4150446215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0101590 0x7f94f0101970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009910 tx=0x7f94e4009c20 comp rx=0 tx=0).stop 2024-04-21T08:38:06.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.761+0000 7f94f4d81700 1 -- 172.21.15.134:0/4150446215 shutdown_connections 2024-04-21T08:38:06.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.761+0000 7f94f4d81700 1 --2- 172.21.15.134:0/4150446215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0101590 0x7f94f0101970 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.761+0000 7f94f4d81700 1 -- 172.21.15.134:0/4150446215 >> 172.21.15.134:0/4150446215 conn(0x7f94f00fce40 msgr2=0x7f94f00ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:06.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.761+0000 7f94f4d81700 1 -- 172.21.15.134:0/4150446215 shutdown_connections 2024-04-21T08:38:06.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.762+0000 7f94f4d81700 1 -- 172.21.15.134:0/4150446215 wait complete. 2024-04-21T08:38:06.763 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.762+0000 7f94f4d81700 1 Processor -- start 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.762+0000 7f94f4d81700 1 -- start start 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.763+0000 7f94f4d81700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0101590 0x7f94f0198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.763+0000 7f94f4d81700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f94f010b810 con 0x7f94f0101590 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.763+0000 7f94ee59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0101590 0x7f94f0198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.763+0000 7f94ee59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0101590 0x7f94f0198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.134:51086/0 (socket says 172.21.15.134:51086) 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.763+0000 7f94ee59c700 1 -- 172.21.15.134:0/963245194 learned_addr learned my addr 172.21.15.134:0/963245194 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.763+0000 7f94ee59c700 1 -- 172.21.15.134:0/963245194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f0199230 con 0x7f94f0101590 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.763+0000 7f94ee59c700 1 --2- 172.21.15.134:0/963245194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0101590 0x7f94f0198cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f94e40056d0 tx=0x7f94e40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.763+0000 7f94df7fe700 1 -- 172.21.15.134:0/963245194 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94e4026070 con 0x7f94f0101590 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.763+0000 7f94f4d81700 1 -- 172.21.15.134:0/963245194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f0199520 con 0x7f94f0101590 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.763+0000 7f94df7fe700 1 -- 172.21.15.134:0/963245194 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94e400bba0 con 0x7f94f0101590 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.763+0000 7f94df7fe700 1 -- 172.21.15.134:0/963245194 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94e4003920 con 0x7f94f0101590 2024-04-21T08:38:06.764 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.764+0000 7f94f4d81700 1 -- 172.21.15.134:0/963245194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f01999e0 con 0x7f94f0101590 2024-04-21T08:38:06.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.764+0000 7f94f4d81700 1 -- 172.21.15.134:0/963245194 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f0102a70 con 0x7f94f0101590 2024-04-21T08:38:06.765 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.764+0000 7f94df7fe700 1 -- 172.21.15.134:0/963245194 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f94e4029070 con 0x7f94f0101590 2024-04-21T08:38:06.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.765+0000 7f94df7fe700 1 --2- 172.21.15.134:0/963245194 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94d8039650 0x7f94d803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:06.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.765+0000 7f94df7fe700 1 -- 172.21.15.134:0/963245194 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f94e401e070 con 0x7f94f0101590 2024-04-21T08:38:06.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.765+0000 7f94edd9b700 1 --2- 172.21.15.134:0/963245194 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94d8039650 0x7f94d803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:06.766 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.765+0000 7f94edd9b700 1 --2- 172.21.15.134:0/963245194 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94d8039650 0x7f94d803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f94e0006fd0 tx=0x7f94e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:06.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.770+0000 7f94df7fe700 1 -- 172.21.15.134:0/963245194 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94e4013070 con 0x7f94f0101590 2024-04-21T08:38:06.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:06 smithi082 ceph-mon[27967]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-21T08:38:06.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:06 smithi082 ceph-mon[27967]: Updating smithi134:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:06.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:06 smithi082 ceph-mon[27967]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:06.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:06 smithi082 ceph-mon[27967]: Updating smithi134:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:38:06.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:06 smithi082 ceph-mon[27967]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:06.977 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.977+0000 7f94f4d81700 1 -- 172.21.15.134:0/963245194 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi106:172.21.15.106=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}) -- 0x7f94f006ace0 con 0x7f94d8039650 2024-04-21T08:38:06.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.984+0000 7f94df7fe700 1 -- 172.21.15.134:0/963245194 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f94f006ace0 con 0x7f94d8039650 2024-04-21T08:38:06.983 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mon update... 2024-04-21T08:38:06.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.986+0000 7f94f4d81700 1 -- 172.21.15.134:0/963245194 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94d8039650 msgr2=0x7f94d803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:06.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.986+0000 7f94f4d81700 1 --2- 172.21.15.134:0/963245194 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94d8039650 0x7f94d803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f94e0006fd0 tx=0x7f94e0006e40 comp rx=0 tx=0).stop 2024-04-21T08:38:06.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.986+0000 7f94f4d81700 1 -- 172.21.15.134:0/963245194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0101590 msgr2=0x7f94f0198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:06.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.986+0000 7f94f4d81700 1 --2- 172.21.15.134:0/963245194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0101590 0x7f94f0198cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f94e40056d0 tx=0x7f94e40057b0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.986+0000 7f94f4d81700 1 -- 172.21.15.134:0/963245194 shutdown_connections 2024-04-21T08:38:06.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.986+0000 7f94f4d81700 1 --2- 172.21.15.134:0/963245194 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94d8039650 0x7f94d803bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.986+0000 7f94f4d81700 1 --2- 172.21.15.134:0/963245194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0101590 0x7f94f0198cf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.986+0000 7f94f4d81700 1 -- 172.21.15.134:0/963245194 >> 172.21.15.134:0/963245194 conn(0x7f94f00fce40 msgr2=0x7f94f0102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:06.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.986+0000 7f94f4d81700 1 -- 172.21.15.134:0/963245194 shutdown_connections 2024-04-21T08:38:06.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:06.986+0000 7f94f4d81700 1 -- 172.21.15.134:0/963245194 wait complete. 2024-04-21T08:38:07.616 DEBUG:teuthology.orchestra.run.smithi106:mon.b> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.b.service 2024-04-21T08:38:07.619 DEBUG:teuthology.orchestra.run.smithi134:mon.c> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.c.service 2024-04-21T08:38:07.621 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:38:07.621 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:38:08.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:07 smithi082 ceph-mon[27967]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi106:172.21.15.106=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:08.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:07 smithi082 ceph-mon[27967]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi106:172.21.15.106=b;smithi134:172.21.15.134=c;count:3 2024-04-21T08:38:08.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:07 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:08.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:07 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:08.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:07 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:08.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:07 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:08.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:07 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:08.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:07 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:38:08.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:07 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:08.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:07 smithi082 ceph-mon[27967]: Deploying daemon mon.c on smithi134 2024-04-21T08:38:08.429 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:09.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:08 smithi082 ceph-mon[27967]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:11.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.497+0000 7f7137aa5700 1 -- 172.21.15.134:0/1088582842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71301015b0 msgr2=0x7f7130101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.497+0000 7f7137aa5700 1 --2- 172.21.15.134:0/1088582842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71301015b0 0x7f7130101990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7128009910 tx=0x7f7128009c20 comp rx=0 tx=0).stop 2024-04-21T08:38:11.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.497+0000 7f7137aa5700 1 -- 172.21.15.134:0/1088582842 shutdown_connections 2024-04-21T08:38:11.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.497+0000 7f7137aa5700 1 --2- 172.21.15.134:0/1088582842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71301015b0 0x7f7130101990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.497+0000 7f7137aa5700 1 -- 172.21.15.134:0/1088582842 >> 172.21.15.134:0/1088582842 conn(0x7f71300fce60 msgr2=0x7f71300ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.497+0000 7f7137aa5700 1 -- 172.21.15.134:0/1088582842 shutdown_connections 2024-04-21T08:38:11.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.497+0000 7f7137aa5700 1 -- 172.21.15.134:0/1088582842 wait complete. 2024-04-21T08:38:11.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.497+0000 7f7137aa5700 1 Processor -- start 2024-04-21T08:38:11.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.497+0000 7f7137aa5700 1 -- start start 2024-04-21T08:38:11.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.498+0000 7f7137aa5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71301015b0 0x7f7130196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.498+0000 7f7137aa5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f713010b830 con 0x7f71301015b0 2024-04-21T08:38:11.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.498+0000 7f7135841700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71301015b0 0x7f7130196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.498+0000 7f7135841700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71301015b0 0x7f7130196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.134:51106/0 (socket says 172.21.15.134:51106) 2024-04-21T08:38:11.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.498+0000 7f7135841700 1 -- 172.21.15.134:0/3504097956 learned_addr learned my addr 172.21.15.134:0/3504097956 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-21T08:38:11.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.498+0000 7f7135841700 1 -- 172.21.15.134:0/3504097956 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71301972c0 con 0x7f71301015b0 2024-04-21T08:38:11.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.498+0000 7f7135841700 1 --2- 172.21.15.134:0/3504097956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71301015b0 0x7f7130196d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f71280056d0 tx=0x7f71280057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.499+0000 7f7126ffd700 1 -- 172.21.15.134:0/3504097956 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7128028070 con 0x7f71301015b0 2024-04-21T08:38:11.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.499+0000 7f7137aa5700 1 -- 172.21.15.134:0/3504097956 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7130197550 con 0x7f71301015b0 2024-04-21T08:38:11.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.499+0000 7f7126ffd700 1 -- 172.21.15.134:0/3504097956 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f712800bba0 con 0x7f71301015b0 2024-04-21T08:38:11.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.499+0000 7f7126ffd700 1 -- 172.21.15.134:0/3504097956 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7128003920 con 0x7f71301015b0 2024-04-21T08:38:11.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.499+0000 7f7137aa5700 1 -- 172.21.15.134:0/3504097956 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713019b650 con 0x7f71301015b0 2024-04-21T08:38:11.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.500+0000 7f7126ffd700 1 -- 172.21.15.134:0/3504097956 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f712802b070 con 0x7f71301015b0 2024-04-21T08:38:11.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.500+0000 7f7137aa5700 1 -- 172.21.15.134:0/3504097956 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7130102a90 con 0x7f71301015b0 2024-04-21T08:38:11.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.500+0000 7f7126ffd700 1 --2- 172.21.15.134:0/3504097956 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f711c039650 0x7f711c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.500+0000 7f7126ffd700 1 -- 172.21.15.134:0/3504097956 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7128020070 con 0x7f71301015b0 2024-04-21T08:38:11.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.500+0000 7f7135040700 1 --2- 172.21.15.134:0/3504097956 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f711c039650 0x7f711c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.501+0000 7f7135040700 1 --2- 172.21.15.134:0/3504097956 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f711c039650 0x7f711c03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7120006fd0 tx=0x7f7120006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.505+0000 7f7126ffd700 1 -- 172.21.15.134:0/3504097956 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7128013070 con 0x7f71301015b0 2024-04-21T08:38:11.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:11 smithi082 ceph-mon[27967]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:11.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.802+0000 7f7137aa5700 1 -- 172.21.15.134:0/3504097956 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f71301980e0 con 0x7f71301015b0 2024-04-21T08:38:11.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.803+0000 7f7126ffd700 1 -- 172.21.15.134:0/3504097956 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f712800b6f0 con 0x7f71301015b0 2024-04-21T08:38:11.803 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:38:11.803 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":1,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","modified":"2024-04-21T08:36:22.015155Z","created":"2024-04-21T08:36:22.015155Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-21T08:38:11.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.805+0000 7f7137aa5700 1 -- 172.21.15.134:0/3504097956 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f711c039650 msgr2=0x7f711c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.805+0000 7f7137aa5700 1 --2- 172.21.15.134:0/3504097956 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f711c039650 0x7f711c03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7120006fd0 tx=0x7f7120006e40 comp rx=0 tx=0).stop 2024-04-21T08:38:11.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.805+0000 7f7137aa5700 1 -- 172.21.15.134:0/3504097956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71301015b0 msgr2=0x7f7130196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.805+0000 7f7137aa5700 1 --2- 172.21.15.134:0/3504097956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71301015b0 0x7f7130196d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f71280056d0 tx=0x7f71280057b0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.805+0000 7f7137aa5700 1 -- 172.21.15.134:0/3504097956 shutdown_connections 2024-04-21T08:38:11.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.805+0000 7f7137aa5700 1 --2- 172.21.15.134:0/3504097956 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f711c039650 0x7f711c03bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.805+0000 7f7137aa5700 1 --2- 172.21.15.134:0/3504097956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71301015b0 0x7f7130196d80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.805+0000 7f7137aa5700 1 -- 172.21.15.134:0/3504097956 >> 172.21.15.134:0/3504097956 conn(0x7f71300fce60 msgr2=0x7f7130102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.805+0000 7f7137aa5700 1 -- 172.21.15.134:0/3504097956 shutdown_connections 2024-04-21T08:38:11.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:11.805+0000 7f7137aa5700 1 -- 172.21.15.134:0/3504097956 wait complete. 2024-04-21T08:38:11.806 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 1 2024-04-21T08:38:12.116 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:11 smithi134 ceph-mon[31772]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-21T08:38:12.116 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:12 smithi134 bash[31745]: ae5eb50a307ae116e33b3bc779b985a087ec49a6b5e7a99633cd47afae086a8b 2024-04-21T08:38:12.443 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:12 smithi134 systemd[1]: Started Ceph mon.c for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:38:13.470 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:38:13.470 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:38:13.716 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:38:17.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:38:17.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:17.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: mon.a calling monitor election 2024-04-21T08:38:17.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:17.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: mon.c calling monitor election 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: monmap epoch 2 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: fsid 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: last_changed 2024-04-21T08:38:11.882985+0000 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: created 2024-04-21T08:36:22.015155+0000 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: min_mon_release 19 (squid) 2024-04-21T08:38:17.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: election_strategy: 1 2024-04-21T08:38:17.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-21T08:38:17.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-21T08:38:17.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: fsmap 2024-04-21T08:38:17.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:38:17.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: mgrmap e13: x(active, since 50s) 2024-04-21T08:38:17.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: overall HEALTH_OK 2024-04-21T08:38:17.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:17.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:17.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:17.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:38:17.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:17.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:38:17.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:17.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: mon.a calling monitor election 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: mon.c calling monitor election 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: monmap epoch 2 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: fsid 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: last_changed 2024-04-21T08:38:11.882985+0000 2024-04-21T08:38:17.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: created 2024-04-21T08:36:22.015155+0000 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: min_mon_release 19 (squid) 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: election_strategy: 1 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: fsmap 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: mgrmap e13: x(active, since 50s) 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: overall HEALTH_OK 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:38:17.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:18.088 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.089+0000 7ff6d2373700 1 -- 172.21.15.134:0/3166884483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6b40035c0 msgr2=0x7ff6b4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.089+0000 7ff6d2373700 1 --2- 172.21.15.134:0/3166884483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6b40035c0 0x7ff6b4005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc06ad30 tx=0x7ff6bc00feb0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.089+0000 7ff6d2373700 1 -- 172.21.15.134:0/3166884483 shutdown_connections 2024-04-21T08:38:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.089+0000 7ff6d2373700 1 --2- 172.21.15.134:0/3166884483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6b40035c0 0x7ff6b4005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.089+0000 7ff6d2373700 1 -- 172.21.15.134:0/3166884483 >> 172.21.15.134:0/3166884483 conn(0x7ff6cc0fcf80 msgr2=0x7ff6cc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.089+0000 7ff6d2373700 1 -- 172.21.15.134:0/3166884483 shutdown_connections 2024-04-21T08:38:18.089 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.089+0000 7ff6d2373700 1 -- 172.21.15.134:0/3166884483 wait complete. 2024-04-21T08:38:18.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:17 smithi134 ceph-mon[31772]: Deploying daemon mon.b on smithi106 2024-04-21T08:38:18.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:18.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.090+0000 7ff6d2373700 1 Processor -- start 2024-04-21T08:38:18.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.090+0000 7ff6d2373700 1 -- start start 2024-04-21T08:38:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.090+0000 7ff6d2373700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6b40035c0 0x7ff6cc198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.090+0000 7ff6d2373700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc198550 0x7ff6cc19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.090+0000 7ff6d2373700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc10a4a0 con 0x7ff6b40035c0 2024-04-21T08:38:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.090+0000 7ff6d2373700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc10a320 con 0x7ff6cc198550 2024-04-21T08:38:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.091+0000 7ff6cb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc198550 0x7ff6cc19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.091+0000 7ff6cb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc198550 0x7ff6cc19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:34726/0 (socket says 172.21.15.134:34726) 2024-04-21T08:38:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.091+0000 7ff6cb7fe700 1 -- 172.21.15.134:0/1332418218 learned_addr learned my addr 172.21.15.134:0/1332418218 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-21T08:38:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.091+0000 7ff6cbfff700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6b40035c0 0x7ff6cc198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.091+0000 7ff6cb7fe700 1 -- 172.21.15.134:0/1332418218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc198550 msgr2=0x7ff6cc19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-21T08:38:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.091+0000 7ff6cb7fe700 1 -- 172.21.15.134:0/1332418218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc198550 msgr2=0x7ff6cc19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T08:38:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.091+0000 7ff6cb7fe700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc198550 0x7ff6cc19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T08:38:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.091+0000 7ff6cb7fe700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc198550 0x7ff6cc19c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.091+0000 7ff6cbfff700 1 -- 172.21.15.134:0/1332418218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc198550 msgr2=0x7ff6cc19c8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.091+0000 7ff6cbfff700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc198550 0x7ff6cc19c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.091+0000 7ff6cbfff700 1 -- 172.21.15.134:0/1332418218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6cc198b90 con 0x7ff6b40035c0 2024-04-21T08:38:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.091+0000 7ff6cbfff700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6b40035c0 0x7ff6cc198010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc010010 tx=0x7ff6bc005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.092+0000 7ff6c97fa700 1 -- 172.21.15.134:0/1332418218 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff6bc013960 con 0x7ff6b40035c0 2024-04-21T08:38:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.092+0000 7ff6d2373700 1 -- 172.21.15.134:0/1332418218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6cc19ce20 con 0x7ff6b40035c0 2024-04-21T08:38:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.092+0000 7ff6c97fa700 1 -- 172.21.15.134:0/1332418218 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6bc0044e0 con 0x7ff6b40035c0 2024-04-21T08:38:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.092+0000 7ff6c97fa700 1 -- 172.21.15.134:0/1332418218 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff6bc003ce0 con 0x7ff6b40035c0 2024-04-21T08:38:18.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.092+0000 7ff6d2373700 1 -- 172.21.15.134:0/1332418218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6cc19d380 con 0x7ff6b40035c0 2024-04-21T08:38:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.093+0000 7ff6c97fa700 1 -- 172.21.15.134:0/1332418218 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff6bc023070 con 0x7ff6b40035c0 2024-04-21T08:38:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.093+0000 7ff6d2373700 1 -- 172.21.15.134:0/1332418218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6cc06b530 con 0x7ff6b40035c0 2024-04-21T08:38:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.093+0000 7ff6c97fa700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ff6b8039590 0x7ff6b803ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.093+0000 7ff6c97fa700 1 -- 172.21.15.134:0/1332418218 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff6bc020050 con 0x7ff6b40035c0 2024-04-21T08:38:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.093+0000 7ff6cb7fe700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ff6b8039590 0x7ff6b803ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:18.094 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.094+0000 7ff6cb7fe700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ff6b8039590 0x7ff6b803ba50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009180 tx=0x7ff6c0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:18.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.099+0000 7ff6c97fa700 1 -- 172.21.15.134:0/1332418218 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6bc028020 con 0x7ff6b40035c0 2024-04-21T08:38:18.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:17 smithi082 ceph-mon[27967]: Deploying daemon mon.b on smithi106 2024-04-21T08:38:18.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.383+0000 7ff6d2373700 1 -- 172.21.15.134:0/1332418218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff6cc19d660 con 0x7ff6b40035c0 2024-04-21T08:38:18.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.384+0000 7ff6c97fa700 1 -- 172.21.15.134:0/1332418218 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7ff6bc01a390 con 0x7ff6b40035c0 2024-04-21T08:38:18.384 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:38:18.384 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":2,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","modified":"2024-04-21T08:38:11.882985Z","created":"2024-04-21T08:36:22.015155Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-21T08:38:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.386+0000 7ff6d2373700 1 -- 172.21.15.134:0/1332418218 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ff6b8039590 msgr2=0x7ff6b803ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.386+0000 7ff6d2373700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ff6b8039590 0x7ff6b803ba50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009180 tx=0x7ff6c0008040 comp rx=0 tx=0).stop 2024-04-21T08:38:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.386+0000 7ff6d2373700 1 -- 172.21.15.134:0/1332418218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6b40035c0 msgr2=0x7ff6cc198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.386+0000 7ff6d2373700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6b40035c0 0x7ff6cc198010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc010010 tx=0x7ff6bc005cc0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.386+0000 7ff6d2373700 1 -- 172.21.15.134:0/1332418218 shutdown_connections 2024-04-21T08:38:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.386+0000 7ff6d2373700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ff6b8039590 0x7ff6b803ba50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.386+0000 7ff6d2373700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6b40035c0 0x7ff6cc198010 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.386+0000 7ff6d2373700 1 --2- 172.21.15.134:0/1332418218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc198550 0x7ff6cc19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.386+0000 7ff6d2373700 1 -- 172.21.15.134:0/1332418218 >> 172.21.15.134:0/1332418218 conn(0x7ff6cc0fcf80 msgr2=0x7ff6cc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.386+0000 7ff6d2373700 1 -- 172.21.15.134:0/1332418218 shutdown_connections 2024-04-21T08:38:18.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:18.386+0000 7ff6d2373700 1 -- 172.21.15.134:0/1332418218 wait complete. 2024-04-21T08:38:18.387 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 2 2024-04-21T08:38:19.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:18 smithi134 ceph-mon[31772]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:19.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:18 smithi134 ceph-mon[31772]: from='client.? 172.21.15.134:0/1332418218' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:38:19.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:18 smithi082 ceph-mon[27967]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:19.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:18 smithi082 ceph-mon[27967]: from='client.? 172.21.15.134:0/1332418218' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:38:20.009 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:38:20.010 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:38:20.273 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:38:20.711 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:20 smithi106 ceph-mon[31072]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-21T08:38:21.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:20 smithi106 bash[31053]: 1eaa36030e11e0b04af2cdd6cf84cdb282cbe2ff9adc09c02cf8284d4c0ac196 2024-04-21T08:38:21.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:20 smithi106 systemd[1]: Started Ceph mon.b for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:38:25.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.964+0000 7f0865229700 1 -- 172.21.15.134:0/1086025976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08480035c0 msgr2=0x7f0848005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:25.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.964+0000 7f0865229700 1 --2- 172.21.15.134:0/1086025976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08480035c0 0x7f0848005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0860077b80 tx=0x7f085400fe60 comp rx=0 tx=0).stop 2024-04-21T08:38:25.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.964+0000 7f0865229700 1 -- 172.21.15.134:0/1086025976 shutdown_connections 2024-04-21T08:38:25.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.964+0000 7f0865229700 1 --2- 172.21.15.134:0/1086025976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08480035c0 0x7f0848005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.964+0000 7f0865229700 1 -- 172.21.15.134:0/1086025976 >> 172.21.15.134:0/1086025976 conn(0x7f08600767a0 msgr2=0x7f0860076bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:25.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.964+0000 7f0865229700 1 -- 172.21.15.134:0/1086025976 shutdown_connections 2024-04-21T08:38:25.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.964+0000 7f0865229700 1 -- 172.21.15.134:0/1086025976 wait complete. 2024-04-21T08:38:25.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.965+0000 7f0865229700 1 Processor -- start 2024-04-21T08:38:25.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.965+0000 7f0865229700 1 -- start start 2024-04-21T08:38:25.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f0865229700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08480035c0 0x7f0860101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:25.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f0865229700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08601023a0 0x7f0860102820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:25.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f0865229700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08601a42f0 0x7f08601a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:25.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f0865229700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f086010a460 con 0x7f08601a42f0 2024-04-21T08:38:25.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f0865229700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f086010a2e0 con 0x7f08480035c0 2024-04-21T08:38:25.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f0865229700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f086010a5e0 con 0x7f08601023a0 2024-04-21T08:38:25.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f085e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08601023a0 0x7f0860102820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:25.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f085e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08601023a0 0x7f0860102820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36660/0 (socket says 172.21.15.134:36660) 2024-04-21T08:38:25.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f085e59c700 1 -- 172.21.15.134:0/1995400825 learned_addr learned my addr 172.21.15.134:0/1995400825 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-21T08:38:25.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f085f59e700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08601a42f0 0x7f08601a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:25.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f085e59c700 1 -- 172.21.15.134:0/1995400825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08601023a0 msgr2=0x7f0860102820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-21T08:38:25.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f085e59c700 1 -- 172.21.15.134:0/1995400825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08601023a0 msgr2=0x7f0860102820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T08:38:25.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f085e59c700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08601023a0 0x7f0860102820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T08:38:25.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f085e59c700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08601023a0 0x7f0860102820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:25.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.966+0000 7f085ed9d700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08480035c0 0x7f0860101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:25.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f085ed9d700 1 -- 172.21.15.134:0/1995400825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08480035c0 msgr2=0x7f0860101e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-21T08:38:25.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f085ed9d700 1 -- 172.21.15.134:0/1995400825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08480035c0 msgr2=0x7f0860101e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T08:38:25.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f085ed9d700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08480035c0 0x7f0860101e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T08:38:25.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f085ed9d700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08480035c0 0x7f0860101e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:25.968 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f085f59e700 1 -- 172.21.15.134:0/1995400825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08601023a0 msgr2=0x7f0860102820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:25.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f085f59e700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08601023a0 0x7f0860102820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f085f59e700 1 -- 172.21.15.134:0/1995400825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08480035c0 msgr2=0x7f0860101e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:25.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f085f59e700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08480035c0 0x7f0860101e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:25.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f085f59e700 1 -- 172.21.15.134:0/1995400825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08601a6e10 con 0x7f08601a42f0 2024-04-21T08:38:25.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f085f59e700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08601a42f0 0x7f08601a66e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f085800bd50 tx=0x7f085800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:25.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f0847fff700 1 -- 172.21.15.134:0/1995400825 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858019070 con 0x7f08601a42f0 2024-04-21T08:38:25.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f0847fff700 1 -- 172.21.15.134:0/1995400825 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f085800cd10 con 0x7f08601a42f0 2024-04-21T08:38:25.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f0865229700 1 -- 172.21.15.134:0/1995400825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08601a7100 con 0x7f08601a42f0 2024-04-21T08:38:25.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f0847fff700 1 -- 172.21.15.134:0/1995400825 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858012700 con 0x7f08601a42f0 2024-04-21T08:38:25.969 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.967+0000 7f0865229700 1 -- 172.21.15.134:0/1995400825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08601a7640 con 0x7f08601a42f0 2024-04-21T08:38:25.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.968+0000 7f0847fff700 1 -- 172.21.15.134:0/1995400825 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f085801c070 con 0x7f08601a42f0 2024-04-21T08:38:25.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.968+0000 7f0847fff700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f084c039ae0 0x7f084c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:25.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.968+0000 7f0847fff700 1 -- 172.21.15.134:0/1995400825 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f085804d160 con 0x7f08601a42f0 2024-04-21T08:38:25.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.969+0000 7f085ed9d700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f084c039ae0 0x7f084c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:25.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.969+0000 7f0865229700 1 -- 172.21.15.134:0/1995400825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08600640a0 con 0x7f08601a42f0 2024-04-21T08:38:25.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.969+0000 7f085ed9d700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f084c039ae0 0x7f084c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f085400fa10 tx=0x7f0854005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:25.973 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:25.974+0000 7f0847fff700 1 -- 172.21.15.134:0/1995400825 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f085800e0a0 con 0x7f08601a42f0 2024-04-21T08:38:26.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:38:26.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: mon.c calling monitor election 2024-04-21T08:38:26.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: mon.a calling monitor election 2024-04-21T08:38:26.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:26.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:26.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:26.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: monmap epoch 3 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: fsid 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: last_changed 2024-04-21T08:38:20.685186+0000 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: created 2024-04-21T08:36:22.015155+0000 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: min_mon_release 19 (squid) 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: election_strategy: 1 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: fsmap 2024-04-21T08:38:26.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:38:26.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: mgrmap e13: x(active, since 59s) 2024-04-21T08:38:26.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: overall HEALTH_OK 2024-04-21T08:38:26.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:26.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:26.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:26.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:26.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:38:26.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: mon.c calling monitor election 2024-04-21T08:38:26.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: mon.a calling monitor election 2024-04-21T08:38:26.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:26.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: monmap epoch 3 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: fsid 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: last_changed 2024-04-21T08:38:20.685186+0000 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: created 2024-04-21T08:36:22.015155+0000 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: min_mon_release 19 (squid) 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: election_strategy: 1 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-21T08:38:26.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-21T08:38:26.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-21T08:38:26.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: fsmap 2024-04-21T08:38:26.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:38:26.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: mgrmap e13: x(active, since 59s) 2024-04-21T08:38:26.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: overall HEALTH_OK 2024-04-21T08:38:26.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:26.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:26.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:26.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:26.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.267+0000 7f0865229700 1 -- 172.21.15.134:0/1995400825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f086002df80 con 0x7f08601a42f0 2024-04-21T08:38:26.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.267+0000 7f0847fff700 1 -- 172.21.15.134:0/1995400825 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f0858007290 con 0x7f08601a42f0 2024-04-21T08:38:26.267 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-21T08:38:26.267 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":3,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","modified":"2024-04-21T08:38:20.685186Z","created":"2024-04-21T08:36:22.015155Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-21T08:38:26.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.269+0000 7f0865229700 1 -- 172.21.15.134:0/1995400825 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f084c039ae0 msgr2=0x7f084c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:26.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.269+0000 7f0865229700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f084c039ae0 0x7f084c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f085400fa10 tx=0x7f0854005de0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.269+0000 7f0865229700 1 -- 172.21.15.134:0/1995400825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08601a42f0 msgr2=0x7f08601a66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:26.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.269+0000 7f0865229700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08601a42f0 0x7f08601a66e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f085800bd50 tx=0x7f085800be30 comp rx=0 tx=0).stop 2024-04-21T08:38:26.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.270+0000 7f0865229700 1 -- 172.21.15.134:0/1995400825 shutdown_connections 2024-04-21T08:38:26.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.270+0000 7f0865229700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f084c039ae0 0x7f084c03bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.270+0000 7f0865229700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08601a42f0 0x7f08601a66e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.270+0000 7f0865229700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08480035c0 0x7f0860101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.270+0000 7f0865229700 1 --2- 172.21.15.134:0/1995400825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08601023a0 0x7f0860102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.270+0000 7f0865229700 1 -- 172.21.15.134:0/1995400825 >> 172.21.15.134:0/1995400825 conn(0x7f08600767a0 msgr2=0x7f0860107270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:26.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.270+0000 7f0865229700 1 -- 172.21.15.134:0/1995400825 shutdown_connections 2024-04-21T08:38:26.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:26.270+0000 7f0865229700 1 -- 172.21.15.134:0/1995400825 wait complete. 2024-04-21T08:38:26.270 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 3 2024-04-21T08:38:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:26 smithi134 ceph-mon[31772]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:26 smithi134 ceph-mon[31772]: from='client.? 172.21.15.134:0/1995400825' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:38:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:26.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:26.879 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-21T08:38:26.879 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-21T08:38:27.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:27.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:26 smithi082 ceph-mon[27967]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:27.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:26 smithi082 ceph-mon[27967]: from='client.? 172.21.15.134:0/1995400825' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:38:27.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:27.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:27.124 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:28.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:38:28.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: mon.c calling monitor election 2024-04-21T08:38:28.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: mon.a calling monitor election 2024-04-21T08:38:28.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:28.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:28.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:28.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: monmap epoch 3 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: fsid 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: last_changed 2024-04-21T08:38:20.685186+0000 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: created 2024-04-21T08:36:22.015155+0000 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: min_mon_release 19 (squid) 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: election_strategy: 1 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-21T08:38:28.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: fsmap 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: mgrmap e13: x(active, since 59s) 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: overall HEALTH_OK 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='client.? 172.21.15.134:0/1995400825' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:28.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.738+0000 7fbe4c881700 1 -- 172.21.15.82:0/1991535321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe48101590 msgr2=0x7fbe48101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:28.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.738+0000 7fbe4c881700 1 --2- 172.21.15.82:0/1991535321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe48101590 0x7fbe48101970 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009910 tx=0x7fbe3c009c20 comp rx=0 tx=0).stop 2024-04-21T08:38:28.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.738+0000 7fbe4c881700 1 -- 172.21.15.82:0/1991535321 shutdown_connections 2024-04-21T08:38:28.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.738+0000 7fbe4c881700 1 --2- 172.21.15.82:0/1991535321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe48101590 0x7fbe48101970 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.738+0000 7fbe4c881700 1 -- 172.21.15.82:0/1991535321 >> 172.21.15.82:0/1991535321 conn(0x7fbe480fce40 msgr2=0x7fbe480ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:28.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.738+0000 7fbe4c881700 1 -- 172.21.15.82:0/1991535321 shutdown_connections 2024-04-21T08:38:28.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.738+0000 7fbe4c881700 1 -- 172.21.15.82:0/1991535321 wait complete. 2024-04-21T08:38:28.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4c881700 1 Processor -- start 2024-04-21T08:38:28.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4c881700 1 -- start start 2024-04-21T08:38:28.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4c881700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe48101590 0x7fbe48198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4c881700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe48199530 0x7fbe481999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4c881700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe481a2060 0x7fbe481a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4c881700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4810b810 con 0x7fbe48101590 2024-04-21T08:38:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4c881700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4810b690 con 0x7fbe48199530 2024-04-21T08:38:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4c881700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4810b990 con 0x7fbe481a2060 2024-04-21T08:38:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe48101590 0x7fbe48198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe48101590 0x7fbe48198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55424/0 (socket says 172.21.15.82:55424) 2024-04-21T08:38:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4659c700 1 -- 172.21.15.82:0/758839108 learned_addr learned my addr 172.21.15.82:0/758839108 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:38:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe45d9b700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe48199530 0x7fbe481999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe46d9d700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe481a2060 0x7fbe481a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4659c700 1 -- 172.21.15.82:0/758839108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe481a2060 msgr2=0x7fbe481a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:28.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4659c700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe481a2060 0x7fbe481a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4659c700 1 -- 172.21.15.82:0/758839108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe48199530 msgr2=0x7fbe481999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:28.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4659c700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe48199530 0x7fbe481999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.739+0000 7fbe4659c700 1 -- 172.21.15.82:0/758839108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe481a4c60 con 0x7fbe48101590 2024-04-21T08:38:28.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.740+0000 7fbe4659c700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe48101590 0x7fbe48198ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c005b40 tx=0x7fbe3c004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:28.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.740+0000 7fbe377fe700 1 -- 172.21.15.82:0/758839108 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c026070 con 0x7fbe48101590 2024-04-21T08:38:28.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.740+0000 7fbe377fe700 1 -- 172.21.15.82:0/758839108 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe3c00b6f0 con 0x7fbe48101590 2024-04-21T08:38:28.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.740+0000 7fbe4c881700 1 -- 172.21.15.82:0/758839108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe481a4f50 con 0x7fbe48101590 2024-04-21T08:38:28.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.740+0000 7fbe377fe700 1 -- 172.21.15.82:0/758839108 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c0038e0 con 0x7fbe48101590 2024-04-21T08:38:28.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.740+0000 7fbe4c881700 1 -- 172.21.15.82:0/758839108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe481a53b0 con 0x7fbe48101590 2024-04-21T08:38:28.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.741+0000 7fbe377fe700 1 -- 172.21.15.82:0/758839108 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbe3c029070 con 0x7fbe48101590 2024-04-21T08:38:28.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.741+0000 7fbe4c881700 1 -- 172.21.15.82:0/758839108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe480640a0 con 0x7fbe48101590 2024-04-21T08:38:28.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.741+0000 7fbe377fe700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbe30039ae0 0x7fbe3003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:28.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.741+0000 7fbe377fe700 1 -- 172.21.15.82:0/758839108 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbe3c04d380 con 0x7fbe48101590 2024-04-21T08:38:28.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.741+0000 7fbe45d9b700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbe30039ae0 0x7fbe3003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:28.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.742+0000 7fbe45d9b700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbe30039ae0 0x7fbe3003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009ad0 tx=0x7fbe38009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:28.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.746+0000 7fbe377fe700 1 -- 172.21.15.82:0/758839108 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe3c013070 con 0x7fbe48101590 2024-04-21T08:38:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: mon.b calling monitor election 2024-04-21T08:38:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: mon.b calling monitor election 2024-04-21T08:38:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: mon.c calling monitor election 2024-04-21T08:38:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: mon.a calling monitor election 2024-04-21T08:38:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-21T08:38:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-21T08:38:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-21T08:38:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T08:38:28.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: monmap epoch 3 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: fsid 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: last_changed 2024-04-21T08:38:20.685186+0000 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: created 2024-04-21T08:36:22.015155+0000 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: min_mon_release 19 (squid) 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: election_strategy: 1 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: fsmap 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: mgrmap e13: x(active, since 61s) 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: overall HEALTH_OK 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: Updating smithi082:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:28.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: Updating smithi106:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: Updating smithi134:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T08:38:28.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:38:28.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:38:28.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:28.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: Reconfiguring daemon mon.a on smithi082 2024-04-21T08:38:28.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:28.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.964+0000 7fbe4c881700 1 -- 172.21.15.82:0/758839108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fbe481a56f0 con 0x7fbe48101590 2024-04-21T08:38:28.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.964+0000 7fbe377fe700 1 -- 172.21.15.82:0/758839108 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fbe3c030090 con 0x7fbe48101590 2024-04-21T08:38:28.964 INFO:teuthology.orchestra.run.smithi082.stdout:# minimal ceph.conf for 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:38:28.964 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-04-21T08:38:28.965 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:38:28.965 INFO:teuthology.orchestra.run.smithi082.stdout: mon_host = [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-04-21T08:38:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.966+0000 7fbe4c881700 1 -- 172.21.15.82:0/758839108 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbe30039ae0 msgr2=0x7fbe3003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.966+0000 7fbe4c881700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbe30039ae0 0x7fbe3003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009ad0 tx=0x7fbe38009210 comp rx=0 tx=0).stop 2024-04-21T08:38:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.966+0000 7fbe4c881700 1 -- 172.21.15.82:0/758839108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe48101590 msgr2=0x7fbe48198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.966+0000 7fbe4c881700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe48101590 0x7fbe48198ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c005b40 tx=0x7fbe3c004f80 comp rx=0 tx=0).stop 2024-04-21T08:38:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.966+0000 7fbe4c881700 1 -- 172.21.15.82:0/758839108 shutdown_connections 2024-04-21T08:38:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.966+0000 7fbe4c881700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbe30039ae0 0x7fbe3003bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.966+0000 7fbe4c881700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe481a2060 0x7fbe481a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.966+0000 7fbe4c881700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe48101590 0x7fbe48198ff0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.966+0000 7fbe4c881700 1 --2- 172.21.15.82:0/758839108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe48199530 0x7fbe481999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.966+0000 7fbe4c881700 1 -- 172.21.15.82:0/758839108 >> 172.21.15.82:0/758839108 conn(0x7fbe480fce40 msgr2=0x7fbe48102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.967+0000 7fbe4c881700 1 -- 172.21.15.82:0/758839108 shutdown_connections 2024-04-21T08:38:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:28.967+0000 7fbe4c881700 1 -- 172.21.15.82:0/758839108 wait complete. 2024-04-21T08:38:28.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: mon.b calling monitor election 2024-04-21T08:38:28.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: mon.b calling monitor election 2024-04-21T08:38:28.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: mon.c calling monitor election 2024-04-21T08:38:28.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: mon.a calling monitor election 2024-04-21T08:38:28.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-21T08:38:28.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-21T08:38:28.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-21T08:38:28.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T08:38:28.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: monmap epoch 3 2024-04-21T08:38:28.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: fsid 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:38:28.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: last_changed 2024-04-21T08:38:20.685186+0000 2024-04-21T08:38:28.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: created 2024-04-21T08:36:22.015155+0000 2024-04-21T08:38:28.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: min_mon_release 19 (squid) 2024-04-21T08:38:28.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: election_strategy: 1 2024-04-21T08:38:28.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-21T08:38:28.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-21T08:38:28.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-21T08:38:28.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: fsmap 2024-04-21T08:38:28.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:38:28.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: mgrmap e13: x(active, since 61s) 2024-04-21T08:38:28.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: overall HEALTH_OK 2024-04-21T08:38:28.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: Updating smithi082:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:28.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: Updating smithi106:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:28.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: Updating smithi134:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:28.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:28.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:28.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T08:38:28.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:38:28.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:38:28.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:28.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: Reconfiguring daemon mon.a on smithi082 2024-04-21T08:38:28.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:29.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: mon.b calling monitor election 2024-04-21T08:38:29.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: mon.b calling monitor election 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: mon.c calling monitor election 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: mon.a calling monitor election 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: monmap epoch 3 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: fsid 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: last_changed 2024-04-21T08:38:20.685186+0000 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: created 2024-04-21T08:36:22.015155+0000 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: min_mon_release 19 (squid) 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: election_strategy: 1 2024-04-21T08:38:29.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: fsmap 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: mgrmap e13: x(active, since 61s) 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: overall HEALTH_OK 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: Updating smithi082:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: Updating smithi106:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: Updating smithi134:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:29.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: Reconfiguring daemon mon.a on smithi082 2024-04-21T08:38:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:29.618 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-21T08:38:29.618 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:38:29.618 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T08:38:29.643 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:38:29.643 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:29.708 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-21T08:38:29.708 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T08:38:29.735 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-21T08:38:29.735 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:29.795 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:29 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/758839108' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:29.799 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-21T08:38:29.799 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T08:38:29.830 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-21T08:38:29.830 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:29.897 INFO:tasks.cephadm:Adding mgr.x on smithi082 2024-04-21T08:38:29.897 INFO:tasks.cephadm:Adding mgr.y on smithi106 2024-04-21T08:38:29.897 INFO:tasks.cephadm:Adding mgr.z on smithi134 2024-04-21T08:38:29.897 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi082=x;smithi106=y;smithi134=z' 2024-04-21T08:38:29.961 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:29 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/758839108' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:29.976 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:29 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/758839108' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:30.175 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:38:30.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:30 smithi082 ceph-mon[27967]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:30.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:30 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:30.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:30 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:30.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:30 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:38:30.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:30 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:38:30.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:30 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:30.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:30 smithi134 ceph-mon[31772]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:30.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:30 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:30.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:30 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:30.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:30 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:38:30.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:30 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:38:30.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:30 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:31.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:30 smithi106 ceph-mon[31072]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:31.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:30 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:31.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:30 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:31.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:30 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:38:31.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:30 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:38:31.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:30 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:31.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.641+0000 7fdc1475e700 1 -- 172.21.15.134:0/3810044073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc0c1030d0 msgr2=0x7fdc0c1054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:31.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.641+0000 7fdc1475e700 1 --2- 172.21.15.134:0/3810044073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc0c1030d0 0x7fdc0c1054c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009910 tx=0x7fdbfc009c20 comp rx=0 tx=0).stop 2024-04-21T08:38:31.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.641+0000 7fdc1475e700 1 -- 172.21.15.134:0/3810044073 shutdown_connections 2024-04-21T08:38:31.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.641+0000 7fdc1475e700 1 --2- 172.21.15.134:0/3810044073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc0c1030d0 0x7fdc0c1054c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.641+0000 7fdc1475e700 1 -- 172.21.15.134:0/3810044073 >> 172.21.15.134:0/3810044073 conn(0x7fdc0c0fca80 msgr2=0x7fdc0c0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:31.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.641+0000 7fdc1475e700 1 -- 172.21.15.134:0/3810044073 shutdown_connections 2024-04-21T08:38:31.642 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.641+0000 7fdc1475e700 1 -- 172.21.15.134:0/3810044073 wait complete. 2024-04-21T08:38:31.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.641+0000 7fdc1475e700 1 Processor -- start 2024-04-21T08:38:31.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc1475e700 1 -- start start 2024-04-21T08:38:31.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc1475e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c1030d0 0x7fdc0c102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:31.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc1475e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc0c101080 0x7fdc0c101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:31.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc1475e700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc0c101a40 0x7fdc0c1a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:31.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc1475e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c108530 con 0x7fdc0c101080 2024-04-21T08:38:31.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc1475e700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1083b0 con 0x7fdc0c101a40 2024-04-21T08:38:31.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc1475e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1086b0 con 0x7fdc0c1030d0 2024-04-21T08:38:31.645 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc124fa700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c1030d0 0x7fdc0c102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc124fa700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c1030d0 0x7fdc0c102a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:36672/0 (socket says 172.21.15.134:36672) 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc124fa700 1 -- 172.21.15.134:0/1272421616 learned_addr learned my addr 172.21.15.134:0/1272421616 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc11cf9700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc0c101080 0x7fdc0c101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc12cfb700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc0c101a40 0x7fdc0c1a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc124fa700 1 -- 172.21.15.134:0/1272421616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc0c101a40 msgr2=0x7fdc0c1a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc124fa700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc0c101a40 0x7fdc0c1a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc124fa700 1 -- 172.21.15.134:0/1272421616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc0c101080 msgr2=0x7fdc0c101500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc124fa700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc0c101080 0x7fdc0c101500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc124fa700 1 -- 172.21.15.134:0/1272421616 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c1a2380 con 0x7fdc0c1030d0 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.642+0000 7fdc124fa700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c1030d0 0x7fdc0c102a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0092e0 tx=0x7fdbfc004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.643+0000 7fdc037fe700 1 -- 172.21.15.134:0/1272421616 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc026070 con 0x7fdc0c1030d0 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.643+0000 7fdc037fe700 1 -- 172.21.15.134:0/1272421616 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbfc00b750 con 0x7fdc0c1030d0 2024-04-21T08:38:31.646 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.643+0000 7fdc037fe700 1 -- 172.21.15.134:0/1272421616 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc003a00 con 0x7fdc0c1030d0 2024-04-21T08:38:31.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.643+0000 7fdc1475e700 1 -- 172.21.15.134:0/1272421616 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0c1a2610 con 0x7fdc0c1030d0 2024-04-21T08:38:31.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.643+0000 7fdc1475e700 1 -- 172.21.15.134:0/1272421616 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc0c1a2b70 con 0x7fdc0c1030d0 2024-04-21T08:38:31.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.643+0000 7fdc037fe700 1 -- 172.21.15.134:0/1272421616 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdbfc029070 con 0x7fdc0c1030d0 2024-04-21T08:38:31.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.644+0000 7fdc1475e700 1 -- 172.21.15.134:0/1272421616 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0c0640a0 con 0x7fdc0c1030d0 2024-04-21T08:38:31.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.644+0000 7fdc037fe700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdbf8039ae0 0x7fdbf803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:31.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.644+0000 7fdc037fe700 1 -- 172.21.15.134:0/1272421616 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdbfc04d470 con 0x7fdc0c1030d0 2024-04-21T08:38:31.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.644+0000 7fdc11cf9700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdbf8039ae0 0x7fdbf803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:31.647 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.644+0000 7fdc11cf9700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdbf8039ae0 0x7fdbf803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009ad0 tx=0x7fdc08009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:31.648 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.649+0000 7fdc037fe700 1 -- 172.21.15.134:0/1272421616 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdbfc013070 con 0x7fdc0c1030d0 2024-04-21T08:38:31.858 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.858+0000 7fdc1475e700 1 -- 172.21.15.134:0/1272421616 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi106=y;smithi134=z", "target": ["mon-mgr", ""]}) -- 0x7fdc0c062e80 con 0x7fdbf8039ae0 2024-04-21T08:38:31.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.865+0000 7fdc037fe700 1 -- 172.21.15.134:0/1272421616 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdc0c062e80 con 0x7fdbf8039ae0 2024-04-21T08:38:31.864 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mgr update... 2024-04-21T08:38:31.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.867+0000 7fdc1475e700 1 -- 172.21.15.134:0/1272421616 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdbf8039ae0 msgr2=0x7fdbf803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:31.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.867+0000 7fdc1475e700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdbf8039ae0 0x7fdbf803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009ad0 tx=0x7fdc08009210 comp rx=0 tx=0).stop 2024-04-21T08:38:31.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.867+0000 7fdc1475e700 1 -- 172.21.15.134:0/1272421616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c1030d0 msgr2=0x7fdc0c102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:31.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.867+0000 7fdc1475e700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c1030d0 0x7fdc0c102a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0092e0 tx=0x7fdbfc004f80 comp rx=0 tx=0).stop 2024-04-21T08:38:31.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.867+0000 7fdc1475e700 1 -- 172.21.15.134:0/1272421616 shutdown_connections 2024-04-21T08:38:31.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.867+0000 7fdc1475e700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdbf8039ae0 0x7fdbf803bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.867+0000 7fdc1475e700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc0c101a40 0x7fdc0c1a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.867+0000 7fdc1475e700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c1030d0 0x7fdc0c102a30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.867+0000 7fdc1475e700 1 --2- 172.21.15.134:0/1272421616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc0c101080 0x7fdc0c101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:31.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.867+0000 7fdc1475e700 1 -- 172.21.15.134:0/1272421616 >> 172.21.15.134:0/1272421616 conn(0x7fdc0c0fca80 msgr2=0x7fdc0c0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:31.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.867+0000 7fdc1475e700 1 -- 172.21.15.134:0/1272421616 shutdown_connections 2024-04-21T08:38:31.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:38:31.867+0000 7fdc1475e700 1 -- 172.21.15.134:0/1272421616 wait complete. 2024-04-21T08:38:31.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:31 smithi106 ceph-mon[31072]: Reconfiguring mon.b (monmap changed)... 2024-04-21T08:38:31.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:31 smithi106 ceph-mon[31072]: Reconfiguring daemon mon.b on smithi106 2024-04-21T08:38:31.975 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:31 smithi134 ceph-mon[31772]: Reconfiguring mon.b (monmap changed)... 2024-04-21T08:38:31.975 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:31 smithi134 ceph-mon[31772]: Reconfiguring daemon mon.b on smithi106 2024-04-21T08:38:32.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:31 smithi082 ceph-mon[27967]: Reconfiguring mon.b (monmap changed)... 2024-04-21T08:38:32.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:31 smithi082 ceph-mon[27967]: Reconfiguring daemon mon.b on smithi106 2024-04-21T08:38:32.515 DEBUG:teuthology.orchestra.run.smithi106:mgr.y> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.y.service 2024-04-21T08:38:32.518 DEBUG:teuthology.orchestra.run.smithi134:mgr.z> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.z.service 2024-04-21T08:38:32.558 INFO:tasks.cephadm:Deploying OSDs... 2024-04-21T08:38:32.558 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:38:32.559 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T08:38:32.578 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T08:38:32.578 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-04-21T08:38:32.635 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T08:38:32.635 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:38:32.635 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-21T08:38:32.635 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:38:32.635 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:38:32.635 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-21 08:37:38.829011508 +0000 2024-04-21T08:38:32.635 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-21 08:32:33.457231771 +0000 2024-04-21T08:38:32.635 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-21 08:32:33.457231771 +0000 2024-04-21T08:38:32.636 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-21 08:32:33.457231771 +0000 2024-04-21T08:38:32.636 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T08:38:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-21T08:38:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-21T08:38:32.702 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000207347 s, 2.5 MB/s 2024-04-21T08:38:32.703 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T08:38:32.760 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-04-21T08:38:32.816 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T08:38:32.816 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:38:32.816 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-21T08:38:32.816 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:38:32.817 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:38:32.817 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-21 08:37:38.863010829 +0000 2024-04-21T08:38:32.817 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-21 08:32:33.807224652 +0000 2024-04-21T08:38:32.817 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-21 08:32:33.807224652 +0000 2024-04-21T08:38:32.817 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-21 08:32:33.807224652 +0000 2024-04-21T08:38:32.817 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T08:38:32.880 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-21T08:38:32.880 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-21T08:38:32.880 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.00020538 s, 2.5 MB/s 2024-04-21T08:38:32.881 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T08:38:32.937 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-04-21T08:38:32.992 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T08:38:32.992 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:38:32.992 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-21T08:38:32.993 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:38:32.993 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:38:32.993 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-21 08:37:38.900010089 +0000 2024-04-21T08:38:32.993 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-21 08:32:34.133218021 +0000 2024-04-21T08:38:32.993 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-21 08:32:34.133218021 +0000 2024-04-21T08:38:32.993 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-21 08:32:34.133218021 +0000 2024-04-21T08:38:32.993 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T08:38:33.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:32 smithi082 ceph-mon[27967]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi106=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:33.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:32 smithi082 ceph-mon[27967]: Saving service mgr spec with placement smithi082=x;smithi106=y;smithi134=z;count:3 2024-04-21T08:38:33.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:32 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:33.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:32 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:33.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:32 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:33.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:32 smithi082 ceph-mon[27967]: Reconfiguring mon.c (monmap changed)... 2024-04-21T08:38:33.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:32 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:38:33.053 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:32 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:38:33.054 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:32 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:33.054 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:32 smithi082 ceph-mon[27967]: Reconfiguring daemon mon.c on smithi134 2024-04-21T08:38:33.054 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:32 smithi082 ceph-mon[27967]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-21T08:38:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-21T08:38:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000217439 s, 2.4 MB/s 2024-04-21T08:38:33.056 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T08:38:33.112 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-04-21T08:38:33.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:32 smithi106 ceph-mon[31072]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi106=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:33.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:32 smithi106 ceph-mon[31072]: Saving service mgr spec with placement smithi082=x;smithi106=y;smithi134=z;count:3 2024-04-21T08:38:33.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:32 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:33.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:32 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:33.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:32 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:33.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:32 smithi106 ceph-mon[31072]: Reconfiguring mon.c (monmap changed)... 2024-04-21T08:38:33.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:32 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:38:33.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:32 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:38:33.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:32 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:33.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:32 smithi106 ceph-mon[31072]: Reconfiguring daemon mon.c on smithi134 2024-04-21T08:38:33.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:32 smithi106 ceph-mon[31072]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:33.168 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T08:38:33.168 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:38:33.168 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-21T08:38:33.168 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:38:33.168 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:38:33.168 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-21 08:37:38.928009530 +0000 2024-04-21T08:38:33.168 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-21 08:32:34.466211247 +0000 2024-04-21T08:38:33.168 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-21 08:32:34.466211247 +0000 2024-04-21T08:38:33.168 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-21 08:32:34.466211247 +0000 2024-04-21T08:38:33.169 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T08:38:33.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:32 smithi134 ceph-mon[31772]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi106=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:33.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:32 smithi134 ceph-mon[31772]: Saving service mgr spec with placement smithi082=x;smithi106=y;smithi134=z;count:3 2024-04-21T08:38:33.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:32 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:33.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:32 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:33.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:32 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:33.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:32 smithi134 ceph-mon[31772]: Reconfiguring mon.c (monmap changed)... 2024-04-21T08:38:33.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:32 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:38:33.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:32 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:38:33.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:32 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:33.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:32 smithi134 ceph-mon[31772]: Reconfiguring daemon mon.c on smithi134 2024-04-21T08:38:33.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:32 smithi134 ceph-mon[31772]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:33.230 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-21T08:38:33.230 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-21T08:38:33.231 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000212096 s, 2.4 MB/s 2024-04-21T08:38:33.231 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T08:38:33.289 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-21T08:38:33.290 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T08:38:33.308 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T08:38:33.308 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 2024-04-21T08:38:33.365 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T08:38:33.365 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:38:33.365 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-21T08:38:33.365 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:38:33.365 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:38:33.366 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-21 08:38:20.170487390 +0000 2024-04-21T08:38:33.366 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-21 08:32:51.870079156 +0000 2024-04-21T08:38:33.366 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-21 08:32:51.870079156 +0000 2024-04-21T08:38:33.366 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-21 08:32:51.870079156 +0000 2024-04-21T08:38:33.366 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T08:38:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-21T08:38:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-21T08:38:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000284193 s, 1.8 MB/s 2024-04-21T08:38:33.432 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T08:38:33.489 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 2024-04-21T08:38:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T08:38:33.546 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:38:33.546 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-21T08:38:33.546 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:38:33.546 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:38:33.546 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-21 08:38:20.170487390 +0000 2024-04-21T08:38:33.546 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-21 08:32:52.212071058 +0000 2024-04-21T08:38:33.546 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-21 08:32:52.212071058 +0000 2024-04-21T08:38:33.547 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-21 08:32:52.212071058 +0000 2024-04-21T08:38:33.547 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T08:38:33.611 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-21T08:38:33.611 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-21T08:38:33.611 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000323856 s, 1.6 MB/s 2024-04-21T08:38:33.612 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T08:38:33.670 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 2024-04-21T08:38:33.726 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T08:38:33.727 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:38:33.727 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-21T08:38:33.727 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:38:33.727 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:38:33.727 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-21 08:38:20.170487390 +0000 2024-04-21T08:38:33.727 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-21 08:32:52.549063079 +0000 2024-04-21T08:38:33.727 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-21 08:32:52.549063079 +0000 2024-04-21T08:38:33.727 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-21 08:32:52.549063079 +0000 2024-04-21T08:38:33.727 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T08:38:33.790 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-21T08:38:33.791 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-21T08:38:33.791 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000211015 s, 2.4 MB/s 2024-04-21T08:38:33.792 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T08:38:33.849 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 2024-04-21T08:38:33.904 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T08:38:33.905 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:38:33.905 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-21T08:38:33.905 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:38:33.905 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:38:33.905 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-21 08:38:20.171487367 +0000 2024-04-21T08:38:33.905 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-21 08:32:52.902054721 +0000 2024-04-21T08:38:33.905 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-21 08:32:52.902054721 +0000 2024-04-21T08:38:33.905 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-21 08:32:52.902054721 +0000 2024-04-21T08:38:33.905 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-21T08:38:33.967 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000214146 s, 2.4 MB/s 2024-04-21T08:38:33.968 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T08:38:34.025 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-21T08:38:34.025 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T08:38:34.042 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T08:38:34.042 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-04-21T08:38:34.098 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T08:38:34.098 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:38:34.098 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-21T08:38:34.098 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:38:34.098 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:38:34.098 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-21 08:38:10.785657113 +0000 2024-04-21T08:38:34.098 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-21 08:32:50.097283930 +0000 2024-04-21T08:38:34.098 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-21 08:32:50.097283930 +0000 2024-04-21T08:38:34.098 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-21 08:32:50.097283930 +0000 2024-04-21T08:38:34.099 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T08:38:34.162 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-21T08:38:34.162 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-21T08:38:34.162 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000270943 s, 1.9 MB/s 2024-04-21T08:38:34.163 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T08:38:34.219 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-04-21T08:38:34.274 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T08:38:34.274 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:38:34.274 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-21T08:38:34.274 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:38:34.274 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:38:34.274 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-21 08:38:10.785657113 +0000 2024-04-21T08:38:34.275 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-21 08:32:50.433275966 +0000 2024-04-21T08:38:34.275 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-21 08:32:50.433275966 +0000 2024-04-21T08:38:34.275 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-21 08:32:50.433275966 +0000 2024-04-21T08:38:34.275 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T08:38:34.336 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-21T08:38:34.337 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-21T08:38:34.337 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000222221 s, 2.3 MB/s 2024-04-21T08:38:34.338 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T08:38:34.393 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-04-21T08:38:34.449 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T08:38:34.449 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:38:34.449 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-21T08:38:34.449 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:38:34.451 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:38:34.451 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-21 08:38:10.786657089 +0000 2024-04-21T08:38:34.451 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-21 08:32:50.776267838 +0000 2024-04-21T08:38:34.451 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-21 08:32:50.776267838 +0000 2024-04-21T08:38:34.451 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-21 08:32:50.776267838 +0000 2024-04-21T08:38:34.451 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T08:38:34.511 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-21T08:38:34.511 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-21T08:38:34.511 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000227647 s, 2.2 MB/s 2024-04-21T08:38:34.512 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T08:38:34.569 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-04-21T08:38:34.625 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T08:38:34.625 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:38:34.625 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-21T08:38:34.625 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:38:34.625 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:38:34.625 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-21 08:38:10.786657089 +0000 2024-04-21T08:38:34.625 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-21 08:32:51.112259874 +0000 2024-04-21T08:38:34.625 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-21 08:32:51.112259874 +0000 2024-04-21T08:38:34.625 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-21 08:32:51.112259874 +0000 2024-04-21T08:38:34.626 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T08:38:34.687 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-21T08:38:34.687 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-21T08:38:34.688 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000193073 s, 2.7 MB/s 2024-04-21T08:38:34.689 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T08:38:34.745 INFO:tasks.cephadm:Deploying osd.0 on smithi082 with /dev/vg_nvme/lv_4... 2024-04-21T08:38:34.745 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T08:38:34.762 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:34.762 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:34.762 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:34.762 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:34.762 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:34.762 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:34.762 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:38:34.762 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:38:34.762 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:38:34.763 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:34.763 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:34 smithi134 ceph-mon[31772]: Deploying daemon mgr.z on smithi134 2024-04-21T08:38:34.763 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:34 smithi134 ceph-mon[31772]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:34.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:34.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:34.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:34.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:34.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:34.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:34.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:38:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:38:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:38:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:34 smithi082 ceph-mon[27967]: Deploying daemon mgr.z on smithi134 2024-04-21T08:38:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:34 smithi082 ceph-mon[27967]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:34.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:34.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:34.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:38:34.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:38:34.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:38:34.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:34.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:34 smithi106 ceph-mon[31072]: Deploying daemon mgr.z on smithi134 2024-04-21T08:38:34.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:34 smithi106 ceph-mon[31072]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:34.977 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:36.186 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:35 smithi134 systemd[1]: Starting Ceph mgr.z for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:38:36.466 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:36 smithi134 podman[33614]: 2024-04-21 08:38:36.089708284 +0000 UTC m=+0.017860902 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:38:36.466 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:36 smithi134 podman[33614]: 2024-04-21 08:38:36.223251059 +0000 UTC m=+0.151403676 container create c1ec677de54abb1687ebdf47fef959630771a2827ab7f72aa38845b5be97fb21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-21T08:38:36.466 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:36 smithi134 podman[33614]: 2024-04-21 08:38:36.409009134 +0000 UTC m=+0.337161753 container init c1ec677de54abb1687ebdf47fef959630771a2827ab7f72aa38845b5be97fb21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:38:36.466 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:36 smithi134 podman[33614]: 2024-04-21 08:38:36.411446682 +0000 UTC m=+0.339599303 container start c1ec677de54abb1687ebdf47fef959630771a2827ab7f72aa38845b5be97fb21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-21T08:38:36.466 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:36 smithi134 ceph-mgr[33633]: -- 172.21.15.134:0/3325482575 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d15d765a0 con 0x561d15d4ac00 2024-04-21T08:38:36.725 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:36 smithi134 bash[33614]: c1ec677de54abb1687ebdf47fef959630771a2827ab7f72aa38845b5be97fb21 2024-04-21T08:38:36.726 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:36 smithi134 systemd[1]: Started Ceph mgr.z for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:38:36.726 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:36 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:36.573+0000 7f98aa8c0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:38:36.850 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T08:38:36.850 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T08:38:36.850 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-04-21T08:38:36.850 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-04-21T08:38:36.851 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115301 s, 909 MB/s 2024-04-21T08:38:36.851 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-04-21T08:38:37.163 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:36 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:36.756+0000 7f98aa8c0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:37.163 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:36 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:36.858+0000 7f98aa8c0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:38:37.164 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:37 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:37.163+0000 7f98aa8c0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:38:37.424 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:37 smithi082 ceph-mon[27967]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:37.424 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:37.424 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:37.425 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:37.425 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:38:37.425 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:38:37.425 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:38:37.425 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:37.426 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_4 2024-04-21T08:38:37.475 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:37 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:37.247+0000 7f98aa8c0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:38:37.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:37 smithi134 ceph-mon[31772]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:37.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:37.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:37.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:37.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:38:37.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:38:37.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:38:37.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:37.510 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:37 smithi106 ceph-mon[31072]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:37.510 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:37.510 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:37.510 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:37.510 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:38:37.510 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:38:37.510 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:38:37.510 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:37.657 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:38.321 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:38 smithi106 ceph-mon[31072]: Deploying daemon mgr.y on smithi106 2024-04-21T08:38:38.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:38 smithi082 ceph-mon[27967]: Deploying daemon mgr.y on smithi106 2024-04-21T08:38:38.628 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:38 smithi106 systemd[1]: Starting Ceph mgr.y for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:38:38.657 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:38 smithi134 ceph-mon[31772]: Deploying daemon mgr.y on smithi106 2024-04-21T08:38:38.657 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:38 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:38.484+0000 7f98aa8c0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:38:38.657 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:38 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:38.657+0000 7f98aa8c0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:38.975 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:38 smithi106 podman[32432]: 2024-04-21 08:38:38.716085442 +0000 UTC m=+0.083865322 container create 1bf6191a46f517e10c00aaa92e8cd1458410ff0200dd6fdc1b633d9b1ebf34d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-21T08:38:38.975 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:38 smithi106 podman[32432]: 2024-04-21 08:38:38.650396772 +0000 UTC m=+0.018176675 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:38:38.975 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:38 smithi106 podman[32432]: 2024-04-21 08:38:38.901400921 +0000 UTC m=+0.269180800 container init 1bf6191a46f517e10c00aaa92e8cd1458410ff0200dd6fdc1b633d9b1ebf34d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:38:38.976 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:38 smithi106 podman[32432]: 2024-04-21 08:38:38.904311844 +0000 UTC m=+0.272091724 container start 1bf6191a46f517e10c00aaa92e8cd1458410ff0200dd6fdc1b633d9b1ebf34d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-21T08:38:38.976 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:38 smithi106 ceph-mgr[32451]: -- 172.21.15.106:0/1161709781 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e32884a5a0 con 0x55e32881ec00 2024-04-21T08:38:38.976 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:38 smithi106 bash[32432]: 1bf6191a46f517e10c00aaa92e8cd1458410ff0200dd6fdc1b633d9b1ebf34d5 2024-04-21T08:38:38.976 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:38 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:38.833+0000 7f98aa8c0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:38:39.251 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:38 smithi106 systemd[1]: Started Ceph mgr.y for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:38:39.251 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:39.057+0000 7f8acaa6d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:38:39.251 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:39.251+0000 7f8acaa6d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:39.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.271+0000 7fb8e90f4700 1 -- 172.21.15.82:0/2974059488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e4102bd0 msgr2=0x7fb8e4103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:39.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.271+0000 7fb8e90f4700 1 --2- 172.21.15.82:0/2974059488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e4102bd0 0x7fb8e4103030 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0099c0 tx=0x7fb8cc009cd0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.271+0000 7fb8e90f4700 1 -- 172.21.15.82:0/2974059488 shutdown_connections 2024-04-21T08:38:39.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.271+0000 7fb8e90f4700 1 --2- 172.21.15.82:0/2974059488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8e4103570 0x7fb8e4109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.271+0000 7fb8e90f4700 1 --2- 172.21.15.82:0/2974059488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8e41019d0 0x7fb8e4101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.271+0000 7fb8e90f4700 1 --2- 172.21.15.82:0/2974059488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e4102bd0 0x7fb8e4103030 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.271+0000 7fb8e90f4700 1 -- 172.21.15.82:0/2974059488 >> 172.21.15.82:0/2974059488 conn(0x7fb8e40fcf90 msgr2=0x7fb8e40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:39.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.271+0000 7fb8e90f4700 1 -- 172.21.15.82:0/2974059488 shutdown_connections 2024-04-21T08:38:39.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.271+0000 7fb8e90f4700 1 -- 172.21.15.82:0/2974059488 wait complete. 2024-04-21T08:38:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.272+0000 7fb8e90f4700 1 Processor -- start 2024-04-21T08:38:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.272+0000 7fb8e90f4700 1 -- start start 2024-04-21T08:38:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.272+0000 7fb8e90f4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8e41019d0 0x7fb8e4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.272+0000 7fb8e90f4700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8e4102bd0 0x7fb8e4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.272+0000 7fb8e90f4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e4103570 0x7fb8e419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.272+0000 7fb8e90f4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e410d440 con 0x7fb8e4103570 2024-04-21T08:38:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.272+0000 7fb8e90f4700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e410d2c0 con 0x7fb8e4102bd0 2024-04-21T08:38:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.272+0000 7fb8e90f4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e410d5c0 con 0x7fb8e41019d0 2024-04-21T08:38:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.272+0000 7fb8e359e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e4103570 0x7fb8e419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.272+0000 7fb8e359e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e4103570 0x7fb8e419f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47066/0 (socket says 172.21.15.82:47066) 2024-04-21T08:38:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.272+0000 7fb8e359e700 1 -- 172.21.15.82:0/838849190 learned_addr learned my addr 172.21.15.82:0/838849190 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:38:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.272+0000 7fb8e2d9d700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8e41019d0 0x7fb8e4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.273+0000 7fb8da59c700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8e4102bd0 0x7fb8e4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.273+0000 7fb8e359e700 1 -- 172.21.15.82:0/838849190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8e41019d0 msgr2=0x7fb8e4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.273+0000 7fb8e359e700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8e41019d0 0x7fb8e4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.273+0000 7fb8e359e700 1 -- 172.21.15.82:0/838849190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8e4102bd0 msgr2=0x7fb8e4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.273+0000 7fb8e359e700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8e4102bd0 0x7fb8e4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.273+0000 7fb8e359e700 1 -- 172.21.15.82:0/838849190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e419f710 con 0x7fb8e4103570 2024-04-21T08:38:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.273+0000 7fb8e359e700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e4103570 0x7fb8e419f070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc007f10 tx=0x7fb8dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.273+0000 7fb8e0d99700 1 -- 172.21.15.82:0/838849190 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00dd20 con 0x7fb8e4103570 2024-04-21T08:38:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.273+0000 7fb8e90f4700 1 -- 172.21.15.82:0/838849190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e419fa00 con 0x7fb8e4103570 2024-04-21T08:38:39.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.273+0000 7fb8e0d99700 1 -- 172.21.15.82:0/838849190 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8dc004020 con 0x7fb8e4103570 2024-04-21T08:38:39.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.273+0000 7fb8e0d99700 1 -- 172.21.15.82:0/838849190 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc015660 con 0x7fb8e4103570 2024-04-21T08:38:39.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.273+0000 7fb8e90f4700 1 -- 172.21.15.82:0/838849190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e419fef0 con 0x7fb8e4103570 2024-04-21T08:38:39.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.274+0000 7fb8e0d99700 1 -- 172.21.15.82:0/838849190 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb8dc015800 con 0x7fb8e4103570 2024-04-21T08:38:39.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.274+0000 7fb8e90f4700 1 -- 172.21.15.82:0/838849190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e4069730 con 0x7fb8e4103570 2024-04-21T08:38:39.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.274+0000 7fb8e0d99700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb8d0039890 0x7fb8d003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:39.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.274+0000 7fb8e0d99700 1 -- 172.21.15.82:0/838849190 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb8dc04c320 con 0x7fb8e4103570 2024-04-21T08:38:39.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.274+0000 7fb8e2d9d700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb8d0039890 0x7fb8d003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:39.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.275+0000 7fb8e2d9d700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb8d0039890 0x7fb8d003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4102a30 tx=0x7fb8d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:39.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.279+0000 7fb8e0d99700 1 -- 172.21.15.82:0/838849190 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8dc010070 con 0x7fb8e4103570 2024-04-21T08:38:39.431 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:39 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:39.123+0000 7f98aa8c0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:38:39.431 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:39 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:39.215+0000 7f98aa8c0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:38:39.431 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:39 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:39.430+0000 7f98aa8c0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:38:39.431 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:39 smithi134 ceph-mon[31772]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:39.431 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:39 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:39.431 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:39 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:39.431 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:39 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:39.431 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:39 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:39.432 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:39 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:39.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:39.491+0000 7fb8e90f4700 1 -- 172.21.15.82:0/838849190 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb8e4062e80 con 0x7fb8d0039890 2024-04-21T08:38:39.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:39 smithi082 ceph-mon[27967]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:39.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:39 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:39.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:39 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:39.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:39 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:39.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:39 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:39.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:39 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:39 smithi106 ceph-mon[31072]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:39 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:39 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:39 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:39.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:39 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:39.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:39 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:39.629 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:39.361+0000 7f8acaa6d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:38:39.725 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:39 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:39.630+0000 7f98aa8c0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:40.090 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:39.693+0000 7f8acaa6d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:38:40.090 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:39.790+0000 7f8acaa6d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:38:40.304 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:40 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:40.265+0000 7f98aa8c0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:38:40.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:38:40.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:38:40.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:40.442 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:38:40.442 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:38:40.442 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:40.700 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:38:40.701 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:38:40.701 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:40.975 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:40 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:40.700+0000 7f98aa8c0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:38:40.976 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:40 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:40.798+0000 7f98aa8c0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:38:41.305 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:41.001+0000 7f98aa8c0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:38:41.305 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:41.018+0000 7f8acaa6d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:38:41.305 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:41.185+0000 7f8acaa6d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:41.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:41.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:41.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:41.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:41.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:38:41.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:38:41.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:41.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.616 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:41.524+0000 7f98aa8c0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:38:41.616 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:41.615+0000 7f98aa8c0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:38:41.616 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:41.616 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:41.616 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.617 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.617 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.617 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.617 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:41.617 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:41.617 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.617 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:38:41.617 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:38:41.617 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:41.617 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.627 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:41.353+0000 7f8acaa6d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:38:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:41.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:38:41.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:38:41.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:41.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:41.929 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:41.637+0000 7f8acaa6d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:38:41.929 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:41.728+0000 7f8acaa6d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:38:41.929 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:41.929+0000 7f8acaa6d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:38:41.975 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:41.856+0000 7f98aa8c0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:38:42.306 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:41 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:41.990+0000 7f98aa8c0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:38:42.307 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:42 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:42.085+0000 7f98aa8c0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:38:42.307 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:42 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:42.199+0000 7f98aa8c0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:42.307 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:42 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:42.120+0000 7f8acaa6d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:42.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:42 smithi082 ceph-mon[27967]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T08:38:42.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:42 smithi082 ceph-mon[27967]: Reconfiguring daemon mgr.x on smithi082 2024-04-21T08:38:42.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:42 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/121871947' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b97aa94e-52aa-4c7c-8a91-252060989aea"}]: dispatch 2024-04-21T08:38:42.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:42 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/121871947' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b97aa94e-52aa-4c7c-8a91-252060989aea"}]': finished 2024-04-21T08:38:42.557 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:42 smithi082 ceph-mon[27967]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T08:38:42.557 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:42 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:38:42.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:42 smithi106 ceph-mon[31072]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T08:38:42.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:42 smithi106 ceph-mon[31072]: Reconfiguring daemon mgr.x on smithi082 2024-04-21T08:38:42.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:42 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/121871947' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b97aa94e-52aa-4c7c-8a91-252060989aea"}]: dispatch 2024-04-21T08:38:42.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:42 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/121871947' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b97aa94e-52aa-4c7c-8a91-252060989aea"}]': finished 2024-04-21T08:38:42.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:42 smithi106 ceph-mon[31072]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T08:38:42.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:42 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:38:42.726 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:42 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:42.504+0000 7f98aa8c0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:38:42.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:42 smithi134 ceph-mon[31772]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T08:38:42.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:42 smithi134 ceph-mon[31772]: Reconfiguring daemon mgr.x on smithi082 2024-04-21T08:38:42.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:42 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/121871947' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b97aa94e-52aa-4c7c-8a91-252060989aea"}]: dispatch 2024-04-21T08:38:42.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:42 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/121871947' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b97aa94e-52aa-4c7c-8a91-252060989aea"}]': finished 2024-04-21T08:38:42.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:42 smithi134 ceph-mon[31772]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T08:38:42.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:42 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:38:43.128 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:42 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:42.717+0000 7f8acaa6d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:38:43.427 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:43 smithi106 ceph-mon[31072]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:43.428 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:43 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/4143427540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:38:43.428 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:43 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:43.138+0000 7f8acaa6d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:38:43.428 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:43 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:43.231+0000 7f8acaa6d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:38:43.428 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:43 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:43.428+0000 7f8acaa6d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:38:43.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:43 smithi134 ceph-mon[31772]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:43.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:43 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/4143427540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:38:43.476 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:43 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:43.098+0000 7f98aa8c0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:38:43.476 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:38:43 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:38:43.312+0000 7f98aa8c0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:38:43.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:43 smithi082 ceph-mon[27967]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:43.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:43 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4143427540' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:38:44.263 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:43 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:43.938+0000 7f8acaa6d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:38:44.263 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:44 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:44.027+0000 7f8acaa6d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:38:44.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:44.317+0000 7fb8e0d99700 1 -- 172.21.15.82:0/838849190 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fb8dc0148c0 con 0x7fb8e4103570 2024-04-21T08:38:44.596 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:44 smithi106 ceph-mon[31072]: Standby manager daemon z started 2024-04-21T08:38:44.597 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:44 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.134:0/2727363908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:38:44.597 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:44 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.134:0/2727363908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:44.597 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:44 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.134:0/2727363908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:38:44.597 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:44 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.134:0/2727363908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:44.597 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:44 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:44.263+0000 7f8acaa6d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:38:44.597 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:44 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:44.394+0000 7f8acaa6d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:38:44.597 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:44 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:44.485+0000 7f8acaa6d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:38:44.597 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:44 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:44.597+0000 7f8acaa6d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:44 smithi082 ceph-mon[27967]: Standby manager daemon z started 2024-04-21T08:38:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:44 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.134:0/2727363908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:38:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:44 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.134:0/2727363908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:44 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.134:0/2727363908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:38:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:44 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.134:0/2727363908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:44.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:44 smithi134 ceph-mon[31772]: Standby manager daemon z started 2024-04-21T08:38:44.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:44 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.134:0/2727363908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:38:44.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:44 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.134:0/2727363908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:44.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:44 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.134:0/2727363908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:38:44.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:44 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.134:0/2727363908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:45.314 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:44 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:44.892+0000 7f8acaa6d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:38:45.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:45 smithi082 ceph-mon[27967]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:45.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:45 smithi082 ceph-mon[27967]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-21T08:38:45.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:45 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:38:45.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:45 smithi106 ceph-mon[31072]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:45 smithi106 ceph-mon[31072]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-21T08:38:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:45 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:38:45.628 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:45 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:45.480+0000 7f8acaa6d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:38:45.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:45 smithi134 ceph-mon[31772]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:45 smithi134 ceph-mon[31772]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-21T08:38:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:45 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:38:46.127 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:38:45 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:38:45.688+0000 7f8acaa6d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:38:46.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:46.325+0000 7fb8e0d99700 1 -- 172.21.15.82:0/838849190 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fb8dc0143a0 con 0x7fb8e4103570 2024-04-21T08:38:46.589 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:46 smithi082 ceph-mon[27967]: Standby manager daemon y started 2024-04-21T08:38:46.589 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:46 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.106:0/2223819518' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:38:46.589 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:46 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.106:0/2223819518' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:46.589 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:46 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.106:0/2223819518' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:38:46.589 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:46 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.106:0/2223819518' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:46 smithi106 ceph-mon[31072]: Standby manager daemon y started 2024-04-21T08:38:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:46 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.106:0/2223819518' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:38:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:46 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.106:0/2223819518' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:46 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.106:0/2223819518' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:38:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:46 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.106:0/2223819518' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:46.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:46 smithi134 ceph-mon[31772]: Standby manager daemon y started 2024-04-21T08:38:46.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:46 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.106:0/2223819518' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:38:46.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:46 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.106:0/2223819518' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:46.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:46 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.106:0/2223819518' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:38:46.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:46 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.106:0/2223819518' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:47.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:47 smithi082 ceph-mon[27967]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:47.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:47 smithi082 ceph-mon[27967]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-21T08:38:47.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:47 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:38:47.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:47 smithi106 ceph-mon[31072]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:47.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:47 smithi106 ceph-mon[31072]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-21T08:38:47.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:47 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:38:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:47 smithi134 ceph-mon[31772]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:47 smithi134 ceph-mon[31772]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-21T08:38:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:47 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:38:49.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:48 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:49.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:48 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:49.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:48 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:49.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:48 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:49.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:48 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:49.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:48 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:49.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:48 smithi082 ceph-mon[27967]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:49.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:48 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:49.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:48 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:49.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:48 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:49.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:48 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:49.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:48 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:49.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:48 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:49.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:48 smithi106 ceph-mon[31072]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:48 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:48 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:48 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:48 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:48 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:48 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:48 smithi134 ceph-mon[31772]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:50.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T08:38:50.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:50.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T08:38:50.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T08:38:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:51.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:51 smithi082 ceph-mon[27967]: Deploying daemon osd.0 on smithi082 2024-04-21T08:38:51.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:51 smithi082 ceph-mon[27967]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:51.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:51 smithi134 ceph-mon[31772]: Deploying daemon osd.0 on smithi082 2024-04-21T08:38:51.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:51 smithi134 ceph-mon[31772]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:51.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:51 smithi106 ceph-mon[31072]: Deploying daemon osd.0 on smithi082 2024-04-21T08:38:51.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:51 smithi106 ceph-mon[31072]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:53.532 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:53 smithi082 ceph-mon[27967]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:53.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:53 smithi106 ceph-mon[31072]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:53 smithi134 ceph-mon[31772]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:55.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:55 smithi082 ceph-mon[27967]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:55.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:55 smithi106 ceph-mon[31072]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:55.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:55 smithi134 ceph-mon[31772]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:56.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:56 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:56.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:56 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:56.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:56 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:56.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:56 smithi134 ceph-mon[31772]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:57.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:56 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:57.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:56 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:57.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:56 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:57.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:56 smithi082 ceph-mon[27967]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:57.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:56 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:57.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:56 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:57.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:56 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:57.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:56 smithi106 ceph-mon[31072]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 0 on host 'smithi082' 2024-04-21T08:38:59.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.250+0000 7fb8e0d99700 1 -- 172.21.15.82:0/838849190 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8e4062e80 con 0x7fb8d0039890 2024-04-21T08:38:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.253+0000 7fb8e90f4700 1 -- 172.21.15.82:0/838849190 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb8d0039890 msgr2=0x7fb8d003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.253+0000 7fb8e90f4700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb8d0039890 0x7fb8d003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4102a30 tx=0x7fb8d400b410 comp rx=0 tx=0).stop 2024-04-21T08:38:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.253+0000 7fb8e90f4700 1 -- 172.21.15.82:0/838849190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e4103570 msgr2=0x7fb8e419f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.253+0000 7fb8e90f4700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e4103570 0x7fb8e419f070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc007f10 tx=0x7fb8dc00d3b0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.253+0000 7fb8e90f4700 1 -- 172.21.15.82:0/838849190 shutdown_connections 2024-04-21T08:38:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.253+0000 7fb8e90f4700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb8d0039890 0x7fb8d003bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.253+0000 7fb8e90f4700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e4103570 0x7fb8e419f070 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.253+0000 7fb8e90f4700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8e41019d0 0x7fb8e4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.253+0000 7fb8e90f4700 1 --2- 172.21.15.82:0/838849190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8e4102bd0 0x7fb8e4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.253+0000 7fb8e90f4700 1 -- 172.21.15.82:0/838849190 >> 172.21.15.82:0/838849190 conn(0x7fb8e40fcf90 msgr2=0x7fb8e4105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.254+0000 7fb8e90f4700 1 -- 172.21.15.82:0/838849190 shutdown_connections 2024-04-21T08:38:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:38:59.254+0000 7fb8e90f4700 1 -- 172.21.15.82:0/838849190 wait complete. 2024-04-21T08:38:59.539 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:59 smithi082 ceph-mon[27967]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:59.539 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:59 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:59.539 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:38:59 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:59.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:59 smithi106 ceph-mon[31072]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:59.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:59 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:59.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:38:59 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:59 smithi134 ceph-mon[31772]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:38:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:59 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:38:59 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:38:59.884 DEBUG:teuthology.orchestra.run.smithi082:osd.0> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.0.service 2024-04-21T08:38:59.886 INFO:tasks.cephadm:Deploying osd.1 on smithi106 with /dev/vg_nvme/lv_4... 2024-04-21T08:38:59.886 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T08:39:00.129 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:39:00.521 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:00 smithi106 ceph-mon[31072]: from='osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T08:39:00.521 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:00 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:00.522 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:00 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:00 smithi082 ceph-mon[27967]: from='osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T08:39:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:00 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:00 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:00 smithi134 ceph-mon[31772]: from='osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T08:39:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:00 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:00 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:01.416 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:01 smithi106 ceph-mon[31072]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:39:01.417 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:01 smithi106 ceph-mon[31072]: from='osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T08:39:01.417 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:01 smithi106 ceph-mon[31072]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T08:39:01.417 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:01 smithi106 ceph-mon[31072]: from='osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-21T08:39:01.417 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:01 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:39:01.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:01 smithi082 ceph-mon[27967]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:39:01.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:01 smithi082 ceph-mon[27967]: from='osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T08:39:01.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:01 smithi082 ceph-mon[27967]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T08:39:01.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:01 smithi082 ceph-mon[27967]: from='osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-21T08:39:01.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:01 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:39:01.609 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:39:01 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0[42765]: 2024-04-21T08:39:01.310+0000 7f7bf60b5700 -1 osd.0 0 waiting for initial osdmap 2024-04-21T08:39:01.609 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:39:01 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0[42765]: 2024-04-21T08:39:01.313+0000 7f7bf1ea8700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:39:01.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:01 smithi134 ceph-mon[31772]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:39:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:01 smithi134 ceph-mon[31772]: from='osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T08:39:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:01 smithi134 ceph-mon[31772]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T08:39:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:01 smithi134 ceph-mon[31772]: from='osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-21T08:39:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:01 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:39:01.999 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T08:39:01.999 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T08:39:01.999 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-21T08:39:01.999 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-21T08:39:01.999 INFO:teuthology.orchestra.run.smithi106.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113494 s, 924 MB/s 2024-04-21T08:39:01.999 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 2024-04-21T08:39:02.000 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-21T08:39:02.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:02 smithi082 ceph-mon[27967]: from='osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-21T08:39:02.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:02 smithi082 ceph-mon[27967]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T08:39:02.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:02 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:39:02.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:02 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:39:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:02 smithi106 ceph-mon[31072]: from='osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-21T08:39:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:02 smithi106 ceph-mon[31072]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T08:39:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:02 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:39:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:02 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:39:02.662 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_4 2024-04-21T08:39:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:02 smithi134 ceph-mon[31772]: from='osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-21T08:39:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:02 smithi134 ceph-mon[31772]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T08:39:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:02 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:39:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:02 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:39:02.901 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:39:03.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:03 smithi082 ceph-mon[27967]: purged_snaps scrub starts 2024-04-21T08:39:03.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:03 smithi082 ceph-mon[27967]: purged_snaps scrub ok 2024-04-21T08:39:03.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:03 smithi082 ceph-mon[27967]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:39:03.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:03 smithi082 ceph-mon[27967]: osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221] boot 2024-04-21T08:39:03.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:03 smithi082 ceph-mon[27967]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T08:39:03.455 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:39:03.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:03 smithi106 ceph-mon[31072]: purged_snaps scrub starts 2024-04-21T08:39:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:03 smithi106 ceph-mon[31072]: purged_snaps scrub ok 2024-04-21T08:39:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:03 smithi106 ceph-mon[31072]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:39:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:03 smithi106 ceph-mon[31072]: osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221] boot 2024-04-21T08:39:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:03 smithi106 ceph-mon[31072]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T08:39:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:39:03.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:03 smithi134 ceph-mon[31772]: purged_snaps scrub starts 2024-04-21T08:39:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:03 smithi134 ceph-mon[31772]: purged_snaps scrub ok 2024-04-21T08:39:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:03 smithi134 ceph-mon[31772]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:39:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:03 smithi134 ceph-mon[31772]: osd.0 [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221] boot 2024-04-21T08:39:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:03 smithi134 ceph-mon[31772]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T08:39:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:39:04.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.337+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3298797633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0103540 msgr2=0x7f01f0109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.337+0000 7f01f4f8f700 1 --2- 172.21.15.106:0/3298797633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0103540 0x7f01f0109df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f01e800b0d0 tx=0x7f01e800b3e0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.337+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3298797633 shutdown_connections 2024-04-21T08:39:04.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.337+0000 7f01f4f8f700 1 --2- 172.21.15.106:0/3298797633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0103540 0x7f01f0109df0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.337+0000 7f01f4f8f700 1 --2- 172.21.15.106:0/3298797633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f01019a0 0x7f01f0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.337+0000 7f01f4f8f700 1 --2- 172.21.15.106:0/3298797633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f0102ba0 0x7f01f0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.337+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3298797633 >> 172.21.15.106:0/3298797633 conn(0x7f01f00fcf40 msgr2=0x7f01f00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:04.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.337+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3298797633 shutdown_connections 2024-04-21T08:39:04.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.337+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3298797633 wait complete. 2024-04-21T08:39:04.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.338+0000 7f01f4f8f700 1 Processor -- start 2024-04-21T08:39:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.338+0000 7f01f4f8f700 1 -- start start 2024-04-21T08:39:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.338+0000 7f01f4f8f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f01019a0 0x7f01f006b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.338+0000 7f01f4f8f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f0102ba0 0x7f01f019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.338+0000 7f01f4f8f700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0103540 0x7f01f019ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.338+0000 7f01f4f8f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f01f010d490 con 0x7f01f01019a0 2024-04-21T08:39:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.338+0000 7f01f4f8f700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01f010d310 con 0x7f01f0103540 2024-04-21T08:39:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.338+0000 7f01f4f8f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01f010d610 con 0x7f01f0102ba0 2024-04-21T08:39:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.338+0000 7f01eed9d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0103540 0x7f01f019ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01eed9d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0103540 0x7f01f019ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58598/0 (socket says 172.21.15.106:58598) 2024-04-21T08:39:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01eed9d700 1 -- 172.21.15.106:0/3877380601 learned_addr learned my addr 172.21.15.106:0/3877380601 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-21T08:39:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01ee59c700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f01019a0 0x7f01f006b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01edd9b700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f0102ba0 0x7f01f019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01eed9d700 1 -- 172.21.15.106:0/3877380601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f0102ba0 msgr2=0x7f01f019d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01eed9d700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f0102ba0 0x7f01f019d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01eed9d700 1 -- 172.21.15.106:0/3877380601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f01019a0 msgr2=0x7f01f006b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:04.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01eed9d700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f01019a0 0x7f01f006b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:04.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01eed9d700 1 -- 172.21.15.106:0/3877380601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f01a07e0 con 0x7f01f0103540 2024-04-21T08:39:04.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01ee59c700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f01019a0 0x7f01f006b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:04.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01eed9d700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0103540 0x7f01f019ff10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f01e80079a0 tx=0x7f01e8007a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:04.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01df7fe700 1 -- 172.21.15.106:0/3877380601 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8021070 con 0x7f01f0103540 2024-04-21T08:39:04.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01df7fe700 1 -- 172.21.15.106:0/3877380601 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01e801db00 con 0x7f01f0103540 2024-04-21T08:39:04.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3877380601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f01a0a70 con 0x7f01f0103540 2024-04-21T08:39:04.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01df7fe700 1 -- 172.21.15.106:0/3877380601 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e801bc60 con 0x7f01f0103540 2024-04-21T08:39:04.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.339+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3877380601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f01a0f20 con 0x7f01f0103540 2024-04-21T08:39:04.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.340+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3877380601 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f0069730 con 0x7f01f0103540 2024-04-21T08:39:04.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.341+0000 7f01df7fe700 1 -- 172.21.15.106:0/3877380601 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f01e8024070 con 0x7f01f0103540 2024-04-21T08:39:04.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.342+0000 7f01df7fe700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f01d80a3bb0 0x7f01d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:04.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.342+0000 7f01df7fe700 1 -- 172.21.15.106:0/3877380601 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f01e800f020 con 0x7f01f0103540 2024-04-21T08:39:04.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.342+0000 7f01ee59c700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f01d80a3bb0 0x7f01d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:04.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.343+0000 7f01ee59c700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f01d80a3bb0 0x7f01d80a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01e0005fd0 tx=0x7f01e0005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:04.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.347+0000 7f01df7fe700 1 -- 172.21.15.106:0/3877380601 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01e802b020 con 0x7f01f0103540 2024-04-21T08:39:04.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:04.550+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3877380601 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f01f0062f40 con 0x7f01d80a3bb0 2024-04-21T08:39:04.550 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:04 smithi106 ceph-mon[31072]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T08:39:04.550 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.550 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.550 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:39:04.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:04.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:04.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:04.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:04.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:04.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:04 smithi082 ceph-mon[27967]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T08:39:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:39:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:04 smithi134 ceph-mon[31772]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T08:39:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:39:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:04.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:04.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:05.481 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:05 smithi106 ceph-mon[31072]: Detected new or changed devices on smithi082 2024-04-21T08:39:05.481 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:05 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi082 to 17088M 2024-04-21T08:39:05.481 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:05 smithi106 ceph-mon[31072]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:05.481 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:05 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:39:05.481 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:05 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:39:05.481 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:05 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:05.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:05 smithi082 ceph-mon[27967]: Detected new or changed devices on smithi082 2024-04-21T08:39:05.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:05 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi082 to 17088M 2024-04-21T08:39:05.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:05 smithi082 ceph-mon[27967]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:05.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:39:05.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:39:05.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:05 smithi134 ceph-mon[31772]: Detected new or changed devices on smithi082 2024-04-21T08:39:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:05 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi082 to 17088M 2024-04-21T08:39:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:05 smithi134 ceph-mon[31772]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:05 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:39:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:05 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:39:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:05 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:06.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:06 smithi106 ceph-mon[31072]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:06.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:06 smithi082 ceph-mon[27967]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:06.627 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:06 smithi134 ceph-mon[31772]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:07.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:07 smithi082 ceph-mon[27967]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:07.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:07 smithi082 ceph-mon[27967]: from='client.? 172.21.15.106:0/1570039581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "209098d5-cae2-4947-8cb5-93c4c7fea11d"}]: dispatch 2024-04-21T08:39:07.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:07 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "209098d5-cae2-4947-8cb5-93c4c7fea11d"}]: dispatch 2024-04-21T08:39:07.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:07 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "209098d5-cae2-4947-8cb5-93c4c7fea11d"}]': finished 2024-04-21T08:39:07.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:07 smithi082 ceph-mon[27967]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T08:39:07.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:07 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:07.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:07 smithi106 ceph-mon[31072]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:07.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:07 smithi106 ceph-mon[31072]: from='client.? 172.21.15.106:0/1570039581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "209098d5-cae2-4947-8cb5-93c4c7fea11d"}]: dispatch 2024-04-21T08:39:07.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:07 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "209098d5-cae2-4947-8cb5-93c4c7fea11d"}]: dispatch 2024-04-21T08:39:07.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:07 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "209098d5-cae2-4947-8cb5-93c4c7fea11d"}]': finished 2024-04-21T08:39:07.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:07 smithi106 ceph-mon[31072]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T08:39:07.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:07 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:07.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:07 smithi134 ceph-mon[31772]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:07.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:07 smithi134 ceph-mon[31772]: from='client.? 172.21.15.106:0/1570039581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "209098d5-cae2-4947-8cb5-93c4c7fea11d"}]: dispatch 2024-04-21T08:39:07.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:07 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "209098d5-cae2-4947-8cb5-93c4c7fea11d"}]: dispatch 2024-04-21T08:39:07.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:07 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "209098d5-cae2-4947-8cb5-93c4c7fea11d"}]': finished 2024-04-21T08:39:07.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:07 smithi134 ceph-mon[31772]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T08:39:07.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:07 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:08.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:08 smithi082 ceph-mon[27967]: from='client.? 172.21.15.106:0/1769241973' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:39:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:08 smithi106 ceph-mon[31072]: from='client.? 172.21.15.106:0/1769241973' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:39:08.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:08 smithi134 ceph-mon[31772]: from='client.? 172.21.15.106:0/1769241973' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:39:09.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:09 smithi082 ceph-mon[27967]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:09 smithi106 ceph-mon[31072]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:09.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:09 smithi134 ceph-mon[31772]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:11.375 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:11 smithi106 ceph-mon[31072]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:11.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:11 smithi082 ceph-mon[27967]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:11.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:11 smithi134 ceph-mon[31772]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:13.550 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:13 smithi106 ceph-mon[31072]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:13.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:13 smithi082 ceph-mon[27967]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:13.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:13 smithi134 ceph-mon[31772]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:14.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:14 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T08:39:14.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:14 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:14.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:14 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T08:39:14.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:14 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:14.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:14 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T08:39:14.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:14 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:15 smithi082 ceph-mon[27967]: Deploying daemon osd.1 on smithi106 2024-04-21T08:39:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:15 smithi082 ceph-mon[27967]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:15.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:15 smithi106 ceph-mon[31072]: Deploying daemon osd.1 on smithi106 2024-04-21T08:39:15.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:15 smithi106 ceph-mon[31072]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:15.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:15 smithi134 ceph-mon[31772]: Deploying daemon osd.1 on smithi106 2024-04-21T08:39:15.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:15 smithi134 ceph-mon[31772]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:17 smithi082 ceph-mon[27967]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:17 smithi106 ceph-mon[31072]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:17.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:17 smithi134 ceph-mon[31772]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:19.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:19 smithi106 ceph-mon[31072]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:19.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:19 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:19.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:19 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:19.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:19 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:19.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:19 smithi082 ceph-mon[27967]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:19.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:19 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:19.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:19 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:19.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:19 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:19 smithi134 ceph-mon[31772]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:19 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:19 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:19 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:21 smithi106 ceph-mon[31072]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:21.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:21 smithi134 ceph-mon[31772]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:21.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:21 smithi082 ceph-mon[27967]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:22 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:22 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:22 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:22 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:22 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:22.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:22 smithi106 ceph-mon[31072]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:22.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:22 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:22.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:22 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:22.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:22 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:22.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:22 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:22.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:22 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:22.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:22 smithi134 ceph-mon[31772]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:23.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.010+0000 7f01df7fe700 1 -- 172.21.15.106:0/3877380601 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f01f0062f40 con 0x7f01d80a3bb0 2024-04-21T08:39:23.010 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 1 on host 'smithi106' 2024-04-21T08:39:23.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.012+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3877380601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f01d80a3bb0 msgr2=0x7f01d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.012+0000 7f01f4f8f700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f01d80a3bb0 0x7f01d80a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01e0005fd0 tx=0x7f01e0005e40 comp rx=0 tx=0).stop 2024-04-21T08:39:23.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.013+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3877380601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0103540 msgr2=0x7f01f019ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.013+0000 7f01f4f8f700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0103540 0x7f01f019ff10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f01e80079a0 tx=0x7f01e8007a80 comp rx=0 tx=0).stop 2024-04-21T08:39:23.012 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.013+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3877380601 shutdown_connections 2024-04-21T08:39:23.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.013+0000 7f01f4f8f700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f01d80a3bb0 0x7f01d80a6070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.013+0000 7f01f4f8f700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01f0103540 0x7f01f019ff10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.013+0000 7f01f4f8f700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f01019a0 0x7f01f006b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.013+0000 7f01f4f8f700 1 --2- 172.21.15.106:0/3877380601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01f0102ba0 0x7f01f019d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.013+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3877380601 >> 172.21.15.106:0/3877380601 conn(0x7f01f00fcf40 msgr2=0x7f01f0105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.013+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3877380601 shutdown_connections 2024-04-21T08:39:23.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:39:23.013+0000 7f01f4f8f700 1 -- 172.21.15.106:0/3877380601 wait complete. 2024-04-21T08:39:23.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:22 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:23.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:22 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:23.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:22 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:23.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:22 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:23.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:22 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:23.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:22 smithi082 ceph-mon[27967]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:23.555 DEBUG:teuthology.orchestra.run.smithi106:osd.1> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.1.service 2024-04-21T08:39:23.557 INFO:tasks.cephadm:Deploying osd.2 on smithi134 with /dev/vg_nvme/lv_4... 2024-04-21T08:39:23.557 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T08:39:23.804 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:39:24.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:24.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:24.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:24.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:23 smithi106 ceph-mon[31072]: from='osd.1 [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:39:24.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:23 smithi106 ceph-mon[31072]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:39:24.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:24.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:24.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:24.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:23 smithi134 ceph-mon[31772]: from='osd.1 [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:39:24.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:23 smithi134 ceph-mon[31772]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:39:24.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:24.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:24.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:24.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:23 smithi082 ceph-mon[27967]: from='osd.1 [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:39:24.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:23 smithi082 ceph-mon[27967]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:39:25.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:24 smithi134 ceph-mon[31772]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T08:39:25.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:24 smithi134 ceph-mon[31772]: from='osd.1 [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:39:25.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:24 smithi134 ceph-mon[31772]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T08:39:25.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:24 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:25.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:24 smithi134 ceph-mon[31772]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:39:25.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:24 smithi134 ceph-mon[31772]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:25.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:24 smithi082 ceph-mon[27967]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T08:39:25.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:24 smithi082 ceph-mon[27967]: from='osd.1 [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:39:25.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:24 smithi082 ceph-mon[27967]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T08:39:25.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:24 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:25.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:24 smithi082 ceph-mon[27967]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:39:25.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:24 smithi082 ceph-mon[27967]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:24 smithi106 ceph-mon[31072]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T08:39:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:24 smithi106 ceph-mon[31072]: from='osd.1 [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:39:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:24 smithi106 ceph-mon[31072]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T08:39:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:24 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:24 smithi106 ceph-mon[31072]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:39:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:24 smithi106 ceph-mon[31072]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:25.378 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:39:24 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1[35654]: 2024-04-21T08:39:24.980+0000 7ff8e94d2700 -1 osd.1 0 waiting for initial osdmap 2024-04-21T08:39:25.379 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:39:24 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1[35654]: 2024-04-21T08:39:24.986+0000 7ff8e52c5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:39:25.672 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T08:39:25.672 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T08:39:25.672 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-21T08:39:25.673 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-21T08:39:25.673 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112253 s, 934 MB/s 2024-04-21T08:39:25.673 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-21T08:39:26.235 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:25 smithi134 ceph-mon[31772]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-21T08:39:26.235 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:25 smithi134 ceph-mon[31772]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T08:39:26.236 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:26.266 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2024-04-21T08:39:26.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:25 smithi106 ceph-mon[31072]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-21T08:39:26.324 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:25 smithi106 ceph-mon[31072]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T08:39:26.324 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:25 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:26.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:25 smithi082 ceph-mon[27967]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-21T08:39:26.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:25 smithi082 ceph-mon[27967]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T08:39:26.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:26.507 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:39:27.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: purged_snaps scrub starts 2024-04-21T08:39:27.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: purged_snaps scrub ok 2024-04-21T08:39:27.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:27.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: osd.1 [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970] boot 2024-04-21T08:39:27.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T08:39:27.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:27.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:27.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: Detected new or changed devices on smithi106 2024-04-21T08:39:27.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:27.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:27.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:39:27.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:27.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:27.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:27.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:27.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: purged_snaps scrub starts 2024-04-21T08:39:27.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: purged_snaps scrub ok 2024-04-21T08:39:27.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:27.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: osd.1 [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970] boot 2024-04-21T08:39:27.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T08:39:27.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:27.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:27.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: Detected new or changed devices on smithi106 2024-04-21T08:39:27.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:27.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:27.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:39:27.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:27.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:27.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:27.360 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:27.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: purged_snaps scrub starts 2024-04-21T08:39:27.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: purged_snaps scrub ok 2024-04-21T08:39:27.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:27.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: osd.1 [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970] boot 2024-04-21T08:39:27.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T08:39:27.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:39:27.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:39:27.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: Detected new or changed devices on smithi106 2024-04-21T08:39:27.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:27.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:27.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:39:27.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:27.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:27.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:27.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:28.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.139+0000 7fb934dd0700 1 -- 172.21.15.134:0/2859448548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb930102ba0 msgr2=0x7fb930103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:28.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.139+0000 7fb934dd0700 1 --2- 172.21.15.134:0/2859448548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb930102ba0 0x7fb930103000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb918009990 tx=0x7fb918009ca0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.139+0000 7fb934dd0700 1 -- 172.21.15.134:0/2859448548 shutdown_connections 2024-04-21T08:39:28.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.139+0000 7fb934dd0700 1 --2- 172.21.15.134:0/2859448548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930103540 0x7fb930109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.139+0000 7fb934dd0700 1 --2- 172.21.15.134:0/2859448548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9301019a0 0x7fb930101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.139+0000 7fb934dd0700 1 --2- 172.21.15.134:0/2859448548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb930102ba0 0x7fb930103000 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.139+0000 7fb934dd0700 1 -- 172.21.15.134:0/2859448548 >> 172.21.15.134:0/2859448548 conn(0x7fb9300fcf40 msgr2=0x7fb9300ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:28.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:27 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi106 to 17088M 2024-04-21T08:39:28.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:27 smithi134 ceph-mon[31772]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T08:39:28.141 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.139+0000 7fb934dd0700 1 -- 172.21.15.134:0/2859448548 shutdown_connections 2024-04-21T08:39:28.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.139+0000 7fb934dd0700 1 -- 172.21.15.134:0/2859448548 wait complete. 2024-04-21T08:39:28.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.140+0000 7fb934dd0700 1 Processor -- start 2024-04-21T08:39:28.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.140+0000 7fb934dd0700 1 -- start start 2024-04-21T08:39:28.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb934dd0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9301019a0 0x7fb93006b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb934dd0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102ba0 0x7fb93019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb934dd0700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb930103540 0x7fb93019ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb934dd0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb93010d430 con 0x7fb930102ba0 2024-04-21T08:39:28.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb934dd0700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb93010d2b0 con 0x7fb930103540 2024-04-21T08:39:28.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb934dd0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb93010d5b0 con 0x7fb9301019a0 2024-04-21T08:39:28.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb92e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9301019a0 0x7fb93006b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb925d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102ba0 0x7fb93019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.142 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb92e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9301019a0 0x7fb93006b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:35242/0 (socket says 172.21.15.134:35242) 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb92e59c700 1 -- 172.21.15.134:0/2314085661 learned_addr learned my addr 172.21.15.134:0/2314085661 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb92ed9d700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb930103540 0x7fb93019ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb92e59c700 1 -- 172.21.15.134:0/2314085661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb930103540 msgr2=0x7fb93019ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb92e59c700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb930103540 0x7fb93019ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb92e59c700 1 -- 172.21.15.134:0/2314085661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102ba0 msgr2=0x7fb93019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb92e59c700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102ba0 0x7fb93019d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb92e59c700 1 -- 172.21.15.134:0/2314085661 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9301a08b0 con 0x7fb9301019a0 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb925d9b700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102ba0 0x7fb93019d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.141+0000 7fb92e59c700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9301019a0 0x7fb93006b290 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb92000d980 tx=0x7fb92000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.142+0000 7fb927fff700 1 -- 172.21.15.134:0/2314085661 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920017070 con 0x7fb9301019a0 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.142+0000 7fb927fff700 1 -- 172.21.15.134:0/2314085661 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb92000fd40 con 0x7fb9301019a0 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.142+0000 7fb934dd0700 1 -- 172.21.15.134:0/2314085661 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9301a0ba0 con 0x7fb9301019a0 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.142+0000 7fb927fff700 1 -- 172.21.15.134:0/2314085661 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92000e3b0 con 0x7fb9301019a0 2024-04-21T08:39:28.143 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.142+0000 7fb934dd0700 1 -- 172.21.15.134:0/2314085661 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9301a1090 con 0x7fb9301019a0 2024-04-21T08:39:28.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.143+0000 7fb934dd0700 1 -- 172.21.15.134:0/2314085661 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb930069730 con 0x7fb9301019a0 2024-04-21T08:39:28.144 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.144+0000 7fb927fff700 1 -- 172.21.15.134:0/2314085661 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fb92000f890 con 0x7fb9301019a0 2024-04-21T08:39:28.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.144+0000 7fb927fff700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:28.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.144+0000 7fb927fff700 1 -- 172.21.15.134:0/2314085661 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fb9200ce600 con 0x7fb9301019a0 2024-04-21T08:39:28.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.145+0000 7fb925d9b700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:28.145 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.145+0000 7fb925d9b700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb91800b5c0 tx=0x7fb918005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:28.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.149+0000 7fb927fff700 1 -- 172.21.15.134:0/2314085661 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb92009c830 con 0x7fb9301019a0 2024-04-21T08:39:28.355 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:28.355+0000 7fb934dd0700 1 -- 172.21.15.134:0/2314085661 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb930062f40 con 0x7fb91c0a3bb0 2024-04-21T08:39:28.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:27 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi106 to 17088M 2024-04-21T08:39:28.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:27 smithi082 ceph-mon[27967]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T08:39:28.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi106 to 17088M 2024-04-21T08:39:28.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:27 smithi106 ceph-mon[31072]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T08:39:29.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:29 smithi134 ceph-mon[31772]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:29.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:29 smithi134 ceph-mon[31772]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:29.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:29 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:39:29.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:29 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:39:29.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:29 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:29.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:29 smithi082 ceph-mon[27967]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:29.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:29 smithi082 ceph-mon[27967]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:29.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:39:29.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:39:29.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:29.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:29 smithi106 ceph-mon[31072]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:29.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:29 smithi106 ceph-mon[31072]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:29.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:29 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:39:29.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:29 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:39:29.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:29 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:31 smithi082 ceph-mon[27967]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:31 smithi082 ceph-mon[27967]: from='client.? 172.21.15.134:0/1231328194' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dceeca7d-3295-4c47-93f3-694da1b97eff"}]: dispatch 2024-04-21T08:39:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:31 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dceeca7d-3295-4c47-93f3-694da1b97eff"}]: dispatch 2024-04-21T08:39:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:31 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dceeca7d-3295-4c47-93f3-694da1b97eff"}]': finished 2024-04-21T08:39:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:31 smithi082 ceph-mon[27967]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T08:39:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:31.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:31 smithi106 ceph-mon[31072]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:31.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:31 smithi106 ceph-mon[31072]: from='client.? 172.21.15.134:0/1231328194' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dceeca7d-3295-4c47-93f3-694da1b97eff"}]: dispatch 2024-04-21T08:39:31.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:31 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dceeca7d-3295-4c47-93f3-694da1b97eff"}]: dispatch 2024-04-21T08:39:31.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:31 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dceeca7d-3295-4c47-93f3-694da1b97eff"}]': finished 2024-04-21T08:39:31.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:31 smithi106 ceph-mon[31072]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T08:39:31.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:31 smithi134 ceph-mon[31772]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:31 smithi134 ceph-mon[31772]: from='client.? 172.21.15.134:0/1231328194' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dceeca7d-3295-4c47-93f3-694da1b97eff"}]: dispatch 2024-04-21T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:31 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dceeca7d-3295-4c47-93f3-694da1b97eff"}]: dispatch 2024-04-21T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:31 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dceeca7d-3295-4c47-93f3-694da1b97eff"}]': finished 2024-04-21T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:31 smithi134 ceph-mon[31772]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T08:39:31.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:32.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:32 smithi082 ceph-mon[27967]: from='client.? 172.21.15.134:0/3869830580' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:39:32.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:32 smithi106 ceph-mon[31072]: from='client.? 172.21.15.134:0/3869830580' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:39:32.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:32 smithi134 ceph-mon[31772]: from='client.? 172.21.15.134:0/3869830580' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:39:33.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:33 smithi082 ceph-mon[27967]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:33.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:33 smithi106 ceph-mon[31072]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:33.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:33 smithi134 ceph-mon[31772]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:35.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:35 smithi134 ceph-mon[31772]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:35.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:35 smithi082 ceph-mon[27967]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:35.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:35 smithi106 ceph-mon[31072]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:37.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:37 smithi082 ceph-mon[27967]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:37 smithi106 ceph-mon[31072]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:37.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:37 smithi134 ceph-mon[31772]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:38.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:38 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T08:39:38.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:38 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:38.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:38 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T08:39:38.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:38 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:38.652 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:38 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T08:39:38.652 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:38 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:39.523 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:39 smithi134 ceph-mon[31772]: Deploying daemon osd.2 on smithi134 2024-04-21T08:39:39.524 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:39 smithi134 ceph-mon[31772]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:39.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:39 smithi082 ceph-mon[27967]: Deploying daemon osd.2 on smithi134 2024-04-21T08:39:39.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:39 smithi082 ceph-mon[27967]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:39.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:39 smithi106 ceph-mon[31072]: Deploying daemon osd.2 on smithi134 2024-04-21T08:39:39.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:39 smithi106 ceph-mon[31072]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:41.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:41 smithi134 ceph-mon[31772]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:41.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:41 smithi082 ceph-mon[27967]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:41.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:41 smithi106 ceph-mon[31072]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:43.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:43 smithi134 ceph-mon[31772]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:43.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:43 smithi082 ceph-mon[27967]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:43.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:43 smithi106 ceph-mon[31072]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:44 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:44 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:44 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:44 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:44 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:44 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:44.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:44 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:44.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:44 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:44.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:44 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:45.542 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:45 smithi134 ceph-mon[31772]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:45.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:45 smithi082 ceph-mon[27967]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:45 smithi106 ceph-mon[31072]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:46.875 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 2 on host 'smithi134' 2024-04-21T08:39:46.875 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.875+0000 7fb927fff700 1 -- 172.21.15.134:0/2314085661 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb930062f40 con 0x7fb91c0a3bb0 2024-04-21T08:39:46.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.877+0000 7fb934dd0700 1 -- 172.21.15.134:0/2314085661 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb91c0a3bb0 msgr2=0x7fb91c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:46.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.877+0000 7fb934dd0700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb91800b5c0 tx=0x7fb918005fb0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.877+0000 7fb934dd0700 1 -- 172.21.15.134:0/2314085661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9301019a0 msgr2=0x7fb93006b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:46.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.877+0000 7fb934dd0700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9301019a0 0x7fb93006b290 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb92000d980 tx=0x7fb92000dc90 comp rx=0 tx=0).stop 2024-04-21T08:39:46.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.877+0000 7fb934dd0700 1 -- 172.21.15.134:0/2314085661 shutdown_connections 2024-04-21T08:39:46.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.877+0000 7fb934dd0700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.877+0000 7fb934dd0700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb930103540 0x7fb93019ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.877+0000 7fb934dd0700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9301019a0 0x7fb93006b290 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.877+0000 7fb934dd0700 1 --2- 172.21.15.134:0/2314085661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102ba0 0x7fb93019d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.877+0000 7fb934dd0700 1 -- 172.21.15.134:0/2314085661 >> 172.21.15.134:0/2314085661 conn(0x7fb9300fcf40 msgr2=0x7fb930106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:46.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.877+0000 7fb934dd0700 1 -- 172.21.15.134:0/2314085661 shutdown_connections 2024-04-21T08:39:46.878 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:39:46.877+0000 7fb934dd0700 1 -- 172.21.15.134:0/2314085661 wait complete. 2024-04-21T08:39:47.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:47 smithi134 ceph-mon[31772]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:47.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:47 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:47.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:47 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:47.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:47 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:47.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:47 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:47.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:47 smithi134 ceph-mon[31772]: from='osd.2 [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:39:47.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:47 smithi134 ceph-mon[31772]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:39:47.501 DEBUG:teuthology.orchestra.run.smithi134:osd.2> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.2.service 2024-04-21T08:39:47.503 INFO:tasks.cephadm:Deploying osd.3 on smithi082 with /dev/vg_nvme/lv_3... 2024-04-21T08:39:47.503 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T08:39:47.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:47 smithi082 ceph-mon[27967]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:47.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:47 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:47.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:47 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:47.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:47 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:47.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:47 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:47.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:47 smithi082 ceph-mon[27967]: from='osd.2 [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:39:47.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:47 smithi082 ceph-mon[27967]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:39:47.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:47 smithi106 ceph-mon[31072]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:47.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:47 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:47.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:47 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:47.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:47 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:47.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:47 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:47.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:47 smithi106 ceph-mon[31072]: from='osd.2 [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:39:47.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:47 smithi106 ceph-mon[31072]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:39:47.749 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:39:49.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:48 smithi082 ceph-mon[27967]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T08:39:49.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:48 smithi082 ceph-mon[27967]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T08:39:49.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:48 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:49.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:48 smithi082 ceph-mon[27967]: from='osd.2 [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:39:49.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:48 smithi082 ceph-mon[27967]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:39:49.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:48 smithi082 ceph-mon[27967]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:49.127 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:48 smithi106 ceph-mon[31072]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T08:39:49.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:48 smithi106 ceph-mon[31072]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T08:39:49.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:48 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:49.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:48 smithi106 ceph-mon[31072]: from='osd.2 [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:39:49.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:48 smithi106 ceph-mon[31072]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:39:49.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:48 smithi106 ceph-mon[31072]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:48 smithi134 ceph-mon[31772]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T08:39:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:48 smithi134 ceph-mon[31772]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T08:39:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:48 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:48 smithi134 ceph-mon[31772]: from='osd.2 [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:39:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:48 smithi134 ceph-mon[31772]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:39:49.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:48 smithi134 ceph-mon[31772]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:49.227 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:39:48 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2[36939]: 2024-04-21T08:39:48.903+0000 7fbdbe90a700 -1 osd.2 0 waiting for initial osdmap 2024-04-21T08:39:49.227 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:39:48 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2[36939]: 2024-04-21T08:39:48.908+0000 7fbdba6fd700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:39:49.747 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T08:39:49.747 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T08:39:49.747 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-04-21T08:39:49.748 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-04-21T08:39:49.748 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114046 s, 919 MB/s 2024-04-21T08:39:49.748 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-04-21T08:39:49.975 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:49 smithi134 ceph-mon[31772]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-21T08:39:49.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:49 smithi134 ceph-mon[31772]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T08:39:49.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:49 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:49.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:49 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:50.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:49 smithi082 ceph-mon[27967]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-21T08:39:50.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:49 smithi082 ceph-mon[27967]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T08:39:50.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:49 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:50.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:49 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:50.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:49 smithi106 ceph-mon[31072]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-21T08:39:50.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:49 smithi106 ceph-mon[31072]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T08:39:50.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:49 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:50.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:49 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:50.447 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_3 2024-04-21T08:39:50.693 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:39:51.012 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: purged_snaps scrub starts 2024-04-21T08:39:51.012 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: purged_snaps scrub ok 2024-04-21T08:39:51.012 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: osd.2 [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351] boot 2024-04-21T08:39:51.012 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T08:39:51.012 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:51.012 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: Detected new or changed devices on smithi134 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi134 to 17088M 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:51.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: purged_snaps scrub starts 2024-04-21T08:39:51.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: purged_snaps scrub ok 2024-04-21T08:39:51.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: osd.2 [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351] boot 2024-04-21T08:39:51.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T08:39:51.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:51.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:51.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:39:51.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: Detected new or changed devices on smithi134 2024-04-21T08:39:51.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:39:51.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi134 to 17088M 2024-04-21T08:39:51.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:51.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:51.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:51.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:51.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:51.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: purged_snaps scrub starts 2024-04-21T08:39:51.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: purged_snaps scrub ok 2024-04-21T08:39:51.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: osd.2 [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351] boot 2024-04-21T08:39:51.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T08:39:51.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:39:51.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:39:51.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:39:51.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: Detected new or changed devices on smithi134 2024-04-21T08:39:51.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:39:51.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi134 to 17088M 2024-04-21T08:39:51.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:51.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:51.380 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:51.380 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:51.380 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:51.380 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:51.380 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:52.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:51 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T08:39:52.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:51 smithi082 ceph-mon[27967]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T08:39:52.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:51 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:39:52.108 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47132]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:39:52.109 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47132]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:39:52.109 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47132]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:39:52.109 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47132]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:39:52.109 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47135]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:39:52.109 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47135]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:39:52.109 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47135]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:39:52.109 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47135]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:39:52.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:51 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T08:39:52.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:51 smithi134 ceph-mon[31772]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T08:39:52.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:51 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:39:52.178 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40545]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:39:52.178 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40545]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:39:52.178 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40545]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:39:52.178 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40545]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:39:52.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:51 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T08:39:52.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:51 smithi106 ceph-mon[31072]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T08:39:52.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:51 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:39:52.295 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39506]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:39:52.295 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39506]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:39:52.295 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39506]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:39:52.295 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39506]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:39:52.295 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39509]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:39:52.296 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39509]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:39:52.296 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39509]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:39:52.296 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39509]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:39:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.399+0000 7f0c80296700 1 -- 172.21.15.82:0/1429690169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78103ca0 msgr2=0x7f0c78104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.399+0000 7f0c80296700 1 --2- 172.21.15.82:0/1429690169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78103ca0 0x7f0c78104100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0c680099c0 tx=0x7f0c68009cd0 comp rx=0 tx=0).stop 2024-04-21T08:39:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.399+0000 7f0c80296700 1 -- 172.21.15.82:0/1429690169 shutdown_connections 2024-04-21T08:39:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.399+0000 7f0c80296700 1 --2- 172.21.15.82:0/1429690169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c78104640 0x7f0c7810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.399+0000 7f0c80296700 1 --2- 172.21.15.82:0/1429690169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c78102aa0 0x7f0c78102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.399+0000 7f0c80296700 1 --2- 172.21.15.82:0/1429690169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78103ca0 0x7f0c78104100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.399+0000 7f0c80296700 1 -- 172.21.15.82:0/1429690169 >> 172.21.15.82:0/1429690169 conn(0x7f0c780fe040 msgr2=0x7f0c78100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.399+0000 7f0c80296700 1 -- 172.21.15.82:0/1429690169 shutdown_connections 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.399+0000 7f0c80296700 1 -- 172.21.15.82:0/1429690169 wait complete. 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.399+0000 7f0c80296700 1 Processor -- start 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.399+0000 7f0c80296700 1 -- start start 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c80296700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c78102aa0 0x7f0c78198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c80296700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c78103ca0 0x7f0c78199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c80296700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78104640 0x7f0c781a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c80296700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7810e540 con 0x7f0c78104640 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c80296700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7810e3c0 con 0x7f0c78103ca0 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c80296700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7810e6c0 con 0x7f0c78102aa0 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c7e833700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78104640 0x7f0c781a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c7e833700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78104640 0x7f0c781a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40264/0 (socket says 172.21.15.82:40264) 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c7e833700 1 -- 172.21.15.82:0/2593820938 learned_addr learned my addr 172.21.15.82:0/2593820938 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:39:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c7d831700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c78103ca0 0x7f0c78199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c7e032700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c78102aa0 0x7f0c78198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c7e833700 1 -- 172.21.15.82:0/2593820938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c78102aa0 msgr2=0x7f0c78198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c7e833700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c78102aa0 0x7f0c78198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c7e833700 1 -- 172.21.15.82:0/2593820938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c78103ca0 msgr2=0x7f0c78199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c7e833700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c78103ca0 0x7f0c78199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c7e833700 1 -- 172.21.15.82:0/2593820938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c781a0880 con 0x7f0c78104640 2024-04-21T08:39:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c7e833700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78104640 0x7f0c781a01b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000ef30 tx=0x7f0c7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c6f7fe700 1 -- 172.21.15.82:0/2593820938 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c700092e0 con 0x7f0c78104640 2024-04-21T08:39:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c6f7fe700 1 -- 172.21.15.82:0/2593820938 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c70004bb0 con 0x7f0c78104640 2024-04-21T08:39:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c6f7fe700 1 -- 172.21.15.82:0/2593820938 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70004020 con 0x7f0c78104640 2024-04-21T08:39:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.400+0000 7f0c80296700 1 -- 172.21.15.82:0/2593820938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c781a0b10 con 0x7f0c78104640 2024-04-21T08:39:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.401+0000 7f0c80296700 1 -- 172.21.15.82:0/2593820938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c78074c70 con 0x7f0c78104640 2024-04-21T08:39:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.402+0000 7f0c80296700 1 -- 172.21.15.82:0/2593820938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c78069730 con 0x7f0c78104640 2024-04-21T08:39:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.403+0000 7f0c6f7fe700 1 -- 172.21.15.82:0/2593820938 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f0c7001b030 con 0x7f0c78104640 2024-04-21T08:39:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.403+0000 7f0c6f7fe700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0c640a8040 0x7f0c640aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:52.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.403+0000 7f0c7e032700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0c640a8040 0x7f0c640aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:52.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.404+0000 7f0c6f7fe700 1 -- 172.21.15.82:0/2593820938 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f0c700cf2f0 con 0x7f0c78104640 2024-04-21T08:39:52.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.404+0000 7f0c7e032700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0c640a8040 0x7f0c640aa500 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c7819a110 tx=0x7f0c74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:52.405 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47143]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:39:52.405 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47143]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:39:52.405 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47143]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:39:52.405 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47143]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:39:52.405 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47146]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:39:52.405 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47146]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:39:52.405 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47146]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:39:52.405 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 sudo[47146]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:39:52.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.407+0000 7f0c6f7fe700 1 -- 172.21.15.82:0/2593820938 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c70022020 con 0x7f0c78104640 2024-04-21T08:39:52.476 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40548]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:39:52.476 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40548]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:39:52.476 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40548]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:39:52.476 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40548]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:39:52.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40551]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:39:52.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40551]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:39:52.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40551]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:39:52.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40551]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:39:52.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40554]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:39:52.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40554]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:39:52.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40554]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:39:52.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 sudo[40554]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:39:52.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:52.619+0000 7f0c80296700 1 -- 172.21.15.82:0/2593820938 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0c78062f40 con 0x7f0c640a8040 2024-04-21T08:39:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39512]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:39:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39512]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:39:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39512]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:39:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39512]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:39:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39515]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:39:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39515]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:39:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39515]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:39:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 sudo[39515]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:39:53.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:39:53.222+0000 7f0c6f7fe700 1 -- 172.21.15.82:0/2593820938 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0c70012050 con 0x7f0c78104640 2024-04-21T08:39:53.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:39:53.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T08:39:53.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:53.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:53.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:53.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:39:53.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:39:53.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:39:53.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:39:53.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:39:53.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:39:53.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:39:53.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:39:53.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:39:53.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: pgmap v83: 1 pgs: 1 active+clean; 321 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:39:53.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:39:53.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:39:53.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:39:53.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:39:53.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:39:53.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:39:53.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:39:53.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:39:53.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:53.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:39:53.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T08:39:53.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:53.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:53.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:53.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:39:53.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:39:53.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:39:53.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:39:53.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:39:53.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:39:53.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:39:53.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:39:53.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:39:53.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: pgmap v83: 1 pgs: 1 active+clean; 321 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:39:53.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:39:53.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:39:53.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:39:53.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:39:53.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:39:53.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:39:53.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:39:53.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:39:53.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:53.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:39:53.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T08:39:53.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:53.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:53.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: pgmap v83: 1 pgs: 1 active+clean; 321 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:39:53.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:39:53.380 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:39:53.380 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:39:53.380 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:39:53.380 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:39:53.380 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:54.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:53 smithi082 ceph-mon[27967]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:54.225 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:53 smithi082 ceph-mon[27967]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T08:39:54.225 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:53 smithi082 ceph-mon[27967]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T08:39:54.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:53 smithi134 ceph-mon[31772]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:54.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:53 smithi134 ceph-mon[31772]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T08:39:54.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:53 smithi134 ceph-mon[31772]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T08:39:54.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:53 smithi106 ceph-mon[31072]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:39:54.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:53 smithi106 ceph-mon[31072]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T08:39:54.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:53 smithi106 ceph-mon[31072]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T08:39:55.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:54 smithi082 ceph-mon[27967]: pgmap v85: 1 pgs: 1 active+clean; 321 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:39:55.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:54 smithi134 ceph-mon[31772]: pgmap v85: 1 pgs: 1 active+clean; 321 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:39:55.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:54 smithi106 ceph-mon[31072]: pgmap v85: 1 pgs: 1 active+clean; 321 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:39:56.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:55 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1545954160' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db05ff77-7134-4823-a2ed-5f55bbc5d66c"}]: dispatch 2024-04-21T08:39:56.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:55 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1545954160' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db05ff77-7134-4823-a2ed-5f55bbc5d66c"}]': finished 2024-04-21T08:39:56.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:55 smithi082 ceph-mon[27967]: osdmap e22: 4 total, 3 up, 4 in 2024-04-21T08:39:56.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:55 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:39:56.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:55 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1545954160' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db05ff77-7134-4823-a2ed-5f55bbc5d66c"}]: dispatch 2024-04-21T08:39:56.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:55 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1545954160' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db05ff77-7134-4823-a2ed-5f55bbc5d66c"}]': finished 2024-04-21T08:39:56.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:55 smithi134 ceph-mon[31772]: osdmap e22: 4 total, 3 up, 4 in 2024-04-21T08:39:56.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:55 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:39:56.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:55 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1545954160' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db05ff77-7134-4823-a2ed-5f55bbc5d66c"}]: dispatch 2024-04-21T08:39:56.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:55 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1545954160' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db05ff77-7134-4823-a2ed-5f55bbc5d66c"}]': finished 2024-04-21T08:39:56.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:55 smithi106 ceph-mon[31072]: osdmap e22: 4 total, 3 up, 4 in 2024-04-21T08:39:56.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:55 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:39:57.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:56 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2288852091' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:39:57.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:56 smithi134 ceph-mon[31772]: pgmap v87: 1 pgs: 1 active+clean; 321 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:39:57.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:56 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2288852091' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:39:57.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:56 smithi082 ceph-mon[27967]: pgmap v87: 1 pgs: 1 active+clean; 321 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:39:57.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:56 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2288852091' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:39:57.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:56 smithi106 ceph-mon[31072]: pgmap v87: 1 pgs: 1 active+clean; 321 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:39:59.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:39:59 smithi082 ceph-mon[27967]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 392 KiB/s wr, 29 op/s 2024-04-21T08:39:59.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:39:59 smithi106 ceph-mon[31072]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 392 KiB/s wr, 29 op/s 2024-04-21T08:39:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:39:59 smithi134 ceph-mon[31772]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 392 KiB/s wr, 29 op/s 2024-04-21T08:40:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:00 smithi082 ceph-mon[27967]: overall HEALTH_OK 2024-04-21T08:40:00.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:00 smithi106 ceph-mon[31072]: overall HEALTH_OK 2024-04-21T08:40:00.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:00 smithi134 ceph-mon[31772]: overall HEALTH_OK 2024-04-21T08:40:01.520 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:01 smithi082 ceph-mon[27967]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 7.7 KiB/s rd, 294 KiB/s wr, 22 op/s 2024-04-21T08:40:01.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:01 smithi106 ceph-mon[31072]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 7.7 KiB/s rd, 294 KiB/s wr, 22 op/s 2024-04-21T08:40:01.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:01 smithi134 ceph-mon[31772]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 7.7 KiB/s rd, 294 KiB/s wr, 22 op/s 2024-04-21T08:40:03.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:03 smithi082 ceph-mon[27967]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 6.6 KiB/s rd, 252 KiB/s wr, 19 op/s 2024-04-21T08:40:03.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T08:40:03.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:03.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:03 smithi106 ceph-mon[31072]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 6.6 KiB/s rd, 252 KiB/s wr, 19 op/s 2024-04-21T08:40:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T08:40:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:03 smithi134 ceph-mon[31772]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 6.6 KiB/s rd, 252 KiB/s wr, 19 op/s 2024-04-21T08:40:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T08:40:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:04 smithi082 ceph-mon[27967]: Deploying daemon osd.3 on smithi082 2024-04-21T08:40:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:04 smithi106 ceph-mon[31072]: Deploying daemon osd.3 on smithi082 2024-04-21T08:40:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:04 smithi134 ceph-mon[31772]: Deploying daemon osd.3 on smithi082 2024-04-21T08:40:05.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:05 smithi082 ceph-mon[27967]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 6.2 KiB/s rd, 235 KiB/s wr, 17 op/s 2024-04-21T08:40:05.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:05 smithi106 ceph-mon[31072]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 6.2 KiB/s rd, 235 KiB/s wr, 17 op/s 2024-04-21T08:40:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:05 smithi134 ceph-mon[31772]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 6.2 KiB/s rd, 235 KiB/s wr, 17 op/s 2024-04-21T08:40:07.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:07 smithi082 ceph-mon[27967]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 5.7 KiB/s rd, 217 KiB/s wr, 16 op/s 2024-04-21T08:40:07.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:07 smithi106 ceph-mon[31072]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 5.7 KiB/s rd, 217 KiB/s wr, 16 op/s 2024-04-21T08:40:07.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:07 smithi134 ceph-mon[31772]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 5.7 KiB/s rd, 217 KiB/s wr, 16 op/s 2024-04-21T08:40:09.568 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:09 smithi082 ceph-mon[27967]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 196 KiB/s wr, 14 op/s 2024-04-21T08:40:09.568 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:09 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:09.568 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:09 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:09.568 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:09 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:09 smithi106 ceph-mon[31072]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 196 KiB/s wr, 14 op/s 2024-04-21T08:40:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:09 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:09 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:09 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:09 smithi134 ceph-mon[31772]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 196 KiB/s wr, 14 op/s 2024-04-21T08:40:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:09 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:09 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:09 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:11.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:11 smithi082 ceph-mon[27967]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:40:11.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:11 smithi106 ceph-mon[31072]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:40:11.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:11 smithi134 ceph-mon[31772]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:40:12.180 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 3 on host 'smithi082' 2024-04-21T08:40:12.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.179+0000 7f0c6f7fe700 1 -- 172.21.15.82:0/2593820938 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0c78062f40 con 0x7f0c640a8040 2024-04-21T08:40:12.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.182+0000 7f0c80296700 1 -- 172.21.15.82:0/2593820938 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0c640a8040 msgr2=0x7f0c640aa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:12.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.182+0000 7f0c80296700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0c640a8040 0x7f0c640aa500 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c7819a110 tx=0x7f0c74009260 comp rx=0 tx=0).stop 2024-04-21T08:40:12.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.182+0000 7f0c80296700 1 -- 172.21.15.82:0/2593820938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78104640 msgr2=0x7f0c781a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:12.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.182+0000 7f0c80296700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78104640 0x7f0c781a01b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000ef30 tx=0x7f0c7000c3a0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.182+0000 7f0c80296700 1 -- 172.21.15.82:0/2593820938 shutdown_connections 2024-04-21T08:40:12.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.182+0000 7f0c80296700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0c640a8040 0x7f0c640aa500 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.182+0000 7f0c80296700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78104640 0x7f0c781a01b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.182+0000 7f0c80296700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c78102aa0 0x7f0c78198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.182+0000 7f0c80296700 1 --2- 172.21.15.82:0/2593820938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c78103ca0 0x7f0c78199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.182+0000 7f0c80296700 1 -- 172.21.15.82:0/2593820938 >> 172.21.15.82:0/2593820938 conn(0x7f0c780fe040 msgr2=0x7f0c78106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:12.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.182+0000 7f0c80296700 1 -- 172.21.15.82:0/2593820938 shutdown_connections 2024-04-21T08:40:12.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:40:12.182+0000 7f0c80296700 1 -- 172.21.15.82:0/2593820938 wait complete. 2024-04-21T08:40:12.845 DEBUG:teuthology.orchestra.run.smithi082:osd.3> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.3.service 2024-04-21T08:40:12.848 INFO:tasks.cephadm:Deploying osd.4 on smithi106 with /dev/vg_nvme/lv_3... 2024-04-21T08:40:12.848 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T08:40:13.095 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:40:13.449 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:13 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:13.449 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:13 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:13.449 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:13 smithi106 ceph-mon[31072]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:40:13.449 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:13 smithi106 ceph-mon[31072]: from='osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T08:40:13.449 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:13 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:13.449 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:13 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:13.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:13 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:13.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:13 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:13.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:13 smithi134 ceph-mon[31772]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:40:13.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:13 smithi134 ceph-mon[31772]: from='osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T08:40:13.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:13 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:13.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:13 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:13.525 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:13 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:13.525 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:13 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:13.525 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:13 smithi082 ceph-mon[27967]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:40:13.525 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:13 smithi082 ceph-mon[27967]: from='osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T08:40:13.525 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:13 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:13.525 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:13 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:14.472 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:14 smithi106 ceph-mon[31072]: from='osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T08:40:14.472 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:14 smithi106 ceph-mon[31072]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T08:40:14.472 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:14 smithi106 ceph-mon[31072]: from='osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-21T08:40:14.472 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:14 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:40:14.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:14 smithi134 ceph-mon[31772]: from='osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T08:40:14.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:14 smithi134 ceph-mon[31772]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T08:40:14.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:14 smithi134 ceph-mon[31772]: from='osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-21T08:40:14.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:14 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:40:14.525 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:14 smithi082 ceph-mon[27967]: from='osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T08:40:14.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:14 smithi082 ceph-mon[27967]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T08:40:14.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:14 smithi082 ceph-mon[27967]: from='osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-21T08:40:14.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:14 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:40:14.526 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:40:14 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3[49419]: 2024-04-21T08:40:14.164+0000 7f813e5f2700 -1 osd.3 0 waiting for initial osdmap 2024-04-21T08:40:14.526 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:40:14 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3[49419]: 2024-04-21T08:40:14.169+0000 7f813a3e5700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:40:15.035 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T08:40:15.035 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T08:40:15.035 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-21T08:40:15.035 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-21T08:40:15.035 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135812 s, 772 MB/s 2024-04-21T08:40:15.035 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-21T08:40:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:15 smithi134 ceph-mon[31772]: from='osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-21T08:40:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:15 smithi134 ceph-mon[31772]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T08:40:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:40:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:40:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:15 smithi134 ceph-mon[31772]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:40:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:15 smithi134 ceph-mon[31772]: osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685] boot 2024-04-21T08:40:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:15 smithi134 ceph-mon[31772]: osdmap e25: 4 total, 4 up, 4 in 2024-04-21T08:40:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:40:15.507 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:15 smithi106 ceph-mon[31072]: from='osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-21T08:40:15.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:15 smithi106 ceph-mon[31072]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T08:40:15.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:40:15.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:40:15.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:15 smithi106 ceph-mon[31072]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:40:15.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:15 smithi106 ceph-mon[31072]: osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685] boot 2024-04-21T08:40:15.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:15 smithi106 ceph-mon[31072]: osdmap e25: 4 total, 4 up, 4 in 2024-04-21T08:40:15.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:40:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:15 smithi082 ceph-mon[27967]: from='osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-21T08:40:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:15 smithi082 ceph-mon[27967]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T08:40:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:40:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:40:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:15 smithi082 ceph-mon[27967]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:40:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:15 smithi082 ceph-mon[27967]: osd.3 [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685] boot 2024-04-21T08:40:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:15 smithi082 ceph-mon[27967]: osdmap e25: 4 total, 4 up, 4 in 2024-04-21T08:40:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:40:15.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_3 2024-04-21T08:40:15.895 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:40:16.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:16 smithi106 ceph-mon[31072]: purged_snaps scrub starts 2024-04-21T08:40:16.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:16 smithi106 ceph-mon[31072]: purged_snaps scrub ok 2024-04-21T08:40:16.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:16 smithi106 ceph-mon[31072]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T08:40:16.300 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:16 smithi082 ceph-mon[27967]: purged_snaps scrub starts 2024-04-21T08:40:16.300 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:16 smithi082 ceph-mon[27967]: purged_snaps scrub ok 2024-04-21T08:40:16.300 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:16 smithi082 ceph-mon[27967]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T08:40:16.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:16 smithi134 ceph-mon[31772]: purged_snaps scrub starts 2024-04-21T08:40:16.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:16 smithi134 ceph-mon[31772]: purged_snaps scrub ok 2024-04-21T08:40:16.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:16 smithi134 ceph-mon[31772]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T08:40:17.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.366+0000 7fb459042700 1 -- 172.21.15.106:0/1372112057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb454102b90 msgr2=0x7fb454102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:17.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.366+0000 7fb459042700 1 --2- 172.21.15.106:0/1372112057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb454102b90 0x7fb454102ff0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4480099c0 tx=0x7fb448009cd0 comp rx=0 tx=0).stop 2024-04-21T08:40:17.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.366+0000 7fb459042700 1 -- 172.21.15.106:0/1372112057 shutdown_connections 2024-04-21T08:40:17.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.366+0000 7fb459042700 1 --2- 172.21.15.106:0/1372112057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb454103530 0x7fb454109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:17.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.366+0000 7fb459042700 1 --2- 172.21.15.106:0/1372112057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb454101990 0x7fb454101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.366+0000 7fb459042700 1 --2- 172.21.15.106:0/1372112057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb454102b90 0x7fb454102ff0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.366+0000 7fb459042700 1 -- 172.21.15.106:0/1372112057 >> 172.21.15.106:0/1372112057 conn(0x7fb4540fcf30 msgr2=0x7fb4540ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.366+0000 7fb459042700 1 -- 172.21.15.106:0/1372112057 shutdown_connections 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.366+0000 7fb459042700 1 -- 172.21.15.106:0/1372112057 wait complete. 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.366+0000 7fb459042700 1 Processor -- start 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb459042700 1 -- start start 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb459042700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb454101990 0x7fb454198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb459042700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb454102b90 0x7fb454199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb459042700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb454103530 0x7fb4541a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb459042700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb45410d430 con 0x7fb454101990 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb459042700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb45410d2b0 con 0x7fb454102b90 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb459042700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb45410d5b0 con 0x7fb454103530 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb45259c700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb454102b90 0x7fb454199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:17.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb45259c700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb454102b90 0x7fb454199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59796/0 (socket says 172.21.15.106:59796) 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb45259c700 1 -- 172.21.15.106:0/202519752 learned_addr learned my addr 172.21.15.106:0/202519752 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb452d9d700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb454101990 0x7fb454198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb45359e700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb454103530 0x7fb4541a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb45259c700 1 -- 172.21.15.106:0/202519752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb454103530 msgr2=0x7fb4541a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb45259c700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb454103530 0x7fb4541a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb45259c700 1 -- 172.21.15.106:0/202519752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb454101990 msgr2=0x7fb454198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb45259c700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb454101990 0x7fb454198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb45259c700 1 -- 172.21.15.106:0/202519752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4541a0830 con 0x7fb454102b90 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.367+0000 7fb452d9d700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb454101990 0x7fb454198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.368+0000 7fb45259c700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb454102b90 0x7fb454199110 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb448003790 tx=0x7fb448005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.368+0000 7fb43bfff700 1 -- 172.21.15.106:0/202519752 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448027070 con 0x7fb454102b90 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.368+0000 7fb43bfff700 1 -- 172.21.15.106:0/202519752 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4480040e0 con 0x7fb454102b90 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.368+0000 7fb459042700 1 -- 172.21.15.106:0/202519752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4541a0ac0 con 0x7fb454102b90 2024-04-21T08:40:17.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.368+0000 7fb43bfff700 1 -- 172.21.15.106:0/202519752 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448022d10 con 0x7fb454102b90 2024-04-21T08:40:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.368+0000 7fb459042700 1 -- 172.21.15.106:0/202519752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4541a0fa0 con 0x7fb454102b90 2024-04-21T08:40:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.369+0000 7fb459042700 1 -- 172.21.15.106:0/202519752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb454069730 con 0x7fb454102b90 2024-04-21T08:40:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.370+0000 7fb43bfff700 1 -- 172.21.15.106:0/202519752 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb44802a070 con 0x7fb454102b90 2024-04-21T08:40:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.370+0000 7fb43bfff700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb43c0a3c40 0x7fb43c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.371+0000 7fb43bfff700 1 -- 172.21.15.106:0/202519752 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7fb4480d0010 con 0x7fb454102b90 2024-04-21T08:40:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.371+0000 7fb452d9d700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb43c0a3c40 0x7fb43c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:17.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.371+0000 7fb452d9d700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb43c0a3c40 0x7fb43c0a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb4541029f0 tx=0x7fb44400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:17.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.375+0000 7fb43bfff700 1 -- 172.21.15.106:0/202519752 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb44805eff0 con 0x7fb454102b90 2024-04-21T08:40:17.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:17.584+0000 7fb459042700 1 -- 172.21.15.106:0/202519752 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb4540ff160 con 0x7fb43c0a3c40 2024-04-21T08:40:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: Detected new or changed devices on smithi082 2024-04-21T08:40:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:40:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:40:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi082 to 8544M 2024-04-21T08:40:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:17 smithi082 ceph-mon[27967]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T08:40:17.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: Detected new or changed devices on smithi082 2024-04-21T08:40:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:40:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:40:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi082 to 8544M 2024-04-21T08:40:17.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:17.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:17.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:17.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:17.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:17.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:17 smithi106 ceph-mon[31072]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T08:40:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: Detected new or changed devices on smithi082 2024-04-21T08:40:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:40:17.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:40:17.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi082 to 8544M 2024-04-21T08:40:17.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:17.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:17.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:17.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:17.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:17.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:17.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:17 smithi134 ceph-mon[31772]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T08:40:18.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:18 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:40:18.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:18 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:40:18.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:18 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:18.488 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:18 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:40:18.488 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:18 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:40:18.488 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:18 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:18.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:18 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:40:18.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:18 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:40:18.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:18 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:19 smithi106 ceph-mon[31072]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:40:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:19 smithi106 ceph-mon[31072]: pgmap v103: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-21T08:40:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:19 smithi106 ceph-mon[31072]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-21T08:40:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:19 smithi106 ceph-mon[31072]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T08:40:19.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:19 smithi134 ceph-mon[31772]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:40:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:19 smithi134 ceph-mon[31772]: pgmap v103: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-21T08:40:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:19 smithi134 ceph-mon[31772]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-21T08:40:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:19 smithi134 ceph-mon[31772]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T08:40:19.791 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:19 smithi082 ceph-mon[27967]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:40:19.792 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:19 smithi082 ceph-mon[27967]: pgmap v103: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-21T08:40:19.792 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:19 smithi082 ceph-mon[27967]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-21T08:40:19.792 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:19 smithi082 ceph-mon[27967]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T08:40:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:20 smithi106 ceph-mon[31072]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T08:40:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:20 smithi106 ceph-mon[31072]: from='client.? 172.21.15.106:0/3489335878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56e090a3-ed81-46b8-8c14-a6fdbd57c344"}]: dispatch 2024-04-21T08:40:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:20 smithi106 ceph-mon[31072]: from='client.? 172.21.15.106:0/3489335878' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56e090a3-ed81-46b8-8c14-a6fdbd57c344"}]': finished 2024-04-21T08:40:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:20 smithi106 ceph-mon[31072]: osdmap e30: 5 total, 4 up, 5 in 2024-04-21T08:40:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:20 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:20.720 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:20 smithi082 ceph-mon[27967]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T08:40:20.720 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:20 smithi082 ceph-mon[27967]: from='client.? 172.21.15.106:0/3489335878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56e090a3-ed81-46b8-8c14-a6fdbd57c344"}]: dispatch 2024-04-21T08:40:20.720 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:20 smithi082 ceph-mon[27967]: from='client.? 172.21.15.106:0/3489335878' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56e090a3-ed81-46b8-8c14-a6fdbd57c344"}]': finished 2024-04-21T08:40:20.720 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:20 smithi082 ceph-mon[27967]: osdmap e30: 5 total, 4 up, 5 in 2024-04-21T08:40:20.720 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:20 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:20.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:20 smithi134 ceph-mon[31772]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T08:40:20.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:20 smithi134 ceph-mon[31772]: from='client.? 172.21.15.106:0/3489335878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56e090a3-ed81-46b8-8c14-a6fdbd57c344"}]: dispatch 2024-04-21T08:40:20.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:20 smithi134 ceph-mon[31772]: from='client.? 172.21.15.106:0/3489335878' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56e090a3-ed81-46b8-8c14-a6fdbd57c344"}]': finished 2024-04-21T08:40:20.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:20 smithi134 ceph-mon[31772]: osdmap e30: 5 total, 4 up, 5 in 2024-04-21T08:40:20.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:20 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:21.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:21 smithi106 ceph-mon[31072]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-21T08:40:21.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:21 smithi106 ceph-mon[31072]: from='client.? 172.21.15.106:0/1073574681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:40:21.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:21 smithi134 ceph-mon[31772]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-21T08:40:21.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:21 smithi134 ceph-mon[31772]: from='client.? 172.21.15.106:0/1073574681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:40:21.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:21 smithi082 ceph-mon[27967]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-21T08:40:21.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:21 smithi082 ceph-mon[27967]: from='client.? 172.21.15.106:0/1073574681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:40:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:22 smithi134 ceph-mon[31772]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-21T08:40:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:22 smithi134 ceph-mon[31772]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-21T08:40:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:22 smithi134 ceph-mon[31772]: Cluster is now healthy 2024-04-21T08:40:22.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:22 smithi082 ceph-mon[27967]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-21T08:40:22.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:22 smithi082 ceph-mon[27967]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-21T08:40:22.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:22 smithi082 ceph-mon[27967]: Cluster is now healthy 2024-04-21T08:40:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:22 smithi106 ceph-mon[31072]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-21T08:40:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:22 smithi106 ceph-mon[31072]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-21T08:40:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:22 smithi106 ceph-mon[31072]: Cluster is now healthy 2024-04-21T08:40:25.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:25 smithi082 ceph-mon[27967]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:40:25.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:25 smithi106 ceph-mon[31072]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:40:25.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:25 smithi134 ceph-mon[31772]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:40:27.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:27 smithi082 ceph-mon[27967]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-21T08:40:27.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:27 smithi106 ceph-mon[31072]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-21T08:40:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:27 smithi134 ceph-mon[31772]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-21T08:40:28.506 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T08:40:28.506 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:28.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T08:40:28.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:28.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T08:40:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:29.562 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:29 smithi106 ceph-mon[31072]: Deploying daemon osd.4 on smithi106 2024-04-21T08:40:29.562 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:29 smithi106 ceph-mon[31072]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-21T08:40:29.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:29 smithi082 ceph-mon[27967]: Deploying daemon osd.4 on smithi106 2024-04-21T08:40:29.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:29 smithi082 ceph-mon[27967]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-21T08:40:29.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:29 smithi134 ceph-mon[31772]: Deploying daemon osd.4 on smithi106 2024-04-21T08:40:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:29 smithi134 ceph-mon[31772]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-21T08:40:31.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:31 smithi106 ceph-mon[31072]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-21T08:40:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:31 smithi082 ceph-mon[27967]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-21T08:40:31.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:31 smithi134 ceph-mon[31772]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-21T08:40:33.594 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:33 smithi106 ceph-mon[31072]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-21T08:40:33.594 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:33.594 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:33.594 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:33.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:33 smithi082 ceph-mon[27967]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-21T08:40:33.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:33.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:33.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:33.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:33 smithi134 ceph-mon[31772]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-21T08:40:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:35.540 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:35 smithi106 ceph-mon[31072]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:35.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:35 smithi082 ceph-mon[27967]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:35.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:35 smithi134 ceph-mon[31772]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:36.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.547+0000 7fb43bfff700 1 -- 172.21.15.106:0/202519752 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb4540ff160 con 0x7fb43c0a3c40 2024-04-21T08:40:36.548 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 4 on host 'smithi106' 2024-04-21T08:40:36.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.550+0000 7fb459042700 1 -- 172.21.15.106:0/202519752 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb43c0a3c40 msgr2=0x7fb43c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.550+0000 7fb459042700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb43c0a3c40 0x7fb43c0a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb4541029f0 tx=0x7fb44400b410 comp rx=0 tx=0).stop 2024-04-21T08:40:36.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.550+0000 7fb459042700 1 -- 172.21.15.106:0/202519752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb454102b90 msgr2=0x7fb454199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:36.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.550+0000 7fb459042700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb454102b90 0x7fb454199110 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb448003790 tx=0x7fb448005b00 comp rx=0 tx=0).stop 2024-04-21T08:40:36.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.550+0000 7fb459042700 1 -- 172.21.15.106:0/202519752 shutdown_connections 2024-04-21T08:40:36.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.550+0000 7fb459042700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb43c0a3c40 0x7fb43c0a6100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.550+0000 7fb459042700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb454103530 0x7fb4541a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.550+0000 7fb459042700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb454101990 0x7fb454198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.550+0000 7fb459042700 1 --2- 172.21.15.106:0/202519752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb454102b90 0x7fb454199110 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:36.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.550+0000 7fb459042700 1 -- 172.21.15.106:0/202519752 >> 172.21.15.106:0/202519752 conn(0x7fb4540fcf30 msgr2=0x7fb4540fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:36.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.550+0000 7fb459042700 1 -- 172.21.15.106:0/202519752 shutdown_connections 2024-04-21T08:40:36.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:40:36.550+0000 7fb459042700 1 -- 172.21.15.106:0/202519752 wait complete. 2024-04-21T08:40:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:36 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:36 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:36 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:36 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:36 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:36 smithi106 ceph-mon[31072]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:36 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:36 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:36 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:36.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:36 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:36.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:36 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:36.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:36 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:36.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:36 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:36.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:36 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:36.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:36 smithi134 ceph-mon[31772]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:36.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:36 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:36.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:36 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:36.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:36 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:37.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:36 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:37.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:36 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:37.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:36 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:37.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:36 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:37.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:36 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:37.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:36 smithi082 ceph-mon[27967]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:37.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:36 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:37.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:36 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:37.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:36 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:37.230 DEBUG:teuthology.orchestra.run.smithi106:osd.4> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.4.service 2024-04-21T08:40:37.233 INFO:tasks.cephadm:Deploying osd.5 on smithi134 with /dev/vg_nvme/lv_3... 2024-04-21T08:40:37.233 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T08:40:37.479 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:40:37.777 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:37 smithi106 ceph-mon[31072]: from='osd.4 [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:40:37.777 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:37 smithi106 ceph-mon[31072]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:40:37.872 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:37 smithi134 ceph-mon[31772]: from='osd.4 [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:40:37.872 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:37 smithi134 ceph-mon[31772]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:40:38.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:37 smithi082 ceph-mon[27967]: from='osd.4 [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:40:38.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:37 smithi082 ceph-mon[27967]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:40:38.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:38 smithi134 ceph-mon[31772]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T08:40:38.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:38 smithi134 ceph-mon[31772]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T08:40:38.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:38 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:38.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:38 smithi134 ceph-mon[31772]: from='osd.4 [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:40:38.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:38 smithi134 ceph-mon[31772]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:40:38.810 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:38 smithi134 ceph-mon[31772]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:38 smithi106 ceph-mon[31072]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T08:40:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:38 smithi106 ceph-mon[31072]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T08:40:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:38 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:38 smithi106 ceph-mon[31072]: from='osd.4 [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:40:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:38 smithi106 ceph-mon[31072]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:40:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:38 smithi106 ceph-mon[31072]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:38.879 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:40:38 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4[42330]: 2024-04-21T08:40:38.683+0000 7f2beac08700 -1 osd.4 0 waiting for initial osdmap 2024-04-21T08:40:38.879 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:40:38 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4[42330]: 2024-04-21T08:40:38.691+0000 7f2be69fb700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:40:39.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:38 smithi082 ceph-mon[27967]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T08:40:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:38 smithi082 ceph-mon[27967]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T08:40:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:38 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:38 smithi082 ceph-mon[27967]: from='osd.4 [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:40:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:38 smithi082 ceph-mon[27967]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:40:39.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:38 smithi082 ceph-mon[27967]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:39.359 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T08:40:39.359 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T08:40:39.359 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-21T08:40:39.359 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-21T08:40:39.360 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011291 s, 929 MB/s 2024-04-21T08:40:39.360 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-21T08:40:39.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:39 smithi134 ceph-mon[31772]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-21T08:40:39.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:39 smithi134 ceph-mon[31772]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T08:40:39.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:39 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:39.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:39 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:40.022 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:39 smithi106 ceph-mon[31072]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-21T08:40:40.022 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:39 smithi106 ceph-mon[31072]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T08:40:40.022 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:39 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:40.022 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:39 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:40.033 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2024-04-21T08:40:40.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:39 smithi082 ceph-mon[27967]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-21T08:40:40.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:39 smithi082 ceph-mon[27967]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T08:40:40.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:39 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:40.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:39 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:40.268 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:40:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: purged_snaps scrub starts 2024-04-21T08:40:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: purged_snaps scrub ok 2024-04-21T08:40:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: osd.4 [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110] boot 2024-04-21T08:40:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: osdmap e33: 5 total, 5 up, 5 in 2024-04-21T08:40:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: Detected new or changed devices on smithi106 2024-04-21T08:40:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:40:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:40:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi106 to 8544M 2024-04-21T08:40:40.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:40.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:40.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:40.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:40.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:40 smithi134 ceph-mon[31772]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:41.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: purged_snaps scrub starts 2024-04-21T08:40:41.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: purged_snaps scrub ok 2024-04-21T08:40:41.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: osd.4 [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110] boot 2024-04-21T08:40:41.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: osdmap e33: 5 total, 5 up, 5 in 2024-04-21T08:40:41.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:41.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: Detected new or changed devices on smithi106 2024-04-21T08:40:41.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:41.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:41.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:40:41.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:40:41.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi106 to 8544M 2024-04-21T08:40:41.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:41.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:41.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:41.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:41.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:40 smithi082 ceph-mon[27967]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:41.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: purged_snaps scrub starts 2024-04-21T08:40:41.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: purged_snaps scrub ok 2024-04-21T08:40:41.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: osd.4 [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110] boot 2024-04-21T08:40:41.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: osdmap e33: 5 total, 5 up, 5 in 2024-04-21T08:40:41.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:40:41.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: Detected new or changed devices on smithi106 2024-04-21T08:40:41.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:41.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:41.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:40:41.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:40:41.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi106 to 8544M 2024-04-21T08:40:41.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:41.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:41.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:41.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:41.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:40 smithi106 ceph-mon[31072]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-21T08:40:41.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.827+0000 7f016f51f700 1 -- 172.21.15.134:0/836890565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0168102ba0 msgr2=0x7f0168103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:41.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.827+0000 7f016f51f700 1 --2- 172.21.15.134:0/836890565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0168102ba0 0x7f0168103000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0158009990 tx=0x7f0158009ca0 comp rx=0 tx=0).stop 2024-04-21T08:40:41.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.827+0000 7f016f51f700 1 -- 172.21.15.134:0/836890565 shutdown_connections 2024-04-21T08:40:41.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.827+0000 7f016f51f700 1 --2- 172.21.15.134:0/836890565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0168103540 0x7f0168109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.827+0000 7f016f51f700 1 --2- 172.21.15.134:0/836890565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01681019a0 0x7f0168101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.827+0000 7f016f51f700 1 --2- 172.21.15.134:0/836890565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0168102ba0 0x7f0168103000 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.827+0000 7f016f51f700 1 -- 172.21.15.134:0/836890565 >> 172.21.15.134:0/836890565 conn(0x7f01680fcf40 msgr2=0x7f01680ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.827+0000 7f016f51f700 1 -- 172.21.15.134:0/836890565 shutdown_connections 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.827+0000 7f016f51f700 1 -- 172.21.15.134:0/836890565 wait complete. 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.827+0000 7f016f51f700 1 Processor -- start 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.827+0000 7f016f51f700 1 -- start start 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016f51f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01681019a0 0x7f016806b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016f51f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0168102ba0 0x7f016819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016f51f700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0168103540 0x7f016819fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016f51f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f016810d440 con 0x7f01681019a0 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016f51f700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f016810d2c0 con 0x7f0168103540 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016f51f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f016810d5c0 con 0x7f0168102ba0 2024-04-21T08:40:41.829 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016caba700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0168102ba0 0x7f016819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016caba700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0168102ba0 0x7f016819d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33418/0 (socket says 172.21.15.134:33418) 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016caba700 1 -- 172.21.15.134:0/3167917349 learned_addr learned my addr 172.21.15.134:0/3167917349 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016d2bb700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01681019a0 0x7f016806b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016caba700 1 -- 172.21.15.134:0/3167917349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0168103540 msgr2=0x7f016819fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016caba700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0168103540 0x7f016819fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016caba700 1 -- 172.21.15.134:0/3167917349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01681019a0 msgr2=0x7f016806b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016caba700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01681019a0 0x7f016806b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016caba700 1 -- 172.21.15.134:0/3167917349 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01681a08c0 con 0x7f0168102ba0 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016caba700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0168102ba0 0x7f016819d9b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0158006010 tx=0x7f01580039f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f015e7fc700 1 -- 172.21.15.134:0/3167917349 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0158027070 con 0x7f0168102ba0 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f015e7fc700 1 -- 172.21.15.134:0/3167917349 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0158022470 con 0x7f0168102ba0 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f016f51f700 1 -- 172.21.15.134:0/3167917349 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01681a0b50 con 0x7f0168102ba0 2024-04-21T08:40:41.830 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.828+0000 7f015e7fc700 1 -- 172.21.15.134:0/3167917349 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015800f6c0 con 0x7f0168102ba0 2024-04-21T08:40:41.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.829+0000 7f016f51f700 1 -- 172.21.15.134:0/3167917349 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01681a0fa0 con 0x7f0168102ba0 2024-04-21T08:40:41.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.829+0000 7f016f51f700 1 -- 172.21.15.134:0/3167917349 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016802df80 con 0x7f0168102ba0 2024-04-21T08:40:41.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.830+0000 7f015e7fc700 1 -- 172.21.15.134:0/3167917349 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f015802a070 con 0x7f0168102ba0 2024-04-21T08:40:41.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.831+0000 7f015e7fc700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f01540a3c40 0x7f01540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:41.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.831+0000 7f015e7fc700 1 -- 172.21.15.134:0/3167917349 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f01580d0230 con 0x7f0168102ba0 2024-04-21T08:40:41.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.831+0000 7f016d2bb700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f01540a3c40 0x7f01540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:41.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.832+0000 7f016d2bb700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f01540a3c40 0x7f01540a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0164005fd0 tx=0x7f0164005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:41.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:41.836+0000 7f015e7fc700 1 -- 172.21.15.134:0/3167917349 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0158013070 con 0x7f0168102ba0 2024-04-21T08:40:41.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:41 smithi134 ceph-mon[31772]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T08:40:42.042 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:40:42.043+0000 7f016f51f700 1 -- 172.21.15.134:0/3167917349 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0168062e80 con 0x7f01540a3c40 2024-04-21T08:40:42.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:41 smithi082 ceph-mon[27967]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T08:40:42.127 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:41 smithi106 ceph-mon[31072]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T08:40:42.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:42 smithi134 ceph-mon[31772]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:40:42.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:42 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:40:42.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:42 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:40:42.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:42 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:42.934 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:42 smithi134 ceph-mon[31772]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:42 smithi082 ceph-mon[27967]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:40:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:42 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:40:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:42 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:40:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:42 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:42 smithi082 ceph-mon[27967]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:42 smithi106 ceph-mon[31072]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:40:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:42 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:40:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:42 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:40:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:42 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:42 smithi106 ceph-mon[31072]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:45.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:45 smithi082 ceph-mon[27967]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:45.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:45 smithi082 ceph-mon[27967]: from='client.? 172.21.15.134:0/1767051461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c045cd42-40b5-4b5c-9d69-69a7ec4f0bef"}]: dispatch 2024-04-21T08:40:45.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:45 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c045cd42-40b5-4b5c-9d69-69a7ec4f0bef"}]: dispatch 2024-04-21T08:40:45.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:45 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c045cd42-40b5-4b5c-9d69-69a7ec4f0bef"}]': finished 2024-04-21T08:40:45.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:45 smithi082 ceph-mon[27967]: osdmap e35: 6 total, 5 up, 6 in 2024-04-21T08:40:45.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:45 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:40:45.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:45 smithi082 ceph-mon[27967]: from='client.? 172.21.15.134:0/3576125548' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:40:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:45 smithi106 ceph-mon[31072]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:45 smithi106 ceph-mon[31072]: from='client.? 172.21.15.134:0/1767051461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c045cd42-40b5-4b5c-9d69-69a7ec4f0bef"}]: dispatch 2024-04-21T08:40:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:45 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c045cd42-40b5-4b5c-9d69-69a7ec4f0bef"}]: dispatch 2024-04-21T08:40:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:45 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c045cd42-40b5-4b5c-9d69-69a7ec4f0bef"}]': finished 2024-04-21T08:40:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:45 smithi106 ceph-mon[31072]: osdmap e35: 6 total, 5 up, 6 in 2024-04-21T08:40:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:45 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:40:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:45 smithi106 ceph-mon[31072]: from='client.? 172.21.15.134:0/3576125548' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:40:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:45 smithi134 ceph-mon[31772]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:45 smithi134 ceph-mon[31772]: from='client.? 172.21.15.134:0/1767051461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c045cd42-40b5-4b5c-9d69-69a7ec4f0bef"}]: dispatch 2024-04-21T08:40:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:45 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c045cd42-40b5-4b5c-9d69-69a7ec4f0bef"}]: dispatch 2024-04-21T08:40:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:45 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c045cd42-40b5-4b5c-9d69-69a7ec4f0bef"}]': finished 2024-04-21T08:40:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:45 smithi134 ceph-mon[31772]: osdmap e35: 6 total, 5 up, 6 in 2024-04-21T08:40:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:45 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:40:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:45 smithi134 ceph-mon[31772]: from='client.? 172.21.15.134:0/3576125548' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:40:47.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:47 smithi082 ceph-mon[27967]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:47.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:47 smithi106 ceph-mon[31072]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:47.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:47 smithi134 ceph-mon[31772]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:49.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:49 smithi082 ceph-mon[27967]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:49.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:49 smithi106 ceph-mon[31072]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:49.633 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:49 smithi134 ceph-mon[31772]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:51.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:51 smithi082 ceph-mon[27967]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:51.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:51 smithi106 ceph-mon[31072]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:51.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:51 smithi134 ceph-mon[31772]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:52.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T08:40:52.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T08:40:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:52.642 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T08:40:52.642 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:53.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:53 smithi134 ceph-mon[31772]: Deploying daemon osd.5 on smithi134 2024-04-21T08:40:53.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:53 smithi134 ceph-mon[31772]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:53.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:53 smithi082 ceph-mon[27967]: Deploying daemon osd.5 on smithi134 2024-04-21T08:40:53.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:53 smithi082 ceph-mon[27967]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:53.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:53 smithi106 ceph-mon[31072]: Deploying daemon osd.5 on smithi134 2024-04-21T08:40:53.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:53 smithi106 ceph-mon[31072]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:55.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:55 smithi134 ceph-mon[31772]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:55.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:55 smithi082 ceph-mon[27967]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:55.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:55 smithi106 ceph-mon[31072]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:57.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:57 smithi082 ceph-mon[27967]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:57.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:57 smithi106 ceph-mon[31072]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:57.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:57 smithi134 ceph-mon[31772]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:58.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:58 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:58.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:58 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:58.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:58 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:58.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:40:58 smithi082 ceph-mon[27967]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:58 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:58 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:58 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:40:58 smithi106 ceph-mon[31072]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:40:58.967 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:58 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:58.968 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:58 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:40:58.968 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:58 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:58.968 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:40:58 smithi134 ceph-mon[31772]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:41:01.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.408+0000 7f015e7fc700 1 -- 172.21.15.134:0/3167917349 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0168062e80 con 0x7f01540a3c40 2024-04-21T08:41:01.408 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 5 on host 'smithi134' 2024-04-21T08:41:01.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.411+0000 7f016f51f700 1 -- 172.21.15.134:0/3167917349 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f01540a3c40 msgr2=0x7f01540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.411+0000 7f016f51f700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f01540a3c40 0x7f01540a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0164005fd0 tx=0x7f0164005dd0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.411+0000 7f016f51f700 1 -- 172.21.15.134:0/3167917349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0168102ba0 msgr2=0x7f016819d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.411+0000 7f016f51f700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0168102ba0 0x7f016819d9b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0158006010 tx=0x7f01580039f0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.411+0000 7f016f51f700 1 -- 172.21.15.134:0/3167917349 shutdown_connections 2024-04-21T08:41:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.411+0000 7f016f51f700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f01540a3c40 0x7f01540a6100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.411+0000 7f016f51f700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0168103540 0x7f016819fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.411+0000 7f016f51f700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01681019a0 0x7f016806b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.411+0000 7f016f51f700 1 --2- 172.21.15.134:0/3167917349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0168102ba0 0x7f016819d9b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.411+0000 7f016f51f700 1 -- 172.21.15.134:0/3167917349 >> 172.21.15.134:0/3167917349 conn(0x7f01680fcf40 msgr2=0x7f0168105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.411+0000 7f016f51f700 1 -- 172.21.15.134:0/3167917349 shutdown_connections 2024-04-21T08:41:01.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:01.411+0000 7f016f51f700 1 -- 172.21.15.134:0/3167917349 wait complete. 2024-04-21T08:41:01.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:01 smithi082 ceph-mon[27967]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:41:01.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:01 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:01.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:01 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:01.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:01 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:01.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:01 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:01.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:01 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:01.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:01 smithi106 ceph-mon[31072]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:41:01.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:01 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:01.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:01 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:01.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:01 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:01.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:01 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:01.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:01 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:01.698 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:01 smithi134 ceph-mon[31772]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:41:01.698 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:01 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:01.698 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:01 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:01.698 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:01 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:01.698 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:01 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:01.699 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:01 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:01.983 DEBUG:teuthology.orchestra.run.smithi134:osd.5> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.5.service 2024-04-21T08:41:01.986 INFO:tasks.cephadm:Deploying osd.6 on smithi082 with /dev/vg_nvme/lv_2... 2024-04-21T08:41:01.986 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T08:41:02.232 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:41:02.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:02 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:02.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:02 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:02.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:02 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:02.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:02 smithi082 ceph-mon[27967]: from='osd.5 [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:41:02.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:02 smithi082 ceph-mon[27967]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:41:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:02 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:02 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:02 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:02 smithi106 ceph-mon[31072]: from='osd.5 [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:41:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:02 smithi106 ceph-mon[31072]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:41:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:02 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:02 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:02 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:02 smithi134 ceph-mon[31772]: from='osd.5 [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:41:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:02 smithi134 ceph-mon[31772]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:41:03.723 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:03 smithi082 ceph-mon[27967]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:41:03.723 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:03 smithi082 ceph-mon[27967]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T08:41:03.723 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:03 smithi082 ceph-mon[27967]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T08:41:03.723 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:03.723 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:03 smithi082 ceph-mon[27967]: from='osd.5 [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:41:03.723 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:03 smithi082 ceph-mon[27967]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:41:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:03 smithi134 ceph-mon[31772]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:41:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:03 smithi134 ceph-mon[31772]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T08:41:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:03 smithi134 ceph-mon[31772]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T08:41:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:03 smithi134 ceph-mon[31772]: from='osd.5 [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:41:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:03 smithi134 ceph-mon[31772]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:41:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:03 smithi106 ceph-mon[31072]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:41:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:03 smithi106 ceph-mon[31072]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T08:41:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:03 smithi106 ceph-mon[31072]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T08:41:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:03 smithi106 ceph-mon[31072]: from='osd.5 [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:41:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:03 smithi106 ceph-mon[31072]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:41:04.314 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T08:41:04.314 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T08:41:04.314 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-04-21T08:41:04.314 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-04-21T08:41:04.314 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114058 s, 919 MB/s 2024-04-21T08:41:04.314 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-04-21T08:41:04.357 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:41:04 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:41:04.090+0000 7fe70ad70700 -1 osd.5 0 waiting for initial osdmap 2024-04-21T08:41:04.357 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:41:04 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:41:04.098+0000 7fe702b5b700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:41:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:04 smithi082 ceph-mon[27967]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-21T08:41:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:04 smithi082 ceph-mon[27967]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T08:41:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:04 smithi082 ceph-mon[27967]: from='osd.5 ' entity='osd.5' 2024-04-21T08:41:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:04 smithi134 ceph-mon[31772]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-21T08:41:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:04 smithi134 ceph-mon[31772]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T08:41:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:04 smithi134 ceph-mon[31772]: from='osd.5 ' entity='osd.5' 2024-04-21T08:41:04.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:04 smithi106 ceph-mon[31072]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-21T08:41:04.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:04 smithi106 ceph-mon[31072]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T08:41:04.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:04.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:04.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:04 smithi106 ceph-mon[31072]: from='osd.5 ' entity='osd.5' 2024-04-21T08:41:05.039 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_2 2024-04-21T08:41:05.287 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:41:05.697 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:05 smithi082 ceph-mon[27967]: purged_snaps scrub starts 2024-04-21T08:41:05.697 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:05 smithi082 ceph-mon[27967]: purged_snaps scrub ok 2024-04-21T08:41:05.697 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:05 smithi082 ceph-mon[27967]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:41:05.697 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:05.697 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:05 smithi082 ceph-mon[27967]: osd.5 [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096] boot 2024-04-21T08:41:05.697 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:05 smithi082 ceph-mon[27967]: osdmap e38: 6 total, 6 up, 6 in 2024-04-21T08:41:05.697 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:05 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:05.697 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:05 smithi082 ceph-mon[27967]: osdmap e39: 6 total, 6 up, 6 in 2024-04-21T08:41:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:05 smithi134 ceph-mon[31772]: purged_snaps scrub starts 2024-04-21T08:41:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:05 smithi134 ceph-mon[31772]: purged_snaps scrub ok 2024-04-21T08:41:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:05 smithi134 ceph-mon[31772]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:41:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:05 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:05 smithi134 ceph-mon[31772]: osd.5 [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096] boot 2024-04-21T08:41:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:05 smithi134 ceph-mon[31772]: osdmap e38: 6 total, 6 up, 6 in 2024-04-21T08:41:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:05 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:05 smithi134 ceph-mon[31772]: osdmap e39: 6 total, 6 up, 6 in 2024-04-21T08:41:05.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:05 smithi106 ceph-mon[31072]: purged_snaps scrub starts 2024-04-21T08:41:05.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:05 smithi106 ceph-mon[31072]: purged_snaps scrub ok 2024-04-21T08:41:05.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:05 smithi106 ceph-mon[31072]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:41:05.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:05 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:05.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:05 smithi106 ceph-mon[31072]: osd.5 [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096] boot 2024-04-21T08:41:05.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:05 smithi106 ceph-mon[31072]: osdmap e38: 6 total, 6 up, 6 in 2024-04-21T08:41:05.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:05 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:41:05.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:05 smithi106 ceph-mon[31072]: osdmap e39: 6 total, 6 up, 6 in 2024-04-21T08:41:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:06 smithi134 ceph-mon[31772]: Detected new or changed devices on smithi134 2024-04-21T08:41:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:06 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi134 to 8544M 2024-04-21T08:41:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:06.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:06.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:06.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:06 smithi134 ceph-mon[31772]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T08:41:06.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:06 smithi134 ceph-mon[31772]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:06.771 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:06 smithi082 ceph-mon[27967]: Detected new or changed devices on smithi134 2024-04-21T08:41:06.771 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:06.771 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:06.771 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:06.771 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:06.771 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:06 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi134 to 8544M 2024-04-21T08:41:06.771 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:06.771 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:06.771 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:06.771 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:06.771 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:06 smithi082 ceph-mon[27967]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T08:41:06.771 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:06 smithi082 ceph-mon[27967]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:06 smithi106 ceph-mon[31072]: Detected new or changed devices on smithi134 2024-04-21T08:41:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:06 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi134 to 8544M 2024-04-21T08:41:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:06.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:06 smithi106 ceph-mon[31072]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T08:41:06.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:06 smithi106 ceph-mon[31072]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.018+0000 7fc793c48700 1 -- 172.21.15.82:0/2596268963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78c102ab0 msgr2=0x7fc78c102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.018+0000 7fc793c48700 1 --2- 172.21.15.82:0/2596268963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78c102ab0 0x7fc78c102ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0099c0 tx=0x7fc77c009cd0 comp rx=0 tx=0).stop 2024-04-21T08:41:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.018+0000 7fc793c48700 1 -- 172.21.15.82:0/2596268963 shutdown_connections 2024-04-21T08:41:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.018+0000 7fc793c48700 1 --2- 172.21.15.82:0/2596268963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc78c104650 0x7fc78c10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.018+0000 7fc793c48700 1 --2- 172.21.15.82:0/2596268963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78c102ab0 0x7fc78c102ed0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.018+0000 7fc793c48700 1 --2- 172.21.15.82:0/2596268963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc78c103cb0 0x7fc78c104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.018+0000 7fc793c48700 1 -- 172.21.15.82:0/2596268963 >> 172.21.15.82:0/2596268963 conn(0x7fc78c0fe050 msgr2=0x7fc78c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.018+0000 7fc793c48700 1 -- 172.21.15.82:0/2596268963 shutdown_connections 2024-04-21T08:41:07.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.018+0000 7fc793c48700 1 -- 172.21.15.82:0/2596268963 wait complete. 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc793c48700 1 Processor -- start 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc793c48700 1 -- start start 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc793c48700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc78c102ab0 0x7fc78c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc793c48700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78c103cb0 0x7fc78c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc793c48700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc78c104650 0x7fc78c1a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc793c48700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c10e620 con 0x7fc78c103cb0 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc793c48700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c10e4a0 con 0x7fc78c102ab0 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc793c48700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c10e7a0 con 0x7fc78c104650 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc7911e3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78c103cb0 0x7fc78c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc7911e3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78c103cb0 0x7fc78c1991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53088/0 (socket says 172.21.15.82:53088) 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc7911e3700 1 -- 172.21.15.82:0/573585861 learned_addr learned my addr 172.21.15.82:0/573585861 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc7911e3700 1 -- 172.21.15.82:0/573585861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc78c104650 msgr2=0x7fc78c1a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc7919e4700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc78c102ab0 0x7fc78c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:07.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.019+0000 7fc7921e5700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc78c104650 0x7fc78c1a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.020+0000 7fc7911e3700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc78c104650 0x7fc78c1a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.020+0000 7fc7911e3700 1 -- 172.21.15.82:0/573585861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc78c102ab0 msgr2=0x7fc78c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.020+0000 7fc7911e3700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc78c102ab0 0x7fc78c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.020+0000 7fc7911e3700 1 -- 172.21.15.82:0/573585861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c1a0910 con 0x7fc78c103cb0 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.020+0000 7fc7919e4700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc78c102ab0 0x7fc78c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.020+0000 7fc7921e5700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc78c104650 0x7fc78c1a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.020+0000 7fc7911e3700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78c103cb0 0x7fc78c1991d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc78800b530 tx=0x7fc78800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.020+0000 7fc782ffd700 1 -- 172.21.15.82:0/573585861 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788015020 con 0x7fc78c103cb0 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.020+0000 7fc793c48700 1 -- 172.21.15.82:0/573585861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c1a0c00 con 0x7fc78c103cb0 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.020+0000 7fc782ffd700 1 -- 172.21.15.82:0/573585861 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc788011ce0 con 0x7fc78c103cb0 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.020+0000 7fc782ffd700 1 -- 172.21.15.82:0/573585861 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788010d20 con 0x7fc78c103cb0 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.020+0000 7fc793c48700 1 -- 172.21.15.82:0/573585861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c1a10f0 con 0x7fc78c103cb0 2024-04-21T08:41:07.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.021+0000 7fc793c48700 1 -- 172.21.15.82:0/573585861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc78c069730 con 0x7fc78c103cb0 2024-04-21T08:41:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.023+0000 7fc782ffd700 1 -- 172.21.15.82:0/573585861 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc788018020 con 0x7fc78c103cb0 2024-04-21T08:41:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.023+0000 7fc782ffd700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc7780a3c40 0x7fc7780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.024+0000 7fc7919e4700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc7780a3c40 0x7fc7780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:07.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.024+0000 7fc782ffd700 1 -- 172.21.15.82:0/573585861 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4533+0+0 (secure 0 0 0) 0x7fc7880cf300 con 0x7fc78c103cb0 2024-04-21T08:41:07.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.024+0000 7fc7919e4700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc7780a3c40 0x7fc7780a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000c00 tx=0x7fc77c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:07.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.027+0000 7fc782ffd700 1 -- 172.21.15.82:0/573585861 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc78801f020 con 0x7fc78c103cb0 2024-04-21T08:41:07.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:07.235+0000 7fc793c48700 1 -- 172.21.15.82:0/573585861 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc78c062e80 con 0x7fc7780a3c40 2024-04-21T08:41:08.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:07 smithi082 ceph-mon[27967]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T08:41:08.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:07 smithi082 ceph-mon[27967]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:41:08.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:07 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:41:08.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:07 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:41:08.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:07 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:08.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:07 smithi134 ceph-mon[31772]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T08:41:08.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:07 smithi134 ceph-mon[31772]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:41:08.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:07 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:41:08.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:07 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:41:08.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:07 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:08.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:07 smithi106 ceph-mon[31072]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T08:41:08.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:07 smithi106 ceph-mon[31072]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:41:08.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:07 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:41:08.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:07 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:41:08.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:07 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:09.096 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:08 smithi082 ceph-mon[27967]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T08:41:09.096 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:08 smithi082 ceph-mon[27967]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-04-21T08:41:09.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:08 smithi134 ceph-mon[31772]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T08:41:09.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:08 smithi134 ceph-mon[31772]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-04-21T08:41:09.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:08 smithi106 ceph-mon[31072]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T08:41:09.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:08 smithi106 ceph-mon[31072]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-04-21T08:41:10.412 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:10 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1279551928' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1f3d11-aa46-4d37-9e6c-9b2778d3306c"}]: dispatch 2024-04-21T08:41:10.412 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:10 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1279551928' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c1f3d11-aa46-4d37-9e6c-9b2778d3306c"}]': finished 2024-04-21T08:41:10.412 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:10 smithi134 ceph-mon[31772]: osdmap e43: 7 total, 6 up, 7 in 2024-04-21T08:41:10.412 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:10 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:10.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:10 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1279551928' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1f3d11-aa46-4d37-9e6c-9b2778d3306c"}]: dispatch 2024-04-21T08:41:10.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:10 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1279551928' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c1f3d11-aa46-4d37-9e6c-9b2778d3306c"}]': finished 2024-04-21T08:41:10.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:10 smithi082 ceph-mon[27967]: osdmap e43: 7 total, 6 up, 7 in 2024-04-21T08:41:10.456 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:10 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:10.585 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:10 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1279551928' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c1f3d11-aa46-4d37-9e6c-9b2778d3306c"}]: dispatch 2024-04-21T08:41:10.585 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:10 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1279551928' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c1f3d11-aa46-4d37-9e6c-9b2778d3306c"}]': finished 2024-04-21T08:41:10.585 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:10 smithi106 ceph-mon[31072]: osdmap e43: 7 total, 6 up, 7 in 2024-04-21T08:41:10.585 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:10 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:11.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:11 smithi134 ceph-mon[31772]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-04-21T08:41:11.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:11 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1874485692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:41:11.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:11 smithi082 ceph-mon[27967]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-04-21T08:41:11.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:11 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1874485692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:41:11.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:11 smithi106 ceph-mon[31072]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-04-21T08:41:11.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:11 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1874485692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:41:13.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:13 smithi082 ceph-mon[27967]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:41:13.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:13 smithi106 ceph-mon[31072]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:41:13.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:13 smithi134 ceph-mon[31772]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:41:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:15 smithi082 ceph-mon[27967]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-21T08:41:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:15 smithi106 ceph-mon[31072]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-21T08:41:15.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:15 smithi134 ceph-mon[31772]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-21T08:41:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:17 smithi082 ceph-mon[27967]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:17.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:17 smithi106 ceph-mon[31072]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:17 smithi134 ceph-mon[31772]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:18.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:18 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T08:41:18.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:18 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:18.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:18 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T08:41:18.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:18 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:18.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:18 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T08:41:18.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:18 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:19.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:19 smithi082 ceph-mon[27967]: Deploying daemon osd.6 on smithi082 2024-04-21T08:41:19.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:19 smithi082 ceph-mon[27967]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:19.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:19 smithi106 ceph-mon[31072]: Deploying daemon osd.6 on smithi082 2024-04-21T08:41:19.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:19 smithi106 ceph-mon[31072]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:19.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:19 smithi134 ceph-mon[31772]: Deploying daemon osd.6 on smithi082 2024-04-21T08:41:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:19 smithi134 ceph-mon[31772]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:21.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:21 smithi082 ceph-mon[27967]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:21.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:21 smithi106 ceph-mon[31072]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:21.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:21 smithi134 ceph-mon[31772]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:23.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:23 smithi082 ceph-mon[27967]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:23.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:23 smithi106 ceph-mon[31072]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:23.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:23 smithi134 ceph-mon[31772]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:25.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:25.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:25.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:25.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:25 smithi082 ceph-mon[27967]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:25.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:25 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:25 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:25 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:25 smithi106 ceph-mon[31072]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:25.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:25.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:25.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:25.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:25 smithi134 ceph-mon[31772]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:27.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:27 smithi082 ceph-mon[27967]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:27.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:27.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:27.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:27.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:27.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:27.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:27 smithi106 ceph-mon[31072]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:27.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:27.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:27.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:27.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:27.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:27 smithi134 ceph-mon[31772]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:28.258 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 6 on host 'smithi082' 2024-04-21T08:41:28.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.257+0000 7fc782ffd700 1 -- 172.21.15.82:0/573585861 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc78c062e80 con 0x7fc7780a3c40 2024-04-21T08:41:28.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.260+0000 7fc793c48700 1 -- 172.21.15.82:0/573585861 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc7780a3c40 msgr2=0x7fc7780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:28.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.260+0000 7fc793c48700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc7780a3c40 0x7fc7780a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc77c000c00 tx=0x7fc77c009f10 comp rx=0 tx=0).stop 2024-04-21T08:41:28.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.260+0000 7fc793c48700 1 -- 172.21.15.82:0/573585861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78c103cb0 msgr2=0x7fc78c1991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:28.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.260+0000 7fc793c48700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78c103cb0 0x7fc78c1991d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc78800b530 tx=0x7fc78800b8f0 comp rx=0 tx=0).stop 2024-04-21T08:41:28.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.260+0000 7fc793c48700 1 -- 172.21.15.82:0/573585861 shutdown_connections 2024-04-21T08:41:28.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.260+0000 7fc793c48700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc7780a3c40 0x7fc7780a6100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:28.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.260+0000 7fc793c48700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc78c104650 0x7fc78c1a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:28.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.260+0000 7fc793c48700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc78c102ab0 0x7fc78c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:28.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.260+0000 7fc793c48700 1 --2- 172.21.15.82:0/573585861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78c103cb0 0x7fc78c1991d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:28.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.260+0000 7fc793c48700 1 -- 172.21.15.82:0/573585861 >> 172.21.15.82:0/573585861 conn(0x7fc78c0fe050 msgr2=0x7fc78c107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:28.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.260+0000 7fc793c48700 1 -- 172.21.15.82:0/573585861 shutdown_connections 2024-04-21T08:41:28.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:41:28.260+0000 7fc793c48700 1 -- 172.21.15.82:0/573585861 wait complete. 2024-04-21T08:41:28.580 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:28 smithi082 ceph-mon[27967]: from='osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T08:41:28.580 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:28.580 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:28.580 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:28 smithi106 ceph-mon[31072]: from='osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T08:41:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:28 smithi134 ceph-mon[31772]: from='osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T08:41:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:29.007 DEBUG:teuthology.orchestra.run.smithi082:osd.6> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.6.service 2024-04-21T08:41:29.010 INFO:tasks.cephadm:Deploying osd.7 on smithi106 with /dev/vg_nvme/lv_2... 2024-04-21T08:41:29.010 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T08:41:29.257 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:41:29.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:29 smithi082 ceph-mon[27967]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:29.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:29 smithi082 ceph-mon[27967]: from='osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T08:41:29.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:29 smithi082 ceph-mon[27967]: osdmap e44: 7 total, 6 up, 7 in 2024-04-21T08:41:29.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:29 smithi082 ceph-mon[27967]: from='osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-21T08:41:29.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:29.502 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:41:29 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:41:29.344+0000 7f6b344e0700 -1 osd.6 0 waiting for initial osdmap 2024-04-21T08:41:29.502 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:41:29 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:41:29.355+0000 7f6b302d3700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:41:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:29 smithi106 ceph-mon[31072]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:29 smithi106 ceph-mon[31072]: from='osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T08:41:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:29 smithi106 ceph-mon[31072]: osdmap e44: 7 total, 6 up, 7 in 2024-04-21T08:41:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:29 smithi106 ceph-mon[31072]: from='osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-21T08:41:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:29 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:29 smithi134 ceph-mon[31772]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:29 smithi134 ceph-mon[31772]: from='osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T08:41:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:29 smithi134 ceph-mon[31772]: osdmap e44: 7 total, 6 up, 7 in 2024-04-21T08:41:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:29 smithi134 ceph-mon[31772]: from='osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-21T08:41:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:29 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:30.566 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:30 smithi082 ceph-mon[27967]: from='osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-21T08:41:30.566 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:30 smithi082 ceph-mon[27967]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T08:41:30.566 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:30 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:30.566 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:30 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:30 smithi106 ceph-mon[31072]: from='osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-21T08:41:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:30 smithi106 ceph-mon[31072]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T08:41:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:30 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:30 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:30.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:30 smithi134 ceph-mon[31772]: from='osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-21T08:41:30.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:30 smithi134 ceph-mon[31772]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T08:41:30.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:30 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:30.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:30 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:31.293 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T08:41:31.293 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T08:41:31.294 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-21T08:41:31.294 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-21T08:41:31.294 INFO:teuthology.orchestra.run.smithi106.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115666 s, 907 MB/s 2024-04-21T08:41:31.294 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-21T08:41:31.595 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:31 smithi106 ceph-mon[31072]: purged_snaps scrub starts 2024-04-21T08:41:31.595 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:31 smithi106 ceph-mon[31072]: purged_snaps scrub ok 2024-04-21T08:41:31.595 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:31 smithi106 ceph-mon[31072]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:31.595 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:31.595 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:31 smithi106 ceph-mon[31072]: osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969] boot 2024-04-21T08:41:31.595 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:31 smithi106 ceph-mon[31072]: osdmap e46: 7 total, 7 up, 7 in 2024-04-21T08:41:31.595 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:31 smithi082 ceph-mon[27967]: purged_snaps scrub starts 2024-04-21T08:41:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:31 smithi082 ceph-mon[27967]: purged_snaps scrub ok 2024-04-21T08:41:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:31 smithi082 ceph-mon[27967]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:31 smithi082 ceph-mon[27967]: osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969] boot 2024-04-21T08:41:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:31 smithi082 ceph-mon[27967]: osdmap e46: 7 total, 7 up, 7 in 2024-04-21T08:41:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:31.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:31 smithi134 ceph-mon[31772]: purged_snaps scrub starts 2024-04-21T08:41:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:31 smithi134 ceph-mon[31772]: purged_snaps scrub ok 2024-04-21T08:41:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:31 smithi134 ceph-mon[31772]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:41:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:31 smithi134 ceph-mon[31772]: osd.6 [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969] boot 2024-04-21T08:41:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:31 smithi134 ceph-mon[31772]: osdmap e46: 7 total, 7 up, 7 in 2024-04-21T08:41:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:41:31.947 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_2 2024-04-21T08:41:32.185 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:41:32.564 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:32 smithi106 ceph-mon[31072]: osdmap e47: 7 total, 7 up, 7 in 2024-04-21T08:41:32.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:32 smithi082 ceph-mon[27967]: osdmap e47: 7 total, 7 up, 7 in 2024-04-21T08:41:32.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:32 smithi134 ceph-mon[31772]: osdmap e47: 7 total, 7 up, 7 in 2024-04-21T08:41:33.582 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:33 smithi106 ceph-mon[31072]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:33.582 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:33 smithi106 ceph-mon[31072]: Detected new or changed devices on smithi082 2024-04-21T08:41:33.582 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:33.582 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:33.582 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:33.582 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:33.582 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:33.582 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:33 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi082 to 5696M 2024-04-21T08:41:33.582 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:33.583 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:33.583 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:33.583 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:33 smithi134 ceph-mon[31772]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:33 smithi134 ceph-mon[31772]: Detected new or changed devices on smithi082 2024-04-21T08:41:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:33 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi082 to 5696M 2024-04-21T08:41:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:33.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:33.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:33.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.810+0000 7f8c71495700 1 -- 172.21.15.106:0/4141145529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c103ce0 msgr2=0x7f8c6c104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.810+0000 7f8c71495700 1 --2- 172.21.15.106:0/4141145529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c103ce0 0x7f8c6c104140 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8c600099c0 tx=0x7f8c60009cd0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.810+0000 7f8c71495700 1 -- 172.21.15.106:0/4141145529 shutdown_connections 2024-04-21T08:41:33.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.810+0000 7f8c71495700 1 --2- 172.21.15.106:0/4141145529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c6c104680 0x7f8c6c10af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.810+0000 7f8c71495700 1 --2- 172.21.15.106:0/4141145529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c6c102ae0 0x7f8c6c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.810+0000 7f8c71495700 1 --2- 172.21.15.106:0/4141145529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c103ce0 0x7f8c6c104140 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.810+0000 7f8c71495700 1 -- 172.21.15.106:0/4141145529 >> 172.21.15.106:0/4141145529 conn(0x7f8c6c0fe080 msgr2=0x7f8c6c1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.810+0000 7f8c71495700 1 -- 172.21.15.106:0/4141145529 shutdown_connections 2024-04-21T08:41:33.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.810+0000 7f8c71495700 1 -- 172.21.15.106:0/4141145529 wait complete. 2024-04-21T08:41:33.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.811+0000 7f8c71495700 1 Processor -- start 2024-04-21T08:41:33.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.811+0000 7f8c71495700 1 -- start start 2024-04-21T08:41:33.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.811+0000 7f8c71495700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c6c102ae0 0x7f8c6c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.811+0000 7f8c71495700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c6c103ce0 0x7f8c6c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.811+0000 7f8c71495700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c104680 0x7f8c6c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.811+0000 7f8c71495700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c10e5d0 con 0x7f8c6c103ce0 2024-04-21T08:41:33.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.811+0000 7f8c71495700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c10e450 con 0x7f8c6c104680 2024-04-21T08:41:33.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.811+0000 7f8c71495700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c10e750 con 0x7f8c6c102ae0 2024-04-21T08:41:33.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.811+0000 7f8c6b7fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c104680 0x7f8c6c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.811+0000 7f8c6b7fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c104680 0x7f8c6c075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51404/0 (socket says 172.21.15.106:51404) 2024-04-21T08:41:33.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.811+0000 7f8c6b7fe700 1 -- 172.21.15.106:0/1303267375 learned_addr learned my addr 172.21.15.106:0/1303267375 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c6a7fc700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c6c103ce0 0x7f8c6c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c6affd700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c6c102ae0 0x7f8c6c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c6b7fe700 1 -- 172.21.15.106:0/1303267375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c6c102ae0 msgr2=0x7f8c6c0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c6b7fe700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c6c102ae0 0x7f8c6c0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c6b7fe700 1 -- 172.21.15.106:0/1303267375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c6c103ce0 msgr2=0x7f8c6c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c6b7fe700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c6c103ce0 0x7f8c6c074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c6b7fe700 1 -- 172.21.15.106:0/1303267375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6c075c20 con 0x7f8c6c104680 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c6affd700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c6c102ae0 0x7f8c6c0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c6b7fe700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c104680 0x7f8c6c075370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400b970 tx=0x7f8c6400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c53fff700 1 -- 172.21.15.106:0/1303267375 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64019070 con 0x7f8c6c104680 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c53fff700 1 -- 172.21.15.106:0/1303267375 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c6400cde0 con 0x7f8c6c104680 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c71495700 1 -- 172.21.15.106:0/1303267375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6c1a2d80 con 0x7f8c6c104680 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c53fff700 1 -- 172.21.15.106:0/1303267375 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64007370 con 0x7f8c6c104680 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.812+0000 7f8c71495700 1 -- 172.21.15.106:0/1303267375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c6c1a3230 con 0x7f8c6c104680 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.813+0000 7f8c71495700 1 -- 172.21.15.106:0/1303267375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c6c069730 con 0x7f8c6c104680 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.814+0000 7f8c53fff700 1 -- 172.21.15.106:0/1303267375 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8c6401c070 con 0x7f8c6c104680 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.815+0000 7f8c53fff700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8c540a3c40 0x7f8c540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.815+0000 7f8c53fff700 1 -- 172.21.15.106:0/1303267375 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f8c640cf2b0 con 0x7f8c6c104680 2024-04-21T08:41:33.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.815+0000 7f8c6affd700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8c540a3c40 0x7f8c540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.816 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.816+0000 7f8c6affd700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8c540a3c40 0x7f8c540a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c005950 tx=0x7f8c5c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:33.818+0000 7f8c53fff700 1 -- 172.21.15.106:0/1303267375 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c64023070 con 0x7f8c6c104680 2024-04-21T08:41:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:33 smithi082 ceph-mon[27967]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:33 smithi082 ceph-mon[27967]: Detected new or changed devices on smithi082 2024-04-21T08:41:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:33 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi082 to 5696M 2024-04-21T08:41:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:34.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:34.028+0000 7f8c71495700 1 -- 172.21.15.106:0/1303267375 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8c6c062e80 con 0x7f8c540a3c40 2024-04-21T08:41:34.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:41:34.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:41:34.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:34.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:41:34.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:41:34.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:34.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:41:34.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:41:34.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:35.577 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:35 smithi106 ceph-mon[31072]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:41:35.577 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:35 smithi106 ceph-mon[31072]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:35.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:35 smithi134 ceph-mon[31772]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:41:35.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:35 smithi134 ceph-mon[31772]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:35.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:35 smithi082 ceph-mon[27967]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:41:35.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:35 smithi082 ceph-mon[27967]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:37 smithi106 ceph-mon[31072]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:37 smithi106 ceph-mon[31072]: from='client.? 172.21.15.106:0/662398554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62f2af9e-02e2-436b-9041-60e4523eeeff"}]: dispatch 2024-04-21T08:41:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:37 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62f2af9e-02e2-436b-9041-60e4523eeeff"}]: dispatch 2024-04-21T08:41:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:37 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62f2af9e-02e2-436b-9041-60e4523eeeff"}]': finished 2024-04-21T08:41:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:37 smithi106 ceph-mon[31072]: osdmap e48: 8 total, 7 up, 8 in 2024-04-21T08:41:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:37 smithi106 ceph-mon[31072]: from='client.? 172.21.15.106:0/2378670914' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:41:37.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:37 smithi134 ceph-mon[31772]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:37.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:37 smithi134 ceph-mon[31772]: from='client.? 172.21.15.106:0/662398554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62f2af9e-02e2-436b-9041-60e4523eeeff"}]: dispatch 2024-04-21T08:41:37.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:37 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62f2af9e-02e2-436b-9041-60e4523eeeff"}]: dispatch 2024-04-21T08:41:37.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:37 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62f2af9e-02e2-436b-9041-60e4523eeeff"}]': finished 2024-04-21T08:41:37.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:37 smithi134 ceph-mon[31772]: osdmap e48: 8 total, 7 up, 8 in 2024-04-21T08:41:37.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:37.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:37 smithi134 ceph-mon[31772]: from='client.? 172.21.15.106:0/2378670914' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:41:37.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:37 smithi082 ceph-mon[27967]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:37.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:37 smithi082 ceph-mon[27967]: from='client.? 172.21.15.106:0/662398554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62f2af9e-02e2-436b-9041-60e4523eeeff"}]: dispatch 2024-04-21T08:41:37.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:37 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62f2af9e-02e2-436b-9041-60e4523eeeff"}]: dispatch 2024-04-21T08:41:37.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:37 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62f2af9e-02e2-436b-9041-60e4523eeeff"}]': finished 2024-04-21T08:41:37.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:37 smithi082 ceph-mon[27967]: osdmap e48: 8 total, 7 up, 8 in 2024-04-21T08:41:37.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:37.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:37 smithi082 ceph-mon[27967]: from='client.? 172.21.15.106:0/2378670914' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:41:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:39 smithi106 ceph-mon[31072]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:39.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:39 smithi134 ceph-mon[31772]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:39.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:39 smithi082 ceph-mon[27967]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:41.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:41 smithi106 ceph-mon[31072]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:41.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:41 smithi134 ceph-mon[31772]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:41.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:41 smithi082 ceph-mon[27967]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:43.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:43 smithi106 ceph-mon[31072]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:43.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:43 smithi134 ceph-mon[31772]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:43.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:43 smithi082 ceph-mon[27967]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:44.707 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:44 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T08:41:44.708 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:44 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:44.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:44 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T08:41:44.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:44 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:44.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:44 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T08:41:44.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:44 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:45.692 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:45 smithi106 ceph-mon[31072]: Deploying daemon osd.7 on smithi106 2024-04-21T08:41:45.692 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:45 smithi106 ceph-mon[31072]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:45.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:45 smithi134 ceph-mon[31772]: Deploying daemon osd.7 on smithi106 2024-04-21T08:41:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:45 smithi134 ceph-mon[31772]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:45.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:45 smithi082 ceph-mon[27967]: Deploying daemon osd.7 on smithi106 2024-04-21T08:41:45.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:45 smithi082 ceph-mon[27967]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:47.652 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:47 smithi106 ceph-mon[31072]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:47 smithi134 ceph-mon[31772]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:47.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:47 smithi082 ceph-mon[27967]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:49.475 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:49 smithi106 ceph-mon[31072]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:49.660 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:49 smithi082 ceph-mon[27967]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:49 smithi134 ceph-mon[31772]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:50 smithi134 ceph-mon[31772]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:50.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:50.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:50.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:50.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:50 smithi082 ceph-mon[27967]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:50.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:50.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:50.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:50.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:50 smithi106 ceph-mon[31072]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:53.244 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.244+0000 7f8c53fff700 1 -- 172.21.15.106:0/1303267375 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8c6c062e80 con 0x7f8c540a3c40 2024-04-21T08:41:53.244 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 7 on host 'smithi106' 2024-04-21T08:41:53.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.246+0000 7f8c71495700 1 -- 172.21.15.106:0/1303267375 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8c540a3c40 msgr2=0x7f8c540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:53.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.246+0000 7f8c71495700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8c540a3c40 0x7f8c540a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c005950 tx=0x7f8c5c0058e0 comp rx=0 tx=0).stop 2024-04-21T08:41:53.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.246+0000 7f8c71495700 1 -- 172.21.15.106:0/1303267375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c104680 msgr2=0x7f8c6c075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:53.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.246+0000 7f8c71495700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c104680 0x7f8c6c075370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400b970 tx=0x7f8c6400bc80 comp rx=0 tx=0).stop 2024-04-21T08:41:53.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.246+0000 7f8c71495700 1 -- 172.21.15.106:0/1303267375 shutdown_connections 2024-04-21T08:41:53.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.246+0000 7f8c71495700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8c540a3c40 0x7f8c540a6100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:53.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.246+0000 7f8c71495700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c104680 0x7f8c6c075370 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:53.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.246+0000 7f8c71495700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c6c102ae0 0x7f8c6c0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:53.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.246+0000 7f8c71495700 1 --2- 172.21.15.106:0/1303267375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c6c103ce0 0x7f8c6c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:53.246 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.246+0000 7f8c71495700 1 -- 172.21.15.106:0/1303267375 >> 172.21.15.106:0/1303267375 conn(0x7f8c6c0fe080 msgr2=0x7f8c6c106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:53.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.246+0000 7f8c71495700 1 -- 172.21.15.106:0/1303267375 shutdown_connections 2024-04-21T08:41:53.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:41:53.246+0000 7f8c71495700 1 -- 172.21.15.106:0/1303267375 wait complete. 2024-04-21T08:41:53.565 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:53 smithi106 ceph-mon[31072]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:53.565 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:53 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.566 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:53 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.566 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:53 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:53.566 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:53 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:53.566 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:53 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.566 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:53 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.566 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:53 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.566 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:53 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:53.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:53 smithi082 ceph-mon[27967]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:53.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:53 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:53 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:53 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:53.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:53 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:53.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:53 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:53 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:53 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:53 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:53 smithi134 ceph-mon[31772]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:53 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:53 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:53 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:53 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:53 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:53 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:53 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:53 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:53.910 DEBUG:teuthology.orchestra.run.smithi106:osd.7> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.7.service 2024-04-21T08:41:53.913 INFO:tasks.cephadm:Deploying osd.8 on smithi134 with /dev/vg_nvme/lv_2... 2024-04-21T08:41:53.913 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T08:41:54.159 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:41:54.539 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:54 smithi134 ceph-mon[31772]: from='osd.7 [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:41:54.540 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:54 smithi134 ceph-mon[31772]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:41:54.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:54 smithi082 ceph-mon[27967]: from='osd.7 [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:41:54.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:54 smithi082 ceph-mon[27967]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:41:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:54 smithi106 ceph-mon[31072]: from='osd.7 [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:41:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:54 smithi106 ceph-mon[31072]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:41:55.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:55 smithi082 ceph-mon[27967]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:55.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:55 smithi082 ceph-mon[27967]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T08:41:55.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:55 smithi082 ceph-mon[27967]: from='osd.7 [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:41:55.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:55 smithi082 ceph-mon[27967]: osdmap e49: 8 total, 7 up, 8 in 2024-04-21T08:41:55.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:55 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:55.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:55 smithi082 ceph-mon[27967]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:41:55.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:55 smithi106 ceph-mon[31072]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:55.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:55 smithi106 ceph-mon[31072]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T08:41:55.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:55 smithi106 ceph-mon[31072]: from='osd.7 [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:41:55.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:55 smithi106 ceph-mon[31072]: osdmap e49: 8 total, 7 up, 8 in 2024-04-21T08:41:55.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:55 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:55.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:55 smithi106 ceph-mon[31072]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:41:55.629 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:41:55 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:41:55.352+0000 7f893434e700 -1 osd.7 0 waiting for initial osdmap 2024-04-21T08:41:55.629 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:41:55 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:41:55.363+0000 7f8930141700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:41:55.692 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:55 smithi134 ceph-mon[31772]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:55.692 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:55 smithi134 ceph-mon[31772]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T08:41:55.692 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:55 smithi134 ceph-mon[31772]: from='osd.7 [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:41:55.692 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:55 smithi134 ceph-mon[31772]: osdmap e49: 8 total, 7 up, 8 in 2024-04-21T08:41:55.692 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:55 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:55.692 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:55 smithi134 ceph-mon[31772]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:41:56.235 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T08:41:56.235 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T08:41:56.235 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-21T08:41:56.235 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-21T08:41:56.236 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111456 s, 941 MB/s 2024-04-21T08:41:56.236 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-21T08:41:56.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:56 smithi106 ceph-mon[31072]: purged_snaps scrub starts 2024-04-21T08:41:56.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:56 smithi106 ceph-mon[31072]: purged_snaps scrub ok 2024-04-21T08:41:56.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:56 smithi106 ceph-mon[31072]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-21T08:41:56.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:56 smithi106 ceph-mon[31072]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T08:41:56.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:56 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:56.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:56 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:56.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:56 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:56.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:56 smithi134 ceph-mon[31772]: purged_snaps scrub starts 2024-04-21T08:41:56.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:56 smithi134 ceph-mon[31772]: purged_snaps scrub ok 2024-04-21T08:41:56.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:56 smithi134 ceph-mon[31772]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-21T08:41:56.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:56 smithi134 ceph-mon[31772]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T08:41:56.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:56 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:56.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:56 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:56.538 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:56 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:56.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:56 smithi082 ceph-mon[27967]: purged_snaps scrub starts 2024-04-21T08:41:56.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:56 smithi082 ceph-mon[27967]: purged_snaps scrub ok 2024-04-21T08:41:56.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:56 smithi082 ceph-mon[27967]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-21T08:41:56.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:56 smithi082 ceph-mon[27967]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T08:41:56.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:56 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:56.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:56 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:56.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:56 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:56.913 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2024-04-21T08:41:57.159 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:41:57.630 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:57.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: osd.7 [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036] boot 2024-04-21T08:41:57.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: osdmap e51: 8 total, 8 up, 8 in 2024-04-21T08:41:57.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:57.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:57.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:57.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:57.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:57.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:57.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:57.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:57.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:57.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:57.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:57.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: osd.7 [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036] boot 2024-04-21T08:41:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: osdmap e51: 8 total, 8 up, 8 in 2024-04-21T08:41:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:57.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:41:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: osd.7 [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036] boot 2024-04-21T08:41:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: osdmap e51: 8 total, 8 up, 8 in 2024-04-21T08:41:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:41:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:57.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:57.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:41:57.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:57.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:57.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:57.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:41:58.586 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:58 smithi134 ceph-mon[31772]: Detected new or changed devices on smithi106 2024-04-21T08:41:58.586 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:58 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi106 to 5696M 2024-04-21T08:41:58.586 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:58 smithi134 ceph-mon[31772]: osdmap e52: 8 total, 8 up, 8 in 2024-04-21T08:41:58.586 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:58 smithi134 ceph-mon[31772]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T08:41:58.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 -- 172.21.15.134:0/3306746461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30c103560 msgr2=0x7fa30c109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 --2- 172.21.15.134:0/3306746461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30c103560 0x7fa30c109e10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa30400b0a0 tx=0x7fa30400b3b0 comp rx=0 tx=0).stop 2024-04-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 -- 172.21.15.134:0/3306746461 shutdown_connections 2024-04-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 --2- 172.21.15.134:0/3306746461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30c103560 0x7fa30c109e10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 --2- 172.21.15.134:0/3306746461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa30c1019c0 0x7fa30c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 --2- 172.21.15.134:0/3306746461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa30c102bc0 0x7fa30c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 -- 172.21.15.134:0/3306746461 >> 172.21.15.134:0/3306746461 conn(0x7fa30c0fcfa0 msgr2=0x7fa30c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 -- 172.21.15.134:0/3306746461 shutdown_connections 2024-04-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 -- 172.21.15.134:0/3306746461 wait complete. 2024-04-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 Processor -- start 2024-04-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 -- start start 2024-04-21T08:41:58.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa30c1019c0 0x7fa30c105950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.817+0000 7fa313813700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30c102bc0 0x7fa30c103fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa313813700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa30c103560 0x7fa30c1044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa313813700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c10d460 con 0x7fa30c1019c0 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa313813700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c10d2e0 con 0x7fa30c103560 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa313813700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c10d5e0 con 0x7fa30c102bc0 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa310dae700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30c102bc0 0x7fa30c103fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa310dae700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30c102bc0 0x7fa30c103fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:40094/0 (socket says 172.21.15.134:40094) 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa310dae700 1 -- 172.21.15.134:0/1604408757 learned_addr learned my addr 172.21.15.134:0/1604408757 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa3115af700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa30c1019c0 0x7fa30c105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa310dae700 1 -- 172.21.15.134:0/1604408757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa30c103560 msgr2=0x7fa30c1044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa310dae700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa30c103560 0x7fa30c1044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa310dae700 1 -- 172.21.15.134:0/1604408757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa30c1019c0 msgr2=0x7fa30c105950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa310dae700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa30c1019c0 0x7fa30c105950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:58.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa310dae700 1 -- 172.21.15.134:0/1604408757 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c104d90 con 0x7fa30c102bc0 2024-04-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa3115af700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa30c1019c0 0x7fa30c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa310dae700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30c102bc0 0x7fa30c103fa0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00c920 tx=0x7fa2fc00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa3027fc700 1 -- 172.21.15.134:0/1604408757 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc007b00 con 0x7fa30c102bc0 2024-04-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa313813700 1 -- 172.21.15.134:0/1604408757 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30c1a2d10 con 0x7fa30c102bc0 2024-04-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa3027fc700 1 -- 172.21.15.134:0/1604408757 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2fc004ce0 con 0x7fa30c102bc0 2024-04-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa3027fc700 1 -- 172.21.15.134:0/1604408757 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc005650 con 0x7fa30c102bc0 2024-04-21T08:41:58.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.818+0000 7fa313813700 1 -- 172.21.15.134:0/1604408757 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30c1a31c0 con 0x7fa30c102bc0 2024-04-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.819+0000 7fa313813700 1 -- 172.21.15.134:0/1604408757 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa30c1a3450 con 0x7fa30c102bc0 2024-04-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.820+0000 7fa3027fc700 1 -- 172.21.15.134:0/1604408757 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa2fc007ca0 con 0x7fa30c102bc0 2024-04-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.821+0000 7fa3027fc700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa2f80a3c40 0x7fa2f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.821+0000 7fa3027fc700 1 -- 172.21.15.134:0/1604408757 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7fa2fc0cf830 con 0x7fa30c102bc0 2024-04-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.821+0000 7fa3115af700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa2f80a3c40 0x7fa2f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:58.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.822+0000 7fa3115af700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa2f80a3c40 0x7fa2f80a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa308005fd0 tx=0x7fa308005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:58.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:58.825+0000 7fa3027fc700 1 -- 172.21.15.134:0/1604408757 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2fc014030 con 0x7fa30c102bc0 2024-04-21T08:41:58.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:58 smithi082 ceph-mon[27967]: Detected new or changed devices on smithi106 2024-04-21T08:41:58.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:58 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi106 to 5696M 2024-04-21T08:41:58.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:58 smithi082 ceph-mon[27967]: osdmap e52: 8 total, 8 up, 8 in 2024-04-21T08:41:58.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:58 smithi082 ceph-mon[27967]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T08:41:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:58 smithi106 ceph-mon[31072]: Detected new or changed devices on smithi106 2024-04-21T08:41:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:58 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi106 to 5696M 2024-04-21T08:41:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:58 smithi106 ceph-mon[31072]: osdmap e52: 8 total, 8 up, 8 in 2024-04-21T08:41:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:58 smithi106 ceph-mon[31072]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T08:41:59.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:41:59.030+0000 7fa313813700 1 -- 172.21.15.134:0/1604408757 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa30c0ff130 con 0x7fa2f80a3c40 2024-04-21T08:41:59.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:59 smithi134 ceph-mon[31772]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:41:59.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:59 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:41:59.385 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:59 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:41:59.646 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:41:59 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:59.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:59 smithi082 ceph-mon[27967]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:41:59.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:59 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:41:59.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:59 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:41:59.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:41:59 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:59 smithi106 ceph-mon[31072]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:41:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:59 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:41:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:59 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:41:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:41:59 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:00.661 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:00 smithi134 ceph-mon[31772]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:42:00.662 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:00 smithi134 ceph-mon[31772]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T08:42:00.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:00 smithi082 ceph-mon[27967]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:42:00.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:00 smithi082 ceph-mon[27967]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T08:42:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:00 smithi106 ceph-mon[31072]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:42:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:00 smithi106 ceph-mon[31072]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T08:42:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:01 smithi134 ceph-mon[31772]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:01 smithi134 ceph-mon[31772]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T08:42:01.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:01 smithi082 ceph-mon[27967]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:01.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:01 smithi082 ceph-mon[27967]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T08:42:01.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:01 smithi106 ceph-mon[31072]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:01.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:01 smithi106 ceph-mon[31072]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T08:42:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:02 smithi134 ceph-mon[31772]: from='client.? 172.21.15.134:0/76564351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e39282b2-0f53-4c57-a68c-771857b35413"}]: dispatch 2024-04-21T08:42:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:02 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e39282b2-0f53-4c57-a68c-771857b35413"}]: dispatch 2024-04-21T08:42:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:02 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e39282b2-0f53-4c57-a68c-771857b35413"}]': finished 2024-04-21T08:42:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:02 smithi134 ceph-mon[31772]: osdmap e56: 9 total, 8 up, 9 in 2024-04-21T08:42:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:02 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:02.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:02 smithi082 ceph-mon[27967]: from='client.? 172.21.15.134:0/76564351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e39282b2-0f53-4c57-a68c-771857b35413"}]: dispatch 2024-04-21T08:42:02.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:02 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e39282b2-0f53-4c57-a68c-771857b35413"}]: dispatch 2024-04-21T08:42:02.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:02 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e39282b2-0f53-4c57-a68c-771857b35413"}]': finished 2024-04-21T08:42:02.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:02 smithi082 ceph-mon[27967]: osdmap e56: 9 total, 8 up, 9 in 2024-04-21T08:42:02.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:02 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:02.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:02 smithi106 ceph-mon[31072]: from='client.? 172.21.15.134:0/76564351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e39282b2-0f53-4c57-a68c-771857b35413"}]: dispatch 2024-04-21T08:42:02.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:02 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e39282b2-0f53-4c57-a68c-771857b35413"}]: dispatch 2024-04-21T08:42:02.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:02 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e39282b2-0f53-4c57-a68c-771857b35413"}]': finished 2024-04-21T08:42:02.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:02 smithi106 ceph-mon[31072]: osdmap e56: 9 total, 8 up, 9 in 2024-04-21T08:42:02.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:02 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:03 smithi134 ceph-mon[31772]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:03 smithi134 ceph-mon[31772]: from='client.? 172.21.15.134:0/1565992709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:42:03.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:03 smithi082 ceph-mon[27967]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:03.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:03 smithi082 ceph-mon[27967]: from='client.? 172.21.15.134:0/1565992709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:42:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:03 smithi106 ceph-mon[31072]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:03 smithi106 ceph-mon[31072]: from='client.? 172.21.15.134:0/1565992709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:42:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:05 smithi134 ceph-mon[31772]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:05.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:05 smithi082 ceph-mon[27967]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:05.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:05 smithi106 ceph-mon[31072]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:07.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:07 smithi134 ceph-mon[31772]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:07.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:07 smithi082 ceph-mon[27967]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:07.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:07 smithi106 ceph-mon[31072]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:09.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:09 smithi134 ceph-mon[31772]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:09.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:09 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T08:42:09.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:09 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:09.680 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:09 smithi106 ceph-mon[31072]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:09.680 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:09 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T08:42:09.680 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:09 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:09.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:09 smithi082 ceph-mon[27967]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:09.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:09 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T08:42:09.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:09 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:10.701 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:10 smithi106 ceph-mon[31072]: Deploying daemon osd.8 on smithi134 2024-04-21T08:42:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:10 smithi134 ceph-mon[31772]: Deploying daemon osd.8 on smithi134 2024-04-21T08:42:10.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:10 smithi082 ceph-mon[27967]: Deploying daemon osd.8 on smithi134 2024-04-21T08:42:11.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:11 smithi134 ceph-mon[31772]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:11.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:11 smithi082 ceph-mon[27967]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:11.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:11 smithi106 ceph-mon[31072]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:13.651 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:13 smithi134 ceph-mon[31772]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:13.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:13 smithi082 ceph-mon[27967]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:13.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:13 smithi106 ceph-mon[31072]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:15.622 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:15 smithi134 ceph-mon[31772]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:15.622 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:15.622 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:15.622 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:15.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:15 smithi082 ceph-mon[27967]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:15.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:15.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:15.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:15.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:15 smithi106 ceph-mon[31072]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:15.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:15.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:15.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:16.724 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:16 smithi134 ceph-mon[31772]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:16.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:16 smithi082 ceph-mon[27967]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:16.880 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:16 smithi106 ceph-mon[31072]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:18.851 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.851+0000 7fa3027fc700 1 -- 172.21.15.134:0/1604408757 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa30c0ff130 con 0x7fa2f80a3c40 2024-04-21T08:42:18.851 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 8 on host 'smithi134' 2024-04-21T08:42:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.854+0000 7fa313813700 1 -- 172.21.15.134:0/1604408757 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa2f80a3c40 msgr2=0x7fa2f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.854+0000 7fa313813700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa2f80a3c40 0x7fa2f80a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa308005fd0 tx=0x7fa308005f00 comp rx=0 tx=0).stop 2024-04-21T08:42:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.854+0000 7fa313813700 1 -- 172.21.15.134:0/1604408757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30c102bc0 msgr2=0x7fa30c103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:18.853 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.854+0000 7fa313813700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30c102bc0 0x7fa30c103fa0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00c920 tx=0x7fa2fc00cce0 comp rx=0 tx=0).stop 2024-04-21T08:42:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.854+0000 7fa313813700 1 -- 172.21.15.134:0/1604408757 shutdown_connections 2024-04-21T08:42:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.854+0000 7fa313813700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa2f80a3c40 0x7fa2f80a6100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.854+0000 7fa313813700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa30c103560 0x7fa30c1044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.854+0000 7fa313813700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa30c1019c0 0x7fa30c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.854+0000 7fa313813700 1 --2- 172.21.15.134:0/1604408757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa30c102bc0 0x7fa30c103fa0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.854+0000 7fa313813700 1 -- 172.21.15.134:0/1604408757 >> 172.21.15.134:0/1604408757 conn(0x7fa30c0fcfa0 msgr2=0x7fa30c0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.854+0000 7fa313813700 1 -- 172.21.15.134:0/1604408757 shutdown_connections 2024-04-21T08:42:18.854 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:42:18.854+0000 7fa313813700 1 -- 172.21.15.134:0/1604408757 wait complete. 2024-04-21T08:42:19.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:19 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:19 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:19 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:19.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:19 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:19.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:19 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:19 smithi082 ceph-mon[27967]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:19.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:19 smithi082 ceph-mon[27967]: from='osd.8 [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:42:19.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:19 smithi082 ceph-mon[27967]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:42:19.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:19 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:19 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:19 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:19 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:19 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:19 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:19 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:19 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:19 smithi106 ceph-mon[31072]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:19 smithi106 ceph-mon[31072]: from='osd.8 [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:42:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:19 smithi106 ceph-mon[31072]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:42:19.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:19 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:19 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:19 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:19 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:19 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:19 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:19 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:19 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:19 smithi134 ceph-mon[31772]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:19 smithi134 ceph-mon[31772]: from='osd.8 [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:42:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:19 smithi134 ceph-mon[31772]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:42:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:19 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:19 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:19.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:19 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:19.556 DEBUG:teuthology.orchestra.run.smithi134:osd.8> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.8.service 2024-04-21T08:42:19.558 INFO:tasks.cephadm:Deploying osd.9 on smithi082 with /dev/vg_nvme/lv_1... 2024-04-21T08:42:19.559 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T08:42:19.803 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:42:20.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:20 smithi082 ceph-mon[27967]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T08:42:20.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:20 smithi082 ceph-mon[27967]: from='osd.8 [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:42:20.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:20 smithi082 ceph-mon[27967]: osdmap e57: 9 total, 8 up, 9 in 2024-04-21T08:42:20.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:20 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:20.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:20 smithi082 ceph-mon[27967]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:42:20.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:20 smithi106 ceph-mon[31072]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T08:42:20.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:20 smithi106 ceph-mon[31072]: from='osd.8 [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:42:20.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:20 smithi106 ceph-mon[31072]: osdmap e57: 9 total, 8 up, 9 in 2024-04-21T08:42:20.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:20 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:20.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:20 smithi106 ceph-mon[31072]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:42:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:20 smithi134 ceph-mon[31772]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T08:42:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:20 smithi134 ceph-mon[31772]: from='osd.8 [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:42:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:20 smithi134 ceph-mon[31772]: osdmap e57: 9 total, 8 up, 9 in 2024-04-21T08:42:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:20 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:20 smithi134 ceph-mon[31772]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:42:20.476 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:42:20 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:42:20.098+0000 7f90fcad6700 -1 osd.8 0 waiting for initial osdmap 2024-04-21T08:42:20.476 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:42:20 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:42:20.107+0000 7f90f88c9700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:42:21.351 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:21 smithi082 ceph-mon[27967]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-21T08:42:21.352 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:21 smithi082 ceph-mon[27967]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T08:42:21.352 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:21 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:21.352 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:21 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:21.352 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:21 smithi082 ceph-mon[27967]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:21.352 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:21 smithi082 ceph-mon[27967]: osd.8 [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684] boot 2024-04-21T08:42:21.352 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:21 smithi082 ceph-mon[27967]: osdmap e59: 9 total, 9 up, 9 in 2024-04-21T08:42:21.352 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:21 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:21.354 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:21 smithi134 ceph-mon[31772]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-21T08:42:21.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:21 smithi134 ceph-mon[31772]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T08:42:21.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:21 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:21.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:21 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:21.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:21 smithi134 ceph-mon[31772]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:21.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:21 smithi134 ceph-mon[31772]: osd.8 [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684] boot 2024-04-21T08:42:21.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:21 smithi134 ceph-mon[31772]: osdmap e59: 9 total, 9 up, 9 in 2024-04-21T08:42:21.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:21 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:21 smithi106 ceph-mon[31072]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-21T08:42:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:21 smithi106 ceph-mon[31072]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T08:42:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:21 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:21 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:21 smithi106 ceph-mon[31072]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:42:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:21 smithi106 ceph-mon[31072]: osd.8 [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684] boot 2024-04-21T08:42:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:21 smithi106 ceph-mon[31072]: osdmap e59: 9 total, 9 up, 9 in 2024-04-21T08:42:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:21 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:42:22.048 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T08:42:22.048 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T08:42:22.048 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-04-21T08:42:22.048 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-04-21T08:42:22.048 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112665 s, 931 MB/s 2024-04-21T08:42:22.048 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-04-21T08:42:22.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:22 smithi134 ceph-mon[31772]: purged_snaps scrub starts 2024-04-21T08:42:22.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:22 smithi134 ceph-mon[31772]: purged_snaps scrub ok 2024-04-21T08:42:22.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:22 smithi134 ceph-mon[31772]: osdmap e60: 9 total, 9 up, 9 in 2024-04-21T08:42:22.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:22 smithi082 ceph-mon[27967]: purged_snaps scrub starts 2024-04-21T08:42:22.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:22 smithi082 ceph-mon[27967]: purged_snaps scrub ok 2024-04-21T08:42:22.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:22 smithi082 ceph-mon[27967]: osdmap e60: 9 total, 9 up, 9 in 2024-04-21T08:42:22.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:22 smithi106 ceph-mon[31072]: purged_snaps scrub starts 2024-04-21T08:42:22.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:22 smithi106 ceph-mon[31072]: purged_snaps scrub ok 2024-04-21T08:42:22.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:22 smithi106 ceph-mon[31072]: osdmap e60: 9 total, 9 up, 9 in 2024-04-21T08:42:22.743 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_1 2024-04-21T08:42:22.979 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:42:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:23 smithi082 ceph-mon[27967]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:23 smithi082 ceph-mon[27967]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T08:42:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:23 smithi106 ceph-mon[31072]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:23 smithi106 ceph-mon[31072]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T08:42:23.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:23 smithi134 ceph-mon[31772]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:23.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:23.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:23.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:23.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:23.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:23.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:23.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:23.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:23.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:23.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:23 smithi134 ceph-mon[31772]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T08:42:24.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:24 smithi082 ceph-mon[27967]: Detected new or changed devices on smithi134 2024-04-21T08:42:24.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:24 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi134 to 5696M 2024-04-21T08:42:24.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:24 smithi106 ceph-mon[31072]: Detected new or changed devices on smithi134 2024-04-21T08:42:24.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:24 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi134 to 5696M 2024-04-21T08:42:24.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:24 smithi134 ceph-mon[31772]: Detected new or changed devices on smithi134 2024-04-21T08:42:24.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:24 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi134 to 5696M 2024-04-21T08:42:24.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.615+0000 7fd500154700 1 -- 172.21.15.82:0/1723282866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8102ad0 msgr2=0x7fd4f8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:24.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.615+0000 7fd500154700 1 --2- 172.21.15.82:0/1723282866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8102ad0 0x7fd4f8102ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40099c0 tx=0x7fd4f4009cd0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.615+0000 7fd500154700 1 -- 172.21.15.82:0/1723282866 shutdown_connections 2024-04-21T08:42:24.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.615+0000 7fd500154700 1 --2- 172.21.15.82:0/1723282866 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4f8104670 0x7fd4f810af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.615+0000 7fd500154700 1 --2- 172.21.15.82:0/1723282866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8102ad0 0x7fd4f8102ef0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.615+0000 7fd500154700 1 --2- 172.21.15.82:0/1723282866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f8103cd0 0x7fd4f8104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.615+0000 7fd500154700 1 -- 172.21.15.82:0/1723282866 >> 172.21.15.82:0/1723282866 conn(0x7fd4f80fe090 msgr2=0x7fd4f81004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:24.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.615+0000 7fd500154700 1 -- 172.21.15.82:0/1723282866 shutdown_connections 2024-04-21T08:42:24.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.615+0000 7fd500154700 1 -- 172.21.15.82:0/1723282866 wait complete. 2024-04-21T08:42:24.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd500154700 1 Processor -- start 2024-04-21T08:42:24.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd500154700 1 -- start start 2024-04-21T08:42:24.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd500154700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4f8102ad0 0x7fd4f80767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd500154700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f8103cd0 0x7fd4f8074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd500154700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8104670 0x7fd4f8075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd500154700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f810e640 con 0x7fd4f8104670 2024-04-21T08:42:24.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd500154700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f810e4c0 con 0x7fd4f8102ad0 2024-04-21T08:42:24.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd500154700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f810e7c0 con 0x7fd4f8103cd0 2024-04-21T08:42:24.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4fe6f1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8104670 0x7fd4f8075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4fe6f1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8104670 0x7fd4f8075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58702/0 (socket says 172.21.15.82:58702) 2024-04-21T08:42:24.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4fe6f1700 1 -- 172.21.15.82:0/1997791812 learned_addr learned my addr 172.21.15.82:0/1997791812 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:42:24.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4fdef0700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4f8102ad0 0x7fd4f80767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4fd6ef700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f8103cd0 0x7fd4f8074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4fe6f1700 1 -- 172.21.15.82:0/1997791812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f8103cd0 msgr2=0x7fd4f8074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:24.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4fe6f1700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f8103cd0 0x7fd4f8074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4fe6f1700 1 -- 172.21.15.82:0/1997791812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4f8102ad0 msgr2=0x7fd4f80767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:24.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4fe6f1700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4f8102ad0 0x7fd4f80767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4fe6f1700 1 -- 172.21.15.82:0/1997791812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f8075c20 con 0x7fd4f8104670 2024-04-21T08:42:24.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4fe6f1700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8104670 0x7fd4f8075370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000b970 tx=0x7fd4f000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:24.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4eeffd700 1 -- 172.21.15.82:0/1997791812 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0019070 con 0x7fd4f8104670 2024-04-21T08:42:24.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4eeffd700 1 -- 172.21.15.82:0/1997791812 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f000cde0 con 0x7fd4f8104670 2024-04-21T08:42:24.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd500154700 1 -- 172.21.15.82:0/1997791812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f81a2e40 con 0x7fd4f8104670 2024-04-21T08:42:24.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.616+0000 7fd4eeffd700 1 -- 172.21.15.82:0/1997791812 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0007370 con 0x7fd4f8104670 2024-04-21T08:42:24.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.617+0000 7fd500154700 1 -- 172.21.15.82:0/1997791812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f81a3320 con 0x7fd4f8104670 2024-04-21T08:42:24.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.617+0000 7fd500154700 1 -- 172.21.15.82:0/1997791812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f8069730 con 0x7fd4f8104670 2024-04-21T08:42:24.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.619+0000 7fd4eeffd700 1 -- 172.21.15.82:0/1997791812 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd4f001c070 con 0x7fd4f8104670 2024-04-21T08:42:24.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.619+0000 7fd4eeffd700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd4e40a3c40 0x7fd4e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.619+0000 7fd4eeffd700 1 -- 172.21.15.82:0/1997791812 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7fd4f00cf630 con 0x7fd4f8104670 2024-04-21T08:42:24.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.619+0000 7fd4fdef0700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd4e40a3c40 0x7fd4e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.620+0000 7fd4fdef0700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd4e40a3c40 0x7fd4e40a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40038c0 tx=0x7fd4f400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:24.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.623+0000 7fd4eeffd700 1 -- 172.21.15.82:0/1997791812 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4f0023070 con 0x7fd4f8104670 2024-04-21T08:42:24.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:24.830+0000 7fd500154700 1 -- 172.21.15.82:0/1997791812 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd4f8062e80 con 0x7fd4e40a3c40 2024-04-21T08:42:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:25 smithi106 ceph-mon[31072]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T08:42:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:25 smithi106 ceph-mon[31072]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:25 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:42:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:25 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:42:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:25 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:25.445 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:25 smithi082 ceph-mon[27967]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T08:42:25.445 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:25 smithi082 ceph-mon[27967]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:25.445 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:42:25.446 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:42:25.446 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:25 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:25.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:25 smithi134 ceph-mon[31772]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T08:42:25.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:25 smithi134 ceph-mon[31772]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:25.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:42:25.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:42:25.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:25 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:26.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:26 smithi134 ceph-mon[31772]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:42:26.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:26 smithi134 ceph-mon[31772]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T08:42:26.519 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:26 smithi082 ceph-mon[27967]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:42:26.519 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:26 smithi082 ceph-mon[27967]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T08:42:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:26 smithi106 ceph-mon[31072]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:42:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:26 smithi106 ceph-mon[31072]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T08:42:27.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:27 smithi134 ceph-mon[31772]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:27.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:27 smithi082 ceph-mon[27967]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:27.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:27 smithi106 ceph-mon[31072]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:28.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:28 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/250303568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8c9e2a-4cb6-442a-a767-9452c6cb7f06"}]: dispatch 2024-04-21T08:42:28.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:28 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/250303568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d8c9e2a-4cb6-442a-a767-9452c6cb7f06"}]': finished 2024-04-21T08:42:28.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:28 smithi082 ceph-mon[27967]: osdmap e64: 10 total, 9 up, 10 in 2024-04-21T08:42:28.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:28.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:28 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/250303568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8c9e2a-4cb6-442a-a767-9452c6cb7f06"}]: dispatch 2024-04-21T08:42:28.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:28 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/250303568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d8c9e2a-4cb6-442a-a767-9452c6cb7f06"}]': finished 2024-04-21T08:42:28.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:28 smithi134 ceph-mon[31772]: osdmap e64: 10 total, 9 up, 10 in 2024-04-21T08:42:28.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:28 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/250303568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8c9e2a-4cb6-442a-a767-9452c6cb7f06"}]: dispatch 2024-04-21T08:42:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:28 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/250303568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d8c9e2a-4cb6-442a-a767-9452c6cb7f06"}]': finished 2024-04-21T08:42:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:28 smithi106 ceph-mon[31072]: osdmap e64: 10 total, 9 up, 10 in 2024-04-21T08:42:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:29.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:29 smithi134 ceph-mon[31772]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-21T08:42:29.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:29 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/189809066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:42:29.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:29 smithi082 ceph-mon[27967]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-21T08:42:29.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:29 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/189809066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:42:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:29 smithi106 ceph-mon[31072]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-21T08:42:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:29 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/189809066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:42:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:31 smithi082 ceph-mon[27967]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-21T08:42:31.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:31 smithi106 ceph-mon[31072]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-21T08:42:31.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:31 smithi134 ceph-mon[31772]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-21T08:42:33.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:33 smithi082 ceph-mon[27967]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-21T08:42:33.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:33 smithi106 ceph-mon[31072]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-21T08:42:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:33 smithi134 ceph-mon[31772]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-21T08:42:35.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:35 smithi082 ceph-mon[27967]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-21T08:42:35.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:35 smithi106 ceph-mon[31072]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-21T08:42:35.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:35 smithi134 ceph-mon[31772]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-21T08:42:36.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:36 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T08:42:36.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:36 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:36.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:36 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T08:42:36.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:36 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:36.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:36 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T08:42:36.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:36 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:37.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:37 smithi082 ceph-mon[27967]: Deploying daemon osd.9 on smithi082 2024-04-21T08:42:37.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:37 smithi082 ceph-mon[27967]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:42:37.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:37 smithi106 ceph-mon[31072]: Deploying daemon osd.9 on smithi082 2024-04-21T08:42:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:37 smithi106 ceph-mon[31072]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:42:37.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:37 smithi134 ceph-mon[31772]: Deploying daemon osd.9 on smithi082 2024-04-21T08:42:37.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:37 smithi134 ceph-mon[31772]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:42:39.567 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:39 smithi082 ceph-mon[27967]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:39 smithi106 ceph-mon[31072]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:39.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:39 smithi134 ceph-mon[31772]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:41.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:41 smithi082 ceph-mon[27967]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:41.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:41 smithi106 ceph-mon[31072]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:41.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:41 smithi134 ceph-mon[31772]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:42 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:42 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:42 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:42 smithi082 ceph-mon[27967]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:42 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:42 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:42 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:42 smithi106 ceph-mon[31072]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:43.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:42 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:43.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:42 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:43.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:42 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:43.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:42 smithi134 ceph-mon[31772]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:45.592 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:45 smithi082 ceph-mon[27967]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:45.592 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:45 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:45.593 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:45 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:45.593 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:45 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:45.593 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:45 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:45.593 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:45 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:45 smithi106 ceph-mon[31072]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:45 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:45 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:45 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:45 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:45 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:45 smithi134 ceph-mon[31772]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:45 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:45 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:45 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:45 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:45 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:45.973 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 9 on host 'smithi082' 2024-04-21T08:42:45.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.974+0000 7fd4eeffd700 1 -- 172.21.15.82:0/1997791812 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd4f8062e80 con 0x7fd4e40a3c40 2024-04-21T08:42:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.976+0000 7fd500154700 1 -- 172.21.15.82:0/1997791812 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd4e40a3c40 msgr2=0x7fd4e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.976+0000 7fd500154700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd4e40a3c40 0x7fd4e40a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40038c0 tx=0x7fd4f400b540 comp rx=0 tx=0).stop 2024-04-21T08:42:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.976+0000 7fd500154700 1 -- 172.21.15.82:0/1997791812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8104670 msgr2=0x7fd4f8075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.976+0000 7fd500154700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8104670 0x7fd4f8075370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000b970 tx=0x7fd4f000bc80 comp rx=0 tx=0).stop 2024-04-21T08:42:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.976+0000 7fd500154700 1 -- 172.21.15.82:0/1997791812 shutdown_connections 2024-04-21T08:42:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.976+0000 7fd500154700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd4e40a3c40 0x7fd4e40a6100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.976+0000 7fd500154700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8104670 0x7fd4f8075370 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.976+0000 7fd500154700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4f8102ad0 0x7fd4f80767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.976+0000 7fd500154700 1 --2- 172.21.15.82:0/1997791812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f8103cd0 0x7fd4f8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.976+0000 7fd500154700 1 -- 172.21.15.82:0/1997791812 >> 172.21.15.82:0/1997791812 conn(0x7fd4f80fe090 msgr2=0x7fd4f8106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.976+0000 7fd500154700 1 -- 172.21.15.82:0/1997791812 shutdown_connections 2024-04-21T08:42:45.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:42:45.976+0000 7fd500154700 1 -- 172.21.15.82:0/1997791812 wait complete. 2024-04-21T08:42:46.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:46 smithi082 ceph-mon[27967]: from='osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T08:42:46.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:46 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:46.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:46 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:46.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:46 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:46.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:46 smithi106 ceph-mon[31072]: from='osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T08:42:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:46 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:46 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:46 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:46.716 DEBUG:teuthology.orchestra.run.smithi082:osd.9> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.9.service 2024-04-21T08:42:46.718 INFO:tasks.cephadm:Deploying osd.10 on smithi106 with /dev/vg_nvme/lv_1... 2024-04-21T08:42:46.718 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T08:42:46.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:46 smithi134 ceph-mon[31772]: from='osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T08:42:46.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:46 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:46.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:46 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:46.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:46 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:46.968 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:42:47.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:47 smithi082 ceph-mon[27967]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:47.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:47 smithi082 ceph-mon[27967]: from='osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T08:42:47.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:47 smithi082 ceph-mon[27967]: osdmap e65: 10 total, 9 up, 10 in 2024-04-21T08:42:47.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:47 smithi082 ceph-mon[27967]: from='osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-21T08:42:47.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:47 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:47.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:42:47 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:42:47.355+0000 7f923caa7700 -1 osd.9 0 waiting for initial osdmap 2024-04-21T08:42:47.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:42:47 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:42:47.368+0000 7f923889a700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:42:47.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:47 smithi106 ceph-mon[31072]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:47.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:47 smithi106 ceph-mon[31072]: from='osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T08:42:47.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:47 smithi106 ceph-mon[31072]: osdmap e65: 10 total, 9 up, 10 in 2024-04-21T08:42:47.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:47 smithi106 ceph-mon[31072]: from='osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-21T08:42:47.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:47 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:47 smithi134 ceph-mon[31772]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:47 smithi134 ceph-mon[31772]: from='osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T08:42:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:47 smithi134 ceph-mon[31772]: osdmap e65: 10 total, 9 up, 10 in 2024-04-21T08:42:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:47 smithi134 ceph-mon[31772]: from='osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-21T08:42:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:47 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:48.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:48 smithi082 ceph-mon[27967]: from='osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-21T08:42:48.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:48 smithi082 ceph-mon[27967]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T08:42:48.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:48 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:48.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:48 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:48.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:48 smithi106 ceph-mon[31072]: from='osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-21T08:42:48.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:48 smithi106 ceph-mon[31072]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T08:42:48.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:48 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:48.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:48 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:48 smithi134 ceph-mon[31772]: from='osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-21T08:42:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:48 smithi134 ceph-mon[31772]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T08:42:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:48 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:48 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:48.920 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T08:42:48.920 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T08:42:48.920 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-21T08:42:48.920 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-21T08:42:48.920 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126553 s, 829 MB/s 2024-04-21T08:42:48.920 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-21T08:42:49.579 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:49 smithi106 ceph-mon[31072]: purged_snaps scrub starts 2024-04-21T08:42:49.579 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:49 smithi106 ceph-mon[31072]: purged_snaps scrub ok 2024-04-21T08:42:49.579 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:49 smithi106 ceph-mon[31072]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:49.579 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:49 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:49.579 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:49 smithi106 ceph-mon[31072]: osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646] boot 2024-04-21T08:42:49.579 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:49 smithi106 ceph-mon[31072]: osdmap e67: 10 total, 10 up, 10 in 2024-04-21T08:42:49.579 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:49 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:49.606 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_1 2024-04-21T08:42:49.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:49 smithi082 ceph-mon[27967]: purged_snaps scrub starts 2024-04-21T08:42:49.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:49 smithi082 ceph-mon[27967]: purged_snaps scrub ok 2024-04-21T08:42:49.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:49 smithi082 ceph-mon[27967]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:49.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:49 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:49.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:49 smithi082 ceph-mon[27967]: osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646] boot 2024-04-21T08:42:49.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:49 smithi082 ceph-mon[27967]: osdmap e67: 10 total, 10 up, 10 in 2024-04-21T08:42:49.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:49 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:49 smithi134 ceph-mon[31772]: purged_snaps scrub starts 2024-04-21T08:42:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:49 smithi134 ceph-mon[31772]: purged_snaps scrub ok 2024-04-21T08:42:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:49 smithi134 ceph-mon[31772]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:42:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:49 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:49 smithi134 ceph-mon[31772]: osd.9 [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646] boot 2024-04-21T08:42:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:49 smithi134 ceph-mon[31772]: osdmap e67: 10 total, 10 up, 10 in 2024-04-21T08:42:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:49 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:42:49.840 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:42:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:50 smithi106 ceph-mon[31072]: osdmap e68: 10 total, 10 up, 10 in 2024-04-21T08:42:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:50.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:50.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:50.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:50 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:50 smithi134 ceph-mon[31772]: osdmap e68: 10 total, 10 up, 10 in 2024-04-21T08:42:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:50.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:50 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:50.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:50 smithi082 ceph-mon[27967]: osdmap e68: 10 total, 10 up, 10 in 2024-04-21T08:42:50.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:50.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:50.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:50.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:50.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:50.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:42:50.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:50.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:50.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:50.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:50 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:42:51.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.342+0000 7feb3f6d0700 1 -- 172.21.15.106:0/3275925032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb38102aa0 msgr2=0x7feb38102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:51.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 --2- 172.21.15.106:0/3275925032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb38102aa0 0x7feb38102ec0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feb340099c0 tx=0x7feb34009cd0 comp rx=0 tx=0).stop 2024-04-21T08:42:51.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 -- 172.21.15.106:0/3275925032 shutdown_connections 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 --2- 172.21.15.106:0/3275925032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb38104640 0x7feb3810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 --2- 172.21.15.106:0/3275925032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb38102aa0 0x7feb38102ec0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 --2- 172.21.15.106:0/3275925032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb38103ca0 0x7feb38104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 -- 172.21.15.106:0/3275925032 >> 172.21.15.106:0/3275925032 conn(0x7feb380fe040 msgr2=0x7feb38100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 -- 172.21.15.106:0/3275925032 shutdown_connections 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 -- 172.21.15.106:0/3275925032 wait complete. 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 Processor -- start 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 -- start start 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb38102aa0 0x7feb380767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb38103ca0 0x7feb38074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb38104640 0x7feb38075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7feb3810e5b0 con 0x7feb38103ca0 2024-04-21T08:42:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feb3810e430 con 0x7feb38102aa0 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.343+0000 7feb3f6d0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb3810e730 con 0x7feb38104640 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb3d46c700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb38102aa0 0x7feb380767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb3d46c700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb38102aa0 0x7feb380767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37690/0 (socket says 172.21.15.106:37690) 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb3d46c700 1 -- 172.21.15.106:0/2580092910 learned_addr learned my addr 172.21.15.106:0/2580092910 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb3cc6b700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb38103ca0 0x7feb38074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb3d46c700 1 -- 172.21.15.106:0/2580092910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb38104640 msgr2=0x7feb38075370 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb3d46c700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb38104640 0x7feb38075370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb3d46c700 1 -- 172.21.15.106:0/2580092910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb38103ca0 msgr2=0x7feb38074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb3d46c700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb38103ca0 0x7feb38074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb3d46c700 1 -- 172.21.15.106:0/2580092910 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb38075c20 con 0x7feb38102aa0 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb3d46c700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb38102aa0 0x7feb380767e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb3400b580 tx=0x7feb34009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb2e7fc700 1 -- 172.21.15.106:0/2580092910 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34027070 con 0x7feb38102aa0 2024-04-21T08:42:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb2e7fc700 1 -- 172.21.15.106:0/2580092910 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb34004120 con 0x7feb38102aa0 2024-04-21T08:42:51.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb2e7fc700 1 -- 172.21.15.106:0/2580092910 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34022ce0 con 0x7feb38102aa0 2024-04-21T08:42:51.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb3f6d0700 1 -- 172.21.15.106:0/2580092910 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb381a2db0 con 0x7feb38102aa0 2024-04-21T08:42:51.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.344+0000 7feb3f6d0700 1 -- 172.21.15.106:0/2580092910 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb381a31b0 con 0x7feb38102aa0 2024-04-21T08:42:51.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.345+0000 7feb3f6d0700 1 -- 172.21.15.106:0/2580092910 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb3802df80 con 0x7feb38102aa0 2024-04-21T08:42:51.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.346+0000 7feb2e7fc700 1 -- 172.21.15.106:0/2580092910 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feb3402a070 con 0x7feb38102aa0 2024-04-21T08:42:51.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.347+0000 7feb2e7fc700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7feb240a3c40 0x7feb240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:51.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.347+0000 7feb2e7fc700 1 -- 172.21.15.106:0/2580092910 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7feb340d12d0 con 0x7feb38102aa0 2024-04-21T08:42:51.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.347+0000 7feb3cc6b700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7feb240a3c40 0x7feb240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:51.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.348+0000 7feb3cc6b700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7feb240a3c40 0x7feb240a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feb28007c60 tx=0x7feb280073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.351+0000 7feb2e7fc700 1 -- 172.21.15.106:0/2580092910 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb34031020 con 0x7feb38102aa0 2024-04-21T08:42:51.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:51 smithi106 ceph-mon[31072]: Detected new or changed devices on smithi082 2024-04-21T08:42:51.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:51 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi082 to 4272M 2024-04-21T08:42:51.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:51 smithi106 ceph-mon[31072]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:51.557 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:42:51.558+0000 7feb3f6d0700 1 -- 172.21.15.106:0/2580092910 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7feb38062e80 con 0x7feb240a3c40 2024-04-21T08:42:51.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:51 smithi082 ceph-mon[27967]: Detected new or changed devices on smithi082 2024-04-21T08:42:51.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:51 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi082 to 4272M 2024-04-21T08:42:51.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:51 smithi082 ceph-mon[27967]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:51.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:51 smithi134 ceph-mon[31772]: Detected new or changed devices on smithi082 2024-04-21T08:42:51.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:51 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi082 to 4272M 2024-04-21T08:42:51.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:51 smithi134 ceph-mon[31772]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:52.427 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:42:52.427 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:42:52.427 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:52 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:52.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:42:52.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:42:52.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:52 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:52.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:42:52.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:42:52.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:52 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:53.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:53 smithi106 ceph-mon[31072]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:42:53.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:53 smithi106 ceph-mon[31072]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:53 smithi134 ceph-mon[31772]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:42:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:53 smithi134 ceph-mon[31772]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:53.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:53 smithi082 ceph-mon[27967]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:42:53.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:53 smithi082 ceph-mon[27967]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:55.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:55 smithi134 ceph-mon[31772]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:55.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:55 smithi134 ceph-mon[31772]: from='client.? 172.21.15.106:0/597737311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73ee0364-4255-443c-ac99-c7f6286afeee"}]: dispatch 2024-04-21T08:42:55.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:55 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73ee0364-4255-443c-ac99-c7f6286afeee"}]: dispatch 2024-04-21T08:42:55.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:55 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73ee0364-4255-443c-ac99-c7f6286afeee"}]': finished 2024-04-21T08:42:55.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:55 smithi134 ceph-mon[31772]: osdmap e69: 11 total, 10 up, 11 in 2024-04-21T08:42:55.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:55 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:42:55.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:55 smithi134 ceph-mon[31772]: from='client.? 172.21.15.106:0/1713542433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:42:55.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:55 smithi082 ceph-mon[27967]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:55.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:55 smithi082 ceph-mon[27967]: from='client.? 172.21.15.106:0/597737311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73ee0364-4255-443c-ac99-c7f6286afeee"}]: dispatch 2024-04-21T08:42:55.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:55 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73ee0364-4255-443c-ac99-c7f6286afeee"}]: dispatch 2024-04-21T08:42:55.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:55 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73ee0364-4255-443c-ac99-c7f6286afeee"}]': finished 2024-04-21T08:42:55.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:55 smithi082 ceph-mon[27967]: osdmap e69: 11 total, 10 up, 11 in 2024-04-21T08:42:55.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:55 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:42:55.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:55 smithi082 ceph-mon[27967]: from='client.? 172.21.15.106:0/1713542433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:42:55.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:55 smithi106 ceph-mon[31072]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:55.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:55 smithi106 ceph-mon[31072]: from='client.? 172.21.15.106:0/597737311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73ee0364-4255-443c-ac99-c7f6286afeee"}]: dispatch 2024-04-21T08:42:55.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:55 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73ee0364-4255-443c-ac99-c7f6286afeee"}]: dispatch 2024-04-21T08:42:55.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:55 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73ee0364-4255-443c-ac99-c7f6286afeee"}]': finished 2024-04-21T08:42:55.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:55 smithi106 ceph-mon[31072]: osdmap e69: 11 total, 10 up, 11 in 2024-04-21T08:42:55.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:55 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:42:55.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:55 smithi106 ceph-mon[31072]: from='client.? 172.21.15.106:0/1713542433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:42:57.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:57 smithi134 ceph-mon[31772]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:57.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:57 smithi082 ceph-mon[27967]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:57 smithi106 ceph-mon[31072]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:59.646 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:42:59 smithi106 ceph-mon[31072]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:42:59 smithi134 ceph-mon[31772]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:42:59.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:42:59 smithi082 ceph-mon[27967]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:01 smithi134 ceph-mon[31772]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:01.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:01 smithi082 ceph-mon[27967]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:01.775 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:01 smithi106 ceph-mon[31072]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:02.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:02 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T08:43:02.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:02 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:02.669 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:02 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T08:43:02.669 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:02 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:02 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T08:43:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:02 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:03.675 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:03 smithi106 ceph-mon[31072]: Deploying daemon osd.10 on smithi106 2024-04-21T08:43:03.675 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:03 smithi106 ceph-mon[31072]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:03 smithi134 ceph-mon[31772]: Deploying daemon osd.10 on smithi106 2024-04-21T08:43:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:03 smithi134 ceph-mon[31772]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:03.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:03 smithi082 ceph-mon[27967]: Deploying daemon osd.10 on smithi106 2024-04-21T08:43:03.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:03 smithi082 ceph-mon[27967]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:05.601 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:05 smithi106 ceph-mon[31072]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:05 smithi134 ceph-mon[31772]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:05.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:05 smithi082 ceph-mon[27967]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:07.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:07 smithi106 ceph-mon[31072]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:07.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:07 smithi134 ceph-mon[31772]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:07.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:07 smithi082 ceph-mon[27967]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:08.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:08 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:08.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:08 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:08.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:08 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:08.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:08 smithi082 ceph-mon[27967]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:08.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:08 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:08.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:08 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:08.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:08 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:08.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:08 smithi106 ceph-mon[31072]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:08.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:08 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:08.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:08 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:08.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:08 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:08.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:08 smithi134 ceph-mon[31772]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.222+0000 7feb2e7fc700 1 -- 172.21.15.106:0/2580092910 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7feb38062e80 con 0x7feb240a3c40 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 10 on host 'smithi106' 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.225+0000 7feb3f6d0700 1 -- 172.21.15.106:0/2580092910 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7feb240a3c40 msgr2=0x7feb240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.225+0000 7feb3f6d0700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7feb240a3c40 0x7feb240a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feb28007c60 tx=0x7feb280073d0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.226+0000 7feb3f6d0700 1 -- 172.21.15.106:0/2580092910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb38102aa0 msgr2=0x7feb380767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.226+0000 7feb3f6d0700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb38102aa0 0x7feb380767e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb3400b580 tx=0x7feb34009f10 comp rx=0 tx=0).stop 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.226+0000 7feb3f6d0700 1 -- 172.21.15.106:0/2580092910 shutdown_connections 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.226+0000 7feb3f6d0700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7feb240a3c40 0x7feb240a6100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.226+0000 7feb3f6d0700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb38104640 0x7feb38075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.226+0000 7feb3f6d0700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb38102aa0 0x7feb380767e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.226+0000 7feb3f6d0700 1 --2- 172.21.15.106:0/2580092910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb38103ca0 0x7feb38074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.226+0000 7feb3f6d0700 1 -- 172.21.15.106:0/2580092910 >> 172.21.15.106:0/2580092910 conn(0x7feb380fe040 msgr2=0x7feb381070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.226+0000 7feb3f6d0700 1 -- 172.21.15.106:0/2580092910 shutdown_connections 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-21T08:43:11.226+0000 7feb3f6d0700 1 -- 172.21.15.106:0/2580092910 wait complete. 2024-04-21T08:43:11.522 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:11 smithi106 ceph-mon[31072]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:11.522 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:11 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:11.523 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:11 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:11.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:11 smithi082 ceph-mon[27967]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:11.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:11 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:11.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:11 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:11 smithi134 ceph-mon[31772]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:11 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:11 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:12.002 DEBUG:teuthology.orchestra.run.smithi106:osd.10> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.10.service 2024-04-21T08:43:12.005 INFO:tasks.cephadm:Deploying osd.11 on smithi134 with /dev/vg_nvme/lv_1... 2024-04-21T08:43:12.005 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T08:43:12.254 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:43:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:12 smithi082 ceph-mon[27967]: from='osd.10 [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:43:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:12 smithi082 ceph-mon[27967]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:43:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:12 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:12 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:12.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:12 smithi106 ceph-mon[31072]: from='osd.10 [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:43:12.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:12 smithi106 ceph-mon[31072]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:43:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:12 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:12 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:12.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:12 smithi134 ceph-mon[31772]: from='osd.10 [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:43:12.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:12 smithi134 ceph-mon[31772]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:43:12.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:12 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:12.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:12 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:13.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:13 smithi082 ceph-mon[27967]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:13.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:13 smithi082 ceph-mon[27967]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T08:43:13.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:13 smithi082 ceph-mon[27967]: from='osd.10 [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:43:13.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:13 smithi082 ceph-mon[27967]: osdmap e70: 11 total, 10 up, 11 in 2024-04-21T08:43:13.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:13 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:13.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:13 smithi082 ceph-mon[27967]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:43:13.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:13 smithi082 ceph-mon[27967]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-21T08:43:13.628 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:43:13 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:43:13.359+0000 7ff8237f5700 -1 osd.10 0 waiting for initial osdmap 2024-04-21T08:43:13.628 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:43:13 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:43:13.370+0000 7ff81f5e8700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:43:13.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:13 smithi106 ceph-mon[31072]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:13.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:13 smithi106 ceph-mon[31072]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T08:43:13.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:13 smithi106 ceph-mon[31072]: from='osd.10 [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:43:13.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:13 smithi106 ceph-mon[31072]: osdmap e70: 11 total, 10 up, 11 in 2024-04-21T08:43:13.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:13 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:13.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:13 smithi106 ceph-mon[31072]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:43:13.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:13 smithi106 ceph-mon[31072]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-21T08:43:13.653 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:13 smithi134 ceph-mon[31772]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:13.653 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:13 smithi134 ceph-mon[31772]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T08:43:13.654 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:13 smithi134 ceph-mon[31772]: from='osd.10 [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:43:13.654 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:13 smithi134 ceph-mon[31772]: osdmap e70: 11 total, 10 up, 11 in 2024-04-21T08:43:13.654 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:13 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:13.654 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:13 smithi134 ceph-mon[31772]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-21T08:43:13.654 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:13 smithi134 ceph-mon[31772]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-21T08:43:14.385 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T08:43:14.385 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T08:43:14.385 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-21T08:43:14.386 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-21T08:43:14.386 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112172 s, 935 MB/s 2024-04-21T08:43:14.386 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 2024-04-21T08:43:14.386 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-21T08:43:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:14 smithi106 ceph-mon[31072]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T08:43:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:14 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:14 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:14.688 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:14 smithi134 ceph-mon[31772]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T08:43:14.688 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:14 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:14.688 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:14 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:14.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:14 smithi082 ceph-mon[27967]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T08:43:14.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:14 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:14.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:14 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:15.163 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2024-04-21T08:43:15.418 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:43:15.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:15 smithi134 ceph-mon[31772]: purged_snaps scrub starts 2024-04-21T08:43:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:15 smithi134 ceph-mon[31772]: purged_snaps scrub ok 2024-04-21T08:43:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:15 smithi134 ceph-mon[31772]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:15 smithi134 ceph-mon[31772]: osd.10 [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808] boot 2024-04-21T08:43:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:15 smithi134 ceph-mon[31772]: osdmap e72: 11 total, 11 up, 11 in 2024-04-21T08:43:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:15 smithi134 ceph-mon[31772]: osdmap e73: 11 total, 11 up, 11 in 2024-04-21T08:43:15.486 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:15 smithi106 ceph-mon[31072]: purged_snaps scrub starts 2024-04-21T08:43:15.486 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:15 smithi106 ceph-mon[31072]: purged_snaps scrub ok 2024-04-21T08:43:15.486 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:15 smithi106 ceph-mon[31072]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:15.486 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:15.486 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:15 smithi106 ceph-mon[31072]: osd.10 [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808] boot 2024-04-21T08:43:15.486 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:15 smithi106 ceph-mon[31072]: osdmap e72: 11 total, 11 up, 11 in 2024-04-21T08:43:15.486 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:15.486 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:15 smithi106 ceph-mon[31072]: osdmap e73: 11 total, 11 up, 11 in 2024-04-21T08:43:15.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:15 smithi082 ceph-mon[27967]: purged_snaps scrub starts 2024-04-21T08:43:15.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:15 smithi082 ceph-mon[27967]: purged_snaps scrub ok 2024-04-21T08:43:15.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:15 smithi082 ceph-mon[27967]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:43:15.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:15.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:15 smithi082 ceph-mon[27967]: osd.10 [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808] boot 2024-04-21T08:43:15.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:15 smithi082 ceph-mon[27967]: osdmap e72: 11 total, 11 up, 11 in 2024-04-21T08:43:15.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:43:15.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:15 smithi082 ceph-mon[27967]: osdmap e73: 11 total, 11 up, 11 in 2024-04-21T08:43:16.846 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: Detected new or changed devices on smithi106 2024-04-21T08:43:16.846 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.846 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.846 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:16.847 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:16.847 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:16.847 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:16.847 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi106 to 4272M 2024-04-21T08:43:16.847 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.847 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:16.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:16.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:16.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T08:43:16.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:16.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:16.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:16.848 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:16 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: Detected new or changed devices on smithi106 2024-04-21T08:43:16.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:16.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:16.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:16.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:16.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi106 to 4272M 2024-04-21T08:43:16.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:16.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:16.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:16.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T08:43:16.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:16.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:16.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:16.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:16 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: Detected new or changed devices on smithi106 2024-04-21T08:43:16.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:16.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:16.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:16.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:16.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi106 to 4272M 2024-04-21T08:43:16.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:16.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:16.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:16.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:16.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T08:43:16.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:16.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:16.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:16.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:16 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:17.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.067+0000 7ffba5e24700 1 -- 172.21.15.134:0/2200599554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0104640 msgr2=0x7ffba010aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:17.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.067+0000 7ffba5e24700 1 --2- 172.21.15.134:0/2200599554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0104640 0x7ffba010aef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800b0a0 tx=0x7ffb9800b3b0 comp rx=0 tx=0).stop 2024-04-21T08:43:17.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.067+0000 7ffba5e24700 1 -- 172.21.15.134:0/2200599554 shutdown_connections 2024-04-21T08:43:17.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.067+0000 7ffba5e24700 1 --2- 172.21.15.134:0/2200599554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0104640 0x7ffba010aef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:17.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.067+0000 7ffba5e24700 1 --2- 172.21.15.134:0/2200599554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffba0102aa0 0x7ffba0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:17.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.067+0000 7ffba5e24700 1 --2- 172.21.15.134:0/2200599554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffba0103ca0 0x7ffba0104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:17.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.067+0000 7ffba5e24700 1 -- 172.21.15.134:0/2200599554 >> 172.21.15.134:0/2200599554 conn(0x7ffba00fe040 msgr2=0x7ffba0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:17.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.067+0000 7ffba5e24700 1 -- 172.21.15.134:0/2200599554 shutdown_connections 2024-04-21T08:43:17.067 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.067+0000 7ffba5e24700 1 -- 172.21.15.134:0/2200599554 wait complete. 2024-04-21T08:43:17.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.068+0000 7ffba5e24700 1 Processor -- start 2024-04-21T08:43:17.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.068+0000 7ffba5e24700 1 -- start start 2024-04-21T08:43:17.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.068+0000 7ffba5e24700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffba0102aa0 0x7ffba00767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:17.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.068+0000 7ffba5e24700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffba0103ca0 0x7ffba0074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:17.069 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.068+0000 7ffba5e24700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0104640 0x7ffba0075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:17.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.068+0000 7ffba5e24700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffba010e520 con 0x7ffba0102aa0 2024-04-21T08:43:17.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.068+0000 7ffba5e24700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffba010e3a0 con 0x7ffba0103ca0 2024-04-21T08:43:17.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffba5e24700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffba010e6a0 con 0x7ffba0104640 2024-04-21T08:43:17.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0104640 0x7ffba0075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:17.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0104640 0x7ffba0075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:34990/0 (socket says 172.21.15.134:34990) 2024-04-21T08:43:17.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9ffff700 1 -- 172.21.15.134:0/3245289708 learned_addr learned my addr 172.21.15.134:0/3245289708 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-21T08:43:17.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9f7fe700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffba0102aa0 0x7ffba00767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:17.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9effd700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffba0103ca0 0x7ffba0074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:17.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9ffff700 1 -- 172.21.15.134:0/3245289708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffba0103ca0 msgr2=0x7ffba0074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:17.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9ffff700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffba0103ca0 0x7ffba0074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:17.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9ffff700 1 -- 172.21.15.134:0/3245289708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffba0102aa0 msgr2=0x7ffba00767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:17.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9ffff700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffba0102aa0 0x7ffba00767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:17.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9ffff700 1 -- 172.21.15.134:0/3245289708 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba0075c20 con 0x7ffba0104640 2024-04-21T08:43:17.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9f7fe700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffba0102aa0 0x7ffba00767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:17.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9ffff700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0104640 0x7ffba0075370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009be0 tx=0x7ffb98004e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:17.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9cff9700 1 -- 172.21.15.134:0/3245289708 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98021070 con 0x7ffba0104640 2024-04-21T08:43:17.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffba5e24700 1 -- 172.21.15.134:0/3245289708 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba01a2c90 con 0x7ffba0104640 2024-04-21T08:43:17.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9cff9700 1 -- 172.21.15.134:0/3245289708 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb98007c80 con 0x7ffba0104640 2024-04-21T08:43:17.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffb9cff9700 1 -- 172.21.15.134:0/3245289708 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98005040 con 0x7ffba0104640 2024-04-21T08:43:17.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.069+0000 7ffba5e24700 1 -- 172.21.15.134:0/3245289708 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba01a30e0 con 0x7ffba0104640 2024-04-21T08:43:17.071 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.070+0000 7ffba5e24700 1 -- 172.21.15.134:0/3245289708 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba002df80 con 0x7ffba0104640 2024-04-21T08:43:17.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.071+0000 7ffb9cff9700 1 -- 172.21.15.134:0/3245289708 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffb980040b0 con 0x7ffba0104640 2024-04-21T08:43:17.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.072+0000 7ffb9cff9700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ffb880a3c40 0x7ffb880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:17.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.072+0000 7ffb9cff9700 1 -- 172.21.15.134:0/3245289708 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7ffb9800f020 con 0x7ffba0104640 2024-04-21T08:43:17.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.072+0000 7ffb9f7fe700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ffb880a3c40 0x7ffb880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:17.072 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.073+0000 7ffb9f7fe700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ffb880a3c40 0x7ffb880a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffba0103b00 tx=0x7ffb90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:17.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.076+0000 7ffb9cff9700 1 -- 172.21.15.134:0/3245289708 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb9802a020 con 0x7ffba0104640 2024-04-21T08:43:17.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:17.284+0000 7ffba5e24700 1 -- 172.21.15.134:0/3245289708 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ffba0062e80 con 0x7ffb880a3c40 2024-04-21T08:43:18.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:17 smithi134 ceph-mon[31772]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T08:43:18.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:17 smithi134 ceph-mon[31772]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:43:18.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:43:18.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:43:18.219 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:18.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:17 smithi082 ceph-mon[27967]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T08:43:18.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:17 smithi082 ceph-mon[27967]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:43:18.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:43:18.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:43:18.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:18.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:17 smithi106 ceph-mon[31072]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T08:43:18.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:17 smithi106 ceph-mon[31072]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:43:18.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:43:18.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:43:18.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:19.155 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:18 smithi134 ceph-mon[31772]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T08:43:19.155 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:18 smithi134 ceph-mon[31772]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 304 MiB used, 983 GiB / 983 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-21T08:43:19.260 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:18 smithi082 ceph-mon[27967]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T08:43:19.260 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:18 smithi082 ceph-mon[27967]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 304 MiB used, 983 GiB / 983 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-21T08:43:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:18 smithi106 ceph-mon[31072]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T08:43:19.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:18 smithi106 ceph-mon[31072]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 304 MiB used, 983 GiB / 983 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-21T08:43:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:20 smithi134 ceph-mon[31772]: from='client.? 172.21.15.134:0/1590715258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f38b227-d954-4575-aced-ffd87058bf16"}]: dispatch 2024-04-21T08:43:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:20 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f38b227-d954-4575-aced-ffd87058bf16"}]: dispatch 2024-04-21T08:43:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:20 smithi134 ceph-mon[31772]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f38b227-d954-4575-aced-ffd87058bf16"}]': finished 2024-04-21T08:43:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:20 smithi134 ceph-mon[31772]: osdmap e77: 12 total, 11 up, 12 in 2024-04-21T08:43:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:20 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:20.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:20 smithi082 ceph-mon[27967]: from='client.? 172.21.15.134:0/1590715258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f38b227-d954-4575-aced-ffd87058bf16"}]: dispatch 2024-04-21T08:43:20.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:20 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f38b227-d954-4575-aced-ffd87058bf16"}]: dispatch 2024-04-21T08:43:20.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:20 smithi082 ceph-mon[27967]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f38b227-d954-4575-aced-ffd87058bf16"}]': finished 2024-04-21T08:43:20.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:20 smithi082 ceph-mon[27967]: osdmap e77: 12 total, 11 up, 12 in 2024-04-21T08:43:20.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:20 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:20 smithi106 ceph-mon[31072]: from='client.? 172.21.15.134:0/1590715258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f38b227-d954-4575-aced-ffd87058bf16"}]: dispatch 2024-04-21T08:43:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:20 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f38b227-d954-4575-aced-ffd87058bf16"}]: dispatch 2024-04-21T08:43:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:20 smithi106 ceph-mon[31072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f38b227-d954-4575-aced-ffd87058bf16"}]': finished 2024-04-21T08:43:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:20 smithi106 ceph-mon[31072]: osdmap e77: 12 total, 11 up, 12 in 2024-04-21T08:43:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:20 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:21.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:21 smithi134 ceph-mon[31772]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 304 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-04-21T08:43:21.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:21 smithi134 ceph-mon[31772]: from='client.? 172.21.15.134:0/3646720873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:43:21.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:21 smithi082 ceph-mon[27967]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 304 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-04-21T08:43:21.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:21 smithi082 ceph-mon[27967]: from='client.? 172.21.15.134:0/3646720873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:43:21.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:21 smithi106 ceph-mon[31072]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 304 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-04-21T08:43:21.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:21 smithi106 ceph-mon[31072]: from='client.? 172.21.15.134:0/3646720873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:43:23.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:23 smithi082 ceph-mon[27967]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 304 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:43:23.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:23 smithi106 ceph-mon[31072]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 304 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:43:23.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:23 smithi134 ceph-mon[31772]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 304 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:43:25.387 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:25 smithi134 ceph-mon[31772]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-21T08:43:25.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:25 smithi082 ceph-mon[27967]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-21T08:43:25.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:25 smithi106 ceph-mon[31072]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-21T08:43:27.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:27 smithi082 ceph-mon[27967]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:27.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:27 smithi106 ceph-mon[31072]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:27.709 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:27 smithi134 ceph-mon[31772]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T08:43:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:28 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:28.672 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T08:43:28.673 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:28 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:28.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T08:43:28.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:28 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:29 smithi106 ceph-mon[31072]: Deploying daemon osd.11 on smithi134 2024-04-21T08:43:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:29 smithi106 ceph-mon[31072]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:29.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:29 smithi134 ceph-mon[31772]: Deploying daemon osd.11 on smithi134 2024-04-21T08:43:29.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:29 smithi134 ceph-mon[31772]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:29.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:29 smithi082 ceph-mon[27967]: Deploying daemon osd.11 on smithi134 2024-04-21T08:43:29.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:29 smithi082 ceph-mon[27967]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:31.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:31 smithi134 ceph-mon[31772]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:31.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:31 smithi106 ceph-mon[31072]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:31.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:31 smithi082 ceph-mon[27967]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:33.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:33 smithi106 ceph-mon[31072]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:33.631 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:33 smithi134 ceph-mon[31772]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:33 smithi082 ceph-mon[27967]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:34.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:34.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:34.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:34 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:34.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:34 smithi134 ceph-mon[31772]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:35.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:35.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:35.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:34 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:35.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:34 smithi082 ceph-mon[27967]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:35.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:35.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:35.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:34 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:35.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:34 smithi106 ceph-mon[31072]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:37.556 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:37 smithi134 ceph-mon[31772]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:37.557 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:37.557 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:37.557 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:37.557 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:37.557 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:37 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:37.602 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 11 on host 'smithi134' 2024-04-21T08:43:37.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.601+0000 7ffb9cff9700 1 -- 172.21.15.134:0/3245289708 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ffba0062e80 con 0x7ffb880a3c40 2024-04-21T08:43:37.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.603+0000 7ffba5e24700 1 -- 172.21.15.134:0/3245289708 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ffb880a3c40 msgr2=0x7ffb880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:37.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.603+0000 7ffba5e24700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ffb880a3c40 0x7ffb880a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffba0103b00 tx=0x7ffb90009260 comp rx=0 tx=0).stop 2024-04-21T08:43:37.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.603+0000 7ffba5e24700 1 -- 172.21.15.134:0/3245289708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0104640 msgr2=0x7ffba0075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:37.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.603+0000 7ffba5e24700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0104640 0x7ffba0075370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009be0 tx=0x7ffb98004e20 comp rx=0 tx=0).stop 2024-04-21T08:43:37.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.603+0000 7ffba5e24700 1 -- 172.21.15.134:0/3245289708 shutdown_connections 2024-04-21T08:43:37.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.603+0000 7ffba5e24700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ffb880a3c40 0x7ffb880a6100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:37.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.603+0000 7ffba5e24700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0104640 0x7ffba0075370 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:37.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.603+0000 7ffba5e24700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffba0102aa0 0x7ffba00767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:37.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.603+0000 7ffba5e24700 1 --2- 172.21.15.134:0/3245289708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffba0103ca0 0x7ffba0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:37.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.603+0000 7ffba5e24700 1 -- 172.21.15.134:0/3245289708 >> 172.21.15.134:0/3245289708 conn(0x7ffba00fe040 msgr2=0x7ffba01070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:37.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.604+0000 7ffba5e24700 1 -- 172.21.15.134:0/3245289708 shutdown_connections 2024-04-21T08:43:37.605 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:37.604+0000 7ffba5e24700 1 -- 172.21.15.134:0/3245289708 wait complete. 2024-04-21T08:43:37.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:37 smithi082 ceph-mon[27967]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:37.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:37.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:37.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:37.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:37.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:37 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:37 smithi106 ceph-mon[31072]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:37.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:37 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:38.220 DEBUG:teuthology.orchestra.run.smithi134:osd.11> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.11.service 2024-04-21T08:43:38.222 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-21T08:43:38.222 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-21T08:43:38.474 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:43:38.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:38 smithi134 ceph-mon[31772]: from='osd.11 [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:43:38.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:38 smithi134 ceph-mon[31772]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:43:38.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:38 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:38.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:38 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:38.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:38 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:38.597 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:38 smithi082 ceph-mon[27967]: from='osd.11 [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:43:38.597 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:38 smithi082 ceph-mon[27967]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:43:38.598 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:38 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:38.598 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:38 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:38.598 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:38 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:38.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:38 smithi106 ceph-mon[31072]: from='osd.11 [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:43:38.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:38 smithi106 ceph-mon[31072]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:43:38.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:38 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:38.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:38 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:38.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:38 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:39.618 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:39 smithi134 ceph-mon[31772]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:39.618 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:39 smithi134 ceph-mon[31772]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T08:43:39.618 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:39 smithi134 ceph-mon[31772]: osdmap e78: 12 total, 11 up, 12 in 2024-04-21T08:43:39.618 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:39 smithi134 ceph-mon[31772]: from='osd.11 [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:43:39.618 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:39 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:39.618 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:39 smithi134 ceph-mon[31772]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:43:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:39 smithi106 ceph-mon[31072]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:39 smithi106 ceph-mon[31072]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T08:43:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:39 smithi106 ceph-mon[31072]: osdmap e78: 12 total, 11 up, 12 in 2024-04-21T08:43:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:39 smithi106 ceph-mon[31072]: from='osd.11 [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:43:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:39 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:39 smithi106 ceph-mon[31072]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:43:39.643 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:39 smithi082 ceph-mon[27967]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:39.643 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:39 smithi082 ceph-mon[27967]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T08:43:39.643 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:39 smithi082 ceph-mon[27967]: osdmap e78: 12 total, 11 up, 12 in 2024-04-21T08:43:39.643 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:39 smithi082 ceph-mon[27967]: from='osd.11 [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:43:39.643 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:39 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:39.643 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:39 smithi082 ceph-mon[27967]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-21T08:43:40.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 -- 172.21.15.82:0/3248637274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408103d30 msgr2=0x7fe408104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:40.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 --2- 172.21.15.82:0/3248637274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408103d30 0x7fe408104190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009990 tx=0x7fe3f8009ca0 comp rx=0 tx=0).stop 2024-04-21T08:43:40.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 -- 172.21.15.82:0/3248637274 shutdown_connections 2024-04-21T08:43:40.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 --2- 172.21.15.82:0/3248637274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4081046d0 0x7fe40810af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:40.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 --2- 172.21.15.82:0/3248637274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe408100d50 0x7fe408101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:40.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 --2- 172.21.15.82:0/3248637274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408103d30 0x7fe408104190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:40.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 -- 172.21.15.82:0/3248637274 >> 172.21.15.82:0/3248637274 conn(0x7fe4080fc6d0 msgr2=0x7fe4080feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:40.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 -- 172.21.15.82:0/3248637274 shutdown_connections 2024-04-21T08:43:40.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 -- 172.21.15.82:0/3248637274 wait complete. 2024-04-21T08:43:40.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 Processor -- start 2024-04-21T08:43:40.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 -- start start 2024-04-21T08:43:40.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408100d50 0x7fe408103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:40.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe408103d30 0x7fe408101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:40.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.280+0000 7fe40fd11700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4081046d0 0x7fe408102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:40.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40fd11700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe40810e540 con 0x7fe408100d50 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40fd11700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe40810e3c0 con 0x7fe4081046d0 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40fd11700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe40810e6c0 con 0x7fe408103d30 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40daad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408100d50 0x7fe408103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40d2ac700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe408103d30 0x7fe408101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40daad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408100d50 0x7fe408103690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37942/0 (socket says 172.21.15.82:37942) 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40d2ac700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe408103d30 0x7fe408101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:53564/0 (socket says 172.21.15.82:53564) 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40daad700 1 -- 172.21.15.82:0/1726374765 learned_addr learned my addr 172.21.15.82:0/1726374765 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40e2ae700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4081046d0 0x7fe408102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40daad700 1 -- 172.21.15.82:0/1726374765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe408103d30 msgr2=0x7fe408101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40daad700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe408103d30 0x7fe408101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40daad700 1 -- 172.21.15.82:0/1726374765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4081046d0 msgr2=0x7fe408102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40daad700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4081046d0 0x7fe408102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40daad700 1 -- 172.21.15.82:0/1726374765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe408102ad0 con 0x7fe408100d50 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40daad700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408100d50 0x7fe408103690 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe40400c970 tx=0x7fe40400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe3feffd700 1 -- 172.21.15.82:0/1726374765 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404004220 con 0x7fe408100d50 2024-04-21T08:43:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe3feffd700 1 -- 172.21.15.82:0/1726374765 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4040043c0 con 0x7fe408100d50 2024-04-21T08:43:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe3feffd700 1 -- 172.21.15.82:0/1726374765 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404003c20 con 0x7fe408100d50 2024-04-21T08:43:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.281+0000 7fe40fd11700 1 -- 172.21.15.82:0/1726374765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4081a2d40 con 0x7fe408100d50 2024-04-21T08:43:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.282+0000 7fe40fd11700 1 -- 172.21.15.82:0/1726374765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4081a31a0 con 0x7fe408100d50 2024-04-21T08:43:40.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.282+0000 7fe40fd11700 1 -- 172.21.15.82:0/1726374765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe40802df80 con 0x7fe408100d50 2024-04-21T08:43:40.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.284+0000 7fe3feffd700 1 -- 172.21.15.82:0/1726374765 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe4040076d0 con 0x7fe408100d50 2024-04-21T08:43:40.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.285+0000 7fe3feffd700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe3f40a3cc0 0x7fe3f40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:40.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.285+0000 7fe40d2ac700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe3f40a3cc0 0x7fe3f40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:40.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.285+0000 7fe3feffd700 1 -- 172.21.15.82:0/1726374765 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7fe4040cfc40 con 0x7fe408100d50 2024-04-21T08:43:40.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.285+0000 7fe40d2ac700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe3f40a3cc0 0x7fe3f40a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800b5c0 tx=0x7fe3f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:40.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.289+0000 7fe3feffd700 1 -- 172.21.15.82:0/1726374765 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe404014030 con 0x7fe408100d50 2024-04-21T08:43:40.446 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:40 smithi082 ceph-mon[27967]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-21T08:43:40.446 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:40 smithi082 ceph-mon[27967]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T08:43:40.446 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:40.446 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:40 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:40.446 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:40 smithi082 ceph-mon[27967]: from='osd.11 ' entity='osd.11' 2024-04-21T08:43:40.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:40 smithi134 ceph-mon[31772]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-21T08:43:40.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:40 smithi134 ceph-mon[31772]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T08:43:40.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:40.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:40 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:40.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:40 smithi134 ceph-mon[31772]: from='osd.11 ' entity='osd.11' 2024-04-21T08:43:40.476 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:43:40 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:43:40.121+0000 7fcafae01700 -1 osd.11 0 waiting for initial osdmap 2024-04-21T08:43:40.476 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:43:40 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:43:40.138+0000 7fcaf2bec700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:43:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.496+0000 7fe40fd11700 1 -- 172.21.15.82:0/1726374765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe4080637a0 con 0x7fe408100d50 2024-04-21T08:43:40.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.497+0000 7fe3feffd700 1 -- 172.21.15.82:0/1726374765 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7fe40409cc60 con 0x7fe408100d50 2024-04-21T08:43:40.496 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:43:40.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.499+0000 7fe40fd11700 1 -- 172.21.15.82:0/1726374765 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe3f40a3cc0 msgr2=0x7fe3f40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:40.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.499+0000 7fe40fd11700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe3f40a3cc0 0x7fe3f40a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800b5c0 tx=0x7fe3f8005fb0 comp rx=0 tx=0).stop 2024-04-21T08:43:40.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.499+0000 7fe40fd11700 1 -- 172.21.15.82:0/1726374765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408100d50 msgr2=0x7fe408103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:40.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.499+0000 7fe40fd11700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408100d50 0x7fe408103690 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe40400c970 tx=0x7fe40400cc80 comp rx=0 tx=0).stop 2024-04-21T08:43:40.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.499+0000 7fe40fd11700 1 -- 172.21.15.82:0/1726374765 shutdown_connections 2024-04-21T08:43:40.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.499+0000 7fe40fd11700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe3f40a3cc0 0x7fe3f40a6180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:40.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.499+0000 7fe40fd11700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4081046d0 0x7fe408102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:40.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.499+0000 7fe40fd11700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408100d50 0x7fe408103690 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:40.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.499+0000 7fe40fd11700 1 --2- 172.21.15.82:0/1726374765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe408103d30 0x7fe408101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:40.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.499+0000 7fe40fd11700 1 -- 172.21.15.82:0/1726374765 >> 172.21.15.82:0/1726374765 conn(0x7fe4080fc6d0 msgr2=0x7fe4081070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:40.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.499+0000 7fe40fd11700 1 -- 172.21.15.82:0/1726374765 shutdown_connections 2024-04-21T08:43:40.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:40.499+0000 7fe40fd11700 1 -- 172.21.15.82:0/1726374765 wait complete. 2024-04-21T08:43:40.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:40 smithi106 ceph-mon[31072]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-21T08:43:40.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:40 smithi106 ceph-mon[31072]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T08:43:40.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:40.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:40 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:40.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:40 smithi106 ceph-mon[31072]: from='osd.11 ' entity='osd.11' 2024-04-21T08:43:41.263 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1713688994,"num_in_osds":12,"osd_in_since":1713689000,"num_remapped_pgs":0} 2024-04-21T08:43:41.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: purged_snaps scrub starts 2024-04-21T08:43:41.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: purged_snaps scrub ok 2024-04-21T08:43:41.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:41.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:41.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1726374765' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T08:43:41.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: osd.11 [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879] boot 2024-04-21T08:43:41.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: osdmap e80: 12 total, 12 up, 12 in 2024-04-21T08:43:41.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:41.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:41.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:41.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:41.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:41.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:41.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:41.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:41.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:41.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:41.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:41 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:41.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: purged_snaps scrub starts 2024-04-21T08:43:41.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: purged_snaps scrub ok 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1726374765' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: osd.11 [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879] boot 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: osdmap e80: 12 total, 12 up, 12 in 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:41.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:41 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: purged_snaps scrub starts 2024-04-21T08:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: purged_snaps scrub ok 2024-04-21T08:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1726374765' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T08:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: osd.11 [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879] boot 2024-04-21T08:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: osdmap e80: 12 total, 12 up, 12 in 2024-04-21T08:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:41.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:41.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:43:41.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:41.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:41.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:41.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:41 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:42.264 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-21T08:43:42.518 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:43:42.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:42 smithi082 ceph-mon[27967]: Detected new or changed devices on smithi134 2024-04-21T08:43:42.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:42 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi134 to 4272M 2024-04-21T08:43:42.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:42 smithi134 ceph-mon[31772]: Detected new or changed devices on smithi134 2024-04-21T08:43:42.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:42 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi134 to 4272M 2024-04-21T08:43:42.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:42 smithi106 ceph-mon[31072]: Detected new or changed devices on smithi134 2024-04-21T08:43:42.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:42 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi134 to 4272M 2024-04-21T08:43:43.671 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:43 smithi082 ceph-mon[27967]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:43.672 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:43 smithi082 ceph-mon[27967]: osdmap e81: 12 total, 12 up, 12 in 2024-04-21T08:43:43.672 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:43 smithi134 ceph-mon[31772]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:43.672 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:43 smithi134 ceph-mon[31772]: osdmap e81: 12 total, 12 up, 12 in 2024-04-21T08:43:43.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:43 smithi106 ceph-mon[31072]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:43.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:43 smithi106 ceph-mon[31072]: osdmap e81: 12 total, 12 up, 12 in 2024-04-21T08:43:44.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.231+0000 7f13e793c700 1 -- 172.21.15.82:0/3708543347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e0104640 msgr2=0x7f13e010aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.231+0000 7f13e793c700 1 --2- 172.21.15.82:0/3708543347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e0104640 0x7f13e010aef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f13d800b0d0 tx=0x7f13d800b3e0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.231+0000 7f13e793c700 1 -- 172.21.15.82:0/3708543347 shutdown_connections 2024-04-21T08:43:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.231+0000 7f13e793c700 1 --2- 172.21.15.82:0/3708543347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e0104640 0x7f13e010aef0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.231+0000 7f13e793c700 1 --2- 172.21.15.82:0/3708543347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13e0102aa0 0x7f13e0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.231+0000 7f13e793c700 1 --2- 172.21.15.82:0/3708543347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13e0103ca0 0x7f13e0104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.231+0000 7f13e793c700 1 -- 172.21.15.82:0/3708543347 >> 172.21.15.82:0/3708543347 conn(0x7f13e00fe040 msgr2=0x7f13e0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.232+0000 7f13e793c700 1 -- 172.21.15.82:0/3708543347 shutdown_connections 2024-04-21T08:43:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.232+0000 7f13e793c700 1 -- 172.21.15.82:0/3708543347 wait complete. 2024-04-21T08:43:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.232+0000 7f13e793c700 1 Processor -- start 2024-04-21T08:43:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.232+0000 7f13e793c700 1 -- start start 2024-04-21T08:43:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.232+0000 7f13e793c700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13e0102aa0 0x7f13e0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.232+0000 7f13e793c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13e0103ca0 0x7f13e0199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.232+0000 7f13e793c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e0104640 0x7f13e01a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.232+0000 7f13e793c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f13e010e5b0 con 0x7f13e0104640 2024-04-21T08:43:44.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e793c700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f13e010e430 con 0x7f13e0102aa0 2024-04-21T08:43:44.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e793c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13e010e730 con 0x7f13e0103ca0 2024-04-21T08:43:44.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e5ed9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e0104640 0x7f13e01a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e5ed9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e0104640 0x7f13e01a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42878/0 (socket says 172.21.15.82:42878) 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e5ed9700 1 -- 172.21.15.82:0/60405905 learned_addr learned my addr 172.21.15.82:0/60405905 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e56d8700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13e0102aa0 0x7f13e0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e4ed7700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13e0103ca0 0x7f13e0199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e5ed9700 1 -- 172.21.15.82:0/60405905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13e0103ca0 msgr2=0x7f13e0199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e5ed9700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13e0103ca0 0x7f13e0199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e5ed9700 1 -- 172.21.15.82:0/60405905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13e0102aa0 msgr2=0x7f13e0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e5ed9700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13e0102aa0 0x7f13e0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e5ed9700 1 -- 172.21.15.82:0/60405905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e01a0860 con 0x7f13e0104640 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e56d8700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13e0102aa0 0x7f13e0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e5ed9700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e0104640 0x7f13e01a0190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009fd0 tx=0x7f13d8004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13d67fc700 1 -- 172.21.15.82:0/60405905 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d801f030 con 0x7f13e0104640 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13d67fc700 1 -- 172.21.15.82:0/60405905 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13d8007bb0 con 0x7f13e0104640 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e793c700 1 -- 172.21.15.82:0/60405905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e01a0af0 con 0x7f13e0104640 2024-04-21T08:43:44.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13d67fc700 1 -- 172.21.15.82:0/60405905 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8004820 con 0x7f13e0104640 2024-04-21T08:43:44.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.233+0000 7f13e793c700 1 -- 172.21.15.82:0/60405905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e01a0fc0 con 0x7f13e0104640 2024-04-21T08:43:44.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.234+0000 7f13e793c700 1 -- 172.21.15.82:0/60405905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e0069730 con 0x7f13e0104640 2024-04-21T08:43:44.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.235+0000 7f13d67fc700 1 -- 172.21.15.82:0/60405905 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f13d8003a00 con 0x7f13e0104640 2024-04-21T08:43:44.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.236+0000 7f13d67fc700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13cc0a3cc0 0x7f13cc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.236+0000 7f13e56d8700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13cc0a3cc0 0x7f13cc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.236+0000 7f13d67fc700 1 -- 172.21.15.82:0/60405905 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f13d80d0ea0 con 0x7f13e0104640 2024-04-21T08:43:44.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.236+0000 7f13e56d8700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13cc0a3cc0 0x7f13cc0a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f13e019a0f0 tx=0x7f13dc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.241+0000 7f13d67fc700 1 -- 172.21.15.82:0/60405905 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13d809ddb0 con 0x7f13e0104640 2024-04-21T08:43:44.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.446+0000 7f13e793c700 1 -- 172.21.15.82:0/60405905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f13e0063650 con 0x7f13e0104640 2024-04-21T08:43:44.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.446+0000 7f13d67fc700 1 -- 172.21.15.82:0/60405905 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f13d809d6b0 con 0x7f13e0104640 2024-04-21T08:43:44.446 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:43:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.448+0000 7f13e793c700 1 -- 172.21.15.82:0/60405905 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13cc0a3cc0 msgr2=0x7f13cc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.448+0000 7f13e793c700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13cc0a3cc0 0x7f13cc0a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f13e019a0f0 tx=0x7f13dc008040 comp rx=0 tx=0).stop 2024-04-21T08:43:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.448+0000 7f13e793c700 1 -- 172.21.15.82:0/60405905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e0104640 msgr2=0x7f13e01a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.448+0000 7f13e793c700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e0104640 0x7f13e01a0190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009fd0 tx=0x7f13d8004ef0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.448+0000 7f13e793c700 1 -- 172.21.15.82:0/60405905 shutdown_connections 2024-04-21T08:43:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.448+0000 7f13e793c700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13cc0a3cc0 0x7f13cc0a6180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.448+0000 7f13e793c700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e0104640 0x7f13e01a0190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.448+0000 7f13e793c700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13e0102aa0 0x7f13e0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.448+0000 7f13e793c700 1 --2- 172.21.15.82:0/60405905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13e0103ca0 0x7f13e0199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.448+0000 7f13e793c700 1 -- 172.21.15.82:0/60405905 >> 172.21.15.82:0/60405905 conn(0x7f13e00fe040 msgr2=0x7f13e0107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.448+0000 7f13e793c700 1 -- 172.21.15.82:0/60405905 shutdown_connections 2024-04-21T08:43:44.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:44.448+0000 7f13e793c700 1 -- 172.21.15.82:0/60405905 wait complete. 2024-04-21T08:43:45.171 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713689021,"num_in_osds":12,"osd_in_since":1713689000,"num_remapped_pgs":0} 2024-04-21T08:43:45.172 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T08:43:45.406 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:43:45.527 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:45 smithi082 ceph-mon[27967]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:45.528 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:45 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/60405905' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T08:43:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:45 smithi134 ceph-mon[31772]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:45 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/60405905' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T08:43:45.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:45 smithi106 ceph-mon[31072]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:45.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:45 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/60405905' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T08:43:47.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.203+0000 7f4cb2481700 1 -- 172.21.15.82:0/1662126859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cac06a750 msgr2=0x7f4cac06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:47.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.203+0000 7f4cb2481700 1 --2- 172.21.15.82:0/1662126859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cac06a750 0x7f4cac06abb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c009990 tx=0x7f4c9c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:43:47.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.203+0000 7f4cb2481700 1 -- 172.21.15.82:0/1662126859 shutdown_connections 2024-04-21T08:43:47.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.203+0000 7f4cb2481700 1 --2- 172.21.15.82:0/1662126859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cac06b0f0 0x7f4cac10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:47.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.203+0000 7f4cb2481700 1 --2- 172.21.15.82:0/1662126859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cac108a30 0x7f4cac108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:47.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.203+0000 7f4cb2481700 1 --2- 172.21.15.82:0/1662126859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cac06a750 0x7f4cac06abb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:47.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.203+0000 7f4cb2481700 1 -- 172.21.15.82:0/1662126859 >> 172.21.15.82:0/1662126859 conn(0x7f4cac0788f0 msgr2=0x7f4cac078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:47.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.203+0000 7f4cb2481700 1 -- 172.21.15.82:0/1662126859 shutdown_connections 2024-04-21T08:43:47.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.203+0000 7f4cb2481700 1 -- 172.21.15.82:0/1662126859 wait complete. 2024-04-21T08:43:47.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb2481700 1 Processor -- start 2024-04-21T08:43:47.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb2481700 1 -- start start 2024-04-21T08:43:47.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb2481700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cac06a750 0x7f4cac101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:47.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb2481700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cac06b0f0 0x7f4cac101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:47.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb2481700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cac108a30 0x7f4cac102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:47.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb2481700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac112c50 con 0x7f4cac108a30 2024-04-21T08:43:47.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb2481700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac112ad0 con 0x7f4cac06a750 2024-04-21T08:43:47.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb2481700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac112dd0 con 0x7f4cac06b0f0 2024-04-21T08:43:47.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb0a1e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cac108a30 0x7f4cac102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:47.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb0a1e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cac108a30 0x7f4cac102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42896/0 (socket says 172.21.15.82:42896) 2024-04-21T08:43:47.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb0a1e700 1 -- 172.21.15.82:0/3935181421 learned_addr learned my addr 172.21.15.82:0/3935181421 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:43:47.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cab7fe700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cac06b0f0 0x7f4cac101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:47.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cabfff700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cac06a750 0x7f4cac101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:47.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb0a1e700 1 -- 172.21.15.82:0/3935181421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cac06b0f0 msgr2=0x7f4cac101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:47.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb0a1e700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cac06b0f0 0x7f4cac101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:47.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb0a1e700 1 -- 172.21.15.82:0/3935181421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cac06a750 msgr2=0x7f4cac101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:47.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.204+0000 7f4cb0a1e700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cac06a750 0x7f4cac101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:47.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.205+0000 7f4cb0a1e700 1 -- 172.21.15.82:0/3935181421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac1a76c0 con 0x7f4cac108a30 2024-04-21T08:43:47.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.205+0000 7f4cb0a1e700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cac108a30 0x7f4cac102490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4007ee0 tx=0x7f4ca400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:47.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.205+0000 7f4ca97fa700 1 -- 172.21.15.82:0/3935181421 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca400dbd0 con 0x7f4cac108a30 2024-04-21T08:43:47.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.205+0000 7f4ca97fa700 1 -- 172.21.15.82:0/3935181421 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca400dd70 con 0x7f4cac108a30 2024-04-21T08:43:47.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.205+0000 7f4cb2481700 1 -- 172.21.15.82:0/3935181421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cac1a79b0 con 0x7f4cac108a30 2024-04-21T08:43:47.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.205+0000 7f4ca97fa700 1 -- 172.21.15.82:0/3935181421 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4005990 con 0x7f4cac108a30 2024-04-21T08:43:47.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.205+0000 7f4cb2481700 1 -- 172.21.15.82:0/3935181421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cac1a7f40 con 0x7f4cac108a30 2024-04-21T08:43:47.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.206+0000 7f4cb2481700 1 -- 172.21.15.82:0/3935181421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cac10b030 con 0x7f4cac108a30 2024-04-21T08:43:47.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.207+0000 7f4ca97fa700 1 -- 172.21.15.82:0/3935181421 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4ca401a020 con 0x7f4cac108a30 2024-04-21T08:43:47.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.207+0000 7f4ca97fa700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f4c940a3cc0 0x7f4c940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:47.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.208+0000 7f4cabfff700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f4c940a3cc0 0x7f4c940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:47.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.208+0000 7f4ca97fa700 1 -- 172.21.15.82:0/3935181421 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4ca40cf6e0 con 0x7f4cac108a30 2024-04-21T08:43:47.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.208+0000 7f4cabfff700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f4c940a3cc0 0x7f4c940a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0006fd0 tx=0x7f4ca0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:47.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.212+0000 7f4ca97fa700 1 -- 172.21.15.82:0/3935181421 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ca4010070 con 0x7f4cac108a30 2024-04-21T08:43:47.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.411+0000 7f4cb2481700 1 -- 172.21.15.82:0/3935181421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cac10aa10 con 0x7f4cac108a30 2024-04-21T08:43:47.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.412+0000 7f4ca97fa700 1 -- 172.21.15.82:0/3935181421 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18468 (secure 0 0 0) 0x7f4ca409c560 con 0x7f4cac108a30 2024-04-21T08:43:47.412 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:43:47.412 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":81,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:43:42.372753+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:43:47.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.414+0000 7f4cb2481700 1 -- 172.21.15.82:0/3935181421 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f4c940a3cc0 msgr2=0x7f4c940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:47.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.414+0000 7f4cb2481700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f4c940a3cc0 0x7f4c940a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0006fd0 tx=0x7f4ca0008040 comp rx=0 tx=0).stop 2024-04-21T08:43:47.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.414+0000 7f4cb2481700 1 -- 172.21.15.82:0/3935181421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cac108a30 msgr2=0x7f4cac102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:47.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.414+0000 7f4cb2481700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cac108a30 0x7f4cac102490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4007ee0 tx=0x7f4ca400d3b0 comp rx=0 tx=0).stop 2024-04-21T08:43:47.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.414+0000 7f4cb2481700 1 -- 172.21.15.82:0/3935181421 shutdown_connections 2024-04-21T08:43:47.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.414+0000 7f4cb2481700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f4c940a3cc0 0x7f4c940a6180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:47.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.414+0000 7f4cb2481700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cac108a30 0x7f4cac102490 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:47.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.414+0000 7f4cb2481700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4cac06a750 0x7f4cac101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:47.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.414+0000 7f4cb2481700 1 --2- 172.21.15.82:0/3935181421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cac06b0f0 0x7f4cac101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:47.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.414+0000 7f4cb2481700 1 -- 172.21.15.82:0/3935181421 >> 172.21.15.82:0/3935181421 conn(0x7f4cac0788f0 msgr2=0x7f4cac0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:47.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.414+0000 7f4cb2481700 1 -- 172.21.15.82:0/3935181421 shutdown_connections 2024-04-21T08:43:47.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:47.414+0000 7f4cb2481700 1 -- 172.21.15.82:0/3935181421 wait complete. 2024-04-21T08:43:47.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:47 smithi134 ceph-mon[31772]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:47.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:47 smithi082 ceph-mon[27967]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:47.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:47 smithi106 ceph-mon[31072]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:48.197 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T08:43:48.197 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-21T08:43:48.433 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:43:48.554 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:48 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3935181421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:43:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:48 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3935181421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:43:48.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:48 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3935181421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:43:49.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:49 smithi082 ceph-mon[27967]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:49.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:49 smithi134 ceph-mon[31772]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:49.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:49 smithi106 ceph-mon[31072]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:50.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.322+0000 7fa51b2de700 1 -- 172.21.15.82:0/1903538751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa514108a50 msgr2=0x7fa514108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:50.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.322+0000 7fa51b2de700 1 --2- 172.21.15.82:0/1903538751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa514108a50 0x7fa514108e30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa510009990 tx=0x7fa510009ca0 comp rx=0 tx=0).stop 2024-04-21T08:43:50.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.322+0000 7fa51b2de700 1 -- 172.21.15.82:0/1903538751 shutdown_connections 2024-04-21T08:43:50.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.322+0000 7fa51b2de700 1 --2- 172.21.15.82:0/1903538751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa514103450 0x7fa51410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:50.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.322+0000 7fa51b2de700 1 --2- 172.21.15.82:0/1903538751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa514108a50 0x7fa514108e30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:50.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.322+0000 7fa51b2de700 1 --2- 172.21.15.82:0/1903538751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa514102a90 0x7fa514102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:50.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.322+0000 7fa51b2de700 1 -- 172.21.15.82:0/1903538751 >> 172.21.15.82:0/1903538751 conn(0x7fa514079ba0 msgr2=0x7fa514079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:50.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.322+0000 7fa51b2de700 1 -- 172.21.15.82:0/1903538751 shutdown_connections 2024-04-21T08:43:50.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.322+0000 7fa51b2de700 1 -- 172.21.15.82:0/1903538751 wait complete. 2024-04-21T08:43:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa51b2de700 1 Processor -- start 2024-04-21T08:43:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa51b2de700 1 -- start start 2024-04-21T08:43:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa51b2de700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa514102a90 0x7fa514199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa51b2de700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa514103450 0x7fa5141996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa51b2de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa514108a50 0x7fa51419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa51b2de700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa514112c80 con 0x7fa514108a50 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa51b2de700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa514112b00 con 0x7fa514102a90 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa51b2de700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa514112e00 con 0x7fa514103450 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa51987b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa514108a50 0x7fa51419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa51987b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa514108a50 0x7fa51419da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42924/0 (socket says 172.21.15.82:42924) 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa51987b700 1 -- 172.21.15.82:0/4232365736 learned_addr learned my addr 172.21.15.82:0/4232365736 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa51907a700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa514102a90 0x7fa514199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.323+0000 7fa518879700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa514103450 0x7fa5141996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.324+0000 7fa51987b700 1 -- 172.21.15.82:0/4232365736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa514103450 msgr2=0x7fa5141996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.324+0000 7fa51987b700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa514103450 0x7fa5141996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.324+0000 7fa51987b700 1 -- 172.21.15.82:0/4232365736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa514102a90 msgr2=0x7fa514199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.324+0000 7fa51987b700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa514102a90 0x7fa514199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.324+0000 7fa51987b700 1 -- 172.21.15.82:0/4232365736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51419e1a0 con 0x7fa514108a50 2024-04-21T08:43:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.324+0000 7fa51987b700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa514108a50 0x7fa51419da60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00b970 tx=0x7fa50c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.324+0000 7fa50a7fc700 1 -- 172.21.15.82:0/4232365736 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c019070 con 0x7fa514108a50 2024-04-21T08:43:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.324+0000 7fa50a7fc700 1 -- 172.21.15.82:0/4232365736 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa50c00cde0 con 0x7fa514108a50 2024-04-21T08:43:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.324+0000 7fa51b2de700 1 -- 172.21.15.82:0/4232365736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51419e490 con 0x7fa514108a50 2024-04-21T08:43:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.324+0000 7fa50a7fc700 1 -- 172.21.15.82:0/4232365736 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c007370 con 0x7fa514108a50 2024-04-21T08:43:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.324+0000 7fa51b2de700 1 -- 172.21.15.82:0/4232365736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5141a5d10 con 0x7fa514108a50 2024-04-21T08:43:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.325+0000 7fa51b2de700 1 -- 172.21.15.82:0/4232365736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa514069730 con 0x7fa514108a50 2024-04-21T08:43:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.326+0000 7fa50a7fc700 1 -- 172.21.15.82:0/4232365736 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa50c01c070 con 0x7fa514108a50 2024-04-21T08:43:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.326+0000 7fa50a7fc700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa5000a3c40 0x7fa5000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.327+0000 7fa51907a700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa5000a3c40 0x7fa5000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.327+0000 7fa50a7fc700 1 -- 172.21.15.82:0/4232365736 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa50c0cf410 con 0x7fa514108a50 2024-04-21T08:43:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.327+0000 7fa51907a700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa5000a3c40 0x7fa5000a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa51000b5c0 tx=0x7fa510005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:50.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.331+0000 7fa50a7fc700 1 -- 172.21.15.82:0/4232365736 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa50c09c290 con 0x7fa514108a50 2024-04-21T08:43:50.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.528+0000 7fa51b2de700 1 -- 172.21.15.82:0/4232365736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa51419a4f0 con 0x7fa514108a50 2024-04-21T08:43:50.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.529+0000 7fa50a7fc700 1 -- 172.21.15.82:0/4232365736 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fa50c09bb90 con 0x7fa514108a50 2024-04-21T08:43:50.528 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:43:50.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.530+0000 7fa51b2de700 1 -- 172.21.15.82:0/4232365736 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa5000a3c40 msgr2=0x7fa5000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:50.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.530+0000 7fa51b2de700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa5000a3c40 0x7fa5000a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa51000b5c0 tx=0x7fa510005960 comp rx=0 tx=0).stop 2024-04-21T08:43:50.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.530+0000 7fa51b2de700 1 -- 172.21.15.82:0/4232365736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa514108a50 msgr2=0x7fa51419da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:50.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.530+0000 7fa51b2de700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa514108a50 0x7fa51419da60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00b970 tx=0x7fa50c00bc80 comp rx=0 tx=0).stop 2024-04-21T08:43:50.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.531+0000 7fa51b2de700 1 -- 172.21.15.82:0/4232365736 shutdown_connections 2024-04-21T08:43:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.531+0000 7fa51b2de700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa5000a3c40 0x7fa5000a6100 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.531+0000 7fa51b2de700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa514108a50 0x7fa51419da60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.531+0000 7fa51b2de700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa514102a90 0x7fa514199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.531+0000 7fa51b2de700 1 --2- 172.21.15.82:0/4232365736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa514103450 0x7fa5141996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.531+0000 7fa51b2de700 1 -- 172.21.15.82:0/4232365736 >> 172.21.15.82:0/4232365736 conn(0x7fa514079ba0 msgr2=0x7fa5141018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.531+0000 7fa51b2de700 1 -- 172.21.15.82:0/4232365736 shutdown_connections 2024-04-21T08:43:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:50.531+0000 7fa51b2de700 1 -- 172.21.15.82:0/4232365736 wait complete. 2024-04-21T08:43:51.463 INFO:tasks.cephadm:Adding mds.a on smithi082 2024-04-21T08:43:51.464 INFO:tasks.cephadm:Adding mds.d on smithi082 2024-04-21T08:43:51.464 INFO:tasks.cephadm:Adding mds.g on smithi082 2024-04-21T08:43:51.464 INFO:tasks.cephadm:Adding mds.j on smithi082 2024-04-21T08:43:51.464 INFO:tasks.cephadm:Adding mds.b on smithi106 2024-04-21T08:43:51.464 INFO:tasks.cephadm:Adding mds.e on smithi106 2024-04-21T08:43:51.464 INFO:tasks.cephadm:Adding mds.h on smithi106 2024-04-21T08:43:51.464 INFO:tasks.cephadm:Adding mds.k on smithi106 2024-04-21T08:43:51.464 INFO:tasks.cephadm:Adding mds.c on smithi134 2024-04-21T08:43:51.464 INFO:tasks.cephadm:Adding mds.f on smithi134 2024-04-21T08:43:51.464 INFO:tasks.cephadm:Adding mds.i on smithi134 2024-04-21T08:43:51.464 INFO:tasks.cephadm:Adding mds.l on smithi134 2024-04-21T08:43:51.464 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l' 2024-04-21T08:43:51.715 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:43:51.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:51 smithi134 ceph-mon[31772]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:51.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:51 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/4232365736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:43:51.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:51 smithi082 ceph-mon[27967]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:51.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:51 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4232365736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:43:51.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:51 smithi106 ceph-mon[31072]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:51.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:51 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/4232365736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:43:53.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.292+0000 7f8bf91f7700 1 -- 172.21.15.134:0/2492752364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf41014f0 msgr2=0x7f8bf410b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.292+0000 7f8bf91f7700 1 --2- 172.21.15.134:0/2492752364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf41014f0 0x7f8bf410b0a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00b0a0 tx=0x7f8bec00b3b0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.292+0000 7f8bf91f7700 1 -- 172.21.15.134:0/2492752364 shutdown_connections 2024-04-21T08:43:53.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.292+0000 7f8bf91f7700 1 --2- 172.21.15.134:0/2492752364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf41014f0 0x7f8bf410b0a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.292+0000 7f8bf91f7700 1 --2- 172.21.15.134:0/2492752364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf41040b0 0x7f8bf41044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.292+0000 7f8bf91f7700 1 --2- 172.21.15.134:0/2492752364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8bf4100b30 0x7f8bf4100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.292+0000 7f8bf91f7700 1 -- 172.21.15.134:0/2492752364 >> 172.21.15.134:0/2492752364 conn(0x7f8bf40fc6b0 msgr2=0x7f8bf40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:53.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.292+0000 7f8bf91f7700 1 -- 172.21.15.134:0/2492752364 shutdown_connections 2024-04-21T08:43:53.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.292+0000 7f8bf91f7700 1 -- 172.21.15.134:0/2492752364 wait complete. 2024-04-21T08:43:53.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf91f7700 1 Processor -- start 2024-04-21T08:43:53.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf91f7700 1 -- start start 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf91f7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf4100b30 0x7f8bf41037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf91f7700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8bf41014f0 0x7f8bf4101e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf91f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf41040b0 0x7f8bf4102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf91f7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf410e590 con 0x7f8bf41040b0 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf91f7700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf410e410 con 0x7f8bf41014f0 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf91f7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf410e710 con 0x7f8bf4100b30 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf2d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf4100b30 0x7f8bf41037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf2d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf4100b30 0x7f8bf41037b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:50600/0 (socket says 172.21.15.134:50600) 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf2d9d700 1 -- 172.21.15.134:0/1757867660 learned_addr learned my addr 172.21.15.134:0/1757867660 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf359e700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf41040b0 0x7f8bf4102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bea59c700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8bf41014f0 0x7f8bf4101e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf2d9d700 1 -- 172.21.15.134:0/1757867660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8bf41014f0 msgr2=0x7f8bf4101e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf2d9d700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8bf41014f0 0x7f8bf4101e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf2d9d700 1 -- 172.21.15.134:0/1757867660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf41040b0 msgr2=0x7f8bf4102340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf2d9d700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf41040b0 0x7f8bf4102340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.293+0000 7f8bf2d9d700 1 -- 172.21.15.134:0/1757867660 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf4102880 con 0x7f8bf4100b30 2024-04-21T08:43:53.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.294+0000 7f8bf2d9d700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf4100b30 0x7f8bf41037b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8be400e970 tx=0x7f8be400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:53.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.294+0000 7f8bf0d99700 1 -- 172.21.15.134:0/1757867660 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be400cbd0 con 0x7f8bf4100b30 2024-04-21T08:43:53.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.294+0000 7f8bf0d99700 1 -- 172.21.15.134:0/1757867660 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be40044d0 con 0x7f8bf4100b30 2024-04-21T08:43:53.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.294+0000 7f8bf91f7700 1 -- 172.21.15.134:0/1757867660 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf4102b70 con 0x7f8bf4100b30 2024-04-21T08:43:53.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.294+0000 7f8bf0d99700 1 -- 172.21.15.134:0/1757867660 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4003df0 con 0x7f8bf4100b30 2024-04-21T08:43:53.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.294+0000 7f8bf91f7700 1 -- 172.21.15.134:0/1757867660 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf41a7660 con 0x7f8bf4100b30 2024-04-21T08:43:53.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.295+0000 7f8bf91f7700 1 -- 172.21.15.134:0/1757867660 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf4069730 con 0x7f8bf4100b30 2024-04-21T08:43:53.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.296+0000 7f8bf0d99700 1 -- 172.21.15.134:0/1757867660 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8be4003f90 con 0x7f8bf4100b30 2024-04-21T08:43:53.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.296+0000 7f8bf0d99700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8be00a3c40 0x7f8be00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:53.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.297+0000 7f8bf0d99700 1 -- 172.21.15.134:0/1757867660 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8be40d01d0 con 0x7f8bf4100b30 2024-04-21T08:43:53.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.297+0000 7f8bea59c700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8be00a3c40 0x7f8be00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:53.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.297+0000 7f8bea59c700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8be00a3c40 0x7f8be00a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc005950 tx=0x7f8bdc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:53.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.301+0000 7f8bf0d99700 1 -- 172.21.15.134:0/1757867660 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8be409d050 con 0x7f8bf4100b30 2024-04-21T08:43:53.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.511+0000 7f8bf91f7700 1 -- 172.21.15.134:0/1757867660 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}) -- 0x7f8bf4062f80 con 0x7f8be00a3c40 2024-04-21T08:43:53.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.519+0000 7f8bf0d99700 1 -- 172.21.15.134:0/1757867660 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8bf4062f80 con 0x7f8be00a3c40 2024-04-21T08:43:53.518 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mds.all update... 2024-04-21T08:43:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.521+0000 7f8bf91f7700 1 -- 172.21.15.134:0/1757867660 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8be00a3c40 msgr2=0x7f8be00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.521+0000 7f8bf91f7700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8be00a3c40 0x7f8be00a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc005950 tx=0x7f8bdc0058e0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.521+0000 7f8bf91f7700 1 -- 172.21.15.134:0/1757867660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf4100b30 msgr2=0x7f8bf41037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:53.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.521+0000 7f8bf91f7700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf4100b30 0x7f8bf41037b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8be400e970 tx=0x7f8be400ec80 comp rx=0 tx=0).stop 2024-04-21T08:43:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.521+0000 7f8bf91f7700 1 -- 172.21.15.134:0/1757867660 shutdown_connections 2024-04-21T08:43:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.521+0000 7f8bf91f7700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8be00a3c40 0x7f8be00a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.521+0000 7f8bf91f7700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf41040b0 0x7f8bf4102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.521+0000 7f8bf91f7700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bf4100b30 0x7f8bf41037b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.521+0000 7f8bf91f7700 1 --2- 172.21.15.134:0/1757867660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8bf41014f0 0x7f8bf4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.521+0000 7f8bf91f7700 1 -- 172.21.15.134:0/1757867660 >> 172.21.15.134:0/1757867660 conn(0x7f8bf40fc6b0 msgr2=0x7f8bf4107150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.521+0000 7f8bf91f7700 1 -- 172.21.15.134:0/1757867660 shutdown_connections 2024-04-21T08:43:53.521 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-21T08:43:53.521+0000 7f8bf91f7700 1 -- 172.21.15.134:0/1757867660 wait complete. 2024-04-21T08:43:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:53 smithi134 ceph-mon[31772]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:53.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:53 smithi082 ceph-mon[27967]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:53.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:53 smithi106 ceph-mon[31072]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:54.185 DEBUG:teuthology.orchestra.run.smithi082:mds.a> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.a.service 2024-04-21T08:43:54.189 DEBUG:teuthology.orchestra.run.smithi082:mds.d> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.d.service 2024-04-21T08:43:54.231 DEBUG:teuthology.orchestra.run.smithi082:mds.g> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.g.service 2024-04-21T08:43:54.276 DEBUG:teuthology.orchestra.run.smithi082:mds.j> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.j.service 2024-04-21T08:43:54.319 DEBUG:teuthology.orchestra.run.smithi106:mds.b> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.b.service 2024-04-21T08:43:54.321 DEBUG:teuthology.orchestra.run.smithi106:mds.e> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.e.service 2024-04-21T08:43:54.365 DEBUG:teuthology.orchestra.run.smithi106:mds.h> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.h.service 2024-04-21T08:43:54.407 DEBUG:teuthology.orchestra.run.smithi106:mds.k> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.k.service 2024-04-21T08:43:54.450 DEBUG:teuthology.orchestra.run.smithi134:mds.c> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.c.service 2024-04-21T08:43:54.453 DEBUG:teuthology.orchestra.run.smithi134:mds.f> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.f.service 2024-04-21T08:43:54.496 DEBUG:teuthology.orchestra.run.smithi134:mds.i> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.i.service 2024-04-21T08:43:54.539 DEBUG:teuthology.orchestra.run.smithi134:mds.l> sudo journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.l.service 2024-04-21T08:43:54.582 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-21T08:43:54.582 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:43:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:54 smithi106 ceph-mon[31072]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:43:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:54 smithi106 ceph-mon[31072]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-04-21T08:43:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:54 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:54 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:54 smithi106 ceph-mon[31072]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:54 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:54 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:54 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:54 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:43:54.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:54 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:43:54.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:54 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:54.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:54 smithi106 ceph-mon[31072]: Deploying daemon mds.l on smithi134 2024-04-21T08:43:54.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.691+0000 7f0d239b2640 1 -- 172.21.15.82:0/3736392855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1c101fd0 msgr2=0x7f0d1c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.691+0000 7f0d239b2640 1 --2- 172.21.15.82:0/3736392855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1c101fd0 0x7f0d1c102450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009900 tx=0x7f0d0c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:54.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.691+0000 7f0d239b2640 1 -- 172.21.15.82:0/3736392855 shutdown_connections 2024-04-21T08:43:54.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.691+0000 7f0d239b2640 1 --2- 172.21.15.82:0/3736392855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d1c102990 0x7f0d1c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.691+0000 7f0d239b2640 1 --2- 172.21.15.82:0/3736392855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1c101fd0 0x7f0d1c102450 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.691+0000 7f0d239b2640 1 --2- 172.21.15.82:0/3736392855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d1c100e80 0x7f0d1c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.691+0000 7f0d239b2640 1 -- 172.21.15.82:0/3736392855 >> 172.21.15.82:0/3736392855 conn(0x7f0d1c0fc410 msgr2=0x7f0d1c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:54.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d239b2640 1 -- 172.21.15.82:0/3736392855 shutdown_connections 2024-04-21T08:43:54.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d239b2640 1 -- 172.21.15.82:0/3736392855 wait complete. 2024-04-21T08:43:54.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d239b2640 1 Processor -- start 2024-04-21T08:43:54.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d239b2640 1 -- start start 2024-04-21T08:43:54.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d239b2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d1c100e80 0x7f0d1c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d239b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1c101fd0 0x7f0d1c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d239b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d1c102990 0x7f0d1c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d239b2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c10c9c0 con 0x7f0d1c101fd0 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d239b2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c10c840 con 0x7f0d1c100e80 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d239b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c10cb40 con 0x7f0d1c102990 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d20f4d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1c101fd0 0x7f0d1c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d20f4d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1c101fd0 0x7f0d1c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34314/0 (socket says 172.21.15.82:34314) 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d20f4d640 1 -- 172.21.15.82:0/1632641228 learned_addr learned my addr 172.21.15.82:0/1632641228 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d21f4f640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d1c102990 0x7f0d1c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.692+0000 7f0d2174e640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d1c100e80 0x7f0d1c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.693+0000 7f0d20f4d640 1 -- 172.21.15.82:0/1632641228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d1c102990 msgr2=0x7f0d1c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.693+0000 7f0d20f4d640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d1c102990 0x7f0d1c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.693+0000 7f0d20f4d640 1 -- 172.21.15.82:0/1632641228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d1c100e80 msgr2=0x7f0d1c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.693+0000 7f0d20f4d640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d1c100e80 0x7f0d1c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.693+0000 7f0d20f4d640 1 -- 172.21.15.82:0/1632641228 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c19fdd0 con 0x7f0d1c101fd0 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.693+0000 7f0d21f4f640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d1c102990 0x7f0d1c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:54.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.693+0000 7f0d20f4d640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1c101fd0 0x7f0d1c1986f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000c00 tx=0x7f0d0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:54.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.693+0000 7f0d127fc640 1 -- 172.21.15.82:0/1632641228 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c043020 con 0x7f0d1c101fd0 2024-04-21T08:43:54.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.693+0000 7f0d127fc640 1 -- 172.21.15.82:0/1632641228 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d0c004aa0 con 0x7f0d1c101fd0 2024-04-21T08:43:54.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.693+0000 7f0d239b2640 1 -- 172.21.15.82:0/1632641228 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1c1a0060 con 0x7f0d1c101fd0 2024-04-21T08:43:54.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.693+0000 7f0d127fc640 1 -- 172.21.15.82:0/1632641228 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c003d00 con 0x7f0d1c101fd0 2024-04-21T08:43:54.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.693+0000 7f0d239b2640 1 -- 172.21.15.82:0/1632641228 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1c1a05b0 con 0x7f0d1c101fd0 2024-04-21T08:43:54.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.694+0000 7f0d239b2640 1 -- 172.21.15.82:0/1632641228 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1c1a07b0 con 0x7f0d1c101fd0 2024-04-21T08:43:54.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.695+0000 7f0d127fc640 1 -- 172.21.15.82:0/1632641228 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0d0c046020 con 0x7f0d1c101fd0 2024-04-21T08:43:54.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.695+0000 7f0d127fc640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0d080a3bb0 0x7f0d080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:54.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.695+0000 7f0d2174e640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0d080a3bb0 0x7f0d080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:54.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.696+0000 7f0d127fc640 1 -- 172.21.15.82:0/1632641228 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0d0c0f25a0 con 0x7f0d1c101fd0 2024-04-21T08:43:54.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.696+0000 7f0d2174e640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0d080a3bb0 0x7f0d080a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0d180058c0 tx=0x7f0d1800a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:54.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.700+0000 7f0d127fc640 1 -- 172.21.15.82:0/1632641228 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d0c04d020 con 0x7f0d1c101fd0 2024-04-21T08:43:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:54 smithi134 ceph-mon[31772]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:43:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:54 smithi134 ceph-mon[31772]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-04-21T08:43:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:54 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:54 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:54 smithi134 ceph-mon[31772]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:54 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:54 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:54 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:54 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:43:54.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:54 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:43:54.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:54 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:54.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:54 smithi134 ceph-mon[31772]: Deploying daemon mds.l on smithi134 2024-04-21T08:43:54.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:54 smithi082 ceph-mon[27967]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:43:54.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:54 smithi082 ceph-mon[27967]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-04-21T08:43:54.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:54 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:54.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:54 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:54.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:54 smithi082 ceph-mon[27967]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:54.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:54 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:54.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:54 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:54.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:54 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:54.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:54 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:43:54.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:54 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:43:54.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:54 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:54.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:54 smithi082 ceph-mon[27967]: Deploying daemon mds.l on smithi134 2024-04-21T08:43:54.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.880+0000 7f0d239b2640 1 -- 172.21.15.82:0/1632641228 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d1c1a0e80 con 0x7f0d1c101fd0 2024-04-21T08:43:54.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.881+0000 7f0d127fc640 1 -- 172.21.15.82:0/1632641228 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18468 (secure 0 0 0) 0x7f0d0c0bf420 con 0x7f0d1c101fd0 2024-04-21T08:43:54.881 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:43:54.881 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":81,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:43:42.372753+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:43:54.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.883+0000 7f0d239b2640 1 -- 172.21.15.82:0/1632641228 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0d080a3bb0 msgr2=0x7f0d080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.883+0000 7f0d239b2640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0d080a3bb0 0x7f0d080a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0d180058c0 tx=0x7f0d1800a4c0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.883+0000 7f0d239b2640 1 -- 172.21.15.82:0/1632641228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1c101fd0 msgr2=0x7f0d1c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:54.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.883+0000 7f0d239b2640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1c101fd0 0x7f0d1c1986f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000c00 tx=0x7f0d0c004270 comp rx=0 tx=0).stop 2024-04-21T08:43:54.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.884+0000 7f0d239b2640 1 -- 172.21.15.82:0/1632641228 shutdown_connections 2024-04-21T08:43:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.884+0000 7f0d239b2640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0d080a3bb0 0x7f0d080a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.884+0000 7f0d239b2640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d1c102990 0x7f0d1c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.884+0000 7f0d239b2640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d1c101fd0 0x7f0d1c1986f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.884+0000 7f0d239b2640 1 --2- 172.21.15.82:0/1632641228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d1c100e80 0x7f0d1c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.884+0000 7f0d239b2640 1 -- 172.21.15.82:0/1632641228 >> 172.21.15.82:0/1632641228 conn(0x7f0d1c0fc410 msgr2=0x7f0d1c06a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.884+0000 7f0d239b2640 1 -- 172.21.15.82:0/1632641228 shutdown_connections 2024-04-21T08:43:54.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:54.884+0000 7f0d239b2640 1 -- 172.21.15.82:0/1632641228 wait complete. 2024-04-21T08:43:54.898 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T08:43:54.898 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:43:55.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.000+0000 7f948f73d640 1 -- 172.21.15.82:0/734788148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9488107c90 msgr2=0x7f948810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:55.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.000+0000 7f948f73d640 1 --2- 172.21.15.82:0/734788148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9488107c90 0x7f948810a0e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9480009ca0 tx=0x7f948002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.000+0000 7f948f73d640 1 -- 172.21.15.82:0/734788148 shutdown_connections 2024-04-21T08:43:55.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.000+0000 7f948f73d640 1 --2- 172.21.15.82:0/734788148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9488107c90 0x7f948810a0e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.000+0000 7f948f73d640 1 --2- 172.21.15.82:0/734788148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9488105360 0x7f9488107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.000+0000 7f948f73d640 1 --2- 172.21.15.82:0/734788148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9488102a30 0x7f9488104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.000+0000 7f948f73d640 1 -- 172.21.15.82:0/734788148 >> 172.21.15.82:0/734788148 conn(0x7f94880fc440 msgr2=0x7f94880fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:55.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.000+0000 7f948f73d640 1 -- 172.21.15.82:0/734788148 shutdown_connections 2024-04-21T08:43:55.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.000+0000 7f948f73d640 1 -- 172.21.15.82:0/734788148 wait complete. 2024-04-21T08:43:55.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948f73d640 1 Processor -- start 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948f73d640 1 -- start start 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948f73d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9488102a30 0x7f94881981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948f73d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9488105360 0x7f9488198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948f73d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9488107c90 0x7f948819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948f73d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f948810daf0 con 0x7f9488102a30 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948f73d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f948810d970 con 0x7f9488105360 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948f73d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f948810dc70 con 0x7f9488107c90 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948d4d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9488102a30 0x7f94881981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948d4d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9488102a30 0x7f94881981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34328/0 (socket says 172.21.15.82:34328) 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948d4d9640 1 -- 172.21.15.82:0/3872817357 learned_addr learned my addr 172.21.15.82:0/3872817357 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948ccd8640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9488105360 0x7f9488198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.001+0000 7f948dcda640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9488107c90 0x7f948819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:55.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f948d4d9640 1 -- 172.21.15.82:0/3872817357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9488107c90 msgr2=0x7f948819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f948d4d9640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9488107c90 0x7f948819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f948d4d9640 1 -- 172.21.15.82:0/3872817357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9488105360 msgr2=0x7f9488198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f948d4d9640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9488105360 0x7f9488198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f948d4d9640 1 -- 172.21.15.82:0/3872817357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948819fdf0 con 0x7f9488102a30 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f948ccd8640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9488105360 0x7f9488198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f948dcda640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9488107c90 0x7f948819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f948d4d9640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9488102a30 0x7f94881981d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f948400d950 tx=0x7f948400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f947e7fc640 1 -- 172.21.15.82:0/3872817357 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484017030 con 0x7f9488102a30 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f947e7fc640 1 -- 172.21.15.82:0/3872817357 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9484004020 con 0x7f9488102a30 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f947e7fc640 1 -- 172.21.15.82:0/3872817357 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94840047a0 con 0x7f9488102a30 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f948f73d640 1 -- 172.21.15.82:0/3872817357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94881a00e0 con 0x7f9488102a30 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.002+0000 7f948f73d640 1 -- 172.21.15.82:0/3872817357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94881a05e0 con 0x7f9488102a30 2024-04-21T08:43:55.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.003+0000 7f948f73d640 1 -- 172.21.15.82:0/3872817357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94880fdf30 con 0x7f9488102a30 2024-04-21T08:43:55.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.004+0000 7f947e7fc640 1 -- 172.21.15.82:0/3872817357 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9484009a30 con 0x7f9488102a30 2024-04-21T08:43:55.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.005+0000 7f947e7fc640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94740a3c30 0x7f94740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:55.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.005+0000 7f948ccd8640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94740a3c30 0x7f94740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:55.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.005+0000 7f947e7fc640 1 -- 172.21.15.82:0/3872817357 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f94840d00d0 con 0x7f9488102a30 2024-04-21T08:43:55.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.005+0000 7f948ccd8640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94740a3c30 0x7f94740a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9478005fd0 tx=0x7f9478005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:55.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.008+0000 7f947e7fc640 1 -- 172.21.15.82:0/3872817357 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f948409cfd0 con 0x7f9488102a30 2024-04-21T08:43:55.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.190+0000 7f948f73d640 1 -- 172.21.15.82:0/3872817357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f948802d830 con 0x7f9488102a30 2024-04-21T08:43:55.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.191+0000 7f947e7fc640 1 -- 172.21.15.82:0/3872817357 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f948409c8d0 con 0x7f9488102a30 2024-04-21T08:43:55.191 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:43:55.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.193+0000 7f948f73d640 1 -- 172.21.15.82:0/3872817357 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94740a3c30 msgr2=0x7f94740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.193+0000 7f948f73d640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94740a3c30 0x7f94740a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9478005fd0 tx=0x7f9478005850 comp rx=0 tx=0).stop 2024-04-21T08:43:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.193+0000 7f948f73d640 1 -- 172.21.15.82:0/3872817357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9488102a30 msgr2=0x7f94881981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.193+0000 7f948f73d640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9488102a30 0x7f94881981d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f948400d950 tx=0x7f948400de10 comp rx=0 tx=0).stop 2024-04-21T08:43:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.193+0000 7f948f73d640 1 -- 172.21.15.82:0/3872817357 shutdown_connections 2024-04-21T08:43:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.193+0000 7f948f73d640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94740a3c30 0x7f94740a60f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.193+0000 7f948f73d640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9488107c90 0x7f948819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.193+0000 7f948f73d640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9488105360 0x7f9488198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.193+0000 7f948f73d640 1 --2- 172.21.15.82:0/3872817357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9488102a30 0x7f94881981d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.193+0000 7f948f73d640 1 -- 172.21.15.82:0/3872817357 >> 172.21.15.82:0/3872817357 conn(0x7f94880fc440 msgr2=0x7f94881032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.193+0000 7f948f73d640 1 -- 172.21.15.82:0/3872817357 shutdown_connections 2024-04-21T08:43:55.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.193+0000 7f948f73d640 1 -- 172.21.15.82:0/3872817357 wait complete. 2024-04-21T08:43:55.203 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:43:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.301+0000 7fce33b50640 1 -- 172.21.15.82:0/2154416594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce2c101fd0 msgr2=0x7fce2c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.301+0000 7fce33b50640 1 --2- 172.21.15.82:0/2154416594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce2c101fd0 0x7fce2c102450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fce20009900 tx=0x7fce2002bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.301+0000 7fce33b50640 1 -- 172.21.15.82:0/2154416594 shutdown_connections 2024-04-21T08:43:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.301+0000 7fce33b50640 1 --2- 172.21.15.82:0/2154416594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce2c102990 0x7fce2c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.301+0000 7fce33b50640 1 --2- 172.21.15.82:0/2154416594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce2c101fd0 0x7fce2c102450 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.301+0000 7fce33b50640 1 --2- 172.21.15.82:0/2154416594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c100e80 0x7fce2c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.301+0000 7fce33b50640 1 -- 172.21.15.82:0/2154416594 >> 172.21.15.82:0/2154416594 conn(0x7fce2c0fc410 msgr2=0x7fce2c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.301+0000 7fce33b50640 1 -- 172.21.15.82:0/2154416594 shutdown_connections 2024-04-21T08:43:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.301+0000 7fce33b50640 1 -- 172.21.15.82:0/2154416594 wait complete. 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce33b50640 1 Processor -- start 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce33b50640 1 -- start start 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce33b50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce2c100e80 0x7fce2c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce33b50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c101fd0 0x7fce2c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce33b50640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce2c102990 0x7fce2c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce33b50640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c10ca30 con 0x7fce2c100e80 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce33b50640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c10c8b0 con 0x7fce2c102990 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce33b50640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c10cbb0 con 0x7fce2c101fd0 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce318ec640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce2c100e80 0x7fce2c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce318ec640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce2c100e80 0x7fce2c1982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34336/0 (socket says 172.21.15.82:34336) 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce318ec640 1 -- 172.21.15.82:0/2401024502 learned_addr learned my addr 172.21.15.82:0/2401024502 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce310eb640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c101fd0 0x7fce2c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce320ed640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce2c102990 0x7fce2c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce318ec640 1 -- 172.21.15.82:0/2401024502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c101fd0 msgr2=0x7fce2c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce318ec640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c101fd0 0x7fce2c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce318ec640 1 -- 172.21.15.82:0/2401024502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce2c102990 msgr2=0x7fce2c19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce318ec640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce2c102990 0x7fce2c19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce318ec640 1 -- 172.21.15.82:0/2401024502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c19fe90 con 0x7fce2c100e80 2024-04-21T08:43:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce320ed640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce2c102990 0x7fce2c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce318ec640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce2c100e80 0x7fce2c1982c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce2400b4a0 tx=0x7fce2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce2affd640 1 -- 172.21.15.82:0/2401024502 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24014070 con 0x7fce2c100e80 2024-04-21T08:43:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce2affd640 1 -- 172.21.15.82:0/2401024502 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce24010ac0 con 0x7fce2c100e80 2024-04-21T08:43:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce2affd640 1 -- 172.21.15.82:0/2401024502 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24017c30 con 0x7fce2c100e80 2024-04-21T08:43:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.302+0000 7fce33b50640 1 -- 172.21.15.82:0/2401024502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2c1a0180 con 0x7fce2c100e80 2024-04-21T08:43:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.303+0000 7fce33b50640 1 -- 172.21.15.82:0/2401024502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2c1a06f0 con 0x7fce2c100e80 2024-04-21T08:43:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.303+0000 7fce33b50640 1 -- 172.21.15.82:0/2401024502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2c1a09b0 con 0x7fce2c100e80 2024-04-21T08:43:55.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.304+0000 7fce2affd640 1 -- 172.21.15.82:0/2401024502 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fce24017070 con 0x7fce2c100e80 2024-04-21T08:43:55.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.305+0000 7fce2affd640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fce140a3c30 0x7fce140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:55.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.305+0000 7fce310eb640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fce140a3c30 0x7fce140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:55.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.305+0000 7fce2affd640 1 -- 172.21.15.82:0/2401024502 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fce240cf680 con 0x7fce2c100e80 2024-04-21T08:43:55.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.305+0000 7fce310eb640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fce140a3c30 0x7fce140a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce20006010 tx=0x7fce20005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:55.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.309+0000 7fce2affd640 1 -- 172.21.15.82:0/2401024502 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce2409c500 con 0x7fce2c100e80 2024-04-21T08:43:55.523 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:55 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1632641228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:43:55.523 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:55 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3872817357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:43:55.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.528+0000 7fce33b50640 1 -- 172.21.15.82:0/2401024502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce2c1a0b80 con 0x7fce2c100e80 2024-04-21T08:43:55.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.528+0000 7fce2affd640 1 -- 172.21.15.82:0/2401024502 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fce2409be00 con 0x7fce2c100e80 2024-04-21T08:43:55.528 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:43:55.528 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-21T08:43:55.528 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 1 2024-04-21T08:43:55.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.531+0000 7fce33b50640 1 -- 172.21.15.82:0/2401024502 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fce140a3c30 msgr2=0x7fce140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:55.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.531+0000 7fce33b50640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fce140a3c30 0x7fce140a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce20006010 tx=0x7fce20005be0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.531+0000 7fce33b50640 1 -- 172.21.15.82:0/2401024502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce2c100e80 msgr2=0x7fce2c1982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:55.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.531+0000 7fce33b50640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce2c100e80 0x7fce2c1982c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce2400b4a0 tx=0x7fce2400b960 comp rx=0 tx=0).stop 2024-04-21T08:43:55.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.531+0000 7fce33b50640 1 -- 172.21.15.82:0/2401024502 shutdown_connections 2024-04-21T08:43:55.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.531+0000 7fce33b50640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fce140a3c30 0x7fce140a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.531+0000 7fce33b50640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce2c102990 0x7fce2c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.531+0000 7fce33b50640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c101fd0 0x7fce2c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.531+0000 7fce33b50640 1 --2- 172.21.15.82:0/2401024502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce2c100e80 0x7fce2c1982c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:55.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.531+0000 7fce33b50640 1 -- 172.21.15.82:0/2401024502 >> 172.21.15.82:0/2401024502 conn(0x7fce2c0fc410 msgr2=0x7fce2c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:55.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.531+0000 7fce33b50640 1 -- 172.21.15.82:0/2401024502 shutdown_connections 2024-04-21T08:43:55.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:55.531+0000 7fce33b50640 1 -- 172.21.15.82:0/2401024502 wait complete. 2024-04-21T08:43:55.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:55 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1632641228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:43:55.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:55 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3872817357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:43:55.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:55 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1632641228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:43:55.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:55 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3872817357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:43:56.685 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:56 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2401024502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:56.685 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:56 smithi134 ceph-mon[31772]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:56.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:56 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2401024502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:56.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:56 smithi082 ceph-mon[27967]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:56.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:56 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2401024502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:56.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:56 smithi106 ceph-mon[31072]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:56.949 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 08:43:56 smithi134 podman[61910]: 2024-04-21 08:43:56.927756341 +0000 UTC m=+0.356178449 container start c8fd270304aacba838a25333ca58c975eafe6888f7f1fc3617340a063d7c8f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-21T08:43:57.226 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 08:43:56 smithi134 ceph-mds[61929]: -- 172.21.15.134:0/3545320769 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ff70e7a40 con 0x558ff70ba800 2024-04-21T08:43:57.226 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 08:43:56 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-l[61925]: starting mds.l at 2024-04-21T08:43:57.226 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 08:43:57 smithi134 bash[61910]: c8fd270304aacba838a25333ca58c975eafe6888f7f1fc3617340a063d7c8f6c 2024-04-21T08:43:57.226 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 08:43:57 smithi134 systemd[1]: Started Ceph mds.l for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:43:57.542 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:43:57.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.653+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2655600700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0101ff0 msgr2=0x7f8fd0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.653+0000 7f8fd87c3640 1 --2- 172.21.15.82:0/2655600700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0101ff0 0x7f8fd0102470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009900 tx=0x7f8fcc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.653+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2655600700 shutdown_connections 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.653+0000 7f8fd87c3640 1 --2- 172.21.15.82:0/2655600700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fd01029b0 0x7f8fd01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.653+0000 7f8fd87c3640 1 --2- 172.21.15.82:0/2655600700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0101ff0 0x7f8fd0102470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.653+0000 7f8fd87c3640 1 --2- 172.21.15.82:0/2655600700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8fd0100ea0 0x7f8fd01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.653+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2655600700 >> 172.21.15.82:0/2655600700 conn(0x7f8fd00fc410 msgr2=0x7f8fd00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.653+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2655600700 shutdown_connections 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.653+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2655600700 wait complete. 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.653+0000 7f8fd87c3640 1 Processor -- start 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.653+0000 7f8fd87c3640 1 -- start start 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd87c3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8fd0100ea0 0x7f8fd01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd87c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0101ff0 0x7f8fd01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:57.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd87c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fd01029b0 0x7f8fd019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd87c3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd010c9e0 con 0x7f8fd0101ff0 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd87c3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd010c860 con 0x7f8fd0100ea0 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd87c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd010cb60 con 0x7f8fd01029b0 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd5d5e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0101ff0 0x7f8fd01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd5d5e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0101ff0 0x7f8fd01986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34350/0 (socket says 172.21.15.82:34350) 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd5d5e640 1 -- 172.21.15.82:0/2123437323 learned_addr learned my addr 172.21.15.82:0/2123437323 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd655f640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8fd0100ea0 0x7f8fd01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd6d60640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fd01029b0 0x7f8fd019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd5d5e640 1 -- 172.21.15.82:0/2123437323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fd01029b0 msgr2=0x7f8fd019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd5d5e640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fd01029b0 0x7f8fd019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd5d5e640 1 -- 172.21.15.82:0/2123437323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8fd0100ea0 msgr2=0x7f8fd01981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd5d5e640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8fd0100ea0 0x7f8fd01981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd5d5e640 1 -- 172.21.15.82:0/2123437323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd019fdd0 con 0x7f8fd0101ff0 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd5d5e640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0101ff0 0x7f8fd01986f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc000c00 tx=0x7f8fcc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:57.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fc77fe640 1 -- 172.21.15.82:0/2123437323 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc043020 con 0x7f8fd0101ff0 2024-04-21T08:43:57.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2123437323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd01a0060 con 0x7f8fd0101ff0 2024-04-21T08:43:57.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fc77fe640 1 -- 172.21.15.82:0/2123437323 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fcc004aa0 con 0x7f8fd0101ff0 2024-04-21T08:43:57.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fc77fe640 1 -- 172.21.15.82:0/2123437323 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc003d00 con 0x7f8fd0101ff0 2024-04-21T08:43:57.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.654+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2123437323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd01a05b0 con 0x7f8fd0101ff0 2024-04-21T08:43:57.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.655+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2123437323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd01a07b0 con 0x7f8fd0101ff0 2024-04-21T08:43:57.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.656+0000 7f8fc77fe640 1 -- 172.21.15.82:0/2123437323 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8fcc046020 con 0x7f8fd0101ff0 2024-04-21T08:43:57.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.657+0000 7f8fc77fe640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8fbc0a3bb0 0x7f8fbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:57.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.657+0000 7f8fd655f640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8fbc0a3bb0 0x7f8fbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:57.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.657+0000 7f8fc77fe640 1 -- 172.21.15.82:0/2123437323 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8fcc0f2500 con 0x7f8fd0101ff0 2024-04-21T08:43:57.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.658+0000 7f8fd655f640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8fbc0a3bb0 0x7f8fbc0a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005d00 tx=0x7f8fc000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:57.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.662+0000 7f8fc77fe640 1 -- 172.21.15.82:0/2123437323 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fcc04d020 con 0x7f8fd0101ff0 2024-04-21T08:43:57.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.878+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2123437323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fd01a0b40 con 0x7f8fd0101ff0 2024-04-21T08:43:57.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.879+0000 7f8fc77fe640 1 -- 172.21.15.82:0/2123437323 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f8fcc0bf380 con 0x7f8fd0101ff0 2024-04-21T08:43:57.879 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:43:57.879 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T08:43:57.879 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 2 2024-04-21T08:43:57.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.881+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2123437323 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8fbc0a3bb0 msgr2=0x7f8fbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:57.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.881+0000 7f8fd87c3640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8fbc0a3bb0 0x7f8fbc0a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005d00 tx=0x7f8fc000b410 comp rx=0 tx=0).stop 2024-04-21T08:43:57.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.881+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2123437323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0101ff0 msgr2=0x7f8fd01986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:57.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.881+0000 7f8fd87c3640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0101ff0 0x7f8fd01986f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc000c00 tx=0x7f8fcc004270 comp rx=0 tx=0).stop 2024-04-21T08:43:57.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.881+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2123437323 shutdown_connections 2024-04-21T08:43:57.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.881+0000 7f8fd87c3640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8fbc0a3bb0 0x7f8fbc0a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:57.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.881+0000 7f8fd87c3640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fd01029b0 0x7f8fd019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:57.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.881+0000 7f8fd87c3640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fd0101ff0 0x7f8fd01986f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:57.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.881+0000 7f8fd87c3640 1 --2- 172.21.15.82:0/2123437323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8fd0100ea0 0x7f8fd01981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:57.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.881+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2123437323 >> 172.21.15.82:0/2123437323 conn(0x7f8fd00fc410 msgr2=0x7f8fd00fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:57.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.881+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2123437323 shutdown_connections 2024-04-21T08:43:57.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:43:57.881+0000 7f8fd87c3640 1 -- 172.21.15.82:0/2123437323 wait complete. 2024-04-21T08:43:58.080 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:57 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] up:boot 2024-04-21T08:43:58.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:57 smithi106 ceph-mon[31072]: fsmap 1 up:standby 2024-04-21T08:43:58.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:43:58.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:58.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:58.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:58.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:43:58.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:43:58.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:57 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:58.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:57 smithi106 ceph-mon[31072]: Deploying daemon mds.k on smithi106 2024-04-21T08:43:58.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:57 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2123437323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:58.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:57 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] up:boot 2024-04-21T08:43:58.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:57 smithi134 ceph-mon[31772]: fsmap 1 up:standby 2024-04-21T08:43:58.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:43:58.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:58.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:58.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:58.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:43:58.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:43:58.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:57 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:58.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:57 smithi134 ceph-mon[31772]: Deploying daemon mds.k on smithi106 2024-04-21T08:43:58.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:57 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2123437323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:58.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:57 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] up:boot 2024-04-21T08:43:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:57 smithi082 ceph-mon[27967]: fsmap 1 up:standby 2024-04-21T08:43:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:43:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:43:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:43:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:43:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:57 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:57 smithi082 ceph-mon[27967]: Deploying daemon mds.k on smithi106 2024-04-21T08:43:58.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:57 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2123437323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:59.127 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:43:58 smithi106 ceph-mon[31072]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:59.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:43:58 smithi134 ceph-mon[31772]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:59.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:43:58 smithi082 ceph-mon[27967]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:43:59.538 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:43:59 smithi106 systemd[1]: Starting Ceph mds.k for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:43:59.798 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:43:59 smithi106 podman[59885]: 2024-04-21 08:43:59.441879427 +0000 UTC m=+0.018109601 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:43:59.798 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:43:59 smithi106 podman[59885]: 2024-04-21 08:43:59.580548211 +0000 UTC m=+0.156778368 container create 723e3ca6f5c51798d7c6cb5b13c19edaa9ed2f79f79bd04393dc719a47c1fe06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-21T08:43:59.799 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:43:59 smithi106 podman[59885]: 2024-04-21 08:43:59.799876792 +0000 UTC m=+0.376106947 container init 723e3ca6f5c51798d7c6cb5b13c19edaa9ed2f79f79bd04393dc719a47c1fe06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-21T08:43:59.891 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:00.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.004+0000 7f54fc37e640 1 -- 172.21.15.82:0/1430599110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f4101ff0 msgr2=0x7f54f4102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:00.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.004+0000 7f54fc37e640 1 --2- 172.21.15.82:0/1430599110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f4101ff0 0x7f54f4102470 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009900 tx=0x7f54e402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:00.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.004+0000 7f54fc37e640 1 -- 172.21.15.82:0/1430599110 shutdown_connections 2024-04-21T08:44:00.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.004+0000 7f54fc37e640 1 --2- 172.21.15.82:0/1430599110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f41029b0 0x7f54f41092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:00.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.004+0000 7f54fc37e640 1 --2- 172.21.15.82:0/1430599110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f4101ff0 0x7f54f4102470 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:00.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.004+0000 7f54fc37e640 1 --2- 172.21.15.82:0/1430599110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4100ea0 0x7f54f41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:00.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.004+0000 7f54fc37e640 1 -- 172.21.15.82:0/1430599110 >> 172.21.15.82:0/1430599110 conn(0x7f54f40fc410 msgr2=0x7f54f40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.004+0000 7f54fc37e640 1 -- 172.21.15.82:0/1430599110 shutdown_connections 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.004+0000 7f54fc37e640 1 -- 172.21.15.82:0/1430599110 wait complete. 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fc37e640 1 Processor -- start 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fc37e640 1 -- start start 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fc37e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4100ea0 0x7f54f41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fc37e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f4101ff0 0x7f54f4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fc37e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f41029b0 0x7f54f419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fc37e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f54f410db40 con 0x7f54f41029b0 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fc37e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f54f410d9c0 con 0x7f54f4101ff0 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fc37e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54f410dcc0 con 0x7f54f4100ea0 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fa91b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f41029b0 0x7f54f419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fa91b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f41029b0 0x7f54f419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34372/0 (socket says 172.21.15.82:34372) 2024-04-21T08:44:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fa91b640 1 -- 172.21.15.82:0/379378372 learned_addr learned my addr 172.21.15.82:0/379378372 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54f9919640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f4101ff0 0x7f54f4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fa11a640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4100ea0 0x7f54f41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fa91b640 1 -- 172.21.15.82:0/379378372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4100ea0 msgr2=0x7f54f41981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fa91b640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4100ea0 0x7f54f41981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fa91b640 1 -- 172.21.15.82:0/379378372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f4101ff0 msgr2=0x7f54f4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.005+0000 7f54fa91b640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f4101ff0 0x7f54f4198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.006+0000 7f54fa91b640 1 -- 172.21.15.82:0/379378372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f419fde0 con 0x7f54f41029b0 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.006+0000 7f54f9919640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f4101ff0 0x7f54f4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.006+0000 7f54fa91b640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f41029b0 0x7f54f419f770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00cad0 tx=0x7f54ec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.006+0000 7f54eb7fe640 1 -- 172.21.15.82:0/379378372 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00ae20 con 0x7f54f41029b0 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.006+0000 7f54eb7fe640 1 -- 172.21.15.82:0/379378372 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54ec005020 con 0x7f54f41029b0 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.006+0000 7f54fc37e640 1 -- 172.21.15.82:0/379378372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f41a00d0 con 0x7f54f41029b0 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.006+0000 7f54eb7fe640 1 -- 172.21.15.82:0/379378372 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00f6b0 con 0x7f54f41029b0 2024-04-21T08:44:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.006+0000 7f54fc37e640 1 -- 172.21.15.82:0/379378372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f41a05a0 con 0x7f54f41029b0 2024-04-21T08:44:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.007+0000 7f54fc37e640 1 -- 172.21.15.82:0/379378372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f41081b0 con 0x7f54f41029b0 2024-04-21T08:44:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.008+0000 7f54eb7fe640 1 -- 172.21.15.82:0/379378372 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f54ec0051d0 con 0x7f54f41029b0 2024-04-21T08:44:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.009+0000 7f54eb7fe640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f54e00a3bb0 0x7f54e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.009+0000 7f54fa11a640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f54e00a3bb0 0x7f54e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.009+0000 7f54eb7fe640 1 -- 172.21.15.82:0/379378372 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f54ec0d03b0 con 0x7f54f41029b0 2024-04-21T08:44:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.010+0000 7f54fa11a640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f54e00a3bb0 0x7f54e00a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f54f00058c0 tx=0x7f54f000c4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.013+0000 7f54eb7fe640 1 -- 172.21.15.82:0/379378372 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54ec014030 con 0x7f54f41029b0 2024-04-21T08:44:00.128 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:43:59 smithi106 podman[59885]: 2024-04-21 08:43:59.802691142 +0000 UTC m=+0.378921307 container start 723e3ca6f5c51798d7c6cb5b13c19edaa9ed2f79f79bd04393dc719a47c1fe06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-21T08:44:00.128 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:43:59 smithi106 ceph-mds[59904]: -- 172.21.15.106:0/3030307894 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b39647fa40 con 0x55b396452c00 2024-04-21T08:44:00.128 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:43:59 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-k[59900]: starting mds.k at 2024-04-21T08:44:00.128 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:43:59 smithi106 bash[59885]: 723e3ca6f5c51798d7c6cb5b13c19edaa9ed2f79f79bd04393dc719a47c1fe06 2024-04-21T08:44:00.128 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:43:59 smithi106 systemd[1]: Started Ceph mds.k for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:44:00.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.264+0000 7f54fc37e640 1 -- 172.21.15.82:0/379378372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54f402d5c0 con 0x7f54f41029b0 2024-04-21T08:44:00.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.265+0000 7f54eb7fe640 1 -- 172.21.15.82:0/379378372 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f54ec09d230 con 0x7f54f41029b0 2024-04-21T08:44:00.264 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:00.265 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T08:44:00.265 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 3 2024-04-21T08:44:00.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.267+0000 7f54fc37e640 1 -- 172.21.15.82:0/379378372 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f54e00a3bb0 msgr2=0x7f54e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:00.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.267+0000 7f54fc37e640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f54e00a3bb0 0x7f54e00a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f54f00058c0 tx=0x7f54f000c4a0 comp rx=0 tx=0).stop 2024-04-21T08:44:00.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.267+0000 7f54fc37e640 1 -- 172.21.15.82:0/379378372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f41029b0 msgr2=0x7f54f419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:00.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.267+0000 7f54fc37e640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f41029b0 0x7f54f419f770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00cad0 tx=0x7f54ec00cf90 comp rx=0 tx=0).stop 2024-04-21T08:44:00.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.267+0000 7f54fc37e640 1 -- 172.21.15.82:0/379378372 shutdown_connections 2024-04-21T08:44:00.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.267+0000 7f54fc37e640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f54e00a3bb0 0x7f54e00a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:00.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.267+0000 7f54fc37e640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f41029b0 0x7f54f419f770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:00.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.267+0000 7f54fc37e640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54f4101ff0 0x7f54f4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:00.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.267+0000 7f54fc37e640 1 --2- 172.21.15.82:0/379378372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54f4100ea0 0x7f54f41981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:00.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.267+0000 7f54fc37e640 1 -- 172.21.15.82:0/379378372 >> 172.21.15.82:0/379378372 conn(0x7f54f40fc410 msgr2=0x7f54f4105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:00.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.267+0000 7f54fc37e640 1 -- 172.21.15.82:0/379378372 shutdown_connections 2024-04-21T08:44:00.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:00.267+0000 7f54fc37e640 1 -- 172.21.15.82:0/379378372 wait complete. 2024-04-21T08:44:01.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:00 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] up:boot 2024-04-21T08:44:01.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:00 smithi134 ceph-mon[31772]: fsmap 2 up:standby 2024-04-21T08:44:01.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:00 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:44:01.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:00 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:01.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:00 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:01.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:00 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:01.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:00 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:01.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:00 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:01.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:00 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:01.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:00 smithi134 ceph-mon[31772]: Deploying daemon mds.j on smithi082 2024-04-21T08:44:01.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:00 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/379378372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:01.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:00 smithi134 ceph-mon[31772]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:01.352 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:00 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] up:boot 2024-04-21T08:44:01.353 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:00 smithi082 ceph-mon[27967]: fsmap 2 up:standby 2024-04-21T08:44:01.353 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:00 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:44:01.353 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:00 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:01.353 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:00 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:01.353 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:00 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:01.353 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:00 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:01.353 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:00 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:01.353 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:00 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:01.353 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:00 smithi082 ceph-mon[27967]: Deploying daemon mds.j on smithi082 2024-04-21T08:44:01.353 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:00 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/379378372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:01.353 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:00 smithi082 ceph-mon[27967]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:01.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:00 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] up:boot 2024-04-21T08:44:01.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:00 smithi106 ceph-mon[31072]: fsmap 2 up:standby 2024-04-21T08:44:01.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:00 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:44:01.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:00 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:01.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:00 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:01.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:00 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:01.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:00 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:01.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:00 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:01.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:00 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:01.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:00 smithi106 ceph-mon[31072]: Deploying daemon mds.j on smithi082 2024-04-21T08:44:01.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:00 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/379378372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:01.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:00 smithi106 ceph-mon[31072]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:02.278 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:02.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.386+0000 7f6d96cc7640 1 -- 172.21.15.82:0/1205037763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d90107cb0 msgr2=0x7f6d9010a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:02.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.386+0000 7f6d96cc7640 1 --2- 172.21.15.82:0/1205037763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d90107cb0 0x7f6d9010a0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009ca0 tx=0x7f6d8802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:02.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.386+0000 7f6d96cc7640 1 -- 172.21.15.82:0/1205037763 shutdown_connections 2024-04-21T08:44:02.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.386+0000 7f6d96cc7640 1 --2- 172.21.15.82:0/1205037763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d90107cb0 0x7f6d9010a0a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:02.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.386+0000 7f6d96cc7640 1 --2- 172.21.15.82:0/1205037763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d90105380 0x7f6d90107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:02.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.386+0000 7f6d96cc7640 1 --2- 172.21.15.82:0/1205037763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d90102a50 0x7f6d90104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:02.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.386+0000 7f6d96cc7640 1 -- 172.21.15.82:0/1205037763 >> 172.21.15.82:0/1205037763 conn(0x7f6d900fc3e0 msgr2=0x7f6d900fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:02.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.386+0000 7f6d96cc7640 1 -- 172.21.15.82:0/1205037763 shutdown_connections 2024-04-21T08:44:02.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.386+0000 7f6d96cc7640 1 -- 172.21.15.82:0/1205037763 wait complete. 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d96cc7640 1 Processor -- start 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d96cc7640 1 -- start start 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d96cc7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d90102a50 0x7f6d90198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d96cc7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d90105380 0x7f6d90198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d96cc7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d90107cb0 0x7f6d9019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d96cc7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9010dac0 con 0x7f6d90105380 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d96cc7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9010d940 con 0x7f6d90107cb0 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d96cc7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9010dc40 con 0x7f6d90102a50 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d8ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d90105380 0x7f6d90198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d8ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d90105380 0x7f6d90198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34390/0 (socket says 172.21.15.82:34390) 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d8ffff640 1 -- 172.21.15.82:0/3566747748 learned_addr learned my addr 172.21.15.82:0/3566747748 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d94a63640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d90102a50 0x7f6d90198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:02.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d95264640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d90107cb0 0x7f6d9019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:02.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d8ffff640 1 -- 172.21.15.82:0/3566747748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d90102a50 msgr2=0x7f6d90198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:02.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d8ffff640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d90102a50 0x7f6d90198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:02.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d8ffff640 1 -- 172.21.15.82:0/3566747748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d90107cb0 msgr2=0x7f6d9019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:02.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d8ffff640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d90107cb0 0x7f6d9019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:02.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.387+0000 7f6d8ffff640 1 -- 172.21.15.82:0/3566747748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9019fd70 con 0x7f6d90105380 2024-04-21T08:44:02.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.388+0000 7f6d8ffff640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d90105380 0x7f6d90198690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005fd0 tx=0x7f6d7c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:02.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.388+0000 7f6d8dffb640 1 -- 172.21.15.82:0/3566747748 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c017070 con 0x7f6d90105380 2024-04-21T08:44:02.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.388+0000 7f6d8dffb640 1 -- 172.21.15.82:0/3566747748 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d7c009c30 con 0x7f6d90105380 2024-04-21T08:44:02.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.388+0000 7f6d96cc7640 1 -- 172.21.15.82:0/3566747748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d901a0060 con 0x7f6d90105380 2024-04-21T08:44:02.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.388+0000 7f6d8dffb640 1 -- 172.21.15.82:0/3566747748 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c003d30 con 0x7f6d90105380 2024-04-21T08:44:02.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.388+0000 7f6d96cc7640 1 -- 172.21.15.82:0/3566747748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d901a05b0 con 0x7f6d90105380 2024-04-21T08:44:02.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.389+0000 7f6d96cc7640 1 -- 172.21.15.82:0/3566747748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d900fded0 con 0x7f6d90105380 2024-04-21T08:44:02.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.390+0000 7f6d8dffb640 1 -- 172.21.15.82:0/3566747748 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6d7c003ed0 con 0x7f6d90105380 2024-04-21T08:44:02.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.391+0000 7f6d8dffb640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6d800a3bb0 0x7f6d800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:02.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.391+0000 7f6d94a63640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6d800a3bb0 0x7f6d800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:02.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.391+0000 7f6d8dffb640 1 -- 172.21.15.82:0/3566747748 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6d7c0cfb50 con 0x7f6d90105380 2024-04-21T08:44:02.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.391+0000 7f6d94a63640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6d800a3bb0 0x7f6d800a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6d84005fd0 tx=0x7f6d8400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:02.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.395+0000 7f6d8dffb640 1 -- 172.21.15.82:0/3566747748 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d7c020030 con 0x7f6d90105380 2024-04-21T08:44:02.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.619+0000 7f6d96cc7640 1 -- 172.21.15.82:0/3566747748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d9002d810 con 0x7f6d90105380 2024-04-21T08:44:02.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.620+0000 7f6d8dffb640 1 -- 172.21.15.82:0/3566747748 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f6d7c009de0 con 0x7f6d90105380 2024-04-21T08:44:02.620 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:02.620 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T08:44:02.620 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 3 2024-04-21T08:44:02.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.622+0000 7f6d96cc7640 1 -- 172.21.15.82:0/3566747748 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6d800a3bb0 msgr2=0x7f6d800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:02.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.622+0000 7f6d96cc7640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6d800a3bb0 0x7f6d800a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6d84005fd0 tx=0x7f6d8400b390 comp rx=0 tx=0).stop 2024-04-21T08:44:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.622+0000 7f6d96cc7640 1 -- 172.21.15.82:0/3566747748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d90105380 msgr2=0x7f6d90198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.622+0000 7f6d96cc7640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d90105380 0x7f6d90198690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005fd0 tx=0x7f6d7c00dde0 comp rx=0 tx=0).stop 2024-04-21T08:44:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.622+0000 7f6d96cc7640 1 -- 172.21.15.82:0/3566747748 shutdown_connections 2024-04-21T08:44:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.622+0000 7f6d96cc7640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6d800a3bb0 0x7f6d800a6070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.622+0000 7f6d96cc7640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d90107cb0 0x7f6d9019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.622+0000 7f6d96cc7640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d90105380 0x7f6d90198690 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.622+0000 7f6d96cc7640 1 --2- 172.21.15.82:0/3566747748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d90102a50 0x7f6d90198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.622+0000 7f6d96cc7640 1 -- 172.21.15.82:0/3566747748 >> 172.21.15.82:0/3566747748 conn(0x7f6d900fc3e0 msgr2=0x7f6d901032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.622+0000 7f6d96cc7640 1 -- 172.21.15.82:0/3566747748 shutdown_connections 2024-04-21T08:44:02.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:02.622+0000 7f6d96cc7640 1 -- 172.21.15.82:0/3566747748 wait complete. 2024-04-21T08:44:02.858 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 21 08:44:02 smithi082 systemd[1]: Started Ceph mds.j for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:44:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:03 smithi106 ceph-mon[31072]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:03 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3566747748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:03 smithi082 ceph-mon[27967]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:03 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3566747748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:03.705 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:03.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:03 smithi134 ceph-mon[31772]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:03 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3566747748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:04.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:04 smithi082 ceph-mon[27967]: Deploying daemon mds.d on smithi082 2024-04-21T08:44:04.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:04 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.82:6834/467976762,v1:172.21.15.82:6835/467976762] up:boot 2024-04-21T08:44:04.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:04 smithi082 ceph-mon[27967]: fsmap 3 up:standby 2024-04-21T08:44:04.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:04 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:44:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:04 smithi106 ceph-mon[31072]: Deploying daemon mds.d on smithi082 2024-04-21T08:44:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:04 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.82:6834/467976762,v1:172.21.15.82:6835/467976762] up:boot 2024-04-21T08:44:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:04 smithi106 ceph-mon[31072]: fsmap 3 up:standby 2024-04-21T08:44:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:04 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:44:04.632 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:04 smithi134 ceph-mon[31772]: Deploying daemon mds.d on smithi082 2024-04-21T08:44:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:04 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.82:6834/467976762,v1:172.21.15.82:6835/467976762] up:boot 2024-04-21T08:44:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:04 smithi134 ceph-mon[31772]: fsmap 3 up:standby 2024-04-21T08:44:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:04 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:44:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.905+0000 7fa47a97e640 1 -- 172.21.15.82:0/1362154251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa474107c50 msgr2=0x7fa47410a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.905+0000 7fa47a97e640 1 --2- 172.21.15.82:0/1362154251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa474107c50 0x7fa47410a0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009ca0 tx=0x7fa46c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.905+0000 7fa47a97e640 1 -- 172.21.15.82:0/1362154251 shutdown_connections 2024-04-21T08:44:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.905+0000 7fa47a97e640 1 --2- 172.21.15.82:0/1362154251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa474107c50 0x7fa47410a0a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.905+0000 7fa47a97e640 1 --2- 172.21.15.82:0/1362154251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa474105320 0x7fa474107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.905+0000 7fa47a97e640 1 --2- 172.21.15.82:0/1362154251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4741029f0 0x7fa474104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.905+0000 7fa47a97e640 1 -- 172.21.15.82:0/1362154251 >> 172.21.15.82:0/1362154251 conn(0x7fa4740fc3e0 msgr2=0x7fa4740fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.905+0000 7fa47a97e640 1 -- 172.21.15.82:0/1362154251 shutdown_connections 2024-04-21T08:44:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.905+0000 7fa47a97e640 1 -- 172.21.15.82:0/1362154251 wait complete. 2024-04-21T08:44:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.906+0000 7fa47a97e640 1 Processor -- start 2024-04-21T08:44:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.906+0000 7fa47a97e640 1 -- start start 2024-04-21T08:44:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.906+0000 7fa47a97e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4741029f0 0x7fa4741981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.906+0000 7fa47a97e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa474105320 0x7fa4741986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.906+0000 7fa47a97e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa474107c50 0x7fa47419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.906+0000 7fa47a97e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa47410db10 con 0x7fa4741029f0 2024-04-21T08:44:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.906+0000 7fa47a97e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa47410d990 con 0x7fa474105320 2024-04-21T08:44:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.906+0000 7fa47a97e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa47410dc90 con 0x7fa474107c50 2024-04-21T08:44:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.906+0000 7fa473fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4741029f0 0x7fa4741981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.906+0000 7fa473fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4741029f0 0x7fa4741981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41312/0 (socket says 172.21.15.82:41312) 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.906+0000 7fa473fff640 1 -- 172.21.15.82:0/2566196826 learned_addr learned my addr 172.21.15.82:0/2566196826 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa478f1b640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa474107c50 0x7fa47419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa4737fe640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa474105320 0x7fa4741986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa473fff640 1 -- 172.21.15.82:0/2566196826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa474107c50 msgr2=0x7fa47419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa473fff640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa474107c50 0x7fa47419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa473fff640 1 -- 172.21.15.82:0/2566196826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa474105320 msgr2=0x7fa4741986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa473fff640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa474105320 0x7fa4741986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa473fff640 1 -- 172.21.15.82:0/2566196826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47419fdd0 con 0x7fa4741029f0 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa4737fe640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa474105320 0x7fa4741986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa473fff640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4741029f0 0x7fa4741981b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa46400d950 tx=0x7fa46400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa4717fa640 1 -- 172.21.15.82:0/2566196826 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464017070 con 0x7fa4741029f0 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa4717fa640 1 -- 172.21.15.82:0/2566196826 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa464009c30 con 0x7fa4741029f0 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa47a97e640 1 -- 172.21.15.82:0/2566196826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4741a00c0 con 0x7fa4741029f0 2024-04-21T08:44:04.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa4717fa640 1 -- 172.21.15.82:0/2566196826 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46400f620 con 0x7fa4741029f0 2024-04-21T08:44:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.907+0000 7fa47a97e640 1 -- 172.21.15.82:0/2566196826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4741a0610 con 0x7fa4741029f0 2024-04-21T08:44:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.908+0000 7fa47a97e640 1 -- 172.21.15.82:0/2566196826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4740fded0 con 0x7fa4741029f0 2024-04-21T08:44:04.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.909+0000 7fa4717fa640 1 -- 172.21.15.82:0/2566196826 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa464010460 con 0x7fa4741029f0 2024-04-21T08:44:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.910+0000 7fa4717fa640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa45c0a3bb0 0x7fa45c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.910+0000 7fa4737fe640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa45c0a3bb0 0x7fa45c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.910+0000 7fa4717fa640 1 -- 172.21.15.82:0/2566196826 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa4640cfc40 con 0x7fa4741029f0 2024-04-21T08:44:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.910+0000 7fa4737fe640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa45c0a3bb0 0x7fa45c0a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa4741996c0 tx=0x7fa468008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:04.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:04.914+0000 7fa4717fa640 1 -- 172.21.15.82:0/2566196826 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa46409cac0 con 0x7fa4741029f0 2024-04-21T08:44:05.108 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:44:05 smithi082 systemd[1]: Starting Ceph mds.d for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:44:05.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.159+0000 7fa47a97e640 1 -- 172.21.15.82:0/2566196826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa47402d5c0 con 0x7fa4741029f0 2024-04-21T08:44:05.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.160+0000 7fa4717fa640 1 -- 172.21.15.82:0/2566196826 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fa464009de0 con 0x7fa4741029f0 2024-04-21T08:44:05.159 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:05.159 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T08:44:05.160 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 4 2024-04-21T08:44:05.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.162+0000 7fa47a97e640 1 -- 172.21.15.82:0/2566196826 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa45c0a3bb0 msgr2=0x7fa45c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:05.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.162+0000 7fa47a97e640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa45c0a3bb0 0x7fa45c0a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa4741996c0 tx=0x7fa468008040 comp rx=0 tx=0).stop 2024-04-21T08:44:05.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.162+0000 7fa47a97e640 1 -- 172.21.15.82:0/2566196826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4741029f0 msgr2=0x7fa4741981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:05.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.162+0000 7fa47a97e640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4741029f0 0x7fa4741981b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa46400d950 tx=0x7fa46400de10 comp rx=0 tx=0).stop 2024-04-21T08:44:05.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.162+0000 7fa47a97e640 1 -- 172.21.15.82:0/2566196826 shutdown_connections 2024-04-21T08:44:05.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.162+0000 7fa47a97e640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa45c0a3bb0 0x7fa45c0a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:05.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.162+0000 7fa47a97e640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa474107c50 0x7fa47419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:05.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.162+0000 7fa47a97e640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa474105320 0x7fa4741986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:05.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.162+0000 7fa47a97e640 1 --2- 172.21.15.82:0/2566196826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4741029f0 0x7fa4741981b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:05.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.162+0000 7fa47a97e640 1 -- 172.21.15.82:0/2566196826 >> 172.21.15.82:0/2566196826 conn(0x7fa4740fc3e0 msgr2=0x7fa4741034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:05.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.162+0000 7fa47a97e640 1 -- 172.21.15.82:0/2566196826 shutdown_connections 2024-04-21T08:44:05.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:05.162+0000 7fa47a97e640 1 -- 172.21.15.82:0/2566196826 wait complete. 2024-04-21T08:44:05.435 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:05 smithi082 ceph-mon[27967]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:05.435 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:05 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2566196826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:05.436 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:44:05 smithi082 podman[69273]: 2024-04-21 08:44:05.172713695 +0000 UTC m=+0.019884015 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:44:05.436 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:44:05 smithi082 podman[69273]: 2024-04-21 08:44:05.322865616 +0000 UTC m=+0.170035927 container create 01b15b9008c67a1bcd886cc11099a025f7ee882ba804fd89a4f1d08ba7cae350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-21T08:44:05.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:05 smithi106 ceph-mon[31072]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:05.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:05 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2566196826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:05 smithi134 ceph-mon[31772]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:05 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2566196826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:05.858 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:44:05 smithi082 podman[69273]: 2024-04-21 08:44:05.550469369 +0000 UTC m=+0.397639677 container init 01b15b9008c67a1bcd886cc11099a025f7ee882ba804fd89a4f1d08ba7cae350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:44:05.858 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:44:05 smithi082 podman[69273]: 2024-04-21 08:44:05.553141857 +0000 UTC m=+0.400312165 container start 01b15b9008c67a1bcd886cc11099a025f7ee882ba804fd89a4f1d08ba7cae350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-21T08:44:05.858 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:44:05 smithi082 ceph-mds[69294]: -- 172.21.15.82:0/2684680035 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b18b81da40 con 0x55b18b7f0c00 2024-04-21T08:44:05.858 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:44:05 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-d[69290]: starting mds.d at 2024-04-21T08:44:05.859 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:44:05 smithi082 bash[69273]: 01b15b9008c67a1bcd886cc11099a025f7ee882ba804fd89a4f1d08ba7cae350 2024-04-21T08:44:05.859 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:44:05 smithi082 systemd[1]: Started Ceph mds.d for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:44:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:06 smithi106 ceph-mon[31072]: Deploying daemon mds.e on smithi106 2024-04-21T08:44:06.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:06 smithi106 ceph-mon[31072]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:06.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:06 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.82:6836/3751789078,v1:172.21.15.82:6837/3751789078] up:boot 2024-04-21T08:44:06.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:06 smithi106 ceph-mon[31072]: fsmap 4 up:standby 2024-04-21T08:44:06.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:06 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:44:06.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:06.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:06.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:06.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:06.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:06.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:06.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:06 smithi134 ceph-mon[31772]: Deploying daemon mds.e on smithi106 2024-04-21T08:44:06.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:06 smithi134 ceph-mon[31772]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:06.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:06 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.82:6836/3751789078,v1:172.21.15.82:6837/3751789078] up:boot 2024-04-21T08:44:06.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:06 smithi134 ceph-mon[31772]: fsmap 4 up:standby 2024-04-21T08:44:06.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:06 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:44:07.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:07.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:07.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:06 smithi082 ceph-mon[27967]: Deploying daemon mds.e on smithi106 2024-04-21T08:44:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:06 smithi082 ceph-mon[27967]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:06 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.82:6836/3751789078,v1:172.21.15.82:6837/3751789078] up:boot 2024-04-21T08:44:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:06 smithi082 ceph-mon[27967]: fsmap 4 up:standby 2024-04-21T08:44:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:06 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:44:07.176 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:07.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.289+0000 7faad665c640 1 -- 172.21.15.82:0/3391603849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faad0103b40 msgr2=0x7faad010a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:07.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.289+0000 7faad665c640 1 --2- 172.21.15.82:0/3391603849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faad0103b40 0x7faad010a3d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faac8009ca0 tx=0x7faac802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.289+0000 7faad665c640 1 -- 172.21.15.82:0/3391603849 shutdown_connections 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.289+0000 7faad665c640 1 --2- 172.21.15.82:0/3391603849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faad0103b40 0x7faad010a3d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.289+0000 7faad665c640 1 --2- 172.21.15.82:0/3391603849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad0103180 0x7faad0103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.289+0000 7faad665c640 1 --2- 172.21.15.82:0/3391603849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0101f90 0x7faad01023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.289+0000 7faad665c640 1 -- 172.21.15.82:0/3391603849 >> 172.21.15.82:0/3391603849 conn(0x7faad00fd500 msgr2=0x7faad00ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.289+0000 7faad665c640 1 -- 172.21.15.82:0/3391603849 shutdown_connections 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.289+0000 7faad665c640 1 -- 172.21.15.82:0/3391603849 wait complete. 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.290+0000 7faad665c640 1 Processor -- start 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.290+0000 7faad665c640 1 -- start start 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.290+0000 7faad665c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faad0101f90 0x7faad0075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.290+0000 7faad665c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad0103180 0x7faad0074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.290+0000 7faad665c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0103b40 0x7faad0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.290+0000 7faad665c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faad010dad0 con 0x7faad0101f90 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.290+0000 7faad665c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faad010d950 con 0x7faad0103b40 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.290+0000 7faad665c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faad010dc50 con 0x7faad0103180 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.290+0000 7faacffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faad0101f90 0x7faad0075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.290+0000 7faacffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faad0101f90 0x7faad0075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41340/0 (socket says 172.21.15.82:41340) 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.290+0000 7faacffff640 1 -- 172.21.15.82:0/1701499765 learned_addr learned my addr 172.21.15.82:0/1701499765 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faacf7fe640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad0103180 0x7faad0074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faad4bf9640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0103b40 0x7faad0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faacffff640 1 -- 172.21.15.82:0/1701499765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad0103180 msgr2=0x7faad0074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faacffff640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad0103180 0x7faad0074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faacffff640 1 -- 172.21.15.82:0/1701499765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0103b40 msgr2=0x7faad0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faacffff640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0103b40 0x7faad0074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faacffff640 1 -- 172.21.15.82:0/1701499765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad0075110 con 0x7faad0101f90 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faacffff640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faad0101f90 0x7faad0075d00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faac400c970 tx=0x7faac400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faacd7fa640 1 -- 172.21.15.82:0/1701499765 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4007c10 con 0x7faad0101f90 2024-04-21T08:44:07.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faacd7fa640 1 -- 172.21.15.82:0/1701499765 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faac4007db0 con 0x7faad0101f90 2024-04-21T08:44:07.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faad665c640 1 -- 172.21.15.82:0/1701499765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad01a21f0 con 0x7faad0101f90 2024-04-21T08:44:07.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faacd7fa640 1 -- 172.21.15.82:0/1701499765 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac400f870 con 0x7faad0101f90 2024-04-21T08:44:07.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.291+0000 7faad665c640 1 -- 172.21.15.82:0/1701499765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad01a2790 con 0x7faad0101f90 2024-04-21T08:44:07.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.292+0000 7faad665c640 1 -- 172.21.15.82:0/1701499765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad0109250 con 0x7faad0101f90 2024-04-21T08:44:07.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.293+0000 7faacd7fa640 1 -- 172.21.15.82:0/1701499765 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faac400fa10 con 0x7faad0101f90 2024-04-21T08:44:07.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.294+0000 7faacd7fa640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7faab80a3c30 0x7faab80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:07.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.294+0000 7faacf7fe640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7faab80a3c30 0x7faab80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:07.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.294+0000 7faacd7fa640 1 -- 172.21.15.82:0/1701499765 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7faac40d05c0 con 0x7faad0101f90 2024-04-21T08:44:07.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.294+0000 7faacf7fe640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7faab80a3c30 0x7faab80a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faac0005fd0 tx=0x7faac0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:07.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.297+0000 7faacd7fa640 1 -- 172.21.15.82:0/1701499765 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faac409d560 con 0x7faad0101f90 2024-04-21T08:44:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.513+0000 7faad665c640 1 -- 172.21.15.82:0/1701499765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faad002d5c0 con 0x7faad0101f90 2024-04-21T08:44:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.514+0000 7faacd7fa640 1 -- 172.21.15.82:0/1701499765 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7faac409ce60 con 0x7faad0101f90 2024-04-21T08:44:07.514 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:07.515 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T08:44:07.515 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 5 2024-04-21T08:44:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.516+0000 7faad665c640 1 -- 172.21.15.82:0/1701499765 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7faab80a3c30 msgr2=0x7faab80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.516+0000 7faad665c640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7faab80a3c30 0x7faab80a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faac0005fd0 tx=0x7faac0005850 comp rx=0 tx=0).stop 2024-04-21T08:44:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.516+0000 7faad665c640 1 -- 172.21.15.82:0/1701499765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faad0101f90 msgr2=0x7faad0075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.516+0000 7faad665c640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faad0101f90 0x7faad0075d00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faac400c970 tx=0x7faac400ce30 comp rx=0 tx=0).stop 2024-04-21T08:44:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.516+0000 7faad665c640 1 -- 172.21.15.82:0/1701499765 shutdown_connections 2024-04-21T08:44:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.516+0000 7faad665c640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7faab80a3c30 0x7faab80a60f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.516+0000 7faad665c640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0103b40 0x7faad0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.516+0000 7faad665c640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad0103180 0x7faad0074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.516+0000 7faad665c640 1 --2- 172.21.15.82:0/1701499765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faad0101f90 0x7faad0075d00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.516+0000 7faad665c640 1 -- 172.21.15.82:0/1701499765 >> 172.21.15.82:0/1701499765 conn(0x7faad00fd500 msgr2=0x7faad0106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.516+0000 7faad665c640 1 -- 172.21.15.82:0/1701499765 shutdown_connections 2024-04-21T08:44:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:07.516+0000 7faad665c640 1 -- 172.21.15.82:0/1701499765 wait complete. 2024-04-21T08:44:07.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:07 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1701499765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:08.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:07 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1701499765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:08.128 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:44:07 smithi106 systemd[1]: Starting Ceph mds.e for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:44:08.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:07 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1701499765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:08.413 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:44:08 smithi106 podman[60205]: 2024-04-21 08:44:08.043707395 +0000 UTC m=+0.018224743 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:44:08.414 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:44:08 smithi106 podman[60205]: 2024-04-21 08:44:08.161946721 +0000 UTC m=+0.136464056 container create 251b53e53e7370bdf6db1b62ebac839f0b8fbc68ba6035a46ef7b8f6272f4a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-21T08:44:08.414 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:44:08 smithi106 podman[60205]: 2024-04-21 08:44:08.414788885 +0000 UTC m=+0.389306220 container init 251b53e53e7370bdf6db1b62ebac839f0b8fbc68ba6035a46ef7b8f6272f4a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:44:08.737 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:44:08 smithi106 podman[60205]: 2024-04-21 08:44:08.417492953 +0000 UTC m=+0.392010298 container start 251b53e53e7370bdf6db1b62ebac839f0b8fbc68ba6035a46ef7b8f6272f4a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-21T08:44:08.738 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:44:08 smithi106 ceph-mds[60224]: -- 172.21.15.106:0/3012649459 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56013be9da40 con 0x56013be71000 2024-04-21T08:44:08.738 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:44:08 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-e[60220]: starting mds.e at 2024-04-21T08:44:08.738 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:44:08 smithi106 bash[60205]: 251b53e53e7370bdf6db1b62ebac839f0b8fbc68ba6035a46ef7b8f6272f4a9a 2024-04-21T08:44:08.738 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:44:08 smithi106 systemd[1]: Started Ceph mds.e for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:44:09.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:08 smithi082 ceph-mon[27967]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:09.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:08 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:09.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:08 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:09.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:08 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:09.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:08 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:09.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:08 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:09.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:08 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:09.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:08 smithi106 ceph-mon[31072]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:09.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:08 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:09.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:08 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:09.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:08 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:09.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:08 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:09.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:08 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:09.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:08 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:09.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:08 smithi134 ceph-mon[31772]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:09.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:08 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:09.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:08 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:09.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:08 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:09.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:08 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:09.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:08 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:09.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:08 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:09.528 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:09.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.638+0000 7f208b8b2640 1 -- 172.21.15.82:0/1621536437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2084100e80 msgr2=0x7f20841012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.638+0000 7f208b8b2640 1 --2- 172.21.15.82:0/1621536437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2084100e80 0x7f20841012a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2074009900 tx=0x7f207402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:09.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.638+0000 7f208b8b2640 1 -- 172.21.15.82:0/1621536437 shutdown_connections 2024-04-21T08:44:09.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.638+0000 7f208b8b2640 1 --2- 172.21.15.82:0/1621536437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2084102990 0x7f20841092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.638+0000 7f208b8b2640 1 --2- 172.21.15.82:0/1621536437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2084101fd0 0x7f2084102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.638+0000 7f208b8b2640 1 --2- 172.21.15.82:0/1621536437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2084100e80 0x7f20841012a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.638+0000 7f208b8b2640 1 -- 172.21.15.82:0/1621536437 >> 172.21.15.82:0/1621536437 conn(0x7f20840fc410 msgr2=0x7f20840fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:09.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.638+0000 7f208b8b2640 1 -- 172.21.15.82:0/1621536437 shutdown_connections 2024-04-21T08:44:09.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.638+0000 7f208b8b2640 1 -- 172.21.15.82:0/1621536437 wait complete. 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.638+0000 7f208b8b2640 1 Processor -- start 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.638+0000 7f208b8b2640 1 -- start start 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208b8b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2084100e80 0x7f20841981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208b8b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2084101fd0 0x7f2084198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208b8b2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2084102990 0x7f208419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208b8b2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f208410c9d0 con 0x7f2084100e80 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208b8b2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f208410c850 con 0x7f2084102990 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208b8b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f208410cb50 con 0x7f2084101fd0 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208964e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2084100e80 0x7f20841981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208964e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2084100e80 0x7f20841981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41358/0 (socket says 172.21.15.82:41358) 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208964e640 1 -- 172.21.15.82:0/3476158961 learned_addr learned my addr 172.21.15.82:0/3476158961 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f2089e4f640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2084102990 0x7f208419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f2088e4d640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2084101fd0 0x7f2084198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208964e640 1 -- 172.21.15.82:0/3476158961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2084101fd0 msgr2=0x7f2084198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208964e640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2084101fd0 0x7f2084198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208964e640 1 -- 172.21.15.82:0/3476158961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2084102990 msgr2=0x7f208419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208964e640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2084102990 0x7f208419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208964e640 1 -- 172.21.15.82:0/3476158961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208419fd80 con 0x7f2084100e80 2024-04-21T08:44:09.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f2089e4f640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2084102990 0x7f208419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:09.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208964e640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2084100e80 0x7f20841981d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2074006010 tx=0x7f2074005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:09.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f207a7fc640 1 -- 172.21.15.82:0/3476158961 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074043020 con 0x7f2084100e80 2024-04-21T08:44:09.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208b8b2640 1 -- 172.21.15.82:0/3476158961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20841a0010 con 0x7f2084100e80 2024-04-21T08:44:09.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f207a7fc640 1 -- 172.21.15.82:0/3476158961 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20740043d0 con 0x7f2084100e80 2024-04-21T08:44:09.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f207a7fc640 1 -- 172.21.15.82:0/3476158961 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074003b80 con 0x7f2084100e80 2024-04-21T08:44:09.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.639+0000 7f208b8b2640 1 -- 172.21.15.82:0/3476158961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20841a0560 con 0x7f2084100e80 2024-04-21T08:44:09.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.640+0000 7f208b8b2640 1 -- 172.21.15.82:0/3476158961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20841a0760 con 0x7f2084100e80 2024-04-21T08:44:09.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.641+0000 7f207a7fc640 1 -- 172.21.15.82:0/3476158961 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2074046020 con 0x7f2084100e80 2024-04-21T08:44:09.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.642+0000 7f207a7fc640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f20700a3bb0 0x7f20700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.642+0000 7f2088e4d640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f20700a3bb0 0x7f20700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.642+0000 7f207a7fc640 1 -- 172.21.15.82:0/3476158961 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f20740f2810 con 0x7f2084100e80 2024-04-21T08:44:09.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.642+0000 7f2088e4d640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f20700a3bb0 0x7f20700a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2084199670 tx=0x7f2080005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:09.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.646+0000 7f207a7fc640 1 -- 172.21.15.82:0/3476158961 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f207404d020 con 0x7f2084100e80 2024-04-21T08:44:09.738 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:09 smithi134 ceph-mon[31772]: Deploying daemon mds.f on smithi134 2024-04-21T08:44:09.739 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:09 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.106:6834/3901395826,v1:172.21.15.106:6835/3901395826] up:boot 2024-04-21T08:44:09.739 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:09 smithi134 ceph-mon[31772]: fsmap 5 up:standby 2024-04-21T08:44:09.739 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:09 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:44:09.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:09 smithi082 ceph-mon[27967]: Deploying daemon mds.f on smithi134 2024-04-21T08:44:09.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:09 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.106:6834/3901395826,v1:172.21.15.106:6835/3901395826] up:boot 2024-04-21T08:44:09.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:09 smithi082 ceph-mon[27967]: fsmap 5 up:standby 2024-04-21T08:44:09.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:09 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:44:09.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.863+0000 7f208b8b2640 1 -- 172.21.15.82:0/3476158961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20841a0930 con 0x7f2084100e80 2024-04-21T08:44:09.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.864+0000 7f207a7fc640 1 -- 172.21.15.82:0/3476158961 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f20740bf690 con 0x7f2084100e80 2024-04-21T08:44:09.864 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:09.864 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-21T08:44:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 6 2024-04-21T08:44:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.866+0000 7f208b8b2640 1 -- 172.21.15.82:0/3476158961 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f20700a3bb0 msgr2=0x7f20700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.866+0000 7f208b8b2640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f20700a3bb0 0x7f20700a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2084199670 tx=0x7f2080005f90 comp rx=0 tx=0).stop 2024-04-21T08:44:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.866+0000 7f208b8b2640 1 -- 172.21.15.82:0/3476158961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2084100e80 msgr2=0x7f20841981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.866+0000 7f208b8b2640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2084100e80 0x7f20841981d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2074006010 tx=0x7f2074005e40 comp rx=0 tx=0).stop 2024-04-21T08:44:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.866+0000 7f208b8b2640 1 -- 172.21.15.82:0/3476158961 shutdown_connections 2024-04-21T08:44:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.866+0000 7f208b8b2640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f20700a3bb0 0x7f20700a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.866+0000 7f208b8b2640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2084102990 0x7f208419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.866+0000 7f208b8b2640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2084101fd0 0x7f2084198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.866+0000 7f208b8b2640 1 --2- 172.21.15.82:0/3476158961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2084100e80 0x7f20841981d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.866+0000 7f208b8b2640 1 -- 172.21.15.82:0/3476158961 >> 172.21.15.82:0/3476158961 conn(0x7f20840fc410 msgr2=0x7f20840fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:09.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.866+0000 7f208b8b2640 1 -- 172.21.15.82:0/3476158961 shutdown_connections 2024-04-21T08:44:09.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:09.866+0000 7f208b8b2640 1 -- 172.21.15.82:0/3476158961 wait complete. 2024-04-21T08:44:10.029 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:09 smithi106 ceph-mon[31072]: Deploying daemon mds.f on smithi134 2024-04-21T08:44:10.030 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:09 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.106:6834/3901395826,v1:172.21.15.106:6835/3901395826] up:boot 2024-04-21T08:44:10.030 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:09 smithi106 ceph-mon[31072]: fsmap 5 up:standby 2024-04-21T08:44:10.030 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:09 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:44:10.907 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:10 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3476158961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:10.907 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:10 smithi134 ceph-mon[31772]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:11.040 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:10 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3476158961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:11.041 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:10 smithi106 ceph-mon[31072]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:11.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:10 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3476158961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:11.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:10 smithi082 ceph-mon[27967]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:11.475 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 21 08:44:11 smithi134 systemd[1]: Started Ceph mds.f for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:44:11.877 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:11.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.988+0000 7f190894b640 1 -- 172.21.15.82:0/2864382739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904101fa0 msgr2=0x7f19041023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:11.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.988+0000 7f190894b640 1 --2- 172.21.15.82:0/2864382739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904101fa0 0x7f19041023c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009900 tx=0x7f18f402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:11.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.988+0000 7f190894b640 1 -- 172.21.15.82:0/2864382739 shutdown_connections 2024-04-21T08:44:11.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.988+0000 7f190894b640 1 --2- 172.21.15.82:0/2864382739 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1904103b50 0x7f190410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:11.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.988+0000 7f190894b640 1 --2- 172.21.15.82:0/2864382739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1904103190 0x7f1904103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:11.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.988+0000 7f190894b640 1 --2- 172.21.15.82:0/2864382739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904101fa0 0x7f19041023c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:11.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.988+0000 7f190894b640 1 -- 172.21.15.82:0/2864382739 >> 172.21.15.82:0/2864382739 conn(0x7f19040fd530 msgr2=0x7f19040ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:11.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.988+0000 7f190894b640 1 -- 172.21.15.82:0/2864382739 shutdown_connections 2024-04-21T08:44:11.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.988+0000 7f190894b640 1 -- 172.21.15.82:0/2864382739 wait complete. 2024-04-21T08:44:11.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.989+0000 7f190894b640 1 Processor -- start 2024-04-21T08:44:11.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.989+0000 7f190894b640 1 -- start start 2024-04-21T08:44:11.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.989+0000 7f190894b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1904101fa0 0x7f1904198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:11.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.989+0000 7f190894b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904103190 0x7f19041986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:11.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.989+0000 7f190894b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1904103b50 0x7f190419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:11.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.989+0000 7f190894b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f190410dae0 con 0x7f1904103190 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.989+0000 7f190894b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f190410d960 con 0x7f1904103b50 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.989+0000 7f190894b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f190410dc60 con 0x7f1904101fa0 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.989+0000 7f1901d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904103190 0x7f19041986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f1901d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904103190 0x7f19041986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41386/0 (socket says 172.21.15.82:41386) 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f1901d9b640 1 -- 172.21.15.82:0/1843302979 learned_addr learned my addr 172.21.15.82:0/1843302979 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f190259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1904101fa0 0x7f1904198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f1902d9d640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1904103b50 0x7f190419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f1901d9b640 1 -- 172.21.15.82:0/1843302979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1904101fa0 msgr2=0x7f1904198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f1901d9b640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1904101fa0 0x7f1904198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f1901d9b640 1 -- 172.21.15.82:0/1843302979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1904103b50 msgr2=0x7f190419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f1901d9b640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1904103b50 0x7f190419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f1901d9b640 1 -- 172.21.15.82:0/1843302979 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190419fd80 con 0x7f1904103190 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f1901d9b640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904103190 0x7f19041986a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c970 tx=0x7f18f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f18f37fe640 1 -- 172.21.15.82:0/1843302979 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8007bb0 con 0x7f1904103190 2024-04-21T08:44:11.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f18f37fe640 1 -- 172.21.15.82:0/1843302979 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f8007d50 con 0x7f1904103190 2024-04-21T08:44:11.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f190894b640 1 -- 172.21.15.82:0/1843302979 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19041a0070 con 0x7f1904103190 2024-04-21T08:44:11.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f18f37fe640 1 -- 172.21.15.82:0/1843302979 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f800f450 con 0x7f1904103190 2024-04-21T08:44:11.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.990+0000 7f190894b640 1 -- 172.21.15.82:0/1843302979 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19041a0570 con 0x7f1904103190 2024-04-21T08:44:11.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.991+0000 7f190894b640 1 -- 172.21.15.82:0/1843302979 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19041092c0 con 0x7f1904103190 2024-04-21T08:44:11.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.993+0000 7f18f37fe640 1 -- 172.21.15.82:0/1843302979 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f18f8004f00 con 0x7f1904103190 2024-04-21T08:44:11.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.994+0000 7f18f37fe640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f18ec0a3bb0 0x7f18ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:11.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.994+0000 7f190259c640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f18ec0a3bb0 0x7f18ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:11.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.994+0000 7f18f37fe640 1 -- 172.21.15.82:0/1843302979 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f18f80d0000 con 0x7f1904103190 2024-04-21T08:44:11.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.994+0000 7f190259c640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f18ec0a3bb0 0x7f18ec0a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009230 tx=0x7f18f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:11.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:11.997+0000 7f18f37fe640 1 -- 172.21.15.82:0/1843302979 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18f8014030 con 0x7f1904103190 2024-04-21T08:44:12.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.216+0000 7f190894b640 1 -- 172.21.15.82:0/1843302979 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f190402d810 con 0x7f1904103190 2024-04-21T08:44:12.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.217+0000 7f18f37fe640 1 -- 172.21.15.82:0/1843302979 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7f18f809c830 con 0x7f1904103190 2024-04-21T08:44:12.216 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:12.217 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-21T08:44:12.217 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2024-04-21T08:44:12.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.219+0000 7f190894b640 1 -- 172.21.15.82:0/1843302979 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f18ec0a3bb0 msgr2=0x7f18ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:12.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.219+0000 7f190894b640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f18ec0a3bb0 0x7f18ec0a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009230 tx=0x7f18f4005fd0 comp rx=0 tx=0).stop 2024-04-21T08:44:12.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.219+0000 7f190894b640 1 -- 172.21.15.82:0/1843302979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904103190 msgr2=0x7f19041986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:12.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.219+0000 7f190894b640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904103190 0x7f19041986a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c970 tx=0x7f18f800ce30 comp rx=0 tx=0).stop 2024-04-21T08:44:12.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.219+0000 7f190894b640 1 -- 172.21.15.82:0/1843302979 shutdown_connections 2024-04-21T08:44:12.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.219+0000 7f190894b640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f18ec0a3bb0 0x7f18ec0a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:12.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.219+0000 7f190894b640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1904103b50 0x7f190419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:12.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.219+0000 7f190894b640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1904103190 0x7f19041986a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:12.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.219+0000 7f190894b640 1 --2- 172.21.15.82:0/1843302979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1904101fa0 0x7f1904198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:12.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.219+0000 7f190894b640 1 -- 172.21.15.82:0/1843302979 >> 172.21.15.82:0/1843302979 conn(0x7f19040fd530 msgr2=0x7f1904106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:12.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.219+0000 7f190894b640 1 -- 172.21.15.82:0/1843302979 shutdown_connections 2024-04-21T08:44:12.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:12.219+0000 7f190894b640 1 -- 172.21.15.82:0/1843302979 wait complete. 2024-04-21T08:44:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:12 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:12 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:12 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:12 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:12 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:12.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:12 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:12.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:12 smithi082 ceph-mon[27967]: Deploying daemon mds.c on smithi134 2024-04-21T08:44:12.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:12 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.134:6834/2962604291,v1:172.21.15.134:6835/2962604291] up:boot 2024-04-21T08:44:12.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:12 smithi082 ceph-mon[27967]: fsmap 6 up:standby 2024-04-21T08:44:12.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:12 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:44:12.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:12 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1843302979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:12 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:12 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:12 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:12 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:12 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:12 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:12 smithi106 ceph-mon[31072]: Deploying daemon mds.c on smithi134 2024-04-21T08:44:12.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:12 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.134:6834/2962604291,v1:172.21.15.134:6835/2962604291] up:boot 2024-04-21T08:44:12.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:12 smithi106 ceph-mon[31072]: fsmap 6 up:standby 2024-04-21T08:44:12.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:12 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:44:12.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:12 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1843302979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:12.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:12 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:12.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:12 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:12.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:12 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:12.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:12 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:12.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:12 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:12.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:12 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:12.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:12 smithi134 ceph-mon[31772]: Deploying daemon mds.c on smithi134 2024-04-21T08:44:12.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:12 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.134:6834/2962604291,v1:172.21.15.134:6835/2962604291] up:boot 2024-04-21T08:44:12.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:12 smithi134 ceph-mon[31772]: fsmap 6 up:standby 2024-04-21T08:44:12.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:12 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:44:12.695 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:12 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1843302979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:13.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:13 smithi082 ceph-mon[27967]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:13.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:13 smithi106 ceph-mon[31072]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:13.687 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:13 smithi134 ceph-mon[31772]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:13.687 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 08:44:13 smithi134 systemd[1]: Starting Ceph mds.c for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:44:13.976 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 08:44:13 smithi134 podman[62539]: 2024-04-21 08:44:13.591334198 +0000 UTC m=+0.018675532 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:44:13.976 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 08:44:13 smithi134 podman[62539]: 2024-04-21 08:44:13.731796526 +0000 UTC m=+0.159137889 container create 3b91812a31b32032dfd74397be3dc46cf38c82b70d6ff7d151531a662dd1f482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-21T08:44:14.230 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:14.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.336+0000 7f942d06d640 1 -- 172.21.15.82:0/3643183285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9428101fd0 msgr2=0x7f9428102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:14.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.336+0000 7f942d06d640 1 --2- 172.21.15.82:0/3643183285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9428101fd0 0x7f9428102450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f941c009900 tx=0x7f941c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:14.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.336+0000 7f942d06d640 1 -- 172.21.15.82:0/3643183285 shutdown_connections 2024-04-21T08:44:14.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.336+0000 7f942d06d640 1 --2- 172.21.15.82:0/3643183285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9428102990 0x7f94281092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:14.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.336+0000 7f942d06d640 1 --2- 172.21.15.82:0/3643183285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9428101fd0 0x7f9428102450 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:14.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.336+0000 7f942d06d640 1 --2- 172.21.15.82:0/3643183285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9428100e80 0x7f94281012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:14.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.336+0000 7f942d06d640 1 -- 172.21.15.82:0/3643183285 >> 172.21.15.82:0/3643183285 conn(0x7f94280fc410 msgr2=0x7f94280fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:14.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.336+0000 7f942d06d640 1 -- 172.21.15.82:0/3643183285 shutdown_connections 2024-04-21T08:44:14.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.336+0000 7f942d06d640 1 -- 172.21.15.82:0/3643183285 wait complete. 2024-04-21T08:44:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942d06d640 1 Processor -- start 2024-04-21T08:44:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942d06d640 1 -- start start 2024-04-21T08:44:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942d06d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9428100e80 0x7f942806b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942d06d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9428101fd0 0x7f942819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942d06d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9428102990 0x7f942819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942d06d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f942810c9d0 con 0x7f9428102990 2024-04-21T08:44:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942d06d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f942810c850 con 0x7f9428101fd0 2024-04-21T08:44:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942d06d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f942810cb50 con 0x7f9428100e80 2024-04-21T08:44:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9428102990 0x7f942819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9428102990 0x7f942819f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47388/0 (socket says 172.21.15.82:47388) 2024-04-21T08:44:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942759e640 1 -- 172.21.15.82:0/1692860090 learned_addr learned my addr 172.21.15.82:0/1692860090 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942659c640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9428101fd0 0x7f942819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f9426d9d640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9428100e80 0x7f942806b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942759e640 1 -- 172.21.15.82:0/1692860090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9428100e80 msgr2=0x7f942806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942759e640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9428100e80 0x7f942806b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942759e640 1 -- 172.21.15.82:0/1692860090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9428101fd0 msgr2=0x7f942819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942759e640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9428101fd0 0x7f942819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.337+0000 7f942759e640 1 -- 172.21.15.82:0/1692860090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942819fdd0 con 0x7f9428102990 2024-04-21T08:44:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.338+0000 7f942759e640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9428102990 0x7f942819f510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f942000eb10 tx=0x7f942000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.338+0000 7f940ffff640 1 -- 172.21.15.82:0/1692860090 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420004b20 con 0x7f9428102990 2024-04-21T08:44:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.338+0000 7f940ffff640 1 -- 172.21.15.82:0/1692860090 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9420004cc0 con 0x7f9428102990 2024-04-21T08:44:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.338+0000 7f942d06d640 1 -- 172.21.15.82:0/1692860090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94281a00c0 con 0x7f9428102990 2024-04-21T08:44:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.338+0000 7f940ffff640 1 -- 172.21.15.82:0/1692860090 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420010670 con 0x7f9428102990 2024-04-21T08:44:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.338+0000 7f942d06d640 1 -- 172.21.15.82:0/1692860090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94281a05e0 con 0x7f9428102990 2024-04-21T08:44:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.339+0000 7f942d06d640 1 -- 172.21.15.82:0/1692860090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9428108190 con 0x7f9428102990 2024-04-21T08:44:14.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.340+0000 7f940ffff640 1 -- 172.21.15.82:0/1692860090 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9420005020 con 0x7f9428102990 2024-04-21T08:44:14.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.340+0000 7f940ffff640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94100a3bb0 0x7f94100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:14.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.340+0000 7f9426d9d640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94100a3bb0 0x7f94100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:14.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.341+0000 7f940ffff640 1 -- 172.21.15.82:0/1692860090 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f94200d0480 con 0x7f9428102990 2024-04-21T08:44:14.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.341+0000 7f9426d9d640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94100a3bb0 0x7f94100a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9418009bd0 tx=0x7f9418005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:14.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.345+0000 7f940ffff640 1 -- 172.21.15.82:0/1692860090 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f942009d300 con 0x7f9428102990 2024-04-21T08:44:14.476 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 08:44:14 smithi134 podman[62539]: 2024-04-21 08:44:14.001363701 +0000 UTC m=+0.428705033 container init 3b91812a31b32032dfd74397be3dc46cf38c82b70d6ff7d151531a662dd1f482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-21T08:44:14.476 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 08:44:14 smithi134 podman[62539]: 2024-04-21 08:44:14.004027669 +0000 UTC m=+0.431368993 container start 3b91812a31b32032dfd74397be3dc46cf38c82b70d6ff7d151531a662dd1f482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-21T08:44:14.476 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 08:44:14 smithi134 ceph-mds[62558]: -- 172.21.15.134:0/3625315763 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560bdf4d1a40 con 0x560bdf4a4800 2024-04-21T08:44:14.476 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 08:44:14 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c[62554]: starting mds.c at 2024-04-21T08:44:14.476 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 08:44:14 smithi134 bash[62539]: 3b91812a31b32032dfd74397be3dc46cf38c82b70d6ff7d151531a662dd1f482 2024-04-21T08:44:14.476 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 08:44:14 smithi134 systemd[1]: Started Ceph mds.c for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:44:14.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.563+0000 7f942d06d640 1 -- 172.21.15.82:0/1692860090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f942802d730 con 0x7f9428102990 2024-04-21T08:44:14.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.563+0000 7f940ffff640 1 -- 172.21.15.82:0/1692860090 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f942009cc00 con 0x7f9428102990 2024-04-21T08:44:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:14.563 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-21T08:44:14.563 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-04-21T08:44:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.565+0000 7f942d06d640 1 -- 172.21.15.82:0/1692860090 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94100a3bb0 msgr2=0x7f94100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.565+0000 7f942d06d640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94100a3bb0 0x7f94100a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9418009bd0 tx=0x7f9418005f90 comp rx=0 tx=0).stop 2024-04-21T08:44:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.565+0000 7f942d06d640 1 -- 172.21.15.82:0/1692860090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9428102990 msgr2=0x7f942819f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.565+0000 7f942d06d640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9428102990 0x7f942819f510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f942000eb10 tx=0x7f942000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:44:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.565+0000 7f942d06d640 1 -- 172.21.15.82:0/1692860090 shutdown_connections 2024-04-21T08:44:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.565+0000 7f942d06d640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f94100a3bb0 0x7f94100a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.565+0000 7f942d06d640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9428102990 0x7f942819f510 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.565+0000 7f942d06d640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9428101fd0 0x7f942819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.565+0000 7f942d06d640 1 --2- 172.21.15.82:0/1692860090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9428100e80 0x7f942806b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.565+0000 7f942d06d640 1 -- 172.21.15.82:0/1692860090 >> 172.21.15.82:0/1692860090 conn(0x7f94280fc410 msgr2=0x7f9428105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.565+0000 7f942d06d640 1 -- 172.21.15.82:0/1692860090 shutdown_connections 2024-04-21T08:44:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:14.565+0000 7f942d06d640 1 -- 172.21.15.82:0/1692860090 wait complete. 2024-04-21T08:44:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:15.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:15 smithi134 ceph-mon[31772]: Deploying daemon mds.h on smithi106 2024-04-21T08:44:15.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:15 smithi134 ceph-mon[31772]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:15.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:15 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.134:6836/1870695477,v1:172.21.15.134:6837/1870695477] up:boot 2024-04-21T08:44:15.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:15 smithi134 ceph-mon[31772]: fsmap 7 up:standby 2024-04-21T08:44:15.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:15 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:44:15.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:15 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1692860090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:15 smithi082 ceph-mon[27967]: Deploying daemon mds.h on smithi106 2024-04-21T08:44:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:15 smithi082 ceph-mon[27967]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:15 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.134:6836/1870695477,v1:172.21.15.134:6837/1870695477] up:boot 2024-04-21T08:44:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:15 smithi082 ceph-mon[27967]: fsmap 7 up:standby 2024-04-21T08:44:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:15 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:44:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:15 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1692860090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:15 smithi106 ceph-mon[31072]: Deploying daemon mds.h on smithi106 2024-04-21T08:44:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:15 smithi106 ceph-mon[31072]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:15 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.134:6836/1870695477,v1:172.21.15.134:6837/1870695477] up:boot 2024-04-21T08:44:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:15 smithi106 ceph-mon[31072]: fsmap 7 up:standby 2024-04-21T08:44:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:15 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:44:15.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:15 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1692860090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:16.577 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:16.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.686+0000 7fc68363a640 1 -- 172.21.15.82:0/675486069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c101f90 msgr2=0x7fc67c1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.686+0000 7fc68363a640 1 --2- 172.21.15.82:0/675486069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c101f90 0x7fc67c1023b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc678009900 tx=0x7fc67802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:16.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.686+0000 7fc68363a640 1 -- 172.21.15.82:0/675486069 shutdown_connections 2024-04-21T08:44:16.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.686+0000 7fc68363a640 1 --2- 172.21.15.82:0/675486069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc67c103b40 0x7fc67c10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.686+0000 7fc68363a640 1 --2- 172.21.15.82:0/675486069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc67c103180 0x7fc67c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.686+0000 7fc68363a640 1 --2- 172.21.15.82:0/675486069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c101f90 0x7fc67c1023b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.686+0000 7fc68363a640 1 -- 172.21.15.82:0/675486069 >> 172.21.15.82:0/675486069 conn(0x7fc67c0fd500 msgr2=0x7fc67c0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.686+0000 7fc68363a640 1 -- 172.21.15.82:0/675486069 shutdown_connections 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.686+0000 7fc68363a640 1 -- 172.21.15.82:0/675486069 wait complete. 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc68363a640 1 Processor -- start 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc68363a640 1 -- start start 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc68363a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c101f90 0x7fc67c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc68363a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc67c103180 0x7fc67c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc68363a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc67c103b40 0x7fc67c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc68363a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c10dad0 con 0x7fc67c101f90 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc68363a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c10d950 con 0x7fc67c103180 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc68363a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c10dc50 con 0x7fc67c103b40 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc6813d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c101f90 0x7fc67c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc6813d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c101f90 0x7fc67c198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47406/0 (socket says 172.21.15.82:47406) 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc6813d6640 1 -- 172.21.15.82:0/2633946547 learned_addr learned my addr 172.21.15.82:0/2633946547 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc680bd5640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc67c103180 0x7fc67c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc6813d6640 1 -- 172.21.15.82:0/2633946547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc67c103b40 msgr2=0x7fc67c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc681bd7640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc67c103b40 0x7fc67c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc6813d6640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc67c103b40 0x7fc67c19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc6813d6640 1 -- 172.21.15.82:0/2633946547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc67c103180 msgr2=0x7fc67c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc6813d6640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc67c103180 0x7fc67c1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.687+0000 7fc6813d6640 1 -- 172.21.15.82:0/2633946547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67c19fdb0 con 0x7fc67c101f90 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.688+0000 7fc6813d6640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c101f90 0x7fc67c198190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc678000c00 tx=0x7fc678004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.688+0000 7fc6727fc640 1 -- 172.21.15.82:0/2633946547 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678043020 con 0x7fc67c101f90 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.688+0000 7fc6727fc640 1 -- 172.21.15.82:0/2633946547 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc678004440 con 0x7fc67c101f90 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.688+0000 7fc6727fc640 1 -- 172.21.15.82:0/2633946547 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678003c90 con 0x7fc67c101f90 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.688+0000 7fc68363a640 1 -- 172.21.15.82:0/2633946547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67c1a0040 con 0x7fc67c101f90 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.688+0000 7fc68363a640 1 -- 172.21.15.82:0/2633946547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc67c1a0590 con 0x7fc67c101f90 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.689+0000 7fc68363a640 1 -- 172.21.15.82:0/2633946547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc67c1092d0 con 0x7fc67c101f90 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.690+0000 7fc6727fc640 1 -- 172.21.15.82:0/2633946547 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc678046020 con 0x7fc67c101f90 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.690+0000 7fc6727fc640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc6680a3bb0 0x7fc6680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.691+0000 7fc680bd5640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc6680a3bb0 0x7fc6680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.691+0000 7fc6727fc640 1 -- 172.21.15.82:0/2633946547 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc6780f2600 con 0x7fc67c101f90 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.691+0000 7fc680bd5640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc6680a3bb0 0x7fc6680a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc67c1996a0 tx=0x7fc66c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.694+0000 7fc6727fc640 1 -- 172.21.15.82:0/2633946547 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc67804d020 con 0x7fc67c101f90 2024-04-21T08:44:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.911+0000 7fc68363a640 1 -- 172.21.15.82:0/2633946547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc67c02d5c0 con 0x7fc67c101f90 2024-04-21T08:44:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.912+0000 7fc6727fc640 1 -- 172.21.15.82:0/2633946547 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7fc6780bf480 con 0x7fc67c101f90 2024-04-21T08:44:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:16.911 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-21T08:44:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-04-21T08:44:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.914+0000 7fc68363a640 1 -- 172.21.15.82:0/2633946547 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc6680a3bb0 msgr2=0x7fc6680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.914+0000 7fc68363a640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc6680a3bb0 0x7fc6680a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc67c1996a0 tx=0x7fc66c005f90 comp rx=0 tx=0).stop 2024-04-21T08:44:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.914+0000 7fc68363a640 1 -- 172.21.15.82:0/2633946547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c101f90 msgr2=0x7fc67c198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.914+0000 7fc68363a640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c101f90 0x7fc67c198190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc678000c00 tx=0x7fc678004040 comp rx=0 tx=0).stop 2024-04-21T08:44:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.914+0000 7fc68363a640 1 -- 172.21.15.82:0/2633946547 shutdown_connections 2024-04-21T08:44:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.914+0000 7fc68363a640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc6680a3bb0 0x7fc6680a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.914+0000 7fc68363a640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc67c103b40 0x7fc67c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.914+0000 7fc68363a640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc67c103180 0x7fc67c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.914+0000 7fc68363a640 1 --2- 172.21.15.82:0/2633946547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c101f90 0x7fc67c198190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.914+0000 7fc68363a640 1 -- 172.21.15.82:0/2633946547 >> 172.21.15.82:0/2633946547 conn(0x7fc67c0fd500 msgr2=0x7fc67c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.914+0000 7fc68363a640 1 -- 172.21.15.82:0/2633946547 shutdown_connections 2024-04-21T08:44:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:16.914+0000 7fc68363a640 1 -- 172.21.15.82:0/2633946547 wait complete. 2024-04-21T08:44:17.365 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 21 08:44:17 smithi106 systemd[1]: Started Ceph mds.h for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:44:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:17 smithi106 ceph-mon[31072]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:17 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2633946547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:17 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:17 smithi134 ceph-mon[31772]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:17 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2633946547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:17 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:17.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:17 smithi082 ceph-mon[27967]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:17.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:17 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2633946547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:17.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:17.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:17.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:17 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:18.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:18 smithi106 ceph-mon[31072]: Deploying daemon mds.i on smithi134 2024-04-21T08:44:18.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:18 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] up:boot 2024-04-21T08:44:18.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:18 smithi106 ceph-mon[31072]: fsmap 8 up:standby 2024-04-21T08:44:18.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:18 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:44:18.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:18 smithi134 ceph-mon[31772]: Deploying daemon mds.i on smithi134 2024-04-21T08:44:18.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:18 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] up:boot 2024-04-21T08:44:18.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:18 smithi134 ceph-mon[31772]: fsmap 8 up:standby 2024-04-21T08:44:18.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:18 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:44:18.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:18 smithi082 ceph-mon[27967]: Deploying daemon mds.i on smithi134 2024-04-21T08:44:18.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:18 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] up:boot 2024-04-21T08:44:18.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:18 smithi082 ceph-mon[27967]: fsmap 8 up:standby 2024-04-21T08:44:18.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:18 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:44:18.925 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:19.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.035+0000 7fe458031640 1 -- 172.21.15.82:0/4205557434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe450102a10 msgr2=0x7fe450104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:19.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.035+0000 7fe458031640 1 --2- 172.21.15.82:0/4205557434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe450102a10 0x7fe450104e00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009900 tx=0x7fe44c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:19.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.035+0000 7fe458031640 1 -- 172.21.15.82:0/4205557434 shutdown_connections 2024-04-21T08:44:19.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.035+0000 7fe458031640 1 --2- 172.21.15.82:0/4205557434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe450107c70 0x7fe45010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:19.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.035+0000 7fe458031640 1 --2- 172.21.15.82:0/4205557434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450105340 0x7fe450107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:19.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.035+0000 7fe458031640 1 --2- 172.21.15.82:0/4205557434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe450102a10 0x7fe450104e00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:19.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.035+0000 7fe458031640 1 -- 172.21.15.82:0/4205557434 >> 172.21.15.82:0/4205557434 conn(0x7fe4500fc420 msgr2=0x7fe4500fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.035+0000 7fe458031640 1 -- 172.21.15.82:0/4205557434 shutdown_connections 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.035+0000 7fe458031640 1 -- 172.21.15.82:0/4205557434 wait complete. 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.035+0000 7fe458031640 1 Processor -- start 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.035+0000 7fe458031640 1 -- start start 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.035+0000 7fe458031640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe450102a10 0x7fe450102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe458031640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe450105340 0x7fe4501009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe458031640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450107c70 0x7fe450100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe458031640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe45010dad0 con 0x7fe450102a10 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe458031640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe45010d950 con 0x7fe450105340 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe458031640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe45010dc50 con 0x7fe450107c70 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe455dcd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe450102a10 0x7fe450102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe455dcd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe450102a10 0x7fe450102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47432/0 (socket says 172.21.15.82:47432) 2024-04-21T08:44:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe455dcd640 1 -- 172.21.15.82:0/3215178899 learned_addr learned my addr 172.21.15.82:0/3215178899 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe4565ce640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450107c70 0x7fe450100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe4555cc640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe450105340 0x7fe4501009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe455dcd640 1 -- 172.21.15.82:0/3215178899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450107c70 msgr2=0x7fe450100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe455dcd640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450107c70 0x7fe450100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe455dcd640 1 -- 172.21.15.82:0/3215178899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe450105340 msgr2=0x7fe4501009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe455dcd640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe450105340 0x7fe4501009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe455dcd640 1 -- 172.21.15.82:0/3215178899 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe450101790 con 0x7fe450102a10 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe4555cc640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe450105340 0x7fe4501009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe4565ce640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450107c70 0x7fe450100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe455dcd640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe450102a10 0x7fe450102380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe44c006010 tx=0x7fe44c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe446ffd640 1 -- 172.21.15.82:0/3215178899 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c043020 con 0x7fe450102a10 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe458031640 1 -- 172.21.15.82:0/3215178899 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4501a2290 con 0x7fe450102a10 2024-04-21T08:44:19.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe446ffd640 1 -- 172.21.15.82:0/3215178899 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe44c004440 con 0x7fe450102a10 2024-04-21T08:44:19.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe446ffd640 1 -- 172.21.15.82:0/3215178899 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c003bf0 con 0x7fe450102a10 2024-04-21T08:44:19.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.036+0000 7fe458031640 1 -- 172.21.15.82:0/3215178899 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4501a27e0 con 0x7fe450102a10 2024-04-21T08:44:19.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.037+0000 7fe458031640 1 -- 172.21.15.82:0/3215178899 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4500fdf10 con 0x7fe450102a10 2024-04-21T08:44:19.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.038+0000 7fe446ffd640 1 -- 172.21.15.82:0/3215178899 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe44c046020 con 0x7fe450102a10 2024-04-21T08:44:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.039+0000 7fe446ffd640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe43c0a3bb0 0x7fe43c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.039+0000 7fe4555cc640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe43c0a3bb0 0x7fe43c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.039+0000 7fe446ffd640 1 -- 172.21.15.82:0/3215178899 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe44c0f23a0 con 0x7fe450102a10 2024-04-21T08:44:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.039+0000 7fe4555cc640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe43c0a3bb0 0x7fe43c0a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe4501020f0 tx=0x7fe440005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:19.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.043+0000 7fe446ffd640 1 -- 172.21.15.82:0/3215178899 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe44c04d020 con 0x7fe450102a10 2024-04-21T08:44:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.260+0000 7fe458031640 1 -- 172.21.15.82:0/3215178899 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe45002d5c0 con 0x7fe450102a10 2024-04-21T08:44:19.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.260+0000 7fe446ffd640 1 -- 172.21.15.82:0/3215178899 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7fe44c0bf220 con 0x7fe450102a10 2024-04-21T08:44:19.260 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:19.260 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-21T08:44:19.260 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 9 2024-04-21T08:44:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.262+0000 7fe458031640 1 -- 172.21.15.82:0/3215178899 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe43c0a3bb0 msgr2=0x7fe43c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.262+0000 7fe458031640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe43c0a3bb0 0x7fe43c0a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe4501020f0 tx=0x7fe440005f90 comp rx=0 tx=0).stop 2024-04-21T08:44:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.262+0000 7fe458031640 1 -- 172.21.15.82:0/3215178899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe450102a10 msgr2=0x7fe450102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.262+0000 7fe458031640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe450102a10 0x7fe450102380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe44c006010 tx=0x7fe44c004040 comp rx=0 tx=0).stop 2024-04-21T08:44:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.262+0000 7fe458031640 1 -- 172.21.15.82:0/3215178899 shutdown_connections 2024-04-21T08:44:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.262+0000 7fe458031640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe43c0a3bb0 0x7fe43c0a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.262+0000 7fe458031640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe450107c70 0x7fe450100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.262+0000 7fe458031640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe450105340 0x7fe4501009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.262+0000 7fe458031640 1 --2- 172.21.15.82:0/3215178899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe450102a10 0x7fe450102380 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.262+0000 7fe458031640 1 -- 172.21.15.82:0/3215178899 >> 172.21.15.82:0/3215178899 conn(0x7fe4500fc420 msgr2=0x7fe450103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.262+0000 7fe458031640 1 -- 172.21.15.82:0/3215178899 shutdown_connections 2024-04-21T08:44:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:19.262+0000 7fe458031640 1 -- 172.21.15.82:0/3215178899 wait complete. 2024-04-21T08:44:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:19 smithi134 ceph-mon[31772]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:19 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3215178899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:19.476 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:44:19 smithi134 systemd[1]: Starting Ceph mds.i for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:44:19.536 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:19 smithi082 ceph-mon[27967]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:19.536 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:19 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3215178899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:19.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:19 smithi106 ceph-mon[31072]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:19.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:19 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3215178899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:19.880 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:44:19 smithi134 podman[62841]: 2024-04-21 08:44:19.519548687 +0000 UTC m=+0.017873996 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:44:19.880 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:44:19 smithi134 podman[62841]: 2024-04-21 08:44:19.628589664 +0000 UTC m=+0.126914959 container create 51575d3f162f0b56a52f4faa1aedef3d2667a16cb425309c3319fcd92c801074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:44:19.880 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:44:19 smithi134 podman[62841]: 2024-04-21 08:44:19.881269291 +0000 UTC m=+0.379594585 container init 51575d3f162f0b56a52f4faa1aedef3d2667a16cb425309c3319fcd92c801074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:44:20.226 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:44:19 smithi134 podman[62841]: 2024-04-21 08:44:19.88394564 +0000 UTC m=+0.382270938 container start 51575d3f162f0b56a52f4faa1aedef3d2667a16cb425309c3319fcd92c801074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True) 2024-04-21T08:44:20.226 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:44:19 smithi134 ceph-mds[62860]: -- 172.21.15.134:0/1142763377 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558249729a40 con 0x5582496fcc00 2024-04-21T08:44:20.226 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:44:19 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-i[62856]: starting mds.i at 2024-04-21T08:44:20.226 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:44:19 smithi134 bash[62841]: 51575d3f162f0b56a52f4faa1aedef3d2667a16cb425309c3319fcd92c801074 2024-04-21T08:44:20.226 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:44:19 smithi134 systemd[1]: Started Ceph mds.i for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:44:21.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:21 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:21.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:21 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:21.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:21 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:21.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:21 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:21.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:21 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:21 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:21 smithi082 ceph-mon[27967]: Deploying daemon mds.g on smithi082 2024-04-21T08:44:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:21 smithi082 ceph-mon[27967]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:21 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] up:boot 2024-04-21T08:44:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:21 smithi082 ceph-mon[27967]: fsmap 9 up:standby 2024-04-21T08:44:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:21 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:44:21.273 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:21 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:21 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:21 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:21 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:21 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:21 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:21.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:21 smithi106 ceph-mon[31072]: Deploying daemon mds.g on smithi082 2024-04-21T08:44:21.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:21 smithi106 ceph-mon[31072]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:21.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:21 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] up:boot 2024-04-21T08:44:21.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:21 smithi106 ceph-mon[31072]: fsmap 9 up:standby 2024-04-21T08:44:21.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:21 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:44:21.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.378+0000 7fdd7e96d640 1 -- 172.21.15.82:0/4030216000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd78100e70 msgr2=0x7fdd78101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.378+0000 7fdd7e96d640 1 --2- 172.21.15.82:0/4030216000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd78100e70 0x7fdd78101290 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009900 tx=0x7fdd6802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.378+0000 7fdd7e96d640 1 -- 172.21.15.82:0/4030216000 shutdown_connections 2024-04-21T08:44:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.378+0000 7fdd7e96d640 1 --2- 172.21.15.82:0/4030216000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd78102980 0x7fdd781092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.378+0000 7fdd7e96d640 1 --2- 172.21.15.82:0/4030216000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd78101fc0 0x7fdd78102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.378+0000 7fdd7e96d640 1 --2- 172.21.15.82:0/4030216000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd78100e70 0x7fdd78101290 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.378+0000 7fdd7e96d640 1 -- 172.21.15.82:0/4030216000 >> 172.21.15.82:0/4030216000 conn(0x7fdd780fc3e0 msgr2=0x7fdd780fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.378+0000 7fdd7e96d640 1 -- 172.21.15.82:0/4030216000 shutdown_connections 2024-04-21T08:44:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.378+0000 7fdd7e96d640 1 -- 172.21.15.82:0/4030216000 wait complete. 2024-04-21T08:44:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd7e96d640 1 Processor -- start 2024-04-21T08:44:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd7e96d640 1 -- start start 2024-04-21T08:44:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd7e96d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd78100e70 0x7fdd78198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:21.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd7e96d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd78101fc0 0x7fdd78198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd7e96d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd78102980 0x7fdd7819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd7e96d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7810ca20 con 0x7fdd78101fc0 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd7e96d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7810c8a0 con 0x7fdd78102980 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd7e96d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7810cba0 con 0x7fdd78100e70 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd777fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd78101fc0 0x7fdd78198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd777fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd78101fc0 0x7fdd78198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47450/0 (socket says 172.21.15.82:47450) 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd777fe640 1 -- 172.21.15.82:0/2471185582 learned_addr learned my addr 172.21.15.82:0/2471185582 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd7cf0a640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd78102980 0x7fdd7819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd777fe640 1 -- 172.21.15.82:0/2471185582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd78100e70 msgr2=0x7fdd78198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd77fff640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd78100e70 0x7fdd78198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd777fe640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd78100e70 0x7fdd78198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd777fe640 1 -- 172.21.15.82:0/2471185582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd78102980 msgr2=0x7fdd7819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd777fe640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd78102980 0x7fdd7819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd777fe640 1 -- 172.21.15.82:0/2471185582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7819fe40 con 0x7fdd78101fc0 2024-04-21T08:44:21.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.379+0000 7fdd7cf0a640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd78102980 0x7fdd7819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.380+0000 7fdd777fe640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd78101fc0 0x7fdd78198760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00c970 tx=0x7fdd6c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.380+0000 7fdd757fa640 1 -- 172.21.15.82:0/2471185582 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c004220 con 0x7fdd78101fc0 2024-04-21T08:44:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.380+0000 7fdd757fa640 1 -- 172.21.15.82:0/2471185582 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd6c0043c0 con 0x7fdd78101fc0 2024-04-21T08:44:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.380+0000 7fdd7e96d640 1 -- 172.21.15.82:0/2471185582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd781a0130 con 0x7fdd78101fc0 2024-04-21T08:44:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.380+0000 7fdd757fa640 1 -- 172.21.15.82:0/2471185582 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c00f450 con 0x7fdd78101fc0 2024-04-21T08:44:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.380+0000 7fdd7e96d640 1 -- 172.21.15.82:0/2471185582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd781a0700 con 0x7fdd78101fc0 2024-04-21T08:44:21.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.381+0000 7fdd7e96d640 1 -- 172.21.15.82:0/2471185582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd781a09c0 con 0x7fdd78101fc0 2024-04-21T08:44:21.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.382+0000 7fdd757fa640 1 -- 172.21.15.82:0/2471185582 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdd6c003610 con 0x7fdd78101fc0 2024-04-21T08:44:21.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.383+0000 7fdd757fa640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdd600a3bb0 0x7fdd600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:21.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.383+0000 7fdd77fff640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdd600a3bb0 0x7fdd600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:21.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.383+0000 7fdd757fa640 1 -- 172.21.15.82:0/2471185582 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fdd6c0cfe90 con 0x7fdd78101fc0 2024-04-21T08:44:21.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.383+0000 7fdd77fff640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdd600a3bb0 0x7fdd600a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd680096c0 tx=0x7fdd68000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:21.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.387+0000 7fdd757fa640 1 -- 172.21.15.82:0/2471185582 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd6c014030 con 0x7fdd78101fc0 2024-04-21T08:44:21.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:21 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:21.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:21 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:21.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:21 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:21.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:21 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:21.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:21 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:21.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:21 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:21.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:21 smithi134 ceph-mon[31772]: Deploying daemon mds.g on smithi082 2024-04-21T08:44:21.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:21 smithi134 ceph-mon[31772]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:21.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:21 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] up:boot 2024-04-21T08:44:21.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:21 smithi134 ceph-mon[31772]: fsmap 9 up:standby 2024-04-21T08:44:21.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:21 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:44:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.607+0000 7fdd7e96d640 1 -- 172.21.15.82:0/2471185582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd781a0b90 con 0x7fdd78101fc0 2024-04-21T08:44:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.608+0000 7fdd757fa640 1 -- 172.21.15.82:0/2471185582 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7fdd6c09cd10 con 0x7fdd78101fc0 2024-04-21T08:44:21.609 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:21.609 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T08:44:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 10 2024-04-21T08:44:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.610+0000 7fdd7e96d640 1 -- 172.21.15.82:0/2471185582 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdd600a3bb0 msgr2=0x7fdd600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.610+0000 7fdd7e96d640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdd600a3bb0 0x7fdd600a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd680096c0 tx=0x7fdd68000bc0 comp rx=0 tx=0).stop 2024-04-21T08:44:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.610+0000 7fdd7e96d640 1 -- 172.21.15.82:0/2471185582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd78101fc0 msgr2=0x7fdd78198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.610+0000 7fdd7e96d640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd78101fc0 0x7fdd78198760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00c970 tx=0x7fdd6c00ce30 comp rx=0 tx=0).stop 2024-04-21T08:44:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.610+0000 7fdd7e96d640 1 -- 172.21.15.82:0/2471185582 shutdown_connections 2024-04-21T08:44:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.610+0000 7fdd7e96d640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdd600a3bb0 0x7fdd600a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.610+0000 7fdd7e96d640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd78102980 0x7fdd7819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.610+0000 7fdd7e96d640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd78101fc0 0x7fdd78198760 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.610+0000 7fdd7e96d640 1 --2- 172.21.15.82:0/2471185582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd78100e70 0x7fdd78198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.610+0000 7fdd7e96d640 1 -- 172.21.15.82:0/2471185582 >> 172.21.15.82:0/2471185582 conn(0x7fdd780fc3e0 msgr2=0x7fdd780690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.610+0000 7fdd7e96d640 1 -- 172.21.15.82:0/2471185582 shutdown_connections 2024-04-21T08:44:21.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:21.610+0000 7fdd7e96d640 1 -- 172.21.15.82:0/2471185582 wait complete. 2024-04-21T08:44:22.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:22 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2471185582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:22.359 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:44:22 smithi082 systemd[1]: Starting Ceph mds.g for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:44:22.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:22 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2471185582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:22.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:22 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2471185582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:22.713 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:44:22 smithi082 podman[69884]: 2024-04-21 08:44:22.294636897 +0000 UTC m=+0.017861049 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:44:22.713 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:44:22 smithi082 podman[69884]: 2024-04-21 08:44:22.411406527 +0000 UTC m=+0.134630669 container create 81d7a4d8f12c2ff7ff9ef2f04854687bea804a1d523b0c4b56864d4f9abe19d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-21T08:44:22.714 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:44:22 smithi082 podman[69884]: 2024-04-21 08:44:22.614079711 +0000 UTC m=+0.337303851 container init 81d7a4d8f12c2ff7ff9ef2f04854687bea804a1d523b0c4b56864d4f9abe19d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-21T08:44:22.714 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:44:22 smithi082 podman[69884]: 2024-04-21 08:44:22.616590505 +0000 UTC m=+0.339814649 container start 81d7a4d8f12c2ff7ff9ef2f04854687bea804a1d523b0c4b56864d4f9abe19d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-g, ceph=True, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-21T08:44:22.714 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:44:22 smithi082 ceph-mds[69903]: -- 172.21.15.82:0/4047336656 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559bbb531a40 con 0x559bbb504800 2024-04-21T08:44:22.714 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:44:22 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-g[69899]: starting mds.g at 2024-04-21T08:44:22.714 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:44:22 smithi082 bash[69884]: 81d7a4d8f12c2ff7ff9ef2f04854687bea804a1d523b0c4b56864d4f9abe19d8 2024-04-21T08:44:23.039 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:44:22 smithi082 systemd[1]: Started Ceph mds.g for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:44:23.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:23 smithi134 ceph-mon[31772]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:23.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:23.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:23.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:23.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:23.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:23.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:23 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:23.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:23 smithi082 ceph-mon[27967]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:23.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:23.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:23 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:23 smithi106 ceph-mon[31072]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:23.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:23 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:23.622 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:23.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 -- 172.21.15.82:0/2307960800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98141011d0 msgr2=0x7f9814101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:23.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 --2- 172.21.15.82:0/2307960800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98141011d0 0x7f9814101630 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9804009900 tx=0x7f980402bcd0 comp rx=0 tx=0).stop 2024-04-21T08:44:23.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 -- 172.21.15.82:0/2307960800 shutdown_connections 2024-04-21T08:44:23.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 --2- 172.21.15.82:0/2307960800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9814101b70 0x7f9814101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:23.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 --2- 172.21.15.82:0/2307960800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98141011d0 0x7f9814101630 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:23.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 --2- 172.21.15.82:0/2307960800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9814100870 0x7f9814100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:23.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 -- 172.21.15.82:0/2307960800 >> 172.21.15.82:0/2307960800 conn(0x7f98140fc410 msgr2=0x7f98140fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:23.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 -- 172.21.15.82:0/2307960800 shutdown_connections 2024-04-21T08:44:23.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 -- 172.21.15.82:0/2307960800 wait complete. 2024-04-21T08:44:23.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 Processor -- start 2024-04-21T08:44:23.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 -- start start 2024-04-21T08:44:23.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9814100870 0x7f981419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:23.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98141011d0 0x7f981419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:23.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9814101b70 0x7f98141a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:23.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f981410fcc0 con 0x7f9814100870 2024-04-21T08:44:23.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f981410fb40 con 0x7f9814101b70 2024-04-21T08:44:23.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981c414640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f981410fe40 con 0x7f98141011d0 2024-04-21T08:44:23.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981a1b0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9814100870 0x7f981419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:23.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981a1b0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9814100870 0x7f981419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44096/0 (socket says 172.21.15.82:44096) 2024-04-21T08:44:23.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.728+0000 7f981a1b0640 1 -- 172.21.15.82:0/1831082140 learned_addr learned my addr 172.21.15.82:0/1831082140 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:23.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f981a9b1640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9814101b70 0x7f98141a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:23.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f98199af640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98141011d0 0x7f981419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:23.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f981a1b0640 1 -- 172.21.15.82:0/1831082140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98141011d0 msgr2=0x7f981419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:23.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f981a1b0640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98141011d0 0x7f981419a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:23.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f981a1b0640 1 -- 172.21.15.82:0/1831082140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9814101b70 msgr2=0x7f98141a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:23.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f981a1b0640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9814101b70 0x7f98141a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:23.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f981a1b0640 1 -- 172.21.15.82:0/1831082140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98141a1fd0 con 0x7f9814100870 2024-04-21T08:44:23.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f981a9b1640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9814101b70 0x7f98141a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:23.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f981a1b0640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9814100870 0x7f981419a3b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f981000d950 tx=0x7f981000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:23.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f980b7fe640 1 -- 172.21.15.82:0/1831082140 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810017070 con 0x7f9814100870 2024-04-21T08:44:23.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f980b7fe640 1 -- 172.21.15.82:0/1831082140 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9810009c30 con 0x7f9814100870 2024-04-21T08:44:23.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f981c414640 1 -- 172.21.15.82:0/1831082140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98141a22c0 con 0x7f9814100870 2024-04-21T08:44:23.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f980b7fe640 1 -- 172.21.15.82:0/1831082140 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981000f620 con 0x7f9814100870 2024-04-21T08:44:23.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.729+0000 7f981c414640 1 -- 172.21.15.82:0/1831082140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98141a2810 con 0x7f9814100870 2024-04-21T08:44:23.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.730+0000 7f981c414640 1 -- 172.21.15.82:0/1831082140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f981410b800 con 0x7f9814100870 2024-04-21T08:44:23.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.731+0000 7f980b7fe640 1 -- 172.21.15.82:0/1831082140 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9810010460 con 0x7f9814100870 2024-04-21T08:44:23.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.732+0000 7f980b7fe640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f98000a3bb0 0x7f98000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:23.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.732+0000 7f98199af640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f98000a3bb0 0x7f98000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:23.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.732+0000 7f980b7fe640 1 -- 172.21.15.82:0/1831082140 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f98100cfc40 con 0x7f9814100870 2024-04-21T08:44:23.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.732+0000 7f98199af640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f98000a3bb0 0x7f98000a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9804036040 tx=0x7f9804000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:23.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.736+0000 7f980b7fe640 1 -- 172.21.15.82:0/1831082140 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f981009cac0 con 0x7f9814100870 2024-04-21T08:44:23.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.953+0000 7f981c414640 1 -- 172.21.15.82:0/1831082140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f981402d5c0 con 0x7f9814100870 2024-04-21T08:44:23.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.953+0000 7f980b7fe640 1 -- 172.21.15.82:0/1831082140 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f9810009de0 con 0x7f9814100870 2024-04-21T08:44:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:23.953 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T08:44:23.954 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 11 2024-04-21T08:44:23.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.955+0000 7f981c414640 1 -- 172.21.15.82:0/1831082140 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f98000a3bb0 msgr2=0x7f98000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:23.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.955+0000 7f981c414640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f98000a3bb0 0x7f98000a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9804036040 tx=0x7f9804000bc0 comp rx=0 tx=0).stop 2024-04-21T08:44:23.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.955+0000 7f981c414640 1 -- 172.21.15.82:0/1831082140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9814100870 msgr2=0x7f981419a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:23.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.955+0000 7f981c414640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9814100870 0x7f981419a3b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f981000d950 tx=0x7f981000de10 comp rx=0 tx=0).stop 2024-04-21T08:44:23.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.955+0000 7f981c414640 1 -- 172.21.15.82:0/1831082140 shutdown_connections 2024-04-21T08:44:23.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.955+0000 7f981c414640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f98000a3bb0 0x7f98000a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:23.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.955+0000 7f981c414640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9814101b70 0x7f98141a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:23.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.955+0000 7f981c414640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98141011d0 0x7f981419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:23.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.955+0000 7f981c414640 1 --2- 172.21.15.82:0/1831082140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9814100870 0x7f981419a3b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:23.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.955+0000 7f981c414640 1 -- 172.21.15.82:0/1831082140 >> 172.21.15.82:0/1831082140 conn(0x7f98140fc410 msgr2=0x7f98140fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:23.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.956+0000 7f981c414640 1 -- 172.21.15.82:0/1831082140 shutdown_connections 2024-04-21T08:44:23.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:23.956+0000 7f981c414640 1 -- 172.21.15.82:0/1831082140 wait complete. 2024-04-21T08:44:24.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:24 smithi082 ceph-mon[27967]: Deploying daemon mds.a on smithi082 2024-04-21T08:44:24.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:24 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.82:6838/3697606919,v1:172.21.15.82:6839/3697606919] up:boot 2024-04-21T08:44:24.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:24 smithi082 ceph-mon[27967]: fsmap 10 up:standby 2024-04-21T08:44:24.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:24 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:44:24.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:24 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1831082140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:24.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:24 smithi106 ceph-mon[31072]: Deploying daemon mds.a on smithi082 2024-04-21T08:44:24.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:24 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.82:6838/3697606919,v1:172.21.15.82:6839/3697606919] up:boot 2024-04-21T08:44:24.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:24 smithi106 ceph-mon[31072]: fsmap 10 up:standby 2024-04-21T08:44:24.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:24 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:44:24.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:24 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1831082140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:24.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:24 smithi134 ceph-mon[31772]: Deploying daemon mds.a on smithi082 2024-04-21T08:44:24.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:24 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.82:6838/3697606919,v1:172.21.15.82:6839/3697606919] up:boot 2024-04-21T08:44:24.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:24 smithi134 ceph-mon[31772]: fsmap 10 up:standby 2024-04-21T08:44:24.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:24 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:44:24.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:24 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1831082140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:25.048 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:44:24 smithi082 systemd[1]: Starting Ceph mds.a for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:44:25.300 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:25 smithi082 ceph-mon[27967]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:25.301 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:44:25 smithi082 podman[70231]: 2024-04-21 08:44:25.027502285 +0000 UTC m=+0.017189972 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:44:25.301 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:44:25 smithi082 podman[70231]: 2024-04-21 08:44:25.160157178 +0000 UTC m=+0.149844841 container create 54b972facf11646326bc478b5486664c7527f68404d28b9af29865023ceb0edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-21T08:44:25.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:25 smithi106 ceph-mon[31072]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:25.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:25 smithi134 ceph-mon[31772]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:25.858 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:44:25 smithi082 podman[70231]: 2024-04-21 08:44:25.40455489 +0000 UTC m=+0.394242553 container init 54b972facf11646326bc478b5486664c7527f68404d28b9af29865023ceb0edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-21T08:44:25.858 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:44:25 smithi082 podman[70231]: 2024-04-21 08:44:25.407085565 +0000 UTC m=+0.396773229 container start 54b972facf11646326bc478b5486664c7527f68404d28b9af29865023ceb0edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-21T08:44:25.859 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:44:25 smithi082 ceph-mds[70250]: -- 172.21.15.82:0/1453874348 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f50f4da40 con 0x562f50f20c00 2024-04-21T08:44:25.859 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:44:25 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-a[70246]: starting mds.a at 2024-04-21T08:44:25.859 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:44:25 smithi082 bash[70231]: 54b972facf11646326bc478b5486664c7527f68404d28b9af29865023ceb0edd 2024-04-21T08:44:25.859 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:44:25 smithi082 systemd[1]: Started Ceph mds.a for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:44:25.966 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:26.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.071+0000 7f66a7281640 1 -- 172.21.15.82:0/2117063687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100ea0 msgr2=0x7f66a01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.071+0000 7f66a7281640 1 --2- 172.21.15.82:0/2117063687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100ea0 0x7f66a01012c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f669c009900 tx=0x7f669c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:26.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.071+0000 7f66a7281640 1 -- 172.21.15.82:0/2117063687 shutdown_connections 2024-04-21T08:44:26.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.071+0000 7f66a7281640 1 --2- 172.21.15.82:0/2117063687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a01029b0 0x7f66a01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.071+0000 7f66a7281640 1 --2- 172.21.15.82:0/2117063687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a0101ff0 0x7f66a0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.071+0000 7f66a7281640 1 --2- 172.21.15.82:0/2117063687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100ea0 0x7f66a01012c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.071+0000 7f66a7281640 1 -- 172.21.15.82:0/2117063687 >> 172.21.15.82:0/2117063687 conn(0x7f66a00fc410 msgr2=0x7f66a00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.071+0000 7f66a7281640 1 -- 172.21.15.82:0/2117063687 shutdown_connections 2024-04-21T08:44:26.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.071+0000 7f66a7281640 1 -- 172.21.15.82:0/2117063687 wait complete. 2024-04-21T08:44:26.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a7281640 1 Processor -- start 2024-04-21T08:44:26.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a7281640 1 -- start start 2024-04-21T08:44:26.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a7281640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100ea0 0x7f66a0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a7281640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a0101ff0 0x7f66a0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a7281640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a01029b0 0x7f66a019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a7281640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f66a010db00 con 0x7f66a0100ea0 2024-04-21T08:44:26.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a7281640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66a010d980 con 0x7f66a01029b0 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a7281640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66a010dc80 con 0x7f66a0101ff0 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a501d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100ea0 0x7f66a0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a501d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100ea0 0x7f66a0198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44122/0 (socket says 172.21.15.82:44122) 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a501d640 1 -- 172.21.15.82:0/803743622 learned_addr learned my addr 172.21.15.82:0/803743622 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a581e640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a01029b0 0x7f66a019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a481c640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a0101ff0 0x7f66a0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a501d640 1 -- 172.21.15.82:0/803743622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a0101ff0 msgr2=0x7f66a0198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a501d640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a0101ff0 0x7f66a0198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a501d640 1 -- 172.21.15.82:0/803743622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a01029b0 msgr2=0x7f66a019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a501d640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a01029b0 0x7f66a019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a501d640 1 -- 172.21.15.82:0/803743622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a019fd50 con 0x7f66a0100ea0 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a481c640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a0101ff0 0x7f66a0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:26.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.072+0000 7f66a501d640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100ea0 0x7f66a0198130 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f669c006010 tx=0x7f669c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.073+0000 7f66967fc640 1 -- 172.21.15.82:0/803743622 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c043020 con 0x7f66a0100ea0 2024-04-21T08:44:26.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.073+0000 7f66967fc640 1 -- 172.21.15.82:0/803743622 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669c004aa0 con 0x7f66a0100ea0 2024-04-21T08:44:26.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.073+0000 7f66a7281640 1 -- 172.21.15.82:0/803743622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a019ffe0 con 0x7f66a0100ea0 2024-04-21T08:44:26.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.073+0000 7f66967fc640 1 -- 172.21.15.82:0/803743622 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c003c70 con 0x7f66a0100ea0 2024-04-21T08:44:26.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.073+0000 7f66a7281640 1 -- 172.21.15.82:0/803743622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a01a04a0 con 0x7f66a0100ea0 2024-04-21T08:44:26.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.074+0000 7f66a7281640 1 -- 172.21.15.82:0/803743622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a01081b0 con 0x7f66a0100ea0 2024-04-21T08:44:26.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.076+0000 7f66967fc640 1 -- 172.21.15.82:0/803743622 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f669c046020 con 0x7f66a0100ea0 2024-04-21T08:44:26.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.076+0000 7f66967fc640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f668c0a3c30 0x7f668c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:26.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.076+0000 7f66a481c640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f668c0a3c30 0x7f668c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:26.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.076+0000 7f66967fc640 1 -- 172.21.15.82:0/803743622 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f669c0f2930 con 0x7f66a0100ea0 2024-04-21T08:44:26.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.077+0000 7f66a481c640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f668c0a3c30 0x7f668c0a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f66900058c0 tx=0x7f6690005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.080+0000 7f66967fc640 1 -- 172.21.15.82:0/803743622 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f669c04d020 con 0x7f66a0100ea0 2024-04-21T08:44:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.295+0000 7f66a7281640 1 -- 172.21.15.82:0/803743622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66a002d5c0 con 0x7f66a0100ea0 2024-04-21T08:44:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.296+0000 7f66967fc640 1 -- 172.21.15.82:0/803743622 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7f669c0bf7b0 con 0x7f66a0100ea0 2024-04-21T08:44:26.296 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:26.296 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T08:44:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 12 2024-04-21T08:44:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.298+0000 7f66a7281640 1 -- 172.21.15.82:0/803743622 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f668c0a3c30 msgr2=0x7f668c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.298+0000 7f66a7281640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f668c0a3c30 0x7f668c0a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f66900058c0 tx=0x7f6690005850 comp rx=0 tx=0).stop 2024-04-21T08:44:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.298+0000 7f66a7281640 1 -- 172.21.15.82:0/803743622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100ea0 msgr2=0x7f66a0198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.298+0000 7f66a7281640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100ea0 0x7f66a0198130 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f669c006010 tx=0x7f669c0042f0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.298+0000 7f66a7281640 1 -- 172.21.15.82:0/803743622 shutdown_connections 2024-04-21T08:44:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.298+0000 7f66a7281640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f668c0a3c30 0x7f668c0a60f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.298+0000 7f66a7281640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66a01029b0 0x7f66a019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.298+0000 7f66a7281640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a0101ff0 0x7f66a0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.298+0000 7f66a7281640 1 --2- 172.21.15.82:0/803743622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100ea0 0x7f66a0198130 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:26.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.298+0000 7f66a7281640 1 -- 172.21.15.82:0/803743622 >> 172.21.15.82:0/803743622 conn(0x7f66a00fc410 msgr2=0x7f66a0105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:26.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.298+0000 7f66a7281640 1 -- 172.21.15.82:0/803743622 shutdown_connections 2024-04-21T08:44:26.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:26.298+0000 7f66a7281640 1 -- 172.21.15.82:0/803743622 wait complete. 2024-04-21T08:44:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:26 smithi082 ceph-mon[27967]: Deploying daemon mds.b on smithi106 2024-04-21T08:44:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:26 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.82:6840/1327149046,v1:172.21.15.82:6841/1327149046] up:boot 2024-04-21T08:44:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:26 smithi082 ceph-mon[27967]: fsmap 11 up:standby 2024-04-21T08:44:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:44:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:26 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/803743622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:26 smithi082 ceph-mon[27967]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:26 smithi106 ceph-mon[31072]: Deploying daemon mds.b on smithi106 2024-04-21T08:44:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:26 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.82:6840/1327149046,v1:172.21.15.82:6841/1327149046] up:boot 2024-04-21T08:44:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:26 smithi106 ceph-mon[31072]: fsmap 11 up:standby 2024-04-21T08:44:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:44:26.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:26 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/803743622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:26.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:26 smithi106 ceph-mon[31072]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:26.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:26.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:26.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:26.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:44:26.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:44:26.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:26.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:26 smithi134 ceph-mon[31772]: Deploying daemon mds.b on smithi106 2024-04-21T08:44:26.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:26 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.82:6840/1327149046,v1:172.21.15.82:6841/1327149046] up:boot 2024-04-21T08:44:26.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:26 smithi134 ceph-mon[31772]: fsmap 11 up:standby 2024-04-21T08:44:26.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:44:26.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:26 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/803743622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:26.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:26 smithi134 ceph-mon[31772]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:28.122 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:44:27 smithi106 systemd[1]: Starting Ceph mds.b for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:44:28.310 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:28.378 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:44:28 smithi106 podman[60818]: 2024-04-21 08:44:28.025665908 +0000 UTC m=+0.018000526 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:44:28.378 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:44:28 smithi106 podman[60818]: 2024-04-21 08:44:28.151269011 +0000 UTC m=+0.143603654 container create 6327164844ccc216ff5ab8fa5f075bf1d48bca5a93cf23834a6a0f9ab610c2f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-21T08:44:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.423+0000 7f257dd75640 1 -- 172.21.15.82:0/487928043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2578100e70 msgr2=0x7f2578101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.423+0000 7f257dd75640 1 --2- 172.21.15.82:0/487928043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2578100e70 0x7f2578101290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f256c009900 tx=0x7f256c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.423+0000 7f257dd75640 1 -- 172.21.15.82:0/487928043 shutdown_connections 2024-04-21T08:44:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.423+0000 7f257dd75640 1 --2- 172.21.15.82:0/487928043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2578102980 0x7f25781092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.423+0000 7f257dd75640 1 --2- 172.21.15.82:0/487928043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2578101fc0 0x7f2578102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.423+0000 7f257dd75640 1 --2- 172.21.15.82:0/487928043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2578100e70 0x7f2578101290 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.423+0000 7f257dd75640 1 -- 172.21.15.82:0/487928043 >> 172.21.15.82:0/487928043 conn(0x7f25780fc3e0 msgr2=0x7f25780fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.423+0000 7f257dd75640 1 -- 172.21.15.82:0/487928043 shutdown_connections 2024-04-21T08:44:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.423+0000 7f257dd75640 1 -- 172.21.15.82:0/487928043 wait complete. 2024-04-21T08:44:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.423+0000 7f257dd75640 1 Processor -- start 2024-04-21T08:44:28.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.423+0000 7f257dd75640 1 -- start start 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f257dd75640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2578100e70 0x7f25781981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f257dd75640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2578101fc0 0x7f25781986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f257dd75640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2578102980 0x7f257819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f257dd75640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f257810c9a0 con 0x7f2578100e70 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f257dd75640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f257810c820 con 0x7f2578101fc0 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f257dd75640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f257810cb20 con 0x7f2578102980 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f25777fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2578100e70 0x7f25781981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f25777fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2578100e70 0x7f25781981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44138/0 (socket says 172.21.15.82:44138) 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f25777fe640 1 -- 172.21.15.82:0/1300033120 learned_addr learned my addr 172.21.15.82:0/1300033120 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f2576ffd640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2578101fc0 0x7f25781986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f2577fff640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2578102980 0x7f257819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f25777fe640 1 -- 172.21.15.82:0/1300033120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2578102980 msgr2=0x7f257819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:28.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f25777fe640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2578102980 0x7f257819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f25777fe640 1 -- 172.21.15.82:0/1300033120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2578101fc0 msgr2=0x7f25781986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f25777fe640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2578101fc0 0x7f25781986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f25777fe640 1 -- 172.21.15.82:0/1300033120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257819fdc0 con 0x7f2578100e70 2024-04-21T08:44:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f25777fe640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2578100e70 0x7f25781981a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f256c006010 tx=0x7f256c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f2574ff9640 1 -- 172.21.15.82:0/1300033120 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c043020 con 0x7f2578100e70 2024-04-21T08:44:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f2574ff9640 1 -- 172.21.15.82:0/1300033120 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f256c004440 con 0x7f2578100e70 2024-04-21T08:44:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f257dd75640 1 -- 172.21.15.82:0/1300033120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25781a0050 con 0x7f2578100e70 2024-04-21T08:44:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f2574ff9640 1 -- 172.21.15.82:0/1300033120 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c003bf0 con 0x7f2578100e70 2024-04-21T08:44:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.424+0000 7f257dd75640 1 -- 172.21.15.82:0/1300033120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25781a05a0 con 0x7f2578100e70 2024-04-21T08:44:28.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.425+0000 7f257dd75640 1 -- 172.21.15.82:0/1300033120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25781a07a0 con 0x7f2578100e70 2024-04-21T08:44:28.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.427+0000 7f2574ff9640 1 -- 172.21.15.82:0/1300033120 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f256c046020 con 0x7f2578100e70 2024-04-21T08:44:28.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.427+0000 7f2574ff9640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f25600a3bb0 0x7f25600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:28.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.427+0000 7f2576ffd640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f25600a3bb0 0x7f25600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:28.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.427+0000 7f2574ff9640 1 -- 172.21.15.82:0/1300033120 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f256c0f2980 con 0x7f2578100e70 2024-04-21T08:44:28.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.428+0000 7f2576ffd640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f25600a3bb0 0x7f25600a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f25781996b0 tx=0x7f2568005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:28.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.431+0000 7f2574ff9640 1 -- 172.21.15.82:0/1300033120 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f256c04d020 con 0x7f2578100e70 2024-04-21T08:44:28.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.662+0000 7f257dd75640 1 -- 172.21.15.82:0/1300033120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25781a0970 con 0x7f2578100e70 2024-04-21T08:44:28.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.663+0000 7f2574ff9640 1 -- 172.21.15.82:0/1300033120 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7f256c0bf800 con 0x7f2578100e70 2024-04-21T08:44:28.663 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:28.663 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T08:44:28.663 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 12 2024-04-21T08:44:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.665+0000 7f257dd75640 1 -- 172.21.15.82:0/1300033120 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f25600a3bb0 msgr2=0x7f25600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.665+0000 7f257dd75640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f25600a3bb0 0x7f25600a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f25781996b0 tx=0x7f2568005f90 comp rx=0 tx=0).stop 2024-04-21T08:44:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.665+0000 7f257dd75640 1 -- 172.21.15.82:0/1300033120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2578100e70 msgr2=0x7f25781981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.665+0000 7f257dd75640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2578100e70 0x7f25781981a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f256c006010 tx=0x7f256c004040 comp rx=0 tx=0).stop 2024-04-21T08:44:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.665+0000 7f257dd75640 1 -- 172.21.15.82:0/1300033120 shutdown_connections 2024-04-21T08:44:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.665+0000 7f257dd75640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f25600a3bb0 0x7f25600a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:28.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.665+0000 7f257dd75640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2578102980 0x7f257819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:28.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.665+0000 7f257dd75640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2578101fc0 0x7f25781986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:28.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.665+0000 7f257dd75640 1 --2- 172.21.15.82:0/1300033120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2578100e70 0x7f25781981a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:28.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.665+0000 7f257dd75640 1 -- 172.21.15.82:0/1300033120 >> 172.21.15.82:0/1300033120 conn(0x7f25780fc3e0 msgr2=0x7f25780fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:28.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.665+0000 7f257dd75640 1 -- 172.21.15.82:0/1300033120 shutdown_connections 2024-04-21T08:44:28.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:28.665+0000 7f257dd75640 1 -- 172.21.15.82:0/1300033120 wait complete. 2024-04-21T08:44:28.878 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:44:28 smithi106 podman[60818]: 2024-04-21 08:44:28.38743315 +0000 UTC m=+0.379767751 container init 6327164844ccc216ff5ab8fa5f075bf1d48bca5a93cf23834a6a0f9ab610c2f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-21T08:44:28.878 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:44:28 smithi106 podman[60818]: 2024-04-21 08:44:28.390370127 +0000 UTC m=+0.382704738 container start 6327164844ccc216ff5ab8fa5f075bf1d48bca5a93cf23834a6a0f9ab610c2f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-21T08:44:28.878 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:44:28 smithi106 ceph-mds[60838]: -- 172.21.15.106:0/1145980351 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56520a9c3a40 con 0x56520a997000 2024-04-21T08:44:28.878 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:44:28 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b[60834]: starting mds.b at 2024-04-21T08:44:28.878 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:44:28 smithi106 bash[60818]: 6327164844ccc216ff5ab8fa5f075bf1d48bca5a93cf23834a6a0f9ab610c2f3 2024-04-21T08:44:28.878 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:44:28 smithi106 systemd[1]: Started Ceph mds.b for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:44:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:29 smithi106 ceph-mon[31072]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:29 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:29 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:29 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:29 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:29 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:29 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1300033120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:29 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:44:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:29 smithi134 ceph-mon[31772]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:29 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:29 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:29 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:29 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:29 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:29 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1300033120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:29 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:44:29.740 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:29 smithi082 ceph-mon[27967]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:29.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:29.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:29 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1300033120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:29.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:29 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:44:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:30 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] up:boot 2024-04-21T08:44:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:30 smithi106 ceph-mon[31072]: fsmap 12 up:standby 2024-04-21T08:44:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:30 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:44:30.678 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:30.699 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:30 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] up:boot 2024-04-21T08:44:30.700 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:30 smithi082 ceph-mon[27967]: fsmap 12 up:standby 2024-04-21T08:44:30.700 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:30 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:44:30.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:30 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] up:boot 2024-04-21T08:44:30.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:30 smithi134 ceph-mon[31772]: fsmap 12 up:standby 2024-04-21T08:44:30.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:30 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:44:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.780+0000 7f0227cc1640 1 -- 172.21.15.82:0/3721729722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0220100e70 msgr2=0x7f0220101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.780+0000 7f0227cc1640 1 --2- 172.21.15.82:0/3721729722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0220100e70 0x7f0220101290 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f021c009900 tx=0x7f021c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.780+0000 7f0227cc1640 1 -- 172.21.15.82:0/3721729722 shutdown_connections 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.780+0000 7f0227cc1640 1 --2- 172.21.15.82:0/3721729722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0220102980 0x7f02201092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.780+0000 7f0227cc1640 1 --2- 172.21.15.82:0/3721729722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0220101fc0 0x7f0220102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.780+0000 7f0227cc1640 1 --2- 172.21.15.82:0/3721729722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0220100e70 0x7f0220101290 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.780+0000 7f0227cc1640 1 -- 172.21.15.82:0/3721729722 >> 172.21.15.82:0/3721729722 conn(0x7f02200fc3e0 msgr2=0x7f02200fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.780+0000 7f0227cc1640 1 -- 172.21.15.82:0/3721729722 shutdown_connections 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.780+0000 7f0227cc1640 1 -- 172.21.15.82:0/3721729722 wait complete. 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f0227cc1640 1 Processor -- start 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f0227cc1640 1 -- start start 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f0227cc1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0220100e70 0x7f02201981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f0227cc1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0220101fc0 0x7f02201986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f0227cc1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0220102980 0x7f022019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f0227cc1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f022010c9c0 con 0x7f0220101fc0 2024-04-21T08:44:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f0227cc1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f022010c840 con 0x7f0220100e70 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f0227cc1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f022010cb40 con 0x7f0220102980 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f022525c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0220101fc0 0x7f02201986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f022525c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0220101fc0 0x7f02201986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44162/0 (socket says 172.21.15.82:44162) 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f022525c640 1 -- 172.21.15.82:0/1438382120 learned_addr learned my addr 172.21.15.82:0/1438382120 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f022625e640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0220102980 0x7f022019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f0225a5d640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0220100e70 0x7f02201981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f022525c640 1 -- 172.21.15.82:0/1438382120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0220102980 msgr2=0x7f022019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f022525c640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0220102980 0x7f022019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f022525c640 1 -- 172.21.15.82:0/1438382120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0220100e70 msgr2=0x7f02201981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.781+0000 7f022525c640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0220100e70 0x7f02201981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.782+0000 7f022525c640 1 -- 172.21.15.82:0/1438382120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022019fdd0 con 0x7f0220101fc0 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.782+0000 7f022525c640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0220101fc0 0x7f02201986f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f021000d770 tx=0x7f021000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.782+0000 7f0216ffd640 1 -- 172.21.15.82:0/1438382120 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210017070 con 0x7f0220101fc0 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.782+0000 7f0227cc1640 1 -- 172.21.15.82:0/1438382120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02201a00c0 con 0x7f0220101fc0 2024-04-21T08:44:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.782+0000 7f0216ffd640 1 -- 172.21.15.82:0/1438382120 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0210009c30 con 0x7f0220101fc0 2024-04-21T08:44:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.782+0000 7f0216ffd640 1 -- 172.21.15.82:0/1438382120 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210003cc0 con 0x7f0220101fc0 2024-04-21T08:44:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.782+0000 7f0227cc1640 1 -- 172.21.15.82:0/1438382120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02201a0610 con 0x7f0220101fc0 2024-04-21T08:44:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.783+0000 7f0227cc1640 1 -- 172.21.15.82:0/1438382120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02201a0870 con 0x7f0220101fc0 2024-04-21T08:44:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.784+0000 7f0216ffd640 1 -- 172.21.15.82:0/1438382120 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0210003e60 con 0x7f0220101fc0 2024-04-21T08:44:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.785+0000 7f0216ffd640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f020c0a3bb0 0x7f020c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:30.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.785+0000 7f0216ffd640 1 -- 172.21.15.82:0/1438382120 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f02100cfd30 con 0x7f0220101fc0 2024-04-21T08:44:30.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.785+0000 7f0225a5d640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f020c0a3bb0 0x7f020c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:30.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.785+0000 7f0225a5d640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f020c0a3bb0 0x7f020c0a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f021c000c00 tx=0x7f021c005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:30.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:30.789+0000 7f0216ffd640 1 -- 172.21.15.82:0/1438382120 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0210009de0 con 0x7f0220101fc0 2024-04-21T08:44:31.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.009+0000 7f0227cc1640 1 -- 172.21.15.82:0/1438382120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02201a0a40 con 0x7f0220101fc0 2024-04-21T08:44:31.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.010+0000 7f0216ffd640 1 -- 172.21.15.82:0/1438382120 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7f021009cbb0 con 0x7f0220101fc0 2024-04-21T08:44:31.010 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:31.010 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-21T08:44:31.010 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 13 2024-04-21T08:44:31.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.012+0000 7f0227cc1640 1 -- 172.21.15.82:0/1438382120 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f020c0a3bb0 msgr2=0x7f020c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.012+0000 7f0227cc1640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f020c0a3bb0 0x7f020c0a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f021c000c00 tx=0x7f021c005be0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.012+0000 7f0227cc1640 1 -- 172.21.15.82:0/1438382120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0220101fc0 msgr2=0x7f02201986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.012+0000 7f0227cc1640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0220101fc0 0x7f02201986f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f021000d770 tx=0x7f021000dc30 comp rx=0 tx=0).stop 2024-04-21T08:44:31.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.012+0000 7f0227cc1640 1 -- 172.21.15.82:0/1438382120 shutdown_connections 2024-04-21T08:44:31.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.012+0000 7f0227cc1640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f020c0a3bb0 0x7f020c0a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.012+0000 7f0227cc1640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0220102980 0x7f022019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.012+0000 7f0227cc1640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0220101fc0 0x7f02201986f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.012+0000 7f0227cc1640 1 --2- 172.21.15.82:0/1438382120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0220100e70 0x7f02201981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.012+0000 7f0227cc1640 1 -- 172.21.15.82:0/1438382120 >> 172.21.15.82:0/1438382120 conn(0x7f02200fc3e0 msgr2=0x7f02200fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:31.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.013+0000 7f0227cc1640 1 -- 172.21.15.82:0/1438382120 shutdown_connections 2024-04-21T08:44:31.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.013+0000 7f0227cc1640 1 -- 172.21.15.82:0/1438382120 wait complete. 2024-04-21T08:44:31.024 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:44:31.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.166+0000 7f78cca4e640 1 -- 172.21.15.82:0/3914312783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c8103b40 msgr2=0x7f78c810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.166+0000 7f78cca4e640 1 --2- 172.21.15.82:0/3914312783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c8103b40 0x7f78c810a3d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009ca0 tx=0x7f78c002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.166+0000 7f78cca4e640 1 -- 172.21.15.82:0/3914312783 shutdown_connections 2024-04-21T08:44:31.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.166+0000 7f78cca4e640 1 --2- 172.21.15.82:0/3914312783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c8103b40 0x7f78c810a3d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.166+0000 7f78cca4e640 1 --2- 172.21.15.82:0/3914312783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c8103180 0x7f78c8103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.166+0000 7f78cca4e640 1 --2- 172.21.15.82:0/3914312783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78c8101f90 0x7f78c81023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.166+0000 7f78cca4e640 1 -- 172.21.15.82:0/3914312783 >> 172.21.15.82:0/3914312783 conn(0x7f78c80fd500 msgr2=0x7f78c80ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:31.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.166+0000 7f78cca4e640 1 -- 172.21.15.82:0/3914312783 shutdown_connections 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.166+0000 7f78cca4e640 1 -- 172.21.15.82:0/3914312783 wait complete. 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78cca4e640 1 Processor -- start 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78cca4e640 1 -- start start 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78cca4e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78c8101f90 0x7f78c8198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78cca4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c8103180 0x7f78c8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78cca4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c8103b40 0x7f78c819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78cca4e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f78c810dad0 con 0x7f78c8103b40 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78cca4e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f78c810d950 con 0x7f78c8101f90 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78cca4e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78c810dc50 con 0x7f78c8103180 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78c6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c8103b40 0x7f78c819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78c6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c8103b40 0x7f78c819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44172/0 (socket says 172.21.15.82:44172) 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78c6d9d640 1 -- 172.21.15.82:0/2948157837 learned_addr learned my addr 172.21.15.82:0/2948157837 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78bffff640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c8103180 0x7f78c8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:31.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78c659c640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78c8101f90 0x7f78c8198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78c6d9d640 1 -- 172.21.15.82:0/2948157837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c8103180 msgr2=0x7f78c8198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78c6d9d640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c8103180 0x7f78c8198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78c6d9d640 1 -- 172.21.15.82:0/2948157837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78c8101f90 msgr2=0x7f78c8198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78c6d9d640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78c8101f90 0x7f78c8198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78c6d9d640 1 -- 172.21.15.82:0/2948157837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c819fd70 con 0x7f78c8103b40 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78c659c640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78c8101f90 0x7f78c8198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.167+0000 7f78bffff640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c8103180 0x7f78c8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.168+0000 7f78c6d9d640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c8103b40 0x7f78c819f700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f78c002c700 tx=0x7f78c00093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.168+0000 7f78bf7fe640 1 -- 172.21.15.82:0/2948157837 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0041020 con 0x7f78c8103b40 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.168+0000 7f78bf7fe640 1 -- 172.21.15.82:0/2948157837 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c0034d60 con 0x7f78c8103b40 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.168+0000 7f78cca4e640 1 -- 172.21.15.82:0/2948157837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c81a0000 con 0x7f78c8103b40 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.168+0000 7f78bf7fe640 1 -- 172.21.15.82:0/2948157837 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0039040 con 0x7f78c8103b40 2024-04-21T08:44:31.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.168+0000 7f78cca4e640 1 -- 172.21.15.82:0/2948157837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c81a0540 con 0x7f78c8103b40 2024-04-21T08:44:31.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.169+0000 7f78cca4e640 1 -- 172.21.15.82:0/2948157837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c81092b0 con 0x7f78c8103b40 2024-04-21T08:44:31.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.170+0000 7f78bf7fe640 1 -- 172.21.15.82:0/2948157837 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f78c0044020 con 0x7f78c8103b40 2024-04-21T08:44:31.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.171+0000 7f78bf7fe640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f78b40a3c30 0x7f78b40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:31.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.171+0000 7f78c659c640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f78b40a3c30 0x7f78b40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:31.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.171+0000 7f78bf7fe640 1 -- 172.21.15.82:0/2948157837 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f78c00f1ee0 con 0x7f78c8103b40 2024-04-21T08:44:31.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.171+0000 7f78c659c640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f78b40a3c30 0x7f78b40a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f78b8005d00 tx=0x7f78b8005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:31.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.175+0000 7f78bf7fe640 1 -- 172.21.15.82:0/2948157837 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78c004b020 con 0x7f78c8103b40 2024-04-21T08:44:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.362+0000 7f78cca4e640 1 -- 172.21.15.82:0/2948157837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78c8062b50 con 0x7f78c8103b40 2024-04-21T08:44:31.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.363+0000 7f78bf7fe640 1 -- 172.21.15.82:0/2948157837 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18468 (secure 0 0 0) 0x7f78c00bed60 con 0x7f78c8103b40 2024-04-21T08:44:31.362 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:31.362 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":81,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:43:42.372753+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:44:31.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.365+0000 7f78cca4e640 1 -- 172.21.15.82:0/2948157837 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f78b40a3c30 msgr2=0x7f78b40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.365+0000 7f78cca4e640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f78b40a3c30 0x7f78b40a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f78b8005d00 tx=0x7f78b8005ef0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.365+0000 7f78cca4e640 1 -- 172.21.15.82:0/2948157837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c8103b40 msgr2=0x7f78c819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.365+0000 7f78cca4e640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c8103b40 0x7f78c819f700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f78c002c700 tx=0x7f78c00093f0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.365+0000 7f78cca4e640 1 -- 172.21.15.82:0/2948157837 shutdown_connections 2024-04-21T08:44:31.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.365+0000 7f78cca4e640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f78b40a3c30 0x7f78b40a60f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.365+0000 7f78cca4e640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78c8103b40 0x7f78c819f700 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.365+0000 7f78cca4e640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78c8103180 0x7f78c8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.366+0000 7f78cca4e640 1 --2- 172.21.15.82:0/2948157837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78c8101f90 0x7f78c8198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.366+0000 7f78cca4e640 1 -- 172.21.15.82:0/2948157837 >> 172.21.15.82:0/2948157837 conn(0x7f78c80fd500 msgr2=0x7f78c8106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:31.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.366+0000 7f78cca4e640 1 -- 172.21.15.82:0/2948157837 shutdown_connections 2024-04-21T08:44:31.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.366+0000 7f78cca4e640 1 -- 172.21.15.82:0/2948157837 wait complete. 2024-04-21T08:44:31.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T08:44:31.381 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:44:31.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 -- 172.21.15.82:0/2234873975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6534100e90 msgr2=0x7f65341012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 --2- 172.21.15.82:0/2234873975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6534100e90 0x7f65341012b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6524009900 tx=0x7f652402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:31.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 -- 172.21.15.82:0/2234873975 shutdown_connections 2024-04-21T08:44:31.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 --2- 172.21.15.82:0/2234873975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65341029a0 0x7f65341092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 --2- 172.21.15.82:0/2234873975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6534101fe0 0x7f6534102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 --2- 172.21.15.82:0/2234873975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6534100e90 0x7f65341012b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 -- 172.21.15.82:0/2234873975 >> 172.21.15.82:0/2234873975 conn(0x7f65340fc400 msgr2=0x7f65340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:31.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 -- 172.21.15.82:0/2234873975 shutdown_connections 2024-04-21T08:44:31.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 -- 172.21.15.82:0/2234873975 wait complete. 2024-04-21T08:44:31.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 Processor -- start 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 -- start start 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6534100e90 0x7f6534104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.485+0000 7f653c21e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6534101fe0 0x7f6534103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f653c21e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65341029a0 0x7f6534103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f653c21e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f653410c9e0 con 0x7f6534101fe0 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f653c21e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f653410c860 con 0x7f6534100e90 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f653c21e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f653410cb60 con 0x7f65341029a0 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f65397b9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6534101fe0 0x7f6534103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f65397b9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6534101fe0 0x7f6534103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44180/0 (socket says 172.21.15.82:44180) 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f65397b9640 1 -- 172.21.15.82:0/835621132 learned_addr learned my addr 172.21.15.82:0/835621132 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f6539fba640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6534100e90 0x7f6534104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f653a7bb640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65341029a0 0x7f6534103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f65397b9640 1 -- 172.21.15.82:0/835621132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65341029a0 msgr2=0x7f6534103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f65397b9640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65341029a0 0x7f6534103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f65397b9640 1 -- 172.21.15.82:0/835621132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6534100e90 msgr2=0x7f6534104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f65397b9640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6534100e90 0x7f6534104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f65397b9640 1 -- 172.21.15.82:0/835621132 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6534104220 con 0x7f6534101fe0 2024-04-21T08:44:31.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f6539fba640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6534100e90 0x7f6534104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:31.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f65397b9640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6534101fe0 0x7f6534103450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f653000e9c0 tx=0x7f653000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:31.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f652affd640 1 -- 172.21.15.82:0/835621132 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653000cd80 con 0x7f6534101fe0 2024-04-21T08:44:31.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f652affd640 1 -- 172.21.15.82:0/835621132 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65300035f0 con 0x7f6534101fe0 2024-04-21T08:44:31.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f652affd640 1 -- 172.21.15.82:0/835621132 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530010690 con 0x7f6534101fe0 2024-04-21T08:44:31.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f653c21e640 1 -- 172.21.15.82:0/835621132 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65341a2350 con 0x7f6534101fe0 2024-04-21T08:44:31.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.486+0000 7f653c21e640 1 -- 172.21.15.82:0/835621132 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65341a28a0 con 0x7f6534101fe0 2024-04-21T08:44:31.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.487+0000 7f653c21e640 1 -- 172.21.15.82:0/835621132 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65341a2bc0 con 0x7f6534101fe0 2024-04-21T08:44:31.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.488+0000 7f652affd640 1 -- 172.21.15.82:0/835621132 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6530004000 con 0x7f6534101fe0 2024-04-21T08:44:31.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.489+0000 7f652affd640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f65200a3c30 0x7f65200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:31.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.489+0000 7f6539fba640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f65200a3c30 0x7f65200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:31.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.489+0000 7f652affd640 1 -- 172.21.15.82:0/835621132 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f65300d05c0 con 0x7f6534101fe0 2024-04-21T08:44:31.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.489+0000 7f6539fba640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f65200a3c30 0x7f65200a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6524009230 tx=0x7f6524005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:31.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.493+0000 7f652affd640 1 -- 172.21.15.82:0/835621132 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f653009d440 con 0x7f6534101fe0 2024-04-21T08:44:31.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.673+0000 7f653c21e640 1 -- 172.21.15.82:0/835621132 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65341a2d90 con 0x7f6534101fe0 2024-04-21T08:44:31.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.674+0000 7f652affd640 1 -- 172.21.15.82:0/835621132 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f653009cd40 con 0x7f6534101fe0 2024-04-21T08:44:31.674 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:44:31.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.676+0000 7f653c21e640 1 -- 172.21.15.82:0/835621132 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f65200a3c30 msgr2=0x7f65200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.676+0000 7f653c21e640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f65200a3c30 0x7f65200a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6524009230 tx=0x7f6524005fd0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.676+0000 7f653c21e640 1 -- 172.21.15.82:0/835621132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6534101fe0 msgr2=0x7f6534103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.676+0000 7f653c21e640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6534101fe0 0x7f6534103450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f653000e9c0 tx=0x7f653000ee80 comp rx=0 tx=0).stop 2024-04-21T08:44:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.676+0000 7f653c21e640 1 -- 172.21.15.82:0/835621132 shutdown_connections 2024-04-21T08:44:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.676+0000 7f653c21e640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f65200a3c30 0x7f65200a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.676+0000 7f653c21e640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65341029a0 0x7f6534103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.676+0000 7f653c21e640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6534101fe0 0x7f6534103450 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.676+0000 7f653c21e640 1 --2- 172.21.15.82:0/835621132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6534100e90 0x7f6534104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.676+0000 7f653c21e640 1 -- 172.21.15.82:0/835621132 >> 172.21.15.82:0/835621132 conn(0x7f65340fc400 msgr2=0x7f6534069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.676+0000 7f653c21e640 1 -- 172.21.15.82:0/835621132 shutdown_connections 2024-04-21T08:44:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.676+0000 7f653c21e640 1 -- 172.21.15.82:0/835621132 wait complete. 2024-04-21T08:44:31.686 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:31.686 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.686 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1438382120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:31.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:31.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:44:31.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:44:31.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.687 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:31 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2948157837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:31.688 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-21T08:44:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1438382120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:44:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:44:31.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:31 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2948157837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.797+0000 7f3675932640 1 -- 172.21.15.82:0/1532680496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3670101820 msgr2=0x7f3670101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.797+0000 7f3675932640 1 --2- 172.21.15.82:0/1532680496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3670101820 0x7f3670101ca0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f366400aa10 tx=0x7f366402cf70 comp rx=0 tx=0).stop 2024-04-21T08:44:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.797+0000 7f3675932640 1 -- 172.21.15.82:0/1532680496 shutdown_connections 2024-04-21T08:44:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.797+0000 7f3675932640 1 --2- 172.21.15.82:0/1532680496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36701021e0 0x7f3670108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.797+0000 7f3675932640 1 --2- 172.21.15.82:0/1532680496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3670101820 0x7f3670101ca0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.797+0000 7f3675932640 1 --2- 172.21.15.82:0/1532680496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3670100630 0x7f3670100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.797+0000 7f3675932640 1 -- 172.21.15.82:0/1532680496 >> 172.21.15.82:0/1532680496 conn(0x7f36700fbbc0 msgr2=0x7f36700fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.797+0000 7f3675932640 1 -- 172.21.15.82:0/1532680496 shutdown_connections 2024-04-21T08:44:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.797+0000 7f3675932640 1 -- 172.21.15.82:0/1532680496 wait complete. 2024-04-21T08:44:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f3675932640 1 Processor -- start 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f3675932640 1 -- start start 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f3675932640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3670100630 0x7f3670198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f3675932640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3670101820 0x7f3670198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f3675932640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36701021e0 0x7f367019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f3675932640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f367010b8f0 con 0x7f3670100630 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f3675932640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f367010b770 con 0x7f3670101820 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f3675932640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f367010ba70 con 0x7f36701021e0 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f366effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3670100630 0x7f3670198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f366effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3670100630 0x7f3670198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44206/0 (socket says 172.21.15.82:44206) 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f366e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3670101820 0x7f3670198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f366effd640 1 -- 172.21.15.82:0/1780250355 learned_addr learned my addr 172.21.15.82:0/1780250355 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f366f7fe640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36701021e0 0x7f367019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f366effd640 1 -- 172.21.15.82:0/1780250355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36701021e0 msgr2=0x7f367019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f366effd640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36701021e0 0x7f367019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f366effd640 1 -- 172.21.15.82:0/1780250355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3670101820 msgr2=0x7f3670198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:31.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f366effd640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3670101820 0x7f3670198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:31.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f366effd640 1 -- 172.21.15.82:0/1780250355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367019fe00 con 0x7f3670100630 2024-04-21T08:44:31.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f366e7fc640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3670101820 0x7f3670198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:31.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.798+0000 7f366f7fe640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36701021e0 0x7f367019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:31.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.799+0000 7f366effd640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3670100630 0x7f3670198230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f366000e9c0 tx=0x7f366000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:31.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.799+0000 7f3674930640 1 -- 172.21.15.82:0/1780250355 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366000cd80 con 0x7f3670100630 2024-04-21T08:44:31.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.799+0000 7f3675932640 1 -- 172.21.15.82:0/1780250355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36701a00f0 con 0x7f3670100630 2024-04-21T08:44:31.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.799+0000 7f3674930640 1 -- 172.21.15.82:0/1780250355 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36600035f0 con 0x7f3670100630 2024-04-21T08:44:31.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.799+0000 7f3674930640 1 -- 172.21.15.82:0/1780250355 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660010690 con 0x7f3670100630 2024-04-21T08:44:31.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.799+0000 7f3675932640 1 -- 172.21.15.82:0/1780250355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36701a0660 con 0x7f3670100630 2024-04-21T08:44:31.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.800+0000 7f3675932640 1 -- 172.21.15.82:0/1780250355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36701a0920 con 0x7f3670100630 2024-04-21T08:44:31.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.801+0000 7f3674930640 1 -- 172.21.15.82:0/1780250355 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f36600037a0 con 0x7f3670100630 2024-04-21T08:44:31.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.802+0000 7f3674930640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f36580a3bb0 0x7f36580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:31.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.802+0000 7f3674930640 1 -- 172.21.15.82:0/1780250355 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f36600d02c0 con 0x7f3670100630 2024-04-21T08:44:31.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.802+0000 7f366e7fc640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f36580a3bb0 0x7f36580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:31.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.803+0000 7f366e7fc640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f36580a3bb0 0x7f36580a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3664004c20 tx=0x7f3664005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:31.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:31.806+0000 7f3674930640 1 -- 172.21.15.82:0/1780250355 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f366009d140 con 0x7f3670100630 2024-04-21T08:44:31.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:31.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1438382120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:31.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:31.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:44:31.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:44:31.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:31.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:31 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2948157837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:32.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.014+0000 7f3675932640 1 -- 172.21.15.82:0/1780250355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f36701a0af0 con 0x7f3670100630 2024-04-21T08:44:32.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.014+0000 7f3674930640 1 -- 172.21.15.82:0/1780250355 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f366009d140 con 0x7f3670100630 2024-04-21T08:44:32.014 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:32.014 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-21T08:44:32.014 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:44:32.014 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-04-21T08:44:32.014 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-04-21T08:44:32.014 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:44:32.014 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-21T08:44:32.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.016+0000 7f3675932640 1 -- 172.21.15.82:0/1780250355 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f36580a3bb0 msgr2=0x7f36580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:32.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.016+0000 7f3675932640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f36580a3bb0 0x7f36580a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3664004c20 tx=0x7f3664005fd0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.016+0000 7f3675932640 1 -- 172.21.15.82:0/1780250355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3670100630 msgr2=0x7f3670198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:32.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.016+0000 7f3675932640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3670100630 0x7f3670198230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f366000e9c0 tx=0x7f366000ee80 comp rx=0 tx=0).stop 2024-04-21T08:44:32.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.016+0000 7f3675932640 1 -- 172.21.15.82:0/1780250355 shutdown_connections 2024-04-21T08:44:32.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.016+0000 7f3675932640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f36580a3bb0 0x7f36580a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.016+0000 7f3675932640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36701021e0 0x7f367019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.016+0000 7f3675932640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3670101820 0x7f3670198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.016+0000 7f3675932640 1 --2- 172.21.15.82:0/1780250355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3670100630 0x7f3670198230 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.016+0000 7f3675932640 1 -- 172.21.15.82:0/1780250355 >> 172.21.15.82:0/1780250355 conn(0x7f36700fbbc0 msgr2=0x7f3670106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:32.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.016+0000 7f3675932640 1 -- 172.21.15.82:0/1780250355 shutdown_connections 2024-04-21T08:44:32.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.016+0000 7f3675932640 1 -- 172.21.15.82:0/1780250355 wait complete. 2024-04-21T08:44:32.026 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-21T08:44:32.026 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-21T08:44:32.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.131+0000 7f94ba0d2640 1 -- 172.21.15.82:0/3999422980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94b4100e90 msgr2=0x7f94b41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:32.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.131+0000 7f94ba0d2640 1 --2- 172.21.15.82:0/3999422980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94b4100e90 0x7f94b41012b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009900 tx=0x7f94a802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:32.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 -- 172.21.15.82:0/3999422980 shutdown_connections 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 --2- 172.21.15.82:0/3999422980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94b41029a0 0x7f94b41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 --2- 172.21.15.82:0/3999422980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b4101fe0 0x7f94b4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 --2- 172.21.15.82:0/3999422980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94b4100e90 0x7f94b41012b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 -- 172.21.15.82:0/3999422980 >> 172.21.15.82:0/3999422980 conn(0x7f94b40fc400 msgr2=0x7f94b40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 -- 172.21.15.82:0/3999422980 shutdown_connections 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 -- 172.21.15.82:0/3999422980 wait complete. 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 Processor -- start 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 -- start start 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94b4100e90 0x7f94b4198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94b4101fe0 0x7f94b4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b41029a0 0x7f94b419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f94b410c9e0 con 0x7f94b4101fe0 2024-04-21T08:44:32.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f94b410c860 con 0x7f94b4100e90 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.132+0000 7f94ba0d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94b410cb60 con 0x7f94b41029a0 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94b4101fe0 0x7f94b4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94b4101fe0 0x7f94b4198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44226/0 (socket says 172.21.15.82:44226) 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b2ffd640 1 -- 172.21.15.82:0/2282861228 learned_addr learned my addr 172.21.15.82:0/2282861228 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b37fe640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94b4100e90 0x7f94b4198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b3fff640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b41029a0 0x7f94b419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b37fe640 1 -- 172.21.15.82:0/2282861228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b41029a0 msgr2=0x7f94b419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b37fe640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b41029a0 0x7f94b419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b37fe640 1 -- 172.21.15.82:0/2282861228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94b4101fe0 msgr2=0x7f94b4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b37fe640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94b4101fe0 0x7f94b4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b37fe640 1 -- 172.21.15.82:0/2282861228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b419fe20 con 0x7f94b4100e90 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b2ffd640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94b4101fe0 0x7f94b4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b3fff640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b41029a0 0x7f94b419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b37fe640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94b4100e90 0x7f94b4198200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f94a8000c00 tx=0x7f94a8005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b0ff9640 1 -- 172.21.15.82:0/2282861228 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8043020 con 0x7f94b4100e90 2024-04-21T08:44:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94ba0d2640 1 -- 172.21.15.82:0/2282861228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b41a00b0 con 0x7f94b4100e90 2024-04-21T08:44:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b0ff9640 1 -- 172.21.15.82:0/2282861228 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a8004350 con 0x7f94b4100e90 2024-04-21T08:44:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94b0ff9640 1 -- 172.21.15.82:0/2282861228 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8003b50 con 0x7f94b4100e90 2024-04-21T08:44:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.133+0000 7f94ba0d2640 1 -- 172.21.15.82:0/2282861228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b41a0600 con 0x7f94b4100e90 2024-04-21T08:44:32.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.134+0000 7f94ba0d2640 1 -- 172.21.15.82:0/2282861228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b41a0800 con 0x7f94b4100e90 2024-04-21T08:44:32.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.135+0000 7f94b0ff9640 1 -- 172.21.15.82:0/2282861228 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f94a8046020 con 0x7f94b4100e90 2024-04-21T08:44:32.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.136+0000 7f94b0ff9640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f949c0a3bb0 0x7f949c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:32.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.136+0000 7f94b2ffd640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f949c0a3bb0 0x7f949c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:32.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.136+0000 7f94b0ff9640 1 -- 172.21.15.82:0/2282861228 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f94a80f2580 con 0x7f94b4100e90 2024-04-21T08:44:32.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.136+0000 7f94b2ffd640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f949c0a3bb0 0x7f949c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f94b4199710 tx=0x7f94a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:32.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.140+0000 7f94b0ff9640 1 -- 172.21.15.82:0/2282861228 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94a804d020 con 0x7f94b4100e90 2024-04-21T08:44:32.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.326+0000 7f94ba0d2640 1 -- 172.21.15.82:0/2282861228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f94b41a09d0 con 0x7f94b4100e90 2024-04-21T08:44:32.464 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:32 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/835621132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:44:32.464 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:32 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1780250355' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T08:44:32.464 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:32 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2282861228' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T08:44:32.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.467+0000 7f94b0ff9640 1 -- 172.21.15.82:0/2282861228 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f94a80bf520 con 0x7f94b4100e90 2024-04-21T08:44:32.466 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-04-21T08:44:32.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.471+0000 7f94ba0d2640 1 -- 172.21.15.82:0/2282861228 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f949c0a3bb0 msgr2=0x7f949c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:32.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.471+0000 7f94ba0d2640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f949c0a3bb0 0x7f949c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f94b4199710 tx=0x7f94a4005f90 comp rx=0 tx=0).stop 2024-04-21T08:44:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.471+0000 7f94ba0d2640 1 -- 172.21.15.82:0/2282861228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94b4100e90 msgr2=0x7f94b4198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.471+0000 7f94ba0d2640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94b4100e90 0x7f94b4198200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f94a8000c00 tx=0x7f94a8005dc0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.471+0000 7f94ba0d2640 1 -- 172.21.15.82:0/2282861228 shutdown_connections 2024-04-21T08:44:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.471+0000 7f94ba0d2640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f949c0a3bb0 0x7f949c0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.471+0000 7f94ba0d2640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94b41029a0 0x7f94b419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.471+0000 7f94ba0d2640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94b4101fe0 0x7f94b4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.471+0000 7f94ba0d2640 1 --2- 172.21.15.82:0/2282861228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94b4100e90 0x7f94b4198200 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.471+0000 7f94ba0d2640 1 -- 172.21.15.82:0/2282861228 >> 172.21.15.82:0/2282861228 conn(0x7f94b40fc400 msgr2=0x7f94b4069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.471+0000 7f94ba0d2640 1 -- 172.21.15.82:0/2282861228 shutdown_connections 2024-04-21T08:44:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.471+0000 7f94ba0d2640 1 -- 172.21.15.82:0/2282861228 wait complete. 2024-04-21T08:44:32.481 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-21T08:44:32.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.597+0000 7f0ff994a640 1 -- 172.21.15.82:0/1458173187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff4103190 msgr2=0x7f0ff4103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:32.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.597+0000 7f0ff994a640 1 --2- 172.21.15.82:0/1458173187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff4103190 0x7f0ff4103610 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009900 tx=0x7f0fe802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:32.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.597+0000 7f0ff994a640 1 -- 172.21.15.82:0/1458173187 shutdown_connections 2024-04-21T08:44:32.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.597+0000 7f0ff994a640 1 --2- 172.21.15.82:0/1458173187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ff4103b50 0x7f0ff410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.597+0000 7f0ff994a640 1 --2- 172.21.15.82:0/1458173187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff4103190 0x7f0ff4103610 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.597+0000 7f0ff994a640 1 --2- 172.21.15.82:0/1458173187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ff4101fa0 0x7f0ff41023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.597+0000 7f0ff994a640 1 -- 172.21.15.82:0/1458173187 >> 172.21.15.82:0/1458173187 conn(0x7f0ff40fd530 msgr2=0x7f0ff40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:32.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.597+0000 7f0ff994a640 1 -- 172.21.15.82:0/1458173187 shutdown_connections 2024-04-21T08:44:32.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.597+0000 7f0ff994a640 1 -- 172.21.15.82:0/1458173187 wait complete. 2024-04-21T08:44:32.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.598+0000 7f0ff994a640 1 Processor -- start 2024-04-21T08:44:32.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.598+0000 7f0ff994a640 1 -- start start 2024-04-21T08:44:32.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff994a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff4101fa0 0x7f0ff41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:32.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff994a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ff4103190 0x7f0ff41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:32.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff994a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ff4103b50 0x7f0ff419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:32.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff994a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff410dae0 con 0x7f0ff4101fa0 2024-04-21T08:44:32.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff994a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff410d960 con 0x7f0ff4103b50 2024-04-21T08:44:32.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff994a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff410dc60 con 0x7f0ff4103190 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff4101fa0 0x7f0ff41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff4101fa0 0x7f0ff41981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44242/0 (socket says 172.21.15.82:44242) 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff2ffd640 1 -- 172.21.15.82:0/384004554 learned_addr learned my addr 172.21.15.82:0/384004554 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff27fc640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ff4103190 0x7f0ff41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff37fe640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ff4103b50 0x7f0ff419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff2ffd640 1 -- 172.21.15.82:0/384004554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ff4103190 msgr2=0x7f0ff41986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff2ffd640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ff4103190 0x7f0ff41986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff2ffd640 1 -- 172.21.15.82:0/384004554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ff4103b50 msgr2=0x7f0ff419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff2ffd640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ff4103b50 0x7f0ff419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff2ffd640 1 -- 172.21.15.82:0/384004554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff419fdd0 con 0x7f0ff4101fa0 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff2ffd640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff4101fa0 0x7f0ff41981b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400e9c0 tx=0x7f0fe400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff8948640 1 -- 172.21.15.82:0/384004554 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400cd80 con 0x7f0ff4101fa0 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff994a640 1 -- 172.21.15.82:0/384004554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff41a00c0 con 0x7f0ff4101fa0 2024-04-21T08:44:32.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff994a640 1 -- 172.21.15.82:0/384004554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff41a0630 con 0x7f0ff4101fa0 2024-04-21T08:44:32.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff8948640 1 -- 172.21.15.82:0/384004554 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fe40035f0 con 0x7f0ff4101fa0 2024-04-21T08:44:32.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.599+0000 7f0ff8948640 1 -- 172.21.15.82:0/384004554 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4010690 con 0x7f0ff4101fa0 2024-04-21T08:44:32.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.601+0000 7f0ff994a640 1 -- 172.21.15.82:0/384004554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff40684e0 con 0x7f0ff4101fa0 2024-04-21T08:44:32.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.602+0000 7f0ff8948640 1 -- 172.21.15.82:0/384004554 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0fe4003a60 con 0x7f0ff4101fa0 2024-04-21T08:44:32.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.603+0000 7f0ff8948640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0fdc0a3c30 0x7f0fdc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:32.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.603+0000 7f0ff27fc640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0fdc0a3c30 0x7f0fdc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:32.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.603+0000 7f0ff8948640 1 -- 172.21.15.82:0/384004554 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f0fe40d0740 con 0x7f0ff4101fa0 2024-04-21T08:44:32.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.603+0000 7f0ff27fc640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0fdc0a3c30 0x7f0fdc0a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009230 tx=0x7f0fe8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:32.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.607+0000 7f0ff8948640 1 -- 172.21.15.82:0/384004554 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0fe409d4e0 con 0x7f0ff4101fa0 2024-04-21T08:44:32.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:32 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/835621132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:44:32.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:32 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1780250355' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T08:44:32.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:32 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2282861228' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T08:44:32.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:32 smithi082 ceph-mon[27967]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T08:44:32.725 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:32 smithi134 ceph-mon[31772]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T08:44:32.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:32.861+0000 7f0ff994a640 1 -- 172.21.15.82:0/384004554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f0ff41093b0 con 0x7f0ff4101fa0 2024-04-21T08:44:32.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:32 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/835621132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:44:32.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:32 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1780250355' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T08:44:32.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:32 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2282861228' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T08:44:32.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:32 smithi106 ceph-mon[31072]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T08:44:33.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.468+0000 7f0ff8948640 1 -- 172.21.15.82:0/384004554 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7f0fe409cde0 con 0x7f0ff4101fa0 2024-04-21T08:44:33.468 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-04-21T08:44:33.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.471+0000 7f0ff994a640 1 -- 172.21.15.82:0/384004554 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0fdc0a3c30 msgr2=0x7f0fdc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:33.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.471+0000 7f0ff994a640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0fdc0a3c30 0x7f0fdc0a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009230 tx=0x7f0fe8005fd0 comp rx=0 tx=0).stop 2024-04-21T08:44:33.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.471+0000 7f0ff994a640 1 -- 172.21.15.82:0/384004554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff4101fa0 msgr2=0x7f0ff41981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:33.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.471+0000 7f0ff994a640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff4101fa0 0x7f0ff41981b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400e9c0 tx=0x7f0fe400ee80 comp rx=0 tx=0).stop 2024-04-21T08:44:33.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.471+0000 7f0ff994a640 1 -- 172.21.15.82:0/384004554 shutdown_connections 2024-04-21T08:44:33.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.471+0000 7f0ff994a640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0fdc0a3c30 0x7f0fdc0a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:33.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.471+0000 7f0ff994a640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ff4103b50 0x7f0ff419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:33.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.471+0000 7f0ff994a640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ff4103190 0x7f0ff41986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:33.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.471+0000 7f0ff994a640 1 --2- 172.21.15.82:0/384004554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff4101fa0 0x7f0ff41981b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:33.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.471+0000 7f0ff994a640 1 -- 172.21.15.82:0/384004554 >> 172.21.15.82:0/384004554 conn(0x7f0ff40fd530 msgr2=0x7f0ff4106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:33.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.471+0000 7f0ff994a640 1 -- 172.21.15.82:0/384004554 shutdown_connections 2024-04-21T08:44:33.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.471+0000 7f0ff994a640 1 -- 172.21.15.82:0/384004554 wait complete. 2024-04-21T08:44:33.489 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-21T08:44:33.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 -- 172.21.15.82:0/4122266402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec4103190 msgr2=0x7fbec4103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 --2- 172.21.15.82:0/4122266402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec4103190 0x7fbec4103610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009900 tx=0x7fbeac02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 -- 172.21.15.82:0/4122266402 shutdown_connections 2024-04-21T08:44:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 --2- 172.21.15.82:0/4122266402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec4103b50 0x7fbec410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 --2- 172.21.15.82:0/4122266402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec4103190 0x7fbec4103610 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 --2- 172.21.15.82:0/4122266402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec4101fa0 0x7fbec41023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 -- 172.21.15.82:0/4122266402 >> 172.21.15.82:0/4122266402 conn(0x7fbec40fd530 msgr2=0x7fbec40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 -- 172.21.15.82:0/4122266402 shutdown_connections 2024-04-21T08:44:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 -- 172.21.15.82:0/4122266402 wait complete. 2024-04-21T08:44:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 Processor -- start 2024-04-21T08:44:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 -- start start 2024-04-21T08:44:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec4101fa0 0x7fbec4198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec4103190 0x7fbec4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec4103b50 0x7fbec419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbec410dae0 con 0x7fbec4101fa0 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbec410d960 con 0x7fbec4103190 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec9523640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbec410dc60 con 0x7fbec4103b50 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec4101fa0 0x7fbec4198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec4101fa0 0x7fbec4198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49652/0 (socket says 172.21.15.82:49652) 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.592+0000 7fbec2ffd640 1 -- 172.21.15.82:0/3939664146 learned_addr learned my addr 172.21.15.82:0/3939664146 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbec37fe640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec4103b50 0x7fbec419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbebbfff640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec4103190 0x7fbec4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbec2ffd640 1 -- 172.21.15.82:0/3939664146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec4103b50 msgr2=0x7fbec419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbec2ffd640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec4103b50 0x7fbec419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbec2ffd640 1 -- 172.21.15.82:0/3939664146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec4103190 msgr2=0x7fbec4198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbec2ffd640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec4103190 0x7fbec4198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbec2ffd640 1 -- 172.21.15.82:0/3939664146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec419fd60 con 0x7fbec4101fa0 2024-04-21T08:44:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbebbfff640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec4103190 0x7fbec4198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbec2ffd640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec4101fa0 0x7fbec4198140 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400c970 tx=0x7fbeb400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbec0ff9640 1 -- 172.21.15.82:0/3939664146 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4007c10 con 0x7fbec4101fa0 2024-04-21T08:44:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbec0ff9640 1 -- 172.21.15.82:0/3939664146 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb4007db0 con 0x7fbec4101fa0 2024-04-21T08:44:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbec9523640 1 -- 172.21.15.82:0/3939664146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec419fff0 con 0x7fbec4101fa0 2024-04-21T08:44:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbec0ff9640 1 -- 172.21.15.82:0/3939664146 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400f870 con 0x7fbec4101fa0 2024-04-21T08:44:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.593+0000 7fbec9523640 1 -- 172.21.15.82:0/3939664146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec41a0500 con 0x7fbec4101fa0 2024-04-21T08:44:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.594+0000 7fbec9523640 1 -- 172.21.15.82:0/3939664146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec41092c0 con 0x7fbec4101fa0 2024-04-21T08:44:33.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.595+0000 7fbec0ff9640 1 -- 172.21.15.82:0/3939664146 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbeb400fa10 con 0x7fbec4101fa0 2024-04-21T08:44:33.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.596+0000 7fbec0ff9640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbeb00a3c30 0x7fbeb00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:33.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.596+0000 7fbebbfff640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbeb00a3c30 0x7fbeb00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:33.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.596+0000 7fbec0ff9640 1 -- 172.21.15.82:0/3939664146 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7fbeb40d0870 con 0x7fbec4101fa0 2024-04-21T08:44:33.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.596+0000 7fbebbfff640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbeb00a3c30 0x7fbeb00a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbec4199650 tx=0x7fbeac02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:33.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.599+0000 7fbec0ff9640 1 -- 172.21.15.82:0/3939664146 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbeb409d520 con 0x7fbec4101fa0 2024-04-21T08:44:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T08:44:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T08:44:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/384004554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-21T08:44:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:33.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:44:33.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:33.817+0000 7fbec9523640 1 -- 172.21.15.82:0/3939664146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbec402d5c0 con 0x7fbec4101fa0 2024-04-21T08:44:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T08:44:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T08:44:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/384004554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-21T08:44:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:44:33.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:33.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T08:44:33.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T08:44:33.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/384004554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-21T08:44:33.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:33.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:33.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:44:33.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:44:34.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.480+0000 7fbec0ff9640 1 -- 172.21.15.82:0/3939664146 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fbeb409ce20 con 0x7fbec4101fa0 2024-04-21T08:44:34.479 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 2 and data pool 3 2024-04-21T08:44:34.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.483+0000 7fbec9523640 1 -- 172.21.15.82:0/3939664146 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbeb00a3c30 msgr2=0x7fbeb00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:34.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.483+0000 7fbec9523640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbeb00a3c30 0x7fbeb00a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbec4199650 tx=0x7fbeac02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:44:34.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.483+0000 7fbec9523640 1 -- 172.21.15.82:0/3939664146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec4101fa0 msgr2=0x7fbec4198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:34.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.483+0000 7fbec9523640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec4101fa0 0x7fbec4198140 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400c970 tx=0x7fbeb400ce30 comp rx=0 tx=0).stop 2024-04-21T08:44:34.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.484+0000 7fbec9523640 1 -- 172.21.15.82:0/3939664146 shutdown_connections 2024-04-21T08:44:34.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.484+0000 7fbec9523640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbeb00a3c30 0x7fbeb00a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:34.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.484+0000 7fbec9523640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbec4103b50 0x7fbec419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:34.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.484+0000 7fbec9523640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec4103190 0x7fbec4198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:34.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.484+0000 7fbec9523640 1 --2- 172.21.15.82:0/3939664146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec4101fa0 0x7fbec4198140 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:34.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.484+0000 7fbec9523640 1 -- 172.21.15.82:0/3939664146 >> 172.21.15.82:0/3939664146 conn(0x7fbec40fd530 msgr2=0x7fbec4106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:34.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.484+0000 7fbec9523640 1 -- 172.21.15.82:0/3939664146 shutdown_connections 2024-04-21T08:44:34.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.484+0000 7fbec9523640 1 -- 172.21.15.82:0/3939664146 wait complete. 2024-04-21T08:44:34.505 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-21T08:44:34.505 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-21T08:44:34.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.610+0000 7f932bb2b640 1 -- 172.21.15.82:0/3805831953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9324100880 msgr2=0x7f9324100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:34.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.610+0000 7f932bb2b640 1 --2- 172.21.15.82:0/3805831953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9324100880 0x7f9324100ca0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9320009ca0 tx=0x7f932002c180 comp rx=0 tx=0).stop 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.610+0000 7f932bb2b640 1 -- 172.21.15.82:0/3805831953 shutdown_connections 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.610+0000 7f932bb2b640 1 --2- 172.21.15.82:0/3805831953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9324101b80 0x7f9324102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.610+0000 7f932bb2b640 1 --2- 172.21.15.82:0/3805831953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93241011e0 0x7f9324101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.610+0000 7f932bb2b640 1 --2- 172.21.15.82:0/3805831953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9324100880 0x7f9324100ca0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.610+0000 7f932bb2b640 1 -- 172.21.15.82:0/3805831953 >> 172.21.15.82:0/3805831953 conn(0x7f93240fc400 msgr2=0x7f93240fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.610+0000 7f932bb2b640 1 -- 172.21.15.82:0/3805831953 shutdown_connections 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f932bb2b640 1 -- 172.21.15.82:0/3805831953 wait complete. 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f932bb2b640 1 Processor -- start 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f932bb2b640 1 -- start start 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f932bb2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9324100880 0x7f932419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f932bb2b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93241011e0 0x7f932419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f932bb2b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9324101b80 0x7f93241a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f932bb2b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f932410fcd0 con 0x7f9324100880 2024-04-21T08:44:34.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f932bb2b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f932410fb50 con 0x7f93241011e0 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f932bb2b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f932410fe50 con 0x7f9324101b80 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f93298c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9324100880 0x7f932419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f93298c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9324100880 0x7f932419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49656/0 (socket says 172.21.15.82:49656) 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f93298c7640 1 -- 172.21.15.82:0/3865259232 learned_addr learned my addr 172.21.15.82:0/3865259232 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.611+0000 7f93290c6640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93241011e0 0x7f932419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.612+0000 7f93298c7640 1 -- 172.21.15.82:0/3865259232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9324101b80 msgr2=0x7f93241a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.612+0000 7f93298c7640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9324101b80 0x7f93241a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.612+0000 7f93298c7640 1 -- 172.21.15.82:0/3865259232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93241011e0 msgr2=0x7f932419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.612+0000 7f93298c7640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93241011e0 0x7f932419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.612+0000 7f93298c7640 1 -- 172.21.15.82:0/3865259232 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93241a2090 con 0x7f9324100880 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.612+0000 7f93290c6640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93241011e0 0x7f932419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.612+0000 7f93298c7640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9324100880 0x7f932419a470 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9320005cc0 tx=0x7f9320003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.612+0000 7f931affd640 1 -- 172.21.15.82:0/3865259232 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320039040 con 0x7f9324100880 2024-04-21T08:44:34.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.612+0000 7f931affd640 1 -- 172.21.15.82:0/3865259232 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9320035e20 con 0x7f9324100880 2024-04-21T08:44:34.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.612+0000 7f932bb2b640 1 -- 172.21.15.82:0/3865259232 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93241a2320 con 0x7f9324100880 2024-04-21T08:44:34.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.612+0000 7f931affd640 1 -- 172.21.15.82:0/3865259232 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320032b10 con 0x7f9324100880 2024-04-21T08:44:34.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.612+0000 7f932bb2b640 1 -- 172.21.15.82:0/3865259232 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93241a2810 con 0x7f9324100880 2024-04-21T08:44:34.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.613+0000 7f932bb2b640 1 -- 172.21.15.82:0/3865259232 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932410b8c0 con 0x7f9324100880 2024-04-21T08:44:34.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.614+0000 7f931affd640 1 -- 172.21.15.82:0/3865259232 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9320046030 con 0x7f9324100880 2024-04-21T08:44:34.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.615+0000 7f931affd640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f93100a3bb0 0x7f93100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:34.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.615+0000 7f93290c6640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f93100a3bb0 0x7f93100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:34.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.615+0000 7f931affd640 1 -- 172.21.15.82:0/3865259232 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f93200f2550 con 0x7f9324100880 2024-04-21T08:44:34.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.615+0000 7f93290c6640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f93100a3bb0 0x7f93100a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f932419b980 tx=0x7f9314005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:34.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.618+0000 7f931affd640 1 -- 172.21.15.82:0/3865259232 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f932004d060 con 0x7f9324100880 2024-04-21T08:44:34.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:34 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/384004554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-21T08:44:34.743 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:34 smithi082 ceph-mon[27967]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T08:44:34.743 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:34 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3939664146' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T08:44:34.743 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:34 smithi082 ceph-mon[27967]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-21T08:44:34.743 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:34 smithi082 ceph-mon[27967]: pgmap v287: 97 pgs: 96 unknown, 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:34.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:34.826+0000 7f932bb2b640 1 -- 172.21.15.82:0/3865259232 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f932402d5c0 con 0x7f9324100880 2024-04-21T08:44:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:34 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/384004554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-21T08:44:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:34 smithi106 ceph-mon[31072]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T08:44:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:34 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3939664146' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T08:44:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:34 smithi106 ceph-mon[31072]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-21T08:44:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:34 smithi106 ceph-mon[31072]: pgmap v287: 97 pgs: 96 unknown, 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:34.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:34 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/384004554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-21T08:44:34.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:34 smithi134 ceph-mon[31772]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T08:44:34.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:34 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3939664146' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T08:44:34.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:34 smithi134 ceph-mon[31772]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-21T08:44:34.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:34 smithi134 ceph-mon[31772]: pgmap v287: 97 pgs: 96 unknown, 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:35.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.487+0000 7f931affd640 1 -- 172.21.15.82:0/3865259232 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f93200333a0 con 0x7f9324100880 2024-04-21T08:44:35.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.490+0000 7f932bb2b640 1 -- 172.21.15.82:0/3865259232 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f93100a3bb0 msgr2=0x7f93100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.490+0000 7f932bb2b640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f93100a3bb0 0x7f93100a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f932419b980 tx=0x7f9314005e50 comp rx=0 tx=0).stop 2024-04-21T08:44:35.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.490+0000 7f932bb2b640 1 -- 172.21.15.82:0/3865259232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9324100880 msgr2=0x7f932419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.490+0000 7f932bb2b640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9324100880 0x7f932419a470 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9320005cc0 tx=0x7f9320003c20 comp rx=0 tx=0).stop 2024-04-21T08:44:35.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.490+0000 7f932bb2b640 1 -- 172.21.15.82:0/3865259232 shutdown_connections 2024-04-21T08:44:35.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.490+0000 7f932bb2b640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f93100a3bb0 0x7f93100a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.490+0000 7f932bb2b640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9324101b80 0x7f93241a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.490+0000 7f932bb2b640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93241011e0 0x7f932419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.490+0000 7f932bb2b640 1 --2- 172.21.15.82:0/3865259232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9324100880 0x7f932419a470 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.490+0000 7f932bb2b640 1 -- 172.21.15.82:0/3865259232 >> 172.21.15.82:0/3865259232 conn(0x7f93240fc400 msgr2=0x7f93240fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:35.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.491+0000 7f932bb2b640 1 -- 172.21.15.82:0/3865259232 shutdown_connections 2024-04-21T08:44:35.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.491+0000 7f932bb2b640 1 -- 172.21.15.82:0/3865259232 wait complete. 2024-04-21T08:44:35.504 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-21T08:44:35.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.616+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2591453075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dc0103c00 msgr2=0x7f0dc010a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.616+0000 7f0dc6c77640 1 --2- 172.21.15.82:0/2591453075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dc0103c00 0x7f0dc010a490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009cd0 tx=0x7f0db802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.616+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2591453075 shutdown_connections 2024-04-21T08:44:35.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.616+0000 7f0dc6c77640 1 --2- 172.21.15.82:0/2591453075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dc0103c00 0x7f0dc010a490 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.616+0000 7f0dc6c77640 1 --2- 172.21.15.82:0/2591453075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc0103240 0x7f0dc01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.616+0000 7f0dc6c77640 1 --2- 172.21.15.82:0/2591453075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dc0100270 0x7f0dc0100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.616+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2591453075 >> 172.21.15.82:0/2591453075 conn(0x7f0dc00fbb80 msgr2=0x7f0dc00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:35.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.616+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2591453075 shutdown_connections 2024-04-21T08:44:35.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.616+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2591453075 wait complete. 2024-04-21T08:44:35.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc6c77640 1 Processor -- start 2024-04-21T08:44:35.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc6c77640 1 -- start start 2024-04-21T08:44:35.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc6c77640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dc0100270 0x7f0dc01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc6c77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc0103240 0x7f0dc0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc6c77640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dc0103c00 0x7f0dc019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc6c77640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc010db10 con 0x7f0dc0103c00 2024-04-21T08:44:35.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc6c77640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc010d990 con 0x7f0dc0100270 2024-04-21T08:44:35.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc6c77640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc010dc90 con 0x7f0dc0103240 2024-04-21T08:44:35.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc5214640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dc0103c00 0x7f0dc019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc5214640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dc0103c00 0x7f0dc019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49674/0 (socket says 172.21.15.82:49674) 2024-04-21T08:44:35.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc5214640 1 -- 172.21.15.82:0/2701135257 learned_addr learned my addr 172.21.15.82:0/2701135257 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dbffff640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc0103240 0x7f0dc0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc4a13640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dc0100270 0x7f0dc01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc5214640 1 -- 172.21.15.82:0/2701135257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc0103240 msgr2=0x7f0dc0198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc5214640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc0103240 0x7f0dc0198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc5214640 1 -- 172.21.15.82:0/2701135257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dc0100270 msgr2=0x7f0dc01980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc5214640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dc0100270 0x7f0dc01980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc5214640 1 -- 172.21.15.82:0/2701135257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc019fd00 con 0x7f0dc0103c00 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc5214640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dc0103c00 0x7f0dc019f690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0db8005e30 tx=0x7f0db8034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dbdffb640 1 -- 172.21.15.82:0/2701135257 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8041020 con 0x7f0dc0103c00 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dbdffb640 1 -- 172.21.15.82:0/2701135257 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db8032840 con 0x7f0dc0103c00 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2701135257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc019ff90 con 0x7f0dc0103c00 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.617+0000 7f0dbdffb640 1 -- 172.21.15.82:0/2701135257 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8039040 con 0x7f0dc0103c00 2024-04-21T08:44:35.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.618+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2701135257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc01a03f0 con 0x7f0dc0103c00 2024-04-21T08:44:35.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.619+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2701135257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc0109370 con 0x7f0dc0103c00 2024-04-21T08:44:35.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.620+0000 7f0dbdffb640 1 -- 172.21.15.82:0/2701135257 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0db8044020 con 0x7f0dc0103c00 2024-04-21T08:44:35.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.621+0000 7f0dbdffb640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0db00a3bb0 0x7f0db00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.621+0000 7f0dc4a13640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0db00a3bb0 0x7f0db00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.621+0000 7f0dbdffb640 1 -- 172.21.15.82:0/2701135257 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f0db80f2960 con 0x7f0dc0103c00 2024-04-21T08:44:35.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.621+0000 7f0dc4a13640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0db00a3bb0 0x7f0db00a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0db4005fd0 tx=0x7f0db4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:35.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.625+0000 7f0dbdffb640 1 -- 172.21.15.82:0/2701135257 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0db804b020 con 0x7f0dc0103c00 2024-04-21T08:44:35.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:35.807+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2701135257 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0dc002d5c0 con 0x7f0dc0103c00 2024-04-21T08:44:35.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:35 smithi082 ceph-mon[27967]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:44:35.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:35 smithi082 ceph-mon[27967]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T08:44:35.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:35 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3939664146' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T08:44:35.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:35 smithi082 ceph-mon[27967]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-21T08:44:35.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:35 smithi082 ceph-mon[27967]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-21T08:44:35.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:35 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3865259232' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-21T08:44:35.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:35 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] up:active 2024-04-21T08:44:35.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:35 smithi082 ceph-mon[27967]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:44:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:35 smithi106 ceph-mon[31072]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:44:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:35 smithi106 ceph-mon[31072]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T08:44:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:35 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3939664146' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T08:44:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:35 smithi106 ceph-mon[31072]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-21T08:44:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:35 smithi106 ceph-mon[31072]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-21T08:44:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:35 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3865259232' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-21T08:44:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:35 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] up:active 2024-04-21T08:44:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:35 smithi106 ceph-mon[31072]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:44:35.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:35 smithi134 ceph-mon[31772]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:44:35.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:35 smithi134 ceph-mon[31772]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T08:44:35.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:35 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3939664146' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T08:44:35.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:35 smithi134 ceph-mon[31772]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-21T08:44:35.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:35 smithi134 ceph-mon[31772]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-21T08:44:35.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:35 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3865259232' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-21T08:44:35.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:35 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] up:active 2024-04-21T08:44:35.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:35 smithi134 ceph-mon[31772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:44:36.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:36 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3865259232' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-21T08:44:36.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:36 smithi082 ceph-mon[27967]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T08:44:36.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:36 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2701135257' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T08:44:36.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:36 smithi082 ceph-mon[27967]: pgmap v290: 97 pgs: 2 creating+peering, 53 active+clean, 42 unknown; 578 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-04-21T08:44:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:36 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3865259232' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-21T08:44:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:36 smithi106 ceph-mon[31072]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T08:44:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:36 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2701135257' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T08:44:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:36 smithi106 ceph-mon[31072]: pgmap v290: 97 pgs: 2 creating+peering, 53 active+clean, 42 unknown; 578 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-04-21T08:44:36.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:36 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3865259232' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-21T08:44:36.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:36 smithi134 ceph-mon[31772]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T08:44:36.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:36 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2701135257' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T08:44:36.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:36 smithi134 ceph-mon[31772]: pgmap v290: 97 pgs: 2 creating+peering, 53 active+clean, 42 unknown; 578 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-04-21T08:44:37.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.495+0000 7f0dbdffb640 1 -- 172.21.15.82:0/2701135257 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f0db80bf310 con 0x7f0dc0103c00 2024-04-21T08:44:37.495 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data_ec' created 2024-04-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.499+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2701135257 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0db00a3bb0 msgr2=0x7f0db00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.499+0000 7f0dc6c77640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0db00a3bb0 0x7f0db00a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0db4005fd0 tx=0x7f0db4005850 comp rx=0 tx=0).stop 2024-04-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.499+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2701135257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dc0103c00 msgr2=0x7f0dc019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.499+0000 7f0dc6c77640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dc0103c00 0x7f0dc019f690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0db8005e30 tx=0x7f0db8034760 comp rx=0 tx=0).stop 2024-04-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.499+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2701135257 shutdown_connections 2024-04-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.499+0000 7f0dc6c77640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0db00a3bb0 0x7f0db00a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.499+0000 7f0dc6c77640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dc0103c00 0x7f0dc019f690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.499+0000 7f0dc6c77640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc0103240 0x7f0dc0198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.499+0000 7f0dc6c77640 1 --2- 172.21.15.82:0/2701135257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dc0100270 0x7f0dc01980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.499+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2701135257 >> 172.21.15.82:0/2701135257 conn(0x7f0dc00fbb80 msgr2=0x7f0dc0106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.499+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2701135257 shutdown_connections 2024-04-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.499+0000 7f0dc6c77640 1 -- 172.21.15.82:0/2701135257 wait complete. 2024-04-21T08:44:37.518 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-21T08:44:37.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.625+0000 7f5b69304640 1 -- 172.21.15.82:0/911863112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b64103b50 msgr2=0x7f5b6410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:37.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.625+0000 7f5b69304640 1 --2- 172.21.15.82:0/911863112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b64103b50 0x7f5b6410a3e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009ca0 tx=0x7f5b5c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:37.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.625+0000 7f5b69304640 1 -- 172.21.15.82:0/911863112 shutdown_connections 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.625+0000 7f5b69304640 1 --2- 172.21.15.82:0/911863112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b64103b50 0x7f5b6410a3e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.625+0000 7f5b69304640 1 --2- 172.21.15.82:0/911863112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b64103190 0x7f5b64103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.625+0000 7f5b69304640 1 --2- 172.21.15.82:0/911863112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b64101fa0 0x7f5b641023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.625+0000 7f5b69304640 1 -- 172.21.15.82:0/911863112 >> 172.21.15.82:0/911863112 conn(0x7f5b640fd530 msgr2=0x7f5b640ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.625+0000 7f5b69304640 1 -- 172.21.15.82:0/911863112 shutdown_connections 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.625+0000 7f5b69304640 1 -- 172.21.15.82:0/911863112 wait complete. 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b69304640 1 Processor -- start 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b69304640 1 -- start start 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b69304640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b64101fa0 0x7f5b641981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b69304640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b64103190 0x7f5b64198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b69304640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b64103b50 0x7f5b6419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b69304640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6410db20 con 0x7f5b64103190 2024-04-21T08:44:37.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b69304640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6410d9a0 con 0x7f5b64101fa0 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b69304640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6410dca0 con 0x7f5b64103b50 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b627fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b64103190 0x7f5b64198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b627fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b64103190 0x7f5b64198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49690/0 (socket says 172.21.15.82:49690) 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b627fc640 1 -- 172.21.15.82:0/2210437199 learned_addr learned my addr 172.21.15.82:0/2210437199 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b62ffd640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b64101fa0 0x7f5b641981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b627fc640 1 -- 172.21.15.82:0/2210437199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b64103b50 msgr2=0x7f5b6419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b627fc640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b64103b50 0x7f5b6419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b627fc640 1 -- 172.21.15.82:0/2210437199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b64101fa0 msgr2=0x7f5b641981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b627fc640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b64101fa0 0x7f5b641981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.626+0000 7f5b627fc640 1 -- 172.21.15.82:0/2210437199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6419fe10 con 0x7f5b64103190 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.627+0000 7f5b627fc640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b64103190 0x7f5b64198730 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800b4a0 tx=0x7f5b5800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.627+0000 7f5b4bfff640 1 -- 172.21.15.82:0/2210437199 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58014070 con 0x7f5b64103190 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.627+0000 7f5b4bfff640 1 -- 172.21.15.82:0/2210437199 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b58010ac0 con 0x7f5b64103190 2024-04-21T08:44:37.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.627+0000 7f5b69304640 1 -- 172.21.15.82:0/2210437199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b641a0100 con 0x7f5b64103190 2024-04-21T08:44:37.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.627+0000 7f5b4bfff640 1 -- 172.21.15.82:0/2210437199 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58017c30 con 0x7f5b64103190 2024-04-21T08:44:37.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.627+0000 7f5b69304640 1 -- 172.21.15.82:0/2210437199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b641a0650 con 0x7f5b64103190 2024-04-21T08:44:37.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.628+0000 7f5b69304640 1 -- 172.21.15.82:0/2210437199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b64109310 con 0x7f5b64103190 2024-04-21T08:44:37.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.629+0000 7f5b4bfff640 1 -- 172.21.15.82:0/2210437199 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b58017070 con 0x7f5b64103190 2024-04-21T08:44:37.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.630+0000 7f5b4bfff640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5b4c0a3bb0 0x7f5b4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:37.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.630+0000 7f5b62ffd640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5b4c0a3bb0 0x7f5b4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:37.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.630+0000 7f5b4bfff640 1 -- 172.21.15.82:0/2210437199 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7f5b580cfd50 con 0x7f5b64103190 2024-04-21T08:44:37.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.630+0000 7f5b62ffd640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5b4c0a3bb0 0x7f5b4c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5b64102ff0 tx=0x7f5b5400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:37.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.634+0000 7f5b4bfff640 1 -- 172.21.15.82:0/2210437199 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b5809c510 con 0x7f5b64103190 2024-04-21T08:44:37.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:37.813+0000 7f5b69304640 1 -- 172.21.15.82:0/2210437199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f5b6402d5c0 con 0x7f5b64103190 2024-04-21T08:44:37.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:37 smithi082 ceph-mon[27967]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-21T08:44:37.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:37 smithi082 ceph-mon[27967]: Cluster is now healthy 2024-04-21T08:44:37.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:37 smithi082 ceph-mon[27967]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T08:44:37.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:37 smithi106 ceph-mon[31072]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-21T08:44:37.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:37 smithi106 ceph-mon[31072]: Cluster is now healthy 2024-04-21T08:44:37.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:37 smithi106 ceph-mon[31072]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T08:44:37.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:37 smithi134 ceph-mon[31772]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-21T08:44:37.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:37 smithi134 ceph-mon[31772]: Cluster is now healthy 2024-04-21T08:44:37.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:37 smithi134 ceph-mon[31772]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T08:44:38.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.497+0000 7f5b4bfff640 1 -- 172.21.15.82:0/2210437199 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7f5b5809be10 con 0x7f5b64103190 2024-04-21T08:44:38.497 INFO:teuthology.orchestra.run.smithi082.stderr:set pool 4 allow_ec_overwrites to true 2024-04-21T08:44:38.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.500+0000 7f5b69304640 1 -- 172.21.15.82:0/2210437199 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5b4c0a3bb0 msgr2=0x7f5b4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:38.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.500+0000 7f5b69304640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5b4c0a3bb0 0x7f5b4c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5b64102ff0 tx=0x7f5b5400a2d0 comp rx=0 tx=0).stop 2024-04-21T08:44:38.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.500+0000 7f5b69304640 1 -- 172.21.15.82:0/2210437199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b64103190 msgr2=0x7f5b64198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:38.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.500+0000 7f5b69304640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b64103190 0x7f5b64198730 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800b4a0 tx=0x7f5b5800b960 comp rx=0 tx=0).stop 2024-04-21T08:44:38.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.500+0000 7f5b69304640 1 -- 172.21.15.82:0/2210437199 shutdown_connections 2024-04-21T08:44:38.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.500+0000 7f5b69304640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5b4c0a3bb0 0x7f5b4c0a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:38.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.501+0000 7f5b69304640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b64103b50 0x7f5b6419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:38.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.501+0000 7f5b69304640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b64103190 0x7f5b64198730 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:38.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.501+0000 7f5b69304640 1 --2- 172.21.15.82:0/2210437199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b64101fa0 0x7f5b641981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:38.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.501+0000 7f5b69304640 1 -- 172.21.15.82:0/2210437199 >> 172.21.15.82:0/2210437199 conn(0x7f5b640fd530 msgr2=0x7f5b641064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:38.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.501+0000 7f5b69304640 1 -- 172.21.15.82:0/2210437199 shutdown_connections 2024-04-21T08:44:38.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.501+0000 7f5b69304640 1 -- 172.21.15.82:0/2210437199 wait complete. 2024-04-21T08:44:38.526 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-21T08:44:38.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.631+0000 7f73af359640 1 -- 172.21.15.82:0/3722558478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a8100e90 msgr2=0x7f73a81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:38.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.631+0000 7f73af359640 1 --2- 172.21.15.82:0/3722558478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a8100e90 0x7f73a81012b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009900 tx=0x7f73a402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:38.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.631+0000 7f73af359640 1 -- 172.21.15.82:0/3722558478 shutdown_connections 2024-04-21T08:44:38.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.631+0000 7f73af359640 1 --2- 172.21.15.82:0/3722558478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73a81029a0 0x7f73a81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:38.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.631+0000 7f73af359640 1 --2- 172.21.15.82:0/3722558478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73a8101fe0 0x7f73a8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:38.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.631+0000 7f73af359640 1 --2- 172.21.15.82:0/3722558478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a8100e90 0x7f73a81012b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:38.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.631+0000 7f73af359640 1 -- 172.21.15.82:0/3722558478 >> 172.21.15.82:0/3722558478 conn(0x7f73a80fc400 msgr2=0x7f73a80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:38.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.631+0000 7f73af359640 1 -- 172.21.15.82:0/3722558478 shutdown_connections 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.631+0000 7f73af359640 1 -- 172.21.15.82:0/3722558478 wait complete. 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73af359640 1 Processor -- start 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73af359640 1 -- start start 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73af359640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73a8100e90 0x7f73a81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73af359640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73a8101fe0 0x7f73a8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73af359640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a81029a0 0x7f73a819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73af359640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f73a810daf0 con 0x7f73a81029a0 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73af359640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f73a810d970 con 0x7f73a8101fe0 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73af359640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73a810dc70 con 0x7f73a8100e90 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73ad8f6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a81029a0 0x7f73a819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73ad8f6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a81029a0 0x7f73a819f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49700/0 (socket says 172.21.15.82:49700) 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73ad8f6640 1 -- 172.21.15.82:0/1294909648 learned_addr learned my addr 172.21.15.82:0/1294909648 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:38.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73ad8f6640 1 -- 172.21.15.82:0/1294909648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73a8100e90 msgr2=0x7f73a81981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:38.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73ad8f6640 1 --2- 172.21.15.82:0/1294909648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73a8100e90 0x7f73a81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:38.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73ad8f6640 1 -- 172.21.15.82:0/1294909648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73a8101fe0 msgr2=0x7f73a8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:38.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73ad8f6640 1 --2- 172.21.15.82:0/1294909648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73a8101fe0 0x7f73a8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:38.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.632+0000 7f73ad8f6640 1 -- 172.21.15.82:0/1294909648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a819fdf0 con 0x7f73a81029a0 2024-04-21T08:44:38.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.633+0000 7f73ad8f6640 1 --2- 172.21.15.82:0/1294909648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a81029a0 0x7f73a819f780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73a000cad0 tx=0x7f73a000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:38.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.633+0000 7f739e7fc640 1 -- 172.21.15.82:0/1294909648 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a000ae20 con 0x7f73a81029a0 2024-04-21T08:44:38.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.633+0000 7f73af359640 1 -- 172.21.15.82:0/1294909648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a81a00e0 con 0x7f73a81029a0 2024-04-21T08:44:38.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.633+0000 7f739e7fc640 1 -- 172.21.15.82:0/1294909648 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73a0005020 con 0x7f73a81029a0 2024-04-21T08:44:38.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.633+0000 7f739e7fc640 1 -- 172.21.15.82:0/1294909648 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a000f6b0 con 0x7f73a81029a0 2024-04-21T08:44:38.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.633+0000 7f73af359640 1 -- 172.21.15.82:0/1294909648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a81a05e0 con 0x7f73a81029a0 2024-04-21T08:44:38.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.634+0000 7f73af359640 1 -- 172.21.15.82:0/1294909648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a8108210 con 0x7f73a81029a0 2024-04-21T08:44:38.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.635+0000 7f739e7fc640 1 -- 172.21.15.82:0/1294909648 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f73a00051d0 con 0x7f73a81029a0 2024-04-21T08:44:38.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.635+0000 7f739e7fc640 1 --2- 172.21.15.82:0/1294909648 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f73940a3bb0 0x7f73940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:38.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.635+0000 7f73ad0f5640 1 --2- 172.21.15.82:0/1294909648 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f73940a3bb0 0x7f73940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:38.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.636+0000 7f739e7fc640 1 -- 172.21.15.82:0/1294909648 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f73a00d0b20 con 0x7f73a81029a0 2024-04-21T08:44:38.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.636+0000 7f73ad0f5640 1 --2- 172.21.15.82:0/1294909648 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f73940a3bb0 0x7f73940a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f73a4036040 tx=0x7f73a4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:38.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.639+0000 7f739e7fc640 1 -- 172.21.15.82:0/1294909648 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73a00d0e20 con 0x7f73a81029a0 2024-04-21T08:44:38.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:38.857+0000 7f73af359640 1 -- 172.21.15.82:0/1294909648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f73a802d5c0 con 0x7f73a81029a0 2024-04-21T08:44:38.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:38 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2701135257' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T08:44:38.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:38 smithi082 ceph-mon[27967]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T08:44:38.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:38 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2210437199' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-21T08:44:38.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:38 smithi082 ceph-mon[27967]: pgmap v293: 129 pgs: 2 creating+peering, 87 active+clean, 40 unknown; 578 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-04-21T08:44:38.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:38 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2210437199' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-21T08:44:38.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:38 smithi082 ceph-mon[27967]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T08:44:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:38 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2701135257' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T08:44:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:38 smithi106 ceph-mon[31072]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T08:44:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:38 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2210437199' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-21T08:44:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:38 smithi106 ceph-mon[31072]: pgmap v293: 129 pgs: 2 creating+peering, 87 active+clean, 40 unknown; 578 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-04-21T08:44:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:38 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2210437199' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-21T08:44:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:38 smithi106 ceph-mon[31072]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T08:44:38.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:38 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2701135257' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T08:44:38.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:38 smithi134 ceph-mon[31772]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T08:44:38.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:38 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2210437199' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-21T08:44:38.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:38 smithi134 ceph-mon[31772]: pgmap v293: 129 pgs: 2 creating+peering, 87 active+clean, 40 unknown; 578 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-04-21T08:44:38.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:38 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2210437199' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-21T08:44:38.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:38 smithi134 ceph-mon[31772]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T08:44:39.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.534+0000 7f739e7fc640 1 -- 172.21.15.82:0/1294909648 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f73a0014030 con 0x7f73a81029a0 2024-04-21T08:44:39.534 INFO:teuthology.orchestra.run.smithi082.stderr:added data pool 4 to fsmap 2024-04-21T08:44:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.536+0000 7f73af359640 1 -- 172.21.15.82:0/1294909648 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f73940a3bb0 msgr2=0x7f73940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.536+0000 7f73af359640 1 --2- 172.21.15.82:0/1294909648 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f73940a3bb0 0x7f73940a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f73a4036040 tx=0x7f73a4000bc0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.536+0000 7f73af359640 1 -- 172.21.15.82:0/1294909648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a81029a0 msgr2=0x7f73a819f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.536+0000 7f73af359640 1 --2- 172.21.15.82:0/1294909648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a81029a0 0x7f73a819f780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73a000cad0 tx=0x7f73a000cf90 comp rx=0 tx=0).stop 2024-04-21T08:44:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.536+0000 7f73af359640 1 -- 172.21.15.82:0/1294909648 shutdown_connections 2024-04-21T08:44:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.536+0000 7f73af359640 1 --2- 172.21.15.82:0/1294909648 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f73940a3bb0 0x7f73940a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.536+0000 7f73af359640 1 --2- 172.21.15.82:0/1294909648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a81029a0 0x7f73a819f780 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.536+0000 7f73af359640 1 --2- 172.21.15.82:0/1294909648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73a8101fe0 0x7f73a8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.536+0000 7f73af359640 1 --2- 172.21.15.82:0/1294909648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73a8100e90 0x7f73a81981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.536+0000 7f73af359640 1 -- 172.21.15.82:0/1294909648 >> 172.21.15.82:0/1294909648 conn(0x7f73a80fc400 msgr2=0x7f73a81053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:39.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.536+0000 7f73af359640 1 -- 172.21.15.82:0/1294909648 shutdown_connections 2024-04-21T08:44:39.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.536+0000 7f73af359640 1 -- 172.21.15.82:0/1294909648 wait complete. 2024-04-21T08:44:39.548 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:39 smithi106 ceph-mon[31072]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:44:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:39 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1294909648' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-21T08:44:39.628 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:44:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b[60834]: 2024-04-21T08:44:39.375+0000 7f642eeb7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-21T08:44:39.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.647+0000 7fd99a796640 1 -- 172.21.15.82:0/147096250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994102a20 msgr2=0x7fd994104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:39.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.647+0000 7fd99a796640 1 --2- 172.21.15.82:0/147096250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994102a20 0x7fd994104e10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd988009900 tx=0x7fd98802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:39.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.647+0000 7fd99a796640 1 -- 172.21.15.82:0/147096250 shutdown_connections 2024-04-21T08:44:39.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.647+0000 7fd99a796640 1 --2- 172.21.15.82:0/147096250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd994107c80 0x7fd99410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.647+0000 7fd99a796640 1 --2- 172.21.15.82:0/147096250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd994105350 0x7fd994107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.647+0000 7fd99a796640 1 --2- 172.21.15.82:0/147096250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994102a20 0x7fd994104e10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.647+0000 7fd99a796640 1 -- 172.21.15.82:0/147096250 >> 172.21.15.82:0/147096250 conn(0x7fd9940fc410 msgr2=0x7fd9940fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:39.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.647+0000 7fd99a796640 1 -- 172.21.15.82:0/147096250 shutdown_connections 2024-04-21T08:44:39.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.647+0000 7fd99a796640 1 -- 172.21.15.82:0/147096250 wait complete. 2024-04-21T08:44:39.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.647+0000 7fd99a796640 1 Processor -- start 2024-04-21T08:44:39.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.647+0000 7fd99a796640 1 -- start start 2024-04-21T08:44:39.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd99a796640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd994102a20 0x7fd994198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd99a796640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994105350 0x7fd9941987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd99a796640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd994107c80 0x7fd99419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd99a796640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd99410db60 con 0x7fd994105350 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd99a796640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd99410d9e0 con 0x7fd994102a20 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd99a796640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd99410dce0 con 0x7fd994107c80 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994105350 0x7fd9941987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994105350 0x7fd9941987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49722/0 (socket says 172.21.15.82:49722) 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd9937fe640 1 -- 172.21.15.82:0/1710377628 learned_addr learned my addr 172.21.15.82:0/1710377628 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd998d33640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd994107c80 0x7fd99419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd993fff640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd994102a20 0x7fd994198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd9937fe640 1 -- 172.21.15.82:0/1710377628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd994107c80 msgr2=0x7fd99419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd9937fe640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd994107c80 0x7fd99419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd9937fe640 1 -- 172.21.15.82:0/1710377628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd994102a20 msgr2=0x7fd994198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:39.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd9937fe640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd994102a20 0x7fd994198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd9937fe640 1 -- 172.21.15.82:0/1710377628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99419fe80 con 0x7fd994105350 2024-04-21T08:44:39.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd9937fe640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994105350 0x7fd9941987a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd98400d740 tx=0x7fd98400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:39.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd9917fa640 1 -- 172.21.15.82:0/1710377628 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984017070 con 0x7fd994105350 2024-04-21T08:44:39.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd9917fa640 1 -- 172.21.15.82:0/1710377628 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd98400de50 con 0x7fd994105350 2024-04-21T08:44:39.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd9917fa640 1 -- 172.21.15.82:0/1710377628 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984004ce0 con 0x7fd994105350 2024-04-21T08:44:39.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd99a796640 1 -- 172.21.15.82:0/1710377628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9941a0170 con 0x7fd994105350 2024-04-21T08:44:39.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.648+0000 7fd99a796640 1 -- 172.21.15.82:0/1710377628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9941a06f0 con 0x7fd994105350 2024-04-21T08:44:39.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.649+0000 7fd99a796640 1 -- 172.21.15.82:0/1710377628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9940fdf00 con 0x7fd994105350 2024-04-21T08:44:39.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.650+0000 7fd9917fa640 1 -- 172.21.15.82:0/1710377628 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd984004e80 con 0x7fd994105350 2024-04-21T08:44:39.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.651+0000 7fd9917fa640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd97c0a3bb0 0x7fd97c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:39.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.651+0000 7fd993fff640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd97c0a3bb0 0x7fd97c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:39.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.651+0000 7fd9917fa640 1 -- 172.21.15.82:0/1710377628 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd9840d0090 con 0x7fd994105350 2024-04-21T08:44:39.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.651+0000 7fd993fff640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd97c0a3bb0 0x7fd97c0a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9880096c0 tx=0x7fd988000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:39.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.655+0000 7fd9917fa640 1 -- 172.21.15.82:0/1710377628 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd98409c830 con 0x7fd994105350 2024-04-21T08:44:39.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:39 smithi082 ceph-mon[27967]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:44:39.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:39 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1294909648' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-21T08:44:39.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.869+0000 7fd99a796640 1 -- 172.21.15.82:0/1710377628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd994062b50 con 0x7fd994105350 2024-04-21T08:44:39.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.870+0000 7fd9917fa640 1 -- 172.21.15.82:0/1710377628 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11809 (secure 0 0 0) 0x7fd98409c130 con 0x7fd994105350 2024-04-21T08:44:39.870 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:39.870 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:44:38.858497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24509,"qdb_cluster":[24509]},"id":1}]} 2024-04-21T08:44:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 16 2024-04-21T08:44:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.872+0000 7fd99a796640 1 -- 172.21.15.82:0/1710377628 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd97c0a3bb0 msgr2=0x7fd97c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.872+0000 7fd99a796640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd97c0a3bb0 0x7fd97c0a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9880096c0 tx=0x7fd988000bc0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.872+0000 7fd99a796640 1 -- 172.21.15.82:0/1710377628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994105350 msgr2=0x7fd9941987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.872+0000 7fd99a796640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994105350 0x7fd9941987a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd98400d740 tx=0x7fd98400dc00 comp rx=0 tx=0).stop 2024-04-21T08:44:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.872+0000 7fd99a796640 1 -- 172.21.15.82:0/1710377628 shutdown_connections 2024-04-21T08:44:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.872+0000 7fd99a796640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd97c0a3bb0 0x7fd97c0a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.872+0000 7fd99a796640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd994107c80 0x7fd99419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.872+0000 7fd99a796640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994105350 0x7fd9941987a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.872+0000 7fd99a796640 1 --2- 172.21.15.82:0/1710377628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd994102a20 0x7fd994198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.872+0000 7fd99a796640 1 -- 172.21.15.82:0/1710377628 >> 172.21.15.82:0/1710377628 conn(0x7fd9940fc410 msgr2=0x7fd994103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.872+0000 7fd99a796640 1 -- 172.21.15.82:0/1710377628 shutdown_connections 2024-04-21T08:44:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.872+0000 7fd99a796640 1 -- 172.21.15.82:0/1710377628 wait complete. 2024-04-21T08:44:39.883 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:44:39.975 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:39 smithi134 ceph-mon[31772]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:44:39.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:39 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1294909648' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-21T08:44:39.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.981+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1440375759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae8101810 msgr2=0x7f6ae8101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:39.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.981+0000 7f6aefc3f640 1 --2- 172.21.15.82:0/1440375759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae8101810 0x7f6ae8101c90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009900 tx=0x7f6ae402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:39.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.981+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1440375759 shutdown_connections 2024-04-21T08:44:39.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.981+0000 7f6aefc3f640 1 --2- 172.21.15.82:0/1440375759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ae81021d0 0x7f6ae8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.981+0000 7f6aefc3f640 1 --2- 172.21.15.82:0/1440375759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae8101810 0x7f6ae8101c90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.981+0000 7f6aefc3f640 1 --2- 172.21.15.82:0/1440375759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae8100620 0x7f6ae8100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.981+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1440375759 >> 172.21.15.82:0/1440375759 conn(0x7f6ae80fbb90 msgr2=0x7f6ae80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:39.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.981+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1440375759 shutdown_connections 2024-04-21T08:44:39.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.981+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1440375759 wait complete. 2024-04-21T08:44:39.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.981+0000 7f6aefc3f640 1 Processor -- start 2024-04-21T08:44:39.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.981+0000 7f6aefc3f640 1 -- start start 2024-04-21T08:44:39.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aefc3f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ae8100620 0x7f6ae8197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:39.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aefc3f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae8101810 0x7f6ae81975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:39.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aefc3f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae81021d0 0x7f6ae819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:39.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aefc3f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae810c9e0 con 0x7f6ae8101810 2024-04-21T08:44:39.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aefc3f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae810c860 con 0x7f6ae8100620 2024-04-21T08:44:39.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aefc3f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae810cb60 con 0x7f6ae81021d0 2024-04-21T08:44:39.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aed1da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae8101810 0x7f6ae81975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:39.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aed1da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae8101810 0x7f6ae81975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49750/0 (socket says 172.21.15.82:49750) 2024-04-21T08:44:39.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aed1da640 1 -- 172.21.15.82:0/1991745124 learned_addr learned my addr 172.21.15.82:0/1991745124 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:39.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aed9db640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ae8100620 0x7f6ae8197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:39.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aee1dc640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae81021d0 0x7f6ae819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:39.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aed1da640 1 -- 172.21.15.82:0/1991745124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae81021d0 msgr2=0x7f6ae819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:39.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aed1da640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae81021d0 0x7f6ae819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aed1da640 1 -- 172.21.15.82:0/1991745124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ae8100620 msgr2=0x7f6ae8197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:39.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aed1da640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ae8100620 0x7f6ae8197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:39.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aed1da640 1 -- 172.21.15.82:0/1991745124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae819ec90 con 0x7f6ae8101810 2024-04-21T08:44:39.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aed1da640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae8101810 0x7f6ae81975b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4036040 tx=0x7f6ae40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:39.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6adeffd640 1 -- 172.21.15.82:0/1991745124 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4043020 con 0x7f6ae8101810 2024-04-21T08:44:39.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6adeffd640 1 -- 172.21.15.82:0/1991745124 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae40054c0 con 0x7f6ae8101810 2024-04-21T08:44:39.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1991745124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae819ef20 con 0x7f6ae8101810 2024-04-21T08:44:39.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6adeffd640 1 -- 172.21.15.82:0/1991745124 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4003db0 con 0x7f6ae8101810 2024-04-21T08:44:39.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.982+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1991745124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae819f3e0 con 0x7f6ae8101810 2024-04-21T08:44:39.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.983+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1991745124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae81079c0 con 0x7f6ae8101810 2024-04-21T08:44:39.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.985+0000 7f6adeffd640 1 -- 172.21.15.82:0/1991745124 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6ae4046020 con 0x7f6ae8101810 2024-04-21T08:44:39.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.985+0000 7f6adeffd640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6ad40a3bb0 0x7f6ad40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:39.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.985+0000 7f6aed9db640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6ad40a3bb0 0x7f6ad40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:39.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.985+0000 7f6adeffd640 1 -- 172.21.15.82:0/1991745124 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f6ae40f2f00 con 0x7f6ae8101810 2024-04-21T08:44:39.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.985+0000 7f6aed9db640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6ad40a3bb0 0x7f6ad40a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80058c0 tx=0x7f6ad800a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:39.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:39.989+0000 7f6adeffd640 1 -- 172.21.15.82:0/1991745124 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ae404d020 con 0x7f6ae8101810 2024-04-21T08:44:40.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.168+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1991745124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ae802d7c0 con 0x7f6ae8101810 2024-04-21T08:44:40.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.169+0000 7f6adeffd640 1 -- 172.21.15.82:0/1991745124 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24140 (secure 0 0 0) 0x7f6ae40bf6a0 con 0x7f6ae8101810 2024-04-21T08:44:40.169 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:40.169 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":89,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:44:38.858333+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:44:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.172+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1991745124 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6ad40a3bb0 msgr2=0x7f6ad40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.172+0000 7f6aefc3f640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6ad40a3bb0 0x7f6ad40a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80058c0 tx=0x7f6ad800a4c0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.172+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1991745124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae8101810 msgr2=0x7f6ae81975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.172+0000 7f6aefc3f640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae8101810 0x7f6ae81975b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4036040 tx=0x7f6ae40042f0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.172+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1991745124 shutdown_connections 2024-04-21T08:44:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.172+0000 7f6aefc3f640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6ad40a3bb0 0x7f6ad40a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.172+0000 7f6aefc3f640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae81021d0 0x7f6ae819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.172+0000 7f6aefc3f640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae8101810 0x7f6ae81975b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.172+0000 7f6aefc3f640 1 --2- 172.21.15.82:0/1991745124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ae8100620 0x7f6ae8197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.172+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1991745124 >> 172.21.15.82:0/1991745124 conn(0x7f6ae80fbb90 msgr2=0x7f6ae8104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.172+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1991745124 shutdown_connections 2024-04-21T08:44:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.172+0000 7f6aefc3f640 1 -- 172.21.15.82:0/1991745124 wait complete. 2024-04-21T08:44:40.183 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:44:40.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.281+0000 7fe43777e640 1 -- 172.21.15.82:0/3998982306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4301029b0 msgr2=0x7fe4301092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.281+0000 7fe43777e640 1 --2- 172.21.15.82:0/3998982306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4301029b0 0x7fe4301092d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe420009900 tx=0x7fe42002bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.281+0000 7fe43777e640 1 -- 172.21.15.82:0/3998982306 shutdown_connections 2024-04-21T08:44:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.281+0000 7fe43777e640 1 --2- 172.21.15.82:0/3998982306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4301029b0 0x7fe4301092d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.281+0000 7fe43777e640 1 --2- 172.21.15.82:0/3998982306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe430101ff0 0x7fe430102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.281+0000 7fe43777e640 1 --2- 172.21.15.82:0/3998982306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe430100ea0 0x7fe4301012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.281+0000 7fe43777e640 1 -- 172.21.15.82:0/3998982306 >> 172.21.15.82:0/3998982306 conn(0x7fe4300fc450 msgr2=0x7fe4300fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.281+0000 7fe43777e640 1 -- 172.21.15.82:0/3998982306 shutdown_connections 2024-04-21T08:44:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.281+0000 7fe43777e640 1 -- 172.21.15.82:0/3998982306 wait complete. 2024-04-21T08:44:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.281+0000 7fe43777e640 1 Processor -- start 2024-04-21T08:44:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.281+0000 7fe43777e640 1 -- start start 2024-04-21T08:44:40.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe43777e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe430100ea0 0x7fe430198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe43777e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe430101ff0 0x7fe430198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe43777e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4301029b0 0x7fe43019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe43777e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe43010db60 con 0x7fe430101ff0 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe43777e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe43010d9e0 con 0x7fe430100ea0 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe43777e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe43010dce0 con 0x7fe4301029b0 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe434d19640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe430101ff0 0x7fe430198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe434d19640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe430101ff0 0x7fe430198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49766/0 (socket says 172.21.15.82:49766) 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe434d19640 1 -- 172.21.15.82:0/3672418433 learned_addr learned my addr 172.21.15.82:0/3672418433 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe434d19640 1 -- 172.21.15.82:0/3672418433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4301029b0 msgr2=0x7fe43019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe434d19640 1 --2- 172.21.15.82:0/3672418433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4301029b0 0x7fe43019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe434d19640 1 -- 172.21.15.82:0/3672418433 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe430100ea0 msgr2=0x7fe430198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe434d19640 1 --2- 172.21.15.82:0/3672418433 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe430100ea0 0x7fe430198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe434d19640 1 -- 172.21.15.82:0/3672418433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43019fd20 con 0x7fe430101ff0 2024-04-21T08:44:40.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe434d19640 1 --2- 172.21.15.82:0/3672418433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe430101ff0 0x7fe430198640 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe42400d770 tx=0x7fe42400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:40.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe42e7fc640 1 -- 172.21.15.82:0/3672418433 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424017070 con 0x7fe430101ff0 2024-04-21T08:44:40.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe42e7fc640 1 -- 172.21.15.82:0/3672418433 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe424009c30 con 0x7fe430101ff0 2024-04-21T08:44:40.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe43777e640 1 -- 172.21.15.82:0/3672418433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43019ffb0 con 0x7fe430101ff0 2024-04-21T08:44:40.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe42e7fc640 1 -- 172.21.15.82:0/3672418433 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424003cc0 con 0x7fe430101ff0 2024-04-21T08:44:40.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.282+0000 7fe43777e640 1 -- 172.21.15.82:0/3672418433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4301a04d0 con 0x7fe430101ff0 2024-04-21T08:44:40.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.283+0000 7fe43777e640 1 -- 172.21.15.82:0/3672418433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe430108150 con 0x7fe430101ff0 2024-04-21T08:44:40.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.284+0000 7fe42e7fc640 1 -- 172.21.15.82:0/3672418433 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe424003e60 con 0x7fe430101ff0 2024-04-21T08:44:40.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.285+0000 7fe42e7fc640 1 --2- 172.21.15.82:0/3672418433 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe4180a3bb0 0x7fe4180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:40.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.285+0000 7fe43551a640 1 --2- 172.21.15.82:0/3672418433 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe4180a3bb0 0x7fe4180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:40.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.285+0000 7fe42e7fc640 1 -- 172.21.15.82:0/3672418433 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe4240d0440 con 0x7fe430101ff0 2024-04-21T08:44:40.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.285+0000 7fe43551a640 1 --2- 172.21.15.82:0/3672418433 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe4180a3bb0 0x7fe4180a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe428005fd0 tx=0x7fe4280094b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:40.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.289+0000 7fe42e7fc640 1 -- 172.21.15.82:0/3672418433 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe424009de0 con 0x7fe430101ff0 2024-04-21T08:44:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.467+0000 7fe43777e640 1 -- 172.21.15.82:0/3672418433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe43002d5c0 con 0x7fe430101ff0 2024-04-21T08:44:40.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.468+0000 7fe42e7fc640 1 -- 172.21.15.82:0/3672418433 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24140 (secure 0 0 0) 0x7fe42409cbe0 con 0x7fe430101ff0 2024-04-21T08:44:40.468 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:40.468 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":89,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:44:38.858333+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:44:40.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.471+0000 7fe43777e640 1 -- 172.21.15.82:0/3672418433 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe4180a3bb0 msgr2=0x7fe4180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.471+0000 7fe43777e640 1 --2- 172.21.15.82:0/3672418433 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe4180a3bb0 0x7fe4180a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe428005fd0 tx=0x7fe4280094b0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.471+0000 7fe43777e640 1 -- 172.21.15.82:0/3672418433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe430101ff0 msgr2=0x7fe430198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.471+0000 7fe43777e640 1 --2- 172.21.15.82:0/3672418433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe430101ff0 0x7fe430198640 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe42400d770 tx=0x7fe42400dc30 comp rx=0 tx=0).stop 2024-04-21T08:44:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.471+0000 7fe43777e640 1 -- 172.21.15.82:0/3672418433 shutdown_connections 2024-04-21T08:44:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.471+0000 7fe43777e640 1 --2- 172.21.15.82:0/3672418433 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe4180a3bb0 0x7fe4180a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.471+0000 7fe43777e640 1 --2- 172.21.15.82:0/3672418433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4301029b0 0x7fe43019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.471+0000 7fe43777e640 1 --2- 172.21.15.82:0/3672418433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe430101ff0 0x7fe430198640 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.471+0000 7fe43777e640 1 --2- 172.21.15.82:0/3672418433 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe430100ea0 0x7fe430198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.471+0000 7fe43777e640 1 -- 172.21.15.82:0/3672418433 >> 172.21.15.82:0/3672418433 conn(0x7fe4300fc450 msgr2=0x7fe430105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.471+0000 7fe43777e640 1 -- 172.21.15.82:0/3672418433 shutdown_connections 2024-04-21T08:44:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.471+0000 7fe43777e640 1 -- 172.21.15.82:0/3672418433 wait complete. 2024-04-21T08:44:40.497 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:40.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.595+0000 7f2766796640 1 -- 172.21.15.82:0/140086654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2760102a00 msgr2=0x7f2760104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.595+0000 7f2766796640 1 --2- 172.21.15.82:0/140086654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2760102a00 0x7f2760104df0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2754009900 tx=0x7f275402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:40.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.595+0000 7f2766796640 1 -- 172.21.15.82:0/140086654 shutdown_connections 2024-04-21T08:44:40.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.595+0000 7f2766796640 1 --2- 172.21.15.82:0/140086654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2760107c60 0x7f276010a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.595+0000 7f2766796640 1 --2- 172.21.15.82:0/140086654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2760105330 0x7f2760107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.595+0000 7f2766796640 1 --2- 172.21.15.82:0/140086654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2760102a00 0x7f2760104df0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.595+0000 7f2766796640 1 -- 172.21.15.82:0/140086654 >> 172.21.15.82:0/140086654 conn(0x7f27600fc410 msgr2=0x7f27600fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:40.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.595+0000 7f2766796640 1 -- 172.21.15.82:0/140086654 shutdown_connections 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.595+0000 7f2766796640 1 -- 172.21.15.82:0/140086654 wait complete. 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.595+0000 7f2766796640 1 Processor -- start 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.595+0000 7f2766796640 1 -- start start 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f2766796640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2760102a00 0x7f276019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f2766796640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2760105330 0x7f276019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f2766796640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2760107c60 0x7f27601a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f2766796640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f276010db30 con 0x7f2760105330 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f2766796640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f276010d9b0 con 0x7f2760107c60 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f2766796640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f276010dcb0 con 0x7f2760102a00 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f275f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2760105330 0x7f276019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f275f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2760105330 0x7f276019a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49774/0 (socket says 172.21.15.82:49774) 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f275f7fe640 1 -- 172.21.15.82:0/3907740561 learned_addr learned my addr 172.21.15.82:0/3907740561 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:40.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f275ffff640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2760102a00 0x7f276019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f2764d33640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2760107c60 0x7f27601a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f275f7fe640 1 -- 172.21.15.82:0/3907740561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2760102a00 msgr2=0x7f276019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f275f7fe640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2760102a00 0x7f276019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f275f7fe640 1 -- 172.21.15.82:0/3907740561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2760107c60 msgr2=0x7f27601a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f275f7fe640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2760107c60 0x7f27601a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f275f7fe640 1 -- 172.21.15.82:0/3907740561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27601a2020 con 0x7f2760105330 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.596+0000 7f275f7fe640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2760105330 0x7f276019a940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f275000d770 tx=0x7f275000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.597+0000 7f275d7fa640 1 -- 172.21.15.82:0/3907740561 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750017070 con 0x7f2760105330 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.597+0000 7f2766796640 1 -- 172.21.15.82:0/3907740561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27601a2310 con 0x7f2760105330 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.597+0000 7f275d7fa640 1 -- 172.21.15.82:0/3907740561 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2750009c30 con 0x7f2760105330 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.597+0000 7f275d7fa640 1 -- 172.21.15.82:0/3907740561 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275000f620 con 0x7f2760105330 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.597+0000 7f2766796640 1 -- 172.21.15.82:0/3907740561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27601a2930 con 0x7f2760105330 2024-04-21T08:44:40.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.598+0000 7f2766796640 1 -- 172.21.15.82:0/3907740561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2760109750 con 0x7f2760105330 2024-04-21T08:44:40.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.599+0000 7f275d7fa640 1 -- 172.21.15.82:0/3907740561 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f275001a070 con 0x7f2760105330 2024-04-21T08:44:40.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.599+0000 7f275d7fa640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f27480a3bb0 0x7f27480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:40.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.599+0000 7f275ffff640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f27480a3bb0 0x7f27480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:40.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.599+0000 7f275d7fa640 1 -- 172.21.15.82:0/3907740561 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f27500d00c0 con 0x7f2760105330 2024-04-21T08:44:40.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.600+0000 7f275ffff640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f27480a3bb0 0x7f27480a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2754009230 tx=0x7f2754005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:40.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.603+0000 7f275d7fa640 1 -- 172.21.15.82:0/3907740561 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2750009de0 con 0x7f2760105330 2024-04-21T08:44:40.805 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:40 smithi106 ceph-mon[31072]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T08:44:40.806 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:40 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1294909648' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-21T08:44:40.806 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:40 smithi106 ceph-mon[31072]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:44:40.806 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:40 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1710377628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:40.806 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:40 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1991745124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:40.806 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:40 smithi106 ceph-mon[31072]: pgmap v296: 129 pgs: 2 creating+peering, 103 active+clean, 24 unknown; 579 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-21T08:44:40.806 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:40 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3672418433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:40.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.818+0000 7f2766796640 1 -- 172.21.15.82:0/3907740561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f276002d5c0 con 0x7f2760105330 2024-04-21T08:44:40.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.819+0000 7f275d7fa640 1 -- 172.21.15.82:0/3907740561 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11809 (secure 0 0 0) 0x7f275009c210 con 0x7f2760105330 2024-04-21T08:44:40.819 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:40.819 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:44:38.858497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24509,"qdb_cluster":[24509]},"id":1}]} 2024-04-21T08:44:40.819 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 16 2024-04-21T08:44:40.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.821+0000 7f2766796640 1 -- 172.21.15.82:0/3907740561 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f27480a3bb0 msgr2=0x7f27480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.821+0000 7f2766796640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f27480a3bb0 0x7f27480a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2754009230 tx=0x7f2754005fd0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.821+0000 7f2766796640 1 -- 172.21.15.82:0/3907740561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2760105330 msgr2=0x7f276019a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.821+0000 7f2766796640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2760105330 0x7f276019a940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f275000d770 tx=0x7f275000dc30 comp rx=0 tx=0).stop 2024-04-21T08:44:40.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.821+0000 7f2766796640 1 -- 172.21.15.82:0/3907740561 shutdown_connections 2024-04-21T08:44:40.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.821+0000 7f2766796640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f27480a3bb0 0x7f27480a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.821+0000 7f2766796640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2760107c60 0x7f27601a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.821+0000 7f2766796640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2760105330 0x7f276019a940 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.821+0000 7f2766796640 1 --2- 172.21.15.82:0/3907740561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2760102a00 0x7f276019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.821+0000 7f2766796640 1 -- 172.21.15.82:0/3907740561 >> 172.21.15.82:0/3907740561 conn(0x7f27600fc410 msgr2=0x7f2760102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:40.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.821+0000 7f2766796640 1 -- 172.21.15.82:0/3907740561 shutdown_connections 2024-04-21T08:44:40.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.821+0000 7f2766796640 1 -- 172.21.15.82:0/3907740561 wait complete. 2024-04-21T08:44:40.831 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:40 smithi082 ceph-mon[27967]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T08:44:40.831 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:40 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1294909648' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-21T08:44:40.831 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:40 smithi082 ceph-mon[27967]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:44:40.831 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:40 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1710377628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:40.831 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:40 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1991745124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:40.831 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:40 smithi082 ceph-mon[27967]: pgmap v296: 129 pgs: 2 creating+peering, 103 active+clean, 24 unknown; 579 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-21T08:44:40.831 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:40 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3672418433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:40.832 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:44:40.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.930+0000 7f1d6f59d640 1 -- 172.21.15.82:0/922695052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d68105180 msgr2=0x7f1d68107570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.930+0000 7f1d6f59d640 1 --2- 172.21.15.82:0/922695052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d68105180 0x7f1d68107570 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009900 tx=0x7f1d5802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:40.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.930+0000 7f1d6f59d640 1 -- 172.21.15.82:0/922695052 shutdown_connections 2024-04-21T08:44:40.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.930+0000 7f1d6f59d640 1 --2- 172.21.15.82:0/922695052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d68107ab0 0x7f1d68109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.930+0000 7f1d6f59d640 1 --2- 172.21.15.82:0/922695052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d68105180 0x7f1d68107570 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.930+0000 7f1d6f59d640 1 --2- 172.21.15.82:0/922695052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d68102850 0x7f1d68104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.930+0000 7f1d6f59d640 1 -- 172.21.15.82:0/922695052 >> 172.21.15.82:0/922695052 conn(0x7f1d680fc1e0 msgr2=0x7f1d680fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:40.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.930+0000 7f1d6f59d640 1 -- 172.21.15.82:0/922695052 shutdown_connections 2024-04-21T08:44:40.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.930+0000 7f1d6f59d640 1 -- 172.21.15.82:0/922695052 wait complete. 2024-04-21T08:44:40.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6f59d640 1 Processor -- start 2024-04-21T08:44:40.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6f59d640 1 -- start start 2024-04-21T08:44:40.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6f59d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d68102850 0x7f1d68198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:40.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6f59d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d68105180 0x7f1d68198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:40.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6f59d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d68107ab0 0x7f1d6819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:40.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6f59d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810d890 con 0x7f1d68105180 2024-04-21T08:44:40.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6f59d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810d710 con 0x7f1d68102850 2024-04-21T08:44:40.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6f59d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810da10 con 0x7f1d68107ab0 2024-04-21T08:44:40.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6dd9a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d68105180 0x7f1d68198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:40.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6dd9a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d68105180 0x7f1d68198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49794/0 (socket says 172.21.15.82:49794) 2024-04-21T08:44:40.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6dd9a640 1 -- 172.21.15.82:0/593127100 learned_addr learned my addr 172.21.15.82:0/593127100 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:40.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6e59b640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d68102850 0x7f1d68198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:40.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6ed9c640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d68107ab0 0x7f1d6819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:40.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6dd9a640 1 -- 172.21.15.82:0/593127100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d68107ab0 msgr2=0x7f1d6819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6dd9a640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d68107ab0 0x7f1d6819f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6dd9a640 1 -- 172.21.15.82:0/593127100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d68102850 msgr2=0x7f1d68198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:40.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6dd9a640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d68102850 0x7f1d68198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:40.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.931+0000 7f1d6dd9a640 1 -- 172.21.15.82:0/593127100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6819fbe0 con 0x7f1d68105180 2024-04-21T08:44:40.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.932+0000 7f1d6dd9a640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d68105180 0x7f1d68198550 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1d58000c00 tx=0x7f1d58004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:40.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.932+0000 7f1d5f7fe640 1 -- 172.21.15.82:0/593127100 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58043020 con 0x7f1d68105180 2024-04-21T08:44:40.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.932+0000 7f1d5f7fe640 1 -- 172.21.15.82:0/593127100 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d58004440 con 0x7f1d68105180 2024-04-21T08:44:40.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.932+0000 7f1d6f59d640 1 -- 172.21.15.82:0/593127100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6819fe70 con 0x7f1d68105180 2024-04-21T08:44:40.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.932+0000 7f1d5f7fe640 1 -- 172.21.15.82:0/593127100 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58003c90 con 0x7f1d68105180 2024-04-21T08:44:40.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.932+0000 7f1d6f59d640 1 -- 172.21.15.82:0/593127100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d681a0380 con 0x7f1d68105180 2024-04-21T08:44:40.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.933+0000 7f1d6f59d640 1 -- 172.21.15.82:0/593127100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d680fdcd0 con 0x7f1d68105180 2024-04-21T08:44:40.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.934+0000 7f1d5f7fe640 1 -- 172.21.15.82:0/593127100 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1d58046020 con 0x7f1d68105180 2024-04-21T08:44:40.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.934+0000 7f1d5f7fe640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1d540a3c30 0x7f1d540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:40.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.934+0000 7f1d6e59b640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1d540a3c30 0x7f1d540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:40.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.935+0000 7f1d5f7fe640 1 -- 172.21.15.82:0/593127100 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1d580f2810 con 0x7f1d68105180 2024-04-21T08:44:40.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.935+0000 7f1d6e59b640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1d540a3c30 0x7f1d540a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009bd0 tx=0x7f1d64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:40.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:40.939+0000 7f1d5f7fe640 1 -- 172.21.15.82:0/593127100 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d5804d020 con 0x7f1d68105180 2024-04-21T08:44:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:40 smithi134 ceph-mon[31772]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T08:44:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:40 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1294909648' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-21T08:44:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:40 smithi134 ceph-mon[31772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:44:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:40 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1710377628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:40 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1991745124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:40 smithi134 ceph-mon[31772]: pgmap v296: 129 pgs: 2 creating+peering, 103 active+clean, 24 unknown; 579 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-21T08:44:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:40 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3672418433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:41.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.117+0000 7f1d6f59d640 1 -- 172.21.15.82:0/593127100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d68062bf0 con 0x7f1d68105180 2024-04-21T08:44:41.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.119+0000 7f1d5f7fe640 1 -- 172.21.15.82:0/593127100 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24140 (secure 0 0 0) 0x7f1d580befb0 con 0x7f1d68105180 2024-04-21T08:44:41.119 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:41.119 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":89,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:44:38.858333+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:44:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.121+0000 7f1d6f59d640 1 -- 172.21.15.82:0/593127100 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1d540a3c30 msgr2=0x7f1d540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.121+0000 7f1d6f59d640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1d540a3c30 0x7f1d540a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009bd0 tx=0x7f1d64005f90 comp rx=0 tx=0).stop 2024-04-21T08:44:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.121+0000 7f1d6f59d640 1 -- 172.21.15.82:0/593127100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d68105180 msgr2=0x7f1d68198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.121+0000 7f1d6f59d640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d68105180 0x7f1d68198550 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1d58000c00 tx=0x7f1d58004040 comp rx=0 tx=0).stop 2024-04-21T08:44:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.121+0000 7f1d6f59d640 1 -- 172.21.15.82:0/593127100 shutdown_connections 2024-04-21T08:44:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.121+0000 7f1d6f59d640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1d540a3c30 0x7f1d540a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.121+0000 7f1d6f59d640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d68107ab0 0x7f1d6819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.121+0000 7f1d6f59d640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d68105180 0x7f1d68198550 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.121+0000 7f1d6f59d640 1 --2- 172.21.15.82:0/593127100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d68102850 0x7f1d68198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.121+0000 7f1d6f59d640 1 -- 172.21.15.82:0/593127100 >> 172.21.15.82:0/593127100 conn(0x7f1d680fc1e0 msgr2=0x7f1d68103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:41.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.121+0000 7f1d6f59d640 1 -- 172.21.15.82:0/593127100 shutdown_connections 2024-04-21T08:44:41.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.121+0000 7f1d6f59d640 1 -- 172.21.15.82:0/593127100 wait complete. 2024-04-21T08:44:41.132 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-21T08:44:33.819176+0000', 'modified': '2024-04-21T08:44:38.858497+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24509}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/3796875133', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 3796875133}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 3796875133}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24509, 'qdb_cluster': [24509]} 2024-04-21T08:44:41.132 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-21T08:44:41.132 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T08:44:41.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.232+0000 7fe4de29a640 1 -- 172.21.15.82:0/591572613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d8102a10 msgr2=0x7fe4d8104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:41.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.232+0000 7fe4de29a640 1 --2- 172.21.15.82:0/591572613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d8102a10 0x7fe4d8104e00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009900 tx=0x7fe4c802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:41.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.232+0000 7fe4de29a640 1 -- 172.21.15.82:0/591572613 shutdown_connections 2024-04-21T08:44:41.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.232+0000 7fe4de29a640 1 --2- 172.21.15.82:0/591572613 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4d8107c70 0x7fe4d810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.232+0000 7fe4de29a640 1 --2- 172.21.15.82:0/591572613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8105340 0x7fe4d8107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.232+0000 7fe4de29a640 1 --2- 172.21.15.82:0/591572613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d8102a10 0x7fe4d8104e00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.232+0000 7fe4de29a640 1 -- 172.21.15.82:0/591572613 >> 172.21.15.82:0/591572613 conn(0x7fe4d80fc440 msgr2=0x7fe4d80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:41.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.232+0000 7fe4de29a640 1 -- 172.21.15.82:0/591572613 shutdown_connections 2024-04-21T08:44:41.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.232+0000 7fe4de29a640 1 -- 172.21.15.82:0/591572613 wait complete. 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.232+0000 7fe4de29a640 1 Processor -- start 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.232+0000 7fe4de29a640 1 -- start start 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4de29a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4d8102a10 0x7fe4d81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4de29a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d8105340 0x7fe4d8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4de29a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8107c70 0x7fe4d819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4de29a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d810db40 con 0x7fe4d8105340 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4de29a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d810d9c0 con 0x7fe4d8107c70 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4de29a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d810dcc0 con 0x7fe4d8102a10 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d8105340 0x7fe4d8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d8105340 0x7fe4d8198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49810/0 (socket says 172.21.15.82:49810) 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d77fe640 1 -- 172.21.15.82:0/577991345 learned_addr learned my addr 172.21.15.82:0/577991345 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4dc837640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8107c70 0x7fe4d819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d77fe640 1 -- 172.21.15.82:0/577991345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4d8102a10 msgr2=0x7fe4d81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d7fff640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4d8102a10 0x7fe4d81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d77fe640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4d8102a10 0x7fe4d81982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d77fe640 1 -- 172.21.15.82:0/577991345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8107c70 msgr2=0x7fe4d819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:41.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d77fe640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8107c70 0x7fe4d819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d77fe640 1 -- 172.21.15.82:0/577991345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d819feb0 con 0x7fe4d8105340 2024-04-21T08:44:41.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4dc837640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8107c70 0x7fe4d819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:41.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d77fe640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d8105340 0x7fe4d8198820 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005fd0 tx=0x7fe4cc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:41.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d57fa640 1 -- 172.21.15.82:0/577991345 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc017070 con 0x7fe4d8105340 2024-04-21T08:44:41.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d57fa640 1 -- 172.21.15.82:0/577991345 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4cc009c30 con 0x7fe4d8105340 2024-04-21T08:44:41.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4d57fa640 1 -- 172.21.15.82:0/577991345 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc003d30 con 0x7fe4d8105340 2024-04-21T08:44:41.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4de29a640 1 -- 172.21.15.82:0/577991345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d81a01a0 con 0x7fe4d8105340 2024-04-21T08:44:41.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.233+0000 7fe4de29a640 1 -- 172.21.15.82:0/577991345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d81a0740 con 0x7fe4d8105340 2024-04-21T08:44:41.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.235+0000 7fe4d57fa640 1 -- 172.21.15.82:0/577991345 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe4cc003ed0 con 0x7fe4d8105340 2024-04-21T08:44:41.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4e0f6a740 1 Processor -- start 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4e0f6a740 1 -- start start 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4e0f6a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacc3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4e0f6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc3830 0x55e4aacc3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4e0f6a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc42d0 0x55e4aaccaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4e0f6a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e4aac27b10 con 0x55e4aacc2e40 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4e0f6a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e4aabc5240 con 0x55e4aacc42d0 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4e0f6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4aac23ad0 con 0x55e4aacc3830 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4dc837640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacc3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4d57fa640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe4c00a3c30 0x7fe4c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4dc837640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacc3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49816/0 (socket says 172.21.15.82:49816) 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4dc837640 1 -- 172.21.15.82:0/2380902091 learned_addr learned my addr 172.21.15.82:0/2380902091 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4d77fe640 1 --2- 172.21.15.82:0/2380902091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc42d0 0x55e4aaccaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4dc837640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe4c00a3c30 0x7fe4c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4d7fff640 1 --2- 172.21.15.82:0/2380902091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc3830 0x55e4aacc3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.236+0000 7fe4dc837640 1 -- 172.21.15.82:0/2380902091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc3830 msgr2=0x55e4aacc3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.237+0000 7fe4dc837640 1 --2- 172.21.15.82:0/2380902091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc3830 0x55e4aacc3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.237+0000 7fe4dc837640 1 -- 172.21.15.82:0/2380902091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc42d0 msgr2=0x55e4aaccaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.237+0000 7fe4dc837640 1 --2- 172.21.15.82:0/2380902091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc42d0 0x55e4aaccaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.237+0000 7fe4dc837640 1 -- 172.21.15.82:0/2380902091 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4aa93a2d0 con 0x55e4aacc2e40 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.237+0000 7fe4d7fff640 1 --2- 172.21.15.82:0/2380902091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc3830 0x55e4aacc3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.237+0000 7fe4d57fa640 1 -- 172.21.15.82:0/577991345 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe4cc0d03a0 con 0x7fe4d8105340 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.237+0000 7fe4dc837640 1 --2- 172.21.15.82:0/2380902091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacc3260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000cbf0 tx=0x7fe4d0007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15104f666f58be63 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.237+0000 7fe4be7fc640 1 -- 172.21.15.82:0/2380902091 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0015070 con 0x55e4aacc2e40 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.237+0000 7fe4be7fc640 1 -- 172.21.15.82:0/2380902091 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d0016040 con 0x55e4aacc2e40 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.237+0000 7fe4be7fc640 1 -- 172.21.15.82:0/2380902091 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d001e020 con 0x55e4aacc2e40 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.237+0000 7fe4dc837640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe4c00a3c30 0x7fe4c00a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe4d001f000 tx=0x7fe4d001b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2380902091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 msgr2=0x55e4aacc3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2380902091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacc3260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000cbf0 tx=0x7fe4d0007680 comp rx=0 tx=0).stop 2024-04-21T08:44:41.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2380902091 shutdown_connections 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2380902091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc42d0 0x55e4aaccaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2380902091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc3830 0x55e4aacc3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2380902091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacc3260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2380902091 >> 172.21.15.82:0/2380902091 conn(0x55e4aacbef40 msgr2=0x55e4aacbd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2380902091 shutdown_connections 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2380902091 wait complete. 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 -- start start 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 Processor -- start 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc2e40 0x55e4aacccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc3830 0x55e4aaccd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc42d0 0x55e4aacd4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e4aaccb4b0 con 0x55e4aacc42d0 2024-04-21T08:44:41.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e4aabb0190 con 0x55e4aacc2e40 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4aaccb630 con 0x55e4aacc3830 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.238+0000 7fe4e0f6a740 10 client.0 _ll_register_callbacks cb 0x55e4aabab8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4dc837640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc42d0 0x55e4aacd4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4dc837640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc42d0 0x55e4aacd4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49832/0 (socket says 172.21.15.82:49832) 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4dc837640 1 -- 172.21.15.82:0/2209395823 learned_addr learned my addr 172.21.15.82:0/2209395823 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4d7fff640 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc2e40 0x55e4aacccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4d77fe640 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc3830 0x55e4aaccd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4dc837640 1 -- 172.21.15.82:0/2209395823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc3830 msgr2=0x55e4aaccd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4dc837640 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc3830 0x55e4aaccd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4dc837640 1 -- 172.21.15.82:0/2209395823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc2e40 msgr2=0x55e4aacccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4dc837640 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc2e40 0x55e4aacccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4dc837640 1 -- 172.21.15.82:0/2209395823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4aaa046b0 con 0x55e4aacc42d0 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4d77fe640 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc3830 0x55e4aaccd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:41.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4dc837640 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc42d0 0x55e4aacd4450 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009810 tx=0x7fe4d0009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28690d5cf03520a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4b77fe640 1 -- 172.21.15.82:0/2209395823 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0015050 con 0x55e4aacc42d0 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4b77fe640 1 -- 172.21.15.82:0/2209395823 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe4d001aa10 con 0x55e4aacc42d0 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2209395823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d0016040 con 0x55e4aacc42d0 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4b77fe640 1 -- 172.21.15.82:0/2209395823 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fe4d0038050 con 0x55e4aacc42d0 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4e0f6a740 10 client.14769 fetch_fsmap learned FSMap version 16 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2209395823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55e4aacd77f0 con 0x55e4aacc42d0 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.239+0000 7fe4b77fe640 1 -- 172.21.15.82:0/2209395823 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d0016040 con 0x55e4aacc42d0 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.240+0000 7fe4b77fe640 1 -- 172.21.15.82:0/2209395823 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7fe4d0023540 con 0x55e4aacc42d0 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.240+0000 7fe4e0f6a740 10 client.14769 fetch_fsmap finished waiting for FSMap version 16 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.240+0000 7fe4e0f6a740 10 client.14769 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.240+0000 7fe4e0f6a740 20 client.14769 populate_metadata read hostname 'smithi082' 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.240+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e4aacd9c80 0x55e4aacdb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.240+0000 7fe4e0f6a740 4 client.14769 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.240+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2209395823 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4aac35530 con 0x55e4aacd9c80 2024-04-21T08:44:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.240+0000 7fe4d7fff640 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e4aacd9c80 0x55e4aacdb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.240+0000 7fe4b77fe640 10 client.14769 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.240+0000 7fe4d7fff640 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e4aacd9c80 0x55e4aacdb0f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a026bdb81e4ab0a7 server_cookie=4b0434d1d6400e40 in_seq=0 out_seq=0 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4b77fe640 1 -- 172.21.15.82:0/2209395823 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4aac35530 con 0x55e4aacd9c80 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4b77fe640 10 client.14769 handle_command_reply: tid=0 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4e0f6a740 1 client.14769 shutdown 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2209395823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc42d0 msgr2=0x55e4aacd4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc42d0 0x55e4aacd4450 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009810 tx=0x7fe4d0009d60 comp rx=0 tx=0).stop 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2209395823 shutdown_connections 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e4aacd9c80 0x55e4aacdb0f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc42d0 0x55e4aacd4450 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc3830 0x55e4aaccd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4b77fe640 0 client.14769 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2209395823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc2e40 0x55e4aacccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2209395823 >> 172.21.15.82:0/2209395823 conn(0x55e4aacbef40 msgr2=0x55e4aacbca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:41.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2209395823 shutdown_connections 2024-04-21T08:44:41.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2209395823 wait complete. 2024-04-21T08:44:41.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.241+0000 7fe4e0f6a740 20 client.14769 trim_cache size 0 max 16384 2024-04-21T08:44:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4e0f6a740 1 Processor -- start 2024-04-21T08:44:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4e0f6a740 1 -- start start 2024-04-21T08:44:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4e0f6a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacd8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4e0f6a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc3830 0x55e4aacd4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4e0f6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc42d0 0x55e4aacb8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4e0f6a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e4aacd9430 con 0x55e4aacc2e40 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4e0f6a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e4aacd9b00 con 0x55e4aacc3830 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4e0f6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4aacda1d0 con 0x55e4aacc42d0 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4d7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacd8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4d7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacd8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49840/0 (socket says 172.21.15.82:49840) 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4d7fff640 1 -- 172.21.15.82:0/821663884 learned_addr learned my addr 172.21.15.82:0/821663884 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4d77fe640 1 --2- 172.21.15.82:0/821663884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc3830 0x55e4aacd4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4d7fff640 1 -- 172.21.15.82:0/821663884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc42d0 msgr2=0x55e4aacb8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4d7fff640 1 --2- 172.21.15.82:0/821663884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc42d0 0x55e4aacb8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4d7fff640 1 -- 172.21.15.82:0/821663884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc3830 msgr2=0x55e4aacd4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4d7fff640 1 --2- 172.21.15.82:0/821663884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc3830 0x55e4aacd4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.259+0000 7fe4d7fff640 1 -- 172.21.15.82:0/821663884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4aacbe630 con 0x55e4aacc2e40 2024-04-21T08:44:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4d7fff640 1 --2- 172.21.15.82:0/821663884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacd8bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe4c802e010 tx=0x7fe4c8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110075d2a66d1970 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4bdffb640 1 -- 172.21.15.82:0/821663884 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8048070 con 0x55e4aacc2e40 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4bdffb640 1 -- 172.21.15.82:0/821663884 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c8005040 con 0x55e4aacc2e40 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4bdffb640 1 -- 172.21.15.82:0/821663884 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8044430 con 0x55e4aacc2e40 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/821663884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 msgr2=0x55e4aacd8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/821663884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacd8bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe4c802e010 tx=0x7fe4c8005a20 comp rx=0 tx=0).stop 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/821663884 shutdown_connections 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/821663884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc42d0 0x55e4aacb8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/821663884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc3830 0x55e4aacd4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/821663884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacd8bf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/821663884 >> 172.21.15.82:0/821663884 conn(0x55e4aacbef40 msgr2=0x55e4aacbd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/821663884 shutdown_connections 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.260+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/821663884 wait complete. 2024-04-21T08:44:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4e0f6a740 1 -- start start 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4e0f6a740 1 Processor -- start 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4e0f6a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacd0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4e0f6a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc3830 0x55e4aacd28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4e0f6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc42d0 0x55e4aacd2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4e0f6a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e4aacdaf70 con 0x55e4aacc2e40 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4e0f6a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e4aacda8a0 con 0x55e4aacc3830 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4e0f6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4aacdb640 con 0x55e4aacc42d0 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4e0f6a740 10 client.0 _ll_register_callbacks cb 0x7fe4cc096f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4d7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacd0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4d7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacd0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49846/0 (socket says 172.21.15.82:49846) 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4d7fff640 1 -- 172.21.15.82:0/2732051191 learned_addr learned my addr 172.21.15.82:0/2732051191 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4d77fe640 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc3830 0x55e4aacd28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4dc837640 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc42d0 0x55e4aacd2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4d7fff640 1 -- 172.21.15.82:0/2732051191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc42d0 msgr2=0x55e4aacd2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4d7fff640 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc42d0 0x55e4aacd2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4d7fff640 1 -- 172.21.15.82:0/2732051191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc3830 msgr2=0x55e4aacd28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4d7fff640 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc3830 0x55e4aacd28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4d7fff640 1 -- 172.21.15.82:0/2732051191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4aacd3320 con 0x55e4aacc2e40 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4dc837640 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc42d0 0x55e4aacd2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4d7fff640 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacd0350 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe4c802c170 tx=0x7fe4c802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475cbdb095e17651 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.261+0000 7fe4b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4b77fe640 1 -- 172.21.15.82:0/2732051191 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8048040 con 0x55e4aacc2e40 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4b77fe640 1 -- 172.21.15.82:0/2732051191 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe4c80051f0 con 0x55e4aacc2e40 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2732051191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c8005040 con 0x55e4aacc2e40 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4b77fe640 1 -- 172.21.15.82:0/2732051191 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fe4c80053f0 con 0x55e4aacc2e40 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4e0f6a740 10 client.14775 fetch_fsmap learned FSMap version 16 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2732051191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55e4aacd4620 con 0x55e4aacc2e40 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4b77fe640 1 -- 172.21.15.82:0/2732051191 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c8005040 con 0x55e4aacc2e40 2024-04-21T08:44:41.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4b77fe640 1 -- 172.21.15.82:0/2732051191 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7fe4c80046c0 con 0x55e4aacc2e40 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4e0f6a740 10 client.14775 fetch_fsmap finished waiting for FSMap version 16 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4e0f6a740 10 client.14775 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4e0f6a740 20 client.14775 populate_metadata read hostname 'smithi082' 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e4aad44020 0x55e4aad42d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4e0f6a740 4 client.14775 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2732051191 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "status"}) -- 0x55e4aac2f3a0 con 0x55e4aad44020 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.262+0000 7fe4d77fe640 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e4aad44020 0x55e4aad42d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4d77fe640 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e4aad44020 0x55e4aad42d30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce93121ef8f68cb server_cookie=264a04a271c5290c in_seq=0 out_seq=0 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4b77fe640 10 client.14775 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4b77fe640 1 -- 172.21.15.82:0/2732051191 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e4aac2f3a0 con 0x55e4aad44020 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4b77fe640 10 client.14775 handle_command_reply: tid=0 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4e0f6a740 1 client.14775 shutdown 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2732051191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 msgr2=0x55e4aacd0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacd0350 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe4c802c170 tx=0x7fe4c802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:41.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2732051191 shutdown_connections 2024-04-21T08:44:41.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e4aad44020 0x55e4aad42d30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4aacc42d0 0x55e4aacd2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e4aacc3830 0x55e4aacd28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4b77fe640 0 client.14775 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:44:41.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4e0f6a740 1 --2- 172.21.15.82:0/2732051191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4aacc2e40 0x55e4aacd0350 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2732051191 >> 172.21.15.82:0/2732051191 conn(0x55e4aacbef40 msgr2=0x55e4aacd8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:41.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2732051191 shutdown_connections 2024-04-21T08:44:41.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4e0f6a740 1 -- 172.21.15.82:0/2732051191 wait complete. 2024-04-21T08:44:41.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.263+0000 7fe4e0f6a740 20 client.14775 trim_cache size 0 max 16384 2024-04-21T08:44:41.270 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:44:41.270 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:44:41.271 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-21T08:44:41.271 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24509, 2024-04-21T08:44:41.271 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:44:41.271 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:44:41.271 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:44:41.271 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 6.7809332409999996, 2024-04-21T08:44:41.271 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 16, 2024-04-21T08:44:41.271 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 89, 2024-04-21T08:44:41.271 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:44:41.271 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 12.848479663999999 2024-04-21T08:44:41.271 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:44:41.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.264+0000 7fe4de29a640 1 -- 172.21.15.82:0/577991345 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe4c00a3c30 msgr2=0x7fe4c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:41.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.264+0000 7fe4de29a640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe4c00a3c30 0x7fe4c00a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe4d001f000 tx=0x7fe4d001b000 comp rx=0 tx=0).stop 2024-04-21T08:44:41.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.264+0000 7fe4de29a640 1 -- 172.21.15.82:0/577991345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d8105340 msgr2=0x7fe4d8198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:41.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.264+0000 7fe4de29a640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d8105340 0x7fe4d8198820 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005fd0 tx=0x7fe4cc00dde0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.264+0000 7fe4de29a640 1 -- 172.21.15.82:0/577991345 shutdown_connections 2024-04-21T08:44:41.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.264+0000 7fe4de29a640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe4c00a3c30 0x7fe4c00a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.264+0000 7fe4de29a640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8107c70 0x7fe4d819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.264+0000 7fe4de29a640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4d8105340 0x7fe4d8198820 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.264+0000 7fe4de29a640 1 --2- 172.21.15.82:0/577991345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4d8102a10 0x7fe4d81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:41.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.264+0000 7fe4de29a640 1 -- 172.21.15.82:0/577991345 >> 172.21.15.82:0/577991345 conn(0x7fe4d80fc440 msgr2=0x7fe4d8103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:41.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.264+0000 7fe4de29a640 1 -- 172.21.15.82:0/577991345 shutdown_connections 2024-04-21T08:44:41.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:41.264+0000 7fe4de29a640 1 -- 172.21.15.82:0/577991345 wait complete. 2024-04-21T08:44:41.274 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-21T08:44:41.274 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:44:41.274 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph fs ls 2024-04-21T08:44:41.510 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:44:41.784 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:41 smithi082 ceph-mon[27967]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:44:41.784 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:41 smithi082 ceph-mon[27967]: Cluster is now healthy 2024-04-21T08:44:41.784 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:41 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3907740561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:41.784 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:41 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/593127100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:41.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:41 smithi134 ceph-mon[31772]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:44:41.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:41 smithi134 ceph-mon[31772]: Cluster is now healthy 2024-04-21T08:44:41.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:41 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3907740561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:41.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:41 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/593127100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:41.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:41 smithi106 ceph-mon[31072]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:44:41.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:41 smithi106 ceph-mon[31072]: Cluster is now healthy 2024-04-21T08:44:41.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:41 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3907740561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:41.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:41 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/593127100' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:42.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:42 smithi082 ceph-mon[27967]: pgmap v297: 129 pgs: 2 creating+peering, 126 active+clean, 1 unknown; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 175 B/s wr, 0 op/s 2024-04-21T08:44:42.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:42 smithi106 ceph-mon[31072]: pgmap v297: 129 pgs: 2 creating+peering, 126 active+clean, 1 unknown; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 175 B/s wr, 0 op/s 2024-04-21T08:44:42.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:42 smithi134 ceph-mon[31772]: pgmap v297: 129 pgs: 2 creating+peering, 126 active+clean, 1 unknown; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 175 B/s wr, 0 op/s 2024-04-21T08:44:43.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.230+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/1380173416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd4102b00 msgr2=0x7f5fd4102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.230+0000 7f5fdc5ae700 1 --2- 172.21.15.82:0/1380173416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd4102b00 0x7f5fd4102f80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009990 tx=0x7f5fc4009ca0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.230+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/1380173416 shutdown_connections 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.230+0000 7f5fdc5ae700 1 --2- 172.21.15.82:0/1380173416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fd41034c0 0x7f5fd410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.230+0000 7f5fdc5ae700 1 --2- 172.21.15.82:0/1380173416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd4108a10 0x7f5fd4108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.230+0000 7f5fdc5ae700 1 --2- 172.21.15.82:0/1380173416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd4102b00 0x7f5fd4102f80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.230+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/1380173416 >> 172.21.15.82:0/1380173416 conn(0x7f5fd4079c00 msgr2=0x7f5fd407a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.230+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/1380173416 shutdown_connections 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.230+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/1380173416 wait complete. 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fdc5ae700 1 Processor -- start 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fdc5ae700 1 -- start start 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fdc5ae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd4102b00 0x7f5fd4199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fdc5ae700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fd41034c0 0x7f5fd4199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fdc5ae700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd4108a10 0x7f5fd419d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fdc5ae700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd4112bf0 con 0x7f5fd4102b00 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fdc5ae700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd4112a70 con 0x7f5fd41034c0 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fdc5ae700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd4112d70 con 0x7f5fd4108a10 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fda34a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd4102b00 0x7f5fd4199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fda34a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd4102b00 0x7f5fd4199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35040/0 (socket says 172.21.15.82:35040) 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fda34a700 1 -- 172.21.15.82:0/2369157908 learned_addr learned my addr 172.21.15.82:0/2369157908 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fdab4b700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd4108a10 0x7f5fd419d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fd9b49700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fd41034c0 0x7f5fd4199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fda34a700 1 -- 172.21.15.82:0/2369157908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd4108a10 msgr2=0x7f5fd419d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fda34a700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd4108a10 0x7f5fd419d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fda34a700 1 -- 172.21.15.82:0/2369157908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fd41034c0 msgr2=0x7f5fd4199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fda34a700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fd41034c0 0x7f5fd4199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fda34a700 1 -- 172.21.15.82:0/2369157908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd419e110 con 0x7f5fd4102b00 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.231+0000 7f5fda34a700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd4102b00 0x7f5fd4199100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0060b0 tx=0x7f5fcc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:43.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.232+0000 7f5fcb7fe700 1 -- 172.21.15.82:0/2369157908 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc01a070 con 0x7f5fd4102b00 2024-04-21T08:44:43.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.232+0000 7f5fcb7fe700 1 -- 172.21.15.82:0/2369157908 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fcc00dd20 con 0x7f5fd4102b00 2024-04-21T08:44:43.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.232+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/2369157908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd419e400 con 0x7f5fd4102b00 2024-04-21T08:44:43.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.232+0000 7f5fcb7fe700 1 -- 172.21.15.82:0/2369157908 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc014600 con 0x7f5fd4102b00 2024-04-21T08:44:43.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.232+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/2369157908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd41a5c80 con 0x7f5fd4102b00 2024-04-21T08:44:43.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.232+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/2369157908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd4069730 con 0x7f5fd4102b00 2024-04-21T08:44:43.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.234+0000 7f5fcb7fe700 1 -- 172.21.15.82:0/2369157908 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5fcc01d070 con 0x7f5fd4102b00 2024-04-21T08:44:43.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.234+0000 7f5fcb7fe700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5fc00a3c40 0x7f5fc00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:43.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.234+0000 7f5fd9b49700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5fc00a3c40 0x7f5fc00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:43.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.234+0000 7f5fcb7fe700 1 -- 172.21.15.82:0/2369157908 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5fcc0cfd90 con 0x7f5fd4102b00 2024-04-21T08:44:43.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.235+0000 7f5fd9b49700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5fc00a3c40 0x7f5fc00a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400b5c0 tx=0x7f5fc4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:43.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.239+0000 7f5fcb7fe700 1 -- 172.21.15.82:0/2369157908 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fcc010070 con 0x7f5fd4102b00 2024-04-21T08:44:43.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.481+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/2369157908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5fd419a460 con 0x7f5fd4102b00 2024-04-21T08:44:43.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.482+0000 7f5fcb7fe700 1 -- 172.21.15.82:0/2369157908 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f5fcc09c530 con 0x7f5fd4102b00 2024-04-21T08:44:43.481 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-21T08:44:43.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.484+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/2369157908 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5fc00a3c40 msgr2=0x7f5fc00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.484+0000 7f5fdc5ae700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5fc00a3c40 0x7f5fc00a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400b5c0 tx=0x7f5fc4005960 comp rx=0 tx=0).stop 2024-04-21T08:44:43.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.484+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/2369157908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd4102b00 msgr2=0x7f5fd4199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:43.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.484+0000 7f5fdc5ae700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd4102b00 0x7f5fd4199100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0060b0 tx=0x7f5fcc00d3b0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.484+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/2369157908 shutdown_connections 2024-04-21T08:44:43.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.484+0000 7f5fdc5ae700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5fc00a3c40 0x7f5fc00a6100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.484+0000 7f5fdc5ae700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fd4108a10 0x7f5fd419d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.484+0000 7f5fdc5ae700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd4102b00 0x7f5fd4199100 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.484+0000 7f5fdc5ae700 1 --2- 172.21.15.82:0/2369157908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fd41034c0 0x7f5fd4199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:43.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.484+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/2369157908 >> 172.21.15.82:0/2369157908 conn(0x7f5fd4079c00 msgr2=0x7f5fd4100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:43.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.484+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/2369157908 shutdown_connections 2024-04-21T08:44:43.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:43.484+0000 7f5fdc5ae700 1 -- 172.21.15.82:0/2369157908 wait complete. 2024-04-21T08:44:43.855 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:43 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2369157908' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:44:43.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:43 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2369157908' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:44:43.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:43 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2369157908' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:44:44.226 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-21T08:44:44.226 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-21T08:44:44.226 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-21T08:44:44.226 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-21T08:44:44.226 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:44:44.226 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-21T08:44:44.226 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-21T08:44:44.227 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-21T08:44:44.227 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-21T08:44:44.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:44.227 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-21T08:44:44.243 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-21T08:44:44.243 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-21T08:44:44.243 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-21T08:44:44.243 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:44:44.243 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-21T08:44:44.243 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:44:44.243 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:44:44.243 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:44:44.244 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-21T08:44:44.244 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:44:44.244 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:44:44.244 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-21T08:44:44.244 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:44:44.244 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-21T08:44:44.244 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-21T08:44:44.244 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-21T08:44:44.244 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-21T08:44:44.244 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42200sec preferred_lft 42200sec 2024-04-21T08:44:44.244 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::8fde:e834:f54e:2fd3/64 scope link noprefixroute 2024-04-21T08:44:44.244 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:44:44.245 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-21T08:44:44.245 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:44:44.245 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-21T08:44:44.245 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2024-04-21T08:44:44.245 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2024-04-21T08:44:44.245 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2024-04-21T08:44:44.245 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-21T08:44:44.245 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-21T08:44:44.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:44:44.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:44:44.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:44.424 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-21T08:44:44.447 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-21T08:44:44.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:44.448 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-21T08:44:44.514 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-21T08:44:44.514 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-21T08:44:44.514 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-21T08:44:44.514 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-21T08:44:44.514 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-21T08:44:44.515 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:44:44.515 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-21T08:44:44.515 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-21T08:44:44.515 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-21T08:44:44.515 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-21T08:44:44.515 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-21T08:44:44.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:44:44.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:44:44.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:44.784 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-21T08:44:44.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:44.800 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-04-21T08:44:44.855 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:44:44.855 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-21T08:44:44.855 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:44:44.855 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-21T08:44:44.855 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-21T08:44:44.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:44 smithi082 ceph-mon[27967]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 150 B/s wr, 0 op/s 2024-04-21T08:44:44.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:44 smithi106 ceph-mon[31072]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 150 B/s wr, 0 op/s 2024-04-21T08:44:44.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:44:44.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:44:44.932 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-21T08:44:44.932 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:44:44.932 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-21T08:44:44.932 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-21T08:44:44.932 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-21T08:44:44.932 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-21T08:44:44.933 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-21T08:44:44.933 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-21T08:44:44.933 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-21T08:44:44.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:44 smithi134 ceph-mon[31772]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 150 B/s wr, 0 op/s 2024-04-21T08:44:44.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:44:45.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:44:45.057 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:44:45.057 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-21T08:44:45.057 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-21T08:44:45.057 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2024-04-21T08:44:45.057 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-21T08:44:45.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:44:45.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:44:45.131 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-21T08:44:45.131 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T08:44:45.132 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:44:45.156 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-21T08:44:45.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T08:44:45.157 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:44:45.211 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-21T08:44:45.275 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-21T08:44:45.331 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-21T08:44:45.331 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-21T08:44:45.331 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-21T08:44:45.331 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-21T08:44:45.331 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-21T08:44:45.331 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-21T08:44:45.332 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-21T08:44:45.332 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-21T08:44:45.332 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-21T08:44:45.332 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-21T08:44:45.332 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-21T08:44:45.332 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/selinux 2024-04-21T08:44:45.332 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T08:44:45.332 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-21T08:44:45.332 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-21T08:44:45.332 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-21T08:44:45.332 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-21T08:44:45.333 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-21T08:44:45.333 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-21T08:44:45.333 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-21T08:44:45.333 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysctl.service 2024-04-21T08:44:45.333 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-21T08:44:45.333 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-21T08:44:45.333 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/nfs/rpc_pipefs 2024-04-21T08:44:45.333 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-21T08:44:45.333 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-21T08:44:45.333 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay 2024-04-21T08:44:45.333 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/cb860465d9dd4389648760599048d56f382a9e05c431ddf1e93497513fc6d21f/merged 2024-04-21T08:44:45.333 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/9772f2d860971eb3f19d776615240978c60fc60a6b23215c3653e11e6ae9c55b/merged 2024-04-21T08:44:45.334 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/0 2024-04-21T08:44:45.334 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T08:44:45.334 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/048e109f117dcb565c69f5596238cd17ca948aecad6d90f819b9eb9901bbf747/merged 2024-04-21T08:44:45.334 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/47516ef441c32a032fb0d803b5341fef846267226b85f32a78f6b6a3dc257229/merged 2024-04-21T08:44:45.334 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/9e2762a086d09d81d825d10abe16e18c4b8a90db73dcf39593c3de2a6fc6ffcb/merged 2024-04-21T08:44:45.334 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/b74c628f3b949cbbac1b6398ecaa6af4447ad840c557f71455d0594e3d49d9b1/merged 2024-04-21T08:44:45.334 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/fb33f6b2de769041147303cc1c096df7b0b591936b85c47279341eb5bf473394/merged 2024-04-21T08:44:45.334 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/7eef970bb1ea502f4def775692fe1b07e55604d335bf28438956169f65f6a7dd/merged 2024-04-21T08:44:45.334 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/01ed206f1ea932edd37252b72a020b8e37aff753fdc6ffbf35ef8a637e8378ff/merged 2024-04-21T08:44:45.334 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/6e1048e875105ee30e460d83f1beb92669ddde960c07b462a12a168ecae487f5/merged 2024-04-21T08:44:45.334 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-21T08:44:45.334 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:44:45.335 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:44:45.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:45.335 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-21T08:44:45.386 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-21T08:44:45.386 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-21T08:44:45.428 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-21T08:44:45.451 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-21T08:44:45.506 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-21T08:44:45.507 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-21T08:44:45.507 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-21T08:44:45.507 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-21T08:44:45.507 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-21T08:44:45.507 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-21T08:44:45.507 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-21T08:44:45.507 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-21T08:44:45.507 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-21T08:44:45.507 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-21T08:44:45.507 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-21T08:44:45.507 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/selinux 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysctl.service 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/nfs/rpc_pipefs 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-21T08:44:45.508 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/cb860465d9dd4389648760599048d56f382a9e05c431ddf1e93497513fc6d21f/merged 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/9772f2d860971eb3f19d776615240978c60fc60a6b23215c3653e11e6ae9c55b/merged 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/0 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/048e109f117dcb565c69f5596238cd17ca948aecad6d90f819b9eb9901bbf747/merged 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/47516ef441c32a032fb0d803b5341fef846267226b85f32a78f6b6a3dc257229/merged 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/9e2762a086d09d81d825d10abe16e18c4b8a90db73dcf39593c3de2a6fc6ffcb/merged 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/b74c628f3b949cbbac1b6398ecaa6af4447ad840c557f71455d0594e3d49d9b1/merged 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/fb33f6b2de769041147303cc1c096df7b0b591936b85c47279341eb5bf473394/merged 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/7eef970bb1ea502f4def775692fe1b07e55604d335bf28438956169f65f6a7dd/merged 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/01ed206f1ea932edd37252b72a020b8e37aff753fdc6ffbf35ef8a637e8378ff/merged 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/6e1048e875105ee30e460d83f1beb92669ddde960c07b462a12a168ecae487f5/merged 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-21T08:44:45.509 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:44:45.510 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:44:45.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:45.510 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-21T08:44:45.523 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:2024-04-21T08:44:45.524+0000 7f8dd3035480 -1 init, newargv = 0x56433f459060 newargc=15 2024-04-21T08:44:45.523 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:ceph-fuse[72508]: starting ceph client 2024-04-21T08:44:45.529 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:ceph-fuse[72508]: starting fuse 2024-04-21T08:44:45.537 INFO:teuthology.orchestra.run.smithi082.stdout:115 2024-04-21T08:44:45.538 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-21T08:44:45.538 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-21T08:44:45.538 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-21T08:44:45.538 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-21T08:44:45.538 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-21T08:44:45.538 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-21T08:44:45.538 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-21T08:44:45.538 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-21T08:44:45.538 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-21T08:44:45.539 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-21T08:44:45.540 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-21T08:44:45.540 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.admin")) 2024-04-21T08:44:45.540 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-21T08:44:45.655 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.admin.72508.asok 2024-04-21T08:44:45.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:44:45.665 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72508.asok 2024-04-21T08:44:45.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:45.665 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72508.asok status 2024-04-21T08:44:45.797 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:44:45.798 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-21T08:44:45.798 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "a630765c1f93c6e61d9b40c4b76437611d5ef29e", 2024-04-21T08:44:45.798 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:44:45.798 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "admin", 2024-04-21T08:44:45.798 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-21T08:44:45.798 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-21T08:44:45.798 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "72508", 2024-04-21T08:44:45.798 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-21T08:44:45.798 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:44:45.798 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-21T08:44:45.798 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-21T08:44:45.798 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 14787, 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 14787 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2806850555 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-21T08:44:45.799 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2806850555 2024-04-21T08:44:45.800 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:44:45.800 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.14787 192.168.0.1:0/2806850555", 2024-04-21T08:44:45.800 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/2806850555", 2024-04-21T08:44:45.800 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-21T08:44:45.800 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 16, 2024-04-21T08:44:45.800 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 89, 2024-04-21T08:44:45.800 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-21T08:44:45.800 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-21T08:44:45.801 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-21T08:44:45.801 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:44:45.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:45.807 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-21T08:44:45.807 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:44:45.822 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-21T08:44:45.822 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:44:45.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:45.823 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:44:45.887 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-21T08:44:45.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T08:44:45.990 INFO:teuthology.orchestra.run.smithi082.stdout:# file: . 2024-04-21T08:44:45.991 INFO:teuthology.orchestra.run.smithi082.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-21T08:44:45.991 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:45.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:44:45.997 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:44:45.997 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:44:46.013 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:44:46.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.150+0000 7fdd596c0640 1 -- 172.21.15.82:0/1218884875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd54103180 msgr2=0x7fdd54103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.150+0000 7fdd596c0640 1 --2- 172.21.15.82:0/1218884875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd54103180 0x7fdd54103600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009900 tx=0x7fdd3c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:46.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.150+0000 7fdd596c0640 1 -- 172.21.15.82:0/1218884875 shutdown_connections 2024-04-21T08:44:46.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.150+0000 7fdd596c0640 1 --2- 172.21.15.82:0/1218884875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd54103b40 0x7fdd5410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.150+0000 7fdd596c0640 1 --2- 172.21.15.82:0/1218884875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd54103180 0x7fdd54103600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.150+0000 7fdd596c0640 1 --2- 172.21.15.82:0/1218884875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd54101f90 0x7fdd541023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.150+0000 7fdd596c0640 1 -- 172.21.15.82:0/1218884875 >> 172.21.15.82:0/1218884875 conn(0x7fdd540fd500 msgr2=0x7fdd540ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:46.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.150+0000 7fdd596c0640 1 -- 172.21.15.82:0/1218884875 shutdown_connections 2024-04-21T08:44:46.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.150+0000 7fdd596c0640 1 -- 172.21.15.82:0/1218884875 wait complete. 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd596c0640 1 Processor -- start 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd596c0640 1 -- start start 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd596c0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd54101f90 0x7fdd54198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd596c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd54103180 0x7fdd54198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd596c0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd54103b40 0x7fdd5419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd596c0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5410dbb0 con 0x7fdd54103180 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd596c0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5410da30 con 0x7fdd54103b40 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd596c0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5410dd30 con 0x7fdd54101f90 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd4bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd54103180 0x7fdd54198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd4bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd54103180 0x7fdd54198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35064/0 (socket says 172.21.15.82:35064) 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd4bfff640 1 -- 172.21.15.82:0/390449377 learned_addr learned my addr 172.21.15.82:0/390449377 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd537fe640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd54103b40 0x7fdd5419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd52ffd640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd54101f90 0x7fdd54198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd4bfff640 1 -- 172.21.15.82:0/390449377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd54101f90 msgr2=0x7fdd54198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd4bfff640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd54101f90 0x7fdd54198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd4bfff640 1 -- 172.21.15.82:0/390449377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd54103b40 msgr2=0x7fdd5419f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd4bfff640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd54103b40 0x7fdd5419f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd4bfff640 1 -- 172.21.15.82:0/390449377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5419ff50 con 0x7fdd54103180 2024-04-21T08:44:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.151+0000 7fdd52ffd640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd54101f90 0x7fdd54198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.152+0000 7fdd4bfff640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd54103180 0x7fdd54198870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c036040 tx=0x7fdd3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.152+0000 7fdd50ff9640 1 -- 172.21.15.82:0/390449377 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c043020 con 0x7fdd54103180 2024-04-21T08:44:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.152+0000 7fdd596c0640 1 -- 172.21.15.82:0/390449377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd541a01e0 con 0x7fdd54103180 2024-04-21T08:44:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.152+0000 7fdd50ff9640 1 -- 172.21.15.82:0/390449377 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3c0054c0 con 0x7fdd54103180 2024-04-21T08:44:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.152+0000 7fdd50ff9640 1 -- 172.21.15.82:0/390449377 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c003db0 con 0x7fdd54103180 2024-04-21T08:44:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.152+0000 7fdd596c0640 1 -- 172.21.15.82:0/390449377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd541a07b0 con 0x7fdd54103180 2024-04-21T08:44:46.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.153+0000 7fdd596c0640 1 -- 172.21.15.82:0/390449377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd54109320 con 0x7fdd54103180 2024-04-21T08:44:46.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.154+0000 7fdd50ff9640 1 -- 172.21.15.82:0/390449377 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdd3c046020 con 0x7fdd54103180 2024-04-21T08:44:46.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.154+0000 7fdd50ff9640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdd400a3bb0 0x7fdd400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:46.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.154+0000 7fdd50ff9640 1 -- 172.21.15.82:0/390449377 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fdd3c0f2b20 con 0x7fdd54103180 2024-04-21T08:44:46.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.154+0000 7fdd52ffd640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdd400a3bb0 0x7fdd400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:46.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.155+0000 7fdd52ffd640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdd400a3bb0 0x7fdd400a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd44006fd0 tx=0x7fdd44008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:46.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.159+0000 7fdd50ff9640 1 -- 172.21.15.82:0/390449377 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd3c04d020 con 0x7fdd54103180 2024-04-21T08:44:46.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.337+0000 7fdd596c0640 1 -- 172.21.15.82:0/390449377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd54062b50 con 0x7fdd54103180 2024-04-21T08:44:46.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.338+0000 7fdd50ff9640 1 -- 172.21.15.82:0/390449377 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24140 (secure 0 0 0) 0x7fdd3c0bf2c0 con 0x7fdd54103180 2024-04-21T08:44:46.338 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:46.339 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":89,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:44:38.858333+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:44:46.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.340+0000 7fdd596c0640 1 -- 172.21.15.82:0/390449377 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdd400a3bb0 msgr2=0x7fdd400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.340+0000 7fdd596c0640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdd400a3bb0 0x7fdd400a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd44006fd0 tx=0x7fdd44008040 comp rx=0 tx=0).stop 2024-04-21T08:44:46.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.340+0000 7fdd596c0640 1 -- 172.21.15.82:0/390449377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd54103180 msgr2=0x7fdd54198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.340+0000 7fdd596c0640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd54103180 0x7fdd54198870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c036040 tx=0x7fdd3c0042f0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.341+0000 7fdd596c0640 1 -- 172.21.15.82:0/390449377 shutdown_connections 2024-04-21T08:44:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.341+0000 7fdd596c0640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdd400a3bb0 0x7fdd400a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.341+0000 7fdd596c0640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd54103b40 0x7fdd5419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.341+0000 7fdd596c0640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd54103180 0x7fdd54198870 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.341+0000 7fdd596c0640 1 --2- 172.21.15.82:0/390449377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd54101f90 0x7fdd54198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.341+0000 7fdd596c0640 1 -- 172.21.15.82:0/390449377 >> 172.21.15.82:0/390449377 conn(0x7fdd540fd500 msgr2=0x7fdd541064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.341+0000 7fdd596c0640 1 -- 172.21.15.82:0/390449377 shutdown_connections 2024-04-21T08:44:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.341+0000 7fdd596c0640 1 -- 172.21.15.82:0/390449377 wait complete. 2024-04-21T08:44:46.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:44:46.352 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:44:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.455+0000 7f41c9c11640 1 -- 172.21.15.82:0/2007050609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c4107c90 msgr2=0x7f41c410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.455+0000 7f41c9c11640 1 --2- 172.21.15.82:0/2007050609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c4107c90 0x7f41c410a0e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009ca0 tx=0x7f41bc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.455+0000 7f41c9c11640 1 -- 172.21.15.82:0/2007050609 shutdown_connections 2024-04-21T08:44:46.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.455+0000 7f41c9c11640 1 --2- 172.21.15.82:0/2007050609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c4107c90 0x7f41c410a0e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.455+0000 7f41c9c11640 1 --2- 172.21.15.82:0/2007050609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41c4105360 0x7f41c4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.455+0000 7f41c9c11640 1 --2- 172.21.15.82:0/2007050609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41c4102a30 0x7f41c4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.455+0000 7f41c9c11640 1 -- 172.21.15.82:0/2007050609 >> 172.21.15.82:0/2007050609 conn(0x7f41c40fc440 msgr2=0x7f41c40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.455+0000 7f41c9c11640 1 -- 172.21.15.82:0/2007050609 shutdown_connections 2024-04-21T08:44:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.455+0000 7f41c9c11640 1 -- 172.21.15.82:0/2007050609 wait complete. 2024-04-21T08:44:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c9c11640 1 Processor -- start 2024-04-21T08:44:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c9c11640 1 -- start start 2024-04-21T08:44:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c9c11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41c4102a30 0x7f41c41023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c9c11640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41c4105360 0x7f41c41009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c9c11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c4107c90 0x7f41c4100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c9c11640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410db50 con 0x7f41c4107c90 2024-04-21T08:44:46.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c9c11640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410d9d0 con 0x7f41c4105360 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c9c11640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41c410dcd0 con 0x7f41c4102a30 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c4107c90 0x7f41c4100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c4107c90 0x7f41c4100f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35076/0 (socket says 172.21.15.82:35076) 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c3fff640 1 -- 172.21.15.82:0/3310454969 learned_addr learned my addr 172.21.15.82:0/3310454969 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c2ffd640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41c4105360 0x7f41c41009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.456+0000 7f41c37fe640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41c4102a30 0x7f41c41023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.457+0000 7f41c3fff640 1 -- 172.21.15.82:0/3310454969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41c4102a30 msgr2=0x7f41c41023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.457+0000 7f41c3fff640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41c4102a30 0x7f41c41023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.457+0000 7f41c3fff640 1 -- 172.21.15.82:0/3310454969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41c4105360 msgr2=0x7f41c41009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.457+0000 7f41c3fff640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41c4105360 0x7f41c41009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.457+0000 7f41c3fff640 1 -- 172.21.15.82:0/3310454969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c41017b0 con 0x7f41c4107c90 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.457+0000 7f41c3fff640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c4107c90 0x7f41c4100f20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f41bc02c700 tx=0x7f41bc034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.457+0000 7f41c0ff9640 1 -- 172.21.15.82:0/3310454969 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc039040 con 0x7f41c4107c90 2024-04-21T08:44:46.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.457+0000 7f41c0ff9640 1 -- 172.21.15.82:0/3310454969 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41bc044070 con 0x7f41c4107c90 2024-04-21T08:44:46.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.457+0000 7f41c0ff9640 1 -- 172.21.15.82:0/3310454969 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc0042a0 con 0x7f41c4107c90 2024-04-21T08:44:46.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.457+0000 7f41c9c11640 1 -- 172.21.15.82:0/3310454969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c41a22e0 con 0x7f41c4107c90 2024-04-21T08:44:46.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.457+0000 7f41c9c11640 1 -- 172.21.15.82:0/3310454969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c41a2880 con 0x7f41c4107c90 2024-04-21T08:44:46.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.458+0000 7f41c9c11640 1 -- 172.21.15.82:0/3310454969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c40fdf30 con 0x7f41c4107c90 2024-04-21T08:44:46.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.459+0000 7f41c0ff9640 1 -- 172.21.15.82:0/3310454969 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f41bc0325c0 con 0x7f41c4107c90 2024-04-21T08:44:46.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.459+0000 7f41c0ff9640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f41ac0a3c30 0x7f41ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:46.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.460+0000 7f41c37fe640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f41ac0a3c30 0x7f41ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:46.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.460+0000 7f41c0ff9640 1 -- 172.21.15.82:0/3310454969 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f41bc0f29f0 con 0x7f41c4107c90 2024-04-21T08:44:46.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.460+0000 7f41c37fe640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f41ac0a3c30 0x7f41ac0a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f41b8005fd0 tx=0x7f41b8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:46.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.464+0000 7f41c0ff9640 1 -- 172.21.15.82:0/3310454969 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41bc0bf190 con 0x7f41c4107c90 2024-04-21T08:44:46.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:46 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/390449377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:46.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.648+0000 7f41c9c11640 1 -- 172.21.15.82:0/3310454969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f41c4062b50 con 0x7f41c4107c90 2024-04-21T08:44:46.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.648+0000 7f41c0ff9640 1 -- 172.21.15.82:0/3310454969 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f41bc0bea90 con 0x7f41c4107c90 2024-04-21T08:44:46.648 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:44:46.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.650+0000 7f41c9c11640 1 -- 172.21.15.82:0/3310454969 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f41ac0a3c30 msgr2=0x7f41ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.650+0000 7f41c9c11640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f41ac0a3c30 0x7f41ac0a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f41b8005fd0 tx=0x7f41b8005850 comp rx=0 tx=0).stop 2024-04-21T08:44:46.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.650+0000 7f41c9c11640 1 -- 172.21.15.82:0/3310454969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c4107c90 msgr2=0x7f41c4100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.650+0000 7f41c9c11640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c4107c90 0x7f41c4100f20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f41bc02c700 tx=0x7f41bc034760 comp rx=0 tx=0).stop 2024-04-21T08:44:46.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.650+0000 7f41c9c11640 1 -- 172.21.15.82:0/3310454969 shutdown_connections 2024-04-21T08:44:46.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.650+0000 7f41c9c11640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f41ac0a3c30 0x7f41ac0a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.650+0000 7f41c9c11640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c4107c90 0x7f41c4100f20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.650+0000 7f41c9c11640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41c4105360 0x7f41c41009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.650+0000 7f41c9c11640 1 --2- 172.21.15.82:0/3310454969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41c4102a30 0x7f41c41023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.650+0000 7f41c9c11640 1 -- 172.21.15.82:0/3310454969 >> 172.21.15.82:0/3310454969 conn(0x7f41c40fc440 msgr2=0x7f41c4103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:46.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.650+0000 7f41c9c11640 1 -- 172.21.15.82:0/3310454969 shutdown_connections 2024-04-21T08:44:46.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.650+0000 7f41c9c11640 1 -- 172.21.15.82:0/3310454969 wait complete. 2024-04-21T08:44:46.660 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:44:46.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:46 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/390449377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:46.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.762+0000 7f0c8886d640 1 -- 172.21.15.82:0/3089691208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c84074930 msgr2=0x7f0c84074d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.762+0000 7f0c8886d640 1 --2- 172.21.15.82:0/3089691208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c84074930 0x7f0c84074d90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009900 tx=0x7f0c7802bcd0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.762+0000 7f0c8886d640 1 -- 172.21.15.82:0/3089691208 shutdown_connections 2024-04-21T08:44:46.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.762+0000 7f0c8886d640 1 --2- 172.21.15.82:0/3089691208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c84109e60 0x7f0c8410c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.762+0000 7f0c8886d640 1 --2- 172.21.15.82:0/3089691208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c84074930 0x7f0c84074d90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.762+0000 7f0c8886d640 1 --2- 172.21.15.82:0/3089691208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c84075f10 0x7f0c84074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.762+0000 7f0c8886d640 1 -- 172.21.15.82:0/3089691208 >> 172.21.15.82:0/3089691208 conn(0x7f0c840fd520 msgr2=0x7f0c840ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.762+0000 7f0c8886d640 1 -- 172.21.15.82:0/3089691208 shutdown_connections 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.762+0000 7f0c8886d640 1 -- 172.21.15.82:0/3089691208 wait complete. 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.763+0000 7f0c8886d640 1 Processor -- start 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.763+0000 7f0c8886d640 1 -- start start 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.763+0000 7f0c8886d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c84074930 0x7f0c8419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.763+0000 7f0c8886d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c84075f10 0x7f0c8419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.763+0000 7f0c8886d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c84109e60 0x7f0c841a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.763+0000 7f0c8886d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8410fd00 con 0x7f0c84075f10 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.763+0000 7f0c8886d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8410fb80 con 0x7f0c84074930 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.763+0000 7f0c8886d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8410fe80 con 0x7f0c84109e60 2024-04-21T08:44:46.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.763+0000 7f0c81d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c84075f10 0x7f0c8419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.763+0000 7f0c81d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c84075f10 0x7f0c8419a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35080/0 (socket says 172.21.15.82:35080) 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.763+0000 7f0c81d9b640 1 -- 172.21.15.82:0/595483138 learned_addr learned my addr 172.21.15.82:0/595483138 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.763+0000 7f0c8259c640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c84074930 0x7f0c8419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c82d9d640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c84109e60 0x7f0c841a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c81d9b640 1 -- 172.21.15.82:0/595483138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c84109e60 msgr2=0x7f0c841a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c81d9b640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c84109e60 0x7f0c841a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c81d9b640 1 -- 172.21.15.82:0/595483138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c84074930 msgr2=0x7f0c8419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c81d9b640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c84074930 0x7f0c8419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c81d9b640 1 -- 172.21.15.82:0/595483138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c841a2000 con 0x7f0c84075f10 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c82d9d640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c84109e60 0x7f0c841a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c81d9b640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c84075f10 0x7f0c8419a920 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0c78006010 tx=0x7f0c7802d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c737fe640 1 -- 172.21.15.82:0/595483138 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78043020 con 0x7f0c84075f10 2024-04-21T08:44:46.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c737fe640 1 -- 172.21.15.82:0/595483138 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c7802dd90 con 0x7f0c84075f10 2024-04-21T08:44:46.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c737fe640 1 -- 172.21.15.82:0/595483138 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78003bb0 con 0x7f0c84075f10 2024-04-21T08:44:46.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c8886d640 1 -- 172.21.15.82:0/595483138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c841a2290 con 0x7f0c84075f10 2024-04-21T08:44:46.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.764+0000 7f0c8886d640 1 -- 172.21.15.82:0/595483138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c841a27e0 con 0x7f0c84075f10 2024-04-21T08:44:46.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.765+0000 7f0c8886d640 1 -- 172.21.15.82:0/595483138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c84108c20 con 0x7f0c84075f10 2024-04-21T08:44:46.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.766+0000 7f0c737fe640 1 -- 172.21.15.82:0/595483138 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0c78046020 con 0x7f0c84075f10 2024-04-21T08:44:46.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.767+0000 7f0c737fe640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0c6c0a3bb0 0x7f0c6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:46.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.767+0000 7f0c8259c640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0c6c0a3bb0 0x7f0c6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:46.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.767+0000 7f0c737fe640 1 -- 172.21.15.82:0/595483138 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f0c780f2f90 con 0x7f0c84075f10 2024-04-21T08:44:46.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.767+0000 7f0c8259c640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0c6c0a3bb0 0x7f0c6c0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c74005fd0 tx=0x7f0c74009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:46.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.770+0000 7f0c737fe640 1 -- 172.21.15.82:0/595483138 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c7804d020 con 0x7f0c84075f10 2024-04-21T08:44:46.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:46 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/390449377' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:46.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.946+0000 7f0c8886d640 1 -- 172.21.15.82:0/595483138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0c8402d830 con 0x7f0c84075f10 2024-04-21T08:44:46.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.947+0000 7f0c737fe640 1 -- 172.21.15.82:0/595483138 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f0c780bf730 con 0x7f0c84075f10 2024-04-21T08:44:46.946 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:44:46.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.948+0000 7f0c8886d640 1 -- 172.21.15.82:0/595483138 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0c6c0a3bb0 msgr2=0x7f0c6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.948+0000 7f0c8886d640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0c6c0a3bb0 0x7f0c6c0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c74005fd0 tx=0x7f0c74009760 comp rx=0 tx=0).stop 2024-04-21T08:44:46.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.948+0000 7f0c8886d640 1 -- 172.21.15.82:0/595483138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c84075f10 msgr2=0x7f0c8419a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:46.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.948+0000 7f0c8886d640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c84075f10 0x7f0c8419a920 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0c78006010 tx=0x7f0c7802d990 comp rx=0 tx=0).stop 2024-04-21T08:44:46.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.949+0000 7f0c8886d640 1 -- 172.21.15.82:0/595483138 shutdown_connections 2024-04-21T08:44:46.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.949+0000 7f0c8886d640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0c6c0a3bb0 0x7f0c6c0a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.949+0000 7f0c8886d640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c84109e60 0x7f0c841a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.949+0000 7f0c8886d640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c84075f10 0x7f0c8419a920 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.949+0000 7f0c8886d640 1 --2- 172.21.15.82:0/595483138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c84074930 0x7f0c8419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:46.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.949+0000 7f0c8886d640 1 -- 172.21.15.82:0/595483138 >> 172.21.15.82:0/595483138 conn(0x7f0c840fd520 msgr2=0x7f0c84105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:46.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.949+0000 7f0c8886d640 1 -- 172.21.15.82:0/595483138 shutdown_connections 2024-04-21T08:44:46.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:46.949+0000 7f0c8886d640 1 -- 172.21.15.82:0/595483138 wait complete. 2024-04-21T08:44:46.959 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:44:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.060+0000 7f3042552640 1 -- 172.21.15.82:0/594515764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f303c103b50 msgr2=0x7f303c10a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.060+0000 7f3042552640 1 --2- 172.21.15.82:0/594515764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f303c103b50 0x7f303c10a3e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3034009ca0 tx=0x7f303402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.060+0000 7f3042552640 1 -- 172.21.15.82:0/594515764 shutdown_connections 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.060+0000 7f3042552640 1 --2- 172.21.15.82:0/594515764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f303c103b50 0x7f303c10a3e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.060+0000 7f3042552640 1 --2- 172.21.15.82:0/594515764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f303c103190 0x7f303c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.060+0000 7f3042552640 1 --2- 172.21.15.82:0/594515764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f303c101fa0 0x7f303c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.060+0000 7f3042552640 1 -- 172.21.15.82:0/594515764 >> 172.21.15.82:0/594515764 conn(0x7f303c0fd530 msgr2=0x7f303c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.060+0000 7f3042552640 1 -- 172.21.15.82:0/594515764 shutdown_connections 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.060+0000 7f3042552640 1 -- 172.21.15.82:0/594515764 wait complete. 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.061+0000 7f3042552640 1 Processor -- start 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.061+0000 7f3042552640 1 -- start start 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.061+0000 7f3042552640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f303c101fa0 0x7f303c075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.061+0000 7f3042552640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f303c103190 0x7f303c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.061+0000 7f3042552640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f303c103b50 0x7f303c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.061+0000 7f3042552640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f303c10dae0 con 0x7f303c103190 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.061+0000 7f3042552640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f303c10d960 con 0x7f303c101fa0 2024-04-21T08:44:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.061+0000 7f3042552640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f303c10dc60 con 0x7f303c103b50 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.061+0000 7f303b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f303c103190 0x7f303c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.061+0000 7f303b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f303c103190 0x7f303c074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35106/0 (socket says 172.21.15.82:35106) 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.061+0000 7f303b7fe640 1 -- 172.21.15.82:0/1254940377 learned_addr learned my addr 172.21.15.82:0/1254940377 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.061+0000 7f3040aef640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f303c103b50 0x7f303c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f303bfff640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f303c101fa0 0x7f303c075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f303b7fe640 1 -- 172.21.15.82:0/1254940377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f303c103b50 msgr2=0x7f303c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f303b7fe640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f303c103b50 0x7f303c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f303b7fe640 1 -- 172.21.15.82:0/1254940377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f303c101fa0 msgr2=0x7f303c075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f303b7fe640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f303c101fa0 0x7f303c075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f303b7fe640 1 -- 172.21.15.82:0/1254940377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c075110 con 0x7f303c103190 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f3040aef640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f303c103b50 0x7f303c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f303b7fe640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f303c103190 0x7f303c074340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f303000e9f0 tx=0x7f303000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f30397fa640 1 -- 172.21.15.82:0/1254940377 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303000cd80 con 0x7f303c103190 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f30397fa640 1 -- 172.21.15.82:0/1254940377 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30300108e0 con 0x7f303c103190 2024-04-21T08:44:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f30397fa640 1 -- 172.21.15.82:0/1254940377 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030018980 con 0x7f303c103190 2024-04-21T08:44:47.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f3042552640 1 -- 172.21.15.82:0/1254940377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303c1a22e0 con 0x7f303c103190 2024-04-21T08:44:47.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.062+0000 7f3042552640 1 -- 172.21.15.82:0/1254940377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303c1a2830 con 0x7f303c103190 2024-04-21T08:44:47.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.063+0000 7f3042552640 1 -- 172.21.15.82:0/1254940377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f303c109330 con 0x7f303c103190 2024-04-21T08:44:47.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.064+0000 7f30397fa640 1 -- 172.21.15.82:0/1254940377 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3030010430 con 0x7f303c103190 2024-04-21T08:44:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.065+0000 7f30397fa640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f30240a3bb0 0x7f30240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.065+0000 7f303bfff640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f30240a3bb0 0x7f30240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.065+0000 7f30397fa640 1 -- 172.21.15.82:0/1254940377 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f30300d0a90 con 0x7f303c103190 2024-04-21T08:44:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.065+0000 7f303bfff640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f30240a3bb0 0x7f30240a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f302c005fd0 tx=0x7f302c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:47.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.069+0000 7f30397fa640 1 -- 172.21.15.82:0/1254940377 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f303009d230 con 0x7f303c103190 2024-04-21T08:44:47.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.247+0000 7f3042552640 1 -- 172.21.15.82:0/1254940377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f303c062b50 con 0x7f303c103190 2024-04-21T08:44:47.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.247+0000 7f30397fa640 1 -- 172.21.15.82:0/1254940377 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f303009cb30 con 0x7f303c103190 2024-04-21T08:44:47.247 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:44:47.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.249+0000 7f3042552640 1 -- 172.21.15.82:0/1254940377 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f30240a3bb0 msgr2=0x7f30240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.249+0000 7f3042552640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f30240a3bb0 0x7f30240a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f302c005fd0 tx=0x7f302c005850 comp rx=0 tx=0).stop 2024-04-21T08:44:47.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.249+0000 7f3042552640 1 -- 172.21.15.82:0/1254940377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f303c103190 msgr2=0x7f303c074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.249+0000 7f3042552640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f303c103190 0x7f303c074340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f303000e9f0 tx=0x7f303000eeb0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.249+0000 7f3042552640 1 -- 172.21.15.82:0/1254940377 shutdown_connections 2024-04-21T08:44:47.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.249+0000 7f3042552640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f30240a3bb0 0x7f30240a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.249+0000 7f3042552640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f303c103b50 0x7f303c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.249+0000 7f3042552640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f303c103190 0x7f303c074340 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.249+0000 7f3042552640 1 --2- 172.21.15.82:0/1254940377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f303c101fa0 0x7f303c075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.249+0000 7f3042552640 1 -- 172.21.15.82:0/1254940377 >> 172.21.15.82:0/1254940377 conn(0x7f303c0fd530 msgr2=0x7f303c106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:47.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.249+0000 7f3042552640 1 -- 172.21.15.82:0/1254940377 shutdown_connections 2024-04-21T08:44:47.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.249+0000 7f3042552640 1 -- 172.21.15.82:0/1254940377 wait complete. 2024-04-21T08:44:47.260 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:44:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.361+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/1151517861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce0103b40 msgr2=0x7f9ce010a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.361+0000 7f9ce7bb2640 1 --2- 172.21.15.82:0/1151517861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce0103b40 0x7f9ce010a3d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009ca0 tx=0x7f9cd802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.361+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/1151517861 shutdown_connections 2024-04-21T08:44:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.361+0000 7f9ce7bb2640 1 --2- 172.21.15.82:0/1151517861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce0103b40 0x7f9ce010a3d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.361+0000 7f9ce7bb2640 1 --2- 172.21.15.82:0/1151517861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ce0103180 0x7f9ce0103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.361+0000 7f9ce7bb2640 1 --2- 172.21.15.82:0/1151517861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ce0101f90 0x7f9ce01023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.361+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/1151517861 >> 172.21.15.82:0/1151517861 conn(0x7f9ce00fd500 msgr2=0x7f9ce00ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.361+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/1151517861 shutdown_connections 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.361+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/1151517861 wait complete. 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.361+0000 7f9ce7bb2640 1 Processor -- start 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.361+0000 7f9ce7bb2640 1 -- start start 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce7bb2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce0101f90 0x7f9ce0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce7bb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ce0103180 0x7f9ce0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce7bb2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ce0103b40 0x7f9ce019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce7bb2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce010db10 con 0x7f9ce0101f90 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce7bb2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce010d990 con 0x7f9ce0103180 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce7bb2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce010dc90 con 0x7f9ce0103b40 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce594e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce0101f90 0x7f9ce0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce594e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce0101f90 0x7f9ce0198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35124/0 (socket says 172.21.15.82:35124) 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce594e640 1 -- 172.21.15.82:0/943039217 learned_addr learned my addr 172.21.15.82:0/943039217 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce514d640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ce0103180 0x7f9ce0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce614f640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ce0103b40 0x7f9ce019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce594e640 1 -- 172.21.15.82:0/943039217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ce0103b40 msgr2=0x7f9ce019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce594e640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ce0103b40 0x7f9ce019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce594e640 1 -- 172.21.15.82:0/943039217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ce0103180 msgr2=0x7f9ce0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce594e640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ce0103180 0x7f9ce0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce594e640 1 -- 172.21.15.82:0/943039217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce019fe30 con 0x7f9ce0101f90 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce514d640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ce0103180 0x7f9ce0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce614f640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ce0103b40 0x7f9ce019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce594e640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce0101f90 0x7f9ce0198210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00c970 tx=0x7f9cdc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9cd6ffd640 1 -- 172.21.15.82:0/943039217 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc004220 con 0x7f9ce0101f90 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9cd6ffd640 1 -- 172.21.15.82:0/943039217 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cdc0043c0 con 0x7f9ce0101f90 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/943039217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce01a0120 con 0x7f9ce0101f90 2024-04-21T08:44:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9cd6ffd640 1 -- 172.21.15.82:0/943039217 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00f450 con 0x7f9ce0101f90 2024-04-21T08:44:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.362+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/943039217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce01a06c0 con 0x7f9ce0101f90 2024-04-21T08:44:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.363+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/943039217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce0109300 con 0x7f9ce0101f90 2024-04-21T08:44:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.364+0000 7f9cd6ffd640 1 -- 172.21.15.82:0/943039217 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9cdc003610 con 0x7f9ce0101f90 2024-04-21T08:44:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.365+0000 7f9cd6ffd640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9ccc0a3c30 0x7f9ccc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.365+0000 7f9cd6ffd640 1 -- 172.21.15.82:0/943039217 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9cdc0d0840 con 0x7f9ce0101f90 2024-04-21T08:44:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.365+0000 7f9ce514d640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9ccc0a3c30 0x7f9ccc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.365+0000 7f9ce514d640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9ccc0a3c30 0x7f9ccc0a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0006fd0 tx=0x7f9cd0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:47.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.369+0000 7f9cd6ffd640 1 -- 172.21.15.82:0/943039217 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9cdc014030 con 0x7f9ce0101f90 2024-04-21T08:44:47.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.546+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/943039217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9ce002d870 con 0x7f9ce0101f90 2024-04-21T08:44:47.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.546+0000 7f9cd6ffd640 1 -- 172.21.15.82:0/943039217 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f9cdc09d060 con 0x7f9ce0101f90 2024-04-21T08:44:47.546 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:44:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.548+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/943039217 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9ccc0a3c30 msgr2=0x7f9ccc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.548+0000 7f9ce7bb2640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9ccc0a3c30 0x7f9ccc0a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0006fd0 tx=0x7f9cd0008040 comp rx=0 tx=0).stop 2024-04-21T08:44:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.548+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/943039217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce0101f90 msgr2=0x7f9ce0198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.548+0000 7f9ce7bb2640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce0101f90 0x7f9ce0198210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00c970 tx=0x7f9cdc00ce30 comp rx=0 tx=0).stop 2024-04-21T08:44:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.548+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/943039217 shutdown_connections 2024-04-21T08:44:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.548+0000 7f9ce7bb2640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9ccc0a3c30 0x7f9ccc0a60f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.548+0000 7f9ce7bb2640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ce0103b40 0x7f9ce019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.548+0000 7f9ce7bb2640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ce0103180 0x7f9ce0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.548+0000 7f9ce7bb2640 1 --2- 172.21.15.82:0/943039217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce0101f90 0x7f9ce0198210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.548+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/943039217 >> 172.21.15.82:0/943039217 conn(0x7f9ce00fd500 msgr2=0x7f9ce01064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.548+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/943039217 shutdown_connections 2024-04-21T08:44:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.548+0000 7f9ce7bb2640 1 -- 172.21.15.82:0/943039217 wait complete. 2024-04-21T08:44:47.557 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:47 smithi082 ceph-mon[27967]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-04-21T08:44:47.557 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:47 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3310454969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:44:47.558 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:47 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/595483138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:44:47.558 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:47 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1254940377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:44:47.559 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T08:44:47.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.658+0000 7fed537ca640 1 -- 172.21.15.82:0/2239223337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed4c101fa0 msgr2=0x7fed4c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.658+0000 7fed537ca640 1 --2- 172.21.15.82:0/2239223337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed4c101fa0 0x7fed4c1023c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fed48009900 tx=0x7fed4802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:47.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.658+0000 7fed537ca640 1 -- 172.21.15.82:0/2239223337 shutdown_connections 2024-04-21T08:44:47.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.658+0000 7fed537ca640 1 --2- 172.21.15.82:0/2239223337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed4c103b50 0x7fed4c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.658+0000 7fed537ca640 1 --2- 172.21.15.82:0/2239223337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed4c103190 0x7fed4c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.658+0000 7fed537ca640 1 --2- 172.21.15.82:0/2239223337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed4c101fa0 0x7fed4c1023c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.658+0000 7fed537ca640 1 -- 172.21.15.82:0/2239223337 >> 172.21.15.82:0/2239223337 conn(0x7fed4c0fd530 msgr2=0x7fed4c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.658+0000 7fed537ca640 1 -- 172.21.15.82:0/2239223337 shutdown_connections 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.658+0000 7fed537ca640 1 -- 172.21.15.82:0/2239223337 wait complete. 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed537ca640 1 Processor -- start 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed537ca640 1 -- start start 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed537ca640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed4c101fa0 0x7fed4c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed537ca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed4c103190 0x7fed4c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed537ca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed4c103b50 0x7fed4c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed537ca640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c10dae0 con 0x7fed4c101fa0 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed537ca640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c10d960 con 0x7fed4c103b50 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed537ca640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c10dc60 con 0x7fed4c103190 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed51566640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed4c101fa0 0x7fed4c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed51566640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed4c101fa0 0x7fed4c1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35148/0 (socket says 172.21.15.82:35148) 2024-04-21T08:44:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed51566640 1 -- 172.21.15.82:0/3766056047 learned_addr learned my addr 172.21.15.82:0/3766056047 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed50d65640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed4c103190 0x7fed4c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed51d67640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed4c103b50 0x7fed4c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed51566640 1 -- 172.21.15.82:0/3766056047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed4c103190 msgr2=0x7fed4c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed51566640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed4c103190 0x7fed4c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed51566640 1 -- 172.21.15.82:0/3766056047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed4c103b50 msgr2=0x7fed4c19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed51566640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed4c103b50 0x7fed4c19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.659+0000 7fed51566640 1 -- 172.21.15.82:0/3766056047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4c19fe10 con 0x7fed4c101fa0 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.660+0000 7fed51566640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed4c101fa0 0x7fed4c1981f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fed48000c00 tx=0x7fed48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.660+0000 7fed427fc640 1 -- 172.21.15.82:0/3766056047 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48043020 con 0x7fed4c101fa0 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.660+0000 7fed427fc640 1 -- 172.21.15.82:0/3766056047 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed48004aa0 con 0x7fed4c101fa0 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.660+0000 7fed537ca640 1 -- 172.21.15.82:0/3766056047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4c1a00a0 con 0x7fed4c101fa0 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.660+0000 7fed427fc640 1 -- 172.21.15.82:0/3766056047 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48003d00 con 0x7fed4c101fa0 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.660+0000 7fed537ca640 1 -- 172.21.15.82:0/3766056047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4c1a0640 con 0x7fed4c101fa0 2024-04-21T08:44:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.661+0000 7fed537ca640 1 -- 172.21.15.82:0/3766056047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed4c109330 con 0x7fed4c101fa0 2024-04-21T08:44:47.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.662+0000 7fed427fc640 1 -- 172.21.15.82:0/3766056047 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fed48046020 con 0x7fed4c101fa0 2024-04-21T08:44:47.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.662+0000 7fed427fc640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fed380a3c30 0x7fed380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:47.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.662+0000 7fed50d65640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fed380a3c30 0x7fed380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:47.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.662+0000 7fed427fc640 1 -- 172.21.15.82:0/3766056047 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fed480f2b80 con 0x7fed4c101fa0 2024-04-21T08:44:47.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.663+0000 7fed50d65640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fed380a3c30 0x7fed380a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed4c199700 tx=0x7fed3c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:47.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.667+0000 7fed427fc640 1 -- 172.21.15.82:0/3766056047 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed4804d020 con 0x7fed4c101fa0 2024-04-21T08:44:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:47 smithi134 ceph-mon[31772]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-04-21T08:44:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:47 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3310454969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:44:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:47 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/595483138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:44:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:47 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1254940377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:44:47.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.856+0000 7fed537ca640 1 -- 172.21.15.82:0/3766056047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fed4c02d5c0 con 0x7fed4c101fa0 2024-04-21T08:44:47.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.856+0000 7fed427fc640 1 -- 172.21.15.82:0/3766056047 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+590 (secure 0 0 0) 0x7fed480a9020 con 0x7fed4c101fa0 2024-04-21T08:44:47.856 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/4175378061 2024-04-22T08:37:26.227727+0000 2024-04-21T08:44:47.856 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2889165689 2024-04-22T08:37:26.227727+0000 2024-04-21T08:44:47.856 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/880382736 2024-04-22T08:37:26.227727+0000 2024-04-21T08:44:47.856 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3352705806 2024-04-22T08:37:26.227727+0000 2024-04-21T08:44:47.856 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/2710226711 2024-04-22T08:36:59.578707+0000 2024-04-21T08:44:47.856 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/4224740504 2024-04-22T08:36:59.578707+0000 2024-04-21T08:44:47.856 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2873389149 2024-04-22T08:36:59.578707+0000 2024-04-21T08:44:47.856 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/880382736 2024-04-22T08:37:26.227727+0000 2024-04-21T08:44:47.856 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/2710226711 2024-04-22T08:36:59.578707+0000 2024-04-21T08:44:47.857 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3032518359 2024-04-22T08:36:59.578707+0000 2024-04-21T08:44:47.857 INFO:teuthology.orchestra.run.smithi082.stderr:listed 10 entries 2024-04-21T08:44:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.858+0000 7fed537ca640 1 -- 172.21.15.82:0/3766056047 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fed380a3c30 msgr2=0x7fed380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.858+0000 7fed537ca640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fed380a3c30 0x7fed380a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed4c199700 tx=0x7fed3c00b390 comp rx=0 tx=0).stop 2024-04-21T08:44:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.858+0000 7fed537ca640 1 -- 172.21.15.82:0/3766056047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed4c101fa0 msgr2=0x7fed4c1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.858+0000 7fed537ca640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed4c101fa0 0x7fed4c1981f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fed48000c00 tx=0x7fed48004270 comp rx=0 tx=0).stop 2024-04-21T08:44:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.858+0000 7fed537ca640 1 -- 172.21.15.82:0/3766056047 shutdown_connections 2024-04-21T08:44:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.858+0000 7fed537ca640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fed380a3c30 0x7fed380a60f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.858+0000 7fed537ca640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed4c103b50 0x7fed4c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.858+0000 7fed537ca640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed4c103190 0x7fed4c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.858+0000 7fed537ca640 1 --2- 172.21.15.82:0/3766056047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed4c101fa0 0x7fed4c1981f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:47.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.858+0000 7fed537ca640 1 -- 172.21.15.82:0/3766056047 >> 172.21.15.82:0/3766056047 conn(0x7fed4c0fd530 msgr2=0x7fed4c106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:47.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.858+0000 7fed537ca640 1 -- 172.21.15.82:0/3766056047 shutdown_connections 2024-04-21T08:44:47.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:47.858+0000 7fed537ca640 1 -- 172.21.15.82:0/3766056047 wait complete. 2024-04-21T08:44:47.868 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:44:47.868 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:44:47.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:47 smithi106 ceph-mon[31072]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-04-21T08:44:47.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:47 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3310454969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:44:47.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:47 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/595483138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:44:47.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:47 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1254940377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:44:47.885 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:44:48.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.022+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/1789091579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab01021e0 msgr2=0x7f5ab0108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.022+0000 7f5ab7ed2640 1 --2- 172.21.15.82:0/1789091579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab01021e0 0x7f5ab0108a70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009ca0 tx=0x7f5aa802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.022+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/1789091579 shutdown_connections 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.022+0000 7f5ab7ed2640 1 --2- 172.21.15.82:0/1789091579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab01021e0 0x7f5ab0108a70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.022+0000 7f5ab7ed2640 1 --2- 172.21.15.82:0/1789091579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab0101820 0x7f5ab0101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.022+0000 7f5ab7ed2640 1 --2- 172.21.15.82:0/1789091579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab0100630 0x7f5ab0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.022+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/1789091579 >> 172.21.15.82:0/1789091579 conn(0x7f5ab00fbba0 msgr2=0x7f5ab00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.022+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/1789091579 shutdown_connections 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.022+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/1789091579 wait complete. 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab7ed2640 1 Processor -- start 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab7ed2640 1 -- start start 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab7ed2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab0100630 0x7f5ab0198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab7ed2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab0101820 0x7f5ab01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab7ed2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab01021e0 0x7f5ab019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab7ed2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab010b8b0 con 0x7f5ab0101820 2024-04-21T08:44:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab7ed2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab010b730 con 0x7f5ab01021e0 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab7ed2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab010ba30 con 0x7f5ab0100630 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab546d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab0101820 0x7f5ab01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab546d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab0101820 0x7f5ab01986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35160/0 (socket says 172.21.15.82:35160) 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab546d640 1 -- 172.21.15.82:0/3355366718 learned_addr learned my addr 172.21.15.82:0/3355366718 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab646f640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab01021e0 0x7f5ab019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab5c6e640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab0100630 0x7f5ab0198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab546d640 1 -- 172.21.15.82:0/3355366718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab0100630 msgr2=0x7f5ab0198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab546d640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab0100630 0x7f5ab0198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab546d640 1 -- 172.21.15.82:0/3355366718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab01021e0 msgr2=0x7f5ab019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab546d640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab01021e0 0x7f5ab019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab546d640 1 -- 172.21.15.82:0/3355366718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab019fd80 con 0x7f5ab0101820 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.023+0000 7f5ab546d640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab0101820 0x7f5ab01986a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000ddc0 tx=0x7f5aa000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.024+0000 7f5aa6ffd640 1 -- 172.21.15.82:0/3355366718 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa001b070 con 0x7f5ab0101820 2024-04-21T08:44:48.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.024+0000 7f5aa6ffd640 1 -- 172.21.15.82:0/3355366718 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa000be40 con 0x7f5ab0101820 2024-04-21T08:44:48.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.024+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/3355366718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab01a0070 con 0x7f5ab0101820 2024-04-21T08:44:48.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.024+0000 7f5aa6ffd640 1 -- 172.21.15.82:0/3355366718 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0016a80 con 0x7f5ab0101820 2024-04-21T08:44:48.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.024+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/3355366718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab01a0610 con 0x7f5ab0101820 2024-04-21T08:44:48.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.025+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/3355366718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab01a08d0 con 0x7f5ab0101820 2024-04-21T08:44:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.026+0000 7f5aa6ffd640 1 -- 172.21.15.82:0/3355366718 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5aa001e070 con 0x7f5ab0101820 2024-04-21T08:44:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.027+0000 7f5aa6ffd640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5a9c0a3c30 0x7f5a9c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.027+0000 7f5ab5c6e640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5a9c0a3c30 0x7f5a9c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.027+0000 7f5aa6ffd640 1 -- 172.21.15.82:0/3355366718 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5aa00d02f0 con 0x7f5ab0101820 2024-04-21T08:44:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.027+0000 7f5ab5c6e640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5a9c0a3c30 0x7f5a9c0a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5aac005fd0 tx=0x7f5aac005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.031+0000 7f5aa6ffd640 1 -- 172.21.15.82:0/3355366718 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5aa009c440 con 0x7f5ab0101820 2024-04-21T08:44:48.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.212+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/3355366718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ab01a0aa0 con 0x7f5ab0101820 2024-04-21T08:44:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.213+0000 7f5aa6ffd640 1 -- 172.21.15.82:0/3355366718 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24140 (secure 0 0 0) 0x7f5aa00a0740 con 0x7f5ab0101820 2024-04-21T08:44:48.213 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:48.213 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":89,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:44:38.858333+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:44:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.215+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/3355366718 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5a9c0a3c30 msgr2=0x7f5a9c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.215+0000 7f5ab7ed2640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5a9c0a3c30 0x7f5a9c0a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5aac005fd0 tx=0x7f5aac005c70 comp rx=0 tx=0).stop 2024-04-21T08:44:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.215+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/3355366718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab0101820 msgr2=0x7f5ab01986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.215+0000 7f5ab7ed2640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab0101820 0x7f5ab01986a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000ddc0 tx=0x7f5aa000a620 comp rx=0 tx=0).stop 2024-04-21T08:44:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.215+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/3355366718 shutdown_connections 2024-04-21T08:44:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.215+0000 7f5ab7ed2640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5a9c0a3c30 0x7f5a9c0a60f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.215+0000 7f5ab7ed2640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab01021e0 0x7f5ab019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.215+0000 7f5ab7ed2640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab0101820 0x7f5ab01986a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.215+0000 7f5ab7ed2640 1 --2- 172.21.15.82:0/3355366718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab0100630 0x7f5ab0198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.215+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/3355366718 >> 172.21.15.82:0/3355366718 conn(0x7f5ab00fbba0 msgr2=0x7f5ab0106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.216+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/3355366718 shutdown_connections 2024-04-21T08:44:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.216+0000 7f5ab7ed2640 1 -- 172.21.15.82:0/3355366718 wait complete. 2024-04-21T08:44:48.226 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:44:48.227 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:44:48.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.328+0000 7f10b305c640 1 -- 172.21.15.82:0/1134137141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac105d60 msgr2=0x7f10ac10c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.328+0000 7f10b305c640 1 --2- 172.21.15.82:0/1134137141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac105d60 0x7f10ac10c5f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009ca0 tx=0x7f10a402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.328+0000 7f10b305c640 1 -- 172.21.15.82:0/1134137141 shutdown_connections 2024-04-21T08:44:48.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.328+0000 7f10b305c640 1 --2- 172.21.15.82:0/1134137141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac105d60 0x7f10ac10c5f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.328+0000 7f10b305c640 1 --2- 172.21.15.82:0/1134137141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10ac1053a0 0x7f10ac105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.328+0000 7f10b305c640 1 --2- 172.21.15.82:0/1134137141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac1041b0 0x7f10ac1045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.328+0000 7f10b305c640 1 -- 172.21.15.82:0/1134137141 >> 172.21.15.82:0/1134137141 conn(0x7f10ac0ff700 msgr2=0x7f10ac101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.328+0000 7f10b305c640 1 -- 172.21.15.82:0/1134137141 shutdown_connections 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b305c640 1 -- 172.21.15.82:0/1134137141 wait complete. 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b305c640 1 Processor -- start 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b305c640 1 -- start start 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b305c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac1041b0 0x7f10ac19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b305c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10ac1053a0 0x7f10ac19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b305c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac105d60 0x7f10ac1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b305c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac10fcf0 con 0x7f10ac105d60 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b305c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac10fb70 con 0x7f10ac1041b0 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b305c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac10fe70 con 0x7f10ac1053a0 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b15f9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac105d60 0x7f10ac1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b15f9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac105d60 0x7f10ac1a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35182/0 (socket says 172.21.15.82:35182) 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b15f9640 1 -- 172.21.15.82:0/655040271 learned_addr learned my addr 172.21.15.82:0/655040271 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:48.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b0df8640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac1041b0 0x7f10ac19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b15f9640 1 -- 172.21.15.82:0/655040271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10ac1053a0 msgr2=0x7f10ac19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b15f9640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10ac1053a0 0x7f10ac19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b15f9640 1 -- 172.21.15.82:0/655040271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac1041b0 msgr2=0x7f10ac19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b15f9640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac1041b0 0x7f10ac19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.329+0000 7f10b15f9640 1 -- 172.21.15.82:0/655040271 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac1a1fa0 con 0x7f10ac105d60 2024-04-21T08:44:48.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.330+0000 7f10b15f9640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac105d60 0x7f10ac1a1930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10a402c700 tx=0x7f10a4034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.330+0000 7f10a9ffb640 1 -- 172.21.15.82:0/655040271 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4039040 con 0x7f10ac105d60 2024-04-21T08:44:48.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.330+0000 7f10a9ffb640 1 -- 172.21.15.82:0/655040271 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a4044070 con 0x7f10ac105d60 2024-04-21T08:44:48.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.330+0000 7f10a9ffb640 1 -- 172.21.15.82:0/655040271 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a40042a0 con 0x7f10ac105d60 2024-04-21T08:44:48.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.330+0000 7f10b305c640 1 -- 172.21.15.82:0/655040271 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ac1a2230 con 0x7f10ac105d60 2024-04-21T08:44:48.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.330+0000 7f10b305c640 1 -- 172.21.15.82:0/655040271 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ac1a2720 con 0x7f10ac105d60 2024-04-21T08:44:48.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.331+0000 7f10b305c640 1 -- 172.21.15.82:0/655040271 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ac10b520 con 0x7f10ac105d60 2024-04-21T08:44:48.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.332+0000 7f10a9ffb640 1 -- 172.21.15.82:0/655040271 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f10a4003970 con 0x7f10ac105d60 2024-04-21T08:44:48.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.333+0000 7f10a9ffb640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f10940a3bb0 0x7f10940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.333+0000 7f10a9ffb640 1 -- 172.21.15.82:0/655040271 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f10a40f34c0 con 0x7f10ac105d60 2024-04-21T08:44:48.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.333+0000 7f10b0df8640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f10940a3bb0 0x7f10940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.333+0000 7f10b0df8640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f10940a3bb0 0x7f10940a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f10ac105200 tx=0x7f109c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.336+0000 7f10a9ffb640 1 -- 172.21.15.82:0/655040271 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10a40bfc60 con 0x7f10ac105d60 2024-04-21T08:44:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.517+0000 7f10b305c640 1 -- 172.21.15.82:0/655040271 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10ac02d830 con 0x7f10ac105d60 2024-04-21T08:44:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.517+0000 7f10a9ffb640 1 -- 172.21.15.82:0/655040271 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f10a40bf560 con 0x7f10ac105d60 2024-04-21T08:44:48.517 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:44:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.519+0000 7f10b305c640 1 -- 172.21.15.82:0/655040271 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f10940a3bb0 msgr2=0x7f10940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.519+0000 7f10b305c640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f10940a3bb0 0x7f10940a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f10ac105200 tx=0x7f109c00b390 comp rx=0 tx=0).stop 2024-04-21T08:44:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.519+0000 7f10b305c640 1 -- 172.21.15.82:0/655040271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac105d60 msgr2=0x7f10ac1a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.519+0000 7f10b305c640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac105d60 0x7f10ac1a1930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10a402c700 tx=0x7f10a4034760 comp rx=0 tx=0).stop 2024-04-21T08:44:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.520+0000 7f10b305c640 1 -- 172.21.15.82:0/655040271 shutdown_connections 2024-04-21T08:44:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.520+0000 7f10b305c640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f10940a3bb0 0x7f10940a6070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.520+0000 7f10b305c640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac105d60 0x7f10ac1a1930 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.520+0000 7f10b305c640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10ac1053a0 0x7f10ac19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.520+0000 7f10b305c640 1 --2- 172.21.15.82:0/655040271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac1041b0 0x7f10ac19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.520+0000 7f10b305c640 1 -- 172.21.15.82:0/655040271 >> 172.21.15.82:0/655040271 conn(0x7f10ac0ff700 msgr2=0x7f10ac1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.520+0000 7f10b305c640 1 -- 172.21.15.82:0/655040271 shutdown_connections 2024-04-21T08:44:48.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.520+0000 7f10b305c640 1 -- 172.21.15.82:0/655040271 wait complete. 2024-04-21T08:44:48.529 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:48 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/943039217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:44:48.529 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:48 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3766056047' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:44:48.529 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:48 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3355366718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:48.530 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:44:48.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.631+0000 7fdfca9a4640 1 -- 172.21.15.82:0/3372518620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfc4102a00 msgr2=0x7fdfc4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.631+0000 7fdfca9a4640 1 --2- 172.21.15.82:0/3372518620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfc4102a00 0x7fdfc4109320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009ca0 tx=0x7fdfbc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.631+0000 7fdfca9a4640 1 -- 172.21.15.82:0/3372518620 shutdown_connections 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.631+0000 7fdfca9a4640 1 --2- 172.21.15.82:0/3372518620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfc4102a00 0x7fdfc4109320 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.631+0000 7fdfca9a4640 1 --2- 172.21.15.82:0/3372518620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc4102040 0x7fdfc41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.631+0000 7fdfca9a4640 1 --2- 172.21.15.82:0/3372518620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc4100e60 0x7fdfc4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.631+0000 7fdfca9a4640 1 -- 172.21.15.82:0/3372518620 >> 172.21.15.82:0/3372518620 conn(0x7fdfc40fc420 msgr2=0x7fdfc40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.631+0000 7fdfca9a4640 1 -- 172.21.15.82:0/3372518620 shutdown_connections 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.631+0000 7fdfca9a4640 1 -- 172.21.15.82:0/3372518620 wait complete. 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.631+0000 7fdfca9a4640 1 Processor -- start 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.631+0000 7fdfca9a4640 1 -- start start 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfca9a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc4100e60 0x7fdfc406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfca9a4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc4102040 0x7fdfc419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfca9a4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfc4102a00 0x7fdfc419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfca9a4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc410ca10 con 0x7fdfc4102a00 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfca9a4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc410c890 con 0x7fdfc4102040 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfca9a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc410cb90 con 0x7fdfc4100e60 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfc8f41640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfc4102a00 0x7fdfc419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfc8f41640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfc4102a00 0x7fdfc419f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35192/0 (socket says 172.21.15.82:35192) 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfc8f41640 1 -- 172.21.15.82:0/4214733424 learned_addr learned my addr 172.21.15.82:0/4214733424 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfc37fe640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc4102040 0x7fdfc419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfc8f41640 1 -- 172.21.15.82:0/4214733424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc4100e60 msgr2=0x7fdfc406b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfc3fff640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc4100e60 0x7fdfc406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfc8f41640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc4100e60 0x7fdfc406b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfc8f41640 1 -- 172.21.15.82:0/4214733424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc4102040 msgr2=0x7fdfc419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfc8f41640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc4102040 0x7fdfc419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfc8f41640 1 -- 172.21.15.82:0/4214733424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc419fe00 con 0x7fdfc4102a00 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfc8f41640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfc4102a00 0x7fdfc419f4f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc02c700 tx=0x7fdfbc0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.632+0000 7fdfc17fa640 1 -- 172.21.15.82:0/4214733424 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc041020 con 0x7fdfc4102a00 2024-04-21T08:44:48.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.633+0000 7fdfc17fa640 1 -- 172.21.15.82:0/4214733424 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfbc034d60 con 0x7fdfc4102a00 2024-04-21T08:44:48.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.633+0000 7fdfca9a4640 1 -- 172.21.15.82:0/4214733424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc41a0030 con 0x7fdfc4102a00 2024-04-21T08:44:48.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.633+0000 7fdfc17fa640 1 -- 172.21.15.82:0/4214733424 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc039040 con 0x7fdfc4102a00 2024-04-21T08:44:48.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.633+0000 7fdfca9a4640 1 -- 172.21.15.82:0/4214733424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc41a0520 con 0x7fdfc4102a00 2024-04-21T08:44:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.633+0000 7fdfca9a4640 1 -- 172.21.15.82:0/4214733424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc4108280 con 0x7fdfc4102a00 2024-04-21T08:44:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.634+0000 7fdfc17fa640 1 -- 172.21.15.82:0/4214733424 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdfbc044020 con 0x7fdfc4102a00 2024-04-21T08:44:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.635+0000 7fdfc17fa640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfb00a3bb0 0x7fdfb00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.635+0000 7fdfc3fff640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfb00a3bb0 0x7fdfb00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.635+0000 7fdfc17fa640 1 -- 172.21.15.82:0/4214733424 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fdfbc0f25f0 con 0x7fdfc4102a00 2024-04-21T08:44:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.636+0000 7fdfc3fff640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfb00a3bb0 0x7fdfb00a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdfac005fd0 tx=0x7fdfac005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.640+0000 7fdfc17fa640 1 -- 172.21.15.82:0/4214733424 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfbc04b020 con 0x7fdfc4102a00 2024-04-21T08:44:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:48 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/943039217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:44:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:48 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3766056047' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:44:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:48 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3355366718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:48.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.817+0000 7fdfca9a4640 1 -- 172.21.15.82:0/4214733424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdfc4062b50 con 0x7fdfc4102a00 2024-04-21T08:44:48.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.818+0000 7fdfc17fa640 1 -- 172.21.15.82:0/4214733424 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fdfbc0bed90 con 0x7fdfc4102a00 2024-04-21T08:44:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:44:48.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.820+0000 7fdfca9a4640 1 -- 172.21.15.82:0/4214733424 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfb00a3bb0 msgr2=0x7fdfb00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.820+0000 7fdfca9a4640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfb00a3bb0 0x7fdfb00a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdfac005fd0 tx=0x7fdfac005850 comp rx=0 tx=0).stop 2024-04-21T08:44:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.820+0000 7fdfca9a4640 1 -- 172.21.15.82:0/4214733424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfc4102a00 msgr2=0x7fdfc419f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.820+0000 7fdfca9a4640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfc4102a00 0x7fdfc419f4f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc02c700 tx=0x7fdfbc0093f0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.820+0000 7fdfca9a4640 1 -- 172.21.15.82:0/4214733424 shutdown_connections 2024-04-21T08:44:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.820+0000 7fdfca9a4640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfb00a3bb0 0x7fdfb00a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.820+0000 7fdfca9a4640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfc4102a00 0x7fdfc419f4f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.820+0000 7fdfca9a4640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfc4102040 0x7fdfc419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.820+0000 7fdfca9a4640 1 --2- 172.21.15.82:0/4214733424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc4100e60 0x7fdfc406b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.820+0000 7fdfca9a4640 1 -- 172.21.15.82:0/4214733424 >> 172.21.15.82:0/4214733424 conn(0x7fdfc40fc420 msgr2=0x7fdfc4105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.820+0000 7fdfca9a4640 1 -- 172.21.15.82:0/4214733424 shutdown_connections 2024-04-21T08:44:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.820+0000 7fdfca9a4640 1 -- 172.21.15.82:0/4214733424 wait complete. 2024-04-21T08:44:48.830 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:44:48.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:48 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/943039217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:44:48.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:48 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3766056047' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:44:48.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:48 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3355366718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:48.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.932+0000 7fe2a6470640 1 -- 172.21.15.82:0/1489856083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a01011e0 msgr2=0x7fe2a0101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.932+0000 7fe2a6470640 1 --2- 172.21.15.82:0/1489856083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a01011e0 0x7fe2a0101640 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe294009900 tx=0x7fe29402bcd0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.932+0000 7fe2a6470640 1 -- 172.21.15.82:0/1489856083 shutdown_connections 2024-04-21T08:44:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.932+0000 7fe2a6470640 1 --2- 172.21.15.82:0/1489856083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2a0101b80 0x7fe2a0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.932+0000 7fe2a6470640 1 --2- 172.21.15.82:0/1489856083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a01011e0 0x7fe2a0101640 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.932+0000 7fe2a6470640 1 --2- 172.21.15.82:0/1489856083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2a0100880 0x7fe2a0100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.932+0000 7fe2a6470640 1 -- 172.21.15.82:0/1489856083 >> 172.21.15.82:0/1489856083 conn(0x7fe2a00fc400 msgr2=0x7fe2a00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.932+0000 7fe2a6470640 1 -- 172.21.15.82:0/1489856083 shutdown_connections 2024-04-21T08:44:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.932+0000 7fe2a6470640 1 -- 172.21.15.82:0/1489856083 wait complete. 2024-04-21T08:44:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.932+0000 7fe2a6470640 1 Processor -- start 2024-04-21T08:44:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a6470640 1 -- start start 2024-04-21T08:44:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a6470640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2a0100880 0x7fe2a019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a6470640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2a01011e0 0x7fe2a019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a6470640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0101b80 0x7fe2a01a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a6470640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a010fca0 con 0x7fe2a0101b80 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a6470640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a010fb20 con 0x7fe2a0100880 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a6470640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a010fe20 con 0x7fe2a01011e0 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a4a0d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0101b80 0x7fe2a01a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a4a0d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0101b80 0x7fe2a01a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35202/0 (socket says 172.21.15.82:35202) 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a4a0d640 1 -- 172.21.15.82:0/660966752 learned_addr learned my addr 172.21.15.82:0/660966752 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe29ffff640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2a0100880 0x7fe2a019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a4a0d640 1 -- 172.21.15.82:0/660966752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2a01011e0 msgr2=0x7fe2a019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a4a0d640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2a01011e0 0x7fe2a019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a4a0d640 1 -- 172.21.15.82:0/660966752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2a0100880 msgr2=0x7fe2a019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a4a0d640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2a0100880 0x7fe2a019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a4a0d640 1 -- 172.21.15.82:0/660966752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a01a2030 con 0x7fe2a0101b80 2024-04-21T08:44:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.933+0000 7fe2a4a0d640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0101b80 0x7fe2a01a19c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe29800eaa0 tx=0x7fe29800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.934+0000 7fe29d7fa640 1 -- 172.21.15.82:0/660966752 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29800cdb0 con 0x7fe2a0101b80 2024-04-21T08:44:48.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.934+0000 7fe29d7fa640 1 -- 172.21.15.82:0/660966752 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe298005020 con 0x7fe2a0101b80 2024-04-21T08:44:48.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.934+0000 7fe2a6470640 1 -- 172.21.15.82:0/660966752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a01a2320 con 0x7fe2a0101b80 2024-04-21T08:44:48.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.934+0000 7fe29d7fa640 1 -- 172.21.15.82:0/660966752 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2980116d0 con 0x7fe2a0101b80 2024-04-21T08:44:48.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.934+0000 7fe2a6470640 1 -- 172.21.15.82:0/660966752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a01a2820 con 0x7fe2a0101b80 2024-04-21T08:44:48.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.935+0000 7fe2a6470640 1 -- 172.21.15.82:0/660966752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a010b870 con 0x7fe2a0101b80 2024-04-21T08:44:48.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.936+0000 7fe29d7fa640 1 -- 172.21.15.82:0/660966752 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe2980051d0 con 0x7fe2a0101b80 2024-04-21T08:44:48.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.936+0000 7fe29d7fa640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe2880a3bb0 0x7fe2880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.937+0000 7fe29ffff640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe2880a3bb0 0x7fe2880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.937+0000 7fe29d7fa640 1 -- 172.21.15.82:0/660966752 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe2980d07d0 con 0x7fe2a0101b80 2024-04-21T08:44:48.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.937+0000 7fe29ffff640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe2880a3bb0 0x7fe2880a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe290005d00 tx=0x7fe29000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:48.941+0000 7fe29d7fa640 1 -- 172.21.15.82:0/660966752 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe298016030 con 0x7fe2a0101b80 2024-04-21T08:44:49.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.117+0000 7fe2a6470640 1 -- 172.21.15.82:0/660966752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe2a002d5c0 con 0x7fe2a0101b80 2024-04-21T08:44:49.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.117+0000 7fe29d7fa640 1 -- 172.21.15.82:0/660966752 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fe29809cf70 con 0x7fe2a0101b80 2024-04-21T08:44:49.117 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:44:49.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.119+0000 7fe2a6470640 1 -- 172.21.15.82:0/660966752 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe2880a3bb0 msgr2=0x7fe2880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.119+0000 7fe2a6470640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe2880a3bb0 0x7fe2880a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe290005d00 tx=0x7fe29000b410 comp rx=0 tx=0).stop 2024-04-21T08:44:49.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.119+0000 7fe2a6470640 1 -- 172.21.15.82:0/660966752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0101b80 msgr2=0x7fe2a01a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.119+0000 7fe2a6470640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0101b80 0x7fe2a01a19c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe29800eaa0 tx=0x7fe29800ef60 comp rx=0 tx=0).stop 2024-04-21T08:44:49.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.119+0000 7fe2a6470640 1 -- 172.21.15.82:0/660966752 shutdown_connections 2024-04-21T08:44:49.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.119+0000 7fe2a6470640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe2880a3bb0 0x7fe2880a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.119+0000 7fe2a6470640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0101b80 0x7fe2a01a19c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.119+0000 7fe2a6470640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2a01011e0 0x7fe2a019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.119+0000 7fe2a6470640 1 --2- 172.21.15.82:0/660966752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2a0100880 0x7fe2a019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.119+0000 7fe2a6470640 1 -- 172.21.15.82:0/660966752 >> 172.21.15.82:0/660966752 conn(0x7fe2a00fc400 msgr2=0x7fe2a00fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.119+0000 7fe2a6470640 1 -- 172.21.15.82:0/660966752 shutdown_connections 2024-04-21T08:44:49.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.119+0000 7fe2a6470640 1 -- 172.21.15.82:0/660966752 wait complete. 2024-04-21T08:44:49.130 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:44:49.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.234+0000 7f63c64a8640 1 -- 172.21.15.82:0/2977169877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c01029a0 msgr2=0x7f63c01092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.234+0000 7f63c64a8640 1 --2- 172.21.15.82:0/2977169877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c01029a0 0x7f63c01092c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009ca0 tx=0x7f63b802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.234+0000 7f63c64a8640 1 -- 172.21.15.82:0/2977169877 shutdown_connections 2024-04-21T08:44:49.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.234+0000 7f63c64a8640 1 --2- 172.21.15.82:0/2977169877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c01029a0 0x7f63c01092c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.234+0000 7f63c64a8640 1 --2- 172.21.15.82:0/2977169877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63c0101fe0 0x7f63c0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.234+0000 7f63c64a8640 1 --2- 172.21.15.82:0/2977169877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c0100e90 0x7f63c01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.234+0000 7f63c64a8640 1 -- 172.21.15.82:0/2977169877 >> 172.21.15.82:0/2977169877 conn(0x7f63c00fc400 msgr2=0x7f63c00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.234+0000 7f63c64a8640 1 -- 172.21.15.82:0/2977169877 shutdown_connections 2024-04-21T08:44:49.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.234+0000 7f63c64a8640 1 -- 172.21.15.82:0/2977169877 wait complete. 2024-04-21T08:44:49.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63c64a8640 1 Processor -- start 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63c64a8640 1 -- start start 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63c64a8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63c0100e90 0x7f63c01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63c64a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c0101fe0 0x7f63c0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63c64a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c01029a0 0x7f63c019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63c64a8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f63c010c9f0 con 0x7f63c0101fe0 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63c64a8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63c010c870 con 0x7f63c0100e90 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63c64a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63c010cb70 con 0x7f63c01029a0 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63bf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c0101fe0 0x7f63c0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63bf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c0101fe0 0x7f63c0198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35224/0 (socket says 172.21.15.82:35224) 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63bf7fe640 1 -- 172.21.15.82:0/4180392557 learned_addr learned my addr 172.21.15.82:0/4180392557 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63bffff640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63c0100e90 0x7f63c01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63bf7fe640 1 -- 172.21.15.82:0/4180392557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c01029a0 msgr2=0x7f63c019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63bf7fe640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c01029a0 0x7f63c019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63bf7fe640 1 -- 172.21.15.82:0/4180392557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63c0100e90 msgr2=0x7f63c01981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63bf7fe640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63c0100e90 0x7f63c01981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.235+0000 7f63bf7fe640 1 -- 172.21.15.82:0/4180392557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c019fdf0 con 0x7f63c0101fe0 2024-04-21T08:44:49.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.236+0000 7f63bf7fe640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c0101fe0 0x7f63c0198710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f63b000e9c0 tx=0x7f63b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.236+0000 7f63bd7fa640 1 -- 172.21.15.82:0/4180392557 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b000cde0 con 0x7f63c0101fe0 2024-04-21T08:44:49.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.236+0000 7f63c64a8640 1 -- 172.21.15.82:0/4180392557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c01a00e0 con 0x7f63c0101fe0 2024-04-21T08:44:49.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.236+0000 7f63bd7fa640 1 -- 172.21.15.82:0/4180392557 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63b00108e0 con 0x7f63c0101fe0 2024-04-21T08:44:49.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.236+0000 7f63bd7fa640 1 -- 172.21.15.82:0/4180392557 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b00188d0 con 0x7f63c0101fe0 2024-04-21T08:44:49.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.236+0000 7f63c64a8640 1 -- 172.21.15.82:0/4180392557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c01a0680 con 0x7f63c0101fe0 2024-04-21T08:44:49.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.237+0000 7f63c64a8640 1 -- 172.21.15.82:0/4180392557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c01a09a0 con 0x7f63c0101fe0 2024-04-21T08:44:49.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.238+0000 7f63bd7fa640 1 -- 172.21.15.82:0/4180392557 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f63b001f020 con 0x7f63c0101fe0 2024-04-21T08:44:49.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.239+0000 7f63bd7fa640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f63ac0a3bb0 0x7f63ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.239+0000 7f63bffff640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f63ac0a3bb0 0x7f63ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.239+0000 7f63bd7fa640 1 -- 172.21.15.82:0/4180392557 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f63b00d0f30 con 0x7f63c0101fe0 2024-04-21T08:44:49.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.239+0000 7f63bffff640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f63ac0a3bb0 0x7f63ac0a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63a8005fd0 tx=0x7f63a8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.242+0000 7f63bd7fa640 1 -- 172.21.15.82:0/4180392557 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63b009d6d0 con 0x7f63c0101fe0 2024-04-21T08:44:49.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.427+0000 7f63c64a8640 1 -- 172.21.15.82:0/4180392557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f63c01a0b70 con 0x7f63c0101fe0 2024-04-21T08:44:49.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.428+0000 7f63bd7fa640 1 -- 172.21.15.82:0/4180392557 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f63b009cfd0 con 0x7f63c0101fe0 2024-04-21T08:44:49.427 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:44:49.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.430+0000 7f63c64a8640 1 -- 172.21.15.82:0/4180392557 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f63ac0a3bb0 msgr2=0x7f63ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.430+0000 7f63c64a8640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f63ac0a3bb0 0x7f63ac0a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63a8005fd0 tx=0x7f63a8005850 comp rx=0 tx=0).stop 2024-04-21T08:44:49.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.430+0000 7f63c64a8640 1 -- 172.21.15.82:0/4180392557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c0101fe0 msgr2=0x7f63c0198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.430+0000 7f63c64a8640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c0101fe0 0x7f63c0198710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f63b000e9c0 tx=0x7f63b000ee80 comp rx=0 tx=0).stop 2024-04-21T08:44:49.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.430+0000 7f63c64a8640 1 -- 172.21.15.82:0/4180392557 shutdown_connections 2024-04-21T08:44:49.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.430+0000 7f63c64a8640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f63ac0a3bb0 0x7f63ac0a6070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.430+0000 7f63c64a8640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63c01029a0 0x7f63c019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.430+0000 7f63c64a8640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63c0101fe0 0x7f63c0198710 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.430+0000 7f63c64a8640 1 --2- 172.21.15.82:0/4180392557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63c0100e90 0x7f63c01981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.430+0000 7f63c64a8640 1 -- 172.21.15.82:0/4180392557 >> 172.21.15.82:0/4180392557 conn(0x7f63c00fc400 msgr2=0x7f63c00fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.430+0000 7f63c64a8640 1 -- 172.21.15.82:0/4180392557 shutdown_connections 2024-04-21T08:44:49.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.430+0000 7f63c64a8640 1 -- 172.21.15.82:0/4180392557 wait complete. 2024-04-21T08:44:49.441 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T08:44:49.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.544+0000 7f79bb9e6640 1 -- 172.21.15.82:0/999474792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79b4105340 msgr2=0x7f79b4107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.544+0000 7f79bb9e6640 1 --2- 172.21.15.82:0/999474792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79b4105340 0x7f79b4107730 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009900 tx=0x7f79a402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:49.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.544+0000 7f79bb9e6640 1 -- 172.21.15.82:0/999474792 shutdown_connections 2024-04-21T08:44:49.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.544+0000 7f79bb9e6640 1 --2- 172.21.15.82:0/999474792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79b4107c70 0x7f79b410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.544+0000 7f79bb9e6640 1 --2- 172.21.15.82:0/999474792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79b4105340 0x7f79b4107730 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.544+0000 7f79bb9e6640 1 --2- 172.21.15.82:0/999474792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b4102a10 0x7f79b4104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.544+0000 7f79bb9e6640 1 -- 172.21.15.82:0/999474792 >> 172.21.15.82:0/999474792 conn(0x7f79b40fc400 msgr2=0x7f79b40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.544+0000 7f79bb9e6640 1 -- 172.21.15.82:0/999474792 shutdown_connections 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.544+0000 7f79bb9e6640 1 -- 172.21.15.82:0/999474792 wait complete. 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79bb9e6640 1 Processor -- start 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79bb9e6640 1 -- start start 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79bb9e6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b4102a10 0x7f79b41982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79bb9e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79b4105340 0x7f79b41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79bb9e6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79b4107c70 0x7f79b419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79bb9e6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f79b410db10 con 0x7f79b4107c70 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79bb9e6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f79b410d990 con 0x7f79b4102a10 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79bb9e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f79b410dc90 con 0x7f79b4105340 2024-04-21T08:44:49.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79b9f83640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79b4107c70 0x7f79b419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79b9f83640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79b4107c70 0x7f79b419f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35236/0 (socket says 172.21.15.82:35236) 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79b9f83640 1 -- 172.21.15.82:0/1982804466 learned_addr learned my addr 172.21.15.82:0/1982804466 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79b9782640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b4102a10 0x7f79b41982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79b8f81640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79b4105340 0x7f79b41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79b9f83640 1 -- 172.21.15.82:0/1982804466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79b4105340 msgr2=0x7f79b41987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79b9f83640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79b4105340 0x7f79b41987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79b9f83640 1 -- 172.21.15.82:0/1982804466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b4102a10 msgr2=0x7f79b41982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79b9f83640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b4102a10 0x7f79b41982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79b9f83640 1 -- 172.21.15.82:0/1982804466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b419fec0 con 0x7f79b4107c70 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79b9782640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b4102a10 0x7f79b41982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.545+0000 7f79b9f83640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79b4107c70 0x7f79b419f850 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00b970 tx=0x7f79ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.546+0000 7f79aa7fc640 1 -- 172.21.15.82:0/1982804466 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac019070 con 0x7f79b4107c70 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.546+0000 7f79bb9e6640 1 -- 172.21.15.82:0/1982804466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b41a01b0 con 0x7f79b4107c70 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.546+0000 7f79aa7fc640 1 -- 172.21.15.82:0/1982804466 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79ac014440 con 0x7f79b4107c70 2024-04-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.546+0000 7f79aa7fc640 1 -- 172.21.15.82:0/1982804466 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0075d0 con 0x7f79b4107c70 2024-04-21T08:44:49.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.546+0000 7f79bb9e6640 1 -- 172.21.15.82:0/1982804466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b41a07a0 con 0x7f79b4107c70 2024-04-21T08:44:49.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.547+0000 7f79bb9e6640 1 -- 172.21.15.82:0/1982804466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b40fdef0 con 0x7f79b4107c70 2024-04-21T08:44:49.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.548+0000 7f79aa7fc640 1 -- 172.21.15.82:0/1982804466 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f79ac01c070 con 0x7f79b4107c70 2024-04-21T08:44:49.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.548+0000 7f79aa7fc640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f79a00a3bb0 0x7f79a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.549+0000 7f79aa7fc640 1 -- 172.21.15.82:0/1982804466 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f79ac0cfc60 con 0x7f79b4107c70 2024-04-21T08:44:49.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.549+0000 7f79b9782640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f79a00a3bb0 0x7f79a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.549+0000 7f79b9782640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f79a00a3bb0 0x7f79a00a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f79b00058c0 tx=0x7f79b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.553+0000 7f79aa7fc640 1 -- 172.21.15.82:0/1982804466 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79ac09c400 con 0x7f79b4107c70 2024-04-21T08:44:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:49 smithi134 ceph-mon[31772]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 104 B/s wr, 0 op/s 2024-04-21T08:44:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:49 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/655040271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:44:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:49 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/4214733424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:44:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:49 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/660966752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:44:49.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.750+0000 7f79bb9e6640 1 -- 172.21.15.82:0/1982804466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f79b4062b50 con 0x7f79b4107c70 2024-04-21T08:44:49.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.751+0000 7f79aa7fc640 1 -- 172.21.15.82:0/1982804466 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+590 (secure 0 0 0) 0x7f79ac09bd00 con 0x7f79b4107c70 2024-04-21T08:44:49.751 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/4175378061 2024-04-22T08:37:26.227727+0000 2024-04-21T08:44:49.751 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2889165689 2024-04-22T08:37:26.227727+0000 2024-04-21T08:44:49.751 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/880382736 2024-04-22T08:37:26.227727+0000 2024-04-21T08:44:49.751 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3352705806 2024-04-22T08:37:26.227727+0000 2024-04-21T08:44:49.751 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/2710226711 2024-04-22T08:36:59.578707+0000 2024-04-21T08:44:49.751 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/4224740504 2024-04-22T08:36:59.578707+0000 2024-04-21T08:44:49.751 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2873389149 2024-04-22T08:36:59.578707+0000 2024-04-21T08:44:49.751 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/880382736 2024-04-22T08:37:26.227727+0000 2024-04-21T08:44:49.751 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/2710226711 2024-04-22T08:36:59.578707+0000 2024-04-21T08:44:49.751 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3032518359 2024-04-22T08:36:59.578707+0000 2024-04-21T08:44:49.751 INFO:teuthology.orchestra.run.smithi082.stderr:listed 10 entries 2024-04-21T08:44:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.753+0000 7f79bb9e6640 1 -- 172.21.15.82:0/1982804466 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f79a00a3bb0 msgr2=0x7f79a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.753+0000 7f79bb9e6640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f79a00a3bb0 0x7f79a00a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f79b00058c0 tx=0x7f79b0005850 comp rx=0 tx=0).stop 2024-04-21T08:44:49.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.753+0000 7f79bb9e6640 1 -- 172.21.15.82:0/1982804466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79b4107c70 msgr2=0x7f79b419f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.753+0000 7f79bb9e6640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79b4107c70 0x7f79b419f850 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00b970 tx=0x7f79ac00be30 comp rx=0 tx=0).stop 2024-04-21T08:44:49.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.753+0000 7f79bb9e6640 1 -- 172.21.15.82:0/1982804466 shutdown_connections 2024-04-21T08:44:49.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.753+0000 7f79bb9e6640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f79a00a3bb0 0x7f79a00a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.753+0000 7f79bb9e6640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79b4107c70 0x7f79b419f850 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.753+0000 7f79bb9e6640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79b4105340 0x7f79b41987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.753+0000 7f79bb9e6640 1 --2- 172.21.15.82:0/1982804466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79b4102a10 0x7f79b41982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.753+0000 7f79bb9e6640 1 -- 172.21.15.82:0/1982804466 >> 172.21.15.82:0/1982804466 conn(0x7f79b40fc400 msgr2=0x7f79b41034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.753+0000 7f79bb9e6640 1 -- 172.21.15.82:0/1982804466 shutdown_connections 2024-04-21T08:44:49.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:49.753+0000 7f79bb9e6640 1 -- 172.21.15.82:0/1982804466 wait complete. 2024-04-21T08:44:49.763 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:49 smithi082 ceph-mon[27967]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 104 B/s wr, 0 op/s 2024-04-21T08:44:49.763 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:49 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/655040271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:44:49.763 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:49 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4214733424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:44:49.764 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:49 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/660966752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:44:49.765 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-21T08:44:49.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:49.765 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:44:49.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:ceph-fuse[72508]: fuse finished with error 0 and tester_r 0 2024-04-21T08:44:49.790 INFO:teuthology.orchestra.run:waiting for 300 2024-04-21T08:44:49.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:49 smithi106 ceph-mon[31072]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 104 B/s wr, 0 op/s 2024-04-21T08:44:49.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:49 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/655040271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:44:49.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:49 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/4214733424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:44:49.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:49 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/660966752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:44:50.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:50 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4180392557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:44:50.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:50 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1982804466' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:44:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:50 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/4180392557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:44:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:50 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1982804466' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:44:50.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:50 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/4180392557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:44:50.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:50 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1982804466' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:44:51.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:51 smithi106 ceph-mon[31072]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:51.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:51 smithi134 ceph-mon[31772]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:51.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:51 smithi082 ceph-mon[27967]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:44:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:53 smithi134 ceph-mon[31772]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:44:53.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:53 smithi082 ceph-mon[27967]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:44:53.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:53 smithi106 ceph-mon[31072]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:44:55.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:55 smithi134 ceph-mon[31772]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:44:55.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:55 smithi082 ceph-mon[27967]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:44:55.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:55 smithi106 ceph-mon[31072]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:44:55.892 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-21T08:44:55.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:55.892 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-21T08:44:55.917 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:44:56.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.068+0000 7fa1d5406640 1 -- 172.21.15.82:0/4147873265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1d0101fc0 msgr2=0x7fa1d0102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.068+0000 7fa1d5406640 1 --2- 172.21.15.82:0/4147873265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1d0101fc0 0x7fa1d0102440 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009900 tx=0x7fa1c402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:56.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.068+0000 7fa1d5406640 1 -- 172.21.15.82:0/4147873265 shutdown_connections 2024-04-21T08:44:56.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.068+0000 7fa1d5406640 1 --2- 172.21.15.82:0/4147873265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0102980 0x7fa1d01092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.068+0000 7fa1d5406640 1 --2- 172.21.15.82:0/4147873265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1d0101fc0 0x7fa1d0102440 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.068+0000 7fa1d5406640 1 --2- 172.21.15.82:0/4147873265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d0100e70 0x7fa1d0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.068+0000 7fa1d5406640 1 -- 172.21.15.82:0/4147873265 >> 172.21.15.82:0/4147873265 conn(0x7fa1d00fc3e0 msgr2=0x7fa1d00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.068+0000 7fa1d5406640 1 -- 172.21.15.82:0/4147873265 shutdown_connections 2024-04-21T08:44:56.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.068+0000 7fa1d5406640 1 -- 172.21.15.82:0/4147873265 wait complete. 2024-04-21T08:44:56.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.068+0000 7fa1d5406640 1 Processor -- start 2024-04-21T08:44:56.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1d5406640 1 -- start start 2024-04-21T08:44:56.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1d5406640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d0100e70 0x7fa1d01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1d5406640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1d0101fc0 0x7fa1d0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1d5406640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0102980 0x7fa1d019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1d5406640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d010dad0 con 0x7fa1d0101fc0 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1d5406640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d010d950 con 0x7fa1d0100e70 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1d5406640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d010dc50 con 0x7fa1d0102980 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1ce7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1d0101fc0 0x7fa1d0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1ce7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1d0101fc0 0x7fa1d0198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37426/0 (socket says 172.21.15.82:37426) 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1ce7fc640 1 -- 172.21.15.82:0/3296928960 learned_addr learned my addr 172.21.15.82:0/3296928960 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1ceffd640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d0100e70 0x7fa1d01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1cf7fe640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0102980 0x7fa1d019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1ce7fc640 1 -- 172.21.15.82:0/3296928960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0102980 msgr2=0x7fa1d019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1ce7fc640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0102980 0x7fa1d019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1ce7fc640 1 -- 172.21.15.82:0/3296928960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d0100e70 msgr2=0x7fa1d01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1ce7fc640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d0100e70 0x7fa1d01981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1ce7fc640 1 -- 172.21.15.82:0/3296928960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d019fde0 con 0x7fa1d0101fc0 2024-04-21T08:44:56.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.069+0000 7fa1ce7fc640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1d0101fc0 0x7fa1d0198700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000c00 tx=0x7fa1c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.070+0000 7fa1b7fff640 1 -- 172.21.15.82:0/3296928960 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4043020 con 0x7fa1d0101fc0 2024-04-21T08:44:56.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.070+0000 7fa1b7fff640 1 -- 172.21.15.82:0/3296928960 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c4004aa0 con 0x7fa1d0101fc0 2024-04-21T08:44:56.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.070+0000 7fa1d5406640 1 -- 172.21.15.82:0/3296928960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d01a0070 con 0x7fa1d0101fc0 2024-04-21T08:44:56.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.070+0000 7fa1b7fff640 1 -- 172.21.15.82:0/3296928960 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4003d00 con 0x7fa1d0101fc0 2024-04-21T08:44:56.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.070+0000 7fa1d5406640 1 -- 172.21.15.82:0/3296928960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01a05c0 con 0x7fa1d0101fc0 2024-04-21T08:44:56.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.071+0000 7fa1d5406640 1 -- 172.21.15.82:0/3296928960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d0108270 con 0x7fa1d0101fc0 2024-04-21T08:44:56.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.072+0000 7fa1b7fff640 1 -- 172.21.15.82:0/3296928960 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa1c4046020 con 0x7fa1d0101fc0 2024-04-21T08:44:56.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.072+0000 7fa1b7fff640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa1b80a3bb0 0x7fa1b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.072+0000 7fa1ceffd640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa1b80a3bb0 0x7fa1b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.072+0000 7fa1b7fff640 1 -- 172.21.15.82:0/3296928960 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa1c40f2f90 con 0x7fa1d0101fc0 2024-04-21T08:44:56.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.073+0000 7fa1ceffd640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa1b80a3bb0 0x7fa1b80a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0005c40 tx=0x7fa1c000a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.076+0000 7fa1b7fff640 1 -- 172.21.15.82:0/3296928960 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1c404d020 con 0x7fa1d0101fc0 2024-04-21T08:44:56.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.253+0000 7fa1d5406640 1 -- 172.21.15.82:0/3296928960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1d002d810 con 0x7fa1d0101fc0 2024-04-21T08:44:56.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.255+0000 7fa1b7fff640 1 -- 172.21.15.82:0/3296928960 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24140 (secure 0 0 0) 0x7fa1c40bf730 con 0x7fa1d0101fc0 2024-04-21T08:44:56.255 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:56.255 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":89,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:44:38.858333+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:44:56.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.257+0000 7fa1d5406640 1 -- 172.21.15.82:0/3296928960 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa1b80a3bb0 msgr2=0x7fa1b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.257+0000 7fa1d5406640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa1b80a3bb0 0x7fa1b80a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0005c40 tx=0x7fa1c000a590 comp rx=0 tx=0).stop 2024-04-21T08:44:56.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.257+0000 7fa1d5406640 1 -- 172.21.15.82:0/3296928960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1d0101fc0 msgr2=0x7fa1d0198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.257+0000 7fa1d5406640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1d0101fc0 0x7fa1d0198700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000c00 tx=0x7fa1c4004270 comp rx=0 tx=0).stop 2024-04-21T08:44:56.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.257+0000 7fa1d5406640 1 -- 172.21.15.82:0/3296928960 shutdown_connections 2024-04-21T08:44:56.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.257+0000 7fa1d5406640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa1b80a3bb0 0x7fa1b80a6070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.257+0000 7fa1d5406640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1d0102980 0x7fa1d019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.257+0000 7fa1d5406640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1d0101fc0 0x7fa1d0198700 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.257+0000 7fa1d5406640 1 --2- 172.21.15.82:0/3296928960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d0100e70 0x7fa1d01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.257+0000 7fa1d5406640 1 -- 172.21.15.82:0/3296928960 >> 172.21.15.82:0/3296928960 conn(0x7fa1d00fc3e0 msgr2=0x7fa1d0105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.257+0000 7fa1d5406640 1 -- 172.21.15.82:0/3296928960 shutdown_connections 2024-04-21T08:44:56.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.257+0000 7fa1d5406640 1 -- 172.21.15.82:0/3296928960 wait complete. 2024-04-21T08:44:56.268 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:44:56.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 -- 172.21.15.82:0/2746986303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479c100e80 msgr2=0x7f479c1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 --2- 172.21.15.82:0/2746986303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479c100e80 0x7f479c1012a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f478c009900 tx=0x7f478c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:56.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 -- 172.21.15.82:0/2746986303 shutdown_connections 2024-04-21T08:44:56.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 --2- 172.21.15.82:0/2746986303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c102990 0x7f479c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 --2- 172.21.15.82:0/2746986303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479c101fd0 0x7f479c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 --2- 172.21.15.82:0/2746986303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479c100e80 0x7f479c1012a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 -- 172.21.15.82:0/2746986303 >> 172.21.15.82:0/2746986303 conn(0x7f479c0fc410 msgr2=0x7f479c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 -- 172.21.15.82:0/2746986303 shutdown_connections 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 -- 172.21.15.82:0/2746986303 wait complete. 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 Processor -- start 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 -- start start 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c100e80 0x7f479c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479c101fd0 0x7f479c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.366+0000 7f47a31a4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479c102990 0x7f479c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a31a4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f479c10ca50 con 0x7f479c102990 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a31a4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f479c10c8d0 con 0x7f479c100e80 2024-04-21T08:44:56.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a31a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f479c10cbd0 con 0x7f479c101fd0 2024-04-21T08:44:56.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a1741640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479c102990 0x7f479c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a1741640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479c102990 0x7f479c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37434/0 (socket says 172.21.15.82:37434) 2024-04-21T08:44:56.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a1741640 1 -- 172.21.15.82:0/594514113 learned_addr learned my addr 172.21.15.82:0/594514113 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:56.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f479bfff640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479c101fd0 0x7f479c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a0f40640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c100e80 0x7f479c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a1741640 1 -- 172.21.15.82:0/594514113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479c101fd0 msgr2=0x7f479c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a1741640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479c101fd0 0x7f479c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a1741640 1 -- 172.21.15.82:0/594514113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c100e80 msgr2=0x7f479c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a1741640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c100e80 0x7f479c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a1741640 1 -- 172.21.15.82:0/594514113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c19fea0 con 0x7f479c102990 2024-04-21T08:44:56.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a0f40640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c100e80 0x7f479c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:56.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a1741640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479c102990 0x7f479c19f830 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f479400cad0 tx=0x7f479400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f4799ffb640 1 -- 172.21.15.82:0/594514113 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479400ae20 con 0x7f479c102990 2024-04-21T08:44:56.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f4799ffb640 1 -- 172.21.15.82:0/594514113 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4794005020 con 0x7f479c102990 2024-04-21T08:44:56.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a31a4640 1 -- 172.21.15.82:0/594514113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c1a0190 con 0x7f479c102990 2024-04-21T08:44:56.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f4799ffb640 1 -- 172.21.15.82:0/594514113 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479400f6b0 con 0x7f479c102990 2024-04-21T08:44:56.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.367+0000 7f47a31a4640 1 -- 172.21.15.82:0/594514113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c1a0710 con 0x7f479c102990 2024-04-21T08:44:56.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.368+0000 7f47a31a4640 1 -- 172.21.15.82:0/594514113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479c1a09d0 con 0x7f479c102990 2024-04-21T08:44:56.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.369+0000 7f4799ffb640 1 -- 172.21.15.82:0/594514113 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f47940051d0 con 0x7f479c102990 2024-04-21T08:44:56.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.370+0000 7f4799ffb640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f47880a3bb0 0x7f47880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.370+0000 7f47a0f40640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f47880a3bb0 0x7f47880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.370+0000 7f4799ffb640 1 -- 172.21.15.82:0/594514113 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f47940d07b0 con 0x7f479c102990 2024-04-21T08:44:56.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.371+0000 7f47a0f40640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f47880a3bb0 0x7f47880a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f478c0096c0 tx=0x7f478c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.375+0000 7f4799ffb640 1 -- 172.21.15.82:0/594514113 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4794014030 con 0x7f479c102990 2024-04-21T08:44:56.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.557+0000 7f47a31a4640 1 -- 172.21.15.82:0/594514113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f479c1a0ba0 con 0x7f479c102990 2024-04-21T08:44:56.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.559+0000 7f4799ffb640 1 -- 172.21.15.82:0/594514113 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24140 (secure 0 0 0) 0x7f479409cf50 con 0x7f479c102990 2024-04-21T08:44:56.559 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:44:56.559 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":89,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:44:38.858333+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:44:56.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.561+0000 7f47a31a4640 1 -- 172.21.15.82:0/594514113 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f47880a3bb0 msgr2=0x7f47880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.561+0000 7f47a31a4640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f47880a3bb0 0x7f47880a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f478c0096c0 tx=0x7f478c000bc0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.561+0000 7f47a31a4640 1 -- 172.21.15.82:0/594514113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479c102990 msgr2=0x7f479c19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.561+0000 7f47a31a4640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479c102990 0x7f479c19f830 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f479400cad0 tx=0x7f479400cf90 comp rx=0 tx=0).stop 2024-04-21T08:44:56.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.561+0000 7f47a31a4640 1 -- 172.21.15.82:0/594514113 shutdown_connections 2024-04-21T08:44:56.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.561+0000 7f47a31a4640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f47880a3bb0 0x7f47880a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.561+0000 7f47a31a4640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479c102990 0x7f479c19f830 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.561+0000 7f47a31a4640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479c101fd0 0x7f479c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.561+0000 7f47a31a4640 1 --2- 172.21.15.82:0/594514113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f479c100e80 0x7f479c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.561+0000 7f47a31a4640 1 -- 172.21.15.82:0/594514113 >> 172.21.15.82:0/594514113 conn(0x7f479c0fc410 msgr2=0x7f479c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.561+0000 7f47a31a4640 1 -- 172.21.15.82:0/594514113 shutdown_connections 2024-04-21T08:44:56.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.561+0000 7f47a31a4640 1 -- 172.21.15.82:0/594514113 wait complete. 2024-04-21T08:44:56.570 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:56 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3296928960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:56.572 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-21T08:44:56.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.674+0000 7f1550db2640 1 -- 172.21.15.82:0/1098121197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154c1053e0 msgr2=0x7f154c1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.674+0000 7f1550db2640 1 --2- 172.21.15.82:0/1098121197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154c1053e0 0x7f154c1077d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1540009900 tx=0x7f154002bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:56.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.674+0000 7f1550db2640 1 -- 172.21.15.82:0/1098121197 shutdown_connections 2024-04-21T08:44:56.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.674+0000 7f1550db2640 1 --2- 172.21.15.82:0/1098121197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f154c107d10 0x7f154c10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.674+0000 7f1550db2640 1 --2- 172.21.15.82:0/1098121197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154c1053e0 0x7f154c1077d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.674+0000 7f1550db2640 1 --2- 172.21.15.82:0/1098121197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f154c102ab0 0x7f154c104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.674+0000 7f1550db2640 1 -- 172.21.15.82:0/1098121197 >> 172.21.15.82:0/1098121197 conn(0x7f154c0fc440 msgr2=0x7f154c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:56.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.674+0000 7f1550db2640 1 -- 172.21.15.82:0/1098121197 shutdown_connections 2024-04-21T08:44:56.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.674+0000 7f1550db2640 1 -- 172.21.15.82:0/1098121197 wait complete. 2024-04-21T08:44:56.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f1550db2640 1 Processor -- start 2024-04-21T08:44:56.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f1550db2640 1 -- start start 2024-04-21T08:44:56.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f1550db2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154c102ab0 0x7f154c102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f1550db2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f154c1053e0 0x7f154c100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f1550db2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f154c107d10 0x7f154c100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f1550db2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f154c10daf0 con 0x7f154c102ab0 2024-04-21T08:44:56.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f1550db2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f154c10d970 con 0x7f154c107d10 2024-04-21T08:44:56.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f1550db2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f154c10dc70 con 0x7f154c1053e0 2024-04-21T08:44:56.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f154a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154c102ab0 0x7f154c102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f154a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154c102ab0 0x7f154c102420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37458/0 (socket says 172.21.15.82:37458) 2024-04-21T08:44:56.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f154a59c640 1 -- 172.21.15.82:0/767016548 learned_addr learned my addr 172.21.15.82:0/767016548 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:56.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f154a59c640 1 -- 172.21.15.82:0/767016548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f154c1053e0 msgr2=0x7f154c100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f154a59c640 1 --2- 172.21.15.82:0/767016548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f154c1053e0 0x7f154c100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f154a59c640 1 -- 172.21.15.82:0/767016548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f154c107d10 msgr2=0x7f154c100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:56.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f154a59c640 1 --2- 172.21.15.82:0/767016548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f154c107d10 0x7f154c100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:56.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f154a59c640 1 -- 172.21.15.82:0/767016548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154c101830 con 0x7f154c102ab0 2024-04-21T08:44:56.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.675+0000 7f154a59c640 1 --2- 172.21.15.82:0/767016548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154c102ab0 0x7f154c102420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f153c00c970 tx=0x7f153c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.676+0000 7f153b7fe640 1 -- 172.21.15.82:0/767016548 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c007bb0 con 0x7f154c102ab0 2024-04-21T08:44:56.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.676+0000 7f153b7fe640 1 -- 172.21.15.82:0/767016548 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f153c007d50 con 0x7f154c102ab0 2024-04-21T08:44:56.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.676+0000 7f153b7fe640 1 -- 172.21.15.82:0/767016548 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c00f450 con 0x7f154c102ab0 2024-04-21T08:44:56.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.676+0000 7f1550db2640 1 -- 172.21.15.82:0/767016548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154c1a2350 con 0x7f154c102ab0 2024-04-21T08:44:56.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.676+0000 7f1550db2640 1 -- 172.21.15.82:0/767016548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f154c1a28a0 con 0x7f154c102ab0 2024-04-21T08:44:56.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.677+0000 7f1550db2640 1 -- 172.21.15.82:0/767016548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f154c0fdf30 con 0x7f154c102ab0 2024-04-21T08:44:56.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.678+0000 7f153b7fe640 1 -- 172.21.15.82:0/767016548 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f153c004f20 con 0x7f154c102ab0 2024-04-21T08:44:56.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.678+0000 7f153b7fe640 1 --2- 172.21.15.82:0/767016548 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f15340a3c30 0x7f15340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:56.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.679+0000 7f1549d9b640 1 --2- 172.21.15.82:0/767016548 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f15340a3c30 0x7f15340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:56.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.679+0000 7f153b7fe640 1 -- 172.21.15.82:0/767016548 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f153c0d0440 con 0x7f154c102ab0 2024-04-21T08:44:56.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.679+0000 7f1549d9b640 1 --2- 172.21.15.82:0/767016548 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f15340a3c30 0x7f15340a60f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1540036040 tx=0x7f1540000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:56.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.683+0000 7f153b7fe640 1 -- 172.21.15.82:0/767016548 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f153c014030 con 0x7f154c102ab0 2024-04-21T08:44:56.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:56 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3296928960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:56.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:56 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3296928960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:56.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:56.898+0000 7f1550db2640 1 -- 172.21.15.82:0/767016548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f154c062b50 con 0x7f154c102ab0 2024-04-21T08:44:57.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.424+0000 7f153b7fe640 1 -- 172.21.15.82:0/767016548 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f153c09cbe0 con 0x7f154c102ab0 2024-04-21T08:44:57.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.426+0000 7f1550db2640 1 -- 172.21.15.82:0/767016548 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f15340a3c30 msgr2=0x7f15340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.427+0000 7f1550db2640 1 --2- 172.21.15.82:0/767016548 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f15340a3c30 0x7f15340a60f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1540036040 tx=0x7f1540000bc0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.427+0000 7f1550db2640 1 -- 172.21.15.82:0/767016548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154c102ab0 msgr2=0x7f154c102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.427+0000 7f1550db2640 1 --2- 172.21.15.82:0/767016548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154c102ab0 0x7f154c102420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f153c00c970 tx=0x7f153c00ce30 comp rx=0 tx=0).stop 2024-04-21T08:44:57.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.427+0000 7f1550db2640 1 -- 172.21.15.82:0/767016548 shutdown_connections 2024-04-21T08:44:57.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.427+0000 7f1550db2640 1 --2- 172.21.15.82:0/767016548 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f15340a3c30 0x7f15340a60f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.427+0000 7f1550db2640 1 --2- 172.21.15.82:0/767016548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f154c107d10 0x7f154c100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.427+0000 7f1550db2640 1 --2- 172.21.15.82:0/767016548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f154c1053e0 0x7f154c100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.427+0000 7f1550db2640 1 --2- 172.21.15.82:0/767016548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f154c102ab0 0x7f154c102420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.427+0000 7f1550db2640 1 -- 172.21.15.82:0/767016548 >> 172.21.15.82:0/767016548 conn(0x7f154c0fc440 msgr2=0x7f154c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.427+0000 7f1550db2640 1 -- 172.21.15.82:0/767016548 shutdown_connections 2024-04-21T08:44:57.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.427+0000 7f1550db2640 1 -- 172.21.15.82:0/767016548 wait complete. 2024-04-21T08:44:57.440 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-21T08:44:57.440 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-21T08:44:57.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.548+0000 7f6e64a81640 1 -- 172.21.15.82:0/3027297044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e601053c0 msgr2=0x7f6e60105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.548+0000 7f6e64a81640 1 --2- 172.21.15.82:0/3027297044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e601053c0 0x7f6e60105840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009900 tx=0x7f6e4802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:57.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.548+0000 7f6e64a81640 1 -- 172.21.15.82:0/3027297044 shutdown_connections 2024-04-21T08:44:57.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.548+0000 7f6e64a81640 1 --2- 172.21.15.82:0/3027297044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e60105d80 0x7f6e6010c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.548+0000 7f6e64a81640 1 --2- 172.21.15.82:0/3027297044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e601053c0 0x7f6e60105840 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.548+0000 7f6e64a81640 1 --2- 172.21.15.82:0/3027297044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e601041d0 0x7f6e601045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.548+0000 7f6e64a81640 1 -- 172.21.15.82:0/3027297044 >> 172.21.15.82:0/3027297044 conn(0x7f6e600ff740 msgr2=0x7f6e60101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:57.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.548+0000 7f6e64a81640 1 -- 172.21.15.82:0/3027297044 shutdown_connections 2024-04-21T08:44:57.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.548+0000 7f6e64a81640 1 -- 172.21.15.82:0/3027297044 wait complete. 2024-04-21T08:44:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.549+0000 7f6e64a81640 1 Processor -- start 2024-04-21T08:44:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.549+0000 7f6e64a81640 1 -- start start 2024-04-21T08:44:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.549+0000 7f6e64a81640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e601041d0 0x7f6e6019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.549+0000 7f6e64a81640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601053c0 0x7f6e6019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.549+0000 7f6e64a81640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e60105d80 0x7f6e601a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.549+0000 7f6e64a81640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010fdf0 con 0x7f6e60105d80 2024-04-21T08:44:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.549+0000 7f6e64a81640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010fc70 con 0x7f6e601041d0 2024-04-21T08:44:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.549+0000 7f6e64a81640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010ff70 con 0x7f6e601053c0 2024-04-21T08:44:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.549+0000 7f6e5ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e60105d80 0x7f6e601a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.549+0000 7f6e5ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e60105d80 0x7f6e601a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37470/0 (socket says 172.21.15.82:37470) 2024-04-21T08:44:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.549+0000 7f6e5ed9d640 1 -- 172.21.15.82:0/573054086 learned_addr learned my addr 172.21.15.82:0/573054086 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e5ed9d640 1 -- 172.21.15.82:0/573054086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601053c0 msgr2=0x7f6e6019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e57fff640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601053c0 0x7f6e6019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e5e59c640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e601041d0 0x7f6e6019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e5ed9d640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601053c0 0x7f6e6019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e5ed9d640 1 -- 172.21.15.82:0/573054086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e601041d0 msgr2=0x7f6e6019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e5ed9d640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e601041d0 0x7f6e6019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e5ed9d640 1 -- 172.21.15.82:0/573054086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e601a2180 con 0x7f6e60105d80 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e57fff640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601053c0 0x7f6e6019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e5e59c640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e601041d0 0x7f6e6019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e5ed9d640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e60105d80 0x7f6e601a1b10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800b970 tx=0x7f6e5800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e577fe640 1 -- 172.21.15.82:0/573054086 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58019070 con 0x7f6e60105d80 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e577fe640 1 -- 172.21.15.82:0/573054086 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e58014440 con 0x7f6e60105d80 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e64a81640 1 -- 172.21.15.82:0/573054086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e601a2470 con 0x7f6e60105d80 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e577fe640 1 -- 172.21.15.82:0/573054086 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e580075d0 con 0x7f6e60105d80 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.550+0000 7f6e64a81640 1 -- 172.21.15.82:0/573054086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e601a2a40 con 0x7f6e60105d80 2024-04-21T08:44:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.551+0000 7f6e64a81640 1 -- 172.21.15.82:0/573054086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6010b560 con 0x7f6e60105d80 2024-04-21T08:44:57.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.552+0000 7f6e577fe640 1 -- 172.21.15.82:0/573054086 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6e5801c070 con 0x7f6e60105d80 2024-04-21T08:44:57.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.553+0000 7f6e577fe640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6e4c0a3bb0 0x7f6e4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:57.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.553+0000 7f6e5e59c640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6e4c0a3bb0 0x7f6e4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:57.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.553+0000 7f6e577fe640 1 -- 172.21.15.82:0/573054086 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f6e580d00a0 con 0x7f6e60105d80 2024-04-21T08:44:57.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.553+0000 7f6e5e59c640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6e4c0a3bb0 0x7f6e4c0a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6e50006fd0 tx=0x7f6e50008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:57.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.556+0000 7f6e577fe640 1 -- 172.21.15.82:0/573054086 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e5809c840 con 0x7f6e60105d80 2024-04-21T08:44:57.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:57 smithi134 ceph-mon[31772]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:44:57.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:57 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/594514113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:57.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:57 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/767016548' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T08:44:57.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:57.773+0000 7f6e64a81640 1 -- 172.21.15.82:0/573054086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f6e6002d5c0 con 0x7f6e60105d80 2024-04-21T08:44:57.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:57 smithi082 ceph-mon[27967]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:44:57.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:57 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/594514113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:57.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:57 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/767016548' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T08:44:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:57 smithi106 ceph-mon[31072]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:44:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:57 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/594514113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:44:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:57 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/767016548' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T08:44:58.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.436+0000 7f6e577fe640 1 -- 172.21.15.82:0/573054086 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f6e5809c140 con 0x7f6e60105d80 2024-04-21T08:44:58.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.439+0000 7f6e64a81640 1 -- 172.21.15.82:0/573054086 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6e4c0a3bb0 msgr2=0x7f6e4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:58.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.439+0000 7f6e64a81640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6e4c0a3bb0 0x7f6e4c0a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6e50006fd0 tx=0x7f6e50008040 comp rx=0 tx=0).stop 2024-04-21T08:44:58.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.439+0000 7f6e64a81640 1 -- 172.21.15.82:0/573054086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e60105d80 msgr2=0x7f6e601a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:58.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.439+0000 7f6e64a81640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e60105d80 0x7f6e601a1b10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800b970 tx=0x7f6e5800be30 comp rx=0 tx=0).stop 2024-04-21T08:44:58.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.439+0000 7f6e64a81640 1 -- 172.21.15.82:0/573054086 shutdown_connections 2024-04-21T08:44:58.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.439+0000 7f6e64a81640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6e4c0a3bb0 0x7f6e4c0a6070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:58.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.439+0000 7f6e64a81640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e60105d80 0x7f6e601a1b10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:58.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.439+0000 7f6e64a81640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e601053c0 0x7f6e6019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:58.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.439+0000 7f6e64a81640 1 --2- 172.21.15.82:0/573054086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e601041d0 0x7f6e6019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:58.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.439+0000 7f6e64a81640 1 -- 172.21.15.82:0/573054086 >> 172.21.15.82:0/573054086 conn(0x7f6e600ff740 msgr2=0x7f6e60108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:58.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.439+0000 7f6e64a81640 1 -- 172.21.15.82:0/573054086 shutdown_connections 2024-04-21T08:44:58.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.439+0000 7f6e64a81640 1 -- 172.21.15.82:0/573054086 wait complete. 2024-04-21T08:44:58.455 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-21T08:44:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.564+0000 7f31c5406640 1 -- 172.21.15.82:0/3070258942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102040 msgr2=0x7f31c01024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.564+0000 7f31c5406640 1 --2- 172.21.15.82:0/3070258942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102040 0x7f31c01024c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009900 tx=0x7f31b402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.564+0000 7f31c5406640 1 -- 172.21.15.82:0/3070258942 shutdown_connections 2024-04-21T08:44:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.564+0000 7f31c5406640 1 --2- 172.21.15.82:0/3070258942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c0102a00 0x7f31c0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.564+0000 7f31c5406640 1 --2- 172.21.15.82:0/3070258942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102040 0x7f31c01024c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.564+0000 7f31c5406640 1 --2- 172.21.15.82:0/3070258942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c0100e60 0x7f31c0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.564+0000 7f31c5406640 1 -- 172.21.15.82:0/3070258942 >> 172.21.15.82:0/3070258942 conn(0x7f31c00fc440 msgr2=0x7f31c00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.564+0000 7f31c5406640 1 -- 172.21.15.82:0/3070258942 shutdown_connections 2024-04-21T08:44:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.564+0000 7f31c5406640 1 -- 172.21.15.82:0/3070258942 wait complete. 2024-04-21T08:44:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.564+0000 7f31c5406640 1 Processor -- start 2024-04-21T08:44:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31c5406640 1 -- start start 2024-04-21T08:44:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31c5406640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c0100e60 0x7f31c0197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31c5406640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c0102040 0x7f31c0197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31c5406640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102a00 0x7f31c019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31c5406640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f31c010ca70 con 0x7f31c0102a00 2024-04-21T08:44:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31c5406640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f31c010c8f0 con 0x7f31c0102040 2024-04-21T08:44:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31c5406640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f31c010cbf0 con 0x7f31c0100e60 2024-04-21T08:44:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31bf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102a00 0x7f31c019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31bf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102a00 0x7f31c019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37476/0 (socket says 172.21.15.82:37476) 2024-04-21T08:44:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31bf7fe640 1 -- 172.21.15.82:0/3346767157 learned_addr learned my addr 172.21.15.82:0/3346767157 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31beffd640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c0100e60 0x7f31c0197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31be7fc640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c0102040 0x7f31c0197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31bf7fe640 1 -- 172.21.15.82:0/3346767157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c0100e60 msgr2=0x7f31c0197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31bf7fe640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c0100e60 0x7f31c0197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31bf7fe640 1 -- 172.21.15.82:0/3346767157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c0102040 msgr2=0x7f31c0197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31bf7fe640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c0102040 0x7f31c0197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31bf7fe640 1 -- 172.21.15.82:0/3346767157 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c019ed60 con 0x7f31c0102a00 2024-04-21T08:44:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31be7fc640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c0102040 0x7f31c0197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:58.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31bf7fe640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102a00 0x7f31c019e6f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f31b800cce0 tx=0x7f31b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:58.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31a7fff640 1 -- 172.21.15.82:0/3346767157 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b800f450 con 0x7f31c0102a00 2024-04-21T08:44:58.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31c5406640 1 -- 172.21.15.82:0/3346767157 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c019eff0 con 0x7f31c0102a00 2024-04-21T08:44:58.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31a7fff640 1 -- 172.21.15.82:0/3346767157 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31b800faa0 con 0x7f31c0102a00 2024-04-21T08:44:58.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.565+0000 7f31a7fff640 1 -- 172.21.15.82:0/3346767157 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8018b70 con 0x7f31c0102a00 2024-04-21T08:44:58.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.566+0000 7f31c5406640 1 -- 172.21.15.82:0/3346767157 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c019f540 con 0x7f31c0102a00 2024-04-21T08:44:58.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.567+0000 7f31c5406640 1 -- 172.21.15.82:0/3346767157 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c0108250 con 0x7f31c0102a00 2024-04-21T08:44:58.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.568+0000 7f31a7fff640 1 -- 172.21.15.82:0/3346767157 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f31b8018d10 con 0x7f31c0102a00 2024-04-21T08:44:58.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.568+0000 7f31a7fff640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f31a80a3bb0 0x7f31a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:58.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.568+0000 7f31beffd640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f31a80a3bb0 0x7f31a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:58.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.569+0000 7f31a7fff640 1 -- 172.21.15.82:0/3346767157 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f31b80d0b30 con 0x7f31c0102a00 2024-04-21T08:44:58.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.569+0000 7f31beffd640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f31a80a3bb0 0x7f31a80a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f31c0101bb0 tx=0x7f31b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.573+0000 7f31a7fff640 1 -- 172.21.15.82:0/3346767157 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31b8014030 con 0x7f31c0102a00 2024-04-21T08:44:58.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:58 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/767016548' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T08:44:58.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:58 smithi134 ceph-mon[31772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:44:58.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:58 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/573054086' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-21T08:44:58.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:58.795+0000 7f31c5406640 1 -- 172.21.15.82:0/3346767157 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f31c002d5c0 con 0x7f31c0102a00 2024-04-21T08:44:58.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:58 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/767016548' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T08:44:58.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:58 smithi082 ceph-mon[27967]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:44:58.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:58 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/573054086' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-21T08:44:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:58 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/767016548' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T08:44:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:58 smithi106 ceph-mon[31072]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:44:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:58 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/573054086' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-21T08:44:59.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.443+0000 7f31a7fff640 1 -- 172.21.15.82:0/3346767157 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f31b809d2d0 con 0x7f31c0102a00 2024-04-21T08:44:59.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.446+0000 7f31c5406640 1 -- 172.21.15.82:0/3346767157 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f31a80a3bb0 msgr2=0x7f31a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.446+0000 7f31c5406640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f31a80a3bb0 0x7f31a80a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f31c0101bb0 tx=0x7f31b0008040 comp rx=0 tx=0).stop 2024-04-21T08:44:59.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.446+0000 7f31c5406640 1 -- 172.21.15.82:0/3346767157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102a00 msgr2=0x7f31c019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.446+0000 7f31c5406640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102a00 0x7f31c019e6f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f31b800cce0 tx=0x7f31b800a620 comp rx=0 tx=0).stop 2024-04-21T08:44:59.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.446+0000 7f31c5406640 1 -- 172.21.15.82:0/3346767157 shutdown_connections 2024-04-21T08:44:59.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.446+0000 7f31c5406640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f31a80a3bb0 0x7f31a80a6070 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.446+0000 7f31c5406640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31c0102a00 0x7f31c019e6f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.446+0000 7f31c5406640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31c0102040 0x7f31c0197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.446+0000 7f31c5406640 1 --2- 172.21.15.82:0/3346767157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c0100e60 0x7f31c0197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.446+0000 7f31c5406640 1 -- 172.21.15.82:0/3346767157 >> 172.21.15.82:0/3346767157 conn(0x7f31c00fc440 msgr2=0x7f31c0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.447+0000 7f31c5406640 1 -- 172.21.15.82:0/3346767157 shutdown_connections 2024-04-21T08:44:59.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.447+0000 7f31c5406640 1 -- 172.21.15.82:0/3346767157 wait complete. 2024-04-21T08:44:59.462 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-21T08:44:59.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.567+0000 7fb935ffb640 1 -- 172.21.15.82:0/674313655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102040 msgr2=0x7fb9301024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.567+0000 7fb935ffb640 1 --2- 172.21.15.82:0/674313655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102040 0x7fb9301024c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb924009900 tx=0x7fb92402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:59.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.567+0000 7fb935ffb640 1 -- 172.21.15.82:0/674313655 shutdown_connections 2024-04-21T08:44:59.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.567+0000 7fb935ffb640 1 --2- 172.21.15.82:0/674313655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb930102a00 0x7fb930109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.567+0000 7fb935ffb640 1 --2- 172.21.15.82:0/674313655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102040 0x7fb9301024c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.567+0000 7fb935ffb640 1 --2- 172.21.15.82:0/674313655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb930100e60 0x7fb930101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.567+0000 7fb935ffb640 1 -- 172.21.15.82:0/674313655 >> 172.21.15.82:0/674313655 conn(0x7fb9300fc420 msgr2=0x7fb9300fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.567+0000 7fb935ffb640 1 -- 172.21.15.82:0/674313655 shutdown_connections 2024-04-21T08:44:59.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.567+0000 7fb935ffb640 1 -- 172.21.15.82:0/674313655 wait complete. 2024-04-21T08:44:59.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.567+0000 7fb935ffb640 1 Processor -- start 2024-04-21T08:44:59.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb935ffb640 1 -- start start 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb935ffb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb930100e60 0x7fb930198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb935ffb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102040 0x7fb9301987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb935ffb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb930102a00 0x7fb93019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb935ffb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb93010ca10 con 0x7fb930102040 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb935ffb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb93010c890 con 0x7fb930100e60 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb935ffb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb93010cb90 con 0x7fb930102a00 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb92effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102040 0x7fb9301987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb92f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb930100e60 0x7fb930198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb92effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102040 0x7fb9301987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37498/0 (socket says 172.21.15.82:37498) 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb92effd640 1 -- 172.21.15.82:0/283777098 learned_addr learned my addr 172.21.15.82:0/283777098 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb92ffff640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb930102a00 0x7fb93019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb92effd640 1 -- 172.21.15.82:0/283777098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb930102a00 msgr2=0x7fb93019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb92effd640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb930102a00 0x7fb93019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb92effd640 1 -- 172.21.15.82:0/283777098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb930100e60 msgr2=0x7fb930198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb92effd640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb930100e60 0x7fb930198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb92effd640 1 -- 172.21.15.82:0/283777098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93019fe60 con 0x7fb930102040 2024-04-21T08:44:59.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb92f7fe640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb930100e60 0x7fb930198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:59.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.568+0000 7fb92effd640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102040 0x7fb9301987d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb924006010 tx=0x7fb924004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.569+0000 7fb92cff9640 1 -- 172.21.15.82:0/283777098 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924043020 con 0x7fb930102040 2024-04-21T08:44:59.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.569+0000 7fb92cff9640 1 -- 172.21.15.82:0/283777098 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb924004440 con 0x7fb930102040 2024-04-21T08:44:59.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.569+0000 7fb935ffb640 1 -- 172.21.15.82:0/283777098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9301a00f0 con 0x7fb930102040 2024-04-21T08:44:59.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.569+0000 7fb92cff9640 1 -- 172.21.15.82:0/283777098 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924003bf0 con 0x7fb930102040 2024-04-21T08:44:59.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.569+0000 7fb935ffb640 1 -- 172.21.15.82:0/283777098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9301a0640 con 0x7fb930102040 2024-04-21T08:44:59.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.570+0000 7fb935ffb640 1 -- 172.21.15.82:0/283777098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9301a0960 con 0x7fb930102040 2024-04-21T08:44:59.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.571+0000 7fb92cff9640 1 -- 172.21.15.82:0/283777098 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb924046020 con 0x7fb930102040 2024-04-21T08:44:59.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.571+0000 7fb92cff9640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb9180a3bb0 0x7fb9180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:59.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.572+0000 7fb92f7fe640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb9180a3bb0 0x7fb9180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:59.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.572+0000 7fb92cff9640 1 -- 172.21.15.82:0/283777098 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb9240f2fb0 con 0x7fb930102040 2024-04-21T08:44:59.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.572+0000 7fb92f7fe640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb9180a3bb0 0x7fb9180a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb920005fd0 tx=0x7fb92000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:59.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.575+0000 7fb92cff9640 1 -- 172.21.15.82:0/283777098 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb92404d020 con 0x7fb930102040 2024-04-21T08:44:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:59 smithi134 ceph-mon[31772]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:44:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:59 smithi134 ceph-mon[31772]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T08:44:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:59 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/573054086' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-21T08:44:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:59 smithi134 ceph-mon[31772]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T08:44:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:59 smithi134 ceph-mon[31772]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:44:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:59 smithi134 ceph-mon[31772]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-21T08:44:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:59 smithi134 ceph-mon[31772]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-21T08:44:59.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:44:59 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3346767157' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T08:44:59.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.806+0000 7fb935ffb640 1 -- 172.21.15.82:0/283777098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb9301a0cf0 con 0x7fb930102040 2024-04-21T08:44:59.819 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:59 smithi082 ceph-mon[27967]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:44:59.819 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:59 smithi082 ceph-mon[27967]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T08:44:59.819 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:59 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/573054086' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-21T08:44:59.819 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:59 smithi082 ceph-mon[27967]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T08:44:59.819 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:59 smithi082 ceph-mon[27967]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:44:59.819 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:59 smithi082 ceph-mon[27967]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-21T08:44:59.819 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:59 smithi082 ceph-mon[27967]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-21T08:44:59.820 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:44:59 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3346767157' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T08:44:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:59 smithi106 ceph-mon[31072]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-21T08:44:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:59 smithi106 ceph-mon[31072]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T08:44:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:59 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/573054086' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-21T08:44:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:59 smithi106 ceph-mon[31072]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T08:44:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:59 smithi106 ceph-mon[31072]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:44:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:59 smithi106 ceph-mon[31072]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-21T08:44:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:59 smithi106 ceph-mon[31072]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-21T08:44:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:44:59 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3346767157' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T08:44:59.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.981+0000 7fb92cff9640 1 -- 172.21.15.82:0/283777098 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7fb9240bf750 con 0x7fb930102040 2024-04-21T08:44:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.984+0000 7fb935ffb640 1 -- 172.21.15.82:0/283777098 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb9180a3bb0 msgr2=0x7fb9180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.984+0000 7fb935ffb640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb9180a3bb0 0x7fb9180a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb920005fd0 tx=0x7fb92000b390 comp rx=0 tx=0).stop 2024-04-21T08:44:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.984+0000 7fb935ffb640 1 -- 172.21.15.82:0/283777098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102040 msgr2=0x7fb9301987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.984+0000 7fb935ffb640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102040 0x7fb9301987d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb924006010 tx=0x7fb924004040 comp rx=0 tx=0).stop 2024-04-21T08:44:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.984+0000 7fb935ffb640 1 -- 172.21.15.82:0/283777098 shutdown_connections 2024-04-21T08:44:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.984+0000 7fb935ffb640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb9180a3bb0 0x7fb9180a6070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.984+0000 7fb935ffb640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb930102a00 0x7fb93019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.984+0000 7fb935ffb640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb930102040 0x7fb9301987d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.984+0000 7fb935ffb640 1 --2- 172.21.15.82:0/283777098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb930100e60 0x7fb930198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.984+0000 7fb935ffb640 1 -- 172.21.15.82:0/283777098 >> 172.21.15.82:0/283777098 conn(0x7fb9300fc420 msgr2=0x7fb9300fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:59.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.984+0000 7fb935ffb640 1 -- 172.21.15.82:0/283777098 shutdown_connections 2024-04-21T08:44:59.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:44:59.984+0000 7fb935ffb640 1 -- 172.21.15.82:0/283777098 wait complete. 2024-04-21T08:44:59.997 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-21T08:44:59.997 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-21T08:45:00.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.099+0000 7f2835806640 1 -- 172.21.15.82:0/2731691058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2830107c60 msgr2=0x7f283010a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.099+0000 7f2835806640 1 --2- 172.21.15.82:0/2731691058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2830107c60 0x7f283010a0b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2828009ca0 tx=0x7f282802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.099+0000 7f2835806640 1 -- 172.21.15.82:0/2731691058 shutdown_connections 2024-04-21T08:45:00.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.099+0000 7f2835806640 1 --2- 172.21.15.82:0/2731691058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2830107c60 0x7f283010a0b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.099+0000 7f2835806640 1 --2- 172.21.15.82:0/2731691058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2830105330 0x7f2830107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.099+0000 7f2835806640 1 --2- 172.21.15.82:0/2731691058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2830102a00 0x7f2830104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.099+0000 7f2835806640 1 -- 172.21.15.82:0/2731691058 >> 172.21.15.82:0/2731691058 conn(0x7f28300fc3f0 msgr2=0x7f28300fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:00.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.099+0000 7f2835806640 1 -- 172.21.15.82:0/2731691058 shutdown_connections 2024-04-21T08:45:00.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.099+0000 7f2835806640 1 -- 172.21.15.82:0/2731691058 wait complete. 2024-04-21T08:45:00.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f2835806640 1 Processor -- start 2024-04-21T08:45:00.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f2835806640 1 -- start start 2024-04-21T08:45:00.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f2835806640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2830102a00 0x7f2830198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f2835806640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2830105330 0x7f28301986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f2835806640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2830107c60 0x7f283019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f2835806640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f283010dab0 con 0x7f2830102a00 2024-04-21T08:45:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f2835806640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f283010d930 con 0x7f2830105330 2024-04-21T08:45:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f2835806640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f283010dc30 con 0x7f2830107c60 2024-04-21T08:45:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f282effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2830102a00 0x7f2830198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f282effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2830102a00 0x7f2830198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37506/0 (socket says 172.21.15.82:37506) 2024-04-21T08:45:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f282effd640 1 -- 172.21.15.82:0/4235994443 learned_addr learned my addr 172.21.15.82:0/4235994443 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f282e7fc640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2830105330 0x7f28301986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f282f7fe640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2830107c60 0x7f283019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f282effd640 1 -- 172.21.15.82:0/4235994443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2830107c60 msgr2=0x7f283019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f282effd640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2830107c60 0x7f283019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f282effd640 1 -- 172.21.15.82:0/4235994443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2830105330 msgr2=0x7f28301986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:00.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f282effd640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2830105330 0x7f28301986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:00.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f282effd640 1 -- 172.21.15.82:0/4235994443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283019fda0 con 0x7f2830102a00 2024-04-21T08:45:00.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f282f7fe640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2830107c60 0x7f283019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:00.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.100+0000 7f282e7fc640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2830105330 0x7f28301986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:00.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.101+0000 7f282effd640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2830102a00 0x7f2830198180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f282000d950 tx=0x7f282000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.101+0000 7f2834804640 1 -- 172.21.15.82:0/4235994443 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2820017030 con 0x7f2830102a00 2024-04-21T08:45:00.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.101+0000 7f2835806640 1 -- 172.21.15.82:0/4235994443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28301a0090 con 0x7f2830102a00 2024-04-21T08:45:00.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.101+0000 7f2834804640 1 -- 172.21.15.82:0/4235994443 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2820004020 con 0x7f2830102a00 2024-04-21T08:45:00.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.101+0000 7f2834804640 1 -- 172.21.15.82:0/4235994443 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28200047a0 con 0x7f2830102a00 2024-04-21T08:45:00.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.101+0000 7f2835806640 1 -- 172.21.15.82:0/4235994443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28301a05e0 con 0x7f2830102a00 2024-04-21T08:45:00.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.102+0000 7f2835806640 1 -- 172.21.15.82:0/4235994443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28300fdee0 con 0x7f2830102a00 2024-04-21T08:45:00.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.103+0000 7f2834804640 1 -- 172.21.15.82:0/4235994443 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2820009a30 con 0x7f2830102a00 2024-04-21T08:45:00.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.103+0000 7f2834804640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f28180a3bb0 0x7f28180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:00.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.103+0000 7f282e7fc640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f28180a3bb0 0x7f28180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:00.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.103+0000 7f2834804640 1 -- 172.21.15.82:0/4235994443 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f28200d07a0 con 0x7f2830102a00 2024-04-21T08:45:00.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.104+0000 7f282e7fc640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f28180a3bb0 0x7f28180a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2830199690 tx=0x7f2824005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:00.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.107+0000 7f2834804640 1 -- 172.21.15.82:0/4235994443 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f282009cf40 con 0x7f2830102a00 2024-04-21T08:45:00.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:00.313+0000 7f2835806640 1 -- 172.21.15.82:0/4235994443 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2830062390 con 0x7f28180a3bb0 2024-04-21T08:45:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3346767157' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T08:45:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] up:active 2024-04-21T08:45:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T08:45:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-21T08:45:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-21T08:45:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-21T08:45:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/283777098' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T08:45:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/283777098' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T08:45:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] up:active 2024-04-21T08:45:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-21T08:45:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T08:45:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-04-21T08:45:00.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-mon[27967]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-04-21T08:45:00.610 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:45:00.318+0000 7fcdf25ca700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:45:00.610 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:45:00.318+0000 7fcdf25ca700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:45:00.610 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:45:00.318+0000 7fcdf25ca700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:45:00.610 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:45:00.318+0000 7fcdf25ca700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:45:00.610 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:45:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:45:00.318+0000 7fcdf25ca700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:45:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3346767157' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T08:45:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] up:active 2024-04-21T08:45:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T08:45:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-21T08:45:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-21T08:45:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-21T08:45:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/283777098' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T08:45:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/283777098' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T08:45:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] up:active 2024-04-21T08:45:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-21T08:45:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T08:45:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-04-21T08:45:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:00 smithi134 ceph-mon[31772]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-04-21T08:45:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3346767157' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T08:45:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] up:active 2024-04-21T08:45:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T08:45:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-21T08:45:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-21T08:45:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-21T08:45:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/283777098' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T08:45:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/283777098' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T08:45:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] up:active 2024-04-21T08:45:00.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-21T08:45:00.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T08:45:00.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-04-21T08:45:00.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:00 smithi106 ceph-mon[31072]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-04-21T08:45:01.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.015+0000 7f2834804640 1 -- 172.21.15.82:0/4235994443 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2830062390 con 0x7f28180a3bb0 2024-04-21T08:45:01.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.017+0000 7f2835806640 1 -- 172.21.15.82:0/4235994443 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f28180a3bb0 msgr2=0x7f28180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.017+0000 7f2835806640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f28180a3bb0 0x7f28180a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2830199690 tx=0x7f2824005f90 comp rx=0 tx=0).stop 2024-04-21T08:45:01.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.017+0000 7f2835806640 1 -- 172.21.15.82:0/4235994443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2830102a00 msgr2=0x7f2830198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.017+0000 7f2835806640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2830102a00 0x7f2830198180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f282000d950 tx=0x7f282000de10 comp rx=0 tx=0).stop 2024-04-21T08:45:01.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.017+0000 7f2835806640 1 -- 172.21.15.82:0/4235994443 shutdown_connections 2024-04-21T08:45:01.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.017+0000 7f2835806640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f28180a3bb0 0x7f28180a6070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.017+0000 7f2835806640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2830107c60 0x7f283019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.017+0000 7f2835806640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2830105330 0x7f28301986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.017+0000 7f2835806640 1 --2- 172.21.15.82:0/4235994443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2830102a00 0x7f2830198180 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.017+0000 7f2835806640 1 -- 172.21.15.82:0/4235994443 >> 172.21.15.82:0/4235994443 conn(0x7f28300fc3f0 msgr2=0x7f28301033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.017+0000 7f2835806640 1 -- 172.21.15.82:0/4235994443 shutdown_connections 2024-04-21T08:45:01.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.017+0000 7f2835806640 1 -- 172.21.15.82:0/4235994443 wait complete. 2024-04-21T08:45:01.029 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-21T08:45:01.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 -- 172.21.15.82:0/648887677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a901029a0 msgr2=0x7f1a901092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 --2- 172.21.15.82:0/648887677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a901029a0 0x7f1a901092c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009ca0 tx=0x7f1a8802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 -- 172.21.15.82:0/648887677 shutdown_connections 2024-04-21T08:45:01.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 --2- 172.21.15.82:0/648887677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a901029a0 0x7f1a901092c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 --2- 172.21.15.82:0/648887677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a90101fe0 0x7f1a90102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 --2- 172.21.15.82:0/648887677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a90100e90 0x7f1a901012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 -- 172.21.15.82:0/648887677 >> 172.21.15.82:0/648887677 conn(0x7f1a900fc400 msgr2=0x7f1a900fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 -- 172.21.15.82:0/648887677 shutdown_connections 2024-04-21T08:45:01.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 -- 172.21.15.82:0/648887677 wait complete. 2024-04-21T08:45:01.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 Processor -- start 2024-04-21T08:45:01.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 -- start start 2024-04-21T08:45:01.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a90100e90 0x7f1a90198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.135+0000 7f1a96cf2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a90101fe0 0x7f1a90198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a96cf2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a901029a0 0x7f1a9019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a96cf2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9010c9d0 con 0x7f1a90101fe0 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a96cf2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9010c850 con 0x7f1a90100e90 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a96cf2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9010cb50 con 0x7f1a901029a0 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a8ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a90101fe0 0x7f1a90198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a8ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a90101fe0 0x7f1a90198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37536/0 (socket says 172.21.15.82:37536) 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a8ffff640 1 -- 172.21.15.82:0/2010717102 learned_addr learned my addr 172.21.15.82:0/2010717102 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a94a8e640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a90100e90 0x7f1a90198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a8ffff640 1 -- 172.21.15.82:0/2010717102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a901029a0 msgr2=0x7f1a9019f700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a8ffff640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a901029a0 0x7f1a9019f700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a8ffff640 1 -- 172.21.15.82:0/2010717102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a90100e90 msgr2=0x7f1a90198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a8ffff640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a90100e90 0x7f1a90198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a8ffff640 1 -- 172.21.15.82:0/2010717102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9019fd70 con 0x7f1a90101fe0 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a8ffff640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a90101fe0 0x7f1a90198690 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400e9c0 tx=0x7f1a8400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a8dffb640 1 -- 172.21.15.82:0/2010717102 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8400ccc0 con 0x7f1a90101fe0 2024-04-21T08:45:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a8dffb640 1 -- 172.21.15.82:0/2010717102 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a840035f0 con 0x7f1a90101fe0 2024-04-21T08:45:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a96cf2640 1 -- 172.21.15.82:0/2010717102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a901a0060 con 0x7f1a90101fe0 2024-04-21T08:45:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a8dffb640 1 -- 172.21.15.82:0/2010717102 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8400ce60 con 0x7f1a90101fe0 2024-04-21T08:45:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.136+0000 7f1a96cf2640 1 -- 172.21.15.82:0/2010717102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a901a05b0 con 0x7f1a90101fe0 2024-04-21T08:45:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.137+0000 7f1a96cf2640 1 -- 172.21.15.82:0/2010717102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a901a0870 con 0x7f1a90101fe0 2024-04-21T08:45:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.138+0000 7f1a8dffb640 1 -- 172.21.15.82:0/2010717102 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1a840037a0 con 0x7f1a90101fe0 2024-04-21T08:45:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.139+0000 7f1a8dffb640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1a780a3bb0 0x7f1a780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.139+0000 7f1a94a8e640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1a780a3bb0 0x7f1a780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.139+0000 7f1a8dffb640 1 -- 172.21.15.82:0/2010717102 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1a840d0aa0 con 0x7f1a90101fe0 2024-04-21T08:45:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.140+0000 7f1a94a8e640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1a780a3bb0 0x7f1a780a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000abe0 tx=0x7f1a8000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.144+0000 7f1a8dffb640 1 -- 172.21.15.82:0/2010717102 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a8409d240 con 0x7f1a90101fe0 2024-04-21T08:45:01.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.349+0000 7f1a96cf2640 1 -- 172.21.15.82:0/2010717102 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1a900697f0 con 0x7f1a780a3bb0 2024-04-21T08:45:01.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.376+0000 7f1a8dffb640 1 -- 172.21.15.82:0/2010717102 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1a900697f0 con 0x7f1a780a3bb0 2024-04-21T08:45:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.378+0000 7f1a96cf2640 1 -- 172.21.15.82:0/2010717102 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1a780a3bb0 msgr2=0x7f1a780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.378+0000 7f1a96cf2640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1a780a3bb0 0x7f1a780a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000abe0 tx=0x7f1a8000a2d0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.378+0000 7f1a96cf2640 1 -- 172.21.15.82:0/2010717102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a90101fe0 msgr2=0x7f1a90198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.378+0000 7f1a96cf2640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a90101fe0 0x7f1a90198690 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400e9c0 tx=0x7f1a8400edc0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.378+0000 7f1a96cf2640 1 -- 172.21.15.82:0/2010717102 shutdown_connections 2024-04-21T08:45:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.378+0000 7f1a96cf2640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1a780a3bb0 0x7f1a780a6070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.378+0000 7f1a96cf2640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a901029a0 0x7f1a9019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.378+0000 7f1a96cf2640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a90101fe0 0x7f1a90198690 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.378+0000 7f1a96cf2640 1 --2- 172.21.15.82:0/2010717102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a90100e90 0x7f1a90198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.378+0000 7f1a96cf2640 1 -- 172.21.15.82:0/2010717102 >> 172.21.15.82:0/2010717102 conn(0x7f1a900fc400 msgr2=0x7f1a900690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.378+0000 7f1a96cf2640 1 -- 172.21.15.82:0/2010717102 shutdown_connections 2024-04-21T08:45:01.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.378+0000 7f1a96cf2640 1 -- 172.21.15.82:0/2010717102 wait complete. 2024-04-21T08:45:01.389 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-21T08:45:01.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.493+0000 7f9dfdbce640 1 -- 172.21.15.82:0/2581763016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9df8101b90 msgr2=0x7f9df8102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.493+0000 7f9dfdbce640 1 --2- 172.21.15.82:0/2581763016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9df8101b90 0x7f9df8102010 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009ca0 tx=0x7f9df002c180 comp rx=0 tx=0).stop 2024-04-21T08:45:01.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.493+0000 7f9dfdbce640 1 -- 172.21.15.82:0/2581763016 shutdown_connections 2024-04-21T08:45:01.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.493+0000 7f9dfdbce640 1 --2- 172.21.15.82:0/2581763016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9df8101b90 0x7f9df8102010 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.493+0000 7f9dfdbce640 1 --2- 172.21.15.82:0/2581763016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9df81011f0 0x7f9df8101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.493+0000 7f9dfdbce640 1 --2- 172.21.15.82:0/2581763016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9df8100890 0x7f9df8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.493+0000 7f9dfdbce640 1 -- 172.21.15.82:0/2581763016 >> 172.21.15.82:0/2581763016 conn(0x7f9df80fc430 msgr2=0x7f9df80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.493+0000 7f9dfdbce640 1 -- 172.21.15.82:0/2581763016 shutdown_connections 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.493+0000 7f9dfdbce640 1 -- 172.21.15.82:0/2581763016 wait complete. 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.494+0000 7f9dfdbce640 1 Processor -- start 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.494+0000 7f9dfdbce640 1 -- start start 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.494+0000 7f9dfdbce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9df8100890 0x7f9df819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.494+0000 7f9dfdbce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9df81011f0 0x7f9df819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.494+0000 7f9dfdbce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9df8101b90 0x7f9df81a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.494+0000 7f9dfdbce640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9df810fd20 con 0x7f9df8101b90 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.494+0000 7f9dfdbce640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9df810fba0 con 0x7f9df8100890 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.494+0000 7f9dfdbce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9df810fea0 con 0x7f9df81011f0 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.494+0000 7f9df7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9df8101b90 0x7f9df81a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.494+0000 7f9df7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9df8101b90 0x7f9df81a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37558/0 (socket says 172.21.15.82:37558) 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.494+0000 7f9df7fff640 1 -- 172.21.15.82:0/53021553 learned_addr learned my addr 172.21.15.82:0/53021553 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.494+0000 7f9df77fe640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9df8100890 0x7f9df819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.495+0000 7f9df7fff640 1 -- 172.21.15.82:0/53021553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9df81011f0 msgr2=0x7f9df819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.495+0000 7f9df7fff640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9df81011f0 0x7f9df819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.495+0000 7f9df7fff640 1 -- 172.21.15.82:0/53021553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9df8100890 msgr2=0x7f9df819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.495+0000 7f9df7fff640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9df8100890 0x7f9df819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.495+0000 7f9df7fff640 1 -- 172.21.15.82:0/53021553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df81a1fb0 con 0x7f9df8101b90 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.495+0000 7f9df7fff640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9df8101b90 0x7f9df81a1940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9df0005e30 tx=0x7f9df0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.495+0000 7f9df57fa640 1 -- 172.21.15.82:0/53021553 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0039040 con 0x7f9df8101b90 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.495+0000 7f9df57fa640 1 -- 172.21.15.82:0/53021553 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9df0044070 con 0x7f9df8101b90 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.495+0000 7f9df57fa640 1 -- 172.21.15.82:0/53021553 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df00328a0 con 0x7f9df8101b90 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.495+0000 7f9dfdbce640 1 -- 172.21.15.82:0/53021553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df81a2240 con 0x7f9df8101b90 2024-04-21T08:45:01.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.495+0000 7f9dfdbce640 1 -- 172.21.15.82:0/53021553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df81a27b0 con 0x7f9df8101b90 2024-04-21T08:45:01.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.496+0000 7f9dfdbce640 1 -- 172.21.15.82:0/53021553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df810b720 con 0x7f9df8101b90 2024-04-21T08:45:01.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.497+0000 7f9df57fa640 1 -- 172.21.15.82:0/53021553 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9df0032b00 con 0x7f9df8101b90 2024-04-21T08:45:01.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.498+0000 7f9df57fa640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9de40a3bb0 0x7f9de40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.498+0000 7f9df77fe640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9de40a3bb0 0x7f9de40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.498+0000 7f9df57fa640 1 -- 172.21.15.82:0/53021553 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9df00f3690 con 0x7f9df8101b90 2024-04-21T08:45:01.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.498+0000 7f9df77fe640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9de40a3bb0 0x7f9de40a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009bd0 tx=0x7f9de8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.501+0000 7f9df57fa640 1 -- 172.21.15.82:0/53021553 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9df00bfe30 con 0x7f9df8101b90 2024-04-21T08:45:01.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.710+0000 7f9dfdbce640 1 -- 172.21.15.82:0/53021553 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f9df8062420 con 0x7f9de40a3bb0 2024-04-21T08:45:01.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.715+0000 7f9df57fa640 1 -- 172.21.15.82:0/53021553 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9df8062420 con 0x7f9de40a3bb0 2024-04-21T08:45:01.715 INFO:teuthology.orchestra.run.smithi082.stdout:/volumes/qa/sv_0/59bce23f-7bb6-4400-b27d-e29a7521613e 2024-04-21T08:45:01.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.717+0000 7f9dfdbce640 1 -- 172.21.15.82:0/53021553 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9de40a3bb0 msgr2=0x7f9de40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.717+0000 7f9dfdbce640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9de40a3bb0 0x7f9de40a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009bd0 tx=0x7f9de8005f90 comp rx=0 tx=0).stop 2024-04-21T08:45:01.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.717+0000 7f9dfdbce640 1 -- 172.21.15.82:0/53021553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9df8101b90 msgr2=0x7f9df81a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.717+0000 7f9dfdbce640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9df8101b90 0x7f9df81a1940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9df0005e30 tx=0x7f9df0003c20 comp rx=0 tx=0).stop 2024-04-21T08:45:01.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.717+0000 7f9dfdbce640 1 -- 172.21.15.82:0/53021553 shutdown_connections 2024-04-21T08:45:01.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.717+0000 7f9dfdbce640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9de40a3bb0 0x7f9de40a6070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.717+0000 7f9dfdbce640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9df8101b90 0x7f9df81a1940 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.717+0000 7f9dfdbce640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9df81011f0 0x7f9df819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.717+0000 7f9dfdbce640 1 --2- 172.21.15.82:0/53021553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9df8100890 0x7f9df819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.717+0000 7f9dfdbce640 1 -- 172.21.15.82:0/53021553 >> 172.21.15.82:0/53021553 conn(0x7f9df80fc430 msgr2=0x7f9df80fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.717+0000 7f9dfdbce640 1 -- 172.21.15.82:0/53021553 shutdown_connections 2024-04-21T08:45:01.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.717+0000 7f9dfdbce640 1 -- 172.21.15.82:0/53021553 wait complete. 2024-04-21T08:45:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:01 smithi134 ceph-mon[31772]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:01 smithi134 ceph-mon[31772]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-21T08:45:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:01 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] up:active 2024-04-21T08:45:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:01 smithi134 ceph-mon[31772]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-21T08:45:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:01 smithi134 ceph-mon[31772]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T08:45:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:01 smithi134 ceph-mon[31772]: Cluster is now healthy 2024-04-21T08:45:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:01 smithi134 ceph-mon[31772]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-04-21T08:45:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:01 smithi134 ceph-mon[31772]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-21T08:45:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:01 smithi134 ceph-mon[31772]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-04-21T08:45:01.727 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:01 smithi082 ceph-mon[27967]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:01.727 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:01 smithi082 ceph-mon[27967]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-21T08:45:01.727 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:01 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] up:active 2024-04-21T08:45:01.727 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:01 smithi082 ceph-mon[27967]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-21T08:45:01.727 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:01 smithi082 ceph-mon[27967]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T08:45:01.727 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:01 smithi082 ceph-mon[27967]: Cluster is now healthy 2024-04-21T08:45:01.727 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:01 smithi082 ceph-mon[27967]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-04-21T08:45:01.727 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:01 smithi082 ceph-mon[27967]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-21T08:45:01.727 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:01 smithi082 ceph-mon[27967]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-04-21T08:45:01.728 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-21T08:45:01.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.831+0000 7f1ae0754640 1 -- 172.21.15.82:0/2377595544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad8101fa0 msgr2=0x7f1ad81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.831+0000 7f1ae0754640 1 --2- 172.21.15.82:0/2377595544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad8101fa0 0x7f1ad81023c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009900 tx=0x7f1ad402bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:01.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 -- 172.21.15.82:0/2377595544 shutdown_connections 2024-04-21T08:45:01.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 --2- 172.21.15.82:0/2377595544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ad8103b50 0x7f1ad810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 --2- 172.21.15.82:0/2377595544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ad8103190 0x7f1ad8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 --2- 172.21.15.82:0/2377595544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad8101fa0 0x7f1ad81023c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 -- 172.21.15.82:0/2377595544 >> 172.21.15.82:0/2377595544 conn(0x7f1ad80fd530 msgr2=0x7f1ad80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 -- 172.21.15.82:0/2377595544 shutdown_connections 2024-04-21T08:45:01.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 -- 172.21.15.82:0/2377595544 wait complete. 2024-04-21T08:45:01.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 Processor -- start 2024-04-21T08:45:01.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 -- start start 2024-04-21T08:45:01.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ad8101fa0 0x7f1ad81981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad8103190 0x7f1ad8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ad8103b50 0x7f1ad819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad810dae0 con 0x7f1ad8103190 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad810d960 con 0x7f1ad8103b50 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1ae0754640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad810dc60 con 0x7f1ad8101fa0 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1addcef640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad8103190 0x7f1ad8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1addcef640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad8103190 0x7f1ad8198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37572/0 (socket says 172.21.15.82:37572) 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1addcef640 1 -- 172.21.15.82:0/1111413155 learned_addr learned my addr 172.21.15.82:0/1111413155 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.832+0000 7f1adecf1640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ad8103b50 0x7f1ad819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.833+0000 7f1ade4f0640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ad8101fa0 0x7f1ad81981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.833+0000 7f1addcef640 1 -- 172.21.15.82:0/1111413155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ad8101fa0 msgr2=0x7f1ad81981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.833+0000 7f1addcef640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ad8101fa0 0x7f1ad81981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.833+0000 7f1addcef640 1 -- 172.21.15.82:0/1111413155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ad8103b50 msgr2=0x7f1ad819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.833+0000 7f1addcef640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ad8103b50 0x7f1ad819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.833+0000 7f1addcef640 1 -- 172.21.15.82:0/1111413155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad819fde0 con 0x7f1ad8103190 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.833+0000 7f1addcef640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad8103190 0x7f1ad8198700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800c920 tx=0x7f1ac800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.833+0000 7f1acf7fe640 1 -- 172.21.15.82:0/1111413155 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8007bb0 con 0x7f1ad8103190 2024-04-21T08:45:01.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.833+0000 7f1ae0754640 1 -- 172.21.15.82:0/1111413155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad81a00d0 con 0x7f1ad8103190 2024-04-21T08:45:01.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.833+0000 7f1acf7fe640 1 -- 172.21.15.82:0/1111413155 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac8007d50 con 0x7f1ad8103190 2024-04-21T08:45:01.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.833+0000 7f1acf7fe640 1 -- 172.21.15.82:0/1111413155 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8018730 con 0x7f1ad8103190 2024-04-21T08:45:01.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.833+0000 7f1ae0754640 1 -- 172.21.15.82:0/1111413155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad81a0620 con 0x7f1ad8103190 2024-04-21T08:45:01.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.834+0000 7f1ae0754640 1 -- 172.21.15.82:0/1111413155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad8109380 con 0x7f1ad8103190 2024-04-21T08:45:01.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.835+0000 7f1acf7fe640 1 -- 172.21.15.82:0/1111413155 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1ac800f450 con 0x7f1ad8103190 2024-04-21T08:45:01.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.836+0000 7f1acf7fe640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1ac40a3bb0 0x7f1ac40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.836+0000 7f1ade4f0640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1ac40a3bb0 0x7f1ac40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.836+0000 7f1acf7fe640 1 -- 172.21.15.82:0/1111413155 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1ac80d0770 con 0x7f1ad8103190 2024-04-21T08:45:01.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.836+0000 7f1ade4f0640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1ac40a3bb0 0x7f1ac40a6070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4006010 tx=0x7f1ad4005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:01.840+0000 7f1acf7fe640 1 -- 172.21.15.82:0/1111413155 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ac8014030 con 0x7f1ad8103190 2024-04-21T08:45:01.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:01 smithi106 ceph-mon[31072]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:01.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:01 smithi106 ceph-mon[31072]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-21T08:45:01.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:01 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] up:active 2024-04-21T08:45:01.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:01 smithi106 ceph-mon[31072]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-21T08:45:01.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:01 smithi106 ceph-mon[31072]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T08:45:01.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:01 smithi106 ceph-mon[31072]: Cluster is now healthy 2024-04-21T08:45:01.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:01 smithi106 ceph-mon[31072]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-04-21T08:45:01.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:01 smithi106 ceph-mon[31072]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-21T08:45:01.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:01 smithi106 ceph-mon[31072]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-04-21T08:45:02.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.049+0000 7f1ae0754640 1 -- 172.21.15.82:0/1111413155 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1ad8062390 con 0x7f1ac40a3bb0 2024-04-21T08:45:02.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.072+0000 7f1acf7fe640 1 -- 172.21.15.82:0/1111413155 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1ad8062390 con 0x7f1ac40a3bb0 2024-04-21T08:45:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.074+0000 7f1ae0754640 1 -- 172.21.15.82:0/1111413155 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1ac40a3bb0 msgr2=0x7f1ac40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.074+0000 7f1ae0754640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1ac40a3bb0 0x7f1ac40a6070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4006010 tx=0x7f1ad4005be0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.074+0000 7f1ae0754640 1 -- 172.21.15.82:0/1111413155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad8103190 msgr2=0x7f1ad8198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.074+0000 7f1ae0754640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad8103190 0x7f1ad8198700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800c920 tx=0x7f1ac800cd20 comp rx=0 tx=0).stop 2024-04-21T08:45:02.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.074+0000 7f1ae0754640 1 -- 172.21.15.82:0/1111413155 shutdown_connections 2024-04-21T08:45:02.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.074+0000 7f1ae0754640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f1ac40a3bb0 0x7f1ac40a6070 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.074+0000 7f1ae0754640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ad8103b50 0x7f1ad819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.074+0000 7f1ae0754640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad8103190 0x7f1ad8198700 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.074+0000 7f1ae0754640 1 --2- 172.21.15.82:0/1111413155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ad8101fa0 0x7f1ad81981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.074+0000 7f1ae0754640 1 -- 172.21.15.82:0/1111413155 >> 172.21.15.82:0/1111413155 conn(0x7f1ad80fd530 msgr2=0x7f1ad8106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:02.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.075+0000 7f1ae0754640 1 -- 172.21.15.82:0/1111413155 shutdown_connections 2024-04-21T08:45:02.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.075+0000 7f1ae0754640 1 -- 172.21.15.82:0/1111413155 wait complete. 2024-04-21T08:45:02.085 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-21T08:45:02.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.188+0000 7f9ca73e6640 1 -- 172.21.15.82:0/2171261166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca0100020 msgr2=0x7f9ca01004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.188+0000 7f9ca73e6640 1 --2- 172.21.15.82:0/2171261166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca0100020 0x7f9ca01004a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009900 tx=0x7f9c9002bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:02.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.188+0000 7f9ca73e6640 1 -- 172.21.15.82:0/2171261166 shutdown_connections 2024-04-21T08:45:02.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.188+0000 7f9ca73e6640 1 --2- 172.21.15.82:0/2171261166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca01009e0 0x7f9ca010a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.188+0000 7f9ca73e6640 1 --2- 172.21.15.82:0/2171261166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca0100020 0x7f9ca01004a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.188+0000 7f9ca73e6640 1 --2- 172.21.15.82:0/2171261166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca01035b0 0x7f9ca01039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.188+0000 7f9ca73e6640 1 -- 172.21.15.82:0/2171261166 >> 172.21.15.82:0/2171261166 conn(0x7f9ca00fbba0 msgr2=0x7f9ca00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.188+0000 7f9ca73e6640 1 -- 172.21.15.82:0/2171261166 shutdown_connections 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.188+0000 7f9ca73e6640 1 -- 172.21.15.82:0/2171261166 wait complete. 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca73e6640 1 Processor -- start 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca73e6640 1 -- start start 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca73e6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca0100020 0x7f9ca0102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca73e6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca01009e0 0x7f9ca01012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca73e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca01035b0 0x7f9ca0101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca73e6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca010db40 con 0x7f9ca0100020 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca73e6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca010d9c0 con 0x7f9ca01009e0 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca73e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca010dcc0 con 0x7f9ca01035b0 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca5182640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca0100020 0x7f9ca0102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca5182640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca0100020 0x7f9ca0102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37604/0 (socket says 172.21.15.82:37604) 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca5182640 1 -- 172.21.15.82:0/701194249 learned_addr learned my addr 172.21.15.82:0/701194249 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:02.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca4981640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca01009e0 0x7f9ca01012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:02.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca5182640 1 -- 172.21.15.82:0/701194249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca01035b0 msgr2=0x7f9ca0101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.189+0000 7f9ca5182640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca01035b0 0x7f9ca0101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.190+0000 7f9ca5182640 1 -- 172.21.15.82:0/701194249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca01009e0 msgr2=0x7f9ca01012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.190+0000 7f9ca5182640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca01009e0 0x7f9ca01012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.190+0000 7f9ca5182640 1 -- 172.21.15.82:0/701194249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca01020c0 con 0x7f9ca0100020 2024-04-21T08:45:02.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.190+0000 7f9ca5182640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca0100020 0x7f9ca0102cb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00d950 tx=0x7f9c9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:02.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.190+0000 7f9c967fc640 1 -- 172.21.15.82:0/701194249 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c017070 con 0x7f9ca0100020 2024-04-21T08:45:02.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.190+0000 7f9ca73e6640 1 -- 172.21.15.82:0/701194249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca01a22e0 con 0x7f9ca0100020 2024-04-21T08:45:02.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.190+0000 7f9c967fc640 1 -- 172.21.15.82:0/701194249 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c9c009c30 con 0x7f9ca0100020 2024-04-21T08:45:02.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.190+0000 7f9c967fc640 1 -- 172.21.15.82:0/701194249 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c00f620 con 0x7f9ca0100020 2024-04-21T08:45:02.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.190+0000 7f9ca73e6640 1 -- 172.21.15.82:0/701194249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca01a2800 con 0x7f9ca0100020 2024-04-21T08:45:02.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.191+0000 7f9ca73e6640 1 -- 172.21.15.82:0/701194249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca00691c0 con 0x7f9ca0100020 2024-04-21T08:45:02.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.193+0000 7f9c967fc640 1 -- 172.21.15.82:0/701194249 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9c9c01a070 con 0x7f9ca0100020 2024-04-21T08:45:02.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.193+0000 7f9c967fc640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9c8c0a3bb0 0x7f9c8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:02.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.193+0000 7f9ca4981640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9c8c0a3bb0 0x7f9c8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:02.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.193+0000 7f9c967fc640 1 -- 172.21.15.82:0/701194249 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9c9c0d00d0 con 0x7f9ca0100020 2024-04-21T08:45:02.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.194+0000 7f9ca4981640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9c8c0a3bb0 0x7f9c8c0a6070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009230 tx=0x7f9c90005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:02.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.197+0000 7f9c967fc640 1 -- 172.21.15.82:0/701194249 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c9c009de0 con 0x7f9ca0100020 2024-04-21T08:45:02.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.404+0000 7f9ca73e6640 1 -- 172.21.15.82:0/701194249 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f9ca0062420 con 0x7f9c8c0a3bb0 2024-04-21T08:45:02.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.408+0000 7f9c967fc640 1 -- 172.21.15.82:0/701194249 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9ca0062420 con 0x7f9c8c0a3bb0 2024-04-21T08:45:02.408 INFO:teuthology.orchestra.run.smithi082.stdout:/volumes/qa/sv_1/6bfc719b-e3be-4e08-984a-1bbd1bc02cbc 2024-04-21T08:45:02.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.410+0000 7f9ca73e6640 1 -- 172.21.15.82:0/701194249 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9c8c0a3bb0 msgr2=0x7f9c8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.410+0000 7f9ca73e6640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9c8c0a3bb0 0x7f9c8c0a6070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009230 tx=0x7f9c90005fd0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.410+0000 7f9ca73e6640 1 -- 172.21.15.82:0/701194249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca0100020 msgr2=0x7f9ca0102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.410+0000 7f9ca73e6640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca0100020 0x7f9ca0102cb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c00d950 tx=0x7f9c9c00de10 comp rx=0 tx=0).stop 2024-04-21T08:45:02.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.410+0000 7f9ca73e6640 1 -- 172.21.15.82:0/701194249 shutdown_connections 2024-04-21T08:45:02.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.410+0000 7f9ca73e6640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9c8c0a3bb0 0x7f9c8c0a6070 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.410+0000 7f9ca73e6640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca01035b0 0x7f9ca0101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.410+0000 7f9ca73e6640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ca01009e0 0x7f9ca01012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.410+0000 7f9ca73e6640 1 --2- 172.21.15.82:0/701194249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ca0100020 0x7f9ca0102cb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.411+0000 7f9ca73e6640 1 -- 172.21.15.82:0/701194249 >> 172.21.15.82:0/701194249 conn(0x7f9ca00fbba0 msgr2=0x7f9ca0106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:02.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.411+0000 7f9ca73e6640 1 -- 172.21.15.82:0/701194249 shutdown_connections 2024-04-21T08:45:02.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.411+0000 7f9ca73e6640 1 -- 172.21.15.82:0/701194249 wait complete. 2024-04-21T08:45:02.421 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:45:02.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.522+0000 7fe7858ba640 1 -- 172.21.15.82:0/2402908237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe780105320 msgr2=0x7fe780107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.522+0000 7fe7858ba640 1 --2- 172.21.15.82:0/2402908237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe780105320 0x7fe780107710 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe768009860 tx=0x7fe76802bb70 comp rx=0 tx=0).stop 2024-04-21T08:45:02.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.522+0000 7fe7858ba640 1 -- 172.21.15.82:0/2402908237 shutdown_connections 2024-04-21T08:45:02.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.522+0000 7fe7858ba640 1 --2- 172.21.15.82:0/2402908237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe780107c50 0x7fe78010a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.522+0000 7fe7858ba640 1 --2- 172.21.15.82:0/2402908237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe780105320 0x7fe780107710 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.522+0000 7fe7858ba640 1 --2- 172.21.15.82:0/2402908237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7801029f0 0x7fe780104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.522+0000 7fe7858ba640 1 -- 172.21.15.82:0/2402908237 >> 172.21.15.82:0/2402908237 conn(0x7fe7800fc3e0 msgr2=0x7fe7800fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:02.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.522+0000 7fe7858ba640 1 -- 172.21.15.82:0/2402908237 shutdown_connections 2024-04-21T08:45:02.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.522+0000 7fe7858ba640 1 -- 172.21.15.82:0/2402908237 wait complete. 2024-04-21T08:45:02.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe7858ba640 1 Processor -- start 2024-04-21T08:45:02.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe7858ba640 1 -- start start 2024-04-21T08:45:02.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe7858ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7801029f0 0x7fe78019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:02.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe7858ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe780105320 0x7fe78019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:02.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe7858ba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe780107c50 0x7fe7801a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:02.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe7858ba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7801024a0 con 0x7fe780107c50 2024-04-21T08:45:02.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe7858ba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe780102320 con 0x7fe780105320 2024-04-21T08:45:02.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe7858ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe780102620 con 0x7fe7801029f0 2024-04-21T08:45:02.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe77f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe780107c50 0x7fe7801a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:02.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe77e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe780105320 0x7fe78019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:02.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe77f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe780107c50 0x7fe7801a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37624/0 (socket says 172.21.15.82:37624) 2024-04-21T08:45:02.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe77f7fe640 1 -- 172.21.15.82:0/2388589415 learned_addr learned my addr 172.21.15.82:0/2388589415 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:02.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.523+0000 7fe77effd640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7801029f0 0x7fe78019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe77f7fe640 1 -- 172.21.15.82:0/2388589415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7801029f0 msgr2=0x7fe78019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe77f7fe640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7801029f0 0x7fe78019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe77f7fe640 1 -- 172.21.15.82:0/2388589415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe780105320 msgr2=0x7fe78019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe77f7fe640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe780105320 0x7fe78019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe77f7fe640 1 -- 172.21.15.82:0/2388589415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7801a1fd0 con 0x7fe780107c50 2024-04-21T08:45:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe77effd640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7801029f0 0x7fe78019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe77e7fc640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe780105320 0x7fe78019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe77f7fe640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe780107c50 0x7fe7801a1960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe77800e970 tx=0x7fe77800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe7848b8640 1 -- 172.21.15.82:0/2388589415 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77800ccc0 con 0x7fe780107c50 2024-04-21T08:45:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe7848b8640 1 -- 172.21.15.82:0/2388589415 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77800ce60 con 0x7fe780107c50 2024-04-21T08:45:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe7858ba640 1 -- 172.21.15.82:0/2388589415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7801a22c0 con 0x7fe780107c50 2024-04-21T08:45:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe7848b8640 1 -- 172.21.15.82:0/2388589415 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778010430 con 0x7fe780107c50 2024-04-21T08:45:02.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.524+0000 7fe7858ba640 1 -- 172.21.15.82:0/2388589415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7801a2830 con 0x7fe780107c50 2024-04-21T08:45:02.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.525+0000 7fe7858ba640 1 -- 172.21.15.82:0/2388589415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7800fded0 con 0x7fe780107c50 2024-04-21T08:45:02.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.526+0000 7fe7848b8640 1 -- 172.21.15.82:0/2388589415 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe778004f90 con 0x7fe780107c50 2024-04-21T08:45:02.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.526+0000 7fe7848b8640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe76c0a3c30 0x7fe76c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:02.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.527+0000 7fe77effd640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe76c0a3c30 0x7fe76c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:02.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.527+0000 7fe7848b8640 1 -- 172.21.15.82:0/2388589415 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe7780d0a40 con 0x7fe780107c50 2024-04-21T08:45:02.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.527+0000 7fe77effd640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe76c0a3c30 0x7fe76c0a60f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe770005fd0 tx=0x7fe7700095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:02.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.531+0000 7fe7848b8640 1 -- 172.21.15.82:0/2388589415 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe77809d1e0 con 0x7fe780107c50 2024-04-21T08:45:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:02 smithi134 ceph-mon[31772]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:02 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] up:active 2024-04-21T08:45:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:02 smithi134 ceph-mon[31772]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-21T08:45:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:02 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:45:02.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.748+0000 7fe7858ba640 1 -- 172.21.15.82:0/2388589415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe78002d5c0 con 0x7fe780107c50 2024-04-21T08:45:02.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.748+0000 7fe7848b8640 1 -- 172.21.15.82:0/2388589415 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7fe77809cae0 con 0x7fe780107c50 2024-04-21T08:45:02.748 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:45:02.748 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:45:02.005557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24481,24509,24476,24409,24407]},"id":1}]} 2024-04-21T08:45:02.749 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-04-21T08:45:02.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.750+0000 7fe7858ba640 1 -- 172.21.15.82:0/2388589415 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe76c0a3c30 msgr2=0x7fe76c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.750+0000 7fe7858ba640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe76c0a3c30 0x7fe76c0a60f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe770005fd0 tx=0x7fe7700095c0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.750+0000 7fe7858ba640 1 -- 172.21.15.82:0/2388589415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe780107c50 msgr2=0x7fe7801a1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.750+0000 7fe7858ba640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe780107c50 0x7fe7801a1960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe77800e970 tx=0x7fe77800ed70 comp rx=0 tx=0).stop 2024-04-21T08:45:02.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.750+0000 7fe7858ba640 1 -- 172.21.15.82:0/2388589415 shutdown_connections 2024-04-21T08:45:02.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.750+0000 7fe7858ba640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe76c0a3c30 0x7fe76c0a60f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.750+0000 7fe7858ba640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe780107c50 0x7fe7801a1960 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.750+0000 7fe7858ba640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe780105320 0x7fe78019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.750+0000 7fe7858ba640 1 --2- 172.21.15.82:0/2388589415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7801029f0 0x7fe78019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.750+0000 7fe7858ba640 1 -- 172.21.15.82:0/2388589415 >> 172.21.15.82:0/2388589415 conn(0x7fe7800fc3e0 msgr2=0x7fe780104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:02.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.751+0000 7fe7858ba640 1 -- 172.21.15.82:0/2388589415 shutdown_connections 2024-04-21T08:45:02.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.751+0000 7fe7858ba640 1 -- 172.21.15.82:0/2388589415 wait complete. 2024-04-21T08:45:02.760 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:02 smithi082 ceph-mon[27967]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:02.760 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:02 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] up:active 2024-04-21T08:45:02.760 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:02 smithi082 ceph-mon[27967]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-21T08:45:02.760 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:02 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:45:02.762 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:45:02.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.859+0000 7fecf4115640 1 -- 172.21.15.82:0/2472985690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecec101fd0 msgr2=0x7fecec102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.859+0000 7fecf4115640 1 --2- 172.21.15.82:0/2472985690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecec101fd0 0x7fecec102450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fece8009900 tx=0x7fece802bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:02.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.859+0000 7fecf4115640 1 -- 172.21.15.82:0/2472985690 shutdown_connections 2024-04-21T08:45:02.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.859+0000 7fecf4115640 1 --2- 172.21.15.82:0/2472985690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecec102990 0x7fecec1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.859+0000 7fecf4115640 1 --2- 172.21.15.82:0/2472985690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecec101fd0 0x7fecec102450 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.859+0000 7fecf4115640 1 --2- 172.21.15.82:0/2472985690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecec100e80 0x7fecec1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.859+0000 7fecf4115640 1 -- 172.21.15.82:0/2472985690 >> 172.21.15.82:0/2472985690 conn(0x7fecec0fc410 msgr2=0x7fecec0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:02.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf4115640 1 -- 172.21.15.82:0/2472985690 shutdown_connections 2024-04-21T08:45:02.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf4115640 1 -- 172.21.15.82:0/2472985690 wait complete. 2024-04-21T08:45:02.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf4115640 1 Processor -- start 2024-04-21T08:45:02.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf4115640 1 -- start start 2024-04-21T08:45:02.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf4115640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecec100e80 0x7fecec06b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:02.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf4115640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecec101fd0 0x7fecec19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf4115640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecec102990 0x7fecec19f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf4115640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fecec10c9c0 con 0x7fecec101fd0 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf4115640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fecec10c840 con 0x7fecec102990 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf4115640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fecec10cb40 con 0x7fecec100e80 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf16b0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecec101fd0 0x7fecec19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf16b0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecec101fd0 0x7fecec19ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58340/0 (socket says 172.21.15.82:58340) 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf16b0640 1 -- 172.21.15.82:0/3837380000 learned_addr learned my addr 172.21.15.82:0/3837380000 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.860+0000 7fecf26b2640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecec102990 0x7fecec19f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fecf1eb1640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecec100e80 0x7fecec06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fecf16b0640 1 -- 172.21.15.82:0/3837380000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecec100e80 msgr2=0x7fecec06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fecf16b0640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecec100e80 0x7fecec06b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fecf16b0640 1 -- 172.21.15.82:0/3837380000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecec102990 msgr2=0x7fecec19f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fecf16b0640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecec102990 0x7fecec19f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:02.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fecf16b0640 1 -- 172.21.15.82:0/3837380000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec19fdd0 con 0x7fecec101fd0 2024-04-21T08:45:02.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fecf1eb1640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecec100e80 0x7fecec06b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:02.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fecf16b0640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecec101fd0 0x7fecec19ced0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fece8000c00 tx=0x7fece8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:02.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fece2ffd640 1 -- 172.21.15.82:0/3837380000 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8043020 con 0x7fecec101fd0 2024-04-21T08:45:02.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fece2ffd640 1 -- 172.21.15.82:0/3837380000 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece8004440 con 0x7fecec101fd0 2024-04-21T08:45:02.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fecf4115640 1 -- 172.21.15.82:0/3837380000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec1a0060 con 0x7fecec101fd0 2024-04-21T08:45:02.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fece2ffd640 1 -- 172.21.15.82:0/3837380000 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8003c90 con 0x7fecec101fd0 2024-04-21T08:45:02.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.861+0000 7fecf4115640 1 -- 172.21.15.82:0/3837380000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec1a05b0 con 0x7fecec101fd0 2024-04-21T08:45:02.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.862+0000 7fecf4115640 1 -- 172.21.15.82:0/3837380000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecec1081b0 con 0x7fecec101fd0 2024-04-21T08:45:02.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.863+0000 7fece2ffd640 1 -- 172.21.15.82:0/3837380000 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fece8046020 con 0x7fecec101fd0 2024-04-21T08:45:02.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.864+0000 7fece2ffd640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fecd80a3bb0 0x7fecd80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:02.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.864+0000 7fecf1eb1640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fecd80a3bb0 0x7fecd80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:02.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.864+0000 7fece2ffd640 1 -- 172.21.15.82:0/3837380000 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fece80f2860 con 0x7fecec101fd0 2024-04-21T08:45:02.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.864+0000 7fecf1eb1640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fecd80a3bb0 0x7fecd80a6070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fecdc005d00 tx=0x7fecdc005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:02.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:02.868+0000 7fece2ffd640 1 -- 172.21.15.82:0/3837380000 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fece804d020 con 0x7fecec101fd0 2024-04-21T08:45:02.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:02 smithi106 ceph-mon[31072]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:02.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:02 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] up:active 2024-04-21T08:45:02.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:02 smithi106 ceph-mon[31072]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-21T08:45:02.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:02 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:45:03.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.022+0000 7fece2ffd640 1 -- 172.21.15.82:0/3837380000 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fece80baf70 con 0x7fecec101fd0 2024-04-21T08:45:03.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.064+0000 7fecf4115640 1 -- 172.21.15.82:0/3837380000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecec062b50 con 0x7fecec101fd0 2024-04-21T08:45:03.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.066+0000 7fece2ffd640 1 -- 172.21.15.82:0/3837380000 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24140 (secure 0 0 0) 0x7fece80bbf50 con 0x7fecec101fd0 2024-04-21T08:45:03.066 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:45:03.066 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":89,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:44:38.858333+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:45:03.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.068+0000 7fecf4115640 1 -- 172.21.15.82:0/3837380000 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fecd80a3bb0 msgr2=0x7fecd80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:03.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.068+0000 7fecf4115640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fecd80a3bb0 0x7fecd80a6070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fecdc005d00 tx=0x7fecdc005ed0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.068+0000 7fecf4115640 1 -- 172.21.15.82:0/3837380000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecec101fd0 msgr2=0x7fecec19ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:03.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.068+0000 7fecf4115640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecec101fd0 0x7fecec19ced0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fece8000c00 tx=0x7fece8004040 comp rx=0 tx=0).stop 2024-04-21T08:45:03.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.068+0000 7fecf4115640 1 -- 172.21.15.82:0/3837380000 shutdown_connections 2024-04-21T08:45:03.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.068+0000 7fecf4115640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fecd80a3bb0 0x7fecd80a6070 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.068+0000 7fecf4115640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecec102990 0x7fecec19f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.068+0000 7fecf4115640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecec101fd0 0x7fecec19ced0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.068+0000 7fecf4115640 1 --2- 172.21.15.82:0/3837380000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecec100e80 0x7fecec06b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.068+0000 7fecf4115640 1 -- 172.21.15.82:0/3837380000 >> 172.21.15.82:0/3837380000 conn(0x7fecec0fc410 msgr2=0x7fecec105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:03.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.068+0000 7fecf4115640 1 -- 172.21.15.82:0/3837380000 shutdown_connections 2024-04-21T08:45:03.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.068+0000 7fecf4115640 1 -- 172.21.15.82:0/3837380000 wait complete. 2024-04-21T08:45:03.079 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-21T08:45:03.079 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:45:03.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.177+0000 7f5ecedf4640 1 -- 172.21.15.82:0/307477461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec8107ca0 msgr2=0x7f5ec810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:03.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.177+0000 7f5ecedf4640 1 --2- 172.21.15.82:0/307477461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec8107ca0 0x7f5ec810a0f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009ca0 tx=0x7f5ec002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.177+0000 7f5ecedf4640 1 -- 172.21.15.82:0/307477461 shutdown_connections 2024-04-21T08:45:03.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.177+0000 7f5ecedf4640 1 --2- 172.21.15.82:0/307477461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec8107ca0 0x7f5ec810a0f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.177+0000 7f5ecedf4640 1 --2- 172.21.15.82:0/307477461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ec8105370 0x7f5ec8107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.177+0000 7f5ecedf4640 1 --2- 172.21.15.82:0/307477461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ec8102a40 0x7f5ec8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.177+0000 7f5ecedf4640 1 -- 172.21.15.82:0/307477461 >> 172.21.15.82:0/307477461 conn(0x7f5ec80fc470 msgr2=0x7f5ec80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:03.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.177+0000 7f5ecedf4640 1 -- 172.21.15.82:0/307477461 shutdown_connections 2024-04-21T08:45:03.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.177+0000 7f5ecedf4640 1 -- 172.21.15.82:0/307477461 wait complete. 2024-04-21T08:45:03.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.177+0000 7f5ecedf4640 1 Processor -- start 2024-04-21T08:45:03.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.177+0000 7f5ecedf4640 1 -- start start 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ecedf4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ec8102a40 0x7f5ec81023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ecedf4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec8105370 0x7f5ec81009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ecedf4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ec8107ca0 0x7f5ec8100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ecedf4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec810db00 con 0x7f5ec8105370 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ecedf4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec810d980 con 0x7f5ec8107ca0 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ecedf4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec810dc80 con 0x7f5ec8102a40 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec8105370 0x7f5ec81009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec8105370 0x7f5ec81009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58358/0 (socket says 172.21.15.82:58358) 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ec7fff640 1 -- 172.21.15.82:0/924626028 learned_addr learned my addr 172.21.15.82:0/924626028 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5eccb90640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ec8102a40 0x7f5ec81023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ecd391640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ec8107ca0 0x7f5ec8100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ec7fff640 1 -- 172.21.15.82:0/924626028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ec8102a40 msgr2=0x7f5ec81023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:03.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ec7fff640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ec8102a40 0x7f5ec81023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ec7fff640 1 -- 172.21.15.82:0/924626028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ec8107ca0 msgr2=0x7f5ec8100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:03.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ec7fff640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ec8107ca0 0x7f5ec8100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ec7fff640 1 -- 172.21.15.82:0/924626028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec81017c0 con 0x7f5ec8105370 2024-04-21T08:45:03.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ec7fff640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec8105370 0x7f5ec81009f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400d740 tx=0x7f5eb400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:03.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ec5ffb640 1 -- 172.21.15.82:0/924626028 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4017070 con 0x7f5ec8105370 2024-04-21T08:45:03.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ec5ffb640 1 -- 172.21.15.82:0/924626028 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eb400de50 con 0x7f5ec8105370 2024-04-21T08:45:03.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ecedf4640 1 -- 172.21.15.82:0/924626028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec81a2290 con 0x7f5ec8105370 2024-04-21T08:45:03.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.178+0000 7f5ec5ffb640 1 -- 172.21.15.82:0/924626028 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4004ce0 con 0x7f5ec8105370 2024-04-21T08:45:03.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.179+0000 7f5ecedf4640 1 -- 172.21.15.82:0/924626028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec81a2790 con 0x7f5ec8105370 2024-04-21T08:45:03.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.179+0000 7f5ecedf4640 1 -- 172.21.15.82:0/924626028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec80fdf60 con 0x7f5ec8105370 2024-04-21T08:45:03.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.180+0000 7f5ec5ffb640 1 -- 172.21.15.82:0/924626028 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5eb4004e80 con 0x7f5ec8105370 2024-04-21T08:45:03.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.181+0000 7f5ec5ffb640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5eb80a3c40 0x7f5eb80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:03.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.181+0000 7f5eccb90640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5eb80a3c40 0x7f5eb80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:03.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.181+0000 7f5ec5ffb640 1 -- 172.21.15.82:0/924626028 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5eb40d00c0 con 0x7f5ec8105370 2024-04-21T08:45:03.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.181+0000 7f5eccb90640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5eb80a3c40 0x7f5eb80a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009bd0 tx=0x7f5ebc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:03.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.185+0000 7f5ec5ffb640 1 -- 172.21.15.82:0/924626028 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5eb409c0b0 con 0x7f5ec8105370 2024-04-21T08:45:03.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.359+0000 7f5ecedf4640 1 -- 172.21.15.82:0/924626028 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ec8062390 con 0x7f5eb80a3c40 2024-04-21T08:45:03.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.369+0000 7f5ec5ffb640 1 -- 172.21.15.82:0/924626028 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340333 (secure 0 0 0) 0x7f5ec8062390 con 0x7f5eb80a3c40 2024-04-21T08:45:03.373 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:45:03.375 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-21T08:45:03.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.373+0000 7f5ecedf4640 1 -- 172.21.15.82:0/924626028 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5eb80a3c40 msgr2=0x7f5eb80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:03.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.373+0000 7f5ecedf4640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5eb80a3c40 0x7f5eb80a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009bd0 tx=0x7f5ebc005f90 comp rx=0 tx=0).stop 2024-04-21T08:45:03.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.373+0000 7f5ecedf4640 1 -- 172.21.15.82:0/924626028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec8105370 msgr2=0x7f5ec81009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:03.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.373+0000 7f5ecedf4640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec8105370 0x7f5ec81009f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400d740 tx=0x7f5eb400dc00 comp rx=0 tx=0).stop 2024-04-21T08:45:03.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.373+0000 7f5ecedf4640 1 -- 172.21.15.82:0/924626028 shutdown_connections 2024-04-21T08:45:03.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.373+0000 7f5ecedf4640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f5eb80a3c40 0x7f5eb80a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.373+0000 7f5ecedf4640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ec8107ca0 0x7f5ec8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.373+0000 7f5ecedf4640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ec8105370 0x7f5ec81009f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.373+0000 7f5ecedf4640 1 --2- 172.21.15.82:0/924626028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ec8102a40 0x7f5ec81023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.373+0000 7f5ecedf4640 1 -- 172.21.15.82:0/924626028 >> 172.21.15.82:0/924626028 conn(0x7f5ec80fc470 msgr2=0x7f5ec8103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:03.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.373+0000 7f5ecedf4640 1 -- 172.21.15.82:0/924626028 shutdown_connections 2024-04-21T08:45:03.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.373+0000 7f5ecedf4640 1 -- 172.21.15.82:0/924626028 wait complete. 2024-04-21T08:45:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-04-21T08:45:02.318637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599920,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":162,"num_read_kb":145,"num_write":342,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":354352,"kb_used_data":13512,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124587472,"statfs":{"total":1151940427776,"available":1151577571328,"internally_reserved":0,"allocated":13836288,"data_stored":7527764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7142,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":116,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003041"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288627+0000","last_change":"2024-04-21T08:44:34.494393+0000","last_active":"2024-04-21T08:44:40.288627+0000","last_peered":"2024-04-21T08:44:40.288627+0000","last_clean":"2024-04-21T08:44:40.288627+0000","last_became_active":"2024-04-21T08:44:34.493583+0000","last_became_peered":"2024-04-21T08:44:34.493583+0000","last_unstale":"2024-04-21T08:44:40.288627+0000","last_undegraded":"2024-04-21T08:44:40.288627+0000","last_fullsized":"2024-04-21T08:44:40.288627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:37.211676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883666+0000","last_change":"2024-04-21T08:44:34.496291+0000","last_active":"2024-04-21T08:44:41.883666+0000","last_peered":"2024-04-21T08:44:41.883666+0000","last_clean":"2024-04-21T08:44:41.883666+0000","last_became_active":"2024-04-21T08:44:34.495019+0000","last_became_peered":"2024-04-21T08:44:34.495019+0000","last_unstale":"2024-04-21T08:44:41.883666+0000","last_undegraded":"2024-04-21T08:44:41.883666+0000","last_fullsized":"2024-04-21T08:44:41.883666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:05.309847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883447+0000","last_change":"2024-04-21T08:44:34.497948+0000","last_active":"2024-04-21T08:44:41.883447+0000","last_peered":"2024-04-21T08:44:41.883447+0000","last_clean":"2024-04-21T08:44:41.883447+0000","last_became_active":"2024-04-21T08:44:34.497657+0000","last_became_peered":"2024-04-21T08:44:34.497657+0000","last_unstale":"2024-04-21T08:44:41.883447+0000","last_undegraded":"2024-04-21T08:44:41.883447+0000","last_fullsized":"2024-04-21T08:44:41.883447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:11.463715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.664151+0000","last_change":"2024-04-21T08:44:34.492682+0000","last_active":"2024-04-21T08:44:41.664151+0000","last_peered":"2024-04-21T08:44:41.664151+0000","last_clean":"2024-04-21T08:44:41.664151+0000","last_became_active":"2024-04-21T08:44:34.492230+0000","last_became_peered":"2024-04-21T08:44:34.492230+0000","last_unstale":"2024-04-21T08:44:41.664151+0000","last_undegraded":"2024-04-21T08:44:41.664151+0000","last_fullsized":"2024-04-21T08:44:41.664151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:40.365608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667826+0000","last_change":"2024-04-21T08:44:34.492160+0000","last_active":"2024-04-21T08:44:41.667826+0000","last_peered":"2024-04-21T08:44:41.667826+0000","last_clean":"2024-04-21T08:44:41.667826+0000","last_became_active":"2024-04-21T08:44:34.491701+0000","last_became_peered":"2024-04-21T08:44:34.491701+0000","last_unstale":"2024-04-21T08:44:41.667826+0000","last_undegraded":"2024-04-21T08:44:41.667826+0000","last_fullsized":"2024-04-21T08:44:41.667826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:50.542661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667591+0000","last_change":"2024-04-21T08:44:34.497341+0000","last_active":"2024-04-21T08:44:41.667591+0000","last_peered":"2024-04-21T08:44:41.667591+0000","last_clean":"2024-04-21T08:44:41.667591+0000","last_became_active":"2024-04-21T08:44:34.495794+0000","last_became_peered":"2024-04-21T08:44:34.495794+0000","last_unstale":"2024-04-21T08:44:41.667591+0000","last_undegraded":"2024-04-21T08:44:41.667591+0000","last_fullsized":"2024-04-21T08:44:41.667591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:13.854753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.531922+0000","last_change":"2024-04-21T08:44:34.501578+0000","last_active":"2024-04-21T08:44:39.531922+0000","last_peered":"2024-04-21T08:44:39.531922+0000","last_clean":"2024-04-21T08:44:39.531922+0000","last_became_active":"2024-04-21T08:44:34.500848+0000","last_became_peered":"2024-04-21T08:44:34.500848+0000","last_unstale":"2024-04-21T08:44:39.531922+0000","last_undegraded":"2024-04-21T08:44:39.531922+0000","last_fullsized":"2024-04-21T08:44:39.531922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:53:27.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175188+0000","last_change":"2024-04-21T08:44:34.502975+0000","last_active":"2024-04-21T08:44:41.175188+0000","last_peered":"2024-04-21T08:44:41.175188+0000","last_clean":"2024-04-21T08:44:41.175188+0000","last_became_active":"2024-04-21T08:44:34.502676+0000","last_became_peered":"2024-04-21T08:44:34.502676+0000","last_unstale":"2024-04-21T08:44:41.175188+0000","last_undegraded":"2024-04-21T08:44:41.175188+0000","last_fullsized":"2024-04-21T08:44:41.175188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:32:13.881603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667470+0000","last_change":"2024-04-21T08:44:34.495703+0000","last_active":"2024-04-21T08:44:41.667470+0000","last_peered":"2024-04-21T08:44:41.667470+0000","last_clean":"2024-04-21T08:44:41.667470+0000","last_became_active":"2024-04-21T08:44:34.495380+0000","last_became_peered":"2024-04-21T08:44:34.495380+0000","last_unstale":"2024-04-21T08:44:41.667470+0000","last_undegraded":"2024-04-21T08:44:41.667470+0000","last_fullsized":"2024-04-21T08:44:41.667470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:45:28.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668103+0000","last_change":"2024-04-21T08:44:34.499361+0000","last_active":"2024-04-21T08:44:41.668103+0000","last_peered":"2024-04-21T08:44:41.668103+0000","last_clean":"2024-04-21T08:44:41.668103+0000","last_became_active":"2024-04-21T08:44:34.499044+0000","last_became_peered":"2024-04-21T08:44:34.499044+0000","last_unstale":"2024-04-21T08:44:41.668103+0000","last_undegraded":"2024-04-21T08:44:41.668103+0000","last_fullsized":"2024-04-21T08:44:41.668103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:12.118985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532452+0000","last_change":"2024-04-21T08:44:34.499631+0000","last_active":"2024-04-21T08:44:39.532452+0000","last_peered":"2024-04-21T08:44:39.532452+0000","last_clean":"2024-04-21T08:44:39.532452+0000","last_became_active":"2024-04-21T08:44:34.499316+0000","last_became_peered":"2024-04-21T08:44:34.499316+0000","last_unstale":"2024-04-21T08:44:39.532452+0000","last_undegraded":"2024-04-21T08:44:39.532452+0000","last_fullsized":"2024-04-21T08:44:39.532452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:26.123017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.664097+0000","last_change":"2024-04-21T08:44:34.504857+0000","last_active":"2024-04-21T08:44:41.664097+0000","last_peered":"2024-04-21T08:44:41.664097+0000","last_clean":"2024-04-21T08:44:41.664097+0000","last_became_active":"2024-04-21T08:44:34.504564+0000","last_became_peered":"2024-04-21T08:44:34.504564+0000","last_unstale":"2024-04-21T08:44:41.664097+0000","last_undegraded":"2024-04-21T08:44:41.664097+0000","last_fullsized":"2024-04-21T08:44:41.664097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:32:05.216162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666811+0000","last_change":"2024-04-21T08:44:34.484074+0000","last_active":"2024-04-21T08:44:41.666811+0000","last_peered":"2024-04-21T08:44:41.666811+0000","last_clean":"2024-04-21T08:44:41.666811+0000","last_became_active":"2024-04-21T08:44:34.483309+0000","last_became_peered":"2024-04-21T08:44:34.483309+0000","last_unstale":"2024-04-21T08:44:41.666811+0000","last_undegraded":"2024-04-21T08:44:41.666811+0000","last_fullsized":"2024-04-21T08:44:41.666811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:39.686048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883370+0000","last_change":"2024-04-21T08:44:34.499159+0000","last_active":"2024-04-21T08:44:41.883370+0000","last_peered":"2024-04-21T08:44:41.883370+0000","last_clean":"2024-04-21T08:44:41.883370+0000","last_became_active":"2024-04-21T08:44:34.496609+0000","last_became_peered":"2024-04-21T08:44:34.496609+0000","last_unstale":"2024-04-21T08:44:41.883370+0000","last_undegraded":"2024-04-21T08:44:41.883370+0000","last_fullsized":"2024-04-21T08:44:41.883370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:11.716321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883288+0000","last_change":"2024-04-21T08:44:34.499004+0000","last_active":"2024-04-21T08:44:41.883288+0000","last_peered":"2024-04-21T08:44:41.883288+0000","last_clean":"2024-04-21T08:44:41.883288+0000","last_became_active":"2024-04-21T08:44:34.498652+0000","last_became_peered":"2024-04-21T08:44:34.498652+0000","last_unstale":"2024-04-21T08:44:41.883288+0000","last_undegraded":"2024-04-21T08:44:41.883288+0000","last_fullsized":"2024-04-21T08:44:41.883288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:48:24.154896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666623+0000","last_change":"2024-04-21T08:44:34.494018+0000","last_active":"2024-04-21T08:44:41.666623+0000","last_peered":"2024-04-21T08:44:41.666623+0000","last_clean":"2024-04-21T08:44:41.666623+0000","last_became_active":"2024-04-21T08:44:34.493692+0000","last_became_peered":"2024-04-21T08:44:34.493692+0000","last_unstale":"2024-04-21T08:44:41.666623+0000","last_undegraded":"2024-04-21T08:44:41.666623+0000","last_fullsized":"2024-04-21T08:44:41.666623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:25:16.944160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288456+0000","last_change":"2024-04-21T08:44:34.493231+0000","last_active":"2024-04-21T08:44:40.288456+0000","last_peered":"2024-04-21T08:44:40.288456+0000","last_clean":"2024-04-21T08:44:40.288456+0000","last_became_active":"2024-04-21T08:44:34.492826+0000","last_became_peered":"2024-04-21T08:44:34.492826+0000","last_unstale":"2024-04-21T08:44:40.288456+0000","last_undegraded":"2024-04-21T08:44:40.288456+0000","last_fullsized":"2024-04-21T08:44:40.288456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:36.815012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526864+0000","last_change":"2024-04-21T08:44:34.495142+0000","last_active":"2024-04-21T08:44:39.526864+0000","last_peered":"2024-04-21T08:44:39.526864+0000","last_clean":"2024-04-21T08:44:39.526864+0000","last_became_active":"2024-04-21T08:44:34.494803+0000","last_became_peered":"2024-04-21T08:44:34.494803+0000","last_unstale":"2024-04-21T08:44:39.526864+0000","last_undegraded":"2024-04-21T08:44:39.526864+0000","last_fullsized":"2024-04-21T08:44:39.526864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:22.974053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527704+0000","last_change":"2024-04-21T08:44:34.491252+0000","last_active":"2024-04-21T08:44:39.527704+0000","last_peered":"2024-04-21T08:44:39.527704+0000","last_clean":"2024-04-21T08:44:39.527704+0000","last_became_active":"2024-04-21T08:44:34.490814+0000","last_became_peered":"2024-04-21T08:44:34.490814+0000","last_unstale":"2024-04-21T08:44:39.527704+0000","last_undegraded":"2024-04-21T08:44:39.527704+0000","last_fullsized":"2024-04-21T08:44:39.527704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:54:52.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527559+0000","last_change":"2024-04-21T08:44:38.518647+0000","last_active":"2024-04-21T08:44:39.527559+0000","last_peered":"2024-04-21T08:44:39.527559+0000","last_clean":"2024-04-21T08:44:39.527559+0000","last_became_active":"2024-04-21T08:44:38.518239+0000","last_became_peered":"2024-04-21T08:44:38.518239+0000","last_unstale":"2024-04-21T08:44:39.527559+0000","last_undegraded":"2024-04-21T08:44:39.527559+0000","last_fullsized":"2024-04-21T08:44:39.527559+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:44.781542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526856+0000","last_change":"2024-04-21T08:44:33.484484+0000","last_active":"2024-04-21T08:44:39.526856+0000","last_peered":"2024-04-21T08:44:39.526856+0000","last_clean":"2024-04-21T08:44:39.526856+0000","last_became_active":"2024-04-21T08:44:33.484160+0000","last_became_peered":"2024-04-21T08:44:33.484160+0000","last_unstale":"2024-04-21T08:44:39.526856+0000","last_undegraded":"2024-04-21T08:44:39.526856+0000","last_fullsized":"2024-04-21T08:44:39.526856+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:45.962598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175630+0000","last_change":"2024-04-21T08:44:34.500722+0000","last_active":"2024-04-21T08:44:41.175630+0000","last_peered":"2024-04-21T08:44:41.175630+0000","last_clean":"2024-04-21T08:44:41.175630+0000","last_became_active":"2024-04-21T08:44:34.500388+0000","last_became_peered":"2024-04-21T08:44:34.500388+0000","last_unstale":"2024-04-21T08:44:41.175630+0000","last_undegraded":"2024-04-21T08:44:41.175630+0000","last_fullsized":"2024-04-21T08:44:41.175630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:05.876636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.663609+0000","last_change":"2024-04-21T08:44:38.521925+0000","last_active":"2024-04-21T08:44:41.663609+0000","last_peered":"2024-04-21T08:44:41.663609+0000","last_clean":"2024-04-21T08:44:41.663609+0000","last_became_active":"2024-04-21T08:44:38.520375+0000","last_became_peered":"2024-04-21T08:44:38.520375+0000","last_unstale":"2024-04-21T08:44:41.663609+0000","last_undegraded":"2024-04-21T08:44:41.663609+0000","last_fullsized":"2024-04-21T08:44:41.663609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:15:15.558088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.176075+0000","last_change":"2024-04-21T08:44:33.490508+0000","last_active":"2024-04-21T08:44:41.176075+0000","last_peered":"2024-04-21T08:44:41.176075+0000","last_clean":"2024-04-21T08:44:41.176075+0000","last_became_active":"2024-04-21T08:44:33.490084+0000","last_became_peered":"2024-04-21T08:44:33.490084+0000","last_unstale":"2024-04-21T08:44:41.176075+0000","last_undegraded":"2024-04-21T08:44:41.176075+0000","last_fullsized":"2024-04-21T08:44:41.176075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:21.553435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883522+0000","last_change":"2024-04-21T08:44:34.491843+0000","last_active":"2024-04-21T08:44:41.883522+0000","last_peered":"2024-04-21T08:44:41.883522+0000","last_clean":"2024-04-21T08:44:41.883522+0000","last_became_active":"2024-04-21T08:44:34.490578+0000","last_became_peered":"2024-04-21T08:44:34.490578+0000","last_unstale":"2024-04-21T08:44:41.883522+0000","last_undegraded":"2024-04-21T08:44:41.883522+0000","last_fullsized":"2024-04-21T08:44:41.883522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:05:12.272330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.663767+0000","last_change":"2024-04-21T08:44:38.520258+0000","last_active":"2024-04-21T08:44:41.663767+0000","last_peered":"2024-04-21T08:44:41.663767+0000","last_clean":"2024-04-21T08:44:41.663767+0000","last_became_active":"2024-04-21T08:44:38.519824+0000","last_became_peered":"2024-04-21T08:44:38.519824+0000","last_unstale":"2024-04-21T08:44:41.663767+0000","last_undegraded":"2024-04-21T08:44:41.663767+0000","last_fullsized":"2024-04-21T08:44:41.663767+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:39.716745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'11","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:01.014251+0000","last_change":"2024-04-21T08:44:33.484934+0000","last_active":"2024-04-21T08:45:01.014251+0000","last_peered":"2024-04-21T08:45:01.014251+0000","last_clean":"2024-04-21T08:45:01.014251+0000","last_became_active":"2024-04-21T08:44:33.484591+0000","last_became_peered":"2024-04-21T08:44:33.484591+0000","last_unstale":"2024-04-21T08:45:01.014251+0000","last_undegraded":"2024-04-21T08:45:01.014251+0000","last_fullsized":"2024-04-21T08:45:01.014251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:48.826611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667054+0000","last_change":"2024-04-21T08:44:34.501109+0000","last_active":"2024-04-21T08:44:41.667054+0000","last_peered":"2024-04-21T08:44:41.667054+0000","last_clean":"2024-04-21T08:44:41.667054+0000","last_became_active":"2024-04-21T08:44:34.500750+0000","last_became_peered":"2024-04-21T08:44:34.500750+0000","last_unstale":"2024-04-21T08:44:41.667054+0000","last_undegraded":"2024-04-21T08:44:41.667054+0000","last_fullsized":"2024-04-21T08:44:41.667054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:29:28.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.663702+0000","last_change":"2024-04-21T08:44:38.519633+0000","last_active":"2024-04-21T08:44:41.663702+0000","last_peered":"2024-04-21T08:44:41.663702+0000","last_clean":"2024-04-21T08:44:41.663702+0000","last_became_active":"2024-04-21T08:44:38.519215+0000","last_became_peered":"2024-04-21T08:44:38.519215+0000","last_unstale":"2024-04-21T08:44:41.663702+0000","last_undegraded":"2024-04-21T08:44:41.663702+0000","last_fullsized":"2024-04-21T08:44:41.663702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:43:59.695844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:01.013231+0000","last_change":"2024-04-21T08:44:33.483267+0000","last_active":"2024-04-21T08:45:01.013231+0000","last_peered":"2024-04-21T08:45:01.013231+0000","last_clean":"2024-04-21T08:45:01.013231+0000","last_became_active":"2024-04-21T08:44:33.482932+0000","last_became_peered":"2024-04-21T08:44:33.482932+0000","last_unstale":"2024-04-21T08:45:01.013231+0000","last_undegraded":"2024-04-21T08:45:01.013231+0000","last_fullsized":"2024-04-21T08:45:01.013231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:53.311550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668877+0000","last_change":"2024-04-21T08:44:34.497389+0000","last_active":"2024-04-21T08:44:41.668877+0000","last_peered":"2024-04-21T08:44:41.668877+0000","last_clean":"2024-04-21T08:44:41.668877+0000","last_became_active":"2024-04-21T08:44:34.497030+0000","last_became_peered":"2024-04-21T08:44:34.497030+0000","last_unstale":"2024-04-21T08:44:41.668877+0000","last_undegraded":"2024-04-21T08:44:41.668877+0000","last_fullsized":"2024-04-21T08:44:41.668877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:20.371888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527623+0000","last_change":"2024-04-21T08:44:38.518095+0000","last_active":"2024-04-21T08:44:39.527623+0000","last_peered":"2024-04-21T08:44:39.527623+0000","last_clean":"2024-04-21T08:44:39.527623+0000","last_became_active":"2024-04-21T08:44:38.517010+0000","last_became_peered":"2024-04-21T08:44:38.517010+0000","last_unstale":"2024-04-21T08:44:39.527623+0000","last_undegraded":"2024-04-21T08:44:39.527623+0000","last_fullsized":"2024-04-21T08:44:39.527623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:02.379512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.991761+0000","last_change":"2024-04-21T08:44:33.487999+0000","last_active":"2024-04-21T08:44:59.991761+0000","last_peered":"2024-04-21T08:44:59.991761+0000","last_clean":"2024-04-21T08:44:59.991761+0000","last_became_active":"2024-04-21T08:44:33.487666+0000","last_became_peered":"2024-04-21T08:44:33.487666+0000","last_unstale":"2024-04-21T08:44:59.991761+0000","last_undegraded":"2024-04-21T08:44:59.991761+0000","last_fullsized":"2024-04-21T08:44:59.991761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:01.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668218+0000","last_change":"2024-04-21T08:44:34.494213+0000","last_active":"2024-04-21T08:44:41.668218+0000","last_peered":"2024-04-21T08:44:41.668218+0000","last_clean":"2024-04-21T08:44:41.668218+0000","last_became_active":"2024-04-21T08:44:34.493853+0000","last_became_peered":"2024-04-21T08:44:34.493853+0000","last_unstale":"2024-04-21T08:44:41.668218+0000","last_undegraded":"2024-04-21T08:44:41.668218+0000","last_fullsized":"2024-04-21T08:44:41.668218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:42.499131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527869+0000","last_change":"2024-04-21T08:44:38.519612+0000","last_active":"2024-04-21T08:44:39.527869+0000","last_peered":"2024-04-21T08:44:39.527869+0000","last_clean":"2024-04-21T08:44:39.527869+0000","last_became_active":"2024-04-21T08:44:38.518946+0000","last_became_peered":"2024-04-21T08:44:38.518946+0000","last_unstale":"2024-04-21T08:44:39.527869+0000","last_undegraded":"2024-04-21T08:44:39.527869+0000","last_fullsized":"2024-04-21T08:44:39.527869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:04.982377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:00.002549+0000","last_change":"2024-04-21T08:44:33.488751+0000","last_active":"2024-04-21T08:45:00.002549+0000","last_peered":"2024-04-21T08:45:00.002549+0000","last_clean":"2024-04-21T08:45:00.002549+0000","last_became_active":"2024-04-21T08:44:33.488425+0000","last_became_peered":"2024-04-21T08:44:33.488425+0000","last_unstale":"2024-04-21T08:45:00.002549+0000","last_undegraded":"2024-04-21T08:45:00.002549+0000","last_fullsized":"2024-04-21T08:45:00.002549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:28:47.868814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668668+0000","last_change":"2024-04-21T08:44:34.492770+0000","last_active":"2024-04-21T08:44:41.668668+0000","last_peered":"2024-04-21T08:44:41.668668+0000","last_clean":"2024-04-21T08:44:41.668668+0000","last_became_active":"2024-04-21T08:44:34.491340+0000","last_became_peered":"2024-04-21T08:44:34.491340+0000","last_unstale":"2024-04-21T08:44:41.668668+0000","last_undegraded":"2024-04-21T08:44:41.668668+0000","last_fullsized":"2024-04-21T08:44:41.668668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:47:54.280990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288500+0000","last_change":"2024-04-21T08:44:38.518888+0000","last_active":"2024-04-21T08:44:40.288500+0000","last_peered":"2024-04-21T08:44:40.288500+0000","last_clean":"2024-04-21T08:44:40.288500+0000","last_became_active":"2024-04-21T08:44:38.518421+0000","last_became_peered":"2024-04-21T08:44:38.518421+0000","last_unstale":"2024-04-21T08:44:40.288500+0000","last_undegraded":"2024-04-21T08:44:40.288500+0000","last_fullsized":"2024-04-21T08:44:40.288500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:21.119364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.462823+0000","last_change":"2024-04-21T08:44:33.484383+0000","last_active":"2024-04-21T08:44:59.462823+0000","last_peered":"2024-04-21T08:44:59.462823+0000","last_clean":"2024-04-21T08:44:59.462823+0000","last_became_active":"2024-04-21T08:44:33.484083+0000","last_became_peered":"2024-04-21T08:44:33.484083+0000","last_unstale":"2024-04-21T08:44:59.462823+0000","last_undegraded":"2024-04-21T08:44:59.462823+0000","last_fullsized":"2024-04-21T08:44:59.462823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:13:14.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175324+0000","last_change":"2024-04-21T08:44:34.495636+0000","last_active":"2024-04-21T08:44:41.175324+0000","last_peered":"2024-04-21T08:44:41.175324+0000","last_clean":"2024-04-21T08:44:41.175324+0000","last_became_active":"2024-04-21T08:44:34.494967+0000","last_became_peered":"2024-04-21T08:44:34.494967+0000","last_unstale":"2024-04-21T08:44:41.175324+0000","last_undegraded":"2024-04-21T08:44:41.175324+0000","last_fullsized":"2024-04-21T08:44:41.175324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:47:02.379208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883626+0000","last_change":"2024-04-21T08:44:38.515145+0000","last_active":"2024-04-21T08:44:41.883626+0000","last_peered":"2024-04-21T08:44:41.883626+0000","last_clean":"2024-04-21T08:44:41.883626+0000","last_became_active":"2024-04-21T08:44:38.514755+0000","last_became_peered":"2024-04-21T08:44:38.514755+0000","last_unstale":"2024-04-21T08:44:41.883626+0000","last_undegraded":"2024-04-21T08:44:41.883626+0000","last_fullsized":"2024-04-21T08:44:41.883626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:38:15.826293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:58.454221+0000","last_change":"2024-04-21T08:44:33.484714+0000","last_active":"2024-04-21T08:44:58.454221+0000","last_peered":"2024-04-21T08:44:58.454221+0000","last_clean":"2024-04-21T08:44:58.454221+0000","last_became_active":"2024-04-21T08:44:33.484385+0000","last_became_peered":"2024-04-21T08:44:33.484385+0000","last_unstale":"2024-04-21T08:44:58.454221+0000","last_undegraded":"2024-04-21T08:44:58.454221+0000","last_fullsized":"2024-04-21T08:44:58.454221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:03.406860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526310+0000","last_change":"2024-04-21T08:44:34.490989+0000","last_active":"2024-04-21T08:44:39.526310+0000","last_peered":"2024-04-21T08:44:39.526310+0000","last_clean":"2024-04-21T08:44:39.526310+0000","last_became_active":"2024-04-21T08:44:34.490658+0000","last_became_peered":"2024-04-21T08:44:34.490658+0000","last_unstale":"2024-04-21T08:44:39.526310+0000","last_undegraded":"2024-04-21T08:44:39.526310+0000","last_fullsized":"2024-04-21T08:44:39.526310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:42:15.537913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175918+0000","last_change":"2024-04-21T08:44:38.519723+0000","last_active":"2024-04-21T08:44:41.175918+0000","last_peered":"2024-04-21T08:44:41.175918+0000","last_clean":"2024-04-21T08:44:41.175918+0000","last_became_active":"2024-04-21T08:44:38.519336+0000","last_became_peered":"2024-04-21T08:44:38.519336+0000","last_unstale":"2024-04-21T08:44:41.175918+0000","last_undegraded":"2024-04-21T08:44:41.175918+0000","last_fullsized":"2024-04-21T08:44:41.175918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:00.332153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.991050+0000","last_change":"2024-04-21T08:44:33.484404+0000","last_active":"2024-04-21T08:44:59.991050+0000","last_peered":"2024-04-21T08:44:59.991050+0000","last_clean":"2024-04-21T08:44:59.991050+0000","last_became_active":"2024-04-21T08:44:33.484088+0000","last_became_peered":"2024-04-21T08:44:33.484088+0000","last_unstale":"2024-04-21T08:44:59.991050+0000","last_undegraded":"2024-04-21T08:44:59.991050+0000","last_fullsized":"2024-04-21T08:44:59.991050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:41.948622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667459+0000","last_change":"2024-04-21T08:44:34.502598+0000","last_active":"2024-04-21T08:44:41.667459+0000","last_peered":"2024-04-21T08:44:41.667459+0000","last_clean":"2024-04-21T08:44:41.667459+0000","last_became_active":"2024-04-21T08:44:34.502057+0000","last_became_peered":"2024-04-21T08:44:34.502057+0000","last_unstale":"2024-04-21T08:44:41.667459+0000","last_undegraded":"2024-04-21T08:44:41.667459+0000","last_fullsized":"2024-04-21T08:44:41.667459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:24.052873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526419+0000","last_change":"2024-04-21T08:44:38.519699+0000","last_active":"2024-04-21T08:44:39.526419+0000","last_peered":"2024-04-21T08:44:39.526419+0000","last_clean":"2024-04-21T08:44:39.526419+0000","last_became_active":"2024-04-21T08:44:38.517655+0000","last_became_peered":"2024-04-21T08:44:38.517655+0000","last_unstale":"2024-04-21T08:44:39.526419+0000","last_undegraded":"2024-04-21T08:44:39.526419+0000","last_fullsized":"2024-04-21T08:44:39.526419+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:12.784479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532460+0000","last_change":"2024-04-21T08:44:33.488799+0000","last_active":"2024-04-21T08:44:39.532460+0000","last_peered":"2024-04-21T08:44:39.532460+0000","last_clean":"2024-04-21T08:44:39.532460+0000","last_became_active":"2024-04-21T08:44:33.487948+0000","last_became_peered":"2024-04-21T08:44:33.487948+0000","last_unstale":"2024-04-21T08:44:39.532460+0000","last_undegraded":"2024-04-21T08:44:39.532460+0000","last_fullsized":"2024-04-21T08:44:39.532460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:57.347588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883571+0000","last_change":"2024-04-21T08:44:34.511133+0000","last_active":"2024-04-21T08:44:41.883571+0000","last_peered":"2024-04-21T08:44:41.883571+0000","last_clean":"2024-04-21T08:44:41.883571+0000","last_became_active":"2024-04-21T08:44:34.510350+0000","last_became_peered":"2024-04-21T08:44:34.510350+0000","last_unstale":"2024-04-21T08:44:41.883571+0000","last_undegraded":"2024-04-21T08:44:41.883571+0000","last_fullsized":"2024-04-21T08:44:41.883571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:30:53.317856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.663964+0000","last_change":"2024-04-21T08:44:38.517956+0000","last_active":"2024-04-21T08:44:41.663964+0000","last_peered":"2024-04-21T08:44:41.663964+0000","last_clean":"2024-04-21T08:44:41.663964+0000","last_became_active":"2024-04-21T08:44:38.517422+0000","last_became_peered":"2024-04-21T08:44:38.517422+0000","last_unstale":"2024-04-21T08:44:41.663964+0000","last_undegraded":"2024-04-21T08:44:41.663964+0000","last_fullsized":"2024-04-21T08:44:41.663964+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:04.650784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.993047+0000","last_change":"2024-04-21T08:44:33.489151+0000","last_active":"2024-04-21T08:44:59.993047+0000","last_peered":"2024-04-21T08:44:59.993047+0000","last_clean":"2024-04-21T08:44:59.993047+0000","last_became_active":"2024-04-21T08:44:33.488781+0000","last_became_peered":"2024-04-21T08:44:33.488781+0000","last_unstale":"2024-04-21T08:44:59.993047+0000","last_undegraded":"2024-04-21T08:44:59.993047+0000","last_fullsized":"2024-04-21T08:44:59.993047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:40.675100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667148+0000","last_change":"2024-04-21T08:44:34.496528+0000","last_active":"2024-04-21T08:44:41.667148+0000","last_peered":"2024-04-21T08:44:41.667148+0000","last_clean":"2024-04-21T08:44:41.667148+0000","last_became_active":"2024-04-21T08:44:34.496223+0000","last_became_peered":"2024-04-21T08:44:34.496223+0000","last_unstale":"2024-04-21T08:44:41.667148+0000","last_undegraded":"2024-04-21T08:44:41.667148+0000","last_fullsized":"2024-04-21T08:44:41.667148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:56.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526362+0000","last_change":"2024-04-21T08:44:38.517537+0000","last_active":"2024-04-21T08:44:39.526362+0000","last_peered":"2024-04-21T08:44:39.526362+0000","last_clean":"2024-04-21T08:44:39.526362+0000","last_became_active":"2024-04-21T08:44:38.517134+0000","last_became_peered":"2024-04-21T08:44:38.517134+0000","last_unstale":"2024-04-21T08:44:39.526362+0000","last_undegraded":"2024-04-21T08:44:39.526362+0000","last_fullsized":"2024-04-21T08:44:39.526362+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:38.634418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.993248+0000","last_change":"2024-04-21T08:44:33.485582+0000","last_active":"2024-04-21T08:44:59.993248+0000","last_peered":"2024-04-21T08:44:59.993248+0000","last_clean":"2024-04-21T08:44:59.993248+0000","last_became_active":"2024-04-21T08:44:33.485249+0000","last_became_peered":"2024-04-21T08:44:33.485249+0000","last_unstale":"2024-04-21T08:44:59.993248+0000","last_undegraded":"2024-04-21T08:44:59.993248+0000","last_fullsized":"2024-04-21T08:44:59.993248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:16:10.571694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668221+0000","last_change":"2024-04-21T08:44:34.491361+0000","last_active":"2024-04-21T08:44:41.668221+0000","last_peered":"2024-04-21T08:44:41.668221+0000","last_clean":"2024-04-21T08:44:41.668221+0000","last_became_active":"2024-04-21T08:44:34.491050+0000","last_became_peered":"2024-04-21T08:44:34.491050+0000","last_unstale":"2024-04-21T08:44:41.668221+0000","last_undegraded":"2024-04-21T08:44:41.668221+0000","last_fullsized":"2024-04-21T08:44:41.668221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:39.082448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.669089+0000","last_change":"2024-04-21T08:44:38.528745+0000","last_active":"2024-04-21T08:44:41.669089+0000","last_peered":"2024-04-21T08:44:41.669089+0000","last_clean":"2024-04-21T08:44:41.669089+0000","last_became_active":"2024-04-21T08:44:38.528275+0000","last_became_peered":"2024-04-21T08:44:38.528275+0000","last_unstale":"2024-04-21T08:44:41.669089+0000","last_undegraded":"2024-04-21T08:44:41.669089+0000","last_fullsized":"2024-04-21T08:44:41.669089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:06:11.338420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.989320+0000","last_change":"2024-04-21T08:44:33.482059+0000","last_active":"2024-04-21T08:44:59.989320+0000","last_peered":"2024-04-21T08:44:59.989320+0000","last_clean":"2024-04-21T08:44:59.989320+0000","last_became_active":"2024-04-21T08:44:33.481663+0000","last_became_peered":"2024-04-21T08:44:33.481663+0000","last_unstale":"2024-04-21T08:44:59.989320+0000","last_undegraded":"2024-04-21T08:44:59.989320+0000","last_fullsized":"2024-04-21T08:44:59.989320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:01.240452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666788+0000","last_change":"2024-04-21T08:44:34.491398+0000","last_active":"2024-04-21T08:44:41.666788+0000","last_peered":"2024-04-21T08:44:41.666788+0000","last_clean":"2024-04-21T08:44:41.666788+0000","last_became_active":"2024-04-21T08:44:34.491073+0000","last_became_peered":"2024-04-21T08:44:34.491073+0000","last_unstale":"2024-04-21T08:44:41.666788+0000","last_undegraded":"2024-04-21T08:44:41.666788+0000","last_fullsized":"2024-04-21T08:44:41.666788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:22.566063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288552+0000","last_change":"2024-04-21T08:44:38.515653+0000","last_active":"2024-04-21T08:44:40.288552+0000","last_peered":"2024-04-21T08:44:40.288552+0000","last_clean":"2024-04-21T08:44:40.288552+0000","last_became_active":"2024-04-21T08:44:38.514796+0000","last_became_peered":"2024-04-21T08:44:38.514796+0000","last_unstale":"2024-04-21T08:44:40.288552+0000","last_undegraded":"2024-04-21T08:44:40.288552+0000","last_fullsized":"2024-04-21T08:44:40.288552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:00:22.662136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:58.454089+0000","last_change":"2024-04-21T08:44:33.487073+0000","last_active":"2024-04-21T08:44:58.454089+0000","last_peered":"2024-04-21T08:44:58.454089+0000","last_clean":"2024-04-21T08:44:58.454089+0000","last_became_active":"2024-04-21T08:44:33.486740+0000","last_became_peered":"2024-04-21T08:44:33.486740+0000","last_unstale":"2024-04-21T08:44:58.454089+0000","last_undegraded":"2024-04-21T08:44:58.454089+0000","last_fullsized":"2024-04-21T08:44:58.454089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:32.673074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532074+0000","last_change":"2024-04-21T08:44:34.496757+0000","last_active":"2024-04-21T08:44:39.532074+0000","last_peered":"2024-04-21T08:44:39.532074+0000","last_clean":"2024-04-21T08:44:39.532074+0000","last_became_active":"2024-04-21T08:44:34.496422+0000","last_became_peered":"2024-04-21T08:44:34.496422+0000","last_unstale":"2024-04-21T08:44:39.532074+0000","last_undegraded":"2024-04-21T08:44:39.532074+0000","last_fullsized":"2024-04-21T08:44:39.532074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:08.686810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526775+0000","last_change":"2024-04-21T08:44:38.520935+0000","last_active":"2024-04-21T08:44:39.526775+0000","last_peered":"2024-04-21T08:44:39.526775+0000","last_clean":"2024-04-21T08:44:39.526775+0000","last_became_active":"2024-04-21T08:44:38.520481+0000","last_became_peered":"2024-04-21T08:44:38.520481+0000","last_unstale":"2024-04-21T08:44:39.526775+0000","last_undegraded":"2024-04-21T08:44:39.526775+0000","last_fullsized":"2024-04-21T08:44:39.526775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:12.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.998062+0000","last_change":"2024-04-21T08:44:33.485797+0000","last_active":"2024-04-21T08:44:59.998062+0000","last_peered":"2024-04-21T08:44:59.998062+0000","last_clean":"2024-04-21T08:44:59.998062+0000","last_became_active":"2024-04-21T08:44:33.485496+0000","last_became_peered":"2024-04-21T08:44:33.485496+0000","last_unstale":"2024-04-21T08:44:59.998062+0000","last_undegraded":"2024-04-21T08:44:59.998062+0000","last_fullsized":"2024-04-21T08:44:59.998062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:11:34.604611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532243+0000","last_change":"2024-04-21T08:44:34.506453+0000","last_active":"2024-04-21T08:44:39.532243+0000","last_peered":"2024-04-21T08:44:39.532243+0000","last_clean":"2024-04-21T08:44:39.532243+0000","last_became_active":"2024-04-21T08:44:34.506121+0000","last_became_peered":"2024-04-21T08:44:34.506121+0000","last_unstale":"2024-04-21T08:44:39.532243+0000","last_undegraded":"2024-04-21T08:44:39.532243+0000","last_fullsized":"2024-04-21T08:44:39.532243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:07.927366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667077+0000","last_change":"2024-04-21T08:44:38.523718+0000","last_active":"2024-04-21T08:44:41.667077+0000","last_peered":"2024-04-21T08:44:41.667077+0000","last_clean":"2024-04-21T08:44:41.667077+0000","last_became_active":"2024-04-21T08:44:38.523003+0000","last_became_peered":"2024-04-21T08:44:38.523003+0000","last_unstale":"2024-04-21T08:44:41.667077+0000","last_undegraded":"2024-04-21T08:44:41.667077+0000","last_fullsized":"2024-04-21T08:44:41.667077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:20.837450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:00.002286+0000","last_change":"2024-04-21T08:44:33.482029+0000","last_active":"2024-04-21T08:45:00.002286+0000","last_peered":"2024-04-21T08:45:00.002286+0000","last_clean":"2024-04-21T08:45:00.002286+0000","last_became_active":"2024-04-21T08:44:33.481713+0000","last_became_peered":"2024-04-21T08:44:33.481713+0000","last_unstale":"2024-04-21T08:45:00.002286+0000","last_undegraded":"2024-04-21T08:45:00.002286+0000","last_fullsized":"2024-04-21T08:45:00.002286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:46:34.717244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668579+0000","last_change":"2024-04-21T08:44:34.491944+0000","last_active":"2024-04-21T08:44:41.668579+0000","last_peered":"2024-04-21T08:44:41.668579+0000","last_clean":"2024-04-21T08:44:41.668579+0000","last_became_active":"2024-04-21T08:44:34.491638+0000","last_became_peered":"2024-04-21T08:44:34.491638+0000","last_unstale":"2024-04-21T08:44:41.668579+0000","last_undegraded":"2024-04-21T08:44:41.668579+0000","last_fullsized":"2024-04-21T08:44:41.668579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:08:07.925937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668596+0000","last_change":"2024-04-21T08:44:38.528971+0000","last_active":"2024-04-21T08:44:41.668596+0000","last_peered":"2024-04-21T08:44:41.668596+0000","last_clean":"2024-04-21T08:44:41.668596+0000","last_became_active":"2024-04-21T08:44:38.528453+0000","last_became_peered":"2024-04-21T08:44:38.528453+0000","last_unstale":"2024-04-21T08:44:41.668596+0000","last_undegraded":"2024-04-21T08:44:41.668596+0000","last_fullsized":"2024-04-21T08:44:41.668596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:50:31.237749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.990116+0000","last_change":"2024-04-21T08:44:33.479262+0000","last_active":"2024-04-21T08:44:59.990116+0000","last_peered":"2024-04-21T08:44:59.990116+0000","last_clean":"2024-04-21T08:44:59.990116+0000","last_became_active":"2024-04-21T08:44:33.478907+0000","last_became_peered":"2024-04-21T08:44:33.478907+0000","last_unstale":"2024-04-21T08:44:59.990116+0000","last_undegraded":"2024-04-21T08:44:59.990116+0000","last_fullsized":"2024-04-21T08:44:59.990116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:33:43.879126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288287+0000","last_change":"2024-04-21T08:44:34.498503+0000","last_active":"2024-04-21T08:44:40.288287+0000","last_peered":"2024-04-21T08:44:40.288287+0000","last_clean":"2024-04-21T08:44:40.288287+0000","last_became_active":"2024-04-21T08:44:34.495140+0000","last_became_peered":"2024-04-21T08:44:34.495140+0000","last_unstale":"2024-04-21T08:44:40.288287+0000","last_undegraded":"2024-04-21T08:44:40.288287+0000","last_fullsized":"2024-04-21T08:44:40.288287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:24.518349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668163+0000","last_change":"2024-04-21T08:44:38.517717+0000","last_active":"2024-04-21T08:44:41.668163+0000","last_peered":"2024-04-21T08:44:41.668163+0000","last_clean":"2024-04-21T08:44:41.668163+0000","last_became_active":"2024-04-21T08:44:38.517272+0000","last_became_peered":"2024-04-21T08:44:38.517272+0000","last_unstale":"2024-04-21T08:44:41.668163+0000","last_undegraded":"2024-04-21T08:44:41.668163+0000","last_fullsized":"2024-04-21T08:44:41.668163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.664130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526633+0000","last_change":"2024-04-21T08:44:33.477498+0000","last_active":"2024-04-21T08:44:39.526633+0000","last_peered":"2024-04-21T08:44:39.526633+0000","last_clean":"2024-04-21T08:44:39.526633+0000","last_became_active":"2024-04-21T08:44:33.477139+0000","last_became_peered":"2024-04-21T08:44:33.477139+0000","last_unstale":"2024-04-21T08:44:39.526633+0000","last_undegraded":"2024-04-21T08:44:39.526633+0000","last_fullsized":"2024-04-21T08:44:39.526633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:26:59.390583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667976+0000","last_change":"2024-04-21T08:44:34.493586+0000","last_active":"2024-04-21T08:44:41.667976+0000","last_peered":"2024-04-21T08:44:41.667976+0000","last_clean":"2024-04-21T08:44:41.667976+0000","last_became_active":"2024-04-21T08:44:34.493258+0000","last_became_peered":"2024-04-21T08:44:34.493258+0000","last_unstale":"2024-04-21T08:44:41.667976+0000","last_undegraded":"2024-04-21T08:44:41.667976+0000","last_fullsized":"2024-04-21T08:44:41.667976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:11:18.783332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666892+0000","last_change":"2024-04-21T08:44:38.519783+0000","last_active":"2024-04-21T08:44:41.666892+0000","last_peered":"2024-04-21T08:44:41.666892+0000","last_clean":"2024-04-21T08:44:41.666892+0000","last_became_active":"2024-04-21T08:44:38.519397+0000","last_became_peered":"2024-04-21T08:44:38.519397+0000","last_unstale":"2024-04-21T08:44:41.666892+0000","last_undegraded":"2024-04-21T08:44:41.666892+0000","last_fullsized":"2024-04-21T08:44:41.666892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:08.740240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532012+0000","last_change":"2024-04-21T08:44:33.490098+0000","last_active":"2024-04-21T08:44:39.532012+0000","last_peered":"2024-04-21T08:44:39.532012+0000","last_clean":"2024-04-21T08:44:39.532012+0000","last_became_active":"2024-04-21T08:44:33.489691+0000","last_became_peered":"2024-04-21T08:44:33.489691+0000","last_unstale":"2024-04-21T08:44:39.532012+0000","last_undegraded":"2024-04-21T08:44:39.532012+0000","last_fullsized":"2024-04-21T08:44:39.532012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:44.965998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667817+0000","last_change":"2024-04-21T08:44:34.498543+0000","last_active":"2024-04-21T08:44:41.667817+0000","last_peered":"2024-04-21T08:44:41.667817+0000","last_clean":"2024-04-21T08:44:41.667817+0000","last_became_active":"2024-04-21T08:44:34.498211+0000","last_became_peered":"2024-04-21T08:44:34.498211+0000","last_unstale":"2024-04-21T08:44:41.667817+0000","last_undegraded":"2024-04-21T08:44:41.667817+0000","last_fullsized":"2024-04-21T08:44:41.667817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:16.506695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667029+0000","last_change":"2024-04-21T08:44:38.521010+0000","last_active":"2024-04-21T08:44:41.667029+0000","last_peered":"2024-04-21T08:44:41.667029+0000","last_clean":"2024-04-21T08:44:41.667029+0000","last_became_active":"2024-04-21T08:44:38.520605+0000","last_became_peered":"2024-04-21T08:44:38.520605+0000","last_unstale":"2024-04-21T08:44:41.667029+0000","last_undegraded":"2024-04-21T08:44:41.667029+0000","last_fullsized":"2024-04-21T08:44:41.667029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:50.378803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668008+0000","last_change":"2024-04-21T08:44:34.496437+0000","last_active":"2024-04-21T08:44:41.668008+0000","last_peered":"2024-04-21T08:44:41.668008+0000","last_clean":"2024-04-21T08:44:41.668008+0000","last_became_active":"2024-04-21T08:44:34.496108+0000","last_became_peered":"2024-04-21T08:44:34.496108+0000","last_unstale":"2024-04-21T08:44:41.668008+0000","last_undegraded":"2024-04-21T08:44:41.668008+0000","last_fullsized":"2024-04-21T08:44:41.668008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:15:55.237988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:00.994306+0000","last_change":"2024-04-21T08:44:33.494031+0000","last_active":"2024-04-21T08:45:00.994306+0000","last_peered":"2024-04-21T08:45:00.994306+0000","last_clean":"2024-04-21T08:45:00.994306+0000","last_became_active":"2024-04-21T08:44:33.493580+0000","last_became_peered":"2024-04-21T08:44:33.493580+0000","last_unstale":"2024-04-21T08:45:00.994306+0000","last_undegraded":"2024-04-21T08:45:00.994306+0000","last_fullsized":"2024-04-21T08:45:00.994306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:39:48.230616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532479+0000","last_change":"2024-04-21T08:44:38.518067+0000","last_active":"2024-04-21T08:44:39.532479+0000","last_peered":"2024-04-21T08:44:39.532479+0000","last_clean":"2024-04-21T08:44:39.532479+0000","last_became_active":"2024-04-21T08:44:38.517594+0000","last_became_peered":"2024-04-21T08:44:38.517594+0000","last_unstale":"2024-04-21T08:44:39.532479+0000","last_undegraded":"2024-04-21T08:44:39.532479+0000","last_fullsized":"2024-04-21T08:44:39.532479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:35.261582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175439+0000","last_change":"2024-04-21T08:44:34.503645+0000","last_active":"2024-04-21T08:44:41.175439+0000","last_peered":"2024-04-21T08:44:41.175439+0000","last_clean":"2024-04-21T08:44:41.175439+0000","last_became_active":"2024-04-21T08:44:34.500736+0000","last_became_peered":"2024-04-21T08:44:34.500736+0000","last_unstale":"2024-04-21T08:44:41.175439+0000","last_undegraded":"2024-04-21T08:44:41.175439+0000","last_fullsized":"2024-04-21T08:44:41.175439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:46.084991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.998088+0000","last_change":"2024-04-21T08:44:33.485110+0000","last_active":"2024-04-21T08:44:59.998088+0000","last_peered":"2024-04-21T08:44:59.998088+0000","last_clean":"2024-04-21T08:44:59.998088+0000","last_became_active":"2024-04-21T08:44:33.484722+0000","last_became_peered":"2024-04-21T08:44:33.484722+0000","last_unstale":"2024-04-21T08:44:59.998088+0000","last_undegraded":"2024-04-21T08:44:59.998088+0000","last_fullsized":"2024-04-21T08:44:59.998088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:47:16.773560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532053+0000","last_change":"2024-04-21T08:43:17.984594+0000","last_active":"2024-04-21T08:44:39.532053+0000","last_peered":"2024-04-21T08:44:39.532053+0000","last_clean":"2024-04-21T08:44:39.532053+0000","last_became_active":"2024-04-21T08:43:17.984207+0000","last_became_peered":"2024-04-21T08:43:17.984207+0000","last_unstale":"2024-04-21T08:44:39.532053+0000","last_undegraded":"2024-04-21T08:44:39.532053+0000","last_fullsized":"2024-04-21T08:44:39.532053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_clean_scrub_stamp":"2024-04-21T08:39:50.910206+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:35.260499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668991+0000","last_change":"2024-04-21T08:44:38.516792+0000","last_active":"2024-04-21T08:44:41.668991+0000","last_peered":"2024-04-21T08:44:41.668991+0000","last_clean":"2024-04-21T08:44:41.668991+0000","last_became_active":"2024-04-21T08:44:38.516369+0000","last_became_peered":"2024-04-21T08:44:38.516369+0000","last_unstale":"2024-04-21T08:44:41.668991+0000","last_undegraded":"2024-04-21T08:44:41.668991+0000","last_fullsized":"2024-04-21T08:44:41.668991+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:29:12.278396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666626+0000","last_change":"2024-04-21T08:44:34.499975+0000","last_active":"2024-04-21T08:44:41.666626+0000","last_peered":"2024-04-21T08:44:41.666626+0000","last_clean":"2024-04-21T08:44:41.666626+0000","last_became_active":"2024-04-21T08:44:34.499646+0000","last_became_peered":"2024-04-21T08:44:34.499646+0000","last_unstale":"2024-04-21T08:44:41.666626+0000","last_undegraded":"2024-04-21T08:44:41.666626+0000","last_fullsized":"2024-04-21T08:44:41.666626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:19:38.082502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532189+0000","last_change":"2024-04-21T08:44:33.484449+0000","last_active":"2024-04-21T08:44:39.532189+0000","last_peered":"2024-04-21T08:44:39.532189+0000","last_clean":"2024-04-21T08:44:39.532189+0000","last_became_active":"2024-04-21T08:44:33.484155+0000","last_became_peered":"2024-04-21T08:44:33.484155+0000","last_unstale":"2024-04-21T08:44:39.532189+0000","last_undegraded":"2024-04-21T08:44:39.532189+0000","last_fullsized":"2024-04-21T08:44:39.532189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:03.480474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288364+0000","last_change":"2024-04-21T08:44:38.515283+0000","last_active":"2024-04-21T08:44:40.288364+0000","last_peered":"2024-04-21T08:44:40.288364+0000","last_clean":"2024-04-21T08:44:40.288364+0000","last_became_active":"2024-04-21T08:44:38.514764+0000","last_became_peered":"2024-04-21T08:44:38.514764+0000","last_unstale":"2024-04-21T08:44:40.288364+0000","last_undegraded":"2024-04-21T08:44:40.288364+0000","last_fullsized":"2024-04-21T08:44:40.288364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:17:36.378392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:00.002206+0000","last_change":"2024-04-21T08:44:33.489198+0000","last_active":"2024-04-21T08:45:00.002206+0000","last_peered":"2024-04-21T08:45:00.002206+0000","last_clean":"2024-04-21T08:45:00.002206+0000","last_became_active":"2024-04-21T08:44:33.488794+0000","last_became_peered":"2024-04-21T08:44:33.488794+0000","last_unstale":"2024-04-21T08:45:00.002206+0000","last_undegraded":"2024-04-21T08:45:00.002206+0000","last_fullsized":"2024-04-21T08:45:00.002206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:24.948283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666921+0000","last_change":"2024-04-21T08:44:34.488811+0000","last_active":"2024-04-21T08:44:41.666921+0000","last_peered":"2024-04-21T08:44:41.666921+0000","last_clean":"2024-04-21T08:44:41.666921+0000","last_became_active":"2024-04-21T08:44:34.488472+0000","last_became_peered":"2024-04-21T08:44:34.488472+0000","last_unstale":"2024-04-21T08:44:41.666921+0000","last_undegraded":"2024-04-21T08:44:41.666921+0000","last_fullsized":"2024-04-21T08:44:41.666921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:44.779492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175359+0000","last_change":"2024-04-21T08:44:38.522711+0000","last_active":"2024-04-21T08:44:41.175359+0000","last_peered":"2024-04-21T08:44:41.175359+0000","last_clean":"2024-04-21T08:44:41.175359+0000","last_became_active":"2024-04-21T08:44:38.522322+0000","last_became_peered":"2024-04-21T08:44:38.522322+0000","last_unstale":"2024-04-21T08:44:41.175359+0000","last_undegraded":"2024-04-21T08:44:41.175359+0000","last_fullsized":"2024-04-21T08:44:41.175359+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:26:13.323283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526855+0000","last_change":"2024-04-21T08:44:33.481815+0000","last_active":"2024-04-21T08:44:39.526855+0000","last_peered":"2024-04-21T08:44:39.526855+0000","last_clean":"2024-04-21T08:44:39.526855+0000","last_became_active":"2024-04-21T08:44:33.481469+0000","last_became_peered":"2024-04-21T08:44:33.481469+0000","last_unstale":"2024-04-21T08:44:39.526855+0000","last_undegraded":"2024-04-21T08:44:39.526855+0000","last_fullsized":"2024-04-21T08:44:39.526855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:13.970821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175969+0000","last_change":"2024-04-21T08:44:34.496996+0000","last_active":"2024-04-21T08:44:41.175969+0000","last_peered":"2024-04-21T08:44:41.175969+0000","last_clean":"2024-04-21T08:44:41.175969+0000","last_became_active":"2024-04-21T08:44:34.496676+0000","last_became_peered":"2024-04-21T08:44:34.496676+0000","last_unstale":"2024-04-21T08:44:41.175969+0000","last_undegraded":"2024-04-21T08:44:41.175969+0000","last_fullsized":"2024-04-21T08:44:41.175969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:43:08.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527305+0000","last_change":"2024-04-21T08:44:38.516240+0000","last_active":"2024-04-21T08:44:39.527305+0000","last_peered":"2024-04-21T08:44:39.527305+0000","last_clean":"2024-04-21T08:44:39.527305+0000","last_became_active":"2024-04-21T08:44:38.515801+0000","last_became_peered":"2024-04-21T08:44:38.515801+0000","last_unstale":"2024-04-21T08:44:39.527305+0000","last_undegraded":"2024-04-21T08:44:39.527305+0000","last_fullsized":"2024-04-21T08:44:39.527305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:46.520373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:58.452060+0000","last_change":"2024-04-21T08:44:33.488952+0000","last_active":"2024-04-21T08:44:58.452060+0000","last_peered":"2024-04-21T08:44:58.452060+0000","last_clean":"2024-04-21T08:44:58.452060+0000","last_became_active":"2024-04-21T08:44:33.488617+0000","last_became_peered":"2024-04-21T08:44:33.488617+0000","last_unstale":"2024-04-21T08:44:58.452060+0000","last_undegraded":"2024-04-21T08:44:58.452060+0000","last_fullsized":"2024-04-21T08:44:58.452060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:37:24.679132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883556+0000","last_change":"2024-04-21T08:44:34.510568+0000","last_active":"2024-04-21T08:44:41.883556+0000","last_peered":"2024-04-21T08:44:41.883556+0000","last_clean":"2024-04-21T08:44:41.883556+0000","last_became_active":"2024-04-21T08:44:34.509900+0000","last_became_peered":"2024-04-21T08:44:34.509900+0000","last_unstale":"2024-04-21T08:44:41.883556+0000","last_undegraded":"2024-04-21T08:44:41.883556+0000","last_fullsized":"2024-04-21T08:44:41.883556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:08.426141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.663826+0000","last_change":"2024-04-21T08:44:38.517513+0000","last_active":"2024-04-21T08:44:41.663826+0000","last_peered":"2024-04-21T08:44:41.663826+0000","last_clean":"2024-04-21T08:44:41.663826+0000","last_became_active":"2024-04-21T08:44:38.517089+0000","last_became_peered":"2024-04-21T08:44:38.517089+0000","last_unstale":"2024-04-21T08:44:41.663826+0000","last_undegraded":"2024-04-21T08:44:41.663826+0000","last_fullsized":"2024-04-21T08:44:41.663826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:02:32.588353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:58.453812+0000","last_change":"2024-04-21T08:44:33.481803+0000","last_active":"2024-04-21T08:44:58.453812+0000","last_peered":"2024-04-21T08:44:58.453812+0000","last_clean":"2024-04-21T08:44:58.453812+0000","last_became_active":"2024-04-21T08:44:33.481256+0000","last_became_peered":"2024-04-21T08:44:33.481256+0000","last_unstale":"2024-04-21T08:44:58.453812+0000","last_undegraded":"2024-04-21T08:44:58.453812+0000","last_fullsized":"2024-04-21T08:44:58.453812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:17.804557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667445+0000","last_change":"2024-04-21T08:44:34.490206+0000","last_active":"2024-04-21T08:44:41.667445+0000","last_peered":"2024-04-21T08:44:41.667445+0000","last_clean":"2024-04-21T08:44:41.667445+0000","last_became_active":"2024-04-21T08:44:34.489797+0000","last_became_peered":"2024-04-21T08:44:34.489797+0000","last_unstale":"2024-04-21T08:44:41.667445+0000","last_undegraded":"2024-04-21T08:44:41.667445+0000","last_fullsized":"2024-04-21T08:44:41.667445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:09.055429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526919+0000","last_change":"2024-04-21T08:44:38.517385+0000","last_active":"2024-04-21T08:44:39.526919+0000","last_peered":"2024-04-21T08:44:39.526919+0000","last_clean":"2024-04-21T08:44:39.526919+0000","last_became_active":"2024-04-21T08:44:38.516964+0000","last_became_peered":"2024-04-21T08:44:38.516964+0000","last_unstale":"2024-04-21T08:44:39.526919+0000","last_undegraded":"2024-04-21T08:44:39.526919+0000","last_fullsized":"2024-04-21T08:44:39.526919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:33.364076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.999998+0000","last_change":"2024-04-21T08:44:33.490586+0000","last_active":"2024-04-21T08:44:59.999998+0000","last_peered":"2024-04-21T08:44:59.999998+0000","last_clean":"2024-04-21T08:44:59.999998+0000","last_became_active":"2024-04-21T08:44:33.489281+0000","last_became_peered":"2024-04-21T08:44:33.489281+0000","last_unstale":"2024-04-21T08:44:59.999998+0000","last_undegraded":"2024-04-21T08:44:59.999998+0000","last_fullsized":"2024-04-21T08:44:59.999998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:41:29.856165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.287994+0000","last_change":"2024-04-21T08:44:34.493394+0000","last_active":"2024-04-21T08:44:40.287994+0000","last_peered":"2024-04-21T08:44:40.287994+0000","last_clean":"2024-04-21T08:44:40.287994+0000","last_became_active":"2024-04-21T08:44:34.492367+0000","last_became_peered":"2024-04-21T08:44:34.492367+0000","last_unstale":"2024-04-21T08:44:40.287994+0000","last_undegraded":"2024-04-21T08:44:40.287994+0000","last_fullsized":"2024-04-21T08:44:40.287994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:13.111266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.664078+0000","last_change":"2024-04-21T08:44:38.519113+0000","last_active":"2024-04-21T08:44:41.664078+0000","last_peered":"2024-04-21T08:44:41.664078+0000","last_clean":"2024-04-21T08:44:41.664078+0000","last_became_active":"2024-04-21T08:44:38.518469+0000","last_became_peered":"2024-04-21T08:44:38.518469+0000","last_unstale":"2024-04-21T08:44:41.664078+0000","last_undegraded":"2024-04-21T08:44:41.664078+0000","last_fullsized":"2024-04-21T08:44:41.664078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:39:22.200041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.448872+0000","last_change":"2024-04-21T08:44:33.481184+0000","last_active":"2024-04-21T08:44:59.448872+0000","last_peered":"2024-04-21T08:44:59.448872+0000","last_clean":"2024-04-21T08:44:59.448872+0000","last_became_active":"2024-04-21T08:44:33.480835+0000","last_became_peered":"2024-04-21T08:44:33.480835+0000","last_unstale":"2024-04-21T08:44:59.448872+0000","last_undegraded":"2024-04-21T08:44:59.448872+0000","last_fullsized":"2024-04-21T08:44:59.448872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:56:49.463576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175552+0000","last_change":"2024-04-21T08:44:34.502226+0000","last_active":"2024-04-21T08:44:41.175552+0000","last_peered":"2024-04-21T08:44:41.175552+0000","last_clean":"2024-04-21T08:44:41.175552+0000","last_became_active":"2024-04-21T08:44:34.501775+0000","last_became_peered":"2024-04-21T08:44:34.501775+0000","last_unstale":"2024-04-21T08:44:41.175552+0000","last_undegraded":"2024-04-21T08:44:41.175552+0000","last_fullsized":"2024-04-21T08:44:41.175552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:58.325267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.663797+0000","last_change":"2024-04-21T08:44:38.522250+0000","last_active":"2024-04-21T08:44:41.663797+0000","last_peered":"2024-04-21T08:44:41.663797+0000","last_clean":"2024-04-21T08:44:41.663797+0000","last_became_active":"2024-04-21T08:44:38.520870+0000","last_became_peered":"2024-04-21T08:44:38.520870+0000","last_unstale":"2024-04-21T08:44:41.663797+0000","last_undegraded":"2024-04-21T08:44:41.663797+0000","last_fullsized":"2024-04-21T08:44:41.663797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:02.391785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:58.453592+0000","last_change":"2024-04-21T08:44:33.486323+0000","last_active":"2024-04-21T08:44:58.453592+0000","last_peered":"2024-04-21T08:44:58.453592+0000","last_clean":"2024-04-21T08:44:58.453592+0000","last_became_active":"2024-04-21T08:44:33.486016+0000","last_became_peered":"2024-04-21T08:44:33.486016+0000","last_unstale":"2024-04-21T08:44:58.453592+0000","last_undegraded":"2024-04-21T08:44:58.453592+0000","last_fullsized":"2024-04-21T08:44:58.453592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:46.645314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532223+0000","last_change":"2024-04-21T08:44:34.496034+0000","last_active":"2024-04-21T08:44:39.532223+0000","last_peered":"2024-04-21T08:44:39.532223+0000","last_clean":"2024-04-21T08:44:39.532223+0000","last_became_active":"2024-04-21T08:44:34.495641+0000","last_became_peered":"2024-04-21T08:44:34.495641+0000","last_unstale":"2024-04-21T08:44:39.532223+0000","last_undegraded":"2024-04-21T08:44:39.532223+0000","last_fullsized":"2024-04-21T08:44:39.532223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:21:18.893326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666869+0000","last_change":"2024-04-21T08:44:38.517288+0000","last_active":"2024-04-21T08:44:41.666869+0000","last_peered":"2024-04-21T08:44:41.666869+0000","last_clean":"2024-04-21T08:44:41.666869+0000","last_became_active":"2024-04-21T08:44:38.516853+0000","last_became_peered":"2024-04-21T08:44:38.516853+0000","last_unstale":"2024-04-21T08:44:41.666869+0000","last_undegraded":"2024-04-21T08:44:41.666869+0000","last_fullsized":"2024-04-21T08:44:41.666869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:27:22.371703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.999908+0000","last_change":"2024-04-21T08:44:33.491578+0000","last_active":"2024-04-21T08:44:59.999908+0000","last_peered":"2024-04-21T08:44:59.999908+0000","last_clean":"2024-04-21T08:44:59.999908+0000","last_became_active":"2024-04-21T08:44:33.491238+0000","last_became_peered":"2024-04-21T08:44:33.491238+0000","last_unstale":"2024-04-21T08:44:59.999908+0000","last_undegraded":"2024-04-21T08:44:59.999908+0000","last_fullsized":"2024-04-21T08:44:59.999908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:50.399031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532567+0000","last_change":"2024-04-21T08:44:34.497632+0000","last_active":"2024-04-21T08:44:39.532567+0000","last_peered":"2024-04-21T08:44:39.532567+0000","last_clean":"2024-04-21T08:44:39.532567+0000","last_became_active":"2024-04-21T08:44:34.497318+0000","last_became_peered":"2024-04-21T08:44:34.497318+0000","last_unstale":"2024-04-21T08:44:39.532567+0000","last_undegraded":"2024-04-21T08:44:39.532567+0000","last_fullsized":"2024-04-21T08:44:39.532567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:11:33.918530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883445+0000","last_change":"2024-04-21T08:44:38.518414+0000","last_active":"2024-04-21T08:44:41.883445+0000","last_peered":"2024-04-21T08:44:41.883445+0000","last_clean":"2024-04-21T08:44:41.883445+0000","last_became_active":"2024-04-21T08:44:38.517953+0000","last_became_peered":"2024-04-21T08:44:38.517953+0000","last_unstale":"2024-04-21T08:44:41.883445+0000","last_undegraded":"2024-04-21T08:44:41.883445+0000","last_fullsized":"2024-04-21T08:44:41.883445+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:48.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.989232+0000","last_change":"2024-04-21T08:44:33.493896+0000","last_active":"2024-04-21T08:44:59.989232+0000","last_peered":"2024-04-21T08:44:59.989232+0000","last_clean":"2024-04-21T08:44:59.989232+0000","last_became_active":"2024-04-21T08:44:33.493566+0000","last_became_peered":"2024-04-21T08:44:33.493566+0000","last_unstale":"2024-04-21T08:44:59.989232+0000","last_undegraded":"2024-04-21T08:44:59.989232+0000","last_fullsized":"2024-04-21T08:44:59.989232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:47:17.307690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532112+0000","last_change":"2024-04-21T08:44:34.500406+0000","last_active":"2024-04-21T08:44:39.532112+0000","last_peered":"2024-04-21T08:44:39.532112+0000","last_clean":"2024-04-21T08:44:39.532112+0000","last_became_active":"2024-04-21T08:44:34.500048+0000","last_became_peered":"2024-04-21T08:44:34.500048+0000","last_unstale":"2024-04-21T08:44:39.532112+0000","last_undegraded":"2024-04-21T08:44:39.532112+0000","last_fullsized":"2024-04-21T08:44:39.532112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:10:58.166155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667651+0000","last_change":"2024-04-21T08:44:38.514812+0000","last_active":"2024-04-21T08:44:41.667651+0000","last_peered":"2024-04-21T08:44:41.667651+0000","last_clean":"2024-04-21T08:44:41.667651+0000","last_became_active":"2024-04-21T08:44:38.514247+0000","last_became_peered":"2024-04-21T08:44:38.514247+0000","last_unstale":"2024-04-21T08:44:41.667651+0000","last_undegraded":"2024-04-21T08:44:41.667651+0000","last_fullsized":"2024-04-21T08:44:41.667651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:06:10.588453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:01.010669+0000","last_change":"2024-04-21T08:44:33.493887+0000","last_active":"2024-04-21T08:45:01.010669+0000","last_peered":"2024-04-21T08:45:01.010669+0000","last_clean":"2024-04-21T08:45:01.010669+0000","last_became_active":"2024-04-21T08:44:33.493555+0000","last_became_peered":"2024-04-21T08:44:33.493555+0000","last_unstale":"2024-04-21T08:45:01.010669+0000","last_undegraded":"2024-04-21T08:45:01.010669+0000","last_fullsized":"2024-04-21T08:45:01.010669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:18.089102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532632+0000","last_change":"2024-04-21T08:44:34.503259+0000","last_active":"2024-04-21T08:44:39.532632+0000","last_peered":"2024-04-21T08:44:39.532632+0000","last_clean":"2024-04-21T08:44:39.532632+0000","last_became_active":"2024-04-21T08:44:34.502845+0000","last_became_peered":"2024-04-21T08:44:34.502845+0000","last_unstale":"2024-04-21T08:44:39.532632+0000","last_undegraded":"2024-04-21T08:44:39.532632+0000","last_fullsized":"2024-04-21T08:44:39.532632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:19.902178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288148+0000","last_change":"2024-04-21T08:44:34.497608+0000","last_active":"2024-04-21T08:44:40.288148+0000","last_peered":"2024-04-21T08:44:40.288148+0000","last_clean":"2024-04-21T08:44:40.288148+0000","last_became_active":"2024-04-21T08:44:34.497255+0000","last_became_peered":"2024-04-21T08:44:34.497255+0000","last_unstale":"2024-04-21T08:44:40.288148+0000","last_undegraded":"2024-04-21T08:44:40.288148+0000","last_fullsized":"2024-04-21T08:44:40.288148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:51:15.181837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527317+0000","last_change":"2024-04-21T08:44:34.495408+0000","last_active":"2024-04-21T08:44:39.527317+0000","last_peered":"2024-04-21T08:44:39.527317+0000","last_clean":"2024-04-21T08:44:39.527317+0000","last_became_active":"2024-04-21T08:44:34.495038+0000","last_became_peered":"2024-04-21T08:44:34.495038+0000","last_unstale":"2024-04-21T08:44:39.527317+0000","last_undegraded":"2024-04-21T08:44:39.527317+0000","last_fullsized":"2024-04-21T08:44:39.527317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:59.512245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527170+0000","last_change":"2024-04-21T08:44:34.493292+0000","last_active":"2024-04-21T08:44:39.527170+0000","last_peered":"2024-04-21T08:44:39.527170+0000","last_clean":"2024-04-21T08:44:39.527170+0000","last_became_active":"2024-04-21T08:44:34.492913+0000","last_became_peered":"2024-04-21T08:44:34.492913+0000","last_unstale":"2024-04-21T08:44:39.527170+0000","last_undegraded":"2024-04-21T08:44:39.527170+0000","last_fullsized":"2024-04-21T08:44:39.527170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:24.271859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883555+0000","last_change":"2024-04-21T08:44:34.510448+0000","last_active":"2024-04-21T08:44:41.883555+0000","last_peered":"2024-04-21T08:44:41.883555+0000","last_clean":"2024-04-21T08:44:41.883555+0000","last_became_active":"2024-04-21T08:44:34.509958+0000","last_became_peered":"2024-04-21T08:44:34.509958+0000","last_unstale":"2024-04-21T08:44:41.883555+0000","last_undegraded":"2024-04-21T08:44:41.883555+0000","last_fullsized":"2024-04-21T08:44:41.883555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:36.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667008+0000","last_change":"2024-04-21T08:44:34.497415+0000","last_active":"2024-04-21T08:44:41.667008+0000","last_peered":"2024-04-21T08:44:41.667008+0000","last_clean":"2024-04-21T08:44:41.667008+0000","last_became_active":"2024-04-21T08:44:34.497120+0000","last_became_peered":"2024-04-21T08:44:34.497120+0000","last_unstale":"2024-04-21T08:44:41.667008+0000","last_undegraded":"2024-04-21T08:44:41.667008+0000","last_fullsized":"2024-04-21T08:44:41.667008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:30.232338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288466+0000","last_change":"2024-04-21T08:44:34.500356+0000","last_active":"2024-04-21T08:44:40.288466+0000","last_peered":"2024-04-21T08:44:40.288466+0000","last_clean":"2024-04-21T08:44:40.288466+0000","last_became_active":"2024-04-21T08:44:34.500032+0000","last_became_peered":"2024-04-21T08:44:34.500032+0000","last_unstale":"2024-04-21T08:44:40.288466+0000","last_undegraded":"2024-04-21T08:44:40.288466+0000","last_fullsized":"2024-04-21T08:44:40.288466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:24.900221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527921+0000","last_change":"2024-04-21T08:44:34.493629+0000","last_active":"2024-04-21T08:44:39.527921+0000","last_peered":"2024-04-21T08:44:39.527921+0000","last_clean":"2024-04-21T08:44:39.527921+0000","last_became_active":"2024-04-21T08:44:34.493273+0000","last_became_peered":"2024-04-21T08:44:34.493273+0000","last_unstale":"2024-04-21T08:44:39.527921+0000","last_undegraded":"2024-04-21T08:44:39.527921+0000","last_fullsized":"2024-04-21T08:44:39.527921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:22:42.698957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.176063+0000","last_change":"2024-04-21T08:44:34.493427+0000","last_active":"2024-04-21T08:44:41.176063+0000","last_peered":"2024-04-21T08:44:41.176063+0000","last_clean":"2024-04-21T08:44:41.176063+0000","last_became_active":"2024-04-21T08:44:34.493105+0000","last_became_peered":"2024-04-21T08:44:34.493105+0000","last_unstale":"2024-04-21T08:44:41.176063+0000","last_undegraded":"2024-04-21T08:44:41.176063+0000","last_fullsized":"2024-04-21T08:44:41.176063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:37.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667898+0000","last_change":"2024-04-21T08:44:34.493930+0000","last_active":"2024-04-21T08:44:41.667898+0000","last_peered":"2024-04-21T08:44:41.667898+0000","last_clean":"2024-04-21T08:44:41.667898+0000","last_became_active":"2024-04-21T08:44:34.491734+0000","last_became_peered":"2024-04-21T08:44:34.491734+0000","last_unstale":"2024-04-21T08:44:41.667898+0000","last_undegraded":"2024-04-21T08:44:41.667898+0000","last_fullsized":"2024-04-21T08:44:41.667898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:23:07.483103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175373+0000","last_change":"2024-04-21T08:44:34.499420+0000","last_active":"2024-04-21T08:44:41.175373+0000","last_peered":"2024-04-21T08:44:41.175373+0000","last_clean":"2024-04-21T08:44:41.175373+0000","last_became_active":"2024-04-21T08:44:34.499079+0000","last_became_peered":"2024-04-21T08:44:34.499079+0000","last_unstale":"2024-04-21T08:44:41.175373+0000","last_undegraded":"2024-04-21T08:44:41.175373+0000","last_fullsized":"2024-04-21T08:44:41.175373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:45.351422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667070+0000","last_change":"2024-04-21T08:44:34.493420+0000","last_active":"2024-04-21T08:44:41.667070+0000","last_peered":"2024-04-21T08:44:41.667070+0000","last_clean":"2024-04-21T08:44:41.667070+0000","last_became_active":"2024-04-21T08:44:34.493098+0000","last_became_peered":"2024-04-21T08:44:34.493098+0000","last_unstale":"2024-04-21T08:44:41.667070+0000","last_undegraded":"2024-04-21T08:44:41.667070+0000","last_fullsized":"2024-04-21T08:44:41.667070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:22:34.316822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666858+0000","last_change":"2024-04-21T08:44:34.494760+0000","last_active":"2024-04-21T08:44:41.666858+0000","last_peered":"2024-04-21T08:44:41.666858+0000","last_clean":"2024-04-21T08:44:41.666858+0000","last_became_active":"2024-04-21T08:44:34.494265+0000","last_became_peered":"2024-04-21T08:44:34.494265+0000","last_unstale":"2024-04-21T08:44:41.666858+0000","last_undegraded":"2024-04-21T08:44:41.666858+0000","last_fullsized":"2024-04-21T08:44:41.666858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:16.035030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288182+0000","last_change":"2024-04-21T08:44:34.498785+0000","last_active":"2024-04-21T08:44:40.288182+0000","last_peered":"2024-04-21T08:44:40.288182+0000","last_clean":"2024-04-21T08:44:40.288182+0000","last_became_active":"2024-04-21T08:44:34.495159+0000","last_became_peered":"2024-04-21T08:44:34.495159+0000","last_unstale":"2024-04-21T08:44:40.288182+0000","last_undegraded":"2024-04-21T08:44:40.288182+0000","last_fullsized":"2024-04-21T08:44:40.288182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:03:20.095016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9552,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":161,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":44433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383699,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":478407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645336,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1613824,"data_stored":1072381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":477999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070498,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":477953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1581056,"data_stored":1068165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":477937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":489409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":477305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":483029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":477499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":477859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36916,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708236,"statfs":{"total":95995035648,"available":95957233664,"internally_reserved":0,"allocated":1617920,"data_stored":1069821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":5544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:45:03.391 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:45:03.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.523+0000 7f7d17104640 1 -- 172.21.15.82:0/1941513994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10101fd0 msgr2=0x7f7d10102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:03.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.523+0000 7f7d17104640 1 --2- 172.21.15.82:0/1941513994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10101fd0 0x7f7d10102450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009900 tx=0x7f7d0402bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:03.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.523+0000 7f7d17104640 1 -- 172.21.15.82:0/1941513994 shutdown_connections 2024-04-21T08:45:03.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.523+0000 7f7d17104640 1 --2- 172.21.15.82:0/1941513994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d10102990 0x7f7d101092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.523+0000 7f7d17104640 1 --2- 172.21.15.82:0/1941513994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10101fd0 0x7f7d10102450 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.523+0000 7f7d17104640 1 --2- 172.21.15.82:0/1941513994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d10100e80 0x7f7d101012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.523+0000 7f7d17104640 1 -- 172.21.15.82:0/1941513994 >> 172.21.15.82:0/1941513994 conn(0x7f7d100fc410 msgr2=0x7f7d100fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:03.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.523+0000 7f7d17104640 1 -- 172.21.15.82:0/1941513994 shutdown_connections 2024-04-21T08:45:03.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.523+0000 7f7d17104640 1 -- 172.21.15.82:0/1941513994 wait complete. 2024-04-21T08:45:03.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d17104640 1 Processor -- start 2024-04-21T08:45:03.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d17104640 1 -- start start 2024-04-21T08:45:03.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d17104640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d10100e80 0x7f7d10198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:03.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d17104640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d10101fd0 0x7f7d10198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:03.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d17104640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10102990 0x7f7d1019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:03.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d17104640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1010db40 con 0x7f7d10102990 2024-04-21T08:45:03.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d17104640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1010d9c0 con 0x7f7d10100e80 2024-04-21T08:45:03.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d17104640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1010dcc0 con 0x7f7d10101fd0 2024-04-21T08:45:03.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d156a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10102990 0x7f7d1019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d156a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10102990 0x7f7d1019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58386/0 (socket says 172.21.15.82:58386) 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d156a1640 1 -- 172.21.15.82:0/2827399956 learned_addr learned my addr 172.21.15.82:0/2827399956 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d14ea0640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d10100e80 0x7f7d10198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d156a1640 1 -- 172.21.15.82:0/2827399956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d10101fd0 msgr2=0x7f7d10198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d156a1640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d10101fd0 0x7f7d10198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d156a1640 1 -- 172.21.15.82:0/2827399956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d10100e80 msgr2=0x7f7d10198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d156a1640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d10100e80 0x7f7d10198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d156a1640 1 -- 172.21.15.82:0/2827399956 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1019fe20 con 0x7f7d10102990 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.524+0000 7f7d156a1640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10102990 0x7f7d1019f7b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800b970 tx=0x7f7d0800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.525+0000 7f7d0dffb640 1 -- 172.21.15.82:0/2827399956 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08019070 con 0x7f7d10102990 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.525+0000 7f7d0dffb640 1 -- 172.21.15.82:0/2827399956 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d08014440 con 0x7f7d10102990 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.525+0000 7f7d17104640 1 -- 172.21.15.82:0/2827399956 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d101a0110 con 0x7f7d10102990 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.525+0000 7f7d0dffb640 1 -- 172.21.15.82:0/2827399956 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d080075d0 con 0x7f7d10102990 2024-04-21T08:45:03.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.525+0000 7f7d17104640 1 -- 172.21.15.82:0/2827399956 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d101a0690 con 0x7f7d10102990 2024-04-21T08:45:03.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.526+0000 7f7d17104640 1 -- 172.21.15.82:0/2827399956 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d10108260 con 0x7f7d10102990 2024-04-21T08:45:03.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.527+0000 7f7d0dffb640 1 -- 172.21.15.82:0/2827399956 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7d0801c070 con 0x7f7d10102990 2024-04-21T08:45:03.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.527+0000 7f7d0dffb640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7cf80a3c40 0x7f7cf80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:03.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.527+0000 7f7d14ea0640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7cf80a3c40 0x7f7cf80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:03.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.527+0000 7f7d0dffb640 1 -- 172.21.15.82:0/2827399956 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7d080cfcf0 con 0x7f7d10102990 2024-04-21T08:45:03.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.528+0000 7f7d14ea0640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7cf80a3c40 0x7f7cf80a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d000058c0 tx=0x7f7d00005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:03.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.532+0000 7f7d0dffb640 1 -- 172.21.15.82:0/2827399956 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d0809c440 con 0x7f7d10102990 2024-04-21T08:45:03.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.710+0000 7f7d17104640 1 -- 172.21.15.82:0/2827399956 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d10062390 con 0x7f7cf80a3c40 2024-04-21T08:45:03.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.719+0000 7f7d0dffb640 1 -- 172.21.15.82:0/2827399956 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340333 (secure 0 0 0) 0x7f7d10062390 con 0x7f7cf80a3c40 2024-04-21T08:45:03.722 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:45:03.724 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-21T08:45:03.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.722+0000 7f7d17104640 1 -- 172.21.15.82:0/2827399956 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7cf80a3c40 msgr2=0x7f7cf80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:03.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.722+0000 7f7d17104640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7cf80a3c40 0x7f7cf80a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d000058c0 tx=0x7f7d00005c60 comp rx=0 tx=0).stop 2024-04-21T08:45:03.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.722+0000 7f7d17104640 1 -- 172.21.15.82:0/2827399956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10102990 msgr2=0x7f7d1019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:03.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.722+0000 7f7d17104640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10102990 0x7f7d1019f7b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800b970 tx=0x7f7d0800be30 comp rx=0 tx=0).stop 2024-04-21T08:45:03.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.722+0000 7f7d17104640 1 -- 172.21.15.82:0/2827399956 shutdown_connections 2024-04-21T08:45:03.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.722+0000 7f7d17104640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7cf80a3c40 0x7f7cf80a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.722+0000 7f7d17104640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10102990 0x7f7d1019f7b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.722+0000 7f7d17104640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d10101fd0 0x7f7d10198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.722+0000 7f7d17104640 1 --2- 172.21.15.82:0/2827399956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d10100e80 0x7f7d10198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:03.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.722+0000 7f7d17104640 1 -- 172.21.15.82:0/2827399956 >> 172.21.15.82:0/2827399956 conn(0x7f7d100fc410 msgr2=0x7f7d10105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:03.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.722+0000 7f7d17104640 1 -- 172.21.15.82:0/2827399956 shutdown_connections 2024-04-21T08:45:03.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:03.722+0000 7f7d17104640 1 -- 172.21.15.82:0/2827399956 wait complete. 2024-04-21T08:45:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:03 smithi134 ceph-mon[31772]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:03.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:03 smithi134 ceph-mon[31772]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:03.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:03 smithi134 ceph-mon[31772]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-04-21T08:45:03.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:03 smithi134 ceph-mon[31772]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:03.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:03 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2388589415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:03.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:03 smithi134 ceph-mon[31772]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T08:45:03.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:03 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3837380000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:45:03.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:03 smithi082 ceph-mon[27967]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:03.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:03 smithi082 ceph-mon[27967]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:03.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:03 smithi082 ceph-mon[27967]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-04-21T08:45:03.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:03 smithi082 ceph-mon[27967]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:03.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:03 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2388589415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:03.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:03 smithi082 ceph-mon[27967]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T08:45:03.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:03 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3837380000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:45:03.733 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-04-21T08:45:02.318637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599920,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":162,"num_read_kb":145,"num_write":342,"num_write_kb":3002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":354352,"kb_used_data":13512,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124587472,"statfs":{"total":1151940427776,"available":1151577571328,"internally_reserved":0,"allocated":13836288,"data_stored":7527764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7142,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":116,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003041"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288627+0000","last_change":"2024-04-21T08:44:34.494393+0000","last_active":"2024-04-21T08:44:40.288627+0000","last_peered":"2024-04-21T08:44:40.288627+0000","last_clean":"2024-04-21T08:44:40.288627+0000","last_became_active":"2024-04-21T08:44:34.493583+0000","last_became_peered":"2024-04-21T08:44:34.493583+0000","last_unstale":"2024-04-21T08:44:40.288627+0000","last_undegraded":"2024-04-21T08:44:40.288627+0000","last_fullsized":"2024-04-21T08:44:40.288627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:37.211676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883666+0000","last_change":"2024-04-21T08:44:34.496291+0000","last_active":"2024-04-21T08:44:41.883666+0000","last_peered":"2024-04-21T08:44:41.883666+0000","last_clean":"2024-04-21T08:44:41.883666+0000","last_became_active":"2024-04-21T08:44:34.495019+0000","last_became_peered":"2024-04-21T08:44:34.495019+0000","last_unstale":"2024-04-21T08:44:41.883666+0000","last_undegraded":"2024-04-21T08:44:41.883666+0000","last_fullsized":"2024-04-21T08:44:41.883666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:05.309847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883447+0000","last_change":"2024-04-21T08:44:34.497948+0000","last_active":"2024-04-21T08:44:41.883447+0000","last_peered":"2024-04-21T08:44:41.883447+0000","last_clean":"2024-04-21T08:44:41.883447+0000","last_became_active":"2024-04-21T08:44:34.497657+0000","last_became_peered":"2024-04-21T08:44:34.497657+0000","last_unstale":"2024-04-21T08:44:41.883447+0000","last_undegraded":"2024-04-21T08:44:41.883447+0000","last_fullsized":"2024-04-21T08:44:41.883447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:11.463715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.664151+0000","last_change":"2024-04-21T08:44:34.492682+0000","last_active":"2024-04-21T08:44:41.664151+0000","last_peered":"2024-04-21T08:44:41.664151+0000","last_clean":"2024-04-21T08:44:41.664151+0000","last_became_active":"2024-04-21T08:44:34.492230+0000","last_became_peered":"2024-04-21T08:44:34.492230+0000","last_unstale":"2024-04-21T08:44:41.664151+0000","last_undegraded":"2024-04-21T08:44:41.664151+0000","last_fullsized":"2024-04-21T08:44:41.664151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:40.365608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667826+0000","last_change":"2024-04-21T08:44:34.492160+0000","last_active":"2024-04-21T08:44:41.667826+0000","last_peered":"2024-04-21T08:44:41.667826+0000","last_clean":"2024-04-21T08:44:41.667826+0000","last_became_active":"2024-04-21T08:44:34.491701+0000","last_became_peered":"2024-04-21T08:44:34.491701+0000","last_unstale":"2024-04-21T08:44:41.667826+0000","last_undegraded":"2024-04-21T08:44:41.667826+0000","last_fullsized":"2024-04-21T08:44:41.667826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:50.542661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667591+0000","last_change":"2024-04-21T08:44:34.497341+0000","last_active":"2024-04-21T08:44:41.667591+0000","last_peered":"2024-04-21T08:44:41.667591+0000","last_clean":"2024-04-21T08:44:41.667591+0000","last_became_active":"2024-04-21T08:44:34.495794+0000","last_became_peered":"2024-04-21T08:44:34.495794+0000","last_unstale":"2024-04-21T08:44:41.667591+0000","last_undegraded":"2024-04-21T08:44:41.667591+0000","last_fullsized":"2024-04-21T08:44:41.667591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:13.854753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.531922+0000","last_change":"2024-04-21T08:44:34.501578+0000","last_active":"2024-04-21T08:44:39.531922+0000","last_peered":"2024-04-21T08:44:39.531922+0000","last_clean":"2024-04-21T08:44:39.531922+0000","last_became_active":"2024-04-21T08:44:34.500848+0000","last_became_peered":"2024-04-21T08:44:34.500848+0000","last_unstale":"2024-04-21T08:44:39.531922+0000","last_undegraded":"2024-04-21T08:44:39.531922+0000","last_fullsized":"2024-04-21T08:44:39.531922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:53:27.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175188+0000","last_change":"2024-04-21T08:44:34.502975+0000","last_active":"2024-04-21T08:44:41.175188+0000","last_peered":"2024-04-21T08:44:41.175188+0000","last_clean":"2024-04-21T08:44:41.175188+0000","last_became_active":"2024-04-21T08:44:34.502676+0000","last_became_peered":"2024-04-21T08:44:34.502676+0000","last_unstale":"2024-04-21T08:44:41.175188+0000","last_undegraded":"2024-04-21T08:44:41.175188+0000","last_fullsized":"2024-04-21T08:44:41.175188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:32:13.881603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667470+0000","last_change":"2024-04-21T08:44:34.495703+0000","last_active":"2024-04-21T08:44:41.667470+0000","last_peered":"2024-04-21T08:44:41.667470+0000","last_clean":"2024-04-21T08:44:41.667470+0000","last_became_active":"2024-04-21T08:44:34.495380+0000","last_became_peered":"2024-04-21T08:44:34.495380+0000","last_unstale":"2024-04-21T08:44:41.667470+0000","last_undegraded":"2024-04-21T08:44:41.667470+0000","last_fullsized":"2024-04-21T08:44:41.667470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:45:28.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668103+0000","last_change":"2024-04-21T08:44:34.499361+0000","last_active":"2024-04-21T08:44:41.668103+0000","last_peered":"2024-04-21T08:44:41.668103+0000","last_clean":"2024-04-21T08:44:41.668103+0000","last_became_active":"2024-04-21T08:44:34.499044+0000","last_became_peered":"2024-04-21T08:44:34.499044+0000","last_unstale":"2024-04-21T08:44:41.668103+0000","last_undegraded":"2024-04-21T08:44:41.668103+0000","last_fullsized":"2024-04-21T08:44:41.668103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:12.118985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532452+0000","last_change":"2024-04-21T08:44:34.499631+0000","last_active":"2024-04-21T08:44:39.532452+0000","last_peered":"2024-04-21T08:44:39.532452+0000","last_clean":"2024-04-21T08:44:39.532452+0000","last_became_active":"2024-04-21T08:44:34.499316+0000","last_became_peered":"2024-04-21T08:44:34.499316+0000","last_unstale":"2024-04-21T08:44:39.532452+0000","last_undegraded":"2024-04-21T08:44:39.532452+0000","last_fullsized":"2024-04-21T08:44:39.532452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:26.123017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.664097+0000","last_change":"2024-04-21T08:44:34.504857+0000","last_active":"2024-04-21T08:44:41.664097+0000","last_peered":"2024-04-21T08:44:41.664097+0000","last_clean":"2024-04-21T08:44:41.664097+0000","last_became_active":"2024-04-21T08:44:34.504564+0000","last_became_peered":"2024-04-21T08:44:34.504564+0000","last_unstale":"2024-04-21T08:44:41.664097+0000","last_undegraded":"2024-04-21T08:44:41.664097+0000","last_fullsized":"2024-04-21T08:44:41.664097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:32:05.216162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666811+0000","last_change":"2024-04-21T08:44:34.484074+0000","last_active":"2024-04-21T08:44:41.666811+0000","last_peered":"2024-04-21T08:44:41.666811+0000","last_clean":"2024-04-21T08:44:41.666811+0000","last_became_active":"2024-04-21T08:44:34.483309+0000","last_became_peered":"2024-04-21T08:44:34.483309+0000","last_unstale":"2024-04-21T08:44:41.666811+0000","last_undegraded":"2024-04-21T08:44:41.666811+0000","last_fullsized":"2024-04-21T08:44:41.666811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:39.686048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883370+0000","last_change":"2024-04-21T08:44:34.499159+0000","last_active":"2024-04-21T08:44:41.883370+0000","last_peered":"2024-04-21T08:44:41.883370+0000","last_clean":"2024-04-21T08:44:41.883370+0000","last_became_active":"2024-04-21T08:44:34.496609+0000","last_became_peered":"2024-04-21T08:44:34.496609+0000","last_unstale":"2024-04-21T08:44:41.883370+0000","last_undegraded":"2024-04-21T08:44:41.883370+0000","last_fullsized":"2024-04-21T08:44:41.883370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:11.716321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883288+0000","last_change":"2024-04-21T08:44:34.499004+0000","last_active":"2024-04-21T08:44:41.883288+0000","last_peered":"2024-04-21T08:44:41.883288+0000","last_clean":"2024-04-21T08:44:41.883288+0000","last_became_active":"2024-04-21T08:44:34.498652+0000","last_became_peered":"2024-04-21T08:44:34.498652+0000","last_unstale":"2024-04-21T08:44:41.883288+0000","last_undegraded":"2024-04-21T08:44:41.883288+0000","last_fullsized":"2024-04-21T08:44:41.883288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:48:24.154896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666623+0000","last_change":"2024-04-21T08:44:34.494018+0000","last_active":"2024-04-21T08:44:41.666623+0000","last_peered":"2024-04-21T08:44:41.666623+0000","last_clean":"2024-04-21T08:44:41.666623+0000","last_became_active":"2024-04-21T08:44:34.493692+0000","last_became_peered":"2024-04-21T08:44:34.493692+0000","last_unstale":"2024-04-21T08:44:41.666623+0000","last_undegraded":"2024-04-21T08:44:41.666623+0000","last_fullsized":"2024-04-21T08:44:41.666623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:25:16.944160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288456+0000","last_change":"2024-04-21T08:44:34.493231+0000","last_active":"2024-04-21T08:44:40.288456+0000","last_peered":"2024-04-21T08:44:40.288456+0000","last_clean":"2024-04-21T08:44:40.288456+0000","last_became_active":"2024-04-21T08:44:34.492826+0000","last_became_peered":"2024-04-21T08:44:34.492826+0000","last_unstale":"2024-04-21T08:44:40.288456+0000","last_undegraded":"2024-04-21T08:44:40.288456+0000","last_fullsized":"2024-04-21T08:44:40.288456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:36.815012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526864+0000","last_change":"2024-04-21T08:44:34.495142+0000","last_active":"2024-04-21T08:44:39.526864+0000","last_peered":"2024-04-21T08:44:39.526864+0000","last_clean":"2024-04-21T08:44:39.526864+0000","last_became_active":"2024-04-21T08:44:34.494803+0000","last_became_peered":"2024-04-21T08:44:34.494803+0000","last_unstale":"2024-04-21T08:44:39.526864+0000","last_undegraded":"2024-04-21T08:44:39.526864+0000","last_fullsized":"2024-04-21T08:44:39.526864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:22.974053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527704+0000","last_change":"2024-04-21T08:44:34.491252+0000","last_active":"2024-04-21T08:44:39.527704+0000","last_peered":"2024-04-21T08:44:39.527704+0000","last_clean":"2024-04-21T08:44:39.527704+0000","last_became_active":"2024-04-21T08:44:34.490814+0000","last_became_peered":"2024-04-21T08:44:34.490814+0000","last_unstale":"2024-04-21T08:44:39.527704+0000","last_undegraded":"2024-04-21T08:44:39.527704+0000","last_fullsized":"2024-04-21T08:44:39.527704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:54:52.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527559+0000","last_change":"2024-04-21T08:44:38.518647+0000","last_active":"2024-04-21T08:44:39.527559+0000","last_peered":"2024-04-21T08:44:39.527559+0000","last_clean":"2024-04-21T08:44:39.527559+0000","last_became_active":"2024-04-21T08:44:38.518239+0000","last_became_peered":"2024-04-21T08:44:38.518239+0000","last_unstale":"2024-04-21T08:44:39.527559+0000","last_undegraded":"2024-04-21T08:44:39.527559+0000","last_fullsized":"2024-04-21T08:44:39.527559+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:44.781542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526856+0000","last_change":"2024-04-21T08:44:33.484484+0000","last_active":"2024-04-21T08:44:39.526856+0000","last_peered":"2024-04-21T08:44:39.526856+0000","last_clean":"2024-04-21T08:44:39.526856+0000","last_became_active":"2024-04-21T08:44:33.484160+0000","last_became_peered":"2024-04-21T08:44:33.484160+0000","last_unstale":"2024-04-21T08:44:39.526856+0000","last_undegraded":"2024-04-21T08:44:39.526856+0000","last_fullsized":"2024-04-21T08:44:39.526856+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:45.962598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175630+0000","last_change":"2024-04-21T08:44:34.500722+0000","last_active":"2024-04-21T08:44:41.175630+0000","last_peered":"2024-04-21T08:44:41.175630+0000","last_clean":"2024-04-21T08:44:41.175630+0000","last_became_active":"2024-04-21T08:44:34.500388+0000","last_became_peered":"2024-04-21T08:44:34.500388+0000","last_unstale":"2024-04-21T08:44:41.175630+0000","last_undegraded":"2024-04-21T08:44:41.175630+0000","last_fullsized":"2024-04-21T08:44:41.175630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:05.876636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.663609+0000","last_change":"2024-04-21T08:44:38.521925+0000","last_active":"2024-04-21T08:44:41.663609+0000","last_peered":"2024-04-21T08:44:41.663609+0000","last_clean":"2024-04-21T08:44:41.663609+0000","last_became_active":"2024-04-21T08:44:38.520375+0000","last_became_peered":"2024-04-21T08:44:38.520375+0000","last_unstale":"2024-04-21T08:44:41.663609+0000","last_undegraded":"2024-04-21T08:44:41.663609+0000","last_fullsized":"2024-04-21T08:44:41.663609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:15:15.558088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.176075+0000","last_change":"2024-04-21T08:44:33.490508+0000","last_active":"2024-04-21T08:44:41.176075+0000","last_peered":"2024-04-21T08:44:41.176075+0000","last_clean":"2024-04-21T08:44:41.176075+0000","last_became_active":"2024-04-21T08:44:33.490084+0000","last_became_peered":"2024-04-21T08:44:33.490084+0000","last_unstale":"2024-04-21T08:44:41.176075+0000","last_undegraded":"2024-04-21T08:44:41.176075+0000","last_fullsized":"2024-04-21T08:44:41.176075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:21.553435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883522+0000","last_change":"2024-04-21T08:44:34.491843+0000","last_active":"2024-04-21T08:44:41.883522+0000","last_peered":"2024-04-21T08:44:41.883522+0000","last_clean":"2024-04-21T08:44:41.883522+0000","last_became_active":"2024-04-21T08:44:34.490578+0000","last_became_peered":"2024-04-21T08:44:34.490578+0000","last_unstale":"2024-04-21T08:44:41.883522+0000","last_undegraded":"2024-04-21T08:44:41.883522+0000","last_fullsized":"2024-04-21T08:44:41.883522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:05:12.272330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.663767+0000","last_change":"2024-04-21T08:44:38.520258+0000","last_active":"2024-04-21T08:44:41.663767+0000","last_peered":"2024-04-21T08:44:41.663767+0000","last_clean":"2024-04-21T08:44:41.663767+0000","last_became_active":"2024-04-21T08:44:38.519824+0000","last_became_peered":"2024-04-21T08:44:38.519824+0000","last_unstale":"2024-04-21T08:44:41.663767+0000","last_undegraded":"2024-04-21T08:44:41.663767+0000","last_fullsized":"2024-04-21T08:44:41.663767+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:39.716745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'11","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:01.014251+0000","last_change":"2024-04-21T08:44:33.484934+0000","last_active":"2024-04-21T08:45:01.014251+0000","last_peered":"2024-04-21T08:45:01.014251+0000","last_clean":"2024-04-21T08:45:01.014251+0000","last_became_active":"2024-04-21T08:44:33.484591+0000","last_became_peered":"2024-04-21T08:44:33.484591+0000","last_unstale":"2024-04-21T08:45:01.014251+0000","last_undegraded":"2024-04-21T08:45:01.014251+0000","last_fullsized":"2024-04-21T08:45:01.014251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:48.826611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667054+0000","last_change":"2024-04-21T08:44:34.501109+0000","last_active":"2024-04-21T08:44:41.667054+0000","last_peered":"2024-04-21T08:44:41.667054+0000","last_clean":"2024-04-21T08:44:41.667054+0000","last_became_active":"2024-04-21T08:44:34.500750+0000","last_became_peered":"2024-04-21T08:44:34.500750+0000","last_unstale":"2024-04-21T08:44:41.667054+0000","last_undegraded":"2024-04-21T08:44:41.667054+0000","last_fullsized":"2024-04-21T08:44:41.667054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:29:28.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.663702+0000","last_change":"2024-04-21T08:44:38.519633+0000","last_active":"2024-04-21T08:44:41.663702+0000","last_peered":"2024-04-21T08:44:41.663702+0000","last_clean":"2024-04-21T08:44:41.663702+0000","last_became_active":"2024-04-21T08:44:38.519215+0000","last_became_peered":"2024-04-21T08:44:38.519215+0000","last_unstale":"2024-04-21T08:44:41.663702+0000","last_undegraded":"2024-04-21T08:44:41.663702+0000","last_fullsized":"2024-04-21T08:44:41.663702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:43:59.695844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:01.013231+0000","last_change":"2024-04-21T08:44:33.483267+0000","last_active":"2024-04-21T08:45:01.013231+0000","last_peered":"2024-04-21T08:45:01.013231+0000","last_clean":"2024-04-21T08:45:01.013231+0000","last_became_active":"2024-04-21T08:44:33.482932+0000","last_became_peered":"2024-04-21T08:44:33.482932+0000","last_unstale":"2024-04-21T08:45:01.013231+0000","last_undegraded":"2024-04-21T08:45:01.013231+0000","last_fullsized":"2024-04-21T08:45:01.013231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:53.311550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668877+0000","last_change":"2024-04-21T08:44:34.497389+0000","last_active":"2024-04-21T08:44:41.668877+0000","last_peered":"2024-04-21T08:44:41.668877+0000","last_clean":"2024-04-21T08:44:41.668877+0000","last_became_active":"2024-04-21T08:44:34.497030+0000","last_became_peered":"2024-04-21T08:44:34.497030+0000","last_unstale":"2024-04-21T08:44:41.668877+0000","last_undegraded":"2024-04-21T08:44:41.668877+0000","last_fullsized":"2024-04-21T08:44:41.668877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:20.371888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527623+0000","last_change":"2024-04-21T08:44:38.518095+0000","last_active":"2024-04-21T08:44:39.527623+0000","last_peered":"2024-04-21T08:44:39.527623+0000","last_clean":"2024-04-21T08:44:39.527623+0000","last_became_active":"2024-04-21T08:44:38.517010+0000","last_became_peered":"2024-04-21T08:44:38.517010+0000","last_unstale":"2024-04-21T08:44:39.527623+0000","last_undegraded":"2024-04-21T08:44:39.527623+0000","last_fullsized":"2024-04-21T08:44:39.527623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:02.379512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.991761+0000","last_change":"2024-04-21T08:44:33.487999+0000","last_active":"2024-04-21T08:44:59.991761+0000","last_peered":"2024-04-21T08:44:59.991761+0000","last_clean":"2024-04-21T08:44:59.991761+0000","last_became_active":"2024-04-21T08:44:33.487666+0000","last_became_peered":"2024-04-21T08:44:33.487666+0000","last_unstale":"2024-04-21T08:44:59.991761+0000","last_undegraded":"2024-04-21T08:44:59.991761+0000","last_fullsized":"2024-04-21T08:44:59.991761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:01.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668218+0000","last_change":"2024-04-21T08:44:34.494213+0000","last_active":"2024-04-21T08:44:41.668218+0000","last_peered":"2024-04-21T08:44:41.668218+0000","last_clean":"2024-04-21T08:44:41.668218+0000","last_became_active":"2024-04-21T08:44:34.493853+0000","last_became_peered":"2024-04-21T08:44:34.493853+0000","last_unstale":"2024-04-21T08:44:41.668218+0000","last_undegraded":"2024-04-21T08:44:41.668218+0000","last_fullsized":"2024-04-21T08:44:41.668218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:42.499131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527869+0000","last_change":"2024-04-21T08:44:38.519612+0000","last_active":"2024-04-21T08:44:39.527869+0000","last_peered":"2024-04-21T08:44:39.527869+0000","last_clean":"2024-04-21T08:44:39.527869+0000","last_became_active":"2024-04-21T08:44:38.518946+0000","last_became_peered":"2024-04-21T08:44:38.518946+0000","last_unstale":"2024-04-21T08:44:39.527869+0000","last_undegraded":"2024-04-21T08:44:39.527869+0000","last_fullsized":"2024-04-21T08:44:39.527869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:04.982377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:00.002549+0000","last_change":"2024-04-21T08:44:33.488751+0000","last_active":"2024-04-21T08:45:00.002549+0000","last_peered":"2024-04-21T08:45:00.002549+0000","last_clean":"2024-04-21T08:45:00.002549+0000","last_became_active":"2024-04-21T08:44:33.488425+0000","last_became_peered":"2024-04-21T08:44:33.488425+0000","last_unstale":"2024-04-21T08:45:00.002549+0000","last_undegraded":"2024-04-21T08:45:00.002549+0000","last_fullsized":"2024-04-21T08:45:00.002549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:28:47.868814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668668+0000","last_change":"2024-04-21T08:44:34.492770+0000","last_active":"2024-04-21T08:44:41.668668+0000","last_peered":"2024-04-21T08:44:41.668668+0000","last_clean":"2024-04-21T08:44:41.668668+0000","last_became_active":"2024-04-21T08:44:34.491340+0000","last_became_peered":"2024-04-21T08:44:34.491340+0000","last_unstale":"2024-04-21T08:44:41.668668+0000","last_undegraded":"2024-04-21T08:44:41.668668+0000","last_fullsized":"2024-04-21T08:44:41.668668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:47:54.280990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288500+0000","last_change":"2024-04-21T08:44:38.518888+0000","last_active":"2024-04-21T08:44:40.288500+0000","last_peered":"2024-04-21T08:44:40.288500+0000","last_clean":"2024-04-21T08:44:40.288500+0000","last_became_active":"2024-04-21T08:44:38.518421+0000","last_became_peered":"2024-04-21T08:44:38.518421+0000","last_unstale":"2024-04-21T08:44:40.288500+0000","last_undegraded":"2024-04-21T08:44:40.288500+0000","last_fullsized":"2024-04-21T08:44:40.288500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:21.119364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.462823+0000","last_change":"2024-04-21T08:44:33.484383+0000","last_active":"2024-04-21T08:44:59.462823+0000","last_peered":"2024-04-21T08:44:59.462823+0000","last_clean":"2024-04-21T08:44:59.462823+0000","last_became_active":"2024-04-21T08:44:33.484083+0000","last_became_peered":"2024-04-21T08:44:33.484083+0000","last_unstale":"2024-04-21T08:44:59.462823+0000","last_undegraded":"2024-04-21T08:44:59.462823+0000","last_fullsized":"2024-04-21T08:44:59.462823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:13:14.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175324+0000","last_change":"2024-04-21T08:44:34.495636+0000","last_active":"2024-04-21T08:44:41.175324+0000","last_peered":"2024-04-21T08:44:41.175324+0000","last_clean":"2024-04-21T08:44:41.175324+0000","last_became_active":"2024-04-21T08:44:34.494967+0000","last_became_peered":"2024-04-21T08:44:34.494967+0000","last_unstale":"2024-04-21T08:44:41.175324+0000","last_undegraded":"2024-04-21T08:44:41.175324+0000","last_fullsized":"2024-04-21T08:44:41.175324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:47:02.379208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883626+0000","last_change":"2024-04-21T08:44:38.515145+0000","last_active":"2024-04-21T08:44:41.883626+0000","last_peered":"2024-04-21T08:44:41.883626+0000","last_clean":"2024-04-21T08:44:41.883626+0000","last_became_active":"2024-04-21T08:44:38.514755+0000","last_became_peered":"2024-04-21T08:44:38.514755+0000","last_unstale":"2024-04-21T08:44:41.883626+0000","last_undegraded":"2024-04-21T08:44:41.883626+0000","last_fullsized":"2024-04-21T08:44:41.883626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:38:15.826293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:58.454221+0000","last_change":"2024-04-21T08:44:33.484714+0000","last_active":"2024-04-21T08:44:58.454221+0000","last_peered":"2024-04-21T08:44:58.454221+0000","last_clean":"2024-04-21T08:44:58.454221+0000","last_became_active":"2024-04-21T08:44:33.484385+0000","last_became_peered":"2024-04-21T08:44:33.484385+0000","last_unstale":"2024-04-21T08:44:58.454221+0000","last_undegraded":"2024-04-21T08:44:58.454221+0000","last_fullsized":"2024-04-21T08:44:58.454221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:03.406860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526310+0000","last_change":"2024-04-21T08:44:34.490989+0000","last_active":"2024-04-21T08:44:39.526310+0000","last_peered":"2024-04-21T08:44:39.526310+0000","last_clean":"2024-04-21T08:44:39.526310+0000","last_became_active":"2024-04-21T08:44:34.490658+0000","last_became_peered":"2024-04-21T08:44:34.490658+0000","last_unstale":"2024-04-21T08:44:39.526310+0000","last_undegraded":"2024-04-21T08:44:39.526310+0000","last_fullsized":"2024-04-21T08:44:39.526310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:42:15.537913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175918+0000","last_change":"2024-04-21T08:44:38.519723+0000","last_active":"2024-04-21T08:44:41.175918+0000","last_peered":"2024-04-21T08:44:41.175918+0000","last_clean":"2024-04-21T08:44:41.175918+0000","last_became_active":"2024-04-21T08:44:38.519336+0000","last_became_peered":"2024-04-21T08:44:38.519336+0000","last_unstale":"2024-04-21T08:44:41.175918+0000","last_undegraded":"2024-04-21T08:44:41.175918+0000","last_fullsized":"2024-04-21T08:44:41.175918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:00.332153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.991050+0000","last_change":"2024-04-21T08:44:33.484404+0000","last_active":"2024-04-21T08:44:59.991050+0000","last_peered":"2024-04-21T08:44:59.991050+0000","last_clean":"2024-04-21T08:44:59.991050+0000","last_became_active":"2024-04-21T08:44:33.484088+0000","last_became_peered":"2024-04-21T08:44:33.484088+0000","last_unstale":"2024-04-21T08:44:59.991050+0000","last_undegraded":"2024-04-21T08:44:59.991050+0000","last_fullsized":"2024-04-21T08:44:59.991050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:41.948622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667459+0000","last_change":"2024-04-21T08:44:34.502598+0000","last_active":"2024-04-21T08:44:41.667459+0000","last_peered":"2024-04-21T08:44:41.667459+0000","last_clean":"2024-04-21T08:44:41.667459+0000","last_became_active":"2024-04-21T08:44:34.502057+0000","last_became_peered":"2024-04-21T08:44:34.502057+0000","last_unstale":"2024-04-21T08:44:41.667459+0000","last_undegraded":"2024-04-21T08:44:41.667459+0000","last_fullsized":"2024-04-21T08:44:41.667459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:24.052873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526419+0000","last_change":"2024-04-21T08:44:38.519699+0000","last_active":"2024-04-21T08:44:39.526419+0000","last_peered":"2024-04-21T08:44:39.526419+0000","last_clean":"2024-04-21T08:44:39.526419+0000","last_became_active":"2024-04-21T08:44:38.517655+0000","last_became_peered":"2024-04-21T08:44:38.517655+0000","last_unstale":"2024-04-21T08:44:39.526419+0000","last_undegraded":"2024-04-21T08:44:39.526419+0000","last_fullsized":"2024-04-21T08:44:39.526419+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:12.784479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532460+0000","last_change":"2024-04-21T08:44:33.488799+0000","last_active":"2024-04-21T08:44:39.532460+0000","last_peered":"2024-04-21T08:44:39.532460+0000","last_clean":"2024-04-21T08:44:39.532460+0000","last_became_active":"2024-04-21T08:44:33.487948+0000","last_became_peered":"2024-04-21T08:44:33.487948+0000","last_unstale":"2024-04-21T08:44:39.532460+0000","last_undegraded":"2024-04-21T08:44:39.532460+0000","last_fullsized":"2024-04-21T08:44:39.532460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:57.347588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883571+0000","last_change":"2024-04-21T08:44:34.511133+0000","last_active":"2024-04-21T08:44:41.883571+0000","last_peered":"2024-04-21T08:44:41.883571+0000","last_clean":"2024-04-21T08:44:41.883571+0000","last_became_active":"2024-04-21T08:44:34.510350+0000","last_became_peered":"2024-04-21T08:44:34.510350+0000","last_unstale":"2024-04-21T08:44:41.883571+0000","last_undegraded":"2024-04-21T08:44:41.883571+0000","last_fullsized":"2024-04-21T08:44:41.883571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:30:53.317856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.663964+0000","last_change":"2024-04-21T08:44:38.517956+0000","last_active":"2024-04-21T08:44:41.663964+0000","last_peered":"2024-04-21T08:44:41.663964+0000","last_clean":"2024-04-21T08:44:41.663964+0000","last_became_active":"2024-04-21T08:44:38.517422+0000","last_became_peered":"2024-04-21T08:44:38.517422+0000","last_unstale":"2024-04-21T08:44:41.663964+0000","last_undegraded":"2024-04-21T08:44:41.663964+0000","last_fullsized":"2024-04-21T08:44:41.663964+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:04.650784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.993047+0000","last_change":"2024-04-21T08:44:33.489151+0000","last_active":"2024-04-21T08:44:59.993047+0000","last_peered":"2024-04-21T08:44:59.993047+0000","last_clean":"2024-04-21T08:44:59.993047+0000","last_became_active":"2024-04-21T08:44:33.488781+0000","last_became_peered":"2024-04-21T08:44:33.488781+0000","last_unstale":"2024-04-21T08:44:59.993047+0000","last_undegraded":"2024-04-21T08:44:59.993047+0000","last_fullsized":"2024-04-21T08:44:59.993047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:40.675100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667148+0000","last_change":"2024-04-21T08:44:34.496528+0000","last_active":"2024-04-21T08:44:41.667148+0000","last_peered":"2024-04-21T08:44:41.667148+0000","last_clean":"2024-04-21T08:44:41.667148+0000","last_became_active":"2024-04-21T08:44:34.496223+0000","last_became_peered":"2024-04-21T08:44:34.496223+0000","last_unstale":"2024-04-21T08:44:41.667148+0000","last_undegraded":"2024-04-21T08:44:41.667148+0000","last_fullsized":"2024-04-21T08:44:41.667148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:56.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526362+0000","last_change":"2024-04-21T08:44:38.517537+0000","last_active":"2024-04-21T08:44:39.526362+0000","last_peered":"2024-04-21T08:44:39.526362+0000","last_clean":"2024-04-21T08:44:39.526362+0000","last_became_active":"2024-04-21T08:44:38.517134+0000","last_became_peered":"2024-04-21T08:44:38.517134+0000","last_unstale":"2024-04-21T08:44:39.526362+0000","last_undegraded":"2024-04-21T08:44:39.526362+0000","last_fullsized":"2024-04-21T08:44:39.526362+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:38.634418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.993248+0000","last_change":"2024-04-21T08:44:33.485582+0000","last_active":"2024-04-21T08:44:59.993248+0000","last_peered":"2024-04-21T08:44:59.993248+0000","last_clean":"2024-04-21T08:44:59.993248+0000","last_became_active":"2024-04-21T08:44:33.485249+0000","last_became_peered":"2024-04-21T08:44:33.485249+0000","last_unstale":"2024-04-21T08:44:59.993248+0000","last_undegraded":"2024-04-21T08:44:59.993248+0000","last_fullsized":"2024-04-21T08:44:59.993248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:16:10.571694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668221+0000","last_change":"2024-04-21T08:44:34.491361+0000","last_active":"2024-04-21T08:44:41.668221+0000","last_peered":"2024-04-21T08:44:41.668221+0000","last_clean":"2024-04-21T08:44:41.668221+0000","last_became_active":"2024-04-21T08:44:34.491050+0000","last_became_peered":"2024-04-21T08:44:34.491050+0000","last_unstale":"2024-04-21T08:44:41.668221+0000","last_undegraded":"2024-04-21T08:44:41.668221+0000","last_fullsized":"2024-04-21T08:44:41.668221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:39.082448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.669089+0000","last_change":"2024-04-21T08:44:38.528745+0000","last_active":"2024-04-21T08:44:41.669089+0000","last_peered":"2024-04-21T08:44:41.669089+0000","last_clean":"2024-04-21T08:44:41.669089+0000","last_became_active":"2024-04-21T08:44:38.528275+0000","last_became_peered":"2024-04-21T08:44:38.528275+0000","last_unstale":"2024-04-21T08:44:41.669089+0000","last_undegraded":"2024-04-21T08:44:41.669089+0000","last_fullsized":"2024-04-21T08:44:41.669089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:06:11.338420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.989320+0000","last_change":"2024-04-21T08:44:33.482059+0000","last_active":"2024-04-21T08:44:59.989320+0000","last_peered":"2024-04-21T08:44:59.989320+0000","last_clean":"2024-04-21T08:44:59.989320+0000","last_became_active":"2024-04-21T08:44:33.481663+0000","last_became_peered":"2024-04-21T08:44:33.481663+0000","last_unstale":"2024-04-21T08:44:59.989320+0000","last_undegraded":"2024-04-21T08:44:59.989320+0000","last_fullsized":"2024-04-21T08:44:59.989320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:01.240452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666788+0000","last_change":"2024-04-21T08:44:34.491398+0000","last_active":"2024-04-21T08:44:41.666788+0000","last_peered":"2024-04-21T08:44:41.666788+0000","last_clean":"2024-04-21T08:44:41.666788+0000","last_became_active":"2024-04-21T08:44:34.491073+0000","last_became_peered":"2024-04-21T08:44:34.491073+0000","last_unstale":"2024-04-21T08:44:41.666788+0000","last_undegraded":"2024-04-21T08:44:41.666788+0000","last_fullsized":"2024-04-21T08:44:41.666788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:22.566063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288552+0000","last_change":"2024-04-21T08:44:38.515653+0000","last_active":"2024-04-21T08:44:40.288552+0000","last_peered":"2024-04-21T08:44:40.288552+0000","last_clean":"2024-04-21T08:44:40.288552+0000","last_became_active":"2024-04-21T08:44:38.514796+0000","last_became_peered":"2024-04-21T08:44:38.514796+0000","last_unstale":"2024-04-21T08:44:40.288552+0000","last_undegraded":"2024-04-21T08:44:40.288552+0000","last_fullsized":"2024-04-21T08:44:40.288552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:00:22.662136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:58.454089+0000","last_change":"2024-04-21T08:44:33.487073+0000","last_active":"2024-04-21T08:44:58.454089+0000","last_peered":"2024-04-21T08:44:58.454089+0000","last_clean":"2024-04-21T08:44:58.454089+0000","last_became_active":"2024-04-21T08:44:33.486740+0000","last_became_peered":"2024-04-21T08:44:33.486740+0000","last_unstale":"2024-04-21T08:44:58.454089+0000","last_undegraded":"2024-04-21T08:44:58.454089+0000","last_fullsized":"2024-04-21T08:44:58.454089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:32.673074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532074+0000","last_change":"2024-04-21T08:44:34.496757+0000","last_active":"2024-04-21T08:44:39.532074+0000","last_peered":"2024-04-21T08:44:39.532074+0000","last_clean":"2024-04-21T08:44:39.532074+0000","last_became_active":"2024-04-21T08:44:34.496422+0000","last_became_peered":"2024-04-21T08:44:34.496422+0000","last_unstale":"2024-04-21T08:44:39.532074+0000","last_undegraded":"2024-04-21T08:44:39.532074+0000","last_fullsized":"2024-04-21T08:44:39.532074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:08.686810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526775+0000","last_change":"2024-04-21T08:44:38.520935+0000","last_active":"2024-04-21T08:44:39.526775+0000","last_peered":"2024-04-21T08:44:39.526775+0000","last_clean":"2024-04-21T08:44:39.526775+0000","last_became_active":"2024-04-21T08:44:38.520481+0000","last_became_peered":"2024-04-21T08:44:38.520481+0000","last_unstale":"2024-04-21T08:44:39.526775+0000","last_undegraded":"2024-04-21T08:44:39.526775+0000","last_fullsized":"2024-04-21T08:44:39.526775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:12.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.998062+0000","last_change":"2024-04-21T08:44:33.485797+0000","last_active":"2024-04-21T08:44:59.998062+0000","last_peered":"2024-04-21T08:44:59.998062+0000","last_clean":"2024-04-21T08:44:59.998062+0000","last_became_active":"2024-04-21T08:44:33.485496+0000","last_became_peered":"2024-04-21T08:44:33.485496+0000","last_unstale":"2024-04-21T08:44:59.998062+0000","last_undegraded":"2024-04-21T08:44:59.998062+0000","last_fullsized":"2024-04-21T08:44:59.998062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:11:34.604611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532243+0000","last_change":"2024-04-21T08:44:34.506453+0000","last_active":"2024-04-21T08:44:39.532243+0000","last_peered":"2024-04-21T08:44:39.532243+0000","last_clean":"2024-04-21T08:44:39.532243+0000","last_became_active":"2024-04-21T08:44:34.506121+0000","last_became_peered":"2024-04-21T08:44:34.506121+0000","last_unstale":"2024-04-21T08:44:39.532243+0000","last_undegraded":"2024-04-21T08:44:39.532243+0000","last_fullsized":"2024-04-21T08:44:39.532243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:07.927366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667077+0000","last_change":"2024-04-21T08:44:38.523718+0000","last_active":"2024-04-21T08:44:41.667077+0000","last_peered":"2024-04-21T08:44:41.667077+0000","last_clean":"2024-04-21T08:44:41.667077+0000","last_became_active":"2024-04-21T08:44:38.523003+0000","last_became_peered":"2024-04-21T08:44:38.523003+0000","last_unstale":"2024-04-21T08:44:41.667077+0000","last_undegraded":"2024-04-21T08:44:41.667077+0000","last_fullsized":"2024-04-21T08:44:41.667077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:20.837450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:00.002286+0000","last_change":"2024-04-21T08:44:33.482029+0000","last_active":"2024-04-21T08:45:00.002286+0000","last_peered":"2024-04-21T08:45:00.002286+0000","last_clean":"2024-04-21T08:45:00.002286+0000","last_became_active":"2024-04-21T08:44:33.481713+0000","last_became_peered":"2024-04-21T08:44:33.481713+0000","last_unstale":"2024-04-21T08:45:00.002286+0000","last_undegraded":"2024-04-21T08:45:00.002286+0000","last_fullsized":"2024-04-21T08:45:00.002286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:46:34.717244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668579+0000","last_change":"2024-04-21T08:44:34.491944+0000","last_active":"2024-04-21T08:44:41.668579+0000","last_peered":"2024-04-21T08:44:41.668579+0000","last_clean":"2024-04-21T08:44:41.668579+0000","last_became_active":"2024-04-21T08:44:34.491638+0000","last_became_peered":"2024-04-21T08:44:34.491638+0000","last_unstale":"2024-04-21T08:44:41.668579+0000","last_undegraded":"2024-04-21T08:44:41.668579+0000","last_fullsized":"2024-04-21T08:44:41.668579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:08:07.925937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668596+0000","last_change":"2024-04-21T08:44:38.528971+0000","last_active":"2024-04-21T08:44:41.668596+0000","last_peered":"2024-04-21T08:44:41.668596+0000","last_clean":"2024-04-21T08:44:41.668596+0000","last_became_active":"2024-04-21T08:44:38.528453+0000","last_became_peered":"2024-04-21T08:44:38.528453+0000","last_unstale":"2024-04-21T08:44:41.668596+0000","last_undegraded":"2024-04-21T08:44:41.668596+0000","last_fullsized":"2024-04-21T08:44:41.668596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:50:31.237749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.990116+0000","last_change":"2024-04-21T08:44:33.479262+0000","last_active":"2024-04-21T08:44:59.990116+0000","last_peered":"2024-04-21T08:44:59.990116+0000","last_clean":"2024-04-21T08:44:59.990116+0000","last_became_active":"2024-04-21T08:44:33.478907+0000","last_became_peered":"2024-04-21T08:44:33.478907+0000","last_unstale":"2024-04-21T08:44:59.990116+0000","last_undegraded":"2024-04-21T08:44:59.990116+0000","last_fullsized":"2024-04-21T08:44:59.990116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:33:43.879126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288287+0000","last_change":"2024-04-21T08:44:34.498503+0000","last_active":"2024-04-21T08:44:40.288287+0000","last_peered":"2024-04-21T08:44:40.288287+0000","last_clean":"2024-04-21T08:44:40.288287+0000","last_became_active":"2024-04-21T08:44:34.495140+0000","last_became_peered":"2024-04-21T08:44:34.495140+0000","last_unstale":"2024-04-21T08:44:40.288287+0000","last_undegraded":"2024-04-21T08:44:40.288287+0000","last_fullsized":"2024-04-21T08:44:40.288287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:24.518349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668163+0000","last_change":"2024-04-21T08:44:38.517717+0000","last_active":"2024-04-21T08:44:41.668163+0000","last_peered":"2024-04-21T08:44:41.668163+0000","last_clean":"2024-04-21T08:44:41.668163+0000","last_became_active":"2024-04-21T08:44:38.517272+0000","last_became_peered":"2024-04-21T08:44:38.517272+0000","last_unstale":"2024-04-21T08:44:41.668163+0000","last_undegraded":"2024-04-21T08:44:41.668163+0000","last_fullsized":"2024-04-21T08:44:41.668163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.664130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526633+0000","last_change":"2024-04-21T08:44:33.477498+0000","last_active":"2024-04-21T08:44:39.526633+0000","last_peered":"2024-04-21T08:44:39.526633+0000","last_clean":"2024-04-21T08:44:39.526633+0000","last_became_active":"2024-04-21T08:44:33.477139+0000","last_became_peered":"2024-04-21T08:44:33.477139+0000","last_unstale":"2024-04-21T08:44:39.526633+0000","last_undegraded":"2024-04-21T08:44:39.526633+0000","last_fullsized":"2024-04-21T08:44:39.526633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:26:59.390583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667976+0000","last_change":"2024-04-21T08:44:34.493586+0000","last_active":"2024-04-21T08:44:41.667976+0000","last_peered":"2024-04-21T08:44:41.667976+0000","last_clean":"2024-04-21T08:44:41.667976+0000","last_became_active":"2024-04-21T08:44:34.493258+0000","last_became_peered":"2024-04-21T08:44:34.493258+0000","last_unstale":"2024-04-21T08:44:41.667976+0000","last_undegraded":"2024-04-21T08:44:41.667976+0000","last_fullsized":"2024-04-21T08:44:41.667976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:11:18.783332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666892+0000","last_change":"2024-04-21T08:44:38.519783+0000","last_active":"2024-04-21T08:44:41.666892+0000","last_peered":"2024-04-21T08:44:41.666892+0000","last_clean":"2024-04-21T08:44:41.666892+0000","last_became_active":"2024-04-21T08:44:38.519397+0000","last_became_peered":"2024-04-21T08:44:38.519397+0000","last_unstale":"2024-04-21T08:44:41.666892+0000","last_undegraded":"2024-04-21T08:44:41.666892+0000","last_fullsized":"2024-04-21T08:44:41.666892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:08.740240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532012+0000","last_change":"2024-04-21T08:44:33.490098+0000","last_active":"2024-04-21T08:44:39.532012+0000","last_peered":"2024-04-21T08:44:39.532012+0000","last_clean":"2024-04-21T08:44:39.532012+0000","last_became_active":"2024-04-21T08:44:33.489691+0000","last_became_peered":"2024-04-21T08:44:33.489691+0000","last_unstale":"2024-04-21T08:44:39.532012+0000","last_undegraded":"2024-04-21T08:44:39.532012+0000","last_fullsized":"2024-04-21T08:44:39.532012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:44.965998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667817+0000","last_change":"2024-04-21T08:44:34.498543+0000","last_active":"2024-04-21T08:44:41.667817+0000","last_peered":"2024-04-21T08:44:41.667817+0000","last_clean":"2024-04-21T08:44:41.667817+0000","last_became_active":"2024-04-21T08:44:34.498211+0000","last_became_peered":"2024-04-21T08:44:34.498211+0000","last_unstale":"2024-04-21T08:44:41.667817+0000","last_undegraded":"2024-04-21T08:44:41.667817+0000","last_fullsized":"2024-04-21T08:44:41.667817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:16.506695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667029+0000","last_change":"2024-04-21T08:44:38.521010+0000","last_active":"2024-04-21T08:44:41.667029+0000","last_peered":"2024-04-21T08:44:41.667029+0000","last_clean":"2024-04-21T08:44:41.667029+0000","last_became_active":"2024-04-21T08:44:38.520605+0000","last_became_peered":"2024-04-21T08:44:38.520605+0000","last_unstale":"2024-04-21T08:44:41.667029+0000","last_undegraded":"2024-04-21T08:44:41.667029+0000","last_fullsized":"2024-04-21T08:44:41.667029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:50.378803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668008+0000","last_change":"2024-04-21T08:44:34.496437+0000","last_active":"2024-04-21T08:44:41.668008+0000","last_peered":"2024-04-21T08:44:41.668008+0000","last_clean":"2024-04-21T08:44:41.668008+0000","last_became_active":"2024-04-21T08:44:34.496108+0000","last_became_peered":"2024-04-21T08:44:34.496108+0000","last_unstale":"2024-04-21T08:44:41.668008+0000","last_undegraded":"2024-04-21T08:44:41.668008+0000","last_fullsized":"2024-04-21T08:44:41.668008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:15:55.237988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:00.994306+0000","last_change":"2024-04-21T08:44:33.494031+0000","last_active":"2024-04-21T08:45:00.994306+0000","last_peered":"2024-04-21T08:45:00.994306+0000","last_clean":"2024-04-21T08:45:00.994306+0000","last_became_active":"2024-04-21T08:44:33.493580+0000","last_became_peered":"2024-04-21T08:44:33.493580+0000","last_unstale":"2024-04-21T08:45:00.994306+0000","last_undegraded":"2024-04-21T08:45:00.994306+0000","last_fullsized":"2024-04-21T08:45:00.994306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:39:48.230616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532479+0000","last_change":"2024-04-21T08:44:38.518067+0000","last_active":"2024-04-21T08:44:39.532479+0000","last_peered":"2024-04-21T08:44:39.532479+0000","last_clean":"2024-04-21T08:44:39.532479+0000","last_became_active":"2024-04-21T08:44:38.517594+0000","last_became_peered":"2024-04-21T08:44:38.517594+0000","last_unstale":"2024-04-21T08:44:39.532479+0000","last_undegraded":"2024-04-21T08:44:39.532479+0000","last_fullsized":"2024-04-21T08:44:39.532479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:35.261582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175439+0000","last_change":"2024-04-21T08:44:34.503645+0000","last_active":"2024-04-21T08:44:41.175439+0000","last_peered":"2024-04-21T08:44:41.175439+0000","last_clean":"2024-04-21T08:44:41.175439+0000","last_became_active":"2024-04-21T08:44:34.500736+0000","last_became_peered":"2024-04-21T08:44:34.500736+0000","last_unstale":"2024-04-21T08:44:41.175439+0000","last_undegraded":"2024-04-21T08:44:41.175439+0000","last_fullsized":"2024-04-21T08:44:41.175439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:46.084991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.998088+0000","last_change":"2024-04-21T08:44:33.485110+0000","last_active":"2024-04-21T08:44:59.998088+0000","last_peered":"2024-04-21T08:44:59.998088+0000","last_clean":"2024-04-21T08:44:59.998088+0000","last_became_active":"2024-04-21T08:44:33.484722+0000","last_became_peered":"2024-04-21T08:44:33.484722+0000","last_unstale":"2024-04-21T08:44:59.998088+0000","last_undegraded":"2024-04-21T08:44:59.998088+0000","last_fullsized":"2024-04-21T08:44:59.998088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:47:16.773560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532053+0000","last_change":"2024-04-21T08:43:17.984594+0000","last_active":"2024-04-21T08:44:39.532053+0000","last_peered":"2024-04-21T08:44:39.532053+0000","last_clean":"2024-04-21T08:44:39.532053+0000","last_became_active":"2024-04-21T08:43:17.984207+0000","last_became_peered":"2024-04-21T08:43:17.984207+0000","last_unstale":"2024-04-21T08:44:39.532053+0000","last_undegraded":"2024-04-21T08:44:39.532053+0000","last_fullsized":"2024-04-21T08:44:39.532053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_clean_scrub_stamp":"2024-04-21T08:39:50.910206+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:35.260499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.668991+0000","last_change":"2024-04-21T08:44:38.516792+0000","last_active":"2024-04-21T08:44:41.668991+0000","last_peered":"2024-04-21T08:44:41.668991+0000","last_clean":"2024-04-21T08:44:41.668991+0000","last_became_active":"2024-04-21T08:44:38.516369+0000","last_became_peered":"2024-04-21T08:44:38.516369+0000","last_unstale":"2024-04-21T08:44:41.668991+0000","last_undegraded":"2024-04-21T08:44:41.668991+0000","last_fullsized":"2024-04-21T08:44:41.668991+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:29:12.278396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666626+0000","last_change":"2024-04-21T08:44:34.499975+0000","last_active":"2024-04-21T08:44:41.666626+0000","last_peered":"2024-04-21T08:44:41.666626+0000","last_clean":"2024-04-21T08:44:41.666626+0000","last_became_active":"2024-04-21T08:44:34.499646+0000","last_became_peered":"2024-04-21T08:44:34.499646+0000","last_unstale":"2024-04-21T08:44:41.666626+0000","last_undegraded":"2024-04-21T08:44:41.666626+0000","last_fullsized":"2024-04-21T08:44:41.666626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:19:38.082502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532189+0000","last_change":"2024-04-21T08:44:33.484449+0000","last_active":"2024-04-21T08:44:39.532189+0000","last_peered":"2024-04-21T08:44:39.532189+0000","last_clean":"2024-04-21T08:44:39.532189+0000","last_became_active":"2024-04-21T08:44:33.484155+0000","last_became_peered":"2024-04-21T08:44:33.484155+0000","last_unstale":"2024-04-21T08:44:39.532189+0000","last_undegraded":"2024-04-21T08:44:39.532189+0000","last_fullsized":"2024-04-21T08:44:39.532189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:03.480474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288364+0000","last_change":"2024-04-21T08:44:38.515283+0000","last_active":"2024-04-21T08:44:40.288364+0000","last_peered":"2024-04-21T08:44:40.288364+0000","last_clean":"2024-04-21T08:44:40.288364+0000","last_became_active":"2024-04-21T08:44:38.514764+0000","last_became_peered":"2024-04-21T08:44:38.514764+0000","last_unstale":"2024-04-21T08:44:40.288364+0000","last_undegraded":"2024-04-21T08:44:40.288364+0000","last_fullsized":"2024-04-21T08:44:40.288364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:17:36.378392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:00.002206+0000","last_change":"2024-04-21T08:44:33.489198+0000","last_active":"2024-04-21T08:45:00.002206+0000","last_peered":"2024-04-21T08:45:00.002206+0000","last_clean":"2024-04-21T08:45:00.002206+0000","last_became_active":"2024-04-21T08:44:33.488794+0000","last_became_peered":"2024-04-21T08:44:33.488794+0000","last_unstale":"2024-04-21T08:45:00.002206+0000","last_undegraded":"2024-04-21T08:45:00.002206+0000","last_fullsized":"2024-04-21T08:45:00.002206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:24.948283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666921+0000","last_change":"2024-04-21T08:44:34.488811+0000","last_active":"2024-04-21T08:44:41.666921+0000","last_peered":"2024-04-21T08:44:41.666921+0000","last_clean":"2024-04-21T08:44:41.666921+0000","last_became_active":"2024-04-21T08:44:34.488472+0000","last_became_peered":"2024-04-21T08:44:34.488472+0000","last_unstale":"2024-04-21T08:44:41.666921+0000","last_undegraded":"2024-04-21T08:44:41.666921+0000","last_fullsized":"2024-04-21T08:44:41.666921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:44.779492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175359+0000","last_change":"2024-04-21T08:44:38.522711+0000","last_active":"2024-04-21T08:44:41.175359+0000","last_peered":"2024-04-21T08:44:41.175359+0000","last_clean":"2024-04-21T08:44:41.175359+0000","last_became_active":"2024-04-21T08:44:38.522322+0000","last_became_peered":"2024-04-21T08:44:38.522322+0000","last_unstale":"2024-04-21T08:44:41.175359+0000","last_undegraded":"2024-04-21T08:44:41.175359+0000","last_fullsized":"2024-04-21T08:44:41.175359+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:26:13.323283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526855+0000","last_change":"2024-04-21T08:44:33.481815+0000","last_active":"2024-04-21T08:44:39.526855+0000","last_peered":"2024-04-21T08:44:39.526855+0000","last_clean":"2024-04-21T08:44:39.526855+0000","last_became_active":"2024-04-21T08:44:33.481469+0000","last_became_peered":"2024-04-21T08:44:33.481469+0000","last_unstale":"2024-04-21T08:44:39.526855+0000","last_undegraded":"2024-04-21T08:44:39.526855+0000","last_fullsized":"2024-04-21T08:44:39.526855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:13.970821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175969+0000","last_change":"2024-04-21T08:44:34.496996+0000","last_active":"2024-04-21T08:44:41.175969+0000","last_peered":"2024-04-21T08:44:41.175969+0000","last_clean":"2024-04-21T08:44:41.175969+0000","last_became_active":"2024-04-21T08:44:34.496676+0000","last_became_peered":"2024-04-21T08:44:34.496676+0000","last_unstale":"2024-04-21T08:44:41.175969+0000","last_undegraded":"2024-04-21T08:44:41.175969+0000","last_fullsized":"2024-04-21T08:44:41.175969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:43:08.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527305+0000","last_change":"2024-04-21T08:44:38.516240+0000","last_active":"2024-04-21T08:44:39.527305+0000","last_peered":"2024-04-21T08:44:39.527305+0000","last_clean":"2024-04-21T08:44:39.527305+0000","last_became_active":"2024-04-21T08:44:38.515801+0000","last_became_peered":"2024-04-21T08:44:38.515801+0000","last_unstale":"2024-04-21T08:44:39.527305+0000","last_undegraded":"2024-04-21T08:44:39.527305+0000","last_fullsized":"2024-04-21T08:44:39.527305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:46.520373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:58.452060+0000","last_change":"2024-04-21T08:44:33.488952+0000","last_active":"2024-04-21T08:44:58.452060+0000","last_peered":"2024-04-21T08:44:58.452060+0000","last_clean":"2024-04-21T08:44:58.452060+0000","last_became_active":"2024-04-21T08:44:33.488617+0000","last_became_peered":"2024-04-21T08:44:33.488617+0000","last_unstale":"2024-04-21T08:44:58.452060+0000","last_undegraded":"2024-04-21T08:44:58.452060+0000","last_fullsized":"2024-04-21T08:44:58.452060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:37:24.679132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883556+0000","last_change":"2024-04-21T08:44:34.510568+0000","last_active":"2024-04-21T08:44:41.883556+0000","last_peered":"2024-04-21T08:44:41.883556+0000","last_clean":"2024-04-21T08:44:41.883556+0000","last_became_active":"2024-04-21T08:44:34.509900+0000","last_became_peered":"2024-04-21T08:44:34.509900+0000","last_unstale":"2024-04-21T08:44:41.883556+0000","last_undegraded":"2024-04-21T08:44:41.883556+0000","last_fullsized":"2024-04-21T08:44:41.883556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:08.426141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.663826+0000","last_change":"2024-04-21T08:44:38.517513+0000","last_active":"2024-04-21T08:44:41.663826+0000","last_peered":"2024-04-21T08:44:41.663826+0000","last_clean":"2024-04-21T08:44:41.663826+0000","last_became_active":"2024-04-21T08:44:38.517089+0000","last_became_peered":"2024-04-21T08:44:38.517089+0000","last_unstale":"2024-04-21T08:44:41.663826+0000","last_undegraded":"2024-04-21T08:44:41.663826+0000","last_fullsized":"2024-04-21T08:44:41.663826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:02:32.588353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:58.453812+0000","last_change":"2024-04-21T08:44:33.481803+0000","last_active":"2024-04-21T08:44:58.453812+0000","last_peered":"2024-04-21T08:44:58.453812+0000","last_clean":"2024-04-21T08:44:58.453812+0000","last_became_active":"2024-04-21T08:44:33.481256+0000","last_became_peered":"2024-04-21T08:44:33.481256+0000","last_unstale":"2024-04-21T08:44:58.453812+0000","last_undegraded":"2024-04-21T08:44:58.453812+0000","last_fullsized":"2024-04-21T08:44:58.453812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:17.804557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667445+0000","last_change":"2024-04-21T08:44:34.490206+0000","last_active":"2024-04-21T08:44:41.667445+0000","last_peered":"2024-04-21T08:44:41.667445+0000","last_clean":"2024-04-21T08:44:41.667445+0000","last_became_active":"2024-04-21T08:44:34.489797+0000","last_became_peered":"2024-04-21T08:44:34.489797+0000","last_unstale":"2024-04-21T08:44:41.667445+0000","last_undegraded":"2024-04-21T08:44:41.667445+0000","last_fullsized":"2024-04-21T08:44:41.667445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:09.055429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.526919+0000","last_change":"2024-04-21T08:44:38.517385+0000","last_active":"2024-04-21T08:44:39.526919+0000","last_peered":"2024-04-21T08:44:39.526919+0000","last_clean":"2024-04-21T08:44:39.526919+0000","last_became_active":"2024-04-21T08:44:38.516964+0000","last_became_peered":"2024-04-21T08:44:38.516964+0000","last_unstale":"2024-04-21T08:44:39.526919+0000","last_undegraded":"2024-04-21T08:44:39.526919+0000","last_fullsized":"2024-04-21T08:44:39.526919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:33.364076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.999998+0000","last_change":"2024-04-21T08:44:33.490586+0000","last_active":"2024-04-21T08:44:59.999998+0000","last_peered":"2024-04-21T08:44:59.999998+0000","last_clean":"2024-04-21T08:44:59.999998+0000","last_became_active":"2024-04-21T08:44:33.489281+0000","last_became_peered":"2024-04-21T08:44:33.489281+0000","last_unstale":"2024-04-21T08:44:59.999998+0000","last_undegraded":"2024-04-21T08:44:59.999998+0000","last_fullsized":"2024-04-21T08:44:59.999998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:41:29.856165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.287994+0000","last_change":"2024-04-21T08:44:34.493394+0000","last_active":"2024-04-21T08:44:40.287994+0000","last_peered":"2024-04-21T08:44:40.287994+0000","last_clean":"2024-04-21T08:44:40.287994+0000","last_became_active":"2024-04-21T08:44:34.492367+0000","last_became_peered":"2024-04-21T08:44:34.492367+0000","last_unstale":"2024-04-21T08:44:40.287994+0000","last_undegraded":"2024-04-21T08:44:40.287994+0000","last_fullsized":"2024-04-21T08:44:40.287994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:13.111266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.664078+0000","last_change":"2024-04-21T08:44:38.519113+0000","last_active":"2024-04-21T08:44:41.664078+0000","last_peered":"2024-04-21T08:44:41.664078+0000","last_clean":"2024-04-21T08:44:41.664078+0000","last_became_active":"2024-04-21T08:44:38.518469+0000","last_became_peered":"2024-04-21T08:44:38.518469+0000","last_unstale":"2024-04-21T08:44:41.664078+0000","last_undegraded":"2024-04-21T08:44:41.664078+0000","last_fullsized":"2024-04-21T08:44:41.664078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:39:22.200041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.448872+0000","last_change":"2024-04-21T08:44:33.481184+0000","last_active":"2024-04-21T08:44:59.448872+0000","last_peered":"2024-04-21T08:44:59.448872+0000","last_clean":"2024-04-21T08:44:59.448872+0000","last_became_active":"2024-04-21T08:44:33.480835+0000","last_became_peered":"2024-04-21T08:44:33.480835+0000","last_unstale":"2024-04-21T08:44:59.448872+0000","last_undegraded":"2024-04-21T08:44:59.448872+0000","last_fullsized":"2024-04-21T08:44:59.448872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:56:49.463576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175552+0000","last_change":"2024-04-21T08:44:34.502226+0000","last_active":"2024-04-21T08:44:41.175552+0000","last_peered":"2024-04-21T08:44:41.175552+0000","last_clean":"2024-04-21T08:44:41.175552+0000","last_became_active":"2024-04-21T08:44:34.501775+0000","last_became_peered":"2024-04-21T08:44:34.501775+0000","last_unstale":"2024-04-21T08:44:41.175552+0000","last_undegraded":"2024-04-21T08:44:41.175552+0000","last_fullsized":"2024-04-21T08:44:41.175552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:58.325267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.663797+0000","last_change":"2024-04-21T08:44:38.522250+0000","last_active":"2024-04-21T08:44:41.663797+0000","last_peered":"2024-04-21T08:44:41.663797+0000","last_clean":"2024-04-21T08:44:41.663797+0000","last_became_active":"2024-04-21T08:44:38.520870+0000","last_became_peered":"2024-04-21T08:44:38.520870+0000","last_unstale":"2024-04-21T08:44:41.663797+0000","last_undegraded":"2024-04-21T08:44:41.663797+0000","last_fullsized":"2024-04-21T08:44:41.663797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:02.391785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:58.453592+0000","last_change":"2024-04-21T08:44:33.486323+0000","last_active":"2024-04-21T08:44:58.453592+0000","last_peered":"2024-04-21T08:44:58.453592+0000","last_clean":"2024-04-21T08:44:58.453592+0000","last_became_active":"2024-04-21T08:44:33.486016+0000","last_became_peered":"2024-04-21T08:44:33.486016+0000","last_unstale":"2024-04-21T08:44:58.453592+0000","last_undegraded":"2024-04-21T08:44:58.453592+0000","last_fullsized":"2024-04-21T08:44:58.453592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:46.645314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532223+0000","last_change":"2024-04-21T08:44:34.496034+0000","last_active":"2024-04-21T08:44:39.532223+0000","last_peered":"2024-04-21T08:44:39.532223+0000","last_clean":"2024-04-21T08:44:39.532223+0000","last_became_active":"2024-04-21T08:44:34.495641+0000","last_became_peered":"2024-04-21T08:44:34.495641+0000","last_unstale":"2024-04-21T08:44:39.532223+0000","last_undegraded":"2024-04-21T08:44:39.532223+0000","last_fullsized":"2024-04-21T08:44:39.532223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:21:18.893326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666869+0000","last_change":"2024-04-21T08:44:38.517288+0000","last_active":"2024-04-21T08:44:41.666869+0000","last_peered":"2024-04-21T08:44:41.666869+0000","last_clean":"2024-04-21T08:44:41.666869+0000","last_became_active":"2024-04-21T08:44:38.516853+0000","last_became_peered":"2024-04-21T08:44:38.516853+0000","last_unstale":"2024-04-21T08:44:41.666869+0000","last_undegraded":"2024-04-21T08:44:41.666869+0000","last_fullsized":"2024-04-21T08:44:41.666869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:27:22.371703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.999908+0000","last_change":"2024-04-21T08:44:33.491578+0000","last_active":"2024-04-21T08:44:59.999908+0000","last_peered":"2024-04-21T08:44:59.999908+0000","last_clean":"2024-04-21T08:44:59.999908+0000","last_became_active":"2024-04-21T08:44:33.491238+0000","last_became_peered":"2024-04-21T08:44:33.491238+0000","last_unstale":"2024-04-21T08:44:59.999908+0000","last_undegraded":"2024-04-21T08:44:59.999908+0000","last_fullsized":"2024-04-21T08:44:59.999908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:50.399031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532567+0000","last_change":"2024-04-21T08:44:34.497632+0000","last_active":"2024-04-21T08:44:39.532567+0000","last_peered":"2024-04-21T08:44:39.532567+0000","last_clean":"2024-04-21T08:44:39.532567+0000","last_became_active":"2024-04-21T08:44:34.497318+0000","last_became_peered":"2024-04-21T08:44:34.497318+0000","last_unstale":"2024-04-21T08:44:39.532567+0000","last_undegraded":"2024-04-21T08:44:39.532567+0000","last_fullsized":"2024-04-21T08:44:39.532567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:11:33.918530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883445+0000","last_change":"2024-04-21T08:44:38.518414+0000","last_active":"2024-04-21T08:44:41.883445+0000","last_peered":"2024-04-21T08:44:41.883445+0000","last_clean":"2024-04-21T08:44:41.883445+0000","last_became_active":"2024-04-21T08:44:38.517953+0000","last_became_peered":"2024-04-21T08:44:38.517953+0000","last_unstale":"2024-04-21T08:44:41.883445+0000","last_undegraded":"2024-04-21T08:44:41.883445+0000","last_fullsized":"2024-04-21T08:44:41.883445+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:48.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:59.989232+0000","last_change":"2024-04-21T08:44:33.493896+0000","last_active":"2024-04-21T08:44:59.989232+0000","last_peered":"2024-04-21T08:44:59.989232+0000","last_clean":"2024-04-21T08:44:59.989232+0000","last_became_active":"2024-04-21T08:44:33.493566+0000","last_became_peered":"2024-04-21T08:44:33.493566+0000","last_unstale":"2024-04-21T08:44:59.989232+0000","last_undegraded":"2024-04-21T08:44:59.989232+0000","last_fullsized":"2024-04-21T08:44:59.989232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:47:17.307690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532112+0000","last_change":"2024-04-21T08:44:34.500406+0000","last_active":"2024-04-21T08:44:39.532112+0000","last_peered":"2024-04-21T08:44:39.532112+0000","last_clean":"2024-04-21T08:44:39.532112+0000","last_became_active":"2024-04-21T08:44:34.500048+0000","last_became_peered":"2024-04-21T08:44:34.500048+0000","last_unstale":"2024-04-21T08:44:39.532112+0000","last_undegraded":"2024-04-21T08:44:39.532112+0000","last_fullsized":"2024-04-21T08:44:39.532112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:10:58.166155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667651+0000","last_change":"2024-04-21T08:44:38.514812+0000","last_active":"2024-04-21T08:44:41.667651+0000","last_peered":"2024-04-21T08:44:41.667651+0000","last_clean":"2024-04-21T08:44:41.667651+0000","last_became_active":"2024-04-21T08:44:38.514247+0000","last_became_peered":"2024-04-21T08:44:38.514247+0000","last_unstale":"2024-04-21T08:44:41.667651+0000","last_undegraded":"2024-04-21T08:44:41.667651+0000","last_fullsized":"2024-04-21T08:44:41.667651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:06:10.588453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:45:01.010669+0000","last_change":"2024-04-21T08:44:33.493887+0000","last_active":"2024-04-21T08:45:01.010669+0000","last_peered":"2024-04-21T08:45:01.010669+0000","last_clean":"2024-04-21T08:45:01.010669+0000","last_became_active":"2024-04-21T08:44:33.493555+0000","last_became_peered":"2024-04-21T08:44:33.493555+0000","last_unstale":"2024-04-21T08:45:01.010669+0000","last_undegraded":"2024-04-21T08:45:01.010669+0000","last_fullsized":"2024-04-21T08:45:01.010669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:18.089102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.532632+0000","last_change":"2024-04-21T08:44:34.503259+0000","last_active":"2024-04-21T08:44:39.532632+0000","last_peered":"2024-04-21T08:44:39.532632+0000","last_clean":"2024-04-21T08:44:39.532632+0000","last_became_active":"2024-04-21T08:44:34.502845+0000","last_became_peered":"2024-04-21T08:44:34.502845+0000","last_unstale":"2024-04-21T08:44:39.532632+0000","last_undegraded":"2024-04-21T08:44:39.532632+0000","last_fullsized":"2024-04-21T08:44:39.532632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:19.902178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288148+0000","last_change":"2024-04-21T08:44:34.497608+0000","last_active":"2024-04-21T08:44:40.288148+0000","last_peered":"2024-04-21T08:44:40.288148+0000","last_clean":"2024-04-21T08:44:40.288148+0000","last_became_active":"2024-04-21T08:44:34.497255+0000","last_became_peered":"2024-04-21T08:44:34.497255+0000","last_unstale":"2024-04-21T08:44:40.288148+0000","last_undegraded":"2024-04-21T08:44:40.288148+0000","last_fullsized":"2024-04-21T08:44:40.288148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:51:15.181837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527317+0000","last_change":"2024-04-21T08:44:34.495408+0000","last_active":"2024-04-21T08:44:39.527317+0000","last_peered":"2024-04-21T08:44:39.527317+0000","last_clean":"2024-04-21T08:44:39.527317+0000","last_became_active":"2024-04-21T08:44:34.495038+0000","last_became_peered":"2024-04-21T08:44:34.495038+0000","last_unstale":"2024-04-21T08:44:39.527317+0000","last_undegraded":"2024-04-21T08:44:39.527317+0000","last_fullsized":"2024-04-21T08:44:39.527317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:59.512245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527170+0000","last_change":"2024-04-21T08:44:34.493292+0000","last_active":"2024-04-21T08:44:39.527170+0000","last_peered":"2024-04-21T08:44:39.527170+0000","last_clean":"2024-04-21T08:44:39.527170+0000","last_became_active":"2024-04-21T08:44:34.492913+0000","last_became_peered":"2024-04-21T08:44:34.492913+0000","last_unstale":"2024-04-21T08:44:39.527170+0000","last_undegraded":"2024-04-21T08:44:39.527170+0000","last_fullsized":"2024-04-21T08:44:39.527170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:24.271859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.883555+0000","last_change":"2024-04-21T08:44:34.510448+0000","last_active":"2024-04-21T08:44:41.883555+0000","last_peered":"2024-04-21T08:44:41.883555+0000","last_clean":"2024-04-21T08:44:41.883555+0000","last_became_active":"2024-04-21T08:44:34.509958+0000","last_became_peered":"2024-04-21T08:44:34.509958+0000","last_unstale":"2024-04-21T08:44:41.883555+0000","last_undegraded":"2024-04-21T08:44:41.883555+0000","last_fullsized":"2024-04-21T08:44:41.883555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:36.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667008+0000","last_change":"2024-04-21T08:44:34.497415+0000","last_active":"2024-04-21T08:44:41.667008+0000","last_peered":"2024-04-21T08:44:41.667008+0000","last_clean":"2024-04-21T08:44:41.667008+0000","last_became_active":"2024-04-21T08:44:34.497120+0000","last_became_peered":"2024-04-21T08:44:34.497120+0000","last_unstale":"2024-04-21T08:44:41.667008+0000","last_undegraded":"2024-04-21T08:44:41.667008+0000","last_fullsized":"2024-04-21T08:44:41.667008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:30.232338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288466+0000","last_change":"2024-04-21T08:44:34.500356+0000","last_active":"2024-04-21T08:44:40.288466+0000","last_peered":"2024-04-21T08:44:40.288466+0000","last_clean":"2024-04-21T08:44:40.288466+0000","last_became_active":"2024-04-21T08:44:34.500032+0000","last_became_peered":"2024-04-21T08:44:34.500032+0000","last_unstale":"2024-04-21T08:44:40.288466+0000","last_undegraded":"2024-04-21T08:44:40.288466+0000","last_fullsized":"2024-04-21T08:44:40.288466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:24.900221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:39.527921+0000","last_change":"2024-04-21T08:44:34.493629+0000","last_active":"2024-04-21T08:44:39.527921+0000","last_peered":"2024-04-21T08:44:39.527921+0000","last_clean":"2024-04-21T08:44:39.527921+0000","last_became_active":"2024-04-21T08:44:34.493273+0000","last_became_peered":"2024-04-21T08:44:34.493273+0000","last_unstale":"2024-04-21T08:44:39.527921+0000","last_undegraded":"2024-04-21T08:44:39.527921+0000","last_fullsized":"2024-04-21T08:44:39.527921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:22:42.698957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.176063+0000","last_change":"2024-04-21T08:44:34.493427+0000","last_active":"2024-04-21T08:44:41.176063+0000","last_peered":"2024-04-21T08:44:41.176063+0000","last_clean":"2024-04-21T08:44:41.176063+0000","last_became_active":"2024-04-21T08:44:34.493105+0000","last_became_peered":"2024-04-21T08:44:34.493105+0000","last_unstale":"2024-04-21T08:44:41.176063+0000","last_undegraded":"2024-04-21T08:44:41.176063+0000","last_fullsized":"2024-04-21T08:44:41.176063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:37.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667898+0000","last_change":"2024-04-21T08:44:34.493930+0000","last_active":"2024-04-21T08:44:41.667898+0000","last_peered":"2024-04-21T08:44:41.667898+0000","last_clean":"2024-04-21T08:44:41.667898+0000","last_became_active":"2024-04-21T08:44:34.491734+0000","last_became_peered":"2024-04-21T08:44:34.491734+0000","last_unstale":"2024-04-21T08:44:41.667898+0000","last_undegraded":"2024-04-21T08:44:41.667898+0000","last_fullsized":"2024-04-21T08:44:41.667898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:23:07.483103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.175373+0000","last_change":"2024-04-21T08:44:34.499420+0000","last_active":"2024-04-21T08:44:41.175373+0000","last_peered":"2024-04-21T08:44:41.175373+0000","last_clean":"2024-04-21T08:44:41.175373+0000","last_became_active":"2024-04-21T08:44:34.499079+0000","last_became_peered":"2024-04-21T08:44:34.499079+0000","last_unstale":"2024-04-21T08:44:41.175373+0000","last_undegraded":"2024-04-21T08:44:41.175373+0000","last_fullsized":"2024-04-21T08:44:41.175373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:45.351422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.667070+0000","last_change":"2024-04-21T08:44:34.493420+0000","last_active":"2024-04-21T08:44:41.667070+0000","last_peered":"2024-04-21T08:44:41.667070+0000","last_clean":"2024-04-21T08:44:41.667070+0000","last_became_active":"2024-04-21T08:44:34.493098+0000","last_became_peered":"2024-04-21T08:44:34.493098+0000","last_unstale":"2024-04-21T08:44:41.667070+0000","last_undegraded":"2024-04-21T08:44:41.667070+0000","last_fullsized":"2024-04-21T08:44:41.667070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:22:34.316822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:41.666858+0000","last_change":"2024-04-21T08:44:34.494760+0000","last_active":"2024-04-21T08:44:41.666858+0000","last_peered":"2024-04-21T08:44:41.666858+0000","last_clean":"2024-04-21T08:44:41.666858+0000","last_became_active":"2024-04-21T08:44:34.494265+0000","last_became_peered":"2024-04-21T08:44:34.494265+0000","last_unstale":"2024-04-21T08:44:41.666858+0000","last_undegraded":"2024-04-21T08:44:41.666858+0000","last_fullsized":"2024-04-21T08:44:41.666858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:16.035030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:44:40.288182+0000","last_change":"2024-04-21T08:44:34.498785+0000","last_active":"2024-04-21T08:44:40.288182+0000","last_peered":"2024-04-21T08:44:40.288182+0000","last_clean":"2024-04-21T08:44:40.288182+0000","last_became_active":"2024-04-21T08:44:34.495159+0000","last_became_peered":"2024-04-21T08:44:34.495159+0000","last_unstale":"2024-04-21T08:44:40.288182+0000","last_undegraded":"2024-04-21T08:44:40.288182+0000","last_fullsized":"2024-04-21T08:44:40.288182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:03:20.095016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9552,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":161,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":44433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383699,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":478407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645336,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1613824,"data_stored":1072381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":477999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070498,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":477953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1581056,"data_stored":1068165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":477937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":489409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":477305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":483029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":477499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":477859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36916,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708236,"statfs":{"total":95995035648,"available":95957233664,"internally_reserved":0,"allocated":1617920,"data_stored":1069821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":5544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:45:03.738 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-21T08:45:03.738 INFO:tasks.cephadm:Setting up client nodes... 2024-04-21T08:45:03.739 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-21T08:45:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:03 smithi106 ceph-mon[31072]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:03 smithi106 ceph-mon[31072]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:03 smithi106 ceph-mon[31072]: pgmap v307: 129 pgs: 129 active+clean; 586 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-04-21T08:45:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:03 smithi106 ceph-mon[31072]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:45:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:03 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2388589415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:03 smithi106 ceph-mon[31072]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T08:45:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:03 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3837380000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:45:04.012 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:04 smithi106 ceph-mon[31072]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:45:04.628 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:45:04 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b[60834]: 2024-04-21T08:45:04.377+0000 7f642eeb7700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-21T08:45:04.628 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:45:04 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b[60834]: 2024-04-21T08:45:04.377+0000 7f642eeb7700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-21T08:45:04.628 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:45:04 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b[60834]: 2024-04-21T08:45:04.377+0000 7f642eeb7700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-21T08:45:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:04 smithi134 ceph-mon[31772]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:45:04.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:04 smithi082 ceph-mon[27967]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:45:05.533 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:05 smithi082 ceph-mon[27967]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:45:05.533 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:05 smithi082 ceph-mon[27967]: pgmap v308: 129 pgs: 129 active+clean; 587 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-04-21T08:45:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:05 smithi134 ceph-mon[31772]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:45:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:05 smithi134 ceph-mon[31772]: pgmap v308: 129 pgs: 129 active+clean; 587 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-04-21T08:45:05.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:05 smithi106 ceph-mon[31072]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:45:05.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:05 smithi106 ceph-mon[31072]: pgmap v308: 129 pgs: 129 active+clean; 587 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-04-21T08:45:05.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.931+0000 7f2f5498d700 1 -- 172.21.15.82:0/2055535175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f50102af0 msgr2=0x7f2f50102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:05.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.931+0000 7f2f5498d700 1 --2- 172.21.15.82:0/2055535175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f50102af0 0x7f2f50102f10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2f400099c0 tx=0x7f2f40009cd0 comp rx=0 tx=0).stop 2024-04-21T08:45:05.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.932+0000 7f2f5498d700 1 -- 172.21.15.82:0/2055535175 shutdown_connections 2024-04-21T08:45:05.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.932+0000 7f2f5498d700 1 --2- 172.21.15.82:0/2055535175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f50104690 0x7f2f5010af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:05.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.932+0000 7f2f5498d700 1 --2- 172.21.15.82:0/2055535175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f50102af0 0x7f2f50102f10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:05.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.932+0000 7f2f5498d700 1 --2- 172.21.15.82:0/2055535175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f50103cf0 0x7f2f50104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:05.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.932+0000 7f2f5498d700 1 -- 172.21.15.82:0/2055535175 >> 172.21.15.82:0/2055535175 conn(0x7f2f500fe090 msgr2=0x7f2f501004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:05.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.932+0000 7f2f5498d700 1 -- 172.21.15.82:0/2055535175 shutdown_connections 2024-04-21T08:45:05.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.932+0000 7f2f5498d700 1 -- 172.21.15.82:0/2055535175 wait complete. 2024-04-21T08:45:05.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.932+0000 7f2f5498d700 1 Processor -- start 2024-04-21T08:45:05.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.932+0000 7f2f5498d700 1 -- start start 2024-04-21T08:45:05.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f5498d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f50102af0 0x7f2f50198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:05.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f5498d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f50103cf0 0x7f2f50199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f5498d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f50104690 0x7f2f501a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f5498d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5010e560 con 0x7f2f50102af0 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f5498d700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5010e3e0 con 0x7f2f50104690 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f5498d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5010e6e0 con 0x7f2f50103cf0 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f50102af0 0x7f2f50198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f50102af0 0x7f2f50198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58406/0 (socket says 172.21.15.82:58406) 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4e59c700 1 -- 172.21.15.82:0/1932438410 learned_addr learned my addr 172.21.15.82:0/1932438410 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4ed9d700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f50104690 0x7f2f501a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4dd9b700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f50103cf0 0x7f2f50199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4e59c700 1 -- 172.21.15.82:0/1932438410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f50103cf0 msgr2=0x7f2f50199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4e59c700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f50103cf0 0x7f2f50199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4e59c700 1 -- 172.21.15.82:0/1932438410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f50104690 msgr2=0x7f2f501a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4e59c700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f50104690 0x7f2f501a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4e59c700 1 -- 172.21.15.82:0/1932438410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f501a07a0 con 0x7f2f50102af0 2024-04-21T08:45:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4dd9b700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f50103cf0 0x7f2f50199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4ed9d700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f50104690 0x7f2f501a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f4e59c700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f50102af0 0x7f2f50198b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009f50 tx=0x7f2f400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.933+0000 7f2f3f7fe700 1 -- 172.21.15.82:0/1932438410 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40027070 con 0x7f2f50102af0 2024-04-21T08:45:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.934+0000 7f2f5498d700 1 -- 172.21.15.82:0/1932438410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f501a0a30 con 0x7f2f50102af0 2024-04-21T08:45:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.934+0000 7f2f3f7fe700 1 -- 172.21.15.82:0/1932438410 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f400041a0 con 0x7f2f50102af0 2024-04-21T08:45:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.934+0000 7f2f3f7fe700 1 -- 172.21.15.82:0/1932438410 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000f460 con 0x7f2f50102af0 2024-04-21T08:45:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.934+0000 7f2f5498d700 1 -- 172.21.15.82:0/1932438410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f501a0f60 con 0x7f2f50102af0 2024-04-21T08:45:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.935+0000 7f2f5498d700 1 -- 172.21.15.82:0/1932438410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f50050f80 con 0x7f2f50102af0 2024-04-21T08:45:05.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.936+0000 7f2f3f7fe700 1 -- 172.21.15.82:0/1932438410 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2f4002a070 con 0x7f2f50102af0 2024-04-21T08:45:05.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.936+0000 7f2f3f7fe700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2f380a3d80 0x7f2f380a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:05.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.936+0000 7f2f4dd9b700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2f380a3d80 0x7f2f380a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:05.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.936+0000 7f2f3f7fe700 1 -- 172.21.15.82:0/1932438410 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2f400d1720 con 0x7f2f50102af0 2024-04-21T08:45:05.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.937+0000 7f2f4dd9b700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2f380a3d80 0x7f2f380a6240 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2f44005fd0 tx=0x7f2f44005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:05.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:05.941+0000 7f2f3f7fe700 1 -- 172.21.15.82:0/1932438410 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f4005eff0 con 0x7f2f50102af0 2024-04-21T08:45:06.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.231+0000 7f2f5498d700 1 -- 172.21.15.82:0/1932438410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2f5002dfb0 con 0x7f2f50102af0 2024-04-21T08:45:06.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.234+0000 7f2f3f7fe700 1 -- 172.21.15.82:0/1932438410 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f2f40013070 con 0x7f2f50102af0 2024-04-21T08:45:06.233 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-04-21T08:45:06.233 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAS0iRmuprnDRAAEMMExblibKpI7zjCLKcYPw== 2024-04-21T08:45:06.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.236+0000 7f2f5498d700 1 -- 172.21.15.82:0/1932438410 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2f380a3d80 msgr2=0x7f2f380a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:06.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.236+0000 7f2f5498d700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2f380a3d80 0x7f2f380a6240 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2f44005fd0 tx=0x7f2f44005ee0 comp rx=0 tx=0).stop 2024-04-21T08:45:06.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.236+0000 7f2f5498d700 1 -- 172.21.15.82:0/1932438410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f50102af0 msgr2=0x7f2f50198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:06.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.236+0000 7f2f5498d700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f50102af0 0x7f2f50198b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009f50 tx=0x7f2f400047b0 comp rx=0 tx=0).stop 2024-04-21T08:45:06.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.236+0000 7f2f5498d700 1 -- 172.21.15.82:0/1932438410 shutdown_connections 2024-04-21T08:45:06.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.236+0000 7f2f5498d700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2f380a3d80 0x7f2f380a6240 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:06.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.236+0000 7f2f5498d700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f50104690 0x7f2f501a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:06.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.236+0000 7f2f5498d700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f50102af0 0x7f2f50198b40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:06.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.236+0000 7f2f5498d700 1 --2- 172.21.15.82:0/1932438410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f50103cf0 0x7f2f50199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:06.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.236+0000 7f2f5498d700 1 -- 172.21.15.82:0/1932438410 >> 172.21.15.82:0/1932438410 conn(0x7f2f500fe090 msgr2=0x7f2f50106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:06.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.236+0000 7f2f5498d700 1 -- 172.21.15.82:0/1932438410 shutdown_connections 2024-04-21T08:45:06.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:06.236+0000 7f2f5498d700 1 -- 172.21.15.82:0/1932438410 wait complete. 2024-04-21T08:45:06.527 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:06 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1932438410' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T08:45:06.527 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:06 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1932438410' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T08:45:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:06 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1932438410' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T08:45:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:06 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1932438410' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T08:45:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:06 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1932438410' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T08:45:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:06 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1932438410' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T08:45:07.005 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:45:07.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-21T08:45:07.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-21T08:45:07.042 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-21T08:45:07.043 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-21T08:45:07.043 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-21T08:45:07.333 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:07.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:07 smithi134 ceph-mon[31772]: pgmap v309: 129 pgs: 129 active+clean; 628 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.2 KiB/s wr, 21 op/s 2024-04-21T08:45:07.739 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:07 smithi082 ceph-mon[27967]: pgmap v309: 129 pgs: 129 active+clean; 628 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.2 KiB/s wr, 21 op/s 2024-04-21T08:45:07.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:07 smithi106 ceph-mon[31072]: pgmap v309: 129 pgs: 129 active+clean; 628 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.2 KiB/s wr, 21 op/s 2024-04-21T08:45:09.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.267+0000 7f63a8317700 1 -- 172.21.15.82:0/349133820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a0101340 msgr2=0x7f63a0101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:09.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.267+0000 7f63a8317700 1 --2- 172.21.15.82:0/349133820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a0101340 0x7f63a0101720 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f639000aad0 tx=0x7f639000ade0 comp rx=0 tx=0).stop 2024-04-21T08:45:09.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.267+0000 7f63a8317700 1 -- 172.21.15.82:0/349133820 shutdown_connections 2024-04-21T08:45:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.267+0000 7f63a8317700 1 --2- 172.21.15.82:0/349133820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63a0106420 0x7f63a010dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.267+0000 7f63a8317700 1 --2- 172.21.15.82:0/349133820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a0101340 0x7f63a0101720 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.267+0000 7f63a8317700 1 --2- 172.21.15.82:0/349133820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63a0101cf0 0x7f63a0105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.267+0000 7f63a8317700 1 -- 172.21.15.82:0/349133820 >> 172.21.15.82:0/349133820 conn(0x7f63a0077f40 msgr2=0x7f63a00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.267+0000 7f63a8317700 1 -- 172.21.15.82:0/349133820 shutdown_connections 2024-04-21T08:45:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.267+0000 7f63a8317700 1 -- 172.21.15.82:0/349133820 wait complete. 2024-04-21T08:45:09.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.268+0000 7f63a8317700 1 Processor -- start 2024-04-21T08:45:09.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.268+0000 7f63a8317700 1 -- start start 2024-04-21T08:45:09.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.268+0000 7f63a8317700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63a0101340 0x7f63a0197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:09.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.268+0000 7f63a8317700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63a0101cf0 0x7f63a0197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:09.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.268+0000 7f63a8317700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a0106420 0x7f63a019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:09.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a8317700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f63a0110b20 con 0x7f63a0106420 2024-04-21T08:45:09.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a8317700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01109a0 con 0x7f63a0101340 2024-04-21T08:45:09.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a8317700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63a0110ca0 con 0x7f63a0101cf0 2024-04-21T08:45:09.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a68b4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a0106420 0x7f63a019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:09.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a68b4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a0106420 0x7f63a019b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58432/0 (socket says 172.21.15.82:58432) 2024-04-21T08:45:09.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a68b4700 1 -- 172.21.15.82:0/35757336 learned_addr learned my addr 172.21.15.82:0/35757336 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:09.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a58b2700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63a0101cf0 0x7f63a0197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a60b3700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63a0101340 0x7f63a0197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a68b4700 1 -- 172.21.15.82:0/35757336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63a0101cf0 msgr2=0x7f63a0197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a68b4700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63a0101cf0 0x7f63a0197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a68b4700 1 -- 172.21.15.82:0/35757336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63a0101340 msgr2=0x7f63a0197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a68b4700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63a0101340 0x7f63a0197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a68b4700 1 -- 172.21.15.82:0/35757336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a019c040 con 0x7f63a0106420 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a68b4700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a0106420 0x7f63a019b900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f639800b9e0 tx=0x7f639800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63977fe700 1 -- 172.21.15.82:0/35757336 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398019070 con 0x7f63a0106420 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63977fe700 1 -- 172.21.15.82:0/35757336 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f639800ce00 con 0x7f63a0106420 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a8317700 1 -- 172.21.15.82:0/35757336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a019c330 con 0x7f63a0106420 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63977fe700 1 -- 172.21.15.82:0/35757336 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398007370 con 0x7f63a0106420 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.269+0000 7f63a8317700 1 -- 172.21.15.82:0/35757336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a01a3bb0 con 0x7f63a0106420 2024-04-21T08:45:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.270+0000 7f63a8317700 1 -- 172.21.15.82:0/35757336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a0069730 con 0x7f63a0106420 2024-04-21T08:45:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.272+0000 7f63977fe700 1 -- 172.21.15.82:0/35757336 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f639801c070 con 0x7f63a0106420 2024-04-21T08:45:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.273+0000 7f63977fe700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f638c0a3d50 0x7f638c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.273+0000 7f63a60b3700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f638c0a3d50 0x7f638c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.273+0000 7f63977fe700 1 -- 172.21.15.82:0/35757336 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f63980d05b0 con 0x7f63a0106420 2024-04-21T08:45:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.273+0000 7f63a60b3700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f638c0a3d50 0x7f638c0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6390005f50 tx=0x7f6390005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:09.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.277+0000 7f63977fe700 1 -- 172.21.15.82:0/35757336 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6398023030 con 0x7f63a0106420 2024-04-21T08:45:09.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.531+0000 7f63a8317700 1 -- 172.21.15.82:0/35757336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f63a0198390 con 0x7f63a0106420 2024-04-21T08:45:09.532 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:09 smithi082 ceph-mon[27967]: pgmap v310: 129 pgs: 129 active+clean; 628 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-04-21T08:45:09.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.536+0000 7f63977fe700 1 -- 172.21.15.82:0/35757336 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7f639809cd00 con 0x7f63a0106420 2024-04-21T08:45:09.540 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:45:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.539+0000 7f63a8317700 1 -- 172.21.15.82:0/35757336 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f638c0a3d50 msgr2=0x7f638c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.539+0000 7f63a8317700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f638c0a3d50 0x7f638c0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6390005f50 tx=0x7f6390005e80 comp rx=0 tx=0).stop 2024-04-21T08:45:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.539+0000 7f63a8317700 1 -- 172.21.15.82:0/35757336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a0106420 msgr2=0x7f63a019b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.539+0000 7f63a8317700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a0106420 0x7f63a019b900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f639800b9e0 tx=0x7f639800bda0 comp rx=0 tx=0).stop 2024-04-21T08:45:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.539+0000 7f63a8317700 1 -- 172.21.15.82:0/35757336 shutdown_connections 2024-04-21T08:45:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.539+0000 7f63a8317700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f638c0a3d50 0x7f638c0a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.539+0000 7f63a8317700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a0106420 0x7f63a019b900 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.539+0000 7f63a8317700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63a0101340 0x7f63a0197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.539+0000 7f63a8317700 1 --2- 172.21.15.82:0/35757336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63a0101cf0 0x7f63a0197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:09.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.539+0000 7f63a8317700 1 -- 172.21.15.82:0/35757336 >> 172.21.15.82:0/35757336 conn(0x7f63a0077f40 msgr2=0x7f63a00fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:09.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.539+0000 7f63a8317700 1 -- 172.21.15.82:0/35757336 shutdown_connections 2024-04-21T08:45:09.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:09.539+0000 7f63a8317700 1 -- 172.21.15.82:0/35757336 wait complete. 2024-04-21T08:45:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:09 smithi134 ceph-mon[31772]: pgmap v310: 129 pgs: 129 active+clean; 628 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-04-21T08:45:09.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:09 smithi106 ceph-mon[31072]: pgmap v310: 129 pgs: 129 active+clean; 628 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-04-21T08:45:10.412 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2138749385},{"type":"v1","addr":"172.21.15.82:6801","nonce":2138749385}]},"active_addr":"172.21.15.82:6801/2138749385","active_change":"2024-04-21T08:37:26.227881+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.82:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2119539425}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2883183315}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2275507980}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":539366680}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2330723242}]}]} 2024-04-21T08:45:10.416 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-21T08:45:10.417 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-21T08:45:10.417 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T08:45:10.663 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:10 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/35757336' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T08:45:10.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:10 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/35757336' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T08:45:10.785 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:10 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/35757336' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T08:45:11.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:11 smithi134 ceph-mon[31772]: pgmap v311: 129 pgs: 129 active+clean; 628 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-04-21T08:45:11.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:11 smithi082 ceph-mon[27967]: pgmap v311: 129 pgs: 129 active+clean; 628 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-04-21T08:45:11.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:11 smithi106 ceph-mon[31072]: pgmap v311: 129 pgs: 129 active+clean; 628 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-04-21T08:45:12.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.534+0000 7fbc0760f700 1 -- 172.21.15.82:0/2757570997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc00102a90 msgr2=0x7fbc00102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:12.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.534+0000 7fbc0760f700 1 --2- 172.21.15.82:0/2757570997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc00102a90 0x7fbc00102f10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009990 tx=0x7fbbf0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:45:12.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.534+0000 7fbc0760f700 1 -- 172.21.15.82:0/2757570997 shutdown_connections 2024-04-21T08:45:12.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.534+0000 7fbc0760f700 1 --2- 172.21.15.82:0/2757570997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc00103450 0x7fbc0010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:12.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.534+0000 7fbc0760f700 1 --2- 172.21.15.82:0/2757570997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc00108a50 0x7fbc00108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:12.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc0760f700 1 --2- 172.21.15.82:0/2757570997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc00102a90 0x7fbc00102f10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:12.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc0760f700 1 -- 172.21.15.82:0/2757570997 >> 172.21.15.82:0/2757570997 conn(0x7fbc00079ba0 msgr2=0x7fbc00079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:12.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc0760f700 1 -- 172.21.15.82:0/2757570997 shutdown_connections 2024-04-21T08:45:12.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc0760f700 1 -- 172.21.15.82:0/2757570997 wait complete. 2024-04-21T08:45:12.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc0760f700 1 Processor -- start 2024-04-21T08:45:12.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc0760f700 1 -- start start 2024-04-21T08:45:12.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc0760f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc00102a90 0x7fbc001992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc0760f700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc00103450 0x7fbc00199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc0760f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc00108a50 0x7fbc0019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc0760f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00112ce0 con 0x7fbc00108a50 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc0760f700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00112b60 con 0x7fbc00103450 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc0760f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00112e60 con 0x7fbc00102a90 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.535+0000 7fbc05bac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc00108a50 0x7fbc0019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbc05bac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc00108a50 0x7fbc0019dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58438/0 (socket says 172.21.15.82:58438) 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbc05bac700 1 -- 172.21.15.82:0/2951988476 learned_addr learned my addr 172.21.15.82:0/2951988476 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbc04baa700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc00103450 0x7fbc00199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbc053ab700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc00102a90 0x7fbc001992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbc05bac700 1 -- 172.21.15.82:0/2951988476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc00102a90 msgr2=0x7fbc001992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbc05bac700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc00102a90 0x7fbc001992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:12.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbc05bac700 1 -- 172.21.15.82:0/2951988476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc00103450 msgr2=0x7fbc00199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbc05bac700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc00103450 0x7fbc00199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbc05bac700 1 -- 172.21.15.82:0/2951988476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0019e2f0 con 0x7fbc00108a50 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbc05bac700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc00108a50 0x7fbc0019dbb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800b970 tx=0x7fbbf800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbbf67fc700 1 -- 172.21.15.82:0/2951988476 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800d860 con 0x7fbc00108a50 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbbf67fc700 1 -- 172.21.15.82:0/2951988476 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbf800e430 con 0x7fbc00108a50 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbc0760f700 1 -- 172.21.15.82:0/2951988476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0019e5e0 con 0x7fbc00108a50 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbbf67fc700 1 -- 172.21.15.82:0/2951988476 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8017660 con 0x7fbc00108a50 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.536+0000 7fbc0760f700 1 -- 172.21.15.82:0/2951988476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc001a5e60 con 0x7fbc00108a50 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.537+0000 7fbc0760f700 1 -- 172.21.15.82:0/2951988476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc00069730 con 0x7fbc00108a50 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.538+0000 7fbbf67fc700 1 -- 172.21.15.82:0/2951988476 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbbf800da00 con 0x7fbc00108a50 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.539+0000 7fbbf67fc700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbbec0a3cd0 0x7fbbec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.539+0000 7fbc053ab700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbbec0a3cd0 0x7fbbec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.539+0000 7fbbf67fc700 1 -- 172.21.15.82:0/2951988476 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fbbf80d0770 con 0x7fbc00108a50 2024-04-21T08:45:12.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.539+0000 7fbc053ab700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbbec0a3cd0 0x7fbbec0a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009a10 tx=0x7fbbfc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:12.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.543+0000 7fbbf67fc700 1 -- 172.21.15.82:0/2951988476 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbbf8013030 con 0x7fbc00108a50 2024-04-21T08:45:12.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:12 smithi082 ceph-mon[27967]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2024-04-21T08:45:12.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:12 smithi134 ceph-mon[31772]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2024-04-21T08:45:12.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.742+0000 7fbc0760f700 1 -- 172.21.15.82:0/2951988476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc0019a640 con 0x7fbc00108a50 2024-04-21T08:45:12.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.743+0000 7fbbf67fc700 1 -- 172.21.15.82:0/2951988476 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24140 (secure 0 0 0) 0x7fbbf809cdb0 con 0x7fbc00108a50 2024-04-21T08:45:12.744 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:45:12.744 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":89,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:44:38.858333+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:45:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.745+0000 7fbc0760f700 1 -- 172.21.15.82:0/2951988476 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbbec0a3cd0 msgr2=0x7fbbec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.745+0000 7fbc0760f700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbbec0a3cd0 0x7fbbec0a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009a10 tx=0x7fbbfc009190 comp rx=0 tx=0).stop 2024-04-21T08:45:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.745+0000 7fbc0760f700 1 -- 172.21.15.82:0/2951988476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc00108a50 msgr2=0x7fbc0019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.745+0000 7fbc0760f700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc00108a50 0x7fbc0019dbb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800b970 tx=0x7fbbf800bc80 comp rx=0 tx=0).stop 2024-04-21T08:45:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.745+0000 7fbc0760f700 1 -- 172.21.15.82:0/2951988476 shutdown_connections 2024-04-21T08:45:12.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.745+0000 7fbc0760f700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbbec0a3cd0 0x7fbbec0a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:12.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.745+0000 7fbc0760f700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc00108a50 0x7fbc0019dbb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:12.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.745+0000 7fbc0760f700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc00102a90 0x7fbc001992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:12.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.745+0000 7fbc0760f700 1 --2- 172.21.15.82:0/2951988476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc00103450 0x7fbc00199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:12.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.745+0000 7fbc0760f700 1 -- 172.21.15.82:0/2951988476 >> 172.21.15.82:0/2951988476 conn(0x7fbc00079ba0 msgr2=0x7fbc001019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:12.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.745+0000 7fbc0760f700 1 -- 172.21.15.82:0/2951988476 shutdown_connections 2024-04-21T08:45:12.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:12.746+0000 7fbc0760f700 1 -- 172.21.15.82:0/2951988476 wait complete. 2024-04-21T08:45:12.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:12 smithi106 ceph-mon[31072]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2024-04-21T08:45:13.550 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-21T08:45:13.551 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T08:45:13.789 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:13.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:13 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2951988476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:45:13.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:13 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2951988476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:45:13.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:13 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2951988476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:45:14.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:14 smithi082 ceph-mon[27967]: pgmap v313: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.0 KiB/s wr, 15 op/s 2024-04-21T08:45:14.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:14 smithi106 ceph-mon[31072]: pgmap v313: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.0 KiB/s wr, 15 op/s 2024-04-21T08:45:14.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:14 smithi134 ceph-mon[31772]: pgmap v313: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.0 KiB/s wr, 15 op/s 2024-04-21T08:45:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.619+0000 7f9ded6c8700 1 -- 172.21.15.82:0/3312974836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9de8101970 msgr2=0x7f9de8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.619+0000 7f9ded6c8700 1 --2- 172.21.15.82:0/3312974836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9de8101970 0x7f9de8101df0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009990 tx=0x7f9dd8009ca0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.619+0000 7f9ded6c8700 1 -- 172.21.15.82:0/3312974836 shutdown_connections 2024-04-21T08:45:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.619+0000 7f9ded6c8700 1 --2- 172.21.15.82:0/3312974836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8102330 0x7f9de810e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.619+0000 7f9ded6c8700 1 --2- 172.21.15.82:0/3312974836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9de8103760 0x7f9de8103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.619+0000 7f9ded6c8700 1 --2- 172.21.15.82:0/3312974836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9de8101970 0x7f9de8101df0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.619+0000 7f9ded6c8700 1 -- 172.21.15.82:0/3312974836 >> 172.21.15.82:0/3312974836 conn(0x7f9de80779b0 msgr2=0x7f9de8077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.619+0000 7f9ded6c8700 1 -- 172.21.15.82:0/3312974836 shutdown_connections 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.619+0000 7f9ded6c8700 1 -- 172.21.15.82:0/3312974836 wait complete. 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.620+0000 7f9ded6c8700 1 Processor -- start 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.620+0000 7f9ded6c8700 1 -- start start 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.620+0000 7f9ded6c8700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9de8101970 0x7f9de8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.620+0000 7f9ded6c8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9de8102330 0x7f9de81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.620+0000 7f9ded6c8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8103760 0x7f9de819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.620+0000 7f9ded6c8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8111bd0 con 0x7f9de8102330 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.620+0000 7f9ded6c8700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8111a50 con 0x7f9de8101970 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.620+0000 7f9ded6c8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8111d50 con 0x7f9de8103760 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.620+0000 7f9de67fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9de8102330 0x7f9de81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.620+0000 7f9de67fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9de8102330 0x7f9de81985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35470/0 (socket says 172.21.15.82:35470) 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.620+0000 7f9de67fc700 1 -- 172.21.15.82:0/2148011571 learned_addr learned my addr 172.21.15.82:0/2148011571 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.620+0000 7f9de6ffd700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9de8101970 0x7f9de8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.621+0000 7f9de77fe700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8103760 0x7f9de819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.621+0000 7f9de67fc700 1 -- 172.21.15.82:0/2148011571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8103760 msgr2=0x7f9de819c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.621+0000 7f9de67fc700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8103760 0x7f9de819c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.621+0000 7f9de67fc700 1 -- 172.21.15.82:0/2148011571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9de8101970 msgr2=0x7f9de8198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.621+0000 7f9de67fc700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9de8101970 0x7f9de8198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.621+0000 7f9de67fc700 1 -- 172.21.15.82:0/2148011571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de819d090 con 0x7f9de8102330 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.621+0000 7f9de67fc700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9de8102330 0x7f9de81985c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80057c0 tx=0x7f9dd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.621+0000 7f9dcffff700 1 -- 172.21.15.82:0/2148011571 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8027070 con 0x7f9de8102330 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.621+0000 7f9dcffff700 1 -- 172.21.15.82:0/2148011571 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd8003f20 con 0x7f9de8102330 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.621+0000 7f9ded6c8700 1 -- 172.21.15.82:0/2148011571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de819d320 con 0x7f9de8102330 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.621+0000 7f9dcffff700 1 -- 172.21.15.82:0/2148011571 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8022c30 con 0x7f9de8102330 2024-04-21T08:45:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.621+0000 7f9ded6c8700 1 -- 172.21.15.82:0/2148011571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de806b1f0 con 0x7f9de8102330 2024-04-21T08:45:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.622+0000 7f9ded6c8700 1 -- 172.21.15.82:0/2148011571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de8103060 con 0x7f9de8102330 2024-04-21T08:45:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.623+0000 7f9dcffff700 1 -- 172.21.15.82:0/2148011571 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9dd802a070 con 0x7f9de8102330 2024-04-21T08:45:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.624+0000 7f9dcffff700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9dd00a3cd0 0x7f9dd00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.624+0000 7f9de6ffd700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9dd00a3cd0 0x7f9dd00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.624+0000 7f9dcffff700 1 -- 172.21.15.82:0/2148011571 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9dd80d1ed0 con 0x7f9de8102330 2024-04-21T08:45:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.625+0000 7f9de6ffd700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9dd00a3cd0 0x7f9dd00a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc005950 tx=0x7f9ddc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:15.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.628+0000 7f9dcffff700 1 -- 172.21.15.82:0/2148011571 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9dd8031020 con 0x7f9de8102330 2024-04-21T08:45:15.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.827+0000 7f9ded6c8700 1 -- 172.21.15.82:0/2148011571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9de81993e0 con 0x7f9de8102330 2024-04-21T08:45:15.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.828+0000 7f9dcffff700 1 -- 172.21.15.82:0/2148011571 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24140 (secure 0 0 0) 0x7f9dd8013070 con 0x7f9de8102330 2024-04-21T08:45:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:45:15.829 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":89,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:44:38.858333+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:45:15.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.830+0000 7f9ded6c8700 1 -- 172.21.15.82:0/2148011571 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9dd00a3cd0 msgr2=0x7f9dd00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:15.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.830+0000 7f9ded6c8700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9dd00a3cd0 0x7f9dd00a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc005950 tx=0x7f9ddc00b410 comp rx=0 tx=0).stop 2024-04-21T08:45:15.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.830+0000 7f9ded6c8700 1 -- 172.21.15.82:0/2148011571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9de8102330 msgr2=0x7f9de81985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:15.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.830+0000 7f9ded6c8700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9de8102330 0x7f9de81985c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80057c0 tx=0x7f9dd80057f0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.830+0000 7f9ded6c8700 1 -- 172.21.15.82:0/2148011571 shutdown_connections 2024-04-21T08:45:15.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.830+0000 7f9ded6c8700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9dd00a3cd0 0x7f9dd00a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.830+0000 7f9ded6c8700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9de8103760 0x7f9de819c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.830+0000 7f9ded6c8700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9de8101970 0x7f9de8198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.830+0000 7f9ded6c8700 1 --2- 172.21.15.82:0/2148011571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9de8102330 0x7f9de81985c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.830+0000 7f9ded6c8700 1 -- 172.21.15.82:0/2148011571 >> 172.21.15.82:0/2148011571 conn(0x7f9de80779b0 msgr2=0x7f9de8106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:15.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.830+0000 7f9ded6c8700 1 -- 172.21.15.82:0/2148011571 shutdown_connections 2024-04-21T08:45:15.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:15.830+0000 7f9ded6c8700 1 -- 172.21.15.82:0/2148011571 wait complete. 2024-04-21T08:45:16.205 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:15 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2148011571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:45:16.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:15 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2148011571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:45:16.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:15 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2148011571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:45:16.647 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-21T08:45:16.647 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-21T08:45:16.648 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-21T08:45:16.649 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-21T08:45:16.649 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-21T08:45:16.650 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-21T08:45:16.650 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-21T08:45:16.651 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-21T08:45:16.652 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-21T08:45:16.652 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-21T08:45:16.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-21T08:45:16.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-21T08:45:17.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:16 smithi134 ceph-mon[31772]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 KiB/s wr, 17 op/s 2024-04-21T08:45:17.349 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:17.349 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:17.352 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:17.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:16 smithi082 ceph-mon[27967]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 KiB/s wr, 17 op/s 2024-04-21T08:45:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:17.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:16 smithi106 ceph-mon[31072]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 KiB/s wr, 17 op/s 2024-04-21T08:45:17.399 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:18.350 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:18.350 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:18.350 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:18.351 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:19.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:19 smithi106 ceph-mon[31072]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-21T08:45:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:19 smithi134 ceph-mon[31772]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-21T08:45:19.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:19 smithi082 ceph-mon[27967]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:21.153 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:21.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:21 smithi082 ceph-mon[27967]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T08:45:21.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:21 smithi134 ceph-mon[31772]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T08:45:21.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:21 smithi106 ceph-mon[31072]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T08:45:23.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:23 smithi082 ceph-mon[27967]: pgmap v317: 129 pgs: 129 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-21T08:45:23.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:23 smithi134 ceph-mon[31772]: pgmap v317: 129 pgs: 129 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-21T08:45:23.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:23 smithi106 ceph-mon[31072]: pgmap v317: 129 pgs: 129 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-21T08:45:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.288+0000 7fd4fed37700 1 -- 172.21.15.82:0/944935056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f806bb30 msgr2=0x7fd4f810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.288+0000 7fd4fed37700 1 --2- 172.21.15.82:0/944935056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f806bb30 0x7fd4f810f900 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000b0a0 tx=0x7fd4f000b3b0 comp rx=0 tx=0).stop 2024-04-21T08:45:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.288+0000 7fd4fed37700 1 -- 172.21.15.82:0/944935056 shutdown_connections 2024-04-21T08:45:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.288+0000 7fd4fed37700 1 --2- 172.21.15.82:0/944935056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f806bb30 0x7fd4f810f900 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:24.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.288+0000 7fd4fed37700 1 --2- 172.21.15.82:0/944935056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f8108a30 0x7fd4f8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.288+0000 7fd4fed37700 1 --2- 172.21.15.82:0/944935056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4f806b190 0x7fd4f806b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.288+0000 7fd4fed37700 1 -- 172.21.15.82:0/944935056 >> 172.21.15.82:0/944935056 conn(0x7fd4f8078910 msgr2=0x7fd4f8078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.288+0000 7fd4fed37700 1 -- 172.21.15.82:0/944935056 shutdown_connections 2024-04-21T08:45:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.288+0000 7fd4fed37700 1 -- 172.21.15.82:0/944935056 wait complete. 2024-04-21T08:45:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.288+0000 7fd4fed37700 1 Processor -- start 2024-04-21T08:45:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.288+0000 7fd4fed37700 1 -- start start 2024-04-21T08:45:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fed37700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4f806b190 0x7fd4f819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fed37700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f806bb30 0x7fd4f819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fed37700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8108a30 0x7fd4f819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fed37700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8112bd0 con 0x7fd4f8108a30 2024-04-21T08:45:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fed37700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8112a50 con 0x7fd4f806b190 2024-04-21T08:45:24.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fed37700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8112d50 con 0x7fd4f806bb30 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fd2d4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8108a30 0x7fd4f819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fd2d4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8108a30 0x7fd4f819fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59518/0 (socket says 172.21.15.82:59518) 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fd2d4700 1 -- 172.21.15.82:0/376766987 learned_addr learned my addr 172.21.15.82:0/376766987 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4f7fff700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f806bb30 0x7fd4f819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fcad3700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4f806b190 0x7fd4f819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fd2d4700 1 -- 172.21.15.82:0/376766987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f806bb30 msgr2=0x7fd4f819b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fd2d4700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4f806bb30 0x7fd4f819b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fd2d4700 1 -- 172.21.15.82:0/376766987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4f806b190 msgr2=0x7fd4f819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fd2d4700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4f806b190 0x7fd4f819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fd2d4700 1 -- 172.21.15.82:0/376766987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f81a0260 con 0x7fd4f8108a30 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4fd2d4700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8108a30 0x7fd4f819fb20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00062a0 tx=0x7fd4f0004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.289+0000 7fd4f5ffb700 1 -- 172.21.15.82:0/376766987 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0021070 con 0x7fd4f8108a30 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.290+0000 7fd4f5ffb700 1 -- 172.21.15.82:0/376766987 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4f0003c00 con 0x7fd4f8108a30 2024-04-21T08:45:24.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.290+0000 7fd4fed37700 1 -- 172.21.15.82:0/376766987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f81a04f0 con 0x7fd4f8108a30 2024-04-21T08:45:24.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.290+0000 7fd4f5ffb700 1 -- 172.21.15.82:0/376766987 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0004880 con 0x7fd4f8108a30 2024-04-21T08:45:24.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.290+0000 7fd4fed37700 1 -- 172.21.15.82:0/376766987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f81a7dd0 con 0x7fd4f8108a30 2024-04-21T08:45:24.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.290+0000 7fd4fed37700 1 -- 172.21.15.82:0/376766987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd4f8063190 con 0x7fd4f8108a30 2024-04-21T08:45:24.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.292+0000 7fd4f5ffb700 1 -- 172.21.15.82:0/376766987 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd4f0007a00 con 0x7fd4f8108a30 2024-04-21T08:45:24.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.292+0000 7fd4f5ffb700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd4e80a3cd0 0x7fd4e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:24.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.292+0000 7fd4fcad3700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd4e80a3cd0 0x7fd4e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:24.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.292+0000 7fd4f5ffb700 1 -- 172.21.15.82:0/376766987 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd4f000f020 con 0x7fd4f8108a30 2024-04-21T08:45:24.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.293+0000 7fd4f5ffb700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969] conn(0x7fd4e80ae990 0x7fd4e80b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:24.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.293+0000 7fd4f5ffb700 1 -- 172.21.15.82:0/376766987 --> [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd4e80b14a0 con 0x7fd4e80ae990 2024-04-21T08:45:24.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.293+0000 7fd4f5ffb700 1 -- 172.21.15.82:0/376766987 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd4f00d23d0 con 0x7fd4f8108a30 2024-04-21T08:45:24.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.293+0000 7fd4f7fff700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969] conn(0x7fd4e80ae990 0x7fd4e80b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:24.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.293+0000 7fd4fcad3700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd4e80a3cd0 0x7fd4e80a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec005950 tx=0x7fd4ec00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:24.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.293+0000 7fd4f7fff700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969] conn(0x7fd4e80ae990 0x7fd4e80b0dd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:24.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.294+0000 7fd4f5ffb700 1 -- 172.21.15.82:0/376766987 <== osd.6 v2:172.21.15.82:6818/1243429969 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd4e80b14a0 con 0x7fd4e80ae990 2024-04-21T08:45:24.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.310+0000 7fd4fed37700 1 -- 172.21.15.82:0/376766987 --> [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd4f8063610 con 0x7fd4e80ae990 2024-04-21T08:45:24.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.310+0000 7fd4f5ffb700 1 -- 172.21.15.82:0/376766987 <== osd.6 v2:172.21.15.82:6818/1243429969 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd4f8063610 con 0x7fd4e80ae990 2024-04-21T08:45:24.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.310+0000 7fd4fed37700 1 -- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969] conn(0x7fd4e80ae990 msgr2=0x7fd4e80b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:24.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.310+0000 7fd4fed37700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969] conn(0x7fd4e80ae990 0x7fd4e80b0dd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:24.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.310+0000 7fd4fed37700 1 -- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd4e80a3cd0 msgr2=0x7fd4e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:24.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.310+0000 7fd4fed37700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd4e80a3cd0 0x7fd4e80a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec005950 tx=0x7fd4ec00a300 comp rx=0 tx=0).stop 2024-04-21T08:45:24.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.310+0000 7fd4fed37700 1 -- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8108a30 msgr2=0x7fd4f819fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:24.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.310+0000 7fd4fed37700 1 --2- 172.21.15.82:0/376766987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4f8108a30 0x7fd4f819fb20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00062a0 tx=0x7fd4f0004e20 comp rx=0 tx=0).stop 2024-04-21T08:45:24.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.310+0000 7fd4fd2d4700 1 -- 172.21.15.82:0/376766987 reap_dead start 2024-04-21T08:45:24.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.311+0000 7fd4fed37700 1 -- 172.21.15.82:0/376766987 shutdown_connections 2024-04-21T08:45:24.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.311+0000 7fd4fed37700 1 -- 172.21.15.82:0/376766987 >> 172.21.15.82:0/376766987 conn(0x7fd4f8078910 msgr2=0x7fd4f810de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:24.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.311+0000 7fd4fed37700 1 -- 172.21.15.82:0/376766987 shutdown_connections 2024-04-21T08:45:24.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:24.311+0000 7fd4fed37700 1 -- 172.21.15.82:0/376766987 wait complete. 2024-04-21T08:45:25.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:25 smithi082 ceph-mon[27967]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-21T08:45:25.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:25 smithi134 ceph-mon[31772]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-21T08:45:25.782 INFO:teuthology.orchestra.run.smithi082.stdout:197568495665 2024-04-21T08:45:25.782 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-21T08:45:25.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:25 smithi106 ceph-mon[31072]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-21T08:45:26.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.379+0000 7f14177bf700 1 -- 172.21.15.82:0/3936984149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1410101940 msgr2=0x7f1410101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:26.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.379+0000 7f14177bf700 1 --2- 172.21.15.82:0/3936984149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1410101940 0x7f1410101dc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f140c009990 tx=0x7f140c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:45:26.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.379+0000 7f14177bf700 1 -- 172.21.15.82:0/3936984149 shutdown_connections 2024-04-21T08:45:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.379+0000 7f14177bf700 1 --2- 172.21.15.82:0/3936984149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1410102300 0x7f141010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.379+0000 7f14177bf700 1 --2- 172.21.15.82:0/3936984149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1410107990 0x7f1410107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.379+0000 7f14177bf700 1 --2- 172.21.15.82:0/3936984149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1410101940 0x7f1410101dc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.379+0000 7f14177bf700 1 -- 172.21.15.82:0/3936984149 >> 172.21.15.82:0/3936984149 conn(0x7f14100788f0 msgr2=0x7f1410078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.379+0000 7f14177bf700 1 -- 172.21.15.82:0/3936984149 shutdown_connections 2024-04-21T08:45:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.379+0000 7f14177bf700 1 -- 172.21.15.82:0/3936984149 wait complete. 2024-04-21T08:45:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f14177bf700 1 Processor -- start 2024-04-21T08:45:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f14177bf700 1 -- start start 2024-04-21T08:45:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f14177bf700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1410101940 0x7f1410198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f14177bf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1410102300 0x7f1410198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f14177bf700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1410107990 0x7f141019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f14177bf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1410111b00 con 0x7f1410102300 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f14177bf700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1410111980 con 0x7f1410107990 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f14177bf700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1410111c80 con 0x7f1410101940 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f1414d5a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1410102300 0x7f1410198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f1414d5a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1410102300 0x7f1410198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59538/0 (socket says 172.21.15.82:59538) 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f1414d5a700 1 -- 172.21.15.82:0/2346440378 learned_addr learned my addr 172.21.15.82:0/2346440378 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f141555b700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1410101940 0x7f1410198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f1415d5c700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1410107990 0x7f141019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.380+0000 7f1414d5a700 1 -- 172.21.15.82:0/2346440378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1410101940 msgr2=0x7f1410198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.381+0000 7f1414d5a700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1410101940 0x7f1410198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.381+0000 7f1414d5a700 1 -- 172.21.15.82:0/2346440378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1410107990 msgr2=0x7f141019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.381+0000 7f1414d5a700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1410107990 0x7f141019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.381+0000 7f1414d5a700 1 -- 172.21.15.82:0/2346440378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141019d020 con 0x7f1410102300 2024-04-21T08:45:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.381+0000 7f1414d5a700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1410102300 0x7f1410198550 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f140c000c00 tx=0x7f140c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.381+0000 7f14067fc700 1 -- 172.21.15.82:0/2346440378 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c027070 con 0x7f1410102300 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.381+0000 7f14177bf700 1 -- 172.21.15.82:0/2346440378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141019d2b0 con 0x7f1410102300 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.381+0000 7f14067fc700 1 -- 172.21.15.82:0/2346440378 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f140c003f20 con 0x7f1410102300 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.381+0000 7f14067fc700 1 -- 172.21.15.82:0/2346440378 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c022ca0 con 0x7f1410102300 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.381+0000 7f14177bf700 1 -- 172.21.15.82:0/2346440378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141006a720 con 0x7f1410102300 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.382+0000 7f14177bf700 1 -- 172.21.15.82:0/2346440378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f141006ac90 con 0x7f1410102300 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.383+0000 7f14067fc700 1 -- 172.21.15.82:0/2346440378 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f140c02a070 con 0x7f1410102300 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.383+0000 7f14067fc700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13fc0a3cd0 0x7f13fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.384+0000 7f141555b700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13fc0a3cd0 0x7f13fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.384+0000 7f14067fc700 1 -- 172.21.15.82:0/2346440378 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f140c0d1d00 con 0x7f1410102300 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.384+0000 7f14067fc700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110] conn(0x7f13fc0ae990 0x7f13fc0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.384+0000 7f14067fc700 1 -- 172.21.15.82:0/2346440378 --> [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f140c0095d0 con 0x7f13fc0ae990 2024-04-21T08:45:26.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.384+0000 7f14067fc700 1 -- 172.21.15.82:0/2346440378 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f140c0d2150 con 0x7f1410102300 2024-04-21T08:45:26.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.384+0000 7f141555b700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13fc0a3cd0 0x7f13fc0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f14000045b0 tx=0x7f140000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:26.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.384+0000 7f1415d5c700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110] conn(0x7f13fc0ae990 0x7f13fc0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:26.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.385+0000 7f1415d5c700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110] conn(0x7f13fc0ae990 0x7f13fc0b0dd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:26.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.385+0000 7f14067fc700 1 -- 172.21.15.82:0/2346440378 <== osd.4 v2:172.21.15.106:6808/1658170110 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f140c0095d0 con 0x7f13fc0ae990 2024-04-21T08:45:26.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.412+0000 7f14177bf700 1 -- 172.21.15.82:0/2346440378 --> [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f141006b110 con 0x7f13fc0ae990 2024-04-21T08:45:26.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.412+0000 7f14067fc700 1 -- 172.21.15.82:0/2346440378 <== osd.4 v2:172.21.15.106:6808/1658170110 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f141006b110 con 0x7f13fc0ae990 2024-04-21T08:45:26.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.413+0000 7f14177bf700 1 -- 172.21.15.82:0/2346440378 >> [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110] conn(0x7f13fc0ae990 msgr2=0x7f13fc0b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:26.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.413+0000 7f14177bf700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110] conn(0x7f13fc0ae990 0x7f13fc0b0dd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:26.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.413+0000 7f14177bf700 1 -- 172.21.15.82:0/2346440378 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13fc0a3cd0 msgr2=0x7f13fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:26.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.413+0000 7f14177bf700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13fc0a3cd0 0x7f13fc0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f14000045b0 tx=0x7f140000b3c0 comp rx=0 tx=0).stop 2024-04-21T08:45:26.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.413+0000 7f14177bf700 1 -- 172.21.15.82:0/2346440378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1410102300 msgr2=0x7f1410198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:26.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.413+0000 7f14177bf700 1 --2- 172.21.15.82:0/2346440378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1410102300 0x7f1410198550 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f140c000c00 tx=0x7f140c0047b0 comp rx=0 tx=0).stop 2024-04-21T08:45:26.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.413+0000 7f1415d5c700 1 -- 172.21.15.82:0/2346440378 reap_dead start 2024-04-21T08:45:26.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.413+0000 7f14177bf700 1 -- 172.21.15.82:0/2346440378 shutdown_connections 2024-04-21T08:45:26.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.413+0000 7f14177bf700 1 -- 172.21.15.82:0/2346440378 >> 172.21.15.82:0/2346440378 conn(0x7f14100788f0 msgr2=0x7f14100ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:26.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.413+0000 7f14177bf700 1 -- 172.21.15.82:0/2346440378 shutdown_connections 2024-04-21T08:45:26.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:26.413+0000 7f14177bf700 1 -- 172.21.15.82:0/2346440378 wait complete. 2024-04-21T08:45:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-21T08:45:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-21T08:45:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 7]}]: dispatch 2024-04-21T08:45:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-21T08:45:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-04-21T08:45:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-21T08:45:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-04-21T08:45:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-04-21T08:45:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-21T08:45:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-21T08:45:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-21T08:45:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 7]}]: dispatch 2024-04-21T08:45:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-21T08:45:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-04-21T08:45:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-21T08:45:26.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-04-21T08:45:26.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-04-21T08:45:26.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-21T08:45:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-21T08:45:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-21T08:45:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 7]}]: dispatch 2024-04-21T08:45:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-21T08:45:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-04-21T08:45:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-21T08:45:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-04-21T08:45:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-04-21T08:45:26.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-21T08:45:27.520 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:27 smithi082 ceph-mon[27967]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-21T08:45:27.521 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-21T08:45:27.521 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-21T08:45:27.521 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 7]}]': finished 2024-04-21T08:45:27.521 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-21T08:45:27.521 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-04-21T08:45:27.521 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-21T08:45:27.521 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-04-21T08:45:27.521 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-04-21T08:45:27.521 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-21T08:45:27.521 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:27 smithi082 ceph-mon[27967]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T08:45:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:27 smithi134 ceph-mon[31772]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-21T08:45:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-21T08:45:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-21T08:45:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 7]}]': finished 2024-04-21T08:45:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-21T08:45:27.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-04-21T08:45:27.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-21T08:45:27.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-04-21T08:45:27.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-04-21T08:45:27.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-21T08:45:27.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:27 smithi134 ceph-mon[31772]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T08:45:27.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:27 smithi106 ceph-mon[31072]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-21T08:45:27.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-21T08:45:27.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-21T08:45:27.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 7]}]': finished 2024-04-21T08:45:27.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-21T08:45:27.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-04-21T08:45:27.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-21T08:45:27.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-04-21T08:45:27.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-04-21T08:45:27.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-21T08:45:27.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:27 smithi106 ceph-mon[31072]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T08:45:28.686 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:28 smithi082 ceph-mon[27967]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T08:45:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:28 smithi134 ceph-mon[31772]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T08:45:28.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:28 smithi106 ceph-mon[31072]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T08:45:29.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:29 smithi082 ceph-mon[27967]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-21T08:45:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:29 smithi134 ceph-mon[31772]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-21T08:45:29.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:29 smithi106 ceph-mon[31072]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-21T08:45:30.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.437+0000 7fe7e3c34700 1 -- 172.21.15.82:0/67489779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7dc1011f0 msgr2=0x7fe7dc1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:30.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.437+0000 7fe7e3c34700 1 --2- 172.21.15.82:0/67489779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7dc1011f0 0x7fe7dc1015d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80099c0 tx=0x7fe7d8009cd0 comp rx=0 tx=0).stop 2024-04-21T08:45:30.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.437+0000 7fe7e3c34700 1 -- 172.21.15.82:0/67489779 shutdown_connections 2024-04-21T08:45:30.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.437+0000 7fe7e3c34700 1 --2- 172.21.15.82:0/67489779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7dc107e10 0x7fe7dc10f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:30.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.437+0000 7fe7e3c34700 1 --2- 172.21.15.82:0/67489779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7dc1011f0 0x7fe7dc1015d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:30.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.437+0000 7fe7e3c34700 1 --2- 172.21.15.82:0/67489779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7dc101ba0 0x7fe7dc1076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:30.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.437+0000 7fe7e3c34700 1 -- 172.21.15.82:0/67489779 >> 172.21.15.82:0/67489779 conn(0x7fe7dc077f40 msgr2=0x7fe7dc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:30.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.437+0000 7fe7e3c34700 1 -- 172.21.15.82:0/67489779 shutdown_connections 2024-04-21T08:45:30.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.437+0000 7fe7e3c34700 1 -- 172.21.15.82:0/67489779 wait complete. 2024-04-21T08:45:30.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.437+0000 7fe7e3c34700 1 Processor -- start 2024-04-21T08:45:30.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.437+0000 7fe7e3c34700 1 -- start start 2024-04-21T08:45:30.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e3c34700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7dc1011f0 0x7fe7dc199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:30.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e3c34700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7dc101ba0 0x7fe7dc1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:30.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e3c34700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7dc107e10 0x7fe7dc19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:30.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e3c34700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc112c60 con 0x7fe7dc1011f0 2024-04-21T08:45:30.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e3c34700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc112ae0 con 0x7fe7dc101ba0 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e3c34700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc112de0 con 0x7fe7dc107e10 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e19d0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7dc1011f0 0x7fe7dc199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e19d0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7dc1011f0 0x7fe7dc199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59562/0 (socket says 172.21.15.82:59562) 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e19d0700 1 -- 172.21.15.82:0/1444257174 learned_addr learned my addr 172.21.15.82:0/1444257174 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e11cf700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7dc101ba0 0x7fe7dc1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e21d1700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7dc107e10 0x7fe7dc19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e19d0700 1 -- 172.21.15.82:0/1444257174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7dc107e10 msgr2=0x7fe7dc19da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e19d0700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7dc107e10 0x7fe7dc19da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e19d0700 1 -- 172.21.15.82:0/1444257174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7dc101ba0 msgr2=0x7fe7dc1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e19d0700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7dc101ba0 0x7fe7dc1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e19d0700 1 -- 172.21.15.82:0/1444257174 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc19e180 con 0x7fe7dc1011f0 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e19d0700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7dc1011f0 0x7fe7dc199170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800b580 tx=0x7fe7d80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:30.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7d2ffd700 1 -- 172.21.15.82:0/1444257174 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8027070 con 0x7fe7dc1011f0 2024-04-21T08:45:30.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e3c34700 1 -- 172.21.15.82:0/1444257174 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7dc19e410 con 0x7fe7dc1011f0 2024-04-21T08:45:30.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7d2ffd700 1 -- 172.21.15.82:0/1444257174 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d8004020 con 0x7fe7dc1011f0 2024-04-21T08:45:30.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7d2ffd700 1 -- 172.21.15.82:0/1444257174 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8022e10 con 0x7fe7dc1011f0 2024-04-21T08:45:30.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.438+0000 7fe7e3c34700 1 -- 172.21.15.82:0/1444257174 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7dc1a5cf0 con 0x7fe7dc1011f0 2024-04-21T08:45:30.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.439+0000 7fe7d0ff9700 1 -- 172.21.15.82:0/1444257174 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7c4001030 con 0x7fe7dc1011f0 2024-04-21T08:45:30.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.441+0000 7fe7d2ffd700 1 -- 172.21.15.82:0/1444257174 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe7d802a070 con 0x7fe7dc1011f0 2024-04-21T08:45:30.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.441+0000 7fe7d2ffd700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe7c80a3cd0 0x7fe7c80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:30.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.441+0000 7fe7e11cf700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe7c80a3cd0 0x7fe7c80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:30.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.441+0000 7fe7d2ffd700 1 -- 172.21.15.82:0/1444257174 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fe7d80d1a60 con 0x7fe7dc1011f0 2024-04-21T08:45:30.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.441+0000 7fe7d2ffd700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879] conn(0x7fe7c80aedb0 0x7fe7c80b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:30.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.442+0000 7fe7d2ffd700 1 -- 172.21.15.82:0/1444257174 --> [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7c80b1920 con 0x7fe7c80aedb0 2024-04-21T08:45:30.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.442+0000 7fe7d2ffd700 1 -- 172.21.15.82:0/1444257174 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe7d80d1eb0 con 0x7fe7dc1011f0 2024-04-21T08:45:30.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.442+0000 7fe7e11cf700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe7c80a3cd0 0x7fe7c80a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc109690 tx=0x7fe7cc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:30.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.442+0000 7fe7e21d1700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879] conn(0x7fe7c80aedb0 0x7fe7c80b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:30.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.442+0000 7fe7e21d1700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879] conn(0x7fe7c80aedb0 0x7fe7c80b1250 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:30.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.443+0000 7fe7d2ffd700 1 -- 172.21.15.82:0/1444257174 <== osd.11 v2:172.21.15.134:6824/1549345879 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7c80b1920 con 0x7fe7c80aedb0 2024-04-21T08:45:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.463+0000 7fe7d0ff9700 1 -- 172.21.15.82:0/1444257174 --> [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7c4002e60 con 0x7fe7c80aedb0 2024-04-21T08:45:30.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.464+0000 7fe7d2ffd700 1 -- 172.21.15.82:0/1444257174 <== osd.11 v2:172.21.15.134:6824/1549345879 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe7c4002e60 con 0x7fe7c80aedb0 2024-04-21T08:45:30.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.465+0000 7fe7d0ff9700 1 -- 172.21.15.82:0/1444257174 >> [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879] conn(0x7fe7c80aedb0 msgr2=0x7fe7c80b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:30.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.465+0000 7fe7d0ff9700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879] conn(0x7fe7c80aedb0 0x7fe7c80b1250 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:30.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.465+0000 7fe7d0ff9700 1 -- 172.21.15.82:0/1444257174 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe7c80a3cd0 msgr2=0x7fe7c80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:30.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.465+0000 7fe7d0ff9700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe7c80a3cd0 0x7fe7c80a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc109690 tx=0x7fe7cc00a380 comp rx=0 tx=0).stop 2024-04-21T08:45:30.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.465+0000 7fe7d0ff9700 1 -- 172.21.15.82:0/1444257174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7dc1011f0 msgr2=0x7fe7dc199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:30.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.465+0000 7fe7d0ff9700 1 --2- 172.21.15.82:0/1444257174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7dc1011f0 0x7fe7dc199170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800b580 tx=0x7fe7d80047b0 comp rx=0 tx=0).stop 2024-04-21T08:45:30.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.465+0000 7fe7e21d1700 1 -- 172.21.15.82:0/1444257174 reap_dead start 2024-04-21T08:45:30.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.465+0000 7fe7d0ff9700 1 -- 172.21.15.82:0/1444257174 shutdown_connections 2024-04-21T08:45:30.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.465+0000 7fe7d0ff9700 1 -- 172.21.15.82:0/1444257174 >> 172.21.15.82:0/1444257174 conn(0x7fe7dc077f40 msgr2=0x7fe7dc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:30.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.465+0000 7fe7d0ff9700 1 -- 172.21.15.82:0/1444257174 shutdown_connections 2024-04-21T08:45:30.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:30.465+0000 7fe7d0ff9700 1 -- 172.21.15.82:0/1444257174 wait complete. 2024-04-21T08:45:31.578 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:31 smithi082 ceph-mon[27967]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T08:45:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:31 smithi134 ceph-mon[31772]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T08:45:31.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:31 smithi106 ceph-mon[31072]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 643 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-21T08:45:32.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.856+0000 7fecc67ab700 1 -- 172.21.15.82:0/2023163040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0102b00 msgr2=0x7fecc0102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.856+0000 7fecc67ab700 1 --2- 172.21.15.82:0/2023163040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0102b00 0x7fecc0102f80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009990 tx=0x7fecb4009ca0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.856+0000 7fecc67ab700 1 -- 172.21.15.82:0/2023163040 shutdown_connections 2024-04-21T08:45:32.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.856+0000 7fecc67ab700 1 --2- 172.21.15.82:0/2023163040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecc01034c0 0x7fecc010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.856+0000 7fecc67ab700 1 --2- 172.21.15.82:0/2023163040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecc0108a10 0x7fecc0108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.856+0000 7fecc67ab700 1 --2- 172.21.15.82:0/2023163040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0102b00 0x7fecc0102f80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.856+0000 7fecc67ab700 1 -- 172.21.15.82:0/2023163040 >> 172.21.15.82:0/2023163040 conn(0x7fecc0079c00 msgr2=0x7fecc007a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.856+0000 7fecc67ab700 1 -- 172.21.15.82:0/2023163040 shutdown_connections 2024-04-21T08:45:32.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.856+0000 7fecc67ab700 1 -- 172.21.15.82:0/2023163040 wait complete. 2024-04-21T08:45:32.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.856+0000 7fecc67ab700 1 Processor -- start 2024-04-21T08:45:32.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.856+0000 7fecc67ab700 1 -- start start 2024-04-21T08:45:32.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc67ab700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecc0102b00 0x7fecc0199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc67ab700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecc01034c0 0x7fecc0199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc67ab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0108a10 0x7fecc019da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc67ab700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0112c50 con 0x7fecc0108a10 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc67ab700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0112ad0 con 0x7fecc0102b00 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc67ab700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0112dd0 con 0x7fecc01034c0 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc4d48700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0108a10 0x7fecc019da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc4d48700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0108a10 0x7fecc019da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49568/0 (socket says 172.21.15.82:49568) 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc4d48700 1 -- 172.21.15.82:0/388849702 learned_addr learned my addr 172.21.15.82:0/388849702 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecbf7fe700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecc01034c0 0x7fecc0199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecbffff700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecc0102b00 0x7fecc0199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc4d48700 1 -- 172.21.15.82:0/388849702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecc01034c0 msgr2=0x7fecc0199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc4d48700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecc01034c0 0x7fecc0199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc4d48700 1 -- 172.21.15.82:0/388849702 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecc0102b00 msgr2=0x7fecc0199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc4d48700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecc0102b00 0x7fecc0199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc4d48700 1 -- 172.21.15.82:0/388849702 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc019e150 con 0x7fecc0108a10 2024-04-21T08:45:32.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc4d48700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0108a10 0x7fecc019da10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fecb8007ee0 tx=0x7fecb800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecbd7fa700 1 -- 172.21.15.82:0/388849702 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb800dbd0 con 0x7fecc0108a10 2024-04-21T08:45:32.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.857+0000 7fecc67ab700 1 -- 172.21.15.82:0/388849702 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc019e440 con 0x7fecc0108a10 2024-04-21T08:45:32.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.858+0000 7fecbd7fa700 1 -- 172.21.15.82:0/388849702 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb800dd70 con 0x7fecc0108a10 2024-04-21T08:45:32.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.858+0000 7fecbd7fa700 1 -- 172.21.15.82:0/388849702 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8005990 con 0x7fecc0108a10 2024-04-21T08:45:32.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.858+0000 7fecc67ab700 1 -- 172.21.15.82:0/388849702 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc0074c70 con 0x7fecc0108a10 2024-04-21T08:45:32.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.858+0000 7fecc67ab700 1 -- 172.21.15.82:0/388849702 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fecc00631d0 con 0x7fecc0108a10 2024-04-21T08:45:32.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.860+0000 7fecbd7fa700 1 -- 172.21.15.82:0/388849702 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fecb801a020 con 0x7fecc0108a10 2024-04-21T08:45:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.860+0000 7fecbd7fa700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7feca80a3cd0 0x7feca80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.860+0000 7fecbffff700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7feca80a3cd0 0x7feca80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.860+0000 7fecbd7fa700 1 -- 172.21.15.82:0/388849702 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fecb80d0a20 con 0x7fecc0108a10 2024-04-21T08:45:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.860+0000 7fecbd7fa700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036] conn(0x7feca80aedb0 0x7feca80b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.860+0000 7fecbd7fa700 1 -- 172.21.15.82:0/388849702 --> [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fecb800be50 con 0x7feca80aedb0 2024-04-21T08:45:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.860+0000 7fecbd7fa700 1 -- 172.21.15.82:0/388849702 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fecb80d0e70 con 0x7fecc0108a10 2024-04-21T08:45:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.861+0000 7fecbffff700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7feca80a3cd0 0x7feca80a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fecb0006fd0 tx=0x7fecb0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.861+0000 7fecbf7fe700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036] conn(0x7feca80aedb0 0x7feca80b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.861+0000 7fecbf7fe700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036] conn(0x7feca80aedb0 0x7feca80b1250 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:32.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.862+0000 7fecbd7fa700 1 -- 172.21.15.82:0/388849702 <== osd.7 v2:172.21.15.106:6816/3692373036 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fecb800be50 con 0x7feca80aedb0 2024-04-21T08:45:32.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.878+0000 7fecc67ab700 1 -- 172.21.15.82:0/388849702 --> [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fecc0063650 con 0x7feca80aedb0 2024-04-21T08:45:32.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.879+0000 7fecbd7fa700 1 -- 172.21.15.82:0/388849702 <== osd.7 v2:172.21.15.106:6816/3692373036 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fecc0063650 con 0x7feca80aedb0 2024-04-21T08:45:32.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.879+0000 7fecc67ab700 1 -- 172.21.15.82:0/388849702 >> [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036] conn(0x7feca80aedb0 msgr2=0x7feca80b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.879+0000 7fecc67ab700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036] conn(0x7feca80aedb0 0x7feca80b1250 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.879+0000 7fecc67ab700 1 -- 172.21.15.82:0/388849702 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7feca80a3cd0 msgr2=0x7feca80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.879+0000 7fecc67ab700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7feca80a3cd0 0x7feca80a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fecb0006fd0 tx=0x7fecb0008040 comp rx=0 tx=0).stop 2024-04-21T08:45:32.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.879+0000 7fecc67ab700 1 -- 172.21.15.82:0/388849702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0108a10 msgr2=0x7fecc019da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:32.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.879+0000 7fecc67ab700 1 --2- 172.21.15.82:0/388849702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0108a10 0x7fecc019da10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fecb8007ee0 tx=0x7fecb800d3b0 comp rx=0 tx=0).stop 2024-04-21T08:45:32.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.879+0000 7fecc4d48700 1 -- 172.21.15.82:0/388849702 reap_dead start 2024-04-21T08:45:32.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.879+0000 7fecc67ab700 1 -- 172.21.15.82:0/388849702 shutdown_connections 2024-04-21T08:45:32.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.879+0000 7fecc67ab700 1 -- 172.21.15.82:0/388849702 >> 172.21.15.82:0/388849702 conn(0x7fecc0079c00 msgr2=0x7fecc0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:32.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.879+0000 7fecc67ab700 1 -- 172.21.15.82:0/388849702 shutdown_connections 2024-04-21T08:45:32.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.879+0000 7fecc67ab700 1 -- 172.21.15.82:0/388849702 wait complete. 2024-04-21T08:45:33.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.980+0000 7f5303d54700 1 -- 172.21.15.82:0/3225622927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52fc1022f0 msgr2=0x7f52fc10e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:33.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.980+0000 7f5303d54700 1 --2- 172.21.15.82:0/3225622927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52fc1022f0 0x7f52fc10e7e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f52f400b0a0 tx=0x7f52f400b3b0 comp rx=0 tx=0).stop 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.980+0000 7f5303d54700 1 -- 172.21.15.82:0/3225622927 shutdown_connections 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.980+0000 7f5303d54700 1 --2- 172.21.15.82:0/3225622927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52fc1022f0 0x7f52fc10e7e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.980+0000 7f5303d54700 1 --2- 172.21.15.82:0/3225622927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52fc107980 0x7f52fc107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.980+0000 7f5303d54700 1 --2- 172.21.15.82:0/3225622927 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52fc101930 0x7f52fc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.980+0000 7f5303d54700 1 -- 172.21.15.82:0/3225622927 >> 172.21.15.82:0/3225622927 conn(0x7f52fc0788b0 msgr2=0x7f52fc078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.980+0000 7f5303d54700 1 -- 172.21.15.82:0/3225622927 shutdown_connections 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.980+0000 7f5303d54700 1 -- 172.21.15.82:0/3225622927 wait complete. 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5303d54700 1 Processor -- start 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5303d54700 1 -- start start 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5303d54700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52fc101930 0x7f52fc199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5303d54700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52fc1022f0 0x7f52fc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5303d54700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52fc107980 0x7f52fc1a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:33.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5303d54700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc111ad0 con 0x7f52fc101930 2024-04-21T08:45:33.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5303d54700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc111950 con 0x7f52fc1022f0 2024-04-21T08:45:33.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5303d54700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc111c50 con 0x7f52fc107980 2024-04-21T08:45:33.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5301af0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52fc101930 0x7f52fc199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:33.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5301af0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52fc101930 0x7f52fc199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59586/0 (socket says 172.21.15.82:59586) 2024-04-21T08:45:33.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5301af0700 1 -- 172.21.15.82:0/1554483100 learned_addr learned my addr 172.21.15.82:0/1554483100 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:33.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f53022f1700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52fc107980 0x7f52fc1a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:33.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5301af0700 1 -- 172.21.15.82:0/1554483100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52fc107980 msgr2=0x7f52fc1a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:33.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5301af0700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52fc107980 0x7f52fc1a4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5301af0700 1 -- 172.21.15.82:0/1554483100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52fc1022f0 msgr2=0x7f52fc19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5301af0700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52fc1022f0 0x7f52fc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f5301af0700 1 -- 172.21.15.82:0/1554483100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52fc1a5510 con 0x7f52fc101930 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.981+0000 7f53022f1700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52fc107980 0x7f52fc1a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.982+0000 7f5301af0700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52fc101930 0x7f52fc199f50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00d950 tx=0x7f52ec00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.982+0000 7f52f2ffd700 1 -- 172.21.15.82:0/1554483100 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec016070 con 0x7f52fc101930 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.982+0000 7f52f2ffd700 1 -- 172.21.15.82:0/1554483100 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ec010460 con 0x7f52fc101930 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.982+0000 7f5303d54700 1 -- 172.21.15.82:0/1554483100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52fc1a5740 con 0x7f52fc101930 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.982+0000 7f52f2ffd700 1 -- 172.21.15.82:0/1554483100 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec00f620 con 0x7f52fc101930 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.982+0000 7f5303d54700 1 -- 172.21.15.82:0/1554483100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52fc1a5c50 con 0x7f52fc101930 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.982+0000 7f5303d54700 1 -- 172.21.15.82:0/1554483100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52fc063150 con 0x7f52fc101930 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.984+0000 7f52f2ffd700 1 -- 172.21.15.82:0/1554483100 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f52ec019070 con 0x7f52fc101930 2024-04-21T08:45:33.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.984+0000 7f52f2ffd700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f52e80a3cd0 0x7f52e80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.985+0000 7f53012ef700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f52e80a3cd0 0x7f52e80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.985+0000 7f52f2ffd700 1 -- 172.21.15.82:0/1554483100 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f52ec0d09d0 con 0x7f52fc101930 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.985+0000 7f52f2ffd700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685] conn(0x7f52e80aed30 0x7f52e80b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.985+0000 7f52f2ffd700 1 -- 172.21.15.82:0/1554483100 --> [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52e80b18a0 con 0x7f52e80aed30 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.985+0000 7f52f2ffd700 1 -- 172.21.15.82:0/1554483100 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f52ec0d0e20 con 0x7f52fc101930 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.985+0000 7f53012ef700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f52e80a3cd0 0x7f52e80a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52f8005950 tx=0x7f52f80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.985+0000 7f53022f1700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685] conn(0x7f52e80aed30 0x7f52e80b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.985+0000 7f53022f1700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685] conn(0x7f52e80aed30 0x7f52e80b11d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:31.986+0000 7f52f2ffd700 1 -- 172.21.15.82:0/1554483100 <== osd.3 v2:172.21.15.82:6810/2296521685 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52e80b18a0 con 0x7f52e80aed30 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.003+0000 7f5303d54700 1 -- 172.21.15.82:0/1554483100 --> [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f52fc063550 con 0x7f52e80aed30 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.004+0000 7f52f2ffd700 1 -- 172.21.15.82:0/1554483100 <== osd.3 v2:172.21.15.82:6810/2296521685 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f52fc063550 con 0x7f52e80aed30 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.004+0000 7f5303d54700 1 -- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685] conn(0x7f52e80aed30 msgr2=0x7f52e80b11d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:33.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.004+0000 7f5303d54700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685] conn(0x7f52e80aed30 0x7f52e80b11d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:33.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.004+0000 7f5303d54700 1 -- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f52e80a3cd0 msgr2=0x7f52e80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:33.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.004+0000 7f5303d54700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f52e80a3cd0 0x7f52e80a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52f8005950 tx=0x7f52f80058e0 comp rx=0 tx=0).stop 2024-04-21T08:45:33.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.004+0000 7f5303d54700 1 -- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52fc101930 msgr2=0x7f52fc199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:33.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.004+0000 7f5303d54700 1 --2- 172.21.15.82:0/1554483100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52fc101930 0x7f52fc199f50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00d950 tx=0x7f52ec00dd10 comp rx=0 tx=0).stop 2024-04-21T08:45:33.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.004+0000 7f53022f1700 1 -- 172.21.15.82:0/1554483100 reap_dead start 2024-04-21T08:45:33.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.004+0000 7f5303d54700 1 -- 172.21.15.82:0/1554483100 shutdown_connections 2024-04-21T08:45:33.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.004+0000 7f5303d54700 1 -- 172.21.15.82:0/1554483100 >> 172.21.15.82:0/1554483100 conn(0x7f52fc0788b0 msgr2=0x7f52fc0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:33.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.004+0000 7f5303d54700 1 -- 172.21.15.82:0/1554483100 shutdown_connections 2024-04-21T08:45:33.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:32.004+0000 7f5303d54700 1 -- 172.21.15.82:0/1554483100 wait complete. 2024-04-21T08:45:33.536 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:33 smithi082 ceph-mon[27967]: pgmap v324: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-21T08:45:33.536 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:33 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:45:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:33 smithi134 ceph-mon[31772]: pgmap v324: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-21T08:45:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:33 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:45:33.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:33 smithi106 ceph-mon[31072]: pgmap v324: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-04-21T08:45:33.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:33 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:45:33.894 INFO:teuthology.orchestra.run.smithi082.stdout:343597383705 2024-04-21T08:45:33.895 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-21T08:45:34.428 INFO:teuthology.orchestra.run.smithi082.stdout:219043332143 2024-04-21T08:45:34.429 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-21T08:45:34.824 INFO:teuthology.orchestra.run.smithi082.stdout:107374182466 2024-04-21T08:45:34.824 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-21T08:45:35.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:35 smithi082 ceph-mon[27967]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2024-04-21T08:45:35.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:35 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:45:35.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:35 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:45:35.635 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:35 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:45:35.635 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:35 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:45:35.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:35 smithi134 ceph-mon[31772]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2024-04-21T08:45:35.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:35 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:45:35.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:35 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:45:35.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:35 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:45:35.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:35 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:45:35.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.792+0000 7fed996f2700 1 -- 172.21.15.82:0/1857477881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed941019d0 msgr2=0x7fed9410f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.792+0000 7fed996f2700 1 --2- 172.21.15.82:0/1857477881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed941019d0 0x7fed9410f910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00b0a0 tx=0x7fed8c00b3b0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.793+0000 7fed996f2700 1 -- 172.21.15.82:0/1857477881 shutdown_connections 2024-04-21T08:45:35.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.793+0000 7fed996f2700 1 --2- 172.21.15.82:0/1857477881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed941019d0 0x7fed9410f910 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.793+0000 7fed996f2700 1 --2- 172.21.15.82:0/1857477881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed94108ad0 0x7fed94108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.793+0000 7fed996f2700 1 --2- 172.21.15.82:0/1857477881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed94101030 0x7fed94101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.793+0000 7fed996f2700 1 -- 172.21.15.82:0/1857477881 >> 172.21.15.82:0/1857477881 conn(0x7fed94077fb0 msgr2=0x7fed940ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.793+0000 7fed996f2700 1 -- 172.21.15.82:0/1857477881 shutdown_connections 2024-04-21T08:45:35.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.793+0000 7fed996f2700 1 -- 172.21.15.82:0/1857477881 wait complete. 2024-04-21T08:45:35.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.793+0000 7fed996f2700 1 Processor -- start 2024-04-21T08:45:35.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.794+0000 7fed996f2700 1 -- start start 2024-04-21T08:45:35.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.794+0000 7fed996f2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed94101030 0x7fed94198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.794+0000 7fed996f2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed941019d0 0x7fed94199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.794+0000 7fed996f2700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed94108ad0 0x7fed9419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.794+0000 7fed996f2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fed94112be0 con 0x7fed941019d0 2024-04-21T08:45:35.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.794+0000 7fed996f2700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fed94112a60 con 0x7fed94108ad0 2024-04-21T08:45:35.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.794+0000 7fed996f2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed94112d60 con 0x7fed94101030 2024-04-21T08:45:35.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.794+0000 7fed927fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed941019d0 0x7fed94199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.794+0000 7fed927fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed941019d0 0x7fed94199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49598/0 (socket says 172.21.15.82:49598) 2024-04-21T08:45:35.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.794+0000 7fed927fc700 1 -- 172.21.15.82:0/4052103290 learned_addr learned my addr 172.21.15.82:0/4052103290 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:35.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.794+0000 7fed92ffd700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed94101030 0x7fed94198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.794+0000 7fed937fe700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed94108ad0 0x7fed9419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.795+0000 7fed92ffd700 1 -- 172.21.15.82:0/4052103290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed94108ad0 msgr2=0x7fed9419d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.795+0000 7fed92ffd700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed94108ad0 0x7fed9419d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.795+0000 7fed92ffd700 1 -- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed941019d0 msgr2=0x7fed94199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.795+0000 7fed92ffd700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed941019d0 0x7fed94199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.795+0000 7fed92ffd700 1 -- 172.21.15.82:0/4052103290 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9419e000 con 0x7fed94101030 2024-04-21T08:45:35.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.795+0000 7fed927fc700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed941019d0 0x7fed94199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:35.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.795+0000 7fed92ffd700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed94101030 0x7fed94198ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fed8400c970 tx=0x7fed8400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.795+0000 7fed7bfff700 1 -- 172.21.15.82:0/4052103290 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84014070 con 0x7fed94101030 2024-04-21T08:45:35.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.795+0000 7fed7bfff700 1 -- 172.21.15.82:0/4052103290 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed84004440 con 0x7fed94101030 2024-04-21T08:45:35.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.795+0000 7fed996f2700 1 -- 172.21.15.82:0/4052103290 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9419e2f0 con 0x7fed94101030 2024-04-21T08:45:35.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.795+0000 7fed7bfff700 1 -- 172.21.15.82:0/4052103290 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84007a30 con 0x7fed94101030 2024-04-21T08:45:35.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.795+0000 7fed996f2700 1 -- 172.21.15.82:0/4052103290 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed94102140 con 0x7fed94101030 2024-04-21T08:45:35.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.796+0000 7fed996f2700 1 -- 172.21.15.82:0/4052103290 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed940631d0 con 0x7fed94101030 2024-04-21T08:45:35.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.797+0000 7fed7bfff700 1 -- 172.21.15.82:0/4052103290 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fed84017070 con 0x7fed94101030 2024-04-21T08:45:35.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.798+0000 7fed7bfff700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fed7c0a3d50 0x7fed7c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.798+0000 7fed7bfff700 1 -- 172.21.15.82:0/4052103290 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fed840d03f0 con 0x7fed94101030 2024-04-21T08:45:35.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.798+0000 7fed927fc700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fed7c0a3d50 0x7fed7c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.798+0000 7fed7bfff700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221] conn(0x7fed7c0aee30 0x7fed7c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:35.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.798+0000 7fed7bfff700 1 -- 172.21.15.82:0/4052103290 --> [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed84007e50 con 0x7fed7c0aee30 2024-04-21T08:45:35.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.798+0000 7fed7bfff700 1 -- 172.21.15.82:0/4052103290 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fed840d0840 con 0x7fed94101030 2024-04-21T08:45:35.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.799+0000 7fed937fe700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221] conn(0x7fed7c0aee30 0x7fed7c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:35.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.799+0000 7fed927fc700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fed7c0a3d50 0x7fed7c0a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fed941081c0 tx=0x7fed8800a400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.799+0000 7fed937fe700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221] conn(0x7fed7c0aee30 0x7fed7c0b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:35.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.800+0000 7fed7bfff700 1 -- 172.21.15.82:0/4052103290 <== osd.0 v2:172.21.15.82:6802/1328184221 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed84007e50 con 0x7fed7c0aee30 2024-04-21T08:45:35.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.816+0000 7fed996f2700 1 -- 172.21.15.82:0/4052103290 --> [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed940635d0 con 0x7fed7c0aee30 2024-04-21T08:45:35.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.817+0000 7fed7bfff700 1 -- 172.21.15.82:0/4052103290 <== osd.0 v2:172.21.15.82:6802/1328184221 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fed940635d0 con 0x7fed7c0aee30 2024-04-21T08:45:35.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.817+0000 7fed996f2700 1 -- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221] conn(0x7fed7c0aee30 msgr2=0x7fed7c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.817+0000 7fed996f2700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221] conn(0x7fed7c0aee30 0x7fed7c0b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:35.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.817+0000 7fed996f2700 1 -- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fed7c0a3d50 msgr2=0x7fed7c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.817+0000 7fed996f2700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fed7c0a3d50 0x7fed7c0a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fed941081c0 tx=0x7fed8800a400 comp rx=0 tx=0).stop 2024-04-21T08:45:35.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.817+0000 7fed996f2700 1 -- 172.21.15.82:0/4052103290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed94101030 msgr2=0x7fed94198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:35.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.817+0000 7fed996f2700 1 --2- 172.21.15.82:0/4052103290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed94101030 0x7fed94198ff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fed8400c970 tx=0x7fed8400cc80 comp rx=0 tx=0).stop 2024-04-21T08:45:35.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.817+0000 7fed937fe700 1 -- 172.21.15.82:0/4052103290 reap_dead start 2024-04-21T08:45:35.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.817+0000 7fed996f2700 1 -- 172.21.15.82:0/4052103290 shutdown_connections 2024-04-21T08:45:35.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.817+0000 7fed996f2700 1 -- 172.21.15.82:0/4052103290 >> 172.21.15.82:0/4052103290 conn(0x7fed94077fb0 msgr2=0x7fed940ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:35.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.818+0000 7fed996f2700 1 -- 172.21.15.82:0/4052103290 shutdown_connections 2024-04-21T08:45:35.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.818+0000 7fed996f2700 1 -- 172.21.15.82:0/4052103290 wait complete. 2024-04-21T08:45:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:35 smithi106 ceph-mon[31072]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2024-04-21T08:45:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:35 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:45:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:35 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:45:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:35 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:45:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:35 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:45:36.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.688+0000 7f989610d700 1 -- 172.21.15.82:0/2596755312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9890103460 msgr2=0x7f989010f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.688+0000 7f989610d700 1 --2- 172.21.15.82:0/2596755312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9890103460 0x7f989010f940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f988800b0a0 tx=0x7f988800b3b0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.688+0000 7f989610d700 1 -- 172.21.15.82:0/2596755312 shutdown_connections 2024-04-21T08:45:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.688+0000 7f989610d700 1 --2- 172.21.15.82:0/2596755312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9890103460 0x7f989010f940 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.688+0000 7f989610d700 1 --2- 172.21.15.82:0/2596755312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9890108a70 0x7f9890108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.688+0000 7f989610d700 1 --2- 172.21.15.82:0/2596755312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9890102ac0 0x7f9890102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.688+0000 7f989610d700 1 -- 172.21.15.82:0/2596755312 >> 172.21.15.82:0/2596755312 conn(0x7f9890079bf0 msgr2=0x7f989007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.688+0000 7f989610d700 1 -- 172.21.15.82:0/2596755312 shutdown_connections 2024-04-21T08:45:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.689+0000 7f989610d700 1 -- 172.21.15.82:0/2596755312 wait complete. 2024-04-21T08:45:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.689+0000 7f989610d700 1 Processor -- start 2024-04-21T08:45:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.689+0000 7f989610d700 1 -- start start 2024-04-21T08:45:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f989610d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9890102ac0 0x7f9890199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f989610d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9890103460 0x7f98901995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f989610d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9890108a70 0x7f989019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f989610d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9890112c10 con 0x7f9890103460 2024-04-21T08:45:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f989610d700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9890112a90 con 0x7f9890108a70 2024-04-21T08:45:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f989610d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9890112d90 con 0x7f9890102ac0 2024-04-21T08:45:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f988effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9890103460 0x7f98901995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f988effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9890103460 0x7f98901995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49574/0 (socket says 172.21.15.82:49574) 2024-04-21T08:45:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f988effd700 1 -- 172.21.15.82:0/172324614 learned_addr learned my addr 172.21.15.82:0/172324614 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f988ffff700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9890108a70 0x7f989019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f988effd700 1 -- 172.21.15.82:0/172324614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9890102ac0 msgr2=0x7f9890199080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:45:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f988effd700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9890102ac0 0x7f9890199080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f988effd700 1 -- 172.21.15.82:0/172324614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9890108a70 msgr2=0x7f989019d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f988effd700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9890108a70 0x7f989019d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f988effd700 1 -- 172.21.15.82:0/172324614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989019e040 con 0x7f9890103460 2024-04-21T08:45:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.690+0000 7f988effd700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9890103460 0x7f98901995c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f988400c9b0 tx=0x7f988400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.691+0000 7f988cff9700 1 -- 172.21.15.82:0/172324614 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884007b00 con 0x7f9890103460 2024-04-21T08:45:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.691+0000 7f988cff9700 1 -- 172.21.15.82:0/172324614 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9884007ca0 con 0x7f9890103460 2024-04-21T08:45:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.691+0000 7f989610d700 1 -- 172.21.15.82:0/172324614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989019e330 con 0x7f9890103460 2024-04-21T08:45:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.691+0000 7f988cff9700 1 -- 172.21.15.82:0/172324614 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884018660 con 0x7f9890103460 2024-04-21T08:45:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.691+0000 7f989610d700 1 -- 172.21.15.82:0/172324614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98901a5bb0 con 0x7f9890103460 2024-04-21T08:45:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.692+0000 7f989610d700 1 -- 172.21.15.82:0/172324614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98900631d0 con 0x7f9890103460 2024-04-21T08:45:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.693+0000 7f988cff9700 1 -- 172.21.15.82:0/172324614 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f988400f450 con 0x7f9890103460 2024-04-21T08:45:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.693+0000 7f988cff9700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f98780a3cd0 0x7f98780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.694+0000 7f988f7fe700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f98780a3cd0 0x7f98780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.694+0000 7f988cff9700 1 -- 172.21.15.82:0/172324614 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f98840d08b0 con 0x7f9890103460 2024-04-21T08:45:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.694+0000 7f988cff9700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684] conn(0x7f98780aedb0 0x7f98780b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.694+0000 7f988cff9700 1 -- 172.21.15.82:0/172324614 --> [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9884012ec0 con 0x7f98780aedb0 2024-04-21T08:45:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.694+0000 7f988cff9700 1 -- 172.21.15.82:0/172324614 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f98840d52c0 con 0x7f9890103460 2024-04-21T08:45:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.694+0000 7f988ffff700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684] conn(0x7f98780aedb0 0x7f98780b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.694+0000 7f988f7fe700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f98780a3cd0 0x7f98780a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9880009a70 tx=0x7f9880009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.695+0000 7f988ffff700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684] conn(0x7f98780aedb0 0x7f98780b1250 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.696+0000 7f988cff9700 1 -- 172.21.15.82:0/172324614 <== osd.8 v2:172.21.15.134:6816/1700011684 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9884012ec0 con 0x7f98780aedb0 2024-04-21T08:45:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.722+0000 7f989610d700 1 -- 172.21.15.82:0/172324614 --> [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9890063650 con 0x7f98780aedb0 2024-04-21T08:45:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.722+0000 7f988cff9700 1 -- 172.21.15.82:0/172324614 <== osd.8 v2:172.21.15.134:6816/1700011684 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9890063650 con 0x7f98780aedb0 2024-04-21T08:45:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.723+0000 7f989610d700 1 -- 172.21.15.82:0/172324614 >> [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684] conn(0x7f98780aedb0 msgr2=0x7f98780b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.723+0000 7f989610d700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684] conn(0x7f98780aedb0 0x7f98780b1250 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.723+0000 7f989610d700 1 -- 172.21.15.82:0/172324614 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f98780a3cd0 msgr2=0x7f98780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.723+0000 7f989610d700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f98780a3cd0 0x7f98780a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9880009a70 tx=0x7f9880009260 comp rx=0 tx=0).stop 2024-04-21T08:45:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.723+0000 7f989610d700 1 -- 172.21.15.82:0/172324614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9890103460 msgr2=0x7f98901995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.723+0000 7f989610d700 1 --2- 172.21.15.82:0/172324614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9890103460 0x7f98901995c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f988400c9b0 tx=0x7f988400ccc0 comp rx=0 tx=0).stop 2024-04-21T08:45:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.723+0000 7f988ffff700 1 -- 172.21.15.82:0/172324614 reap_dead start 2024-04-21T08:45:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.723+0000 7f989610d700 1 -- 172.21.15.82:0/172324614 shutdown_connections 2024-04-21T08:45:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.723+0000 7f989610d700 1 -- 172.21.15.82:0/172324614 >> 172.21.15.82:0/172324614 conn(0x7f9890079bf0 msgr2=0x7f9890100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.723+0000 7f989610d700 1 -- 172.21.15.82:0/172324614 shutdown_connections 2024-04-21T08:45:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:35.723+0000 7f989610d700 1 -- 172.21.15.82:0/172324614 wait complete. 2024-04-21T08:45:37.277 INFO:teuthology.orchestra.run.smithi082.stdout:141733920828 2024-04-21T08:45:37.277 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-21T08:45:37.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:37 smithi082 ceph-mon[27967]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:45:37.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:37 smithi134 ceph-mon[31772]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:45:37.825 INFO:teuthology.orchestra.run.smithi082.stdout:34359738449 2024-04-21T08:45:37.826 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-21T08:45:37.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:37 smithi106 ceph-mon[31072]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:45:38.294 INFO:teuthology.orchestra.run.smithi082.stdout:253403070505 2024-04-21T08:45:38.294 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-21T08:45:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.808+0000 7f8419f58700 1 -- 172.21.15.82:0/130234119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8414101950 msgr2=0x7f841410f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.808+0000 7f8419f58700 1 --2- 172.21.15.82:0/130234119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8414101950 0x7f841410f920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f840c00b0d0 tx=0x7f840c00b3e0 comp rx=0 tx=0).stop 2024-04-21T08:45:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.808+0000 7f8419f58700 1 -- 172.21.15.82:0/130234119 shutdown_connections 2024-04-21T08:45:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.808+0000 7f8419f58700 1 --2- 172.21.15.82:0/130234119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8414101950 0x7f841410f920 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.808+0000 7f8419f58700 1 --2- 172.21.15.82:0/130234119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8414107a60 0x7f8414107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.808+0000 7f8419f58700 1 --2- 172.21.15.82:0/130234119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8414100f90 0x7f8414101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.808+0000 7f8419f58700 1 -- 172.21.15.82:0/130234119 >> 172.21.15.82:0/130234119 conn(0x7f8414077f40 msgr2=0x7f84140ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.808+0000 7f8419f58700 1 -- 172.21.15.82:0/130234119 shutdown_connections 2024-04-21T08:45:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.808+0000 7f8419f58700 1 -- 172.21.15.82:0/130234119 wait complete. 2024-04-21T08:45:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.808+0000 7f8419f58700 1 Processor -- start 2024-04-21T08:45:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f8419f58700 1 -- start start 2024-04-21T08:45:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f8419f58700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8414100f90 0x7f8414198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f8419f58700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8414101950 0x7f8414199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f8419f58700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8414107a60 0x7f841419d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f8419f58700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8414112d10 con 0x7f8414101950 2024-04-21T08:45:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f8419f58700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8414112b90 con 0x7f8414107a60 2024-04-21T08:45:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f8419f58700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8414112e90 con 0x7f8414100f90 2024-04-21T08:45:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f840affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8414101950 0x7f8414199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f840affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8414101950 0x7f8414199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49612/0 (socket says 172.21.15.82:49612) 2024-04-21T08:45:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f840affd700 1 -- 172.21.15.82:0/4110604475 learned_addr learned my addr 172.21.15.82:0/4110604475 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f8413fff700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8414107a60 0x7f841419d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f84137fe700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8414100f90 0x7f8414198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f840affd700 1 -- 172.21.15.82:0/4110604475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8414100f90 msgr2=0x7f8414198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f840affd700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8414100f90 0x7f8414198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f840affd700 1 -- 172.21.15.82:0/4110604475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8414107a60 msgr2=0x7f841419d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f840affd700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8414107a60 0x7f841419d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f840affd700 1 -- 172.21.15.82:0/4110604475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841419dff0 con 0x7f8414101950 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f84137fe700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8414100f90 0x7f8414198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.809+0000 7f840affd700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8414101950 0x7f8414199520 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00c9b0 tx=0x7f83fc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.810+0000 7f84117fa700 1 -- 172.21.15.82:0/4110604475 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc007b00 con 0x7f8414101950 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.810+0000 7f84117fa700 1 -- 172.21.15.82:0/4110604475 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83fc007ca0 con 0x7f8414101950 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.810+0000 7f8419f58700 1 -- 172.21.15.82:0/4110604475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841419e2e0 con 0x7f8414101950 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.810+0000 7f84117fa700 1 -- 172.21.15.82:0/4110604475 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc018660 con 0x7f8414101950 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.810+0000 7f8419f58700 1 -- 172.21.15.82:0/4110604475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84141a5b60 con 0x7f8414101950 2024-04-21T08:45:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.810+0000 7f8419f58700 1 -- 172.21.15.82:0/4110604475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8414106190 con 0x7f8414101950 2024-04-21T08:45:38.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.812+0000 7f84117fa700 1 -- 172.21.15.82:0/4110604475 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f83fc00f450 con 0x7f8414101950 2024-04-21T08:45:38.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.812+0000 7f84117fa700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f84000a3cd0 0x7f84000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:38.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.812+0000 7f84137fe700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f84000a3cd0 0x7f84000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:38.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.812+0000 7f84117fa700 1 -- 172.21.15.82:0/4110604475 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f83fc0d0d00 con 0x7f8414101950 2024-04-21T08:45:38.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.813+0000 7f84117fa700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351] conn(0x7f84000aedb0 0x7f84000b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:38.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.813+0000 7f84117fa700 1 -- 172.21.15.82:0/4110604475 --> [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84000b1920 con 0x7f84000aedb0 2024-04-21T08:45:38.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.813+0000 7f84117fa700 1 -- 172.21.15.82:0/4110604475 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f83fc0d1150 con 0x7f8414101950 2024-04-21T08:45:38.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.813+0000 7f84137fe700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f84000a3cd0 0x7f84000a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8404005fd0 tx=0x7f8404005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:38.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.813+0000 7f8413fff700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351] conn(0x7f84000aedb0 0x7f84000b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:38.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.813+0000 7f8413fff700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351] conn(0x7f84000aedb0 0x7f84000b1250 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:38.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.814+0000 7f84117fa700 1 -- 172.21.15.82:0/4110604475 <== osd.2 v2:172.21.15.134:6800/3690916351 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84000b1920 con 0x7f84000aedb0 2024-04-21T08:45:38.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.830+0000 7f8419f58700 1 -- 172.21.15.82:0/4110604475 --> [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f84141042b0 con 0x7f84000aedb0 2024-04-21T08:45:38.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.831+0000 7f84117fa700 1 -- 172.21.15.82:0/4110604475 <== osd.2 v2:172.21.15.134:6800/3690916351 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f84141042b0 con 0x7f84000aedb0 2024-04-21T08:45:38.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.831+0000 7f8419f58700 1 -- 172.21.15.82:0/4110604475 >> [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351] conn(0x7f84000aedb0 msgr2=0x7f84000b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:38.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.831+0000 7f8419f58700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351] conn(0x7f84000aedb0 0x7f84000b1250 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:38.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.831+0000 7f8419f58700 1 -- 172.21.15.82:0/4110604475 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f84000a3cd0 msgr2=0x7f84000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:38.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.831+0000 7f8419f58700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f84000a3cd0 0x7f84000a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8404005fd0 tx=0x7f8404005e30 comp rx=0 tx=0).stop 2024-04-21T08:45:38.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.831+0000 7f8419f58700 1 -- 172.21.15.82:0/4110604475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8414101950 msgr2=0x7f8414199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:38.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.831+0000 7f8419f58700 1 --2- 172.21.15.82:0/4110604475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8414101950 0x7f8414199520 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00c9b0 tx=0x7f83fc00ccc0 comp rx=0 tx=0).stop 2024-04-21T08:45:38.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.831+0000 7f8413fff700 1 -- 172.21.15.82:0/4110604475 reap_dead start 2024-04-21T08:45:38.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.831+0000 7f8419f58700 1 -- 172.21.15.82:0/4110604475 shutdown_connections 2024-04-21T08:45:38.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.831+0000 7f8419f58700 1 -- 172.21.15.82:0/4110604475 >> 172.21.15.82:0/4110604475 conn(0x7f8414077f40 msgr2=0x7f84141057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:38.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.831+0000 7f8419f58700 1 -- 172.21.15.82:0/4110604475 shutdown_connections 2024-04-21T08:45:38.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:38.831+0000 7f8419f58700 1 -- 172.21.15.82:0/4110604475 wait complete. 2024-04-21T08:45:39.241 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:39.242 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:39.243 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:39.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:39 smithi082 ceph-mon[27967]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:45:39.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:39 smithi134 ceph-mon[31772]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:45:39.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:39 smithi106 ceph-mon[31072]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:45:40.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.685+0000 7fec5c332700 1 -- 172.21.15.82:0/287669320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec54101d20 msgr2=0x7fec541021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:40.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.685+0000 7fec5c332700 1 --2- 172.21.15.82:0/287669320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec54101d20 0x7fec541021a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fec44009990 tx=0x7fec44009ca0 comp rx=0 tx=0).stop 2024-04-21T08:45:40.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 -- 172.21.15.82:0/287669320 shutdown_connections 2024-04-21T08:45:40.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 --2- 172.21.15.82:0/287669320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec541026e0 0x7fec5410e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:40.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 --2- 172.21.15.82:0/287669320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec54106990 0x7fec54106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:40.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 --2- 172.21.15.82:0/287669320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec54101d20 0x7fec541021a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:40.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 -- 172.21.15.82:0/287669320 >> 172.21.15.82:0/287669320 conn(0x7fec540788f0 msgr2=0x7fec54078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 -- 172.21.15.82:0/287669320 shutdown_connections 2024-04-21T08:45:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 -- 172.21.15.82:0/287669320 wait complete. 2024-04-21T08:45:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 Processor -- start 2024-04-21T08:45:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 -- start start 2024-04-21T08:45:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec54101d20 0x7fec54197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec541026e0 0x7fec54198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec54106990 0x7fec5419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fec54111ae0 con 0x7fec541026e0 2024-04-21T08:45:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec54111960 con 0x7fec54101d20 2024-04-21T08:45:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec5c332700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec54111c60 con 0x7fec54106990 2024-04-21T08:45:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec598cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec541026e0 0x7fec54198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec598cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec541026e0 0x7fec54198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49640/0 (socket says 172.21.15.82:49640) 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.686+0000 7fec598cd700 1 -- 172.21.15.82:0/958206155 learned_addr learned my addr 172.21.15.82:0/958206155 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec5a8cf700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec54106990 0x7fec5419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec5a0ce700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec54101d20 0x7fec54197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec598cd700 1 -- 172.21.15.82:0/958206155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec54106990 msgr2=0x7fec5419c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec598cd700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec54106990 0x7fec5419c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec598cd700 1 -- 172.21.15.82:0/958206155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec54101d20 msgr2=0x7fec54197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec598cd700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec54101d20 0x7fec54197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec598cd700 1 -- 172.21.15.82:0/958206155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5419cfe0 con 0x7fec541026e0 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec5a8cf700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec54106990 0x7fec5419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec598cd700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec541026e0 0x7fec54198510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fec440057f0 tx=0x7fec44003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec4b7fe700 1 -- 172.21.15.82:0/958206155 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4402a070 con 0x7fec541026e0 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec4b7fe700 1 -- 172.21.15.82:0/958206155 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec4401ad60 con 0x7fec541026e0 2024-04-21T08:45:40.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec5c332700 1 -- 172.21.15.82:0/958206155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5419d270 con 0x7fec541026e0 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec4b7fe700 1 -- 172.21.15.82:0/958206155 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44004170 con 0x7fec541026e0 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.687+0000 7fec5c332700 1 -- 172.21.15.82:0/958206155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5406b110 con 0x7fec541026e0 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.688+0000 7fec5c332700 1 -- 172.21.15.82:0/958206155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec5406b6c0 con 0x7fec541026e0 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.689+0000 7fec4b7fe700 1 -- 172.21.15.82:0/958206155 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fec4402d070 con 0x7fec541026e0 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.690+0000 7fec4b7fe700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fec400a3cd0 0x7fec400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.690+0000 7fec5a0ce700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fec400a3cd0 0x7fec400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.690+0000 7fec4b7fe700 1 -- 172.21.15.82:0/958206155 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fec440d42f0 con 0x7fec541026e0 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.690+0000 7fec4b7fe700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808] conn(0x7fec400aedb0 0x7fec400b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.690+0000 7fec4b7fe700 1 -- 172.21.15.82:0/958206155 --> [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec44011ec0 con 0x7fec400aedb0 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.690+0000 7fec4b7fe700 1 -- 172.21.15.82:0/958206155 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fec440d4740 con 0x7fec541026e0 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.690+0000 7fec5a0ce700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fec400a3cd0 0x7fec400a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fec50005fd0 tx=0x7fec5000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.690+0000 7fec5a8cf700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808] conn(0x7fec400aedb0 0x7fec400b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.691+0000 7fec5a8cf700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808] conn(0x7fec400aedb0 0x7fec400b1250 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:40.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.692+0000 7fec4b7fe700 1 -- 172.21.15.82:0/958206155 <== osd.10 v2:172.21.15.106:6824/204007808 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec44011ec0 con 0x7fec400aedb0 2024-04-21T08:45:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.709+0000 7fec5c332700 1 -- 172.21.15.82:0/958206155 --> [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec5406bb40 con 0x7fec400aedb0 2024-04-21T08:45:40.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.709+0000 7fec4b7fe700 1 -- 172.21.15.82:0/958206155 <== osd.10 v2:172.21.15.106:6824/204007808 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fec5406bb40 con 0x7fec400aedb0 2024-04-21T08:45:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.709+0000 7fec5c332700 1 -- 172.21.15.82:0/958206155 >> [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808] conn(0x7fec400aedb0 msgr2=0x7fec400b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.709+0000 7fec5c332700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808] conn(0x7fec400aedb0 0x7fec400b1250 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:40.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.709+0000 7fec5c332700 1 -- 172.21.15.82:0/958206155 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fec400a3cd0 msgr2=0x7fec400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.710+0000 7fec5c332700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fec400a3cd0 0x7fec400a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fec50005fd0 tx=0x7fec5000a560 comp rx=0 tx=0).stop 2024-04-21T08:45:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.710+0000 7fec5c332700 1 -- 172.21.15.82:0/958206155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec541026e0 msgr2=0x7fec54198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.710+0000 7fec5c332700 1 --2- 172.21.15.82:0/958206155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec541026e0 0x7fec54198510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fec440057f0 tx=0x7fec44003730 comp rx=0 tx=0).stop 2024-04-21T08:45:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.710+0000 7fec5a8cf700 1 -- 172.21.15.82:0/958206155 reap_dead start 2024-04-21T08:45:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.710+0000 7fec5c332700 1 -- 172.21.15.82:0/958206155 shutdown_connections 2024-04-21T08:45:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.710+0000 7fec5c332700 1 -- 172.21.15.82:0/958206155 >> 172.21.15.82:0/958206155 conn(0x7fec540788f0 msgr2=0x7fec540ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.710+0000 7fec5c332700 1 -- 172.21.15.82:0/958206155 shutdown_connections 2024-04-21T08:45:40.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:40.710+0000 7fec5c332700 1 -- 172.21.15.82:0/958206155 wait complete. 2024-04-21T08:45:41.569 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:41 smithi082 ceph-mon[27967]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:45:41.679 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:41 smithi106 ceph-mon[31072]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:45:41.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:41 smithi134 ceph-mon[31772]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:45:42.208 INFO:teuthology.orchestra.run.smithi082.stdout:77309411401 2024-04-21T08:45:42.208 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-21T08:45:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 -- 172.21.15.82:0/2025107957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c074c70 msgr2=0x7fa18c075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 --2- 172.21.15.82:0/2025107957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c074c70 0x7fa18c075050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa188009990 tx=0x7fa188009ca0 comp rx=0 tx=0).stop 2024-04-21T08:45:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 -- 172.21.15.82:0/2025107957 shutdown_connections 2024-04-21T08:45:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 --2- 172.21.15.82:0/2025107957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa18c113c10 0x7fa18c116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 --2- 172.21.15.82:0/2025107957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c074c70 0x7fa18c075050 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 --2- 172.21.15.82:0/2025107957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa18c075590 0x7fa18c1136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 -- 172.21.15.82:0/2025107957 >> 172.21.15.82:0/2025107957 conn(0x7fa18c079ba0 msgr2=0x7fa18c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 -- 172.21.15.82:0/2025107957 shutdown_connections 2024-04-21T08:45:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 -- 172.21.15.82:0/2025107957 wait complete. 2024-04-21T08:45:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 Processor -- start 2024-04-21T08:45:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 -- start start 2024-04-21T08:45:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa18c074c70 0x7fa18c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.671+0000 7fa194324700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa18c075590 0x7fa18c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa194324700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c113c10 0x7fa18c1a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa194324700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c1191f0 con 0x7fa18c113c10 2024-04-21T08:45:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa194324700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c119070 con 0x7fa18c074c70 2024-04-21T08:45:43.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa194324700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa18c119370 con 0x7fa18c075590 2024-04-21T08:45:43.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1928c1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c113c10 0x7fa18c1a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:43.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1928c1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c113c10 0x7fa18c1a3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49656/0 (socket says 172.21.15.82:49656) 2024-04-21T08:45:43.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1928c1700 1 -- 172.21.15.82:0/2894162376 learned_addr learned my addr 172.21.15.82:0/2894162376 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:43.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1918bf700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa18c075590 0x7fa18c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:43.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1920c0700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa18c074c70 0x7fa18c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:43.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1928c1700 1 -- 172.21.15.82:0/2894162376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa18c075590 msgr2=0x7fa18c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1928c1700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa18c075590 0x7fa18c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1928c1700 1 -- 172.21.15.82:0/2894162376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa18c074c70 msgr2=0x7fa18c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1928c1700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa18c074c70 0x7fa18c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1928c1700 1 -- 172.21.15.82:0/2894162376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18c1a4730 con 0x7fa18c113c10 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1928c1700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c113c10 0x7fa18c1a3ff0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa18400ef90 tx=0x7fa18400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1837fe700 1 -- 172.21.15.82:0/2894162376 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1840092e0 con 0x7fa18c113c10 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1837fe700 1 -- 172.21.15.82:0/2894162376 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa184004bb0 con 0x7fa18c113c10 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa194324700 1 -- 172.21.15.82:0/2894162376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18c1a49c0 con 0x7fa18c113c10 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa1837fe700 1 -- 172.21.15.82:0/2894162376 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184004280 con 0x7fa18c113c10 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.672+0000 7fa194324700 1 -- 172.21.15.82:0/2894162376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa18c1ac2a0 con 0x7fa18c113c10 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.673+0000 7fa194324700 1 -- 172.21.15.82:0/2894162376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa18c113510 con 0x7fa18c113c10 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.674+0000 7fa1837fe700 1 -- 172.21.15.82:0/2894162376 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa18401b030 con 0x7fa18c113c10 2024-04-21T08:45:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.675+0000 7fa1837fe700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa1780a3cd0 0x7fa1780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.675+0000 7fa1920c0700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa1780a3cd0 0x7fa1780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.675+0000 7fa1837fe700 1 -- 172.21.15.82:0/2894162376 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fa1840d0d00 con 0x7fa18c113c10 2024-04-21T08:45:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.675+0000 7fa1837fe700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970] conn(0x7fa1780aedb0 0x7fa1780b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.675+0000 7fa1837fe700 1 -- 172.21.15.82:0/2894162376 --> [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa184005ab0 con 0x7fa1780aedb0 2024-04-21T08:45:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.675+0000 7fa1837fe700 1 -- 172.21.15.82:0/2894162376 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fa1840d1150 con 0x7fa18c113c10 2024-04-21T08:45:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.675+0000 7fa1920c0700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa1780a3cd0 0x7fa1780a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa18800b2e0 tx=0x7fa188005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.676+0000 7fa1918bf700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970] conn(0x7fa1780aedb0 0x7fa1780b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.676+0000 7fa1918bf700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970] conn(0x7fa1780aedb0 0x7fa1780b1250 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.677+0000 7fa1837fe700 1 -- 172.21.15.82:0/2894162376 <== osd.1 v2:172.21.15.106:6800/1176470970 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa184005ab0 con 0x7fa1780aedb0 2024-04-21T08:45:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.693+0000 7fa194324700 1 -- 172.21.15.82:0/2894162376 --> [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa18c076360 con 0x7fa1780aedb0 2024-04-21T08:45:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.694+0000 7fa1837fe700 1 -- 172.21.15.82:0/2894162376 <== osd.1 v2:172.21.15.106:6800/1176470970 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa18c076360 con 0x7fa1780aedb0 2024-04-21T08:45:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.694+0000 7fa194324700 1 -- 172.21.15.82:0/2894162376 >> [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970] conn(0x7fa1780aedb0 msgr2=0x7fa1780b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.694+0000 7fa194324700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970] conn(0x7fa1780aedb0 0x7fa1780b1250 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.694+0000 7fa194324700 1 -- 172.21.15.82:0/2894162376 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa1780a3cd0 msgr2=0x7fa1780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.694+0000 7fa194324700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa1780a3cd0 0x7fa1780a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa18800b2e0 tx=0x7fa188005960 comp rx=0 tx=0).stop 2024-04-21T08:45:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.694+0000 7fa194324700 1 -- 172.21.15.82:0/2894162376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c113c10 msgr2=0x7fa18c1a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.694+0000 7fa194324700 1 --2- 172.21.15.82:0/2894162376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c113c10 0x7fa18c1a3ff0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa18400ef90 tx=0x7fa18400c3a0 comp rx=0 tx=0).stop 2024-04-21T08:45:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.694+0000 7fa1928c1700 1 -- 172.21.15.82:0/2894162376 reap_dead start 2024-04-21T08:45:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.694+0000 7fa194324700 1 -- 172.21.15.82:0/2894162376 shutdown_connections 2024-04-21T08:45:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.694+0000 7fa194324700 1 -- 172.21.15.82:0/2894162376 >> 172.21.15.82:0/2894162376 conn(0x7fa18c079ba0 msgr2=0x7fa18c112ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.694+0000 7fa194324700 1 -- 172.21.15.82:0/2894162376 shutdown_connections 2024-04-21T08:45:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:41.694+0000 7fa194324700 1 -- 172.21.15.82:0/2894162376 wait complete. 2024-04-21T08:45:43.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:43 smithi134 ceph-mon[31772]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:45:43.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:43 smithi082 ceph-mon[27967]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:45:43.764 INFO:teuthology.orchestra.run.smithi082.stdout:55834574925 2024-04-21T08:45:43.764 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-21T08:45:43.873 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:43 smithi106 ceph-mon[31072]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:45:44.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.811+0000 7f474b3c5700 1 -- 172.21.15.82:0/2698066566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4744108a50 msgr2=0x7f4744108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:44.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.811+0000 7f474b3c5700 1 --2- 172.21.15.82:0/2698066566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4744108a50 0x7f4744108e30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4740009990 tx=0x7f4740009ca0 comp rx=0 tx=0).stop 2024-04-21T08:45:44.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.811+0000 7f474b3c5700 1 -- 172.21.15.82:0/2698066566 shutdown_connections 2024-04-21T08:45:44.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.811+0000 7f474b3c5700 1 --2- 172.21.15.82:0/2698066566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4744103450 0x7f474410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:44.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.811+0000 7f474b3c5700 1 --2- 172.21.15.82:0/2698066566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4744108a50 0x7f4744108e30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:44.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.811+0000 7f474b3c5700 1 --2- 172.21.15.82:0/2698066566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4744102a90 0x7f4744102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:44.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.811+0000 7f474b3c5700 1 -- 172.21.15.82:0/2698066566 >> 172.21.15.82:0/2698066566 conn(0x7f4744079ba0 msgr2=0x7f4744079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:44.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.812+0000 7f474b3c5700 1 -- 172.21.15.82:0/2698066566 shutdown_connections 2024-04-21T08:45:44.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.812+0000 7f474b3c5700 1 -- 172.21.15.82:0/2698066566 wait complete. 2024-04-21T08:45:44.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.812+0000 7f474b3c5700 1 Processor -- start 2024-04-21T08:45:44.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.812+0000 7f474b3c5700 1 -- start start 2024-04-21T08:45:44.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.812+0000 7f474b3c5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4744102a90 0x7f4744074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.812+0000 7f474b3c5700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4744103450 0x7f4744075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.812+0000 7f474b3c5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4744108a50 0x7f47440757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.812+0000 7f474b3c5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4744112c80 con 0x7f4744102a90 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.812+0000 7f474b3c5700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4744112b00 con 0x7f4744103450 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.812+0000 7f474b3c5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4744112e00 con 0x7f4744108a50 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.812+0000 7f4749161700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4744102a90 0x7f4744074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f4749161700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4744102a90 0x7f4744074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50992/0 (socket says 172.21.15.82:50992) 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f4749161700 1 -- 172.21.15.82:0/3900671537 learned_addr learned my addr 172.21.15.82:0/3900671537 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f4748960700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4744103450 0x7f4744075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f4749962700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4744108a50 0x7f47440757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f4749161700 1 -- 172.21.15.82:0/3900671537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4744108a50 msgr2=0x7f47440757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f4749161700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4744108a50 0x7f47440757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:44.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f4749161700 1 -- 172.21.15.82:0/3900671537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4744103450 msgr2=0x7f4744075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:44.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f4749161700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4744103450 0x7f4744075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:44.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f4749161700 1 -- 172.21.15.82:0/3900671537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47441a76a0 con 0x7f4744102a90 2024-04-21T08:45:44.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f4749161700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4744102a90 0x7f4744074d40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f47400057f0 tx=0x7f4740003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:44.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f473a7fc700 1 -- 172.21.15.82:0/3900671537 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740027070 con 0x7f4744102a90 2024-04-21T08:45:44.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f473a7fc700 1 -- 172.21.15.82:0/3900671537 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4740004020 con 0x7f4744102a90 2024-04-21T08:45:44.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f474b3c5700 1 -- 172.21.15.82:0/3900671537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47441a7930 con 0x7f4744102a90 2024-04-21T08:45:44.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f473a7fc700 1 -- 172.21.15.82:0/3900671537 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740022d60 con 0x7f4744102a90 2024-04-21T08:45:44.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.813+0000 7f474b3c5700 1 -- 172.21.15.82:0/3900671537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47441a7e60 con 0x7f4744102a90 2024-04-21T08:45:44.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.814+0000 7f474b3c5700 1 -- 172.21.15.82:0/3900671537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4744063110 con 0x7f4744102a90 2024-04-21T08:45:44.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.815+0000 7f473a7fc700 1 -- 172.21.15.82:0/3900671537 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f474002a070 con 0x7f4744102a90 2024-04-21T08:45:44.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.816+0000 7f473a7fc700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f47300a3d50 0x7f47300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:44.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.816+0000 7f4748960700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f47300a3d50 0x7f47300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:44.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.816+0000 7f473a7fc700 1 -- 172.21.15.82:0/3900671537 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f47400d2160 con 0x7f4744102a90 2024-04-21T08:45:44.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.816+0000 7f473a7fc700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646] conn(0x7f47300aee30 0x7f47300b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:44.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.816+0000 7f473a7fc700 1 -- 172.21.15.82:0/3900671537 --> [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47300b19a0 con 0x7f47300aee30 2024-04-21T08:45:44.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.816+0000 7f473a7fc700 1 -- 172.21.15.82:0/3900671537 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f47400d25b0 con 0x7f4744102a90 2024-04-21T08:45:44.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.816+0000 7f4749962700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646] conn(0x7f47300aee30 0x7f47300b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:44.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.816+0000 7f4748960700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f47300a3d50 0x7f47300a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f474407a280 tx=0x7f473400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:44.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.817+0000 7f4749962700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646] conn(0x7f47300aee30 0x7f47300b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:44.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.817+0000 7f473a7fc700 1 -- 172.21.15.82:0/3900671537 <== osd.9 v2:172.21.15.82:6826/112604646 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47300b19a0 con 0x7f47300aee30 2024-04-21T08:45:44.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.833+0000 7f474b3c5700 1 -- 172.21.15.82:0/3900671537 --> [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4744063510 con 0x7f47300aee30 2024-04-21T08:45:44.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.833+0000 7f473a7fc700 1 -- 172.21.15.82:0/3900671537 <== osd.9 v2:172.21.15.82:6826/112604646 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4744063510 con 0x7f47300aee30 2024-04-21T08:45:44.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.834+0000 7f474b3c5700 1 -- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646] conn(0x7f47300aee30 msgr2=0x7f47300b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:44.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.834+0000 7f474b3c5700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646] conn(0x7f47300aee30 0x7f47300b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:44.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.834+0000 7f474b3c5700 1 -- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f47300a3d50 msgr2=0x7f47300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:44.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.834+0000 7f474b3c5700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f47300a3d50 0x7f47300a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f474407a280 tx=0x7f473400a300 comp rx=0 tx=0).stop 2024-04-21T08:45:44.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.834+0000 7f474b3c5700 1 -- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4744102a90 msgr2=0x7f4744074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:44.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.834+0000 7f474b3c5700 1 --2- 172.21.15.82:0/3900671537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4744102a90 0x7f4744074d40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f47400057f0 tx=0x7f4740003680 comp rx=0 tx=0).stop 2024-04-21T08:45:44.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.834+0000 7f4749962700 1 -- 172.21.15.82:0/3900671537 reap_dead start 2024-04-21T08:45:44.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.834+0000 7f474b3c5700 1 -- 172.21.15.82:0/3900671537 shutdown_connections 2024-04-21T08:45:44.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.834+0000 7f474b3c5700 1 -- 172.21.15.82:0/3900671537 >> 172.21.15.82:0/3900671537 conn(0x7f4744079ba0 msgr2=0x7f47441018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:44.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.834+0000 7f474b3c5700 1 -- 172.21.15.82:0/3900671537 shutdown_connections 2024-04-21T08:45:44.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:44.834+0000 7f474b3c5700 1 -- 172.21.15.82:0/3900671537 wait complete. 2024-04-21T08:45:45.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:45 smithi082 ceph-mon[27967]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:45:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:45 smithi134 ceph-mon[31772]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:45:45.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:45 smithi106 ceph-mon[31072]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:45:45.947 INFO:teuthology.orchestra.run.smithi082.stdout:309237645345 2024-04-21T08:45:45.947 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-21T08:45:46.685 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:47.110 INFO:teuthology.orchestra.run.smithi082.stdout:287762808871 2024-04-21T08:45:47.110 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-21T08:45:47.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.348+0000 7fb266f39700 1 -- 172.21.15.82:0/813280723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb260102330 msgr2=0x7fb26010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.348+0000 7fb266f39700 1 --2- 172.21.15.82:0/813280723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb260102330 0x7fb26010e7f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb250009990 tx=0x7fb250009ca0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.348+0000 7fb266f39700 1 -- 172.21.15.82:0/813280723 shutdown_connections 2024-04-21T08:45:47.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.348+0000 7fb266f39700 1 --2- 172.21.15.82:0/813280723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb260102330 0x7fb26010e7f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.348+0000 7fb266f39700 1 --2- 172.21.15.82:0/813280723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb260107920 0x7fb260107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.348+0000 7fb266f39700 1 --2- 172.21.15.82:0/813280723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb260101970 0x7fb260101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.348+0000 7fb266f39700 1 -- 172.21.15.82:0/813280723 >> 172.21.15.82:0/813280723 conn(0x7fb260078910 msgr2=0x7fb260078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.348+0000 7fb266f39700 1 -- 172.21.15.82:0/813280723 shutdown_connections 2024-04-21T08:45:47.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.348+0000 7fb266f39700 1 -- 172.21.15.82:0/813280723 wait complete. 2024-04-21T08:45:47.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.348+0000 7fb266f39700 1 Processor -- start 2024-04-21T08:45:47.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.348+0000 7fb266f39700 1 -- start start 2024-04-21T08:45:47.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb266f39700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb260101970 0x7fb26019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb266f39700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb260102330 0x7fb26019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb266f39700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb260107920 0x7fb2601a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb266f39700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb260111bc0 con 0x7fb260107920 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb266f39700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb260111a40 con 0x7fb260102330 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb266f39700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb260111d40 con 0x7fb260101970 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb2654d6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb260107920 0x7fb2601a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb2654d6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb260107920 0x7fb2601a5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51022/0 (socket says 172.21.15.82:51022) 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb2654d6700 1 -- 172.21.15.82:0/2488826311 learned_addr learned my addr 172.21.15.82:0/2488826311 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb25ffff700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb260102330 0x7fb26019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb264cd5700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb260101970 0x7fb26019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb2654d6700 1 -- 172.21.15.82:0/2488826311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb260101970 msgr2=0x7fb26019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb2654d6700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb260101970 0x7fb26019a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb2654d6700 1 -- 172.21.15.82:0/2488826311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb260102330 msgr2=0x7fb26019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb2654d6700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb260102330 0x7fb26019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb2654d6700 1 -- 172.21.15.82:0/2488826311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2601a5660 con 0x7fb260107920 2024-04-21T08:45:47.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb25ffff700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb260102330 0x7fb26019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:47.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb2654d6700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb260107920 0x7fb2601a5120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb250005910 tx=0x7fb2500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb25dffb700 1 -- 172.21.15.82:0/2488826311 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250027070 con 0x7fb260107920 2024-04-21T08:45:47.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.349+0000 7fb25dffb700 1 -- 172.21.15.82:0/2488826311 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb250004020 con 0x7fb260107920 2024-04-21T08:45:47.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.350+0000 7fb25dffb700 1 -- 172.21.15.82:0/2488826311 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250022d40 con 0x7fb260107920 2024-04-21T08:45:47.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.350+0000 7fb266f39700 1 -- 172.21.15.82:0/2488826311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2601a5830 con 0x7fb260107920 2024-04-21T08:45:47.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.350+0000 7fb266f39700 1 -- 172.21.15.82:0/2488826311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2601a5c30 con 0x7fb260107920 2024-04-21T08:45:47.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.351+0000 7fb266f39700 1 -- 172.21.15.82:0/2488826311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb260069730 con 0x7fb260107920 2024-04-21T08:45:47.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.352+0000 7fb25dffb700 1 -- 172.21.15.82:0/2488826311 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb25002a070 con 0x7fb260107920 2024-04-21T08:45:47.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.352+0000 7fb25dffb700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb2480a3d50 0x7fb2480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:47.273 INFO:teuthology.orchestra.run.smithi082.stdout:343597383708 2024-04-21T08:45:47.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.352+0000 7fb264cd5700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb2480a3d50 0x7fb2480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:47.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.352+0000 7fb25dffb700 1 -- 172.21.15.82:0/2488826311 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fb2500d1f20 con 0x7fb260107920 2024-04-21T08:45:47.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.353+0000 7fb264cd5700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb2480a3d50 0x7fb2480a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb258007dc0 tx=0x7fb2580072f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:47.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.356+0000 7fb25dffb700 1 -- 172.21.15.82:0/2488826311 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb250031070 con 0x7fb260107920 2024-04-21T08:45:47.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.565+0000 7fb266f39700 1 -- 172.21.15.82:0/2488826311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb26002e140 con 0x7fb260107920 2024-04-21T08:45:47.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.565+0000 7fb25dffb700 1 -- 172.21.15.82:0/2488826311 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb250013070 con 0x7fb260107920 2024-04-21T08:45:47.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.567+0000 7fb266f39700 1 -- 172.21.15.82:0/2488826311 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb2480a3d50 msgr2=0x7fb2480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.567+0000 7fb266f39700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb2480a3d50 0x7fb2480a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb258007dc0 tx=0x7fb2580072f0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.567+0000 7fb266f39700 1 -- 172.21.15.82:0/2488826311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb260107920 msgr2=0x7fb2601a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:47.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.567+0000 7fb266f39700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb260107920 0x7fb2601a5120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb250005910 tx=0x7fb2500047b0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.567+0000 7fb266f39700 1 -- 172.21.15.82:0/2488826311 shutdown_connections 2024-04-21T08:45:47.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.567+0000 7fb266f39700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb2480a3d50 0x7fb2480a6210 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.567+0000 7fb266f39700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb260107920 0x7fb2601a5120 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.567+0000 7fb266f39700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb260101970 0x7fb26019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.567+0000 7fb266f39700 1 --2- 172.21.15.82:0/2488826311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb260102330 0x7fb26019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:47.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.567+0000 7fb266f39700 1 -- 172.21.15.82:0/2488826311 >> 172.21.15.82:0/2488826311 conn(0x7fb260078910 msgr2=0x7fb2600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:47.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.567+0000 7fb266f39700 1 -- 172.21.15.82:0/2488826311 shutdown_connections 2024-04-21T08:45:47.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:46.567+0000 7fb266f39700 1 -- 172.21.15.82:0/2488826311 wait complete. 2024-04-21T08:45:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:47 smithi134 ceph-mon[31772]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:45:47.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:47 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2488826311' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T08:45:47.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:47 smithi082 ceph-mon[27967]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:45:47.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:47 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2488826311' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T08:45:47.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:47 smithi106 ceph-mon[31072]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:45:47.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:47 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2488826311' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T08:45:48.026 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383705 got 343597383708 for osd.11 2024-04-21T08:45:48.027 DEBUG:teuthology.parallel:result is None 2024-04-21T08:45:48.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.283+0000 7fbb8c31b700 1 -- 172.21.15.82:0/1491523541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb84108a80 msgr2=0x7fbb84108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:48.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.283+0000 7fbb8c31b700 1 --2- 172.21.15.82:0/1491523541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb84108a80 0x7fbb84108e60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009990 tx=0x7fbb80009ca0 comp rx=0 tx=0).stop 2024-04-21T08:45:48.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.283+0000 7fbb8c31b700 1 -- 172.21.15.82:0/1491523541 shutdown_connections 2024-04-21T08:45:48.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.283+0000 7fbb8c31b700 1 --2- 172.21.15.82:0/1491523541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb841034b0 0x7fbb8410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:48.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.283+0000 7fbb8c31b700 1 --2- 172.21.15.82:0/1491523541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb84108a80 0x7fbb84108e60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:48.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.283+0000 7fbb8c31b700 1 --2- 172.21.15.82:0/1491523541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb84102af0 0x7fbb84102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:48.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.283+0000 7fbb8c31b700 1 -- 172.21.15.82:0/1491523541 >> 172.21.15.82:0/1491523541 conn(0x7fbb84079ad0 msgr2=0x7fbb84079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.283+0000 7fbb8c31b700 1 -- 172.21.15.82:0/1491523541 shutdown_connections 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.283+0000 7fbb8c31b700 1 -- 172.21.15.82:0/1491523541 wait complete. 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.284+0000 7fbb8c31b700 1 Processor -- start 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.284+0000 7fbb8c31b700 1 -- start start 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.284+0000 7fbb8c31b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb84102af0 0x7fbb841a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.284+0000 7fbb8c31b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb841034b0 0x7fbb841a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.284+0000 7fbb8c31b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb84108a80 0x7fbb8410c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.284+0000 7fbb8c31b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84112c30 con 0x7fbb84102af0 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.284+0000 7fbb8c31b700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84112ab0 con 0x7fbb84108a80 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.284+0000 7fbb8c31b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84112db0 con 0x7fbb841034b0 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb8a0b7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb84102af0 0x7fbb841a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb8a0b7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb84102af0 0x7fbb841a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51030/0 (socket says 172.21.15.82:51030) 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb8a0b7700 1 -- 172.21.15.82:0/435356431 learned_addr learned my addr 172.21.15.82:0/435356431 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb898b6700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb841034b0 0x7fbb841a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb8a8b8700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb84108a80 0x7fbb8410c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb8a0b7700 1 -- 172.21.15.82:0/435356431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb841034b0 msgr2=0x7fbb841a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb8a0b7700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb841034b0 0x7fbb841a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb8a0b7700 1 -- 172.21.15.82:0/435356431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb84108a80 msgr2=0x7fbb8410c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb8a0b7700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb84108a80 0x7fbb8410c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb8a0b7700 1 -- 172.21.15.82:0/435356431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8410cfe0 con 0x7fbb84102af0 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb8a0b7700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb84102af0 0x7fbb841a4320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbb80000c00 tx=0x7fbb800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb7b7fe700 1 -- 172.21.15.82:0/435356431 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80027070 con 0x7fbb84102af0 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb7b7fe700 1 -- 172.21.15.82:0/435356431 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb80003f20 con 0x7fbb84102af0 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb8c31b700 1 -- 172.21.15.82:0/435356431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8410d270 con 0x7fbb84102af0 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.285+0000 7fbb7b7fe700 1 -- 172.21.15.82:0/435356431 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80022cf0 con 0x7fbb84102af0 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.286+0000 7fbb8c31b700 1 -- 172.21.15.82:0/435356431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb841ac250 con 0x7fbb84102af0 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.286+0000 7fbb8c31b700 1 -- 172.21.15.82:0/435356431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb84101db0 con 0x7fbb84102af0 2024-04-21T08:45:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.288+0000 7fbb7b7fe700 1 -- 172.21.15.82:0/435356431 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbb8002a070 con 0x7fbb84102af0 2024-04-21T08:45:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.289+0000 7fbb7b7fe700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbb700a3d50 0x7fbb700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.289+0000 7fbb898b6700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbb700a3d50 0x7fbb700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.289+0000 7fbb7b7fe700 1 -- 172.21.15.82:0/435356431 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fbb800d1ca0 con 0x7fbb84102af0 2024-04-21T08:45:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.289+0000 7fbb7b7fe700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096] conn(0x7fbb700aee30 0x7fbb700b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.289+0000 7fbb7b7fe700 1 -- 172.21.15.82:0/435356431 --> [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb800095d0 con 0x7fbb700aee30 2024-04-21T08:45:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.289+0000 7fbb7b7fe700 1 -- 172.21.15.82:0/435356431 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fbb800d2050 con 0x7fbb84102af0 2024-04-21T08:45:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.289+0000 7fbb898b6700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbb700a3d50 0x7fbb700a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb8410da10 tx=0x7fbb7400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.289+0000 7fbb8a8b8700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096] conn(0x7fbb700aee30 0x7fbb700b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.290+0000 7fbb8a8b8700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096] conn(0x7fbb700aee30 0x7fbb700b12d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.291+0000 7fbb7b7fe700 1 -- 172.21.15.82:0/435356431 <== osd.5 v2:172.21.15.134:6808/1529948096 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb800095d0 con 0x7fbb700aee30 2024-04-21T08:45:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.307+0000 7fbb8c31b700 1 -- 172.21.15.82:0/435356431 --> [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb841041b0 con 0x7fbb700aee30 2024-04-21T08:45:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.308+0000 7fbb7b7fe700 1 -- 172.21.15.82:0/435356431 <== osd.5 v2:172.21.15.134:6808/1529948096 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbb841041b0 con 0x7fbb700aee30 2024-04-21T08:45:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.308+0000 7fbb8c31b700 1 -- 172.21.15.82:0/435356431 >> [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096] conn(0x7fbb700aee30 msgr2=0x7fbb700b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.308+0000 7fbb8c31b700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096] conn(0x7fbb700aee30 0x7fbb700b12d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.308+0000 7fbb8c31b700 1 -- 172.21.15.82:0/435356431 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbb700a3d50 msgr2=0x7fbb700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.308+0000 7fbb8c31b700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbb700a3d50 0x7fbb700a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb8410da10 tx=0x7fbb7400a380 comp rx=0 tx=0).stop 2024-04-21T08:45:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.308+0000 7fbb8c31b700 1 -- 172.21.15.82:0/435356431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb84102af0 msgr2=0x7fbb841a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.308+0000 7fbb8c31b700 1 --2- 172.21.15.82:0/435356431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb84102af0 0x7fbb841a4320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbb80000c00 tx=0x7fbb800047b0 comp rx=0 tx=0).stop 2024-04-21T08:45:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.308+0000 7fbb8a8b8700 1 -- 172.21.15.82:0/435356431 reap_dead start 2024-04-21T08:45:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.308+0000 7fbb8c31b700 1 -- 172.21.15.82:0/435356431 shutdown_connections 2024-04-21T08:45:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.308+0000 7fbb8c31b700 1 -- 172.21.15.82:0/435356431 >> 172.21.15.82:0/435356431 conn(0x7fbb84079ad0 msgr2=0x7fbb841013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.308+0000 7fbb8c31b700 1 -- 172.21.15.82:0/435356431 shutdown_connections 2024-04-21T08:45:48.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:47.308+0000 7fbb8c31b700 1 -- 172.21.15.82:0/435356431 wait complete. 2024-04-21T08:45:49.556 INFO:teuthology.orchestra.run.smithi082.stdout:163208757308 2024-04-21T08:45:49.557 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-21T08:45:49.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:49 smithi082 ceph-mon[27967]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:45:49.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:49 smithi134 ceph-mon[31772]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:45:49.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:49 smithi106 ceph-mon[31072]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:45:50.735 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:50.737 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:50.738 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:50.738 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:51.086 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.090+0000 7fdde98ef700 1 -- 172.21.15.82:0/2978589704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde4108ad0 msgr2=0x7fdde4108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.090+0000 7fdde98ef700 1 --2- 172.21.15.82:0/2978589704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde4108ad0 0x7fdde4108eb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009990 tx=0x7fddd4009ca0 comp rx=0 tx=0).stop 2024-04-21T08:45:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.090+0000 7fdde98ef700 1 -- 172.21.15.82:0/2978589704 shutdown_connections 2024-04-21T08:45:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.090+0000 7fdde98ef700 1 --2- 172.21.15.82:0/2978589704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdde41019d0 0x7fdde410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.090+0000 7fdde98ef700 1 --2- 172.21.15.82:0/2978589704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde4108ad0 0x7fdde4108eb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.090+0000 7fdde98ef700 1 --2- 172.21.15.82:0/2978589704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde4101030 0x7fdde4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.090+0000 7fdde98ef700 1 -- 172.21.15.82:0/2978589704 >> 172.21.15.82:0/2978589704 conn(0x7fdde4077fb0 msgr2=0x7fdde40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.090+0000 7fdde98ef700 1 -- 172.21.15.82:0/2978589704 shutdown_connections 2024-04-21T08:45:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.090+0000 7fdde98ef700 1 -- 172.21.15.82:0/2978589704 wait complete. 2024-04-21T08:45:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.091+0000 7fdde98ef700 1 Processor -- start 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.091+0000 7fdde98ef700 1 -- start start 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.091+0000 7fdde98ef700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde4101030 0x7fdde4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.091+0000 7fdde98ef700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdde41019d0 0x7fdde41996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.091+0000 7fdde98ef700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde4108ad0 0x7fdde419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.091+0000 7fdde98ef700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4112c40 con 0x7fdde4101030 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.091+0000 7fdde98ef700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4112ac0 con 0x7fdde41019d0 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.091+0000 7fdde98ef700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdde4112dc0 con 0x7fdde4108ad0 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.092+0000 7fdde2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde4101030 0x7fdde4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.092+0000 7fdde37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde4108ad0 0x7fdde419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.092+0000 7fdde37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde4108ad0 0x7fdde419da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:47000/0 (socket says 172.21.15.82:47000) 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.092+0000 7fdde37fe700 1 -- 172.21.15.82:0/3149323446 learned_addr learned my addr 172.21.15.82:0/3149323446 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.092+0000 7fdde27fc700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdde41019d0 0x7fdde41996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.092+0000 7fdde37fe700 1 -- 172.21.15.82:0/3149323446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdde41019d0 msgr2=0x7fdde41996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.092+0000 7fdde37fe700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdde41019d0 0x7fdde41996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.092+0000 7fdde37fe700 1 -- 172.21.15.82:0/3149323446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde4101030 msgr2=0x7fdde4199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.092+0000 7fdde37fe700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde4101030 0x7fdde4199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.092+0000 7fdde37fe700 1 -- 172.21.15.82:0/3149323446 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde419e180 con 0x7fdde4108ad0 2024-04-21T08:45:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.092+0000 7fdde37fe700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde4108ad0 0x7fdde419da40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00b970 tx=0x7fdddc00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.093+0000 7fdde88ed700 1 -- 172.21.15.82:0/3149323446 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc00d860 con 0x7fdde4108ad0 2024-04-21T08:45:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.093+0000 7fdde98ef700 1 -- 172.21.15.82:0/3149323446 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde419e470 con 0x7fdde4108ad0 2024-04-21T08:45:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.093+0000 7fdde98ef700 1 -- 172.21.15.82:0/3149323446 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde41a5cf0 con 0x7fdde4108ad0 2024-04-21T08:45:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.093+0000 7fdde88ed700 1 -- 172.21.15.82:0/3149323446 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdddc00e430 con 0x7fdde4108ad0 2024-04-21T08:45:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.093+0000 7fdde88ed700 1 -- 172.21.15.82:0/3149323446 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc017660 con 0x7fdde4108ad0 2024-04-21T08:45:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.094+0000 7fdde98ef700 1 -- 172.21.15.82:0/3149323446 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde4104350 con 0x7fdde4108ad0 2024-04-21T08:45:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.095+0000 7fdde88ed700 1 -- 172.21.15.82:0/3149323446 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdddc00da00 con 0x7fdde4108ad0 2024-04-21T08:45:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.096+0000 7fdde88ed700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fddcc0a3d50 0x7fddcc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.096+0000 7fdde2ffd700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fddcc0a3d50 0x7fddcc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.096+0000 7fdde88ed700 1 -- 172.21.15.82:0/3149323446 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fdddc0d0610 con 0x7fdde4108ad0 2024-04-21T08:45:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.097+0000 7fdde2ffd700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fddcc0a3d50 0x7fddcc0a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fddd400b5c0 tx=0x7fddd4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:51.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.100+0000 7fdde88ed700 1 -- 172.21.15.82:0/3149323446 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdddc013030 con 0x7fdde4108ad0 2024-04-21T08:45:51.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.318+0000 7fdde98ef700 1 -- 172.21.15.82:0/3149323446 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdde402df80 con 0x7fdde4108ad0 2024-04-21T08:45:51.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.318+0000 7fdde88ed700 1 -- 172.21.15.82:0/3149323446 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdddc09cb70 con 0x7fdde4108ad0 2024-04-21T08:45:51.318 INFO:teuthology.orchestra.run.smithi082.stdout:197568495670 2024-04-21T08:45:51.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.321+0000 7fdde98ef700 1 -- 172.21.15.82:0/3149323446 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fddcc0a3d50 msgr2=0x7fddcc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:51.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.321+0000 7fdde98ef700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fddcc0a3d50 0x7fddcc0a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fddd400b5c0 tx=0x7fddd4005960 comp rx=0 tx=0).stop 2024-04-21T08:45:51.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.321+0000 7fdde98ef700 1 -- 172.21.15.82:0/3149323446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde4108ad0 msgr2=0x7fdde419da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:51.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.321+0000 7fdde98ef700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde4108ad0 0x7fdde419da40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00b970 tx=0x7fdddc00bc80 comp rx=0 tx=0).stop 2024-04-21T08:45:51.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.321+0000 7fdde98ef700 1 -- 172.21.15.82:0/3149323446 shutdown_connections 2024-04-21T08:45:51.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.321+0000 7fdde98ef700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fddcc0a3d50 0x7fddcc0a6210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:51.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.321+0000 7fdde98ef700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde4108ad0 0x7fdde419da40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:51.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.321+0000 7fdde98ef700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde4101030 0x7fdde4199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:51.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.321+0000 7fdde98ef700 1 --2- 172.21.15.82:0/3149323446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdde41019d0 0x7fdde41996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:51.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.321+0000 7fdde98ef700 1 -- 172.21.15.82:0/3149323446 >> 172.21.15.82:0/3149323446 conn(0x7fdde4077fb0 msgr2=0x7fdde40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:51.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.321+0000 7fdde98ef700 1 -- 172.21.15.82:0/3149323446 shutdown_connections 2024-04-21T08:45:51.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:51.321+0000 7fdde98ef700 1 -- 172.21.15.82:0/3149323446 wait complete. 2024-04-21T08:45:51.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:51 smithi082 ceph-mon[27967]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:45:51.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:51 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3149323446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T08:45:51.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:51 smithi134 ceph-mon[31772]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:45:51.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:51 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3149323446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T08:45:51.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:51 smithi106 ceph-mon[31072]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:45:51.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:51 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3149323446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T08:45:53.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:53 smithi106 ceph-mon[31072]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:45:53.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:53 smithi134 ceph-mon[31772]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:45:53.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:53 smithi082 ceph-mon[27967]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:45:55.523 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:55 smithi082 ceph-mon[27967]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:45:55.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:55 smithi134 ceph-mon[31772]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:45:55.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.861+0000 7f3f01779700 1 -- 172.21.15.82:0/2937772005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3efc101940 msgr2=0x7f3efc101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.861+0000 7f3f01779700 1 --2- 172.21.15.82:0/2937772005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3efc101940 0x7f3efc101dc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009990 tx=0x7f3ee4009ca0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.861+0000 7f3f01779700 1 -- 172.21.15.82:0/2937772005 shutdown_connections 2024-04-21T08:45:55.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.861+0000 7f3f01779700 1 --2- 172.21.15.82:0/2937772005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3efc102300 0x7f3efc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.861+0000 7f3f01779700 1 --2- 172.21.15.82:0/2937772005 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3efc107990 0x7f3efc107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.861+0000 7f3f01779700 1 --2- 172.21.15.82:0/2937772005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3efc101940 0x7f3efc101dc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.861+0000 7f3f01779700 1 -- 172.21.15.82:0/2937772005 >> 172.21.15.82:0/2937772005 conn(0x7f3efc0788f0 msgr2=0x7f3efc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.861+0000 7f3f01779700 1 -- 172.21.15.82:0/2937772005 shutdown_connections 2024-04-21T08:45:55.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.861+0000 7f3f01779700 1 -- 172.21.15.82:0/2937772005 wait complete. 2024-04-21T08:45:55.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.862+0000 7f3f01779700 1 Processor -- start 2024-04-21T08:45:55.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.862+0000 7f3f01779700 1 -- start start 2024-04-21T08:45:55.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.862+0000 7f3f01779700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3efc101940 0x7f3efc198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.862+0000 7f3f01779700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3efc102300 0x7f3efc1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.862+0000 7f3f01779700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3efc107990 0x7f3efc19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.862+0000 7f3f01779700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc111b50 con 0x7f3efc101940 2024-04-21T08:45:55.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.862+0000 7f3f01779700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1119d0 con 0x7f3efc102300 2024-04-21T08:45:55.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.862+0000 7f3f01779700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc111cd0 con 0x7f3efc107990 2024-04-21T08:45:55.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.862+0000 7f3efaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3efc101940 0x7f3efc198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.862+0000 7f3efaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3efc101940 0x7f3efc198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53446/0 (socket says 172.21.15.82:53446) 2024-04-21T08:45:55.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.862+0000 7f3efaffd700 1 -- 172.21.15.82:0/181111276 learned_addr learned my addr 172.21.15.82:0/181111276 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:55.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3efb7fe700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3efc107990 0x7f3efc19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3ef27fc700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3efc102300 0x7f3efc1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3efaffd700 1 -- 172.21.15.82:0/181111276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3efc107990 msgr2=0x7f3efc19c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3efaffd700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3efc107990 0x7f3efc19c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3efaffd700 1 -- 172.21.15.82:0/181111276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3efc102300 msgr2=0x7f3efc1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3efaffd700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3efc102300 0x7f3efc1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3efaffd700 1 -- 172.21.15.82:0/181111276 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc19d070 con 0x7f3efc101940 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3ef27fc700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3efc102300 0x7f3efc1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3efaffd700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3efc101940 0x7f3efc198060 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c920 tx=0x7f3eec00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3ef8ff9700 1 -- 172.21.15.82:0/181111276 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec007b00 con 0x7f3efc101940 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3f01779700 1 -- 172.21.15.82:0/181111276 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc19d360 con 0x7f3efc101940 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3ef8ff9700 1 -- 172.21.15.82:0/181111276 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eec004ce0 con 0x7f3efc101940 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3ef8ff9700 1 -- 172.21.15.82:0/181111276 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec005650 con 0x7f3efc101940 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.863+0000 7f3f01779700 1 -- 172.21.15.82:0/181111276 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a4be0 con 0x7f3efc101940 2024-04-21T08:45:55.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.864+0000 7f3f01779700 1 -- 172.21.15.82:0/181111276 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3efc069730 con 0x7f3efc101940 2024-04-21T08:45:55.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.865+0000 7f3ef8ff9700 1 -- 172.21.15.82:0/181111276 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3eec004750 con 0x7f3efc101940 2024-04-21T08:45:55.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.866+0000 7f3ef8ff9700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3ee80a3cd0 0x7f3ee80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.866+0000 7f3ef27fc700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3ee80a3cd0 0x7f3ee80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.866+0000 7f3ef8ff9700 1 -- 172.21.15.82:0/181111276 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f3eec0d0520 con 0x7f3efc101940 2024-04-21T08:45:55.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.866+0000 7f3ef27fc700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3ee80a3cd0 0x7f3ee80a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400b5c0 tx=0x7f3ee4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:55.870+0000 7f3ef8ff9700 1 -- 172.21.15.82:0/181111276 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3eec014030 con 0x7f3efc101940 2024-04-21T08:45:55.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:55 smithi106 ceph-mon[31072]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:45:56.080 INFO:teuthology.orchestra.run.smithi082.stdout:141733920833 2024-04-21T08:45:56.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.080+0000 7f3f01779700 1 -- 172.21.15.82:0/181111276 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3efc1993c0 con 0x7f3efc101940 2024-04-21T08:45:56.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.080+0000 7f3ef8ff9700 1 -- 172.21.15.82:0/181111276 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3eec09ca60 con 0x7f3efc101940 2024-04-21T08:45:56.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7f3f01779700 1 -- 172.21.15.82:0/181111276 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3ee80a3cd0 msgr2=0x7f3ee80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:56.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7f3f01779700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3ee80a3cd0 0x7f3ee80a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400b5c0 tx=0x7f3ee4005fb0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7f3f01779700 1 -- 172.21.15.82:0/181111276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3efc101940 msgr2=0x7f3efc198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:56.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7f3f01779700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3efc101940 0x7f3efc198060 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c920 tx=0x7f3eec00cce0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7f3f01779700 1 -- 172.21.15.82:0/181111276 shutdown_connections 2024-04-21T08:45:56.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7f3f01779700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3ee80a3cd0 0x7f3ee80a6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7f3f01779700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3efc107990 0x7f3efc19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7f3f01779700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3efc101940 0x7f3efc198060 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7f3f01779700 1 --2- 172.21.15.82:0/181111276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3efc102300 0x7f3efc1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7f3f01779700 1 -- 172.21.15.82:0/181111276 >> 172.21.15.82:0/181111276 conn(0x7f3efc0788f0 msgr2=0x7f3efc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:56.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7f3f01779700 1 -- 172.21.15.82:0/181111276 shutdown_connections 2024-04-21T08:45:56.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7f3f01779700 1 -- 172.21.15.82:0/181111276 wait complete. 2024-04-21T08:45:56.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2151180964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f0101930 msgr2=0x7fd6f0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:56.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7fd6f77e7700 1 --2- 172.21.15.82:0/2151180964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f0101930 0x7fd6f0101db0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009990 tx=0x7fd6e0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2151180964 shutdown_connections 2024-04-21T08:45:56.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7fd6f77e7700 1 --2- 172.21.15.82:0/2151180964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6f01022f0 0x7fd6f010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7fd6f77e7700 1 --2- 172.21.15.82:0/2151180964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6f0107980 0x7fd6f0107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7fd6f77e7700 1 --2- 172.21.15.82:0/2151180964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f0101930 0x7fd6f0101db0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2151180964 >> 172.21.15.82:0/2151180964 conn(0x7fd6f00788b0 msgr2=0x7fd6f0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2151180964 shutdown_connections 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.083+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2151180964 wait complete. 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.084+0000 7fd6f77e7700 1 Processor -- start 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.084+0000 7fd6f77e7700 1 -- start start 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.084+0000 7fd6f77e7700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6f0101930 0x7fd6f01980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.084+0000 7fd6f77e7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f01022f0 0x7fd6f0198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.084+0000 7fd6f77e7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6f0107980 0x7fd6f019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.084+0000 7fd6f77e7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0111af0 con 0x7fd6f01022f0 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.084+0000 7fd6f77e7700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0111970 con 0x7fd6f0101930 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.084+0000 7fd6f77e7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0111c70 con 0x7fd6f0107980 2024-04-21T08:45:56.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.084+0000 7fd6f4d82700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f01022f0 0x7fd6f0198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.084+0000 7fd6f4d82700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f01022f0 0x7fd6f0198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53472/0 (socket says 172.21.15.82:53472) 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.084+0000 7fd6f4d82700 1 -- 172.21.15.82:0/2343728500 learned_addr learned my addr 172.21.15.82:0/2343728500 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.084+0000 7fd6f5583700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6f0101930 0x7fd6f01980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.085+0000 7fd6f5d84700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6f0107980 0x7fd6f019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.085+0000 7fd6f4d82700 1 -- 172.21.15.82:0/2343728500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6f0107980 msgr2=0x7fd6f019c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.085+0000 7fd6f4d82700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6f0107980 0x7fd6f019c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.085+0000 7fd6f4d82700 1 -- 172.21.15.82:0/2343728500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6f0101930 msgr2=0x7fd6f01980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.085+0000 7fd6f4d82700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6f0101930 0x7fd6f01980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.085+0000 7fd6f4d82700 1 -- 172.21.15.82:0/2343728500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f019d100 con 0x7fd6f01022f0 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.085+0000 7fd6f5583700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6f0101930 0x7fd6f01980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.085+0000 7fd6f4d82700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f01022f0 0x7fd6f0198630 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0000c00 tx=0x7fd6e00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.085+0000 7fd6e67fc700 1 -- 172.21.15.82:0/2343728500 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0027070 con 0x7fd6f01022f0 2024-04-21T08:45:56.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.085+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2343728500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f019d390 con 0x7fd6f01022f0 2024-04-21T08:45:56.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.085+0000 7fd6e67fc700 1 -- 172.21.15.82:0/2343728500 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e0003f20 con 0x7fd6f01022f0 2024-04-21T08:45:56.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.086+0000 7fd6e67fc700 1 -- 172.21.15.82:0/2343728500 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0022cf0 con 0x7fd6f01022f0 2024-04-21T08:45:56.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.086+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2343728500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f01a4c70 con 0x7fd6f01022f0 2024-04-21T08:45:56.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.087+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2343728500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f0109ea0 con 0x7fd6f01022f0 2024-04-21T08:45:56.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.087+0000 7fd6e67fc700 1 -- 172.21.15.82:0/2343728500 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd6e002a070 con 0x7fd6f01022f0 2024-04-21T08:45:56.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.088+0000 7fd6e67fc700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd6dc0a3d50 0x7fd6dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:56.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.088+0000 7fd6e67fc700 1 -- 172.21.15.82:0/2343728500 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fd6e00d15c0 con 0x7fd6f01022f0 2024-04-21T08:45:56.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.088+0000 7fd6f5583700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd6dc0a3d50 0x7fd6dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:56.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.089+0000 7fd6f5583700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd6dc0a3d50 0x7fd6dc0a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0103d0 tx=0x7fd6ec00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:56.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.093+0000 7fd6e67fc700 1 -- 172.21.15.82:0/2343728500 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6e0013070 con 0x7fd6f01022f0 2024-04-21T08:45:56.305 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:56.307 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:45:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.310+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2343728500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd6f0199450 con 0x7fd6f01022f0 2024-04-21T08:45:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.310+0000 7fd6e67fc700 1 -- 172.21.15.82:0/2343728500 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd6e009db90 con 0x7fd6f01022f0 2024-04-21T08:45:56.310 INFO:teuthology.orchestra.run.smithi082.stdout:219043332147 2024-04-21T08:45:56.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.313+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2343728500 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd6dc0a3d50 msgr2=0x7fd6dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:56.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.313+0000 7fd6f77e7700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd6dc0a3d50 0x7fd6dc0a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0103d0 tx=0x7fd6ec00b3c0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.313+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2343728500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f01022f0 msgr2=0x7fd6f0198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:56.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.313+0000 7fd6f77e7700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f01022f0 0x7fd6f0198630 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0000c00 tx=0x7fd6e00047b0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.313+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2343728500 shutdown_connections 2024-04-21T08:45:56.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.313+0000 7fd6f77e7700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd6dc0a3d50 0x7fd6dc0a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.313+0000 7fd6f77e7700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6f0107980 0x7fd6f019c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.313+0000 7fd6f77e7700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6f0101930 0x7fd6f01980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.313+0000 7fd6f77e7700 1 --2- 172.21.15.82:0/2343728500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f01022f0 0x7fd6f0198630 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.313+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2343728500 >> 172.21.15.82:0/2343728500 conn(0x7fd6f00788b0 msgr2=0x7fd6f00ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:56.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.313+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2343728500 shutdown_connections 2024-04-21T08:45:56.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.313+0000 7fd6f77e7700 1 -- 172.21.15.82:0/2343728500 wait complete. 2024-04-21T08:45:56.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:56 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/181111276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T08:45:56.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:56 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2343728500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T08:45:56.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:56 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/181111276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T08:45:56.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:56 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2343728500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T08:45:56.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.843+0000 7f4266485700 1 -- 172.21.15.82:0/1162921035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4260102330 msgr2=0x7f426010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:56.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.843+0000 7f4266485700 1 --2- 172.21.15.82:0/1162921035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4260102330 0x7f426010e820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f425800b0d0 tx=0x7f425800b3e0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.843+0000 7f4266485700 1 -- 172.21.15.82:0/1162921035 shutdown_connections 2024-04-21T08:45:56.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.843+0000 7f4266485700 1 --2- 172.21.15.82:0/1162921035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4260102330 0x7f426010e820 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.843+0000 7f4266485700 1 --2- 172.21.15.82:0/1162921035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42601079c0 0x7f4260107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.843+0000 7f4266485700 1 --2- 172.21.15.82:0/1162921035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4260101970 0x7f4260101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.843+0000 7f4266485700 1 -- 172.21.15.82:0/1162921035 >> 172.21.15.82:0/1162921035 conn(0x7f4260078980 msgr2=0x7f4260078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:56.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.843+0000 7f4266485700 1 -- 172.21.15.82:0/1162921035 shutdown_connections 2024-04-21T08:45:56.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.843+0000 7f4266485700 1 -- 172.21.15.82:0/1162921035 wait complete. 2024-04-21T08:45:56.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f4266485700 1 Processor -- start 2024-04-21T08:45:56.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f4266485700 1 -- start start 2024-04-21T08:45:56.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f4266485700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4260101970 0x7f4260197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:56.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f4266485700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4260102330 0x7f42601984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f4266485700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42601079c0 0x7f426019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f4266485700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4260111b40 con 0x7f4260101970 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f4266485700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f42601119c0 con 0x7f4260102330 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f4266485700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4260111cc0 con 0x7f42601079c0 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f425ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4260101970 0x7f4260197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f425ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4260101970 0x7f4260197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53502/0 (socket says 172.21.15.82:53502) 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f425ffff700 1 -- 172.21.15.82:0/2205246601 learned_addr learned my addr 172.21.15.82:0/2205246601 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f4264a22700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42601079c0 0x7f426019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f425f7fe700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4260102330 0x7f42601984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f425ffff700 1 -- 172.21.15.82:0/2205246601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42601079c0 msgr2=0x7f426019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f425ffff700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42601079c0 0x7f426019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f425ffff700 1 -- 172.21.15.82:0/2205246601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4260102330 msgr2=0x7f42601984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:56.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f425ffff700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4260102330 0x7f42601984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:56.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f425ffff700 1 -- 172.21.15.82:0/2205246601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426019cfc0 con 0x7f4260101970 2024-04-21T08:45:56.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.844+0000 7f4264a22700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42601079c0 0x7f426019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:56.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.845+0000 7f425f7fe700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4260102330 0x7f42601984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:56.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.845+0000 7f425ffff700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4260101970 0x7f4260197fb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f425000c9b0 tx=0x7f425000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:56.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.845+0000 7f425d7fa700 1 -- 172.21.15.82:0/2205246601 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250007b00 con 0x7f4260101970 2024-04-21T08:45:56.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.845+0000 7f425d7fa700 1 -- 172.21.15.82:0/2205246601 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4250007ca0 con 0x7f4260101970 2024-04-21T08:45:56.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.845+0000 7f4266485700 1 -- 172.21.15.82:0/2205246601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426019d2b0 con 0x7f4260101970 2024-04-21T08:45:56.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.845+0000 7f425d7fa700 1 -- 172.21.15.82:0/2205246601 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250018730 con 0x7f4260101970 2024-04-21T08:45:56.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.845+0000 7f4266485700 1 -- 172.21.15.82:0/2205246601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42601a4b30 con 0x7f4260101970 2024-04-21T08:45:56.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.846+0000 7f4266485700 1 -- 172.21.15.82:0/2205246601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4260103090 con 0x7f4260101970 2024-04-21T08:45:56.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.847+0000 7f425d7fa700 1 -- 172.21.15.82:0/2205246601 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f425000f450 con 0x7f4260101970 2024-04-21T08:45:56.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.848+0000 7f425d7fa700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f42480a3d50 0x7f42480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:56.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.848+0000 7f425f7fe700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f42480a3d50 0x7f42480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:56.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.848+0000 7f425d7fa700 1 -- 172.21.15.82:0/2205246601 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f42500d0880 con 0x7f4260101970 2024-04-21T08:45:56.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.848+0000 7f425f7fe700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f42480a3d50 0x7f42480a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4254005fd0 tx=0x7f4254005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:56.852+0000 7f425d7fa700 1 -- 172.21.15.82:0/2205246601 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4250014030 con 0x7f4260101970 2024-04-21T08:45:56.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:56 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/181111276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T08:45:56.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:56 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2343728500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T08:45:56.991 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495665 got 197568495670 for osd.6 2024-04-21T08:45:56.991 DEBUG:teuthology.parallel:result is None 2024-04-21T08:45:57.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.061+0000 7f4266485700 1 -- 172.21.15.82:0/2205246601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f426002df80 con 0x7f4260101970 2024-04-21T08:45:57.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.062+0000 7f425d7fa700 1 -- 172.21.15.82:0/2205246601 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f425009cdc0 con 0x7f4260101970 2024-04-21T08:45:57.062 INFO:teuthology.orchestra.run.smithi082.stdout:77309411405 2024-04-21T08:45:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.064+0000 7f4266485700 1 -- 172.21.15.82:0/2205246601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f42480a3d50 msgr2=0x7f42480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.064+0000 7f4266485700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f42480a3d50 0x7f42480a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4254005fd0 tx=0x7f4254005ee0 comp rx=0 tx=0).stop 2024-04-21T08:45:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.064+0000 7f4266485700 1 -- 172.21.15.82:0/2205246601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4260101970 msgr2=0x7f4260197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.064+0000 7f4266485700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4260101970 0x7f4260197fb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f425000c9b0 tx=0x7f425000cd70 comp rx=0 tx=0).stop 2024-04-21T08:45:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.064+0000 7f4266485700 1 -- 172.21.15.82:0/2205246601 shutdown_connections 2024-04-21T08:45:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.064+0000 7f4266485700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f42480a3d50 0x7f42480a6210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.064+0000 7f4266485700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42601079c0 0x7f426019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.064+0000 7f4266485700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4260101970 0x7f4260197fb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.064+0000 7f4266485700 1 --2- 172.21.15.82:0/2205246601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4260102330 0x7f42601984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.064+0000 7f4266485700 1 -- 172.21.15.82:0/2205246601 >> 172.21.15.82:0/2205246601 conn(0x7f4260078980 msgr2=0x7f42600ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:57.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.064+0000 7f4266485700 1 -- 172.21.15.82:0/2205246601 shutdown_connections 2024-04-21T08:45:57.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:57.064+0000 7f4266485700 1 -- 172.21.15.82:0/2205246601 wait complete. 2024-04-21T08:45:57.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:57 smithi134 ceph-mon[31772]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:45:57.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:57 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2205246601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T08:45:57.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:57 smithi082 ceph-mon[27967]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:45:57.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:57 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2205246601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T08:45:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:57 smithi106 ceph-mon[31072]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:45:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:57 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2205246601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T08:45:58.196 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920828 got 141733920833 for osd.4 2024-04-21T08:45:58.196 DEBUG:teuthology.parallel:result is None 2024-04-21T08:45:58.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.850+0000 7f2940be4700 1 -- 172.21.15.82:0/2407437163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f293c06bb10 msgr2=0x7f293c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:58.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.850+0000 7f2940be4700 1 --2- 172.21.15.82:0/2407437163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f293c06bb10 0x7f293c10f900 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f293400b0a0 tx=0x7f293400b3b0 comp rx=0 tx=0).stop 2024-04-21T08:45:58.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.851+0000 7f2940be4700 1 -- 172.21.15.82:0/2407437163 shutdown_connections 2024-04-21T08:45:58.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.851+0000 7f2940be4700 1 --2- 172.21.15.82:0/2407437163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f293c06bb10 0x7f293c10f900 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:58.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.851+0000 7f2940be4700 1 --2- 172.21.15.82:0/2407437163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f293c108a30 0x7f293c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:58.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.851+0000 7f2940be4700 1 --2- 172.21.15.82:0/2407437163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f293c06b170 0x7f293c06b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:58.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.851+0000 7f2940be4700 1 -- 172.21.15.82:0/2407437163 >> 172.21.15.82:0/2407437163 conn(0x7f293c0788f0 msgr2=0x7f293c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:58.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.851+0000 7f2940be4700 1 -- 172.21.15.82:0/2407437163 shutdown_connections 2024-04-21T08:45:58.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.851+0000 7f2940be4700 1 -- 172.21.15.82:0/2407437163 wait complete. 2024-04-21T08:45:58.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.851+0000 7f2940be4700 1 Processor -- start 2024-04-21T08:45:58.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.851+0000 7f2940be4700 1 -- start start 2024-04-21T08:45:58.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2940be4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f293c06b170 0x7f293c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:58.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2940be4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f293c06bb10 0x7f293c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:58.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2940be4700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f293c108a30 0x7f293c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:58.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2940be4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f293c112cb0 con 0x7f293c06bb10 2024-04-21T08:45:58.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2940be4700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f293c112b30 con 0x7f293c108a30 2024-04-21T08:45:58.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2940be4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f293c112e30 con 0x7f293c06b170 2024-04-21T08:45:58.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2939d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f293c06bb10 0x7f293c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2939d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f293c06bb10 0x7f293c19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53518/0 (socket says 172.21.15.82:53518) 2024-04-21T08:45:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2939d9b700 1 -- 172.21.15.82:0/1410592908 learned_addr learned my addr 172.21.15.82:0/1410592908 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:45:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f293ad9d700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f293c108a30 0x7f293c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f293a59c700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f293c06b170 0x7f293c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2939d9b700 1 -- 172.21.15.82:0/1410592908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f293c06b170 msgr2=0x7f293c19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2939d9b700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f293c06b170 0x7f293c19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2939d9b700 1 -- 172.21.15.82:0/1410592908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f293c108a30 msgr2=0x7f293c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2939d9b700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f293c108a30 0x7f293c19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2939d9b700 1 -- 172.21.15.82:0/1410592908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293c1a0420 con 0x7f293c06bb10 2024-04-21T08:45:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f293a59c700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f293c06b170 0x7f293c19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2939d9b700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f293c06bb10 0x7f293c19b9a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f293000b530 tx=0x7f293000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f292b7fe700 1 -- 172.21.15.82:0/1410592908 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930015020 con 0x7f293c06bb10 2024-04-21T08:45:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2940be4700 1 -- 172.21.15.82:0/1410592908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293c1a0710 con 0x7f293c06bb10 2024-04-21T08:45:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f292b7fe700 1 -- 172.21.15.82:0/1410592908 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2930011ce0 con 0x7f293c06bb10 2024-04-21T08:45:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f292b7fe700 1 -- 172.21.15.82:0/1410592908 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930010e40 con 0x7f293c06bb10 2024-04-21T08:45:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.852+0000 7f2940be4700 1 -- 172.21.15.82:0/1410592908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293c1a7f90 con 0x7f293c06bb10 2024-04-21T08:45:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.853+0000 7f2940be4700 1 -- 172.21.15.82:0/1410592908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f293c069730 con 0x7f293c06bb10 2024-04-21T08:45:58.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.855+0000 7f292b7fe700 1 -- 172.21.15.82:0/1410592908 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2930018020 con 0x7f293c06bb10 2024-04-21T08:45:58.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.855+0000 7f292b7fe700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f29240a3cd0 0x7f29240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:58.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.855+0000 7f293a59c700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f29240a3cd0 0x7f29240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:58.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.855+0000 7f292b7fe700 1 -- 172.21.15.82:0/1410592908 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f29300d05a0 con 0x7f293c06bb10 2024-04-21T08:45:58.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.856+0000 7f293a59c700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f29240a3cd0 0x7f29240a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f292c0099d0 tx=0x7f292c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:58.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:58.859+0000 7f292b7fe700 1 -- 172.21.15.82:0/1410592908 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f293001f020 con 0x7f293c06bb10 2024-04-21T08:45:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.074+0000 7f2940be4700 1 -- 172.21.15.82:0/1410592908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f293c02df80 con 0x7f293c06bb10 2024-04-21T08:45:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.074+0000 7f292b7fe700 1 -- 172.21.15.82:0/1410592908 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f293009cbf0 con 0x7f293c06bb10 2024-04-21T08:45:59.074 INFO:teuthology.orchestra.run.smithi082.stdout:107374182471 2024-04-21T08:45:59.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.077+0000 7f2940be4700 1 -- 172.21.15.82:0/1410592908 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f29240a3cd0 msgr2=0x7f29240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:59.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.077+0000 7f2940be4700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f29240a3cd0 0x7f29240a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f292c0099d0 tx=0x7f292c008040 comp rx=0 tx=0).stop 2024-04-21T08:45:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.077+0000 7f2940be4700 1 -- 172.21.15.82:0/1410592908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f293c06bb10 msgr2=0x7f293c19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.077+0000 7f2940be4700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f293c06bb10 0x7f293c19b9a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f293000b530 tx=0x7f293000b840 comp rx=0 tx=0).stop 2024-04-21T08:45:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.077+0000 7f2940be4700 1 -- 172.21.15.82:0/1410592908 shutdown_connections 2024-04-21T08:45:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.077+0000 7f2940be4700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f29240a3cd0 0x7f29240a6190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.077+0000 7f2940be4700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f293c108a30 0x7f293c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.077+0000 7f2940be4700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f293c06b170 0x7f293c19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.077+0000 7f2940be4700 1 --2- 172.21.15.82:0/1410592908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f293c06bb10 0x7f293c19b9a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.077+0000 7f2940be4700 1 -- 172.21.15.82:0/1410592908 >> 172.21.15.82:0/1410592908 conn(0x7f293c0788f0 msgr2=0x7f293c10ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.077+0000 7f2940be4700 1 -- 172.21.15.82:0/1410592908 shutdown_connections 2024-04-21T08:45:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:45:59.077+0000 7f2940be4700 1 -- 172.21.15.82:0/1410592908 wait complete. 2024-04-21T08:45:59.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:59 smithi082 ceph-mon[27967]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:45:59.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:45:59 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1410592908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T08:45:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:59 smithi134 ceph-mon[31772]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:45:59.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:45:59 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1410592908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T08:45:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:59 smithi106 ceph-mon[31072]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:45:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:45:59 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1410592908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T08:46:01.210 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182466 got 107374182471 for osd.3 2024-04-21T08:46:01.211 DEBUG:teuthology.parallel:result is None 2024-04-21T08:46:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:01 smithi134 ceph-mon[31772]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:46:01.803 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:01 smithi082 ceph-mon[27967]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:46:01.829 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:46:01.835 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332143 got 219043332147 for osd.7 2024-04-21T08:46:01.835 DEBUG:teuthology.parallel:result is None 2024-04-21T08:46:01.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:01 smithi106 ceph-mon[31072]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:46:02.378 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411405 for osd.2 2024-04-21T08:46:02.378 DEBUG:teuthology.parallel:result is None 2024-04-21T08:46:03.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.054+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/229947979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8102510 msgr2=0x7fb0f8102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:03.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.054+0000 7fb0ff7b7700 1 --2- 172.21.15.82:0/229947979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8102510 0x7fb0f8102990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80099c0 tx=0x7fb0e8009cd0 comp rx=0 tx=0).stop 2024-04-21T08:46:03.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.054+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/229947979 shutdown_connections 2024-04-21T08:46:03.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.054+0000 7fb0ff7b7700 1 --2- 172.21.15.82:0/229947979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0f8102ed0 0x7fb0f810e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:03.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.054+0000 7fb0ff7b7700 1 --2- 172.21.15.82:0/229947979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0f8107c30 0x7fb0f8108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:03.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.054+0000 7fb0ff7b7700 1 --2- 172.21.15.82:0/229947979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8102510 0x7fb0f8102990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:03.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.054+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/229947979 >> 172.21.15.82:0/229947979 conn(0x7fb0f8076c60 msgr2=0x7fb0f8077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:03.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.054+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/229947979 shutdown_connections 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.054+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/229947979 wait complete. 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0ff7b7700 1 Processor -- start 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0ff7b7700 1 -- start start 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0ff7b7700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0f8102510 0x7fb0f8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0ff7b7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0f8102ed0 0x7fb0f8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0ff7b7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8107c30 0x7fb0f8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0ff7b7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f8111cf0 con 0x7fb0f8107c30 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0ff7b7700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f8111b70 con 0x7fb0f8102510 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0ff7b7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f8111e70 con 0x7fb0f8102ed0 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0fdd54700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8107c30 0x7fb0f8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0fdd54700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8107c30 0x7fb0f8198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50990/0 (socket says 172.21.15.82:50990) 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0fdd54700 1 -- 172.21.15.82:0/4268283242 learned_addr learned my addr 172.21.15.82:0/4268283242 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0fcd52700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0f8102ed0 0x7fb0f8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:03.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0fd553700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0f8102510 0x7fb0f8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:03.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0fdd54700 1 -- 172.21.15.82:0/4268283242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0f8102ed0 msgr2=0x7fb0f8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:03.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0fdd54700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0f8102ed0 0x7fb0f8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:03.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0fdd54700 1 -- 172.21.15.82:0/4268283242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0f8102510 msgr2=0x7fb0f8197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:03.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0fdd54700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0f8102510 0x7fb0f8197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:03.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.055+0000 7fb0fdd54700 1 -- 172.21.15.82:0/4268283242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f81a7730 con 0x7fb0f8107c30 2024-04-21T08:46:03.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.056+0000 7fb0fdd54700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8107c30 0x7fb0f8198940 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000b9e0 tx=0x7fb0f000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:03.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.056+0000 7fb0ee7fc700 1 -- 172.21.15.82:0/4268283242 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000d8e0 con 0x7fb0f8107c30 2024-04-21T08:46:03.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.056+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/4268283242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f81a7a20 con 0x7fb0f8107c30 2024-04-21T08:46:03.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.056+0000 7fb0ee7fc700 1 -- 172.21.15.82:0/4268283242 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0f0015420 con 0x7fb0f8107c30 2024-04-21T08:46:03.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.056+0000 7fb0ee7fc700 1 -- 172.21.15.82:0/4268283242 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0014630 con 0x7fb0f8107c30 2024-04-21T08:46:03.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.056+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/4268283242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f81a7f30 con 0x7fb0f8107c30 2024-04-21T08:46:03.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.057+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/4268283242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f8069730 con 0x7fb0f8107c30 2024-04-21T08:46:03.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.058+0000 7fb0ee7fc700 1 -- 172.21.15.82:0/4268283242 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb0f00155d0 con 0x7fb0f8107c30 2024-04-21T08:46:03.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.059+0000 7fb0ee7fc700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb0e40a3cd0 0x7fb0e40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:03.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.059+0000 7fb0ee7fc700 1 -- 172.21.15.82:0/4268283242 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fb0f00d01f0 con 0x7fb0f8107c30 2024-04-21T08:46:03.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.059+0000 7fb0fd553700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb0e40a3cd0 0x7fb0e40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:03.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.059+0000 7fb0fd553700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb0e40a3cd0 0x7fb0e40a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4004630 tx=0x7fb0f400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:03.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.064+0000 7fb0ee7fc700 1 -- 172.21.15.82:0/4268283242 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0f009c730 con 0x7fb0f8107c30 2024-04-21T08:46:03.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.290+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/4268283242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb0f802df80 con 0x7fb0f8107c30 2024-04-21T08:46:03.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.290+0000 7fb0ee7fc700 1 -- 172.21.15.82:0/4268283242 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb0f009c030 con 0x7fb0f8107c30 2024-04-21T08:46:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:34359738454 2024-04-21T08:46:03.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.292+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/4268283242 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb0e40a3cd0 msgr2=0x7fb0e40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:03.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.292+0000 7fb0ff7b7700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb0e40a3cd0 0x7fb0e40a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4004630 tx=0x7fb0f400b3c0 comp rx=0 tx=0).stop 2024-04-21T08:46:03.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.292+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/4268283242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8107c30 msgr2=0x7fb0f8198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:03.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.292+0000 7fb0ff7b7700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8107c30 0x7fb0f8198940 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000b9e0 tx=0x7fb0f000bcf0 comp rx=0 tx=0).stop 2024-04-21T08:46:03.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.293+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/4268283242 shutdown_connections 2024-04-21T08:46:03.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.293+0000 7fb0ff7b7700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb0e40a3cd0 0x7fb0e40a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:03.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.293+0000 7fb0ff7b7700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8107c30 0x7fb0f8198940 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:03.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.293+0000 7fb0ff7b7700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0f8102510 0x7fb0f8197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:03.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.293+0000 7fb0ff7b7700 1 --2- 172.21.15.82:0/4268283242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0f8102ed0 0x7fb0f8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:03.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.293+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/4268283242 >> 172.21.15.82:0/4268283242 conn(0x7fb0f8076c60 msgr2=0x7fb0f80ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:03.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.293+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/4268283242 shutdown_connections 2024-04-21T08:46:03.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:03.293+0000 7fb0ff7b7700 1 -- 172.21.15.82:0/4268283242 wait complete. 2024-04-21T08:46:03.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:03 smithi082 ceph-mon[27967]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:03.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:46:03.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:46:03.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:46:03.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:46:03.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:03 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:46:03.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:03 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4268283242' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T08:46:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:03 smithi134 ceph-mon[31772]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:46:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:46:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:46:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:46:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:03 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:46:03.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:03 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/4268283242' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T08:46:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:03 smithi106 ceph-mon[31072]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:46:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:46:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:46:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:46:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:03 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' 2024-04-21T08:46:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:03 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/4268283242' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T08:46:04.500 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738449 got 34359738454 for osd.0 2024-04-21T08:46:04.500 DEBUG:teuthology.parallel:result is None 2024-04-21T08:46:05.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.601+0000 7f130a90d700 1 -- 172.21.15.82:0/1334730569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1304108a50 msgr2=0x7f1304108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:05.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.601+0000 7f130a90d700 1 --2- 172.21.15.82:0/1334730569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1304108a50 0x7f1304108e30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009990 tx=0x7f12f4009ca0 comp rx=0 tx=0).stop 2024-04-21T08:46:05.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.601+0000 7f130a90d700 1 -- 172.21.15.82:0/1334730569 shutdown_connections 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.601+0000 7f130a90d700 1 --2- 172.21.15.82:0/1334730569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1304103450 0x7f130410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.601+0000 7f130a90d700 1 --2- 172.21.15.82:0/1334730569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1304108a50 0x7f1304108e30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.601+0000 7f130a90d700 1 --2- 172.21.15.82:0/1334730569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1304102a90 0x7f1304102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.601+0000 7f130a90d700 1 -- 172.21.15.82:0/1334730569 >> 172.21.15.82:0/1334730569 conn(0x7f1304079ba0 msgr2=0x7f1304079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.601+0000 7f130a90d700 1 -- 172.21.15.82:0/1334730569 shutdown_connections 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.601+0000 7f130a90d700 1 -- 172.21.15.82:0/1334730569 wait complete. 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f130a90d700 1 Processor -- start 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f130a90d700 1 -- start start 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f130a90d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1304102a90 0x7f1304199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f130a90d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1304103450 0x7f1304199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f130a90d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1304108a50 0x7f130419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f130a90d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1304112c00 con 0x7f1304103450 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f130a90d700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1304112a80 con 0x7f1304108a50 2024-04-21T08:46:05.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f130a90d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1304112d80 con 0x7f1304102a90 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f13037fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1304103450 0x7f1304199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f13037fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1304103450 0x7f1304199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51002/0 (socket says 172.21.15.82:51002) 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f13037fe700 1 -- 172.21.15.82:0/2408605601 learned_addr learned my addr 172.21.15.82:0/2408605601 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f1308eaa700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1304108a50 0x7f130419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f13037fe700 1 -- 172.21.15.82:0/2408605601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1304102a90 msgr2=0x7f1304199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f13037fe700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1304102a90 0x7f1304199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f13037fe700 1 -- 172.21.15.82:0/2408605601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1304108a50 msgr2=0x7f130419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f13037fe700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1304108a50 0x7f130419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f13037fe700 1 -- 172.21.15.82:0/2408605601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130419e160 con 0x7f1304103450 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.602+0000 7f1308eaa700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1304108a50 0x7f130419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.603+0000 7f13037fe700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1304103450 0x7f1304199690 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f12f800d980 tx=0x7f12f800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.603+0000 7f13017fa700 1 -- 172.21.15.82:0/2408605601 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8016070 con 0x7f1304103450 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.603+0000 7f13017fa700 1 -- 172.21.15.82:0/2408605601 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f80044d0 con 0x7f1304103450 2024-04-21T08:46:05.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.603+0000 7f13017fa700 1 -- 172.21.15.82:0/2408605601 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8004d00 con 0x7f1304103450 2024-04-21T08:46:05.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.603+0000 7f130a90d700 1 -- 172.21.15.82:0/2408605601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130419e450 con 0x7f1304103450 2024-04-21T08:46:05.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.603+0000 7f130a90d700 1 -- 172.21.15.82:0/2408605601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13041a5cd0 con 0x7f1304103450 2024-04-21T08:46:05.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.604+0000 7f130a90d700 1 -- 172.21.15.82:0/2408605601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1304069730 con 0x7f1304103450 2024-04-21T08:46:05.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.605+0000 7f13017fa700 1 -- 172.21.15.82:0/2408605601 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f12f8004020 con 0x7f1304103450 2024-04-21T08:46:05.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.605+0000 7f13017fa700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f12ec0a3d50 0x7f12ec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:05.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.606+0000 7f1303fff700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f12ec0a3d50 0x7f12ec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:05.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.606+0000 7f13017fa700 1 -- 172.21.15.82:0/2408605601 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f12f80d0700 con 0x7f1304103450 2024-04-21T08:46:05.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.606+0000 7f1303fff700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f12ec0a3d50 0x7f12ec0a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f12f400b5c0 tx=0x7f12f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:05.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.610+0000 7f13017fa700 1 -- 172.21.15.82:0/2408605601 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12f801f020 con 0x7f1304103450 2024-04-21T08:46:05.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:05 smithi134 ceph-mon[31772]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:46:05.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.813+0000 7f130a90d700 1 -- 172.21.15.82:0/2408605601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f130402df80 con 0x7f1304103450 2024-04-21T08:46:05.813 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:05 smithi082 ceph-mon[27967]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:46:05.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.813+0000 7f13017fa700 1 -- 172.21.15.82:0/2408605601 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f12f809cc40 con 0x7f1304103450 2024-04-21T08:46:05.813 INFO:teuthology.orchestra.run.smithi082.stdout:55834574930 2024-04-21T08:46:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.815+0000 7f130a90d700 1 -- 172.21.15.82:0/2408605601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f12ec0a3d50 msgr2=0x7f12ec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.815+0000 7f130a90d700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f12ec0a3d50 0x7f12ec0a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f12f400b5c0 tx=0x7f12f4005fb0 comp rx=0 tx=0).stop 2024-04-21T08:46:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.815+0000 7f130a90d700 1 -- 172.21.15.82:0/2408605601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1304103450 msgr2=0x7f1304199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.815+0000 7f130a90d700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1304103450 0x7f1304199690 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f12f800d980 tx=0x7f12f800dd40 comp rx=0 tx=0).stop 2024-04-21T08:46:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.815+0000 7f130a90d700 1 -- 172.21.15.82:0/2408605601 shutdown_connections 2024-04-21T08:46:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.815+0000 7f130a90d700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f12ec0a3d50 0x7f12ec0a6210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.815+0000 7f130a90d700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1304108a50 0x7f130419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.815+0000 7f130a90d700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1304102a90 0x7f1304199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.815+0000 7f130a90d700 1 --2- 172.21.15.82:0/2408605601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1304103450 0x7f1304199690 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.815+0000 7f130a90d700 1 -- 172.21.15.82:0/2408605601 >> 172.21.15.82:0/2408605601 conn(0x7f1304079ba0 msgr2=0x7f13041017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.815+0000 7f130a90d700 1 -- 172.21.15.82:0/2408605601 shutdown_connections 2024-04-21T08:46:05.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:05.815+0000 7f130a90d700 1 -- 172.21.15.82:0/2408605601 wait complete. 2024-04-21T08:46:05.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:05 smithi106 ceph-mon[31072]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:46:06.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:06 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2408605601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T08:46:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:06 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2408605601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T08:46:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:06 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2408605601' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T08:46:07.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.376+0000 7fe97027a700 1 -- 172.21.15.82:0/4174911278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe968102ac0 msgr2=0x7fe968102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.376+0000 7fe97027a700 1 --2- 172.21.15.82:0/4174911278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe968102ac0 0x7fe968102f40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe958009990 tx=0x7fe958009ca0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.376+0000 7fe97027a700 1 -- 172.21.15.82:0/4174911278 shutdown_connections 2024-04-21T08:46:07.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.376+0000 7fe97027a700 1 --2- 172.21.15.82:0/4174911278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe968103480 0x7fe96810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.376+0000 7fe97027a700 1 --2- 172.21.15.82:0/4174911278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe968108ab0 0x7fe968108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.376+0000 7fe97027a700 1 --2- 172.21.15.82:0/4174911278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe968102ac0 0x7fe968102f40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.376+0000 7fe97027a700 1 -- 172.21.15.82:0/4174911278 >> 172.21.15.82:0/4174911278 conn(0x7fe968079bf0 msgr2=0x7fe96807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.376+0000 7fe97027a700 1 -- 172.21.15.82:0/4174911278 shutdown_connections 2024-04-21T08:46:07.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.376+0000 7fe97027a700 1 -- 172.21.15.82:0/4174911278 wait complete. 2024-04-21T08:46:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.376+0000 7fe97027a700 1 Processor -- start 2024-04-21T08:46:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe97027a700 1 -- start start 2024-04-21T08:46:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe97027a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe968102ac0 0x7fe968199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe97027a700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe968103480 0x7fe9681996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe97027a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe968108ab0 0x7fe96819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe97027a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe968112d40 con 0x7fe968108ab0 2024-04-21T08:46:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe97027a700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe968112bc0 con 0x7fe968103480 2024-04-21T08:46:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe97027a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe968112ec0 con 0x7fe968102ac0 2024-04-21T08:46:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe96e817700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe968108ab0 0x7fe96819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe96e817700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe968108ab0 0x7fe96819da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51032/0 (socket says 172.21.15.82:51032) 2024-04-21T08:46:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe96e817700 1 -- 172.21.15.82:0/2323688915 learned_addr learned my addr 172.21.15.82:0/2323688915 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe96d815700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe968103480 0x7fe9681996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe96e016700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe968102ac0 0x7fe968199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe96e817700 1 -- 172.21.15.82:0/2323688915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe968102ac0 msgr2=0x7fe968199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe96e817700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe968102ac0 0x7fe968199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe96e817700 1 -- 172.21.15.82:0/2323688915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe968103480 msgr2=0x7fe9681996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe96e817700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe968103480 0x7fe9681996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe96e817700 1 -- 172.21.15.82:0/2323688915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96819e1a0 con 0x7fe968108ab0 2024-04-21T08:46:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.377+0000 7fe96e817700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe968108ab0 0x7fe96819da60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe96000b970 tx=0x7fe96000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.378+0000 7fe95f7fe700 1 -- 172.21.15.82:0/2323688915 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96000d860 con 0x7fe968108ab0 2024-04-21T08:46:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.378+0000 7fe95f7fe700 1 -- 172.21.15.82:0/2323688915 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe96000e430 con 0x7fe968108ab0 2024-04-21T08:46:07.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.378+0000 7fe97027a700 1 -- 172.21.15.82:0/2323688915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96819e490 con 0x7fe968108ab0 2024-04-21T08:46:07.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.378+0000 7fe95f7fe700 1 -- 172.21.15.82:0/2323688915 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960017660 con 0x7fe968108ab0 2024-04-21T08:46:07.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.378+0000 7fe97027a700 1 -- 172.21.15.82:0/2323688915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9681a5d10 con 0x7fe968108ab0 2024-04-21T08:46:07.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.378+0000 7fe97027a700 1 -- 172.21.15.82:0/2323688915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe968069730 con 0x7fe968108ab0 2024-04-21T08:46:07.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.380+0000 7fe95f7fe700 1 -- 172.21.15.82:0/2323688915 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe96000da00 con 0x7fe968108ab0 2024-04-21T08:46:07.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.381+0000 7fe95f7fe700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe9540a3cd0 0x7fe9540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.382+0000 7fe95f7fe700 1 -- 172.21.15.82:0/2323688915 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fe9600d08c0 con 0x7fe968108ab0 2024-04-21T08:46:07.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.382+0000 7fe96e016700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe9540a3cd0 0x7fe9540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.382+0000 7fe96e016700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe9540a3cd0 0x7fe9540a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe96400ac50 tx=0x7fe96400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.386+0000 7fe95f7fe700 1 -- 172.21.15.82:0/2323688915 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe960013030 con 0x7fe968108ab0 2024-04-21T08:46:07.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.597+0000 7fe97027a700 1 -- 172.21.15.82:0/2323688915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe96819a4f0 con 0x7fe968108ab0 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.597+0000 7fe95f7fe700 1 -- 172.21.15.82:0/2323688915 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe96009ce00 con 0x7fe968108ab0 2024-04-21T08:46:07.597 INFO:teuthology.orchestra.run.smithi082.stdout:253403070512 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.599+0000 7fe97027a700 1 -- 172.21.15.82:0/2323688915 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe9540a3cd0 msgr2=0x7fe9540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.599+0000 7fe97027a700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe9540a3cd0 0x7fe9540a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe96400ac50 tx=0x7fe96400a380 comp rx=0 tx=0).stop 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.599+0000 7fe97027a700 1 -- 172.21.15.82:0/2323688915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe968108ab0 msgr2=0x7fe96819da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.599+0000 7fe97027a700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe968108ab0 0x7fe96819da60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe96000b970 tx=0x7fe96000bc80 comp rx=0 tx=0).stop 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.599+0000 7fe97027a700 1 -- 172.21.15.82:0/2323688915 shutdown_connections 2024-04-21T08:46:07.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.599+0000 7fe97027a700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe9540a3cd0 0x7fe9540a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.599+0000 7fe97027a700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe968108ab0 0x7fe96819da60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.599+0000 7fe97027a700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe968102ac0 0x7fe968199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.599+0000 7fe97027a700 1 --2- 172.21.15.82:0/2323688915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe968103480 0x7fe9681996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.599+0000 7fe97027a700 1 -- 172.21.15.82:0/2323688915 >> 172.21.15.82:0/2323688915 conn(0x7fe968079bf0 msgr2=0x7fe968100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.599+0000 7fe97027a700 1 -- 172.21.15.82:0/2323688915 shutdown_connections 2024-04-21T08:46:07.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:07.599+0000 7fe97027a700 1 -- 172.21.15.82:0/2323688915 wait complete. 2024-04-21T08:46:07.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:07 smithi134 ceph-mon[31772]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:07.749 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:07 smithi082 ceph-mon[27967]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:07.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:07 smithi106 ceph-mon[31072]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:08.670 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070505 got 253403070512 for osd.8 2024-04-21T08:46:08.670 DEBUG:teuthology.parallel:result is None 2024-04-21T08:46:08.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:08 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2323688915' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T08:46:08.750 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:08 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2323688915' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T08:46:08.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:08 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2323688915' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T08:46:09.014 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574930 for osd.1 2024-04-21T08:46:09.014 DEBUG:teuthology.parallel:result is None 2024-04-21T08:46:09.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:09 smithi082 ceph-mon[27967]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:46:09.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.634+0000 7f6aeaba2700 1 -- 172.21.15.82:0/723070834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4108ad0 msgr2=0x7f6ae4108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.634+0000 7f6aeaba2700 1 --2- 172.21.15.82:0/723070834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4108ad0 0x7f6ae4108eb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009990 tx=0x7f6ad8009ca0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.634+0000 7f6aeaba2700 1 -- 172.21.15.82:0/723070834 shutdown_connections 2024-04-21T08:46:09.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.634+0000 7f6aeaba2700 1 --2- 172.21.15.82:0/723070834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ae41019d0 0x7f6ae410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.634+0000 7f6aeaba2700 1 --2- 172.21.15.82:0/723070834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4108ad0 0x7f6ae4108eb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.634+0000 7f6aeaba2700 1 --2- 172.21.15.82:0/723070834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae4101030 0x7f6ae4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.634+0000 7f6aeaba2700 1 -- 172.21.15.82:0/723070834 >> 172.21.15.82:0/723070834 conn(0x7f6ae4077fb0 msgr2=0x7f6ae40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.634+0000 7f6aeaba2700 1 -- 172.21.15.82:0/723070834 shutdown_connections 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.634+0000 7f6aeaba2700 1 -- 172.21.15.82:0/723070834 wait complete. 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6aeaba2700 1 Processor -- start 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6aeaba2700 1 -- start start 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6aeaba2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4101030 0x7f6ae41990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6aeaba2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae41019d0 0x7f6ae4199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6aeaba2700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ae4108ad0 0x7f6ae419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6aeaba2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4112bc0 con 0x7f6ae4101030 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6aeaba2700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4112a40 con 0x7f6ae4108ad0 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6aeaba2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4112d40 con 0x7f6ae41019d0 2024-04-21T08:46:09.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6ae893e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4101030 0x7f6ae41990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6ae893e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4101030 0x7f6ae41990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51062/0 (socket says 172.21.15.82:51062) 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6ae893e700 1 -- 172.21.15.82:0/4114421638 learned_addr learned my addr 172.21.15.82:0/4114421638 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6ae913f700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ae4108ad0 0x7f6ae419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6ae3fff700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae41019d0 0x7f6ae4199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6ae893e700 1 -- 172.21.15.82:0/4114421638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae41019d0 msgr2=0x7f6ae4199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6ae893e700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae41019d0 0x7f6ae4199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6ae893e700 1 -- 172.21.15.82:0/4114421638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ae4108ad0 msgr2=0x7f6ae419d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6ae893e700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ae4108ad0 0x7f6ae419d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.635+0000 7f6ae893e700 1 -- 172.21.15.82:0/4114421638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae419e100 con 0x7f6ae4101030 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.636+0000 7f6ae893e700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4101030 0x7f6ae41990f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8000c00 tx=0x7f6ad80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.636+0000 7f6ae1ffb700 1 -- 172.21.15.82:0/4114421638 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8027070 con 0x7f6ae4101030 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.636+0000 7f6ae1ffb700 1 -- 172.21.15.82:0/4114421638 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ad8003f20 con 0x7f6ae4101030 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.636+0000 7f6aeaba2700 1 -- 172.21.15.82:0/4114421638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae419e390 con 0x7f6ae4101030 2024-04-21T08:46:09.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.636+0000 7f6ae1ffb700 1 -- 172.21.15.82:0/4114421638 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8022cf0 con 0x7f6ae4101030 2024-04-21T08:46:09.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.636+0000 7f6aeaba2700 1 -- 172.21.15.82:0/4114421638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae41a5c70 con 0x7f6ae4101030 2024-04-21T08:46:09.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.637+0000 7f6aeaba2700 1 -- 172.21.15.82:0/4114421638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae4104350 con 0x7f6ae4101030 2024-04-21T08:46:09.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.638+0000 7f6ae1ffb700 1 -- 172.21.15.82:0/4114421638 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6ad802a070 con 0x7f6ae4101030 2024-04-21T08:46:09.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.639+0000 7f6ae1ffb700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6ad40a3cd0 0x7f6ad40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:09.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.639+0000 7f6ae3fff700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6ad40a3cd0 0x7f6ad40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:09.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.639+0000 7f6ae1ffb700 1 -- 172.21.15.82:0/4114421638 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f6ad80d1aa0 con 0x7f6ae4101030 2024-04-21T08:46:09.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.639+0000 7f6ae3fff700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6ad40a3cd0 0x7f6ad40a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0005950 tx=0x7f6ad00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:09.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.643+0000 7f6ae1ffb700 1 -- 172.21.15.82:0/4114421638 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ad8013070 con 0x7f6ae4101030 2024-04-21T08:46:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:09 smithi134 ceph-mon[31772]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:46:09.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.854+0000 7f6aeaba2700 1 -- 172.21.15.82:0/4114421638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6ae402df80 con 0x7f6ae4101030 2024-04-21T08:46:09.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.855+0000 7f6ae1ffb700 1 -- 172.21.15.82:0/4114421638 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6ad809e010 con 0x7f6ae4101030 2024-04-21T08:46:09.854 INFO:teuthology.orchestra.run.smithi082.stdout:163208757312 2024-04-21T08:46:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.857+0000 7f6aeaba2700 1 -- 172.21.15.82:0/4114421638 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6ad40a3cd0 msgr2=0x7f6ad40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.857+0000 7f6aeaba2700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6ad40a3cd0 0x7f6ad40a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0005950 tx=0x7f6ad00058e0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.857+0000 7f6aeaba2700 1 -- 172.21.15.82:0/4114421638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4101030 msgr2=0x7f6ae41990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.857+0000 7f6aeaba2700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4101030 0x7f6ae41990f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8000c00 tx=0x7f6ad80047b0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.857+0000 7f6aeaba2700 1 -- 172.21.15.82:0/4114421638 shutdown_connections 2024-04-21T08:46:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.857+0000 7f6aeaba2700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f6ad40a3cd0 0x7f6ad40a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.857+0000 7f6aeaba2700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ae4108ad0 0x7f6ae419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.857+0000 7f6aeaba2700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ae4101030 0x7f6ae41990f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.857+0000 7f6aeaba2700 1 --2- 172.21.15.82:0/4114421638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ae41019d0 0x7f6ae4199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:09.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.857+0000 7f6aeaba2700 1 -- 172.21.15.82:0/4114421638 >> 172.21.15.82:0/4114421638 conn(0x7f6ae4077fb0 msgr2=0x7f6ae40ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:09.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.857+0000 7f6aeaba2700 1 -- 172.21.15.82:0/4114421638 shutdown_connections 2024-04-21T08:46:09.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.857+0000 7f6aeaba2700 1 -- 172.21.15.82:0/4114421638 wait complete. 2024-04-21T08:46:09.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:09 smithi106 ceph-mon[31072]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:46:10.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.011+0000 7f15bb82d700 1 -- 172.21.15.82:0/1720078562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15b4102300 msgr2=0x7f15b410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.011+0000 7f15bb82d700 1 --2- 172.21.15.82:0/1720078562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15b4102300 0x7f15b410e7f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009990 tx=0x7f15b0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.011+0000 7f15bb82d700 1 -- 172.21.15.82:0/1720078562 shutdown_connections 2024-04-21T08:46:10.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.011+0000 7f15bb82d700 1 --2- 172.21.15.82:0/1720078562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15b4102300 0x7f15b410e7f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.011+0000 7f15bb82d700 1 --2- 172.21.15.82:0/1720078562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b4107990 0x7f15b4107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.011+0000 7f15bb82d700 1 --2- 172.21.15.82:0/1720078562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b4101940 0x7f15b4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.011+0000 7f15bb82d700 1 -- 172.21.15.82:0/1720078562 >> 172.21.15.82:0/1720078562 conn(0x7f15b40788f0 msgr2=0x7f15b4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.011+0000 7f15bb82d700 1 -- 172.21.15.82:0/1720078562 shutdown_connections 2024-04-21T08:46:10.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.011+0000 7f15bb82d700 1 -- 172.21.15.82:0/1720078562 wait complete. 2024-04-21T08:46:10.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.012+0000 7f15bb82d700 1 Processor -- start 2024-04-21T08:46:10.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.012+0000 7f15bb82d700 1 -- start start 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.012+0000 7f15bb82d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15b4101940 0x7f15b41990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.012+0000 7f15bb82d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b4102300 0x7f15b4199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.012+0000 7f15bb82d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b4107990 0x7f15b419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.012+0000 7f15bb82d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4112c30 con 0x7f15b4101940 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.012+0000 7f15bb82d700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4112ab0 con 0x7f15b4102300 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.012+0000 7f15bb82d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4112db0 con 0x7f15b4107990 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.012+0000 7f15b95c9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15b4101940 0x7f15b41990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.012+0000 7f15b95c9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15b4101940 0x7f15b41990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51068/0 (socket says 172.21.15.82:51068) 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.012+0000 7f15b95c9700 1 -- 172.21.15.82:0/992792282 learned_addr learned my addr 172.21.15.82:0/992792282 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.012+0000 7f15b9dca700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b4107990 0x7f15b419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15b8dc8700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b4102300 0x7f15b4199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15b95c9700 1 -- 172.21.15.82:0/992792282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b4107990 msgr2=0x7f15b419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15b95c9700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b4107990 0x7f15b419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15b95c9700 1 -- 172.21.15.82:0/992792282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b4102300 msgr2=0x7f15b4199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15b95c9700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b4102300 0x7f15b4199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15b95c9700 1 -- 172.21.15.82:0/992792282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b419e080 con 0x7f15b4101940 2024-04-21T08:46:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15b8dc8700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b4102300 0x7f15b4199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15b9dca700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b4107990 0x7f15b419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15b95c9700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15b4101940 0x7f15b41990c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f15ac007f10 tx=0x7f15ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15aa7fc700 1 -- 172.21.15.82:0/992792282 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac003a00 con 0x7f15b4101940 2024-04-21T08:46:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15aa7fc700 1 -- 172.21.15.82:0/992792282 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15ac003ba0 con 0x7f15b4101940 2024-04-21T08:46:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15bb82d700 1 -- 172.21.15.82:0/992792282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b419e370 con 0x7f15b4101940 2024-04-21T08:46:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15aa7fc700 1 -- 172.21.15.82:0/992792282 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac015660 con 0x7f15b4101940 2024-04-21T08:46:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.013+0000 7f15bb82d700 1 -- 172.21.15.82:0/992792282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b41a5bf0 con 0x7f15b4101940 2024-04-21T08:46:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.014+0000 7f15bb82d700 1 -- 172.21.15.82:0/992792282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b4069730 con 0x7f15b4101940 2024-04-21T08:46:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.015+0000 7f15aa7fc700 1 -- 172.21.15.82:0/992792282 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f15ac004000 con 0x7f15b4101940 2024-04-21T08:46:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.016+0000 7f15aa7fc700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f15a00a3cd0 0x7f15a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.016+0000 7f15b8dc8700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f15a00a3cd0 0x7f15a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.016+0000 7f15aa7fc700 1 -- 172.21.15.82:0/992792282 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f15ac0d0cf0 con 0x7f15b4101940 2024-04-21T08:46:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.016+0000 7f15b8dc8700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f15a00a3cd0 0x7f15a00a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15a4005950 tx=0x7f15a40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.020+0000 7f15aa7fc700 1 -- 172.21.15.82:0/992792282 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15ac023070 con 0x7f15b4101940 2024-04-21T08:46:10.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.225+0000 7f15bb82d700 1 -- 172.21.15.82:0/992792282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f15b402df80 con 0x7f15b4101940 2024-04-21T08:46:10.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.225+0000 7f15aa7fc700 1 -- 172.21.15.82:0/992792282 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f15ac010070 con 0x7f15b4101940 2024-04-21T08:46:10.225 INFO:teuthology.orchestra.run.smithi082.stdout:309237645350 2024-04-21T08:46:10.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.228+0000 7f15bb82d700 1 -- 172.21.15.82:0/992792282 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f15a00a3cd0 msgr2=0x7f15a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.228+0000 7f15bb82d700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f15a00a3cd0 0x7f15a00a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f15a4005950 tx=0x7f15a40058e0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.228+0000 7f15bb82d700 1 -- 172.21.15.82:0/992792282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15b4101940 msgr2=0x7f15b41990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.228+0000 7f15bb82d700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15b4101940 0x7f15b41990c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f15ac007f10 tx=0x7f15ac00d3b0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.228+0000 7f15bb82d700 1 -- 172.21.15.82:0/992792282 shutdown_connections 2024-04-21T08:46:10.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.228+0000 7f15bb82d700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f15a00a3cd0 0x7f15a00a6190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.228+0000 7f15bb82d700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b4107990 0x7f15b419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.228+0000 7f15bb82d700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15b4101940 0x7f15b41990c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.228+0000 7f15bb82d700 1 --2- 172.21.15.82:0/992792282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b4102300 0x7f15b4199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.228+0000 7f15bb82d700 1 -- 172.21.15.82:0/992792282 >> 172.21.15.82:0/992792282 conn(0x7f15b40788f0 msgr2=0x7f15b40ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.228+0000 7f15bb82d700 1 -- 172.21.15.82:0/992792282 shutdown_connections 2024-04-21T08:46:10.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:10.228+0000 7f15bb82d700 1 -- 172.21.15.82:0/992792282 wait complete. 2024-04-21T08:46:10.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:10 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/904752474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T08:46:10.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:10 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4114421638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T08:46:10.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:10 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/992792282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T08:46:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:10 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/904752474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T08:46:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:10 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/4114421638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T08:46:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:10 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/992792282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T08:46:10.758 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757308 got 163208757312 for osd.5 2024-04-21T08:46:10.758 DEBUG:teuthology.parallel:result is None 2024-04-21T08:46:10.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.526+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/3846851392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b4101960 msgr2=0x7fc2b4101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.526+0000 7fc2bbc9e700 1 --2- 172.21.15.82:0/3846851392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b4101960 0x7fc2b4101de0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009990 tx=0x7fc2b0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.526+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/3846851392 shutdown_connections 2024-04-21T08:46:10.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.526+0000 7fc2bbc9e700 1 --2- 172.21.15.82:0/3846851392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2b4102320 0x7fc2b410e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.526+0000 7fc2bbc9e700 1 --2- 172.21.15.82:0/3846851392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2b4107960 0x7fc2b4107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.526+0000 7fc2bbc9e700 1 --2- 172.21.15.82:0/3846851392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b4101960 0x7fc2b4101de0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.526+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/3846851392 >> 172.21.15.82:0/3846851392 conn(0x7fc2b40789e0 msgr2=0x7fc2b4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.526+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/3846851392 shutdown_connections 2024-04-21T08:46:10.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.526+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/3846851392 wait complete. 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2bbc9e700 1 Processor -- start 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2bbc9e700 1 -- start start 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2bbc9e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2b4101960 0x7fc2b4199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2bbc9e700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2b4102320 0x7fc2b4199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2bbc9e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b4107960 0x7fc2b419da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2bbc9e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4111ba0 con 0x7fc2b4107960 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2bbc9e700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4111a20 con 0x7fc2b4102320 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2bbc9e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4111d20 con 0x7fc2b4101960 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2ba23b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b4107960 0x7fc2b419da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2ba23b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b4107960 0x7fc2b419da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51044/0 (socket says 172.21.15.82:51044) 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2ba23b700 1 -- 172.21.15.82:0/904752474 learned_addr learned my addr 172.21.15.82:0/904752474 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2b9239700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2b4102320 0x7fc2b4199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2ba23b700 1 -- 172.21.15.82:0/904752474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2b4101960 msgr2=0x7fc2b4199140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:46:10.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2ba23b700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2b4101960 0x7fc2b4199140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2ba23b700 1 -- 172.21.15.82:0/904752474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2b4102320 msgr2=0x7fc2b4199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2ba23b700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2b4102320 0x7fc2b4199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.527+0000 7fc2ba23b700 1 -- 172.21.15.82:0/904752474 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b419e150 con 0x7fc2b4107960 2024-04-21T08:46:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.528+0000 7fc2ba23b700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b4107960 0x7fc2b419da10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac007ee0 tx=0x7fc2ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.528+0000 7fc2aaffd700 1 -- 172.21.15.82:0/904752474 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac00dbd0 con 0x7fc2b4107960 2024-04-21T08:46:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.528+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/904752474 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b419e440 con 0x7fc2b4107960 2024-04-21T08:46:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.528+0000 7fc2aaffd700 1 -- 172.21.15.82:0/904752474 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ac00dd70 con 0x7fc2b4107960 2024-04-21T08:46:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.528+0000 7fc2aaffd700 1 -- 172.21.15.82:0/904752474 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac005990 con 0x7fc2b4107960 2024-04-21T08:46:10.860 INFO:teuthology.orchestra.run.smithi082.stdout:287762808875 2024-04-21T08:46:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.528+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/904752474 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b41a5cc0 con 0x7fc2b4107960 2024-04-21T08:46:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.529+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/904752474 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b4069730 con 0x7fc2b4107960 2024-04-21T08:46:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.530+0000 7fc2aaffd700 1 -- 172.21.15.82:0/904752474 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc2ac01a020 con 0x7fc2b4107960 2024-04-21T08:46:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.530+0000 7fc2aaffd700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc2a00a3cd0 0x7fc2a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.531+0000 7fc2b9a3a700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc2a00a3cd0 0x7fc2a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.531+0000 7fc2aaffd700 1 -- 172.21.15.82:0/904752474 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fc2ac0d0770 con 0x7fc2b4107960 2024-04-21T08:46:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.531+0000 7fc2b9a3a700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc2a00a3cd0 0x7fc2a00a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009730 tx=0x7fc2a4006cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.534+0000 7fc2aaffd700 1 -- 172.21.15.82:0/904752474 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2ac010070 con 0x7fc2b4107960 2024-04-21T08:46:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.744+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/904752474 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fc2b402df80 con 0x7fc2b4107960 2024-04-21T08:46:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.744+0000 7fc2aaffd700 1 -- 172.21.15.82:0/904752474 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc2ac09ccb0 con 0x7fc2b4107960 2024-04-21T08:46:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.746+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/904752474 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc2a00a3cd0 msgr2=0x7fc2a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.746+0000 7fc2bbc9e700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc2a00a3cd0 0x7fc2a00a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009730 tx=0x7fc2a4006cb0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.746+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/904752474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b4107960 msgr2=0x7fc2b419da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:10.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.746+0000 7fc2bbc9e700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b4107960 0x7fc2b419da10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac007ee0 tx=0x7fc2ac00d3b0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.746+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/904752474 shutdown_connections 2024-04-21T08:46:10.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.746+0000 7fc2bbc9e700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc2a00a3cd0 0x7fc2a00a6190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.746+0000 7fc2bbc9e700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b4107960 0x7fc2b419da10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.746+0000 7fc2bbc9e700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2b4101960 0x7fc2b4199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.746+0000 7fc2bbc9e700 1 --2- 172.21.15.82:0/904752474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2b4102320 0x7fc2b4199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:10.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.746+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/904752474 >> 172.21.15.82:0/904752474 conn(0x7fc2b40789e0 msgr2=0x7fc2b406b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:10.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.746+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/904752474 shutdown_connections 2024-04-21T08:46:10.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:09.746+0000 7fc2bbc9e700 1 -- 172.21.15.82:0/904752474 wait complete. 2024-04-21T08:46:10.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:10 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/904752474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T08:46:10.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:10 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/4114421638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T08:46:10.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:10 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/992792282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T08:46:11.504 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:11 smithi082 ceph-mon[27967]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:46:11.531 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808871 got 287762808875 for osd.9 2024-04-21T08:46:11.531 DEBUG:teuthology.parallel:result is None 2024-04-21T08:46:11.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:11 smithi134 ceph-mon[31772]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:46:11.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:11 smithi106 ceph-mon[31072]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:46:12.255 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645345 got 309237645350 for osd.10 2024-04-21T08:46:12.255 DEBUG:teuthology.parallel:result is None 2024-04-21T08:46:12.256 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-21T08:46:12.256 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-21T08:46:12.500 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:46:13.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:13 smithi134 ceph-mon[31772]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:13.785 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:13 smithi082 ceph-mon[27967]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:13.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:13 smithi106 ceph-mon[31072]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:14.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.315+0000 7fef2cd7d700 1 -- 172.21.15.82:0/1472109218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef28102a90 msgr2=0x7fef28102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:14.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.315+0000 7fef2cd7d700 1 --2- 172.21.15.82:0/1472109218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef28102a90 0x7fef28102f10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009990 tx=0x7fef1c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:46:14.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.315+0000 7fef2cd7d700 1 -- 172.21.15.82:0/1472109218 shutdown_connections 2024-04-21T08:46:14.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.315+0000 7fef2cd7d700 1 --2- 172.21.15.82:0/1472109218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef28103450 0x7fef2810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.315+0000 7fef2cd7d700 1 --2- 172.21.15.82:0/1472109218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef28108a50 0x7fef28108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.315+0000 7fef2cd7d700 1 --2- 172.21.15.82:0/1472109218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef28102a90 0x7fef28102f10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.315+0000 7fef2cd7d700 1 -- 172.21.15.82:0/1472109218 >> 172.21.15.82:0/1472109218 conn(0x7fef28079ba0 msgr2=0x7fef28079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.316+0000 7fef2cd7d700 1 -- 172.21.15.82:0/1472109218 shutdown_connections 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.316+0000 7fef2cd7d700 1 -- 172.21.15.82:0/1472109218 wait complete. 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.316+0000 7fef2cd7d700 1 Processor -- start 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.316+0000 7fef2cd7d700 1 -- start start 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef2cd7d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef28102a90 0x7fef281992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef2cd7d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef28103450 0x7fef28199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef2cd7d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef28108a50 0x7fef2819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef2cd7d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fef28112d80 con 0x7fef28103450 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef2cd7d700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fef28112c00 con 0x7fef28102a90 2024-04-21T08:46:14.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef2cd7d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef28112f00 con 0x7fef28108a50 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef25d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef28103450 0x7fef28199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef25d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef28103450 0x7fef28199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38780/0 (socket says 172.21.15.82:38780) 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef25d9b700 1 -- 172.21.15.82:0/3091381054 learned_addr learned my addr 172.21.15.82:0/3091381054 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef26d9d700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef28108a50 0x7fef2819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef2659c700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef28102a90 0x7fef281992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef25d9b700 1 -- 172.21.15.82:0/3091381054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef28108a50 msgr2=0x7fef2819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef25d9b700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef28108a50 0x7fef2819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef25d9b700 1 -- 172.21.15.82:0/3091381054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef28102a90 msgr2=0x7fef281992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef25d9b700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef28102a90 0x7fef281992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef25d9b700 1 -- 172.21.15.82:0/3091381054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2819e300 con 0x7fef28103450 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef25d9b700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef28103450 0x7fef28199830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fef1c005910 tx=0x7fef1c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef177fe700 1 -- 172.21.15.82:0/3091381054 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c026070 con 0x7fef28103450 2024-04-21T08:46:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef2cd7d700 1 -- 172.21.15.82:0/3091381054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2819e590 con 0x7fef28103450 2024-04-21T08:46:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef177fe700 1 -- 172.21.15.82:0/3091381054 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef1c003f50 con 0x7fef28103450 2024-04-21T08:46:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef177fe700 1 -- 172.21.15.82:0/3091381054 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c00fd50 con 0x7fef28103450 2024-04-21T08:46:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.317+0000 7fef2cd7d700 1 -- 172.21.15.82:0/3091381054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef281a5e70 con 0x7fef28103450 2024-04-21T08:46:14.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.318+0000 7fef2cd7d700 1 -- 172.21.15.82:0/3091381054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef28104180 con 0x7fef28103450 2024-04-21T08:46:14.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.320+0000 7fef177fe700 1 -- 172.21.15.82:0/3091381054 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fef1c029070 con 0x7fef28103450 2024-04-21T08:46:14.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.320+0000 7fef177fe700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fef100a3d10 0x7fef100a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:14.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.320+0000 7fef2659c700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fef100a3d10 0x7fef100a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:14.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.320+0000 7fef177fe700 1 -- 172.21.15.82:0/3091381054 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fef1c0d1540 con 0x7fef28103450 2024-04-21T08:46:14.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.321+0000 7fef2659c700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fef100a3d10 0x7fef100a61d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fef18009a10 tx=0x7fef18009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:14.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.324+0000 7fef177fe700 1 -- 172.21.15.82:0/3091381054 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef1c030080 con 0x7fef28103450 2024-04-21T08:46:14.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.519+0000 7fef2cd7d700 1 -- 172.21.15.82:0/3091381054 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef28062e80 con 0x7fef100a3d10 2024-04-21T08:46:14.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.530+0000 7fef177fe700 1 -- 172.21.15.82:0/3091381054 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340392 (secure 0 0 0) 0x7fef28062e80 con 0x7fef100a3d10 2024-04-21T08:46:14.534 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:14.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.533+0000 7fef2cd7d700 1 -- 172.21.15.82:0/3091381054 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fef100a3d10 msgr2=0x7fef100a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:14.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.533+0000 7fef2cd7d700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fef100a3d10 0x7fef100a61d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fef18009a10 tx=0x7fef18009190 comp rx=0 tx=0).stop 2024-04-21T08:46:14.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.533+0000 7fef2cd7d700 1 -- 172.21.15.82:0/3091381054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef28103450 msgr2=0x7fef28199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:14.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.533+0000 7fef2cd7d700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef28103450 0x7fef28199830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fef1c005910 tx=0x7fef1c0047b0 comp rx=0 tx=0).stop 2024-04-21T08:46:14.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.533+0000 7fef2cd7d700 1 -- 172.21.15.82:0/3091381054 shutdown_connections 2024-04-21T08:46:14.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.533+0000 7fef2cd7d700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fef100a3d10 0x7fef100a61d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:14.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.533+0000 7fef2cd7d700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef28108a50 0x7fef2819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:14.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.533+0000 7fef2cd7d700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef28102a90 0x7fef281992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:14.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.533+0000 7fef2cd7d700 1 --2- 172.21.15.82:0/3091381054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef28103450 0x7fef28199830 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:14.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.533+0000 7fef2cd7d700 1 -- 172.21.15.82:0/3091381054 >> 172.21.15.82:0/3091381054 conn(0x7fef28079ba0 msgr2=0x7fef28101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:14.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.533+0000 7fef2cd7d700 1 -- 172.21.15.82:0/3091381054 shutdown_connections 2024-04-21T08:46:14.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:14.533+0000 7fef2cd7d700 1 -- 172.21.15.82:0/3091381054 wait complete. 2024-04-21T08:46:14.539 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-21T08:46:15.357 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-04-21T08:46:14.339185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658799,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":877,"num_read_kb":564,"num_write":421,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355136,"kb_used_data":14228,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586688,"statfs":{"total":1151940427776,"available":1151576768512,"internally_reserved":0,"allocated":14569472,"data_stored":7961447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003601"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.975144+0000","last_change":"2024-04-21T08:44:34.494393+0000","last_active":"2024-04-21T08:45:27.975144+0000","last_peered":"2024-04-21T08:45:27.975144+0000","last_clean":"2024-04-21T08:45:27.975144+0000","last_became_active":"2024-04-21T08:44:34.493583+0000","last_became_peered":"2024-04-21T08:44:34.493583+0000","last_unstale":"2024-04-21T08:45:27.975144+0000","last_undegraded":"2024-04-21T08:45:27.975144+0000","last_fullsized":"2024-04-21T08:45:27.975144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:37.211676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403224+0000","last_change":"2024-04-21T08:44:34.496291+0000","last_active":"2024-04-21T08:45:27.403224+0000","last_peered":"2024-04-21T08:45:27.403224+0000","last_clean":"2024-04-21T08:45:27.403224+0000","last_became_active":"2024-04-21T08:44:34.495019+0000","last_became_peered":"2024-04-21T08:44:34.495019+0000","last_unstale":"2024-04-21T08:45:27.403224+0000","last_undegraded":"2024-04-21T08:45:27.403224+0000","last_fullsized":"2024-04-21T08:45:27.403224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:05.309847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402953+0000","last_change":"2024-04-21T08:44:34.497948+0000","last_active":"2024-04-21T08:45:27.402953+0000","last_peered":"2024-04-21T08:45:27.402953+0000","last_clean":"2024-04-21T08:45:27.402953+0000","last_became_active":"2024-04-21T08:44:34.497657+0000","last_became_peered":"2024-04-21T08:44:34.497657+0000","last_unstale":"2024-04-21T08:45:27.402953+0000","last_undegraded":"2024-04-21T08:45:27.402953+0000","last_fullsized":"2024-04-21T08:45:27.402953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:11.463715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403161+0000","last_change":"2024-04-21T08:44:34.492682+0000","last_active":"2024-04-21T08:45:27.403161+0000","last_peered":"2024-04-21T08:45:27.403161+0000","last_clean":"2024-04-21T08:45:27.403161+0000","last_became_active":"2024-04-21T08:44:34.492230+0000","last_became_peered":"2024-04-21T08:44:34.492230+0000","last_unstale":"2024-04-21T08:45:27.403161+0000","last_undegraded":"2024-04-21T08:45:27.403161+0000","last_fullsized":"2024-04-21T08:45:27.403161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:40.365608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404211+0000","last_change":"2024-04-21T08:44:34.492160+0000","last_active":"2024-04-21T08:45:27.404211+0000","last_peered":"2024-04-21T08:45:27.404211+0000","last_clean":"2024-04-21T08:45:27.404211+0000","last_became_active":"2024-04-21T08:44:34.491701+0000","last_became_peered":"2024-04-21T08:44:34.491701+0000","last_unstale":"2024-04-21T08:45:27.404211+0000","last_undegraded":"2024-04-21T08:45:27.404211+0000","last_fullsized":"2024-04-21T08:45:27.404211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:50.542661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.517055+0000","last_change":"2024-04-21T08:44:34.497341+0000","last_active":"2024-04-21T08:45:27.517055+0000","last_peered":"2024-04-21T08:45:27.517055+0000","last_clean":"2024-04-21T08:45:27.517055+0000","last_became_active":"2024-04-21T08:44:34.495794+0000","last_became_peered":"2024-04-21T08:44:34.495794+0000","last_unstale":"2024-04-21T08:45:27.517055+0000","last_undegraded":"2024-04-21T08:45:27.517055+0000","last_fullsized":"2024-04-21T08:45:27.517055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:13.854753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515447+0000","last_change":"2024-04-21T08:44:34.501578+0000","last_active":"2024-04-21T08:45:27.515447+0000","last_peered":"2024-04-21T08:45:27.515447+0000","last_clean":"2024-04-21T08:45:27.515447+0000","last_became_active":"2024-04-21T08:44:34.500848+0000","last_became_peered":"2024-04-21T08:44:34.500848+0000","last_unstale":"2024-04-21T08:45:27.515447+0000","last_undegraded":"2024-04-21T08:45:27.515447+0000","last_fullsized":"2024-04-21T08:45:27.515447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:53:27.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.514764+0000","last_change":"2024-04-21T08:44:34.502975+0000","last_active":"2024-04-21T08:45:27.514764+0000","last_peered":"2024-04-21T08:45:27.514764+0000","last_clean":"2024-04-21T08:45:27.514764+0000","last_became_active":"2024-04-21T08:44:34.502676+0000","last_became_peered":"2024-04-21T08:44:34.502676+0000","last_unstale":"2024-04-21T08:45:27.514764+0000","last_undegraded":"2024-04-21T08:45:27.514764+0000","last_fullsized":"2024-04-21T08:45:27.514764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:32:13.881603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516244+0000","last_change":"2024-04-21T08:44:34.495703+0000","last_active":"2024-04-21T08:45:27.516244+0000","last_peered":"2024-04-21T08:45:27.516244+0000","last_clean":"2024-04-21T08:45:27.516244+0000","last_became_active":"2024-04-21T08:44:34.495380+0000","last_became_peered":"2024-04-21T08:44:34.495380+0000","last_unstale":"2024-04-21T08:45:27.516244+0000","last_undegraded":"2024-04-21T08:45:27.516244+0000","last_fullsized":"2024-04-21T08:45:27.516244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:45:28.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.517278+0000","last_change":"2024-04-21T08:44:34.499361+0000","last_active":"2024-04-21T08:45:27.517278+0000","last_peered":"2024-04-21T08:45:27.517278+0000","last_clean":"2024-04-21T08:45:27.517278+0000","last_became_active":"2024-04-21T08:44:34.499044+0000","last_became_peered":"2024-04-21T08:44:34.499044+0000","last_unstale":"2024-04-21T08:45:27.517278+0000","last_undegraded":"2024-04-21T08:45:27.517278+0000","last_fullsized":"2024-04-21T08:45:27.517278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:12.118985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516623+0000","last_change":"2024-04-21T08:44:34.499631+0000","last_active":"2024-04-21T08:45:27.516623+0000","last_peered":"2024-04-21T08:45:27.516623+0000","last_clean":"2024-04-21T08:45:27.516623+0000","last_became_active":"2024-04-21T08:44:34.499316+0000","last_became_peered":"2024-04-21T08:44:34.499316+0000","last_unstale":"2024-04-21T08:45:27.516623+0000","last_undegraded":"2024-04-21T08:45:27.516623+0000","last_fullsized":"2024-04-21T08:45:27.516623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:26.123017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403042+0000","last_change":"2024-04-21T08:44:34.504857+0000","last_active":"2024-04-21T08:45:27.403042+0000","last_peered":"2024-04-21T08:45:27.403042+0000","last_clean":"2024-04-21T08:45:27.403042+0000","last_became_active":"2024-04-21T08:44:34.504564+0000","last_became_peered":"2024-04-21T08:44:34.504564+0000","last_unstale":"2024-04-21T08:45:27.403042+0000","last_undegraded":"2024-04-21T08:45:27.403042+0000","last_fullsized":"2024-04-21T08:45:27.403042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:32:05.216162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402190+0000","last_change":"2024-04-21T08:44:34.484074+0000","last_active":"2024-04-21T08:45:27.402190+0000","last_peered":"2024-04-21T08:45:27.402190+0000","last_clean":"2024-04-21T08:45:27.402190+0000","last_became_active":"2024-04-21T08:44:34.483309+0000","last_became_peered":"2024-04-21T08:44:34.483309+0000","last_unstale":"2024-04-21T08:45:27.402190+0000","last_undegraded":"2024-04-21T08:45:27.402190+0000","last_fullsized":"2024-04-21T08:45:27.402190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:39.686048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402799+0000","last_change":"2024-04-21T08:44:34.499159+0000","last_active":"2024-04-21T08:45:27.402799+0000","last_peered":"2024-04-21T08:45:27.402799+0000","last_clean":"2024-04-21T08:45:27.402799+0000","last_became_active":"2024-04-21T08:44:34.496609+0000","last_became_peered":"2024-04-21T08:44:34.496609+0000","last_unstale":"2024-04-21T08:45:27.402799+0000","last_undegraded":"2024-04-21T08:45:27.402799+0000","last_fullsized":"2024-04-21T08:45:27.402799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:11.716321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.406543+0000","last_change":"2024-04-21T08:45:27.406543+0000","last_active":"2024-04-21T08:45:27.406543+0000","last_peered":"2024-04-21T08:45:27.406543+0000","last_clean":"2024-04-21T08:45:27.406543+0000","last_became_active":"2024-04-21T08:45:27.406190+0000","last_became_peered":"2024-04-21T08:45:27.406190+0000","last_unstale":"2024-04-21T08:45:27.406543+0000","last_undegraded":"2024-04-21T08:45:27.406543+0000","last_fullsized":"2024-04-21T08:45:27.406543+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:56:17.760163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516541+0000","last_change":"2024-04-21T08:44:34.494018+0000","last_active":"2024-04-21T08:45:27.516541+0000","last_peered":"2024-04-21T08:45:27.516541+0000","last_clean":"2024-04-21T08:45:27.516541+0000","last_became_active":"2024-04-21T08:44:34.493692+0000","last_became_peered":"2024-04-21T08:44:34.493692+0000","last_unstale":"2024-04-21T08:45:27.516541+0000","last_undegraded":"2024-04-21T08:45:27.516541+0000","last_fullsized":"2024-04-21T08:45:27.516541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:25:16.944160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974962+0000","last_change":"2024-04-21T08:44:34.493231+0000","last_active":"2024-04-21T08:45:27.974962+0000","last_peered":"2024-04-21T08:45:27.974962+0000","last_clean":"2024-04-21T08:45:27.974962+0000","last_became_active":"2024-04-21T08:44:34.492826+0000","last_became_peered":"2024-04-21T08:44:34.492826+0000","last_unstale":"2024-04-21T08:45:27.974962+0000","last_undegraded":"2024-04-21T08:45:27.974962+0000","last_fullsized":"2024-04-21T08:45:27.974962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:36.815012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.400005+0000","last_change":"2024-04-21T08:44:34.495142+0000","last_active":"2024-04-21T08:45:27.400005+0000","last_peered":"2024-04-21T08:45:27.400005+0000","last_clean":"2024-04-21T08:45:27.400005+0000","last_became_active":"2024-04-21T08:44:34.494803+0000","last_became_peered":"2024-04-21T08:44:34.494803+0000","last_unstale":"2024-04-21T08:45:27.400005+0000","last_undegraded":"2024-04-21T08:45:27.400005+0000","last_fullsized":"2024-04-21T08:45:27.400005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:22.974053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404980+0000","last_change":"2024-04-21T08:44:34.491252+0000","last_active":"2024-04-21T08:45:27.404980+0000","last_peered":"2024-04-21T08:45:27.404980+0000","last_clean":"2024-04-21T08:45:27.404980+0000","last_became_active":"2024-04-21T08:44:34.490814+0000","last_became_peered":"2024-04-21T08:44:34.490814+0000","last_unstale":"2024-04-21T08:45:27.404980+0000","last_undegraded":"2024-04-21T08:45:27.404980+0000","last_fullsized":"2024-04-21T08:45:27.404980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:54:52.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404453+0000","last_change":"2024-04-21T08:44:38.518647+0000","last_active":"2024-04-21T08:45:27.404453+0000","last_peered":"2024-04-21T08:45:27.404453+0000","last_clean":"2024-04-21T08:45:27.404453+0000","last_became_active":"2024-04-21T08:44:38.518239+0000","last_became_peered":"2024-04-21T08:44:38.518239+0000","last_unstale":"2024-04-21T08:45:27.404453+0000","last_undegraded":"2024-04-21T08:45:27.404453+0000","last_fullsized":"2024-04-21T08:45:27.404453+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:44.781542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":108,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:08.099422+0000","last_change":"2024-04-21T08:44:33.484484+0000","last_active":"2024-04-21T08:46:08.099422+0000","last_peered":"2024-04-21T08:46:08.099422+0000","last_clean":"2024-04-21T08:46:08.099422+0000","last_became_active":"2024-04-21T08:44:33.484160+0000","last_became_peered":"2024-04-21T08:44:33.484160+0000","last_unstale":"2024-04-21T08:46:08.099422+0000","last_undegraded":"2024-04-21T08:46:08.099422+0000","last_fullsized":"2024-04-21T08:46:08.099422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:45.962598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515007+0000","last_change":"2024-04-21T08:44:34.500722+0000","last_active":"2024-04-21T08:45:27.515007+0000","last_peered":"2024-04-21T08:45:27.515007+0000","last_clean":"2024-04-21T08:45:27.515007+0000","last_became_active":"2024-04-21T08:44:34.500388+0000","last_became_peered":"2024-04-21T08:44:34.500388+0000","last_unstale":"2024-04-21T08:45:27.515007+0000","last_undegraded":"2024-04-21T08:45:27.515007+0000","last_fullsized":"2024-04-21T08:45:27.515007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:05.876636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.518006+0000","last_change":"2024-04-21T08:45:27.518006+0000","last_active":"2024-04-21T08:45:27.518006+0000","last_peered":"2024-04-21T08:45:27.518006+0000","last_clean":"2024-04-21T08:45:27.518006+0000","last_became_active":"2024-04-21T08:45:27.517409+0000","last_became_peered":"2024-04-21T08:45:27.517409+0000","last_unstale":"2024-04-21T08:45:27.518006+0000","last_undegraded":"2024-04-21T08:45:27.518006+0000","last_fullsized":"2024-04-21T08:45:27.518006+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:45:01.253587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515217+0000","last_change":"2024-04-21T08:44:33.490508+0000","last_active":"2024-04-21T08:45:27.515217+0000","last_peered":"2024-04-21T08:45:27.515217+0000","last_clean":"2024-04-21T08:45:27.515217+0000","last_became_active":"2024-04-21T08:44:33.490084+0000","last_became_peered":"2024-04-21T08:44:33.490084+0000","last_unstale":"2024-04-21T08:45:27.515217+0000","last_undegraded":"2024-04-21T08:45:27.515217+0000","last_fullsized":"2024-04-21T08:45:27.515217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:21.553435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403196+0000","last_change":"2024-04-21T08:44:34.491843+0000","last_active":"2024-04-21T08:45:27.403196+0000","last_peered":"2024-04-21T08:45:27.403196+0000","last_clean":"2024-04-21T08:45:27.403196+0000","last_became_active":"2024-04-21T08:44:34.490578+0000","last_became_peered":"2024-04-21T08:44:34.490578+0000","last_unstale":"2024-04-21T08:45:27.403196+0000","last_undegraded":"2024-04-21T08:45:27.403196+0000","last_fullsized":"2024-04-21T08:45:27.403196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:05:12.272330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402322+0000","last_change":"2024-04-21T08:44:38.520258+0000","last_active":"2024-04-21T08:45:27.402322+0000","last_peered":"2024-04-21T08:45:27.402322+0000","last_clean":"2024-04-21T08:45:27.402322+0000","last_became_active":"2024-04-21T08:44:38.519824+0000","last_became_peered":"2024-04-21T08:44:38.519824+0000","last_unstale":"2024-04-21T08:45:27.402322+0000","last_undegraded":"2024-04-21T08:45:27.402322+0000","last_fullsized":"2024-04-21T08:45:27.402322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:39.716745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404909+0000","last_change":"2024-04-21T08:44:33.484934+0000","last_active":"2024-04-21T08:45:27.404909+0000","last_peered":"2024-04-21T08:45:27.404909+0000","last_clean":"2024-04-21T08:45:27.404909+0000","last_became_active":"2024-04-21T08:44:33.484591+0000","last_became_peered":"2024-04-21T08:44:33.484591+0000","last_unstale":"2024-04-21T08:45:27.404909+0000","last_undegraded":"2024-04-21T08:45:27.404909+0000","last_fullsized":"2024-04-21T08:45:27.404909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:48.826611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515909+0000","last_change":"2024-04-21T08:44:34.501109+0000","last_active":"2024-04-21T08:45:27.515909+0000","last_peered":"2024-04-21T08:45:27.515909+0000","last_clean":"2024-04-21T08:45:27.515909+0000","last_became_active":"2024-04-21T08:44:34.500750+0000","last_became_peered":"2024-04-21T08:44:34.500750+0000","last_unstale":"2024-04-21T08:45:27.515909+0000","last_undegraded":"2024-04-21T08:45:27.515909+0000","last_fullsized":"2024-04-21T08:45:27.515909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:29:28.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402344+0000","last_change":"2024-04-21T08:44:38.519633+0000","last_active":"2024-04-21T08:45:27.402344+0000","last_peered":"2024-04-21T08:45:27.402344+0000","last_clean":"2024-04-21T08:45:27.402344+0000","last_became_active":"2024-04-21T08:44:38.519215+0000","last_became_peered":"2024-04-21T08:44:38.519215+0000","last_unstale":"2024-04-21T08:45:27.402344+0000","last_undegraded":"2024-04-21T08:45:27.402344+0000","last_fullsized":"2024-04-21T08:45:27.402344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:43:59.695844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.405067+0000","last_change":"2024-04-21T08:44:33.483267+0000","last_active":"2024-04-21T08:45:27.405067+0000","last_peered":"2024-04-21T08:45:27.405067+0000","last_clean":"2024-04-21T08:45:27.405067+0000","last_became_active":"2024-04-21T08:44:33.482932+0000","last_became_peered":"2024-04-21T08:44:33.482932+0000","last_unstale":"2024-04-21T08:45:27.405067+0000","last_undegraded":"2024-04-21T08:45:27.405067+0000","last_fullsized":"2024-04-21T08:45:27.405067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:53.311550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408529+0000","last_change":"2024-04-21T08:44:34.497389+0000","last_active":"2024-04-21T08:45:27.408529+0000","last_peered":"2024-04-21T08:45:27.408529+0000","last_clean":"2024-04-21T08:45:27.408529+0000","last_became_active":"2024-04-21T08:44:34.497030+0000","last_became_peered":"2024-04-21T08:44:34.497030+0000","last_unstale":"2024-04-21T08:45:27.408529+0000","last_undegraded":"2024-04-21T08:45:27.408529+0000","last_fullsized":"2024-04-21T08:45:27.408529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:20.371888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404791+0000","last_change":"2024-04-21T08:44:38.518095+0000","last_active":"2024-04-21T08:45:27.404791+0000","last_peered":"2024-04-21T08:45:27.404791+0000","last_clean":"2024-04-21T08:45:27.404791+0000","last_became_active":"2024-04-21T08:44:38.517010+0000","last_became_peered":"2024-04-21T08:44:38.517010+0000","last_unstale":"2024-04-21T08:45:27.404791+0000","last_undegraded":"2024-04-21T08:45:27.404791+0000","last_fullsized":"2024-04-21T08:45:27.404791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:02.379512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:09.095019+0000","last_change":"2024-04-21T08:44:33.487999+0000","last_active":"2024-04-21T08:46:09.095019+0000","last_peered":"2024-04-21T08:46:09.095019+0000","last_clean":"2024-04-21T08:46:09.095019+0000","last_became_active":"2024-04-21T08:44:33.487666+0000","last_became_peered":"2024-04-21T08:44:33.487666+0000","last_unstale":"2024-04-21T08:46:09.095019+0000","last_undegraded":"2024-04-21T08:46:09.095019+0000","last_fullsized":"2024-04-21T08:46:09.095019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:01.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404363+0000","last_change":"2024-04-21T08:44:34.494213+0000","last_active":"2024-04-21T08:45:27.404363+0000","last_peered":"2024-04-21T08:45:27.404363+0000","last_clean":"2024-04-21T08:45:27.404363+0000","last_became_active":"2024-04-21T08:44:34.493853+0000","last_became_peered":"2024-04-21T08:44:34.493853+0000","last_unstale":"2024-04-21T08:45:27.404363+0000","last_undegraded":"2024-04-21T08:45:27.404363+0000","last_fullsized":"2024-04-21T08:45:27.404363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:42.499131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.410579+0000","last_change":"2024-04-21T08:45:27.410579+0000","last_active":"2024-04-21T08:45:27.410579+0000","last_peered":"2024-04-21T08:45:27.410579+0000","last_clean":"2024-04-21T08:45:27.410579+0000","last_became_active":"2024-04-21T08:45:27.409929+0000","last_became_peered":"2024-04-21T08:45:27.409929+0000","last_unstale":"2024-04-21T08:45:27.410579+0000","last_undegraded":"2024-04-21T08:45:27.410579+0000","last_fullsized":"2024-04-21T08:45:27.410579+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:35.897482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516909+0000","last_change":"2024-04-21T08:44:33.488751+0000","last_active":"2024-04-21T08:45:27.516909+0000","last_peered":"2024-04-21T08:45:27.516909+0000","last_clean":"2024-04-21T08:45:27.516909+0000","last_became_active":"2024-04-21T08:44:33.488425+0000","last_became_peered":"2024-04-21T08:44:33.488425+0000","last_unstale":"2024-04-21T08:45:27.516909+0000","last_undegraded":"2024-04-21T08:45:27.516909+0000","last_fullsized":"2024-04-21T08:45:27.516909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:28:47.868814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408306+0000","last_change":"2024-04-21T08:44:34.492770+0000","last_active":"2024-04-21T08:45:27.408306+0000","last_peered":"2024-04-21T08:45:27.408306+0000","last_clean":"2024-04-21T08:45:27.408306+0000","last_became_active":"2024-04-21T08:44:34.491340+0000","last_became_peered":"2024-04-21T08:44:34.491340+0000","last_unstale":"2024-04-21T08:45:27.408306+0000","last_undegraded":"2024-04-21T08:45:27.408306+0000","last_fullsized":"2024-04-21T08:45:27.408306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:47:54.280990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974951+0000","last_change":"2024-04-21T08:44:38.518888+0000","last_active":"2024-04-21T08:45:27.974951+0000","last_peered":"2024-04-21T08:45:27.974951+0000","last_clean":"2024-04-21T08:45:27.974951+0000","last_became_active":"2024-04-21T08:44:38.518421+0000","last_became_peered":"2024-04-21T08:44:38.518421+0000","last_unstale":"2024-04-21T08:45:27.974951+0000","last_undegraded":"2024-04-21T08:45:27.974951+0000","last_fullsized":"2024-04-21T08:45:27.974951+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:21.119364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.401525+0000","last_change":"2024-04-21T08:44:33.484383+0000","last_active":"2024-04-21T08:45:27.401525+0000","last_peered":"2024-04-21T08:45:27.401525+0000","last_clean":"2024-04-21T08:45:27.401525+0000","last_became_active":"2024-04-21T08:44:33.484083+0000","last_became_peered":"2024-04-21T08:44:33.484083+0000","last_unstale":"2024-04-21T08:45:27.401525+0000","last_undegraded":"2024-04-21T08:45:27.401525+0000","last_fullsized":"2024-04-21T08:45:27.401525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:13:14.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.514863+0000","last_change":"2024-04-21T08:44:34.495636+0000","last_active":"2024-04-21T08:45:27.514863+0000","last_peered":"2024-04-21T08:45:27.514863+0000","last_clean":"2024-04-21T08:45:27.514863+0000","last_became_active":"2024-04-21T08:44:34.494967+0000","last_became_peered":"2024-04-21T08:44:34.494967+0000","last_unstale":"2024-04-21T08:45:27.514863+0000","last_undegraded":"2024-04-21T08:45:27.514863+0000","last_fullsized":"2024-04-21T08:45:27.514863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:47:02.379208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403129+0000","last_change":"2024-04-21T08:44:38.515145+0000","last_active":"2024-04-21T08:45:27.403129+0000","last_peered":"2024-04-21T08:45:27.403129+0000","last_clean":"2024-04-21T08:45:27.403129+0000","last_became_active":"2024-04-21T08:44:38.514755+0000","last_became_peered":"2024-04-21T08:44:38.514755+0000","last_unstale":"2024-04-21T08:45:27.403129+0000","last_undegraded":"2024-04-21T08:45:27.403129+0000","last_fullsized":"2024-04-21T08:45:27.403129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:38:15.826293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408784+0000","last_change":"2024-04-21T08:44:33.484714+0000","last_active":"2024-04-21T08:45:27.408784+0000","last_peered":"2024-04-21T08:45:27.408784+0000","last_clean":"2024-04-21T08:45:27.408784+0000","last_became_active":"2024-04-21T08:44:33.484385+0000","last_became_peered":"2024-04-21T08:44:33.484385+0000","last_unstale":"2024-04-21T08:45:27.408784+0000","last_undegraded":"2024-04-21T08:45:27.408784+0000","last_fullsized":"2024-04-21T08:45:27.408784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:03.406860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399527+0000","last_change":"2024-04-21T08:44:34.490989+0000","last_active":"2024-04-21T08:45:27.399527+0000","last_peered":"2024-04-21T08:45:27.399527+0000","last_clean":"2024-04-21T08:45:27.399527+0000","last_became_active":"2024-04-21T08:44:34.490658+0000","last_became_peered":"2024-04-21T08:44:34.490658+0000","last_unstale":"2024-04-21T08:45:27.399527+0000","last_undegraded":"2024-04-21T08:45:27.399527+0000","last_fullsized":"2024-04-21T08:45:27.399527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:42:15.537913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515087+0000","last_change":"2024-04-21T08:44:38.519723+0000","last_active":"2024-04-21T08:45:27.515087+0000","last_peered":"2024-04-21T08:45:27.515087+0000","last_clean":"2024-04-21T08:45:27.515087+0000","last_became_active":"2024-04-21T08:44:38.519336+0000","last_became_peered":"2024-04-21T08:44:38.519336+0000","last_unstale":"2024-04-21T08:45:27.515087+0000","last_undegraded":"2024-04-21T08:45:27.515087+0000","last_fullsized":"2024-04-21T08:45:27.515087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:00.332153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":170,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:10.095841+0000","last_change":"2024-04-21T08:44:33.484404+0000","last_active":"2024-04-21T08:46:10.095841+0000","last_peered":"2024-04-21T08:46:10.095841+0000","last_clean":"2024-04-21T08:46:10.095841+0000","last_became_active":"2024-04-21T08:44:33.484088+0000","last_became_peered":"2024-04-21T08:44:33.484088+0000","last_unstale":"2024-04-21T08:46:10.095841+0000","last_undegraded":"2024-04-21T08:46:10.095841+0000","last_fullsized":"2024-04-21T08:46:10.095841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:41.948622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516481+0000","last_change":"2024-04-21T08:44:34.502598+0000","last_active":"2024-04-21T08:45:27.516481+0000","last_peered":"2024-04-21T08:45:27.516481+0000","last_clean":"2024-04-21T08:45:27.516481+0000","last_became_active":"2024-04-21T08:44:34.502057+0000","last_became_peered":"2024-04-21T08:44:34.502057+0000","last_unstale":"2024-04-21T08:45:27.516481+0000","last_undegraded":"2024-04-21T08:45:27.516481+0000","last_fullsized":"2024-04-21T08:45:27.516481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:24.052873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399525+0000","last_change":"2024-04-21T08:44:38.519699+0000","last_active":"2024-04-21T08:45:27.399525+0000","last_peered":"2024-04-21T08:45:27.399525+0000","last_clean":"2024-04-21T08:45:27.399525+0000","last_became_active":"2024-04-21T08:44:38.517655+0000","last_became_peered":"2024-04-21T08:44:38.517655+0000","last_unstale":"2024-04-21T08:45:27.399525+0000","last_undegraded":"2024-04-21T08:45:27.399525+0000","last_fullsized":"2024-04-21T08:45:27.399525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:12.784479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516206+0000","last_change":"2024-04-21T08:44:33.488799+0000","last_active":"2024-04-21T08:45:27.516206+0000","last_peered":"2024-04-21T08:45:27.516206+0000","last_clean":"2024-04-21T08:45:27.516206+0000","last_became_active":"2024-04-21T08:44:33.487948+0000","last_became_peered":"2024-04-21T08:44:33.487948+0000","last_unstale":"2024-04-21T08:45:27.516206+0000","last_undegraded":"2024-04-21T08:45:27.516206+0000","last_fullsized":"2024-04-21T08:45:27.516206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:57.347588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.518395+0000","last_change":"2024-04-21T08:45:27.518395+0000","last_active":"2024-04-21T08:45:27.518395+0000","last_peered":"2024-04-21T08:45:27.518395+0000","last_clean":"2024-04-21T08:45:27.518395+0000","last_became_active":"2024-04-21T08:45:27.517969+0000","last_became_peered":"2024-04-21T08:45:27.517969+0000","last_unstale":"2024-04-21T08:45:27.518395+0000","last_undegraded":"2024-04-21T08:45:27.518395+0000","last_fullsized":"2024-04-21T08:45:27.518395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:55.306556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402827+0000","last_change":"2024-04-21T08:44:38.517956+0000","last_active":"2024-04-21T08:45:27.402827+0000","last_peered":"2024-04-21T08:45:27.402827+0000","last_clean":"2024-04-21T08:45:27.402827+0000","last_became_active":"2024-04-21T08:44:38.517422+0000","last_became_peered":"2024-04-21T08:44:38.517422+0000","last_unstale":"2024-04-21T08:45:27.402827+0000","last_undegraded":"2024-04-21T08:45:27.402827+0000","last_fullsized":"2024-04-21T08:45:27.402827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:04.650784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408163+0000","last_change":"2024-04-21T08:44:33.489151+0000","last_active":"2024-04-21T08:45:27.408163+0000","last_peered":"2024-04-21T08:45:27.408163+0000","last_clean":"2024-04-21T08:45:27.408163+0000","last_became_active":"2024-04-21T08:44:33.488781+0000","last_became_peered":"2024-04-21T08:44:33.488781+0000","last_unstale":"2024-04-21T08:45:27.408163+0000","last_undegraded":"2024-04-21T08:45:27.408163+0000","last_fullsized":"2024-04-21T08:45:27.408163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:40.675100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516109+0000","last_change":"2024-04-21T08:44:34.496528+0000","last_active":"2024-04-21T08:45:27.516109+0000","last_peered":"2024-04-21T08:45:27.516109+0000","last_clean":"2024-04-21T08:45:27.516109+0000","last_became_active":"2024-04-21T08:44:34.496223+0000","last_became_peered":"2024-04-21T08:44:34.496223+0000","last_unstale":"2024-04-21T08:45:27.516109+0000","last_undegraded":"2024-04-21T08:45:27.516109+0000","last_fullsized":"2024-04-21T08:45:27.516109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:56.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399531+0000","last_change":"2024-04-21T08:44:38.517537+0000","last_active":"2024-04-21T08:45:27.399531+0000","last_peered":"2024-04-21T08:45:27.399531+0000","last_clean":"2024-04-21T08:45:27.399531+0000","last_became_active":"2024-04-21T08:44:38.517134+0000","last_became_peered":"2024-04-21T08:44:38.517134+0000","last_unstale":"2024-04-21T08:45:27.399531+0000","last_undegraded":"2024-04-21T08:45:27.399531+0000","last_fullsized":"2024-04-21T08:45:27.399531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:38.634418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":208,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:10.103065+0000","last_change":"2024-04-21T08:44:33.485582+0000","last_active":"2024-04-21T08:46:10.103065+0000","last_peered":"2024-04-21T08:46:10.103065+0000","last_clean":"2024-04-21T08:46:10.103065+0000","last_became_active":"2024-04-21T08:44:33.485249+0000","last_became_peered":"2024-04-21T08:44:33.485249+0000","last_unstale":"2024-04-21T08:46:10.103065+0000","last_undegraded":"2024-04-21T08:46:10.103065+0000","last_fullsized":"2024-04-21T08:46:10.103065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:16:10.571694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61775,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":135,"num_write":35,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404559+0000","last_change":"2024-04-21T08:44:34.491361+0000","last_active":"2024-04-21T08:45:27.404559+0000","last_peered":"2024-04-21T08:45:27.404559+0000","last_clean":"2024-04-21T08:45:27.404559+0000","last_became_active":"2024-04-21T08:44:34.491050+0000","last_became_peered":"2024-04-21T08:44:34.491050+0000","last_unstale":"2024-04-21T08:45:27.404559+0000","last_undegraded":"2024-04-21T08:45:27.404559+0000","last_fullsized":"2024-04-21T08:45:27.404559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:39.082448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408809+0000","last_change":"2024-04-21T08:44:38.528745+0000","last_active":"2024-04-21T08:45:27.408809+0000","last_peered":"2024-04-21T08:45:27.408809+0000","last_clean":"2024-04-21T08:45:27.408809+0000","last_became_active":"2024-04-21T08:44:38.528275+0000","last_became_peered":"2024-04-21T08:44:38.528275+0000","last_unstale":"2024-04-21T08:45:27.408809+0000","last_undegraded":"2024-04-21T08:45:27.408809+0000","last_fullsized":"2024-04-21T08:45:27.408809+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:06:11.338420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403537+0000","last_change":"2024-04-21T08:44:33.482059+0000","last_active":"2024-04-21T08:45:27.403537+0000","last_peered":"2024-04-21T08:45:27.403537+0000","last_clean":"2024-04-21T08:45:27.403537+0000","last_became_active":"2024-04-21T08:44:33.481663+0000","last_became_peered":"2024-04-21T08:44:33.481663+0000","last_unstale":"2024-04-21T08:45:27.403537+0000","last_undegraded":"2024-04-21T08:45:27.403537+0000","last_fullsized":"2024-04-21T08:45:27.403537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:01.240452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.401749+0000","last_change":"2024-04-21T08:44:34.491398+0000","last_active":"2024-04-21T08:45:27.401749+0000","last_peered":"2024-04-21T08:45:27.401749+0000","last_clean":"2024-04-21T08:45:27.401749+0000","last_became_active":"2024-04-21T08:44:34.491073+0000","last_became_peered":"2024-04-21T08:44:34.491073+0000","last_unstale":"2024-04-21T08:45:27.401749+0000","last_undegraded":"2024-04-21T08:45:27.401749+0000","last_fullsized":"2024-04-21T08:45:27.401749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:22.566063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974571+0000","last_change":"2024-04-21T08:44:38.515653+0000","last_active":"2024-04-21T08:45:27.974571+0000","last_peered":"2024-04-21T08:45:27.974571+0000","last_clean":"2024-04-21T08:45:27.974571+0000","last_became_active":"2024-04-21T08:44:38.514796+0000","last_became_peered":"2024-04-21T08:44:38.514796+0000","last_unstale":"2024-04-21T08:45:27.974571+0000","last_undegraded":"2024-04-21T08:45:27.974571+0000","last_fullsized":"2024-04-21T08:45:27.974571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:00:22.662136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408084+0000","last_change":"2024-04-21T08:44:33.487073+0000","last_active":"2024-04-21T08:45:27.408084+0000","last_peered":"2024-04-21T08:45:27.408084+0000","last_clean":"2024-04-21T08:45:27.408084+0000","last_became_active":"2024-04-21T08:44:33.486740+0000","last_became_peered":"2024-04-21T08:44:33.486740+0000","last_unstale":"2024-04-21T08:45:27.408084+0000","last_undegraded":"2024-04-21T08:45:27.408084+0000","last_fullsized":"2024-04-21T08:45:27.408084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:32.673074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516008+0000","last_change":"2024-04-21T08:44:34.496757+0000","last_active":"2024-04-21T08:45:27.516008+0000","last_peered":"2024-04-21T08:45:27.516008+0000","last_clean":"2024-04-21T08:45:27.516008+0000","last_became_active":"2024-04-21T08:44:34.496422+0000","last_became_peered":"2024-04-21T08:44:34.496422+0000","last_unstale":"2024-04-21T08:45:27.516008+0000","last_undegraded":"2024-04-21T08:45:27.516008+0000","last_fullsized":"2024-04-21T08:45:27.516008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:08.686810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399726+0000","last_change":"2024-04-21T08:44:38.520935+0000","last_active":"2024-04-21T08:45:27.399726+0000","last_peered":"2024-04-21T08:45:27.399726+0000","last_clean":"2024-04-21T08:45:27.399726+0000","last_became_active":"2024-04-21T08:44:38.520481+0000","last_became_peered":"2024-04-21T08:44:38.520481+0000","last_unstale":"2024-04-21T08:45:27.399726+0000","last_undegraded":"2024-04-21T08:45:27.399726+0000","last_fullsized":"2024-04-21T08:45:27.399726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:12.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:09.094172+0000","last_change":"2024-04-21T08:44:33.485797+0000","last_active":"2024-04-21T08:46:09.094172+0000","last_peered":"2024-04-21T08:46:09.094172+0000","last_clean":"2024-04-21T08:46:09.094172+0000","last_became_active":"2024-04-21T08:44:33.485496+0000","last_became_peered":"2024-04-21T08:44:33.485496+0000","last_unstale":"2024-04-21T08:46:09.094172+0000","last_undegraded":"2024-04-21T08:46:09.094172+0000","last_fullsized":"2024-04-21T08:46:09.094172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:11:34.604611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515510+0000","last_change":"2024-04-21T08:44:34.506453+0000","last_active":"2024-04-21T08:45:27.515510+0000","last_peered":"2024-04-21T08:45:27.515510+0000","last_clean":"2024-04-21T08:45:27.515510+0000","last_became_active":"2024-04-21T08:44:34.506121+0000","last_became_peered":"2024-04-21T08:44:34.506121+0000","last_unstale":"2024-04-21T08:45:27.515510+0000","last_undegraded":"2024-04-21T08:45:27.515510+0000","last_fullsized":"2024-04-21T08:45:27.515510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:07.927366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402163+0000","last_change":"2024-04-21T08:44:38.523718+0000","last_active":"2024-04-21T08:45:27.402163+0000","last_peered":"2024-04-21T08:45:27.402163+0000","last_clean":"2024-04-21T08:45:27.402163+0000","last_became_active":"2024-04-21T08:44:38.523003+0000","last_became_peered":"2024-04-21T08:44:38.523003+0000","last_unstale":"2024-04-21T08:45:27.402163+0000","last_undegraded":"2024-04-21T08:45:27.402163+0000","last_fullsized":"2024-04-21T08:45:27.402163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:20.837450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404019+0000","last_change":"2024-04-21T08:44:33.482029+0000","last_active":"2024-04-21T08:45:27.404019+0000","last_peered":"2024-04-21T08:45:27.404019+0000","last_clean":"2024-04-21T08:45:27.404019+0000","last_became_active":"2024-04-21T08:44:33.481713+0000","last_became_peered":"2024-04-21T08:44:33.481713+0000","last_unstale":"2024-04-21T08:45:27.404019+0000","last_undegraded":"2024-04-21T08:45:27.404019+0000","last_fullsized":"2024-04-21T08:45:27.404019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:46:34.717244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408714+0000","last_change":"2024-04-21T08:44:34.491944+0000","last_active":"2024-04-21T08:45:27.408714+0000","last_peered":"2024-04-21T08:45:27.408714+0000","last_clean":"2024-04-21T08:45:27.408714+0000","last_became_active":"2024-04-21T08:44:34.491638+0000","last_became_peered":"2024-04-21T08:44:34.491638+0000","last_unstale":"2024-04-21T08:45:27.408714+0000","last_undegraded":"2024-04-21T08:45:27.408714+0000","last_fullsized":"2024-04-21T08:45:27.408714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:08:07.925937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408288+0000","last_change":"2024-04-21T08:44:38.528971+0000","last_active":"2024-04-21T08:45:27.408288+0000","last_peered":"2024-04-21T08:45:27.408288+0000","last_clean":"2024-04-21T08:45:27.408288+0000","last_became_active":"2024-04-21T08:44:38.528453+0000","last_became_peered":"2024-04-21T08:44:38.528453+0000","last_unstale":"2024-04-21T08:45:27.408288+0000","last_undegraded":"2024-04-21T08:45:27.408288+0000","last_fullsized":"2024-04-21T08:45:27.408288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:50:31.237749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.406121+0000","last_change":"2024-04-21T08:44:33.479262+0000","last_active":"2024-04-21T08:45:27.406121+0000","last_peered":"2024-04-21T08:45:27.406121+0000","last_clean":"2024-04-21T08:45:27.406121+0000","last_became_active":"2024-04-21T08:44:33.478907+0000","last_became_peered":"2024-04-21T08:44:33.478907+0000","last_unstale":"2024-04-21T08:45:27.406121+0000","last_undegraded":"2024-04-21T08:45:27.406121+0000","last_fullsized":"2024-04-21T08:45:27.406121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:33:43.879126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974889+0000","last_change":"2024-04-21T08:44:34.498503+0000","last_active":"2024-04-21T08:45:27.974889+0000","last_peered":"2024-04-21T08:45:27.974889+0000","last_clean":"2024-04-21T08:45:27.974889+0000","last_became_active":"2024-04-21T08:44:34.495140+0000","last_became_peered":"2024-04-21T08:44:34.495140+0000","last_unstale":"2024-04-21T08:45:27.974889+0000","last_undegraded":"2024-04-21T08:45:27.974889+0000","last_fullsized":"2024-04-21T08:45:27.974889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:24.518349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404212+0000","last_change":"2024-04-21T08:44:38.517717+0000","last_active":"2024-04-21T08:45:27.404212+0000","last_peered":"2024-04-21T08:45:27.404212+0000","last_clean":"2024-04-21T08:45:27.404212+0000","last_became_active":"2024-04-21T08:44:38.517272+0000","last_became_peered":"2024-04-21T08:44:38.517272+0000","last_unstale":"2024-04-21T08:45:27.404212+0000","last_undegraded":"2024-04-21T08:45:27.404212+0000","last_fullsized":"2024-04-21T08:45:27.404212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.664130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399781+0000","last_change":"2024-04-21T08:44:33.477498+0000","last_active":"2024-04-21T08:45:27.399781+0000","last_peered":"2024-04-21T08:45:27.399781+0000","last_clean":"2024-04-21T08:45:27.399781+0000","last_became_active":"2024-04-21T08:44:33.477139+0000","last_became_peered":"2024-04-21T08:44:33.477139+0000","last_unstale":"2024-04-21T08:45:27.399781+0000","last_undegraded":"2024-04-21T08:45:27.399781+0000","last_fullsized":"2024-04-21T08:45:27.399781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:26:59.390583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.517425+0000","last_change":"2024-04-21T08:44:34.493586+0000","last_active":"2024-04-21T08:45:27.517425+0000","last_peered":"2024-04-21T08:45:27.517425+0000","last_clean":"2024-04-21T08:45:27.517425+0000","last_became_active":"2024-04-21T08:44:34.493258+0000","last_became_peered":"2024-04-21T08:44:34.493258+0000","last_unstale":"2024-04-21T08:45:27.517425+0000","last_undegraded":"2024-04-21T08:45:27.517425+0000","last_fullsized":"2024-04-21T08:45:27.517425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:11:18.783332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.401955+0000","last_change":"2024-04-21T08:44:38.519783+0000","last_active":"2024-04-21T08:45:27.401955+0000","last_peered":"2024-04-21T08:45:27.401955+0000","last_clean":"2024-04-21T08:45:27.401955+0000","last_became_active":"2024-04-21T08:44:38.519397+0000","last_became_peered":"2024-04-21T08:44:38.519397+0000","last_unstale":"2024-04-21T08:45:27.401955+0000","last_undegraded":"2024-04-21T08:45:27.401955+0000","last_fullsized":"2024-04-21T08:45:27.401955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:08.740240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:11.103629+0000","last_change":"2024-04-21T08:44:33.490098+0000","last_active":"2024-04-21T08:46:11.103629+0000","last_peered":"2024-04-21T08:46:11.103629+0000","last_clean":"2024-04-21T08:46:11.103629+0000","last_became_active":"2024-04-21T08:44:33.489691+0000","last_became_peered":"2024-04-21T08:44:33.489691+0000","last_unstale":"2024-04-21T08:46:11.103629+0000","last_undegraded":"2024-04-21T08:46:11.103629+0000","last_fullsized":"2024-04-21T08:46:11.103629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:44.965998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516106+0000","last_change":"2024-04-21T08:44:34.498543+0000","last_active":"2024-04-21T08:45:27.516106+0000","last_peered":"2024-04-21T08:45:27.516106+0000","last_clean":"2024-04-21T08:45:27.516106+0000","last_became_active":"2024-04-21T08:44:34.498211+0000","last_became_peered":"2024-04-21T08:44:34.498211+0000","last_unstale":"2024-04-21T08:45:27.516106+0000","last_undegraded":"2024-04-21T08:45:27.516106+0000","last_fullsized":"2024-04-21T08:45:27.516106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:16.506695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.401604+0000","last_change":"2024-04-21T08:44:38.521010+0000","last_active":"2024-04-21T08:45:27.401604+0000","last_peered":"2024-04-21T08:45:27.401604+0000","last_clean":"2024-04-21T08:45:27.401604+0000","last_became_active":"2024-04-21T08:44:38.520605+0000","last_became_peered":"2024-04-21T08:44:38.520605+0000","last_unstale":"2024-04-21T08:45:27.401604+0000","last_undegraded":"2024-04-21T08:45:27.401604+0000","last_fullsized":"2024-04-21T08:45:27.401604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:50.378803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.410137+0000","last_change":"2024-04-21T08:45:27.410137+0000","last_active":"2024-04-21T08:45:27.410137+0000","last_peered":"2024-04-21T08:45:27.410137+0000","last_clean":"2024-04-21T08:45:27.410137+0000","last_became_active":"2024-04-21T08:45:27.409728+0000","last_became_peered":"2024-04-21T08:45:27.409728+0000","last_unstale":"2024-04-21T08:45:27.410137+0000","last_undegraded":"2024-04-21T08:45:27.410137+0000","last_fullsized":"2024-04-21T08:45:27.410137+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:17.934902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:10.094832+0000","last_change":"2024-04-21T08:44:33.494031+0000","last_active":"2024-04-21T08:46:10.094832+0000","last_peered":"2024-04-21T08:46:10.094832+0000","last_clean":"2024-04-21T08:46:10.094832+0000","last_became_active":"2024-04-21T08:44:33.493580+0000","last_became_peered":"2024-04-21T08:44:33.493580+0000","last_unstale":"2024-04-21T08:46:10.094832+0000","last_undegraded":"2024-04-21T08:46:10.094832+0000","last_fullsized":"2024-04-21T08:46:10.094832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:39:48.230616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515535+0000","last_change":"2024-04-21T08:44:38.518067+0000","last_active":"2024-04-21T08:45:27.515535+0000","last_peered":"2024-04-21T08:45:27.515535+0000","last_clean":"2024-04-21T08:45:27.515535+0000","last_became_active":"2024-04-21T08:44:38.517594+0000","last_became_peered":"2024-04-21T08:44:38.517594+0000","last_unstale":"2024-04-21T08:45:27.515535+0000","last_undegraded":"2024-04-21T08:45:27.515535+0000","last_fullsized":"2024-04-21T08:45:27.515535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:35.261582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.518194+0000","last_change":"2024-04-21T08:45:27.517719+0000","last_active":"2024-04-21T08:45:27.518194+0000","last_peered":"2024-04-21T08:45:27.518194+0000","last_clean":"2024-04-21T08:45:27.518194+0000","last_became_active":"2024-04-21T08:45:27.517429+0000","last_became_peered":"2024-04-21T08:45:27.517429+0000","last_unstale":"2024-04-21T08:45:27.518194+0000","last_undegraded":"2024-04-21T08:45:27.518194+0000","last_fullsized":"2024-04-21T08:45:27.518194+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:49:34.097759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402575+0000","last_change":"2024-04-21T08:44:33.485110+0000","last_active":"2024-04-21T08:45:27.402575+0000","last_peered":"2024-04-21T08:45:27.402575+0000","last_clean":"2024-04-21T08:45:27.402575+0000","last_became_active":"2024-04-21T08:44:33.484722+0000","last_became_peered":"2024-04-21T08:44:33.484722+0000","last_unstale":"2024-04-21T08:45:27.402575+0000","last_undegraded":"2024-04-21T08:45:27.402575+0000","last_fullsized":"2024-04-21T08:45:27.402575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:47:16.773560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":362,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515276+0000","last_change":"2024-04-21T08:43:17.984594+0000","last_active":"2024-04-21T08:45:27.515276+0000","last_peered":"2024-04-21T08:45:27.515276+0000","last_clean":"2024-04-21T08:45:27.515276+0000","last_became_active":"2024-04-21T08:43:17.984207+0000","last_became_peered":"2024-04-21T08:43:17.984207+0000","last_unstale":"2024-04-21T08:45:27.515276+0000","last_undegraded":"2024-04-21T08:45:27.515276+0000","last_fullsized":"2024-04-21T08:45:27.515276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_clean_scrub_stamp":"2024-04-21T08:39:50.910206+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:35.260499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.519344+0000","last_change":"2024-04-21T08:45:27.518834+0000","last_active":"2024-04-21T08:45:27.519344+0000","last_peered":"2024-04-21T08:45:27.519344+0000","last_clean":"2024-04-21T08:45:27.519344+0000","last_became_active":"2024-04-21T08:45:27.518390+0000","last_became_peered":"2024-04-21T08:45:27.518390+0000","last_unstale":"2024-04-21T08:45:27.519344+0000","last_undegraded":"2024-04-21T08:45:27.519344+0000","last_fullsized":"2024-04-21T08:45:27.519344+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:58:32.673008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.401412+0000","last_change":"2024-04-21T08:44:34.499975+0000","last_active":"2024-04-21T08:45:27.401412+0000","last_peered":"2024-04-21T08:45:27.401412+0000","last_clean":"2024-04-21T08:45:27.401412+0000","last_became_active":"2024-04-21T08:44:34.499646+0000","last_became_peered":"2024-04-21T08:44:34.499646+0000","last_unstale":"2024-04-21T08:45:27.401412+0000","last_undegraded":"2024-04-21T08:45:27.401412+0000","last_fullsized":"2024-04-21T08:45:27.401412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:19:38.082502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515346+0000","last_change":"2024-04-21T08:44:33.484449+0000","last_active":"2024-04-21T08:45:27.515346+0000","last_peered":"2024-04-21T08:45:27.515346+0000","last_clean":"2024-04-21T08:45:27.515346+0000","last_became_active":"2024-04-21T08:44:33.484155+0000","last_became_peered":"2024-04-21T08:44:33.484155+0000","last_unstale":"2024-04-21T08:45:27.515346+0000","last_undegraded":"2024-04-21T08:45:27.515346+0000","last_fullsized":"2024-04-21T08:45:27.515346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:03.480474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974663+0000","last_change":"2024-04-21T08:44:38.515283+0000","last_active":"2024-04-21T08:45:27.974663+0000","last_peered":"2024-04-21T08:45:27.974663+0000","last_clean":"2024-04-21T08:45:27.974663+0000","last_became_active":"2024-04-21T08:44:38.514764+0000","last_became_peered":"2024-04-21T08:44:38.514764+0000","last_unstale":"2024-04-21T08:45:27.974663+0000","last_undegraded":"2024-04-21T08:45:27.974663+0000","last_fullsized":"2024-04-21T08:45:27.974663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:17:36.378392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404687+0000","last_change":"2024-04-21T08:44:33.489198+0000","last_active":"2024-04-21T08:45:27.404687+0000","last_peered":"2024-04-21T08:45:27.404687+0000","last_clean":"2024-04-21T08:45:27.404687+0000","last_became_active":"2024-04-21T08:44:33.488794+0000","last_became_peered":"2024-04-21T08:44:33.488794+0000","last_unstale":"2024-04-21T08:45:27.404687+0000","last_undegraded":"2024-04-21T08:45:27.404687+0000","last_fullsized":"2024-04-21T08:45:27.404687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:24.948283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402060+0000","last_change":"2024-04-21T08:44:34.488811+0000","last_active":"2024-04-21T08:45:27.402060+0000","last_peered":"2024-04-21T08:45:27.402060+0000","last_clean":"2024-04-21T08:45:27.402060+0000","last_became_active":"2024-04-21T08:44:34.488472+0000","last_became_peered":"2024-04-21T08:44:34.488472+0000","last_unstale":"2024-04-21T08:45:27.402060+0000","last_undegraded":"2024-04-21T08:45:27.402060+0000","last_fullsized":"2024-04-21T08:45:27.402060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:44.779492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.514792+0000","last_change":"2024-04-21T08:44:38.522711+0000","last_active":"2024-04-21T08:45:27.514792+0000","last_peered":"2024-04-21T08:45:27.514792+0000","last_clean":"2024-04-21T08:45:27.514792+0000","last_became_active":"2024-04-21T08:44:38.522322+0000","last_became_peered":"2024-04-21T08:44:38.522322+0000","last_unstale":"2024-04-21T08:45:27.514792+0000","last_undegraded":"2024-04-21T08:45:27.514792+0000","last_fullsized":"2024-04-21T08:45:27.514792+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:26:13.323283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399984+0000","last_change":"2024-04-21T08:44:33.481815+0000","last_active":"2024-04-21T08:45:27.399984+0000","last_peered":"2024-04-21T08:45:27.399984+0000","last_clean":"2024-04-21T08:45:27.399984+0000","last_became_active":"2024-04-21T08:44:33.481469+0000","last_became_peered":"2024-04-21T08:44:33.481469+0000","last_unstale":"2024-04-21T08:45:27.399984+0000","last_undegraded":"2024-04-21T08:45:27.399984+0000","last_fullsized":"2024-04-21T08:45:27.399984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:13.970821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515241+0000","last_change":"2024-04-21T08:44:34.496996+0000","last_active":"2024-04-21T08:45:27.515241+0000","last_peered":"2024-04-21T08:45:27.515241+0000","last_clean":"2024-04-21T08:45:27.515241+0000","last_became_active":"2024-04-21T08:44:34.496676+0000","last_became_peered":"2024-04-21T08:44:34.496676+0000","last_unstale":"2024-04-21T08:45:27.515241+0000","last_undegraded":"2024-04-21T08:45:27.515241+0000","last_fullsized":"2024-04-21T08:45:27.515241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:43:08.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404218+0000","last_change":"2024-04-21T08:44:38.516240+0000","last_active":"2024-04-21T08:45:27.404218+0000","last_peered":"2024-04-21T08:45:27.404218+0000","last_clean":"2024-04-21T08:45:27.404218+0000","last_became_active":"2024-04-21T08:44:38.515801+0000","last_became_peered":"2024-04-21T08:44:38.515801+0000","last_unstale":"2024-04-21T08:45:27.404218+0000","last_undegraded":"2024-04-21T08:45:27.404218+0000","last_fullsized":"2024-04-21T08:45:27.404218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:46.520373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403282+0000","last_change":"2024-04-21T08:44:33.488952+0000","last_active":"2024-04-21T08:45:27.403282+0000","last_peered":"2024-04-21T08:45:27.403282+0000","last_clean":"2024-04-21T08:45:27.403282+0000","last_became_active":"2024-04-21T08:44:33.488617+0000","last_became_peered":"2024-04-21T08:44:33.488617+0000","last_unstale":"2024-04-21T08:45:27.403282+0000","last_undegraded":"2024-04-21T08:45:27.403282+0000","last_fullsized":"2024-04-21T08:45:27.403282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:37:24.679132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403163+0000","last_change":"2024-04-21T08:44:34.510568+0000","last_active":"2024-04-21T08:45:27.403163+0000","last_peered":"2024-04-21T08:45:27.403163+0000","last_clean":"2024-04-21T08:45:27.403163+0000","last_became_active":"2024-04-21T08:44:34.509900+0000","last_became_peered":"2024-04-21T08:44:34.509900+0000","last_unstale":"2024-04-21T08:45:27.403163+0000","last_undegraded":"2024-04-21T08:45:27.403163+0000","last_fullsized":"2024-04-21T08:45:27.403163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:08.426141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.976874+0000","last_change":"2024-04-21T08:45:27.976874+0000","last_active":"2024-04-21T08:45:27.976874+0000","last_peered":"2024-04-21T08:45:27.976874+0000","last_clean":"2024-04-21T08:45:27.976874+0000","last_became_active":"2024-04-21T08:45:27.976387+0000","last_became_peered":"2024-04-21T08:45:27.976387+0000","last_unstale":"2024-04-21T08:45:27.976874+0000","last_undegraded":"2024-04-21T08:45:27.976874+0000","last_fullsized":"2024-04-21T08:45:27.976874+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:38:18.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515795+0000","last_change":"2024-04-21T08:44:33.481803+0000","last_active":"2024-04-21T08:45:27.515795+0000","last_peered":"2024-04-21T08:45:27.515795+0000","last_clean":"2024-04-21T08:45:27.515795+0000","last_became_active":"2024-04-21T08:44:33.481256+0000","last_became_peered":"2024-04-21T08:44:33.481256+0000","last_unstale":"2024-04-21T08:45:27.515795+0000","last_undegraded":"2024-04-21T08:45:27.515795+0000","last_fullsized":"2024-04-21T08:45:27.515795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:17.804557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404089+0000","last_change":"2024-04-21T08:44:34.490206+0000","last_active":"2024-04-21T08:45:27.404089+0000","last_peered":"2024-04-21T08:45:27.404089+0000","last_clean":"2024-04-21T08:45:27.404089+0000","last_became_active":"2024-04-21T08:44:34.489797+0000","last_became_peered":"2024-04-21T08:44:34.489797+0000","last_unstale":"2024-04-21T08:45:27.404089+0000","last_undegraded":"2024-04-21T08:45:27.404089+0000","last_fullsized":"2024-04-21T08:45:27.404089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:09.055429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399727+0000","last_change":"2024-04-21T08:44:38.517385+0000","last_active":"2024-04-21T08:45:27.399727+0000","last_peered":"2024-04-21T08:45:27.399727+0000","last_clean":"2024-04-21T08:45:27.399727+0000","last_became_active":"2024-04-21T08:44:38.516964+0000","last_became_peered":"2024-04-21T08:44:38.516964+0000","last_unstale":"2024-04-21T08:45:27.399727+0000","last_undegraded":"2024-04-21T08:45:27.399727+0000","last_fullsized":"2024-04-21T08:45:27.399727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:33.364076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.514578+0000","last_change":"2024-04-21T08:44:33.490586+0000","last_active":"2024-04-21T08:45:27.514578+0000","last_peered":"2024-04-21T08:45:27.514578+0000","last_clean":"2024-04-21T08:45:27.514578+0000","last_became_active":"2024-04-21T08:44:33.489281+0000","last_became_peered":"2024-04-21T08:44:33.489281+0000","last_unstale":"2024-04-21T08:45:27.514578+0000","last_undegraded":"2024-04-21T08:45:27.514578+0000","last_fullsized":"2024-04-21T08:45:27.514578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:41:29.856165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974475+0000","last_change":"2024-04-21T08:44:34.493394+0000","last_active":"2024-04-21T08:45:27.974475+0000","last_peered":"2024-04-21T08:45:27.974475+0000","last_clean":"2024-04-21T08:45:27.974475+0000","last_became_active":"2024-04-21T08:44:34.492367+0000","last_became_peered":"2024-04-21T08:44:34.492367+0000","last_unstale":"2024-04-21T08:45:27.974475+0000","last_undegraded":"2024-04-21T08:45:27.974475+0000","last_fullsized":"2024-04-21T08:45:27.974475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:13.111266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402882+0000","last_change":"2024-04-21T08:44:38.519113+0000","last_active":"2024-04-21T08:45:27.402882+0000","last_peered":"2024-04-21T08:45:27.402882+0000","last_clean":"2024-04-21T08:45:27.402882+0000","last_became_active":"2024-04-21T08:44:38.518469+0000","last_became_peered":"2024-04-21T08:44:38.518469+0000","last_unstale":"2024-04-21T08:45:27.402882+0000","last_undegraded":"2024-04-21T08:45:27.402882+0000","last_fullsized":"2024-04-21T08:45:27.402882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:39:22.200041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403988+0000","last_change":"2024-04-21T08:44:33.481184+0000","last_active":"2024-04-21T08:45:27.403988+0000","last_peered":"2024-04-21T08:45:27.403988+0000","last_clean":"2024-04-21T08:45:27.403988+0000","last_became_active":"2024-04-21T08:44:33.480835+0000","last_became_peered":"2024-04-21T08:44:33.480835+0000","last_unstale":"2024-04-21T08:45:27.403988+0000","last_undegraded":"2024-04-21T08:45:27.403988+0000","last_fullsized":"2024-04-21T08:45:27.403988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:56:49.463576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.514740+0000","last_change":"2024-04-21T08:44:34.502226+0000","last_active":"2024-04-21T08:45:27.514740+0000","last_peered":"2024-04-21T08:45:27.514740+0000","last_clean":"2024-04-21T08:45:27.514740+0000","last_became_active":"2024-04-21T08:44:34.501775+0000","last_became_peered":"2024-04-21T08:44:34.501775+0000","last_unstale":"2024-04-21T08:45:27.514740+0000","last_undegraded":"2024-04-21T08:45:27.514740+0000","last_fullsized":"2024-04-21T08:45:27.514740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:58.325267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402941+0000","last_change":"2024-04-21T08:44:38.522250+0000","last_active":"2024-04-21T08:45:27.402941+0000","last_peered":"2024-04-21T08:45:27.402941+0000","last_clean":"2024-04-21T08:45:27.402941+0000","last_became_active":"2024-04-21T08:44:38.520870+0000","last_became_peered":"2024-04-21T08:44:38.520870+0000","last_unstale":"2024-04-21T08:45:27.402941+0000","last_undegraded":"2024-04-21T08:45:27.402941+0000","last_fullsized":"2024-04-21T08:45:27.402941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:02.391785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404318+0000","last_change":"2024-04-21T08:44:33.486323+0000","last_active":"2024-04-21T08:45:27.404318+0000","last_peered":"2024-04-21T08:45:27.404318+0000","last_clean":"2024-04-21T08:45:27.404318+0000","last_became_active":"2024-04-21T08:44:33.486016+0000","last_became_peered":"2024-04-21T08:44:33.486016+0000","last_unstale":"2024-04-21T08:45:27.404318+0000","last_undegraded":"2024-04-21T08:45:27.404318+0000","last_fullsized":"2024-04-21T08:45:27.404318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:46.645314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516011+0000","last_change":"2024-04-21T08:44:34.496034+0000","last_active":"2024-04-21T08:45:27.516011+0000","last_peered":"2024-04-21T08:45:27.516011+0000","last_clean":"2024-04-21T08:45:27.516011+0000","last_became_active":"2024-04-21T08:44:34.495641+0000","last_became_peered":"2024-04-21T08:44:34.495641+0000","last_unstale":"2024-04-21T08:45:27.516011+0000","last_undegraded":"2024-04-21T08:45:27.516011+0000","last_fullsized":"2024-04-21T08:45:27.516011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:21:18.893326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516366+0000","last_change":"2024-04-21T08:44:38.517288+0000","last_active":"2024-04-21T08:45:27.516366+0000","last_peered":"2024-04-21T08:45:27.516366+0000","last_clean":"2024-04-21T08:45:27.516366+0000","last_became_active":"2024-04-21T08:44:38.516853+0000","last_became_peered":"2024-04-21T08:44:38.516853+0000","last_unstale":"2024-04-21T08:45:27.516366+0000","last_undegraded":"2024-04-21T08:45:27.516366+0000","last_fullsized":"2024-04-21T08:45:27.516366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:27:22.371703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":102,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:09.119092+0000","last_change":"2024-04-21T08:44:33.491578+0000","last_active":"2024-04-21T08:46:09.119092+0000","last_peered":"2024-04-21T08:46:09.119092+0000","last_clean":"2024-04-21T08:46:09.119092+0000","last_became_active":"2024-04-21T08:44:33.491238+0000","last_became_peered":"2024-04-21T08:44:33.491238+0000","last_unstale":"2024-04-21T08:46:09.119092+0000","last_undegraded":"2024-04-21T08:46:09.119092+0000","last_fullsized":"2024-04-21T08:46:09.119092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:50.399031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516721+0000","last_change":"2024-04-21T08:44:34.497632+0000","last_active":"2024-04-21T08:45:27.516721+0000","last_peered":"2024-04-21T08:45:27.516721+0000","last_clean":"2024-04-21T08:45:27.516721+0000","last_became_active":"2024-04-21T08:44:34.497318+0000","last_became_peered":"2024-04-21T08:44:34.497318+0000","last_unstale":"2024-04-21T08:45:27.516721+0000","last_undegraded":"2024-04-21T08:45:27.516721+0000","last_fullsized":"2024-04-21T08:45:27.516721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:11:33.918530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402732+0000","last_change":"2024-04-21T08:44:38.518414+0000","last_active":"2024-04-21T08:45:27.402732+0000","last_peered":"2024-04-21T08:45:27.402732+0000","last_clean":"2024-04-21T08:45:27.402732+0000","last_became_active":"2024-04-21T08:44:38.517953+0000","last_became_peered":"2024-04-21T08:44:38.517953+0000","last_unstale":"2024-04-21T08:45:27.402732+0000","last_undegraded":"2024-04-21T08:45:27.402732+0000","last_fullsized":"2024-04-21T08:45:27.402732+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:48.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408673+0000","last_change":"2024-04-21T08:44:33.493896+0000","last_active":"2024-04-21T08:45:27.408673+0000","last_peered":"2024-04-21T08:45:27.408673+0000","last_clean":"2024-04-21T08:45:27.408673+0000","last_became_active":"2024-04-21T08:44:33.493566+0000","last_became_peered":"2024-04-21T08:44:33.493566+0000","last_unstale":"2024-04-21T08:45:27.408673+0000","last_undegraded":"2024-04-21T08:45:27.408673+0000","last_fullsized":"2024-04-21T08:45:27.408673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:47:17.307690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515647+0000","last_change":"2024-04-21T08:44:34.500406+0000","last_active":"2024-04-21T08:45:27.515647+0000","last_peered":"2024-04-21T08:45:27.515647+0000","last_clean":"2024-04-21T08:45:27.515647+0000","last_became_active":"2024-04-21T08:44:34.500048+0000","last_became_peered":"2024-04-21T08:44:34.500048+0000","last_unstale":"2024-04-21T08:45:27.515647+0000","last_undegraded":"2024-04-21T08:45:27.515647+0000","last_fullsized":"2024-04-21T08:45:27.515647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:10:58.166155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.407940+0000","last_change":"2024-04-21T08:44:38.514812+0000","last_active":"2024-04-21T08:45:27.407940+0000","last_peered":"2024-04-21T08:45:27.407940+0000","last_clean":"2024-04-21T08:45:27.407940+0000","last_became_active":"2024-04-21T08:44:38.514247+0000","last_became_peered":"2024-04-21T08:44:38.514247+0000","last_unstale":"2024-04-21T08:45:27.407940+0000","last_undegraded":"2024-04-21T08:45:27.407940+0000","last_fullsized":"2024-04-21T08:45:27.407940+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:06:10.588453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402105+0000","last_change":"2024-04-21T08:44:33.493887+0000","last_active":"2024-04-21T08:45:27.402105+0000","last_peered":"2024-04-21T08:45:27.402105+0000","last_clean":"2024-04-21T08:45:27.402105+0000","last_became_active":"2024-04-21T08:44:33.493555+0000","last_became_peered":"2024-04-21T08:44:33.493555+0000","last_unstale":"2024-04-21T08:45:27.402105+0000","last_undegraded":"2024-04-21T08:45:27.402105+0000","last_fullsized":"2024-04-21T08:45:27.402105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:18.089102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516416+0000","last_change":"2024-04-21T08:44:34.503259+0000","last_active":"2024-04-21T08:45:27.516416+0000","last_peered":"2024-04-21T08:45:27.516416+0000","last_clean":"2024-04-21T08:45:27.516416+0000","last_became_active":"2024-04-21T08:44:34.502845+0000","last_became_peered":"2024-04-21T08:44:34.502845+0000","last_unstale":"2024-04-21T08:45:27.516416+0000","last_undegraded":"2024-04-21T08:45:27.516416+0000","last_fullsized":"2024-04-21T08:45:27.516416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:19.902178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974556+0000","last_change":"2024-04-21T08:44:34.497608+0000","last_active":"2024-04-21T08:45:27.974556+0000","last_peered":"2024-04-21T08:45:27.974556+0000","last_clean":"2024-04-21T08:45:27.974556+0000","last_became_active":"2024-04-21T08:44:34.497255+0000","last_became_peered":"2024-04-21T08:44:34.497255+0000","last_unstale":"2024-04-21T08:45:27.974556+0000","last_undegraded":"2024-04-21T08:45:27.974556+0000","last_fullsized":"2024-04-21T08:45:27.974556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:51:15.181837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404357+0000","last_change":"2024-04-21T08:44:34.495408+0000","last_active":"2024-04-21T08:45:27.404357+0000","last_peered":"2024-04-21T08:45:27.404357+0000","last_clean":"2024-04-21T08:45:27.404357+0000","last_became_active":"2024-04-21T08:44:34.495038+0000","last_became_peered":"2024-04-21T08:44:34.495038+0000","last_unstale":"2024-04-21T08:45:27.404357+0000","last_undegraded":"2024-04-21T08:45:27.404357+0000","last_fullsized":"2024-04-21T08:45:27.404357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:59.512245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403868+0000","last_change":"2024-04-21T08:44:34.493292+0000","last_active":"2024-04-21T08:45:27.403868+0000","last_peered":"2024-04-21T08:45:27.403868+0000","last_clean":"2024-04-21T08:45:27.403868+0000","last_became_active":"2024-04-21T08:44:34.492913+0000","last_became_peered":"2024-04-21T08:44:34.492913+0000","last_unstale":"2024-04-21T08:45:27.403868+0000","last_undegraded":"2024-04-21T08:45:27.403868+0000","last_fullsized":"2024-04-21T08:45:27.403868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:24.271859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402956+0000","last_change":"2024-04-21T08:44:34.510448+0000","last_active":"2024-04-21T08:45:27.402956+0000","last_peered":"2024-04-21T08:45:27.402956+0000","last_clean":"2024-04-21T08:45:27.402956+0000","last_became_active":"2024-04-21T08:44:34.509958+0000","last_became_peered":"2024-04-21T08:44:34.509958+0000","last_unstale":"2024-04-21T08:45:27.402956+0000","last_undegraded":"2024-04-21T08:45:27.402956+0000","last_fullsized":"2024-04-21T08:45:27.402956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:36.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402282+0000","last_change":"2024-04-21T08:44:34.497415+0000","last_active":"2024-04-21T08:45:27.402282+0000","last_peered":"2024-04-21T08:45:27.402282+0000","last_clean":"2024-04-21T08:45:27.402282+0000","last_became_active":"2024-04-21T08:44:34.497120+0000","last_became_peered":"2024-04-21T08:44:34.497120+0000","last_unstale":"2024-04-21T08:45:27.402282+0000","last_undegraded":"2024-04-21T08:45:27.402282+0000","last_fullsized":"2024-04-21T08:45:27.402282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:30.232338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974996+0000","last_change":"2024-04-21T08:44:34.500356+0000","last_active":"2024-04-21T08:45:27.974996+0000","last_peered":"2024-04-21T08:45:27.974996+0000","last_clean":"2024-04-21T08:45:27.974996+0000","last_became_active":"2024-04-21T08:44:34.500032+0000","last_became_peered":"2024-04-21T08:44:34.500032+0000","last_unstale":"2024-04-21T08:45:27.974996+0000","last_undegraded":"2024-04-21T08:45:27.974996+0000","last_fullsized":"2024-04-21T08:45:27.974996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:24.900221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.405726+0000","last_change":"2024-04-21T08:44:34.493629+0000","last_active":"2024-04-21T08:45:27.405726+0000","last_peered":"2024-04-21T08:45:27.405726+0000","last_clean":"2024-04-21T08:45:27.405726+0000","last_became_active":"2024-04-21T08:44:34.493273+0000","last_became_peered":"2024-04-21T08:44:34.493273+0000","last_unstale":"2024-04-21T08:45:27.405726+0000","last_undegraded":"2024-04-21T08:45:27.405726+0000","last_fullsized":"2024-04-21T08:45:27.405726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:22:42.698957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515341+0000","last_change":"2024-04-21T08:44:34.493427+0000","last_active":"2024-04-21T08:45:27.515341+0000","last_peered":"2024-04-21T08:45:27.515341+0000","last_clean":"2024-04-21T08:45:27.515341+0000","last_became_active":"2024-04-21T08:44:34.493105+0000","last_became_peered":"2024-04-21T08:44:34.493105+0000","last_unstale":"2024-04-21T08:45:27.515341+0000","last_undegraded":"2024-04-21T08:45:27.515341+0000","last_fullsized":"2024-04-21T08:45:27.515341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:37.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516355+0000","last_change":"2024-04-21T08:45:27.515899+0000","last_active":"2024-04-21T08:45:27.516355+0000","last_peered":"2024-04-21T08:45:27.516355+0000","last_clean":"2024-04-21T08:45:27.516355+0000","last_became_active":"2024-04-21T08:45:27.515567+0000","last_became_peered":"2024-04-21T08:45:27.515567+0000","last_unstale":"2024-04-21T08:45:27.516355+0000","last_undegraded":"2024-04-21T08:45:27.516355+0000","last_fullsized":"2024-04-21T08:45:27.516355+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:16.645869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.514604+0000","last_change":"2024-04-21T08:44:34.499420+0000","last_active":"2024-04-21T08:45:27.514604+0000","last_peered":"2024-04-21T08:45:27.514604+0000","last_clean":"2024-04-21T08:45:27.514604+0000","last_became_active":"2024-04-21T08:44:34.499079+0000","last_became_peered":"2024-04-21T08:44:34.499079+0000","last_unstale":"2024-04-21T08:45:27.514604+0000","last_undegraded":"2024-04-21T08:45:27.514604+0000","last_fullsized":"2024-04-21T08:45:27.514604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:45.351422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516918+0000","last_change":"2024-04-21T08:44:34.493420+0000","last_active":"2024-04-21T08:45:27.516918+0000","last_peered":"2024-04-21T08:45:27.516918+0000","last_clean":"2024-04-21T08:45:27.516918+0000","last_became_active":"2024-04-21T08:44:34.493098+0000","last_became_peered":"2024-04-21T08:44:34.493098+0000","last_unstale":"2024-04-21T08:45:27.516918+0000","last_undegraded":"2024-04-21T08:45:27.516918+0000","last_fullsized":"2024-04-21T08:45:27.516918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:22:34.316822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402161+0000","last_change":"2024-04-21T08:44:34.494760+0000","last_active":"2024-04-21T08:45:27.402161+0000","last_peered":"2024-04-21T08:45:27.402161+0000","last_clean":"2024-04-21T08:45:27.402161+0000","last_became_active":"2024-04-21T08:44:34.494265+0000","last_became_peered":"2024-04-21T08:44:34.494265+0000","last_unstale":"2024-04-21T08:45:27.402161+0000","last_undegraded":"2024-04-21T08:45:27.402161+0000","last_fullsized":"2024-04-21T08:45:27.402161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:16.035030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974799+0000","last_change":"2024-04-21T08:44:34.498785+0000","last_active":"2024-04-21T08:45:27.974799+0000","last_peered":"2024-04-21T08:45:27.974799+0000","last_clean":"2024-04-21T08:45:27.974799+0000","last_became_active":"2024-04-21T08:44:34.495159+0000","last_became_peered":"2024-04-21T08:44:34.495159+0000","last_unstale":"2024-04-21T08:45:27.974799+0000","last_undegraded":"2024-04-21T08:45:27.974799+0000","last_fullsized":"2024-04-21T08:45:27.974799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:03:20.095016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68197,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":741,"num_read_kb":445,"num_write":238,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":204591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":501370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645351,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1724416,"data_stored":1149961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":505396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":501678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1650688,"data_stored":1097784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":496804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":559609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":498260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1118208,"data_stored":559829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":496512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":500878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36964,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708188,"statfs":{"total":95995035648,"available":95957184512,"internally_reserved":0,"allocated":1658880,"data_stored":1093366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:46:15.364 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-21T08:46:15.606 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:46:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:15 smithi082 ceph-mon[27967]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:46:15.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:15 smithi134 ceph-mon[31772]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:46:15.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:15 smithi106 ceph-mon[31072]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:46:16.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:16 smithi134 ceph-mon[31772]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:46:16.782 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:16 smithi082 ceph-mon[27967]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:46:16.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:16 smithi106 ceph-mon[31072]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:46:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.350+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/1642570596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e8103450 msgr2=0x7fd7e810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.350+0000 7fd7ed8aa700 1 --2- 172.21.15.82:0/1642570596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e8103450 0x7fd7e810f8b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000b0a0 tx=0x7fd7e000b3b0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.350+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/1642570596 shutdown_connections 2024-04-21T08:46:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.350+0000 7fd7ed8aa700 1 --2- 172.21.15.82:0/1642570596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e8103450 0x7fd7e810f8b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.350+0000 7fd7ed8aa700 1 --2- 172.21.15.82:0/1642570596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7e8108a50 0x7fd7e8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.350+0000 7fd7ed8aa700 1 --2- 172.21.15.82:0/1642570596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7e8102a90 0x7fd7e8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.350+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/1642570596 >> 172.21.15.82:0/1642570596 conn(0x7fd7e8079ba0 msgr2=0x7fd7e8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.350+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/1642570596 shutdown_connections 2024-04-21T08:46:17.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.350+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/1642570596 wait complete. 2024-04-21T08:46:17.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7ed8aa700 1 Processor -- start 2024-04-21T08:46:17.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7ed8aa700 1 -- start start 2024-04-21T08:46:17.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7ed8aa700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7e8102a90 0x7fd7e8199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7ed8aa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e8103450 0x7fd7e81996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7ed8aa700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7e8108a50 0x7fd7e819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7ed8aa700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e8112c80 con 0x7fd7e8103450 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7ed8aa700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e8112b00 con 0x7fd7e8102a90 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7ed8aa700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e8112e00 con 0x7fd7e8108a50 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7dffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e8103450 0x7fd7e81996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7dffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e8103450 0x7fd7e81996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38802/0 (socket says 172.21.15.82:38802) 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7dffff700 1 -- 172.21.15.82:0/3523250033 learned_addr learned my addr 172.21.15.82:0/3523250033 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7e6ffd700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7e8102a90 0x7fd7e8199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7dffff700 1 -- 172.21.15.82:0/3523250033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7e8108a50 msgr2=0x7fd7e819da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.351+0000 7fd7dffff700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7e8108a50 0x7fd7e819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.352+0000 7fd7dffff700 1 -- 172.21.15.82:0/3523250033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7e8102a90 msgr2=0x7fd7e8199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.352+0000 7fd7dffff700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7e8102a90 0x7fd7e8199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.352+0000 7fd7dffff700 1 -- 172.21.15.82:0/3523250033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e819e150 con 0x7fd7e8103450 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.352+0000 7fd7dffff700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e8103450 0x7fd7e81996d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000b530 tx=0x7fd7d000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:17.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.352+0000 7fd7ec8a8700 1 -- 172.21.15.82:0/3523250033 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0015020 con 0x7fd7e8103450 2024-04-21T08:46:17.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.352+0000 7fd7ec8a8700 1 -- 172.21.15.82:0/3523250033 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d0011ce0 con 0x7fd7e8103450 2024-04-21T08:46:17.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.352+0000 7fd7ec8a8700 1 -- 172.21.15.82:0/3523250033 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0010d20 con 0x7fd7e8103450 2024-04-21T08:46:17.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.352+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/3523250033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e819e440 con 0x7fd7e8103450 2024-04-21T08:46:17.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.352+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/3523250033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e81a5cc0 con 0x7fd7e8103450 2024-04-21T08:46:17.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.353+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/3523250033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e8069730 con 0x7fd7e8103450 2024-04-21T08:46:17.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.354+0000 7fd7ec8a8700 1 -- 172.21.15.82:0/3523250033 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd7d0018020 con 0x7fd7e8103450 2024-04-21T08:46:17.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.354+0000 7fd7ec8a8700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd7d40a3d50 0x7fd7d40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:17.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.355+0000 7fd7e6ffd700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd7d40a3d50 0x7fd7d40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:17.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.355+0000 7fd7ec8a8700 1 -- 172.21.15.82:0/3523250033 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fd7d00d0560 con 0x7fd7e8103450 2024-04-21T08:46:17.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.355+0000 7fd7e6ffd700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd7d40a3d50 0x7fd7d40a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8005950 tx=0x7fd7d8005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.358+0000 7fd7ec8a8700 1 -- 172.21.15.82:0/3523250033 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7d001f020 con 0x7fd7e8103450 2024-04-21T08:46:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.559+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/3523250033 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd7e8062e80 con 0x7fd7d40a3d50 2024-04-21T08:46:17.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.568+0000 7fd7ec8a8700 1 -- 172.21.15.82:0/3523250033 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340392 (secure 0 0 0) 0x7fd7e8062e80 con 0x7fd7d40a3d50 2024-04-21T08:46:17.572 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:17.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.571+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/3523250033 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd7d40a3d50 msgr2=0x7fd7d40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.571+0000 7fd7ed8aa700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd7d40a3d50 0x7fd7d40a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8005950 tx=0x7fd7d8005cf0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.571+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/3523250033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e8103450 msgr2=0x7fd7e81996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:17.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.571+0000 7fd7ed8aa700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e8103450 0x7fd7e81996d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000b530 tx=0x7fd7d000b8f0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.571+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/3523250033 shutdown_connections 2024-04-21T08:46:17.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.571+0000 7fd7ed8aa700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd7d40a3d50 0x7fd7d40a6210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.571+0000 7fd7ed8aa700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7e8108a50 0x7fd7e819da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.571+0000 7fd7ed8aa700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7e8102a90 0x7fd7e8199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.571+0000 7fd7ed8aa700 1 --2- 172.21.15.82:0/3523250033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e8103450 0x7fd7e81996d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:17.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.571+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/3523250033 >> 172.21.15.82:0/3523250033 conn(0x7fd7e8079ba0 msgr2=0x7fd7e8101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:17.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.572+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/3523250033 shutdown_connections 2024-04-21T08:46:17.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:17.572+0000 7fd7ed8aa700 1 -- 172.21.15.82:0/3523250033 wait complete. 2024-04-21T08:46:17.576 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-21T08:46:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:17 smithi134 ceph-mon[31772]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:17.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:17 smithi082 ceph-mon[27967]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:17.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:17 smithi106 ceph-mon[31072]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:18.348 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-04-21T08:46:16.339547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658799,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":917,"num_read_kb":584,"num_write":421,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355136,"kb_used_data":14228,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124586688,"statfs":{"total":1151940427776,"available":1151576768512,"internally_reserved":0,"allocated":14569472,"data_stored":7961447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003024"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.975144+0000","last_change":"2024-04-21T08:44:34.494393+0000","last_active":"2024-04-21T08:45:27.975144+0000","last_peered":"2024-04-21T08:45:27.975144+0000","last_clean":"2024-04-21T08:45:27.975144+0000","last_became_active":"2024-04-21T08:44:34.493583+0000","last_became_peered":"2024-04-21T08:44:34.493583+0000","last_unstale":"2024-04-21T08:45:27.975144+0000","last_undegraded":"2024-04-21T08:45:27.975144+0000","last_fullsized":"2024-04-21T08:45:27.975144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:37.211676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403224+0000","last_change":"2024-04-21T08:44:34.496291+0000","last_active":"2024-04-21T08:45:27.403224+0000","last_peered":"2024-04-21T08:45:27.403224+0000","last_clean":"2024-04-21T08:45:27.403224+0000","last_became_active":"2024-04-21T08:44:34.495019+0000","last_became_peered":"2024-04-21T08:44:34.495019+0000","last_unstale":"2024-04-21T08:45:27.403224+0000","last_undegraded":"2024-04-21T08:45:27.403224+0000","last_fullsized":"2024-04-21T08:45:27.403224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:05.309847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402953+0000","last_change":"2024-04-21T08:44:34.497948+0000","last_active":"2024-04-21T08:45:27.402953+0000","last_peered":"2024-04-21T08:45:27.402953+0000","last_clean":"2024-04-21T08:45:27.402953+0000","last_became_active":"2024-04-21T08:44:34.497657+0000","last_became_peered":"2024-04-21T08:44:34.497657+0000","last_unstale":"2024-04-21T08:45:27.402953+0000","last_undegraded":"2024-04-21T08:45:27.402953+0000","last_fullsized":"2024-04-21T08:45:27.402953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:11.463715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403161+0000","last_change":"2024-04-21T08:44:34.492682+0000","last_active":"2024-04-21T08:45:27.403161+0000","last_peered":"2024-04-21T08:45:27.403161+0000","last_clean":"2024-04-21T08:45:27.403161+0000","last_became_active":"2024-04-21T08:44:34.492230+0000","last_became_peered":"2024-04-21T08:44:34.492230+0000","last_unstale":"2024-04-21T08:45:27.403161+0000","last_undegraded":"2024-04-21T08:45:27.403161+0000","last_fullsized":"2024-04-21T08:45:27.403161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:40.365608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404211+0000","last_change":"2024-04-21T08:44:34.492160+0000","last_active":"2024-04-21T08:45:27.404211+0000","last_peered":"2024-04-21T08:45:27.404211+0000","last_clean":"2024-04-21T08:45:27.404211+0000","last_became_active":"2024-04-21T08:44:34.491701+0000","last_became_peered":"2024-04-21T08:44:34.491701+0000","last_unstale":"2024-04-21T08:45:27.404211+0000","last_undegraded":"2024-04-21T08:45:27.404211+0000","last_fullsized":"2024-04-21T08:45:27.404211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:50.542661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.517055+0000","last_change":"2024-04-21T08:44:34.497341+0000","last_active":"2024-04-21T08:45:27.517055+0000","last_peered":"2024-04-21T08:45:27.517055+0000","last_clean":"2024-04-21T08:45:27.517055+0000","last_became_active":"2024-04-21T08:44:34.495794+0000","last_became_peered":"2024-04-21T08:44:34.495794+0000","last_unstale":"2024-04-21T08:45:27.517055+0000","last_undegraded":"2024-04-21T08:45:27.517055+0000","last_fullsized":"2024-04-21T08:45:27.517055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:13.854753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515447+0000","last_change":"2024-04-21T08:44:34.501578+0000","last_active":"2024-04-21T08:45:27.515447+0000","last_peered":"2024-04-21T08:45:27.515447+0000","last_clean":"2024-04-21T08:45:27.515447+0000","last_became_active":"2024-04-21T08:44:34.500848+0000","last_became_peered":"2024-04-21T08:44:34.500848+0000","last_unstale":"2024-04-21T08:45:27.515447+0000","last_undegraded":"2024-04-21T08:45:27.515447+0000","last_fullsized":"2024-04-21T08:45:27.515447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:53:27.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.514764+0000","last_change":"2024-04-21T08:44:34.502975+0000","last_active":"2024-04-21T08:45:27.514764+0000","last_peered":"2024-04-21T08:45:27.514764+0000","last_clean":"2024-04-21T08:45:27.514764+0000","last_became_active":"2024-04-21T08:44:34.502676+0000","last_became_peered":"2024-04-21T08:44:34.502676+0000","last_unstale":"2024-04-21T08:45:27.514764+0000","last_undegraded":"2024-04-21T08:45:27.514764+0000","last_fullsized":"2024-04-21T08:45:27.514764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:32:13.881603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516244+0000","last_change":"2024-04-21T08:44:34.495703+0000","last_active":"2024-04-21T08:45:27.516244+0000","last_peered":"2024-04-21T08:45:27.516244+0000","last_clean":"2024-04-21T08:45:27.516244+0000","last_became_active":"2024-04-21T08:44:34.495380+0000","last_became_peered":"2024-04-21T08:44:34.495380+0000","last_unstale":"2024-04-21T08:45:27.516244+0000","last_undegraded":"2024-04-21T08:45:27.516244+0000","last_fullsized":"2024-04-21T08:45:27.516244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:45:28.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.517278+0000","last_change":"2024-04-21T08:44:34.499361+0000","last_active":"2024-04-21T08:45:27.517278+0000","last_peered":"2024-04-21T08:45:27.517278+0000","last_clean":"2024-04-21T08:45:27.517278+0000","last_became_active":"2024-04-21T08:44:34.499044+0000","last_became_peered":"2024-04-21T08:44:34.499044+0000","last_unstale":"2024-04-21T08:45:27.517278+0000","last_undegraded":"2024-04-21T08:45:27.517278+0000","last_fullsized":"2024-04-21T08:45:27.517278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:12.118985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516623+0000","last_change":"2024-04-21T08:44:34.499631+0000","last_active":"2024-04-21T08:45:27.516623+0000","last_peered":"2024-04-21T08:45:27.516623+0000","last_clean":"2024-04-21T08:45:27.516623+0000","last_became_active":"2024-04-21T08:44:34.499316+0000","last_became_peered":"2024-04-21T08:44:34.499316+0000","last_unstale":"2024-04-21T08:45:27.516623+0000","last_undegraded":"2024-04-21T08:45:27.516623+0000","last_fullsized":"2024-04-21T08:45:27.516623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:26.123017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403042+0000","last_change":"2024-04-21T08:44:34.504857+0000","last_active":"2024-04-21T08:45:27.403042+0000","last_peered":"2024-04-21T08:45:27.403042+0000","last_clean":"2024-04-21T08:45:27.403042+0000","last_became_active":"2024-04-21T08:44:34.504564+0000","last_became_peered":"2024-04-21T08:44:34.504564+0000","last_unstale":"2024-04-21T08:45:27.403042+0000","last_undegraded":"2024-04-21T08:45:27.403042+0000","last_fullsized":"2024-04-21T08:45:27.403042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:32:05.216162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402190+0000","last_change":"2024-04-21T08:44:34.484074+0000","last_active":"2024-04-21T08:45:27.402190+0000","last_peered":"2024-04-21T08:45:27.402190+0000","last_clean":"2024-04-21T08:45:27.402190+0000","last_became_active":"2024-04-21T08:44:34.483309+0000","last_became_peered":"2024-04-21T08:44:34.483309+0000","last_unstale":"2024-04-21T08:45:27.402190+0000","last_undegraded":"2024-04-21T08:45:27.402190+0000","last_fullsized":"2024-04-21T08:45:27.402190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:39.686048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402799+0000","last_change":"2024-04-21T08:44:34.499159+0000","last_active":"2024-04-21T08:45:27.402799+0000","last_peered":"2024-04-21T08:45:27.402799+0000","last_clean":"2024-04-21T08:45:27.402799+0000","last_became_active":"2024-04-21T08:44:34.496609+0000","last_became_peered":"2024-04-21T08:44:34.496609+0000","last_unstale":"2024-04-21T08:45:27.402799+0000","last_undegraded":"2024-04-21T08:45:27.402799+0000","last_fullsized":"2024-04-21T08:45:27.402799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:11.716321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.406543+0000","last_change":"2024-04-21T08:45:27.406543+0000","last_active":"2024-04-21T08:45:27.406543+0000","last_peered":"2024-04-21T08:45:27.406543+0000","last_clean":"2024-04-21T08:45:27.406543+0000","last_became_active":"2024-04-21T08:45:27.406190+0000","last_became_peered":"2024-04-21T08:45:27.406190+0000","last_unstale":"2024-04-21T08:45:27.406543+0000","last_undegraded":"2024-04-21T08:45:27.406543+0000","last_fullsized":"2024-04-21T08:45:27.406543+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:56:17.760163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516541+0000","last_change":"2024-04-21T08:44:34.494018+0000","last_active":"2024-04-21T08:45:27.516541+0000","last_peered":"2024-04-21T08:45:27.516541+0000","last_clean":"2024-04-21T08:45:27.516541+0000","last_became_active":"2024-04-21T08:44:34.493692+0000","last_became_peered":"2024-04-21T08:44:34.493692+0000","last_unstale":"2024-04-21T08:45:27.516541+0000","last_undegraded":"2024-04-21T08:45:27.516541+0000","last_fullsized":"2024-04-21T08:45:27.516541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:25:16.944160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974962+0000","last_change":"2024-04-21T08:44:34.493231+0000","last_active":"2024-04-21T08:45:27.974962+0000","last_peered":"2024-04-21T08:45:27.974962+0000","last_clean":"2024-04-21T08:45:27.974962+0000","last_became_active":"2024-04-21T08:44:34.492826+0000","last_became_peered":"2024-04-21T08:44:34.492826+0000","last_unstale":"2024-04-21T08:45:27.974962+0000","last_undegraded":"2024-04-21T08:45:27.974962+0000","last_fullsized":"2024-04-21T08:45:27.974962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:36.815012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.400005+0000","last_change":"2024-04-21T08:44:34.495142+0000","last_active":"2024-04-21T08:45:27.400005+0000","last_peered":"2024-04-21T08:45:27.400005+0000","last_clean":"2024-04-21T08:45:27.400005+0000","last_became_active":"2024-04-21T08:44:34.494803+0000","last_became_peered":"2024-04-21T08:44:34.494803+0000","last_unstale":"2024-04-21T08:45:27.400005+0000","last_undegraded":"2024-04-21T08:45:27.400005+0000","last_fullsized":"2024-04-21T08:45:27.400005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:22.974053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404980+0000","last_change":"2024-04-21T08:44:34.491252+0000","last_active":"2024-04-21T08:45:27.404980+0000","last_peered":"2024-04-21T08:45:27.404980+0000","last_clean":"2024-04-21T08:45:27.404980+0000","last_became_active":"2024-04-21T08:44:34.490814+0000","last_became_peered":"2024-04-21T08:44:34.490814+0000","last_unstale":"2024-04-21T08:45:27.404980+0000","last_undegraded":"2024-04-21T08:45:27.404980+0000","last_fullsized":"2024-04-21T08:45:27.404980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:54:52.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404453+0000","last_change":"2024-04-21T08:44:38.518647+0000","last_active":"2024-04-21T08:45:27.404453+0000","last_peered":"2024-04-21T08:45:27.404453+0000","last_clean":"2024-04-21T08:45:27.404453+0000","last_became_active":"2024-04-21T08:44:38.518239+0000","last_became_peered":"2024-04-21T08:44:38.518239+0000","last_unstale":"2024-04-21T08:45:27.404453+0000","last_undegraded":"2024-04-21T08:45:27.404453+0000","last_fullsized":"2024-04-21T08:45:27.404453+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:44.781542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":113,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:13.105788+0000","last_change":"2024-04-21T08:44:33.484484+0000","last_active":"2024-04-21T08:46:13.105788+0000","last_peered":"2024-04-21T08:46:13.105788+0000","last_clean":"2024-04-21T08:46:13.105788+0000","last_became_active":"2024-04-21T08:44:33.484160+0000","last_became_peered":"2024-04-21T08:44:33.484160+0000","last_unstale":"2024-04-21T08:46:13.105788+0000","last_undegraded":"2024-04-21T08:46:13.105788+0000","last_fullsized":"2024-04-21T08:46:13.105788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:45.962598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515007+0000","last_change":"2024-04-21T08:44:34.500722+0000","last_active":"2024-04-21T08:45:27.515007+0000","last_peered":"2024-04-21T08:45:27.515007+0000","last_clean":"2024-04-21T08:45:27.515007+0000","last_became_active":"2024-04-21T08:44:34.500388+0000","last_became_peered":"2024-04-21T08:44:34.500388+0000","last_unstale":"2024-04-21T08:45:27.515007+0000","last_undegraded":"2024-04-21T08:45:27.515007+0000","last_fullsized":"2024-04-21T08:45:27.515007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:05.876636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.518006+0000","last_change":"2024-04-21T08:45:27.518006+0000","last_active":"2024-04-21T08:45:27.518006+0000","last_peered":"2024-04-21T08:45:27.518006+0000","last_clean":"2024-04-21T08:45:27.518006+0000","last_became_active":"2024-04-21T08:45:27.517409+0000","last_became_peered":"2024-04-21T08:45:27.517409+0000","last_unstale":"2024-04-21T08:45:27.518006+0000","last_undegraded":"2024-04-21T08:45:27.518006+0000","last_fullsized":"2024-04-21T08:45:27.518006+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:45:01.253587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515217+0000","last_change":"2024-04-21T08:44:33.490508+0000","last_active":"2024-04-21T08:45:27.515217+0000","last_peered":"2024-04-21T08:45:27.515217+0000","last_clean":"2024-04-21T08:45:27.515217+0000","last_became_active":"2024-04-21T08:44:33.490084+0000","last_became_peered":"2024-04-21T08:44:33.490084+0000","last_unstale":"2024-04-21T08:45:27.515217+0000","last_undegraded":"2024-04-21T08:45:27.515217+0000","last_fullsized":"2024-04-21T08:45:27.515217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:21.553435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403196+0000","last_change":"2024-04-21T08:44:34.491843+0000","last_active":"2024-04-21T08:45:27.403196+0000","last_peered":"2024-04-21T08:45:27.403196+0000","last_clean":"2024-04-21T08:45:27.403196+0000","last_became_active":"2024-04-21T08:44:34.490578+0000","last_became_peered":"2024-04-21T08:44:34.490578+0000","last_unstale":"2024-04-21T08:45:27.403196+0000","last_undegraded":"2024-04-21T08:45:27.403196+0000","last_fullsized":"2024-04-21T08:45:27.403196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:05:12.272330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402322+0000","last_change":"2024-04-21T08:44:38.520258+0000","last_active":"2024-04-21T08:45:27.402322+0000","last_peered":"2024-04-21T08:45:27.402322+0000","last_clean":"2024-04-21T08:45:27.402322+0000","last_became_active":"2024-04-21T08:44:38.519824+0000","last_became_peered":"2024-04-21T08:44:38.519824+0000","last_unstale":"2024-04-21T08:45:27.402322+0000","last_undegraded":"2024-04-21T08:45:27.402322+0000","last_fullsized":"2024-04-21T08:45:27.402322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:39.716745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404909+0000","last_change":"2024-04-21T08:44:33.484934+0000","last_active":"2024-04-21T08:45:27.404909+0000","last_peered":"2024-04-21T08:45:27.404909+0000","last_clean":"2024-04-21T08:45:27.404909+0000","last_became_active":"2024-04-21T08:44:33.484591+0000","last_became_peered":"2024-04-21T08:44:33.484591+0000","last_unstale":"2024-04-21T08:45:27.404909+0000","last_undegraded":"2024-04-21T08:45:27.404909+0000","last_fullsized":"2024-04-21T08:45:27.404909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:48.826611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515909+0000","last_change":"2024-04-21T08:44:34.501109+0000","last_active":"2024-04-21T08:45:27.515909+0000","last_peered":"2024-04-21T08:45:27.515909+0000","last_clean":"2024-04-21T08:45:27.515909+0000","last_became_active":"2024-04-21T08:44:34.500750+0000","last_became_peered":"2024-04-21T08:44:34.500750+0000","last_unstale":"2024-04-21T08:45:27.515909+0000","last_undegraded":"2024-04-21T08:45:27.515909+0000","last_fullsized":"2024-04-21T08:45:27.515909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:29:28.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402344+0000","last_change":"2024-04-21T08:44:38.519633+0000","last_active":"2024-04-21T08:45:27.402344+0000","last_peered":"2024-04-21T08:45:27.402344+0000","last_clean":"2024-04-21T08:45:27.402344+0000","last_became_active":"2024-04-21T08:44:38.519215+0000","last_became_peered":"2024-04-21T08:44:38.519215+0000","last_unstale":"2024-04-21T08:45:27.402344+0000","last_undegraded":"2024-04-21T08:45:27.402344+0000","last_fullsized":"2024-04-21T08:45:27.402344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:43:59.695844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.405067+0000","last_change":"2024-04-21T08:44:33.483267+0000","last_active":"2024-04-21T08:45:27.405067+0000","last_peered":"2024-04-21T08:45:27.405067+0000","last_clean":"2024-04-21T08:45:27.405067+0000","last_became_active":"2024-04-21T08:44:33.482932+0000","last_became_peered":"2024-04-21T08:44:33.482932+0000","last_unstale":"2024-04-21T08:45:27.405067+0000","last_undegraded":"2024-04-21T08:45:27.405067+0000","last_fullsized":"2024-04-21T08:45:27.405067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:53.311550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408529+0000","last_change":"2024-04-21T08:44:34.497389+0000","last_active":"2024-04-21T08:45:27.408529+0000","last_peered":"2024-04-21T08:45:27.408529+0000","last_clean":"2024-04-21T08:45:27.408529+0000","last_became_active":"2024-04-21T08:44:34.497030+0000","last_became_peered":"2024-04-21T08:44:34.497030+0000","last_unstale":"2024-04-21T08:45:27.408529+0000","last_undegraded":"2024-04-21T08:45:27.408529+0000","last_fullsized":"2024-04-21T08:45:27.408529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:20.371888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404791+0000","last_change":"2024-04-21T08:44:38.518095+0000","last_active":"2024-04-21T08:45:27.404791+0000","last_peered":"2024-04-21T08:45:27.404791+0000","last_clean":"2024-04-21T08:45:27.404791+0000","last_became_active":"2024-04-21T08:44:38.517010+0000","last_became_peered":"2024-04-21T08:44:38.517010+0000","last_unstale":"2024-04-21T08:45:27.404791+0000","last_undegraded":"2024-04-21T08:45:27.404791+0000","last_fullsized":"2024-04-21T08:45:27.404791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:02.379512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":110,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:14.102524+0000","last_change":"2024-04-21T08:44:33.487999+0000","last_active":"2024-04-21T08:46:14.102524+0000","last_peered":"2024-04-21T08:46:14.102524+0000","last_clean":"2024-04-21T08:46:14.102524+0000","last_became_active":"2024-04-21T08:44:33.487666+0000","last_became_peered":"2024-04-21T08:44:33.487666+0000","last_unstale":"2024-04-21T08:46:14.102524+0000","last_undegraded":"2024-04-21T08:46:14.102524+0000","last_fullsized":"2024-04-21T08:46:14.102524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:01.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404363+0000","last_change":"2024-04-21T08:44:34.494213+0000","last_active":"2024-04-21T08:45:27.404363+0000","last_peered":"2024-04-21T08:45:27.404363+0000","last_clean":"2024-04-21T08:45:27.404363+0000","last_became_active":"2024-04-21T08:44:34.493853+0000","last_became_peered":"2024-04-21T08:44:34.493853+0000","last_unstale":"2024-04-21T08:45:27.404363+0000","last_undegraded":"2024-04-21T08:45:27.404363+0000","last_fullsized":"2024-04-21T08:45:27.404363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:42.499131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.410579+0000","last_change":"2024-04-21T08:45:27.410579+0000","last_active":"2024-04-21T08:45:27.410579+0000","last_peered":"2024-04-21T08:45:27.410579+0000","last_clean":"2024-04-21T08:45:27.410579+0000","last_became_active":"2024-04-21T08:45:27.409929+0000","last_became_peered":"2024-04-21T08:45:27.409929+0000","last_unstale":"2024-04-21T08:45:27.410579+0000","last_undegraded":"2024-04-21T08:45:27.410579+0000","last_fullsized":"2024-04-21T08:45:27.410579+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:35.897482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516909+0000","last_change":"2024-04-21T08:44:33.488751+0000","last_active":"2024-04-21T08:45:27.516909+0000","last_peered":"2024-04-21T08:45:27.516909+0000","last_clean":"2024-04-21T08:45:27.516909+0000","last_became_active":"2024-04-21T08:44:33.488425+0000","last_became_peered":"2024-04-21T08:44:33.488425+0000","last_unstale":"2024-04-21T08:45:27.516909+0000","last_undegraded":"2024-04-21T08:45:27.516909+0000","last_fullsized":"2024-04-21T08:45:27.516909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:28:47.868814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408306+0000","last_change":"2024-04-21T08:44:34.492770+0000","last_active":"2024-04-21T08:45:27.408306+0000","last_peered":"2024-04-21T08:45:27.408306+0000","last_clean":"2024-04-21T08:45:27.408306+0000","last_became_active":"2024-04-21T08:44:34.491340+0000","last_became_peered":"2024-04-21T08:44:34.491340+0000","last_unstale":"2024-04-21T08:45:27.408306+0000","last_undegraded":"2024-04-21T08:45:27.408306+0000","last_fullsized":"2024-04-21T08:45:27.408306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:47:54.280990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974951+0000","last_change":"2024-04-21T08:44:38.518888+0000","last_active":"2024-04-21T08:45:27.974951+0000","last_peered":"2024-04-21T08:45:27.974951+0000","last_clean":"2024-04-21T08:45:27.974951+0000","last_became_active":"2024-04-21T08:44:38.518421+0000","last_became_peered":"2024-04-21T08:44:38.518421+0000","last_unstale":"2024-04-21T08:45:27.974951+0000","last_undegraded":"2024-04-21T08:45:27.974951+0000","last_fullsized":"2024-04-21T08:45:27.974951+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:21.119364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.401525+0000","last_change":"2024-04-21T08:44:33.484383+0000","last_active":"2024-04-21T08:45:27.401525+0000","last_peered":"2024-04-21T08:45:27.401525+0000","last_clean":"2024-04-21T08:45:27.401525+0000","last_became_active":"2024-04-21T08:44:33.484083+0000","last_became_peered":"2024-04-21T08:44:33.484083+0000","last_unstale":"2024-04-21T08:45:27.401525+0000","last_undegraded":"2024-04-21T08:45:27.401525+0000","last_fullsized":"2024-04-21T08:45:27.401525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:13:14.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.514863+0000","last_change":"2024-04-21T08:44:34.495636+0000","last_active":"2024-04-21T08:45:27.514863+0000","last_peered":"2024-04-21T08:45:27.514863+0000","last_clean":"2024-04-21T08:45:27.514863+0000","last_became_active":"2024-04-21T08:44:34.494967+0000","last_became_peered":"2024-04-21T08:44:34.494967+0000","last_unstale":"2024-04-21T08:45:27.514863+0000","last_undegraded":"2024-04-21T08:45:27.514863+0000","last_fullsized":"2024-04-21T08:45:27.514863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:47:02.379208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403129+0000","last_change":"2024-04-21T08:44:38.515145+0000","last_active":"2024-04-21T08:45:27.403129+0000","last_peered":"2024-04-21T08:45:27.403129+0000","last_clean":"2024-04-21T08:45:27.403129+0000","last_became_active":"2024-04-21T08:44:38.514755+0000","last_became_peered":"2024-04-21T08:44:38.514755+0000","last_unstale":"2024-04-21T08:45:27.403129+0000","last_undegraded":"2024-04-21T08:45:27.403129+0000","last_fullsized":"2024-04-21T08:45:27.403129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:38:15.826293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408784+0000","last_change":"2024-04-21T08:44:33.484714+0000","last_active":"2024-04-21T08:45:27.408784+0000","last_peered":"2024-04-21T08:45:27.408784+0000","last_clean":"2024-04-21T08:45:27.408784+0000","last_became_active":"2024-04-21T08:44:33.484385+0000","last_became_peered":"2024-04-21T08:44:33.484385+0000","last_unstale":"2024-04-21T08:45:27.408784+0000","last_undegraded":"2024-04-21T08:45:27.408784+0000","last_fullsized":"2024-04-21T08:45:27.408784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:03.406860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399527+0000","last_change":"2024-04-21T08:44:34.490989+0000","last_active":"2024-04-21T08:45:27.399527+0000","last_peered":"2024-04-21T08:45:27.399527+0000","last_clean":"2024-04-21T08:45:27.399527+0000","last_became_active":"2024-04-21T08:44:34.490658+0000","last_became_peered":"2024-04-21T08:44:34.490658+0000","last_unstale":"2024-04-21T08:45:27.399527+0000","last_undegraded":"2024-04-21T08:45:27.399527+0000","last_fullsized":"2024-04-21T08:45:27.399527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:42:15.537913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515087+0000","last_change":"2024-04-21T08:44:38.519723+0000","last_active":"2024-04-21T08:45:27.515087+0000","last_peered":"2024-04-21T08:45:27.515087+0000","last_clean":"2024-04-21T08:45:27.515087+0000","last_became_active":"2024-04-21T08:44:38.519336+0000","last_became_peered":"2024-04-21T08:44:38.519336+0000","last_unstale":"2024-04-21T08:45:27.515087+0000","last_undegraded":"2024-04-21T08:45:27.515087+0000","last_fullsized":"2024-04-21T08:45:27.515087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:00.332153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:15.103391+0000","last_change":"2024-04-21T08:44:33.484404+0000","last_active":"2024-04-21T08:46:15.103391+0000","last_peered":"2024-04-21T08:46:15.103391+0000","last_clean":"2024-04-21T08:46:15.103391+0000","last_became_active":"2024-04-21T08:44:33.484088+0000","last_became_peered":"2024-04-21T08:44:33.484088+0000","last_unstale":"2024-04-21T08:46:15.103391+0000","last_undegraded":"2024-04-21T08:46:15.103391+0000","last_fullsized":"2024-04-21T08:46:15.103391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:41.948622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516481+0000","last_change":"2024-04-21T08:44:34.502598+0000","last_active":"2024-04-21T08:45:27.516481+0000","last_peered":"2024-04-21T08:45:27.516481+0000","last_clean":"2024-04-21T08:45:27.516481+0000","last_became_active":"2024-04-21T08:44:34.502057+0000","last_became_peered":"2024-04-21T08:44:34.502057+0000","last_unstale":"2024-04-21T08:45:27.516481+0000","last_undegraded":"2024-04-21T08:45:27.516481+0000","last_fullsized":"2024-04-21T08:45:27.516481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:24.052873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399525+0000","last_change":"2024-04-21T08:44:38.519699+0000","last_active":"2024-04-21T08:45:27.399525+0000","last_peered":"2024-04-21T08:45:27.399525+0000","last_clean":"2024-04-21T08:45:27.399525+0000","last_became_active":"2024-04-21T08:44:38.517655+0000","last_became_peered":"2024-04-21T08:44:38.517655+0000","last_unstale":"2024-04-21T08:45:27.399525+0000","last_undegraded":"2024-04-21T08:45:27.399525+0000","last_fullsized":"2024-04-21T08:45:27.399525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:12.784479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516206+0000","last_change":"2024-04-21T08:44:33.488799+0000","last_active":"2024-04-21T08:45:27.516206+0000","last_peered":"2024-04-21T08:45:27.516206+0000","last_clean":"2024-04-21T08:45:27.516206+0000","last_became_active":"2024-04-21T08:44:33.487948+0000","last_became_peered":"2024-04-21T08:44:33.487948+0000","last_unstale":"2024-04-21T08:45:27.516206+0000","last_undegraded":"2024-04-21T08:45:27.516206+0000","last_fullsized":"2024-04-21T08:45:27.516206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:10:57.347588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.518395+0000","last_change":"2024-04-21T08:45:27.518395+0000","last_active":"2024-04-21T08:45:27.518395+0000","last_peered":"2024-04-21T08:45:27.518395+0000","last_clean":"2024-04-21T08:45:27.518395+0000","last_became_active":"2024-04-21T08:45:27.517969+0000","last_became_peered":"2024-04-21T08:45:27.517969+0000","last_unstale":"2024-04-21T08:45:27.518395+0000","last_undegraded":"2024-04-21T08:45:27.518395+0000","last_fullsized":"2024-04-21T08:45:27.518395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:55.306556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402827+0000","last_change":"2024-04-21T08:44:38.517956+0000","last_active":"2024-04-21T08:45:27.402827+0000","last_peered":"2024-04-21T08:45:27.402827+0000","last_clean":"2024-04-21T08:45:27.402827+0000","last_became_active":"2024-04-21T08:44:38.517422+0000","last_became_peered":"2024-04-21T08:44:38.517422+0000","last_unstale":"2024-04-21T08:45:27.402827+0000","last_undegraded":"2024-04-21T08:45:27.402827+0000","last_fullsized":"2024-04-21T08:45:27.402827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:04.650784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408163+0000","last_change":"2024-04-21T08:44:33.489151+0000","last_active":"2024-04-21T08:45:27.408163+0000","last_peered":"2024-04-21T08:45:27.408163+0000","last_clean":"2024-04-21T08:45:27.408163+0000","last_became_active":"2024-04-21T08:44:33.488781+0000","last_became_peered":"2024-04-21T08:44:33.488781+0000","last_unstale":"2024-04-21T08:45:27.408163+0000","last_undegraded":"2024-04-21T08:45:27.408163+0000","last_fullsized":"2024-04-21T08:45:27.408163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:40.675100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516109+0000","last_change":"2024-04-21T08:44:34.496528+0000","last_active":"2024-04-21T08:45:27.516109+0000","last_peered":"2024-04-21T08:45:27.516109+0000","last_clean":"2024-04-21T08:45:27.516109+0000","last_became_active":"2024-04-21T08:44:34.496223+0000","last_became_peered":"2024-04-21T08:44:34.496223+0000","last_unstale":"2024-04-21T08:45:27.516109+0000","last_undegraded":"2024-04-21T08:45:27.516109+0000","last_fullsized":"2024-04-21T08:45:27.516109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:56.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399531+0000","last_change":"2024-04-21T08:44:38.517537+0000","last_active":"2024-04-21T08:45:27.399531+0000","last_peered":"2024-04-21T08:45:27.399531+0000","last_clean":"2024-04-21T08:45:27.399531+0000","last_became_active":"2024-04-21T08:44:38.517134+0000","last_became_peered":"2024-04-21T08:44:38.517134+0000","last_unstale":"2024-04-21T08:45:27.399531+0000","last_undegraded":"2024-04-21T08:45:27.399531+0000","last_fullsized":"2024-04-21T08:45:27.399531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:38.634418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":218,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:15.110140+0000","last_change":"2024-04-21T08:44:33.485582+0000","last_active":"2024-04-21T08:46:15.110140+0000","last_peered":"2024-04-21T08:46:15.110140+0000","last_clean":"2024-04-21T08:46:15.110140+0000","last_became_active":"2024-04-21T08:44:33.485249+0000","last_became_peered":"2024-04-21T08:44:33.485249+0000","last_unstale":"2024-04-21T08:46:15.110140+0000","last_undegraded":"2024-04-21T08:46:15.110140+0000","last_fullsized":"2024-04-21T08:46:15.110140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:16:10.571694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61775,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":159,"num_read_kb":140,"num_write":35,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404559+0000","last_change":"2024-04-21T08:44:34.491361+0000","last_active":"2024-04-21T08:45:27.404559+0000","last_peered":"2024-04-21T08:45:27.404559+0000","last_clean":"2024-04-21T08:45:27.404559+0000","last_became_active":"2024-04-21T08:44:34.491050+0000","last_became_peered":"2024-04-21T08:44:34.491050+0000","last_unstale":"2024-04-21T08:45:27.404559+0000","last_undegraded":"2024-04-21T08:45:27.404559+0000","last_fullsized":"2024-04-21T08:45:27.404559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:39.082448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408809+0000","last_change":"2024-04-21T08:44:38.528745+0000","last_active":"2024-04-21T08:45:27.408809+0000","last_peered":"2024-04-21T08:45:27.408809+0000","last_clean":"2024-04-21T08:45:27.408809+0000","last_became_active":"2024-04-21T08:44:38.528275+0000","last_became_peered":"2024-04-21T08:44:38.528275+0000","last_unstale":"2024-04-21T08:45:27.408809+0000","last_undegraded":"2024-04-21T08:45:27.408809+0000","last_fullsized":"2024-04-21T08:45:27.408809+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:06:11.338420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403537+0000","last_change":"2024-04-21T08:44:33.482059+0000","last_active":"2024-04-21T08:45:27.403537+0000","last_peered":"2024-04-21T08:45:27.403537+0000","last_clean":"2024-04-21T08:45:27.403537+0000","last_became_active":"2024-04-21T08:44:33.481663+0000","last_became_peered":"2024-04-21T08:44:33.481663+0000","last_unstale":"2024-04-21T08:45:27.403537+0000","last_undegraded":"2024-04-21T08:45:27.403537+0000","last_fullsized":"2024-04-21T08:45:27.403537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:01.240452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.401749+0000","last_change":"2024-04-21T08:44:34.491398+0000","last_active":"2024-04-21T08:45:27.401749+0000","last_peered":"2024-04-21T08:45:27.401749+0000","last_clean":"2024-04-21T08:45:27.401749+0000","last_became_active":"2024-04-21T08:44:34.491073+0000","last_became_peered":"2024-04-21T08:44:34.491073+0000","last_unstale":"2024-04-21T08:45:27.401749+0000","last_undegraded":"2024-04-21T08:45:27.401749+0000","last_fullsized":"2024-04-21T08:45:27.401749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:22.566063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974571+0000","last_change":"2024-04-21T08:44:38.515653+0000","last_active":"2024-04-21T08:45:27.974571+0000","last_peered":"2024-04-21T08:45:27.974571+0000","last_clean":"2024-04-21T08:45:27.974571+0000","last_became_active":"2024-04-21T08:44:38.514796+0000","last_became_peered":"2024-04-21T08:44:38.514796+0000","last_unstale":"2024-04-21T08:45:27.974571+0000","last_undegraded":"2024-04-21T08:45:27.974571+0000","last_fullsized":"2024-04-21T08:45:27.974571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:00:22.662136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408084+0000","last_change":"2024-04-21T08:44:33.487073+0000","last_active":"2024-04-21T08:45:27.408084+0000","last_peered":"2024-04-21T08:45:27.408084+0000","last_clean":"2024-04-21T08:45:27.408084+0000","last_became_active":"2024-04-21T08:44:33.486740+0000","last_became_peered":"2024-04-21T08:44:33.486740+0000","last_unstale":"2024-04-21T08:45:27.408084+0000","last_undegraded":"2024-04-21T08:45:27.408084+0000","last_fullsized":"2024-04-21T08:45:27.408084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:32.673074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516008+0000","last_change":"2024-04-21T08:44:34.496757+0000","last_active":"2024-04-21T08:45:27.516008+0000","last_peered":"2024-04-21T08:45:27.516008+0000","last_clean":"2024-04-21T08:45:27.516008+0000","last_became_active":"2024-04-21T08:44:34.496422+0000","last_became_peered":"2024-04-21T08:44:34.496422+0000","last_unstale":"2024-04-21T08:45:27.516008+0000","last_undegraded":"2024-04-21T08:45:27.516008+0000","last_fullsized":"2024-04-21T08:45:27.516008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:08.686810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399726+0000","last_change":"2024-04-21T08:44:38.520935+0000","last_active":"2024-04-21T08:45:27.399726+0000","last_peered":"2024-04-21T08:45:27.399726+0000","last_clean":"2024-04-21T08:45:27.399726+0000","last_became_active":"2024-04-21T08:44:38.520481+0000","last_became_peered":"2024-04-21T08:44:38.520481+0000","last_unstale":"2024-04-21T08:45:27.399726+0000","last_undegraded":"2024-04-21T08:45:27.399726+0000","last_fullsized":"2024-04-21T08:45:27.399726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:12.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":110,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:14.101750+0000","last_change":"2024-04-21T08:44:33.485797+0000","last_active":"2024-04-21T08:46:14.101750+0000","last_peered":"2024-04-21T08:46:14.101750+0000","last_clean":"2024-04-21T08:46:14.101750+0000","last_became_active":"2024-04-21T08:44:33.485496+0000","last_became_peered":"2024-04-21T08:44:33.485496+0000","last_unstale":"2024-04-21T08:46:14.101750+0000","last_undegraded":"2024-04-21T08:46:14.101750+0000","last_fullsized":"2024-04-21T08:46:14.101750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:11:34.604611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515510+0000","last_change":"2024-04-21T08:44:34.506453+0000","last_active":"2024-04-21T08:45:27.515510+0000","last_peered":"2024-04-21T08:45:27.515510+0000","last_clean":"2024-04-21T08:45:27.515510+0000","last_became_active":"2024-04-21T08:44:34.506121+0000","last_became_peered":"2024-04-21T08:44:34.506121+0000","last_unstale":"2024-04-21T08:45:27.515510+0000","last_undegraded":"2024-04-21T08:45:27.515510+0000","last_fullsized":"2024-04-21T08:45:27.515510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:07.927366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402163+0000","last_change":"2024-04-21T08:44:38.523718+0000","last_active":"2024-04-21T08:45:27.402163+0000","last_peered":"2024-04-21T08:45:27.402163+0000","last_clean":"2024-04-21T08:45:27.402163+0000","last_became_active":"2024-04-21T08:44:38.523003+0000","last_became_peered":"2024-04-21T08:44:38.523003+0000","last_unstale":"2024-04-21T08:45:27.402163+0000","last_undegraded":"2024-04-21T08:45:27.402163+0000","last_fullsized":"2024-04-21T08:45:27.402163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:20.837450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404019+0000","last_change":"2024-04-21T08:44:33.482029+0000","last_active":"2024-04-21T08:45:27.404019+0000","last_peered":"2024-04-21T08:45:27.404019+0000","last_clean":"2024-04-21T08:45:27.404019+0000","last_became_active":"2024-04-21T08:44:33.481713+0000","last_became_peered":"2024-04-21T08:44:33.481713+0000","last_unstale":"2024-04-21T08:45:27.404019+0000","last_undegraded":"2024-04-21T08:45:27.404019+0000","last_fullsized":"2024-04-21T08:45:27.404019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:46:34.717244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408714+0000","last_change":"2024-04-21T08:44:34.491944+0000","last_active":"2024-04-21T08:45:27.408714+0000","last_peered":"2024-04-21T08:45:27.408714+0000","last_clean":"2024-04-21T08:45:27.408714+0000","last_became_active":"2024-04-21T08:44:34.491638+0000","last_became_peered":"2024-04-21T08:44:34.491638+0000","last_unstale":"2024-04-21T08:45:27.408714+0000","last_undegraded":"2024-04-21T08:45:27.408714+0000","last_fullsized":"2024-04-21T08:45:27.408714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:08:07.925937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408288+0000","last_change":"2024-04-21T08:44:38.528971+0000","last_active":"2024-04-21T08:45:27.408288+0000","last_peered":"2024-04-21T08:45:27.408288+0000","last_clean":"2024-04-21T08:45:27.408288+0000","last_became_active":"2024-04-21T08:44:38.528453+0000","last_became_peered":"2024-04-21T08:44:38.528453+0000","last_unstale":"2024-04-21T08:45:27.408288+0000","last_undegraded":"2024-04-21T08:45:27.408288+0000","last_fullsized":"2024-04-21T08:45:27.408288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:50:31.237749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.406121+0000","last_change":"2024-04-21T08:44:33.479262+0000","last_active":"2024-04-21T08:45:27.406121+0000","last_peered":"2024-04-21T08:45:27.406121+0000","last_clean":"2024-04-21T08:45:27.406121+0000","last_became_active":"2024-04-21T08:44:33.478907+0000","last_became_peered":"2024-04-21T08:44:33.478907+0000","last_unstale":"2024-04-21T08:45:27.406121+0000","last_undegraded":"2024-04-21T08:45:27.406121+0000","last_fullsized":"2024-04-21T08:45:27.406121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:33:43.879126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974889+0000","last_change":"2024-04-21T08:44:34.498503+0000","last_active":"2024-04-21T08:45:27.974889+0000","last_peered":"2024-04-21T08:45:27.974889+0000","last_clean":"2024-04-21T08:45:27.974889+0000","last_became_active":"2024-04-21T08:44:34.495140+0000","last_became_peered":"2024-04-21T08:44:34.495140+0000","last_unstale":"2024-04-21T08:45:27.974889+0000","last_undegraded":"2024-04-21T08:45:27.974889+0000","last_fullsized":"2024-04-21T08:45:27.974889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:24.518349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404212+0000","last_change":"2024-04-21T08:44:38.517717+0000","last_active":"2024-04-21T08:45:27.404212+0000","last_peered":"2024-04-21T08:45:27.404212+0000","last_clean":"2024-04-21T08:45:27.404212+0000","last_became_active":"2024-04-21T08:44:38.517272+0000","last_became_peered":"2024-04-21T08:44:38.517272+0000","last_unstale":"2024-04-21T08:45:27.404212+0000","last_undegraded":"2024-04-21T08:45:27.404212+0000","last_fullsized":"2024-04-21T08:45:27.404212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.664130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399781+0000","last_change":"2024-04-21T08:44:33.477498+0000","last_active":"2024-04-21T08:45:27.399781+0000","last_peered":"2024-04-21T08:45:27.399781+0000","last_clean":"2024-04-21T08:45:27.399781+0000","last_became_active":"2024-04-21T08:44:33.477139+0000","last_became_peered":"2024-04-21T08:44:33.477139+0000","last_unstale":"2024-04-21T08:45:27.399781+0000","last_undegraded":"2024-04-21T08:45:27.399781+0000","last_fullsized":"2024-04-21T08:45:27.399781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:26:59.390583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.517425+0000","last_change":"2024-04-21T08:44:34.493586+0000","last_active":"2024-04-21T08:45:27.517425+0000","last_peered":"2024-04-21T08:45:27.517425+0000","last_clean":"2024-04-21T08:45:27.517425+0000","last_became_active":"2024-04-21T08:44:34.493258+0000","last_became_peered":"2024-04-21T08:44:34.493258+0000","last_unstale":"2024-04-21T08:45:27.517425+0000","last_undegraded":"2024-04-21T08:45:27.517425+0000","last_fullsized":"2024-04-21T08:45:27.517425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:11:18.783332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.401955+0000","last_change":"2024-04-21T08:44:38.519783+0000","last_active":"2024-04-21T08:45:27.401955+0000","last_peered":"2024-04-21T08:45:27.401955+0000","last_clean":"2024-04-21T08:45:27.401955+0000","last_became_active":"2024-04-21T08:44:38.519397+0000","last_became_peered":"2024-04-21T08:44:38.519397+0000","last_unstale":"2024-04-21T08:45:27.401955+0000","last_undegraded":"2024-04-21T08:45:27.401955+0000","last_fullsized":"2024-04-21T08:45:27.401955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:08.740240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:11.103629+0000","last_change":"2024-04-21T08:44:33.490098+0000","last_active":"2024-04-21T08:46:11.103629+0000","last_peered":"2024-04-21T08:46:11.103629+0000","last_clean":"2024-04-21T08:46:11.103629+0000","last_became_active":"2024-04-21T08:44:33.489691+0000","last_became_peered":"2024-04-21T08:44:33.489691+0000","last_unstale":"2024-04-21T08:46:11.103629+0000","last_undegraded":"2024-04-21T08:46:11.103629+0000","last_fullsized":"2024-04-21T08:46:11.103629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:44.965998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516106+0000","last_change":"2024-04-21T08:44:34.498543+0000","last_active":"2024-04-21T08:45:27.516106+0000","last_peered":"2024-04-21T08:45:27.516106+0000","last_clean":"2024-04-21T08:45:27.516106+0000","last_became_active":"2024-04-21T08:44:34.498211+0000","last_became_peered":"2024-04-21T08:44:34.498211+0000","last_unstale":"2024-04-21T08:45:27.516106+0000","last_undegraded":"2024-04-21T08:45:27.516106+0000","last_fullsized":"2024-04-21T08:45:27.516106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:16.506695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.401604+0000","last_change":"2024-04-21T08:44:38.521010+0000","last_active":"2024-04-21T08:45:27.401604+0000","last_peered":"2024-04-21T08:45:27.401604+0000","last_clean":"2024-04-21T08:45:27.401604+0000","last_became_active":"2024-04-21T08:44:38.520605+0000","last_became_peered":"2024-04-21T08:44:38.520605+0000","last_unstale":"2024-04-21T08:45:27.401604+0000","last_undegraded":"2024-04-21T08:45:27.401604+0000","last_fullsized":"2024-04-21T08:45:27.401604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:50.378803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.410137+0000","last_change":"2024-04-21T08:45:27.410137+0000","last_active":"2024-04-21T08:45:27.410137+0000","last_peered":"2024-04-21T08:45:27.410137+0000","last_clean":"2024-04-21T08:45:27.410137+0000","last_became_active":"2024-04-21T08:45:27.409728+0000","last_became_peered":"2024-04-21T08:45:27.409728+0000","last_unstale":"2024-04-21T08:45:27.410137+0000","last_undegraded":"2024-04-21T08:45:27.410137+0000","last_fullsized":"2024-04-21T08:45:27.410137+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:17.934902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:10.094832+0000","last_change":"2024-04-21T08:44:33.494031+0000","last_active":"2024-04-21T08:46:10.094832+0000","last_peered":"2024-04-21T08:46:10.094832+0000","last_clean":"2024-04-21T08:46:10.094832+0000","last_became_active":"2024-04-21T08:44:33.493580+0000","last_became_peered":"2024-04-21T08:44:33.493580+0000","last_unstale":"2024-04-21T08:46:10.094832+0000","last_undegraded":"2024-04-21T08:46:10.094832+0000","last_fullsized":"2024-04-21T08:46:10.094832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:39:48.230616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515535+0000","last_change":"2024-04-21T08:44:38.518067+0000","last_active":"2024-04-21T08:45:27.515535+0000","last_peered":"2024-04-21T08:45:27.515535+0000","last_clean":"2024-04-21T08:45:27.515535+0000","last_became_active":"2024-04-21T08:44:38.517594+0000","last_became_peered":"2024-04-21T08:44:38.517594+0000","last_unstale":"2024-04-21T08:45:27.515535+0000","last_undegraded":"2024-04-21T08:45:27.515535+0000","last_fullsized":"2024-04-21T08:45:27.515535+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:35.261582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.518194+0000","last_change":"2024-04-21T08:45:27.517719+0000","last_active":"2024-04-21T08:45:27.518194+0000","last_peered":"2024-04-21T08:45:27.518194+0000","last_clean":"2024-04-21T08:45:27.518194+0000","last_became_active":"2024-04-21T08:45:27.517429+0000","last_became_peered":"2024-04-21T08:45:27.517429+0000","last_unstale":"2024-04-21T08:45:27.518194+0000","last_undegraded":"2024-04-21T08:45:27.518194+0000","last_fullsized":"2024-04-21T08:45:27.518194+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:49:34.097759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402575+0000","last_change":"2024-04-21T08:44:33.485110+0000","last_active":"2024-04-21T08:45:27.402575+0000","last_peered":"2024-04-21T08:45:27.402575+0000","last_clean":"2024-04-21T08:45:27.402575+0000","last_became_active":"2024-04-21T08:44:33.484722+0000","last_became_peered":"2024-04-21T08:44:33.484722+0000","last_unstale":"2024-04-21T08:45:27.402575+0000","last_undegraded":"2024-04-21T08:45:27.402575+0000","last_fullsized":"2024-04-21T08:45:27.402575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:47:16.773560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":362,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515276+0000","last_change":"2024-04-21T08:43:17.984594+0000","last_active":"2024-04-21T08:45:27.515276+0000","last_peered":"2024-04-21T08:45:27.515276+0000","last_clean":"2024-04-21T08:45:27.515276+0000","last_became_active":"2024-04-21T08:43:17.984207+0000","last_became_peered":"2024-04-21T08:43:17.984207+0000","last_unstale":"2024-04-21T08:45:27.515276+0000","last_undegraded":"2024-04-21T08:45:27.515276+0000","last_fullsized":"2024-04-21T08:45:27.515276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_clean_scrub_stamp":"2024-04-21T08:39:50.910206+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:35.260499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.519344+0000","last_change":"2024-04-21T08:45:27.518834+0000","last_active":"2024-04-21T08:45:27.519344+0000","last_peered":"2024-04-21T08:45:27.519344+0000","last_clean":"2024-04-21T08:45:27.519344+0000","last_became_active":"2024-04-21T08:45:27.518390+0000","last_became_peered":"2024-04-21T08:45:27.518390+0000","last_unstale":"2024-04-21T08:45:27.519344+0000","last_undegraded":"2024-04-21T08:45:27.519344+0000","last_fullsized":"2024-04-21T08:45:27.519344+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:58:32.673008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.401412+0000","last_change":"2024-04-21T08:44:34.499975+0000","last_active":"2024-04-21T08:45:27.401412+0000","last_peered":"2024-04-21T08:45:27.401412+0000","last_clean":"2024-04-21T08:45:27.401412+0000","last_became_active":"2024-04-21T08:44:34.499646+0000","last_became_peered":"2024-04-21T08:44:34.499646+0000","last_unstale":"2024-04-21T08:45:27.401412+0000","last_undegraded":"2024-04-21T08:45:27.401412+0000","last_fullsized":"2024-04-21T08:45:27.401412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:19:38.082502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515346+0000","last_change":"2024-04-21T08:44:33.484449+0000","last_active":"2024-04-21T08:45:27.515346+0000","last_peered":"2024-04-21T08:45:27.515346+0000","last_clean":"2024-04-21T08:45:27.515346+0000","last_became_active":"2024-04-21T08:44:33.484155+0000","last_became_peered":"2024-04-21T08:44:33.484155+0000","last_unstale":"2024-04-21T08:45:27.515346+0000","last_undegraded":"2024-04-21T08:45:27.515346+0000","last_fullsized":"2024-04-21T08:45:27.515346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:03.480474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974663+0000","last_change":"2024-04-21T08:44:38.515283+0000","last_active":"2024-04-21T08:45:27.974663+0000","last_peered":"2024-04-21T08:45:27.974663+0000","last_clean":"2024-04-21T08:45:27.974663+0000","last_became_active":"2024-04-21T08:44:38.514764+0000","last_became_peered":"2024-04-21T08:44:38.514764+0000","last_unstale":"2024-04-21T08:45:27.974663+0000","last_undegraded":"2024-04-21T08:45:27.974663+0000","last_fullsized":"2024-04-21T08:45:27.974663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:17:36.378392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404687+0000","last_change":"2024-04-21T08:44:33.489198+0000","last_active":"2024-04-21T08:45:27.404687+0000","last_peered":"2024-04-21T08:45:27.404687+0000","last_clean":"2024-04-21T08:45:27.404687+0000","last_became_active":"2024-04-21T08:44:33.488794+0000","last_became_peered":"2024-04-21T08:44:33.488794+0000","last_unstale":"2024-04-21T08:45:27.404687+0000","last_undegraded":"2024-04-21T08:45:27.404687+0000","last_fullsized":"2024-04-21T08:45:27.404687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:24.948283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402060+0000","last_change":"2024-04-21T08:44:34.488811+0000","last_active":"2024-04-21T08:45:27.402060+0000","last_peered":"2024-04-21T08:45:27.402060+0000","last_clean":"2024-04-21T08:45:27.402060+0000","last_became_active":"2024-04-21T08:44:34.488472+0000","last_became_peered":"2024-04-21T08:44:34.488472+0000","last_unstale":"2024-04-21T08:45:27.402060+0000","last_undegraded":"2024-04-21T08:45:27.402060+0000","last_fullsized":"2024-04-21T08:45:27.402060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:44.779492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.514792+0000","last_change":"2024-04-21T08:44:38.522711+0000","last_active":"2024-04-21T08:45:27.514792+0000","last_peered":"2024-04-21T08:45:27.514792+0000","last_clean":"2024-04-21T08:45:27.514792+0000","last_became_active":"2024-04-21T08:44:38.522322+0000","last_became_peered":"2024-04-21T08:44:38.522322+0000","last_unstale":"2024-04-21T08:45:27.514792+0000","last_undegraded":"2024-04-21T08:45:27.514792+0000","last_fullsized":"2024-04-21T08:45:27.514792+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:26:13.323283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'12","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399984+0000","last_change":"2024-04-21T08:44:33.481815+0000","last_active":"2024-04-21T08:45:27.399984+0000","last_peered":"2024-04-21T08:45:27.399984+0000","last_clean":"2024-04-21T08:45:27.399984+0000","last_became_active":"2024-04-21T08:44:33.481469+0000","last_became_peered":"2024-04-21T08:44:33.481469+0000","last_unstale":"2024-04-21T08:45:27.399984+0000","last_undegraded":"2024-04-21T08:45:27.399984+0000","last_fullsized":"2024-04-21T08:45:27.399984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:13.970821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515241+0000","last_change":"2024-04-21T08:44:34.496996+0000","last_active":"2024-04-21T08:45:27.515241+0000","last_peered":"2024-04-21T08:45:27.515241+0000","last_clean":"2024-04-21T08:45:27.515241+0000","last_became_active":"2024-04-21T08:44:34.496676+0000","last_became_peered":"2024-04-21T08:44:34.496676+0000","last_unstale":"2024-04-21T08:45:27.515241+0000","last_undegraded":"2024-04-21T08:45:27.515241+0000","last_fullsized":"2024-04-21T08:45:27.515241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:43:08.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404218+0000","last_change":"2024-04-21T08:44:38.516240+0000","last_active":"2024-04-21T08:45:27.404218+0000","last_peered":"2024-04-21T08:45:27.404218+0000","last_clean":"2024-04-21T08:45:27.404218+0000","last_became_active":"2024-04-21T08:44:38.515801+0000","last_became_peered":"2024-04-21T08:44:38.515801+0000","last_unstale":"2024-04-21T08:45:27.404218+0000","last_undegraded":"2024-04-21T08:45:27.404218+0000","last_fullsized":"2024-04-21T08:45:27.404218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:46.520373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403282+0000","last_change":"2024-04-21T08:44:33.488952+0000","last_active":"2024-04-21T08:45:27.403282+0000","last_peered":"2024-04-21T08:45:27.403282+0000","last_clean":"2024-04-21T08:45:27.403282+0000","last_became_active":"2024-04-21T08:44:33.488617+0000","last_became_peered":"2024-04-21T08:44:33.488617+0000","last_unstale":"2024-04-21T08:45:27.403282+0000","last_undegraded":"2024-04-21T08:45:27.403282+0000","last_fullsized":"2024-04-21T08:45:27.403282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:37:24.679132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403163+0000","last_change":"2024-04-21T08:44:34.510568+0000","last_active":"2024-04-21T08:45:27.403163+0000","last_peered":"2024-04-21T08:45:27.403163+0000","last_clean":"2024-04-21T08:45:27.403163+0000","last_became_active":"2024-04-21T08:44:34.509900+0000","last_became_peered":"2024-04-21T08:44:34.509900+0000","last_unstale":"2024-04-21T08:45:27.403163+0000","last_undegraded":"2024-04-21T08:45:27.403163+0000","last_fullsized":"2024-04-21T08:45:27.403163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:08.426141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.976874+0000","last_change":"2024-04-21T08:45:27.976874+0000","last_active":"2024-04-21T08:45:27.976874+0000","last_peered":"2024-04-21T08:45:27.976874+0000","last_clean":"2024-04-21T08:45:27.976874+0000","last_became_active":"2024-04-21T08:45:27.976387+0000","last_became_peered":"2024-04-21T08:45:27.976387+0000","last_unstale":"2024-04-21T08:45:27.976874+0000","last_undegraded":"2024-04-21T08:45:27.976874+0000","last_fullsized":"2024-04-21T08:45:27.976874+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:38:18.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515795+0000","last_change":"2024-04-21T08:44:33.481803+0000","last_active":"2024-04-21T08:45:27.515795+0000","last_peered":"2024-04-21T08:45:27.515795+0000","last_clean":"2024-04-21T08:45:27.515795+0000","last_became_active":"2024-04-21T08:44:33.481256+0000","last_became_peered":"2024-04-21T08:44:33.481256+0000","last_unstale":"2024-04-21T08:45:27.515795+0000","last_undegraded":"2024-04-21T08:45:27.515795+0000","last_fullsized":"2024-04-21T08:45:27.515795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:17.804557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404089+0000","last_change":"2024-04-21T08:44:34.490206+0000","last_active":"2024-04-21T08:45:27.404089+0000","last_peered":"2024-04-21T08:45:27.404089+0000","last_clean":"2024-04-21T08:45:27.404089+0000","last_became_active":"2024-04-21T08:44:34.489797+0000","last_became_peered":"2024-04-21T08:44:34.489797+0000","last_unstale":"2024-04-21T08:45:27.404089+0000","last_undegraded":"2024-04-21T08:45:27.404089+0000","last_fullsized":"2024-04-21T08:45:27.404089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:09.055429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.399727+0000","last_change":"2024-04-21T08:44:38.517385+0000","last_active":"2024-04-21T08:45:27.399727+0000","last_peered":"2024-04-21T08:45:27.399727+0000","last_clean":"2024-04-21T08:45:27.399727+0000","last_became_active":"2024-04-21T08:44:38.516964+0000","last_became_peered":"2024-04-21T08:44:38.516964+0000","last_unstale":"2024-04-21T08:45:27.399727+0000","last_undegraded":"2024-04-21T08:45:27.399727+0000","last_fullsized":"2024-04-21T08:45:27.399727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:33.364076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.514578+0000","last_change":"2024-04-21T08:44:33.490586+0000","last_active":"2024-04-21T08:45:27.514578+0000","last_peered":"2024-04-21T08:45:27.514578+0000","last_clean":"2024-04-21T08:45:27.514578+0000","last_became_active":"2024-04-21T08:44:33.489281+0000","last_became_peered":"2024-04-21T08:44:33.489281+0000","last_unstale":"2024-04-21T08:45:27.514578+0000","last_undegraded":"2024-04-21T08:45:27.514578+0000","last_fullsized":"2024-04-21T08:45:27.514578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:41:29.856165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974475+0000","last_change":"2024-04-21T08:44:34.493394+0000","last_active":"2024-04-21T08:45:27.974475+0000","last_peered":"2024-04-21T08:45:27.974475+0000","last_clean":"2024-04-21T08:45:27.974475+0000","last_became_active":"2024-04-21T08:44:34.492367+0000","last_became_peered":"2024-04-21T08:44:34.492367+0000","last_unstale":"2024-04-21T08:45:27.974475+0000","last_undegraded":"2024-04-21T08:45:27.974475+0000","last_fullsized":"2024-04-21T08:45:27.974475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:13.111266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402882+0000","last_change":"2024-04-21T08:44:38.519113+0000","last_active":"2024-04-21T08:45:27.402882+0000","last_peered":"2024-04-21T08:45:27.402882+0000","last_clean":"2024-04-21T08:45:27.402882+0000","last_became_active":"2024-04-21T08:44:38.518469+0000","last_became_peered":"2024-04-21T08:44:38.518469+0000","last_unstale":"2024-04-21T08:45:27.402882+0000","last_undegraded":"2024-04-21T08:45:27.402882+0000","last_fullsized":"2024-04-21T08:45:27.402882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:39:22.200041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403988+0000","last_change":"2024-04-21T08:44:33.481184+0000","last_active":"2024-04-21T08:45:27.403988+0000","last_peered":"2024-04-21T08:45:27.403988+0000","last_clean":"2024-04-21T08:45:27.403988+0000","last_became_active":"2024-04-21T08:44:33.480835+0000","last_became_peered":"2024-04-21T08:44:33.480835+0000","last_unstale":"2024-04-21T08:45:27.403988+0000","last_undegraded":"2024-04-21T08:45:27.403988+0000","last_fullsized":"2024-04-21T08:45:27.403988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:56:49.463576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.514740+0000","last_change":"2024-04-21T08:44:34.502226+0000","last_active":"2024-04-21T08:45:27.514740+0000","last_peered":"2024-04-21T08:45:27.514740+0000","last_clean":"2024-04-21T08:45:27.514740+0000","last_became_active":"2024-04-21T08:44:34.501775+0000","last_became_peered":"2024-04-21T08:44:34.501775+0000","last_unstale":"2024-04-21T08:45:27.514740+0000","last_undegraded":"2024-04-21T08:45:27.514740+0000","last_fullsized":"2024-04-21T08:45:27.514740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:58.325267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402941+0000","last_change":"2024-04-21T08:44:38.522250+0000","last_active":"2024-04-21T08:45:27.402941+0000","last_peered":"2024-04-21T08:45:27.402941+0000","last_clean":"2024-04-21T08:45:27.402941+0000","last_became_active":"2024-04-21T08:44:38.520870+0000","last_became_peered":"2024-04-21T08:44:38.520870+0000","last_unstale":"2024-04-21T08:45:27.402941+0000","last_undegraded":"2024-04-21T08:45:27.402941+0000","last_fullsized":"2024-04-21T08:45:27.402941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:02.391785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404318+0000","last_change":"2024-04-21T08:44:33.486323+0000","last_active":"2024-04-21T08:45:27.404318+0000","last_peered":"2024-04-21T08:45:27.404318+0000","last_clean":"2024-04-21T08:45:27.404318+0000","last_became_active":"2024-04-21T08:44:33.486016+0000","last_became_peered":"2024-04-21T08:44:33.486016+0000","last_unstale":"2024-04-21T08:45:27.404318+0000","last_undegraded":"2024-04-21T08:45:27.404318+0000","last_fullsized":"2024-04-21T08:45:27.404318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:46.645314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516011+0000","last_change":"2024-04-21T08:44:34.496034+0000","last_active":"2024-04-21T08:45:27.516011+0000","last_peered":"2024-04-21T08:45:27.516011+0000","last_clean":"2024-04-21T08:45:27.516011+0000","last_became_active":"2024-04-21T08:44:34.495641+0000","last_became_peered":"2024-04-21T08:44:34.495641+0000","last_unstale":"2024-04-21T08:45:27.516011+0000","last_undegraded":"2024-04-21T08:45:27.516011+0000","last_fullsized":"2024-04-21T08:45:27.516011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:21:18.893326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516366+0000","last_change":"2024-04-21T08:44:38.517288+0000","last_active":"2024-04-21T08:45:27.516366+0000","last_peered":"2024-04-21T08:45:27.516366+0000","last_clean":"2024-04-21T08:45:27.516366+0000","last_became_active":"2024-04-21T08:44:38.516853+0000","last_became_peered":"2024-04-21T08:44:38.516853+0000","last_unstale":"2024-04-21T08:45:27.516366+0000","last_undegraded":"2024-04-21T08:45:27.516366+0000","last_fullsized":"2024-04-21T08:45:27.516366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:27:22.371703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":107,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:46:14.139457+0000","last_change":"2024-04-21T08:44:33.491578+0000","last_active":"2024-04-21T08:46:14.139457+0000","last_peered":"2024-04-21T08:46:14.139457+0000","last_clean":"2024-04-21T08:46:14.139457+0000","last_became_active":"2024-04-21T08:44:33.491238+0000","last_became_peered":"2024-04-21T08:44:33.491238+0000","last_unstale":"2024-04-21T08:46:14.139457+0000","last_undegraded":"2024-04-21T08:46:14.139457+0000","last_fullsized":"2024-04-21T08:46:14.139457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:50.399031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516721+0000","last_change":"2024-04-21T08:44:34.497632+0000","last_active":"2024-04-21T08:45:27.516721+0000","last_peered":"2024-04-21T08:45:27.516721+0000","last_clean":"2024-04-21T08:45:27.516721+0000","last_became_active":"2024-04-21T08:44:34.497318+0000","last_became_peered":"2024-04-21T08:44:34.497318+0000","last_unstale":"2024-04-21T08:45:27.516721+0000","last_undegraded":"2024-04-21T08:45:27.516721+0000","last_fullsized":"2024-04-21T08:45:27.516721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:11:33.918530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402732+0000","last_change":"2024-04-21T08:44:38.518414+0000","last_active":"2024-04-21T08:45:27.402732+0000","last_peered":"2024-04-21T08:45:27.402732+0000","last_clean":"2024-04-21T08:45:27.402732+0000","last_became_active":"2024-04-21T08:44:38.517953+0000","last_became_peered":"2024-04-21T08:44:38.517953+0000","last_unstale":"2024-04-21T08:45:27.402732+0000","last_undegraded":"2024-04-21T08:45:27.402732+0000","last_fullsized":"2024-04-21T08:45:27.402732+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:48.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.408673+0000","last_change":"2024-04-21T08:44:33.493896+0000","last_active":"2024-04-21T08:45:27.408673+0000","last_peered":"2024-04-21T08:45:27.408673+0000","last_clean":"2024-04-21T08:45:27.408673+0000","last_became_active":"2024-04-21T08:44:33.493566+0000","last_became_peered":"2024-04-21T08:44:33.493566+0000","last_unstale":"2024-04-21T08:45:27.408673+0000","last_undegraded":"2024-04-21T08:45:27.408673+0000","last_fullsized":"2024-04-21T08:45:27.408673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:47:17.307690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515647+0000","last_change":"2024-04-21T08:44:34.500406+0000","last_active":"2024-04-21T08:45:27.515647+0000","last_peered":"2024-04-21T08:45:27.515647+0000","last_clean":"2024-04-21T08:45:27.515647+0000","last_became_active":"2024-04-21T08:44:34.500048+0000","last_became_peered":"2024-04-21T08:44:34.500048+0000","last_unstale":"2024-04-21T08:45:27.515647+0000","last_undegraded":"2024-04-21T08:45:27.515647+0000","last_fullsized":"2024-04-21T08:45:27.515647+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:10:58.166155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.407940+0000","last_change":"2024-04-21T08:44:38.514812+0000","last_active":"2024-04-21T08:45:27.407940+0000","last_peered":"2024-04-21T08:45:27.407940+0000","last_clean":"2024-04-21T08:45:27.407940+0000","last_became_active":"2024-04-21T08:44:38.514247+0000","last_became_peered":"2024-04-21T08:44:38.514247+0000","last_unstale":"2024-04-21T08:45:27.407940+0000","last_undegraded":"2024-04-21T08:45:27.407940+0000","last_fullsized":"2024-04-21T08:45:27.407940+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:06:10.588453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402105+0000","last_change":"2024-04-21T08:44:33.493887+0000","last_active":"2024-04-21T08:45:27.402105+0000","last_peered":"2024-04-21T08:45:27.402105+0000","last_clean":"2024-04-21T08:45:27.402105+0000","last_became_active":"2024-04-21T08:44:33.493555+0000","last_became_peered":"2024-04-21T08:44:33.493555+0000","last_unstale":"2024-04-21T08:45:27.402105+0000","last_undegraded":"2024-04-21T08:45:27.402105+0000","last_fullsized":"2024-04-21T08:45:27.402105+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:18.089102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516416+0000","last_change":"2024-04-21T08:44:34.503259+0000","last_active":"2024-04-21T08:45:27.516416+0000","last_peered":"2024-04-21T08:45:27.516416+0000","last_clean":"2024-04-21T08:45:27.516416+0000","last_became_active":"2024-04-21T08:44:34.502845+0000","last_became_peered":"2024-04-21T08:44:34.502845+0000","last_unstale":"2024-04-21T08:45:27.516416+0000","last_undegraded":"2024-04-21T08:45:27.516416+0000","last_fullsized":"2024-04-21T08:45:27.516416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:19.902178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974556+0000","last_change":"2024-04-21T08:44:34.497608+0000","last_active":"2024-04-21T08:45:27.974556+0000","last_peered":"2024-04-21T08:45:27.974556+0000","last_clean":"2024-04-21T08:45:27.974556+0000","last_became_active":"2024-04-21T08:44:34.497255+0000","last_became_peered":"2024-04-21T08:44:34.497255+0000","last_unstale":"2024-04-21T08:45:27.974556+0000","last_undegraded":"2024-04-21T08:45:27.974556+0000","last_fullsized":"2024-04-21T08:45:27.974556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:51:15.181837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.404357+0000","last_change":"2024-04-21T08:44:34.495408+0000","last_active":"2024-04-21T08:45:27.404357+0000","last_peered":"2024-04-21T08:45:27.404357+0000","last_clean":"2024-04-21T08:45:27.404357+0000","last_became_active":"2024-04-21T08:44:34.495038+0000","last_became_peered":"2024-04-21T08:44:34.495038+0000","last_unstale":"2024-04-21T08:45:27.404357+0000","last_undegraded":"2024-04-21T08:45:27.404357+0000","last_fullsized":"2024-04-21T08:45:27.404357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:59.512245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.403868+0000","last_change":"2024-04-21T08:44:34.493292+0000","last_active":"2024-04-21T08:45:27.403868+0000","last_peered":"2024-04-21T08:45:27.403868+0000","last_clean":"2024-04-21T08:45:27.403868+0000","last_became_active":"2024-04-21T08:44:34.492913+0000","last_became_peered":"2024-04-21T08:44:34.492913+0000","last_unstale":"2024-04-21T08:45:27.403868+0000","last_undegraded":"2024-04-21T08:45:27.403868+0000","last_fullsized":"2024-04-21T08:45:27.403868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:24.271859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402956+0000","last_change":"2024-04-21T08:44:34.510448+0000","last_active":"2024-04-21T08:45:27.402956+0000","last_peered":"2024-04-21T08:45:27.402956+0000","last_clean":"2024-04-21T08:45:27.402956+0000","last_became_active":"2024-04-21T08:44:34.509958+0000","last_became_peered":"2024-04-21T08:44:34.509958+0000","last_unstale":"2024-04-21T08:45:27.402956+0000","last_undegraded":"2024-04-21T08:45:27.402956+0000","last_fullsized":"2024-04-21T08:45:27.402956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:36.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402282+0000","last_change":"2024-04-21T08:44:34.497415+0000","last_active":"2024-04-21T08:45:27.402282+0000","last_peered":"2024-04-21T08:45:27.402282+0000","last_clean":"2024-04-21T08:45:27.402282+0000","last_became_active":"2024-04-21T08:44:34.497120+0000","last_became_peered":"2024-04-21T08:44:34.497120+0000","last_unstale":"2024-04-21T08:45:27.402282+0000","last_undegraded":"2024-04-21T08:45:27.402282+0000","last_fullsized":"2024-04-21T08:45:27.402282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:30.232338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974996+0000","last_change":"2024-04-21T08:44:34.500356+0000","last_active":"2024-04-21T08:45:27.974996+0000","last_peered":"2024-04-21T08:45:27.974996+0000","last_clean":"2024-04-21T08:45:27.974996+0000","last_became_active":"2024-04-21T08:44:34.500032+0000","last_became_peered":"2024-04-21T08:44:34.500032+0000","last_unstale":"2024-04-21T08:45:27.974996+0000","last_undegraded":"2024-04-21T08:45:27.974996+0000","last_fullsized":"2024-04-21T08:45:27.974996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:24.900221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.405726+0000","last_change":"2024-04-21T08:44:34.493629+0000","last_active":"2024-04-21T08:45:27.405726+0000","last_peered":"2024-04-21T08:45:27.405726+0000","last_clean":"2024-04-21T08:45:27.405726+0000","last_became_active":"2024-04-21T08:44:34.493273+0000","last_became_peered":"2024-04-21T08:44:34.493273+0000","last_unstale":"2024-04-21T08:45:27.405726+0000","last_undegraded":"2024-04-21T08:45:27.405726+0000","last_fullsized":"2024-04-21T08:45:27.405726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:22:42.698957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.515341+0000","last_change":"2024-04-21T08:44:34.493427+0000","last_active":"2024-04-21T08:45:27.515341+0000","last_peered":"2024-04-21T08:45:27.515341+0000","last_clean":"2024-04-21T08:45:27.515341+0000","last_became_active":"2024-04-21T08:44:34.493105+0000","last_became_peered":"2024-04-21T08:44:34.493105+0000","last_unstale":"2024-04-21T08:45:27.515341+0000","last_undegraded":"2024-04-21T08:45:27.515341+0000","last_fullsized":"2024-04-21T08:45:27.515341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:37.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516355+0000","last_change":"2024-04-21T08:45:27.515899+0000","last_active":"2024-04-21T08:45:27.516355+0000","last_peered":"2024-04-21T08:45:27.516355+0000","last_clean":"2024-04-21T08:45:27.516355+0000","last_became_active":"2024-04-21T08:45:27.515567+0000","last_became_peered":"2024-04-21T08:45:27.515567+0000","last_unstale":"2024-04-21T08:45:27.516355+0000","last_undegraded":"2024-04-21T08:45:27.516355+0000","last_fullsized":"2024-04-21T08:45:27.516355+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:16.645869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.514604+0000","last_change":"2024-04-21T08:44:34.499420+0000","last_active":"2024-04-21T08:45:27.514604+0000","last_peered":"2024-04-21T08:45:27.514604+0000","last_clean":"2024-04-21T08:45:27.514604+0000","last_became_active":"2024-04-21T08:44:34.499079+0000","last_became_peered":"2024-04-21T08:44:34.499079+0000","last_unstale":"2024-04-21T08:45:27.514604+0000","last_undegraded":"2024-04-21T08:45:27.514604+0000","last_fullsized":"2024-04-21T08:45:27.514604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:45.351422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.516918+0000","last_change":"2024-04-21T08:44:34.493420+0000","last_active":"2024-04-21T08:45:27.516918+0000","last_peered":"2024-04-21T08:45:27.516918+0000","last_clean":"2024-04-21T08:45:27.516918+0000","last_became_active":"2024-04-21T08:44:34.493098+0000","last_became_peered":"2024-04-21T08:44:34.493098+0000","last_unstale":"2024-04-21T08:45:27.516918+0000","last_undegraded":"2024-04-21T08:45:27.516918+0000","last_fullsized":"2024-04-21T08:45:27.516918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:22:34.316822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.402161+0000","last_change":"2024-04-21T08:44:34.494760+0000","last_active":"2024-04-21T08:45:27.402161+0000","last_peered":"2024-04-21T08:45:27.402161+0000","last_clean":"2024-04-21T08:45:27.402161+0000","last_became_active":"2024-04-21T08:44:34.494265+0000","last_became_peered":"2024-04-21T08:44:34.494265+0000","last_unstale":"2024-04-21T08:45:27.402161+0000","last_undegraded":"2024-04-21T08:45:27.402161+0000","last_fullsized":"2024-04-21T08:45:27.402161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:16.035030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:45:27.974799+0000","last_change":"2024-04-21T08:44:34.498785+0000","last_active":"2024-04-21T08:45:27.974799+0000","last_peered":"2024-04-21T08:45:27.974799+0000","last_clean":"2024-04-21T08:45:27.974799+0000","last_became_active":"2024-04-21T08:44:34.495159+0000","last_became_peered":"2024-04-21T08:44:34.495159+0000","last_unstale":"2024-04-21T08:45:27.974799+0000","last_undegraded":"2024-04-21T08:45:27.974799+0000","last_fullsized":"2024-04-21T08:45:27.974799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:03:20.095016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68197,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":781,"num_read_kb":465,"num_write":238,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":204591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":501370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645352,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1724416,"data_stored":1149961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808877,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":505396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":501678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1650688,"data_stored":1097784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":496804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":559609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":498260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1118208,"data_stored":559829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":496512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":500878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36964,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708188,"statfs":{"total":95995035648,"available":95957184512,"internally_reserved":0,"allocated":1658880,"data_stored":1093366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:46:18.354 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-21T08:46:18.354 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-21T08:46:18.354 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-21T08:46:18.354 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph health --format=json 2024-04-21T08:46:18.597 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:46:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:19 smithi134 ceph-mon[31772]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:46:19.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:19 smithi134 ceph-mon[31772]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:46:19.846 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:19 smithi082 ceph-mon[27967]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:46:19.847 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:19 smithi082 ceph-mon[27967]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:46:19.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:19 smithi106 ceph-mon[31072]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:46:19.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:19 smithi106 ceph-mon[31072]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:46:20.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.488+0000 7f39f6318700 1 -- 172.21.15.82:0/1644139535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39f0107990 msgr2=0x7f39f0107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.488+0000 7f39f6318700 1 --2- 172.21.15.82:0/1644139535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39f0107990 0x7f39f0107d70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009990 tx=0x7f39e4009ca0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.488+0000 7f39f6318700 1 -- 172.21.15.82:0/1644139535 shutdown_connections 2024-04-21T08:46:20.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.488+0000 7f39f6318700 1 --2- 172.21.15.82:0/1644139535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0102300 0x7f39f010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.488+0000 7f39f6318700 1 --2- 172.21.15.82:0/1644139535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39f0107990 0x7f39f0107d70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.488+0000 7f39f6318700 1 --2- 172.21.15.82:0/1644139535 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f0101940 0x7f39f0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.488+0000 7f39f6318700 1 -- 172.21.15.82:0/1644139535 >> 172.21.15.82:0/1644139535 conn(0x7f39f00788f0 msgr2=0x7f39f0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:20.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.488+0000 7f39f6318700 1 -- 172.21.15.82:0/1644139535 shutdown_connections 2024-04-21T08:46:20.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.488+0000 7f39f6318700 1 -- 172.21.15.82:0/1644139535 wait complete. 2024-04-21T08:46:20.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.489+0000 7f39f6318700 1 Processor -- start 2024-04-21T08:46:20.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.489+0000 7f39f6318700 1 -- start start 2024-04-21T08:46:20.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.489+0000 7f39f6318700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0101940 0x7f39f01034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.489+0000 7f39f6318700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39f0102300 0x7f39f0103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.489+0000 7f39f6318700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f0107990 0x7f39f0103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.489+0000 7f39f6318700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0111b00 con 0x7f39f0102300 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.489+0000 7f39f6318700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0111980 con 0x7f39f0107990 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.489+0000 7f39f6318700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0111c80 con 0x7f39f0101940 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39ef7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39f0102300 0x7f39f0103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39effff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0101940 0x7f39f01034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39ef7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39f0102300 0x7f39f0103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38828/0 (socket says 172.21.15.82:38828) 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39f48b5700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f0107990 0x7f39f0103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39effff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0101940 0x7f39f01034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:56636/0 (socket says 172.21.15.82:56636) 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39ef7fe700 1 -- 172.21.15.82:0/1595982525 learned_addr learned my addr 172.21.15.82:0/1595982525 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39ef7fe700 1 -- 172.21.15.82:0/1595982525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0101940 msgr2=0x7f39f01034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39ef7fe700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0101940 0x7f39f01034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39ef7fe700 1 -- 172.21.15.82:0/1595982525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f0107990 msgr2=0x7f39f0103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39ef7fe700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f0107990 0x7f39f0103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39ef7fe700 1 -- 172.21.15.82:0/1595982525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f01a7680 con 0x7f39f0102300 2024-04-21T08:46:20.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39effff700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0101940 0x7f39f01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:20.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39f48b5700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f0107990 0x7f39f0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:20.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39ef7fe700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39f0102300 0x7f39f0103a30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f39e000c920 tx=0x7f39e000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:20.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39ed7fa700 1 -- 172.21.15.82:0/1595982525 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0004a30 con 0x7f39f0102300 2024-04-21T08:46:20.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39ed7fa700 1 -- 172.21.15.82:0/1595982525 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39e00054d0 con 0x7f39f0102300 2024-04-21T08:46:20.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39f6318700 1 -- 172.21.15.82:0/1595982525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f01a7970 con 0x7f39f0102300 2024-04-21T08:46:20.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.490+0000 7f39ed7fa700 1 -- 172.21.15.82:0/1595982525 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0007c70 con 0x7f39f0102300 2024-04-21T08:46:20.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.491+0000 7f39f6318700 1 -- 172.21.15.82:0/1595982525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f01a7eb0 con 0x7f39f0102300 2024-04-21T08:46:20.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.491+0000 7f39f6318700 1 -- 172.21.15.82:0/1595982525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f0103060 con 0x7f39f0102300 2024-04-21T08:46:20.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.493+0000 7f39ed7fa700 1 -- 172.21.15.82:0/1595982525 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f39e0004c60 con 0x7f39f0102300 2024-04-21T08:46:20.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.494+0000 7f39ed7fa700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f39d80a3cd0 0x7f39d80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.494+0000 7f39effff700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f39d80a3cd0 0x7f39d80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.494+0000 7f39ed7fa700 1 -- 172.21.15.82:0/1595982525 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f39e00d08e0 con 0x7f39f0102300 2024-04-21T08:46:20.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.495+0000 7f39effff700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f39d80a3cd0 0x7f39d80a6190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f39e4006010 tx=0x7f39e400b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:20.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.498+0000 7f39ed7fa700 1 -- 172.21.15.82:0/1595982525 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f39e0014030 con 0x7f39f0102300 2024-04-21T08:46:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.752+0000 7f39f6318700 1 -- 172.21.15.82:0/1595982525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f39f0069730 con 0x7f39f0102300 2024-04-21T08:46:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.752+0000 7f39ed7fa700 1 -- 172.21.15.82:0/1595982525 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f39e009ce20 con 0x7f39f0102300 2024-04-21T08:46:20.752 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:20.752 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-21T08:46:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.754+0000 7f39f6318700 1 -- 172.21.15.82:0/1595982525 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f39d80a3cd0 msgr2=0x7f39d80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.754+0000 7f39f6318700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f39d80a3cd0 0x7f39d80a6190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f39e4006010 tx=0x7f39e400b560 comp rx=0 tx=0).stop 2024-04-21T08:46:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.754+0000 7f39f6318700 1 -- 172.21.15.82:0/1595982525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39f0102300 msgr2=0x7f39f0103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.754+0000 7f39f6318700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39f0102300 0x7f39f0103a30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f39e000c920 tx=0x7f39e000cc30 comp rx=0 tx=0).stop 2024-04-21T08:46:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.755+0000 7f39f6318700 1 -- 172.21.15.82:0/1595982525 shutdown_connections 2024-04-21T08:46:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.755+0000 7f39f6318700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f39d80a3cd0 0x7f39d80a6190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.755+0000 7f39f6318700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f0107990 0x7f39f0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.755+0000 7f39f6318700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39f0101940 0x7f39f01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.755+0000 7f39f6318700 1 --2- 172.21.15.82:0/1595982525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39f0102300 0x7f39f0103a30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.755+0000 7f39f6318700 1 -- 172.21.15.82:0/1595982525 >> 172.21.15.82:0/1595982525 conn(0x7f39f00788f0 msgr2=0x7f39f006b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.755+0000 7f39f6318700 1 -- 172.21.15.82:0/1595982525 shutdown_connections 2024-04-21T08:46:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:20.755+0000 7f39f6318700 1 -- 172.21.15.82:0/1595982525 wait complete. 2024-04-21T08:46:21.516 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-21T08:46:21.516 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:46:21.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.618+0000 7eff65352640 1 -- 172.21.15.82:0/593724772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff601011c0 msgr2=0x7eff60101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:21.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.618+0000 7eff65352640 1 --2- 172.21.15.82:0/593724772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff601011c0 0x7eff60101620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7eff54009900 tx=0x7eff5402bcd0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.618+0000 7eff65352640 1 -- 172.21.15.82:0/593724772 shutdown_connections 2024-04-21T08:46:21.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.618+0000 7eff65352640 1 --2- 172.21.15.82:0/593724772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff60101b60 0x7eff60101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.618+0000 7eff65352640 1 --2- 172.21.15.82:0/593724772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff601011c0 0x7eff60101620 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.618+0000 7eff65352640 1 --2- 172.21.15.82:0/593724772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff60100860 0x7eff60100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.618+0000 7eff65352640 1 -- 172.21.15.82:0/593724772 >> 172.21.15.82:0/593724772 conn(0x7eff600fc3e0 msgr2=0x7eff600fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.619+0000 7eff65352640 1 -- 172.21.15.82:0/593724772 shutdown_connections 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.619+0000 7eff65352640 1 -- 172.21.15.82:0/593724772 wait complete. 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.619+0000 7eff65352640 1 Processor -- start 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.619+0000 7eff65352640 1 -- start start 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.619+0000 7eff65352640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff60100860 0x7eff6019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.619+0000 7eff65352640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff601011c0 0x7eff6019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.619+0000 7eff65352640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff60101b60 0x7eff601a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.619+0000 7eff65352640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010fcb0 con 0x7eff601011c0 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.619+0000 7eff65352640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010fb30 con 0x7eff60100860 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.619+0000 7eff65352640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010fe30 con 0x7eff60101b60 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff5e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff601011c0 0x7eff6019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff5e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff601011c0 0x7eff6019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38844/0 (socket says 172.21.15.82:38844) 2024-04-21T08:46:21.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff5e7fc640 1 -- 172.21.15.82:0/2284403195 learned_addr learned my addr 172.21.15.82:0/2284403195 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:21.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff5effd640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff60100860 0x7eff6019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:21.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff5f7fe640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff60101b60 0x7eff601a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:21.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff5e7fc640 1 -- 172.21.15.82:0/2284403195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff60101b60 msgr2=0x7eff601a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:21.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff5e7fc640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff60101b60 0x7eff601a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff5e7fc640 1 -- 172.21.15.82:0/2284403195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff60100860 msgr2=0x7eff6019a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:21.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff5e7fc640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff60100860 0x7eff6019a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff5e7fc640 1 -- 172.21.15.82:0/2284403195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff601a1fc0 con 0x7eff601011c0 2024-04-21T08:46:21.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff5f7fe640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff60101b60 0x7eff601a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:21.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff5e7fc640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff601011c0 0x7eff6019a8e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7eff54036040 tx=0x7eff5402da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:21.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff47fff640 1 -- 172.21.15.82:0/2284403195 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54043020 con 0x7eff601011c0 2024-04-21T08:46:21.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff47fff640 1 -- 172.21.15.82:0/2284403195 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff5402de10 con 0x7eff601011c0 2024-04-21T08:46:21.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff65352640 1 -- 172.21.15.82:0/2284403195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff601a2250 con 0x7eff601011c0 2024-04-21T08:46:21.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff47fff640 1 -- 172.21.15.82:0/2284403195 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54003c60 con 0x7eff601011c0 2024-04-21T08:46:21.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.620+0000 7eff65352640 1 -- 172.21.15.82:0/2284403195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff601a27a0 con 0x7eff601011c0 2024-04-21T08:46:21.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.621+0000 7eff65352640 1 -- 172.21.15.82:0/2284403195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6010b7f0 con 0x7eff601011c0 2024-04-21T08:46:21.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.622+0000 7eff47fff640 1 -- 172.21.15.82:0/2284403195 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7eff54046020 con 0x7eff601011c0 2024-04-21T08:46:21.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.623+0000 7eff47fff640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7eff480a3c40 0x7eff480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:21.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.623+0000 7eff5effd640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7eff480a3c40 0x7eff480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:21.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.623+0000 7eff47fff640 1 -- 172.21.15.82:0/2284403195 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7eff540f3070 con 0x7eff601011c0 2024-04-21T08:46:21.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.623+0000 7eff5effd640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7eff480a3c40 0x7eff480a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7eff500058c0 tx=0x7eff5000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:21.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.627+0000 7eff47fff640 1 -- 172.21.15.82:0/2284403195 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff5404d020 con 0x7eff601011c0 2024-04-21T08:46:21.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:21 smithi134 ceph-mon[31772]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:46:21.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:21 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1595982525' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T08:46:21.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.805+0000 7eff65352640 1 -- 172.21.15.82:0/2284403195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff6002d830 con 0x7eff601011c0 2024-04-21T08:46:21.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.806+0000 7eff47fff640 1 -- 172.21.15.82:0/2284403195 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24580 (secure 0 0 0) 0x7eff540bf6c0 con 0x7eff601011c0 2024-04-21T08:46:21.806 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:21.806 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:45:27.389965+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:46:21.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.808+0000 7eff65352640 1 -- 172.21.15.82:0/2284403195 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7eff480a3c40 msgr2=0x7eff480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:21.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.808+0000 7eff65352640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7eff480a3c40 0x7eff480a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7eff500058c0 tx=0x7eff5000a4c0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.808+0000 7eff65352640 1 -- 172.21.15.82:0/2284403195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff601011c0 msgr2=0x7eff6019a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:21.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.808+0000 7eff65352640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff601011c0 0x7eff6019a8e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7eff54036040 tx=0x7eff5402da10 comp rx=0 tx=0).stop 2024-04-21T08:46:21.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.808+0000 7eff65352640 1 -- 172.21.15.82:0/2284403195 shutdown_connections 2024-04-21T08:46:21.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.808+0000 7eff65352640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7eff480a3c40 0x7eff480a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.808+0000 7eff65352640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff60101b60 0x7eff601a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.808+0000 7eff65352640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff601011c0 0x7eff6019a8e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.808+0000 7eff65352640 1 --2- 172.21.15.82:0/2284403195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff60100860 0x7eff6019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.808+0000 7eff65352640 1 -- 172.21.15.82:0/2284403195 >> 172.21.15.82:0/2284403195 conn(0x7eff600fc3e0 msgr2=0x7eff600fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:21.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.808+0000 7eff65352640 1 -- 172.21.15.82:0/2284403195 shutdown_connections 2024-04-21T08:46:21.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.808+0000 7eff65352640 1 -- 172.21.15.82:0/2284403195 wait complete. 2024-04-21T08:46:21.818 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:21 smithi082 ceph-mon[27967]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:46:21.818 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:21 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1595982525' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T08:46:21.819 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:46:21.820 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:46:21.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:21 smithi106 ceph-mon[31072]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:46:21.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:21 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1595982525' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T08:46:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.922+0000 7fcd3074b640 1 -- 172.21.15.82:0/4294462708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28101fa0 msgr2=0x7fcd281023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.922+0000 7fcd3074b640 1 --2- 172.21.15.82:0/4294462708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28101fa0 0x7fcd281023c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcd24009900 tx=0x7fcd2402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.922+0000 7fcd3074b640 1 -- 172.21.15.82:0/4294462708 shutdown_connections 2024-04-21T08:46:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.922+0000 7fcd3074b640 1 --2- 172.21.15.82:0/4294462708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd28103b50 0x7fcd2810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.922+0000 7fcd3074b640 1 --2- 172.21.15.82:0/4294462708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28103190 0x7fcd28103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.922+0000 7fcd3074b640 1 --2- 172.21.15.82:0/4294462708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28101fa0 0x7fcd281023c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.922+0000 7fcd3074b640 1 -- 172.21.15.82:0/4294462708 >> 172.21.15.82:0/4294462708 conn(0x7fcd280fd530 msgr2=0x7fcd280ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.922+0000 7fcd3074b640 1 -- 172.21.15.82:0/4294462708 shutdown_connections 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.922+0000 7fcd3074b640 1 -- 172.21.15.82:0/4294462708 wait complete. 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd3074b640 1 Processor -- start 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd3074b640 1 -- start start 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd3074b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28101fa0 0x7fcd28198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd3074b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28103190 0x7fcd281987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd3074b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd28103b50 0x7fcd2819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd3074b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2810db20 con 0x7fcd28103190 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd3074b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2810d9a0 con 0x7fcd28103b50 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd3074b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2810dca0 con 0x7fcd28101fa0 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2dce6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28103190 0x7fcd281987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:21.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2dce6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28103190 0x7fcd281987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38872/0 (socket says 172.21.15.82:38872) 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2dce6640 1 -- 172.21.15.82:0/1715442288 learned_addr learned my addr 172.21.15.82:0/1715442288 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2ece8640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd28103b50 0x7fcd2819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2dce6640 1 -- 172.21.15.82:0/1715442288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28101fa0 msgr2=0x7fcd28198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2e4e7640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28101fa0 0x7fcd28198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2dce6640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28101fa0 0x7fcd28198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2dce6640 1 -- 172.21.15.82:0/1715442288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd28103b50 msgr2=0x7fcd2819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2dce6640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd28103b50 0x7fcd2819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2dce6640 1 -- 172.21.15.82:0/1715442288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2819fe90 con 0x7fcd28103190 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2e4e7640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28101fa0 0x7fcd28198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2ece8640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd28103b50 0x7fcd2819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.923+0000 7fcd2dce6640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28103190 0x7fcd281987b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800b4a0 tx=0x7fcd1800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.924+0000 7fcd1f7fe640 1 -- 172.21.15.82:0/1715442288 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18014070 con 0x7fcd28103190 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.924+0000 7fcd1f7fe640 1 -- 172.21.15.82:0/1715442288 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd18010ac0 con 0x7fcd28103190 2024-04-21T08:46:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.924+0000 7fcd3074b640 1 -- 172.21.15.82:0/1715442288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd281a0180 con 0x7fcd28103190 2024-04-21T08:46:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.924+0000 7fcd1f7fe640 1 -- 172.21.15.82:0/1715442288 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18017c30 con 0x7fcd28103190 2024-04-21T08:46:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.924+0000 7fcd3074b640 1 -- 172.21.15.82:0/1715442288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd281a0700 con 0x7fcd28103190 2024-04-21T08:46:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.924+0000 7fcd3074b640 1 -- 172.21.15.82:0/1715442288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd28109390 con 0x7fcd28103190 2024-04-21T08:46:21.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.926+0000 7fcd1f7fe640 1 -- 172.21.15.82:0/1715442288 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcd18017070 con 0x7fcd28103190 2024-04-21T08:46:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.926+0000 7fcd1f7fe640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fcd140a3c40 0x7fcd140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.926+0000 7fcd2e4e7640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fcd140a3c40 0x7fcd140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.926+0000 7fcd1f7fe640 1 -- 172.21.15.82:0/1715442288 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fcd180cfc10 con 0x7fcd28103190 2024-04-21T08:46:21.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.927+0000 7fcd2e4e7640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fcd140a3c40 0x7fcd140a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcd240096c0 tx=0x7fcd24000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:21.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:21.931+0000 7fcd1f7fe640 1 -- 172.21.15.82:0/1715442288 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd1809c260 con 0x7fcd28103190 2024-04-21T08:46:22.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.122+0000 7fcd3074b640 1 -- 172.21.15.82:0/1715442288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd2802d810 con 0x7fcd28103190 2024-04-21T08:46:22.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.123+0000 7fcd1f7fe640 1 -- 172.21.15.82:0/1715442288 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fcd1809bb60 con 0x7fcd28103190 2024-04-21T08:46:22.123 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:46:22.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.125+0000 7fcd3074b640 1 -- 172.21.15.82:0/1715442288 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fcd140a3c40 msgr2=0x7fcd140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.125+0000 7fcd3074b640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fcd140a3c40 0x7fcd140a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcd240096c0 tx=0x7fcd24000bc0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.125+0000 7fcd3074b640 1 -- 172.21.15.82:0/1715442288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28103190 msgr2=0x7fcd281987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.125+0000 7fcd3074b640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28103190 0x7fcd281987b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800b4a0 tx=0x7fcd1800b960 comp rx=0 tx=0).stop 2024-04-21T08:46:22.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.125+0000 7fcd3074b640 1 -- 172.21.15.82:0/1715442288 shutdown_connections 2024-04-21T08:46:22.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.125+0000 7fcd3074b640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fcd140a3c40 0x7fcd140a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.125+0000 7fcd3074b640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd28103b50 0x7fcd2819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.125+0000 7fcd3074b640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd28103190 0x7fcd281987b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.125+0000 7fcd3074b640 1 --2- 172.21.15.82:0/1715442288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd28101fa0 0x7fcd28198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.125+0000 7fcd3074b640 1 -- 172.21.15.82:0/1715442288 >> 172.21.15.82:0/1715442288 conn(0x7fcd280fd530 msgr2=0x7fcd28106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:22.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.125+0000 7fcd3074b640 1 -- 172.21.15.82:0/1715442288 shutdown_connections 2024-04-21T08:46:22.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.125+0000 7fcd3074b640 1 -- 172.21.15.82:0/1715442288 wait complete. 2024-04-21T08:46:22.136 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:46:22.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.236+0000 7f03c710d640 1 -- 172.21.15.82:0/888533223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c0100e90 msgr2=0x7f03c01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.236+0000 7f03c710d640 1 --2- 172.21.15.82:0/888533223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c0100e90 0x7f03c01012b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009ca0 tx=0x7f03b802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.236+0000 7f03c710d640 1 -- 172.21.15.82:0/888533223 shutdown_connections 2024-04-21T08:46:22.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.236+0000 7f03c710d640 1 --2- 172.21.15.82:0/888533223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03c01029a0 0x7f03c01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.236+0000 7f03c710d640 1 --2- 172.21.15.82:0/888533223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c0101fe0 0x7f03c0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.236+0000 7f03c710d640 1 --2- 172.21.15.82:0/888533223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c0100e90 0x7f03c01012b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.236+0000 7f03c710d640 1 -- 172.21.15.82:0/888533223 >> 172.21.15.82:0/888533223 conn(0x7f03c00fc400 msgr2=0x7f03c00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.236+0000 7f03c710d640 1 -- 172.21.15.82:0/888533223 shutdown_connections 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.236+0000 7f03c710d640 1 -- 172.21.15.82:0/888533223 wait complete. 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.236+0000 7f03c710d640 1 Processor -- start 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.236+0000 7f03c710d640 1 -- start start 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c710d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03c0100e90 0x7f03c01970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c710d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c0101fe0 0x7f03c01975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c710d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c01029a0 0x7f03c019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c710d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f03c010c9e0 con 0x7f03c01029a0 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c710d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f03c010c860 con 0x7f03c0100e90 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c710d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03c010cb60 con 0x7f03c0101fe0 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c56aa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c01029a0 0x7f03c019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:22.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c56aa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c01029a0 0x7f03c019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38894/0 (socket says 172.21.15.82:38894) 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c56aa640 1 -- 172.21.15.82:0/2651352762 learned_addr learned my addr 172.21.15.82:0/2651352762 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c4ea9640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03c0100e90 0x7f03c01970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03bffff640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c0101fe0 0x7f03c01975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c56aa640 1 -- 172.21.15.82:0/2651352762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c0101fe0 msgr2=0x7f03c01975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c56aa640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c0101fe0 0x7f03c01975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c56aa640 1 -- 172.21.15.82:0/2651352762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03c0100e90 msgr2=0x7f03c01970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c56aa640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03c0100e90 0x7f03c01970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c56aa640 1 -- 172.21.15.82:0/2651352762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c019ecc0 con 0x7f03c01029a0 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c56aa640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c01029a0 0x7f03c019e650 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c970 tx=0x7f03b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03bdffb640 1 -- 172.21.15.82:0/2651352762 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0007bb0 con 0x7f03c01029a0 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03bdffb640 1 -- 172.21.15.82:0/2651352762 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b0007d50 con 0x7f03c01029a0 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03c710d640 1 -- 172.21.15.82:0/2651352762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c019efb0 con 0x7f03c01029a0 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.237+0000 7f03bdffb640 1 -- 172.21.15.82:0/2651352762 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0018770 con 0x7f03c01029a0 2024-04-21T08:46:22.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.238+0000 7f03c710d640 1 -- 172.21.15.82:0/2651352762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c019f470 con 0x7f03c01029a0 2024-04-21T08:46:22.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.239+0000 7f03c710d640 1 -- 172.21.15.82:0/2651352762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c0108270 con 0x7f03c01029a0 2024-04-21T08:46:22.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.240+0000 7f03bdffb640 1 -- 172.21.15.82:0/2651352762 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f03b000f990 con 0x7f03c01029a0 2024-04-21T08:46:22.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.240+0000 7f03bdffb640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f03a80a3cc0 0x7f03a80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:22.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.240+0000 7f03c4ea9640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f03a80a3cc0 0x7f03a80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:22.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.240+0000 7f03bdffb640 1 -- 172.21.15.82:0/2651352762 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f03b00d0960 con 0x7f03c01029a0 2024-04-21T08:46:22.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.241+0000 7f03c4ea9640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f03a80a3cc0 0x7f03a80a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005e30 tx=0x7f03b8000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:22.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.245+0000 7f03bdffb640 1 -- 172.21.15.82:0/2651352762 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03b0014030 con 0x7f03c01029a0 2024-04-21T08:46:22.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.422+0000 7f03c710d640 1 -- 172.21.15.82:0/2651352762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f03c002d810 con 0x7f03c01029a0 2024-04-21T08:46:22.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.422+0000 7f03bdffb640 1 -- 172.21.15.82:0/2651352762 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f03b009cea0 con 0x7f03c01029a0 2024-04-21T08:46:22.422 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:46:22.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.424+0000 7f03c710d640 1 -- 172.21.15.82:0/2651352762 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f03a80a3cc0 msgr2=0x7f03a80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.424+0000 7f03c710d640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f03a80a3cc0 0x7f03a80a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005e30 tx=0x7f03b8000f40 comp rx=0 tx=0).stop 2024-04-21T08:46:22.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.424+0000 7f03c710d640 1 -- 172.21.15.82:0/2651352762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c01029a0 msgr2=0x7f03c019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.424+0000 7f03c710d640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c01029a0 0x7f03c019e650 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c970 tx=0x7f03b000ce30 comp rx=0 tx=0).stop 2024-04-21T08:46:22.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.424+0000 7f03c710d640 1 -- 172.21.15.82:0/2651352762 shutdown_connections 2024-04-21T08:46:22.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.424+0000 7f03c710d640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f03a80a3cc0 0x7f03a80a6180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.424+0000 7f03c710d640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03c01029a0 0x7f03c019e650 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.424+0000 7f03c710d640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03c0101fe0 0x7f03c01975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.424+0000 7f03c710d640 1 --2- 172.21.15.82:0/2651352762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03c0100e90 0x7f03c01970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.424+0000 7f03c710d640 1 -- 172.21.15.82:0/2651352762 >> 172.21.15.82:0/2651352762 conn(0x7f03c00fc400 msgr2=0x7f03c0105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:22.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.425+0000 7f03c710d640 1 -- 172.21.15.82:0/2651352762 shutdown_connections 2024-04-21T08:46:22.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.425+0000 7f03c710d640 1 -- 172.21.15.82:0/2651352762 wait complete. 2024-04-21T08:46:22.435 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:46:22.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.540+0000 7f7c9290d640 1 -- 172.21.15.82:0/3575141485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c103b50 msgr2=0x7f7c8c10a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.540+0000 7f7c9290d640 1 --2- 172.21.15.82:0/3575141485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c103b50 0x7f7c8c10a3e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009ca0 tx=0x7f7c8402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.540+0000 7f7c9290d640 1 -- 172.21.15.82:0/3575141485 shutdown_connections 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.540+0000 7f7c9290d640 1 --2- 172.21.15.82:0/3575141485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c103b50 0x7f7c8c10a3e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.540+0000 7f7c9290d640 1 --2- 172.21.15.82:0/3575141485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c103190 0x7f7c8c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.540+0000 7f7c9290d640 1 --2- 172.21.15.82:0/3575141485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c8c101fa0 0x7f7c8c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.540+0000 7f7c9290d640 1 -- 172.21.15.82:0/3575141485 >> 172.21.15.82:0/3575141485 conn(0x7f7c8c0fd530 msgr2=0x7f7c8c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.540+0000 7f7c9290d640 1 -- 172.21.15.82:0/3575141485 shutdown_connections 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.540+0000 7f7c9290d640 1 -- 172.21.15.82:0/3575141485 wait complete. 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.540+0000 7f7c9290d640 1 Processor -- start 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.540+0000 7f7c9290d640 1 -- start start 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c9290d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c8c101fa0 0x7f7c8c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c9290d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c103190 0x7f7c8c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c9290d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c103b50 0x7f7c8c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c9290d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c10daa0 con 0x7f7c8c103b50 2024-04-21T08:46:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c9290d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c10d920 con 0x7f7c8c101fa0 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c9290d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c10dc20 con 0x7f7c8c103190 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c90eaa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c103b50 0x7f7c8c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c90eaa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c103b50 0x7f7c8c19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38912/0 (socket says 172.21.15.82:38912) 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c90eaa640 1 -- 172.21.15.82:0/1195766632 learned_addr learned my addr 172.21.15.82:0/1195766632 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c8bfff640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c8c101fa0 0x7f7c8c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c8b7fe640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c103190 0x7f7c8c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c90eaa640 1 -- 172.21.15.82:0/1195766632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c103190 msgr2=0x7f7c8c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c90eaa640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c103190 0x7f7c8c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c90eaa640 1 -- 172.21.15.82:0/1195766632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c8c101fa0 msgr2=0x7f7c8c1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c90eaa640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c8c101fa0 0x7f7c8c1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c90eaa640 1 -- 172.21.15.82:0/1195766632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8c19fd10 con 0x7f7c8c103b50 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c8bfff640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c8c101fa0 0x7f7c8c1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c90eaa640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c103b50 0x7f7c8c19f6a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7c8402c700 tx=0x7f7c84035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:22.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.541+0000 7f7c897fa640 1 -- 172.21.15.82:0/1195766632 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84039040 con 0x7f7c8c103b50 2024-04-21T08:46:22.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.542+0000 7f7c9290d640 1 -- 172.21.15.82:0/1195766632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8c19ffa0 con 0x7f7c8c103b50 2024-04-21T08:46:22.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.542+0000 7f7c897fa640 1 -- 172.21.15.82:0/1195766632 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c84044070 con 0x7f7c8c103b50 2024-04-21T08:46:22.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.542+0000 7f7c897fa640 1 -- 172.21.15.82:0/1195766632 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84004440 con 0x7f7c8c103b50 2024-04-21T08:46:22.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.542+0000 7f7c9290d640 1 -- 172.21.15.82:0/1195766632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8c1a0400 con 0x7f7c8c103b50 2024-04-21T08:46:22.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.543+0000 7f7c9290d640 1 -- 172.21.15.82:0/1195766632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c8c109340 con 0x7f7c8c103b50 2024-04-21T08:46:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.544+0000 7f7c897fa640 1 -- 172.21.15.82:0/1195766632 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7c840045e0 con 0x7f7c8c103b50 2024-04-21T08:46:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.545+0000 7f7c897fa640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7c740a3c40 0x7f7c740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.545+0000 7f7c8bfff640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7c740a3c40 0x7f7c740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.545+0000 7f7c897fa640 1 -- 172.21.15.82:0/1195766632 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f7c840f2eb0 con 0x7f7c8c103b50 2024-04-21T08:46:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.545+0000 7f7c8bfff640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7c740a3c40 0x7f7c740a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c102ff0 tx=0x7f7c7c00a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:22.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.549+0000 7f7c897fa640 1 -- 172.21.15.82:0/1195766632 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c840bf500 con 0x7f7c8c103b50 2024-04-21T08:46:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:22 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2284403195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:22 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1715442288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:46:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:22 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2651352762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:46:22.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.729+0000 7f7c9290d640 1 -- 172.21.15.82:0/1195766632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7c8c062bf0 con 0x7f7c8c103b50 2024-04-21T08:46:22.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.730+0000 7f7c897fa640 1 -- 172.21.15.82:0/1195766632 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f7c840bee00 con 0x7f7c8c103b50 2024-04-21T08:46:22.729 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:46:22.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.732+0000 7f7c9290d640 1 -- 172.21.15.82:0/1195766632 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7c740a3c40 msgr2=0x7f7c740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.732+0000 7f7c9290d640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7c740a3c40 0x7f7c740a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c102ff0 tx=0x7f7c7c00a5e0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.732+0000 7f7c9290d640 1 -- 172.21.15.82:0/1195766632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c103b50 msgr2=0x7f7c8c19f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.732+0000 7f7c9290d640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c103b50 0x7f7c8c19f6a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7c8402c700 tx=0x7f7c84035d60 comp rx=0 tx=0).stop 2024-04-21T08:46:22.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.732+0000 7f7c9290d640 1 -- 172.21.15.82:0/1195766632 shutdown_connections 2024-04-21T08:46:22.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.732+0000 7f7c9290d640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7c740a3c40 0x7f7c740a6100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.732+0000 7f7c9290d640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c8c103b50 0x7f7c8c19f6a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.732+0000 7f7c9290d640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c103190 0x7f7c8c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.732+0000 7f7c9290d640 1 --2- 172.21.15.82:0/1195766632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c8c101fa0 0x7f7c8c1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.732+0000 7f7c9290d640 1 -- 172.21.15.82:0/1195766632 >> 172.21.15.82:0/1195766632 conn(0x7f7c8c0fd530 msgr2=0x7f7c8c106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:22.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.732+0000 7f7c9290d640 1 -- 172.21.15.82:0/1195766632 shutdown_connections 2024-04-21T08:46:22.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.732+0000 7f7c9290d640 1 -- 172.21.15.82:0/1195766632 wait complete. 2024-04-21T08:46:22.741 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:22 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2284403195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:22.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:22 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1715442288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:46:22.742 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:22 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2651352762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:46:22.743 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:46:22.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.844+0000 7f7987f2b640 1 -- 172.21.15.82:0/598931958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7980102040 msgr2=0x7f79801024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.844+0000 7f7987f2b640 1 --2- 172.21.15.82:0/598931958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7980102040 0x7f79801024c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7970009900 tx=0x7f797002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:22.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.844+0000 7f7987f2b640 1 -- 172.21.15.82:0/598931958 shutdown_connections 2024-04-21T08:46:22.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.844+0000 7f7987f2b640 1 --2- 172.21.15.82:0/598931958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7980102a00 0x7f7980109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.844+0000 7f7987f2b640 1 --2- 172.21.15.82:0/598931958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7980102040 0x7f79801024c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.844+0000 7f7987f2b640 1 --2- 172.21.15.82:0/598931958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7980100e60 0x7f7980101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.844+0000 7f7987f2b640 1 -- 172.21.15.82:0/598931958 >> 172.21.15.82:0/598931958 conn(0x7f79800fc440 msgr2=0x7f79800fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:22.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.844+0000 7f7987f2b640 1 -- 172.21.15.82:0/598931958 shutdown_connections 2024-04-21T08:46:22.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.844+0000 7f7987f2b640 1 -- 172.21.15.82:0/598931958 wait complete. 2024-04-21T08:46:22.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f7987f2b640 1 Processor -- start 2024-04-21T08:46:22.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f7987f2b640 1 -- start start 2024-04-21T08:46:22.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f7987f2b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7980100e60 0x7f798019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:22.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f7987f2b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7980102040 0x7f798019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:22.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f7987f2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7980102a00 0x7f79801a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:22.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f7987f2b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f798010db30 con 0x7f7980102a00 2024-04-21T08:46:22.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f7987f2b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f798010d9b0 con 0x7f7980100e60 2024-04-21T08:46:22.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f7987f2b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f798010dcb0 con 0x7f7980102040 2024-04-21T08:46:22.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f79864c8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7980102a00 0x7f79801a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:22.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f79864c8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7980102a00 0x7f79801a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36208/0 (socket says 172.21.15.82:36208) 2024-04-21T08:46:22.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f79864c8640 1 -- 172.21.15.82:0/1162731161 learned_addr learned my addr 172.21.15.82:0/1162731161 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:22.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f7985cc7640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7980100e60 0x7f798019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:22.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f79864c8640 1 -- 172.21.15.82:0/1162731161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7980102040 msgr2=0x7f798019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f79864c8640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7980102040 0x7f798019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f79864c8640 1 -- 172.21.15.82:0/1162731161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7980100e60 msgr2=0x7f798019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:22.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f79864c8640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7980100e60 0x7f798019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:22.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f79864c8640 1 -- 172.21.15.82:0/1162731161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79801a2080 con 0x7f7980102a00 2024-04-21T08:46:22.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.845+0000 7f7985cc7640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7980100e60 0x7f798019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:22.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.846+0000 7f79864c8640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7980102a00 0x7f79801a1a10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f797800eaa0 tx=0x7f797800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:22.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.846+0000 7f7976ffd640 1 -- 172.21.15.82:0/1162731161 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797800cdb0 con 0x7f7980102a00 2024-04-21T08:46:22.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.846+0000 7f7987f2b640 1 -- 172.21.15.82:0/1162731161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79801a2370 con 0x7f7980102a00 2024-04-21T08:46:22.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.846+0000 7f7976ffd640 1 -- 172.21.15.82:0/1162731161 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7978005020 con 0x7f7980102a00 2024-04-21T08:46:22.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.846+0000 7f7976ffd640 1 -- 172.21.15.82:0/1162731161 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79780116d0 con 0x7f7980102a00 2024-04-21T08:46:22.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.846+0000 7f7987f2b640 1 -- 172.21.15.82:0/1162731161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79801a2870 con 0x7f7980102a00 2024-04-21T08:46:22.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.847+0000 7f7987f2b640 1 -- 172.21.15.82:0/1162731161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79801a2b30 con 0x7f7980102a00 2024-04-21T08:46:22.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.848+0000 7f7976ffd640 1 -- 172.21.15.82:0/1162731161 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f79780051d0 con 0x7f7980102a00 2024-04-21T08:46:22.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.848+0000 7f7976ffd640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f796c0a3c40 0x7f796c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:22.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.848+0000 7f7985cc7640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f796c0a3c40 0x7f796c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:22.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.848+0000 7f7976ffd640 1 -- 172.21.15.82:0/1162731161 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f79780d0f70 con 0x7f7980102a00 2024-04-21T08:46:22.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.849+0000 7f7985cc7640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f796c0a3c40 0x7f796c0a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f797c0058c0 tx=0x7f797c005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:22.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:22.852+0000 7f7976ffd640 1 -- 172.21.15.82:0/1162731161 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7978016030 con 0x7f7980102a00 2024-04-21T08:46:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:22 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2284403195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:22 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1715442288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:46:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:22 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2651352762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:46:23.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.033+0000 7f7987f2b640 1 -- 172.21.15.82:0/1162731161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f79801a2d00 con 0x7f7980102a00 2024-04-21T08:46:23.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.033+0000 7f7976ffd640 1 -- 172.21.15.82:0/1162731161 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f797809d4b0 con 0x7f7980102a00 2024-04-21T08:46:23.033 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:46:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.035+0000 7f7987f2b640 1 -- 172.21.15.82:0/1162731161 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f796c0a3c40 msgr2=0x7f796c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.035+0000 7f7987f2b640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f796c0a3c40 0x7f796c0a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f797c0058c0 tx=0x7f797c005ec0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.035+0000 7f7987f2b640 1 -- 172.21.15.82:0/1162731161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7980102a00 msgr2=0x7f79801a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.035+0000 7f7987f2b640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7980102a00 0x7f79801a1a10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f797800eaa0 tx=0x7f797800ef60 comp rx=0 tx=0).stop 2024-04-21T08:46:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.035+0000 7f7987f2b640 1 -- 172.21.15.82:0/1162731161 shutdown_connections 2024-04-21T08:46:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.035+0000 7f7987f2b640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f796c0a3c40 0x7f796c0a6100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.035+0000 7f7987f2b640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7980102a00 0x7f79801a1a10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.035+0000 7f7987f2b640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7980102040 0x7f798019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.035+0000 7f7987f2b640 1 --2- 172.21.15.82:0/1162731161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7980100e60 0x7f798019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.035+0000 7f7987f2b640 1 -- 172.21.15.82:0/1162731161 >> 172.21.15.82:0/1162731161 conn(0x7f79800fc440 msgr2=0x7f7980109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:23.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.035+0000 7f7987f2b640 1 -- 172.21.15.82:0/1162731161 shutdown_connections 2024-04-21T08:46:23.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.035+0000 7f7987f2b640 1 -- 172.21.15.82:0/1162731161 wait complete. 2024-04-21T08:46:23.046 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:46:23.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.145+0000 7f99ad354640 1 -- 172.21.15.82:0/2257337235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8107c60 msgr2=0x7f99a810a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.145+0000 7f99ad354640 1 --2- 172.21.15.82:0/2257337235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8107c60 0x7f99a810a0b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f99a0009ca0 tx=0x7f99a002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.145+0000 7f99ad354640 1 -- 172.21.15.82:0/2257337235 shutdown_connections 2024-04-21T08:46:23.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.145+0000 7f99ad354640 1 --2- 172.21.15.82:0/2257337235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8107c60 0x7f99a810a0b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.145+0000 7f99ad354640 1 --2- 172.21.15.82:0/2257337235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a8105330 0x7f99a8107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.145+0000 7f99ad354640 1 --2- 172.21.15.82:0/2257337235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99a8102a00 0x7f99a8104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.145+0000 7f99ad354640 1 -- 172.21.15.82:0/2257337235 >> 172.21.15.82:0/2257337235 conn(0x7f99a80fc410 msgr2=0x7f99a80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.145+0000 7f99ad354640 1 -- 172.21.15.82:0/2257337235 shutdown_connections 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.145+0000 7f99ad354640 1 -- 172.21.15.82:0/2257337235 wait complete. 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.145+0000 7f99ad354640 1 Processor -- start 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.145+0000 7f99ad354640 1 -- start start 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99ad354640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8102a00 0x7f99a819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99ad354640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a8105330 0x7f99a819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99ad354640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99a8107c60 0x7f99a81a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99ad354640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f99a810dab0 con 0x7f99a8102a00 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99ad354640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f99a810d930 con 0x7f99a8105330 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99ad354640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f99a810dc30 con 0x7f99a8107c60 2024-04-21T08:46:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99a6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8102a00 0x7f99a819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99a6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8102a00 0x7f99a819a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36224/0 (socket says 172.21.15.82:36224) 2024-04-21T08:46:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99a6ffd640 1 -- 172.21.15.82:0/3051446715 learned_addr learned my addr 172.21.15.82:0/3051446715 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99a77fe640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99a8107c60 0x7f99a81a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99a67fc640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a8105330 0x7f99a819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99a6ffd640 1 -- 172.21.15.82:0/3051446715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99a8107c60 msgr2=0x7f99a81a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99a6ffd640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99a8107c60 0x7f99a81a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99a6ffd640 1 -- 172.21.15.82:0/3051446715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a8105330 msgr2=0x7f99a819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99a6ffd640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a8105330 0x7f99a819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99a6ffd640 1 -- 172.21.15.82:0/3051446715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a81a1ee0 con 0x7f99a8102a00 2024-04-21T08:46:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99a67fc640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a8105330 0x7f99a819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:23.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99a6ffd640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8102a00 0x7f99a819a2c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c920 tx=0x7f999c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:23.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f998ffff640 1 -- 172.21.15.82:0/3051446715 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c007c70 con 0x7f99a8102a00 2024-04-21T08:46:23.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99ad354640 1 -- 172.21.15.82:0/3051446715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a81a21d0 con 0x7f99a8102a00 2024-04-21T08:46:23.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f998ffff640 1 -- 172.21.15.82:0/3051446715 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f999c007e10 con 0x7f99a8102a00 2024-04-21T08:46:23.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f998ffff640 1 -- 172.21.15.82:0/3051446715 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c018820 con 0x7f99a8102a00 2024-04-21T08:46:23.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.146+0000 7f99ad354640 1 -- 172.21.15.82:0/3051446715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a81a2720 con 0x7f99a8102a00 2024-04-21T08:46:23.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.147+0000 7f99ad354640 1 -- 172.21.15.82:0/3051446715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a8109750 con 0x7f99a8102a00 2024-04-21T08:46:23.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.148+0000 7f998ffff640 1 -- 172.21.15.82:0/3051446715 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f999c00f450 con 0x7f99a8102a00 2024-04-21T08:46:23.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.149+0000 7f998ffff640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f99900a3c40 0x7f99900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:23.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.149+0000 7f99a67fc640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f99900a3c40 0x7f99900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:23.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.149+0000 7f998ffff640 1 -- 172.21.15.82:0/3051446715 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f999c0d0d30 con 0x7f99a8102a00 2024-04-21T08:46:23.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.149+0000 7f99a67fc640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f99900a3c40 0x7f99900a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f99a819b7d0 tx=0x7f9998005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:23.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.152+0000 7f998ffff640 1 -- 172.21.15.82:0/3051446715 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f999c014030 con 0x7f99a8102a00 2024-04-21T08:46:23.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.368+0000 7f99ad354640 1 -- 172.21.15.82:0/3051446715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99a802d870 con 0x7f99a8102a00 2024-04-21T08:46:23.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.369+0000 7f998ffff640 1 -- 172.21.15.82:0/3051446715 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f999c09d300 con 0x7f99a8102a00 2024-04-21T08:46:23.368 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:23.369 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:45:02.005557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24481,24509,24476,24409,24407]},"id":1}]} 2024-04-21T08:46:23.369 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-04-21T08:46:23.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.371+0000 7f99ad354640 1 -- 172.21.15.82:0/3051446715 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f99900a3c40 msgr2=0x7f99900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.371+0000 7f99ad354640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f99900a3c40 0x7f99900a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f99a819b7d0 tx=0x7f9998005e50 comp rx=0 tx=0).stop 2024-04-21T08:46:23.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.371+0000 7f99ad354640 1 -- 172.21.15.82:0/3051446715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8102a00 msgr2=0x7f99a819a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.371+0000 7f99ad354640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8102a00 0x7f99a819a2c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c920 tx=0x7f999c00cde0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.371+0000 7f99ad354640 1 -- 172.21.15.82:0/3051446715 shutdown_connections 2024-04-21T08:46:23.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.371+0000 7f99ad354640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f99900a3c40 0x7f99900a6100 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.371+0000 7f99ad354640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99a8107c60 0x7f99a81a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.371+0000 7f99ad354640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99a8105330 0x7f99a819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.371+0000 7f99ad354640 1 --2- 172.21.15.82:0/3051446715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8102a00 0x7f99a819a2c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.371+0000 7f99ad354640 1 -- 172.21.15.82:0/3051446715 >> 172.21.15.82:0/3051446715 conn(0x7f99a80fc410 msgr2=0x7f99a81020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:23.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.371+0000 7f99ad354640 1 -- 172.21.15.82:0/3051446715 shutdown_connections 2024-04-21T08:46:23.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.371+0000 7f99ad354640 1 -- 172.21.15.82:0/3051446715 wait complete. 2024-04-21T08:46:23.383 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:46:23.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.483+0000 7f2cb2932640 1 -- 172.21.15.82:0/1717529121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac101fc0 msgr2=0x7f2cac102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.483+0000 7f2cb2932640 1 --2- 172.21.15.82:0/1717529121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac101fc0 0x7f2cac102440 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009900 tx=0x7f2c9c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:23.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.483+0000 7f2cb2932640 1 -- 172.21.15.82:0/1717529121 shutdown_connections 2024-04-21T08:46:23.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.483+0000 7f2cb2932640 1 --2- 172.21.15.82:0/1717529121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac102980 0x7f2cac1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.483+0000 7f2cb2932640 1 --2- 172.21.15.82:0/1717529121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac101fc0 0x7f2cac102440 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.483+0000 7f2cb2932640 1 --2- 172.21.15.82:0/1717529121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cac100e70 0x7f2cac101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.483+0000 7f2cb2932640 1 -- 172.21.15.82:0/1717529121 >> 172.21.15.82:0/1717529121 conn(0x7f2cac0fc3e0 msgr2=0x7f2cac0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:23.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.483+0000 7f2cb2932640 1 -- 172.21.15.82:0/1717529121 shutdown_connections 2024-04-21T08:46:23.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.483+0000 7f2cb2932640 1 -- 172.21.15.82:0/1717529121 wait complete. 2024-04-21T08:46:23.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.483+0000 7f2cb2932640 1 Processor -- start 2024-04-21T08:46:23.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.483+0000 7f2cb2932640 1 -- start start 2024-04-21T08:46:23.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cb2932640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac100e70 0x7f2cac196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:23.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cb2932640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac101fc0 0x7f2cac197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:23.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cb2932640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cac102980 0x7f2cac19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:23.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cb2932640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10c9a0 con 0x7f2cac101fc0 2024-04-21T08:46:23.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cb2932640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10c820 con 0x7f2cac100e70 2024-04-21T08:46:23.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cb2932640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10cb20 con 0x7f2cac102980 2024-04-21T08:46:23.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac101fc0 0x7f2cac197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:23.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac101fc0 0x7f2cac197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36254/0 (socket says 172.21.15.82:36254) 2024-04-21T08:46:23.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cab7fe640 1 -- 172.21.15.82:0/3357204040 learned_addr learned my addr 172.21.15.82:0/3357204040 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:23.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cabfff640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac100e70 0x7f2cac196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:23.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cb0ecf640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cac102980 0x7f2cac19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:23.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cab7fe640 1 -- 172.21.15.82:0/3357204040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cac102980 msgr2=0x7f2cac19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cab7fe640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cac102980 0x7f2cac19e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cab7fe640 1 -- 172.21.15.82:0/3357204040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac100e70 msgr2=0x7f2cac196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cab7fe640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac100e70 0x7f2cac196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cab7fe640 1 -- 172.21.15.82:0/3357204040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac19ec00 con 0x7f2cac101fc0 2024-04-21T08:46:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cabfff640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac100e70 0x7f2cac196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:23.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2cab7fe640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac101fc0 0x7f2cac197520 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c036040 tx=0x7f2c9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:23.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.484+0000 7f2ca97fa640 1 -- 172.21.15.82:0/3357204040 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c043020 con 0x7f2cac101fc0 2024-04-21T08:46:23.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.485+0000 7f2ca97fa640 1 -- 172.21.15.82:0/3357204040 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c9c004aa0 con 0x7f2cac101fc0 2024-04-21T08:46:23.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.485+0000 7f2cb2932640 1 -- 172.21.15.82:0/3357204040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac19ee90 con 0x7f2cac101fc0 2024-04-21T08:46:23.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.485+0000 7f2ca97fa640 1 -- 172.21.15.82:0/3357204040 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c003de0 con 0x7f2cac101fc0 2024-04-21T08:46:23.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.485+0000 7f2cb2932640 1 -- 172.21.15.82:0/3357204040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac19f2f0 con 0x7f2cac101fc0 2024-04-21T08:46:23.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.486+0000 7f2cb2932640 1 -- 172.21.15.82:0/3357204040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac108120 con 0x7f2cac101fc0 2024-04-21T08:46:23.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.487+0000 7f2ca97fa640 1 -- 172.21.15.82:0/3357204040 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2c9c046020 con 0x7f2cac101fc0 2024-04-21T08:46:23.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.487+0000 7f2ca97fa640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2c940a3c40 0x7f2c940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:23.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.488+0000 7f2cabfff640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2c940a3c40 0x7f2c940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:23.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.488+0000 7f2ca97fa640 1 -- 172.21.15.82:0/3357204040 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f2c9c0f2c90 con 0x7f2cac101fc0 2024-04-21T08:46:23.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.488+0000 7f2cabfff640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2c940a3c40 0x7f2c940a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000abe0 tx=0x7f2ca000a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:23.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.492+0000 7f2ca97fa640 1 -- 172.21.15.82:0/3357204040 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c9c0f8020 con 0x7f2cac101fc0 2024-04-21T08:46:23.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.671+0000 7f2cb2932640 1 -- 172.21.15.82:0/3357204040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cac02d810 con 0x7f2cac101fc0 2024-04-21T08:46:23.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.672+0000 7f2ca97fa640 1 -- 172.21.15.82:0/3357204040 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24580 (secure 0 0 0) 0x7f2c9c04d020 con 0x7f2cac101fc0 2024-04-21T08:46:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:23.672 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:45:27.389965+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:46:23.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.674+0000 7f2cb2932640 1 -- 172.21.15.82:0/3357204040 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2c940a3c40 msgr2=0x7f2c940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.674+0000 7f2cb2932640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2c940a3c40 0x7f2c940a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000abe0 tx=0x7f2ca000a5c0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.674+0000 7f2cb2932640 1 -- 172.21.15.82:0/3357204040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac101fc0 msgr2=0x7f2cac197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.674+0000 7f2cb2932640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac101fc0 0x7f2cac197520 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c036040 tx=0x7f2c9c0042f0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.675+0000 7f2cb2932640 1 -- 172.21.15.82:0/3357204040 shutdown_connections 2024-04-21T08:46:23.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.675+0000 7f2cb2932640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2c940a3c40 0x7f2c940a6100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.675+0000 7f2cb2932640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cac102980 0x7f2cac19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.675+0000 7f2cb2932640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac101fc0 0x7f2cac197520 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.675+0000 7f2cb2932640 1 --2- 172.21.15.82:0/3357204040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cac100e70 0x7f2cac196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.675+0000 7f2cb2932640 1 -- 172.21.15.82:0/3357204040 >> 172.21.15.82:0/3357204040 conn(0x7f2cac0fc3e0 msgr2=0x7f2cac1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:23.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.675+0000 7f2cb2932640 1 -- 172.21.15.82:0/3357204040 shutdown_connections 2024-04-21T08:46:23.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.675+0000 7f2cb2932640 1 -- 172.21.15.82:0/3357204040 wait complete. 2024-04-21T08:46:23.684 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:23 smithi082 ceph-mon[27967]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:23.684 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:23 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1195766632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:46:23.685 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:23 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1162731161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:46:23.685 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:23 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3051446715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:23.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:46:23.687 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:46:23.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:23 smithi134 ceph-mon[31772]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:23.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:23 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1195766632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:46:23.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:23 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1162731161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:46:23.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:23 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3051446715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.789+0000 7f413b7a4640 1 -- 172.21.15.82:0/3773581637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4134100e80 msgr2=0x7f41341012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.789+0000 7f413b7a4640 1 --2- 172.21.15.82:0/3773581637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4134100e80 0x7f41341012a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4130009900 tx=0x7f413002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.789+0000 7f413b7a4640 1 -- 172.21.15.82:0/3773581637 shutdown_connections 2024-04-21T08:46:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.789+0000 7f413b7a4640 1 --2- 172.21.15.82:0/3773581637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4134102990 0x7f41341092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.789+0000 7f413b7a4640 1 --2- 172.21.15.82:0/3773581637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4134101fd0 0x7f4134102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.789+0000 7f413b7a4640 1 --2- 172.21.15.82:0/3773581637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4134100e80 0x7f41341012a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.789+0000 7f413b7a4640 1 -- 172.21.15.82:0/3773581637 >> 172.21.15.82:0/3773581637 conn(0x7f41340fc3f0 msgr2=0x7f41340fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.789+0000 7f413b7a4640 1 -- 172.21.15.82:0/3773581637 shutdown_connections 2024-04-21T08:46:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.789+0000 7f413b7a4640 1 -- 172.21.15.82:0/3773581637 wait complete. 2024-04-21T08:46:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.789+0000 7f413b7a4640 1 Processor -- start 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.789+0000 7f413b7a4640 1 -- start start 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f413b7a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4134100e80 0x7f4134198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f413b7a4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4134101fd0 0x7f4134198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f413b7a4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4134102990 0x7f413419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f413b7a4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f413410db20 con 0x7f4134101fd0 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f413b7a4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f413410d9a0 con 0x7f4134102990 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f413b7a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f413410dca0 con 0x7f4134100e80 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f4138d3f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4134101fd0 0x7f4134198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f4138d3f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4134101fd0 0x7f4134198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36274/0 (socket says 172.21.15.82:36274) 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f4138d3f640 1 -- 172.21.15.82:0/634537308 learned_addr learned my addr 172.21.15.82:0/634537308 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f4139540640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4134100e80 0x7f4134198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f4139d41640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4134102990 0x7f413419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f4138d3f640 1 -- 172.21.15.82:0/634537308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4134100e80 msgr2=0x7f4134198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f4138d3f640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4134100e80 0x7f4134198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f4138d3f640 1 -- 172.21.15.82:0/634537308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4134102990 msgr2=0x7f413419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f4138d3f640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4134102990 0x7f413419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f4138d3f640 1 -- 172.21.15.82:0/634537308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413419fe20 con 0x7f4134101fd0 2024-04-21T08:46:23.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f4138d3f640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4134101fd0 0x7f4134198740 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f412400d770 tx=0x7f412400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:23.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f412a7fc640 1 -- 172.21.15.82:0/634537308 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124017070 con 0x7f4134101fd0 2024-04-21T08:46:23.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f413b7a4640 1 -- 172.21.15.82:0/634537308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41341a0110 con 0x7f4134101fd0 2024-04-21T08:46:23.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f412a7fc640 1 -- 172.21.15.82:0/634537308 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4124009c30 con 0x7f4134101fd0 2024-04-21T08:46:23.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f412a7fc640 1 -- 172.21.15.82:0/634537308 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124003cc0 con 0x7f4134101fd0 2024-04-21T08:46:23.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.790+0000 7f413b7a4640 1 -- 172.21.15.82:0/634537308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41341a0660 con 0x7f4134101fd0 2024-04-21T08:46:23.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.791+0000 7f413b7a4640 1 -- 172.21.15.82:0/634537308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41341082b0 con 0x7f4134101fd0 2024-04-21T08:46:23.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.792+0000 7f412a7fc640 1 -- 172.21.15.82:0/634537308 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4124003e60 con 0x7f4134101fd0 2024-04-21T08:46:23.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.793+0000 7f412a7fc640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f41200a3c40 0x7f41200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:23.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.793+0000 7f4139540640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f41200a3c40 0x7f41200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:23.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.793+0000 7f412a7fc640 1 -- 172.21.15.82:0/634537308 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f41240d01b0 con 0x7f4134101fd0 2024-04-21T08:46:23.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.793+0000 7f4139540640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f41200a3c40 0x7f41200a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4130009230 tx=0x7f4130005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:23.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.797+0000 7f412a7fc640 1 -- 172.21.15.82:0/634537308 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4124009de0 con 0x7f4134101fd0 2024-04-21T08:46:23.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:23 smithi106 ceph-mon[31072]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:23.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:23 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1195766632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:46:23.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:23 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1162731161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:46:23.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:23 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3051446715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:23.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.976+0000 7f413b7a4640 1 -- 172.21.15.82:0/634537308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f413402d810 con 0x7f4134101fd0 2024-04-21T08:46:23.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.976+0000 7f412a7fc640 1 -- 172.21.15.82:0/634537308 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f41240bc070 con 0x7f4134101fd0 2024-04-21T08:46:23.976 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:46:23.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.978+0000 7f413b7a4640 1 -- 172.21.15.82:0/634537308 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f41200a3c40 msgr2=0x7f41200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.978+0000 7f413b7a4640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f41200a3c40 0x7f41200a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4130009230 tx=0x7f4130005fd0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.978+0000 7f413b7a4640 1 -- 172.21.15.82:0/634537308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4134101fd0 msgr2=0x7f4134198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:23.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.978+0000 7f413b7a4640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4134101fd0 0x7f4134198740 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f412400d770 tx=0x7f412400dc30 comp rx=0 tx=0).stop 2024-04-21T08:46:23.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.978+0000 7f413b7a4640 1 -- 172.21.15.82:0/634537308 shutdown_connections 2024-04-21T08:46:23.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.978+0000 7f413b7a4640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f41200a3c40 0x7f41200a6100 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.978+0000 7f413b7a4640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4134102990 0x7f413419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.978+0000 7f413b7a4640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4134101fd0 0x7f4134198740 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.978+0000 7f413b7a4640 1 --2- 172.21.15.82:0/634537308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4134100e80 0x7f4134198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:23.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.978+0000 7f413b7a4640 1 -- 172.21.15.82:0/634537308 >> 172.21.15.82:0/634537308 conn(0x7f41340fc3f0 msgr2=0x7f4134105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:23.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.978+0000 7f413b7a4640 1 -- 172.21.15.82:0/634537308 shutdown_connections 2024-04-21T08:46:23.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:23.978+0000 7f413b7a4640 1 -- 172.21.15.82:0/634537308 wait complete. 2024-04-21T08:46:23.989 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:46:24.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.090+0000 7f13867ad640 1 -- 172.21.15.82:0/3258273596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1380107c90 msgr2=0x7f138010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.090+0000 7f13867ad640 1 --2- 172.21.15.82:0/3258273596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1380107c90 0x7f138010a0e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1378009ca0 tx=0x7f137802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.090+0000 7f13867ad640 1 -- 172.21.15.82:0/3258273596 shutdown_connections 2024-04-21T08:46:24.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.090+0000 7f13867ad640 1 --2- 172.21.15.82:0/3258273596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1380107c90 0x7f138010a0e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.090+0000 7f13867ad640 1 --2- 172.21.15.82:0/3258273596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1380105360 0x7f1380107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.090+0000 7f13867ad640 1 --2- 172.21.15.82:0/3258273596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1380102a30 0x7f1380104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.090+0000 7f13867ad640 1 -- 172.21.15.82:0/3258273596 >> 172.21.15.82:0/3258273596 conn(0x7f13800fc420 msgr2=0x7f13800fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:24.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.090+0000 7f13867ad640 1 -- 172.21.15.82:0/3258273596 shutdown_connections 2024-04-21T08:46:24.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.090+0000 7f13867ad640 1 -- 172.21.15.82:0/3258273596 wait complete. 2024-04-21T08:46:24.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f13867ad640 1 Processor -- start 2024-04-21T08:46:24.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f13867ad640 1 -- start start 2024-04-21T08:46:24.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f13867ad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1380102a30 0x7f13801981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f13867ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1380105360 0x7f1380198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f13867ad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1380107c90 0x7f138019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f13867ad640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f138010daf0 con 0x7f1380102a30 2024-04-21T08:46:24.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f13867ad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f138010d970 con 0x7f1380105360 2024-04-21T08:46:24.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f13867ad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f138010dc70 con 0x7f1380107c90 2024-04-21T08:46:24.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f137ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1380102a30 0x7f13801981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f137ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1380102a30 0x7f13801981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36302/0 (socket says 172.21.15.82:36302) 2024-04-21T08:46:24.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f137ffff640 1 -- 172.21.15.82:0/34171348 learned_addr learned my addr 172.21.15.82:0/34171348 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:24.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f1384d4a640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1380107c90 0x7f138019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f137ffff640 1 -- 172.21.15.82:0/34171348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1380107c90 msgr2=0x7f138019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f137ffff640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1380107c90 0x7f138019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f137ffff640 1 -- 172.21.15.82:0/34171348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1380105360 msgr2=0x7f1380198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f137ffff640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1380105360 0x7f1380198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.091+0000 7f137ffff640 1 -- 172.21.15.82:0/34171348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138019fde0 con 0x7f1380102a30 2024-04-21T08:46:24.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.092+0000 7f137ffff640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1380102a30 0x7f13801981c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f137400e9f0 tx=0x7f137400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:24.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.092+0000 7f137d7fa640 1 -- 172.21.15.82:0/34171348 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137400cd80 con 0x7f1380102a30 2024-04-21T08:46:24.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.092+0000 7f13867ad640 1 -- 172.21.15.82:0/34171348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13801a00d0 con 0x7f1380102a30 2024-04-21T08:46:24.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.092+0000 7f137d7fa640 1 -- 172.21.15.82:0/34171348 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13740054d0 con 0x7f1380102a30 2024-04-21T08:46:24.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.092+0000 7f137d7fa640 1 -- 172.21.15.82:0/34171348 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374010690 con 0x7f1380102a30 2024-04-21T08:46:24.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.092+0000 7f13867ad640 1 -- 172.21.15.82:0/34171348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13801a05d0 con 0x7f1380102a30 2024-04-21T08:46:24.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.093+0000 7f13867ad640 1 -- 172.21.15.82:0/34171348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13800fdf10 con 0x7f1380102a30 2024-04-21T08:46:24.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.094+0000 7f137d7fa640 1 -- 172.21.15.82:0/34171348 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1374004e90 con 0x7f1380102a30 2024-04-21T08:46:24.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.095+0000 7f137d7fa640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13680a3c40 0x7f13680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.095+0000 7f137f7fe640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13680a3c40 0x7f13680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.095+0000 7f137d7fa640 1 -- 172.21.15.82:0/34171348 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f13740d0e20 con 0x7f1380102a30 2024-04-21T08:46:24.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.095+0000 7f137f7fe640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13680a3c40 0x7f13680a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f13801996d0 tx=0x7f1370005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:24.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.099+0000 7f137d7fa640 1 -- 172.21.15.82:0/34171348 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f137409d360 con 0x7f1380102a30 2024-04-21T08:46:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.275+0000 7f13867ad640 1 -- 172.21.15.82:0/34171348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f138002d830 con 0x7f1380102a30 2024-04-21T08:46:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.276+0000 7f137d7fa640 1 -- 172.21.15.82:0/34171348 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f13740bbe20 con 0x7f1380102a30 2024-04-21T08:46:24.276 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:46:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.278+0000 7f13867ad640 1 -- 172.21.15.82:0/34171348 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13680a3c40 msgr2=0x7f13680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.278+0000 7f13867ad640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13680a3c40 0x7f13680a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f13801996d0 tx=0x7f1370005bf0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.278+0000 7f13867ad640 1 -- 172.21.15.82:0/34171348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1380102a30 msgr2=0x7f13801981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.278+0000 7f13867ad640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1380102a30 0x7f13801981c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f137400e9f0 tx=0x7f137400edf0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.278+0000 7f13867ad640 1 -- 172.21.15.82:0/34171348 shutdown_connections 2024-04-21T08:46:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.278+0000 7f13867ad640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f13680a3c40 0x7f13680a6100 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.278+0000 7f13867ad640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1380107c90 0x7f138019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.278+0000 7f13867ad640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1380105360 0x7f1380198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.278+0000 7f13867ad640 1 --2- 172.21.15.82:0/34171348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1380102a30 0x7f13801981c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.278+0000 7f13867ad640 1 -- 172.21.15.82:0/34171348 >> 172.21.15.82:0/34171348 conn(0x7f13800fc420 msgr2=0x7f1380103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.278+0000 7f13867ad640 1 -- 172.21.15.82:0/34171348 shutdown_connections 2024-04-21T08:46:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.278+0000 7f13867ad640 1 -- 172.21.15.82:0/34171348 wait complete. 2024-04-21T08:46:24.289 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:46:24.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.391+0000 7f0618cba640 1 -- 172.21.15.82:0/1907837419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0614102a00 msgr2=0x7f0614109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.391+0000 7f0618cba640 1 --2- 172.21.15.82:0/1907837419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0614102a00 0x7f0614109320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f060c009ca0 tx=0x7f060c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.391+0000 7f0618cba640 1 -- 172.21.15.82:0/1907837419 shutdown_connections 2024-04-21T08:46:24.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.391+0000 7f0618cba640 1 --2- 172.21.15.82:0/1907837419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0614102a00 0x7f0614109320 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.391+0000 7f0618cba640 1 --2- 172.21.15.82:0/1907837419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0614102040 0x7f06141024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.391+0000 7f0618cba640 1 --2- 172.21.15.82:0/1907837419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0614100e60 0x7f0614101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.391+0000 7f0618cba640 1 -- 172.21.15.82:0/1907837419 >> 172.21.15.82:0/1907837419 conn(0x7f06140fc420 msgr2=0x7f06140fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:24.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.391+0000 7f0618cba640 1 -- 172.21.15.82:0/1907837419 shutdown_connections 2024-04-21T08:46:24.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.391+0000 7f0618cba640 1 -- 172.21.15.82:0/1907837419 wait complete. 2024-04-21T08:46:24.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f0618cba640 1 Processor -- start 2024-04-21T08:46:24.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f0618cba640 1 -- start start 2024-04-21T08:46:24.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f0618cba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0614100e60 0x7f0614198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f0618cba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0614102040 0x7f0614198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f0618cba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0614102a00 0x7f061419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f0618cba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f061410ca00 con 0x7f0614100e60 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f0618cba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f061410c880 con 0x7f0614102a00 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f0618cba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f061410cb80 con 0x7f0614102040 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f061259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0614100e60 0x7f0614198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f061259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0614100e60 0x7f0614198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36310/0 (socket says 172.21.15.82:36310) 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f061259c640 1 -- 172.21.15.82:0/312681513 learned_addr learned my addr 172.21.15.82:0/312681513 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f060bfff640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0614102040 0x7f0614198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f0612d9d640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0614102a00 0x7f061419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f061259c640 1 -- 172.21.15.82:0/312681513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0614102040 msgr2=0x7f0614198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f061259c640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0614102040 0x7f0614198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f061259c640 1 -- 172.21.15.82:0/312681513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0614102a00 msgr2=0x7f061419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f061259c640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0614102a00 0x7f061419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f061259c640 1 -- 172.21.15.82:0/312681513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061419fdb0 con 0x7f0614100e60 2024-04-21T08:46:24.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f0612d9d640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0614102a00 0x7f061419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:24.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.392+0000 7f060bfff640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0614102040 0x7f0614198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:24.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.393+0000 7f061259c640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0614100e60 0x7f0614198200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f060400c970 tx=0x7f060400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:24.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.393+0000 7f060b7fe640 1 -- 172.21.15.82:0/312681513 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604004220 con 0x7f0614100e60 2024-04-21T08:46:24.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.393+0000 7f060b7fe640 1 -- 172.21.15.82:0/312681513 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06040043c0 con 0x7f0614100e60 2024-04-21T08:46:24.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.393+0000 7f0618cba640 1 -- 172.21.15.82:0/312681513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06141a00a0 con 0x7f0614100e60 2024-04-21T08:46:24.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.393+0000 7f060b7fe640 1 -- 172.21.15.82:0/312681513 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060400f450 con 0x7f0614100e60 2024-04-21T08:46:24.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.393+0000 7f0618cba640 1 -- 172.21.15.82:0/312681513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06141a05f0 con 0x7f0614100e60 2024-04-21T08:46:24.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.394+0000 7f0618cba640 1 -- 172.21.15.82:0/312681513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06141a08b0 con 0x7f0614100e60 2024-04-21T08:46:24.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.395+0000 7f060b7fe640 1 -- 172.21.15.82:0/312681513 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0604003610 con 0x7f0614100e60 2024-04-21T08:46:24.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.395+0000 7f060b7fe640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f06000a3c40 0x7f06000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.396+0000 7f060b7fe640 1 -- 172.21.15.82:0/312681513 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f06040d0b50 con 0x7f0614100e60 2024-04-21T08:46:24.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.396+0000 7f060bfff640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f06000a3c40 0x7f06000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.396+0000 7f060bfff640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f06000a3c40 0x7f06000a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0058c0 tx=0x7f05fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:24.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.399+0000 7f060b7fe640 1 -- 172.21.15.82:0/312681513 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0604014030 con 0x7f0614100e60 2024-04-21T08:46:24.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.578+0000 7f0618cba640 1 -- 172.21.15.82:0/312681513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f06141a0a80 con 0x7f0614100e60 2024-04-21T08:46:24.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.578+0000 7f060b7fe640 1 -- 172.21.15.82:0/312681513 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f060409d090 con 0x7f0614100e60 2024-04-21T08:46:24.578 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:46:24.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.580+0000 7f0618cba640 1 -- 172.21.15.82:0/312681513 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f06000a3c40 msgr2=0x7f06000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.580+0000 7f0618cba640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f06000a3c40 0x7f06000a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0058c0 tx=0x7f05fc005850 comp rx=0 tx=0).stop 2024-04-21T08:46:24.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.580+0000 7f0618cba640 1 -- 172.21.15.82:0/312681513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0614100e60 msgr2=0x7f0614198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.580+0000 7f0618cba640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0614100e60 0x7f0614198200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f060400c970 tx=0x7f060400ce30 comp rx=0 tx=0).stop 2024-04-21T08:46:24.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.580+0000 7f0618cba640 1 -- 172.21.15.82:0/312681513 shutdown_connections 2024-04-21T08:46:24.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.580+0000 7f0618cba640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f06000a3c40 0x7f06000a6100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.580+0000 7f0618cba640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0614102a00 0x7f061419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.580+0000 7f0618cba640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0614102040 0x7f0614198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.580+0000 7f0618cba640 1 --2- 172.21.15.82:0/312681513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0614100e60 0x7f0614198200 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.580+0000 7f0618cba640 1 -- 172.21.15.82:0/312681513 >> 172.21.15.82:0/312681513 conn(0x7f06140fc420 msgr2=0x7f06140fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:24.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.580+0000 7f0618cba640 1 -- 172.21.15.82:0/312681513 shutdown_connections 2024-04-21T08:46:24.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.580+0000 7f0618cba640 1 -- 172.21.15.82:0/312681513 wait complete. 2024-04-21T08:46:24.590 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:24 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3357204040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:24.590 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:24 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/634537308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:46:24.590 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:24 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/34171348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:46:24.591 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:46:24.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.692+0000 7f342fa5c640 1 -- 172.21.15.82:0/1178680566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3428103190 msgr2=0x7f3428103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.692+0000 7f342fa5c640 1 --2- 172.21.15.82:0/1178680566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3428103190 0x7f3428103610 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3418009900 tx=0x7f341802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:24.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.692+0000 7f342fa5c640 1 -- 172.21.15.82:0/1178680566 shutdown_connections 2024-04-21T08:46:24.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.692+0000 7f342fa5c640 1 --2- 172.21.15.82:0/1178680566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3428103b50 0x7f342810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.692+0000 7f342fa5c640 1 --2- 172.21.15.82:0/1178680566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3428103190 0x7f3428103610 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.692+0000 7f342fa5c640 1 --2- 172.21.15.82:0/1178680566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3428101fa0 0x7f34281023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.692+0000 7f342fa5c640 1 -- 172.21.15.82:0/1178680566 >> 172.21.15.82:0/1178680566 conn(0x7f34280fd530 msgr2=0x7f34280ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.692+0000 7f342fa5c640 1 -- 172.21.15.82:0/1178680566 shutdown_connections 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.692+0000 7f342fa5c640 1 -- 172.21.15.82:0/1178680566 wait complete. 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342fa5c640 1 Processor -- start 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342fa5c640 1 -- start start 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342fa5c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3428101fa0 0x7f3428198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342fa5c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3428103190 0x7f3428198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342fa5c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3428103b50 0x7f342819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342fa5c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f342810dae0 con 0x7f3428101fa0 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342fa5c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f342810d960 con 0x7f3428103b50 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342fa5c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f342810dc60 con 0x7f3428103190 2024-04-21T08:46:24.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342d7f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3428101fa0 0x7f3428198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342d7f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3428101fa0 0x7f3428198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36338/0 (socket says 172.21.15.82:36338) 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342d7f8640 1 -- 172.21.15.82:0/2323252619 learned_addr learned my addr 172.21.15.82:0/2323252619 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342d7f8640 1 -- 172.21.15.82:0/2323252619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3428103190 msgr2=0x7f3428198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342d7f8640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3428103190 0x7f3428198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342d7f8640 1 -- 172.21.15.82:0/2323252619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3428103b50 msgr2=0x7f342819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342dff9640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3428103b50 0x7f342819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342d7f8640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3428103b50 0x7f342819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342d7f8640 1 -- 172.21.15.82:0/2323252619 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342819fd60 con 0x7f3428101fa0 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342dff9640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3428103b50 0x7f342819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342d7f8640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3428101fa0 0x7f3428198140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f342400c970 tx=0x7f342400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f341e7fc640 1 -- 172.21.15.82:0/2323252619 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424007c10 con 0x7f3428101fa0 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f342fa5c640 1 -- 172.21.15.82:0/2323252619 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342819fff0 con 0x7f3428101fa0 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f341e7fc640 1 -- 172.21.15.82:0/2323252619 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3424007db0 con 0x7f3428101fa0 2024-04-21T08:46:24.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.693+0000 7f341e7fc640 1 -- 172.21.15.82:0/2323252619 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342400f7d0 con 0x7f3428101fa0 2024-04-21T08:46:24.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.694+0000 7f342fa5c640 1 -- 172.21.15.82:0/2323252619 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34281a0510 con 0x7f3428101fa0 2024-04-21T08:46:24.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.694+0000 7f342fa5c640 1 -- 172.21.15.82:0/2323252619 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34281092c0 con 0x7f3428101fa0 2024-04-21T08:46:24.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.696+0000 7f341e7fc640 1 -- 172.21.15.82:0/2323252619 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3424018480 con 0x7f3428101fa0 2024-04-21T08:46:24.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.696+0000 7f341e7fc640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f34140a3c40 0x7f34140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.696+0000 7f342cff7640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f34140a3c40 0x7f34140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.696+0000 7f341e7fc640 1 -- 172.21.15.82:0/2323252619 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f34240d0cb0 con 0x7f3428101fa0 2024-04-21T08:46:24.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.697+0000 7f342cff7640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f34140a3c40 0x7f34140a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3418036040 tx=0x7f3418000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:24.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.701+0000 7f341e7fc640 1 -- 172.21.15.82:0/2323252619 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f342409d280 con 0x7f3428101fa0 2024-04-21T08:46:24.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:24 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3357204040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:24.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:24 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/634537308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:46:24.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:24 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/34171348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:46:24.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.877+0000 7f342fa5c640 1 -- 172.21.15.82:0/2323252619 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f342802d830 con 0x7f3428101fa0 2024-04-21T08:46:24.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.878+0000 7f341e7fc640 1 -- 172.21.15.82:0/2323252619 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f342409cb80 con 0x7f3428101fa0 2024-04-21T08:46:24.878 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:46:24.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:24 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3357204040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:24.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:24 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/634537308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:46:24.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:24 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/34171348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:46:24.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.880+0000 7f342fa5c640 1 -- 172.21.15.82:0/2323252619 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f34140a3c40 msgr2=0x7f34140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.880+0000 7f342fa5c640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f34140a3c40 0x7f34140a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3418036040 tx=0x7f3418000bc0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.880+0000 7f342fa5c640 1 -- 172.21.15.82:0/2323252619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3428101fa0 msgr2=0x7f3428198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.880+0000 7f342fa5c640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3428101fa0 0x7f3428198140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f342400c970 tx=0x7f342400ce30 comp rx=0 tx=0).stop 2024-04-21T08:46:24.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.880+0000 7f342fa5c640 1 -- 172.21.15.82:0/2323252619 shutdown_connections 2024-04-21T08:46:24.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.880+0000 7f342fa5c640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f34140a3c40 0x7f34140a6100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.880+0000 7f342fa5c640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3428103b50 0x7f342819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.880+0000 7f342fa5c640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3428103190 0x7f3428198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.880+0000 7f342fa5c640 1 --2- 172.21.15.82:0/2323252619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3428101fa0 0x7f3428198140 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.880+0000 7f342fa5c640 1 -- 172.21.15.82:0/2323252619 >> 172.21.15.82:0/2323252619 conn(0x7f34280fd530 msgr2=0x7f3428106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:24.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.880+0000 7f342fa5c640 1 -- 172.21.15.82:0/2323252619 shutdown_connections 2024-04-21T08:46:24.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.880+0000 7f342fa5c640 1 -- 172.21.15.82:0/2323252619 wait complete. 2024-04-21T08:46:24.895 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:46:24.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.993+0000 7fa65924e640 1 -- 172.21.15.82:0/3723308726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa654103b50 msgr2=0x7fa65410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.993+0000 7fa65924e640 1 --2- 172.21.15.82:0/3723308726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa654103b50 0x7fa65410a3e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009900 tx=0x7fa64c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:24.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.993+0000 7fa65924e640 1 -- 172.21.15.82:0/3723308726 shutdown_connections 2024-04-21T08:46:24.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.993+0000 7fa65924e640 1 --2- 172.21.15.82:0/3723308726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa654103b50 0x7fa65410a3e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.993+0000 7fa65924e640 1 --2- 172.21.15.82:0/3723308726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa654103190 0x7fa654103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.993+0000 7fa65924e640 1 --2- 172.21.15.82:0/3723308726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa654101fa0 0x7fa6541023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.993+0000 7fa65924e640 1 -- 172.21.15.82:0/3723308726 >> 172.21.15.82:0/3723308726 conn(0x7fa6540fd530 msgr2=0x7fa6540ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:24.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.993+0000 7fa65924e640 1 -- 172.21.15.82:0/3723308726 shutdown_connections 2024-04-21T08:46:24.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.993+0000 7fa65924e640 1 -- 172.21.15.82:0/3723308726 wait complete. 2024-04-21T08:46:24.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.994+0000 7fa65924e640 1 Processor -- start 2024-04-21T08:46:24.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.994+0000 7fa65924e640 1 -- start start 2024-04-21T08:46:24.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.994+0000 7fa65924e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa654101fa0 0x7fa6541982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.994+0000 7fa65924e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa654103190 0x7fa654198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.994+0000 7fa65924e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa654103b50 0x7fa65419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.994+0000 7fa65924e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa65410dbc0 con 0x7fa654103b50 2024-04-21T08:46:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.994+0000 7fa65924e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa65410da40 con 0x7fa654101fa0 2024-04-21T08:46:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.994+0000 7fa65924e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa65410dd40 con 0x7fa654103190 2024-04-21T08:46:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.994+0000 7fa65359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa654103b50 0x7fa65419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.994+0000 7fa65359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa654103b50 0x7fa65419f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36344/0 (socket says 172.21.15.82:36344) 2024-04-21T08:46:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.994+0000 7fa65359e640 1 -- 172.21.15.82:0/1408686877 learned_addr learned my addr 172.21.15.82:0/1408686877 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.994+0000 7fa65259c640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa654103190 0x7fa654198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa65359e640 1 -- 172.21.15.82:0/1408686877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa654103190 msgr2=0x7fa654198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa65359e640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa654103190 0x7fa654198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa652d9d640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa654101fa0 0x7fa6541982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa65359e640 1 -- 172.21.15.82:0/1408686877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa654101fa0 msgr2=0x7fa6541982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa65359e640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa654101fa0 0x7fa6541982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa65359e640 1 -- 172.21.15.82:0/1408686877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65419fee0 con 0x7fa654103b50 2024-04-21T08:46:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa65259c640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa654103190 0x7fa654198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa652d9d640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa654101fa0 0x7fa6541982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa65359e640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa654103b50 0x7fa65419f870 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009230 tx=0x7fa64c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa63bfff640 1 -- 172.21.15.82:0/1408686877 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c043020 con 0x7fa654103b50 2024-04-21T08:46:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa65924e640 1 -- 172.21.15.82:0/1408686877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6541a0170 con 0x7fa654103b50 2024-04-21T08:46:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa63bfff640 1 -- 172.21.15.82:0/1408686877 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa64c0054c0 con 0x7fa654103b50 2024-04-21T08:46:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa63bfff640 1 -- 172.21.15.82:0/1408686877 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c003db0 con 0x7fa654103b50 2024-04-21T08:46:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.995+0000 7fa65924e640 1 -- 172.21.15.82:0/1408686877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6541a0790 con 0x7fa654103b50 2024-04-21T08:46:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.996+0000 7fa65924e640 1 -- 172.21.15.82:0/1408686877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa654109330 con 0x7fa654103b50 2024-04-21T08:46:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.998+0000 7fa63bfff640 1 -- 172.21.15.82:0/1408686877 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa64c046020 con 0x7fa654103b50 2024-04-21T08:46:25.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.998+0000 7fa63bfff640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa63c0a3c40 0x7fa63c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.998+0000 7fa652d9d640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa63c0a3c40 0x7fa63c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.998+0000 7fa63bfff640 1 -- 172.21.15.82:0/1408686877 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fa64c0f3160 con 0x7fa654103b50 2024-04-21T08:46:25.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:24.999+0000 7fa652d9d640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa63c0a3c40 0x7fa63c0a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa654102ff0 tx=0x7fa6440073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.002+0000 7fa63bfff640 1 -- 172.21.15.82:0/1408686877 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa64c04d020 con 0x7fa654103b50 2024-04-21T08:46:25.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.223+0000 7fa65924e640 1 -- 172.21.15.82:0/1408686877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa65402d5c0 con 0x7fa654103b50 2024-04-21T08:46:25.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.223+0000 7fa63bfff640 1 -- 172.21.15.82:0/1408686877 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7fa64c0bf7b0 con 0x7fa654103b50 2024-04-21T08:46:25.223 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:25.223 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:45:02.005557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24481,24509,24476,24409,24407]},"id":1}]} 2024-04-21T08:46:25.223 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-04-21T08:46:25.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.226+0000 7fa65924e640 1 -- 172.21.15.82:0/1408686877 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa63c0a3c40 msgr2=0x7fa63c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.226+0000 7fa65924e640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa63c0a3c40 0x7fa63c0a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa654102ff0 tx=0x7fa6440073c0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.226+0000 7fa65924e640 1 -- 172.21.15.82:0/1408686877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa654103b50 msgr2=0x7fa65419f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.226+0000 7fa65924e640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa654103b50 0x7fa65419f870 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009230 tx=0x7fa64c0042f0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.226+0000 7fa65924e640 1 -- 172.21.15.82:0/1408686877 shutdown_connections 2024-04-21T08:46:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.226+0000 7fa65924e640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fa63c0a3c40 0x7fa63c0a6100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.226+0000 7fa65924e640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa654103b50 0x7fa65419f870 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.226+0000 7fa65924e640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa654103190 0x7fa654198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.226+0000 7fa65924e640 1 --2- 172.21.15.82:0/1408686877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa654101fa0 0x7fa6541982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.226+0000 7fa65924e640 1 -- 172.21.15.82:0/1408686877 >> 172.21.15.82:0/1408686877 conn(0x7fa6540fd530 msgr2=0x7fa654106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.226+0000 7fa65924e640 1 -- 172.21.15.82:0/1408686877 shutdown_connections 2024-04-21T08:46:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.226+0000 7fa65924e640 1 -- 172.21.15.82:0/1408686877 wait complete. 2024-04-21T08:46:25.237 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:46:25.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.334+0000 7f71d4c55640 1 -- 172.21.15.82:0/2416694715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d0105320 msgr2=0x7f71d0107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.334+0000 7f71d4c55640 1 --2- 172.21.15.82:0/2416694715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d0105320 0x7f71d0107710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009ca0 tx=0x7f71c402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 -- 172.21.15.82:0/2416694715 shutdown_connections 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 --2- 172.21.15.82:0/2416694715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71d0107c50 0x7f71d010a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 --2- 172.21.15.82:0/2416694715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d0105320 0x7f71d0107710 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 --2- 172.21.15.82:0/2416694715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71d01029f0 0x7f71d0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 -- 172.21.15.82:0/2416694715 >> 172.21.15.82:0/2416694715 conn(0x7f71d00fc3e0 msgr2=0x7f71d00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 -- 172.21.15.82:0/2416694715 shutdown_connections 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 -- 172.21.15.82:0/2416694715 wait complete. 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 Processor -- start 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 -- start start 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71d01029f0 0x7f71d0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d0105320 0x7f71d0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71d0107c50 0x7f71d019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f71d010da80 con 0x7f71d0105320 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f71d010d900 con 0x7f71d01029f0 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.335+0000 7f71d4c55640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71d010dc00 con 0x7f71d0107c50 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71cdd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d0105320 0x7f71d0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71cdd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d0105320 0x7f71d0198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36360/0 (socket says 172.21.15.82:36360) 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71cdd9b640 1 -- 172.21.15.82:0/1615310810 learned_addr learned my addr 172.21.15.82:0/1615310810 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71ce59c640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71d01029f0 0x7f71d0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71ced9d640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71d0107c50 0x7f71d019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71cdd9b640 1 -- 172.21.15.82:0/1615310810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71d0107c50 msgr2=0x7f71d019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71cdd9b640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71d0107c50 0x7f71d019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71cdd9b640 1 -- 172.21.15.82:0/1615310810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71d01029f0 msgr2=0x7f71d0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71cdd9b640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71d01029f0 0x7f71d0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71cdd9b640 1 -- 172.21.15.82:0/1615310810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d019fdd0 con 0x7f71d0105320 2024-04-21T08:46:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71ced9d640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71d0107c50 0x7f71d019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:25.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71cdd9b640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d0105320 0x7f71d0198740 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f71c4032380 tx=0x7f71c40347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71bf7fe640 1 -- 172.21.15.82:0/1615310810 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4039040 con 0x7f71d0105320 2024-04-21T08:46:25.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71d4c55640 1 -- 172.21.15.82:0/1615310810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d01a0060 con 0x7f71d0105320 2024-04-21T08:46:25.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71bf7fe640 1 -- 172.21.15.82:0/1615310810 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4044070 con 0x7f71d0105320 2024-04-21T08:46:25.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71bf7fe640 1 -- 172.21.15.82:0/1615310810 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c40042a0 con 0x7f71d0105320 2024-04-21T08:46:25.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.336+0000 7f71d4c55640 1 -- 172.21.15.82:0/1615310810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d01a05a0 con 0x7f71d0105320 2024-04-21T08:46:25.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.337+0000 7f71d4c55640 1 -- 172.21.15.82:0/1615310810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d00fded0 con 0x7f71d0105320 2024-04-21T08:46:25.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.339+0000 7f71bf7fe640 1 -- 172.21.15.82:0/1615310810 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f71c4003a00 con 0x7f71d0105320 2024-04-21T08:46:25.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.339+0000 7f71bf7fe640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f71b80a3c40 0x7f71b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.340+0000 7f71ce59c640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f71b80a3c40 0x7f71b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.340+0000 7f71bf7fe640 1 -- 172.21.15.82:0/1615310810 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f71c40f3660 con 0x7f71d0105320 2024-04-21T08:46:25.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.340+0000 7f71ce59c640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f71b80a3c40 0x7f71b80a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f71c800bbe0 tx=0x7f71c800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.343+0000 7f71bf7fe640 1 -- 172.21.15.82:0/1615310810 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71c40bfcb0 con 0x7f71d0105320 2024-04-21T08:46:25.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.529+0000 7f71d4c55640 1 -- 172.21.15.82:0/1615310810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71d002d7c0 con 0x7f71d0105320 2024-04-21T08:46:25.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.530+0000 7f71bf7fe640 1 -- 172.21.15.82:0/1615310810 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24580 (secure 0 0 0) 0x7f71c40bf5b0 con 0x7f71d0105320 2024-04-21T08:46:25.530 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:25.530 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:45:27.389965+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:46:25.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.533+0000 7f71d4c55640 1 -- 172.21.15.82:0/1615310810 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f71b80a3c40 msgr2=0x7f71b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.533+0000 7f71d4c55640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f71b80a3c40 0x7f71b80a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f71c800bbe0 tx=0x7f71c800b390 comp rx=0 tx=0).stop 2024-04-21T08:46:25.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.533+0000 7f71d4c55640 1 -- 172.21.15.82:0/1615310810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d0105320 msgr2=0x7f71d0198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.533+0000 7f71d4c55640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d0105320 0x7f71d0198740 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f71c4032380 tx=0x7f71c40347b0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.533+0000 7f71d4c55640 1 -- 172.21.15.82:0/1615310810 shutdown_connections 2024-04-21T08:46:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.533+0000 7f71d4c55640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f71b80a3c40 0x7f71b80a6100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.533+0000 7f71d4c55640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71d0107c50 0x7f71d019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.533+0000 7f71d4c55640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71d0105320 0x7f71d0198740 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.533+0000 7f71d4c55640 1 --2- 172.21.15.82:0/1615310810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71d01029f0 0x7f71d0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.533+0000 7f71d4c55640 1 -- 172.21.15.82:0/1615310810 >> 172.21.15.82:0/1615310810 conn(0x7f71d00fc3e0 msgr2=0x7f71d0103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.533+0000 7f71d4c55640 1 -- 172.21.15.82:0/1615310810 shutdown_connections 2024-04-21T08:46:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.533+0000 7f71d4c55640 1 -- 172.21.15.82:0/1615310810 wait complete. 2024-04-21T08:46:25.543 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:25 smithi082 ceph-mon[27967]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:46:25.543 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:25 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/312681513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:46:25.543 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:25 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2323252619' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:46:25.543 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:25 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1408686877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:25.546 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:46:25.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.644+0000 7f2180f15640 1 -- 172.21.15.82:0/1028060352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217c102050 msgr2=0x7f217c1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.644+0000 7f2180f15640 1 --2- 172.21.15.82:0/1028060352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217c102050 0x7f217c1024d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2164009900 tx=0x7f216402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:25.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.644+0000 7f2180f15640 1 -- 172.21.15.82:0/1028060352 shutdown_connections 2024-04-21T08:46:25.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.644+0000 7f2180f15640 1 --2- 172.21.15.82:0/1028060352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217c102a10 0x7f217c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.644+0000 7f2180f15640 1 --2- 172.21.15.82:0/1028060352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217c102050 0x7f217c1024d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.644+0000 7f2180f15640 1 --2- 172.21.15.82:0/1028060352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f217c100e70 0x7f217c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.644+0000 7f2180f15640 1 -- 172.21.15.82:0/1028060352 >> 172.21.15.82:0/1028060352 conn(0x7f217c0fc470 msgr2=0x7f217c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.644+0000 7f2180f15640 1 -- 172.21.15.82:0/1028060352 shutdown_connections 2024-04-21T08:46:25.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.644+0000 7f2180f15640 1 -- 172.21.15.82:0/1028060352 wait complete. 2024-04-21T08:46:25.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.645+0000 7f2180f15640 1 Processor -- start 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.645+0000 7f2180f15640 1 -- start start 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.645+0000 7f2180f15640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217c100e70 0x7f217c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.645+0000 7f2180f15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f217c102050 0x7f217c19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.645+0000 7f2180f15640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217c102a10 0x7f217c19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.645+0000 7f2180f15640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f217c10ca80 con 0x7f217c102a10 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.645+0000 7f2180f15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f217c10c900 con 0x7f217c102050 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.645+0000 7f2180f15640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f217c10cc00 con 0x7f217c100e70 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.645+0000 7f217ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217c102a10 0x7f217c19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.645+0000 7f217ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217c102a10 0x7f217c19f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36370/0 (socket says 172.21.15.82:36370) 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.645+0000 7f217ad9d640 1 -- 172.21.15.82:0/2094370677 learned_addr learned my addr 172.21.15.82:0/2094370677 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f2173fff640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f217c102050 0x7f217c19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f217ad9d640 1 -- 172.21.15.82:0/2094370677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217c100e70 msgr2=0x7f217c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f217a59c640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217c100e70 0x7f217c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f217ad9d640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217c100e70 0x7f217c06b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f217ad9d640 1 -- 172.21.15.82:0/2094370677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f217c102050 msgr2=0x7f217c19cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f217ad9d640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f217c102050 0x7f217c19cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f217ad9d640 1 -- 172.21.15.82:0/2094370677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217c19fed0 con 0x7f217c102a10 2024-04-21T08:46:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f217ad9d640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217c102a10 0x7f217c19f5c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f217400cad0 tx=0x7f217400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f21737fe640 1 -- 172.21.15.82:0/2094370677 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217400ae20 con 0x7f217c102a10 2024-04-21T08:46:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f21737fe640 1 -- 172.21.15.82:0/2094370677 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2174005020 con 0x7f217c102a10 2024-04-21T08:46:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f2180f15640 1 -- 172.21.15.82:0/2094370677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217c1a0160 con 0x7f217c102a10 2024-04-21T08:46:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f21737fe640 1 -- 172.21.15.82:0/2094370677 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217400f6b0 con 0x7f217c102a10 2024-04-21T08:46:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.646+0000 7f2180f15640 1 -- 172.21.15.82:0/2094370677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217c1a06e0 con 0x7f217c102a10 2024-04-21T08:46:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.647+0000 7f2180f15640 1 -- 172.21.15.82:0/2094370677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f217c108260 con 0x7f217c102a10 2024-04-21T08:46:25.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.648+0000 7f21737fe640 1 -- 172.21.15.82:0/2094370677 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f21740051d0 con 0x7f217c102a10 2024-04-21T08:46:25.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.649+0000 7f21737fe640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f21680a3c40 0x7f21680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.649+0000 7f217a59c640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f21680a3c40 0x7f21680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.649+0000 7f21737fe640 1 -- 172.21.15.82:0/2094370677 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f21740d0a40 con 0x7f217c102a10 2024-04-21T08:46:25.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.649+0000 7f217a59c640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f21680a3c40 0x7f21680a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f217c101bc0 tx=0x7f216c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.653+0000 7f21737fe640 1 -- 172.21.15.82:0/2094370677 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2174014030 con 0x7f217c102a10 2024-04-21T08:46:25.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:25 smithi134 ceph-mon[31772]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:46:25.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:25 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/312681513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:46:25.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:25 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2323252619' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:46:25.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:25 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1408686877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:25.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.869+0000 7f2180f15640 1 -- 172.21.15.82:0/2094370677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f217c02d5c0 con 0x7f217c102a10 2024-04-21T08:46:25.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.870+0000 7f21737fe640 1 -- 172.21.15.82:0/2094370677 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f217409cf80 con 0x7f217c102a10 2024-04-21T08:46:25.870 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:25.870 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:45:02.005557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24481,24509,24476,24409,24407]},"id":1}]} 2024-04-21T08:46:25.871 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-04-21T08:46:25.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.872+0000 7f2180f15640 1 -- 172.21.15.82:0/2094370677 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f21680a3c40 msgr2=0x7f21680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.872+0000 7f2180f15640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f21680a3c40 0x7f21680a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f217c101bc0 tx=0x7f216c008040 comp rx=0 tx=0).stop 2024-04-21T08:46:25.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.872+0000 7f2180f15640 1 -- 172.21.15.82:0/2094370677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217c102a10 msgr2=0x7f217c19f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.872+0000 7f2180f15640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217c102a10 0x7f217c19f5c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f217400cad0 tx=0x7f217400cf90 comp rx=0 tx=0).stop 2024-04-21T08:46:25.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.872+0000 7f2180f15640 1 -- 172.21.15.82:0/2094370677 shutdown_connections 2024-04-21T08:46:25.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.872+0000 7f2180f15640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f21680a3c40 0x7f21680a6100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.872+0000 7f2180f15640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f217c102a10 0x7f217c19f5c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.872+0000 7f2180f15640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f217c102050 0x7f217c19cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.872+0000 7f2180f15640 1 --2- 172.21.15.82:0/2094370677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f217c100e70 0x7f217c06b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.872+0000 7f2180f15640 1 -- 172.21.15.82:0/2094370677 >> 172.21.15.82:0/2094370677 conn(0x7f217c0fc470 msgr2=0x7f217c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.872+0000 7f2180f15640 1 -- 172.21.15.82:0/2094370677 shutdown_connections 2024-04-21T08:46:25.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.872+0000 7f2180f15640 1 -- 172.21.15.82:0/2094370677 wait complete. 2024-04-21T08:46:25.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:25 smithi106 ceph-mon[31072]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:46:25.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:25 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/312681513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:46:25.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:25 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2323252619' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:46:25.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:25 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1408686877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:25.884 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T08:44:33.819176+0000', 'modified': '2024-04-21T08:45:02.005557+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24509, 'mds_1': 24481, 'mds_2': 24476, 'mds_3': 24409, 'mds_4': 24407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6839/3697606919', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 3697606919}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 3697606919}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6841/1327149046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6840', 'nonce': 1327149046}, {'type': 'v1', 'addr': '172.21.15.82:6841', 'nonce': 1327149046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.106:6833/2583766346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6832', 'nonce': 2583766346}, {'type': 'v1', 'addr': '172.21.15.106:6833', 'nonce': 2583766346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.134:6833/2942732132', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2942732132}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2942732132}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6835/3901395826', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6834', 'nonce': 3901395826}, {'type': 'v1', 'addr': '172.21.15.106:6835', 'nonce': 3901395826}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/2962604291', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 2962604291}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 2962604291}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/1870695477', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 1870695477}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 1870695477}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.106:6837/3731760515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 3731760515}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 3731760515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.134:6839/394037083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 394037083}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 394037083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/3796875133', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 3796875133}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 3796875133}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24481, 24509, 24476, 24409, 24407]} 2024-04-21T08:46:25.884 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-21T08:46:25.884 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-21T08:46:25.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.984+0000 7f8edc89a640 1 -- 172.21.15.82:0/3706180838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ed8105340 msgr2=0x7f8ed8107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.984+0000 7f8edc89a640 1 --2- 172.21.15.82:0/3706180838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ed8105340 0x7f8ed8107730 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009900 tx=0x7f8ec802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.984+0000 7f8edc89a640 1 -- 172.21.15.82:0/3706180838 shutdown_connections 2024-04-21T08:46:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.984+0000 7f8edc89a640 1 --2- 172.21.15.82:0/3706180838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed8107c70 0x7f8ed810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.984+0000 7f8edc89a640 1 --2- 172.21.15.82:0/3706180838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ed8105340 0x7f8ed8107730 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.984+0000 7f8edc89a640 1 --2- 172.21.15.82:0/3706180838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ed8102a10 0x7f8ed8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.984+0000 7f8edc89a640 1 -- 172.21.15.82:0/3706180838 >> 172.21.15.82:0/3706180838 conn(0x7f8ed80fc400 msgr2=0x7f8ed80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.984+0000 7f8edc89a640 1 -- 172.21.15.82:0/3706180838 shutdown_connections 2024-04-21T08:46:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.984+0000 7f8edc89a640 1 -- 172.21.15.82:0/3706180838 wait complete. 2024-04-21T08:46:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8edc89a640 1 Processor -- start 2024-04-21T08:46:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8edc89a640 1 -- start start 2024-04-21T08:46:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8edc89a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed8102a10 0x7f8ed8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8edc89a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ed8105340 0x7f8ed8198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8edc89a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ed8107c70 0x7f8ed819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8edc89a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed810db90 con 0x7f8ed8105340 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8edc89a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed810da10 con 0x7f8ed8107c70 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8edc89a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed810dd10 con 0x7f8ed8102a10 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ed5d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ed8105340 0x7f8ed8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ed5d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ed8105340 0x7f8ed8198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36390/0 (socket says 172.21.15.82:36390) 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ed5d9b640 1 -- 172.21.15.82:0/1500742333 learned_addr learned my addr 172.21.15.82:0/1500742333 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ed659c640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed8102a10 0x7f8ed8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ed6d9d640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ed8107c70 0x7f8ed819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ed5d9b640 1 -- 172.21.15.82:0/1500742333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed8102a10 msgr2=0x7f8ed8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed8102a10 0x7f8ed8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ed5d9b640 1 -- 172.21.15.82:0/1500742333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ed8107c70 msgr2=0x7f8ed819f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ed8107c70 0x7f8ed819f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ed5d9b640 1 -- 172.21.15.82:0/1500742333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed819ff70 con 0x7f8ed8105340 2024-04-21T08:46:25.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ed659c640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed8102a10 0x7f8ed8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:25.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ed8105340 0x7f8ed8198890 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8036040 tx=0x7f8ec80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ec77fe640 1 -- 172.21.15.82:0/1500742333 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8043020 con 0x7f8ed8105340 2024-04-21T08:46:25.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8edc89a640 1 -- 172.21.15.82:0/1500742333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed81a0200 con 0x7f8ed8105340 2024-04-21T08:46:25.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ec77fe640 1 -- 172.21.15.82:0/1500742333 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec8004aa0 con 0x7f8ed8105340 2024-04-21T08:46:25.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.985+0000 7f8ec77fe640 1 -- 172.21.15.82:0/1500742333 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8003de0 con 0x7f8ed8105340 2024-04-21T08:46:25.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.986+0000 7f8edc89a640 1 -- 172.21.15.82:0/1500742333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed81a0820 con 0x7f8ed8105340 2024-04-21T08:46:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.987+0000 7f8ec77fe640 1 -- 172.21.15.82:0/1500742333 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8ec8046020 con 0x7f8ed8105340 2024-04-21T08:46:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8edf56b740 1 Processor -- start 2024-04-21T08:46:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8edf56b740 1 -- start start 2024-04-21T08:46:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8edf56b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf876260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8ec77fe640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8ec00a3c40 0x7f8ec00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8edf56b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf876830 0x5578cf876cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8edf56b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 0x5578cf87deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8edf56b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5578cf7dab10 con 0x5578cf875e40 2024-04-21T08:46:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8edf56b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5578cf778240 con 0x5578cf876830 2024-04-21T08:46:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8edf56b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578cf7d6ad0 con 0x5578cf8772d0 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8ed659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf876260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8ed6d9d640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8ec00a3c40 0x7f8ec00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8ed659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf876260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36396/0 (socket says 172.21.15.82:36396) 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8ed659c640 1 -- 172.21.15.82:0/3913358929 learned_addr learned my addr 172.21.15.82:0/3913358929 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.988+0000 7f8ec77fe640 1 -- 172.21.15.82:0/1500742333 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8ec80f2cb0 con 0x7f8ed8105340 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/3913358929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf876830 0x5578cf876cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ed6d9d640 1 --2- 172.21.15.82:0/3913358929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 0x5578cf87deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ed659c640 1 -- 172.21.15.82:0/3913358929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 msgr2=0x5578cf87deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ed659c640 1 --2- 172.21.15.82:0/3913358929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 0x5578cf87deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ed659c640 1 -- 172.21.15.82:0/3913358929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf876830 msgr2=0x5578cf876cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ed659c640 1 --2- 172.21.15.82:0/3913358929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf876830 0x5578cf876cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ed659c640 1 -- 172.21.15.82:0/3913358929 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578cf4ed2d0 con 0x5578cf875e40 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ed6d9d640 1 --2- 172.21.15.82:0/3913358929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 0x5578cf87deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ed6d9d640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8ec00a3c40 0x7f8ec00a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5578cf871410 tx=0x7f8ed00087c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ed659c640 1 --2- 172.21.15.82:0/3913358929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf876260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc007850 tx=0x7f8ecc007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdd47ca11b6e691 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ec4ff9640 1 -- 172.21.15.82:0/3913358929 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc00d950 con 0x5578cf875e40 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ec4ff9640 1 -- 172.21.15.82:0/3913358929 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ecc018070 con 0x5578cf875e40 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.989+0000 7f8ec4ff9640 1 -- 172.21.15.82:0/3913358929 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc013d90 con 0x5578cf875e40 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 -- 172.21.15.82:0/3913358929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 msgr2=0x5578cf876260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 --2- 172.21.15.82:0/3913358929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf876260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc007850 tx=0x7f8ecc007d10 comp rx=0 tx=0).stop 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 -- 172.21.15.82:0/3913358929 shutdown_connections 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 --2- 172.21.15.82:0/3913358929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 0x5578cf87deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 --2- 172.21.15.82:0/3913358929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf876830 0x5578cf876cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 --2- 172.21.15.82:0/3913358929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf876260 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 -- 172.21.15.82:0/3913358929 >> 172.21.15.82:0/3913358929 conn(0x5578cf871f40 msgr2=0x5578cf870490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 -- 172.21.15.82:0/3913358929 shutdown_connections 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 -- 172.21.15.82:0/3913358929 wait complete. 2024-04-21T08:46:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 -- start start 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 Processor -- start 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf87fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf876830 0x5578cf880390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 0x5578cf8873d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5578cf87e4b0 con 0x5578cf875e40 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5578cf763190 con 0x5578cf876830 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.990+0000 7f8edf56b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578cf87e630 con 0x5578cf8772d0 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8edf56b740 10 client.0 _ll_register_callbacks cb 0x5578cf75e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ed659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf87fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ed659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf87fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36408/0 (socket says 172.21.15.82:36408) 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ed659c640 1 -- 172.21.15.82:0/2839338473 learned_addr learned my addr 172.21.15.82:0/2839338473 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ed6d9d640 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 0x5578cf8873d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf876830 0x5578cf880390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ed659c640 1 -- 172.21.15.82:0/2839338473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 msgr2=0x5578cf8873d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ed659c640 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 0x5578cf8873d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ed659c640 1 -- 172.21.15.82:0/2839338473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf876830 msgr2=0x5578cf880390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ed659c640 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf876830 0x5578cf880390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ed659c640 1 -- 172.21.15.82:0/2839338473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578cf5b76b0 con 0x5578cf875e40 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ed659c640 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf87fe50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc000c00 tx=0x7f8ecc013730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69690d29ba1bbedd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ebdffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ebdffb640 1 -- 172.21.15.82:0/2839338473 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc00d950 con 0x5578cf875e40 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ebdffb640 1 -- 172.21.15.82:0/2839338473 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8ecc00daf0 con 0x5578cf875e40 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8edf56b740 1 -- 172.21.15.82:0/2839338473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ecc018070 con 0x5578cf875e40 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ebdffb640 1 -- 172.21.15.82:0/2839338473 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8ecc00dcf0 con 0x5578cf875e40 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8edf56b740 10 client.15237 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8edf56b740 1 -- 172.21.15.82:0/2839338473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578cf88a6b0 con 0x5578cf875e40 2024-04-21T08:46:25.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.991+0000 7f8ebdffb640 1 -- 172.21.15.82:0/2839338473 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ecc018070 con 0x5578cf875e40 2024-04-21T08:46:26.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.992+0000 7f8ebdffb640 1 -- 172.21.15.82:0/2839338473 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8ecc01d400 con 0x5578cf875e40 2024-04-21T08:46:26.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.992+0000 7f8edf56b740 10 client.15237 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:26.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.992+0000 7f8edf56b740 10 client.15237 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:46:26.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.992+0000 7f8edf56b740 20 client.15237 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:26.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.992+0000 7f8edf56b740 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5578cf88cb40 0x5578cf88dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.992+0000 7f8edf56b740 4 client.15237 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:26.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.992+0000 7f8edf56b740 1 -- 172.21.15.82:0/2839338473 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578cf7e8530 con 0x5578cf88cb40 2024-04-21T08:46:26.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.992+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5578cf88cb40 0x5578cf88dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.992+0000 7f8ebdffb640 10 client.15237 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.992+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5578cf88cb40 0x5578cf88dfb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4fc52d38f527267 server_cookie=fb01db8181e08b8a in_seq=0 out_seq=0 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.993+0000 7f8ebdffb640 1 -- 172.21.15.82:0/2839338473 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578cf7e8530 con 0x5578cf88cb40 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.993+0000 7f8ebdffb640 10 client.15237 handle_command_reply: tid=0 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.993+0000 7f8edf56b740 1 client.15237 shutdown 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.994+0000 7f8edf56b740 1 -- 172.21.15.82:0/2839338473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 msgr2=0x5578cf87fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.994+0000 7f8edf56b740 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf87fe50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc000c00 tx=0x7f8ecc013730 comp rx=0 tx=0).stop 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.994+0000 7f8edf56b740 1 -- 172.21.15.82:0/2839338473 shutdown_connections 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.994+0000 7f8edf56b740 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5578cf88cb40 0x5578cf88dfb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.994+0000 7f8edf56b740 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 0x5578cf8873d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.994+0000 7f8edf56b740 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf876830 0x5578cf880390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.994+0000 7f8edf56b740 1 --2- 172.21.15.82:0/2839338473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf87fe50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.994+0000 7f8ebdffb640 0 client.15237 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:46:26.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.994+0000 7f8edf56b740 1 -- 172.21.15.82:0/2839338473 >> 172.21.15.82:0/2839338473 conn(0x5578cf871f40 msgr2=0x5578cf86fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.994+0000 7f8edf56b740 1 -- 172.21.15.82:0/2839338473 shutdown_connections 2024-04-21T08:46:26.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.994+0000 7f8edf56b740 1 -- 172.21.15.82:0/2839338473 wait complete. 2024-04-21T08:46:26.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:25.994+0000 7f8edf56b740 20 client.15237 trim_cache size 0 max 16384 2024-04-21T08:46:26.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.011+0000 7f8edf56b740 1 Processor -- start 2024-04-21T08:46:26.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.011+0000 7f8edf56b740 1 -- start start 2024-04-21T08:46:26.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8edf56b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf875e40 0x5578cf88bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8edf56b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf876830 0x5578cf887390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8edf56b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 0x5578cf87ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8edf56b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5578cf88c2f0 con 0x5578cf876830 2024-04-21T08:46:26.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8edf56b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5578cf88c9c0 con 0x5578cf875e40 2024-04-21T08:46:26.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8edf56b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578cf88d090 con 0x5578cf8772d0 2024-04-21T08:46:26.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ed5d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf876830 0x5578cf887390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ed5d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf876830 0x5578cf887390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36412/0 (socket says 172.21.15.82:36412) 2024-04-21T08:46:26.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ed5d9b640 1 -- 172.21.15.82:0/2679671545 learned_addr learned my addr 172.21.15.82:0/2679671545 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ed659c640 1 --2- 172.21.15.82:0/2679671545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf875e40 0x5578cf88bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ed5d9b640 1 -- 172.21.15.82:0/2679671545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 msgr2=0x5578cf87ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/2679671545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 0x5578cf87ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ed5d9b640 1 -- 172.21.15.82:0/2679671545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf875e40 msgr2=0x5578cf88bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/2679671545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf875e40 0x5578cf88bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ed5d9b640 1 -- 172.21.15.82:0/2679671545 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578cf871090 con 0x5578cf876830 2024-04-21T08:46:26.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/2679671545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf876830 0x5578cf887390 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80bb060 tx=0x7f8ec80baea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b8b4146b8c3c03 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ebffff640 1 -- 172.21.15.82:0/2679671545 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec803ce30 con 0x5578cf876830 2024-04-21T08:46:26.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ebffff640 1 -- 172.21.15.82:0/2679671545 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec80bec00 con 0x5578cf876830 2024-04-21T08:46:26.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.012+0000 7f8ebffff640 1 -- 172.21.15.82:0/2679671545 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80bff30 con 0x5578cf876830 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 -- 172.21.15.82:0/2679671545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf876830 msgr2=0x5578cf887390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 --2- 172.21.15.82:0/2679671545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf876830 0x5578cf887390 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80bb060 tx=0x7f8ec80baea0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 -- 172.21.15.82:0/2679671545 shutdown_connections 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 --2- 172.21.15.82:0/2679671545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf8772d0 0x5578cf87ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 --2- 172.21.15.82:0/2679671545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf876830 0x5578cf887390 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 --2- 172.21.15.82:0/2679671545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf875e40 0x5578cf88bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 -- 172.21.15.82:0/2679671545 >> 172.21.15.82:0/2679671545 conn(0x5578cf871f40 msgr2=0x5578cf86b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 -- 172.21.15.82:0/2679671545 shutdown_connections 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 -- 172.21.15.82:0/2679671545 wait complete. 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 -- start start 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 Processor -- start 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf882090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf876830 0x5578cf8825d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf8772d0 0x5578cf8f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5578cf88de30 con 0x5578cf875e40 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5578cf88d760 con 0x5578cf8772d0 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578cf88e500 con 0x5578cf876830 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.013+0000 7f8edf56b740 10 client.0 _ll_register_callbacks cb 0x7f8ecc0076d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf882090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf882090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36420/0 (socket says 172.21.15.82:36420) 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed659c640 1 -- 172.21.15.82:0/3442320837 learned_addr learned my addr 172.21.15.82:0/3442320837 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed6d9d640 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf8772d0 0x5578cf8f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf876830 0x5578cf8825d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed659c640 1 -- 172.21.15.82:0/3442320837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf876830 msgr2=0x5578cf8825d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed659c640 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf876830 0x5578cf8825d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed659c640 1 -- 172.21.15.82:0/3442320837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf8772d0 msgr2=0x5578cf8f0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed659c640 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf8772d0 0x5578cf8f0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed659c640 1 -- 172.21.15.82:0/3442320837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578cf882c40 con 0x5578cf875e40 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf876830 0x5578cf8825d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed6d9d640 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf8772d0 0x5578cf8f0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ed659c640 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf882090 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc007d50 tx=0x7f8ecc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4566625773ec8194 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ebdffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ebdffb640 1 -- 172.21.15.82:0/3442320837 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc039060 con 0x5578cf875e40 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ebdffb640 1 -- 172.21.15.82:0/3442320837 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8ecc007140 con 0x5578cf875e40 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8edf56b740 1 -- 172.21.15.82:0/3442320837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec80bec00 con 0x5578cf875e40 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ebdffb640 1 -- 172.21.15.82:0/3442320837 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8ecc040050 con 0x5578cf875e40 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8edf56b740 10 client.15243 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8edf56b740 1 -- 172.21.15.82:0/3442320837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578cf8f21a0 con 0x5578cf875e40 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.014+0000 7f8ebdffb640 1 -- 172.21.15.82:0/3442320837 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec80bec00 con 0x5578cf875e40 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8ebdffb640 1 -- 172.21.15.82:0/3442320837 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8ecc028b70 con 0x5578cf875e40 2024-04-21T08:46:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8edf56b740 10 client.15243 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8edf56b740 10 client.15243 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8edf56b740 20 client.15243 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8edf56b740 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5578cf8f6d80 0x5578cf8f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8edf56b740 4 client.15243 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8edf56b740 1 -- 172.21.15.82:0/3442320837 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "status"}) -- 0x5578cf7e23a0 con 0x5578cf8f6d80 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5578cf8f6d80 0x5578cf8f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8ebdffb640 10 client.15243 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8ed5d9b640 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5578cf8f6d80 0x5578cf8f5a90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bbdd99f933dbfc84 server_cookie=f3c5db0cb64f972 in_seq=0 out_seq=0 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8ebdffb640 1 -- 172.21.15.82:0/3442320837 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5578cf7e23a0 con 0x5578cf8f6d80 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8ebdffb640 10 client.15243 handle_command_reply: tid=0 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.015+0000 7f8edf56b740 1 client.15243 shutdown 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edf56b740 1 -- 172.21.15.82:0/3442320837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 msgr2=0x5578cf882090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edf56b740 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf882090 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc007d50 tx=0x7f8ecc00dc00 comp rx=0 tx=0).stop 2024-04-21T08:46:26.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edf56b740 1 -- 172.21.15.82:0/3442320837 shutdown_connections 2024-04-21T08:46:26.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edf56b740 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5578cf8f6d80 0x5578cf8f5a90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edf56b740 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578cf8772d0 0x5578cf8f0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8ebdffb640 0 client.15243 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:46:26.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edf56b740 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578cf876830 0x5578cf8825d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edf56b740 1 --2- 172.21.15.82:0/3442320837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578cf875e40 0x5578cf882090 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edf56b740 1 -- 172.21.15.82:0/3442320837 >> 172.21.15.82:0/3442320837 conn(0x5578cf871f40 msgr2=0x5578cf87dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edf56b740 1 -- 172.21.15.82:0/3442320837 shutdown_connections 2024-04-21T08:46:26.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edf56b740 1 -- 172.21.15.82:0/3442320837 wait complete. 2024-04-21T08:46:26.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edf56b740 20 client.15243 trim_cache size 0 max 16384 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 4, 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24407, 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 85.013408503999997, 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 91, 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 146.18856673299999 2024-04-21T08:46:26.024 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:46:26.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edc89a640 1 -- 172.21.15.82:0/1500742333 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8ec00a3c40 msgr2=0x7f8ec00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edc89a640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8ec00a3c40 0x7f8ec00a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5578cf871410 tx=0x7f8ed00087c0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edc89a640 1 -- 172.21.15.82:0/1500742333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ed8105340 msgr2=0x7f8ed8198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edc89a640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ed8105340 0x7f8ed8198890 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8036040 tx=0x7f8ec80042f0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edc89a640 1 -- 172.21.15.82:0/1500742333 shutdown_connections 2024-04-21T08:46:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edc89a640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8ec00a3c40 0x7f8ec00a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edc89a640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ed8107c70 0x7f8ed819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edc89a640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ed8105340 0x7f8ed8198890 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edc89a640 1 --2- 172.21.15.82:0/1500742333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ed8102a10 0x7f8ed8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edc89a640 1 -- 172.21.15.82:0/1500742333 >> 172.21.15.82:0/1500742333 conn(0x7f8ed80fc400 msgr2=0x7f8ed8103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edc89a640 1 -- 172.21.15.82:0/1500742333 shutdown_connections 2024-04-21T08:46:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.016+0000 7f8edc89a640 1 -- 172.21.15.82:0/1500742333 wait complete. 2024-04-21T08:46:26.027 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-21T08:46:26.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.127+0000 7f0485366640 1 -- 172.21.15.82:0/1360859031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0480103bf0 msgr2=0x7f048010a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.127+0000 7f0485366640 1 --2- 172.21.15.82:0/1360859031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0480103bf0 0x7f048010a480 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0470009930 tx=0x7f047002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:26.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.127+0000 7f0485366640 1 -- 172.21.15.82:0/1360859031 shutdown_connections 2024-04-21T08:46:26.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.127+0000 7f0485366640 1 --2- 172.21.15.82:0/1360859031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0480103bf0 0x7f048010a480 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.127+0000 7f0485366640 1 --2- 172.21.15.82:0/1360859031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0480103230 0x7f04801036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.127+0000 7f0485366640 1 --2- 172.21.15.82:0/1360859031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0480100260 0x7f0480100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.127+0000 7f0485366640 1 -- 172.21.15.82:0/1360859031 >> 172.21.15.82:0/1360859031 conn(0x7f04800fbb90 msgr2=0x7f04800fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.127+0000 7f0485366640 1 -- 172.21.15.82:0/1360859031 shutdown_connections 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.127+0000 7f0485366640 1 -- 172.21.15.82:0/1360859031 wait complete. 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.127+0000 7f0485366640 1 Processor -- start 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f0485366640 1 -- start start 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f0485366640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0480100260 0x7f0480102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f0485366640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0480103230 0x7f04801011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f0485366640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0480103bf0 0x7f0480101720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f0485366640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f048010db40 con 0x7f0480103bf0 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f0485366640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f048010d9c0 con 0x7f0480100260 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f0485366640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f048010dcc0 con 0x7f0480103230 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0480103bf0 0x7f0480101720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0480103bf0 0x7f0480101720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36448/0 (socket says 172.21.15.82:36448) 2024-04-21T08:46:26.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047f7fe640 1 -- 172.21.15.82:0/2200574801 learned_addr learned my addr 172.21.15.82:0/2200574801 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047effd640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0480100260 0x7f0480102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047e7fc640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0480103230 0x7f04801011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047f7fe640 1 -- 172.21.15.82:0/2200574801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0480103230 msgr2=0x7f04801011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047f7fe640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0480103230 0x7f04801011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047f7fe640 1 -- 172.21.15.82:0/2200574801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0480100260 msgr2=0x7f0480102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047f7fe640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0480100260 0x7f0480102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047f7fe640 1 -- 172.21.15.82:0/2200574801 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0480101fb0 con 0x7f0480103bf0 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047effd640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0480100260 0x7f0480102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047e7fc640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0480103230 0x7f04801011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f047f7fe640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0480103bf0 0x7f0480101720 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0470005ab0 tx=0x7f04700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f0467fff640 1 -- 172.21.15.82:0/2200574801 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470043020 con 0x7f0480103bf0 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f0485366640 1 -- 172.21.15.82:0/2200574801 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04801a2330 con 0x7f0480103bf0 2024-04-21T08:46:26.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f0467fff640 1 -- 172.21.15.82:0/2200574801 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0470004a80 con 0x7f0480103bf0 2024-04-21T08:46:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.128+0000 7f0467fff640 1 -- 172.21.15.82:0/2200574801 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470003d00 con 0x7f0480103bf0 2024-04-21T08:46:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.129+0000 7f0485366640 1 -- 172.21.15.82:0/2200574801 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04801a2950 con 0x7f0480103bf0 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.130+0000 7f0467fff640 1 -- 172.21.15.82:0/2200574801 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0470046020 con 0x7f0480103bf0 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f0488030740 1 Processor -- start 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f0488030740 1 -- start start 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f0488030740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 0x5654567ca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f0488030740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567ca830 0x5654567cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f0488030740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567cb2d0 0x5654567d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f0488030740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56545672eb10 con 0x5654567c9e40 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f0488030740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5654566cc240 con 0x5654567cb2d0 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f0488030740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56545672aad0 con 0x5654567ca830 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f047e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 0x5654567ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f047e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 0x5654567ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36458/0 (socket says 172.21.15.82:36458) 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f047e7fc640 1 -- 172.21.15.82:0/291606160 learned_addr learned my addr 172.21.15.82:0/291606160 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f0467fff640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f04680a3c40 0x7f04680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f047f7fe640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f04680a3c40 0x7f04680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.131+0000 7f047effd640 1 --2- 172.21.15.82:0/291606160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567ca830 0x5654567cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.132+0000 7f047e7fc640 1 -- 172.21.15.82:0/291606160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567ca830 msgr2=0x5654567cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.132+0000 7f047e7fc640 1 --2- 172.21.15.82:0/291606160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567ca830 0x5654567cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.132+0000 7f047e7fc640 1 -- 172.21.15.82:0/291606160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567cb2d0 msgr2=0x5654567d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.132+0000 7f047e7fc640 1 --2- 172.21.15.82:0/291606160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567cb2d0 0x5654567d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.132+0000 7f047e7fc640 1 -- 172.21.15.82:0/291606160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654564412d0 con 0x5654567c9e40 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.132+0000 7f047effd640 1 --2- 172.21.15.82:0/291606160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567ca830 0x5654567cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.132+0000 7f0467fff640 1 -- 172.21.15.82:0/2200574801 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f04700f2c90 con 0x7f0480103bf0 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.132+0000 7f047e7fc640 1 --2- 172.21.15.82:0/291606160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 0x5654567ca260 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f047400c520 tx=0x7f047400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cc432c136cafcb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.132+0000 7f047f7fe640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f04680a3c40 0x7f04680a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f04700be8e0 tx=0x7f047004f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.132+0000 7f04657fa640 1 -- 172.21.15.82:0/291606160 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04740076e0 con 0x5654567c9e40 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.132+0000 7f04657fa640 1 -- 172.21.15.82:0/291606160 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0474013070 con 0x5654567c9e40 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.132+0000 7f04657fa640 1 -- 172.21.15.82:0/291606160 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474018b00 con 0x5654567c9e40 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 -- 172.21.15.82:0/291606160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 msgr2=0x5654567ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 --2- 172.21.15.82:0/291606160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 0x5654567ca260 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f047400c520 tx=0x7f047400c9e0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 -- 172.21.15.82:0/291606160 shutdown_connections 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 --2- 172.21.15.82:0/291606160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567cb2d0 0x5654567d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 --2- 172.21.15.82:0/291606160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567ca830 0x5654567cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 --2- 172.21.15.82:0/291606160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 0x5654567ca260 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 -- 172.21.15.82:0/291606160 >> 172.21.15.82:0/291606160 conn(0x5654567c5f40 msgr2=0x5654567c4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 -- 172.21.15.82:0/291606160 shutdown_connections 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 -- 172.21.15.82:0/291606160 wait complete. 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 -- start start 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 Processor -- start 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567c9e40 0x5654567d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567ca830 0x5654567d43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567cb2d0 0x5654567db430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5654567d24b0 con 0x5654567ca830 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5654566b7190 con 0x5654567c9e40 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5654567d2630 con 0x5654567cb2d0 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f0488030740 10 client.0 _ll_register_callbacks cb 0x5654566b28b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f047effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567ca830 0x5654567d43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f047effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567ca830 0x5654567d43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36470/0 (socket says 172.21.15.82:36470) 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.133+0000 7f047effd640 1 -- 172.21.15.82:0/3704948514 learned_addr learned my addr 172.21.15.82:0/3704948514 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f047e7fc640 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567c9e40 0x5654567d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f047e7fc640 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567cb2d0 0x5654567db430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f047effd640 1 -- 172.21.15.82:0/3704948514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567cb2d0 msgr2=0x5654567db430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f047effd640 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567cb2d0 0x5654567db430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f047effd640 1 -- 172.21.15.82:0/3704948514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567c9e40 msgr2=0x5654567d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f047effd640 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567c9e40 0x5654567d3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f047effd640 1 -- 172.21.15.82:0/3704948514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56545650b6b0 con 0x5654567ca830 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f047effd640 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567ca830 0x5654567d43f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0478010670 tx=0x7f0478010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e08838029f7516 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f0462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f0462ff5640 1 -- 172.21.15.82:0/3704948514 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047800e940 con 0x5654567ca830 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f0462ff5640 1 -- 172.21.15.82:0/3704948514 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f047800eae0 con 0x5654567ca830 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f0488030740 1 -- 172.21.15.82:0/3704948514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0474013070 con 0x5654567ca830 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f0462ff5640 1 -- 172.21.15.82:0/3704948514 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0478025050 con 0x5654567ca830 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f0488030740 10 client.15255 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f0488030740 1 -- 172.21.15.82:0/3704948514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654567de7e0 con 0x5654567ca830 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f0462ff5640 1 -- 172.21.15.82:0/3704948514 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0474013070 con 0x5654567ca830 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f0462ff5640 1 -- 172.21.15.82:0/3704948514 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f047800f040 con 0x5654567ca830 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f0488030740 10 client.15255 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.134+0000 7f0488030740 10 client.15255 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:46:26.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.135+0000 7f0488030740 20 client.15255 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:26.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.135+0000 7f0488030740 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5654567e0eb0 0x5654567e2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.135+0000 7f0488030740 4 client.15255 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:26.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.135+0000 7f0488030740 1 -- 172.21.15.82:0/3704948514 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56545673c530 con 0x5654567e0eb0 2024-04-21T08:46:26.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.135+0000 7f047e7fc640 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5654567e0eb0 0x5654567e2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.135+0000 7f047e7fc640 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5654567e0eb0 0x5654567e2320 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6687ebe53102109c server_cookie=652c2e60da82796a in_seq=0 out_seq=0 2024-04-21T08:46:26.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.135+0000 7f0462ff5640 10 client.15255 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:46:26.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0462ff5640 1 -- 172.21.15.82:0/3704948514 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56545673c530 con 0x5654567e0eb0 2024-04-21T08:46:26.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0462ff5640 10 client.15255 handle_command_reply: tid=0 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0488030740 1 client.15255 shutdown 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0488030740 1 -- 172.21.15.82:0/3704948514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567ca830 msgr2=0x5654567d43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0488030740 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567ca830 0x5654567d43f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0478010670 tx=0x7f0478010a70 comp rx=0 tx=0).stop 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0488030740 1 -- 172.21.15.82:0/3704948514 shutdown_connections 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0488030740 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5654567e0eb0 0x5654567e2320 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0488030740 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567cb2d0 0x5654567db430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0488030740 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567ca830 0x5654567d43f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0462ff5640 0 client.15255 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0488030740 1 --2- 172.21.15.82:0/3704948514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567c9e40 0x5654567d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0488030740 1 -- 172.21.15.82:0/3704948514 >> 172.21.15.82:0/3704948514 conn(0x5654567c5f40 msgr2=0x5654567c3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0488030740 1 -- 172.21.15.82:0/3704948514 shutdown_connections 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.136+0000 7f0488030740 1 -- 172.21.15.82:0/3704948514 wait complete. 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.137+0000 7f0488030740 20 client.15255 trim_cache size 0 max 16384 2024-04-21T08:46:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.154+0000 7f0488030740 1 Processor -- start 2024-04-21T08:46:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.154+0000 7f0488030740 1 -- start start 2024-04-21T08:46:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.154+0000 7f0488030740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 0x5654567cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.154+0000 7f0488030740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567ca830 0x5654567db3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.154+0000 7f0488030740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567cb2d0 0x5654567ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.154+0000 7f0488030740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5654567e0660 con 0x5654567c9e40 2024-04-21T08:46:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.154+0000 7f0488030740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5654567e0d30 con 0x5654567ca830 2024-04-21T08:46:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.154+0000 7f0488030740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5654567e1400 con 0x5654567cb2d0 2024-04-21T08:46:26.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.154+0000 7f047e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 0x5654567cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.154+0000 7f047e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 0x5654567cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36484/0 (socket says 172.21.15.82:36484) 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.154+0000 7f047e7fc640 1 -- 172.21.15.82:0/2797955892 learned_addr learned my addr 172.21.15.82:0/2797955892 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.154+0000 7f047effd640 1 --2- 172.21.15.82:0/2797955892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567ca830 0x5654567db3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f047e7fc640 1 -- 172.21.15.82:0/2797955892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567cb2d0 msgr2=0x5654567ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f047e7fc640 1 --2- 172.21.15.82:0/2797955892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567cb2d0 0x5654567ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f047e7fc640 1 -- 172.21.15.82:0/2797955892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567ca830 msgr2=0x5654567db3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f047e7fc640 1 --2- 172.21.15.82:0/2797955892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567ca830 0x5654567db3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f047e7fc640 1 -- 172.21.15.82:0/2797955892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654567bfe70 con 0x5654567c9e40 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f047effd640 1 --2- 172.21.15.82:0/2797955892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567ca830 0x5654567db3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f047e7fc640 1 --2- 172.21.15.82:0/2797955892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 0x5654567cac10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f047400c4f0 tx=0x7f0474004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f3a91345f4e9eb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f0464ff9640 1 -- 172.21.15.82:0/2797955892 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04740185f0 con 0x5654567c9e40 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f0464ff9640 1 -- 172.21.15.82:0/2797955892 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0474013050 con 0x5654567c9e40 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f0464ff9640 1 -- 172.21.15.82:0/2797955892 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474007d90 con 0x5654567c9e40 2024-04-21T08:46:26.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f0488030740 1 -- 172.21.15.82:0/2797955892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 msgr2=0x5654567cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f0488030740 1 --2- 172.21.15.82:0/2797955892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 0x5654567cac10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f047400c4f0 tx=0x7f0474004a40 comp rx=0 tx=0).stop 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f0488030740 1 -- 172.21.15.82:0/2797955892 shutdown_connections 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f0488030740 1 --2- 172.21.15.82:0/2797955892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567cb2d0 0x5654567ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f0488030740 1 --2- 172.21.15.82:0/2797955892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567ca830 0x5654567db3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f0488030740 1 --2- 172.21.15.82:0/2797955892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567c9e40 0x5654567cac10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f0488030740 1 -- 172.21.15.82:0/2797955892 >> 172.21.15.82:0/2797955892 conn(0x5654567c5f40 msgr2=0x5654567d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f0488030740 1 -- 172.21.15.82:0/2797955892 shutdown_connections 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.155+0000 7f0488030740 1 -- 172.21.15.82:0/2797955892 wait complete. 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f0488030740 1 -- start start 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f0488030740 1 Processor -- start 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f0488030740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567c9e40 0x5654567d6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f0488030740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567ca830 0x5654567d6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f0488030740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567cb2d0 0x5654567d9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f0488030740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5654567e21a0 con 0x5654567cb2d0 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f0488030740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5654567e1ad0 con 0x5654567ca830 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f0488030740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5654567e2870 con 0x5654567c9e40 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f0488030740 10 client.0 _ll_register_callbacks cb 0x7f047800b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f047e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567cb2d0 0x5654567d9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f047e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567cb2d0 0x5654567d9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36488/0 (socket says 172.21.15.82:36488) 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f047e7fc640 1 -- 172.21.15.82:0/1564484067 learned_addr learned my addr 172.21.15.82:0/1564484067 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f047e7fc640 1 -- 172.21.15.82:0/1564484067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567c9e40 msgr2=0x5654567d6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f047e7fc640 1 --2- 172.21.15.82:0/1564484067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567c9e40 0x5654567d6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f047e7fc640 1 -- 172.21.15.82:0/1564484067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567ca830 msgr2=0x5654567d6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f047e7fc640 1 --2- 172.21.15.82:0/1564484067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567ca830 0x5654567d6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f047e7fc640 1 -- 172.21.15.82:0/1564484067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654567da2f0 con 0x5654567cb2d0 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f047e7fc640 1 --2- 172.21.15.82:0/1564484067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567cb2d0 0x5654567d9d00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0474009fe0 tx=0x7f0474009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecf2f7ecd330f26 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f0462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.156+0000 7f0462ff5640 1 -- 172.21.15.82:0/1564484067 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474007b10 con 0x5654567cb2d0 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0462ff5640 1 -- 172.21.15.82:0/1564484067 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f0474007cb0 con 0x5654567cb2d0 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0488030740 1 -- 172.21.15.82:0/1564484067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0474013050 con 0x5654567cb2d0 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0462ff5640 1 -- 172.21.15.82:0/1564484067 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04740185f0 con 0x5654567cb2d0 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0488030740 10 client.15261 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0488030740 1 -- 172.21.15.82:0/1564484067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5654567db560 con 0x5654567cb2d0 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0462ff5640 1 -- 172.21.15.82:0/1564484067 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0474013050 con 0x5654567cb2d0 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0462ff5640 1 -- 172.21.15.82:0/1564484067 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0474018ce0 con 0x5654567cb2d0 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0488030740 10 client.15261 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0488030740 10 client.15261 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0488030740 20 client.15261 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0488030740 1 --2- 172.21.15.82:0/1564484067 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x56545684afe0 0x565456849cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0488030740 4 client.15261 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f0488030740 1 -- 172.21.15.82:0/1564484067 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "status"}) -- 0x5654567363a0 con 0x56545684afe0 2024-04-21T08:46:26.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.157+0000 7f047effd640 1 --2- 172.21.15.82:0/1564484067 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x56545684afe0 0x565456849cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0462ff5640 10 client.15261 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f047effd640 1 --2- 172.21.15.82:0/1564484067 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x56545684afe0 0x565456849cf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5990885bd98fe110 server_cookie=6d5267e8c5024ba in_seq=0 out_seq=0 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0462ff5640 1 -- 172.21.15.82:0/1564484067 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5654567363a0 con 0x56545684afe0 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0462ff5640 10 client.15261 handle_command_reply: tid=0 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0488030740 1 client.15261 shutdown 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0488030740 1 -- 172.21.15.82:0/1564484067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567cb2d0 msgr2=0x5654567d9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0488030740 1 --2- 172.21.15.82:0/1564484067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567cb2d0 0x5654567d9d00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0474009fe0 tx=0x7f0474009d50 comp rx=0 tx=0).stop 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0488030740 1 -- 172.21.15.82:0/1564484067 shutdown_connections 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0488030740 1 --2- 172.21.15.82:0/1564484067 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x56545684afe0 0x565456849cf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0488030740 1 --2- 172.21.15.82:0/1564484067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654567cb2d0 0x5654567d9d00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0488030740 1 --2- 172.21.15.82:0/1564484067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654567ca830 0x5654567d6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0488030740 1 --2- 172.21.15.82:0/1564484067 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654567c9e40 0x5654567d6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0488030740 1 -- 172.21.15.82:0/1564484067 >> 172.21.15.82:0/1564484067 conn(0x5654567c5f40 msgr2=0x5654567d3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0462ff5640 0 client.15261 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:46:26.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0488030740 1 -- 172.21.15.82:0/1564484067 shutdown_connections 2024-04-21T08:46:26.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0488030740 1 -- 172.21.15.82:0/1564484067 wait complete. 2024-04-21T08:46:26.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.158+0000 7f0488030740 20 client.15261 trim_cache size 0 max 16384 2024-04-21T08:46:26.165 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:46:26.165 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:46:26.165 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 3, 2024-04-21T08:46:26.165 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24409, 2024-04-21T08:46:26.165 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:46:26.165 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:46:26.165 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:46:26.166 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 86.169012175000006, 2024-04-21T08:46:26.166 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2024-04-21T08:46:26.166 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 91, 2024-04-21T08:46:26.166 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:46:26.166 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 149.206191897 2024-04-21T08:46:26.166 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:46:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.159+0000 7f0485366640 1 -- 172.21.15.82:0/2200574801 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f04680a3c40 msgr2=0x7f04680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.159+0000 7f0485366640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f04680a3c40 0x7f04680a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f04700be8e0 tx=0x7f047004f000 comp rx=0 tx=0).stop 2024-04-21T08:46:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.159+0000 7f0485366640 1 -- 172.21.15.82:0/2200574801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0480103bf0 msgr2=0x7f0480101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.159+0000 7f0485366640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0480103bf0 0x7f0480101720 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0470005ab0 tx=0x7f04700042f0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.159+0000 7f0485366640 1 -- 172.21.15.82:0/2200574801 shutdown_connections 2024-04-21T08:46:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.159+0000 7f0485366640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f04680a3c40 0x7f04680a6100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.159+0000 7f0485366640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0480103bf0 0x7f0480101720 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.159+0000 7f0485366640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0480103230 0x7f04801011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.159+0000 7f0485366640 1 --2- 172.21.15.82:0/2200574801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0480100260 0x7f0480102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.159+0000 7f0485366640 1 -- 172.21.15.82:0/2200574801 >> 172.21.15.82:0/2200574801 conn(0x7f04800fbb90 msgr2=0x7f0480106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.159+0000 7f0485366640 1 -- 172.21.15.82:0/2200574801 shutdown_connections 2024-04-21T08:46:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.159+0000 7f0485366640 1 -- 172.21.15.82:0/2200574801 wait complete. 2024-04-21T08:46:26.169 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.269+0000 7f8d214b2640 1 -- 172.21.15.82:0/2817680235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c101fd0 msgr2=0x7f8d1c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.269+0000 7f8d214b2640 1 --2- 172.21.15.82:0/2817680235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c101fd0 0x7f8d1c102450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009900 tx=0x7f8d0402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.269+0000 7f8d214b2640 1 -- 172.21.15.82:0/2817680235 shutdown_connections 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.269+0000 7f8d214b2640 1 --2- 172.21.15.82:0/2817680235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c102990 0x7f8d1c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.269+0000 7f8d214b2640 1 --2- 172.21.15.82:0/2817680235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c101fd0 0x7f8d1c102450 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.269+0000 7f8d214b2640 1 --2- 172.21.15.82:0/2817680235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c100e80 0x7f8d1c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.269+0000 7f8d214b2640 1 -- 172.21.15.82:0/2817680235 >> 172.21.15.82:0/2817680235 conn(0x7f8d1c0fc3f0 msgr2=0x7f8d1c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.269+0000 7f8d214b2640 1 -- 172.21.15.82:0/2817680235 shutdown_connections 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.269+0000 7f8d214b2640 1 -- 172.21.15.82:0/2817680235 wait complete. 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.269+0000 7f8d214b2640 1 Processor -- start 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.269+0000 7f8d214b2640 1 -- start start 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d214b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c100e80 0x7f8d1c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d214b2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c101fd0 0x7f8d1c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d214b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c102990 0x7f8d1c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d214b2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10ca50 con 0x7f8d1c102990 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d214b2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10c8d0 con 0x7f8d1c101fd0 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d214b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10cbd0 con 0x7f8d1c100e80 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d1b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c102990 0x7f8d1c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d1b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c102990 0x7f8d1c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36506/0 (socket says 172.21.15.82:36506) 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d1b7fe640 1 -- 172.21.15.82:0/1069647423 learned_addr learned my addr 172.21.15.82:0/1069647423 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c101fd0 0x7f8d1c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d1affd640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c100e80 0x7f8d1c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d1b7fe640 1 -- 172.21.15.82:0/1069647423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c100e80 msgr2=0x7f8d1c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d1b7fe640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c100e80 0x7f8d1c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d1b7fe640 1 -- 172.21.15.82:0/1069647423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c101fd0 msgr2=0x7f8d1c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d1b7fe640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c101fd0 0x7f8d1c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d1b7fe640 1 -- 172.21.15.82:0/1069647423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c19fea0 con 0x7f8d1c102990 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d1affd640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c100e80 0x7f8d1c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d1b7fe640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c102990 0x7f8d1c19f830 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00d750 tx=0x7f8d0c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d13fff640 1 -- 172.21.15.82:0/1069647423 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c017070 con 0x7f8d1c102990 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d13fff640 1 -- 172.21.15.82:0/1069647423 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0c0054d0 con 0x7f8d1c102990 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d214b2640 1 -- 172.21.15.82:0/1069647423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1c1a0190 con 0x7f8d1c102990 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d13fff640 1 -- 172.21.15.82:0/1069647423 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c004c60 con 0x7f8d1c102990 2024-04-21T08:46:26.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.270+0000 7f8d214b2640 1 -- 172.21.15.82:0/1069647423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1c1a0730 con 0x7f8d1c102990 2024-04-21T08:46:26.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.273+0000 7f8d13fff640 1 -- 172.21.15.82:0/1069647423 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8d0c009bd0 con 0x7f8d1c102990 2024-04-21T08:46:26.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.273+0000 7f8d24176740 1 Processor -- start 2024-04-21T08:46:26.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.273+0000 7f8d24176740 1 -- start start 2024-04-21T08:46:26.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.273+0000 7f8d24176740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 0x55c205eaa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.273+0000 7f8d24176740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eaa830 0x55c205eaacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.273+0000 7f8d24176740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eab2d0 0x55c205eb1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.273+0000 7f8d24176740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c205e0eb10 con 0x55c205ea9e40 2024-04-21T08:46:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.273+0000 7f8d24176740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c205dac240 con 0x55c205eab2d0 2024-04-21T08:46:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.273+0000 7f8d24176740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c205e0aad0 con 0x55c205eaa830 2024-04-21T08:46:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.273+0000 7f8d13fff640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8d080a3c40 0x7f8d080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 0x55c205eaa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 0x55c205eaa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36510/0 (socket says 172.21.15.82:36510) 2024-04-21T08:46:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1a7fc640 1 -- 172.21.15.82:0/3182234352 learned_addr learned my addr 172.21.15.82:0/3182234352 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1affd640 1 --2- 172.21.15.82:0/3182234352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eaa830 0x55c205eaacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1b7fe640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8d080a3c40 0x7f8d080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/3182234352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eab2d0 0x55c205eb1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d13fff640 1 -- 172.21.15.82:0/1069647423 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8d0c0d0270 con 0x7f8d1c102990 2024-04-21T08:46:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1a7fc640 1 -- 172.21.15.82:0/3182234352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eaa830 msgr2=0x55c205eaacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/3182234352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eaa830 0x55c205eaacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1a7fc640 1 -- 172.21.15.82:0/3182234352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eab2d0 msgr2=0x55c205eb1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/3182234352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eab2d0 0x55c205eb1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1a7fc640 1 -- 172.21.15.82:0/3182234352 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c205b212d0 con 0x55c205ea9e40 2024-04-21T08:46:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/3182234352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 0x55c205eaa260 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009230 tx=0x7f8d04005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f7ffba698dbe4b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d1b7fe640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8d080a3c40 0x7f8d080a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c098510 tx=0x7f8d0c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d117fa640 1 -- 172.21.15.82:0/3182234352 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04004030 con 0x55c205ea9e40 2024-04-21T08:46:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d117fa640 1 -- 172.21.15.82:0/3182234352 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d04048070 con 0x55c205ea9e40 2024-04-21T08:46:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.274+0000 7f8d117fa640 1 -- 172.21.15.82:0/3182234352 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04004950 con 0x55c205ea9e40 2024-04-21T08:46:26.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.275+0000 7f8d24176740 1 -- 172.21.15.82:0/3182234352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 msgr2=0x55c205eaa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.275+0000 7f8d24176740 1 --2- 172.21.15.82:0/3182234352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 0x55c205eaa260 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009230 tx=0x7f8d04005d90 comp rx=0 tx=0).stop 2024-04-21T08:46:26.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.275+0000 7f8d24176740 1 -- 172.21.15.82:0/3182234352 shutdown_connections 2024-04-21T08:46:26.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.275+0000 7f8d24176740 1 --2- 172.21.15.82:0/3182234352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eab2d0 0x55c205eb1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.275+0000 7f8d24176740 1 --2- 172.21.15.82:0/3182234352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eaa830 0x55c205eaacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.275+0000 7f8d24176740 1 --2- 172.21.15.82:0/3182234352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 0x55c205eaa260 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.275+0000 7f8d24176740 1 -- 172.21.15.82:0/3182234352 >> 172.21.15.82:0/3182234352 conn(0x55c205ea5f40 msgr2=0x55c205ea4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.275+0000 7f8d24176740 1 -- 172.21.15.82:0/3182234352 shutdown_connections 2024-04-21T08:46:26.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.275+0000 7f8d24176740 1 -- 172.21.15.82:0/3182234352 wait complete. 2024-04-21T08:46:26.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.275+0000 7f8d24176740 1 -- start start 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.275+0000 7f8d24176740 1 Processor -- start 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d24176740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 0x55c205eb3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d24176740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eaa830 0x55c205eb43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d24176740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eab2d0 0x55c205ebb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d24176740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c205eb2450 con 0x55c205ea9e40 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d24176740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c205d97190 con 0x55c205eaa830 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d24176740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c205eb25d0 con 0x55c205eab2d0 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d24176740 10 client.0 _ll_register_callbacks cb 0x55c205d928b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d1a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 0x55c205eb3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d1a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 0x55c205eb3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36516/0 (socket says 172.21.15.82:36516) 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d1a7fc640 1 -- 172.21.15.82:0/961235161 learned_addr learned my addr 172.21.15.82:0/961235161 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d1affd640 1 --2- 172.21.15.82:0/961235161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eaa830 0x55c205eb43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d1a7fc640 1 -- 172.21.15.82:0/961235161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eab2d0 msgr2=0x55c205ebb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/961235161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eab2d0 0x55c205ebb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d1a7fc640 1 -- 172.21.15.82:0/961235161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eaa830 msgr2=0x55c205eb43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/961235161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eaa830 0x55c205eb43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d1a7fc640 1 -- 172.21.15.82:0/961235161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c205beb6b0 con 0x55c205ea9e40 2024-04-21T08:46:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/961235161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 0x55c205eb3e60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009690 tx=0x7f8d04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbf7a0908cf4730 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d027fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d027fc640 1 -- 172.21.15.82:0/961235161 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040038b0 con 0x55c205ea9e40 2024-04-21T08:46:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d027fc640 1 -- 172.21.15.82:0/961235161 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8d04003a50 con 0x55c205ea9e40 2024-04-21T08:46:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.276+0000 7f8d24176740 1 -- 172.21.15.82:0/961235161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d04048070 con 0x55c205ea9e40 2024-04-21T08:46:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.277+0000 7f8d027fc640 1 -- 172.21.15.82:0/961235161 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d0405d050 con 0x55c205ea9e40 2024-04-21T08:46:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.277+0000 7f8d24176740 10 client.15273 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.277+0000 7f8d24176740 1 -- 172.21.15.82:0/961235161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c205ebe750 con 0x55c205ea9e40 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.277+0000 7f8d027fc640 1 -- 172.21.15.82:0/961235161 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d04048070 con 0x55c205ea9e40 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.277+0000 7f8d027fc640 1 -- 172.21.15.82:0/961235161 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8d04040040 con 0x55c205ea9e40 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.277+0000 7f8d24176740 10 client.15273 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.277+0000 7f8d24176740 10 client.15273 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.277+0000 7f8d24176740 20 client.15273 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.277+0000 7f8d24176740 1 --2- 172.21.15.82:0/961235161 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55c205ec0970 0x55c205ec1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.277+0000 7f8d24176740 4 client.15273 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.277+0000 7f8d24176740 1 -- 172.21.15.82:0/961235161 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c205e1c530 con 0x55c205ec0970 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.277+0000 7f8d1affd640 1 --2- 172.21.15.82:0/961235161 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55c205ec0970 0x55c205ec1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d027fc640 10 client.15273 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d1affd640 1 --2- 172.21.15.82:0/961235161 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55c205ec0970 0x55c205ec1de0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c3c7a57dc172686 server_cookie=525921f36a24c5f4 in_seq=0 out_seq=0 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d027fc640 1 -- 172.21.15.82:0/961235161 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c205e1c530 con 0x55c205ec0970 2024-04-21T08:46:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d027fc640 10 client.15273 handle_command_reply: tid=0 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d24176740 1 client.15273 shutdown 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d24176740 1 -- 172.21.15.82:0/961235161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 msgr2=0x55c205eb3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d24176740 1 --2- 172.21.15.82:0/961235161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 0x55c205eb3e60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009690 tx=0x7f8d04004270 comp rx=0 tx=0).stop 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d24176740 1 -- 172.21.15.82:0/961235161 shutdown_connections 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d24176740 1 --2- 172.21.15.82:0/961235161 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55c205ec0970 0x55c205ec1de0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d24176740 1 --2- 172.21.15.82:0/961235161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eab2d0 0x55c205ebb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d24176740 1 --2- 172.21.15.82:0/961235161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eaa830 0x55c205eb43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d027fc640 0 client.15273 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d24176740 1 --2- 172.21.15.82:0/961235161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205ea9e40 0x55c205eb3e60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.278+0000 7f8d24176740 1 -- 172.21.15.82:0/961235161 >> 172.21.15.82:0/961235161 conn(0x55c205ea5f40 msgr2=0x55c205ea3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.279+0000 7f8d24176740 1 -- 172.21.15.82:0/961235161 shutdown_connections 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.279+0000 7f8d24176740 1 -- 172.21.15.82:0/961235161 wait complete. 2024-04-21T08:46:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.279+0000 7f8d24176740 20 client.15273 trim_cache size 0 max 16384 2024-04-21T08:46:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.299+0000 7f8d24176740 1 Processor -- start 2024-04-21T08:46:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.299+0000 7f8d24176740 1 -- start start 2024-04-21T08:46:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d24176740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205ea9e40 0x55c205e9f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d24176740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eaa830 0x55c205ea4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d24176740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eab2d0 0x55c205ea4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d24176740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c205ec0120 con 0x55c205eaa830 2024-04-21T08:46:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d24176740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c205ec07f0 con 0x55c205eab2d0 2024-04-21T08:46:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d24176740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c205ec0ec0 con 0x55c205ea9e40 2024-04-21T08:46:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d1affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eaa830 0x55c205ea4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d1affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eaa830 0x55c205ea4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36522/0 (socket says 172.21.15.82:36522) 2024-04-21T08:46:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d1affd640 1 -- 172.21.15.82:0/2193416166 learned_addr learned my addr 172.21.15.82:0/2193416166 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/2193416166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eab2d0 0x55c205ea4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d1affd640 1 -- 172.21.15.82:0/2193416166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205ea9e40 msgr2=0x55c205e9f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d1affd640 1 --2- 172.21.15.82:0/2193416166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205ea9e40 0x55c205e9f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d1affd640 1 -- 172.21.15.82:0/2193416166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eab2d0 msgr2=0x55c205ea4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d1affd640 1 --2- 172.21.15.82:0/2193416166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eab2d0 0x55c205ea4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.300+0000 7f8d1affd640 1 -- 172.21.15.82:0/2193416166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c205ea5320 con 0x55c205eaa830 2024-04-21T08:46:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.301+0000 7f8d1affd640 1 --2- 172.21.15.82:0/2193416166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eaa830 0x55c205ea4350 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x55c205ec2770 tx=0x7f8d140098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b5d414dc16c87d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.302+0000 7f8d10ff9640 1 -- 172.21.15.82:0/2193416166 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400a6c0 con 0x55c205eaa830 2024-04-21T08:46:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.302+0000 7f8d10ff9640 1 -- 172.21.15.82:0/2193416166 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d14022030 con 0x55c205eaa830 2024-04-21T08:46:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.302+0000 7f8d10ff9640 1 -- 172.21.15.82:0/2193416166 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14018b40 con 0x55c205eaa830 2024-04-21T08:46:26.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.303+0000 7f8d24176740 1 -- 172.21.15.82:0/2193416166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eaa830 msgr2=0x55c205ea4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.303+0000 7f8d24176740 1 --2- 172.21.15.82:0/2193416166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eaa830 0x55c205ea4350 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x55c205ec2770 tx=0x7f8d140098d0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.310+0000 7f8d24176740 1 -- 172.21.15.82:0/2193416166 shutdown_connections 2024-04-21T08:46:26.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.310+0000 7f8d24176740 1 --2- 172.21.15.82:0/2193416166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205eab2d0 0x55c205ea4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.310+0000 7f8d24176740 1 --2- 172.21.15.82:0/2193416166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eaa830 0x55c205ea4350 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.310+0000 7f8d24176740 1 --2- 172.21.15.82:0/2193416166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205ea9e40 0x55c205e9f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.310+0000 7f8d24176740 1 -- 172.21.15.82:0/2193416166 >> 172.21.15.82:0/2193416166 conn(0x55c205ea5f40 msgr2=0x55c205eae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.311+0000 7f8d24176740 1 -- 172.21.15.82:0/2193416166 shutdown_connections 2024-04-21T08:46:26.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.311+0000 7f8d24176740 1 -- 172.21.15.82:0/2193416166 wait complete. 2024-04-21T08:46:26.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.312+0000 7f8d24176740 1 -- start start 2024-04-21T08:46:26.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.312+0000 7f8d24176740 1 Processor -- start 2024-04-21T08:46:26.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.312+0000 7f8d24176740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205ea9e40 0x55c205ec9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.312+0000 7f8d24176740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eaa830 0x55c205eca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.312+0000 7f8d24176740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eab2d0 0x55c205eabe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.312+0000 7f8d24176740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c205ec1c60 con 0x55c205eab2d0 2024-04-21T08:46:26.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.312+0000 7f8d24176740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c205ec1590 con 0x55c205ea9e40 2024-04-21T08:46:26.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.313+0000 7f8d24176740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c205ec2330 con 0x55c205eaa830 2024-04-21T08:46:26.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.313+0000 7f8d24176740 10 client.0 _ll_register_callbacks cb 0x7f8d14009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.313+0000 7f8d1a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205ea9e40 0x55c205ec9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.313+0000 7f8d1a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205ea9e40 0x55c205ec9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34778/0 (socket says 172.21.15.82:34778) 2024-04-21T08:46:26.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.313+0000 7f8d1a7fc640 1 -- 172.21.15.82:0/611157854 learned_addr learned my addr 172.21.15.82:0/611157854 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.313+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/611157854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eab2d0 0x55c205eabe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.313+0000 7f8d1a7fc640 1 -- 172.21.15.82:0/611157854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eaa830 msgr2=0x55c205eca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.313+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/611157854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eaa830 0x55c205eca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.313+0000 7f8d1a7fc640 1 -- 172.21.15.82:0/611157854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205ea9e40 msgr2=0x55c205ec9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.313+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/611157854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205ea9e40 0x55c205ec9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.313+0000 7f8d1a7fc640 1 -- 172.21.15.82:0/611157854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c205eb9390 con 0x55c205eab2d0 2024-04-21T08:46:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.314+0000 7f8d1a7fc640 1 --2- 172.21.15.82:0/611157854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eab2d0 0x55c205eabe80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8d0402c170 tx=0x7f8d04005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23035bf4d7075963 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.316+0000 7f8d027fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.316+0000 7f8d027fc640 1 -- 172.21.15.82:0/611157854 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0402c8e0 con 0x55c205eab2d0 2024-04-21T08:46:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.316+0000 7f8d027fc640 1 -- 172.21.15.82:0/611157854 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8d04034ca0 con 0x55c205eab2d0 2024-04-21T08:46:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.316+0000 7f8d24176740 1 -- 172.21.15.82:0/611157854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d14022030 con 0x55c205eab2d0 2024-04-21T08:46:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.317+0000 7f8d027fc640 1 -- 172.21.15.82:0/611157854 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d0404c880 con 0x55c205eab2d0 2024-04-21T08:46:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.317+0000 7f8d24176740 10 client.15279 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:26.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.317+0000 7f8d24176740 1 -- 172.21.15.82:0/611157854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c205f24910 con 0x55c205eab2d0 2024-04-21T08:46:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.320+0000 7f8d027fc640 1 -- 172.21.15.82:0/611157854 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d04048040 con 0x55c205eab2d0 2024-04-21T08:46:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.320+0000 7f8d027fc640 1 -- 172.21.15.82:0/611157854 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8d040345d0 con 0x55c205eab2d0 2024-04-21T08:46:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.320+0000 7f8d24176740 10 client.15279 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.320+0000 7f8d24176740 10 client.15279 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:46:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.320+0000 7f8d24176740 20 client.15279 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.320+0000 7f8d24176740 1 --2- 172.21.15.82:0/611157854 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55c205f29560 0x55c205f28270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.320+0000 7f8d24176740 4 client.15279 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:46:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.320+0000 7f8d24176740 1 -- 172.21.15.82:0/611157854 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "status"}) -- 0x7f8d14022030 con 0x55c205f29560 2024-04-21T08:46:26.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.321+0000 7f8d1affd640 1 --2- 172.21.15.82:0/611157854 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55c205f29560 0x55c205f28270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.322+0000 7f8d1affd640 1 --2- 172.21.15.82:0/611157854 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55c205f29560 0x55c205f28270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fb06f88521575e4 server_cookie=d788cc294e07ae12 in_seq=0 out_seq=0 2024-04-21T08:46:26.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.322+0000 7f8d027fc640 10 client.15279 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:46:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.323+0000 7f8d027fc640 1 -- 172.21.15.82:0/611157854 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x7f8d14022030 con 0x55c205f29560 2024-04-21T08:46:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.323+0000 7f8d027fc640 10 client.15279 handle_command_reply: tid=0 2024-04-21T08:46:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.323+0000 7f8d24176740 1 client.15279 shutdown 2024-04-21T08:46:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.324+0000 7f8d24176740 1 -- 172.21.15.82:0/611157854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eab2d0 msgr2=0x55c205eabe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.324+0000 7f8d24176740 1 --2- 172.21.15.82:0/611157854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eab2d0 0x55c205eabe80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8d0402c170 tx=0x7f8d04005fd0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.324+0000 7f8d24176740 1 -- 172.21.15.82:0/611157854 shutdown_connections 2024-04-21T08:46:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.324+0000 7f8d24176740 1 --2- 172.21.15.82:0/611157854 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55c205f29560 0x55c205f28270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.324+0000 7f8d24176740 1 --2- 172.21.15.82:0/611157854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c205eab2d0 0x55c205eabe80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.324+0000 7f8d24176740 1 --2- 172.21.15.82:0/611157854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c205eaa830 0x55c205eca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.324+0000 7f8d24176740 1 --2- 172.21.15.82:0/611157854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c205ea9e40 0x55c205ec9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.324+0000 7f8d24176740 1 -- 172.21.15.82:0/611157854 >> 172.21.15.82:0/611157854 conn(0x55c205ea5f40 msgr2=0x55c205ea4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.324+0000 7f8d027fc640 0 client.15279 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:46:26.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.324+0000 7f8d24176740 1 -- 172.21.15.82:0/611157854 shutdown_connections 2024-04-21T08:46:26.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.325+0000 7f8d24176740 1 -- 172.21.15.82:0/611157854 wait complete. 2024-04-21T08:46:26.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.325+0000 7f8d24176740 20 client.15279 trim_cache size 0 max 16384 2024-04-21T08:46:26.325 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:46:26.325 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:46:26.325 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 2, 2024-04-21T08:46:26.326 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24476, 2024-04-21T08:46:26.326 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:46:26.326 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:46:26.326 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:46:26.326 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 86.871576734000001, 2024-04-21T08:46:26.326 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2024-04-21T08:46:26.326 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 91, 2024-04-21T08:46:26.326 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:46:26.326 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 129.198541138 2024-04-21T08:46:26.326 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:46:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.327+0000 7f8d214b2640 1 -- 172.21.15.82:0/1069647423 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8d080a3c40 msgr2=0x7f8d080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.327+0000 7f8d214b2640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8d080a3c40 0x7f8d080a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c098510 tx=0x7f8d0c000bc0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.327+0000 7f8d214b2640 1 -- 172.21.15.82:0/1069647423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c102990 msgr2=0x7f8d1c19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.327+0000 7f8d214b2640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c102990 0x7f8d1c19f830 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00d750 tx=0x7f8d0c00dc10 comp rx=0 tx=0).stop 2024-04-21T08:46:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.328+0000 7f8d214b2640 1 -- 172.21.15.82:0/1069647423 shutdown_connections 2024-04-21T08:46:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.328+0000 7f8d214b2640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8d080a3c40 0x7f8d080a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.328+0000 7f8d214b2640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d1c102990 0x7f8d1c19f830 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.328+0000 7f8d214b2640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c101fd0 0x7f8d1c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.328+0000 7f8d214b2640 1 --2- 172.21.15.82:0/1069647423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d1c100e80 0x7f8d1c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.328+0000 7f8d214b2640 1 -- 172.21.15.82:0/1069647423 >> 172.21.15.82:0/1069647423 conn(0x7f8d1c0fc3f0 msgr2=0x7f8d1c06a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.328+0000 7f8d214b2640 1 -- 172.21.15.82:0/1069647423 shutdown_connections 2024-04-21T08:46:26.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.328+0000 7f8d214b2640 1 -- 172.21.15.82:0/1069647423 wait complete. 2024-04-21T08:46:26.347 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T08:46:26.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.448+0000 7f900afd5640 1 -- 172.21.15.82:0/2012906176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9004103190 msgr2=0x7f9004103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.448+0000 7f900afd5640 1 --2- 172.21.15.82:0/2012906176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9004103190 0x7f9004103610 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009900 tx=0x7f8ff002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:26.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.448+0000 7f900afd5640 1 -- 172.21.15.82:0/2012906176 shutdown_connections 2024-04-21T08:46:26.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.448+0000 7f900afd5640 1 --2- 172.21.15.82:0/2012906176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9004103b50 0x7f900410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.448+0000 7f900afd5640 1 --2- 172.21.15.82:0/2012906176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9004103190 0x7f9004103610 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.448+0000 7f900afd5640 1 --2- 172.21.15.82:0/2012906176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9004101fa0 0x7f90041023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.448+0000 7f900afd5640 1 -- 172.21.15.82:0/2012906176 >> 172.21.15.82:0/2012906176 conn(0x7f90040fd530 msgr2=0x7f90040ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.448+0000 7f900afd5640 1 -- 172.21.15.82:0/2012906176 shutdown_connections 2024-04-21T08:46:26.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.448+0000 7f900afd5640 1 -- 172.21.15.82:0/2012906176 wait complete. 2024-04-21T08:46:26.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.448+0000 7f900afd5640 1 Processor -- start 2024-04-21T08:46:26.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.448+0000 7f900afd5640 1 -- start start 2024-04-21T08:46:26.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f900afd5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9004101fa0 0x7f90041983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f900afd5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9004103190 0x7f90041988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f900afd5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9004103b50 0x7f900419f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f900afd5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f900410dbc0 con 0x7f9004103190 2024-04-21T08:46:26.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f900afd5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f900410da40 con 0x7f9004103b50 2024-04-21T08:46:26.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f900afd5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f900410dd40 con 0x7f9004101fa0 2024-04-21T08:46:26.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9003fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9004103190 0x7f90041988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9003fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9004103190 0x7f90041988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36552/0 (socket says 172.21.15.82:36552) 2024-04-21T08:46:26.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9003fff640 1 -- 172.21.15.82:0/281793506 learned_addr learned my addr 172.21.15.82:0/281793506 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9008d71640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9004101fa0 0x7f90041983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9009572640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9004103b50 0x7f900419f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9003fff640 1 -- 172.21.15.82:0/281793506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9004101fa0 msgr2=0x7f90041983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9003fff640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9004101fa0 0x7f90041983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9003fff640 1 -- 172.21.15.82:0/281793506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9004103b50 msgr2=0x7f900419f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9003fff640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9004103b50 0x7f900419f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9003fff640 1 -- 172.21.15.82:0/281793506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900419ffd0 con 0x7f9004103190 2024-04-21T08:46:26.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9003fff640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9004103190 0x7f90041988f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0036040 tx=0x7f8ff00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9001ffb640 1 -- 172.21.15.82:0/281793506 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0043020 con 0x7f9004103190 2024-04-21T08:46:26.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9001ffb640 1 -- 172.21.15.82:0/281793506 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ff00054c0 con 0x7f9004103190 2024-04-21T08:46:26.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f900afd5640 1 -- 172.21.15.82:0/281793506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90041a0260 con 0x7f9004103190 2024-04-21T08:46:26.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.449+0000 7f9001ffb640 1 -- 172.21.15.82:0/281793506 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0003d40 con 0x7f9004103190 2024-04-21T08:46:26.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.450+0000 7f900afd5640 1 -- 172.21.15.82:0/281793506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90041a0830 con 0x7f9004103190 2024-04-21T08:46:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f9001ffb640 1 -- 172.21.15.82:0/281793506 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8ff0046020 con 0x7f9004103190 2024-04-21T08:46:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f900dbc7740 1 Processor -- start 2024-04-21T08:46:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f900dbc7740 1 -- start start 2024-04-21T08:46:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f900dbc7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc27e40 0x5576bfc28260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f900dbc7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc28cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f900dbc7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc292d0 0x5576bfc2feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f900dbc7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5576bfb8cb10 con 0x5576bfc292d0 2024-04-21T08:46:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f900dbc7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5576bfb2a240 con 0x5576bfc28830 2024-04-21T08:46:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f900dbc7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576bfb88ad0 con 0x5576bfc27e40 2024-04-21T08:46:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f9003fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc292d0 0x5576bfc2feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f9003fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc292d0 0x5576bfc2feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36564/0 (socket says 172.21.15.82:36564) 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f9003fff640 1 -- 172.21.15.82:0/2692931880 learned_addr learned my addr 172.21.15.82:0/2692931880 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.452+0000 7f9001ffb640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8ff40a3c40 0x7f8ff40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9009572640 1 --2- 172.21.15.82:0/2692931880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc27e40 0x5576bfc28260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9008d71640 1 --2- 172.21.15.82:0/2692931880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc28cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9009572640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8ff40a3c40 0x7f8ff40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9001ffb640 1 -- 172.21.15.82:0/281793506 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8ff00f2bd0 con 0x7f9004103190 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9003fff640 1 -- 172.21.15.82:0/2692931880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc27e40 msgr2=0x5576bfc28260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9003fff640 1 --2- 172.21.15.82:0/2692931880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc27e40 0x5576bfc28260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9003fff640 1 -- 172.21.15.82:0/2692931880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 msgr2=0x5576bfc28cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9003fff640 1 --2- 172.21.15.82:0/2692931880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc28cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9003fff640 1 -- 172.21.15.82:0/2692931880 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576bf89f2d0 con 0x5576bfc292d0 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9009572640 1 --2- 172.21.15.82:0/2692931880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc27e40 0x5576bfc28260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9008d71640 1 --2- 172.21.15.82:0/2692931880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc28cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9003fff640 1 --2- 172.21.15.82:0/2692931880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc292d0 0x5576bfc2feb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00f02c0 tx=0x7f8ff0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336a0a0b1fcb4037 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f8feeffd640 1 -- 172.21.15.82:0/2692931880 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00be660 con 0x5576bfc292d0 2024-04-21T08:46:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f9009572640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8ff40a3c40 0x7f8ff40a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009fd0 tx=0x7f8ffc009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f8feeffd640 1 -- 172.21.15.82:0/2692931880 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ff00bf220 con 0x5576bfc292d0 2024-04-21T08:46:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.453+0000 7f8feeffd640 1 -- 172.21.15.82:0/2692931880 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00044e0 con 0x5576bfc292d0 2024-04-21T08:46:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.454+0000 7f900dbc7740 1 -- 172.21.15.82:0/2692931880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc292d0 msgr2=0x5576bfc2feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.454+0000 7f900dbc7740 1 --2- 172.21.15.82:0/2692931880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc292d0 0x5576bfc2feb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00f02c0 tx=0x7f8ff0009260 comp rx=0 tx=0).stop 2024-04-21T08:46:26.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.454+0000 7f900dbc7740 1 -- 172.21.15.82:0/2692931880 shutdown_connections 2024-04-21T08:46:26.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.454+0000 7f900dbc7740 1 --2- 172.21.15.82:0/2692931880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc292d0 0x5576bfc2feb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.454+0000 7f900dbc7740 1 --2- 172.21.15.82:0/2692931880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc28cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.454+0000 7f900dbc7740 1 --2- 172.21.15.82:0/2692931880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc27e40 0x5576bfc28260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.454+0000 7f900dbc7740 1 -- 172.21.15.82:0/2692931880 >> 172.21.15.82:0/2692931880 conn(0x5576bfc23f40 msgr2=0x5576bfc22490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.454+0000 7f900dbc7740 1 -- 172.21.15.82:0/2692931880 shutdown_connections 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.454+0000 7f900dbc7740 1 -- 172.21.15.82:0/2692931880 wait complete. 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.454+0000 7f900dbc7740 1 -- start start 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.454+0000 7f900dbc7740 1 Processor -- start 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f900dbc7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc27e40 0x5576bfc31b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f900dbc7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc28830 0x5576bfc32050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f900dbc7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc292d0 0x5576bfc39090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f900dbc7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc29ad0 con 0x5576bfc28830 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f900dbc7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5576bfb15190 con 0x5576bfc292d0 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f900dbc7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc303f0 con 0x5576bfc27e40 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f900dbc7740 10 client.0 _ll_register_callbacks cb 0x5576bfb108b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f9003fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc28830 0x5576bfc32050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f9003fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc28830 0x5576bfc32050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36580/0 (socket says 172.21.15.82:36580) 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f9003fff640 1 -- 172.21.15.82:0/554484682 learned_addr learned my addr 172.21.15.82:0/554484682 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f9008d71640 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc27e40 0x5576bfc31b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f9009572640 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc292d0 0x5576bfc39090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f9003fff640 1 -- 172.21.15.82:0/554484682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc27e40 msgr2=0x5576bfc31b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f9003fff640 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc27e40 0x5576bfc31b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f9003fff640 1 -- 172.21.15.82:0/554484682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc292d0 msgr2=0x5576bfc39090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f9003fff640 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc292d0 0x5576bfc39090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f9003fff640 1 -- 172.21.15.82:0/554484682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576bf9696b0 con 0x5576bfc28830 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f9009572640 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc292d0 0x5576bfc39090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f9003fff640 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc28830 0x5576bfc32050 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0005ec0 tx=0x7f8ff0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef00467a3a49a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f8fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f8fe3fff640 1 -- 172.21.15.82:0/554484682 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00d8060 con 0x5576bfc28830 2024-04-21T08:46:26.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f8fe3fff640 1 -- 172.21.15.82:0/554484682 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8ff0004c00 con 0x5576bfc28830 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f900dbc7740 1 -- 172.21.15.82:0/554484682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ff00bf220 con 0x5576bfc28830 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f8fe3fff640 1 -- 172.21.15.82:0/554484682 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8ff00be480 con 0x5576bfc28830 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f900dbc7740 10 client.15291 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.455+0000 7f900dbc7740 1 -- 172.21.15.82:0/554484682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576bfc3c370 con 0x5576bfc28830 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.456+0000 7f8fe3fff640 1 -- 172.21.15.82:0/554484682 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ff00bf220 con 0x5576bfc28830 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.456+0000 7f8fe3fff640 1 -- 172.21.15.82:0/554484682 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8ff00ddce0 con 0x5576bfc28830 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.456+0000 7f900dbc7740 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.456+0000 7f900dbc7740 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.456+0000 7f900dbc7740 20 client.15291 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.456+0000 7f900dbc7740 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5576bfc3e600 0x5576bfc3fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.456+0000 7f900dbc7740 4 client.15291 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.456+0000 7f900dbc7740 1 -- 172.21.15.82:0/554484682 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576bfb9a530 con 0x5576bfc3e600 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.456+0000 7f9008d71640 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5576bfc3e600 0x5576bfc3fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.456+0000 7f8fe3fff640 10 client.15291 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.456+0000 7f9008d71640 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5576bfc3e600 0x5576bfc3fa70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad5c8588f516739e server_cookie=967829087bbd9bd8 in_seq=0 out_seq=0 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f8fe3fff640 1 -- 172.21.15.82:0/554484682 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576bfb9a530 con 0x5576bfc3e600 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f8fe3fff640 10 client.15291 handle_command_reply: tid=0 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f900dbc7740 1 client.15291 shutdown 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f900dbc7740 1 -- 172.21.15.82:0/554484682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc28830 msgr2=0x5576bfc32050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f900dbc7740 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc28830 0x5576bfc32050 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0005ec0 tx=0x7f8ff0003f60 comp rx=0 tx=0).stop 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f900dbc7740 1 -- 172.21.15.82:0/554484682 shutdown_connections 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f900dbc7740 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5576bfc3e600 0x5576bfc3fa70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f900dbc7740 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc292d0 0x5576bfc39090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f8fe3fff640 0 client.15291 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f900dbc7740 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc28830 0x5576bfc32050 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f900dbc7740 1 --2- 172.21.15.82:0/554484682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc27e40 0x5576bfc31b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f900dbc7740 1 -- 172.21.15.82:0/554484682 >> 172.21.15.82:0/554484682 conn(0x5576bfc23f40 msgr2=0x5576bfc21a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.457+0000 7f900dbc7740 1 -- 172.21.15.82:0/554484682 shutdown_connections 2024-04-21T08:46:26.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.458+0000 7f900dbc7740 1 -- 172.21.15.82:0/554484682 wait complete. 2024-04-21T08:46:26.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.458+0000 7f900dbc7740 20 client.15291 trim_cache size 0 max 16384 2024-04-21T08:46:26.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.479+0000 7f900dbc7740 1 Processor -- start 2024-04-21T08:46:26.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.480+0000 7f900dbc7740 1 -- start start 2024-04-21T08:46:26.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.481+0000 7f900dbc7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc27e40 0x5576bfc3a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.481+0000 7f900dbc7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc39050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.481+0000 7f900dbc7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc292d0 0x5576bfc21170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.481+0000 7f900dbc7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc3ddb0 con 0x5576bfc27e40 2024-04-21T08:46:26.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.481+0000 7f900dbc7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc3e480 con 0x5576bfc28830 2024-04-21T08:46:26.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.481+0000 7f900dbc7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc3eb50 con 0x5576bfc292d0 2024-04-21T08:46:26.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.483+0000 7f9009572640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc292d0 0x5576bfc21170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.483+0000 7f9009572640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc292d0 0x5576bfc21170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:43156/0 (socket says 172.21.15.82:43156) 2024-04-21T08:46:26.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.483+0000 7f9009572640 1 -- 172.21.15.82:0/2612083223 learned_addr learned my addr 172.21.15.82:0/2612083223 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.485+0000 7f9003fff640 1 --2- 172.21.15.82:0/2612083223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc39050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.486+0000 7f9003fff640 1 -- 172.21.15.82:0/2612083223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc292d0 msgr2=0x5576bfc21170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.486+0000 7f9003fff640 1 --2- 172.21.15.82:0/2612083223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc292d0 0x5576bfc21170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.486+0000 7f9003fff640 1 -- 172.21.15.82:0/2612083223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc27e40 msgr2=0x5576bfc3a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.486+0000 7f9003fff640 1 --2- 172.21.15.82:0/2612083223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc27e40 0x5576bfc3a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.486+0000 7f9003fff640 1 -- 172.21.15.82:0/2612083223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576bfc23630 con 0x5576bfc28830 2024-04-21T08:46:26.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.487+0000 7f9001ffb640 1 -- 172.21.15.82:0/281793506 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8ff00f8070 con 0x7f9004103190 2024-04-21T08:46:26.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.487+0000 7f9003fff640 1 --2- 172.21.15.82:0/2612083223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc39050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0005ab0 tx=0x7f8ff00bd030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=890c2caed8648e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.487+0000 7f8fee7fc640 1 -- 172.21.15.82:0/2612083223 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0003d40 con 0x5576bfc28830 2024-04-21T08:46:26.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.487+0000 7f8fee7fc640 1 -- 172.21.15.82:0/2612083223 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ff0100020 con 0x5576bfc28830 2024-04-21T08:46:26.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.487+0000 7f8fee7fc640 1 -- 172.21.15.82:0/2612083223 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00fb3f0 con 0x5576bfc28830 2024-04-21T08:46:26.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.488+0000 7f900dbc7740 1 -- 172.21.15.82:0/2612083223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 msgr2=0x5576bfc39050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.488+0000 7f900dbc7740 1 --2- 172.21.15.82:0/2612083223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc39050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0005ab0 tx=0x7f8ff00bd030 comp rx=0 tx=0).stop 2024-04-21T08:46:26.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.488+0000 7f900dbc7740 1 -- 172.21.15.82:0/2612083223 shutdown_connections 2024-04-21T08:46:26.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.488+0000 7f900dbc7740 1 --2- 172.21.15.82:0/2612083223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc292d0 0x5576bfc21170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.488+0000 7f900dbc7740 1 --2- 172.21.15.82:0/2612083223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc39050 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.488+0000 7f900dbc7740 1 --2- 172.21.15.82:0/2612083223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc27e40 0x5576bfc3a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.488+0000 7f900dbc7740 1 -- 172.21.15.82:0/2612083223 >> 172.21.15.82:0/2612083223 conn(0x5576bfc23f40 msgr2=0x5576bfc39ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.489+0000 7f900dbc7740 1 -- 172.21.15.82:0/2612083223 shutdown_connections 2024-04-21T08:46:26.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.489+0000 7f900dbc7740 1 -- 172.21.15.82:0/2612083223 wait complete. 2024-04-21T08:46:26.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.489+0000 7f900dbc7740 1 -- start start 2024-04-21T08:46:26.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.489+0000 7f900dbc7740 1 Processor -- start 2024-04-21T08:46:26.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f900dbc7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc27e40 0x5576bfc34b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f900dbc7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f900dbc7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc292d0 0x5576bfc38050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f900dbc7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc3f8f0 con 0x5576bfc27e40 2024-04-21T08:46:26.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f900dbc7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc3f220 con 0x5576bfc28830 2024-04-21T08:46:26.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f900dbc7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5576bfc3ffc0 con 0x5576bfc292d0 2024-04-21T08:46:26.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f900dbc7740 10 client.0 _ll_register_callbacks cb 0x7f8ff0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f9008d71640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc27e40 0x5576bfc34b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f9003fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f9003fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc350a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34806/0 (socket says 172.21.15.82:34806) 2024-04-21T08:46:26.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f9008d71640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc27e40 0x5576bfc34b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36602/0 (socket says 172.21.15.82:36602) 2024-04-21T08:46:26.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f9003fff640 1 -- 172.21.15.82:0/1582011080 learned_addr learned my addr 172.21.15.82:0/1582011080 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.490+0000 7f9009572640 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc292d0 0x5576bfc38050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f9003fff640 1 -- 172.21.15.82:0/1582011080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc292d0 msgr2=0x5576bfc38050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f9003fff640 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc292d0 0x5576bfc38050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f9003fff640 1 -- 172.21.15.82:0/1582011080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc27e40 msgr2=0x5576bfc34b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f9003fff640 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc27e40 0x5576bfc34b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f9003fff640 1 -- 172.21.15.82:0/1582011080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576bfc38640 con 0x5576bfc28830 2024-04-21T08:46:26.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f9008d71640 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc27e40 0x5576bfc34b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:26.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f9003fff640 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc350a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00f0380 tx=0x7f8ff0004b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c07fa1a2f7f0d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f8fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:46:26.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f8fe3fff640 1 -- 172.21.15.82:0/1582011080 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00ff070 con 0x5576bfc28830 2024-04-21T08:46:26.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f8fe3fff640 1 -- 172.21.15.82:0/1582011080 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9216+0+0 (secure 0 0 0) 0x7f8ff0004c00 con 0x5576bfc28830 2024-04-21T08:46:26.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f900dbc7740 1 -- 172.21.15.82:0/1582011080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ff0100020 con 0x5576bfc28830 2024-04-21T08:46:26.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f8fe3fff640 1 -- 172.21.15.82:0/1582011080 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8ff00741a0 con 0x5576bfc28830 2024-04-21T08:46:26.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f900dbc7740 10 client.24923 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:26.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f900dbc7740 1 -- 172.21.15.82:0/1582011080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5576bfca4930 con 0x5576bfc28830 2024-04-21T08:46:26.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f8fe3fff640 1 -- 172.21.15.82:0/1582011080 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ff0100020 con 0x5576bfc28830 2024-04-21T08:46:26.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f8fe3fff640 1 -- 172.21.15.82:0/1582011080 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8ff00ddce0 con 0x5576bfc28830 2024-04-21T08:46:26.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f900dbc7740 10 client.24923 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:26.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f900dbc7740 10 client.24923 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:46:26.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.491+0000 7f900dbc7740 20 client.24923 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:26.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.492+0000 7f900dbc7740 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5576bfca8d60 0x5576bfca7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.492+0000 7f900dbc7740 4 client.24923 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:46:26.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.492+0000 7f900dbc7740 1 -- 172.21.15.82:0/1582011080 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "status"}) -- 0x5576bfb943a0 con 0x5576bfca8d60 2024-04-21T08:46:26.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.492+0000 7f9008d71640 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5576bfca8d60 0x5576bfca7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f9008d71640 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5576bfca8d60 0x5576bfca7a70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=635d5c02ff13d009 server_cookie=10b4e4f0657191cf in_seq=0 out_seq=0 2024-04-21T08:46:26.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f8fe3fff640 10 client.24923 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:46:26.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f8fe3fff640 1 -- 172.21.15.82:0/1582011080 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5576bfb943a0 con 0x5576bfca8d60 2024-04-21T08:46:26.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f8fe3fff640 10 client.24923 handle_command_reply: tid=0 2024-04-21T08:46:26.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f900dbc7740 1 client.24923 shutdown 2024-04-21T08:46:26.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f900dbc7740 1 -- 172.21.15.82:0/1582011080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 msgr2=0x5576bfc350a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f900dbc7740 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc350a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00f0380 tx=0x7f8ff0004b70 comp rx=0 tx=0).stop 2024-04-21T08:46:26.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f900dbc7740 1 -- 172.21.15.82:0/1582011080 shutdown_connections 2024-04-21T08:46:26.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f900dbc7740 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5576bfca8d60 0x5576bfca7a70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f900dbc7740 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576bfc292d0 0x5576bfc38050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f900dbc7740 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576bfc28830 0x5576bfc350a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f900dbc7740 1 --2- 172.21.15.82:0/1582011080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576bfc27e40 0x5576bfc34b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f8fe3fff640 0 client.24923 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:46:26.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f900dbc7740 1 -- 172.21.15.82:0/1582011080 >> 172.21.15.82:0/1582011080 conn(0x5576bfc23f40 msgr2=0x5576bfc21170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f900dbc7740 1 -- 172.21.15.82:0/1582011080 shutdown_connections 2024-04-21T08:46:26.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f900dbc7740 1 -- 172.21.15.82:0/1582011080 wait complete. 2024-04-21T08:46:26.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.493+0000 7f900dbc7740 20 client.24923 trim_cache size 0 max 16384 2024-04-21T08:46:26.510 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:46:26.510 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:46:26.510 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 1, 2024-04-21T08:46:26.510 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24481, 2024-04-21T08:46:26.511 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:46:26.511 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:46:26.511 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:46:26.511 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 88.050037236999998, 2024-04-21T08:46:26.511 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2024-04-21T08:46:26.512 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 92, 2024-04-21T08:46:26.512 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:46:26.512 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 126.58470530300001 2024-04-21T08:46:26.512 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:46:26.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.494+0000 7f900afd5640 1 -- 172.21.15.82:0/281793506 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8ff40a3c40 msgr2=0x7f8ff40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.494+0000 7f900afd5640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8ff40a3c40 0x7f8ff40a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009fd0 tx=0x7f8ffc009650 comp rx=0 tx=0).stop 2024-04-21T08:46:26.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.494+0000 7f900afd5640 1 -- 172.21.15.82:0/281793506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9004103190 msgr2=0x7f90041988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.494+0000 7f900afd5640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9004103190 0x7f90041988f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0036040 tx=0x7f8ff00042f0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.494+0000 7f900afd5640 1 -- 172.21.15.82:0/281793506 shutdown_connections 2024-04-21T08:46:26.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.494+0000 7f900afd5640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f8ff40a3c40 0x7f8ff40a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.494+0000 7f900afd5640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9004103b50 0x7f900419f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.494+0000 7f900afd5640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9004103190 0x7f90041988f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.494+0000 7f900afd5640 1 --2- 172.21.15.82:0/281793506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9004101fa0 0x7f90041983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.494+0000 7f900afd5640 1 -- 172.21.15.82:0/281793506 >> 172.21.15.82:0/281793506 conn(0x7f90040fd530 msgr2=0x7f9004106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.494+0000 7f900afd5640 1 -- 172.21.15.82:0/281793506 shutdown_connections 2024-04-21T08:46:26.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.494+0000 7f900afd5640 1 -- 172.21.15.82:0/281793506 wait complete. 2024-04-21T08:46:26.517 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T08:46:26.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 -- 172.21.15.82:0/3002582784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe434100e90 msgr2=0x7fe4341012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 --2- 172.21.15.82:0/3002582784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe434100e90 0x7fe4341012b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009ca0 tx=0x7fe42c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 -- 172.21.15.82:0/3002582784 shutdown_connections 2024-04-21T08:46:26.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 --2- 172.21.15.82:0/3002582784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4341029a0 0x7fe4341092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 --2- 172.21.15.82:0/3002582784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe434101fe0 0x7fe434102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 --2- 172.21.15.82:0/3002582784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe434100e90 0x7fe4341012b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 -- 172.21.15.82:0/3002582784 >> 172.21.15.82:0/3002582784 conn(0x7fe4340fc400 msgr2=0x7fe4340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 -- 172.21.15.82:0/3002582784 shutdown_connections 2024-04-21T08:46:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 -- 172.21.15.82:0/3002582784 wait complete. 2024-04-21T08:46:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 Processor -- start 2024-04-21T08:46:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 -- start start 2024-04-21T08:46:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe434100e90 0x7fe434069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe434101fe0 0x7fe43419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4341029a0 0x7fe43419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe43410c9e0 con 0x7fe434100e90 2024-04-21T08:46:26.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe43410c860 con 0x7fe4341029a0 2024-04-21T08:46:26.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43bae0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe43410cb60 con 0x7fe434101fe0 2024-04-21T08:46:26.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.618+0000 7fe43987c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe434100e90 0x7fe434069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe43987c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe434100e90 0x7fe434069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36618/0 (socket says 172.21.15.82:36618) 2024-04-21T08:46:26.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe43987c640 1 -- 172.21.15.82:0/3553578408 learned_addr learned my addr 172.21.15.82:0/3553578408 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe43907b640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe434101fe0 0x7fe43419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe43a07d640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4341029a0 0x7fe43419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe43987c640 1 -- 172.21.15.82:0/3553578408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe434101fe0 msgr2=0x7fe43419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe43987c640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe434101fe0 0x7fe43419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe43987c640 1 -- 172.21.15.82:0/3553578408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4341029a0 msgr2=0x7fe43419f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe43987c640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4341029a0 0x7fe43419f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe43987c640 1 -- 172.21.15.82:0/3553578408 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43419fe00 con 0x7fe434100e90 2024-04-21T08:46:26.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe43987c640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe434100e90 0x7fe434069c50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe42c005e30 tx=0x7fe42c034830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe432ffd640 1 -- 172.21.15.82:0/3553578408 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c039040 con 0x7fe434100e90 2024-04-21T08:46:26.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe432ffd640 1 -- 172.21.15.82:0/3553578408 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe42c035c30 con 0x7fe434100e90 2024-04-21T08:46:26.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe43bae0640 1 -- 172.21.15.82:0/3553578408 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4341a0030 con 0x7fe434100e90 2024-04-21T08:46:26.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe432ffd640 1 -- 172.21.15.82:0/3553578408 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c0042a0 con 0x7fe434100e90 2024-04-21T08:46:26.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.619+0000 7fe43bae0640 1 -- 172.21.15.82:0/3553578408 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4341a0520 con 0x7fe434100e90 2024-04-21T08:46:26.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.621+0000 7fe432ffd640 1 -- 172.21.15.82:0/3553578408 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe42c046030 con 0x7fe434100e90 2024-04-21T08:46:26.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43e7af740 1 Processor -- start 2024-04-21T08:46:26.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43e7af740 1 -- start start 2024-04-21T08:46:26.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43e7af740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 0x556c35cb4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43e7af740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cb4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43e7af740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb52d0 0x556c35cbbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43e7af740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556c35c18b10 con 0x556c35cb4830 2024-04-21T08:46:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43e7af740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556c35bb6240 con 0x556c35cb52d0 2024-04-21T08:46:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43e7af740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c35c14ad0 con 0x556c35cb3e40 2024-04-21T08:46:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe432ffd640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe41c0a3bb0 0x7fe41c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43987c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cb4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43987c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cb4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36630/0 (socket says 172.21.15.82:36630) 2024-04-21T08:46:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43987c640 1 -- 172.21.15.82:0/868471871 learned_addr learned my addr 172.21.15.82:0/868471871 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43a07d640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe41c0a3bb0 0x7fe41c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.622+0000 7fe43907b640 1 --2- 172.21.15.82:0/868471871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb52d0 0x556c35cbbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe43a07d640 1 --2- 172.21.15.82:0/868471871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 0x556c35cb4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe432ffd640 1 -- 172.21.15.82:0/3553578408 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9216+0+0 (secure 0 0 0) 0x7fe42c0f2160 con 0x7fe434100e90 2024-04-21T08:46:26.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe43987c640 1 -- 172.21.15.82:0/868471871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 msgr2=0x556c35cb4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe43987c640 1 --2- 172.21.15.82:0/868471871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 0x556c35cb4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe43987c640 1 -- 172.21.15.82:0/868471871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb52d0 msgr2=0x556c35cbbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe43987c640 1 --2- 172.21.15.82:0/868471871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb52d0 0x556c35cbbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe43987c640 1 -- 172.21.15.82:0/868471871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3592b2d0 con 0x556c35cb4830 2024-04-21T08:46:26.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe43907b640 1 --2- 172.21.15.82:0/868471871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb52d0 0x556c35cbbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:26.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe43a07d640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe41c0a3bb0 0x7fe41c0a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe42400bfd0 tx=0x7fe42400ba60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe43987c640 1 --2- 172.21.15.82:0/868471871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cb4cb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe42c000f80 tx=0x7fe42c090c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aba24011a3ca102 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe41bfff640 1 -- 172.21.15.82:0/868471871 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c0be800 con 0x556c35cb4830 2024-04-21T08:46:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe41bfff640 1 -- 172.21.15.82:0/868471871 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe42c0be100 con 0x556c35cb4830 2024-04-21T08:46:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.623+0000 7fe41bfff640 1 -- 172.21.15.82:0/868471871 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c0ba770 con 0x556c35cb4830 2024-04-21T08:46:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.624+0000 7fe43e7af740 1 -- 172.21.15.82:0/868471871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 msgr2=0x556c35cb4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.624+0000 7fe43e7af740 1 --2- 172.21.15.82:0/868471871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cb4cb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe42c000f80 tx=0x7fe42c090c40 comp rx=0 tx=0).stop 2024-04-21T08:46:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.624+0000 7fe43e7af740 1 -- 172.21.15.82:0/868471871 shutdown_connections 2024-04-21T08:46:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.624+0000 7fe43e7af740 1 --2- 172.21.15.82:0/868471871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb52d0 0x556c35cbbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.624+0000 7fe43e7af740 1 --2- 172.21.15.82:0/868471871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cb4cb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.624+0000 7fe43e7af740 1 --2- 172.21.15.82:0/868471871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 0x556c35cb4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.624+0000 7fe43e7af740 1 -- 172.21.15.82:0/868471871 >> 172.21.15.82:0/868471871 conn(0x556c35caff40 msgr2=0x556c35cae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.624+0000 7fe43e7af740 1 -- 172.21.15.82:0/868471871 shutdown_connections 2024-04-21T08:46:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.624+0000 7fe43e7af740 1 -- 172.21.15.82:0/868471871 wait complete. 2024-04-21T08:46:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.624+0000 7fe43e7af740 1 -- start start 2024-04-21T08:46:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.624+0000 7fe43e7af740 1 Processor -- start 2024-04-21T08:46:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43e7af740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 0x556c35cbdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43e7af740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cbe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43e7af740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb52d0 0x556c35cc5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43e7af740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556c35cb5ad0 con 0x556c35cb4830 2024-04-21T08:46:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43e7af740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556c35ba1190 con 0x556c35cb52d0 2024-04-21T08:46:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43e7af740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c35cbc3f0 con 0x556c35cb3e40 2024-04-21T08:46:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43e7af740 10 client.0 _ll_register_callbacks cb 0x556c35b9c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43907b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cbe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43907b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cbe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36636/0 (socket says 172.21.15.82:36636) 2024-04-21T08:46:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43907b640 1 -- 172.21.15.82:0/3516423667 learned_addr learned my addr 172.21.15.82:0/3516423667 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43a07d640 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb52d0 0x556c35cc5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43907b640 1 -- 172.21.15.82:0/3516423667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 msgr2=0x556c35cbdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43987c640 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 0x556c35cbdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43907b640 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 0x556c35cbdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43907b640 1 -- 172.21.15.82:0/3516423667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb52d0 msgr2=0x556c35cc5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43907b640 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb52d0 0x556c35cc5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43907b640 1 -- 172.21.15.82:0/3516423667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c359f56b0 con 0x556c35cb4830 2024-04-21T08:46:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43a07d640 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb52d0 0x556c35cc5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43907b640 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cbe140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe428010820 tx=0x7fe428010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2625560821d1a0bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe4197fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe4197fa640 1 -- 172.21.15.82:0/3516423667 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428004260 con 0x556c35cb4830 2024-04-21T08:46:26.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe4197fa640 1 -- 172.21.15.82:0/3516423667 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9216+0+0 (secure 0 0 0) 0x7fe428004400 con 0x556c35cb4830 2024-04-21T08:46:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe43e7af740 1 -- 172.21.15.82:0/3516423667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe42c0be100 con 0x556c35cb4830 2024-04-21T08:46:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.625+0000 7fe4197fa640 1 -- 172.21.15.82:0/3516423667 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe42800f920 con 0x556c35cb4830 2024-04-21T08:46:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe43e7af740 10 client.15306 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe43e7af740 1 -- 172.21.15.82:0/3516423667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556c35cc8610 con 0x556c35cb4830 2024-04-21T08:46:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe4197fa640 1 -- 172.21.15.82:0/3516423667 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe42c0be100 con 0x556c35cb4830 2024-04-21T08:46:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe4197fa640 1 -- 172.21.15.82:0/3516423667 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe42801a420 con 0x556c35cb4830 2024-04-21T08:46:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe43e7af740 10 client.15306 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe43e7af740 10 client.15306 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:46:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe43e7af740 20 client.15306 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe43e7af740 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556c35ccad40 0x556c35ccc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe43e7af740 4 client.15306 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe43e7af740 1 -- 172.21.15.82:0/3516423667 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c35c26530 con 0x556c35ccad40 2024-04-21T08:46:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe43987c640 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556c35ccad40 0x556c35ccc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe43987c640 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556c35ccad40 0x556c35ccc1b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2720223840cd379e server_cookie=c19f4d747cb25846 in_seq=0 out_seq=0 2024-04-21T08:46:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.626+0000 7fe4197fa640 10 client.15306 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:46:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe4197fa640 1 -- 172.21.15.82:0/3516423667 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c35c26530 con 0x556c35ccad40 2024-04-21T08:46:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe4197fa640 10 client.15306 handle_command_reply: tid=0 2024-04-21T08:46:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe43e7af740 1 client.15306 shutdown 2024-04-21T08:46:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe43e7af740 1 -- 172.21.15.82:0/3516423667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 msgr2=0x556c35cbe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe43e7af740 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cbe140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe428010820 tx=0x7fe428010ce0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe43e7af740 1 -- 172.21.15.82:0/3516423667 shutdown_connections 2024-04-21T08:46:26.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe43e7af740 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556c35ccad40 0x556c35ccc1b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe43e7af740 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb52d0 0x556c35cc5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe43e7af740 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cbe140 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe43e7af740 1 --2- 172.21.15.82:0/3516423667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 0x556c35cbdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe4197fa640 0 client.15306 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:46:26.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe43e7af740 1 -- 172.21.15.82:0/3516423667 >> 172.21.15.82:0/3516423667 conn(0x556c35caff40 msgr2=0x556c35cada00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe43e7af740 1 -- 172.21.15.82:0/3516423667 shutdown_connections 2024-04-21T08:46:26.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.627+0000 7fe43e7af740 1 -- 172.21.15.82:0/3516423667 wait complete. 2024-04-21T08:46:26.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.628+0000 7fe43e7af740 20 client.15306 trim_cache size 0 max 16384 2024-04-21T08:46:26.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.645+0000 7fe43e7af740 1 Processor -- start 2024-04-21T08:46:26.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.645+0000 7fe43e7af740 1 -- start start 2024-04-21T08:46:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43e7af740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35caff40 0x556c35cae420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43e7af740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb3e40 0x556c35cae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43e7af740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb4830 0x556c35cc5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43e7af740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556c35cca4f0 con 0x556c35cb3e40 2024-04-21T08:46:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43e7af740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556c35ccabc0 con 0x556c35cb4830 2024-04-21T08:46:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43e7af740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c35ccb290 con 0x556c35caff40 2024-04-21T08:46:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43907b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb3e40 0x556c35cae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43907b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb3e40 0x556c35cae960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36642/0 (socket says 172.21.15.82:36642) 2024-04-21T08:46:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43907b640 1 -- 172.21.15.82:0/3363209984 learned_addr learned my addr 172.21.15.82:0/3363209984 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43a07d640 1 --2- 172.21.15.82:0/3363209984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb4830 0x556c35cc5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43987c640 1 --2- 172.21.15.82:0/3363209984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35caff40 0x556c35cae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43907b640 1 -- 172.21.15.82:0/3363209984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35caff40 msgr2=0x556c35cae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43907b640 1 --2- 172.21.15.82:0/3363209984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35caff40 0x556c35cae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43907b640 1 -- 172.21.15.82:0/3363209984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb4830 msgr2=0x556c35cc5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43907b640 1 --2- 172.21.15.82:0/3363209984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb4830 0x556c35cc5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43907b640 1 -- 172.21.15.82:0/3363209984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c35cbbb20 con 0x556c35cb3e40 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43987c640 1 --2- 172.21.15.82:0/3363209984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35caff40 0x556c35cae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe43907b640 1 --2- 172.21.15.82:0/3363209984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb3e40 0x556c35cae960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe428001eb0 tx=0x7fe42801ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f040580c5fb5e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe41b7fe640 1 -- 172.21.15.82:0/3363209984 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42802c070 con 0x556c35cb3e40 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe41b7fe640 1 -- 172.21.15.82:0/3363209984 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe42801a420 con 0x556c35cb3e40 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.646+0000 7fe41b7fe640 1 -- 172.21.15.82:0/3363209984 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428003830 con 0x556c35cb3e40 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.647+0000 7fe43e7af740 1 -- 172.21.15.82:0/3363209984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb3e40 msgr2=0x556c35cae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.647+0000 7fe43e7af740 1 --2- 172.21.15.82:0/3363209984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb3e40 0x556c35cae960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe428001eb0 tx=0x7fe42801ab50 comp rx=0 tx=0).stop 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.647+0000 7fe43e7af740 1 -- 172.21.15.82:0/3363209984 shutdown_connections 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.647+0000 7fe43e7af740 1 --2- 172.21.15.82:0/3363209984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35cb4830 0x556c35cc5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.647+0000 7fe43e7af740 1 --2- 172.21.15.82:0/3363209984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb3e40 0x556c35cae960 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.647+0000 7fe43e7af740 1 --2- 172.21.15.82:0/3363209984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35caff40 0x556c35cae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.647+0000 7fe43e7af740 1 -- 172.21.15.82:0/3363209984 >> 172.21.15.82:0/3363209984 conn(0x556c35ca9b70 msgr2=0x556c35cb8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.647+0000 7fe43e7af740 1 -- 172.21.15.82:0/3363209984 shutdown_connections 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.647+0000 7fe43e7af740 1 -- 172.21.15.82:0/3363209984 wait complete. 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.647+0000 7fe43e7af740 1 -- start start 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.647+0000 7fe43e7af740 1 Processor -- start 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43e7af740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35caff40 0x556c35cc10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43e7af740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 0x556c35cc3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43e7af740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cc3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43e7af740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556c35ccc030 con 0x556c35cb4830 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43e7af740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556c35ccb960 con 0x556c35caff40 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43e7af740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c35ccc700 con 0x556c35cb3e40 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43e7af740 10 client.0 _ll_register_callbacks cb 0x7fe42800a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43a07d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cc3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43a07d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cc3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36654/0 (socket says 172.21.15.82:36654) 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43a07d640 1 -- 172.21.15.82:0/1079343932 learned_addr learned my addr 172.21.15.82:0/1079343932 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43987c640 1 --2- 172.21.15.82:0/1079343932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35caff40 0x556c35cc10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43a07d640 1 -- 172.21.15.82:0/1079343932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 msgr2=0x556c35cc3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43a07d640 1 --2- 172.21.15.82:0/1079343932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 0x556c35cc3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43a07d640 1 -- 172.21.15.82:0/1079343932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35caff40 msgr2=0x556c35cc10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43a07d640 1 --2- 172.21.15.82:0/1079343932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35caff40 0x556c35cc10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43a07d640 1 -- 172.21.15.82:0/1079343932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c35cc4090 con 0x556c35cb4830 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43a07d640 1 --2- 172.21.15.82:0/1079343932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cc3b50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe4240042b0 tx=0x7fe42400b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4312a7daef4aa00e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe4197fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe4197fa640 1 -- 172.21.15.82:0/1079343932 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424003af0 con 0x556c35cb4830 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43e7af740 1 -- 172.21.15.82:0/1079343932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe42801a420 con 0x556c35cb4830 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe4197fa640 1 -- 172.21.15.82:0/1079343932 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9216+0+0 (secure 0 0 0) 0x7fe42401b040 con 0x556c35cb4830 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe4197fa640 1 -- 172.21.15.82:0/1079343932 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe424034050 con 0x556c35cb4830 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43e7af740 10 client.15312 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe43e7af740 1 -- 172.21.15.82:0/1079343932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556c35cc53f0 con 0x556c35cb4830 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.648+0000 7fe4197fa640 1 -- 172.21.15.82:0/1079343932 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe42801a420 con 0x556c35cb4830 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe4197fa640 1 -- 172.21.15.82:0/1079343932 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe424028070 con 0x556c35cb4830 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe43e7af740 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe43e7af740 10 client.15312 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe43e7af740 20 client.15312 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe43e7af740 1 --2- 172.21.15.82:0/1079343932 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556c35cb52d0 0x556c35d34280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe43e7af740 4 client.15312 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe43e7af740 1 -- 172.21.15.82:0/1079343932 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "status"}) -- 0x556c35c203a0 con 0x556c35cb52d0 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe43987c640 1 --2- 172.21.15.82:0/1079343932 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556c35cb52d0 0x556c35d34280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe43987c640 1 --2- 172.21.15.82:0/1079343932 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556c35cb52d0 0x556c35d34280 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80266661b17fcbad server_cookie=6c3e1f4d93fb6c2d in_seq=0 out_seq=0 2024-04-21T08:46:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe4197fa640 10 client.15312 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:46:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe4197fa640 1 -- 172.21.15.82:0/1079343932 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x556c35c203a0 con 0x556c35cb52d0 2024-04-21T08:46:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe4197fa640 10 client.15312 handle_command_reply: tid=0 2024-04-21T08:46:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.649+0000 7fe43e7af740 1 client.15312 shutdown 2024-04-21T08:46:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43e7af740 1 -- 172.21.15.82:0/1079343932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 msgr2=0x556c35cc3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43e7af740 1 --2- 172.21.15.82:0/1079343932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cc3b50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe4240042b0 tx=0x7fe42400b2e0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43e7af740 1 -- 172.21.15.82:0/1079343932 shutdown_connections 2024-04-21T08:46:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43e7af740 1 --2- 172.21.15.82:0/1079343932 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556c35cb52d0 0x556c35d34280 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43e7af740 1 --2- 172.21.15.82:0/1079343932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c35cb4830 0x556c35cc3b50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe4197fa640 0 client.15312 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:46:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43e7af740 1 --2- 172.21.15.82:0/1079343932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c35cb3e40 0x556c35cc3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43e7af740 1 --2- 172.21.15.82:0/1079343932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c35caff40 0x556c35cc10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43e7af740 1 -- 172.21.15.82:0/1079343932 >> 172.21.15.82:0/1079343932 conn(0x556c35ca9b70 msgr2=0x556c35cad170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43e7af740 1 -- 172.21.15.82:0/1079343932 shutdown_connections 2024-04-21T08:46:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43e7af740 1 -- 172.21.15.82:0/1079343932 wait complete. 2024-04-21T08:46:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43e7af740 20 client.15312 trim_cache size 0 max 16384 2024-04-21T08:46:26.684 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:46:26.684 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:46:26.684 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-21T08:46:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24509, 2024-04-21T08:46:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:46:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:46:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:46:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 112.167557777, 2024-04-21T08:46:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2024-04-21T08:46:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 92, 2024-04-21T08:46:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:46:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 118.235102803 2024-04-21T08:46:26.685 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:46:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43bae0640 1 -- 172.21.15.82:0/3553578408 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe41c0a3bb0 msgr2=0x7fe41c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43bae0640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe41c0a3bb0 0x7fe41c0a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe42400bfd0 tx=0x7fe42400ba60 comp rx=0 tx=0).stop 2024-04-21T08:46:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43bae0640 1 -- 172.21.15.82:0/3553578408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe434100e90 msgr2=0x7fe434069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43bae0640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe434100e90 0x7fe434069c50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe42c005e30 tx=0x7fe42c034830 comp rx=0 tx=0).stop 2024-04-21T08:46:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43bae0640 1 -- 172.21.15.82:0/3553578408 shutdown_connections 2024-04-21T08:46:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43bae0640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe41c0a3bb0 0x7fe41c0a6070 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43bae0640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4341029a0 0x7fe43419f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43bae0640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe434101fe0 0x7fe43419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43bae0640 1 --2- 172.21.15.82:0/3553578408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe434100e90 0x7fe434069c50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43bae0640 1 -- 172.21.15.82:0/3553578408 >> 172.21.15.82:0/3553578408 conn(0x7fe4340fc400 msgr2=0x7fe434105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43bae0640 1 -- 172.21.15.82:0/3553578408 shutdown_connections 2024-04-21T08:46:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:26.650+0000 7fe43bae0640 1 -- 172.21.15.82:0/3553578408 wait complete. 2024-04-21T08:46:26.689 INFO:tasks.cephadm:Setup complete, yielding 2024-04-21T08:46:26.689 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-21T08:46:26.699 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-04-21T08:46:26.699 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-21T08:46:26.720 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:26 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1615310810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:26.720 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:26 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2094370677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:26.720 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-21T08:46:26.721 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-21T08:46:26.721 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-21T08:46:26.721 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-21T08:46:26.721 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:26 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-21T08:46:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:26 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1615310810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:26 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2094370677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-21T08:46:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-21T08:46:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-21T08:46:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-21T08:46:26.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:26 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-21T08:46:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:26 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1615310810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:26 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2094370677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-21T08:46:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-21T08:46:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-21T08:46:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-21T08:46:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:26 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-21T08:46:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:46:27.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:27 smithi082 ceph-mon[27967]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:27.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-21T08:46:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-21T08:46:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-21T08:46:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-21T08:46:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:27 smithi082 ceph-mon[27967]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-21T08:46:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:27 smithi082 ceph-mon[27967]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T08:46:27.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:27 smithi082 ceph-mon[27967]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-21T08:46:27.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:27 smithi106 ceph-mon[31072]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:27.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-21T08:46:27.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-21T08:46:27.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-21T08:46:27.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-21T08:46:27.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:27 smithi106 ceph-mon[31072]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-21T08:46:27.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:27 smithi106 ceph-mon[31072]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T08:46:27.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:27 smithi106 ceph-mon[31072]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-21T08:46:27.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:27 smithi134 ceph-mon[31772]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:27.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-21T08:46:27.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-21T08:46:27.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-21T08:46:27.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-21T08:46:27.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:27 smithi134 ceph-mon[31772]: from='mgr.14152 172.21.15.82:0/454131767' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-21T08:46:27.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:27 smithi134 ceph-mon[31772]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T08:46:27.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:27 smithi134 ceph-mon[31772]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-21T08:46:28.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:28 smithi082 ceph-mon[27967]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T08:46:28.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:28 smithi082 ceph-mon[27967]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-21T08:46:28.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:28 smithi106 ceph-mon[31072]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T08:46:28.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:28 smithi106 ceph-mon[31072]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-21T08:46:28.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.832+0000 7f747a8de700 1 -- 172.21.15.82:0/2097464843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7474103540 msgr2=0x7f7474109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:28.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.832+0000 7f747a8de700 1 --2- 172.21.15.82:0/2097464843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7474103540 0x7f7474109df0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f746c00b0d0 tx=0x7f746c00b3e0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.832+0000 7f747a8de700 1 -- 172.21.15.82:0/2097464843 shutdown_connections 2024-04-21T08:46:28.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.832+0000 7f747a8de700 1 --2- 172.21.15.82:0/2097464843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7474103540 0x7f7474109df0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.832+0000 7f747a8de700 1 --2- 172.21.15.82:0/2097464843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74741019a0 0x7f7474101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.832+0000 7f747a8de700 1 --2- 172.21.15.82:0/2097464843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7474102ba0 0x7f7474103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.832+0000 7f747a8de700 1 -- 172.21.15.82:0/2097464843 >> 172.21.15.82:0/2097464843 conn(0x7f74740fcf40 msgr2=0x7f74740ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.832+0000 7f747a8de700 1 -- 172.21.15.82:0/2097464843 shutdown_connections 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.832+0000 7f747a8de700 1 -- 172.21.15.82:0/2097464843 wait complete. 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f747a8de700 1 Processor -- start 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f747a8de700 1 -- start start 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f747a8de700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74741019a0 0x7f7474198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f747a8de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7474102ba0 0x7f7474199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f747a8de700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7474103540 0x7f74741a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f747a8de700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f747410d4b0 con 0x7f7474102ba0 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f747a8de700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f747410d330 con 0x7f74741019a0 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f747a8de700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f747410d630 con 0x7f7474103540 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f74737fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7474102ba0 0x7f7474199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f74737fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7474102ba0 0x7f7474199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36668/0 (socket says 172.21.15.82:36668) 2024-04-21T08:46:28.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f74737fe700 1 -- 172.21.15.82:0/913947527 learned_addr learned my addr 172.21.15.82:0/913947527 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f7478e7b700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7474103540 0x7f74741a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f7473fff700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74741019a0 0x7f7474198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f74737fe700 1 -- 172.21.15.82:0/913947527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7474103540 msgr2=0x7f74741a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f74737fe700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7474103540 0x7f74741a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f74737fe700 1 -- 172.21.15.82:0/913947527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74741019a0 msgr2=0x7f7474198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f74737fe700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74741019a0 0x7f7474198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f74737fe700 1 -- 172.21.15.82:0/913947527 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74741a0820 con 0x7f7474102ba0 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.833+0000 7f7473fff700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74741019a0 0x7f7474198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.834+0000 7f74737fe700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7474102ba0 0x7f7474199100 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f746400c9e0 tx=0x7f746400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.834+0000 7f74717fa700 1 -- 172.21.15.82:0/913947527 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464007ac0 con 0x7f7474102ba0 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.834+0000 7f747a8de700 1 -- 172.21.15.82:0/913947527 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74741a0b10 con 0x7f7474102ba0 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.834+0000 7f74717fa700 1 -- 172.21.15.82:0/913947527 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7464007c60 con 0x7f7474102ba0 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.834+0000 7f747a8de700 1 -- 172.21.15.82:0/913947527 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74741a10a0 con 0x7f7474102ba0 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.834+0000 7f74717fa700 1 -- 172.21.15.82:0/913947527 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464018620 con 0x7f7474102ba0 2024-04-21T08:46:28.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.835+0000 7f747a8de700 1 -- 172.21.15.82:0/913947527 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7474069760 con 0x7f7474102ba0 2024-04-21T08:46:28.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.836+0000 7f74717fa700 1 -- 172.21.15.82:0/913947527 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f746400f450 con 0x7f7474102ba0 2024-04-21T08:46:28.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.836+0000 7f74717fa700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f745c0a3c40 0x7f745c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:28.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.836+0000 7f7473fff700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f745c0a3c40 0x7f745c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:28.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.836+0000 7f74717fa700 1 -- 172.21.15.82:0/913947527 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f74640d09c0 con 0x7f7474102ba0 2024-04-21T08:46:28.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.837+0000 7f7473fff700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f745c0a3c40 0x7f745c0a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7474102a00 tx=0x7f7468008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:28.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:28.840+0000 7f74717fa700 1 -- 172.21.15.82:0/913947527 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7474069760 con 0x7f7474102ba0 2024-04-21T08:46:28.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:28 smithi134 ceph-mon[31772]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T08:46:28.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:28 smithi134 ceph-mon[31772]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-21T08:46:29.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.056+0000 7f747a8de700 1 -- 172.21.15.82:0/913947527 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f74740ff110 con 0x7f745c0a3c40 2024-04-21T08:46:29.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.058+0000 7f74717fa700 1 -- 172.21.15.82:0/913947527 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f74740ff110 con 0x7f745c0a3c40 2024-04-21T08:46:29.057 INFO:teuthology.orchestra.run.smithi082.stdout:Backend: cephadm 2024-04-21T08:46:29.057 INFO:teuthology.orchestra.run.smithi082.stdout:Available: Yes 2024-04-21T08:46:29.057 INFO:teuthology.orchestra.run.smithi082.stdout:Paused: No 2024-04-21T08:46:29.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.060+0000 7f747a8de700 1 -- 172.21.15.82:0/913947527 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f745c0a3c40 msgr2=0x7f745c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:29.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.060+0000 7f747a8de700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f745c0a3c40 0x7f745c0a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7474102a00 tx=0x7f7468008040 comp rx=0 tx=0).stop 2024-04-21T08:46:29.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.060+0000 7f747a8de700 1 -- 172.21.15.82:0/913947527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7474102ba0 msgr2=0x7f7474199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:29.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.060+0000 7f747a8de700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7474102ba0 0x7f7474199100 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f746400c9e0 tx=0x7f746400cda0 comp rx=0 tx=0).stop 2024-04-21T08:46:29.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.060+0000 7f747a8de700 1 -- 172.21.15.82:0/913947527 shutdown_connections 2024-04-21T08:46:29.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.060+0000 7f747a8de700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f745c0a3c40 0x7f745c0a6100 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:29.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.060+0000 7f747a8de700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7474103540 0x7f74741a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:29.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.060+0000 7f747a8de700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74741019a0 0x7f7474198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:29.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.060+0000 7f747a8de700 1 --2- 172.21.15.82:0/913947527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7474102ba0 0x7f7474199100 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:29.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.060+0000 7f747a8de700 1 -- 172.21.15.82:0/913947527 >> 172.21.15.82:0/913947527 conn(0x7f74740fcf40 msgr2=0x7f74740fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:29.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.060+0000 7f747a8de700 1 -- 172.21.15.82:0/913947527 shutdown_connections 2024-04-21T08:46:29.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:29.060+0000 7f747a8de700 1 -- 172.21.15.82:0/913947527 wait complete. 2024-04-21T08:46:29.851 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:29 smithi082 ceph-mon[27967]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-21T08:46:29.852 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:29 smithi082 ceph-mon[27967]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:29.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:29 smithi106 ceph-mon[31072]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-21T08:46:29.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:29 smithi106 ceph-mon[31072]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:29.882 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-21T08:46:29.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:29 smithi134 ceph-mon[31772]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-21T08:46:29.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:29 smithi134 ceph-mon[31772]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:30.166 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:46:30.562 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:30 smithi082 ceph-mon[27967]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-21T08:46:30.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:30 smithi106 ceph-mon[31072]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-21T08:46:30.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:30 smithi134 ceph-mon[31772]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-21T08:46:32.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.122+0000 7f4654f35700 1 -- 172.21.15.82:0/1142527048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4650101c40 msgr2=0x7f46501020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:32.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.122+0000 7f4654f35700 1 --2- 172.21.15.82:0/1142527048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4650101c40 0x7f46501020a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4638009990 tx=0x7f4638009ca0 comp rx=0 tx=0).stop 2024-04-21T08:46:32.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.122+0000 7f4654f35700 1 -- 172.21.15.82:0/1142527048 shutdown_connections 2024-04-21T08:46:32.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.122+0000 7f4654f35700 1 --2- 172.21.15.82:0/1142527048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46501025e0 0x7f4650109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:32.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.122+0000 7f4654f35700 1 --2- 172.21.15.82:0/1142527048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4650102f90 0x7f46501033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:32.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.122+0000 7f4654f35700 1 --2- 172.21.15.82:0/1142527048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4650101c40 0x7f46501020a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:32.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.122+0000 7f4654f35700 1 -- 172.21.15.82:0/1142527048 >> 172.21.15.82:0/1142527048 conn(0x7f46500fc6b0 msgr2=0x7f46500feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:32.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.122+0000 7f4654f35700 1 -- 172.21.15.82:0/1142527048 shutdown_connections 2024-04-21T08:46:32.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.122+0000 7f4654f35700 1 -- 172.21.15.82:0/1142527048 wait complete. 2024-04-21T08:46:32.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.123+0000 7f4654f35700 1 Processor -- start 2024-04-21T08:46:32.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.123+0000 7f4654f35700 1 -- start start 2024-04-21T08:46:32.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.123+0000 7f4654f35700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4650101c40 0x7f4650198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:32.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.123+0000 7f4654f35700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46501025e0 0x7f4650199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:32.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.123+0000 7f4654f35700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4650102f90 0x7f46501a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:32.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.123+0000 7f4654f35700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f465010e630 con 0x7f46501025e0 2024-04-21T08:46:32.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.123+0000 7f4654f35700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f465010e4b0 con 0x7f4650101c40 2024-04-21T08:46:32.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.123+0000 7f4654f35700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f465010e7b0 con 0x7f4650102f90 2024-04-21T08:46:32.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.123+0000 7f464dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46501025e0 0x7f4650199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:32.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.123+0000 7f464dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46501025e0 0x7f4650199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36684/0 (socket says 172.21.15.82:36684) 2024-04-21T08:46:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.123+0000 7f464dd9b700 1 -- 172.21.15.82:0/3670843167 learned_addr learned my addr 172.21.15.82:0/3670843167 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f464dd9b700 1 -- 172.21.15.82:0/3670843167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4650102f90 msgr2=0x7f46501a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f464ed9d700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4650102f90 0x7f46501a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f464dd9b700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4650102f90 0x7f46501a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f464e59c700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4650101c40 0x7f4650198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f464dd9b700 1 -- 172.21.15.82:0/3670843167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4650101c40 msgr2=0x7f4650198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f464dd9b700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4650101c40 0x7f4650198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f464dd9b700 1 -- 172.21.15.82:0/3670843167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46501a0980 con 0x7f46501025e0 2024-04-21T08:46:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f464ed9d700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4650102f90 0x7f46501a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f464dd9b700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46501025e0 0x7f4650199270 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f46380036f0 tx=0x7f46380039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f46477fe700 1 -- 172.21.15.82:0/3670843167 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638027070 con 0x7f46501025e0 2024-04-21T08:46:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f46477fe700 1 -- 172.21.15.82:0/3670843167 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4638022470 con 0x7f46501025e0 2024-04-21T08:46:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f4654f35700 1 -- 172.21.15.82:0/3670843167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46501a0c10 con 0x7f46501025e0 2024-04-21T08:46:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f46477fe700 1 -- 172.21.15.82:0/3670843167 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463800f6c0 con 0x7f46501025e0 2024-04-21T08:46:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.124+0000 7f4654f35700 1 -- 172.21.15.82:0/3670843167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46501a1060 con 0x7f46501025e0 2024-04-21T08:46:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.125+0000 7f4654f35700 1 -- 172.21.15.82:0/3670843167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465002df80 con 0x7f46501025e0 2024-04-21T08:46:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.126+0000 7f46477fe700 1 -- 172.21.15.82:0/3670843167 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f463802a070 con 0x7f46501025e0 2024-04-21T08:46:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.127+0000 7f46477fe700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f463c0a3c40 0x7f463c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.127+0000 7f464e59c700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f463c0a3c40 0x7f463c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.127+0000 7f46477fe700 1 -- 172.21.15.82:0/3670843167 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f46380d1760 con 0x7f46501025e0 2024-04-21T08:46:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.127+0000 7f464e59c700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f463c0a3c40 0x7f463c0a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f465019a210 tx=0x7f4640007310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.131+0000 7f46477fe700 1 -- 172.21.15.82:0/3670843167 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4638013070 con 0x7f46501025e0 2024-04-21T08:46:32.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.340+0000 7f4654f35700 1 -- 172.21.15.82:0/3670843167 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4650062e80 con 0x7f463c0a3c40 2024-04-21T08:46:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.354+0000 7f46477fe700 1 -- 172.21.15.82:0/3670843167 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f4650062e80 con 0x7f463c0a3c40 2024-04-21T08:46:32.354 INFO:teuthology.orchestra.run.smithi082.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-21T08:46:32.354 INFO:teuthology.orchestra.run.smithi082.stdout:mds.a smithi082 running (2m) 29s ago 2m 16.2M - 19.0.0-3154-ga630765c b45a994450e2 54b972facf11 2024-04-21T08:46:32.354 INFO:teuthology.orchestra.run.smithi082.stdout:mds.b smithi106 running (2m) 57s ago 2m 28.5M - 19.0.0-3154-ga630765c b45a994450e2 6327164844cc 2024-04-21T08:46:32.354 INFO:teuthology.orchestra.run.smithi082.stdout:mds.c smithi134 running (2m) 57s ago 2m 18.6M - 19.0.0-3154-ga630765c b45a994450e2 3b91812a31b3 2024-04-21T08:46:32.354 INFO:teuthology.orchestra.run.smithi082.stdout:mds.d smithi082 running (2m) 29s ago 2m 16.4M - 19.0.0-3154-ga630765c b45a994450e2 01b15b9008c6 2024-04-21T08:46:32.354 INFO:teuthology.orchestra.run.smithi082.stdout:mds.e smithi106 running (2m) 57s ago 2m 17.6M - 19.0.0-3154-ga630765c b45a994450e2 251b53e53e73 2024-04-21T08:46:32.354 INFO:teuthology.orchestra.run.smithi082.stdout:mds.f smithi134 running (2m) 57s ago 2m 15.0M - 19.0.0-3154-ga630765c b45a994450e2 7a919ece608e 2024-04-21T08:46:32.354 INFO:teuthology.orchestra.run.smithi082.stdout:mds.g smithi082 running (2m) 29s ago 2m 16.2M - 19.0.0-3154-ga630765c b45a994450e2 81d7a4d8f12c 2024-04-21T08:46:32.354 INFO:teuthology.orchestra.run.smithi082.stdout:mds.h smithi106 running (2m) 57s ago 2m 18.4M - 19.0.0-3154-ga630765c b45a994450e2 28df946b28d7 2024-04-21T08:46:32.354 INFO:teuthology.orchestra.run.smithi082.stdout:mds.i smithi134 running (2m) 57s ago 2m 16.1M - 19.0.0-3154-ga630765c b45a994450e2 51575d3f162f 2024-04-21T08:46:32.354 INFO:teuthology.orchestra.run.smithi082.stdout:mds.j smithi082 running (2m) 29s ago 2m 15.4M - 19.0.0-3154-ga630765c b45a994450e2 9ea64cc0decb 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:mds.k smithi106 running (2m) 57s ago 2m 18.4M - 19.0.0-3154-ga630765c b45a994450e2 723e3ca6f5c5 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:mds.l smithi134 running (2m) 57s ago 2m 16.4M - 19.0.0-3154-ga630765c b45a994450e2 c8fd270304aa 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.x smithi082 *:9283,8765 running (10m) 29s ago 10m 515M - 19.0.0-3154-ga630765c b45a994450e2 c83669ec547c 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.y smithi106 *:8443,8765 running (7m) 57s ago 7m 452M - 19.0.0-3154-ga630765c b45a994450e2 1bf6191a46f5 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.z smithi134 *:8443,8765 running (7m) 57s ago 7m 452M - 19.0.0-3154-ga630765c b45a994450e2 c1ec677de54a 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:mon.a smithi082 running (10m) 29s ago 10m 55.5M 2048M 19.0.0-3154-ga630765c b45a994450e2 1d8a7d05453a 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:mon.b smithi106 running (8m) 57s ago 8m 38.8M 2048M 19.0.0-3154-ga630765c b45a994450e2 1eaa36030e11 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:mon.c smithi134 running (8m) 57s ago 8m 39.1M 2048M 19.0.0-3154-ga630765c b45a994450e2 ae5eb50a307a 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:osd.0 smithi082 running (7m) 29s ago 7m 45.9M 4272M 19.0.0-3154-ga630765c b45a994450e2 3f35a9d8d562 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:osd.1 smithi106 running (7m) 57s ago 7m 46.0M 4272M 19.0.0-3154-ga630765c b45a994450e2 efa326034deb 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:osd.2 smithi134 running (6m) 57s ago 6m 49.1M 4272M 19.0.0-3154-ga630765c b45a994450e2 009f000ff1bf 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:osd.3 smithi082 running (6m) 29s ago 6m 44.3M 4272M 19.0.0-3154-ga630765c b45a994450e2 35319ec4c4a1 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:osd.4 smithi106 running (5m) 57s ago 5m 45.7M 4272M 19.0.0-3154-ga630765c b45a994450e2 ef418a0520d2 2024-04-21T08:46:32.355 INFO:teuthology.orchestra.run.smithi082.stdout:osd.5 smithi134 running (5m) 57s ago 5m 52.5M 4272M 19.0.0-3154-ga630765c b45a994450e2 3fae4aa88706 2024-04-21T08:46:32.356 INFO:teuthology.orchestra.run.smithi082.stdout:osd.6 smithi082 running (5m) 29s ago 5m 43.2M 4272M 19.0.0-3154-ga630765c b45a994450e2 46384a50a837 2024-04-21T08:46:32.356 INFO:teuthology.orchestra.run.smithi082.stdout:osd.7 smithi106 running (4m) 57s ago 4m 45.7M 4272M 19.0.0-3154-ga630765c b45a994450e2 607674195e28 2024-04-21T08:46:32.356 INFO:teuthology.orchestra.run.smithi082.stdout:osd.8 smithi134 running (4m) 57s ago 4m 46.0M 4272M 19.0.0-3154-ga630765c b45a994450e2 7343d3fb26aa 2024-04-21T08:46:32.356 INFO:teuthology.orchestra.run.smithi082.stdout:osd.9 smithi082 running (3m) 29s ago 3m 41.5M 4272M 19.0.0-3154-ga630765c b45a994450e2 707e1c0f2d35 2024-04-21T08:46:32.356 INFO:teuthology.orchestra.run.smithi082.stdout:osd.10 smithi106 running (3m) 57s ago 3m 46.5M 4272M 19.0.0-3154-ga630765c b45a994450e2 892ee78c46f6 2024-04-21T08:46:32.356 INFO:teuthology.orchestra.run.smithi082.stdout:osd.11 smithi134 running (2m) 57s ago 2m 49.5M 4272M 19.0.0-3154-ga630765c b45a994450e2 d9f9e1cdc8c7 2024-04-21T08:46:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.356+0000 7f4654f35700 1 -- 172.21.15.82:0/3670843167 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f463c0a3c40 msgr2=0x7f463c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.356+0000 7f4654f35700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f463c0a3c40 0x7f463c0a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f465019a210 tx=0x7f4640007310 comp rx=0 tx=0).stop 2024-04-21T08:46:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.356+0000 7f4654f35700 1 -- 172.21.15.82:0/3670843167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46501025e0 msgr2=0x7f4650199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.356+0000 7f4654f35700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46501025e0 0x7f4650199270 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f46380036f0 tx=0x7f46380039f0 comp rx=0 tx=0).stop 2024-04-21T08:46:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.356+0000 7f4654f35700 1 -- 172.21.15.82:0/3670843167 shutdown_connections 2024-04-21T08:46:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.356+0000 7f4654f35700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f463c0a3c40 0x7f463c0a6100 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.356+0000 7f4654f35700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4650102f90 0x7f46501a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.356+0000 7f4654f35700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4650101c40 0x7f4650198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.356+0000 7f4654f35700 1 --2- 172.21.15.82:0/3670843167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46501025e0 0x7f4650199270 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.356+0000 7f4654f35700 1 -- 172.21.15.82:0/3670843167 >> 172.21.15.82:0/3670843167 conn(0x7f46500fc6b0 msgr2=0x7f4650106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.356+0000 7f4654f35700 1 -- 172.21.15.82:0/3670843167 shutdown_connections 2024-04-21T08:46:32.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:32.356+0000 7f4654f35700 1 -- 172.21.15.82:0/3670843167 wait complete. 2024-04-21T08:46:33.012 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-21T08:46:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:46:33.638 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:33 smithi082 ceph-mon[27967]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:33.638 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:33 smithi082 ceph-mon[27967]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 383 B/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:33 smithi134 ceph-mon[31772]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:33.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:33 smithi134 ceph-mon[31772]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 383 B/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:33.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:33 smithi106 ceph-mon[31072]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:33.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:33 smithi106 ceph-mon[31072]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 383 B/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:34.726 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:34 smithi082 ceph-mon[27967]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-21T08:46:34.726 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:34 smithi082 ceph-mon[27967]: Cluster is now healthy 2024-04-21T08:46:34.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:34 smithi134 ceph-mon[31772]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-21T08:46:34.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:34 smithi134 ceph-mon[31772]: Cluster is now healthy 2024-04-21T08:46:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:34 smithi106 ceph-mon[31072]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-21T08:46:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:34 smithi106 ceph-mon[31072]: Cluster is now healthy 2024-04-21T08:46:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.966+0000 7f7ad5842700 1 -- 172.21.15.82:0/1912349955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ad0108850 msgr2=0x7f7ad010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.966+0000 7f7ad5842700 1 --2- 172.21.15.82:0/1912349955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ad0108850 0x7f7ad010ac40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800b0a0 tx=0x7f7ac800b3b0 comp rx=0 tx=0).stop 2024-04-21T08:46:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.966+0000 7f7ad5842700 1 -- 172.21.15.82:0/1912349955 shutdown_connections 2024-04-21T08:46:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.966+0000 7f7ad5842700 1 --2- 172.21.15.82:0/1912349955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ad0108850 0x7f7ad010ac40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.966+0000 7f7ad5842700 1 --2- 172.21.15.82:0/1912349955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ad01035f0 0x7f7ad01059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.966+0000 7f7ad5842700 1 --2- 172.21.15.82:0/1912349955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad0105f20 0x7f7ad0108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.966+0000 7f7ad5842700 1 -- 172.21.15.82:0/1912349955 >> 172.21.15.82:0/1912349955 conn(0x7f7ad00fcf60 msgr2=0x7f7ad00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.966+0000 7f7ad5842700 1 -- 172.21.15.82:0/1912349955 shutdown_connections 2024-04-21T08:46:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.966+0000 7f7ad5842700 1 -- 172.21.15.82:0/1912349955 wait complete. 2024-04-21T08:46:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.966+0000 7f7ad5842700 1 Processor -- start 2024-04-21T08:46:34.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.966+0000 7f7ad5842700 1 -- start start 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ad5842700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ad01035f0 0x7f7ad019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ad5842700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ad0105f20 0x7f7ad019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ad5842700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad0108850 0x7f7ad01a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ad5842700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad010e520 con 0x7f7ad0105f20 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ad5842700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad010e3a0 con 0x7f7ad01035f0 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ad5842700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad010e6a0 con 0x7f7ad0108850 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ace7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ad0105f20 0x7f7ad019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ace7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ad0105f20 0x7f7ad019b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42264/0 (socket says 172.21.15.82:42264) 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ace7fc700 1 -- 172.21.15.82:0/781255334 learned_addr learned my addr 172.21.15.82:0/781255334 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7aceffd700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ad01035f0 0x7f7ad019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7acf7fe700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad0108850 0x7f7ad01a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ace7fc700 1 -- 172.21.15.82:0/781255334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad0108850 msgr2=0x7f7ad01a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ace7fc700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad0108850 0x7f7ad01a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ace7fc700 1 -- 172.21.15.82:0/781255334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ad01035f0 msgr2=0x7f7ad019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:34.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ace7fc700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ad01035f0 0x7f7ad019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:34.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ace7fc700 1 -- 172.21.15.82:0/781255334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad01a29c0 con 0x7f7ad0105f20 2024-04-21T08:46:34.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7aceffd700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ad01035f0 0x7f7ad019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:34.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7acf7fe700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad0108850 0x7f7ad01a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:34.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ace7fc700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ad0105f20 0x7f7ad019b2a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400d9b0 tx=0x7f7ac400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:34.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ad4840700 1 -- 172.21.15.82:0/781255334 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4017070 con 0x7f7ad0105f20 2024-04-21T08:46:34.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ad5842700 1 -- 172.21.15.82:0/781255334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad01a2cb0 con 0x7f7ad0105f20 2024-04-21T08:46:34.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ad4840700 1 -- 172.21.15.82:0/781255334 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac4004830 con 0x7f7ad0105f20 2024-04-21T08:46:34.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ad5842700 1 -- 172.21.15.82:0/781255334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad01a31a0 con 0x7f7ad0105f20 2024-04-21T08:46:34.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.967+0000 7f7ad4840700 1 -- 172.21.15.82:0/781255334 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40053a0 con 0x7f7ad0105f20 2024-04-21T08:46:34.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.969+0000 7f7ad5842700 1 -- 172.21.15.82:0/781255334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad0069730 con 0x7f7ad0105f20 2024-04-21T08:46:34.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.970+0000 7f7ad4840700 1 -- 172.21.15.82:0/781255334 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7ac40049e0 con 0x7f7ad0105f20 2024-04-21T08:46:34.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.970+0000 7f7ad4840700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7ab80a3cc0 0x7f7ab80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:34.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.970+0000 7f7aceffd700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7ab80a3cc0 0x7f7ab80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:34.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.971+0000 7f7ad4840700 1 -- 172.21.15.82:0/781255334 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f7ac40d06d0 con 0x7f7ad0105f20 2024-04-21T08:46:34.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.971+0000 7f7aceffd700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7ab80a3cc0 0x7f7ab80a6180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0005950 tx=0x7f7ac00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:34.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:34.975+0000 7f7ad4840700 1 -- 172.21.15.82:0/781255334 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ac4020020 con 0x7f7ad0105f20 2024-04-21T08:46:35.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.183+0000 7f7ad5842700 1 -- 172.21.15.82:0/781255334 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7ad0062e80 con 0x7f7ab80a3cc0 2024-04-21T08:46:35.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.189+0000 7f7ad4840700 1 -- 172.21.15.82:0/781255334 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7ad0062e80 con 0x7f7ab80a3cc0 2024-04-21T08:46:35.189 INFO:teuthology.orchestra.run.smithi082.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-21T08:46:35.189 INFO:teuthology.orchestra.run.smithi082.stdout:mds.all 12/12 60s ago 2m smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-04-21T08:46:35.189 INFO:teuthology.orchestra.run.smithi082.stdout:mgr 3/3 60s ago 8m smithi082=x;smithi106=y;smithi134=z;count:3 2024-04-21T08:46:35.189 INFO:teuthology.orchestra.run.smithi082.stdout:mon 3/3 60s ago 8m smithi082:172.21.15.82=a;smithi106:172.21.15.106=b;smithi134:172.21.15.134=c;count:3 2024-04-21T08:46:35.189 INFO:teuthology.orchestra.run.smithi082.stdout:osd 12 60s ago - 2024-04-21T08:46:35.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.191+0000 7f7ad5842700 1 -- 172.21.15.82:0/781255334 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7ab80a3cc0 msgr2=0x7f7ab80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:35.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.191+0000 7f7ad5842700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7ab80a3cc0 0x7f7ab80a6180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0005950 tx=0x7f7ac00058e0 comp rx=0 tx=0).stop 2024-04-21T08:46:35.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.191+0000 7f7ad5842700 1 -- 172.21.15.82:0/781255334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ad0105f20 msgr2=0x7f7ad019b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:35.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.191+0000 7f7ad5842700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ad0105f20 0x7f7ad019b2a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400d9b0 tx=0x7f7ac400dd70 comp rx=0 tx=0).stop 2024-04-21T08:46:35.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.191+0000 7f7ad5842700 1 -- 172.21.15.82:0/781255334 shutdown_connections 2024-04-21T08:46:35.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.191+0000 7f7ad5842700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f7ab80a3cc0 0x7f7ab80a6180 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:35.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.191+0000 7f7ad5842700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ad0108850 0x7f7ad01a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:35.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.191+0000 7f7ad5842700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ad01035f0 0x7f7ad019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:35.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.191+0000 7f7ad5842700 1 --2- 172.21.15.82:0/781255334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ad0105f20 0x7f7ad019b2a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:35.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.191+0000 7f7ad5842700 1 -- 172.21.15.82:0/781255334 >> 172.21.15.82:0/781255334 conn(0x7f7ad00fcf60 msgr2=0x7f7ad0102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:35.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.191+0000 7f7ad5842700 1 -- 172.21.15.82:0/781255334 shutdown_connections 2024-04-21T08:46:35.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:35.191+0000 7f7ad5842700 1 -- 172.21.15.82:0/781255334 wait complete. 2024-04-21T08:46:35.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:35 smithi134 ceph-mon[31772]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:35.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:35 smithi082 ceph-mon[27967]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:35.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:35 smithi106 ceph-mon[31072]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:35.897 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-21T08:46:36.139 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:46:36.562 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:36 smithi082 ceph-mon[27967]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:36.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:36 smithi134 ceph-mon[31772]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:36 smithi106 ceph-mon[31072]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:37.606 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:37 smithi082 ceph-mon[27967]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 311 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:37.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:37 smithi134 ceph-mon[31772]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 311 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:37.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.842+0000 7fc39d8b1700 1 -- 172.21.15.82:0/1582931656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc398105e60 msgr2=0x7fc398108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:37.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.842+0000 7fc39d8b1700 1 --2- 172.21.15.82:0/1582931656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc398105e60 0x7fc398108250 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009990 tx=0x7fc38c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:46:37.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.842+0000 7fc39d8b1700 1 -- 172.21.15.82:0/1582931656 shutdown_connections 2024-04-21T08:46:37.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.842+0000 7fc39d8b1700 1 --2- 172.21.15.82:0/1582931656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc398108790 0x7fc39810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:37.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.842+0000 7fc39d8b1700 1 --2- 172.21.15.82:0/1582931656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc398103530 0x7fc398105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:37.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.842+0000 7fc39d8b1700 1 --2- 172.21.15.82:0/1582931656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc398105e60 0x7fc398108250 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:37.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.842+0000 7fc39d8b1700 1 -- 172.21.15.82:0/1582931656 >> 172.21.15.82:0/1582931656 conn(0x7fc3980fcf40 msgr2=0x7fc3980ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:37.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.842+0000 7fc39d8b1700 1 -- 172.21.15.82:0/1582931656 shutdown_connections 2024-04-21T08:46:37.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.842+0000 7fc39d8b1700 1 -- 172.21.15.82:0/1582931656 wait complete. 2024-04-21T08:46:37.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.843+0000 7fc39d8b1700 1 Processor -- start 2024-04-21T08:46:37.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.843+0000 7fc39d8b1700 1 -- start start 2024-04-21T08:46:37.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.843+0000 7fc39d8b1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc398103530 0x7fc398198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:37.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.843+0000 7fc39d8b1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc398105e60 0x7fc398199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:37.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.843+0000 7fc39d8b1700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc398108790 0x7fc3981a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:37.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.843+0000 7fc39d8b1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc39810e5a0 con 0x7fc398105e60 2024-04-21T08:46:37.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.843+0000 7fc39d8b1700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc39810e420 con 0x7fc398108790 2024-04-21T08:46:37.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.843+0000 7fc39d8b1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc39810e720 con 0x7fc398103530 2024-04-21T08:46:37.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc3967fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc398105e60 0x7fc398199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:37.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc3967fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc398105e60 0x7fc398199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42288/0 (socket says 172.21.15.82:42288) 2024-04-21T08:46:37.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc3967fc700 1 -- 172.21.15.82:0/673999998 learned_addr learned my addr 172.21.15.82:0/673999998 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:37.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc396ffd700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc398103530 0x7fc398198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:37.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc3977fe700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc398108790 0x7fc3981a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:37.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc3967fc700 1 -- 172.21.15.82:0/673999998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc398103530 msgr2=0x7fc398198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:37.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc3967fc700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc398103530 0x7fc398198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:37.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc3967fc700 1 -- 172.21.15.82:0/673999998 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc398108790 msgr2=0x7fc3981a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:37.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc3967fc700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc398108790 0x7fc3981a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:37.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc3967fc700 1 -- 172.21.15.82:0/673999998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3981a0930 con 0x7fc398105e60 2024-04-21T08:46:37.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc3977fe700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc398108790 0x7fc3981a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:37.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc3967fc700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc398105e60 0x7fc398199270 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0047b0 tx=0x7fc38c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:37.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc39c8af700 1 -- 172.21.15.82:0/673999998 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c026070 con 0x7fc398105e60 2024-04-21T08:46:37.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc39d8b1700 1 -- 172.21.15.82:0/673999998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3981a0bc0 con 0x7fc398105e60 2024-04-21T08:46:37.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc39c8af700 1 -- 172.21.15.82:0/673999998 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc38c0040b0 con 0x7fc398105e60 2024-04-21T08:46:37.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc39c8af700 1 -- 172.21.15.82:0/673999998 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c005290 con 0x7fc398105e60 2024-04-21T08:46:37.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.844+0000 7fc39d8b1700 1 -- 172.21.15.82:0/673999998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3981a1010 con 0x7fc398105e60 2024-04-21T08:46:37.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.845+0000 7fc39d8b1700 1 -- 172.21.15.82:0/673999998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc398069730 con 0x7fc398105e60 2024-04-21T08:46:37.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.846+0000 7fc39c8af700 1 -- 172.21.15.82:0/673999998 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc38c029070 con 0x7fc398105e60 2024-04-21T08:46:37.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.847+0000 7fc39c8af700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc3800a3c40 0x7fc3800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:37.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.847+0000 7fc396ffd700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc3800a3c40 0x7fc3800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:37.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.847+0000 7fc39c8af700 1 -- 172.21.15.82:0/673999998 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fc38c0d1520 con 0x7fc398105e60 2024-04-21T08:46:37.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.847+0000 7fc396ffd700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc3800a3c40 0x7fc3800a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc39819a1c0 tx=0x7fc3880073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:37.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:37.851+0000 7fc39c8af700 1 -- 172.21.15.82:0/673999998 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc38c030020 con 0x7fc398105e60 2024-04-21T08:46:37.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:37 smithi106 ceph-mon[31072]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 311 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:38.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.054+0000 7fc39d8b1700 1 -- 172.21.15.82:0/673999998 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc398062e80 con 0x7fc3800a3c40 2024-04-21T08:46:38.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.055+0000 7fc39c8af700 1 -- 172.21.15.82:0/673999998 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc398062e80 con 0x7fc3800a3c40 2024-04-21T08:46:38.055 INFO:teuthology.orchestra.run.smithi082.stdout:HOST ADDR LABELS STATUS 2024-04-21T08:46:38.055 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 172.21.15.82 2024-04-21T08:46:38.055 INFO:teuthology.orchestra.run.smithi082.stdout:smithi106 172.21.15.106 2024-04-21T08:46:38.055 INFO:teuthology.orchestra.run.smithi082.stdout:smithi134 172.21.15.134 2024-04-21T08:46:38.056 INFO:teuthology.orchestra.run.smithi082.stdout:3 hosts in cluster 2024-04-21T08:46:38.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.057+0000 7fc39d8b1700 1 -- 172.21.15.82:0/673999998 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc3800a3c40 msgr2=0x7fc3800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:38.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.057+0000 7fc39d8b1700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc3800a3c40 0x7fc3800a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc39819a1c0 tx=0x7fc3880073d0 comp rx=0 tx=0).stop 2024-04-21T08:46:38.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.057+0000 7fc39d8b1700 1 -- 172.21.15.82:0/673999998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc398105e60 msgr2=0x7fc398199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:38.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.057+0000 7fc39d8b1700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc398105e60 0x7fc398199270 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0047b0 tx=0x7fc38c005f10 comp rx=0 tx=0).stop 2024-04-21T08:46:38.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.058+0000 7fc39d8b1700 1 -- 172.21.15.82:0/673999998 shutdown_connections 2024-04-21T08:46:38.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.058+0000 7fc39d8b1700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fc3800a3c40 0x7fc3800a6100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:38.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.058+0000 7fc39d8b1700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc398108790 0x7fc3981a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:38.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.058+0000 7fc39d8b1700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc398103530 0x7fc398198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:38.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.058+0000 7fc39d8b1700 1 --2- 172.21.15.82:0/673999998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc398105e60 0x7fc398199270 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:38.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.058+0000 7fc39d8b1700 1 -- 172.21.15.82:0/673999998 >> 172.21.15.82:0/673999998 conn(0x7fc3980fcf40 msgr2=0x7fc398103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:38.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.058+0000 7fc39d8b1700 1 -- 172.21.15.82:0/673999998 shutdown_connections 2024-04-21T08:46:38.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:38.058+0000 7fc39d8b1700 1 -- 172.21.15.82:0/673999998 wait complete. 2024-04-21T08:46:38.840 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-21T08:46:39.085 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:46:39.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:39 smithi134 ceph-mon[31772]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:39.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:39 smithi134 ceph-mon[31772]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 282 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:39.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:39 smithi082 ceph-mon[27967]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:39.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:39 smithi082 ceph-mon[27967]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 282 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:39.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:39 smithi106 ceph-mon[31072]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:39.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:39 smithi106 ceph-mon[31072]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 282 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:40.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.852+0000 7f851fd21700 1 -- 172.21.15.82:0/793857468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8518105ec0 msgr2=0x7f85181082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:40.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 --2- 172.21.15.82:0/793857468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8518105ec0 0x7f85181082b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8508009990 tx=0x7f8508009ca0 comp rx=0 tx=0).stop 2024-04-21T08:46:40.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 -- 172.21.15.82:0/793857468 shutdown_connections 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 --2- 172.21.15.82:0/793857468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85181087f0 0x7f851810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 --2- 172.21.15.82:0/793857468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8518103590 0x7f8518105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 --2- 172.21.15.82:0/793857468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8518105ec0 0x7f85181082b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 -- 172.21.15.82:0/793857468 >> 172.21.15.82:0/793857468 conn(0x7f85180fcf40 msgr2=0x7f85180ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 -- 172.21.15.82:0/793857468 shutdown_connections 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 -- 172.21.15.82:0/793857468 wait complete. 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 Processor -- start 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 -- start start 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8518103590 0x7f8518198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8518105ec0 0x7f85181991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85181087f0 0x7f85181a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f851810e550 con 0x7f8518105ec0 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f851810e3d0 con 0x7f85181087f0 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.853+0000 7f851fd21700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f851810e6d0 con 0x7f8518103590 2024-04-21T08:46:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851d2bc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8518105ec0 0x7f85181991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851d2bc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8518105ec0 0x7f85181991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42306/0 (socket says 172.21.15.82:42306) 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851d2bc700 1 -- 172.21.15.82:0/278030331 learned_addr learned my addr 172.21.15.82:0/278030331 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851dabd700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8518103590 0x7f8518198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851e2be700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85181087f0 0x7f85181a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851d2bc700 1 -- 172.21.15.82:0/278030331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8518103590 msgr2=0x7f8518198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851d2bc700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8518103590 0x7f8518198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851d2bc700 1 -- 172.21.15.82:0/278030331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85181087f0 msgr2=0x7f85181a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851d2bc700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85181087f0 0x7f85181a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851d2bc700 1 -- 172.21.15.82:0/278030331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85181a08e0 con 0x7f8518105ec0 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851d2bc700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8518105ec0 0x7f85181991a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8508006010 tx=0x7f85080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f850effd700 1 -- 172.21.15.82:0/278030331 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508027070 con 0x7f8518105ec0 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851fd21700 1 -- 172.21.15.82:0/278030331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85181a0b70 con 0x7f8518105ec0 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f850effd700 1 -- 172.21.15.82:0/278030331 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8508004000 con 0x7f8518105ec0 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f850effd700 1 -- 172.21.15.82:0/278030331 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508022c80 con 0x7f8518105ec0 2024-04-21T08:46:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.854+0000 7f851fd21700 1 -- 172.21.15.82:0/278030331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85181a1020 con 0x7f8518105ec0 2024-04-21T08:46:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.855+0000 7f851fd21700 1 -- 172.21.15.82:0/278030331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8518069730 con 0x7f8518105ec0 2024-04-21T08:46:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.856+0000 7f850effd700 1 -- 172.21.15.82:0/278030331 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f850802a070 con 0x7f8518105ec0 2024-04-21T08:46:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.857+0000 7f850effd700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f85040a3c40 0x7f85040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.857+0000 7f851dabd700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f85040a3c40 0x7f85040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.857+0000 7f850effd700 1 -- 172.21.15.82:0/278030331 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f85080d1e50 con 0x7f8518105ec0 2024-04-21T08:46:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.857+0000 7f851dabd700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f85040a3c40 0x7f85040a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8514005950 tx=0x7f851400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:40.861+0000 7f850effd700 1 -- 172.21.15.82:0/278030331 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8508031070 con 0x7f8518105ec0 2024-04-21T08:46:41.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.069+0000 7f851fd21700 1 -- 172.21.15.82:0/278030331 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8518062e80 con 0x7f85040a3c40 2024-04-21T08:46:41.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.071+0000 7f850effd700 1 -- 172.21.15.82:0/278030331 <== mgr.14152 v2:172.21.15.82:6800/2138749385 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8518062e80 con 0x7f85040a3c40 2024-04-21T08:46:41.071 INFO:teuthology.orchestra.run.smithi082.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-21T08:46:41.071 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400SD400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-21T08:46:41.071 INFO:teuthology.orchestra.run.smithi082.stdout:smithi106 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013R400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-21T08:46:41.071 INFO:teuthology.orchestra.run.smithi082.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-21T08:46:41.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.073+0000 7f851fd21700 1 -- 172.21.15.82:0/278030331 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f85040a3c40 msgr2=0x7f85040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.073+0000 7f851fd21700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f85040a3c40 0x7f85040a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8514005950 tx=0x7f851400a300 comp rx=0 tx=0).stop 2024-04-21T08:46:41.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.073+0000 7f851fd21700 1 -- 172.21.15.82:0/278030331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8518105ec0 msgr2=0x7f85181991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:41.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.073+0000 7f851fd21700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8518105ec0 0x7f85181991a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8508006010 tx=0x7f85080047b0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.073+0000 7f851fd21700 1 -- 172.21.15.82:0/278030331 shutdown_connections 2024-04-21T08:46:41.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.073+0000 7f851fd21700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f85040a3c40 0x7f85040a6100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.073+0000 7f851fd21700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85181087f0 0x7f85181a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.073+0000 7f851fd21700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8518103590 0x7f8518198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.073+0000 7f851fd21700 1 --2- 172.21.15.82:0/278030331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8518105ec0 0x7f85181991a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:41.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.073+0000 7f851fd21700 1 -- 172.21.15.82:0/278030331 >> 172.21.15.82:0/278030331 conn(0x7f85180fcf40 msgr2=0x7f8518105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:41.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.073+0000 7f851fd21700 1 -- 172.21.15.82:0/278030331 shutdown_connections 2024-04-21T08:46:41.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:41.073+0000 7f851fd21700 1 -- 172.21.15.82:0/278030331 wait complete. 2024-04-21T08:46:41.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:41 smithi134 ceph-mon[31772]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:41.779 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:41 smithi082 ceph-mon[27967]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:41.806 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-21T08:46:41.816 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-04-21T08:46:41.816 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-21T08:46:41.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:41 smithi106 ceph-mon[31072]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:46:42.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:42 smithi134 ceph-mon[31772]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:42.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:42 smithi082 ceph-mon[27967]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:42.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:42 smithi106 ceph-mon[31072]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:46:43.598 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:43 smithi082 ceph-mon[27967]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:43.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:43 smithi134 ceph-mon[31772]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:43.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.817+0000 7fcc01808700 1 -- 172.21.15.82:0/1180525267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbfc1019b0 msgr2=0x7fcbfc101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.817+0000 7fcc01808700 1 --2- 172.21.15.82:0/1180525267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbfc1019b0 0x7fcbfc101dd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0099c0 tx=0x7fcbec009cd0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.818+0000 7fcc01808700 1 -- 172.21.15.82:0/1180525267 shutdown_connections 2024-04-21T08:46:43.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.818+0000 7fcc01808700 1 --2- 172.21.15.82:0/1180525267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbfc103550 0x7fcbfc109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.818+0000 7fcc01808700 1 --2- 172.21.15.82:0/1180525267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbfc1019b0 0x7fcbfc101dd0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.818+0000 7fcc01808700 1 --2- 172.21.15.82:0/1180525267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbfc102bb0 0x7fcbfc103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.818+0000 7fcc01808700 1 -- 172.21.15.82:0/1180525267 >> 172.21.15.82:0/1180525267 conn(0x7fcbfc0fcf70 msgr2=0x7fcbfc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:43.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.818+0000 7fcc01808700 1 -- 172.21.15.82:0/1180525267 shutdown_connections 2024-04-21T08:46:43.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.818+0000 7fcc01808700 1 -- 172.21.15.82:0/1180525267 wait complete. 2024-04-21T08:46:43.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.818+0000 7fcc01808700 1 Processor -- start 2024-04-21T08:46:43.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcc01808700 1 -- start start 2024-04-21T08:46:43.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcc01808700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbfc1019b0 0x7fcbfc06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcc01808700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbfc102bb0 0x7fcbfc19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcc01808700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbfc103550 0x7fcbfc19fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcc01808700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc10d420 con 0x7fcbfc102bb0 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcc01808700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc10d2a0 con 0x7fcbfc103550 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcc01808700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc10d5a0 con 0x7fcbfc1019b0 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcbfa7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbfc102bb0 0x7fcbfc19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcbfa7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbfc102bb0 0x7fcbfc19d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42068/0 (socket says 172.21.15.82:42068) 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcbfa7fc700 1 -- 172.21.15.82:0/1580788785 learned_addr learned my addr 172.21.15.82:0/1580788785 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcbfaffd700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbfc1019b0 0x7fcbfc06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcbfb7fe700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbfc103550 0x7fcbfc19fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcbfa7fc700 1 -- 172.21.15.82:0/1580788785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbfc1019b0 msgr2=0x7fcbfc06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcbfa7fc700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbfc1019b0 0x7fcbfc06bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcbfa7fc700 1 -- 172.21.15.82:0/1580788785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbfc103550 msgr2=0x7fcbfc19fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:43.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcbfa7fc700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbfc103550 0x7fcbfc19fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:43.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcbfa7fc700 1 -- 172.21.15.82:0/1580788785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc1a08c0 con 0x7fcbfc102bb0 2024-04-21T08:46:43.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcbfb7fe700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbfc103550 0x7fcbfc19fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:43.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.819+0000 7fcbfa7fc700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbfc102bb0 0x7fcbfc19d9b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000d980 tx=0x7fcbf000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.820+0000 7fcc00806700 1 -- 172.21.15.82:0/1580788785 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0016070 con 0x7fcbfc102bb0 2024-04-21T08:46:43.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.820+0000 7fcc00806700 1 -- 172.21.15.82:0/1580788785 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf0004830 con 0x7fcbfc102bb0 2024-04-21T08:46:43.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.820+0000 7fcc01808700 1 -- 172.21.15.82:0/1580788785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbfc1a0bb0 con 0x7fcbfc102bb0 2024-04-21T08:46:43.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.820+0000 7fcc00806700 1 -- 172.21.15.82:0/1580788785 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00054c0 con 0x7fcbfc102bb0 2024-04-21T08:46:43.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.820+0000 7fcc01808700 1 -- 172.21.15.82:0/1580788785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbfc1a10f0 con 0x7fcbfc102bb0 2024-04-21T08:46:43.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.821+0000 7fcc01808700 1 -- 172.21.15.82:0/1580788785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbfc069730 con 0x7fcbfc102bb0 2024-04-21T08:46:43.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.822+0000 7fcc00806700 1 -- 172.21.15.82:0/1580788785 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcbf0004220 con 0x7fcbfc102bb0 2024-04-21T08:46:43.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.822+0000 7fcc00806700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fcbe40a3c40 0x7fcbe40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:43.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.822+0000 7fcbfaffd700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fcbe40a3c40 0x7fcbe40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:43.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.822+0000 7fcc00806700 1 -- 172.21.15.82:0/1580788785 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fcbf00d0b30 con 0x7fcbfc102bb0 2024-04-21T08:46:43.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.823+0000 7fcbfaffd700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fcbe40a3c40 0x7fcbe40a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcbec006010 tx=0x7fcbec0036a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:43.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:43.826+0000 7fcc00806700 1 -- 172.21.15.82:0/1580788785 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcbf001f020 con 0x7fcbfc102bb0 2024-04-21T08:46:43.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:43 smithi106 ceph-mon[31072]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:44.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.071+0000 7fcc01808700 1 -- 172.21.15.82:0/1580788785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fcbfc02e050 con 0x7fcbfc102bb0 2024-04-21T08:46:44.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.071+0000 7fcc00806700 1 -- 172.21.15.82:0/1580788785 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3100 (secure 0 0 0) 0x7fcbf00193b0 con 0x7fcbfc102bb0 2024-04-21T08:46:44.071 INFO:teuthology.orchestra.run.smithi082.stdout:e27 2024-04-21T08:46:44.071 INFO:teuthology.orchestra.run.smithi082.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T08:46:44.071 INFO:teuthology.orchestra.run.smithi082.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T08:46:44.072 INFO:teuthology.orchestra.run.smithi082.stdout:legacy client fscid: 1 2024-04-21T08:46:44.072 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:44.072 INFO:teuthology.orchestra.run.smithi082.stdout:Filesystem 'cephfs' (1) 2024-04-21T08:46:44.072 INFO:teuthology.orchestra.run.smithi082.stdout:fs_name cephfs 2024-04-21T08:46:44.072 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 27 2024-04-21T08:46:44.072 INFO:teuthology.orchestra.run.smithi082.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-21T08:46:44.072 INFO:teuthology.orchestra.run.smithi082.stdout:created 2024-04-21T08:44:33.819176+0000 2024-04-21T08:46:44.072 INFO:teuthology.orchestra.run.smithi082.stdout:modified 2024-04-21T08:45:02.005557+0000 2024-04-21T08:46:44.072 INFO:teuthology.orchestra.run.smithi082.stdout:tableserver 0 2024-04-21T08:46:44.072 INFO:teuthology.orchestra.run.smithi082.stdout:root 0 2024-04-21T08:46:44.072 INFO:teuthology.orchestra.run.smithi082.stdout:session_timeout 300 2024-04-21T08:46:44.072 INFO:teuthology.orchestra.run.smithi082.stdout:session_autoclose 300 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:max_file_size 1099511627776 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:max_xattr_size 65536 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:required_client_features {} 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:last_failure 0 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:last_failure_osd_epoch 0 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:max_mds 5 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:in 0,1,2,3,4 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:up {0=24509,1=24481,2=24476,3=24409,4=24407} 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:failed 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:damaged 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:stopped 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:data_pools [3,4] 2024-04-21T08:46:44.073 INFO:teuthology.orchestra.run.smithi082.stdout:metadata_pool 2 2024-04-21T08:46:44.074 INFO:teuthology.orchestra.run.smithi082.stdout:inline_data disabled 2024-04-21T08:46:44.074 INFO:teuthology.orchestra.run.smithi082.stdout:balancer 2024-04-21T08:46:44.074 INFO:teuthology.orchestra.run.smithi082.stdout:bal_rank_mask -1 2024-04-21T08:46:44.074 INFO:teuthology.orchestra.run.smithi082.stdout:standby_count_wanted 0 2024-04-21T08:46:44.074 INFO:teuthology.orchestra.run.smithi082.stdout:qdb_cluster leader: 24509 members: 24481,24509,24476,24409,24407 2024-04-21T08:46:44.074 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b{0:24509} state up:active seq 3 addr [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:46:44.074 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c{0:24460} state up:standby-replay seq 1 addr [v2:172.21.15.134:6836/1870695477,v1:172.21.15.134:6837/1870695477] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:46:44.074 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.i{1:24481} state up:active seq 11 addr [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:46:44.074 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.f{1:24451} state up:standby-replay seq 1 addr [v2:172.21.15.134:6834/2962604291,v1:172.21.15.134:6835/2962604291] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:46:44.074 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.h{2:24476} state up:active seq 12 addr [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:46:44.074 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.e{2:24440} state up:standby-replay seq 1 addr [v2:172.21.15.106:6834/3901395826,v1:172.21.15.106:6835/3901395826] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:46:44.074 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.l{3:24409} state up:active seq 17 addr [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:46:44.075 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a{3:14643} state up:standby-replay seq 1 addr [v2:172.21.15.82:6840/1327149046,v1:172.21.15.82:6841/1327149046] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:46:44.075 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.k{4:24407} state up:active seq 17 addr [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:46:44.075 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.g{4:14631} state up:standby-replay seq 1 addr [v2:172.21.15.82:6838/3697606919,v1:172.21.15.82:6839/3697606919] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:46:44.075 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:44.075 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:44.075 INFO:teuthology.orchestra.run.smithi082.stdout:Standby daemons: 2024-04-21T08:46:44.075 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:44.075 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.j{-1:14556} state up:standby seq 1 addr [v2:172.21.15.82:6834/467976762,v1:172.21.15.82:6835/467976762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:46:44.075 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d{-1:14568} state up:standby seq 1 addr [v2:172.21.15.82:6836/3751789078,v1:172.21.15.82:6837/3751789078] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:46:44.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.073+0000 7fcc01808700 1 -- 172.21.15.82:0/1580788785 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fcbe40a3c40 msgr2=0x7fcbe40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.073+0000 7fcc01808700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fcbe40a3c40 0x7fcbe40a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcbec006010 tx=0x7fcbec0036a0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.073+0000 7fcc01808700 1 -- 172.21.15.82:0/1580788785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbfc102bb0 msgr2=0x7fcbfc19d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:44.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.073+0000 7fcc01808700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbfc102bb0 0x7fcbfc19d9b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000d980 tx=0x7fcbf000dc90 comp rx=0 tx=0).stop 2024-04-21T08:46:44.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.073+0000 7fcc01808700 1 -- 172.21.15.82:0/1580788785 shutdown_connections 2024-04-21T08:46:44.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.073+0000 7fcc01808700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fcbe40a3c40 0x7fcbe40a6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.073+0000 7fcc01808700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbfc103550 0x7fcbfc19fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.073+0000 7fcc01808700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbfc1019b0 0x7fcbfc06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.073+0000 7fcc01808700 1 --2- 172.21.15.82:0/1580788785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbfc102bb0 0x7fcbfc19d9b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:44.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.073+0000 7fcc01808700 1 -- 172.21.15.82:0/1580788785 >> 172.21.15.82:0/1580788785 conn(0x7fcbfc0fcf70 msgr2=0x7fcbfc106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:44.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.074+0000 7fcc01808700 1 -- 172.21.15.82:0/1580788785 shutdown_connections 2024-04-21T08:46:44.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:44.074+0000 7fcc01808700 1 -- 172.21.15.82:0/1580788785 wait complete. 2024-04-21T08:46:44.077 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-04-21T08:46:44.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:44 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1580788785' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T08:46:44.789 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:44 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1580788785' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T08:46:44.818 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-21T08:46:44.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:44 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1580788785' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T08:46:45.056 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:46:45.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:45 smithi134 ceph-mon[31772]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:45.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:45 smithi082 ceph-mon[27967]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:45.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:45 smithi106 ceph-mon[31072]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-21T08:46:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.815+0000 7f57a2068700 1 -- 172.21.15.82:0/1776871964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c102d10 msgr2=0x7f579c1095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.815+0000 7f57a2068700 1 --2- 172.21.15.82:0/1776871964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c102d10 0x7f579c1095c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f579400b0d0 tx=0x7f579400b3e0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.815+0000 7f57a2068700 1 -- 172.21.15.82:0/1776871964 shutdown_connections 2024-04-21T08:46:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.815+0000 7f57a2068700 1 --2- 172.21.15.82:0/1776871964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c102d10 0x7f579c1095c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.815+0000 7f57a2068700 1 --2- 172.21.15.82:0/1776871964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579c101170 0x7f579c101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.815+0000 7f57a2068700 1 --2- 172.21.15.82:0/1776871964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f579c102370 0x7f579c1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.815+0000 7f57a2068700 1 -- 172.21.15.82:0/1776871964 >> 172.21.15.82:0/1776871964 conn(0x7f579c0fc710 msgr2=0x7f579c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:46.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.815+0000 7f57a2068700 1 -- 172.21.15.82:0/1776871964 shutdown_connections 2024-04-21T08:46:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.815+0000 7f57a2068700 1 -- 172.21.15.82:0/1776871964 wait complete. 2024-04-21T08:46:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.816+0000 7f57a2068700 1 Processor -- start 2024-04-21T08:46:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.816+0000 7f57a2068700 1 -- start start 2024-04-21T08:46:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.816+0000 7f57a2068700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f579c101170 0x7f579c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.816+0000 7f57a2068700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c102370 0x7f579c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.816+0000 7f57a2068700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579c102d10 0x7f579c1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.816+0000 7f57a2068700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f579c10d580 con 0x7f579c102370 2024-04-21T08:46:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.816+0000 7f57a2068700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f579c10d400 con 0x7f579c101170 2024-04-21T08:46:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.816+0000 7f57a2068700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f579c10d700 con 0x7f579c102d10 2024-04-21T08:46:46.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.816+0000 7f579affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c102370 0x7f579c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.816+0000 7f579affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c102370 0x7f579c1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42094/0 (socket says 172.21.15.82:42094) 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.816+0000 7f579affd700 1 -- 172.21.15.82:0/2221941832 learned_addr learned my addr 172.21.15.82:0/2221941832 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f579bfff700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579c102d10 0x7f579c1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f579affd700 1 -- 172.21.15.82:0/2221941832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579c102d10 msgr2=0x7f579c1a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f579b7fe700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f579c101170 0x7f579c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f579affd700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579c102d10 0x7f579c1a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f579affd700 1 -- 172.21.15.82:0/2221941832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f579c101170 msgr2=0x7f579c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f579affd700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f579c101170 0x7f579c198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f579affd700 1 -- 172.21.15.82:0/2221941832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c1a0970 con 0x7f579c102370 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f579bfff700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579c102d10 0x7f579c1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f579affd700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c102370 0x7f579c1992b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f579000b530 tx=0x7f579000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f5798ff9700 1 -- 172.21.15.82:0/2221941832 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790015020 con 0x7f579c102370 2024-04-21T08:46:46.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f57a2068700 1 -- 172.21.15.82:0/2221941832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579c1a0c60 con 0x7f579c102370 2024-04-21T08:46:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f5798ff9700 1 -- 172.21.15.82:0/2221941832 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5790011ce0 con 0x7f579c102370 2024-04-21T08:46:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f5798ff9700 1 -- 172.21.15.82:0/2221941832 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790010d20 con 0x7f579c102370 2024-04-21T08:46:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.817+0000 7f57a2068700 1 -- 172.21.15.82:0/2221941832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f579c1a11a0 con 0x7f579c102370 2024-04-21T08:46:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.818+0000 7f57a2068700 1 -- 172.21.15.82:0/2221941832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f579c069730 con 0x7f579c102370 2024-04-21T08:46:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.819+0000 7f5798ff9700 1 -- 172.21.15.82:0/2221941832 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5790018020 con 0x7f579c102370 2024-04-21T08:46:46.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.820+0000 7f5798ff9700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f57840a3c40 0x7f57840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.820+0000 7f579b7fe700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f57840a3c40 0x7f57840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.820+0000 7f5798ff9700 1 -- 172.21.15.82:0/2221941832 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f57900d0550 con 0x7f579c102370 2024-04-21T08:46:46.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.820+0000 7f579b7fe700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f57840a3c40 0x7f57840a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f579c19a200 tx=0x7f578c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:46.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:46.824+0000 7f5798ff9700 1 -- 172.21.15.82:0/2221941832 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f579001f020 con 0x7f579c102370 2024-04-21T08:46:47.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.023+0000 7f57a2068700 1 -- 172.21.15.82:0/2221941832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f579c1a1480 con 0x7f579c102370 2024-04-21T08:46:47.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.024+0000 7f5798ff9700 1 -- 172.21.15.82:0/2221941832 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5711 (secure 0 0 0) 0x7f579c1a1480 con 0x7f579c102370 2024-04-21T08:46:47.024 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 93 2024-04-21T08:46:47.024 INFO:teuthology.orchestra.run.smithi082.stdout:fsid 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T08:46:47.024 INFO:teuthology.orchestra.run.smithi082.stdout:created 2024-04-21T08:36:25.047081+0000 2024-04-21T08:46:47.024 INFO:teuthology.orchestra.run.smithi082.stdout:modified 2024-04-21T08:46:27.475773+0000 2024-04-21T08:46:47.024 INFO:teuthology.orchestra.run.smithi082.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-21T08:46:47.024 INFO:teuthology.orchestra.run.smithi082.stdout:crush_version 27 2024-04-21T08:46:47.024 INFO:teuthology.orchestra.run.smithi082.stdout:full_ratio 0.95 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:backfillfull_ratio 0.9 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:nearfull_ratio 0.85 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:require_min_compat_client luminous 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:min_compat_client luminous 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:require_osd_release squid 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:stretch_mode_enabled false 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:max_osd 12 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6802/1328184221,v1:172.21.15.82:6803/1328184221] [v2:172.21.15.82:6804/1328184221,v1:172.21.15.82:6805/1328184221] exists,up b97aa94e-52aa-4c7c-8a91-252060989aea 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6800/1176470970,v1:172.21.15.106:6801/1176470970] [v2:172.21.15.106:6802/1176470970,v1:172.21.15.106:6803/1176470970] exists,up 209098d5-cae2-4947-8cb5-93c4c7fea11d 2024-04-21T08:46:47.025 INFO:teuthology.orchestra.run.smithi082.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/3690916351,v1:172.21.15.134:6801/3690916351] [v2:172.21.15.134:6802/3690916351,v1:172.21.15.134:6803/3690916351] exists,up dceeca7d-3295-4c47-93f3-694da1b97eff 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6810/2296521685,v1:172.21.15.82:6811/2296521685] [v2:172.21.15.82:6812/2296521685,v1:172.21.15.82:6813/2296521685] exists,up db05ff77-7134-4823-a2ed-5f55bbc5d66c 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:osd.4 up in weight 1 up_from 33 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6808/1658170110,v1:172.21.15.106:6809/1658170110] [v2:172.21.15.106:6810/1658170110,v1:172.21.15.106:6811/1658170110] exists,up 56e090a3-ed81-46b8-8c14-a6fdbd57c344 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:osd.5 up in weight 1 up_from 38 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/1529948096,v1:172.21.15.134:6809/1529948096] [v2:172.21.15.134:6810/1529948096,v1:172.21.15.134:6811/1529948096] exists,up c045cd42-40b5-4b5c-9d69-69a7ec4f0bef 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:osd.6 up in weight 1 up_from 46 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6818/1243429969,v1:172.21.15.82:6819/1243429969] [v2:172.21.15.82:6820/1243429969,v1:172.21.15.82:6821/1243429969] exists,up 2c1f3d11-aa46-4d37-9e6c-9b2778d3306c 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:osd.7 up in weight 1 up_from 51 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6816/3692373036,v1:172.21.15.106:6817/3692373036] [v2:172.21.15.106:6818/3692373036,v1:172.21.15.106:6819/3692373036] exists,up 62f2af9e-02e2-436b-9041-60e4523eeeff 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:osd.8 up in weight 1 up_from 59 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/1700011684,v1:172.21.15.134:6817/1700011684] [v2:172.21.15.134:6818/1700011684,v1:172.21.15.134:6819/1700011684] exists,up e39282b2-0f53-4c57-a68c-771857b35413 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:osd.9 up in weight 1 up_from 67 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6826/112604646,v1:172.21.15.82:6827/112604646] [v2:172.21.15.82:6828/112604646,v1:172.21.15.82:6829/112604646] exists,up 5d8c9e2a-4cb6-442a-a767-9452c6cb7f06 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:osd.10 up in weight 1 up_from 72 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6824/204007808,v1:172.21.15.106:6825/204007808] [v2:172.21.15.106:6826/204007808,v1:172.21.15.106:6827/204007808] exists,up 73ee0364-4255-443c-ac99-c7f6286afeee 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/1549345879,v1:172.21.15.134:6825/1549345879] [v2:172.21.15.134:6826/1549345879,v1:172.21.15.134:6827/1549345879] exists,up 0f38b227-d954-4575-aced-ffd87058bf16 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.a [2,11] 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.b [8,11] 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.16 [2,4] 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.0 [4,8] 2024-04-21T08:46:47.026 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.2 [3,9] 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.3 [4,9] 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.17 [4,8,5,7] 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.1e [3,9] 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.28 [5,6] 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.31 [4,6] 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 4.6 [5,8] 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 4.e [9,11] 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 4.18 [9,8] 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 4.1c [9,8] 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/4175378061 expires 2024-04-22T08:37:26.227727+0000 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/2889165689 expires 2024-04-22T08:37:26.227727+0000 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6800/880382736 expires 2024-04-22T08:37:26.227727+0000 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/3352705806 expires 2024-04-22T08:37:26.227727+0000 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6800/2710226711 expires 2024-04-22T08:36:59.578707+0000 2024-04-21T08:46:47.027 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/4224740504 expires 2024-04-22T08:36:59.578707+0000 2024-04-21T08:46:47.028 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/2873389149 expires 2024-04-22T08:36:59.578707+0000 2024-04-21T08:46:47.028 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6801/880382736 expires 2024-04-22T08:37:26.227727+0000 2024-04-21T08:46:47.028 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6801/2710226711 expires 2024-04-22T08:36:59.578707+0000 2024-04-21T08:46:47.028 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/3032518359 expires 2024-04-22T08:36:59.578707+0000 2024-04-21T08:46:47.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.026+0000 7f57a2068700 1 -- 172.21.15.82:0/2221941832 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f57840a3c40 msgr2=0x7f57840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:47.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.026+0000 7f57a2068700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f57840a3c40 0x7f57840a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f579c19a200 tx=0x7f578c008040 comp rx=0 tx=0).stop 2024-04-21T08:46:47.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.026+0000 7f57a2068700 1 -- 172.21.15.82:0/2221941832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c102370 msgr2=0x7f579c1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:47.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.026+0000 7f57a2068700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c102370 0x7f579c1992b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f579000b530 tx=0x7f579000b8f0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.026+0000 7f57a2068700 1 -- 172.21.15.82:0/2221941832 shutdown_connections 2024-04-21T08:46:47.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.026+0000 7f57a2068700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f57840a3c40 0x7f57840a6100 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.026+0000 7f57a2068700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579c102d10 0x7f579c1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.026+0000 7f57a2068700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f579c101170 0x7f579c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.026+0000 7f57a2068700 1 --2- 172.21.15.82:0/2221941832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c102370 0x7f579c1992b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.026+0000 7f57a2068700 1 -- 172.21.15.82:0/2221941832 >> 172.21.15.82:0/2221941832 conn(0x7f579c0fc710 msgr2=0x7f579c0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:47.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.027+0000 7f57a2068700 1 -- 172.21.15.82:0/2221941832 shutdown_connections 2024-04-21T08:46:47.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.027+0000 7f57a2068700 1 -- 172.21.15.82:0/2221941832 wait complete. 2024-04-21T08:46:47.724 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:47 smithi082 ceph-mon[27967]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:47.724 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:47 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2221941832' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T08:46:47.754 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-21T08:46:47.798 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:46:47.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:47 smithi106 ceph-mon[31072]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:47.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:47 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2221941832' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T08:46:47.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.906+0000 7f9a2a359640 1 -- 172.21.15.82:0/3061697969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a24105320 msgr2=0x7f9a24107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:47.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.906+0000 7f9a2a359640 1 --2- 172.21.15.82:0/3061697969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a24105320 0x7f9a24107710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009900 tx=0x7f9a1402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:47.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.906+0000 7f9a2a359640 1 -- 172.21.15.82:0/3061697969 shutdown_connections 2024-04-21T08:46:47.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.906+0000 7f9a2a359640 1 --2- 172.21.15.82:0/3061697969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a24107c50 0x7f9a2410a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.906+0000 7f9a2a359640 1 --2- 172.21.15.82:0/3061697969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a24105320 0x7f9a24107710 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.906+0000 7f9a2a359640 1 --2- 172.21.15.82:0/3061697969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a241029f0 0x7f9a24104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.906+0000 7f9a2a359640 1 -- 172.21.15.82:0/3061697969 >> 172.21.15.82:0/3061697969 conn(0x7f9a240fc3e0 msgr2=0x7f9a240fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:47.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.906+0000 7f9a2a359640 1 -- 172.21.15.82:0/3061697969 shutdown_connections 2024-04-21T08:46:47.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.906+0000 7f9a2a359640 1 -- 172.21.15.82:0/3061697969 wait complete. 2024-04-21T08:46:47.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.906+0000 7f9a2a359640 1 Processor -- start 2024-04-21T08:46:47.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.906+0000 7f9a2a359640 1 -- start start 2024-04-21T08:46:47.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a2a359640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a241029f0 0x7f9a24102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:47.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a2a359640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a24105320 0x7f9a241009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:47.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a2a359640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a24107c50 0x7f9a24100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:47.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a2a359640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2410daf0 con 0x7f9a24107c50 2024-04-21T08:46:47.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a2a359640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2410d970 con 0x7f9a241029f0 2024-04-21T08:46:47.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a2a359640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2410dc70 con 0x7f9a24105320 2024-04-21T08:46:47.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a288f6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a24107c50 0x7f9a24100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:47.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a288f6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a24107c50 0x7f9a24100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42102/0 (socket says 172.21.15.82:42102) 2024-04-21T08:46:47.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a288f6640 1 -- 172.21.15.82:0/1662304038 learned_addr learned my addr 172.21.15.82:0/1662304038 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:47.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a23fff640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a241029f0 0x7f9a24102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:47.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a288f6640 1 -- 172.21.15.82:0/1662304038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a24105320 msgr2=0x7f9a241009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a237fe640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a24105320 0x7f9a241009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a288f6640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a24105320 0x7f9a241009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a288f6640 1 -- 172.21.15.82:0/1662304038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a241029f0 msgr2=0x7f9a24102360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a288f6640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a241029f0 0x7f9a24102360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a288f6640 1 -- 172.21.15.82:0/1662304038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a24101770 con 0x7f9a24107c50 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a23fff640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a241029f0 0x7f9a24102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a288f6640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a24107c50 0x7f9a24100ee0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00b970 tx=0x7f9a1c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a217fa640 1 -- 172.21.15.82:0/1662304038 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c019070 con 0x7f9a24107c50 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a217fa640 1 -- 172.21.15.82:0/1662304038 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c014440 con 0x7f9a24107c50 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a2a359640 1 -- 172.21.15.82:0/1662304038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a241a2340 con 0x7f9a24107c50 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a217fa640 1 -- 172.21.15.82:0/1662304038 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0075d0 con 0x7f9a24107c50 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.907+0000 7f9a2a359640 1 -- 172.21.15.82:0/1662304038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a241a28c0 con 0x7f9a24107c50 2024-04-21T08:46:47.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.908+0000 7f9a2a359640 1 -- 172.21.15.82:0/1662304038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a240fded0 con 0x7f9a24107c50 2024-04-21T08:46:47.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.909+0000 7f9a217fa640 1 -- 172.21.15.82:0/1662304038 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9a1c01c070 con 0x7f9a24107c50 2024-04-21T08:46:47.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.910+0000 7f9a217fa640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9a0c0a3bb0 0x7f9a0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:47.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.910+0000 7f9a217fa640 1 -- 172.21.15.82:0/1662304038 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f9a1c0d0360 con 0x7f9a24107c50 2024-04-21T08:46:47.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.910+0000 7f9a23fff640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9a0c0a3bb0 0x7f9a0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:47.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.910+0000 7f9a23fff640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9a0c0a3bb0 0x7f9a0c0a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9a180058c0 tx=0x7f9a18005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:47.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:47.914+0000 7f9a217fa640 1 -- 172.21.15.82:0/1662304038 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a1c09c970 con 0x7f9a24107c50 2024-04-21T08:46:47.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:47 smithi134 ceph-mon[31772]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:47.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:47 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2221941832' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T08:46:48.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.093+0000 7f9a2a359640 1 -- 172.21.15.82:0/1662304038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a24062b50 con 0x7f9a24107c50 2024-04-21T08:46:48.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.094+0000 7f9a217fa640 1 -- 172.21.15.82:0/1662304038 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24870 (secure 0 0 0) 0x7f9a1c09c270 con 0x7f9a24107c50 2024-04-21T08:46:48.094 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:48.094 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":93,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:46:27.475773+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:46:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.096+0000 7f9a2a359640 1 -- 172.21.15.82:0/1662304038 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9a0c0a3bb0 msgr2=0x7f9a0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.096+0000 7f9a2a359640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9a0c0a3bb0 0x7f9a0c0a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9a180058c0 tx=0x7f9a18005d30 comp rx=0 tx=0).stop 2024-04-21T08:46:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.096+0000 7f9a2a359640 1 -- 172.21.15.82:0/1662304038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a24107c50 msgr2=0x7f9a24100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.096+0000 7f9a2a359640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a24107c50 0x7f9a24100ee0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00b970 tx=0x7f9a1c00be30 comp rx=0 tx=0).stop 2024-04-21T08:46:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.096+0000 7f9a2a359640 1 -- 172.21.15.82:0/1662304038 shutdown_connections 2024-04-21T08:46:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.096+0000 7f9a2a359640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f9a0c0a3bb0 0x7f9a0c0a6070 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.096+0000 7f9a2a359640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a24107c50 0x7f9a24100ee0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.096+0000 7f9a2a359640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a24105320 0x7f9a241009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.096+0000 7f9a2a359640 1 --2- 172.21.15.82:0/1662304038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a241029f0 0x7f9a24102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.096+0000 7f9a2a359640 1 -- 172.21.15.82:0/1662304038 >> 172.21.15.82:0/1662304038 conn(0x7f9a240fc3e0 msgr2=0x7f9a24103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.096+0000 7f9a2a359640 1 -- 172.21.15.82:0/1662304038 shutdown_connections 2024-04-21T08:46:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.096+0000 7f9a2a359640 1 -- 172.21.15.82:0/1662304038 wait complete. 2024-04-21T08:46:48.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:46:48.108 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:46:48.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.210+0000 7f75e57e2640 1 -- 172.21.15.82:0/1641271547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0103b50 msgr2=0x7f75e010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.210+0000 7f75e57e2640 1 --2- 172.21.15.82:0/1641271547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0103b50 0x7f75e010a3e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009ca0 tx=0x7f75d802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.210+0000 7f75e57e2640 1 -- 172.21.15.82:0/1641271547 shutdown_connections 2024-04-21T08:46:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.210+0000 7f75e57e2640 1 --2- 172.21.15.82:0/1641271547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0103b50 0x7f75e010a3e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.210+0000 7f75e57e2640 1 --2- 172.21.15.82:0/1641271547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75e0103190 0x7f75e0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.210+0000 7f75e57e2640 1 --2- 172.21.15.82:0/1641271547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75e0101fa0 0x7f75e01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.210+0000 7f75e57e2640 1 -- 172.21.15.82:0/1641271547 >> 172.21.15.82:0/1641271547 conn(0x7f75e00fd530 msgr2=0x7f75e00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.210+0000 7f75e57e2640 1 -- 172.21.15.82:0/1641271547 shutdown_connections 2024-04-21T08:46:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.210+0000 7f75e57e2640 1 -- 172.21.15.82:0/1641271547 wait complete. 2024-04-21T08:46:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.211+0000 7f75e57e2640 1 Processor -- start 2024-04-21T08:46:48.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.211+0000 7f75e57e2640 1 -- start start 2024-04-21T08:46:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.211+0000 7f75e57e2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75e0101fa0 0x7f75e01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.211+0000 7f75e57e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75e0103190 0x7f75e0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.211+0000 7f75e57e2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0103b50 0x7f75e019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.211+0000 7f75e57e2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f75e010dae0 con 0x7f75e0103b50 2024-04-21T08:46:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.211+0000 7f75e57e2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75e010d960 con 0x7f75e0101fa0 2024-04-21T08:46:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.211+0000 7f75e57e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75e010dc60 con 0x7f75e0103190 2024-04-21T08:46:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.211+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0103b50 0x7f75e019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:48.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.211+0000 7f75df7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0103b50 0x7f75e019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42122/0 (socket says 172.21.15.82:42122) 2024-04-21T08:46:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.211+0000 7f75df7fe640 1 -- 172.21.15.82:0/3798505758 learned_addr learned my addr 172.21.15.82:0/3798505758 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75de7fc640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75e0103190 0x7f75e0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75df7fe640 1 -- 172.21.15.82:0/3798505758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75e0103190 msgr2=0x7f75e0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75df7fe640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75e0103190 0x7f75e0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75df7fe640 1 -- 172.21.15.82:0/3798505758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75e0101fa0 msgr2=0x7f75e01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75df7fe640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75e0101fa0 0x7f75e01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75df7fe640 1 -- 172.21.15.82:0/3798505758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e019fde0 con 0x7f75e0103b50 2024-04-21T08:46:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75de7fc640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75e0103190 0x7f75e0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:48.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75df7fe640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0103b50 0x7f75e019f770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f75d802c700 tx=0x7f75d80093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75c7fff640 1 -- 172.21.15.82:0/3798505758 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8041020 con 0x7f75e0103b50 2024-04-21T08:46:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75c7fff640 1 -- 172.21.15.82:0/3798505758 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75d8034d60 con 0x7f75e0103b50 2024-04-21T08:46:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75e57e2640 1 -- 172.21.15.82:0/3798505758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e01a0070 con 0x7f75e0103b50 2024-04-21T08:46:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75c7fff640 1 -- 172.21.15.82:0/3798505758 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8039040 con 0x7f75e0103b50 2024-04-21T08:46:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.212+0000 7f75e57e2640 1 -- 172.21.15.82:0/3798505758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e01a0560 con 0x7f75e0103b50 2024-04-21T08:46:48.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.213+0000 7f75e57e2640 1 -- 172.21.15.82:0/3798505758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e0109330 con 0x7f75e0103b50 2024-04-21T08:46:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.214+0000 7f75c7fff640 1 -- 172.21.15.82:0/3798505758 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f75d8044020 con 0x7f75e0103b50 2024-04-21T08:46:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.214+0000 7f75c7fff640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f75c80a3bb0 0x7f75c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.215+0000 7f75deffd640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f75c80a3bb0 0x7f75c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.215+0000 7f75c7fff640 1 -- 172.21.15.82:0/3798505758 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f75d80f2820 con 0x7f75e0103b50 2024-04-21T08:46:48.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.215+0000 7f75deffd640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f75c80a3bb0 0x7f75c80a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f75e0102ff0 tx=0x7f75d0009630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:48.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.219+0000 7f75c7fff640 1 -- 172.21.15.82:0/3798505758 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75d804b020 con 0x7f75e0103b50 2024-04-21T08:46:48.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.397+0000 7f75e57e2640 1 -- 172.21.15.82:0/3798505758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75e0062b50 con 0x7f75e0103b50 2024-04-21T08:46:48.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.397+0000 7f75c7fff640 1 -- 172.21.15.82:0/3798505758 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f75d80bee30 con 0x7f75e0103b50 2024-04-21T08:46:48.397 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:46:48.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.399+0000 7f75e57e2640 1 -- 172.21.15.82:0/3798505758 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f75c80a3bb0 msgr2=0x7f75c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.399+0000 7f75e57e2640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f75c80a3bb0 0x7f75c80a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f75e0102ff0 tx=0x7f75d0009630 comp rx=0 tx=0).stop 2024-04-21T08:46:48.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.399+0000 7f75e57e2640 1 -- 172.21.15.82:0/3798505758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0103b50 msgr2=0x7f75e019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.399+0000 7f75e57e2640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0103b50 0x7f75e019f770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f75d802c700 tx=0x7f75d80093f0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.400+0000 7f75e57e2640 1 -- 172.21.15.82:0/3798505758 shutdown_connections 2024-04-21T08:46:48.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.400+0000 7f75e57e2640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f75c80a3bb0 0x7f75c80a6070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.400+0000 7f75e57e2640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75e0103b50 0x7f75e019f770 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.400+0000 7f75e57e2640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75e0103190 0x7f75e0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.400+0000 7f75e57e2640 1 --2- 172.21.15.82:0/3798505758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75e0101fa0 0x7f75e01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.400+0000 7f75e57e2640 1 -- 172.21.15.82:0/3798505758 >> 172.21.15.82:0/3798505758 conn(0x7f75e00fd530 msgr2=0x7f75e0106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:48.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.400+0000 7f75e57e2640 1 -- 172.21.15.82:0/3798505758 shutdown_connections 2024-04-21T08:46:48.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.400+0000 7f75e57e2640 1 -- 172.21.15.82:0/3798505758 wait complete. 2024-04-21T08:46:48.411 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:46:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3806918512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff4102a00 msgr2=0x7f3ff4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 --2- 172.21.15.82:0/3806918512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff4102a00 0x7f3ff4109320 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009900 tx=0x7f3fdc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3806918512 shutdown_connections 2024-04-21T08:46:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 --2- 172.21.15.82:0/3806918512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff4102a00 0x7f3ff4109320 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 --2- 172.21.15.82:0/3806918512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff4102040 0x7f3ff41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 --2- 172.21.15.82:0/3806918512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4100e60 0x7f3ff4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3806918512 >> 172.21.15.82:0/3806918512 conn(0x7f3ff40fc420 msgr2=0x7f3ff40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3806918512 shutdown_connections 2024-04-21T08:46:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3806918512 wait complete. 2024-04-21T08:46:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 Processor -- start 2024-04-21T08:46:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 -- start start 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff4100e60 0x7f3ff406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff4102040 0x7f3ff419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4102a00 0x7f3ff419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff410caf0 con 0x7f3ff4100e60 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff410c970 con 0x7f3ff4102040 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ffac3a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff410cc70 con 0x7f3ff4102a00 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.514+0000 7f3ff89d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff4100e60 0x7f3ff406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff89d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff4100e60 0x7f3ff406b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42134/0 (socket says 172.21.15.82:42134) 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff89d6640 1 -- 172.21.15.82:0/3212456524 learned_addr learned my addr 172.21.15.82:0/3212456524 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff91d7640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4102a00 0x7f3ff419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff89d6640 1 -- 172.21.15.82:0/3212456524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4102a00 msgr2=0x7f3ff419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff3fff640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff4102040 0x7f3ff419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff89d6640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4102a00 0x7f3ff419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff89d6640 1 -- 172.21.15.82:0/3212456524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff4102040 msgr2=0x7f3ff419d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff89d6640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff4102040 0x7f3ff419d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff89d6640 1 -- 172.21.15.82:0/3212456524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff419ff70 con 0x7f3ff4100e60 2024-04-21T08:46:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff91d7640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4102a00 0x7f3ff419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff89d6640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff4100e60 0x7f3ff406b0c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00b9c0 tx=0x7f3fec00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff1ffb640 1 -- 172.21.15.82:0/3212456524 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec019070 con 0x7f3ff4100e60 2024-04-21T08:46:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff1ffb640 1 -- 172.21.15.82:0/3212456524 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fec00cba0 con 0x7f3ff4100e60 2024-04-21T08:46:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3212456524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff41a0200 con 0x7f3ff4100e60 2024-04-21T08:46:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ff1ffb640 1 -- 172.21.15.82:0/3212456524 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0125a0 con 0x7f3ff4100e60 2024-04-21T08:46:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.515+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3212456524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff41a0780 con 0x7f3ff4100e60 2024-04-21T08:46:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.517+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3212456524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff41082c0 con 0x7f3ff4100e60 2024-04-21T08:46:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.517+0000 7f3ff1ffb640 1 -- 172.21.15.82:0/3212456524 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3fec01c070 con 0x7f3ff4100e60 2024-04-21T08:46:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.518+0000 7f3ff1ffb640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3fe00a3bb0 0x7f3fe00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.518+0000 7f3ff1ffb640 1 -- 172.21.15.82:0/3212456524 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f3fec0cfe40 con 0x7f3ff4100e60 2024-04-21T08:46:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.518+0000 7f3ff3fff640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3fe00a3bb0 0x7f3fe00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.518+0000 7f3ff3fff640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3fe00a3bb0 0x7f3fe00a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3ff406ae30 tx=0x7f3fe4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:48.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.523+0000 7f3ff1ffb640 1 -- 172.21.15.82:0/3212456524 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fec09c450 con 0x7f3ff4100e60 2024-04-21T08:46:48.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.700+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3212456524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3ff4062b50 con 0x7f3ff4100e60 2024-04-21T08:46:48.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.701+0000 7f3ff1ffb640 1 -- 172.21.15.82:0/3212456524 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f3fec09bd50 con 0x7f3ff4100e60 2024-04-21T08:46:48.701 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:46:48.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.703+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3212456524 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3fe00a3bb0 msgr2=0x7f3fe00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.703+0000 7f3ffac3a640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3fe00a3bb0 0x7f3fe00a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3ff406ae30 tx=0x7f3fe4008040 comp rx=0 tx=0).stop 2024-04-21T08:46:48.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.703+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3212456524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff4100e60 msgr2=0x7f3ff406b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.703+0000 7f3ffac3a640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff4100e60 0x7f3ff406b0c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00b9c0 tx=0x7f3fec00be80 comp rx=0 tx=0).stop 2024-04-21T08:46:48.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.703+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3212456524 shutdown_connections 2024-04-21T08:46:48.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.703+0000 7f3ffac3a640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3fe00a3bb0 0x7f3fe00a6070 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.703+0000 7f3ffac3a640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ff4102a00 0x7f3ff419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.703+0000 7f3ffac3a640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff4102040 0x7f3ff419d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.703+0000 7f3ffac3a640 1 --2- 172.21.15.82:0/3212456524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff4100e60 0x7f3ff406b0c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.703+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3212456524 >> 172.21.15.82:0/3212456524 conn(0x7f3ff40fc420 msgr2=0x7f3ff4105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:48.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.703+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3212456524 shutdown_connections 2024-04-21T08:46:48.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.703+0000 7f3ffac3a640 1 -- 172.21.15.82:0/3212456524 wait complete. 2024-04-21T08:46:48.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:48 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1662304038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:48.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:48 smithi082 ceph-mon[27967]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:46:48.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:48 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3798505758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:46:48.714 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:46:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.818+0000 7fe55f975640 1 -- 172.21.15.82:0/1441719996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe558103190 msgr2=0x7fe558103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.818+0000 7fe55f975640 1 --2- 172.21.15.82:0/1441719996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe558103190 0x7fe558103610 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe548009900 tx=0x7fe54802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.818+0000 7fe55f975640 1 -- 172.21.15.82:0/1441719996 shutdown_connections 2024-04-21T08:46:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.818+0000 7fe55f975640 1 --2- 172.21.15.82:0/1441719996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe558103b50 0x7fe55810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.818+0000 7fe55f975640 1 --2- 172.21.15.82:0/1441719996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe558103190 0x7fe558103610 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.818+0000 7fe55f975640 1 --2- 172.21.15.82:0/1441719996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe558101fa0 0x7fe5581023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.818+0000 7fe55f975640 1 -- 172.21.15.82:0/1441719996 >> 172.21.15.82:0/1441719996 conn(0x7fe5580fd530 msgr2=0x7fe5580ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.818+0000 7fe55f975640 1 -- 172.21.15.82:0/1441719996 shutdown_connections 2024-04-21T08:46:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.818+0000 7fe55f975640 1 -- 172.21.15.82:0/1441719996 wait complete. 2024-04-21T08:46:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.818+0000 7fe55f975640 1 Processor -- start 2024-04-21T08:46:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.818+0000 7fe55f975640 1 -- start start 2024-04-21T08:46:48.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55f975640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe558101fa0 0x7fe5581982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55f975640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe558103190 0x7fe558198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55f975640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe558103b50 0x7fe55819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55f975640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe55810db40 con 0x7fe558103b50 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55f975640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe55810d9c0 con 0x7fe558101fa0 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55f975640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe55810dcc0 con 0x7fe558103190 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55df12640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe558103b50 0x7fe55819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55df12640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe558103b50 0x7fe55819f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42146/0 (socket says 172.21.15.82:42146) 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55df12640 1 -- 172.21.15.82:0/2667058409 learned_addr learned my addr 172.21.15.82:0/2667058409 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55cf10640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe558103190 0x7fe558198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55d711640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe558101fa0 0x7fe5581982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55df12640 1 -- 172.21.15.82:0/2667058409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe558103190 msgr2=0x7fe558198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55df12640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe558103190 0x7fe558198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55df12640 1 -- 172.21.15.82:0/2667058409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe558101fa0 msgr2=0x7fe5581982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:48.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55df12640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe558101fa0 0x7fe5581982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:48.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55df12640 1 -- 172.21.15.82:0/2667058409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55819fef0 con 0x7fe558103b50 2024-04-21T08:46:48.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55df12640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe558103b50 0x7fe55819f880 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe55000cad0 tx=0x7fe55000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:48.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe54e7fc640 1 -- 172.21.15.82:0/2667058409 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55000ae20 con 0x7fe558103b50 2024-04-21T08:46:48.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe54e7fc640 1 -- 172.21.15.82:0/2667058409 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe550005020 con 0x7fe558103b50 2024-04-21T08:46:48.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe54e7fc640 1 -- 172.21.15.82:0/2667058409 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55000f6b0 con 0x7fe558103b50 2024-04-21T08:46:48.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55f975640 1 -- 172.21.15.82:0/2667058409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5581a01e0 con 0x7fe558103b50 2024-04-21T08:46:48.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.819+0000 7fe55f975640 1 -- 172.21.15.82:0/2667058409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5581a0730 con 0x7fe558103b50 2024-04-21T08:46:48.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.821+0000 7fe55f975640 1 -- 172.21.15.82:0/2667058409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5581093e0 con 0x7fe558103b50 2024-04-21T08:46:48.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.822+0000 7fe54e7fc640 1 -- 172.21.15.82:0/2667058409 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe5500051d0 con 0x7fe558103b50 2024-04-21T08:46:48.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.822+0000 7fe54e7fc640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe5440a3bb0 0x7fe5440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:48.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.822+0000 7fe55d711640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe5440a3bb0 0x7fe5440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:48.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.823+0000 7fe54e7fc640 1 -- 172.21.15.82:0/2667058409 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fe5500d0890 con 0x7fe558103b50 2024-04-21T08:46:48.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.823+0000 7fe55d711640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe5440a3bb0 0x7fe5440a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe558102ff0 tx=0x7fe554008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:48.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:48.827+0000 7fe54e7fc640 1 -- 172.21.15.82:0/2667058409 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe550014030 con 0x7fe558103b50 2024-04-21T08:46:48.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:48 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1662304038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:48.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:48 smithi106 ceph-mon[31072]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:46:48.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:48 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3798505758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:46:48.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:48 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1662304038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:48.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:48 smithi134 ceph-mon[31772]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:46:48.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:48 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3798505758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:46:49.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.004+0000 7fe55f975640 1 -- 172.21.15.82:0/2667058409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe558062b50 con 0x7fe558103b50 2024-04-21T08:46:49.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.004+0000 7fe54e7fc640 1 -- 172.21.15.82:0/2667058409 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fe55009cea0 con 0x7fe558103b50 2024-04-21T08:46:49.004 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:46:49.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.006+0000 7fe55f975640 1 -- 172.21.15.82:0/2667058409 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe5440a3bb0 msgr2=0x7fe5440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.006+0000 7fe55f975640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe5440a3bb0 0x7fe5440a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe558102ff0 tx=0x7fe554008040 comp rx=0 tx=0).stop 2024-04-21T08:46:49.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.006+0000 7fe55f975640 1 -- 172.21.15.82:0/2667058409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe558103b50 msgr2=0x7fe55819f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.006+0000 7fe55f975640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe558103b50 0x7fe55819f880 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe55000cad0 tx=0x7fe55000cf90 comp rx=0 tx=0).stop 2024-04-21T08:46:49.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.006+0000 7fe55f975640 1 -- 172.21.15.82:0/2667058409 shutdown_connections 2024-04-21T08:46:49.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.006+0000 7fe55f975640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fe5440a3bb0 0x7fe5440a6070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.006+0000 7fe55f975640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe558103b50 0x7fe55819f880 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.006+0000 7fe55f975640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe558103190 0x7fe558198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.006+0000 7fe55f975640 1 --2- 172.21.15.82:0/2667058409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe558101fa0 0x7fe5581982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.006+0000 7fe55f975640 1 -- 172.21.15.82:0/2667058409 >> 172.21.15.82:0/2667058409 conn(0x7fe5580fd530 msgr2=0x7fe5581065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:49.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.006+0000 7fe55f975640 1 -- 172.21.15.82:0/2667058409 shutdown_connections 2024-04-21T08:46:49.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.006+0000 7fe55f975640 1 -- 172.21.15.82:0/2667058409 wait complete. 2024-04-21T08:46:49.018 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:46:49.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.121+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/2916929959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbdc41029a0 msgr2=0x7fbdc41092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.121+0000 7fbdcc2c1640 1 --2- 172.21.15.82:0/2916929959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbdc41029a0 0x7fbdc41092c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009ca0 tx=0x7fbdbc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.121+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/2916929959 shutdown_connections 2024-04-21T08:46:49.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.121+0000 7fbdcc2c1640 1 --2- 172.21.15.82:0/2916929959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbdc41029a0 0x7fbdc41092c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.121+0000 7fbdcc2c1640 1 --2- 172.21.15.82:0/2916929959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc4101fe0 0x7fbdc4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.121+0000 7fbdcc2c1640 1 --2- 172.21.15.82:0/2916929959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc4100e90 0x7fbdc41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.121+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/2916929959 >> 172.21.15.82:0/2916929959 conn(0x7fbdc40fc400 msgr2=0x7fbdc40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:49.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.121+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/2916929959 shutdown_connections 2024-04-21T08:46:49.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.121+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/2916929959 wait complete. 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdcc2c1640 1 Processor -- start 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdcc2c1640 1 -- start start 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdcc2c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc4100e90 0x7fbdc419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdcc2c1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbdc4101fe0 0x7fbdc419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdcc2c1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc41029a0 0x7fbdc41a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdcc2c1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc410db30 con 0x7fbdc41029a0 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdcc2c1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc410d9b0 con 0x7fbdc4101fe0 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdcc2c1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc410dcb0 con 0x7fbdc4100e90 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdca85e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc41029a0 0x7fbdc41a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdca85e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc41029a0 0x7fbdc41a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42162/0 (socket says 172.21.15.82:42162) 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdca85e640 1 -- 172.21.15.82:0/4106129513 learned_addr learned my addr 172.21.15.82:0/4106129513 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdca05d640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc4100e90 0x7fbdc419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdc985c640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbdc4101fe0 0x7fbdc419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:49.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdca85e640 1 -- 172.21.15.82:0/4106129513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc4100e90 msgr2=0x7fbdc419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdca85e640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc4100e90 0x7fbdc419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdca85e640 1 -- 172.21.15.82:0/4106129513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbdc4101fe0 msgr2=0x7fbdc419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdca85e640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbdc4101fe0 0x7fbdc419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdca85e640 1 -- 172.21.15.82:0/4106129513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc41a1fc0 con 0x7fbdc41029a0 2024-04-21T08:46:49.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.122+0000 7fbdc985c640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbdc4101fe0 0x7fbdc419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:49.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.123+0000 7fbdca85e640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc41029a0 0x7fbdc41a1950 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc02c700 tx=0x7fbdbc0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:49.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.123+0000 7fbdbb7fe640 1 -- 172.21.15.82:0/4106129513 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc041020 con 0x7fbdc41029a0 2024-04-21T08:46:49.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.123+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/4106129513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc41a2250 con 0x7fbdc41029a0 2024-04-21T08:46:49.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.123+0000 7fbdbb7fe640 1 -- 172.21.15.82:0/4106129513 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdbc034ca0 con 0x7fbdc41029a0 2024-04-21T08:46:49.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.123+0000 7fbdbb7fe640 1 -- 172.21.15.82:0/4106129513 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc039040 con 0x7fbdc41029a0 2024-04-21T08:46:49.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.123+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/4106129513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc41a2740 con 0x7fbdc41029a0 2024-04-21T08:46:49.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.124+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/4106129513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc41a29a0 con 0x7fbdc41029a0 2024-04-21T08:46:49.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.125+0000 7fbdbb7fe640 1 -- 172.21.15.82:0/4106129513 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbdbc044020 con 0x7fbdc41029a0 2024-04-21T08:46:49.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.126+0000 7fbdbb7fe640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbdb00a3bb0 0x7fbdb00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:49.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.126+0000 7fbdca05d640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbdb00a3bb0 0x7fbdb00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:49.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.126+0000 7fbdbb7fe640 1 -- 172.21.15.82:0/4106129513 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fbdbc0f2590 con 0x7fbdc41029a0 2024-04-21T08:46:49.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.126+0000 7fbdca05d640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbdb00a3bb0 0x7fbdb00a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009bd0 tx=0x7fbdc0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:49.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.130+0000 7fbdbb7fe640 1 -- 172.21.15.82:0/4106129513 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbdbc04b020 con 0x7fbdc41029a0 2024-04-21T08:46:49.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.321+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/4106129513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbdc41a2b70 con 0x7fbdc41029a0 2024-04-21T08:46:49.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.322+0000 7fbdbb7fe640 1 -- 172.21.15.82:0/4106129513 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fbdbc0beb60 con 0x7fbdc41029a0 2024-04-21T08:46:49.322 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:46:49.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.324+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/4106129513 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbdb00a3bb0 msgr2=0x7fbdb00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.324+0000 7fbdcc2c1640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbdb00a3bb0 0x7fbdb00a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009bd0 tx=0x7fbdc0005f90 comp rx=0 tx=0).stop 2024-04-21T08:46:49.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.324+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/4106129513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc41029a0 msgr2=0x7fbdc41a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.324+0000 7fbdcc2c1640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc41029a0 0x7fbdc41a1950 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc02c700 tx=0x7fbdbc0346d0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.324+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/4106129513 shutdown_connections 2024-04-21T08:46:49.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.324+0000 7fbdcc2c1640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fbdb00a3bb0 0x7fbdb00a6070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.324+0000 7fbdcc2c1640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdc41029a0 0x7fbdc41a1950 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.324+0000 7fbdcc2c1640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbdc4101fe0 0x7fbdc419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.324+0000 7fbdcc2c1640 1 --2- 172.21.15.82:0/4106129513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdc4100e90 0x7fbdc419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.324+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/4106129513 >> 172.21.15.82:0/4106129513 conn(0x7fbdc40fc400 msgr2=0x7fbdc4109cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:49.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.324+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/4106129513 shutdown_connections 2024-04-21T08:46:49.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.324+0000 7fbdcc2c1640 1 -- 172.21.15.82:0/4106129513 wait complete. 2024-04-21T08:46:49.336 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:46:49.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.435+0000 7f0ef46f2640 1 -- 172.21.15.82:0/1562398232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eec100e70 msgr2=0x7f0eec101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.435+0000 7f0ef46f2640 1 --2- 172.21.15.82:0/1562398232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eec100e70 0x7f0eec101290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009900 tx=0x7f0edc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:49.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.435+0000 7f0ef46f2640 1 -- 172.21.15.82:0/1562398232 shutdown_connections 2024-04-21T08:46:49.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.435+0000 7f0ef46f2640 1 --2- 172.21.15.82:0/1562398232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0eec102980 0x7f0eec1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.435+0000 7f0ef46f2640 1 --2- 172.21.15.82:0/1562398232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0eec101fc0 0x7f0eec102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.435+0000 7f0ef46f2640 1 --2- 172.21.15.82:0/1562398232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eec100e70 0x7f0eec101290 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.435+0000 7f0ef46f2640 1 -- 172.21.15.82:0/1562398232 >> 172.21.15.82:0/1562398232 conn(0x7f0eec0fc3e0 msgr2=0x7f0eec0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.435+0000 7f0ef46f2640 1 -- 172.21.15.82:0/1562398232 shutdown_connections 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.435+0000 7f0ef46f2640 1 -- 172.21.15.82:0/1562398232 wait complete. 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef46f2640 1 Processor -- start 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef46f2640 1 -- start start 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef46f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0eec100e70 0x7f0eec069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef46f2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0eec101fc0 0x7f0eec19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef46f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eec102980 0x7f0eec19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef46f2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec10c9c0 con 0x7f0eec102980 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef46f2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec10c840 con 0x7f0eec101fc0 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef46f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec10cb40 con 0x7f0eec100e70 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef2c8f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eec102980 0x7f0eec19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef2c8f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eec102980 0x7f0eec19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42182/0 (socket says 172.21.15.82:42182) 2024-04-21T08:46:49.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef2c8f640 1 -- 172.21.15.82:0/2524692873 learned_addr learned my addr 172.21.15.82:0/2524692873 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef1c8d640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0eec101fc0 0x7f0eec19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef2c8f640 1 -- 172.21.15.82:0/2524692873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0eec100e70 msgr2=0x7f0eec069c70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef2c8f640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0eec100e70 0x7f0eec069c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef2c8f640 1 -- 172.21.15.82:0/2524692873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0eec101fc0 msgr2=0x7f0eec19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef2c8f640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0eec101fc0 0x7f0eec19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef2c8f640 1 -- 172.21.15.82:0/2524692873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eec19fdb0 con 0x7f0eec102980 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.436+0000 7f0ef1c8d640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0eec101fc0 0x7f0eec19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.437+0000 7f0ef2c8f640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eec102980 0x7f0eec19f4f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400eaa0 tx=0x7f0ee400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.437+0000 7f0ee37fe640 1 -- 172.21.15.82:0/2524692873 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400cdb0 con 0x7f0eec102980 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.437+0000 7f0ee37fe640 1 -- 172.21.15.82:0/2524692873 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee4005020 con 0x7f0eec102980 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.437+0000 7f0ef46f2640 1 -- 172.21.15.82:0/2524692873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eec1a00a0 con 0x7f0eec102980 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.437+0000 7f0ee37fe640 1 -- 172.21.15.82:0/2524692873 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee40116d0 con 0x7f0eec102980 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.437+0000 7f0ef46f2640 1 -- 172.21.15.82:0/2524692873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eec1a05a0 con 0x7f0eec102980 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.438+0000 7f0ef46f2640 1 -- 172.21.15.82:0/2524692873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eec108250 con 0x7f0eec102980 2024-04-21T08:46:49.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.439+0000 7f0ee37fe640 1 -- 172.21.15.82:0/2524692873 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0ee40051d0 con 0x7f0eec102980 2024-04-21T08:46:49.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.439+0000 7f0ee37fe640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0ed80a3bb0 0x7f0ed80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:49.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.440+0000 7f0ef248e640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0ed80a3bb0 0x7f0ed80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:49.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.440+0000 7f0ee37fe640 1 -- 172.21.15.82:0/2524692873 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f0ee40d0a80 con 0x7f0eec102980 2024-04-21T08:46:49.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.440+0000 7f0ef248e640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0ed80a3bb0 0x7f0ed80a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0edc036040 tx=0x7f0edc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:49.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.444+0000 7f0ee37fe640 1 -- 172.21.15.82:0/2524692873 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ee4016030 con 0x7f0eec102980 2024-04-21T08:46:49.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:49 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3212456524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:46:49.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:49 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2667058409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:46:49.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:49 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4106129513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:46:49.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.667+0000 7f0ef46f2640 1 -- 172.21.15.82:0/2524692873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0eec062b50 con 0x7f0eec102980 2024-04-21T08:46:49.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.667+0000 7f0ee37fe640 1 -- 172.21.15.82:0/2524692873 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f0ee409d090 con 0x7f0eec102980 2024-04-21T08:46:49.668 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:49.668 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:45:02.005557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24481,24509,24476,24409,24407]},"id":1}]} 2024-04-21T08:46:49.669 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-04-21T08:46:49.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.669+0000 7f0ef46f2640 1 -- 172.21.15.82:0/2524692873 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0ed80a3bb0 msgr2=0x7f0ed80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.669+0000 7f0ef46f2640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0ed80a3bb0 0x7f0ed80a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0edc036040 tx=0x7f0edc005fd0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.669+0000 7f0ef46f2640 1 -- 172.21.15.82:0/2524692873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eec102980 msgr2=0x7f0eec19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.669+0000 7f0ef46f2640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eec102980 0x7f0eec19f4f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400eaa0 tx=0x7f0ee400ef60 comp rx=0 tx=0).stop 2024-04-21T08:46:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.669+0000 7f0ef46f2640 1 -- 172.21.15.82:0/2524692873 shutdown_connections 2024-04-21T08:46:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.669+0000 7f0ef46f2640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0ed80a3bb0 0x7f0ed80a6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.670+0000 7f0ef46f2640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0eec102980 0x7f0eec19f4f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.670+0000 7f0ef46f2640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0eec101fc0 0x7f0eec19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.670+0000 7f0ef46f2640 1 --2- 172.21.15.82:0/2524692873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0eec100e70 0x7f0eec069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.670+0000 7f0ef46f2640 1 -- 172.21.15.82:0/2524692873 >> 172.21.15.82:0/2524692873 conn(0x7f0eec0fc3e0 msgr2=0x7f0eec105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:49.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.670+0000 7f0ef46f2640 1 -- 172.21.15.82:0/2524692873 shutdown_connections 2024-04-21T08:46:49.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.670+0000 7f0ef46f2640 1 -- 172.21.15.82:0/2524692873 wait complete. 2024-04-21T08:46:49.682 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:46:49.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.782+0000 7f2a0425c640 1 -- 172.21.15.82:0/1709070625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29fc103190 msgr2=0x7f29fc103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.782+0000 7f2a0425c640 1 --2- 172.21.15.82:0/1709070625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29fc103190 0x7f29fc103610 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f29f8009900 tx=0x7f29f802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:49.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 -- 172.21.15.82:0/1709070625 shutdown_connections 2024-04-21T08:46:49.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 --2- 172.21.15.82:0/1709070625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29fc103b50 0x7f29fc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 --2- 172.21.15.82:0/1709070625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29fc103190 0x7f29fc103610 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 --2- 172.21.15.82:0/1709070625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29fc101fa0 0x7f29fc1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 -- 172.21.15.82:0/1709070625 >> 172.21.15.82:0/1709070625 conn(0x7f29fc0fd530 msgr2=0x7f29fc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 -- 172.21.15.82:0/1709070625 shutdown_connections 2024-04-21T08:46:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 -- 172.21.15.82:0/1709070625 wait complete. 2024-04-21T08:46:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 Processor -- start 2024-04-21T08:46:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 -- start start 2024-04-21T08:46:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29fc101fa0 0x7f29fc198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29fc103190 0x7f29fc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:49.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29fc103b50 0x7f29fc19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:49.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc10dae0 con 0x7f29fc101fa0 2024-04-21T08:46:49.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc10d960 con 0x7f29fc103190 2024-04-21T08:46:49.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a0425c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc10dc60 con 0x7f29fc103b50 2024-04-21T08:46:49.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a01ff8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29fc101fa0 0x7f29fc198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:49.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a01ff8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29fc101fa0 0x7f29fc198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42186/0 (socket says 172.21.15.82:42186) 2024-04-21T08:46:49.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.783+0000 7f2a01ff8640 1 -- 172.21.15.82:0/3356731311 learned_addr learned my addr 172.21.15.82:0/3356731311 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:49.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f2a027f9640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29fc103b50 0x7f29fc19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:49.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f2a017f7640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29fc103190 0x7f29fc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:49.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f2a01ff8640 1 -- 172.21.15.82:0/3356731311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29fc103b50 msgr2=0x7f29fc19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f2a01ff8640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29fc103b50 0x7f29fc19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f2a01ff8640 1 -- 172.21.15.82:0/3356731311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29fc103190 msgr2=0x7f29fc1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f2a01ff8640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29fc103190 0x7f29fc1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f2a01ff8640 1 -- 172.21.15.82:0/3356731311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc19fdb0 con 0x7f29fc101fa0 2024-04-21T08:46:49.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f2a01ff8640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29fc101fa0 0x7f29fc198190 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00c970 tx=0x7f29ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:49.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f29f2ffd640 1 -- 172.21.15.82:0/3356731311 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec004220 con 0x7f29fc101fa0 2024-04-21T08:46:49.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f2a0425c640 1 -- 172.21.15.82:0/3356731311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29fc1a00a0 con 0x7f29fc101fa0 2024-04-21T08:46:49.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f29f2ffd640 1 -- 172.21.15.82:0/3356731311 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ec0043c0 con 0x7f29fc101fa0 2024-04-21T08:46:49.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f29f2ffd640 1 -- 172.21.15.82:0/3356731311 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec004a00 con 0x7f29fc101fa0 2024-04-21T08:46:49.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.784+0000 7f2a0425c640 1 -- 172.21.15.82:0/3356731311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29fc1a05a0 con 0x7f29fc101fa0 2024-04-21T08:46:49.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.785+0000 7f2a0425c640 1 -- 172.21.15.82:0/3356731311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29fc1093e0 con 0x7f29fc101fa0 2024-04-21T08:46:49.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.787+0000 7f29f2ffd640 1 -- 172.21.15.82:0/3356731311 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f29ec0079b0 con 0x7f29fc101fa0 2024-04-21T08:46:49.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.787+0000 7f29f2ffd640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f29e80a3bb0 0x7f29e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:49.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.788+0000 7f2a017f7640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f29e80a3bb0 0x7f29e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:49.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.788+0000 7f29f2ffd640 1 -- 172.21.15.82:0/3356731311 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f29ec0d0a20 con 0x7f29fc101fa0 2024-04-21T08:46:49.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.788+0000 7f2a017f7640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f29e80a3bb0 0x7f29e80a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f29fc1996a0 tx=0x7f29f802c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:49.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.790+0000 7f29f2ffd640 1 -- 172.21.15.82:0/3356731311 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29ec014030 con 0x7f29fc101fa0 2024-04-21T08:46:49.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:49 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3212456524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:46:49.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:49 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2667058409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:46:49.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:49 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/4106129513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:46:49.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.973+0000 7f2a0425c640 1 -- 172.21.15.82:0/3356731311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29fc062b50 con 0x7f29fc101fa0 2024-04-21T08:46:49.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.975+0000 7f29f2ffd640 1 -- 172.21.15.82:0/3356731311 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24870 (secure 0 0 0) 0x7f29ec09d030 con 0x7f29fc101fa0 2024-04-21T08:46:49.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:49 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3212456524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:46:49.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:49 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2667058409' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:46:49.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:49 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/4106129513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:46:49.977 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:49.977 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":93,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:46:27.475773+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:46:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.977+0000 7f2a0425c640 1 -- 172.21.15.82:0/3356731311 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f29e80a3bb0 msgr2=0x7f29e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.977+0000 7f2a0425c640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f29e80a3bb0 0x7f29e80a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f29fc1996a0 tx=0x7f29f802c800 comp rx=0 tx=0).stop 2024-04-21T08:46:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.977+0000 7f2a0425c640 1 -- 172.21.15.82:0/3356731311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29fc101fa0 msgr2=0x7f29fc198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.977+0000 7f2a0425c640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29fc101fa0 0x7f29fc198190 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00c970 tx=0x7f29ec00ce30 comp rx=0 tx=0).stop 2024-04-21T08:46:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.977+0000 7f2a0425c640 1 -- 172.21.15.82:0/3356731311 shutdown_connections 2024-04-21T08:46:49.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.977+0000 7f2a0425c640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f29e80a3bb0 0x7f29e80a6070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.977+0000 7f2a0425c640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29fc103b50 0x7f29fc19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.977+0000 7f2a0425c640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29fc103190 0x7f29fc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.977+0000 7f2a0425c640 1 --2- 172.21.15.82:0/3356731311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29fc101fa0 0x7f29fc198190 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:49.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.977+0000 7f2a0425c640 1 -- 172.21.15.82:0/3356731311 >> 172.21.15.82:0/3356731311 conn(0x7f29fc0fd530 msgr2=0x7f29fc1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:49.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.977+0000 7f2a0425c640 1 -- 172.21.15.82:0/3356731311 shutdown_connections 2024-04-21T08:46:49.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:49.977+0000 7f2a0425c640 1 -- 172.21.15.82:0/3356731311 wait complete. 2024-04-21T08:46:49.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:46:49.990 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:46:50.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 -- 172.21.15.82:0/3448125865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc0100e90 msgr2=0x7f2dc01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 --2- 172.21.15.82:0/3448125865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc0100e90 0x7f2dc01012b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009900 tx=0x7f2db402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:50.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 -- 172.21.15.82:0/3448125865 shutdown_connections 2024-04-21T08:46:50.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 --2- 172.21.15.82:0/3448125865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dc01029a0 0x7f2dc01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 --2- 172.21.15.82:0/3448125865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dc0101fe0 0x7f2dc0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 --2- 172.21.15.82:0/3448125865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc0100e90 0x7f2dc01012b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 -- 172.21.15.82:0/3448125865 >> 172.21.15.82:0/3448125865 conn(0x7f2dc00fc400 msgr2=0x7f2dc00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 -- 172.21.15.82:0/3448125865 shutdown_connections 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 -- 172.21.15.82:0/3448125865 wait complete. 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 Processor -- start 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 -- start start 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dc0100e90 0x7f2dc0198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc0101fe0 0x7f2dc0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dc01029a0 0x7f2dc019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc010db30 con 0x7f2dc0101fe0 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc010d9b0 con 0x7f2dc01029a0 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dc6c26640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc010dcb0 con 0x7f2dc0100e90 2024-04-21T08:46:50.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dbffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc0101fe0 0x7f2dc0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dbffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc0101fe0 0x7f2dc0198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42210/0 (socket says 172.21.15.82:42210) 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.092+0000 7f2dbffff640 1 -- 172.21.15.82:0/268626108 learned_addr learned my addr 172.21.15.82:0/268626108 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dc49c2640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dc0100e90 0x7f2dc0198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dc51c3640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dc01029a0 0x7f2dc019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dbffff640 1 -- 172.21.15.82:0/268626108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dc0100e90 msgr2=0x7f2dc0198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dbffff640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dc0100e90 0x7f2dc0198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dbffff640 1 -- 172.21.15.82:0/268626108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dc01029a0 msgr2=0x7f2dc019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dbffff640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dc01029a0 0x7f2dc019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dbffff640 1 -- 172.21.15.82:0/268626108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc019fe40 con 0x7f2dc0101fe0 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dbffff640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc0101fe0 0x7f2dc0198760 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00e9c0 tx=0x7f2dac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dbdffb640 1 -- 172.21.15.82:0/268626108 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00cd80 con 0x7f2dc0101fe0 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dbdffb640 1 -- 172.21.15.82:0/268626108 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dac0035f0 con 0x7f2dc0101fe0 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dc6c26640 1 -- 172.21.15.82:0/268626108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc01a0130 con 0x7f2dc0101fe0 2024-04-21T08:46:50.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dbdffb640 1 -- 172.21.15.82:0/268626108 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac010690 con 0x7f2dc0101fe0 2024-04-21T08:46:50.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.093+0000 7f2dc6c26640 1 -- 172.21.15.82:0/268626108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc01a0680 con 0x7f2dc0101fe0 2024-04-21T08:46:50.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.094+0000 7f2dc6c26640 1 -- 172.21.15.82:0/268626108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc0108220 con 0x7f2dc0101fe0 2024-04-21T08:46:50.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.096+0000 7f2dbdffb640 1 -- 172.21.15.82:0/268626108 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2dac0037a0 con 0x7f2dc0101fe0 2024-04-21T08:46:50.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.097+0000 7f2dbdffb640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2db00a3bb0 0x7f2db00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:50.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.097+0000 7f2dc49c2640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2db00a3bb0 0x7f2db00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:50.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.097+0000 7f2dbdffb640 1 -- 172.21.15.82:0/268626108 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f2dac0d0be0 con 0x7f2dc0101fe0 2024-04-21T08:46:50.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.097+0000 7f2dc49c2640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2db00a3bb0 0x7f2db00a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009230 tx=0x7f2db4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:50.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.101+0000 7f2dbdffb640 1 -- 172.21.15.82:0/268626108 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2dac09d1f0 con 0x7f2dc0101fe0 2024-04-21T08:46:50.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.277+0000 7f2dc6c26640 1 -- 172.21.15.82:0/268626108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2dc0062b50 con 0x7f2dc0101fe0 2024-04-21T08:46:50.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.277+0000 7f2dbdffb640 1 -- 172.21.15.82:0/268626108 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f2dac09caf0 con 0x7f2dc0101fe0 2024-04-21T08:46:50.278 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:46:50.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.279+0000 7f2dc6c26640 1 -- 172.21.15.82:0/268626108 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2db00a3bb0 msgr2=0x7f2db00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.279+0000 7f2dc6c26640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2db00a3bb0 0x7f2db00a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009230 tx=0x7f2db4005fd0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.279+0000 7f2dc6c26640 1 -- 172.21.15.82:0/268626108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc0101fe0 msgr2=0x7f2dc0198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.279+0000 7f2dc6c26640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc0101fe0 0x7f2dc0198760 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00e9c0 tx=0x7f2dac00ee80 comp rx=0 tx=0).stop 2024-04-21T08:46:50.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.279+0000 7f2dc6c26640 1 -- 172.21.15.82:0/268626108 shutdown_connections 2024-04-21T08:46:50.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.279+0000 7f2dc6c26640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2db00a3bb0 0x7f2db00a6070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.279+0000 7f2dc6c26640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2dc01029a0 0x7f2dc019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.279+0000 7f2dc6c26640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dc0101fe0 0x7f2dc0198760 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.279+0000 7f2dc6c26640 1 --2- 172.21.15.82:0/268626108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dc0100e90 0x7f2dc0198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.279+0000 7f2dc6c26640 1 -- 172.21.15.82:0/268626108 >> 172.21.15.82:0/268626108 conn(0x7f2dc00fc400 msgr2=0x7f2dc01053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:50.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.279+0000 7f2dc6c26640 1 -- 172.21.15.82:0/268626108 shutdown_connections 2024-04-21T08:46:50.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.279+0000 7f2dc6c26640 1 -- 172.21.15.82:0/268626108 wait complete. 2024-04-21T08:46:50.291 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:46:50.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.392+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2780986332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86f0075f90 msgr2=0x7f86f00743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.392+0000 7f86f4a4e640 1 --2- 172.21.15.82:0/2780986332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86f0075f90 0x7f86f00743e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009900 tx=0x7f86d802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:50.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.392+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2780986332 shutdown_connections 2024-04-21T08:46:50.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.392+0000 7f86f4a4e640 1 --2- 172.21.15.82:0/2780986332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86f0109e60 0x7f86f010c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.392+0000 7f86f4a4e640 1 --2- 172.21.15.82:0/2780986332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f00749b0 0x7f86f0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.392+0000 7f86f4a4e640 1 --2- 172.21.15.82:0/2780986332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86f0075f90 0x7f86f00743e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.392+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2780986332 >> 172.21.15.82:0/2780986332 conn(0x7f86f00fd530 msgr2=0x7f86f00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:50.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.392+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2780986332 shutdown_connections 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.392+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2780986332 wait complete. 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86f4a4e640 1 Processor -- start 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86f4a4e640 1 -- start start 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86f4a4e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86f00749b0 0x7f86f019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86f4a4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f0075f90 0x7f86f019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86f4a4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86f0109e60 0x7f86f01a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86f4a4e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f86f010fd90 con 0x7f86f0109e60 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86f4a4e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f86f010fc10 con 0x7f86f00749b0 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86f4a4e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86f010ff10 con 0x7f86f0075f90 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86eed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86f0109e60 0x7f86f01a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86eed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86f0109e60 0x7f86f01a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42226/0 (socket says 172.21.15.82:42226) 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86eed9d640 1 -- 172.21.15.82:0/2408631390 learned_addr learned my addr 172.21.15.82:0/2408631390 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86ee59c640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86f00749b0 0x7f86f019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:50.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86eed9d640 1 -- 172.21.15.82:0/2408631390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f0075f90 msgr2=0x7f86f019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86eed9d640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f0075f90 0x7f86f019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86eed9d640 1 -- 172.21.15.82:0/2408631390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86f00749b0 msgr2=0x7f86f019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86eed9d640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86f00749b0 0x7f86f019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86eed9d640 1 -- 172.21.15.82:0/2408631390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f01a2190 con 0x7f86f0109e60 2024-04-21T08:46:50.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86eed9d640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86f0109e60 0x7f86f01a1b20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f86e800b970 tx=0x7f86e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:50.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.393+0000 7f86e77fe640 1 -- 172.21.15.82:0/2408631390 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8019070 con 0x7f86f0109e60 2024-04-21T08:46:50.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.394+0000 7f86e77fe640 1 -- 172.21.15.82:0/2408631390 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e8014440 con 0x7f86f0109e60 2024-04-21T08:46:50.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.394+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2408631390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f01a2480 con 0x7f86f0109e60 2024-04-21T08:46:50.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.394+0000 7f86e77fe640 1 -- 172.21.15.82:0/2408631390 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e80075d0 con 0x7f86f0109e60 2024-04-21T08:46:50.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.394+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2408631390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a2a00 con 0x7f86f0109e60 2024-04-21T08:46:50.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.395+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2408631390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f0108b90 con 0x7f86f0109e60 2024-04-21T08:46:50.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.396+0000 7f86e77fe640 1 -- 172.21.15.82:0/2408631390 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f86e801c070 con 0x7f86f0109e60 2024-04-21T08:46:50.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.396+0000 7f86e77fe640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f86dc0a3bb0 0x7f86dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:50.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.396+0000 7f86ee59c640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f86dc0a3bb0 0x7f86dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:50.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.396+0000 7f86e77fe640 1 -- 172.21.15.82:0/2408631390 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f86e80d0340 con 0x7f86f0109e60 2024-04-21T08:46:50.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.397+0000 7f86ee59c640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f86dc0a3bb0 0x7f86dc0a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f86d80096c0 tx=0x7f86d8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:50.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.400+0000 7f86e77fe640 1 -- 172.21.15.82:0/2408631390 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f86e809c950 con 0x7f86f0109e60 2024-04-21T08:46:50.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.580+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2408631390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f86f002d7f0 con 0x7f86f0109e60 2024-04-21T08:46:50.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.581+0000 7f86e77fe640 1 -- 172.21.15.82:0/2408631390 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f86e809c250 con 0x7f86f0109e60 2024-04-21T08:46:50.582 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:46:50.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.583+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2408631390 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f86dc0a3bb0 msgr2=0x7f86dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.583+0000 7f86f4a4e640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f86dc0a3bb0 0x7f86dc0a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f86d80096c0 tx=0x7f86d8000bc0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.583+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2408631390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86f0109e60 msgr2=0x7f86f01a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.583+0000 7f86f4a4e640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86f0109e60 0x7f86f01a1b20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f86e800b970 tx=0x7f86e800be30 comp rx=0 tx=0).stop 2024-04-21T08:46:50.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.583+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2408631390 shutdown_connections 2024-04-21T08:46:50.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.583+0000 7f86f4a4e640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f86dc0a3bb0 0x7f86dc0a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.583+0000 7f86f4a4e640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86f0109e60 0x7f86f01a1b20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.583+0000 7f86f4a4e640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86f0075f90 0x7f86f019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.583+0000 7f86f4a4e640 1 --2- 172.21.15.82:0/2408631390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86f00749b0 0x7f86f019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.583+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2408631390 >> 172.21.15.82:0/2408631390 conn(0x7f86f00fd530 msgr2=0x7f86f0105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:50.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.583+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2408631390 shutdown_connections 2024-04-21T08:46:50.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.583+0000 7f86f4a4e640 1 -- 172.21.15.82:0/2408631390 wait complete. 2024-04-21T08:46:50.593 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:50 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2524692873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:50.594 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:50 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3356731311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:50.594 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:50 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/268626108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:46:50.594 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:50 smithi082 ceph-mon[27967]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:46:50.595 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:46:50.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.695+0000 7f77e9443640 1 -- 172.21.15.82:0/3837453370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77e4102a00 msgr2=0x7f77e4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.695+0000 7f77e9443640 1 --2- 172.21.15.82:0/3837453370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77e4102a00 0x7f77e4109320 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009ca0 tx=0x7f77dc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.695+0000 7f77e9443640 1 -- 172.21.15.82:0/3837453370 shutdown_connections 2024-04-21T08:46:50.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.695+0000 7f77e9443640 1 --2- 172.21.15.82:0/3837453370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77e4102a00 0x7f77e4109320 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.695+0000 7f77e9443640 1 --2- 172.21.15.82:0/3837453370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e4102040 0x7f77e41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.695+0000 7f77e9443640 1 --2- 172.21.15.82:0/3837453370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e4100e60 0x7f77e4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.695+0000 7f77e9443640 1 -- 172.21.15.82:0/3837453370 >> 172.21.15.82:0/3837453370 conn(0x7f77e40fc420 msgr2=0x7f77e40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.695+0000 7f77e9443640 1 -- 172.21.15.82:0/3837453370 shutdown_connections 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.695+0000 7f77e9443640 1 -- 172.21.15.82:0/3837453370 wait complete. 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e9443640 1 Processor -- start 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e9443640 1 -- start start 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e9443640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e4100e60 0x7f77e41981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e9443640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77e4102040 0x7f77e4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e9443640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e4102a00 0x7f77e419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e9443640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f77e410db30 con 0x7f77e4102040 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e9443640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f77e410d9b0 con 0x7f77e4100e60 2024-04-21T08:46:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e9443640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77e410dcb0 con 0x7f77e4102a00 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77e4102040 0x7f77e4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77e4102040 0x7f77e4198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42244/0 (socket says 172.21.15.82:42244) 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e27fc640 1 -- 172.21.15.82:0/3202355176 learned_addr learned my addr 172.21.15.82:0/3202355176 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e37fe640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e4102a00 0x7f77e419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e2ffd640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e4100e60 0x7f77e41981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e27fc640 1 -- 172.21.15.82:0/3202355176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e4102a00 msgr2=0x7f77e419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e27fc640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e4102a00 0x7f77e419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e27fc640 1 -- 172.21.15.82:0/3202355176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e4100e60 msgr2=0x7f77e41981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e27fc640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e4100e60 0x7f77e41981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e27fc640 1 -- 172.21.15.82:0/3202355176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e419fe00 con 0x7f77e4102040 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.696+0000 7f77e2ffd640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e4100e60 0x7f77e41981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.697+0000 7f77e27fc640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77e4102040 0x7f77e4198720 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f77d800d770 tx=0x7f77d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.697+0000 7f77cbfff640 1 -- 172.21.15.82:0/3202355176 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8017070 con 0x7f77e4102040 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.697+0000 7f77cbfff640 1 -- 172.21.15.82:0/3202355176 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77d80054d0 con 0x7f77e4102040 2024-04-21T08:46:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.697+0000 7f77e9443640 1 -- 172.21.15.82:0/3202355176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e41a00f0 con 0x7f77e4102040 2024-04-21T08:46:50.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.697+0000 7f77cbfff640 1 -- 172.21.15.82:0/3202355176 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8004c30 con 0x7f77e4102040 2024-04-21T08:46:50.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.697+0000 7f77e9443640 1 -- 172.21.15.82:0/3202355176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e41a05f0 con 0x7f77e4102040 2024-04-21T08:46:50.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.698+0000 7f77e9443640 1 -- 172.21.15.82:0/3202355176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e4108250 con 0x7f77e4102040 2024-04-21T08:46:50.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.699+0000 7f77cbfff640 1 -- 172.21.15.82:0/3202355176 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f77d8004e10 con 0x7f77e4102040 2024-04-21T08:46:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.699+0000 7f77cbfff640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f77cc0a3bb0 0x7f77cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.699+0000 7f77e2ffd640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f77cc0a3bb0 0x7f77cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.699+0000 7f77cbfff640 1 -- 172.21.15.82:0/3202355176 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f77d80d0290 con 0x7f77e4102040 2024-04-21T08:46:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.700+0000 7f77e2ffd640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f77cc0a3bb0 0x7f77cc0a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77e4101bb0 tx=0x7f77d400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:50.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.704+0000 7f77cbfff640 1 -- 172.21.15.82:0/3202355176 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77d809c8a0 con 0x7f77e4102040 2024-04-21T08:46:50.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:50 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2524692873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:50.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:50 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3356731311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:50.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:50 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/268626108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:46:50.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:50 smithi106 ceph-mon[31072]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:46:50.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.882+0000 7f77e9443640 1 -- 172.21.15.82:0/3202355176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f77e402d5c0 con 0x7f77e4102040 2024-04-21T08:46:50.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.882+0000 7f77cbfff640 1 -- 172.21.15.82:0/3202355176 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f77d809c1a0 con 0x7f77e4102040 2024-04-21T08:46:50.883 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:46:50.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.884+0000 7f77e9443640 1 -- 172.21.15.82:0/3202355176 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f77cc0a3bb0 msgr2=0x7f77cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.884+0000 7f77e9443640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f77cc0a3bb0 0x7f77cc0a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77e4101bb0 tx=0x7f77d400a560 comp rx=0 tx=0).stop 2024-04-21T08:46:50.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.884+0000 7f77e9443640 1 -- 172.21.15.82:0/3202355176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77e4102040 msgr2=0x7f77e4198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:50.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.884+0000 7f77e9443640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77e4102040 0x7f77e4198720 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f77d800d770 tx=0x7f77d800dc30 comp rx=0 tx=0).stop 2024-04-21T08:46:50.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.884+0000 7f77e9443640 1 -- 172.21.15.82:0/3202355176 shutdown_connections 2024-04-21T08:46:50.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.884+0000 7f77e9443640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f77cc0a3bb0 0x7f77cc0a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.884+0000 7f77e9443640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77e4102a00 0x7f77e419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.884+0000 7f77e9443640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77e4102040 0x7f77e4198720 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.884+0000 7f77e9443640 1 --2- 172.21.15.82:0/3202355176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e4100e60 0x7f77e41981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:50.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.884+0000 7f77e9443640 1 -- 172.21.15.82:0/3202355176 >> 172.21.15.82:0/3202355176 conn(0x7f77e40fc420 msgr2=0x7f77e4105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:50.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.884+0000 7f77e9443640 1 -- 172.21.15.82:0/3202355176 shutdown_connections 2024-04-21T08:46:50.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.884+0000 7f77e9443640 1 -- 172.21.15.82:0/3202355176 wait complete. 2024-04-21T08:46:50.896 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:46:50.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:50 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2524692873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:50.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:50 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3356731311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:50.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:50 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/268626108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:46:50.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:50 smithi134 ceph-mon[31772]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-21T08:46:50.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.996+0000 7fceba706640 1 -- 172.21.15.82:0/120190273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4103b50 msgr2=0x7fceb410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.996+0000 7fceba706640 1 --2- 172.21.15.82:0/120190273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4103b50 0x7fceb410a3e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fceac009ca0 tx=0x7fceac02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 -- 172.21.15.82:0/120190273 shutdown_connections 2024-04-21T08:46:51.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 --2- 172.21.15.82:0/120190273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4103b50 0x7fceb410a3e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 --2- 172.21.15.82:0/120190273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceb4103190 0x7fceb4103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 --2- 172.21.15.82:0/120190273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fceb4101fa0 0x7fceb41023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 -- 172.21.15.82:0/120190273 >> 172.21.15.82:0/120190273 conn(0x7fceb40fd530 msgr2=0x7fceb40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:51.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 -- 172.21.15.82:0/120190273 shutdown_connections 2024-04-21T08:46:51.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 -- 172.21.15.82:0/120190273 wait complete. 2024-04-21T08:46:51.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 Processor -- start 2024-04-21T08:46:51.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 -- start start 2024-04-21T08:46:51.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fceb4101fa0 0x7fceb4198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceb4103190 0x7fceb41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4103b50 0x7fceb419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fceb410dae0 con 0x7fceb4103b50 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fceb410d960 con 0x7fceb4101fa0 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.997+0000 7fceba706640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fceb410dc60 con 0x7fceb4103190 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb8ca3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4103b50 0x7fceb419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb8ca3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4103b50 0x7fceb419f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42266/0 (socket says 172.21.15.82:42266) 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb8ca3640 1 -- 172.21.15.82:0/2479111185 learned_addr learned my addr 172.21.15.82:0/2479111185 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb37fe640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceb4103190 0x7fceb41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb3fff640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fceb4101fa0 0x7fceb4198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb8ca3640 1 -- 172.21.15.82:0/2479111185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceb4103190 msgr2=0x7fceb41986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb8ca3640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceb4103190 0x7fceb41986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb8ca3640 1 -- 172.21.15.82:0/2479111185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fceb4101fa0 msgr2=0x7fceb4198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb8ca3640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fceb4101fa0 0x7fceb4198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb8ca3640 1 -- 172.21.15.82:0/2479111185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb419fd80 con 0x7fceb4103b50 2024-04-21T08:46:51.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb3fff640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fceb4101fa0 0x7fceb4198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:51.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb8ca3640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4103b50 0x7fceb419f710 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fceac032520 tx=0x7fceac0347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:51.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb17fa640 1 -- 172.21.15.82:0/2479111185 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac039040 con 0x7fceb4103b50 2024-04-21T08:46:51.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb17fa640 1 -- 172.21.15.82:0/2479111185 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceac044070 con 0x7fceb4103b50 2024-04-21T08:46:51.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceba706640 1 -- 172.21.15.82:0/2479111185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb41a0010 con 0x7fceb4103b50 2024-04-21T08:46:51.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceb17fa640 1 -- 172.21.15.82:0/2479111185 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac0042a0 con 0x7fceb4103b50 2024-04-21T08:46:51.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.998+0000 7fceba706640 1 -- 172.21.15.82:0/2479111185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb41a0500 con 0x7fceb4103b50 2024-04-21T08:46:51.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:50.999+0000 7fceba706640 1 -- 172.21.15.82:0/2479111185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb4109340 con 0x7fceb4103b50 2024-04-21T08:46:51.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.000+0000 7fceb17fa640 1 -- 172.21.15.82:0/2479111185 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fceac003970 con 0x7fceb4103b50 2024-04-21T08:46:51.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.001+0000 7fceb17fa640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fce9c0a3bb0 0x7fce9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.001+0000 7fceb3fff640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fce9c0a3bb0 0x7fce9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.001+0000 7fceb17fa640 1 -- 172.21.15.82:0/2479111185 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fceac0f2a90 con 0x7fceb4103b50 2024-04-21T08:46:51.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.001+0000 7fceb3fff640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fce9c0a3bb0 0x7fce9c0a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fceb4102ff0 tx=0x7fcea4005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:51.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.005+0000 7fceb17fa640 1 -- 172.21.15.82:0/2479111185 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fceac035d30 con 0x7fceb4103b50 2024-04-21T08:46:51.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.182+0000 7fceba706640 1 -- 172.21.15.82:0/2479111185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fceb4062b50 con 0x7fceb4103b50 2024-04-21T08:46:51.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.182+0000 7fceb17fa640 1 -- 172.21.15.82:0/2479111185 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fceac0bf0a0 con 0x7fceb4103b50 2024-04-21T08:46:51.183 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:46:51.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.184+0000 7fceba706640 1 -- 172.21.15.82:0/2479111185 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fce9c0a3bb0 msgr2=0x7fce9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.184+0000 7fceba706640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fce9c0a3bb0 0x7fce9c0a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fceb4102ff0 tx=0x7fcea4005e70 comp rx=0 tx=0).stop 2024-04-21T08:46:51.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.184+0000 7fceba706640 1 -- 172.21.15.82:0/2479111185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4103b50 msgr2=0x7fceb419f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.184+0000 7fceba706640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4103b50 0x7fceb419f710 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fceac032520 tx=0x7fceac0347b0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.184+0000 7fceba706640 1 -- 172.21.15.82:0/2479111185 shutdown_connections 2024-04-21T08:46:51.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.184+0000 7fceba706640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fce9c0a3bb0 0x7fce9c0a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.184+0000 7fceba706640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb4103b50 0x7fceb419f710 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.184+0000 7fceba706640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fceb4103190 0x7fceb41986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.184+0000 7fceba706640 1 --2- 172.21.15.82:0/2479111185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fceb4101fa0 0x7fceb4198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.184+0000 7fceba706640 1 -- 172.21.15.82:0/2479111185 >> 172.21.15.82:0/2479111185 conn(0x7fceb40fd530 msgr2=0x7fceb4106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:51.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.184+0000 7fceba706640 1 -- 172.21.15.82:0/2479111185 shutdown_connections 2024-04-21T08:46:51.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.184+0000 7fceba706640 1 -- 172.21.15.82:0/2479111185 wait complete. 2024-04-21T08:46:51.196 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:46:51.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.294+0000 7f570764e640 1 -- 172.21.15.82:0/3016880728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5700101fa0 msgr2=0x7f57001023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.294+0000 7f570764e640 1 --2- 172.21.15.82:0/3016880728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5700101fa0 0x7f57001023c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009900 tx=0x7f56f002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.294+0000 7f570764e640 1 -- 172.21.15.82:0/3016880728 shutdown_connections 2024-04-21T08:46:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.294+0000 7f570764e640 1 --2- 172.21.15.82:0/3016880728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5700103b50 0x7f570010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.294+0000 7f570764e640 1 --2- 172.21.15.82:0/3016880728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5700103190 0x7f5700103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.294+0000 7f570764e640 1 --2- 172.21.15.82:0/3016880728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5700101fa0 0x7f57001023c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.294+0000 7f570764e640 1 -- 172.21.15.82:0/3016880728 >> 172.21.15.82:0/3016880728 conn(0x7f57000fd530 msgr2=0x7f57000ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.294+0000 7f570764e640 1 -- 172.21.15.82:0/3016880728 shutdown_connections 2024-04-21T08:46:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.294+0000 7f570764e640 1 -- 172.21.15.82:0/3016880728 wait complete. 2024-04-21T08:46:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.294+0000 7f570764e640 1 Processor -- start 2024-04-21T08:46:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.294+0000 7f570764e640 1 -- start start 2024-04-21T08:46:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f570764e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5700101fa0 0x7f5700198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f570764e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5700103190 0x7f57001986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f570764e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5700103b50 0x7f570019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f570764e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f570010db30 con 0x7f5700103190 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f570764e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f570010d9b0 con 0x7f5700103b50 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f570764e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f570010dcb0 con 0x7f5700101fa0 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f5704be9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5700103190 0x7f57001986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f5704be9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5700103190 0x7f57001986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42290/0 (socket says 172.21.15.82:42290) 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f5704be9640 1 -- 172.21.15.82:0/1035098105 learned_addr learned my addr 172.21.15.82:0/1035098105 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f5705beb640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5700103b50 0x7f570019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f5704be9640 1 -- 172.21.15.82:0/1035098105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5700101fa0 msgr2=0x7f5700198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f57053ea640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5700101fa0 0x7f5700198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f5704be9640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5700101fa0 0x7f5700198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f5704be9640 1 -- 172.21.15.82:0/1035098105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5700103b50 msgr2=0x7f570019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f5704be9640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5700103b50 0x7f570019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f5704be9640 1 -- 172.21.15.82:0/1035098105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570019fda0 con 0x7f5700103190 2024-04-21T08:46:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f5704be9640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5700103190 0x7f57001986c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00d950 tx=0x7f56fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f56f67fc640 1 -- 172.21.15.82:0/1035098105 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc017070 con 0x7f5700103190 2024-04-21T08:46:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f56f67fc640 1 -- 172.21.15.82:0/1035098105 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56fc009c30 con 0x7f5700103190 2024-04-21T08:46:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f570764e640 1 -- 172.21.15.82:0/1035098105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57001a0090 con 0x7f5700103190 2024-04-21T08:46:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f56f67fc640 1 -- 172.21.15.82:0/1035098105 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc00f620 con 0x7f5700103190 2024-04-21T08:46:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.295+0000 7f570764e640 1 -- 172.21.15.82:0/1035098105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57001a05e0 con 0x7f5700103190 2024-04-21T08:46:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.296+0000 7f570764e640 1 -- 172.21.15.82:0/1035098105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57001092c0 con 0x7f5700103190 2024-04-21T08:46:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.297+0000 7f56f67fc640 1 -- 172.21.15.82:0/1035098105 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f56fc010460 con 0x7f5700103190 2024-04-21T08:46:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.298+0000 7f56f67fc640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f56ec0a3bb0 0x7f56ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.298+0000 7f57053ea640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f56ec0a3bb0 0x7f56ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.298+0000 7f56f67fc640 1 -- 172.21.15.82:0/1035098105 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f56fc0d04b0 con 0x7f5700103190 2024-04-21T08:46:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.298+0000 7f57053ea640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f56ec0a3bb0 0x7f56ec0a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f56f00096c0 tx=0x7f56f0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.303+0000 7f56f67fc640 1 -- 172.21.15.82:0/1035098105 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56fc09cac0 con 0x7f5700103190 2024-04-21T08:46:51.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.519+0000 7f570764e640 1 -- 172.21.15.82:0/1035098105 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5700062b50 con 0x7f5700103190 2024-04-21T08:46:51.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.519+0000 7f56f67fc640 1 -- 172.21.15.82:0/1035098105 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f56fc009de0 con 0x7f5700103190 2024-04-21T08:46:51.520 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:51.520 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:45:02.005557+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24481,24509,24476,24409,24407]},"id":1}]} 2024-04-21T08:46:51.520 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 27 2024-04-21T08:46:51.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.522+0000 7f570764e640 1 -- 172.21.15.82:0/1035098105 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f56ec0a3bb0 msgr2=0x7f56ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.522+0000 7f570764e640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f56ec0a3bb0 0x7f56ec0a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f56f00096c0 tx=0x7f56f0005fd0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.522+0000 7f570764e640 1 -- 172.21.15.82:0/1035098105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5700103190 msgr2=0x7f57001986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.522+0000 7f570764e640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5700103190 0x7f57001986c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00d950 tx=0x7f56fc00de10 comp rx=0 tx=0).stop 2024-04-21T08:46:51.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.522+0000 7f570764e640 1 -- 172.21.15.82:0/1035098105 shutdown_connections 2024-04-21T08:46:51.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.522+0000 7f570764e640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f56ec0a3bb0 0x7f56ec0a6070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.522+0000 7f570764e640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5700103b50 0x7f570019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.522+0000 7f570764e640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5700103190 0x7f57001986c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.522+0000 7f570764e640 1 --2- 172.21.15.82:0/1035098105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5700101fa0 0x7f5700198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.522+0000 7f570764e640 1 -- 172.21.15.82:0/1035098105 >> 172.21.15.82:0/1035098105 conn(0x7f57000fd530 msgr2=0x7f5700106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:51.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.522+0000 7f570764e640 1 -- 172.21.15.82:0/1035098105 shutdown_connections 2024-04-21T08:46:51.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.522+0000 7f570764e640 1 -- 172.21.15.82:0/1035098105 wait complete. 2024-04-21T08:46:51.534 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:46:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.631+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4024202163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb64100e80 msgr2=0x7ffb641012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.631+0000 7ffb6bd52640 1 --2- 172.21.15.82:0/4024202163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb64100e80 0x7ffb641012a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009900 tx=0x7ffb6002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.631+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4024202163 shutdown_connections 2024-04-21T08:46:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.631+0000 7ffb6bd52640 1 --2- 172.21.15.82:0/4024202163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb64102990 0x7ffb641092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.631+0000 7ffb6bd52640 1 --2- 172.21.15.82:0/4024202163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb64101fd0 0x7ffb64102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.631+0000 7ffb6bd52640 1 --2- 172.21.15.82:0/4024202163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb64100e80 0x7ffb641012a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.631+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4024202163 >> 172.21.15.82:0/4024202163 conn(0x7ffb640fc410 msgr2=0x7ffb640fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:51.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.631+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4024202163 shutdown_connections 2024-04-21T08:46:51.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.631+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4024202163 wait complete. 2024-04-21T08:46:51.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.631+0000 7ffb6bd52640 1 Processor -- start 2024-04-21T08:46:51.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.631+0000 7ffb6bd52640 1 -- start start 2024-04-21T08:46:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.631+0000 7ffb6bd52640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb64100e80 0x7ffb6410b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb6bd52640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb64101fd0 0x7ffb64109b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb6bd52640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb64102990 0x7ffb6410a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb6bd52640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6410dab0 con 0x7ffb64101fd0 2024-04-21T08:46:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb6bd52640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6410d930 con 0x7ffb64102990 2024-04-21T08:46:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb6bd52640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6410dc30 con 0x7ffb64100e80 2024-04-21T08:46:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb692ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb64101fd0 0x7ffb64109b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb692ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb64101fd0 0x7ffb64109b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42302/0 (socket says 172.21.15.82:42302) 2024-04-21T08:46:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb692ed640 1 -- 172.21.15.82:0/4046505063 learned_addr learned my addr 172.21.15.82:0/4046505063 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb69aee640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb64100e80 0x7ffb6410b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb6a2ef640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb64102990 0x7ffb6410a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb692ed640 1 -- 172.21.15.82:0/4046505063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb64100e80 msgr2=0x7ffb6410b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb692ed640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb64100e80 0x7ffb6410b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb692ed640 1 -- 172.21.15.82:0/4046505063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb64102990 msgr2=0x7ffb6410a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb692ed640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb64102990 0x7ffb6410a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb692ed640 1 -- 172.21.15.82:0/4046505063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6410a900 con 0x7ffb64101fd0 2024-04-21T08:46:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb692ed640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb64101fd0 0x7ffb64109b30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400c970 tx=0x7ffb5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:51.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb5affd640 1 -- 172.21.15.82:0/4046505063 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54004220 con 0x7ffb64101fd0 2024-04-21T08:46:51.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb5affd640 1 -- 172.21.15.82:0/4046505063 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb540043c0 con 0x7ffb64101fd0 2024-04-21T08:46:51.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4046505063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb641a22a0 con 0x7ffb64101fd0 2024-04-21T08:46:51.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb5affd640 1 -- 172.21.15.82:0/4046505063 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400f450 con 0x7ffb64101fd0 2024-04-21T08:46:51.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.632+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4046505063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb641a27f0 con 0x7ffb64101fd0 2024-04-21T08:46:51.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.633+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4046505063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb64108230 con 0x7ffb64101fd0 2024-04-21T08:46:51.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.634+0000 7ffb5affd640 1 -- 172.21.15.82:0/4046505063 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffb54003610 con 0x7ffb64101fd0 2024-04-21T08:46:51.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.635+0000 7ffb5affd640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ffb500a3bb0 0x7ffb500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.635+0000 7ffb69aee640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ffb500a3bb0 0x7ffb500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.635+0000 7ffb5affd640 1 -- 172.21.15.82:0/4046505063 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7ffb540d0930 con 0x7ffb64101fd0 2024-04-21T08:46:51.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.635+0000 7ffb69aee640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ffb500a3bb0 0x7ffb500a6070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009230 tx=0x7ffb60005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:51.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.639+0000 7ffb5affd640 1 -- 172.21.15.82:0/4046505063 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb54014030 con 0x7ffb64101fd0 2024-04-21T08:46:51.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.815+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4046505063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb6402d810 con 0x7ffb64101fd0 2024-04-21T08:46:51.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.817+0000 7ffb5affd640 1 -- 172.21.15.82:0/4046505063 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24870 (secure 0 0 0) 0x7ffb5409c8f0 con 0x7ffb64101fd0 2024-04-21T08:46:51.818 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:46:51.818 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":93,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:46:27.475773+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:46:51.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.819+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4046505063 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ffb500a3bb0 msgr2=0x7ffb500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.819+0000 7ffb6bd52640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ffb500a3bb0 0x7ffb500a6070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009230 tx=0x7ffb60005fd0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.819+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4046505063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb64101fd0 msgr2=0x7ffb64109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.819+0000 7ffb6bd52640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb64101fd0 0x7ffb64109b30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400c970 tx=0x7ffb5400ce30 comp rx=0 tx=0).stop 2024-04-21T08:46:51.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.819+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4046505063 shutdown_connections 2024-04-21T08:46:51.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.819+0000 7ffb6bd52640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7ffb500a3bb0 0x7ffb500a6070 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.819+0000 7ffb6bd52640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb64102990 0x7ffb6410a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.819+0000 7ffb6bd52640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb64101fd0 0x7ffb64109b30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.819+0000 7ffb6bd52640 1 --2- 172.21.15.82:0/4046505063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb64100e80 0x7ffb6410b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.819+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4046505063 >> 172.21.15.82:0/4046505063 conn(0x7ffb640fc410 msgr2=0x7ffb64105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:51.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.819+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4046505063 shutdown_connections 2024-04-21T08:46:51.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.819+0000 7ffb6bd52640 1 -- 172.21.15.82:0/4046505063 wait complete. 2024-04-21T08:46:51.830 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:51 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2408631390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:46:51.830 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:51 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3202355176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:46:51.830 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:51 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2479111185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:46:51.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T08:44:33.819176+0000', 'modified': '2024-04-21T08:45:02.005557+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24509, 'mds_1': 24481, 'mds_2': 24476, 'mds_3': 24409, 'mds_4': 24407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6839/3697606919', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 3697606919}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 3697606919}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6841/1327149046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6840', 'nonce': 1327149046}, {'type': 'v1', 'addr': '172.21.15.82:6841', 'nonce': 1327149046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.106:6833/2583766346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6832', 'nonce': 2583766346}, {'type': 'v1', 'addr': '172.21.15.106:6833', 'nonce': 2583766346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.134:6833/2942732132', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2942732132}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2942732132}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6835/3901395826', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6834', 'nonce': 3901395826}, {'type': 'v1', 'addr': '172.21.15.106:6835', 'nonce': 3901395826}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/2962604291', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 2962604291}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 2962604291}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/1870695477', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 1870695477}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 1870695477}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.106:6837/3731760515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 3731760515}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 3731760515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.134:6839/394037083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 394037083}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 394037083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/3796875133', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 3796875133}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 3796875133}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24481, 24509, 24476, 24409, 24407]} 2024-04-21T08:46:51.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-21T08:46:51.832 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-21T08:46:51.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:51 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2408631390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:46:51.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:51 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3202355176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:46:51.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:51 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2479111185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:46:51.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 -- 172.21.15.82:0/581976060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a81029a0 msgr2=0x7fd1a81092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 --2- 172.21.15.82:0/581976060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a81029a0 0x7fd1a81092c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009ca0 tx=0x7fd1a002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 -- 172.21.15.82:0/581976060 shutdown_connections 2024-04-21T08:46:51.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 --2- 172.21.15.82:0/581976060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a81029a0 0x7fd1a81092c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 --2- 172.21.15.82:0/581976060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1a8101fe0 0x7fd1a8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 --2- 172.21.15.82:0/581976060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1a8100e90 0x7fd1a81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 -- 172.21.15.82:0/581976060 >> 172.21.15.82:0/581976060 conn(0x7fd1a80fc400 msgr2=0x7fd1a80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:51.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 -- 172.21.15.82:0/581976060 shutdown_connections 2024-04-21T08:46:51.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 -- 172.21.15.82:0/581976060 wait complete. 2024-04-21T08:46:51.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 Processor -- start 2024-04-21T08:46:51.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 -- start start 2024-04-21T08:46:51.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1a8100e90 0x7fd1a8198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a8101fe0 0x7fd1a81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1a81029a0 0x7fd1a819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a810db30 con 0x7fd1a8101fe0 2024-04-21T08:46:51.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a810d9b0 con 0x7fd1a81029a0 2024-04-21T08:46:51.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1aecc8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a810dcb0 con 0x7fd1a8100e90 2024-04-21T08:46:51.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1a7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a8101fe0 0x7fd1a81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1a7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a8101fe0 0x7fd1a81986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42326/0 (socket says 172.21.15.82:42326) 2024-04-21T08:46:51.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.931+0000 7fd1a7fff640 1 -- 172.21.15.82:0/3324453682 learned_addr learned my addr 172.21.15.82:0/3324453682 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:51.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1aca64640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1a8100e90 0x7fd1a8198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1a7fff640 1 -- 172.21.15.82:0/3324453682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1a8100e90 msgr2=0x7fd1a8198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1ad265640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1a81029a0 0x7fd1a819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1a7fff640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1a8100e90 0x7fd1a8198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1a7fff640 1 -- 172.21.15.82:0/3324453682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1a81029a0 msgr2=0x7fd1a819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1a7fff640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1a81029a0 0x7fd1a819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1a7fff640 1 -- 172.21.15.82:0/3324453682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a819fdb0 con 0x7fd1a8101fe0 2024-04-21T08:46:51.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1ad265640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1a81029a0 0x7fd1a819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:51.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1a7fff640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a8101fe0 0x7fd1a81986d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c970 tx=0x7fd19800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:51.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1a5ffb640 1 -- 172.21.15.82:0/3324453682 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198004220 con 0x7fd1a8101fe0 2024-04-21T08:46:51.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1a5ffb640 1 -- 172.21.15.82:0/3324453682 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1980043c0 con 0x7fd1a8101fe0 2024-04-21T08:46:51.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1a5ffb640 1 -- 172.21.15.82:0/3324453682 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198004a00 con 0x7fd1a8101fe0 2024-04-21T08:46:51.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1aecc8640 1 -- 172.21.15.82:0/3324453682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a81a00a0 con 0x7fd1a8101fe0 2024-04-21T08:46:51.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.932+0000 7fd1aecc8640 1 -- 172.21.15.82:0/3324453682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a81a05a0 con 0x7fd1a8101fe0 2024-04-21T08:46:51.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.934+0000 7fd1a5ffb640 1 -- 172.21.15.82:0/3324453682 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd1980079b0 con 0x7fd1a8101fe0 2024-04-21T08:46:51.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.934+0000 7fd1b1998740 1 Processor -- start 2024-04-21T08:46:51.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1b1998740 1 -- start start 2024-04-21T08:46:51.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1a5ffb640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd1940a3bb0 0x7fd1940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1b1998740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4697e40 0x5626d4698260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1b1998740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d4698830 0x5626d4698cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1b1998740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d46992d0 0x5626d469feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1b1998740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5626d45fcb10 con 0x5626d46992d0 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1b1998740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5626d459a240 con 0x5626d4697e40 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1b1998740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5626d45f8ad0 con 0x5626d4698830 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1ad265640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd1940a3bb0 0x7fd1940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1ad265640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d46992d0 0x5626d469feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1a5ffb640 1 -- 172.21.15.82:0/3324453682 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fd1980d0550 con 0x7fd1a8101fe0 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1ad265640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d46992d0 0x5626d469feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42328/0 (socket says 172.21.15.82:42328) 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1ad265640 1 -- 172.21.15.82:0/4013234254 learned_addr learned my addr 172.21.15.82:0/4013234254 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1aca64640 1 --2- 172.21.15.82:0/4013234254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4697e40 0x5626d4698260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1ad265640 1 -- 172.21.15.82:0/4013234254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d4698830 msgr2=0x5626d4698cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1ad265640 1 --2- 172.21.15.82:0/4013234254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d4698830 0x5626d4698cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1ad265640 1 -- 172.21.15.82:0/4013234254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4697e40 msgr2=0x5626d4698260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:51.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1ad265640 1 --2- 172.21.15.82:0/4013234254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4697e40 0x5626d4698260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1ad265640 1 -- 172.21.15.82:0/4013234254 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626d430f2d0 con 0x5626d46992d0 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1aca64640 1 --2- 172.21.15.82:0/4013234254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4697e40 0x5626d4698260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1ad265640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd1940a3bb0 0x7fd1940a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009ca0 tx=0x7fd1a00099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.935+0000 7fd1ad265640 1 --2- 172.21.15.82:0/4013234254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d46992d0 0x5626d469feb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0036000 tx=0x7fd1a0045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829001eeee9febe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.936+0000 7fd19effd640 1 -- 172.21.15.82:0/4013234254 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0050070 con 0x5626d46992d0 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.936+0000 7fd19effd640 1 -- 172.21.15.82:0/4013234254 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a0004040 con 0x5626d46992d0 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.936+0000 7fd19effd640 1 -- 172.21.15.82:0/4013234254 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0039040 con 0x5626d46992d0 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.936+0000 7fd1b1998740 1 -- 172.21.15.82:0/4013234254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d46992d0 msgr2=0x5626d469feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.936+0000 7fd1b1998740 1 --2- 172.21.15.82:0/4013234254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d46992d0 0x5626d469feb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0036000 tx=0x7fd1a0045000 comp rx=0 tx=0).stop 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.936+0000 7fd1b1998740 1 -- 172.21.15.82:0/4013234254 shutdown_connections 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.936+0000 7fd1b1998740 1 --2- 172.21.15.82:0/4013234254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d46992d0 0x5626d469feb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.936+0000 7fd1b1998740 1 --2- 172.21.15.82:0/4013234254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d4698830 0x5626d4698cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.936+0000 7fd1b1998740 1 --2- 172.21.15.82:0/4013234254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4697e40 0x5626d4698260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.936+0000 7fd1b1998740 1 -- 172.21.15.82:0/4013234254 >> 172.21.15.82:0/4013234254 conn(0x5626d4693f40 msgr2=0x5626d4692490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.936+0000 7fd1b1998740 1 -- 172.21.15.82:0/4013234254 shutdown_connections 2024-04-21T08:46:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.936+0000 7fd1b1998740 1 -- 172.21.15.82:0/4013234254 wait complete. 2024-04-21T08:46:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1b1998740 1 -- start start 2024-04-21T08:46:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1b1998740 1 Processor -- start 2024-04-21T08:46:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1b1998740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4697e40 0x5626d46a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1b1998740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 0x5626d46a2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1b1998740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d46992d0 0x5626d46a9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1b1998740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5626d46a0450 con 0x5626d4698830 2024-04-21T08:46:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1b1998740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5626d4585190 con 0x5626d4697e40 2024-04-21T08:46:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1b1998740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5626d46a05d0 con 0x5626d46992d0 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1b1998740 10 client.0 _ll_register_callbacks cb 0x5626d45808b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1a7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 0x5626d46a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1a7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 0x5626d46a2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42342/0 (socket says 172.21.15.82:42342) 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1a7fff640 1 -- 172.21.15.82:0/707230655 learned_addr learned my addr 172.21.15.82:0/707230655 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1a7fff640 1 -- 172.21.15.82:0/707230655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d46992d0 msgr2=0x5626d46a9470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1a7fff640 1 --2- 172.21.15.82:0/707230655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d46992d0 0x5626d46a9470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1a7fff640 1 -- 172.21.15.82:0/707230655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4697e40 msgr2=0x5626d46a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1a7fff640 1 --2- 172.21.15.82:0/707230655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4697e40 0x5626d46a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.937+0000 7fd1a7fff640 1 -- 172.21.15.82:0/707230655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626d43d96b0 con 0x5626d4698830 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd1a7fff640 1 --2- 172.21.15.82:0/707230655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 0x5626d46a2430 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd19800afd0 tx=0x7fd19809bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1acc1b003107470 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd187fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd187fff640 1 -- 172.21.15.82:0/707230655 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19801d040 con 0x5626d4698830 2024-04-21T08:46:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd187fff640 1 -- 172.21.15.82:0/707230655 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fd198007ca0 con 0x5626d4698830 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd1b1998740 1 -- 172.21.15.82:0/707230655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a0004040 con 0x5626d4698830 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd187fff640 1 -- 172.21.15.82:0/707230655 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd19809c460 con 0x5626d4698830 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd1b1998740 10 client.15444 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd1b1998740 1 -- 172.21.15.82:0/707230655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626d46ac850 con 0x5626d4698830 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd187fff640 1 -- 172.21.15.82:0/707230655 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a0004040 con 0x5626d4698830 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd187fff640 1 -- 172.21.15.82:0/707230655 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd198080370 con 0x5626d4698830 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd1b1998740 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd1b1998740 10 client.15444 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd1b1998740 20 client.15444 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd1b1998740 1 --2- 172.21.15.82:0/707230655 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5626d46aee00 0x5626d46b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd1b1998740 4 client.15444 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.938+0000 7fd1b1998740 1 -- 172.21.15.82:0/707230655 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626d460a530 con 0x5626d46aee00 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.939+0000 7fd1aca64640 1 --2- 172.21.15.82:0/707230655 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5626d46aee00 0x5626d46b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.939+0000 7fd1aca64640 1 --2- 172.21.15.82:0/707230655 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5626d46aee00 0x5626d46b0270 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2aee885678eefb0 server_cookie=ce845094e77eee65 in_seq=0 out_seq=0 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.939+0000 7fd187fff640 10 client.15444 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd187fff640 1 -- 172.21.15.82:0/707230655 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626d460a530 con 0x5626d46aee00 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd187fff640 10 client.15444 handle_command_reply: tid=0 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd1b1998740 1 client.15444 shutdown 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd1b1998740 1 -- 172.21.15.82:0/707230655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 msgr2=0x5626d46a2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd1b1998740 1 --2- 172.21.15.82:0/707230655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 0x5626d46a2430 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd19800afd0 tx=0x7fd19809bdc0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd1b1998740 1 -- 172.21.15.82:0/707230655 shutdown_connections 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd1b1998740 1 --2- 172.21.15.82:0/707230655 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5626d46aee00 0x5626d46b0270 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd1b1998740 1 --2- 172.21.15.82:0/707230655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d46992d0 0x5626d46a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd1b1998740 1 --2- 172.21.15.82:0/707230655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 0x5626d46a2430 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd187fff640 0 client.15444 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd1b1998740 1 --2- 172.21.15.82:0/707230655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4697e40 0x5626d46a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd1b1998740 1 -- 172.21.15.82:0/707230655 >> 172.21.15.82:0/707230655 conn(0x5626d4693f40 msgr2=0x5626d4691a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:51.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd1b1998740 1 -- 172.21.15.82:0/707230655 shutdown_connections 2024-04-21T08:46:51.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd1b1998740 1 -- 172.21.15.82:0/707230655 wait complete. 2024-04-21T08:46:51.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.940+0000 7fd1b1998740 20 client.15444 trim_cache size 0 max 16384 2024-04-21T08:46:51.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1b1998740 1 Processor -- start 2024-04-21T08:46:51.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1b1998740 1 -- start start 2024-04-21T08:46:51.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1b1998740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4697e40 0x5626d469bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1b1998740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4698830 0x5626d46a9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1b1998740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d46992d0 0x5626d469c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1b1998740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5626d46ae5b0 con 0x5626d4697e40 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1b1998740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5626d46aec80 con 0x5626d4698830 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1b1998740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5626d46af350 con 0x5626d46992d0 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1aca64640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4697e40 0x5626d469bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1aca64640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4697e40 0x5626d469bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42358/0 (socket says 172.21.15.82:42358) 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1aca64640 1 -- 172.21.15.82:0/2441756539 learned_addr learned my addr 172.21.15.82:0/2441756539 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1aca64640 1 -- 172.21.15.82:0/2441756539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d46992d0 msgr2=0x5626d469c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1aca64640 1 --2- 172.21.15.82:0/2441756539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d46992d0 0x5626d469c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1aca64640 1 -- 172.21.15.82:0/2441756539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4698830 msgr2=0x5626d46a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1aca64640 1 --2- 172.21.15.82:0/2441756539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4698830 0x5626d46a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1aca64640 1 -- 172.21.15.82:0/2441756539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626d4693230 con 0x5626d4697e40 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd1aca64640 1 --2- 172.21.15.82:0/2441756539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4697e40 0x5626d469bd00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x5626d46a14e0 tx=0x7fd190005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120c285ff02d1fea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd19e7fc640 1 -- 172.21.15.82:0/2441756539 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190004230 con 0x5626d4697e40 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd19e7fc640 1 -- 172.21.15.82:0/2441756539 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1900043d0 con 0x5626d4697e40 2024-04-21T08:46:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.959+0000 7fd19e7fc640 1 -- 172.21.15.82:0/2441756539 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190003d20 con 0x5626d4697e40 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 -- 172.21.15.82:0/2441756539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4697e40 msgr2=0x5626d469bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 --2- 172.21.15.82:0/2441756539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4697e40 0x5626d469bd00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x5626d46a14e0 tx=0x7fd190005ae0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 -- 172.21.15.82:0/2441756539 shutdown_connections 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 --2- 172.21.15.82:0/2441756539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d46992d0 0x5626d469c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 --2- 172.21.15.82:0/2441756539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d4698830 0x5626d46a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 --2- 172.21.15.82:0/2441756539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4697e40 0x5626d469bd00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 -- 172.21.15.82:0/2441756539 >> 172.21.15.82:0/2441756539 conn(0x5626d4693f40 msgr2=0x5626d46a1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 -- 172.21.15.82:0/2441756539 shutdown_connections 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 -- 172.21.15.82:0/2441756539 wait complete. 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 -- start start 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 Processor -- start 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d4697e40 0x5626d46a3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 0x5626d46a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d46992d0 0x5626d46a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5626d46b00f0 con 0x5626d4698830 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5626d46afa20 con 0x5626d46992d0 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.960+0000 7fd1b1998740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5626d46b07c0 con 0x5626d4697e40 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1b1998740 10 client.0 _ll_register_callbacks cb 0x7fd1980058a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1a7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 0x5626d46a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1a7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 0x5626d46a3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42364/0 (socket says 172.21.15.82:42364) 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1a7fff640 1 -- 172.21.15.82:0/552737189 learned_addr learned my addr 172.21.15.82:0/552737189 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1aca64640 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d4697e40 0x5626d46a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1ad265640 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d46992d0 0x5626d46a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1a7fff640 1 -- 172.21.15.82:0/552737189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d4697e40 msgr2=0x5626d46a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1a7fff640 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d4697e40 0x5626d46a3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1a7fff640 1 -- 172.21.15.82:0/552737189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d46992d0 msgr2=0x5626d46a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1a7fff640 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d46992d0 0x5626d46a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1a7fff640 1 -- 172.21.15.82:0/552737189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626d46a8b00 con 0x5626d4698830 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1a7fff640 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 0x5626d46a3bb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd19809c620 tx=0x7fd19809bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70127c3f5aee240f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd187fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd187fff640 1 -- 172.21.15.82:0/552737189 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19801d070 con 0x5626d4698830 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd187fff640 1 -- 172.21.15.82:0/552737189 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fd1980983b0 con 0x5626d4698830 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1b1998740 1 -- 172.21.15.82:0/552737189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1900043d0 con 0x5626d4698830 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd187fff640 1 -- 172.21.15.82:0/552737189 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd1980a50d0 con 0x5626d4698830 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1b1998740 10 client.15450 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd1b1998740 1 -- 172.21.15.82:0/552737189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626d4715180 con 0x5626d4698830 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.961+0000 7fd187fff640 1 -- 172.21.15.82:0/552737189 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1900043d0 con 0x5626d4698830 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd187fff640 1 -- 172.21.15.82:0/552737189 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd1980b5be0 con 0x5626d4698830 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd1b1998740 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd1b1998740 10 client.15450 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:46:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd1b1998740 20 client.15450 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd1b1998740 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5626d4719d20 0x5626d4716830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd1b1998740 4 client.15450 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd1b1998740 1 -- 172.21.15.82:0/552737189 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "status"}) -- 0x5626d46043a0 con 0x5626d4719d20 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd1aca64640 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5626d4719d20 0x5626d4716830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd187fff640 10 client.15450 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd1aca64640 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5626d4719d20 0x5626d4716830 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=120d93d52a930ebd server_cookie=2e715f08706784e5 in_seq=0 out_seq=0 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd187fff640 1 -- 172.21.15.82:0/552737189 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5626d46043a0 con 0x5626d4719d20 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd187fff640 10 client.15450 handle_command_reply: tid=0 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.962+0000 7fd1b1998740 1 client.15450 shutdown 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1b1998740 1 -- 172.21.15.82:0/552737189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 msgr2=0x5626d46a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1b1998740 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 0x5626d46a3bb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd19809c620 tx=0x7fd19809bd10 comp rx=0 tx=0).stop 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1b1998740 1 -- 172.21.15.82:0/552737189 shutdown_connections 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1b1998740 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5626d4719d20 0x5626d4716830 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1b1998740 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5626d46992d0 0x5626d46a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1b1998740 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626d4698830 0x5626d46a3bb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd187fff640 0 client.15450 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:46:51.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1b1998740 1 --2- 172.21.15.82:0/552737189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626d4697e40 0x5626d46a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1b1998740 1 -- 172.21.15.82:0/552737189 >> 172.21.15.82:0/552737189 conn(0x5626d4693f40 msgr2=0x5626d469c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:51.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1b1998740 1 -- 172.21.15.82:0/552737189 shutdown_connections 2024-04-21T08:46:51.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1b1998740 1 -- 172.21.15.82:0/552737189 wait complete. 2024-04-21T08:46:51.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1b1998740 20 client.15450 trim_cache size 0 max 16384 2024-04-21T08:46:51.970 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:46:51.970 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:46:51.970 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 4, 2024-04-21T08:46:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24407, 2024-04-21T08:46:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:46:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:46:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:46:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 110.961408972, 2024-04-21T08:46:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2024-04-21T08:46:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 93, 2024-04-21T08:46:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:46:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 172.136566931 2024-04-21T08:46:51.971 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:46:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1aecc8640 1 -- 172.21.15.82:0/3324453682 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd1940a3bb0 msgr2=0x7fd1940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1aecc8640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd1940a3bb0 0x7fd1940a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009ca0 tx=0x7fd1a00099e0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1aecc8640 1 -- 172.21.15.82:0/3324453682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a8101fe0 msgr2=0x7fd1a81986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1aecc8640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a8101fe0 0x7fd1a81986d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c970 tx=0x7fd19800ce30 comp rx=0 tx=0).stop 2024-04-21T08:46:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1aecc8640 1 -- 172.21.15.82:0/3324453682 shutdown_connections 2024-04-21T08:46:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1aecc8640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fd1940a3bb0 0x7fd1940a6070 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1aecc8640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1a81029a0 0x7fd1a819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1aecc8640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1a8101fe0 0x7fd1a81986d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1aecc8640 1 --2- 172.21.15.82:0/3324453682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1a8100e90 0x7fd1a8198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1aecc8640 1 -- 172.21.15.82:0/3324453682 >> 172.21.15.82:0/3324453682 conn(0x7fd1a80fc400 msgr2=0x7fd1a81053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1aecc8640 1 -- 172.21.15.82:0/3324453682 shutdown_connections 2024-04-21T08:46:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:51.963+0000 7fd1aecc8640 1 -- 172.21.15.82:0/3324453682 wait complete. 2024-04-21T08:46:51.975 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-21T08:46:51.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:51 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2408631390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:46:51.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:51 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3202355176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:46:51.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:51 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2479111185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:46:52.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 -- 172.21.15.82:0/4059755515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85841053a0 msgr2=0x7f8584107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 --2- 172.21.15.82:0/4059755515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85841053a0 0x7f8584107790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8570009900 tx=0x7f857002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:52.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 -- 172.21.15.82:0/4059755515 shutdown_connections 2024-04-21T08:46:52.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 --2- 172.21.15.82:0/4059755515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8584107cd0 0x7f858410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 --2- 172.21.15.82:0/4059755515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85841053a0 0x7f8584107790 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 --2- 172.21.15.82:0/4059755515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8584102a70 0x7f8584104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 -- 172.21.15.82:0/4059755515 >> 172.21.15.82:0/4059755515 conn(0x7f85840fc400 msgr2=0x7f85840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 -- 172.21.15.82:0/4059755515 shutdown_connections 2024-04-21T08:46:52.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 -- 172.21.15.82:0/4059755515 wait complete. 2024-04-21T08:46:52.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 Processor -- start 2024-04-21T08:46:52.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 -- start start 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8584102a70 0x7f8584198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85841053a0 0x7f8584198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8584107cd0 0x7f858419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f858410dac0 con 0x7f8584107cd0 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f858410d940 con 0x7f8584102a70 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.074+0000 7f858ac43640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f858410dc40 con 0x7f85841053a0 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f85891e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8584107cd0 0x7f858419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f85891e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8584107cd0 0x7f858419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42380/0 (socket says 172.21.15.82:42380) 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f85891e0640 1 -- 172.21.15.82:0/3983481019 learned_addr learned my addr 172.21.15.82:0/3983481019 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f8583fff640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85841053a0 0x7f8584198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f85889df640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8584102a70 0x7f8584198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f85891e0640 1 -- 172.21.15.82:0/3983481019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85841053a0 msgr2=0x7f8584198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f85891e0640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85841053a0 0x7f8584198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f85891e0640 1 -- 172.21.15.82:0/3983481019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8584102a70 msgr2=0x7f8584198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f85891e0640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8584102a70 0x7f8584198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f85891e0640 1 -- 172.21.15.82:0/3983481019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858419fe70 con 0x7f8584107cd0 2024-04-21T08:46:52.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f85891e0640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8584107cd0 0x7f858419f800 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f857c00eaa0 tx=0x7f857c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f8581ffb640 1 -- 172.21.15.82:0/3983481019 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c00cdb0 con 0x7f8584107cd0 2024-04-21T08:46:52.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f8581ffb640 1 -- 172.21.15.82:0/3983481019 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857c005020 con 0x7f8584107cd0 2024-04-21T08:46:52.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f8581ffb640 1 -- 172.21.15.82:0/3983481019 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c0116d0 con 0x7f8584107cd0 2024-04-21T08:46:52.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f858ac43640 1 -- 172.21.15.82:0/3983481019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85841a0160 con 0x7f8584107cd0 2024-04-21T08:46:52.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.075+0000 7f858ac43640 1 -- 172.21.15.82:0/3983481019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85841a0630 con 0x7f8584107cd0 2024-04-21T08:46:52.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.077+0000 7f8581ffb640 1 -- 172.21.15.82:0/3983481019 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f857c0051d0 con 0x7f8584107cd0 2024-04-21T08:46:52.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f858d837740 1 Processor -- start 2024-04-21T08:46:52.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f858d837740 1 -- start start 2024-04-21T08:46:52.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f8581ffb640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f85740a3bb0 0x7f85740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f858d837740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f858d837740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cc830 0x557d9e2cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f858d837740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cd2d0 0x557d9e2d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f858d837740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557d9e230b10 con 0x557d9e2cbe40 2024-04-21T08:46:52.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f858d837740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557d9e1ce240 con 0x557d9e2cd2d0 2024-04-21T08:46:52.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f858d837740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d9e22cad0 con 0x557d9e2cc830 2024-04-21T08:46:52.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f8583fff640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f85740a3bb0 0x7f85740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f8581ffb640 1 -- 172.21.15.82:0/3983481019 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f857c0d09d0 con 0x7f8584107cd0 2024-04-21T08:46:52.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f85889df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f85889df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42392/0 (socket says 172.21.15.82:42392) 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f85889df640 1 -- 172.21.15.82:0/729238449 learned_addr learned my addr 172.21.15.82:0/729238449 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f85889df640 1 -- 172.21.15.82:0/729238449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cc830 msgr2=0x557d9e2cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f85889df640 1 --2- 172.21.15.82:0/729238449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cc830 0x557d9e2cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f85889df640 1 -- 172.21.15.82:0/729238449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cd2d0 msgr2=0x557d9e2d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f85889df640 1 --2- 172.21.15.82:0/729238449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cd2d0 0x557d9e2d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f85889df640 1 -- 172.21.15.82:0/729238449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d9df432d0 con 0x557d9e2cbe40 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f8583fff640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f85740a3bb0 0x7f85740a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8570005ab0 tx=0x7f857002c9c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.078+0000 7f85889df640 1 --2- 172.21.15.82:0/729238449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2cc260 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f857800cbe0 tx=0x7f857800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f7758d2eb0d4a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.079+0000 7f856effd640 1 -- 172.21.15.82:0/729238449 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578010630 con 0x557d9e2cbe40 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.079+0000 7f856effd640 1 -- 172.21.15.82:0/729238449 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8578010c80 con 0x557d9e2cbe40 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.079+0000 7f856effd640 1 -- 172.21.15.82:0/729238449 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857801aaf0 con 0x557d9e2cbe40 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.079+0000 7f858d837740 1 -- 172.21.15.82:0/729238449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 msgr2=0x557d9e2cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.079+0000 7f858d837740 1 --2- 172.21.15.82:0/729238449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2cc260 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f857800cbe0 tx=0x7f857800e720 comp rx=0 tx=0).stop 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.079+0000 7f858d837740 1 -- 172.21.15.82:0/729238449 shutdown_connections 2024-04-21T08:46:52.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.079+0000 7f858d837740 1 --2- 172.21.15.82:0/729238449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cd2d0 0x557d9e2d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.079+0000 7f858d837740 1 --2- 172.21.15.82:0/729238449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cc830 0x557d9e2cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.079+0000 7f858d837740 1 --2- 172.21.15.82:0/729238449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2cc260 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.079+0000 7f858d837740 1 -- 172.21.15.82:0/729238449 >> 172.21.15.82:0/729238449 conn(0x557d9e2c7f40 msgr2=0x557d9e2c6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.079+0000 7f858d837740 1 -- 172.21.15.82:0/729238449 shutdown_connections 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.079+0000 7f858d837740 1 -- 172.21.15.82:0/729238449 wait complete. 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f858d837740 1 -- start start 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f858d837740 1 Processor -- start 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f858d837740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f858d837740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cc830 0x557d9e2d6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f858d837740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f858d837740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557d9e2d44b0 con 0x557d9e2cbe40 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f858d837740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557d9e1b9190 con 0x557d9e2cc830 2024-04-21T08:46:52.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f858d837740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d9e2d4630 con 0x557d9e2cd2d0 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f85889df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f858d837740 10 client.0 _ll_register_callbacks cb 0x557d9e1b48b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f85889df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2d5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42408/0 (socket says 172.21.15.82:42408) 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f85889df640 1 -- 172.21.15.82:0/839198218 learned_addr learned my addr 172.21.15.82:0/839198218 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f85891e0640 1 --2- 172.21.15.82:0/839198218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f85889df640 1 -- 172.21.15.82:0/839198218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cd2d0 msgr2=0x557d9e2dd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f85889df640 1 --2- 172.21.15.82:0/839198218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f85889df640 1 -- 172.21.15.82:0/839198218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cc830 msgr2=0x557d9e2d6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f85889df640 1 --2- 172.21.15.82:0/839198218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cc830 0x557d9e2d6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.080+0000 7f85889df640 1 -- 172.21.15.82:0/839198218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d9e00d6b0 con 0x557d9e2cbe40 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f85889df640 1 --2- 172.21.15.82:0/839198218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2d5f40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f85780058c0 tx=0x7f85780056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b657220acb9e4068 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f8563fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:52.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f8563fff640 1 -- 172.21.15.82:0/839198218 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857802a060 con 0x557d9e2cbe40 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f8563fff640 1 -- 172.21.15.82:0/839198218 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f857801a410 con 0x557d9e2cbe40 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f858d837740 1 -- 172.21.15.82:0/839198218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8578010c80 con 0x557d9e2cbe40 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f8563fff640 1 -- 172.21.15.82:0/839198218 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8578031050 con 0x557d9e2cbe40 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f858d837740 10 client.15462 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f858d837740 1 -- 172.21.15.82:0/839198218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d9e2e0830 con 0x557d9e2cbe40 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f8563fff640 1 -- 172.21.15.82:0/839198218 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8578010c80 con 0x557d9e2cbe40 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f8563fff640 1 -- 172.21.15.82:0/839198218 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8578025620 con 0x557d9e2cbe40 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f858d837740 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f858d837740 10 client.15462 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f858d837740 20 client.15462 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f858d837740 1 --2- 172.21.15.82:0/839198218 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557d9e2e2aa0 0x557d9e2e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f858d837740 4 client.15462 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.081+0000 7f858d837740 1 -- 172.21.15.82:0/839198218 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d9e23e530 con 0x557d9e2e2aa0 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.082+0000 7f8583fff640 1 --2- 172.21.15.82:0/839198218 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557d9e2e2aa0 0x557d9e2e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.082+0000 7f8583fff640 1 --2- 172.21.15.82:0/839198218 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557d9e2e2aa0 0x557d9e2e3f10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8836608a3320dd27 server_cookie=6a7da07a117086be in_seq=0 out_seq=0 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.082+0000 7f8563fff640 10 client.15462 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f8563fff640 1 -- 172.21.15.82:0/839198218 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d9e23e530 con 0x557d9e2e2aa0 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f8563fff640 10 client.15462 handle_command_reply: tid=0 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f858d837740 1 client.15462 shutdown 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f858d837740 1 -- 172.21.15.82:0/839198218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 msgr2=0x557d9e2d5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f858d837740 1 --2- 172.21.15.82:0/839198218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2d5f40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f85780058c0 tx=0x7f85780056d0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f858d837740 1 -- 172.21.15.82:0/839198218 shutdown_connections 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f858d837740 1 --2- 172.21.15.82:0/839198218 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557d9e2e2aa0 0x557d9e2e3f10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f858d837740 1 --2- 172.21.15.82:0/839198218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f8563fff640 0 client.15462 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f858d837740 1 --2- 172.21.15.82:0/839198218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cc830 0x557d9e2d6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f858d837740 1 --2- 172.21.15.82:0/839198218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2d5f40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f858d837740 1 -- 172.21.15.82:0/839198218 >> 172.21.15.82:0/839198218 conn(0x557d9e2c7f40 msgr2=0x557d9e2c5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f858d837740 1 -- 172.21.15.82:0/839198218 shutdown_connections 2024-04-21T08:46:52.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f858d837740 1 -- 172.21.15.82:0/839198218 wait complete. 2024-04-21T08:46:52.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.083+0000 7f858d837740 20 client.15462 trim_cache size 0 max 16384 2024-04-21T08:46:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.101+0000 7f858d837740 1 Processor -- start 2024-04-21T08:46:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.101+0000 7f858d837740 1 -- start start 2024-04-21T08:46:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.101+0000 7f858d837740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2c6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.101+0000 7f858d837740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cc830 0x557d9e2c6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.101+0000 7f858d837740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.101+0000 7f858d837740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557d9e2e2250 con 0x557d9e2cbe40 2024-04-21T08:46:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.101+0000 7f858d837740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557d9e2e2920 con 0x557d9e2cc830 2024-04-21T08:46:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.101+0000 7f858d837740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d9e2e2ff0 con 0x557d9e2cd2d0 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.101+0000 7f85889df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.101+0000 7f85889df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42418/0 (socket says 172.21.15.82:42418) 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.101+0000 7f85889df640 1 -- 172.21.15.82:0/1257843868 learned_addr learned my addr 172.21.15.82:0/1257843868 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f85889df640 1 -- 172.21.15.82:0/1257843868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cd2d0 msgr2=0x557d9e2dd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f85889df640 1 --2- 172.21.15.82:0/1257843868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f85889df640 1 -- 172.21.15.82:0/1257843868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cc830 msgr2=0x557d9e2c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f85889df640 1 --2- 172.21.15.82:0/1257843868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cc830 0x557d9e2c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f85889df640 1 -- 172.21.15.82:0/1257843868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d9e2c7630 con 0x557d9e2cbe40 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f85889df640 1 --2- 172.21.15.82:0/1257843868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2c6400 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8578005ab0 tx=0x7f857800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5a34e8f9274aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f856e7fc640 1 -- 172.21.15.82:0/1257843868 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857802a070 con 0x557d9e2cbe40 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f856e7fc640 1 -- 172.21.15.82:0/1257843868 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857800c6f0 con 0x557d9e2cbe40 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f856e7fc640 1 -- 172.21.15.82:0/1257843868 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578005290 con 0x557d9e2cbe40 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f858d837740 1 -- 172.21.15.82:0/1257843868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 msgr2=0x557d9e2c6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f858d837740 1 --2- 172.21.15.82:0/1257843868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2c6400 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8578005ab0 tx=0x7f857800c330 comp rx=0 tx=0).stop 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f858d837740 1 -- 172.21.15.82:0/1257843868 shutdown_connections 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f858d837740 1 --2- 172.21.15.82:0/1257843868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f858d837740 1 --2- 172.21.15.82:0/1257843868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cc830 0x557d9e2c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f858d837740 1 --2- 172.21.15.82:0/1257843868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cbe40 0x557d9e2c6400 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.102+0000 7f858d837740 1 -- 172.21.15.82:0/1257843868 >> 172.21.15.82:0/1257843868 conn(0x557d9e2c7f40 msgr2=0x557d9e2c1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f858d837740 1 -- 172.21.15.82:0/1257843868 shutdown_connections 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f858d837740 1 -- 172.21.15.82:0/1257843868 wait complete. 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f858d837740 1 -- start start 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f858d837740 1 Processor -- start 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f858d837740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cbe40 0x557d9e2d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f858d837740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cc830 0x557d9e2db910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f858d837740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f858d837740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557d9e2e3d90 con 0x557d9e2cd2d0 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f858d837740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557d9e2e36c0 con 0x557d9e2cbe40 2024-04-21T08:46:52.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f858d837740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d9e2e4460 con 0x557d9e2cc830 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f858d837740 10 client.0 _ll_register_callbacks cb 0x7f8578014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f85891e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f85891e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42430/0 (socket says 172.21.15.82:42430) 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.103+0000 7f85891e0640 1 -- 172.21.15.82:0/1229890377 learned_addr learned my addr 172.21.15.82:0/1229890377 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f85889df640 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cbe40 0x557d9e2d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f85891e0640 1 -- 172.21.15.82:0/1229890377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cc830 msgr2=0x557d9e2db910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f8583fff640 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cc830 0x557d9e2db910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f85891e0640 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cc830 0x557d9e2db910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f85891e0640 1 -- 172.21.15.82:0/1229890377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cbe40 msgr2=0x557d9e2d93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f85891e0640 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cbe40 0x557d9e2d93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f85891e0640 1 -- 172.21.15.82:0/1229890377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d9e2dc5c0 con 0x557d9e2cd2d0 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f85889df640 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cbe40 0x557d9e2d93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f85891e0640 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dbe50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f857c098d40 tx=0x7f857c09d120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff54654dfe58ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f8563fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f8563fff640 1 -- 172.21.15.82:0/1229890377 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c01f040 con 0x557d9e2cd2d0 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f8563fff640 1 -- 172.21.15.82:0/1229890377 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f857c0a50c0 con 0x557d9e2cd2d0 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f858d837740 1 -- 172.21.15.82:0/1229890377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f857800c6f0 con 0x557d9e2cd2d0 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f8563fff640 1 -- 172.21.15.82:0/1229890377 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f857c0a52c0 con 0x557d9e2cd2d0 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f858d837740 10 client.15468 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.104+0000 7f858d837740 1 -- 172.21.15.82:0/1229890377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d9e348a50 con 0x557d9e2cd2d0 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.105+0000 7f8563fff640 1 -- 172.21.15.82:0/1229890377 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857800c6f0 con 0x557d9e2cd2d0 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.105+0000 7f8563fff640 1 -- 172.21.15.82:0/1229890377 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f857c00f040 con 0x557d9e2cd2d0 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.105+0000 7f858d837740 10 client.15468 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.105+0000 7f858d837740 10 client.15468 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.105+0000 7f858d837740 20 client.15468 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.105+0000 7f858d837740 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557d9e2e2aa0 0x557d9e34c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.105+0000 7f858d837740 4 client.15468 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.105+0000 7f858d837740 1 -- 172.21.15.82:0/1229890377 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "status"}) -- 0x557d9e2383a0 con 0x557d9e2e2aa0 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.105+0000 7f85889df640 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557d9e2e2aa0 0x557d9e34c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f8563fff640 10 client.15468 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f85889df640 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557d9e2e2aa0 0x557d9e34c340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d22cac459d14c4aa server_cookie=9b757b4c8e10349e in_seq=0 out_seq=0 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f8563fff640 1 -- 172.21.15.82:0/1229890377 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557d9e2383a0 con 0x557d9e2e2aa0 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f8563fff640 10 client.15468 handle_command_reply: tid=0 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f858d837740 1 client.15468 shutdown 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f858d837740 1 -- 172.21.15.82:0/1229890377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cd2d0 msgr2=0x557d9e2dbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f858d837740 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dbe50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f857c098d40 tx=0x7f857c09d120 comp rx=0 tx=0).stop 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f858d837740 1 -- 172.21.15.82:0/1229890377 shutdown_connections 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f858d837740 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557d9e2e2aa0 0x557d9e34c340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f858d837740 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d9e2cd2d0 0x557d9e2dbe50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f8563fff640 0 client.15468 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:46:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f858d837740 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d9e2cc830 0x557d9e2db910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f858d837740 1 --2- 172.21.15.82:0/1229890377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d9e2cbe40 0x557d9e2d93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f858d837740 1 -- 172.21.15.82:0/1229890377 >> 172.21.15.82:0/1229890377 conn(0x557d9e2c7f40 msgr2=0x557d9e2c5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f858d837740 1 -- 172.21.15.82:0/1229890377 shutdown_connections 2024-04-21T08:46:52.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f858d837740 1 -- 172.21.15.82:0/1229890377 wait complete. 2024-04-21T08:46:52.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.106+0000 7f858d837740 20 client.15468 trim_cache size 0 max 16384 2024-04-21T08:46:52.113 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:46:52.113 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:46:52.113 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 3, 2024-04-21T08:46:52.113 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24409, 2024-04-21T08:46:52.113 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:46:52.114 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:46:52.114 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:46:52.114 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 112.11785460500001, 2024-04-21T08:46:52.114 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2024-04-21T08:46:52.114 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 93, 2024-04-21T08:46:52.114 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:46:52.114 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 175.15503038599999 2024-04-21T08:46:52.114 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:46:52.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.107+0000 7f858ac43640 1 -- 172.21.15.82:0/3983481019 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f85740a3bb0 msgr2=0x7f85740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.107+0000 7f858ac43640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f85740a3bb0 0x7f85740a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8570005ab0 tx=0x7f857002c9c0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.107+0000 7f858ac43640 1 -- 172.21.15.82:0/3983481019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8584107cd0 msgr2=0x7f858419f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.107+0000 7f858ac43640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8584107cd0 0x7f858419f800 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f857c00eaa0 tx=0x7f857c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:46:52.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.107+0000 7f858ac43640 1 -- 172.21.15.82:0/3983481019 shutdown_connections 2024-04-21T08:46:52.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.107+0000 7f858ac43640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f85740a3bb0 0x7f85740a6070 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.107+0000 7f858ac43640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8584107cd0 0x7f858419f800 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.107+0000 7f858ac43640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85841053a0 0x7f8584198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.107+0000 7f858ac43640 1 --2- 172.21.15.82:0/3983481019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8584102a70 0x7f8584198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.107+0000 7f858ac43640 1 -- 172.21.15.82:0/3983481019 >> 172.21.15.82:0/3983481019 conn(0x7f85840fc400 msgr2=0x7f8584104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.107+0000 7f858ac43640 1 -- 172.21.15.82:0/3983481019 shutdown_connections 2024-04-21T08:46:52.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.107+0000 7f858ac43640 1 -- 172.21.15.82:0/3983481019 wait complete. 2024-04-21T08:46:52.118 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-21T08:46:52.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.216+0000 7f3e63cc0640 1 -- 172.21.15.82:0/3555687980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c103160 msgr2=0x7f3e5c1055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.216+0000 7f3e63cc0640 1 --2- 172.21.15.82:0/3555687980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c103160 0x7f3e5c1055d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009900 tx=0x7f3e4c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:52.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.216+0000 7f3e63cc0640 1 -- 172.21.15.82:0/3555687980 shutdown_connections 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.216+0000 7f3e63cc0640 1 --2- 172.21.15.82:0/3555687980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e5c105b10 0x7f3e5c105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.216+0000 7f3e63cc0640 1 --2- 172.21.15.82:0/3555687980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c103160 0x7f3e5c1055d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.216+0000 7f3e63cc0640 1 --2- 172.21.15.82:0/3555687980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c1007f0 0x7f3e5c102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.216+0000 7f3e63cc0640 1 -- 172.21.15.82:0/3555687980 >> 172.21.15.82:0/3555687980 conn(0x7f3e5c0fc410 msgr2=0x7f3e5c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.216+0000 7f3e63cc0640 1 -- 172.21.15.82:0/3555687980 shutdown_connections 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.216+0000 7f3e63cc0640 1 -- 172.21.15.82:0/3555687980 wait complete. 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e63cc0640 1 Processor -- start 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e63cc0640 1 -- start start 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e63cc0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e5c1007f0 0x7f3e5c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e63cc0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c103160 0x7f3e5c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e63cc0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c105b10 0x7f3e5c1a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e63cc0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c1140d0 con 0x7f3e5c103160 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e63cc0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c113f50 con 0x7f3e5c1007f0 2024-04-21T08:46:52.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e63cc0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c114250 con 0x7f3e5c105b10 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e6125b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c103160 0x7f3e5c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e6125b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c103160 0x7f3e5c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42446/0 (socket says 172.21.15.82:42446) 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e6125b640 1 -- 172.21.15.82:0/456872521 learned_addr learned my addr 172.21.15.82:0/456872521 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e6225d640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c105b10 0x7f3e5c1a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.217+0000 7f3e61a5c640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e5c1007f0 0x7f3e5c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.218+0000 7f3e6125b640 1 -- 172.21.15.82:0/456872521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c105b10 msgr2=0x7f3e5c1a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.218+0000 7f3e6125b640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c105b10 0x7f3e5c1a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.218+0000 7f3e6125b640 1 -- 172.21.15.82:0/456872521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e5c1007f0 msgr2=0x7f3e5c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.218+0000 7f3e6125b640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e5c1007f0 0x7f3e5c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.218+0000 7f3e6125b640 1 -- 172.21.15.82:0/456872521 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c1a6440 con 0x7f3e5c103160 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.218+0000 7f3e6125b640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c103160 0x7f3e5c19ed60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c006010 tx=0x7f3e4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.218+0000 7f3e52ffd640 1 -- 172.21.15.82:0/456872521 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c043020 con 0x7f3e5c103160 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.218+0000 7f3e52ffd640 1 -- 172.21.15.82:0/456872521 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e4c004aa0 con 0x7f3e5c103160 2024-04-21T08:46:52.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.218+0000 7f3e63cc0640 1 -- 172.21.15.82:0/456872521 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5c1a66d0 con 0x7f3e5c103160 2024-04-21T08:46:52.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.218+0000 7f3e52ffd640 1 -- 172.21.15.82:0/456872521 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c003c70 con 0x7f3e5c103160 2024-04-21T08:46:52.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.218+0000 7f3e63cc0640 1 -- 172.21.15.82:0/456872521 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5c1a6c70 con 0x7f3e5c103160 2024-04-21T08:46:52.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.220+0000 7f3e52ffd640 1 -- 172.21.15.82:0/456872521 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3e4c046020 con 0x7f3e5c103160 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e66982740 1 Processor -- start 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e66982740 1 -- start start 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e66982740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff65fe40 0x55adff660260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e66982740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff660830 0x55adff660cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e66982740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 0x55adff667eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e66982740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55adff5c4b10 con 0x55adff6612d0 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e66982740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55adff562240 con 0x55adff65fe40 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e66982740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55adff5c0ad0 con 0x55adff660830 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e52ffd640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3e480a3c30 0x7f3e480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e6125b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 0x55adff667eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e6125b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 0x55adff667eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42450/0 (socket says 172.21.15.82:42450) 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e6225d640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3e480a3c30 0x7f3e480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e6125b640 1 -- 172.21.15.82:0/2677490553 learned_addr learned my addr 172.21.15.82:0/2677490553 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e61a5c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff660830 0x55adff660cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e6225d640 1 --2- 172.21.15.82:0/2677490553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff65fe40 0x55adff660260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.221+0000 7f3e52ffd640 1 -- 172.21.15.82:0/456872521 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f3e4c0f2af0 con 0x7f3e5c103160 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.222+0000 7f3e61a5c640 1 -- 172.21.15.82:0/2677490553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff65fe40 msgr2=0x55adff660260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.222+0000 7f3e61a5c640 1 --2- 172.21.15.82:0/2677490553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff65fe40 0x55adff660260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.222+0000 7f3e61a5c640 1 -- 172.21.15.82:0/2677490553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 msgr2=0x55adff667eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.222+0000 7f3e61a5c640 1 --2- 172.21.15.82:0/2677490553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 0x55adff667eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.222+0000 7f3e61a5c640 1 -- 172.21.15.82:0/2677490553 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adff2d72d0 con 0x55adff660830 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.222+0000 7f3e6125b640 1 --2- 172.21.15.82:0/2677490553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 0x55adff667eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.222+0000 7f3e6225d640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3e480a3c30 0x7f3e480a60f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3e54005fd0 tx=0x7f3e5400b860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.222+0000 7f3e61a5c640 1 --2- 172.21.15.82:0/2677490553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff660830 0x55adff660cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800adb0 tx=0x7f3e5800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89d2bb28acb22c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.222+0000 7f3e47fff640 1 -- 172.21.15.82:0/2677490553 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58003970 con 0x55adff660830 2024-04-21T08:46:52.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.222+0000 7f3e47fff640 1 -- 172.21.15.82:0/2677490553 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e58004020 con 0x55adff660830 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.222+0000 7f3e47fff640 1 -- 172.21.15.82:0/2677490553 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58011940 con 0x55adff660830 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.223+0000 7f3e66982740 1 -- 172.21.15.82:0/2677490553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff660830 msgr2=0x55adff660cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.223+0000 7f3e66982740 1 --2- 172.21.15.82:0/2677490553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff660830 0x55adff660cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800adb0 tx=0x7f3e5800f730 comp rx=0 tx=0).stop 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.223+0000 7f3e66982740 1 -- 172.21.15.82:0/2677490553 shutdown_connections 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.223+0000 7f3e66982740 1 --2- 172.21.15.82:0/2677490553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 0x55adff667eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.223+0000 7f3e66982740 1 --2- 172.21.15.82:0/2677490553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff660830 0x55adff660cb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.223+0000 7f3e66982740 1 --2- 172.21.15.82:0/2677490553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff65fe40 0x55adff660260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.223+0000 7f3e66982740 1 -- 172.21.15.82:0/2677490553 >> 172.21.15.82:0/2677490553 conn(0x55adff65bf40 msgr2=0x55adff65a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.223+0000 7f3e66982740 1 -- 172.21.15.82:0/2677490553 shutdown_connections 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.223+0000 7f3e66982740 1 -- 172.21.15.82:0/2677490553 wait complete. 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.223+0000 7f3e66982740 1 -- start start 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.223+0000 7f3e66982740 1 Processor -- start 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e66982740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff65fe40 0x55adff669eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e66982740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff660830 0x55adff66a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e66982740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 0x55adff671430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e66982740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55adff6684b0 con 0x55adff6612d0 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e66982740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55adff54d190 con 0x55adff660830 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e66982740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55adff668630 con 0x55adff65fe40 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e66982740 10 client.0 _ll_register_callbacks cb 0x55adff5488b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e6225d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 0x55adff671430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e6225d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 0x55adff671430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42464/0 (socket says 172.21.15.82:42464) 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e6225d640 1 -- 172.21.15.82:0/4142238500 learned_addr learned my addr 172.21.15.82:0/4142238500 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e61a5c640 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff65fe40 0x55adff669eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e6125b640 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff660830 0x55adff66a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e6225d640 1 -- 172.21.15.82:0/4142238500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff65fe40 msgr2=0x55adff669eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e6225d640 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff65fe40 0x55adff669eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e6225d640 1 -- 172.21.15.82:0/4142238500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff660830 msgr2=0x55adff66a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e6225d640 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff660830 0x55adff66a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e6225d640 1 -- 172.21.15.82:0/4142238500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adff3a16b0 con 0x55adff6612d0 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e6225d640 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 0x55adff671430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3e54004440 tx=0x7f3e54005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98053a4d9e5aea61 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e457fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e457fa640 1 -- 172.21.15.82:0/4142238500 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e540108a0 con 0x55adff6612d0 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e457fa640 1 -- 172.21.15.82:0/4142238500 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f3e54010a40 con 0x55adff6612d0 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e66982740 1 -- 172.21.15.82:0/4142238500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e58004020 con 0x55adff6612d0 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e457fa640 1 -- 172.21.15.82:0/4142238500 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e54033050 con 0x55adff6612d0 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e66982740 10 client.15480 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.224+0000 7f3e66982740 1 -- 172.21.15.82:0/4142238500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55adff674810 con 0x55adff6612d0 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.225+0000 7f3e457fa640 1 -- 172.21.15.82:0/4142238500 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e58004020 con 0x55adff6612d0 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.225+0000 7f3e457fa640 1 -- 172.21.15.82:0/4142238500 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3e5401e390 con 0x55adff6612d0 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.225+0000 7f3e66982740 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.225+0000 7f3e66982740 10 client.15480 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.225+0000 7f3e66982740 20 client.15480 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.225+0000 7f3e66982740 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55adff676f60 0x55adff6783d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.225+0000 7f3e66982740 4 client.15480 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:52.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.225+0000 7f3e66982740 1 -- 172.21.15.82:0/4142238500 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adff5d2530 con 0x55adff676f60 2024-04-21T08:46:52.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.225+0000 7f3e61a5c640 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55adff676f60 0x55adff6783d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.226+0000 7f3e457fa640 10 client.15480 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:46:52.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.226+0000 7f3e61a5c640 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55adff676f60 0x55adff6783d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f13b2cbd8303b0db server_cookie=643db32c8c873f6e in_seq=0 out_seq=0 2024-04-21T08:46:52.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.226+0000 7f3e457fa640 1 -- 172.21.15.82:0/4142238500 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55adff5d2530 con 0x55adff676f60 2024-04-21T08:46:52.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.226+0000 7f3e457fa640 10 client.15480 handle_command_reply: tid=0 2024-04-21T08:46:52.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.226+0000 7f3e66982740 1 client.15480 shutdown 2024-04-21T08:46:52.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.226+0000 7f3e66982740 1 -- 172.21.15.82:0/4142238500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 msgr2=0x55adff671430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.226+0000 7f3e66982740 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 0x55adff671430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3e54004440 tx=0x7f3e54005cc0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.227+0000 7f3e66982740 1 -- 172.21.15.82:0/4142238500 shutdown_connections 2024-04-21T08:46:52.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.227+0000 7f3e66982740 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55adff676f60 0x55adff6783d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.227+0000 7f3e66982740 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff6612d0 0x55adff671430 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.227+0000 7f3e457fa640 0 client.15480 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:46:52.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.227+0000 7f3e66982740 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff660830 0x55adff66a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.227+0000 7f3e66982740 1 --2- 172.21.15.82:0/4142238500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff65fe40 0x55adff669eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.227+0000 7f3e66982740 1 -- 172.21.15.82:0/4142238500 >> 172.21.15.82:0/4142238500 conn(0x55adff65bf40 msgr2=0x55adff659a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.227+0000 7f3e66982740 1 -- 172.21.15.82:0/4142238500 shutdown_connections 2024-04-21T08:46:52.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.227+0000 7f3e66982740 1 -- 172.21.15.82:0/4142238500 wait complete. 2024-04-21T08:46:52.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.227+0000 7f3e66982740 20 client.15480 trim_cache size 0 max 16384 2024-04-21T08:46:52.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.244+0000 7f3e66982740 1 Processor -- start 2024-04-21T08:46:52.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.244+0000 7f3e66982740 1 -- start start 2024-04-21T08:46:52.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e66982740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff65fe40 0x55adff65a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e66982740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff660830 0x55adff6713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e66982740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff6612d0 0x55adff65a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e66982740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55adff676710 con 0x55adff65fe40 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e66982740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55adff676de0 con 0x55adff660830 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e66982740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55adff6774b0 con 0x55adff6612d0 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e61a5c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff65fe40 0x55adff65a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e61a5c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff65fe40 0x55adff65a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42468/0 (socket says 172.21.15.82:42468) 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e61a5c640 1 -- 172.21.15.82:0/2436009204 learned_addr learned my addr 172.21.15.82:0/2436009204 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e6125b640 1 --2- 172.21.15.82:0/2436009204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff660830 0x55adff6713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e61a5c640 1 -- 172.21.15.82:0/2436009204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff6612d0 msgr2=0x55adff65a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e61a5c640 1 --2- 172.21.15.82:0/2436009204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff6612d0 0x55adff65a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e61a5c640 1 -- 172.21.15.82:0/2436009204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff660830 msgr2=0x55adff6713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e61a5c640 1 --2- 172.21.15.82:0/2436009204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff660830 0x55adff6713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e61a5c640 1 -- 172.21.15.82:0/2436009204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adff65bd40 con 0x55adff65fe40 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e6125b640 1 --2- 172.21.15.82:0/2436009204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff660830 0x55adff6713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e61a5c640 1 --2- 172.21.15.82:0/2436009204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff65fe40 0x55adff65a350 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800fc30 tx=0x7f3e5800a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7bad3c4f687612 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e477fe640 1 -- 172.21.15.82:0/2436009204 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5802c070 con 0x55adff65fe40 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e477fe640 1 -- 172.21.15.82:0/2436009204 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5800a9b0 con 0x55adff65fe40 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.245+0000 7f3e477fe640 1 -- 172.21.15.82:0/2436009204 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58003d80 con 0x55adff65fe40 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.246+0000 7f3e66982740 1 -- 172.21.15.82:0/2436009204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff65fe40 msgr2=0x55adff65a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.246+0000 7f3e66982740 1 --2- 172.21.15.82:0/2436009204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff65fe40 0x55adff65a350 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800fc30 tx=0x7f3e5800a6d0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.246+0000 7f3e66982740 1 -- 172.21.15.82:0/2436009204 shutdown_connections 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.246+0000 7f3e66982740 1 --2- 172.21.15.82:0/2436009204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff6612d0 0x55adff65a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.246+0000 7f3e66982740 1 --2- 172.21.15.82:0/2436009204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff660830 0x55adff6713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.246+0000 7f3e66982740 1 --2- 172.21.15.82:0/2436009204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff65fe40 0x55adff65a350 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.246+0000 7f3e66982740 1 -- 172.21.15.82:0/2436009204 >> 172.21.15.82:0/2436009204 conn(0x55adff65bf40 msgr2=0x55adff669eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.246+0000 7f3e66982740 1 -- 172.21.15.82:0/2436009204 shutdown_connections 2024-04-21T08:46:52.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.246+0000 7f3e66982740 1 -- 172.21.15.82:0/2436009204 wait complete. 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.246+0000 7f3e66982740 1 -- start start 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.246+0000 7f3e66982740 1 Processor -- start 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e66982740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff65fe40 0x55adff662bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e66982740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff660830 0x55adff6713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e66982740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff6612d0 0x55adff663220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e66982740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55adff678250 con 0x55adff660830 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e66982740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55adff677b80 con 0x55adff6612d0 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e66982740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55adff678920 con 0x55adff65fe40 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e66982740 10 client.0 _ll_register_callbacks cb 0x7f3e54004110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e6125b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff660830 0x55adff6713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e6125b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff660830 0x55adff6713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42482/0 (socket says 172.21.15.82:42482) 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e6125b640 1 -- 172.21.15.82:0/3150264120 learned_addr learned my addr 172.21.15.82:0/3150264120 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e6225d640 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff6612d0 0x55adff663220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e6125b640 1 -- 172.21.15.82:0/3150264120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff65fe40 msgr2=0x55adff662bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e6125b640 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff65fe40 0x55adff662bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e6125b640 1 -- 172.21.15.82:0/3150264120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff6612d0 msgr2=0x55adff663220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e6125b640 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff6612d0 0x55adff663220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e6125b640 1 -- 172.21.15.82:0/3150264120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adff665ca0 con 0x55adff660830 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e6225d640 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff6612d0 0x55adff663220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e6125b640 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff660830 0x55adff6713f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c091510 tx=0x7f3e4c08d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73f894cfbbb3e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.247+0000 7f3e457fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e457fa640 1 -- 172.21.15.82:0/3150264120 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c0be360 con 0x55adff660830 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e457fa640 1 -- 172.21.15.82:0/3150264120 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f3e4c0f0330 con 0x55adff660830 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e66982740 1 -- 172.21.15.82:0/3150264120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e5800a9b0 con 0x55adff660830 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e457fa640 1 -- 172.21.15.82:0/3150264120 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e4c0035f0 con 0x55adff660830 2024-04-21T08:46:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e66982740 10 client.15486 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e66982740 1 -- 172.21.15.82:0/3150264120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55adff666f10 con 0x55adff660830 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e457fa640 1 -- 172.21.15.82:0/3150264120 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5800a9b0 con 0x55adff660830 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e457fa640 1 -- 172.21.15.82:0/3150264120 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3e4c0c71e0 con 0x55adff660830 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e66982740 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e66982740 10 client.15486 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e66982740 20 client.15486 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e66982740 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55adff6e1360 0x55adff6dd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e66982740 4 client.15486 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e66982740 1 -- 172.21.15.82:0/3150264120 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "status"}) -- 0x55adff5cc3a0 con 0x55adff6e1360 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.248+0000 7f3e61a5c640 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55adff6e1360 0x55adff6dd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.249+0000 7f3e457fa640 10 client.15486 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.249+0000 7f3e61a5c640 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55adff6e1360 0x55adff6dd7a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5fb98a4b9da0b17 server_cookie=af0794c879565c2f in_seq=0 out_seq=0 2024-04-21T08:46:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.249+0000 7f3e457fa640 1 -- 172.21.15.82:0/3150264120 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55adff5cc3a0 con 0x55adff6e1360 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.249+0000 7f3e457fa640 10 client.15486 handle_command_reply: tid=0 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.249+0000 7f3e66982740 1 client.15486 shutdown 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.249+0000 7f3e66982740 1 -- 172.21.15.82:0/3150264120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff660830 msgr2=0x55adff6713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.249+0000 7f3e66982740 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff660830 0x55adff6713f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c091510 tx=0x7f3e4c08d400 comp rx=0 tx=0).stop 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.249+0000 7f3e66982740 1 -- 172.21.15.82:0/3150264120 shutdown_connections 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e66982740 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55adff6e1360 0x55adff6dd7a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e66982740 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55adff6612d0 0x55adff663220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e66982740 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adff660830 0x55adff6713f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e66982740 1 --2- 172.21.15.82:0/3150264120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55adff65fe40 0x55adff662bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e457fa640 0 client.15486 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e66982740 1 -- 172.21.15.82:0/3150264120 >> 172.21.15.82:0/3150264120 conn(0x55adff65bf40 msgr2=0x55adff65a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e66982740 1 -- 172.21.15.82:0/3150264120 shutdown_connections 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e66982740 1 -- 172.21.15.82:0/3150264120 wait complete. 2024-04-21T08:46:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e66982740 20 client.15486 trim_cache size 0 max 16384 2024-04-21T08:46:52.257 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:46:52.257 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:46:52.257 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 2, 2024-04-21T08:46:52.257 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24476, 2024-04-21T08:46:52.257 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:46:52.257 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:46:52.257 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:46:52.257 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 112.798933557, 2024-04-21T08:46:52.257 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2024-04-21T08:46:52.257 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 93, 2024-04-21T08:46:52.258 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:46:52.258 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 155.12588662900001 2024-04-21T08:46:52.258 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:46:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e63cc0640 1 -- 172.21.15.82:0/456872521 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3e480a3c30 msgr2=0x7f3e480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e63cc0640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3e480a3c30 0x7f3e480a60f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3e54005fd0 tx=0x7f3e5400b860 comp rx=0 tx=0).stop 2024-04-21T08:46:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e63cc0640 1 -- 172.21.15.82:0/456872521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c103160 msgr2=0x7f3e5c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e63cc0640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c103160 0x7f3e5c19ed60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c006010 tx=0x7f3e4c0042f0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e63cc0640 1 -- 172.21.15.82:0/456872521 shutdown_connections 2024-04-21T08:46:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e63cc0640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f3e480a3c30 0x7f3e480a60f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e63cc0640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e5c105b10 0x7f3e5c1a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e63cc0640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c103160 0x7f3e5c19ed60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e63cc0640 1 --2- 172.21.15.82:0/456872521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e5c1007f0 0x7f3e5c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e63cc0640 1 -- 172.21.15.82:0/456872521 >> 172.21.15.82:0/456872521 conn(0x7f3e5c0fc410 msgr2=0x7f3e5c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e63cc0640 1 -- 172.21.15.82:0/456872521 shutdown_connections 2024-04-21T08:46:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.250+0000 7f3e63cc0640 1 -- 172.21.15.82:0/456872521 wait complete. 2024-04-21T08:46:52.262 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T08:46:52.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.361+0000 7f2d14d7e640 1 -- 172.21.15.82:0/2955102633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d10103b50 msgr2=0x7f2d1010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.361+0000 7f2d14d7e640 1 --2- 172.21.15.82:0/2955102633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d10103b50 0x7f2d1010a3e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009ca0 tx=0x7f2d0802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.361+0000 7f2d14d7e640 1 -- 172.21.15.82:0/2955102633 shutdown_connections 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.361+0000 7f2d14d7e640 1 --2- 172.21.15.82:0/2955102633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d10103b50 0x7f2d1010a3e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 --2- 172.21.15.82:0/2955102633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d10103190 0x7f2d10103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 --2- 172.21.15.82:0/2955102633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d10101fa0 0x7f2d101023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 -- 172.21.15.82:0/2955102633 >> 172.21.15.82:0/2955102633 conn(0x7f2d100fd530 msgr2=0x7f2d100ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 -- 172.21.15.82:0/2955102633 shutdown_connections 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 -- 172.21.15.82:0/2955102633 wait complete. 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 Processor -- start 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 -- start start 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d10101fa0 0x7f2d101981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d10103190 0x7f2d10198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d10103b50 0x7f2d1019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1010db20 con 0x7f2d10101fa0 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1010d9a0 con 0x7f2d10103b50 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d14d7e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1010dca0 con 0x7f2d10103190 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.362+0000 7f2d0e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d10101fa0 0x7f2d101981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2d0e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d10101fa0 0x7f2d101981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42508/0 (socket says 172.21.15.82:42508) 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2d0e59c640 1 -- 172.21.15.82:0/3899180664 learned_addr learned my addr 172.21.15.82:0/3899180664 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2d0ed9d640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d10103b50 0x7f2d1019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2d0dd9b640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d10103190 0x7f2d10198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2d0e59c640 1 -- 172.21.15.82:0/3899180664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d10103190 msgr2=0x7f2d10198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d10103190 0x7f2d10198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2d0e59c640 1 -- 172.21.15.82:0/3899180664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d10103b50 msgr2=0x7f2d1019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d10103b50 0x7f2d1019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2d0e59c640 1 -- 172.21.15.82:0/3899180664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1019fde0 con 0x7f2d10101fa0 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d10101fa0 0x7f2d101981c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000d770 tx=0x7f2d0000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2cff7fe640 1 -- 172.21.15.82:0/3899180664 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00017070 con 0x7f2d10101fa0 2024-04-21T08:46:52.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2cff7fe640 1 -- 172.21.15.82:0/3899180664 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d00009c30 con 0x7f2d10101fa0 2024-04-21T08:46:52.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2d14d7e640 1 -- 172.21.15.82:0/3899180664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d101a00d0 con 0x7f2d10101fa0 2024-04-21T08:46:52.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2cff7fe640 1 -- 172.21.15.82:0/3899180664 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00003cc0 con 0x7f2d10101fa0 2024-04-21T08:46:52.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.363+0000 7f2d14d7e640 1 -- 172.21.15.82:0/3899180664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d101a0620 con 0x7f2d10101fa0 2024-04-21T08:46:52.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.365+0000 7f2cff7fe640 1 -- 172.21.15.82:0/3899180664 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2d00003e60 con 0x7f2d10101fa0 2024-04-21T08:46:52.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d1796f740 1 Processor -- start 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d1796f740 1 -- start start 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d1796f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0b7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d1796f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b7830 0x556dfd0b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d1796f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b82d0 0x556dfd0beeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d1796f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556dfd01bb10 con 0x556dfd0b6e40 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d1796f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556dfcfb9240 con 0x556dfd0b82d0 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d1796f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556dfd017ad0 con 0x556dfd0b7830 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2cff7fe640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2cf80a3bb0 0x7f2cf80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d0ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d0ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0b7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42510/0 (socket says 172.21.15.82:42510) 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d0ed9d640 1 -- 172.21.15.82:0/2471353750 learned_addr learned my addr 172.21.15.82:0/2471353750 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d0dd9b640 1 --2- 172.21.15.82:0/2471353750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b82d0 0x556dfd0beeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d0ed9d640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2cf80a3bb0 0x7f2cf80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2cff7fe640 1 -- 172.21.15.82:0/3899180664 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f2d000d0160 con 0x7f2d10101fa0 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/2471353750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b7830 0x556dfd0b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d0ed9d640 1 -- 172.21.15.82:0/2471353750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b7830 msgr2=0x556dfd0b7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d0ed9d640 1 --2- 172.21.15.82:0/2471353750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b7830 0x556dfd0b7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d0ed9d640 1 -- 172.21.15.82:0/2471353750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b82d0 msgr2=0x556dfd0beeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d0ed9d640 1 --2- 172.21.15.82:0/2471353750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b82d0 0x556dfd0beeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.366+0000 7f2d0ed9d640 1 -- 172.21.15.82:0/2471353750 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dfcd2e2d0 con 0x556dfd0b6e40 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.367+0000 7f2d0dd9b640 1 --2- 172.21.15.82:0/2471353750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b82d0 0x556dfd0beeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.367+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/2471353750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b7830 0x556dfd0b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.367+0000 7f2d0ed9d640 1 --2- 172.21.15.82:0/2471353750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0b7260 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2d08037040 tx=0x7f2d08032780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e123afce574d438 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.367+0000 7f2d0ed9d640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2cf80a3bb0 0x7f2cf80a6070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2d08005e30 tx=0x7f2d08042000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.367+0000 7f2cfcff9640 1 -- 172.21.15.82:0/2471353750 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08051070 con 0x556dfd0b6e40 2024-04-21T08:46:52.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.367+0000 7f2cfcff9640 1 -- 172.21.15.82:0/2471353750 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d08032df0 con 0x556dfd0b6e40 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.367+0000 7f2cfcff9640 1 -- 172.21.15.82:0/2471353750 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08039040 con 0x556dfd0b6e40 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 -- 172.21.15.82:0/2471353750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 msgr2=0x556dfd0b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 --2- 172.21.15.82:0/2471353750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0b7260 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2d08037040 tx=0x7f2d08032780 comp rx=0 tx=0).stop 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 -- 172.21.15.82:0/2471353750 shutdown_connections 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 --2- 172.21.15.82:0/2471353750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b82d0 0x556dfd0beeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 --2- 172.21.15.82:0/2471353750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b7830 0x556dfd0b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 --2- 172.21.15.82:0/2471353750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0b7260 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 -- 172.21.15.82:0/2471353750 >> 172.21.15.82:0/2471353750 conn(0x556dfd0b2f40 msgr2=0x556dfd0b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 -- 172.21.15.82:0/2471353750 shutdown_connections 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 -- 172.21.15.82:0/2471353750 wait complete. 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 -- start start 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 Processor -- start 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0c0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b7830 0x556dfd0c1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 0x556dfd0c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556dfd0bf450 con 0x556dfd0b6e40 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556dfcfa4190 con 0x556dfd0b7830 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556dfd0bf5d0 con 0x556dfd0b82d0 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d1796f740 10 client.0 _ll_register_callbacks cb 0x556dfcf9f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d0e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0c0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d0e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0c0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42518/0 (socket says 172.21.15.82:42518) 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d0e59c640 1 -- 172.21.15.82:0/461698375 learned_addr learned my addr 172.21.15.82:0/461698375 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.368+0000 7f2d0ed9d640 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 0x556dfd0c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d0dd9b640 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b7830 0x556dfd0c1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d0e59c640 1 -- 172.21.15.82:0/461698375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 msgr2=0x556dfd0c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 0x556dfd0c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d0e59c640 1 -- 172.21.15.82:0/461698375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b7830 msgr2=0x556dfd0c1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b7830 0x556dfd0c1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d0e59c640 1 -- 172.21.15.82:0/461698375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dfcdf86b0 con 0x556dfd0b6e40 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d0ed9d640 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 0x556dfd0c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d0dd9b640 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b7830 0x556dfd0c1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0c0ee0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009610 tx=0x7f2d0009bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c34201e538142e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2cf5ffb640 1 -- 172.21.15.82:0/461698375 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00020050 con 0x556dfd0b6e40 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2cf5ffb640 1 -- 172.21.15.82:0/461698375 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f2d000b5560 con 0x556dfd0b6e40 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d1796f740 1 -- 172.21.15.82:0/461698375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556dfd029530 con 0x556dfd0b6e40 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2cf5ffb640 1 -- 172.21.15.82:0/461698375 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d0009c070 con 0x556dfd0b6e40 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d1796f740 10 client.15498 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d1796f740 1 -- 172.21.15.82:0/461698375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556dfd0cb830 con 0x556dfd0b6e40 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2cf5ffb640 1 -- 172.21.15.82:0/461698375 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556dfd029530 con 0x556dfd0b6e40 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2cf5ffb640 1 -- 172.21.15.82:0/461698375 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2d000986e0 con 0x556dfd0b6e40 2024-04-21T08:46:52.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d1796f740 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d1796f740 10 client.15498 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d1796f740 20 client.15498 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.369+0000 7f2d1796f740 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x556dfd0cded0 0x556dfd0cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.370+0000 7f2d1796f740 4 client.15498 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.370+0000 7f2d1796f740 1 -- 172.21.15.82:0/461698375 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dfd0233a0 con 0x556dfd0cded0 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.370+0000 7f2d0dd9b640 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x556dfd0cded0 0x556dfd0cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.370+0000 7f2d0dd9b640 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x556dfd0cded0 0x556dfd0cf340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=913a763ce569c525 server_cookie=c80ea12650b464ae in_seq=0 out_seq=0 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.370+0000 7f2cf5ffb640 10 client.15498 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2cf5ffb640 1 -- 172.21.15.82:0/461698375 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556dfd0233a0 con 0x556dfd0cded0 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2cf5ffb640 10 client.15498 handle_command_reply: tid=0 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2d1796f740 1 client.15498 shutdown 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2d1796f740 1 -- 172.21.15.82:0/461698375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 msgr2=0x556dfd0c0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2d1796f740 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0c0ee0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009610 tx=0x7f2d0009bcb0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2d1796f740 1 -- 172.21.15.82:0/461698375 shutdown_connections 2024-04-21T08:46:52.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2d1796f740 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x556dfd0cded0 0x556dfd0cf340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2d1796f740 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 0x556dfd0c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2d1796f740 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b7830 0x556dfd0c1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2d1796f740 1 --2- 172.21.15.82:0/461698375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0c0ee0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2cf5ffb640 0 client.15498 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:46:52.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2d1796f740 1 -- 172.21.15.82:0/461698375 >> 172.21.15.82:0/461698375 conn(0x556dfd0b2f40 msgr2=0x556dfd0b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2d1796f740 1 -- 172.21.15.82:0/461698375 shutdown_connections 2024-04-21T08:46:52.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2d1796f740 1 -- 172.21.15.82:0/461698375 wait complete. 2024-04-21T08:46:52.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.371+0000 7f2d1796f740 20 client.15498 trim_cache size 0 max 16384 2024-04-21T08:46:52.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.389+0000 7f2d1796f740 1 Processor -- start 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.389+0000 7f2d1796f740 1 -- start start 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.389+0000 7f2d1796f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0bf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.389+0000 7f2d1796f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b7830 0x556dfd0c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.389+0000 7f2d1796f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 0x556dfd0bb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.389+0000 7f2d1796f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556dfd0cd680 con 0x556dfd0b6e40 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.389+0000 7f2d1796f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556dfd0cdd50 con 0x556dfd0b7830 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.389+0000 7f2d1796f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556dfd0ce420 con 0x556dfd0b82d0 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.389+0000 7f2d0e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0bf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2d0e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0bf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42520/0 (socket says 172.21.15.82:42520) 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2d0e59c640 1 -- 172.21.15.82:0/1761631869 learned_addr learned my addr 172.21.15.82:0/1761631869 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2d0ed9d640 1 --2- 172.21.15.82:0/1761631869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 0x556dfd0bb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2d0dd9b640 1 --2- 172.21.15.82:0/1761631869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b7830 0x556dfd0c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2d0e59c640 1 -- 172.21.15.82:0/1761631869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 msgr2=0x556dfd0bb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/1761631869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 0x556dfd0bb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2d0e59c640 1 -- 172.21.15.82:0/1761631869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b7830 msgr2=0x556dfd0c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/1761631869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b7830 0x556dfd0c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2d0e59c640 1 -- 172.21.15.82:0/1761631869 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dfd0b2d40 con 0x556dfd0b6e40 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/1761631869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0bf970 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2d0009c230 tx=0x7f2d00098350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fa67e4478c4ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2cf7fff640 1 -- 172.21.15.82:0/1761631869 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00020070 con 0x556dfd0b6e40 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2cf7fff640 1 -- 172.21.15.82:0/1761631869 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d00015020 con 0x556dfd0b6e40 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.390+0000 7f2cf7fff640 1 -- 172.21.15.82:0/1761631869 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000986e0 con 0x556dfd0b6e40 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 -- 172.21.15.82:0/1761631869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 msgr2=0x556dfd0bf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 --2- 172.21.15.82:0/1761631869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0bf970 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2d0009c230 tx=0x7f2d00098350 comp rx=0 tx=0).stop 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 -- 172.21.15.82:0/1761631869 shutdown_connections 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 --2- 172.21.15.82:0/1761631869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 0x556dfd0bb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 --2- 172.21.15.82:0/1761631869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b7830 0x556dfd0c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 --2- 172.21.15.82:0/1761631869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b6e40 0x556dfd0bf970 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 -- 172.21.15.82:0/1761631869 >> 172.21.15.82:0/1761631869 conn(0x556dfd0b2f40 msgr2=0x556dfd0bef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 -- 172.21.15.82:0/1761631869 shutdown_connections 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 -- 172.21.15.82:0/1761631869 wait complete. 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 -- start start 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 Processor -- start 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b6e40 0x556dfd0c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b7830 0x556dfd0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 0x556dfd0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556dfd0cf1c0 con 0x556dfd0b7830 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556dfd0ceaf0 con 0x556dfd0b6e40 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556dfd0cf890 con 0x556dfd0b82d0 2024-04-21T08:46:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d1796f740 10 client.0 _ll_register_callbacks cb 0x7f2d000046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:52.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d0dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b7830 0x556dfd0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d0dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b7830 0x556dfd0c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42536/0 (socket says 172.21.15.82:42536) 2024-04-21T08:46:52.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.391+0000 7f2d0dd9b640 1 -- 172.21.15.82:0/2402673238 learned_addr learned my addr 172.21.15.82:0/2402673238 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d0dd9b640 1 -- 172.21.15.82:0/2402673238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 msgr2=0x556dfd0c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d0dd9b640 1 --2- 172.21.15.82:0/2402673238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 0x556dfd0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d0dd9b640 1 -- 172.21.15.82:0/2402673238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b6e40 msgr2=0x556dfd0c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d0dd9b640 1 --2- 172.21.15.82:0/2402673238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b6e40 0x556dfd0c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d0dd9b640 1 -- 172.21.15.82:0/2402673238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dfd0c7470 con 0x556dfd0b7830 2024-04-21T08:46:52.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d0dd9b640 1 --2- 172.21.15.82:0/2402673238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b7830 0x556dfd0c6840 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x556dfd0b2410 tx=0x7f2d0400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1aee034d94ab1c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:52.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2cf5ffb640 1 -- 172.21.15.82:0/2402673238 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04014c50 con 0x556dfd0b7830 2024-04-21T08:46:52.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2cf5ffb640 1 -- 172.21.15.82:0/2402673238 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f2d04014df0 con 0x556dfd0b7830 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d1796f740 1 -- 172.21.15.82:0/2402673238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d00015020 con 0x556dfd0b7830 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2cf5ffb640 1 -- 172.21.15.82:0/2402673238 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2d04023440 con 0x556dfd0b7830 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d1796f740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d1796f740 1 -- 172.21.15.82:0/2402673238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556dfd133e80 con 0x556dfd0b7830 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2cf5ffb640 1 -- 172.21.15.82:0/2402673238 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d00015020 con 0x556dfd0b7830 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2cf5ffb640 1 -- 172.21.15.82:0/2402673238 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2d04021070 con 0x556dfd0b7830 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d1796f740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d1796f740 10 client.15504 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d1796f740 20 client.15504 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d1796f740 1 --2- 172.21.15.82:0/2402673238 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x556dfd138a20 0x556dfd137730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d1796f740 4 client.15504 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.392+0000 7f2d1796f740 1 -- 172.21.15.82:0/2402673238 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "status"}) -- 0x556dfcf9f960 con 0x556dfd138a20 2024-04-21T08:46:52.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/2402673238 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x556dfd138a20 0x556dfd137730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d0e59c640 1 --2- 172.21.15.82:0/2402673238 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x556dfd138a20 0x556dfd137730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e1fb40f8e4ead97 server_cookie=d5968d5ec7dfe60a in_seq=0 out_seq=0 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2cf5ffb640 10 client.15504 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2cf5ffb640 1 -- 172.21.15.82:0/2402673238 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556dfcf9f960 con 0x556dfd138a20 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2cf5ffb640 10 client.15504 handle_command_reply: tid=0 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d1796f740 1 client.15504 shutdown 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d1796f740 1 -- 172.21.15.82:0/2402673238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b7830 msgr2=0x556dfd0c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d1796f740 1 --2- 172.21.15.82:0/2402673238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b7830 0x556dfd0c6840 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x556dfd0b2410 tx=0x7f2d0400bec0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d1796f740 1 -- 172.21.15.82:0/2402673238 shutdown_connections 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d1796f740 1 --2- 172.21.15.82:0/2402673238 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x556dfd138a20 0x556dfd137730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d1796f740 1 --2- 172.21.15.82:0/2402673238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556dfd0b82d0 0x556dfd0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2cf5ffb640 0 client.15504 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d1796f740 1 --2- 172.21.15.82:0/2402673238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556dfd0b7830 0x556dfd0c6840 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d1796f740 1 --2- 172.21.15.82:0/2402673238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556dfd0b6e40 0x556dfd0c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d1796f740 1 -- 172.21.15.82:0/2402673238 >> 172.21.15.82:0/2402673238 conn(0x556dfd0b2f40 msgr2=0x556dfd0c0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d1796f740 1 -- 172.21.15.82:0/2402673238 shutdown_connections 2024-04-21T08:46:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.393+0000 7f2d1796f740 1 -- 172.21.15.82:0/2402673238 wait complete. 2024-04-21T08:46:52.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d1796f740 20 client.15504 trim_cache size 0 max 16384 2024-04-21T08:46:52.402 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:46:52.402 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:46:52.402 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 1, 2024-04-21T08:46:52.402 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24481, 2024-04-21T08:46:52.402 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:46:52.402 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:46:52.403 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:46:52.403 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 113.95131249000001, 2024-04-21T08:46:52.403 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2024-04-21T08:46:52.403 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 93, 2024-04-21T08:46:52.403 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:46:52.403 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 152.48597722700001 2024-04-21T08:46:52.403 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:46:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d14d7e640 1 -- 172.21.15.82:0/3899180664 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2cf80a3bb0 msgr2=0x7f2cf80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d14d7e640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2cf80a3bb0 0x7f2cf80a6070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2d08005e30 tx=0x7f2d08042000 comp rx=0 tx=0).stop 2024-04-21T08:46:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d14d7e640 1 -- 172.21.15.82:0/3899180664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d10101fa0 msgr2=0x7f2d101981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d14d7e640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d10101fa0 0x7f2d101981c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000d770 tx=0x7f2d0000dc30 comp rx=0 tx=0).stop 2024-04-21T08:46:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d14d7e640 1 -- 172.21.15.82:0/3899180664 shutdown_connections 2024-04-21T08:46:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d14d7e640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f2cf80a3bb0 0x7f2cf80a6070 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d14d7e640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d10103b50 0x7f2d1019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d14d7e640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d10103190 0x7f2d10198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d14d7e640 1 --2- 172.21.15.82:0/3899180664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d10101fa0 0x7f2d101981c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d14d7e640 1 -- 172.21.15.82:0/3899180664 >> 172.21.15.82:0/3899180664 conn(0x7f2d100fd530 msgr2=0x7f2d10106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d14d7e640 1 -- 172.21.15.82:0/3899180664 shutdown_connections 2024-04-21T08:46:52.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.394+0000 7f2d14d7e640 1 -- 172.21.15.82:0/3899180664 wait complete. 2024-04-21T08:46:52.406 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T08:46:52.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.504+0000 7fb142cc1640 1 -- 172.21.15.82:0/2340665202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13c100e90 msgr2=0x7fb13c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.504+0000 7fb142cc1640 1 --2- 172.21.15.82:0/2340665202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13c100e90 0x7fb13c1012b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb130009900 tx=0x7fb13002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 -- 172.21.15.82:0/2340665202 shutdown_connections 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 --2- 172.21.15.82:0/2340665202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb13c1029a0 0x7fb13c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 --2- 172.21.15.82:0/2340665202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb13c101fe0 0x7fb13c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 --2- 172.21.15.82:0/2340665202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13c100e90 0x7fb13c1012b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 -- 172.21.15.82:0/2340665202 >> 172.21.15.82:0/2340665202 conn(0x7fb13c0fc400 msgr2=0x7fb13c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 -- 172.21.15.82:0/2340665202 shutdown_connections 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 -- 172.21.15.82:0/2340665202 wait complete. 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 Processor -- start 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 -- start start 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13c100e90 0x7fb13c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb13c101fe0 0x7fb13c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb13c1029a0 0x7fb13c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c10ca40 con 0x7fb13c100e90 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c10c8c0 con 0x7fb13c1029a0 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb142cc1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c10cbc0 con 0x7fb13c101fe0 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb140a5d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13c100e90 0x7fb13c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb140a5d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13c100e90 0x7fb13c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42568/0 (socket says 172.21.15.82:42568) 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb140a5d640 1 -- 172.21.15.82:0/3953724782 learned_addr learned my addr 172.21.15.82:0/3953724782 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.505+0000 7fb14125e640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb13c1029a0 0x7fb13c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb140a5d640 1 -- 172.21.15.82:0/3953724782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb13c101fe0 msgr2=0x7fb13c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb13bfff640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb13c101fe0 0x7fb13c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb140a5d640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb13c101fe0 0x7fb13c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb140a5d640 1 -- 172.21.15.82:0/3953724782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb13c1029a0 msgr2=0x7fb13c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb140a5d640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb13c1029a0 0x7fb13c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb140a5d640 1 -- 172.21.15.82:0/3953724782 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c19fe80 con 0x7fb13c100e90 2024-04-21T08:46:52.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb14125e640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb13c1029a0 0x7fb13c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:52.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb140a5d640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13c100e90 0x7fb13c198260 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb130006010 tx=0x7fb1300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb139ffb640 1 -- 172.21.15.82:0/3953724782 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130043020 con 0x7fb13c100e90 2024-04-21T08:46:52.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb139ffb640 1 -- 172.21.15.82:0/3953724782 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb130004aa0 con 0x7fb13c100e90 2024-04-21T08:46:52.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb142cc1640 1 -- 172.21.15.82:0/3953724782 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13c1a0110 con 0x7fb13c100e90 2024-04-21T08:46:52.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb139ffb640 1 -- 172.21.15.82:0/3953724782 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130003c70 con 0x7fb13c100e90 2024-04-21T08:46:52.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.506+0000 7fb142cc1640 1 -- 172.21.15.82:0/3953724782 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb13c1a06e0 con 0x7fb13c100e90 2024-04-21T08:46:52.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.508+0000 7fb139ffb640 1 -- 172.21.15.82:0/3953724782 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb130046020 con 0x7fb13c100e90 2024-04-21T08:46:52.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb145994740 1 Processor -- start 2024-04-21T08:46:52.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb145994740 1 -- start start 2024-04-21T08:46:52.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb145994740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9938260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb145994740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 0x556eb9938cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb145994740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb99392d0 0x556eb993feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb145994740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556eb989cb10 con 0x556eb9938830 2024-04-21T08:46:52.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb145994740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556eb983a240 con 0x556eb99392d0 2024-04-21T08:46:52.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb145994740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556eb9898ad0 con 0x556eb9937e40 2024-04-21T08:46:52.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb139ffb640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb12c0a3bb0 0x7fb12c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb140a5d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 0x556eb9938cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb140a5d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 0x556eb9938cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42576/0 (socket says 172.21.15.82:42576) 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb140a5d640 1 -- 172.21.15.82:0/3215414532 learned_addr learned my addr 172.21.15.82:0/3215414532 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb13bfff640 1 --2- 172.21.15.82:0/3215414532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb99392d0 0x556eb993feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb14125e640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb12c0a3bb0 0x7fb12c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.509+0000 7fb14125e640 1 --2- 172.21.15.82:0/3215414532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9938260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.510+0000 7fb139ffb640 1 -- 172.21.15.82:0/3953724782 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fb1300f2c90 con 0x7fb13c100e90 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.510+0000 7fb140a5d640 1 -- 172.21.15.82:0/3215414532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 msgr2=0x556eb9938260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.510+0000 7fb140a5d640 1 --2- 172.21.15.82:0/3215414532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9938260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.510+0000 7fb140a5d640 1 -- 172.21.15.82:0/3215414532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb99392d0 msgr2=0x556eb993feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.510+0000 7fb140a5d640 1 --2- 172.21.15.82:0/3215414532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb99392d0 0x556eb993feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.510+0000 7fb140a5d640 1 -- 172.21.15.82:0/3215414532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb95af2d0 con 0x556eb9938830 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.510+0000 7fb14125e640 1 --2- 172.21.15.82:0/3215414532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9938260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.510+0000 7fb140a5d640 1 --2- 172.21.15.82:0/3215414532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 0x556eb9938cb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb1300f02c0 tx=0x7fb13002cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68d8451afa0fe26 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.510+0000 7fb14125e640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb12c0a3bb0 0x7fb12c0a6070 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x556eb9933200 tx=0x7fb134009610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.510+0000 7fb126ffd640 1 -- 172.21.15.82:0/3215414532 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300044e0 con 0x556eb9938830 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.510+0000 7fb126ffd640 1 -- 172.21.15.82:0/3215414532 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1300035f0 con 0x556eb9938830 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.510+0000 7fb126ffd640 1 -- 172.21.15.82:0/3215414532 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300f0490 con 0x556eb9938830 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 -- 172.21.15.82:0/3215414532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 msgr2=0x556eb9938cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 --2- 172.21.15.82:0/3215414532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 0x556eb9938cb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb1300f02c0 tx=0x7fb13002cdd0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 -- 172.21.15.82:0/3215414532 shutdown_connections 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 --2- 172.21.15.82:0/3215414532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb99392d0 0x556eb993feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 --2- 172.21.15.82:0/3215414532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 0x556eb9938cb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 --2- 172.21.15.82:0/3215414532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9938260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 -- 172.21.15.82:0/3215414532 >> 172.21.15.82:0/3215414532 conn(0x556eb9933f40 msgr2=0x556eb9932490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 -- 172.21.15.82:0/3215414532 shutdown_connections 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 -- 172.21.15.82:0/3215414532 wait complete. 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 -- start start 2024-04-21T08:46:52.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 Processor -- start 2024-04-21T08:46:52.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9941ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 0x556eb99420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb99392d0 0x556eb9949120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556eb9939ad0 con 0x556eb9938830 2024-04-21T08:46:52.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556eb9825190 con 0x556eb99392d0 2024-04-21T08:46:52.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.511+0000 7fb145994740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556eb99403f0 con 0x556eb9937e40 2024-04-21T08:46:52.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb145994740 10 client.0 _ll_register_callbacks cb 0x556eb98208b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:52.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb13bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 0x556eb99420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb13bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 0x556eb99420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42582/0 (socket says 172.21.15.82:42582) 2024-04-21T08:46:52.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb13bfff640 1 -- 172.21.15.82:0/280876759 learned_addr learned my addr 172.21.15.82:0/280876759 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb140a5d640 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9941ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb14125e640 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb99392d0 0x556eb9949120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb13bfff640 1 -- 172.21.15.82:0/280876759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 msgr2=0x556eb9941ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb13bfff640 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9941ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb13bfff640 1 -- 172.21.15.82:0/280876759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb99392d0 msgr2=0x556eb9949120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb13bfff640 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb99392d0 0x556eb9949120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb13bfff640 1 -- 172.21.15.82:0/280876759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb96796b0 con 0x556eb9938830 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb13bfff640 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 0x556eb99420e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb128009dc0 tx=0x7fb1280117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2c42c096f51fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb11bfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb11bfff640 1 -- 172.21.15.82:0/280876759 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128012860 con 0x556eb9938830 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb11bfff640 1 -- 172.21.15.82:0/280876759 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fb128012a00 con 0x556eb9938830 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb145994740 1 -- 172.21.15.82:0/280876759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556eb98aa530 con 0x556eb9938830 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb11bfff640 1 -- 172.21.15.82:0/280876759 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb128025050 con 0x556eb9938830 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb145994740 10 client.15516 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:52.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.512+0000 7fb145994740 1 -- 172.21.15.82:0/280876759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556eb994c560 con 0x556eb9938830 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.513+0000 7fb11bfff640 1 -- 172.21.15.82:0/280876759 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556eb98aa530 con 0x556eb9938830 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.513+0000 7fb11bfff640 1 -- 172.21.15.82:0/280876759 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb12800f040 con 0x556eb9938830 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.513+0000 7fb145994740 10 client.15516 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.513+0000 7fb145994740 10 client.15516 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.513+0000 7fb145994740 20 client.15516 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.513+0000 7fb145994740 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556eb994ec80 0x556eb99500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.513+0000 7fb145994740 4 client.15516 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.513+0000 7fb145994740 1 -- 172.21.15.82:0/280876759 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eb98a43a0 con 0x556eb994ec80 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.513+0000 7fb140a5d640 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556eb994ec80 0x556eb99500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb140a5d640 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556eb994ec80 0x556eb99500f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb3d5ac525326ce8 server_cookie=97e92234b5e769b0 in_seq=0 out_seq=0 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb11bfff640 10 client.15516 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb11bfff640 1 -- 172.21.15.82:0/280876759 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556eb98a43a0 con 0x556eb994ec80 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb11bfff640 10 client.15516 handle_command_reply: tid=0 2024-04-21T08:46:52.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb145994740 1 client.15516 shutdown 2024-04-21T08:46:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb145994740 1 -- 172.21.15.82:0/280876759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 msgr2=0x556eb99420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb145994740 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 0x556eb99420e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb128009dc0 tx=0x7fb1280117b0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb145994740 1 -- 172.21.15.82:0/280876759 shutdown_connections 2024-04-21T08:46:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb145994740 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556eb994ec80 0x556eb99500f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb145994740 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb99392d0 0x556eb9949120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb145994740 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9938830 0x556eb99420e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb145994740 1 --2- 172.21.15.82:0/280876759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9941ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb11bfff640 0 client.15516 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:46:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.514+0000 7fb145994740 1 -- 172.21.15.82:0/280876759 >> 172.21.15.82:0/280876759 conn(0x556eb9933f40 msgr2=0x556eb9931a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.515+0000 7fb145994740 1 -- 172.21.15.82:0/280876759 shutdown_connections 2024-04-21T08:46:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.515+0000 7fb145994740 1 -- 172.21.15.82:0/280876759 wait complete. 2024-04-21T08:46:52.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.515+0000 7fb145994740 20 client.15516 trim_cache size 0 max 16384 2024-04-21T08:46:52.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.532+0000 7fb145994740 1 Processor -- start 2024-04-21T08:46:52.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.532+0000 7fb145994740 1 -- start start 2024-04-21T08:46:52.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.532+0000 7fb145994740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9933f40 0x556eb9932400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.532+0000 7fb145994740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9932940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.532+0000 7fb145994740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb9938830 0x556eb99490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.532+0000 7fb145994740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556eb994e430 con 0x556eb9933f40 2024-04-21T08:46:52.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.532+0000 7fb145994740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556eb994eb00 con 0x556eb9938830 2024-04-21T08:46:52.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.532+0000 7fb145994740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556eb994f1d0 con 0x556eb9937e40 2024-04-21T08:46:52.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb140a5d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9933f40 0x556eb9932400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb140a5d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9933f40 0x556eb9932400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42584/0 (socket says 172.21.15.82:42584) 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb140a5d640 1 -- 172.21.15.82:0/3034279304 learned_addr learned my addr 172.21.15.82:0/3034279304 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb13bfff640 1 --2- 172.21.15.82:0/3034279304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9932940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb140a5d640 1 -- 172.21.15.82:0/3034279304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 msgr2=0x556eb9932940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb140a5d640 1 --2- 172.21.15.82:0/3034279304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9932940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb140a5d640 1 -- 172.21.15.82:0/3034279304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb9938830 msgr2=0x556eb99490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb140a5d640 1 --2- 172.21.15.82:0/3034279304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb9938830 0x556eb99490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb140a5d640 1 -- 172.21.15.82:0/3034279304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb9942d20 con 0x556eb9933f40 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb13bfff640 1 --2- 172.21.15.82:0/3034279304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9932940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb140a5d640 1 --2- 172.21.15.82:0/3034279304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9933f40 0x556eb9932400 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb130000c00 tx=0x7fb1300bea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7b3fed6d57b414 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb1267fc640 1 -- 172.21.15.82:0/3034279304 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300bf260 con 0x556eb9933f40 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb1267fc640 1 -- 172.21.15.82:0/3034279304 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb130003e10 con 0x556eb9933f40 2024-04-21T08:46:52.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.533+0000 7fb1267fc640 1 -- 172.21.15.82:0/3034279304 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1300df0f0 con 0x556eb9933f40 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 -- 172.21.15.82:0/3034279304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9933f40 msgr2=0x556eb9932400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 --2- 172.21.15.82:0/3034279304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9933f40 0x556eb9932400 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb130000c00 tx=0x7fb1300bea60 comp rx=0 tx=0).stop 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 -- 172.21.15.82:0/3034279304 shutdown_connections 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 --2- 172.21.15.82:0/3034279304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb9938830 0x556eb99490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 --2- 172.21.15.82:0/3034279304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9937e40 0x556eb9932940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 --2- 172.21.15.82:0/3034279304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9933f40 0x556eb9932400 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 -- 172.21.15.82:0/3034279304 >> 172.21.15.82:0/3034279304 conn(0x556eb9940a50 msgr2=0x556eb993c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 -- 172.21.15.82:0/3034279304 shutdown_connections 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 -- 172.21.15.82:0/3034279304 wait complete. 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 -- start start 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 Processor -- start 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb9933f40 0x556eb993e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9937e40 0x556eb993ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9938830 0x556eb99b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556eb994ff70 con 0x556eb9937e40 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556eb994f8a0 con 0x556eb9933f40 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556eb9950640 con 0x556eb9938830 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb145994740 10 client.0 _ll_register_callbacks cb 0x7fb128009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb13bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9937e40 0x556eb993ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb13bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9937e40 0x556eb993ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42600/0 (socket says 172.21.15.82:42600) 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.534+0000 7fb13bfff640 1 -- 172.21.15.82:0/3853791134 learned_addr learned my addr 172.21.15.82:0/3853791134 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb140a5d640 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb9933f40 0x556eb993e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb14125e640 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9938830 0x556eb99b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb13bfff640 1 -- 172.21.15.82:0/3853791134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9938830 msgr2=0x556eb99b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb13bfff640 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9938830 0x556eb99b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb13bfff640 1 -- 172.21.15.82:0/3853791134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb9933f40 msgr2=0x556eb993e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb13bfff640 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb9933f40 0x556eb993e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb13bfff640 1 -- 172.21.15.82:0/3853791134 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb99b3160 con 0x556eb9937e40 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb14125e640 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9938830 0x556eb99b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb13bfff640 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9937e40 0x556eb993ec20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb12800a010 tx=0x7fb128009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3a70625c6a43e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb11bfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb11bfff640 1 -- 172.21.15.82:0/3853791134 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128005240 con 0x556eb9937e40 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb11bfff640 1 -- 172.21.15.82:0/3853791134 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fb1280053e0 con 0x556eb9937e40 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb145994740 1 -- 172.21.15.82:0/3853791134 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb130003e10 con 0x556eb9937e40 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb11bfff640 1 -- 172.21.15.82:0/3853791134 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb128039050 con 0x556eb9937e40 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb145994740 10 client.15522 fetch_fsmap learned FSMap version 27 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb145994740 1 -- 172.21.15.82:0/3853791134 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556eb99b43a0 con 0x556eb9937e40 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb11bfff640 1 -- 172.21.15.82:0/3853791134 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb130003e10 con 0x556eb9937e40 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.535+0000 7fb11bfff640 1 -- 172.21.15.82:0/3853791134 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb128012a20 con 0x556eb9937e40 2024-04-21T08:46:52.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 10 client.15522 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 20 client.15522 populate_metadata read hostname 'smithi082' 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556eb99392d0 0x556eb99b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 4 client.15522 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 1 -- 172.21.15.82:0/3853791134 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "status"}) -- 0x556eb9820960 con 0x556eb99392d0 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb140a5d640 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556eb99392d0 0x556eb99b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb11bfff640 10 client.15522 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb140a5d640 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556eb99392d0 0x556eb99b7c90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c887335da6610fb5 server_cookie=c84e0139d151be in_seq=0 out_seq=0 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb11bfff640 1 -- 172.21.15.82:0/3853791134 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556eb9820960 con 0x556eb99392d0 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb11bfff640 10 client.15522 handle_command_reply: tid=0 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 1 client.15522 shutdown 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 1 -- 172.21.15.82:0/3853791134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9937e40 msgr2=0x556eb993ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9937e40 0x556eb993ec20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb12800a010 tx=0x7fb128009980 comp rx=0 tx=0).stop 2024-04-21T08:46:52.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 1 -- 172.21.15.82:0/3853791134 shutdown_connections 2024-04-21T08:46:52.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x556eb99392d0 0x556eb99b7c90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556eb9938830 0x556eb99b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556eb9937e40 0x556eb993ec20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb11bfff640 0 client.15522 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:46:52.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.536+0000 7fb145994740 1 --2- 172.21.15.82:0/3853791134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556eb9933f40 0x556eb993e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb145994740 1 -- 172.21.15.82:0/3853791134 >> 172.21.15.82:0/3853791134 conn(0x556eb9940a50 msgr2=0x556eb9931170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb145994740 1 -- 172.21.15.82:0/3853791134 shutdown_connections 2024-04-21T08:46:52.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb145994740 1 -- 172.21.15.82:0/3853791134 wait complete. 2024-04-21T08:46:52.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb145994740 20 client.15522 trim_cache size 0 max 16384 2024-04-21T08:46:52.546 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:46:52.546 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:46:52.546 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-21T08:46:52.546 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24509, 2024-04-21T08:46:52.546 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:46:52.546 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:46:52.546 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:46:52.546 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 138.055425386, 2024-04-21T08:46:52.546 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 27, 2024-04-21T08:46:52.546 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 93, 2024-04-21T08:46:52.546 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:46:52.546 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 144.12297201600001 2024-04-21T08:46:52.547 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:46:52.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb142cc1640 1 -- 172.21.15.82:0/3953724782 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb12c0a3bb0 msgr2=0x7fb12c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb142cc1640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb12c0a3bb0 0x7fb12c0a6070 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x556eb9933200 tx=0x7fb134009610 comp rx=0 tx=0).stop 2024-04-21T08:46:52.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb142cc1640 1 -- 172.21.15.82:0/3953724782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13c100e90 msgr2=0x7fb13c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb142cc1640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13c100e90 0x7fb13c198260 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb130006010 tx=0x7fb1300042f0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb142cc1640 1 -- 172.21.15.82:0/3953724782 shutdown_connections 2024-04-21T08:46:52.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb142cc1640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fb12c0a3bb0 0x7fb12c0a6070 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb142cc1640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb13c1029a0 0x7fb13c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb142cc1640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb13c101fe0 0x7fb13c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb142cc1640 1 --2- 172.21.15.82:0/3953724782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13c100e90 0x7fb13c198260 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb142cc1640 1 -- 172.21.15.82:0/3953724782 >> 172.21.15.82:0/3953724782 conn(0x7fb13c0fc400 msgr2=0x7fb13c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb142cc1640 1 -- 172.21.15.82:0/3953724782 shutdown_connections 2024-04-21T08:46:52.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.537+0000 7fb142cc1640 1 -- 172.21.15.82:0/3953724782 wait complete. 2024-04-21T08:46:52.550 INFO:teuthology.run_tasks:Running task sequential... 2024-04-21T08:46:52.560 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-21T08:46:52.560 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-21T08:46:52.560 INFO:teuthology.task.print:Enabling mgr modules 2024-04-21T08:46:52.560 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-21T08:46:52.561 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T08:46:52.561 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-04-21T08:46:52.561 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-21T08:46:52.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.660+0000 7f7026632640 1 -- 172.21.15.82:0/2492578419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70201031a0 msgr2=0x7f7020103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.660+0000 7f7026632640 1 --2- 172.21.15.82:0/2492578419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70201031a0 0x7f7020103620 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7014009900 tx=0x7f701402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:52.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.660+0000 7f7026632640 1 -- 172.21.15.82:0/2492578419 shutdown_connections 2024-04-21T08:46:52.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.660+0000 7f7026632640 1 --2- 172.21.15.82:0/2492578419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7020103b60 0x7f702010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.660+0000 7f7026632640 1 --2- 172.21.15.82:0/2492578419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70201031a0 0x7f7020103620 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.660+0000 7f7026632640 1 --2- 172.21.15.82:0/2492578419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7020101fb0 0x7f70201023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.660+0000 7f7026632640 1 -- 172.21.15.82:0/2492578419 >> 172.21.15.82:0/2492578419 conn(0x7f70200fd540 msgr2=0x7f70200ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:52.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.660+0000 7f7026632640 1 -- 172.21.15.82:0/2492578419 shutdown_connections 2024-04-21T08:46:52.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.660+0000 7f7026632640 1 -- 172.21.15.82:0/2492578419 wait complete. 2024-04-21T08:46:52.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.660+0000 7f7026632640 1 Processor -- start 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.660+0000 7f7026632640 1 -- start start 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f7026632640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7020101fb0 0x7f7020198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f7026632640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70201031a0 0x7f70201987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f7026632640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7020103b60 0x7f702019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f7026632640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f702010dbd0 con 0x7f70201031a0 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f7026632640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f702010da50 con 0x7f7020103b60 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f7026632640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f702010dd50 con 0x7f7020101fb0 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70201031a0 0x7f70201987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70201031a0 0x7f70201987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34720/0 (socket says 172.21.15.82:34720) 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701f7fe640 1 -- 172.21.15.82:0/253543879 learned_addr learned my addr 172.21.15.82:0/253543879 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f7024bcf640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7020103b60 0x7f702019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701ffff640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7020101fb0 0x7f7020198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701f7fe640 1 -- 172.21.15.82:0/253543879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7020101fb0 msgr2=0x7f7020198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701f7fe640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7020101fb0 0x7f7020198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701f7fe640 1 -- 172.21.15.82:0/253543879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7020103b60 msgr2=0x7f702019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:52.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701f7fe640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7020103b60 0x7f702019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:52.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701f7fe640 1 -- 172.21.15.82:0/253543879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702019feb0 con 0x7f70201031a0 2024-04-21T08:46:52.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701ffff640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7020101fb0 0x7f7020198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:52.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701f7fe640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70201031a0 0x7f70201987d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7014036040 tx=0x7f70140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701d7fa640 1 -- 172.21.15.82:0/253543879 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014043020 con 0x7f70201031a0 2024-04-21T08:46:52.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701d7fa640 1 -- 172.21.15.82:0/253543879 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70140054c0 con 0x7f70201031a0 2024-04-21T08:46:52.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f7026632640 1 -- 172.21.15.82:0/253543879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70201a0140 con 0x7f70201031a0 2024-04-21T08:46:52.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f701d7fa640 1 -- 172.21.15.82:0/253543879 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014003db0 con 0x7f70201031a0 2024-04-21T08:46:52.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.661+0000 7f7026632640 1 -- 172.21.15.82:0/253543879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70201a0710 con 0x7f70201031a0 2024-04-21T08:46:52.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.662+0000 7f7026632640 1 -- 172.21.15.82:0/253543879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70201092d0 con 0x7f70201031a0 2024-04-21T08:46:52.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.663+0000 7f701d7fa640 1 -- 172.21.15.82:0/253543879 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7014046020 con 0x7f70201031a0 2024-04-21T08:46:52.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.664+0000 7f701d7fa640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f70080a3c50 0x7f70080a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:52.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.664+0000 7f701ffff640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f70080a3c50 0x7f70080a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:52.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.664+0000 7f701d7fa640 1 -- 172.21.15.82:0/253543879 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f70140f2d70 con 0x7f70201031a0 2024-04-21T08:46:52.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.664+0000 7f701ffff640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f70080a3c50 0x7f70080a6110 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7010006fd0 tx=0x7f7010008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:52.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.669+0000 7f701d7fa640 1 -- 172.21.15.82:0/253543879 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f70140f7070 con 0x7f70201031a0 2024-04-21T08:46:52.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:52 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1035098105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:52.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:52 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4046505063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:52.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:52 smithi082 ceph-mon[27967]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:52.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:52 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1035098105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:52.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:52 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/4046505063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:52.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:52 smithi106 ceph-mon[31072]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:52.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:52.894+0000 7f7026632640 1 -- 172.21.15.82:0/253543879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f7020062b50 con 0x7f70201031a0 2024-04-21T08:46:52.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:52 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1035098105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:52.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:52 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/4046505063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:46:52.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:52 smithi134 ceph-mon[31772]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:46:53.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.505+0000 7f701d7fa640 1 -- 172.21.15.82:0/253543879 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f701404d020 con 0x7f70201031a0 2024-04-21T08:46:53.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.509+0000 7f7026632640 1 -- 172.21.15.82:0/253543879 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f70080a3c50 msgr2=0x7f70080a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:53.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.509+0000 7f7026632640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f70080a3c50 0x7f70080a6110 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7010006fd0 tx=0x7f7010008040 comp rx=0 tx=0).stop 2024-04-21T08:46:53.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.509+0000 7f7026632640 1 -- 172.21.15.82:0/253543879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70201031a0 msgr2=0x7f70201987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:53.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.509+0000 7f7026632640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70201031a0 0x7f70201987d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7014036040 tx=0x7f70140042f0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.509+0000 7f7026632640 1 -- 172.21.15.82:0/253543879 shutdown_connections 2024-04-21T08:46:53.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.509+0000 7f7026632640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f70080a3c50 0x7f70080a6110 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.509+0000 7f7026632640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7020103b60 0x7f702019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.509+0000 7f7026632640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70201031a0 0x7f70201987d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.509+0000 7f7026632640 1 --2- 172.21.15.82:0/253543879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7020101fb0 0x7f7020198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.509+0000 7f7026632640 1 -- 172.21.15.82:0/253543879 >> 172.21.15.82:0/253543879 conn(0x7f70200fd540 msgr2=0x7f70201064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:53.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.509+0000 7f7026632640 1 -- 172.21.15.82:0/253543879 shutdown_connections 2024-04-21T08:46:53.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.511+0000 7f7026632640 1 -- 172.21.15.82:0/253543879 wait complete. 2024-04-21T08:46:53.527 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-21T08:46:53.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.633+0000 7fe014a81640 1 -- 172.21.15.82:0/142810759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010102000 msgr2=0x7fe010102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.633+0000 7fe014a81640 1 --2- 172.21.15.82:0/142810759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010102000 0x7fe010102420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe004009900 tx=0x7fe00402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.633+0000 7fe014a81640 1 -- 172.21.15.82:0/142810759 shutdown_connections 2024-04-21T08:46:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.633+0000 7fe014a81640 1 --2- 172.21.15.82:0/142810759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe010103bb0 0x7fe01010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.633+0000 7fe014a81640 1 --2- 172.21.15.82:0/142810759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0101031f0 0x7fe010103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.633+0000 7fe014a81640 1 --2- 172.21.15.82:0/142810759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010102000 0x7fe010102420 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.633+0000 7fe014a81640 1 -- 172.21.15.82:0/142810759 >> 172.21.15.82:0/142810759 conn(0x7fe0100fd570 msgr2=0x7fe0100ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.633+0000 7fe014a81640 1 -- 172.21.15.82:0/142810759 shutdown_connections 2024-04-21T08:46:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.633+0000 7fe014a81640 1 -- 172.21.15.82:0/142810759 wait complete. 2024-04-21T08:46:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe014a81640 1 Processor -- start 2024-04-21T08:46:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe014a81640 1 -- start start 2024-04-21T08:46:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe014a81640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe010102000 0x7fe01019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:53.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe014a81640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0101031f0 0x7fe01019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe014a81640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010103bb0 0x7fe0101a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe014a81640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe01010db40 con 0x7fe010103bb0 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe014a81640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe01010d9c0 con 0x7fe0101031f0 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe014a81640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe01010dcc0 con 0x7fe010102000 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe00ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010103bb0 0x7fe0101a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe00ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010103bb0 0x7fe0101a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34758/0 (socket says 172.21.15.82:34758) 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe00ed9d640 1 -- 172.21.15.82:0/23749064 learned_addr learned my addr 172.21.15.82:0/23749064 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe00dd9b640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0101031f0 0x7fe01019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe00e59c640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe010102000 0x7fe01019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe00ed9d640 1 -- 172.21.15.82:0/23749064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe010102000 msgr2=0x7fe01019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe00ed9d640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe010102000 0x7fe01019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe00ed9d640 1 -- 172.21.15.82:0/23749064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0101031f0 msgr2=0x7fe01019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe00ed9d640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0101031f0 0x7fe01019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe00ed9d640 1 -- 172.21.15.82:0/23749064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0101a1fa0 con 0x7fe010103bb0 2024-04-21T08:46:53.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.634+0000 7fe00ed9d640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010103bb0 0x7fe0101a1930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe00800eaa0 tx=0x7fe00800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:53.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.635+0000 7fdfff7fe640 1 -- 172.21.15.82:0/23749064 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00800cdb0 con 0x7fe010103bb0 2024-04-21T08:46:53.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.635+0000 7fe014a81640 1 -- 172.21.15.82:0/23749064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0101a2290 con 0x7fe010103bb0 2024-04-21T08:46:53.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.635+0000 7fdfff7fe640 1 -- 172.21.15.82:0/23749064 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe008005020 con 0x7fe010103bb0 2024-04-21T08:46:53.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.635+0000 7fdfff7fe640 1 -- 172.21.15.82:0/23749064 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0080116d0 con 0x7fe010103bb0 2024-04-21T08:46:53.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.635+0000 7fe014a81640 1 -- 172.21.15.82:0/23749064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0101a27e0 con 0x7fe010103bb0 2024-04-21T08:46:53.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.636+0000 7fe014a81640 1 -- 172.21.15.82:0/23749064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0101a2aa0 con 0x7fe010103bb0 2024-04-21T08:46:53.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.637+0000 7fdfff7fe640 1 -- 172.21.15.82:0/23749064 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fe0080051d0 con 0x7fe010103bb0 2024-04-21T08:46:53.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.637+0000 7fdfff7fe640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdff80a3ca0 0x7fdff80a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:53.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.638+0000 7fe00e59c640 1 -- 172.21.15.82:0/23749064 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdff80a3ca0 msgr2=0x7fdff80a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2138749385 2024-04-21T08:46:53.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.638+0000 7fdfff7fe640 1 -- 172.21.15.82:0/23749064 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fe0080d0910 con 0x7fe010103bb0 2024-04-21T08:46:53.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.638+0000 7fe00e59c640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdff80a3ca0 0x7fdff80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:46:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.642+0000 7fdfff7fe640 1 -- 172.21.15.82:0/23749064 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe008016030 con 0x7fe010103bb0 2024-04-21T08:46:53.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.839+0000 7fe00e59c640 1 -- 172.21.15.82:0/23749064 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdff80a3ca0 msgr2=0x7fdff80a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2138749385 2024-04-21T08:46:53.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.839+0000 7fe00e59c640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdff80a3ca0 0x7fdff80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:46:53.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.847+0000 7fe014a81640 1 -- 172.21.15.82:0/23749064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fe0101a2c70 con 0x7fe010103bb0 2024-04-21T08:46:53.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.850+0000 7fdfff7fe640 1 -- 172.21.15.82:0/23749064 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v25)=0 v25) ==== 138+0+0 (secure 0 0 0) 0x7fe00809c8d0 con 0x7fe010103bb0 2024-04-21T08:46:53.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.854+0000 7fe014a81640 1 -- 172.21.15.82:0/23749064 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdff80a3ca0 msgr2=0x7fdff80a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:46:53.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.854+0000 7fe014a81640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdff80a3ca0 0x7fdff80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.854+0000 7fe014a81640 1 -- 172.21.15.82:0/23749064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010103bb0 msgr2=0x7fe0101a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:53.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.854+0000 7fe014a81640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010103bb0 0x7fe0101a1930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe00800eaa0 tx=0x7fe00800ef60 comp rx=0 tx=0).stop 2024-04-21T08:46:53.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.854+0000 7fe014a81640 1 -- 172.21.15.82:0/23749064 shutdown_connections 2024-04-21T08:46:53.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.854+0000 7fe014a81640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdff80a3ca0 0x7fdff80a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.854+0000 7fe014a81640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010103bb0 0x7fe0101a1930 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.854+0000 7fe014a81640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0101031f0 0x7fe01019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.854+0000 7fe014a81640 1 --2- 172.21.15.82:0/23749064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe010102000 0x7fe01019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.854+0000 7fe014a81640 1 -- 172.21.15.82:0/23749064 >> 172.21.15.82:0/23749064 conn(0x7fe0100fd570 msgr2=0x7fe010074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:53.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.854+0000 7fe014a81640 1 -- 172.21.15.82:0/23749064 shutdown_connections 2024-04-21T08:46:53.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.854+0000 7fe014a81640 1 -- 172.21.15.82:0/23749064 wait complete. 2024-04-21T08:46:53.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:53 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/253543879' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T08:46:53.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: ignoring --setuser ceph since I am not root 2024-04-21T08:46:53.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: ignoring --setgroup ceph since I am not root 2024-04-21T08:46:53.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:53.709+0000 7fe17b99f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:46:53.869 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-21T08:46:53.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:53 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/253543879' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T08:46:53.872 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:53 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: ignoring --setuser ceph since I am not root 2024-04-21T08:46:53.872 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:53 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: ignoring --setgroup ceph since I am not root 2024-04-21T08:46:53.872 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:53 smithi106 ceph-mgr[32451]: -- 172.21.15.106:0/2600350149 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c411dce5a0 con 0x55c411da3000 2024-04-21T08:46:53.872 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:53 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:53.685+0000 7f479326f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:46:53.872 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:53 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:53.871+0000 7f479326f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:46:53.873 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:53 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/253543879' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T08:46:53.873 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:53 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: ignoring --setuser ceph since I am not root 2024-04-21T08:46:53.873 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:53 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: ignoring --setgroup ceph since I am not root 2024-04-21T08:46:53.873 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:53 smithi134 ceph-mgr[33633]: -- 172.21.15.134:0/1239001910 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557d13a525a0 con 0x557d13a27000 2024-04-21T08:46:53.873 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:53 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:53.688+0000 7ff6318ca200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:46:53.873 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:53 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:53.873+0000 7ff6318ca200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:46:53.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.970+0000 7f029ce43640 1 -- 172.21.15.82:0/2810220083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0298102a00 msgr2=0x7f0298109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:53.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.970+0000 7f029ce43640 1 --2- 172.21.15.82:0/2810220083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0298102a00 0x7f0298109320 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0290009ca0 tx=0x7f029002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.970+0000 7f029ce43640 1 -- 172.21.15.82:0/2810220083 shutdown_connections 2024-04-21T08:46:53.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.970+0000 7f029ce43640 1 --2- 172.21.15.82:0/2810220083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0298102a00 0x7f0298109320 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.970+0000 7f029ce43640 1 --2- 172.21.15.82:0/2810220083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0298102040 0x7f02981024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.970+0000 7f029ce43640 1 --2- 172.21.15.82:0/2810220083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0298100e60 0x7f0298101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.970+0000 7f029ce43640 1 -- 172.21.15.82:0/2810220083 >> 172.21.15.82:0/2810220083 conn(0x7f02980fc420 msgr2=0x7f02980fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.970+0000 7f029ce43640 1 -- 172.21.15.82:0/2810220083 shutdown_connections 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.970+0000 7f029ce43640 1 -- 172.21.15.82:0/2810220083 wait complete. 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f029ce43640 1 Processor -- start 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f029ce43640 1 -- start start 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f029ce43640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0298100e60 0x7f0298196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f029ce43640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0298102040 0x7f0298197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f029ce43640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0298102a00 0x7f029819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f029ce43640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f029810c9f0 con 0x7f0298102040 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f029ce43640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f029810c870 con 0x7f0298102a00 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f029ce43640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f029810cb70 con 0x7f0298100e60 2024-04-21T08:46:53.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f0295d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0298102040 0x7f0298197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:53.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f0295d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0298102040 0x7f0298197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34772/0 (socket says 172.21.15.82:34772) 2024-04-21T08:46:53.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f0295d9b640 1 -- 172.21.15.82:0/3742972461 learned_addr learned my addr 172.21.15.82:0/3742972461 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:53.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f0296d9d640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0298102a00 0x7f029819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:53.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.971+0000 7f029659c640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0298100e60 0x7f0298196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:53.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.972+0000 7f0295d9b640 1 -- 172.21.15.82:0/3742972461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0298100e60 msgr2=0x7f0298196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:53.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.972+0000 7f0295d9b640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0298100e60 0x7f0298196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.972+0000 7f0295d9b640 1 -- 172.21.15.82:0/3742972461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0298102a00 msgr2=0x7f029819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:53.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.972+0000 7f0295d9b640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0298102a00 0x7f029819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:53.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.972+0000 7f0295d9b640 1 -- 172.21.15.82:0/3742972461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029819ec00 con 0x7f0298102040 2024-04-21T08:46:53.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.972+0000 7f0295d9b640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0298102040 0x7f0298197520 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f028c00d950 tx=0x7f028c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:53.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.972+0000 7f02877fe640 1 -- 172.21.15.82:0/3742972461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c017070 con 0x7f0298102040 2024-04-21T08:46:53.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.972+0000 7f029ce43640 1 -- 172.21.15.82:0/3742972461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029819eef0 con 0x7f0298102040 2024-04-21T08:46:53.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.972+0000 7f02877fe640 1 -- 172.21.15.82:0/3742972461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f028c009c30 con 0x7f0298102040 2024-04-21T08:46:53.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.972+0000 7f02877fe640 1 -- 172.21.15.82:0/3742972461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c00f620 con 0x7f0298102040 2024-04-21T08:46:53.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.972+0000 7f029ce43640 1 -- 172.21.15.82:0/3742972461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f029819f400 con 0x7f0298102040 2024-04-21T08:46:53.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.973+0000 7f029ce43640 1 -- 172.21.15.82:0/3742972461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0298108200 con 0x7f0298102040 2024-04-21T08:46:53.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.974+0000 7f02877fe640 1 -- 172.21.15.82:0/3742972461 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f028c005030 con 0x7f0298102040 2024-04-21T08:46:53.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.975+0000 7f02877fe640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f02800a3ca0 0x7f02800a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:53.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.975+0000 7f029659c640 1 -- 172.21.15.82:0/3742972461 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f02800a3ca0 msgr2=0x7f02800a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2138749385 2024-04-21T08:46:53.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.975+0000 7f02877fe640 1 -- 172.21.15.82:0/3742972461 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f028c0d0380 con 0x7f0298102040 2024-04-21T08:46:53.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.975+0000 7f029659c640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f02800a3ca0 0x7f02800a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:46:53.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:53.979+0000 7f02877fe640 1 -- 172.21.15.82:0/3742972461 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f028c009de0 con 0x7f0298102040 2024-04-21T08:46:54.128 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:53 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:53.975+0000 7f479326f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:46:54.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.163+0000 7f029ce43640 1 -- 172.21.15.82:0/3742972461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f029802d660 con 0x7f0298102040 2024-04-21T08:46:54.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.167+0000 7f02877fe640 1 -- 172.21.15.82:0/3742972461 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v26)=0 v26) ==== 133+0+0 (secure 0 0 0) 0x7f028c09c340 con 0x7f0298102040 2024-04-21T08:46:54.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.169+0000 7f029ce43640 1 -- 172.21.15.82:0/3742972461 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f02800a3ca0 msgr2=0x7f02800a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:46:54.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.169+0000 7f029ce43640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f02800a3ca0 0x7f02800a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:54.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.169+0000 7f029ce43640 1 -- 172.21.15.82:0/3742972461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0298102040 msgr2=0x7f0298197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:54.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.169+0000 7f029ce43640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0298102040 0x7f0298197520 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f028c00d950 tx=0x7f028c00de10 comp rx=0 tx=0).stop 2024-04-21T08:46:54.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.169+0000 7f029ce43640 1 -- 172.21.15.82:0/3742972461 shutdown_connections 2024-04-21T08:46:54.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.169+0000 7f029ce43640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f02800a3ca0 0x7f02800a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:54.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.169+0000 7f029ce43640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0298102a00 0x7f029819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:54.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.169+0000 7f029ce43640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0298102040 0x7f0298197520 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:54.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.169+0000 7f029ce43640 1 --2- 172.21.15.82:0/3742972461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0298100e60 0x7f0298196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:54.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.169+0000 7f029ce43640 1 -- 172.21.15.82:0/3742972461 >> 172.21.15.82:0/3742972461 conn(0x7f02980fc420 msgr2=0x7f02981053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:54.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.169+0000 7f029ce43640 1 -- 172.21.15.82:0/3742972461 shutdown_connections 2024-04-21T08:46:54.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:54.169+0000 7f029ce43640 1 -- 172.21.15.82:0/3742972461 wait complete. 2024-04-21T08:46:54.179 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:53.905+0000 7fe17b99f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:46:54.179 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:54 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:54.015+0000 7fe17b99f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:46:54.180 INFO:teuthology.run_tasks:Running task kclient... 2024-04-21T08:46:54.194 INFO:tasks.kclient:Mounting kernel clients... 2024-04-21T08:46:54.194 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-21T08:46:54.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:54.194 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-21T08:46:54.209 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-21T08:46:54.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:54.210 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:46:54.226 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:53 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:53.976+0000 7ff6318ca200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:46:54.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:54.274 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-04-21T08:46:54.398 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-21T08:46:54.398 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-21T08:46:54.398 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-21T08:46:54.399 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph fs ls 2024-04-21T08:46:54.430 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:54 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:54.339+0000 7fe17b99f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:46:54.430 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:54 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:54.430+0000 7fe17b99f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:46:54.628 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:54 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:54.286+0000 7f479326f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:46:54.628 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:54 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:54.371+0000 7f479326f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:46:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:54 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/253543879' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T08:46:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:54 smithi106 ceph-mon[31072]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-21T08:46:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:54 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/23749064' entity='client.admin' 2024-04-21T08:46:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:54 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3742972461' entity='client.admin' 2024-04-21T08:46:54.666 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:46:54.726 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:54 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:54.289+0000 7ff6318ca200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:46:54.726 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:54 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:54.373+0000 7ff6318ca200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:46:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:54 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/253543879' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T08:46:54.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:54 smithi134 ceph-mon[31772]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-21T08:46:54.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:54 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/23749064' entity='client.admin' 2024-04-21T08:46:54.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:54 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3742972461' entity='client.admin' 2024-04-21T08:46:54.789 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:54 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/253543879' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T08:46:54.789 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:54 smithi082 ceph-mon[27967]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-21T08:46:54.789 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:54 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/23749064' entity='client.admin' 2024-04-21T08:46:54.789 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:54 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3742972461' entity='client.admin' 2024-04-21T08:46:55.818 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:55.644+0000 7fe17b99f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:46:55.878 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:55 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:55.550+0000 7f479326f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:46:55.878 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:55 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:55.719+0000 7f479326f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:46:55.901 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:55 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:55.560+0000 7ff6318ca200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:46:55.901 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:55 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:55.730+0000 7ff6318ca200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:46:56.076 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:55.818+0000 7fe17b99f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:46:56.077 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:55.995+0000 7fe17b99f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:46:56.178 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:55 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:55.890+0000 7f479326f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:46:56.189 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:55 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:55.902+0000 7ff6318ca200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:46:56.337 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:56 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:56.293+0000 7fe17b99f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:46:56.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.441+0000 7fdfe8549700 1 -- 172.21.15.82:0/3116691247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfe0108a20 msgr2=0x7fdfe0108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.441+0000 7fdfe8549700 1 --2- 172.21.15.82:0/3116691247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfe0108a20 0x7fdfe0108e00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009990 tx=0x7fdfdc009ca0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.441+0000 7fdfe8549700 1 -- 172.21.15.82:0/3116691247 shutdown_connections 2024-04-21T08:46:56.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.441+0000 7fdfe8549700 1 --2- 172.21.15.82:0/3116691247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe006b980 0x7fdfe010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.441+0000 7fdfe8549700 1 --2- 172.21.15.82:0/3116691247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfe0108a20 0x7fdfe0108e00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.441+0000 7fdfe8549700 1 --2- 172.21.15.82:0/3116691247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe006afe0 0x7fdfe006b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.441+0000 7fdfe8549700 1 -- 172.21.15.82:0/3116691247 >> 172.21.15.82:0/3116691247 conn(0x7fdfe00788b0 msgr2=0x7fdfe0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:56.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.441+0000 7fdfe8549700 1 -- 172.21.15.82:0/3116691247 shutdown_connections 2024-04-21T08:46:56.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.441+0000 7fdfe8549700 1 -- 172.21.15.82:0/3116691247 wait complete. 2024-04-21T08:46:56.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe8549700 1 Processor -- start 2024-04-21T08:46:56.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe8549700 1 -- start start 2024-04-21T08:46:56.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe8549700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe006afe0 0x7fdfe01990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe8549700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfe006b980 0x7fdfe0199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe8549700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe0108a20 0x7fdfe019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe8549700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0112ba0 con 0x7fdfe006b980 2024-04-21T08:46:56.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe8549700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0112a20 con 0x7fdfe0108a20 2024-04-21T08:46:56.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe8549700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0112d20 con 0x7fdfe006afe0 2024-04-21T08:46:56.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe5ae4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfe006b980 0x7fdfe0199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe62e5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe006afe0 0x7fdfe01990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe62e5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe006afe0 0x7fdfe01990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52070/0 (socket says 172.21.15.82:52070) 2024-04-21T08:46:56.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe62e5700 1 -- 172.21.15.82:0/3446837793 learned_addr learned my addr 172.21.15.82:0/3446837793 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe6ae6700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe0108a20 0x7fdfe019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe5ae4700 1 -- 172.21.15.82:0/3446837793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe006afe0 msgr2=0x7fdfe01990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe5ae4700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe006afe0 0x7fdfe01990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe5ae4700 1 -- 172.21.15.82:0/3446837793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe0108a20 msgr2=0x7fdfe019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe5ae4700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe0108a20 0x7fdfe019d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.442+0000 7fdfe5ae4700 1 -- 172.21.15.82:0/3446837793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe019e0e0 con 0x7fdfe006b980 2024-04-21T08:46:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.443+0000 7fdfe5ae4700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfe006b980 0x7fdfe0199610 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000d950 tx=0x7fdfd000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.443+0000 7fdfd77fe700 1 -- 172.21.15.82:0/3446837793 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0016070 con 0x7fdfe006b980 2024-04-21T08:46:56.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.443+0000 7fdfe8549700 1 -- 172.21.15.82:0/3446837793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe019e3d0 con 0x7fdfe006b980 2024-04-21T08:46:56.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.443+0000 7fdfd77fe700 1 -- 172.21.15.82:0/3446837793 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd000fd40 con 0x7fdfe006b980 2024-04-21T08:46:56.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.443+0000 7fdfd77fe700 1 -- 172.21.15.82:0/3446837793 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd000e3b0 con 0x7fdfe006b980 2024-04-21T08:46:56.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.443+0000 7fdfe8549700 1 -- 172.21.15.82:0/3446837793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe01a5c50 con 0x7fdfe006b980 2024-04-21T08:46:56.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.444+0000 7fdfe8549700 1 -- 172.21.15.82:0/3446837793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe0069730 con 0x7fdfe006b980 2024-04-21T08:46:56.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.445+0000 7fdfd77fe700 1 -- 172.21.15.82:0/3446837793 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fdfd0019070 con 0x7fdfe006b980 2024-04-21T08:46:56.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.446+0000 7fdfd77fe700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfcc0a3c90 0x7fdfcc0a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:56.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.446+0000 7fdfe62e5700 1 -- 172.21.15.82:0/3446837793 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfcc0a3c90 msgr2=0x7fdfcc0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2138749385 2024-04-21T08:46:56.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.446+0000 7fdfd77fe700 1 -- 172.21.15.82:0/3446837793 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fdfd00d0760 con 0x7fdfe006b980 2024-04-21T08:46:56.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.446+0000 7fdfe62e5700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfcc0a3c90 0x7fdfcc0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:46:56.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.450+0000 7fdfd77fe700 1 -- 172.21.15.82:0/3446837793 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfd009cd70 con 0x7fdfe006b980 2024-04-21T08:46:56.470 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:56 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:56.178+0000 7f479326f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:46:56.471 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:56 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:56.270+0000 7f479326f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:46:56.475 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:56 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:56.190+0000 7ff6318ca200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:46:56.476 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:56 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:56.283+0000 7ff6318ca200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:46:56.599 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:56 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:56.389+0000 7fe17b99f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:46:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.647+0000 7fdfe62e5700 1 -- 172.21.15.82:0/3446837793 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfcc0a3c90 msgr2=0x7fdfcc0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2138749385 2024-04-21T08:46:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.647+0000 7fdfe62e5700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfcc0a3c90 0x7fdfcc0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:46:56.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.703+0000 7fdfe8549700 1 -- 172.21.15.82:0/3446837793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdfe002df80 con 0x7fdfe006b980 2024-04-21T08:46:56.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.703+0000 7fdfd77fe700 1 -- 172.21.15.82:0/3446837793 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7fdfd009c670 con 0x7fdfe006b980 2024-04-21T08:46:56.703 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-21T08:46:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.707+0000 7fdfe8549700 1 -- 172.21.15.82:0/3446837793 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfcc0a3c90 msgr2=0x7fdfcc0a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:46:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.707+0000 7fdfe8549700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfcc0a3c90 0x7fdfcc0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.707+0000 7fdfe8549700 1 -- 172.21.15.82:0/3446837793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfe006b980 msgr2=0x7fdfe0199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.707+0000 7fdfe8549700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfe006b980 0x7fdfe0199610 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000d950 tx=0x7fdfd000dc60 comp rx=0 tx=0).stop 2024-04-21T08:46:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.707+0000 7fdfe8549700 1 -- 172.21.15.82:0/3446837793 shutdown_connections 2024-04-21T08:46:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.707+0000 7fdfe8549700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7fdfcc0a3c90 0x7fdfcc0a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.707+0000 7fdfe8549700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe0108a20 0x7fdfe019d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.707+0000 7fdfe8549700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfe006afe0 0x7fdfe01990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.707+0000 7fdfe8549700 1 --2- 172.21.15.82:0/3446837793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfe006b980 0x7fdfe0199610 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.707+0000 7fdfe8549700 1 -- 172.21.15.82:0/3446837793 >> 172.21.15.82:0/3446837793 conn(0x7fdfe00788b0 msgr2=0x7fdfe00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.707+0000 7fdfe8549700 1 -- 172.21.15.82:0/3446837793 shutdown_connections 2024-04-21T08:46:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:56.707+0000 7fdfe8549700 1 -- 172.21.15.82:0/3446837793 wait complete. 2024-04-21T08:46:56.755 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:56 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:56.470+0000 7f479326f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:46:56.755 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:56 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:56.663+0000 7f479326f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:46:56.755 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:56 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:56.486+0000 7ff6318ca200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:46:56.755 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:56 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:56.679+0000 7ff6318ca200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:46:56.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:56 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3446837793' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:46:56.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:56 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:56.599+0000 7fe17b99f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:46:56.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:56 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:56.810+0000 7fe17b99f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:46:57.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:56 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3446837793' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:46:57.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:56 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3446837793' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:46:57.439 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-21T08:46:57.439 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-21T08:46:57.439 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-21T08:46:57.439 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-21T08:46:57.439 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-21T08:46:57.439 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-21T08:46:57.439 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-21T08:46:57.439 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-21T08:46:57.439 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-21T08:46:57.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:57.440 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-21T08:46:57.497 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-21T08:46:57.497 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-21T08:46:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-21T08:46:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:46:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-21T08:46:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:46:57.498 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:46:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:46:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-21T08:46:57.498 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:46:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:46:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-21T08:46:57.498 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:46:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-21T08:46:57.499 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-21T08:46:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-21T08:46:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-21T08:46:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42067sec preferred_lft 42067sec 2024-04-21T08:46:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::8fde:e834:f54e:2fd3/64 scope link noprefixroute 2024-04-21T08:46:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:46:57.499 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-21T08:46:57.499 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:46:57.499 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-21T08:46:57.499 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2024-04-21T08:46:57.500 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2024-04-21T08:46:57.500 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2024-04-21T08:46:57.500 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-21T08:46:57.500 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-21T08:46:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:46:57.628 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:57 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:57.260+0000 7f479326f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:46:57.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:46:57.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:57.638 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-21T08:46:57.659 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:57 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:57.414+0000 7fe17b99f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:46:57.660 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-21T08:46:57.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:57.663 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-21T08:46:57.709 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:57 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:57.283+0000 7ff6318ca200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:46:57.718 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-21T08:46:57.719 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-21T08:46:57.719 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-21T08:46:57.719 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-21T08:46:57.719 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-21T08:46:57.720 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:46:57.720 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-21T08:46:57.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-21T08:46:57.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-21T08:46:57.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-21T08:46:57.720 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-21T08:46:57.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:46:57.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:46:57.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:57.845 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-21T08:46:57.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:57.861 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-04-21T08:46:57.917 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:46:57.918 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-21T08:46:57.918 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-21T08:46:57.918 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-21T08:46:57.918 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-21T08:46:57.968 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:57 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:57.868+0000 7fe17b99f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:46:57.968 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:57 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:57.968+0000 7fe17b99f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:46:57.976 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:57 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:57.709+0000 7ff6318ca200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:46:57.976 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:57 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:57.807+0000 7ff6318ca200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:46:57.982 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:57 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:57.683+0000 7f479326f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:46:57.982 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:57 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:57.779+0000 7f479326f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:46:57.982 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:57 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:57.982+0000 7f479326f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:46:58.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:46:58.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:46:58.034 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-21T08:46:58.034 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:46:58.034 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-21T08:46:58.034 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-21T08:46:58.034 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-21T08:46:58.034 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-21T08:46:58.034 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-21T08:46:58.034 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-21T08:46:58.034 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-21T08:46:58.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:46:58.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:46:58.185 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:46:58.185 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-21T08:46:58.185 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-21T08:46:58.185 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2024-04-21T08:46:58.185 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-21T08:46:58.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:46:58.241 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:58.188+0000 7fe17b99f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:46:58.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:46:58.261 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T08:46:58.261 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-21T08:46:58.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T08:46:58.277 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-21T08:46:58.333 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-21T08:46:58.334 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-21T08:46:58.334 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/6bfc719b-e3be-4e08-984a-1bbd1bc02cbc 2024-04-21T08:46:58.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:58.334 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/6bfc719b-e3be-4e08-984a-1bbd1bc02cbc /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-21T08:46:58.418 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-21T08:46:58.418 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-21T08:46:58.418 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-04-21T08:46:58.476 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:58 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:58.011+0000 7ff6318ca200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:46:58.543 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-21T08:46:58.543 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-21T08:46:58.543 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-04-21T08:46:58.543 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.106:3300,172.21.15.134:3300" 2024-04-21T08:46:58.543 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.82:3300,172.21.15.106:3300,172.21.15.134:3300:/volumes/qa/sv_1/6bfc719b-e3be-4e08-984a-1bbd1bc02cbc 2024-04-21T08:46:58.543 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=24d05dee-ffba-11ee-bc93-c7b262605968" will pass to kernel 2024-04-21T08:46:58.544 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-21T08:46:58.544 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-04-21T08:46:58.544 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-21T08:46:58.544 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-04-21T08:46:58.544 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-21T08:46:58.546 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-21T08:46:58.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:58.546 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-21T08:46:58.614 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T08:46:58.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T08:46:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:46:58.711 INFO:teuthology.orchestra.run.smithi082.stdout:client25091 2024-04-21T08:46:58.712 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:46:58.712 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:46:58.728 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T08:46:58.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T08:46:58.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:46:58.825 INFO:teuthology.orchestra.run.smithi082.stdout:client25091 2024-04-21T08:46:58.825 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T08:46:58.837 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:58 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:58.503+0000 7f479326f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:46:58.837 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:58 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:58.596+0000 7f479326f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:46:58.839 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:58.742+0000 7fe17b99f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:46:58.839 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:58.840+0000 7fe17b99f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:46:58.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T08:46:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:46:58.878 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:58 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:58.541+0000 7ff6318ca200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:46:58.878 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:58 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:58.634+0000 7ff6318ca200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:46:58.882 INFO:teuthology.orchestra.run.smithi082.stdout:client25091 2024-04-21T08:46:58.882 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/24d05dee-ffba-11ee-bc93-c7b262605968.client25091/status') 2024-04-21T08:46:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:46:58.979 INFO:teuthology.orchestra.run.smithi082.stdout:instance: client.25091 (3)172.21.15.82:0/1607505684 2024-04-21T08:46:58.980 INFO:teuthology.orchestra.run.smithi082.stdout:blocklisted: false 2024-04-21T08:46:58.980 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records in 2024-04-21T08:46:58.980 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records out 2024-04-21T08:46:58.980 INFO:teuthology.orchestra.run.smithi082.stderr:71 bytes copied, 2.1721e-05 s, 3.3 MB/s 2024-04-21T08:46:58.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:46:58.987 INFO:teuthology.run_tasks:Running task exec... 2024-04-21T08:46:58.996 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T08:46:58.996 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-04-21T08:46:58.996 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-21T08:46:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.098+0000 7f0f34304640 1 -- 172.21.15.82:0/4267286464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c105360 msgr2=0x7f0f2c107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.098+0000 7f0f34304640 1 --2- 172.21.15.82:0/4267286464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c105360 0x7f0f2c107750 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009900 tx=0x7f0f2802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.098+0000 7f0f34304640 1 -- 172.21.15.82:0/4267286464 shutdown_connections 2024-04-21T08:46:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.098+0000 7f0f34304640 1 --2- 172.21.15.82:0/4267286464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f2c107c90 0x7f0f2c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.098+0000 7f0f34304640 1 --2- 172.21.15.82:0/4267286464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c105360 0x7f0f2c107750 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.098+0000 7f0f34304640 1 --2- 172.21.15.82:0/4267286464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c102a30 0x7f0f2c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.098+0000 7f0f34304640 1 -- 172.21.15.82:0/4267286464 >> 172.21.15.82:0/4267286464 conn(0x7f0f2c0fc420 msgr2=0x7f0f2c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.098+0000 7f0f34304640 1 -- 172.21.15.82:0/4267286464 shutdown_connections 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.098+0000 7f0f34304640 1 -- 172.21.15.82:0/4267286464 wait complete. 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f34304640 1 Processor -- start 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f34304640 1 -- start start 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f34304640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c102a30 0x7f0f2c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f34304640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f2c105360 0x7f0f2c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f34304640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c107c90 0x7f0f2c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f34304640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c10db40 con 0x7f0f2c107c90 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f34304640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c10d9c0 con 0x7f0f2c105360 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f34304640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c10dcc0 con 0x7f0f2c102a30 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f328a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c107c90 0x7f0f2c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f3189f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f2c105360 0x7f0f2c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f328a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c107c90 0x7f0f2c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34810/0 (socket says 172.21.15.82:34810) 2024-04-21T08:46:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f328a1640 1 -- 172.21.15.82:0/563517093 learned_addr learned my addr 172.21.15.82:0/563517093 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.099+0000 7f0f320a0640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c102a30 0x7f0f2c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f3189f640 1 -- 172.21.15.82:0/563517093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c102a30 msgr2=0x7f0f2c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f3189f640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c102a30 0x7f0f2c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f3189f640 1 -- 172.21.15.82:0/563517093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c107c90 msgr2=0x7f0f2c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f3189f640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c107c90 0x7f0f2c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f3189f640 1 -- 172.21.15.82:0/563517093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c19fdc0 con 0x7f0f2c105360 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f328a1640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c107c90 0x7f0f2c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f320a0640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c102a30 0x7f0f2c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f3189f640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f2c105360 0x7f0f2c1986e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f28036040 tx=0x7f0f280042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f237fe640 1 -- 172.21.15.82:0/563517093 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28043020 con 0x7f0f2c105360 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f34304640 1 -- 172.21.15.82:0/563517093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c1a0050 con 0x7f0f2c105360 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f237fe640 1 -- 172.21.15.82:0/563517093 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f28004a50 con 0x7f0f2c105360 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f237fe640 1 -- 172.21.15.82:0/563517093 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28003b30 con 0x7f0f2c105360 2024-04-21T08:46:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.100+0000 7f0f34304640 1 -- 172.21.15.82:0/563517093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c1a05c0 con 0x7f0f2c105360 2024-04-21T08:46:59.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.101+0000 7f0f34304640 1 -- 172.21.15.82:0/563517093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2c0fdf10 con 0x7f0f2c105360 2024-04-21T08:46:59.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.102+0000 7f0f237fe640 1 -- 172.21.15.82:0/563517093 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0f28046020 con 0x7f0f2c105360 2024-04-21T08:46:59.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.103+0000 7f0f237fe640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0f180a3ca0 0x7f0f180a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.103+0000 7f0f320a0640 1 -- 172.21.15.82:0/563517093 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0f180a3ca0 msgr2=0x7f0f180a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2138749385 2024-04-21T08:46:59.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.103+0000 7f0f237fe640 1 -- 172.21.15.82:0/563517093 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f0f280f2f00 con 0x7f0f2c105360 2024-04-21T08:46:59.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.103+0000 7f0f320a0640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0f180a3ca0 0x7f0f180a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:46:59.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.106+0000 7f0f237fe640 1 -- 172.21.15.82:0/563517093 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f2804d020 con 0x7f0f2c105360 2024-04-21T08:46:59.128 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:58 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:58.838+0000 7f479326f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:46:59.128 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:58 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:58.969+0000 7f479326f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:46:59.128 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:59 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:59.062+0000 7f479326f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:46:59.225 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:58 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:58.879+0000 7ff6318ca200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:46:59.225 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:59 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:59.013+0000 7ff6318ca200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:46:59.225 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:59 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:59.109+0000 7ff6318ca200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:46:59.225 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:59 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:59.225+0000 7ff6318ca200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:46:59.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.304+0000 7f0f320a0640 1 -- 172.21.15.82:0/563517093 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0f180a3ca0 msgr2=0x7f0f180a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2138749385 2024-04-21T08:46:59.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.304+0000 7f0f320a0640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0f180a3ca0 0x7f0f180a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:46:59.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.332+0000 7f0f34304640 1 -- 172.21.15.82:0/563517093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f0f2c02d5c0 con 0x7f0f2c105360 2024-04-21T08:46:59.358 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:59 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:59.097+0000 7fe17b99f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:46:59.358 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:59 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:59.240+0000 7fe17b99f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:46:59.358 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:59 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:59.342+0000 7fe17b99f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:46:59.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.388+0000 7f0f237fe640 1 -- 172.21.15.82:0/563517093 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f0f280bf510 con 0x7f0f2c105360 2024-04-21T08:46:59.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.390+0000 7f0f34304640 1 -- 172.21.15.82:0/563517093 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0f180a3ca0 msgr2=0x7f0f180a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:46:59.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.390+0000 7f0f34304640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0f180a3ca0 0x7f0f180a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.391+0000 7f0f34304640 1 -- 172.21.15.82:0/563517093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f2c105360 msgr2=0x7f0f2c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.391+0000 7f0f34304640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f2c105360 0x7f0f2c1986e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f28036040 tx=0x7f0f280042f0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.391+0000 7f0f34304640 1 -- 172.21.15.82:0/563517093 shutdown_connections 2024-04-21T08:46:59.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.391+0000 7f0f34304640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f0f180a3ca0 0x7f0f180a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.391+0000 7f0f34304640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f2c107c90 0x7f0f2c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.391+0000 7f0f34304640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f2c105360 0x7f0f2c1986e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.391+0000 7f0f34304640 1 --2- 172.21.15.82:0/563517093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f2c102a30 0x7f0f2c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.391+0000 7f0f34304640 1 -- 172.21.15.82:0/563517093 >> 172.21.15.82:0/563517093 conn(0x7f0f2c0fc420 msgr2=0x7f0f2c103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.391+0000 7f0f34304640 1 -- 172.21.15.82:0/563517093 shutdown_connections 2024-04-21T08:46:59.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.391+0000 7f0f34304640 1 -- 172.21.15.82:0/563517093 wait complete. 2024-04-21T08:46:59.401 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-21T08:46:59.475 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:59 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:59.179+0000 7f479326f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:46:59.476 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:59 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/563517093' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-21T08:46:59.476 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:46:59 smithi106 ceph-mon[31072]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-21T08:46:59.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:59 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/563517093' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-21T08:46:59.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:46:59 smithi134 ceph-mon[31772]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-21T08:46:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.505+0000 7f8086488640 1 -- 172.21.15.82:0/1877045519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80801031b0 msgr2=0x7f8080103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.505+0000 7f8086488640 1 --2- 172.21.15.82:0/1877045519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80801031b0 0x7f8080103630 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8070009900 tx=0x7f807002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.506+0000 7f8086488640 1 -- 172.21.15.82:0/1877045519 shutdown_connections 2024-04-21T08:46:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.506+0000 7f8086488640 1 --2- 172.21.15.82:0/1877045519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8080103b70 0x7f808010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.506+0000 7f8086488640 1 --2- 172.21.15.82:0/1877045519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80801031b0 0x7f8080103630 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.506+0000 7f8086488640 1 --2- 172.21.15.82:0/1877045519 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8080101fc0 0x7f80801023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.506+0000 7f8086488640 1 -- 172.21.15.82:0/1877045519 >> 172.21.15.82:0/1877045519 conn(0x7f80800fd530 msgr2=0x7f80800ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.506+0000 7f8086488640 1 -- 172.21.15.82:0/1877045519 shutdown_connections 2024-04-21T08:46:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.506+0000 7f8086488640 1 -- 172.21.15.82:0/1877045519 wait complete. 2024-04-21T08:46:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.506+0000 7f8086488640 1 Processor -- start 2024-04-21T08:46:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.506+0000 7f8086488640 1 -- start start 2024-04-21T08:46:59.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f8086488640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8080101fc0 0x7f8080075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f8086488640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80801031b0 0x7f8080074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f8086488640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8080103b70 0x7f8080074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f8086488640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f808010db00 con 0x7f80801031b0 2024-04-21T08:46:59.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f8086488640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f808010d980 con 0x7f8080103b70 2024-04-21T08:46:59.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f8086488640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f808010dc80 con 0x7f8080101fc0 2024-04-21T08:46:59.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f8084a25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8080103b70 0x7f8080074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f8084a25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8080103b70 0x7f8080074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:52386/0 (socket says 172.21.15.82:52386) 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f8084a25640 1 -- 172.21.15.82:0/2567918149 learned_addr learned my addr 172.21.15.82:0/2567918149 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f807ffff640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8080101fc0 0x7f8080075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f807f7fe640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80801031b0 0x7f8080074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f807ffff640 1 -- 172.21.15.82:0/2567918149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8080103b70 msgr2=0x7f8080074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f807ffff640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8080103b70 0x7f8080074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f807ffff640 1 -- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80801031b0 msgr2=0x7f8080074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f807ffff640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80801031b0 0x7f8080074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f807ffff640 1 -- 172.21.15.82:0/2567918149 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8080075110 con 0x7f8080101fc0 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.507+0000 7f807ffff640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8080101fc0 0x7f8080075d00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f806800e9c0 tx=0x7f806800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.508+0000 7f807d7fa640 1 -- 172.21.15.82:0/2567918149 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806800ccc0 con 0x7f8080101fc0 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.508+0000 7f807d7fa640 1 -- 172.21.15.82:0/2567918149 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80680035f0 con 0x7f8080101fc0 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.508+0000 7f8086488640 1 -- 172.21.15.82:0/2567918149 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80801a2270 con 0x7f8080101fc0 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.508+0000 7f807d7fa640 1 -- 172.21.15.82:0/2567918149 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806800ce60 con 0x7f8080101fc0 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.508+0000 7f8086488640 1 -- 172.21.15.82:0/2567918149 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80801a2790 con 0x7f8080101fc0 2024-04-21T08:46:59.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.509+0000 7f8086488640 1 -- 172.21.15.82:0/2567918149 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808004fce0 con 0x7f8080101fc0 2024-04-21T08:46:59.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.510+0000 7f807d7fa640 1 -- 172.21.15.82:0/2567918149 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f8068004000 con 0x7f8080101fc0 2024-04-21T08:46:59.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.510+0000 7f807d7fa640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f806c0a3ca0 0x7f806c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.510+0000 7f807f7fe640 1 -- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f806c0a3ca0 msgr2=0x7f806c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2138749385 2024-04-21T08:46:59.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.511+0000 7f807d7fa640 1 -- 172.21.15.82:0/2567918149 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f80680d0ea0 con 0x7f8080101fc0 2024-04-21T08:46:59.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.511+0000 7f807f7fe640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f806c0a3ca0 0x7f806c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:46:59.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.515+0000 7f807d7fa640 1 -- 172.21.15.82:0/2567918149 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8068043e20 con 0x7f8080101fc0 2024-04-21T08:46:59.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.712+0000 7f807f7fe640 1 -- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f806c0a3ca0 msgr2=0x7f806c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2138749385 2024-04-21T08:46:59.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.712+0000 7f807f7fe640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f806c0a3ca0 0x7f806c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:46:59.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:46:59.724+0000 7f8086488640 1 -- 172.21.15.82:0/2567918149 --> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f8080062430 con 0x7f806c0a3ca0 2024-04-21T08:46:59.774 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:59 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/563517093' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-21T08:46:59.774 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:46:59 smithi082 ceph-mon[27967]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-21T08:46:59.774 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:59 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:59.465+0000 7fe17b99f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:46:59.878 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:46:59 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:46:59.476+0000 7f479326f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:46:59.976 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:46:59 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:46:59.536+0000 7ff6318ca200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:47:00.108 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:46:59 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:46:59.774+0000 7fe17b99f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:47:00.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:00.113+0000 7f807f7fe640 1 -- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f806c0a3ca0 msgr2=0x7f806c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/2138749385 2024-04-21T08:47:00.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:00.113+0000 7f807f7fe640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f806c0a3ca0 0x7f806c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-21T08:47:00.378 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:47:00.075+0000 7f479326f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:47:00.378 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y[32447]: 2024-04-21T08:47:00.288+0000 7f479326f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:47:00.390 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:47:00.136+0000 7ff6318ca200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:47:00.390 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z[33629]: 2024-04-21T08:47:00.350+0000 7ff6318ca200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:47:00.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:00.403+0000 7f807d7fa640 1 -- 172.21.15.82:0/2567918149 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f8068099420 con 0x7f8080101fc0 2024-04-21T08:47:00.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:00.631+0000 7f807d7fa640 1 -- 172.21.15.82:0/2567918149 <== mon.1 v2:172.21.15.134:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f80680ce650 con 0x7f8080101fc0 2024-04-21T08:47:00.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:00.632+0000 7f807d7fa640 1 -- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f806c0a3ca0 msgr2=0x7f806c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:47:00.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:00.632+0000 7f807d7fa640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/2138749385,v1:172.21.15.82:6801/2138749385] conn(0x7f806c0a3ca0 0x7f806c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-21T08:47:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:47:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: Standby manager daemon y restarted 2024-04-21T08:47:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: Standby manager daemon y started 2024-04-21T08:47:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.106:0/3992988652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:47:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.106:0/3992988652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:47:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.106:0/3992988652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:47:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.106:0/3992988652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:47:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: Standby manager daemon z restarted 2024-04-21T08:47:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: Standby manager daemon z started 2024-04-21T08:47:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.134:0/4020475147' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:47:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.134:0/4020475147' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:47:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.134:0/4020475147' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:47:00.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:00 smithi134 ceph-mon[31772]: from='mgr.? 172.21.15.134:0/4020475147' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:47:00.762 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: Standby manager daemon y restarted 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: Standby manager daemon y started 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.106:0/3992988652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.106:0/3992988652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.106:0/3992988652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.106:0/3992988652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: Standby manager daemon z restarted 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: Standby manager daemon z started 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.134:0/4020475147' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.134:0/4020475147' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.134:0/4020475147' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:47:00.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:00 smithi106 ceph-mon[31072]: from='mgr.? 172.21.15.134:0/4020475147' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:47:00.764 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-21T08:47:00.764 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:47:00.764 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: Standby manager daemon y restarted 2024-04-21T08:47:00.764 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: Standby manager daemon y started 2024-04-21T08:47:00.764 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.106:0/3992988652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:47:00.764 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.106:0/3992988652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:47:00.764 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.106:0/3992988652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:47:00.764 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.106:0/3992988652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:47:00.764 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: Standby manager daemon z restarted 2024-04-21T08:47:00.765 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: Standby manager daemon z started 2024-04-21T08:47:00.765 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.134:0/4020475147' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:47:00.765 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.134:0/4020475147' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:47:00.765 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.134:0/4020475147' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:47:00.765 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-mon[27967]: from='mgr.? 172.21.15.134:0/4020475147' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:47:00.765 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:00.390+0000 7fe17b99f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:47:00.765 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:00.615+0000 7fe17b99f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:47:01.108 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:00.874+0000 7fe0ddad8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:01.109 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:00.874+0000 7fe0ddad8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:01.109 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:00.874+0000 7fe0ddad8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:01.109 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:00.874+0000 7fe0ddad8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:01.109 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:00.874+0000 7fe0ddad8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:01.109 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:00.875+0000 7fe0db2d3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:01.109 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:00.875+0000 7fe0db2d3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:01.109 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:00.875+0000 7fe0db2d3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:01.109 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:00.875+0000 7fe0db2d3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:01.109 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:00.875+0000 7fe0db2d3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:01.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.635+0000 7f807d7fa640 1 -- 172.21.15.82:0/2567918149 <== mon.1 v2:172.21.15.134:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f806800ccc0 con 0x7f8080101fc0 2024-04-21T08:47:01.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.635+0000 7f807d7fa640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f806c035890 0x7f806c035c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:01.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.635+0000 7f807d7fa640 1 -- 172.21.15.82:0/2567918149 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f8068043c70 con 0x7f806c035890 2024-04-21T08:47:01.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.641+0000 7f807f7fe640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f806c035890 0x7f806c035c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.644+0000 7f807f7fe640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f806c035890 0x7f806c035c70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8070009230 tx=0x7f8070005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:01.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-21T08:47:01.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: Active manager daemon x restarted 2024-04-21T08:47:01.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: Activating manager daemon x 2024-04-21T08:47:01.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T08:47:01.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: mgrmap e21: x(active, starting, since 0.0114416s), standbys: z, y 2024-04-21T08:47:01.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:47:01.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:47:01.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:47:01.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:47:01.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:47:01.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:47:01.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:47:01.655 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: Manager daemon x is now available 2024-04-21T08:47:01.656 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:47:01.656 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:47:01.656 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:47:01.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.668+0000 7f807d7fa640 1 -- 172.21.15.82:0/2567918149 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f8068043c70 con 0x7f806c035890 2024-04-21T08:47:01.667 INFO:teuthology.orchestra.run.smithi082.stdout:{} 2024-04-21T08:47:01.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.670+0000 7f8086488640 1 -- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f806c035890 msgr2=0x7f806c035c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:01.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.670+0000 7f8086488640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f806c035890 0x7f806c035c70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8070009230 tx=0x7f8070005fd0 comp rx=0 tx=0).stop 2024-04-21T08:47:01.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.670+0000 7f8086488640 1 -- 172.21.15.82:0/2567918149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8080101fc0 msgr2=0x7f8080075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:01.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.670+0000 7f8086488640 1 --2- 172.21.15.82:0/2567918149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8080101fc0 0x7f8080075d00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f806800e9c0 tx=0x7f806800ee80 comp rx=0 tx=0).stop 2024-04-21T08:47:01.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.670+0000 7f8084a25640 1 -- 172.21.15.82:0/2567918149 reap_dead start 2024-04-21T08:47:01.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.670+0000 7f8086488640 1 -- 172.21.15.82:0/2567918149 shutdown_connections 2024-04-21T08:47:01.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.670+0000 7f8086488640 1 -- 172.21.15.82:0/2567918149 >> 172.21.15.82:0/2567918149 conn(0x7f80800fd530 msgr2=0x7f8080106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:01.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.670+0000 7f8086488640 1 -- 172.21.15.82:0/2567918149 shutdown_connections 2024-04-21T08:47:01.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.670+0000 7f8086488640 1 -- 172.21.15.82:0/2567918149 wait complete. 2024-04-21T08:47:01.680 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-21T08:47:01.692 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-21T08:47:01.701 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:47:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-21T08:47:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: Active manager daemon x restarted 2024-04-21T08:47:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: Activating manager daemon x 2024-04-21T08:47:01.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T08:47:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: mgrmap e21: x(active, starting, since 0.0114416s), standbys: z, y 2024-04-21T08:47:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:47:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:47:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:47:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:47:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:47:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:47:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:47:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:47:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:47:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:47:01.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:47:01.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:47:01.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:47:01.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:47:01.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:47:01.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:47:01.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:47:01.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:47:01.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:47:01.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:47:01.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: Manager daemon x is now available 2024-04-21T08:47:01.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:47:01.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:47:01.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:01 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:47:01.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.801+0000 7f137f481640 1 -- 172.21.15.82:0/2191455826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1378103180 msgr2=0x7f1378103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.801+0000 7f137f481640 1 --2- 172.21.15.82:0/2191455826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1378103180 0x7f1378103600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1368009860 tx=0x7f136802bb70 comp rx=0 tx=0).stop 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 -- 172.21.15.82:0/2191455826 shutdown_connections 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 --2- 172.21.15.82:0/2191455826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1378103b40 0x7f137810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 --2- 172.21.15.82:0/2191455826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1378103180 0x7f1378103600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 --2- 172.21.15.82:0/2191455826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1378101f90 0x7f13781023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 -- 172.21.15.82:0/2191455826 >> 172.21.15.82:0/2191455826 conn(0x7f13780fd500 msgr2=0x7f13780ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 -- 172.21.15.82:0/2191455826 shutdown_connections 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 -- 172.21.15.82:0/2191455826 wait complete. 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 Processor -- start 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 -- start start 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1378101f90 0x7f1378198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1378103180 0x7f13781986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:01.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1378103b40 0x7f137819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f137810da80 con 0x7f1378103b40 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f137810d900 con 0x7f1378103180 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137f481640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f137810dc00 con 0x7f1378101f90 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137da1e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1378103b40 0x7f137819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.802+0000 7f137d21d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1378101f90 0x7f1378198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f137d21d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1378101f90 0x7f1378198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52258/0 (socket says 172.21.15.82:52258) 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f137d21d640 1 -- 172.21.15.82:0/2034269903 learned_addr learned my addr 172.21.15.82:0/2034269903 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f137ca1c640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1378103180 0x7f13781986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f137d21d640 1 -- 172.21.15.82:0/2034269903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1378103180 msgr2=0x7f13781986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f137d21d640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1378103180 0x7f13781986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f137d21d640 1 -- 172.21.15.82:0/2034269903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1378103b40 msgr2=0x7f137819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f137d21d640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1378103b40 0x7f137819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:01.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f137d21d640 1 -- 172.21.15.82:0/2034269903 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137819fd80 con 0x7f1378101f90 2024-04-21T08:47:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f137da1e640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1378103b40 0x7f137819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f137d21d640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1378101f90 0x7f1378198160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f137400e9c0 tx=0x7f137400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f136e7fc640 1 -- 172.21.15.82:0/2034269903 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137400cd80 con 0x7f1378101f90 2024-04-21T08:47:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f137f481640 1 -- 172.21.15.82:0/2034269903 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13781a0070 con 0x7f1378101f90 2024-04-21T08:47:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f137f481640 1 -- 172.21.15.82:0/2034269903 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13781a05e0 con 0x7f1378101f90 2024-04-21T08:47:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f136e7fc640 1 -- 172.21.15.82:0/2034269903 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13740218e0 con 0x7f1378101f90 2024-04-21T08:47:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.803+0000 7f136e7fc640 1 -- 172.21.15.82:0/2034269903 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374010810 con 0x7f1378101f90 2024-04-21T08:47:01.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.804+0000 7f137f481640 1 -- 172.21.15.82:0/2034269903 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f135c0052c0 con 0x7f1378101f90 2024-04-21T08:47:01.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.805+0000 7f136e7fc640 1 -- 172.21.15.82:0/2034269903 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f1374021430 con 0x7f1378101f90 2024-04-21T08:47:01.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.806+0000 7f136e7fc640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f13640a39b0 0x7f13640a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:01.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.806+0000 7f137ca1c640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f13640a39b0 0x7f13640a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:01.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.806+0000 7f136e7fc640 1 -- 172.21.15.82:0/2034269903 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f13740d07f0 con 0x7f1378101f90 2024-04-21T08:47:01.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.806+0000 7f137ca1c640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f13640a39b0 0x7f13640a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f136802c180 tx=0x7f13680349b0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:01.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:01.810+0000 7f136e7fc640 1 -- 172.21.15.82:0/2034269903 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f137409cfe0 con 0x7f1378101f90 2024-04-21T08:47:01.828 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: Active manager daemon x restarted 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: Activating manager daemon x 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: mgrmap e21: x(active, starting, since 0.0114416s), standbys: z, y 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:47:01.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:47:01.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: Manager daemon x is now available 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:47:01.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:47:01.832 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:01 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:47:02.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.010+0000 7f137f481640 1 -- 172.21.15.82:0/2034269903 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f135c005880 con 0x7f1378101f90 2024-04-21T08:47:02.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.012+0000 7f136e7fc640 1 -- 172.21.15.82:0/2034269903 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25247 (secure 0 0 0) 0x7f137409c8e0 con 0x7f1378101f90 2024-04-21T08:47:02.012 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:02.012 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:47:00.617381+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2119539425":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6800/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/539366680":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/2275507980":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2883183315":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:47:02.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.014+0000 7f137f481640 1 -- 172.21.15.82:0/2034269903 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f13640a39b0 msgr2=0x7f13640a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.014+0000 7f137f481640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f13640a39b0 0x7f13640a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f136802c180 tx=0x7f13680349b0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.014+0000 7f137f481640 1 -- 172.21.15.82:0/2034269903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1378101f90 msgr2=0x7f1378198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.014+0000 7f137f481640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1378101f90 0x7f1378198160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f137400e9c0 tx=0x7f137400ee80 comp rx=0 tx=0).stop 2024-04-21T08:47:02.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.014+0000 7f137f481640 1 -- 172.21.15.82:0/2034269903 shutdown_connections 2024-04-21T08:47:02.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.014+0000 7f137f481640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f13640a39b0 0x7f13640a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.014+0000 7f137f481640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1378103b40 0x7f137819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.014+0000 7f137f481640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1378103180 0x7f13781986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.014+0000 7f137f481640 1 --2- 172.21.15.82:0/2034269903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1378101f90 0x7f1378198160 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.014+0000 7f137f481640 1 -- 172.21.15.82:0/2034269903 >> 172.21.15.82:0/2034269903 conn(0x7f13780fd500 msgr2=0x7f1378106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:02.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.014+0000 7f137f481640 1 -- 172.21.15.82:0/2034269903 shutdown_connections 2024-04-21T08:47:02.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.014+0000 7f137f481640 1 -- 172.21.15.82:0/2034269903 wait complete. 2024-04-21T08:47:02.024 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:01.652+0000 7fe0d82cd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:02.024 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:01.652+0000 7fe0d82cd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:02.024 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:01.652+0000 7fe0d82cd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:02.024 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:01.652+0000 7fe0d82cd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:02.024 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:47:01 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:47:01.652+0000 7fe0d82cd700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:47:02.026 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:47:02.027 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:47:02.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 -- 172.21.15.82:0/2287789221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b41029b0 msgr2=0x7fd5b41092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 --2- 172.21.15.82:0/2287789221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b41029b0 0x7fd5b41092d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009ca0 tx=0x7fd5ac02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 -- 172.21.15.82:0/2287789221 shutdown_connections 2024-04-21T08:47:02.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 --2- 172.21.15.82:0/2287789221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b41029b0 0x7fd5b41092d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 --2- 172.21.15.82:0/2287789221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5b4101ff0 0x7fd5b4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 --2- 172.21.15.82:0/2287789221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5b4100ea0 0x7fd5b41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 -- 172.21.15.82:0/2287789221 >> 172.21.15.82:0/2287789221 conn(0x7fd5b40fc410 msgr2=0x7fd5b40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:02.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 -- 172.21.15.82:0/2287789221 shutdown_connections 2024-04-21T08:47:02.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 -- 172.21.15.82:0/2287789221 wait complete. 2024-04-21T08:47:02.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 Processor -- start 2024-04-21T08:47:02.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 -- start start 2024-04-21T08:47:02.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5b4100ea0 0x7fd5b41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:02.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5b4101ff0 0x7fd5b4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:02.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b41029b0 0x7fd5b419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:02.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.131+0000 7fd5baad2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b410c9f0 con 0x7fd5b41029b0 2024-04-21T08:47:02.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5baad2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b410c870 con 0x7fd5b4100ea0 2024-04-21T08:47:02.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5baad2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b410cb70 con 0x7fd5b4101ff0 2024-04-21T08:47:02.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b906f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b41029b0 0x7fd5b419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:02.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b906f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b41029b0 0x7fd5b419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34980/0 (socket says 172.21.15.82:34980) 2024-04-21T08:47:02.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b906f640 1 -- 172.21.15.82:0/1915503565 learned_addr learned my addr 172.21.15.82:0/1915503565 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b886e640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5b4100ea0 0x7fd5b41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b3fff640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5b4101ff0 0x7fd5b4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b906f640 1 -- 172.21.15.82:0/1915503565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5b4101ff0 msgr2=0x7fd5b4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b906f640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5b4101ff0 0x7fd5b4198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b906f640 1 -- 172.21.15.82:0/1915503565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5b4100ea0 msgr2=0x7fd5b41981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b906f640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5b4100ea0 0x7fd5b41981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b906f640 1 -- 172.21.15.82:0/1915503565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b419fde0 con 0x7fd5b41029b0 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b886e640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5b4100ea0 0x7fd5b41981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b906f640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b41029b0 0x7fd5b419f770 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac02c700 tx=0x7fd5ac0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b1ffb640 1 -- 172.21.15.82:0/1915503565 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac041020 con 0x7fd5b41029b0 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b1ffb640 1 -- 172.21.15.82:0/1915503565 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ac034ca0 con 0x7fd5b41029b0 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5baad2640 1 -- 172.21.15.82:0/1915503565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b41a0070 con 0x7fd5b41029b0 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5b1ffb640 1 -- 172.21.15.82:0/1915503565 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac039040 con 0x7fd5b41029b0 2024-04-21T08:47:02.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.132+0000 7fd5baad2640 1 -- 172.21.15.82:0/1915503565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b41a0560 con 0x7fd5b41029b0 2024-04-21T08:47:02.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.133+0000 7fd5baad2640 1 -- 172.21.15.82:0/1915503565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b41a0880 con 0x7fd5b41029b0 2024-04-21T08:47:02.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.134+0000 7fd5b1ffb640 1 -- 172.21.15.82:0/1915503565 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fd5ac044020 con 0x7fd5b41029b0 2024-04-21T08:47:02.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.135+0000 7fd5b1ffb640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5a40a39b0 0x7fd5a40a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:02.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.135+0000 7fd5b886e640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5a40a39b0 0x7fd5a40a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:02.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.135+0000 7fd5b1ffb640 1 -- 172.21.15.82:0/1915503565 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fd5ac0f2390 con 0x7fd5b41029b0 2024-04-21T08:47:02.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.135+0000 7fd5b886e640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5a40a39b0 0x7fd5a40a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009c20 tx=0x7fd5a8005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:02.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.140+0000 7fd5b1ffb640 1 -- 172.21.15.82:0/1915503565 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5ac04b020 con 0x7fd5b41029b0 2024-04-21T08:47:02.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.318+0000 7fd5baad2640 1 -- 172.21.15.82:0/1915503565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd5b41a0c10 con 0x7fd5b41029b0 2024-04-21T08:47:02.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.318+0000 7fd5b1ffb640 1 -- 172.21.15.82:0/1915503565 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fd5ac0beb80 con 0x7fd5b41029b0 2024-04-21T08:47:02.318 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:47:02.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.320+0000 7fd5baad2640 1 -- 172.21.15.82:0/1915503565 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5a40a39b0 msgr2=0x7fd5a40a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.320+0000 7fd5baad2640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5a40a39b0 0x7fd5a40a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009c20 tx=0x7fd5a8005f90 comp rx=0 tx=0).stop 2024-04-21T08:47:02.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.320+0000 7fd5baad2640 1 -- 172.21.15.82:0/1915503565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b41029b0 msgr2=0x7fd5b419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.320+0000 7fd5baad2640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b41029b0 0x7fd5b419f770 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac02c700 tx=0x7fd5ac0346d0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.320+0000 7fd5baad2640 1 -- 172.21.15.82:0/1915503565 shutdown_connections 2024-04-21T08:47:02.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.320+0000 7fd5baad2640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5a40a39b0 0x7fd5a40a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.320+0000 7fd5baad2640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b41029b0 0x7fd5b419f770 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.320+0000 7fd5baad2640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5b4101ff0 0x7fd5b4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.320+0000 7fd5baad2640 1 --2- 172.21.15.82:0/1915503565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5b4100ea0 0x7fd5b41981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.320+0000 7fd5baad2640 1 -- 172.21.15.82:0/1915503565 >> 172.21.15.82:0/1915503565 conn(0x7fd5b40fc410 msgr2=0x7fd5b40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:02.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.320+0000 7fd5baad2640 1 -- 172.21.15.82:0/1915503565 shutdown_connections 2024-04-21T08:47:02.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.320+0000 7fd5baad2640 1 -- 172.21.15.82:0/1915503565 wait complete. 2024-04-21T08:47:02.331 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:47:02.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.433+0000 7f842b2d2640 1 -- 172.21.15.82:0/941726541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8424101f90 msgr2=0x7f84241023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.433+0000 7f842b2d2640 1 --2- 172.21.15.82:0/941726541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8424101f90 0x7f84241023b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8420009900 tx=0x7f842002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.433+0000 7f842b2d2640 1 -- 172.21.15.82:0/941726541 shutdown_connections 2024-04-21T08:47:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.433+0000 7f842b2d2640 1 --2- 172.21.15.82:0/941726541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8424103b40 0x7f842410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.433+0000 7f842b2d2640 1 --2- 172.21.15.82:0/941726541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8424103180 0x7f8424103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.433+0000 7f842b2d2640 1 --2- 172.21.15.82:0/941726541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8424101f90 0x7f84241023b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.433+0000 7f842b2d2640 1 -- 172.21.15.82:0/941726541 >> 172.21.15.82:0/941726541 conn(0x7f84240fd500 msgr2=0x7f84240ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842b2d2640 1 -- 172.21.15.82:0/941726541 shutdown_connections 2024-04-21T08:47:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842b2d2640 1 -- 172.21.15.82:0/941726541 wait complete. 2024-04-21T08:47:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842b2d2640 1 Processor -- start 2024-04-21T08:47:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842b2d2640 1 -- start start 2024-04-21T08:47:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842b2d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8424101f90 0x7f8424198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:02.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842b2d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8424103180 0x7f84241987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842b2d2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8424103b40 0x7f842419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842b2d2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f842410dad0 con 0x7f8424103b40 2024-04-21T08:47:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842b2d2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f842410d950 con 0x7f8424103180 2024-04-21T08:47:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842b2d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f842410dc50 con 0x7f8424101f90 2024-04-21T08:47:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842986f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8424103b40 0x7f842419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842986f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8424103b40 0x7f842419f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35002/0 (socket says 172.21.15.82:35002) 2024-04-21T08:47:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842986f640 1 -- 172.21.15.82:0/3975814241 learned_addr learned my addr 172.21.15.82:0/3975814241 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842886d640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8424103180 0x7f84241987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.434+0000 7f842906e640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8424101f90 0x7f8424198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.435+0000 7f842986f640 1 -- 172.21.15.82:0/3975814241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8424101f90 msgr2=0x7f8424198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.435+0000 7f842986f640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8424101f90 0x7f8424198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.435+0000 7f842986f640 1 -- 172.21.15.82:0/3975814241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8424103180 msgr2=0x7f84241987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.435+0000 7f842986f640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8424103180 0x7f84241987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.435+0000 7f842986f640 1 -- 172.21.15.82:0/3975814241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842419fe90 con 0x7f8424103b40 2024-04-21T08:47:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.435+0000 7f842986f640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8424103b40 0x7f842419f820 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f841c009fd0 tx=0x7f841c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.435+0000 7f841a7fc640 1 -- 172.21.15.82:0/3975814241 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c017070 con 0x7f8424103b40 2024-04-21T08:47:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.435+0000 7f841a7fc640 1 -- 172.21.15.82:0/3975814241 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f841c003e20 con 0x7f8424103b40 2024-04-21T08:47:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.435+0000 7f841a7fc640 1 -- 172.21.15.82:0/3975814241 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c010460 con 0x7f8424103b40 2024-04-21T08:47:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.435+0000 7f842b2d2640 1 -- 172.21.15.82:0/3975814241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84241a0180 con 0x7f8424103b40 2024-04-21T08:47:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.435+0000 7f842b2d2640 1 -- 172.21.15.82:0/3975814241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84241a0680 con 0x7f8424103b40 2024-04-21T08:47:02.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.436+0000 7f842b2d2640 1 -- 172.21.15.82:0/3975814241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84241093d0 con 0x7f8424103b40 2024-04-21T08:47:02.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.437+0000 7f841a7fc640 1 -- 172.21.15.82:0/3975814241 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f841c003970 con 0x7f8424103b40 2024-04-21T08:47:02.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.438+0000 7f841a7fc640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f84100a39b0 0x7f84100a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:02.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.438+0000 7f841a7fc640 1 -- 172.21.15.82:0/3975814241 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f841c0d0190 con 0x7f8424103b40 2024-04-21T08:47:02.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.438+0000 7f842906e640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f84100a39b0 0x7f84100a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:02.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.438+0000 7f842906e640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f84100a39b0 0x7f84100a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8420005ab0 tx=0x7f8420005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:02.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.441+0000 7f841a7fc640 1 -- 172.21.15.82:0/3975814241 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f841c09c330 con 0x7f8424103b40 2024-04-21T08:47:02.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.639+0000 7f842b2d2640 1 -- 172.21.15.82:0/3975814241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f842402d5c0 con 0x7f8424103b40 2024-04-21T08:47:02.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.641+0000 7f841a7fc640 1 -- 172.21.15.82:0/3975814241 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f841c0989a0 con 0x7f8424103b40 2024-04-21T08:47:02.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.643+0000 7f841a7fc640 1 -- 172.21.15.82:0/3975814241 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f841c099980 con 0x7f8424103b40 2024-04-21T08:47:02.643 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:47:02.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.646+0000 7f842b2d2640 1 -- 172.21.15.82:0/3975814241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f84100a39b0 msgr2=0x7f84100a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.646+0000 7f842b2d2640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f84100a39b0 0x7f84100a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8420005ab0 tx=0x7f8420005fd0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.646+0000 7f842b2d2640 1 -- 172.21.15.82:0/3975814241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8424103b40 msgr2=0x7f842419f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.646+0000 7f842b2d2640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8424103b40 0x7f842419f820 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f841c009fd0 tx=0x7f841c00b260 comp rx=0 tx=0).stop 2024-04-21T08:47:02.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.646+0000 7f842b2d2640 1 -- 172.21.15.82:0/3975814241 shutdown_connections 2024-04-21T08:47:02.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.646+0000 7f842b2d2640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f84100a39b0 0x7f84100a5e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.646+0000 7f842b2d2640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8424103b40 0x7f842419f820 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.646+0000 7f842b2d2640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8424103180 0x7f84241987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.646+0000 7f842b2d2640 1 --2- 172.21.15.82:0/3975814241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8424101f90 0x7f8424198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.646+0000 7f842b2d2640 1 -- 172.21.15.82:0/3975814241 >> 172.21.15.82:0/3975814241 conn(0x7f84240fd500 msgr2=0x7f84241065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:02.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.646+0000 7f842b2d2640 1 -- 172.21.15.82:0/3975814241 shutdown_connections 2024-04-21T08:47:02.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.646+0000 7f842b2d2640 1 -- 172.21.15.82:0/3975814241 wait complete. 2024-04-21T08:47:02.659 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:47:02.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.780+0000 7fefe517e640 1 -- 172.21.15.82:0/856693943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefe0102a10 msgr2=0x7fefe0104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.780+0000 7fefe517e640 1 --2- 172.21.15.82:0/856693943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefe0102a10 0x7fefe0104e00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009900 tx=0x7fefd402bc50 comp rx=0 tx=0).stop 2024-04-21T08:47:02.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.780+0000 7fefe517e640 1 -- 172.21.15.82:0/856693943 shutdown_connections 2024-04-21T08:47:02.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.780+0000 7fefe517e640 1 --2- 172.21.15.82:0/856693943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefe0107c70 0x7fefe010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.780+0000 7fefe517e640 1 --2- 172.21.15.82:0/856693943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefe0105340 0x7fefe0107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.780+0000 7fefe517e640 1 --2- 172.21.15.82:0/856693943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefe0102a10 0x7fefe0104e00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.780+0000 7fefe517e640 1 -- 172.21.15.82:0/856693943 >> 172.21.15.82:0/856693943 conn(0x7fefe00fc440 msgr2=0x7fefe00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:02.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.780+0000 7fefe517e640 1 -- 172.21.15.82:0/856693943 shutdown_connections 2024-04-21T08:47:02.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.780+0000 7fefe517e640 1 -- 172.21.15.82:0/856693943 wait complete. 2024-04-21T08:47:02.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.780+0000 7fefe517e640 1 Processor -- start 2024-04-21T08:47:02.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefe517e640 1 -- start start 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefe517e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefe0102a10 0x7fefe0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefe517e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefe0105340 0x7fefe0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefe517e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefe0107c70 0x7fefe019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefe517e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fefe010dbb0 con 0x7fefe0102a10 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefe517e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fefe010da30 con 0x7fefe0107c70 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefe517e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fefe010dd30 con 0x7fefe0105340 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefe0102a10 0x7fefe0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefded9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefe0102a10 0x7fefe0198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44174/0 (socket says 172.21.15.82:44174) 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefded9d640 1 -- 172.21.15.82:0/573367850 learned_addr learned my addr 172.21.15.82:0/573367850 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefdf59e640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefe0107c70 0x7fefe019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefded9d640 1 -- 172.21.15.82:0/573367850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefe0105340 msgr2=0x7fefe0198840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefded9d640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefe0105340 0x7fefe0198840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefded9d640 1 -- 172.21.15.82:0/573367850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefe0107c70 msgr2=0x7fefe019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:02.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefded9d640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefe0107c70 0x7fefe019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:02.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefded9d640 1 -- 172.21.15.82:0/573367850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe019fed0 con 0x7fefe0102a10 2024-04-21T08:47:02.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.781+0000 7fefded9d640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefe0102a10 0x7fefe0198300 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fefd4000c00 tx=0x7fefd4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:02.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.782+0000 7fefc7fff640 1 -- 172.21.15.82:0/573367850 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4043020 con 0x7fefe0102a10 2024-04-21T08:47:02.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.782+0000 7fefe517e640 1 -- 172.21.15.82:0/573367850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe01a0160 con 0x7fefe0102a10 2024-04-21T08:47:02.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.782+0000 7fefc7fff640 1 -- 172.21.15.82:0/573367850 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefd4004440 con 0x7fefe0102a10 2024-04-21T08:47:02.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.782+0000 7fefc7fff640 1 -- 172.21.15.82:0/573367850 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4003b90 con 0x7fefe0102a10 2024-04-21T08:47:02.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.782+0000 7fefe517e640 1 -- 172.21.15.82:0/573367850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe01a0730 con 0x7fefe0102a10 2024-04-21T08:47:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.783+0000 7fefe517e640 1 -- 172.21.15.82:0/573367850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe00fdf30 con 0x7fefe0102a10 2024-04-21T08:47:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.785+0000 7fefc7fff640 1 -- 172.21.15.82:0/573367850 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fefd4046020 con 0x7fefe0102a10 2024-04-21T08:47:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.785+0000 7fefc7fff640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fefc80a3db0 0x7fefc80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.785+0000 7fefc7fff640 1 -- 172.21.15.82:0/573367850 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fefd40f32e0 con 0x7fefe0102a10 2024-04-21T08:47:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.785+0000 7fefde59c640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fefc80a3db0 0x7fefc80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.786+0000 7fefde59c640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fefc80a3db0 0x7fefc80a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fefe01997c0 tx=0x7fefd0005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:02.791+0000 7fefc7fff640 1 -- 172.21.15.82:0/573367850 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefd404d020 con 0x7fefe0102a10 2024-04-21T08:47:02.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:02 smithi134 ceph-mon[31772]: mgrmap e22: x(active, since 1.01906s), standbys: z, y 2024-04-21T08:47:02.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:02 smithi134 ceph-mon[31772]: [21/Apr/2024:08:47:01] ENGINE Bus STARTING 2024-04-21T08:47:02.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:02 smithi134 ceph-mon[31772]: [21/Apr/2024:08:47:01] ENGINE Serving on http://172.21.15.82:8765 2024-04-21T08:47:02.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:02 smithi134 ceph-mon[31772]: [21/Apr/2024:08:47:01] ENGINE Serving on https://172.21.15.82:7150 2024-04-21T08:47:02.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:02 smithi134 ceph-mon[31772]: [21/Apr/2024:08:47:01] ENGINE Bus STARTED 2024-04-21T08:47:02.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:02 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2034269903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:02.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:02 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1915503565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:03.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.016+0000 7fefe517e640 1 -- 172.21.15.82:0/573367850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fefe002d830 con 0x7fefe0102a10 2024-04-21T08:47:03.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.016+0000 7fefc7fff640 1 -- 172.21.15.82:0/573367850 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fefd40bf560 con 0x7fefe0102a10 2024-04-21T08:47:03.016 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:47:03.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.018+0000 7fefe517e640 1 -- 172.21.15.82:0/573367850 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fefc80a3db0 msgr2=0x7fefc80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.018+0000 7fefe517e640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fefc80a3db0 0x7fefc80a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fefe01997c0 tx=0x7fefd0005f90 comp rx=0 tx=0).stop 2024-04-21T08:47:03.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.018+0000 7fefe517e640 1 -- 172.21.15.82:0/573367850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefe0102a10 msgr2=0x7fefe0198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.018+0000 7fefe517e640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefe0102a10 0x7fefe0198300 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fefd4000c00 tx=0x7fefd4004040 comp rx=0 tx=0).stop 2024-04-21T08:47:03.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.018+0000 7fefe517e640 1 -- 172.21.15.82:0/573367850 shutdown_connections 2024-04-21T08:47:03.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.018+0000 7fefe517e640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fefe0107c70 0x7fefe019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.018+0000 7fefe517e640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefe0105340 0x7fefe0198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.018+0000 7fefe517e640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fefc80a3db0 0x7fefc80a6270 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.018+0000 7fefe517e640 1 --2- 172.21.15.82:0/573367850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefe0102a10 0x7fefe0198300 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.018+0000 7fefe517e640 1 -- 172.21.15.82:0/573367850 >> 172.21.15.82:0/573367850 conn(0x7fefe00fc440 msgr2=0x7fefe0103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:03.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.018+0000 7fefe517e640 1 -- 172.21.15.82:0/573367850 shutdown_connections 2024-04-21T08:47:03.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.019+0000 7fefe517e640 1 -- 172.21.15.82:0/573367850 wait complete. 2024-04-21T08:47:03.029 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:02 smithi082 ceph-mon[27967]: mgrmap e22: x(active, since 1.01906s), standbys: z, y 2024-04-21T08:47:03.029 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:02 smithi082 ceph-mon[27967]: [21/Apr/2024:08:47:01] ENGINE Bus STARTING 2024-04-21T08:47:03.029 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:02 smithi082 ceph-mon[27967]: [21/Apr/2024:08:47:01] ENGINE Serving on http://172.21.15.82:8765 2024-04-21T08:47:03.029 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:02 smithi082 ceph-mon[27967]: [21/Apr/2024:08:47:01] ENGINE Serving on https://172.21.15.82:7150 2024-04-21T08:47:03.029 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:02 smithi082 ceph-mon[27967]: [21/Apr/2024:08:47:01] ENGINE Bus STARTED 2024-04-21T08:47:03.029 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:02 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2034269903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:03.029 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:02 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1915503565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:03.030 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:47:03.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:02 smithi106 ceph-mon[31072]: mgrmap e22: x(active, since 1.01906s), standbys: z, y 2024-04-21T08:47:03.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:02 smithi106 ceph-mon[31072]: [21/Apr/2024:08:47:01] ENGINE Bus STARTING 2024-04-21T08:47:03.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:02 smithi106 ceph-mon[31072]: [21/Apr/2024:08:47:01] ENGINE Serving on http://172.21.15.82:8765 2024-04-21T08:47:03.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:02 smithi106 ceph-mon[31072]: [21/Apr/2024:08:47:01] ENGINE Serving on https://172.21.15.82:7150 2024-04-21T08:47:03.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:02 smithi106 ceph-mon[31072]: [21/Apr/2024:08:47:01] ENGINE Bus STARTED 2024-04-21T08:47:03.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:02 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2034269903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:03.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:02 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1915503565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:03.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.137+0000 7f60a9406640 1 -- 172.21.15.82:0/4228430478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60a4103b50 msgr2=0x7f60a410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.137+0000 7f60a9406640 1 --2- 172.21.15.82:0/4228430478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60a4103b50 0x7f60a410a3e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f609c009be0 tx=0x7f609c02bef0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.137+0000 7f60a9406640 1 -- 172.21.15.82:0/4228430478 shutdown_connections 2024-04-21T08:47:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.137+0000 7f60a9406640 1 --2- 172.21.15.82:0/4228430478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60a4103b50 0x7f60a410a3e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.137+0000 7f60a9406640 1 --2- 172.21.15.82:0/4228430478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a4103190 0x7f60a4103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.137+0000 7f60a9406640 1 --2- 172.21.15.82:0/4228430478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a4101fa0 0x7f60a41023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.137+0000 7f60a9406640 1 -- 172.21.15.82:0/4228430478 >> 172.21.15.82:0/4228430478 conn(0x7f60a40fd530 msgr2=0x7f60a40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.137+0000 7f60a9406640 1 -- 172.21.15.82:0/4228430478 shutdown_connections 2024-04-21T08:47:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.137+0000 7f60a9406640 1 -- 172.21.15.82:0/4228430478 wait complete. 2024-04-21T08:47:03.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.137+0000 7f60a9406640 1 Processor -- start 2024-04-21T08:47:03.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a9406640 1 -- start start 2024-04-21T08:47:03.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a9406640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a4101fa0 0x7f60a4075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:03.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a9406640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a4103190 0x7f60a4074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:03.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a9406640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60a4103b50 0x7f60a4074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:03.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a9406640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f60a410dae0 con 0x7f60a4103190 2024-04-21T08:47:03.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a9406640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f60a410d960 con 0x7f60a4103b50 2024-04-21T08:47:03.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a9406640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60a410dc60 con 0x7f60a4101fa0 2024-04-21T08:47:03.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a4103190 0x7f60a4074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:03.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60a4103b50 0x7f60a4074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:03.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60a4103b50 0x7f60a4074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:39646/0 (socket says 172.21.15.82:39646) 2024-04-21T08:47:03.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a37fe640 1 -- 172.21.15.82:0/1377019124 learned_addr learned my addr 172.21.15.82:0/1377019124 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:03.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a4103190 0x7f60a4074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44204/0 (socket says 172.21.15.82:44204) 2024-04-21T08:47:03.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a2ffd640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a4101fa0 0x7f60a4075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:03.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a37fe640 1 -- 172.21.15.82:0/1377019124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a4101fa0 msgr2=0x7f60a4075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a37fe640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a4101fa0 0x7f60a4075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a37fe640 1 -- 172.21.15.82:0/1377019124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a4103190 msgr2=0x7f60a4074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a37fe640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a4103190 0x7f60a4074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a37fe640 1 -- 172.21.15.82:0/1377019124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a4075110 con 0x7f60a4103b50 2024-04-21T08:47:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f60a37fe640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60a4103b50 0x7f60a4074880 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f609c004af0 tx=0x7f609c033c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.138+0000 7f608bfff640 1 -- 172.21.15.82:0/1377019124 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c043020 con 0x7f60a4103b50 2024-04-21T08:47:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.139+0000 7f608bfff640 1 -- 172.21.15.82:0/1377019124 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f609c02ca60 con 0x7f60a4103b50 2024-04-21T08:47:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.139+0000 7f60a9406640 1 -- 172.21.15.82:0/1377019124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a41a2280 con 0x7f60a4103b50 2024-04-21T08:47:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.139+0000 7f608bfff640 1 -- 172.21.15.82:0/1377019124 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c034690 con 0x7f60a4103b50 2024-04-21T08:47:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.139+0000 7f60a9406640 1 -- 172.21.15.82:0/1377019124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a41a2820 con 0x7f60a4103b50 2024-04-21T08:47:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.140+0000 7f60a9406640 1 -- 172.21.15.82:0/1377019124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a4109340 con 0x7f60a4103b50 2024-04-21T08:47:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.141+0000 7f608bfff640 1 -- 172.21.15.82:0/1377019124 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f609c046020 con 0x7f60a4103b50 2024-04-21T08:47:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.141+0000 7f608bfff640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f608c0a3e30 0x7f608c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.141+0000 7f60a2ffd640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f608c0a3e30 0x7f608c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.141+0000 7f608bfff640 1 -- 172.21.15.82:0/1377019124 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f609c0f3b80 con 0x7f60a4103b50 2024-04-21T08:47:03.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.142+0000 7f60a2ffd640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f608c0a3e30 0x7f608c0a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f60a4102ff0 tx=0x7f6098005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:03.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.145+0000 7f608bfff640 1 -- 172.21.15.82:0/1377019124 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f609c04d020 con 0x7f60a4103b50 2024-04-21T08:47:03.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.329+0000 7f60a9406640 1 -- 172.21.15.82:0/1377019124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f60a402d5c0 con 0x7f60a4103b50 2024-04-21T08:47:03.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.330+0000 7f608bfff640 1 -- 172.21.15.82:0/1377019124 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f609c0bfe00 con 0x7f60a4103b50 2024-04-21T08:47:03.329 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:47:03.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.332+0000 7f60a9406640 1 -- 172.21.15.82:0/1377019124 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f608c0a3e30 msgr2=0x7f608c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.332+0000 7f60a9406640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f608c0a3e30 0x7f608c0a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f60a4102ff0 tx=0x7f6098005f90 comp rx=0 tx=0).stop 2024-04-21T08:47:03.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.332+0000 7f60a9406640 1 -- 172.21.15.82:0/1377019124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60a4103b50 msgr2=0x7f60a4074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.332+0000 7f60a9406640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60a4103b50 0x7f60a4074880 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f609c004af0 tx=0x7f609c033c90 comp rx=0 tx=0).stop 2024-04-21T08:47:03.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.332+0000 7f60a9406640 1 -- 172.21.15.82:0/1377019124 shutdown_connections 2024-04-21T08:47:03.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.332+0000 7f60a9406640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f60a4103b50 0x7f60a4074880 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.332+0000 7f60a9406640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f608c0a3e30 0x7f608c0a62f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.332+0000 7f60a9406640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a4103190 0x7f60a4074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.332+0000 7f60a9406640 1 --2- 172.21.15.82:0/1377019124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a4101fa0 0x7f60a4075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.332+0000 7f60a9406640 1 -- 172.21.15.82:0/1377019124 >> 172.21.15.82:0/1377019124 conn(0x7f60a40fd530 msgr2=0x7f60a4106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:03.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.332+0000 7f60a9406640 1 -- 172.21.15.82:0/1377019124 shutdown_connections 2024-04-21T08:47:03.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.332+0000 7f60a9406640 1 -- 172.21.15.82:0/1377019124 wait complete. 2024-04-21T08:47:03.343 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:47:03.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.445+0000 7f5aba175640 1 -- 172.21.15.82:0/1894763657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab4103b30 msgr2=0x7f5ab410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.445+0000 7f5aba175640 1 --2- 172.21.15.82:0/1894763657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab4103b30 0x7f5ab410a3c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009ca0 tx=0x7f5aac02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.445+0000 7f5aba175640 1 -- 172.21.15.82:0/1894763657 shutdown_connections 2024-04-21T08:47:03.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.445+0000 7f5aba175640 1 --2- 172.21.15.82:0/1894763657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab4103b30 0x7f5ab410a3c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.445+0000 7f5aba175640 1 --2- 172.21.15.82:0/1894763657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab4103170 0x7f5ab41035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.445+0000 7f5aba175640 1 --2- 172.21.15.82:0/1894763657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab4101f80 0x7f5ab41023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.445+0000 7f5aba175640 1 -- 172.21.15.82:0/1894763657 >> 172.21.15.82:0/1894763657 conn(0x7f5ab40fd530 msgr2=0x7f5ab40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.445+0000 7f5aba175640 1 -- 172.21.15.82:0/1894763657 shutdown_connections 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.445+0000 7f5aba175640 1 -- 172.21.15.82:0/1894763657 wait complete. 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5aba175640 1 Processor -- start 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5aba175640 1 -- start start 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5aba175640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab4101f80 0x7f5ab4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5aba175640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab4103170 0x7f5ab4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5aba175640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab4103b30 0x7f5ab419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5aba175640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab410da80 con 0x7f5ab4103b30 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5aba175640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab410d900 con 0x7f5ab4103170 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5aba175640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab410dc00 con 0x7f5ab4101f80 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5ab2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab4103170 0x7f5ab4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5ab37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab4101f80 0x7f5ab4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:03.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab4103b30 0x7f5ab419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5ab37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab4101f80 0x7f5ab4198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:47926/0 (socket says 172.21.15.82:47926) 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5ab2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab4103170 0x7f5ab4198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:39662/0 (socket says 172.21.15.82:39662) 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.446+0000 7f5ab2ffd640 1 -- 172.21.15.82:0/3060234602 learned_addr learned my addr 172.21.15.82:0/3060234602 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5ab3fff640 1 -- 172.21.15.82:0/3060234602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab4101f80 msgr2=0x7f5ab4198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5ab3fff640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab4101f80 0x7f5ab4198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5ab3fff640 1 -- 172.21.15.82:0/3060234602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab4103170 msgr2=0x7f5ab4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5ab3fff640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab4103170 0x7f5ab4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5ab3fff640 1 -- 172.21.15.82:0/3060234602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab419fd50 con 0x7f5ab4103b30 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5ab2ffd640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab4103170 0x7f5ab4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5ab37fe640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab4101f80 0x7f5ab4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5ab3fff640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab4103b30 0x7f5ab419f6e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5aac02c700 tx=0x7f5aac034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5ab0ff9640 1 -- 172.21.15.82:0/3060234602 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac039040 con 0x7f5ab4103b30 2024-04-21T08:47:03.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5aba175640 1 -- 172.21.15.82:0/3060234602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab419ffe0 con 0x7f5ab4103b30 2024-04-21T08:47:03.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5ab0ff9640 1 -- 172.21.15.82:0/3060234602 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aac0325c0 con 0x7f5ab4103b30 2024-04-21T08:47:03.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5ab0ff9640 1 -- 172.21.15.82:0/3060234602 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac0042a0 con 0x7f5ab4103b30 2024-04-21T08:47:03.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.447+0000 7f5aba175640 1 -- 172.21.15.82:0/3060234602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab41a0440 con 0x7f5ab4103b30 2024-04-21T08:47:03.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.448+0000 7f5aba175640 1 -- 172.21.15.82:0/3060234602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab4109310 con 0x7f5ab4103b30 2024-04-21T08:47:03.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.449+0000 7f5ab0ff9640 1 -- 172.21.15.82:0/3060234602 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f5aac046030 con 0x7f5ab4103b30 2024-04-21T08:47:03.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.450+0000 7f5ab0ff9640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5a9c0a3db0 0x7f5a9c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:03.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.450+0000 7f5ab37fe640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5a9c0a3db0 0x7f5a9c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:03.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.450+0000 7f5ab0ff9640 1 -- 172.21.15.82:0/3060234602 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f5aac0f2b70 con 0x7f5ab4103b30 2024-04-21T08:47:03.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.450+0000 7f5ab37fe640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5a9c0a3db0 0x7f5a9c0a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4102fd0 tx=0x7f5aa400b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:03.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.453+0000 7f5ab0ff9640 1 -- 172.21.15.82:0/3060234602 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5aac0bee80 con 0x7f5ab4103b30 2024-04-21T08:47:03.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.643+0000 7f5aba175640 1 -- 172.21.15.82:0/3060234602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ab402d790 con 0x7f5ab4103b30 2024-04-21T08:47:03.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.645+0000 7f5ab0ff9640 1 -- 172.21.15.82:0/3060234602 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25247 (secure 0 0 0) 0x7f5aac0be780 con 0x7f5ab4103b30 2024-04-21T08:47:03.644 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:03.645 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:47:00.617381+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2119539425":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6800/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/539366680":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/2275507980":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2883183315":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:47:03.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.647+0000 7f5aba175640 1 -- 172.21.15.82:0/3060234602 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5a9c0a3db0 msgr2=0x7f5a9c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.647+0000 7f5aba175640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5a9c0a3db0 0x7f5a9c0a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4102fd0 tx=0x7f5aa400b390 comp rx=0 tx=0).stop 2024-04-21T08:47:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.647+0000 7f5aba175640 1 -- 172.21.15.82:0/3060234602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab4103b30 msgr2=0x7f5ab419f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.647+0000 7f5aba175640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab4103b30 0x7f5ab419f6e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5aac02c700 tx=0x7f5aac034760 comp rx=0 tx=0).stop 2024-04-21T08:47:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.647+0000 7f5aba175640 1 -- 172.21.15.82:0/3060234602 shutdown_connections 2024-04-21T08:47:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.647+0000 7f5aba175640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5a9c0a3db0 0x7f5a9c0a6270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.647+0000 7f5aba175640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ab4103b30 0x7f5ab419f6e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.647+0000 7f5aba175640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ab4103170 0x7f5ab4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.647+0000 7f5aba175640 1 --2- 172.21.15.82:0/3060234602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab4101f80 0x7f5ab4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.647+0000 7f5aba175640 1 -- 172.21.15.82:0/3060234602 >> 172.21.15.82:0/3060234602 conn(0x7f5ab40fd530 msgr2=0x7f5ab41064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.647+0000 7f5aba175640 1 -- 172.21.15.82:0/3060234602 shutdown_connections 2024-04-21T08:47:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.647+0000 7f5aba175640 1 -- 172.21.15.82:0/3060234602 wait complete. 2024-04-21T08:47:03.658 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:47:03.659 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:47:03.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.763+0000 7fdd92c32640 1 -- 172.21.15.82:0/3398566075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd8c101fa0 msgr2=0x7fdd8c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.763+0000 7fdd92c32640 1 --2- 172.21.15.82:0/3398566075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd8c101fa0 0x7fdd8c1023c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009900 tx=0x7fdd8002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.763+0000 7fdd92c32640 1 -- 172.21.15.82:0/3398566075 shutdown_connections 2024-04-21T08:47:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.763+0000 7fdd92c32640 1 --2- 172.21.15.82:0/3398566075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd8c103b50 0x7fdd8c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.763+0000 7fdd92c32640 1 --2- 172.21.15.82:0/3398566075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd8c103190 0x7fdd8c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.763+0000 7fdd92c32640 1 --2- 172.21.15.82:0/3398566075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd8c101fa0 0x7fdd8c1023c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.763+0000 7fdd92c32640 1 -- 172.21.15.82:0/3398566075 >> 172.21.15.82:0/3398566075 conn(0x7fdd8c0fd530 msgr2=0x7fdd8c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.763+0000 7fdd92c32640 1 -- 172.21.15.82:0/3398566075 shutdown_connections 2024-04-21T08:47:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.763+0000 7fdd92c32640 1 -- 172.21.15.82:0/3398566075 wait complete. 2024-04-21T08:47:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.763+0000 7fdd92c32640 1 Processor -- start 2024-04-21T08:47:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.763+0000 7fdd92c32640 1 -- start start 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd92c32640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd8c101fa0 0x7fdd8c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd92c32640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd8c103190 0x7fdd8c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd92c32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd8c103b50 0x7fdd8c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd92c32640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10dae0 con 0x7fdd8c101fa0 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd92c32640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10d960 con 0x7fdd8c103190 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd92c32640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10dc60 con 0x7fdd8c103b50 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd909ce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd8c101fa0 0x7fdd8c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd909ce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd8c101fa0 0x7fdd8c1981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44238/0 (socket says 172.21.15.82:44238) 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd909ce640 1 -- 172.21.15.82:0/2220971817 learned_addr learned my addr 172.21.15.82:0/2220971817 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd8bfff640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd8c103190 0x7fdd8c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd911cf640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd8c103b50 0x7fdd8c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd909ce640 1 -- 172.21.15.82:0/2220971817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd8c103b50 msgr2=0x7fdd8c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd909ce640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd8c103b50 0x7fdd8c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd909ce640 1 -- 172.21.15.82:0/2220971817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd8c103190 msgr2=0x7fdd8c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd909ce640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd8c103190 0x7fdd8c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd909ce640 1 -- 172.21.15.82:0/2220971817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c19fe00 con 0x7fdd8c101fa0 2024-04-21T08:47:03.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd911cf640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd8c103b50 0x7fdd8c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:03.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd8bfff640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd8c103190 0x7fdd8c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:03.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd909ce640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd8c101fa0 0x7fdd8c1981e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdd80000c00 tx=0x7fdd80004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:03.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.764+0000 7fdd89ffb640 1 -- 172.21.15.82:0/2220971817 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80043020 con 0x7fdd8c101fa0 2024-04-21T08:47:03.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.765+0000 7fdd92c32640 1 -- 172.21.15.82:0/2220971817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8c1a0090 con 0x7fdd8c101fa0 2024-04-21T08:47:03.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.765+0000 7fdd89ffb640 1 -- 172.21.15.82:0/2220971817 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd80004440 con 0x7fdd8c101fa0 2024-04-21T08:47:03.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.765+0000 7fdd89ffb640 1 -- 172.21.15.82:0/2220971817 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80003c90 con 0x7fdd8c101fa0 2024-04-21T08:47:03.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.765+0000 7fdd92c32640 1 -- 172.21.15.82:0/2220971817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8c1a0630 con 0x7fdd8c101fa0 2024-04-21T08:47:03.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.766+0000 7fdd92c32640 1 -- 172.21.15.82:0/2220971817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd8c1093e0 con 0x7fdd8c101fa0 2024-04-21T08:47:03.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.768+0000 7fdd89ffb640 1 -- 172.21.15.82:0/2220971817 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fdd80046020 con 0x7fdd8c101fa0 2024-04-21T08:47:03.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.768+0000 7fdd89ffb640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdd7c0a3db0 0x7fdd7c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:03.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.768+0000 7fdd89ffb640 1 -- 172.21.15.82:0/2220971817 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fdd800f3300 con 0x7fdd8c101fa0 2024-04-21T08:47:03.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.768+0000 7fdd8bfff640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdd7c0a3db0 0x7fdd7c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:03.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.769+0000 7fdd8bfff640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdd7c0a3db0 0x7fdd7c0a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c1996f0 tx=0x7fdd7800b410 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:03.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.771+0000 7fdd89ffb640 1 -- 172.21.15.82:0/2220971817 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd8004d020 con 0x7fdd8c101fa0 2024-04-21T08:47:03.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.957+0000 7fdd92c32640 1 -- 172.21.15.82:0/2220971817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdd8c062b50 con 0x7fdd8c101fa0 2024-04-21T08:47:03.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.957+0000 7fdd89ffb640 1 -- 172.21.15.82:0/2220971817 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fdd800bf580 con 0x7fdd8c101fa0 2024-04-21T08:47:03.957 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:47:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.959+0000 7fdd92c32640 1 -- 172.21.15.82:0/2220971817 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdd7c0a3db0 msgr2=0x7fdd7c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.959+0000 7fdd92c32640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdd7c0a3db0 0x7fdd7c0a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c1996f0 tx=0x7fdd7800b410 comp rx=0 tx=0).stop 2024-04-21T08:47:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.959+0000 7fdd92c32640 1 -- 172.21.15.82:0/2220971817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd8c101fa0 msgr2=0x7fdd8c1981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.959+0000 7fdd92c32640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd8c101fa0 0x7fdd8c1981e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdd80000c00 tx=0x7fdd80004040 comp rx=0 tx=0).stop 2024-04-21T08:47:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.959+0000 7fdd92c32640 1 -- 172.21.15.82:0/2220971817 shutdown_connections 2024-04-21T08:47:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.959+0000 7fdd92c32640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd8c103b50 0x7fdd8c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.959+0000 7fdd92c32640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd8c103190 0x7fdd8c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.959+0000 7fdd92c32640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdd7c0a3db0 0x7fdd7c0a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.959+0000 7fdd92c32640 1 --2- 172.21.15.82:0/2220971817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd8c101fa0 0x7fdd8c1981e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.959+0000 7fdd92c32640 1 -- 172.21.15.82:0/2220971817 >> 172.21.15.82:0/2220971817 conn(0x7fdd8c0fd530 msgr2=0x7fdd8c1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.959+0000 7fdd92c32640 1 -- 172.21.15.82:0/2220971817 shutdown_connections 2024-04-21T08:47:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:03.959+0000 7fdd92c32640 1 -- 172.21.15.82:0/2220971817 wait complete. 2024-04-21T08:47:03.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:03 smithi082 ceph-mon[27967]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:47:03.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:03 smithi082 ceph-mon[27967]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-21T08:47:03.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:03 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3975814241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:03.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:03 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:03.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:03 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:03.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:03 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:03.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:03 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:03.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:03 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:03.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:03 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:03.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:03 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/573367850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:03.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:03 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1377019124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:03.971 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:47:03.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:03 smithi134 ceph-mon[31772]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:47:03.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:03 smithi134 ceph-mon[31772]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-21T08:47:03.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:03 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3975814241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:03.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:03 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:03.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:03 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:03.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:03 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:03.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:03 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:03.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:03 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:03.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:03 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:03.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:03 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/573367850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:03.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:03 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1377019124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.077+0000 7f3d86fd5640 1 -- 172.21.15.82:0/114827466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d80103190 msgr2=0x7f3d80103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.077+0000 7f3d86fd5640 1 --2- 172.21.15.82:0/114827466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d80103190 0x7f3d80103610 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009900 tx=0x7f3d6c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.077+0000 7f3d86fd5640 1 -- 172.21.15.82:0/114827466 shutdown_connections 2024-04-21T08:47:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.077+0000 7f3d86fd5640 1 --2- 172.21.15.82:0/114827466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d80103b50 0x7f3d8010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.077+0000 7f3d86fd5640 1 --2- 172.21.15.82:0/114827466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d80103190 0x7f3d80103610 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.077+0000 7f3d86fd5640 1 --2- 172.21.15.82:0/114827466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d80101fa0 0x7f3d801023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.077+0000 7f3d86fd5640 1 -- 172.21.15.82:0/114827466 >> 172.21.15.82:0/114827466 conn(0x7f3d800fd530 msgr2=0x7f3d800ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.077+0000 7f3d86fd5640 1 -- 172.21.15.82:0/114827466 shutdown_connections 2024-04-21T08:47:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.077+0000 7f3d86fd5640 1 -- 172.21.15.82:0/114827466 wait complete. 2024-04-21T08:47:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d86fd5640 1 Processor -- start 2024-04-21T08:47:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d86fd5640 1 -- start start 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d86fd5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d80101fa0 0x7f3d80198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d86fd5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d80103190 0x7f3d80198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d86fd5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d80103b50 0x7f3d8019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d86fd5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8010dae0 con 0x7f3d80103190 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d86fd5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8010d960 con 0x7f3d80103b50 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d86fd5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8010dc60 con 0x7f3d80101fa0 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d7ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d80103190 0x7f3d80198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d7ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d80103190 0x7f3d80198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44252/0 (socket says 172.21.15.82:44252) 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d7ffff640 1 -- 172.21.15.82:0/1906355602 learned_addr learned my addr 172.21.15.82:0/1906355602 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d84d71640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d80101fa0 0x7f3d80198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.078+0000 7f3d85572640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d80103b50 0x7f3d8019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.079+0000 7f3d7ffff640 1 -- 172.21.15.82:0/1906355602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d80101fa0 msgr2=0x7f3d80198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.079+0000 7f3d7ffff640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d80101fa0 0x7f3d80198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.079+0000 7f3d7ffff640 1 -- 172.21.15.82:0/1906355602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d80103b50 msgr2=0x7f3d8019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.079+0000 7f3d7ffff640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d80103b50 0x7f3d8019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.079+0000 7f3d7ffff640 1 -- 172.21.15.82:0/1906355602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8019fd60 con 0x7f3d80103190 2024-04-21T08:47:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.079+0000 7f3d84d71640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d80101fa0 0x7f3d80198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.079+0000 7f3d7ffff640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d80103190 0x7f3d80198680 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c006010 tx=0x7f3d6c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.079+0000 7f3d7dffb640 1 -- 172.21.15.82:0/1906355602 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c043020 con 0x7f3d80103190 2024-04-21T08:47:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.079+0000 7f3d86fd5640 1 -- 172.21.15.82:0/1906355602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8019fff0 con 0x7f3d80103190 2024-04-21T08:47:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.079+0000 7f3d7dffb640 1 -- 172.21.15.82:0/1906355602 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d6c004440 con 0x7f3d80103190 2024-04-21T08:47:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.079+0000 7f3d7dffb640 1 -- 172.21.15.82:0/1906355602 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c003bf0 con 0x7f3d80103190 2024-04-21T08:47:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.079+0000 7f3d86fd5640 1 -- 172.21.15.82:0/1906355602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d801a0500 con 0x7f3d80103190 2024-04-21T08:47:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.080+0000 7f3d86fd5640 1 -- 172.21.15.82:0/1906355602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d80109340 con 0x7f3d80103190 2024-04-21T08:47:04.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.081+0000 7f3d7dffb640 1 -- 172.21.15.82:0/1906355602 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f3d6c046020 con 0x7f3d80103190 2024-04-21T08:47:04.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.082+0000 7f3d7dffb640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3d700a3db0 0x7f3d700a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:04.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.082+0000 7f3d7dffb640 1 -- 172.21.15.82:0/1906355602 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f3d6c0f34f0 con 0x7f3d80103190 2024-04-21T08:47:04.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.082+0000 7f3d84d71640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3d700a3db0 0x7f3d700a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:04.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.082+0000 7f3d84d71640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3d700a3db0 0x7f3d700a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d80102ff0 tx=0x7f3d74005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:04.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.086+0000 7f3d7dffb640 1 -- 172.21.15.82:0/1906355602 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d6c04d020 con 0x7f3d80103190 2024-04-21T08:47:04.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:03 smithi106 ceph-mon[31072]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:47:04.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:03 smithi106 ceph-mon[31072]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-21T08:47:04.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:03 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3975814241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:04.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:03 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:03 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:03 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:03 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:03 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:03 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:03 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/573367850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:04.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:03 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1377019124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:04.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.287+0000 7f3d86fd5640 1 -- 172.21.15.82:0/1906355602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3d8002d810 con 0x7f3d80103190 2024-04-21T08:47:04.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.288+0000 7f3d7dffb640 1 -- 172.21.15.82:0/1906355602 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f3d6c0bf770 con 0x7f3d80103190 2024-04-21T08:47:04.288 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:47:04.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.290+0000 7f3d86fd5640 1 -- 172.21.15.82:0/1906355602 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3d700a3db0 msgr2=0x7f3d700a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.290+0000 7f3d86fd5640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3d700a3db0 0x7f3d700a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d80102ff0 tx=0x7f3d74005f90 comp rx=0 tx=0).stop 2024-04-21T08:47:04.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.290+0000 7f3d86fd5640 1 -- 172.21.15.82:0/1906355602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d80103190 msgr2=0x7f3d80198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.290+0000 7f3d86fd5640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d80103190 0x7f3d80198680 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c006010 tx=0x7f3d6c004040 comp rx=0 tx=0).stop 2024-04-21T08:47:04.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.290+0000 7f3d86fd5640 1 -- 172.21.15.82:0/1906355602 shutdown_connections 2024-04-21T08:47:04.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.290+0000 7f3d86fd5640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d80103b50 0x7f3d8019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.290+0000 7f3d86fd5640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3d700a3db0 0x7f3d700a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.290+0000 7f3d86fd5640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d80103190 0x7f3d80198680 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.290+0000 7f3d86fd5640 1 --2- 172.21.15.82:0/1906355602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d80101fa0 0x7f3d80198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.290+0000 7f3d86fd5640 1 -- 172.21.15.82:0/1906355602 >> 172.21.15.82:0/1906355602 conn(0x7f3d800fd530 msgr2=0x7f3d80106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:04.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.290+0000 7f3d86fd5640 1 -- 172.21.15.82:0/1906355602 shutdown_connections 2024-04-21T08:47:04.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.290+0000 7f3d86fd5640 1 -- 172.21.15.82:0/1906355602 wait complete. 2024-04-21T08:47:04.301 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:47:04.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.405+0000 7f86cadce640 1 -- 172.21.15.82:0/1015711748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c41009a0 msgr2=0x7f86c410a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.405+0000 7f86cadce640 1 --2- 172.21.15.82:0/1015711748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c41009a0 0x7f86c410a550 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009ca0 tx=0x7f86bc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.405+0000 7f86cadce640 1 -- 172.21.15.82:0/1015711748 shutdown_connections 2024-04-21T08:47:04.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.405+0000 7f86cadce640 1 --2- 172.21.15.82:0/1015711748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c41009a0 0x7f86c410a550 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.405+0000 7f86cadce640 1 --2- 172.21.15.82:0/1015711748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86c40fffe0 0x7f86c4100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.405+0000 7f86cadce640 1 --2- 172.21.15.82:0/1015711748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86c4103570 0x7f86c4103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.405+0000 7f86cadce640 1 -- 172.21.15.82:0/1015711748 >> 172.21.15.82:0/1015711748 conn(0x7f86c40fbb80 msgr2=0x7f86c40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.405+0000 7f86cadce640 1 -- 172.21.15.82:0/1015711748 shutdown_connections 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.405+0000 7f86cadce640 1 -- 172.21.15.82:0/1015711748 wait complete. 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.406+0000 7f86cadce640 1 Processor -- start 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.406+0000 7f86cadce640 1 -- start start 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.406+0000 7f86cadce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86c40fffe0 0x7f86c4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.406+0000 7f86cadce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c41009a0 0x7f86c41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.406+0000 7f86cadce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86c4103570 0x7f86c419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.406+0000 7f86cadce640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f86c410dac0 con 0x7f86c41009a0 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.406+0000 7f86cadce640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f86c410d940 con 0x7f86c40fffe0 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.406+0000 7f86cadce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86c410dc40 con 0x7f86c4103570 2024-04-21T08:47:04.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.406+0000 7f86c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c41009a0 0x7f86c41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.406+0000 7f86c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c41009a0 0x7f86c41986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44274/0 (socket says 172.21.15.82:44274) 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.406+0000 7f86c3fff640 1 -- 172.21.15.82:0/3971159137 learned_addr learned my addr 172.21.15.82:0/3971159137 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.406+0000 7f86c8b6a640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86c40fffe0 0x7f86c4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.407+0000 7f86c936b640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86c4103570 0x7f86c419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.407+0000 7f86c3fff640 1 -- 172.21.15.82:0/3971159137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86c4103570 msgr2=0x7f86c419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.407+0000 7f86c3fff640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86c4103570 0x7f86c419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.407+0000 7f86c3fff640 1 -- 172.21.15.82:0/3971159137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86c40fffe0 msgr2=0x7f86c4198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.407+0000 7f86c3fff640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86c40fffe0 0x7f86c4198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.407+0000 7f86c3fff640 1 -- 172.21.15.82:0/3971159137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c419fda0 con 0x7f86c41009a0 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.407+0000 7f86c3fff640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c41009a0 0x7f86c41986c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f86b000d950 tx=0x7f86b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.407+0000 7f86c1ffb640 1 -- 172.21.15.82:0/3971159137 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0017070 con 0x7f86c41009a0 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.407+0000 7f86cadce640 1 -- 172.21.15.82:0/3971159137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c41a0090 con 0x7f86c41009a0 2024-04-21T08:47:04.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.407+0000 7f86c1ffb640 1 -- 172.21.15.82:0/3971159137 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86b0009c30 con 0x7f86c41009a0 2024-04-21T08:47:04.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.407+0000 7f86c1ffb640 1 -- 172.21.15.82:0/3971159137 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b000f620 con 0x7f86c41009a0 2024-04-21T08:47:04.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.407+0000 7f86cadce640 1 -- 172.21.15.82:0/3971159137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c41a0630 con 0x7f86c41009a0 2024-04-21T08:47:04.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.408+0000 7f86cadce640 1 -- 172.21.15.82:0/3971159137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c41094b0 con 0x7f86c41009a0 2024-04-21T08:47:04.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.409+0000 7f86c1ffb640 1 -- 172.21.15.82:0/3971159137 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f86b0010460 con 0x7f86c41009a0 2024-04-21T08:47:04.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.410+0000 7f86c1ffb640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f86b40a3db0 0x7f86b40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:04.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.410+0000 7f86c8b6a640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f86b40a3db0 0x7f86b40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:04.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.410+0000 7f86c1ffb640 1 -- 172.21.15.82:0/3971159137 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f86b00d08d0 con 0x7f86c41009a0 2024-04-21T08:47:04.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.410+0000 7f86c8b6a640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f86b40a3db0 0x7f86b40a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f86c4068e00 tx=0x7f86b8005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:04.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.414+0000 7f86c1ffb640 1 -- 172.21.15.82:0/3971159137 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f86b0009de0 con 0x7f86c41009a0 2024-04-21T08:47:04.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.616+0000 7f86cadce640 1 -- 172.21.15.82:0/3971159137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f86c402d7e0 con 0x7f86c41009a0 2024-04-21T08:47:04.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.617+0000 7f86c1ffb640 1 -- 172.21.15.82:0/3971159137 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f86b009c3f0 con 0x7f86c41009a0 2024-04-21T08:47:04.616 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:47:04.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.619+0000 7f86cadce640 1 -- 172.21.15.82:0/3971159137 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f86b40a3db0 msgr2=0x7f86b40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.619+0000 7f86cadce640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f86b40a3db0 0x7f86b40a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f86c4068e00 tx=0x7f86b8005f90 comp rx=0 tx=0).stop 2024-04-21T08:47:04.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.619+0000 7f86cadce640 1 -- 172.21.15.82:0/3971159137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c41009a0 msgr2=0x7f86c41986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.619+0000 7f86cadce640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c41009a0 0x7f86c41986c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f86b000d950 tx=0x7f86b000de10 comp rx=0 tx=0).stop 2024-04-21T08:47:04.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.619+0000 7f86cadce640 1 -- 172.21.15.82:0/3971159137 shutdown_connections 2024-04-21T08:47:04.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.619+0000 7f86cadce640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86c4103570 0x7f86c419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.619+0000 7f86cadce640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f86b40a3db0 0x7f86b40a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.619+0000 7f86cadce640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c41009a0 0x7f86c41986c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.619+0000 7f86cadce640 1 --2- 172.21.15.82:0/3971159137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86c40fffe0 0x7f86c4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.619+0000 7f86cadce640 1 -- 172.21.15.82:0/3971159137 >> 172.21.15.82:0/3971159137 conn(0x7f86c40fbb80 msgr2=0x7f86c4106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:04.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.619+0000 7f86cadce640 1 -- 172.21.15.82:0/3971159137 shutdown_connections 2024-04-21T08:47:04.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.619+0000 7f86cadce640 1 -- 172.21.15.82:0/3971159137 wait complete. 2024-04-21T08:47:04.630 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:47:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.750+0000 7f92ad7f4640 1 -- 172.21.15.82:0/4009526761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a8103190 msgr2=0x7f92a8103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.750+0000 7f92ad7f4640 1 --2- 172.21.15.82:0/4009526761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a8103190 0x7f92a8103610 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f929c009900 tx=0x7f929c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.750+0000 7f92ad7f4640 1 -- 172.21.15.82:0/4009526761 shutdown_connections 2024-04-21T08:47:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.750+0000 7f92ad7f4640 1 --2- 172.21.15.82:0/4009526761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92a8103b50 0x7f92a810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.750+0000 7f92ad7f4640 1 --2- 172.21.15.82:0/4009526761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a8103190 0x7f92a8103610 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.750+0000 7f92ad7f4640 1 --2- 172.21.15.82:0/4009526761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92a8101fa0 0x7f92a81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.750+0000 7f92ad7f4640 1 -- 172.21.15.82:0/4009526761 >> 172.21.15.82:0/4009526761 conn(0x7f92a80fd530 msgr2=0x7f92a80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.750+0000 7f92ad7f4640 1 -- 172.21.15.82:0/4009526761 shutdown_connections 2024-04-21T08:47:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.750+0000 7f92ad7f4640 1 -- 172.21.15.82:0/4009526761 wait complete. 2024-04-21T08:47:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.750+0000 7f92ad7f4640 1 Processor -- start 2024-04-21T08:47:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92ad7f4640 1 -- start start 2024-04-21T08:47:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92ad7f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92a8101fa0 0x7f92a8198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92ad7f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a8103190 0x7f92a8198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92ad7f4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92a8103b50 0x7f92a819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92ad7f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f92a810db20 con 0x7f92a8103190 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92ad7f4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f92a810d9a0 con 0x7f92a8103b50 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92ad7f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92a810dca0 con 0x7f92a8101fa0 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92a67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a8103190 0x7f92a8198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92a67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a8103190 0x7f92a8198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44300/0 (socket says 172.21.15.82:44300) 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92a67fc640 1 -- 172.21.15.82:0/155889491 learned_addr learned my addr 172.21.15.82:0/155889491 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92a6ffd640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92a8101fa0 0x7f92a8198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92a77fe640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92a8103b50 0x7f92a819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92a67fc640 1 -- 172.21.15.82:0/155889491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92a8101fa0 msgr2=0x7f92a8198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92a67fc640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92a8101fa0 0x7f92a8198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92a67fc640 1 -- 172.21.15.82:0/155889491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92a8103b50 msgr2=0x7f92a819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92a67fc640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92a8103b50 0x7f92a819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92a67fc640 1 -- 172.21.15.82:0/155889491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a819fe60 con 0x7f92a8103190 2024-04-21T08:47:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.751+0000 7f92a67fc640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a8103190 0x7f92a8198780 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f929c006010 tx=0x7f929c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.752+0000 7f928ffff640 1 -- 172.21.15.82:0/155889491 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c043020 con 0x7f92a8103190 2024-04-21T08:47:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.752+0000 7f928ffff640 1 -- 172.21.15.82:0/155889491 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f929c004440 con 0x7f92a8103190 2024-04-21T08:47:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.752+0000 7f92ad7f4640 1 -- 172.21.15.82:0/155889491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a81a00f0 con 0x7f92a8103190 2024-04-21T08:47:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.752+0000 7f928ffff640 1 -- 172.21.15.82:0/155889491 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c003bf0 con 0x7f92a8103190 2024-04-21T08:47:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.752+0000 7f92ad7f4640 1 -- 172.21.15.82:0/155889491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a81a06c0 con 0x7f92a8103190 2024-04-21T08:47:04.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.753+0000 7f92ad7f4640 1 -- 172.21.15.82:0/155889491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a8109390 con 0x7f92a8103190 2024-04-21T08:47:04.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.754+0000 7f928ffff640 1 -- 172.21.15.82:0/155889491 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f929c046020 con 0x7f92a8103190 2024-04-21T08:47:04.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.754+0000 7f928ffff640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f92900a3db0 0x7f92900a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:04.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.754+0000 7f928ffff640 1 -- 172.21.15.82:0/155889491 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f929c0f2f70 con 0x7f92a8103190 2024-04-21T08:47:04.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.754+0000 7f92a6ffd640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f92900a3db0 0x7f92900a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:04.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.755+0000 7f92a6ffd640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f92900a3db0 0x7f92900a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f92a8102ff0 tx=0x7f9298005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:04.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.759+0000 7f928ffff640 1 -- 172.21.15.82:0/155889491 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f929c04d020 con 0x7f92a8103190 2024-04-21T08:47:04.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3060234602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:04.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi134 to 176.2M 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: Unable to set osd_memory_target on smithi134 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-21T08:47:04.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: Unable to set osd_memory_target on smithi106 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2220971817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi082 to 176.2M 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: Unable to set osd_memory_target on smithi082 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-21T08:47:04.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-21T08:47:04.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-21T08:47:04.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1906355602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:04.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: Updating smithi106:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:47:04.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: Updating smithi082:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:47:04.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: Updating smithi134:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:47:04.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:04 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3971159137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:04.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.986+0000 7f92ad7f4640 1 -- 172.21.15.82:0/155889491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f92a802d770 con 0x7f92a8103190 2024-04-21T08:47:04.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.987+0000 7f928ffff640 1 -- 172.21.15.82:0/155889491 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f929c0bf1f0 con 0x7f92a8103190 2024-04-21T08:47:04.986 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:47:04.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.989+0000 7f92ad7f4640 1 -- 172.21.15.82:0/155889491 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f92900a3db0 msgr2=0x7f92900a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.989+0000 7f92ad7f4640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f92900a3db0 0x7f92900a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f92a8102ff0 tx=0x7f9298005f90 comp rx=0 tx=0).stop 2024-04-21T08:47:04.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.989+0000 7f92ad7f4640 1 -- 172.21.15.82:0/155889491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a8103190 msgr2=0x7f92a8198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:04.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.989+0000 7f92ad7f4640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a8103190 0x7f92a8198780 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f929c006010 tx=0x7f929c004040 comp rx=0 tx=0).stop 2024-04-21T08:47:04.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.989+0000 7f92ad7f4640 1 -- 172.21.15.82:0/155889491 shutdown_connections 2024-04-21T08:47:04.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.989+0000 7f92ad7f4640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92a8103b50 0x7f92a819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.989+0000 7f92ad7f4640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f92900a3db0 0x7f92900a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.989+0000 7f92ad7f4640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92a8103190 0x7f92a8198780 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.989+0000 7f92ad7f4640 1 --2- 172.21.15.82:0/155889491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92a8101fa0 0x7f92a8198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:04.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.989+0000 7f92ad7f4640 1 -- 172.21.15.82:0/155889491 >> 172.21.15.82:0/155889491 conn(0x7f92a80fd530 msgr2=0x7f92a8106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:04.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.989+0000 7f92ad7f4640 1 -- 172.21.15.82:0/155889491 shutdown_connections 2024-04-21T08:47:04.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:04.989+0000 7f92ad7f4640 1 -- 172.21.15.82:0/155889491 wait complete. 2024-04-21T08:47:04.999 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3060234602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:04.999 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:04.999 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi134 to 176.2M 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: Unable to set osd_memory_target on smithi134 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: Unable to set osd_memory_target on smithi106 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-21T08:47:05.000 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2220971817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:05.001 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.001 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.001 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.001 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.001 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.001 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.001 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi082 to 176.2M 2024-04-21T08:47:05.001 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: Unable to set osd_memory_target on smithi082 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-21T08:47:05.002 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-21T08:47:05.002 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:47:05.003 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:47:05.003 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:47:05.003 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-21T08:47:05.003 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-21T08:47:05.003 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-21T08:47:05.003 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1906355602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:05.003 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: Updating smithi106:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:47:05.003 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: Updating smithi082:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:47:05.003 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: Updating smithi134:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:47:05.003 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:04 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3971159137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:05.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.104+0000 7f41e1e14640 1 -- 172.21.15.82:0/1384446592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41dc1010b0 msgr2=0x7f41dc101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.104+0000 7f41e1e14640 1 --2- 172.21.15.82:0/1384446592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41dc1010b0 0x7f41dc101490 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009900 tx=0x7f41cc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:05.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.104+0000 7f41e1e14640 1 -- 172.21.15.82:0/1384446592 shutdown_connections 2024-04-21T08:47:05.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.104+0000 7f41e1e14640 1 --2- 172.21.15.82:0/1384446592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41dc106180 0x7f41dc10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.104+0000 7f41e1e14640 1 --2- 172.21.15.82:0/1384446592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41dc101a60 0x7f41dc105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.104+0000 7f41e1e14640 1 --2- 172.21.15.82:0/1384446592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41dc1010b0 0x7f41dc101490 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.104+0000 7f41e1e14640 1 -- 172.21.15.82:0/1384446592 >> 172.21.15.82:0/1384446592 conn(0x7f41dc0771d0 msgr2=0x7f41dc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.104+0000 7f41e1e14640 1 -- 172.21.15.82:0/1384446592 shutdown_connections 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.104+0000 7f41e1e14640 1 -- 172.21.15.82:0/1384446592 wait complete. 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41e1e14640 1 Processor -- start 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41e1e14640 1 -- start start 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41e1e14640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41dc1010b0 0x7f41dc197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41e1e14640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41dc106180 0x7f41dc1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41e1e14640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41dc197df0 0x7f41dc1a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41e1e14640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc1110c0 con 0x7f41dc1010b0 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41e1e14640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc110f40 con 0x7f41dc197df0 2024-04-21T08:47:05.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41e1e14640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc111240 con 0x7f41dc106180 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41db7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41dc1010b0 0x7f41dc197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41db7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41dc1010b0 0x7f41dc197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44328/0 (socket says 172.21.15.82:44328) 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41db7fe640 1 -- 172.21.15.82:0/1880972080 learned_addr learned my addr 172.21.15.82:0/1880972080 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41dbfff640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41dc197df0 0x7f41dc1a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.105+0000 7f41daffd640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41dc106180 0x7f41dc1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.106+0000 7f41db7fe640 1 -- 172.21.15.82:0/1880972080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41dc106180 msgr2=0x7f41dc1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.106+0000 7f41db7fe640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41dc106180 0x7f41dc1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.106+0000 7f41db7fe640 1 -- 172.21.15.82:0/1880972080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41dc197df0 msgr2=0x7f41dc1a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.106+0000 7f41db7fe640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41dc197df0 0x7f41dc1a69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.106+0000 7f41db7fe640 1 -- 172.21.15.82:0/1880972080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc1a6fb0 con 0x7f41dc1010b0 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.106+0000 7f41db7fe640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41dc1010b0 0x7f41dc197370 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f41cc006010 tx=0x7f41cc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.106+0000 7f41d8ff9640 1 -- 172.21.15.82:0/1880972080 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc043020 con 0x7f41dc1010b0 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.106+0000 7f41e1e14640 1 -- 172.21.15.82:0/1880972080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41dc1a7240 con 0x7f41dc1010b0 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.106+0000 7f41d8ff9640 1 -- 172.21.15.82:0/1880972080 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41cc02caf0 con 0x7f41dc1010b0 2024-04-21T08:47:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.106+0000 7f41d8ff9640 1 -- 172.21.15.82:0/1880972080 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc003af0 con 0x7f41dc1010b0 2024-04-21T08:47:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.106+0000 7f41e1e14640 1 -- 172.21.15.82:0/1880972080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41dc1a7780 con 0x7f41dc1010b0 2024-04-21T08:47:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.107+0000 7f41e1e14640 1 -- 172.21.15.82:0/1880972080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41dc069230 con 0x7f41dc1010b0 2024-04-21T08:47:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.108+0000 7f41d8ff9640 1 -- 172.21.15.82:0/1880972080 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f41cc046020 con 0x7f41dc1010b0 2024-04-21T08:47:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.109+0000 7f41d8ff9640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f41c40a3db0 0x7f41c40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.109+0000 7f41daffd640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f41c40a3db0 0x7f41c40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.109+0000 7f41d8ff9640 1 -- 172.21.15.82:0/1880972080 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f41cc0f2800 con 0x7f41dc1010b0 2024-04-21T08:47:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.110+0000 7f41daffd640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f41c40a3db0 0x7f41c40a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f41dc1994d0 tx=0x7f41d0005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:05.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.114+0000 7f41d8ff9640 1 -- 172.21.15.82:0/1880972080 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41cc04d070 con 0x7f41dc1010b0 2024-04-21T08:47:05.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3060234602' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:05.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi134 to 176.2M 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: Unable to set osd_memory_target on smithi134 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: Unable to set osd_memory_target on smithi106 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2220971817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi082 to 176.2M 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: Unable to set osd_memory_target on smithi082 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1906355602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:05.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: Updating smithi106:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:47:05.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: Updating smithi082:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:47:05.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: Updating smithi134:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:47:05.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:04 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3971159137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:05.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.161+0000 7f41d8ff9640 1 -- 172.21.15.82:0/1880972080 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f41cc0ba9f0 con 0x7f41dc1010b0 2024-04-21T08:47:05.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.352+0000 7f41e1e14640 1 -- 172.21.15.82:0/1880972080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41dc109410 con 0x7f41dc1010b0 2024-04-21T08:47:05.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.352+0000 7f41d8ff9640 1 -- 172.21.15.82:0/1880972080 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f41cc0bea80 con 0x7f41dc1010b0 2024-04-21T08:47:05.353 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:05.353 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:46:59.334314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:47:05.353 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 28 2024-04-21T08:47:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.354+0000 7f41e1e14640 1 -- 172.21.15.82:0/1880972080 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f41c40a3db0 msgr2=0x7f41c40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.354+0000 7f41e1e14640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f41c40a3db0 0x7f41c40a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f41dc1994d0 tx=0x7f41d0005f90 comp rx=0 tx=0).stop 2024-04-21T08:47:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.354+0000 7f41e1e14640 1 -- 172.21.15.82:0/1880972080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41dc1010b0 msgr2=0x7f41dc197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.354+0000 7f41e1e14640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41dc1010b0 0x7f41dc197370 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f41cc006010 tx=0x7f41cc0045e0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.355+0000 7f41e1e14640 1 -- 172.21.15.82:0/1880972080 shutdown_connections 2024-04-21T08:47:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.355+0000 7f41e1e14640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41dc197df0 0x7f41dc1a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.355+0000 7f41e1e14640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41dc106180 0x7f41dc1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.355+0000 7f41e1e14640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f41c40a3db0 0x7f41c40a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.355+0000 7f41e1e14640 1 --2- 172.21.15.82:0/1880972080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41dc1010b0 0x7f41dc197370 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.355+0000 7f41e1e14640 1 -- 172.21.15.82:0/1880972080 >> 172.21.15.82:0/1880972080 conn(0x7f41dc0771d0 msgr2=0x7f41dc0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.355+0000 7f41e1e14640 1 -- 172.21.15.82:0/1880972080 shutdown_connections 2024-04-21T08:47:05.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.355+0000 7f41e1e14640 1 -- 172.21.15.82:0/1880972080 wait complete. 2024-04-21T08:47:05.366 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-21T08:47:05.366 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-21T08:47:05.367 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:47:05.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.468+0000 7f25feb9d640 1 -- 172.21.15.82:0/2743986685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f81010b0 msgr2=0x7f25f8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.468+0000 7f25feb9d640 1 --2- 172.21.15.82:0/2743986685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f81010b0 0x7f25f8101490 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f25ec009900 tx=0x7f25ec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:05.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.468+0000 7f25feb9d640 1 -- 172.21.15.82:0/2743986685 shutdown_connections 2024-04-21T08:47:05.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.468+0000 7f25feb9d640 1 --2- 172.21.15.82:0/2743986685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25f8106180 0x7f25f810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.468+0000 7f25feb9d640 1 --2- 172.21.15.82:0/2743986685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25f8101a60 0x7f25f8105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.468+0000 7f25feb9d640 1 --2- 172.21.15.82:0/2743986685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f81010b0 0x7f25f8101490 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.468+0000 7f25feb9d640 1 -- 172.21.15.82:0/2743986685 >> 172.21.15.82:0/2743986685 conn(0x7f25f80771d0 msgr2=0x7f25f80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:05.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.468+0000 7f25feb9d640 1 -- 172.21.15.82:0/2743986685 shutdown_connections 2024-04-21T08:47:05.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.468+0000 7f25feb9d640 1 -- 172.21.15.82:0/2743986685 wait complete. 2024-04-21T08:47:05.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25feb9d640 1 Processor -- start 2024-04-21T08:47:05.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25feb9d640 1 -- start start 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25feb9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25f81010b0 0x7f25f81995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25feb9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f8106180 0x7f25f8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25feb9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25f819a070 0x7f25f81a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25feb9d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f25f81110e0 con 0x7f25f8106180 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25feb9d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f25f8110f60 con 0x7f25f81010b0 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25feb9d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f25f8111260 con 0x7f25f819a070 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25f7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f8106180 0x7f25f8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25f7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f8106180 0x7f25f8199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44346/0 (socket says 172.21.15.82:44346) 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25f7fff640 1 -- 172.21.15.82:0/4142643910 learned_addr learned my addr 172.21.15.82:0/4142643910 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25fc939640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25f81010b0 0x7f25f81995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25fd13a640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25f819a070 0x7f25f81a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25f7fff640 1 -- 172.21.15.82:0/4142643910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25f819a070 msgr2=0x7f25f81a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25f7fff640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25f819a070 0x7f25f81a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25f7fff640 1 -- 172.21.15.82:0/4142643910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25f81010b0 msgr2=0x7f25f81995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25f7fff640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25f81010b0 0x7f25f81995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.469+0000 7f25f7fff640 1 -- 172.21.15.82:0/4142643910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f81a4df0 con 0x7f25f8106180 2024-04-21T08:47:05.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.470+0000 7f25f7fff640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f8106180 0x7f25f8199ae0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f25e400e9f0 tx=0x7f25e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:05.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.470+0000 7f25f5ffb640 1 -- 172.21.15.82:0/4142643910 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e400ccc0 con 0x7f25f8106180 2024-04-21T08:47:05.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.470+0000 7f25f5ffb640 1 -- 172.21.15.82:0/4142643910 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25e40054d0 con 0x7f25f8106180 2024-04-21T08:47:05.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.470+0000 7f25f5ffb640 1 -- 172.21.15.82:0/4142643910 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4010430 con 0x7f25f8106180 2024-04-21T08:47:05.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.470+0000 7f25feb9d640 1 -- 172.21.15.82:0/4142643910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f81a50e0 con 0x7f25f8106180 2024-04-21T08:47:05.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.470+0000 7f25feb9d640 1 -- 172.21.15.82:0/4142643910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f81a55e0 con 0x7f25f8106180 2024-04-21T08:47:05.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.471+0000 7f25feb9d640 1 -- 172.21.15.82:0/4142643910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f8069230 con 0x7f25f8106180 2024-04-21T08:47:05.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.472+0000 7f25f5ffb640 1 -- 172.21.15.82:0/4142643910 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f25e40046c0 con 0x7f25f8106180 2024-04-21T08:47:05.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.473+0000 7f25f5ffb640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f25e80a3e40 0x7f25e80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:05.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.473+0000 7f25fc939640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f25e80a3e40 0x7f25e80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:05.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.473+0000 7f25f5ffb640 1 -- 172.21.15.82:0/4142643910 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f25e40d11c0 con 0x7f25f8106180 2024-04-21T08:47:05.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.473+0000 7f25fc939640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f25e80a3e40 0x7f25e80a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f25ec006010 tx=0x7f25ec005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:05.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.477+0000 7f25f5ffb640 1 -- 172.21.15.82:0/4142643910 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25e409d250 con 0x7f25f8106180 2024-04-21T08:47:05.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.657+0000 7f25feb9d640 1 -- 172.21.15.82:0/4142643910 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25f8101a60 con 0x7f25e80a3e40 2024-04-21T08:47:05.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.666+0000 7f25f5ffb640 1 -- 172.21.15.82:0/4142643910 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339428 (secure 0 0 0) 0x7f25f8101a60 con 0x7f25e80a3e40 2024-04-21T08:47:05.670 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:05.672 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-21T08:47:05.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.670+0000 7f25feb9d640 1 -- 172.21.15.82:0/4142643910 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f25e80a3e40 msgr2=0x7f25e80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.670+0000 7f25feb9d640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f25e80a3e40 0x7f25e80a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f25ec006010 tx=0x7f25ec005e40 comp rx=0 tx=0).stop 2024-04-21T08:47:05.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.670+0000 7f25feb9d640 1 -- 172.21.15.82:0/4142643910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f8106180 msgr2=0x7f25f8199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.670+0000 7f25feb9d640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f8106180 0x7f25f8199ae0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f25e400e9f0 tx=0x7f25e400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.670+0000 7f25feb9d640 1 -- 172.21.15.82:0/4142643910 shutdown_connections 2024-04-21T08:47:05.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.670+0000 7f25feb9d640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25f819a070 0x7f25f81a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.670+0000 7f25feb9d640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f25e80a3e40 0x7f25e80a6300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.670+0000 7f25feb9d640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f8106180 0x7f25f8199ae0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.670+0000 7f25feb9d640 1 --2- 172.21.15.82:0/4142643910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f25f81010b0 0x7f25f81995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.670+0000 7f25feb9d640 1 -- 172.21.15.82:0/4142643910 >> 172.21.15.82:0/4142643910 conn(0x7f25f80771d0 msgr2=0x7f25f80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:05.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.670+0000 7f25feb9d640 1 -- 172.21.15.82:0/4142643910 shutdown_connections 2024-04-21T08:47:05.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.670+0000 7f25feb9d640 1 -- 172.21.15.82:0/4142643910 wait complete. 2024-04-21T08:47:05.681 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-21T08:47:04.633865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191531,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1482,"num_read_kb":907,"num_write":555,"num_write_kb":4406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":939,"ondisk_log_size":939,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357300,"kb_used_data":16376,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124584524,"statfs":{"total":1151940427776,"available":1151574552576,"internally_reserved":0,"allocated":16769024,"data_stored":9906153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.985540"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748276+0000","last_change":"2024-04-21T08:44:34.494393+0000","last_active":"2024-04-21T08:47:00.748276+0000","last_peered":"2024-04-21T08:47:00.748276+0000","last_clean":"2024-04-21T08:47:00.748276+0000","last_became_active":"2024-04-21T08:44:34.493583+0000","last_became_peered":"2024-04-21T08:44:34.493583+0000","last_unstale":"2024-04-21T08:47:00.748276+0000","last_undegraded":"2024-04-21T08:47:00.748276+0000","last_fullsized":"2024-04-21T08:47:00.748276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:37.211676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747826+0000","last_change":"2024-04-21T08:44:34.496291+0000","last_active":"2024-04-21T08:47:00.747826+0000","last_peered":"2024-04-21T08:47:00.747826+0000","last_clean":"2024-04-21T08:47:00.747826+0000","last_became_active":"2024-04-21T08:44:34.495019+0000","last_became_peered":"2024-04-21T08:44:34.495019+0000","last_unstale":"2024-04-21T08:47:00.747826+0000","last_undegraded":"2024-04-21T08:47:00.747826+0000","last_fullsized":"2024-04-21T08:47:00.747826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:05.309847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747464+0000","last_change":"2024-04-21T08:44:34.497948+0000","last_active":"2024-04-21T08:47:00.747464+0000","last_peered":"2024-04-21T08:47:00.747464+0000","last_clean":"2024-04-21T08:47:00.747464+0000","last_became_active":"2024-04-21T08:44:34.497657+0000","last_became_peered":"2024-04-21T08:44:34.497657+0000","last_unstale":"2024-04-21T08:47:00.747464+0000","last_undegraded":"2024-04-21T08:47:00.747464+0000","last_fullsized":"2024-04-21T08:47:00.747464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:11.463715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748377+0000","last_change":"2024-04-21T08:44:34.492682+0000","last_active":"2024-04-21T08:47:00.748377+0000","last_peered":"2024-04-21T08:47:00.748377+0000","last_clean":"2024-04-21T08:47:00.748377+0000","last_became_active":"2024-04-21T08:44:34.492230+0000","last_became_peered":"2024-04-21T08:44:34.492230+0000","last_unstale":"2024-04-21T08:47:00.748377+0000","last_undegraded":"2024-04-21T08:47:00.748377+0000","last_fullsized":"2024-04-21T08:47:00.748377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:40.365608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721320+0000","last_change":"2024-04-21T08:44:34.492160+0000","last_active":"2024-04-21T08:47:00.721320+0000","last_peered":"2024-04-21T08:47:00.721320+0000","last_clean":"2024-04-21T08:47:00.721320+0000","last_became_active":"2024-04-21T08:44:34.491701+0000","last_became_peered":"2024-04-21T08:44:34.491701+0000","last_unstale":"2024-04-21T08:47:00.721320+0000","last_undegraded":"2024-04-21T08:47:00.721320+0000","last_fullsized":"2024-04-21T08:47:00.721320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:50.542661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626504+0000","last_change":"2024-04-21T08:44:34.497341+0000","last_active":"2024-04-21T08:47:00.626504+0000","last_peered":"2024-04-21T08:47:00.626504+0000","last_clean":"2024-04-21T08:47:00.626504+0000","last_became_active":"2024-04-21T08:44:34.495794+0000","last_became_peered":"2024-04-21T08:44:34.495794+0000","last_unstale":"2024-04-21T08:47:00.626504+0000","last_undegraded":"2024-04-21T08:47:00.626504+0000","last_fullsized":"2024-04-21T08:47:00.626504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:13.854753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748474+0000","last_change":"2024-04-21T08:44:34.501578+0000","last_active":"2024-04-21T08:47:00.748474+0000","last_peered":"2024-04-21T08:47:00.748474+0000","last_clean":"2024-04-21T08:47:00.748474+0000","last_became_active":"2024-04-21T08:44:34.500848+0000","last_became_peered":"2024-04-21T08:44:34.500848+0000","last_unstale":"2024-04-21T08:47:00.748474+0000","last_undegraded":"2024-04-21T08:47:00.748474+0000","last_fullsized":"2024-04-21T08:47:00.748474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:53:27.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.621951+0000","last_change":"2024-04-21T08:44:34.502975+0000","last_active":"2024-04-21T08:47:00.621951+0000","last_peered":"2024-04-21T08:47:00.621951+0000","last_clean":"2024-04-21T08:47:00.621951+0000","last_became_active":"2024-04-21T08:44:34.502676+0000","last_became_peered":"2024-04-21T08:44:34.502676+0000","last_unstale":"2024-04-21T08:47:00.621951+0000","last_undegraded":"2024-04-21T08:47:00.621951+0000","last_fullsized":"2024-04-21T08:47:00.621951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:32:13.881603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748249+0000","last_change":"2024-04-21T08:44:34.495703+0000","last_active":"2024-04-21T08:47:00.748249+0000","last_peered":"2024-04-21T08:47:00.748249+0000","last_clean":"2024-04-21T08:47:00.748249+0000","last_became_active":"2024-04-21T08:44:34.495380+0000","last_became_peered":"2024-04-21T08:44:34.495380+0000","last_unstale":"2024-04-21T08:47:00.748249+0000","last_undegraded":"2024-04-21T08:47:00.748249+0000","last_fullsized":"2024-04-21T08:47:00.748249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:45:28.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626928+0000","last_change":"2024-04-21T08:44:34.499361+0000","last_active":"2024-04-21T08:47:00.626928+0000","last_peered":"2024-04-21T08:47:00.626928+0000","last_clean":"2024-04-21T08:47:00.626928+0000","last_became_active":"2024-04-21T08:44:34.499044+0000","last_became_peered":"2024-04-21T08:44:34.499044+0000","last_unstale":"2024-04-21T08:47:00.626928+0000","last_undegraded":"2024-04-21T08:47:00.626928+0000","last_fullsized":"2024-04-21T08:47:00.626928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:12.118985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748894+0000","last_change":"2024-04-21T08:44:34.499631+0000","last_active":"2024-04-21T08:47:00.748894+0000","last_peered":"2024-04-21T08:47:00.748894+0000","last_clean":"2024-04-21T08:47:00.748894+0000","last_became_active":"2024-04-21T08:44:34.499316+0000","last_became_peered":"2024-04-21T08:44:34.499316+0000","last_unstale":"2024-04-21T08:47:00.748894+0000","last_undegraded":"2024-04-21T08:47:00.748894+0000","last_fullsized":"2024-04-21T08:47:00.748894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:26.123017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748326+0000","last_change":"2024-04-21T08:44:34.504857+0000","last_active":"2024-04-21T08:47:00.748326+0000","last_peered":"2024-04-21T08:47:00.748326+0000","last_clean":"2024-04-21T08:47:00.748326+0000","last_became_active":"2024-04-21T08:44:34.504564+0000","last_became_peered":"2024-04-21T08:44:34.504564+0000","last_unstale":"2024-04-21T08:47:00.748326+0000","last_undegraded":"2024-04-21T08:47:00.748326+0000","last_fullsized":"2024-04-21T08:47:00.748326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:32:05.216162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874684+0000","last_change":"2024-04-21T08:44:34.484074+0000","last_active":"2024-04-21T08:47:00.874684+0000","last_peered":"2024-04-21T08:47:00.874684+0000","last_clean":"2024-04-21T08:47:00.874684+0000","last_became_active":"2024-04-21T08:44:34.483309+0000","last_became_peered":"2024-04-21T08:44:34.483309+0000","last_unstale":"2024-04-21T08:47:00.874684+0000","last_undegraded":"2024-04-21T08:47:00.874684+0000","last_fullsized":"2024-04-21T08:47:00.874684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:39.686048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747340+0000","last_change":"2024-04-21T08:44:34.499159+0000","last_active":"2024-04-21T08:47:00.747340+0000","last_peered":"2024-04-21T08:47:00.747340+0000","last_clean":"2024-04-21T08:47:00.747340+0000","last_became_active":"2024-04-21T08:44:34.496609+0000","last_became_peered":"2024-04-21T08:44:34.496609+0000","last_unstale":"2024-04-21T08:47:00.747340+0000","last_undegraded":"2024-04-21T08:47:00.747340+0000","last_fullsized":"2024-04-21T08:47:00.747340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:11.716321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747233+0000","last_change":"2024-04-21T08:45:27.406543+0000","last_active":"2024-04-21T08:47:00.747233+0000","last_peered":"2024-04-21T08:47:00.747233+0000","last_clean":"2024-04-21T08:47:00.747233+0000","last_became_active":"2024-04-21T08:45:27.406190+0000","last_became_peered":"2024-04-21T08:45:27.406190+0000","last_unstale":"2024-04-21T08:47:00.747233+0000","last_undegraded":"2024-04-21T08:47:00.747233+0000","last_fullsized":"2024-04-21T08:47:00.747233+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:56:17.760163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747327+0000","last_change":"2024-04-21T08:44:34.494018+0000","last_active":"2024-04-21T08:47:00.747327+0000","last_peered":"2024-04-21T08:47:00.747327+0000","last_clean":"2024-04-21T08:47:00.747327+0000","last_became_active":"2024-04-21T08:44:34.493692+0000","last_became_peered":"2024-04-21T08:44:34.493692+0000","last_unstale":"2024-04-21T08:47:00.747327+0000","last_undegraded":"2024-04-21T08:47:00.747327+0000","last_fullsized":"2024-04-21T08:47:00.747327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:25:16.944160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748187+0000","last_change":"2024-04-21T08:44:34.493231+0000","last_active":"2024-04-21T08:47:00.748187+0000","last_peered":"2024-04-21T08:47:00.748187+0000","last_clean":"2024-04-21T08:47:00.748187+0000","last_became_active":"2024-04-21T08:44:34.492826+0000","last_became_peered":"2024-04-21T08:44:34.492826+0000","last_unstale":"2024-04-21T08:47:00.748187+0000","last_undegraded":"2024-04-21T08:47:00.748187+0000","last_fullsized":"2024-04-21T08:47:00.748187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:36.815012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719352+0000","last_change":"2024-04-21T08:44:34.495142+0000","last_active":"2024-04-21T08:47:00.719352+0000","last_peered":"2024-04-21T08:47:00.719352+0000","last_clean":"2024-04-21T08:47:00.719352+0000","last_became_active":"2024-04-21T08:44:34.494803+0000","last_became_peered":"2024-04-21T08:44:34.494803+0000","last_unstale":"2024-04-21T08:47:00.719352+0000","last_undegraded":"2024-04-21T08:47:00.719352+0000","last_fullsized":"2024-04-21T08:47:00.719352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:22.974053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627186+0000","last_change":"2024-04-21T08:44:34.491252+0000","last_active":"2024-04-21T08:47:00.627186+0000","last_peered":"2024-04-21T08:47:00.627186+0000","last_clean":"2024-04-21T08:47:00.627186+0000","last_became_active":"2024-04-21T08:44:34.490814+0000","last_became_peered":"2024-04-21T08:44:34.490814+0000","last_unstale":"2024-04-21T08:47:00.627186+0000","last_undegraded":"2024-04-21T08:47:00.627186+0000","last_fullsized":"2024-04-21T08:47:00.627186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:54:52.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626862+0000","last_change":"2024-04-21T08:44:38.518647+0000","last_active":"2024-04-21T08:47:00.626862+0000","last_peered":"2024-04-21T08:47:00.626862+0000","last_clean":"2024-04-21T08:47:00.626862+0000","last_became_active":"2024-04-21T08:44:38.518239+0000","last_became_peered":"2024-04-21T08:44:38.518239+0000","last_unstale":"2024-04-21T08:47:00.626862+0000","last_undegraded":"2024-04-21T08:47:00.626862+0000","last_fullsized":"2024-04-21T08:47:00.626862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:44.781542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":167,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719204+0000","last_change":"2024-04-21T08:44:33.484484+0000","last_active":"2024-04-21T08:47:00.719204+0000","last_peered":"2024-04-21T08:47:00.719204+0000","last_clean":"2024-04-21T08:47:00.719204+0000","last_became_active":"2024-04-21T08:44:33.484160+0000","last_became_peered":"2024-04-21T08:44:33.484160+0000","last_unstale":"2024-04-21T08:47:00.719204+0000","last_undegraded":"2024-04-21T08:47:00.719204+0000","last_fullsized":"2024-04-21T08:47:00.719204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:45.962598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718920+0000","last_change":"2024-04-21T08:46:27.484947+0000","last_active":"2024-04-21T08:47:00.718920+0000","last_peered":"2024-04-21T08:47:00.718920+0000","last_clean":"2024-04-21T08:47:00.718920+0000","last_became_active":"2024-04-21T08:46:27.484530+0000","last_became_peered":"2024-04-21T08:46:27.484530+0000","last_unstale":"2024-04-21T08:47:00.718920+0000","last_undegraded":"2024-04-21T08:47:00.718920+0000","last_fullsized":"2024-04-21T08:47:00.718920+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:59.248176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747712+0000","last_change":"2024-04-21T08:45:27.518006+0000","last_active":"2024-04-21T08:47:00.747712+0000","last_peered":"2024-04-21T08:47:00.747712+0000","last_clean":"2024-04-21T08:47:00.747712+0000","last_became_active":"2024-04-21T08:45:27.517409+0000","last_became_peered":"2024-04-21T08:45:27.517409+0000","last_unstale":"2024-04-21T08:47:00.747712+0000","last_undegraded":"2024-04-21T08:47:00.747712+0000","last_fullsized":"2024-04-21T08:47:00.747712+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:45:01.253587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623343+0000","last_change":"2024-04-21T08:44:33.490508+0000","last_active":"2024-04-21T08:47:00.623343+0000","last_peered":"2024-04-21T08:47:00.623343+0000","last_clean":"2024-04-21T08:47:00.623343+0000","last_became_active":"2024-04-21T08:44:33.490084+0000","last_became_peered":"2024-04-21T08:44:33.490084+0000","last_unstale":"2024-04-21T08:47:00.623343+0000","last_undegraded":"2024-04-21T08:47:00.623343+0000","last_fullsized":"2024-04-21T08:47:00.623343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:21.553435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747770+0000","last_change":"2024-04-21T08:44:34.491843+0000","last_active":"2024-04-21T08:47:00.747770+0000","last_peered":"2024-04-21T08:47:00.747770+0000","last_clean":"2024-04-21T08:47:00.747770+0000","last_became_active":"2024-04-21T08:44:34.490578+0000","last_became_peered":"2024-04-21T08:44:34.490578+0000","last_unstale":"2024-04-21T08:47:00.747770+0000","last_undegraded":"2024-04-21T08:47:00.747770+0000","last_fullsized":"2024-04-21T08:47:00.747770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:05:12.272330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748032+0000","last_change":"2024-04-21T08:44:38.520258+0000","last_active":"2024-04-21T08:47:00.748032+0000","last_peered":"2024-04-21T08:47:00.748032+0000","last_clean":"2024-04-21T08:47:00.748032+0000","last_became_active":"2024-04-21T08:44:38.519824+0000","last_became_peered":"2024-04-21T08:44:38.519824+0000","last_unstale":"2024-04-21T08:47:00.748032+0000","last_undegraded":"2024-04-21T08:47:00.748032+0000","last_fullsized":"2024-04-21T08:47:00.748032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:39.716745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721769+0000","last_change":"2024-04-21T08:44:33.484934+0000","last_active":"2024-04-21T08:47:00.721769+0000","last_peered":"2024-04-21T08:47:00.721769+0000","last_clean":"2024-04-21T08:47:00.721769+0000","last_became_active":"2024-04-21T08:44:33.484591+0000","last_became_peered":"2024-04-21T08:44:33.484591+0000","last_unstale":"2024-04-21T08:47:00.721769+0000","last_undegraded":"2024-04-21T08:47:00.721769+0000","last_fullsized":"2024-04-21T08:47:00.721769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:48.826611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747621+0000","last_change":"2024-04-21T08:44:34.501109+0000","last_active":"2024-04-21T08:47:00.747621+0000","last_peered":"2024-04-21T08:47:00.747621+0000","last_clean":"2024-04-21T08:47:00.747621+0000","last_became_active":"2024-04-21T08:44:34.500750+0000","last_became_peered":"2024-04-21T08:44:34.500750+0000","last_unstale":"2024-04-21T08:47:00.747621+0000","last_undegraded":"2024-04-21T08:47:00.747621+0000","last_fullsized":"2024-04-21T08:47:00.747621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:29:28.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747712+0000","last_change":"2024-04-21T08:44:38.519633+0000","last_active":"2024-04-21T08:47:00.747712+0000","last_peered":"2024-04-21T08:47:00.747712+0000","last_clean":"2024-04-21T08:47:00.747712+0000","last_became_active":"2024-04-21T08:44:38.519215+0000","last_became_peered":"2024-04-21T08:44:38.519215+0000","last_unstale":"2024-04-21T08:47:00.747712+0000","last_undegraded":"2024-04-21T08:47:00.747712+0000","last_fullsized":"2024-04-21T08:47:00.747712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:43:59.695844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721688+0000","last_change":"2024-04-21T08:44:33.483267+0000","last_active":"2024-04-21T08:47:00.721688+0000","last_peered":"2024-04-21T08:47:00.721688+0000","last_clean":"2024-04-21T08:47:00.721688+0000","last_became_active":"2024-04-21T08:44:33.482932+0000","last_became_peered":"2024-04-21T08:44:33.482932+0000","last_unstale":"2024-04-21T08:47:00.721688+0000","last_undegraded":"2024-04-21T08:47:00.721688+0000","last_fullsized":"2024-04-21T08:47:00.721688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:53.311550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729235+0000","last_change":"2024-04-21T08:44:34.497389+0000","last_active":"2024-04-21T08:47:00.729235+0000","last_peered":"2024-04-21T08:47:00.729235+0000","last_clean":"2024-04-21T08:47:00.729235+0000","last_became_active":"2024-04-21T08:44:34.497030+0000","last_became_peered":"2024-04-21T08:44:34.497030+0000","last_unstale":"2024-04-21T08:47:00.729235+0000","last_undegraded":"2024-04-21T08:47:00.729235+0000","last_fullsized":"2024-04-21T08:47:00.729235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:20.371888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626918+0000","last_change":"2024-04-21T08:44:38.518095+0000","last_active":"2024-04-21T08:47:00.626918+0000","last_peered":"2024-04-21T08:47:00.626918+0000","last_clean":"2024-04-21T08:47:00.626918+0000","last_became_active":"2024-04-21T08:44:38.517010+0000","last_became_peered":"2024-04-21T08:44:38.517010+0000","last_unstale":"2024-04-21T08:47:00.626918+0000","last_undegraded":"2024-04-21T08:47:00.626918+0000","last_fullsized":"2024-04-21T08:47:00.626918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:02.379512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'8","reported_seq":167,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.167577+0000","last_change":"2024-04-21T08:44:33.487999+0000","last_active":"2024-04-21T08:47:01.167577+0000","last_peered":"2024-04-21T08:47:01.167577+0000","last_clean":"2024-04-21T08:47:01.167577+0000","last_became_active":"2024-04-21T08:44:33.487666+0000","last_became_peered":"2024-04-21T08:44:33.487666+0000","last_unstale":"2024-04-21T08:47:01.167577+0000","last_undegraded":"2024-04-21T08:47:01.167577+0000","last_fullsized":"2024-04-21T08:47:01.167577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:01.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1712,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721477+0000","last_change":"2024-04-21T08:44:34.494213+0000","last_active":"2024-04-21T08:47:00.721477+0000","last_peered":"2024-04-21T08:47:00.721477+0000","last_clean":"2024-04-21T08:47:00.721477+0000","last_became_active":"2024-04-21T08:44:34.493853+0000","last_became_peered":"2024-04-21T08:44:34.493853+0000","last_unstale":"2024-04-21T08:47:00.721477+0000","last_undegraded":"2024-04-21T08:47:00.721477+0000","last_fullsized":"2024-04-21T08:47:00.721477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:42.499131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627012+0000","last_change":"2024-04-21T08:45:27.410579+0000","last_active":"2024-04-21T08:47:00.627012+0000","last_peered":"2024-04-21T08:47:00.627012+0000","last_clean":"2024-04-21T08:47:00.627012+0000","last_became_active":"2024-04-21T08:45:27.409929+0000","last_became_peered":"2024-04-21T08:45:27.409929+0000","last_unstale":"2024-04-21T08:47:00.627012+0000","last_undegraded":"2024-04-21T08:47:00.627012+0000","last_fullsized":"2024-04-21T08:47:00.627012+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:35.897482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626508+0000","last_change":"2024-04-21T08:44:33.488751+0000","last_active":"2024-04-21T08:47:00.626508+0000","last_peered":"2024-04-21T08:47:00.626508+0000","last_clean":"2024-04-21T08:47:00.626508+0000","last_became_active":"2024-04-21T08:44:33.488425+0000","last_became_peered":"2024-04-21T08:44:33.488425+0000","last_unstale":"2024-04-21T08:47:00.626508+0000","last_undegraded":"2024-04-21T08:47:00.626508+0000","last_fullsized":"2024-04-21T08:47:00.626508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:28:47.868814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729103+0000","last_change":"2024-04-21T08:44:34.492770+0000","last_active":"2024-04-21T08:47:00.729103+0000","last_peered":"2024-04-21T08:47:00.729103+0000","last_clean":"2024-04-21T08:47:00.729103+0000","last_became_active":"2024-04-21T08:44:34.491340+0000","last_became_peered":"2024-04-21T08:44:34.491340+0000","last_unstale":"2024-04-21T08:47:00.729103+0000","last_undegraded":"2024-04-21T08:47:00.729103+0000","last_fullsized":"2024-04-21T08:47:00.729103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:47:54.280990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748097+0000","last_change":"2024-04-21T08:44:38.518888+0000","last_active":"2024-04-21T08:47:00.748097+0000","last_peered":"2024-04-21T08:47:00.748097+0000","last_clean":"2024-04-21T08:47:00.748097+0000","last_became_active":"2024-04-21T08:44:38.518421+0000","last_became_peered":"2024-04-21T08:44:38.518421+0000","last_unstale":"2024-04-21T08:47:00.748097+0000","last_undegraded":"2024-04-21T08:47:00.748097+0000","last_fullsized":"2024-04-21T08:47:00.748097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:21.119364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874482+0000","last_change":"2024-04-21T08:44:33.484383+0000","last_active":"2024-04-21T08:47:00.874482+0000","last_peered":"2024-04-21T08:47:00.874482+0000","last_clean":"2024-04-21T08:47:00.874482+0000","last_became_active":"2024-04-21T08:44:33.484083+0000","last_became_peered":"2024-04-21T08:44:33.484083+0000","last_unstale":"2024-04-21T08:47:00.874482+0000","last_undegraded":"2024-04-21T08:47:00.874482+0000","last_fullsized":"2024-04-21T08:47:00.874482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:13:14.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622126+0000","last_change":"2024-04-21T08:44:34.495636+0000","last_active":"2024-04-21T08:47:00.622126+0000","last_peered":"2024-04-21T08:47:00.622126+0000","last_clean":"2024-04-21T08:47:00.622126+0000","last_became_active":"2024-04-21T08:44:34.494967+0000","last_became_peered":"2024-04-21T08:44:34.494967+0000","last_unstale":"2024-04-21T08:47:00.622126+0000","last_undegraded":"2024-04-21T08:47:00.622126+0000","last_fullsized":"2024-04-21T08:47:00.622126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:47:02.379208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747724+0000","last_change":"2024-04-21T08:44:38.515145+0000","last_active":"2024-04-21T08:47:00.747724+0000","last_peered":"2024-04-21T08:47:00.747724+0000","last_clean":"2024-04-21T08:47:00.747724+0000","last_became_active":"2024-04-21T08:44:38.514755+0000","last_became_peered":"2024-04-21T08:44:38.514755+0000","last_unstale":"2024-04-21T08:47:00.747724+0000","last_undegraded":"2024-04-21T08:47:00.747724+0000","last_fullsized":"2024-04-21T08:47:00.747724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:38:15.826293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728869+0000","last_change":"2024-04-21T08:44:33.484714+0000","last_active":"2024-04-21T08:47:00.728869+0000","last_peered":"2024-04-21T08:47:00.728869+0000","last_clean":"2024-04-21T08:47:00.728869+0000","last_became_active":"2024-04-21T08:44:33.484385+0000","last_became_peered":"2024-04-21T08:44:33.484385+0000","last_unstale":"2024-04-21T08:47:00.728869+0000","last_undegraded":"2024-04-21T08:47:00.728869+0000","last_fullsized":"2024-04-21T08:47:00.728869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:03.406860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718544+0000","last_change":"2024-04-21T08:44:34.490989+0000","last_active":"2024-04-21T08:47:00.718544+0000","last_peered":"2024-04-21T08:47:00.718544+0000","last_clean":"2024-04-21T08:47:00.718544+0000","last_became_active":"2024-04-21T08:44:34.490658+0000","last_became_peered":"2024-04-21T08:44:34.490658+0000","last_unstale":"2024-04-21T08:47:00.718544+0000","last_undegraded":"2024-04-21T08:47:00.718544+0000","last_fullsized":"2024-04-21T08:47:00.718544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:42:15.537913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622288+0000","last_change":"2024-04-21T08:44:38.519723+0000","last_active":"2024-04-21T08:47:00.622288+0000","last_peered":"2024-04-21T08:47:00.622288+0000","last_clean":"2024-04-21T08:47:00.622288+0000","last_became_active":"2024-04-21T08:44:38.519336+0000","last_became_peered":"2024-04-21T08:44:38.519336+0000","last_unstale":"2024-04-21T08:47:00.622288+0000","last_undegraded":"2024-04-21T08:47:00.622288+0000","last_fullsized":"2024-04-21T08:47:00.622288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:00.332153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'6","reported_seq":281,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.237217+0000","last_change":"2024-04-21T08:44:33.484404+0000","last_active":"2024-04-21T08:47:01.237217+0000","last_peered":"2024-04-21T08:47:01.237217+0000","last_clean":"2024-04-21T08:47:01.237217+0000","last_became_active":"2024-04-21T08:44:33.484088+0000","last_became_peered":"2024-04-21T08:44:33.484088+0000","last_unstale":"2024-04-21T08:47:01.237217+0000","last_undegraded":"2024-04-21T08:47:01.237217+0000","last_fullsized":"2024-04-21T08:47:01.237217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:41.948622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":242,"num_read_kb":242,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748214+0000","last_change":"2024-04-21T08:44:34.502598+0000","last_active":"2024-04-21T08:47:00.748214+0000","last_peered":"2024-04-21T08:47:00.748214+0000","last_clean":"2024-04-21T08:47:00.748214+0000","last_became_active":"2024-04-21T08:44:34.502057+0000","last_became_peered":"2024-04-21T08:44:34.502057+0000","last_unstale":"2024-04-21T08:47:00.748214+0000","last_undegraded":"2024-04-21T08:47:00.748214+0000","last_fullsized":"2024-04-21T08:47:00.748214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:24.052873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718327+0000","last_change":"2024-04-21T08:44:38.519699+0000","last_active":"2024-04-21T08:47:00.718327+0000","last_peered":"2024-04-21T08:47:00.718327+0000","last_clean":"2024-04-21T08:47:00.718327+0000","last_became_active":"2024-04-21T08:44:38.517655+0000","last_became_peered":"2024-04-21T08:44:38.517655+0000","last_unstale":"2024-04-21T08:47:00.718327+0000","last_undegraded":"2024-04-21T08:47:00.718327+0000","last_fullsized":"2024-04-21T08:47:00.718327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:12.784479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748806+0000","last_change":"2024-04-21T08:46:27.494747+0000","last_active":"2024-04-21T08:47:00.748806+0000","last_peered":"2024-04-21T08:47:00.748806+0000","last_clean":"2024-04-21T08:47:00.748806+0000","last_became_active":"2024-04-21T08:46:27.490469+0000","last_became_peered":"2024-04-21T08:46:27.490469+0000","last_unstale":"2024-04-21T08:47:00.748806+0000","last_undegraded":"2024-04-21T08:47:00.748806+0000","last_fullsized":"2024-04-21T08:47:00.748806+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:16:35.257267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747850+0000","last_change":"2024-04-21T08:45:27.518395+0000","last_active":"2024-04-21T08:47:00.747850+0000","last_peered":"2024-04-21T08:47:00.747850+0000","last_clean":"2024-04-21T08:47:00.747850+0000","last_became_active":"2024-04-21T08:45:27.517969+0000","last_became_peered":"2024-04-21T08:45:27.517969+0000","last_unstale":"2024-04-21T08:47:00.747850+0000","last_undegraded":"2024-04-21T08:47:00.747850+0000","last_fullsized":"2024-04-21T08:47:00.747850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:55.306556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748406+0000","last_change":"2024-04-21T08:44:38.517956+0000","last_active":"2024-04-21T08:47:00.748406+0000","last_peered":"2024-04-21T08:47:00.748406+0000","last_clean":"2024-04-21T08:47:00.748406+0000","last_became_active":"2024-04-21T08:44:38.517422+0000","last_became_peered":"2024-04-21T08:44:38.517422+0000","last_unstale":"2024-04-21T08:47:00.748406+0000","last_undegraded":"2024-04-21T08:47:00.748406+0000","last_fullsized":"2024-04-21T08:47:00.748406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:04.650784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729223+0000","last_change":"2024-04-21T08:44:33.489151+0000","last_active":"2024-04-21T08:47:00.729223+0000","last_peered":"2024-04-21T08:47:00.729223+0000","last_clean":"2024-04-21T08:47:00.729223+0000","last_became_active":"2024-04-21T08:44:33.488781+0000","last_became_peered":"2024-04-21T08:44:33.488781+0000","last_unstale":"2024-04-21T08:47:00.729223+0000","last_undegraded":"2024-04-21T08:47:00.729223+0000","last_fullsized":"2024-04-21T08:47:00.729223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:40.675100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747905+0000","last_change":"2024-04-21T08:44:34.496528+0000","last_active":"2024-04-21T08:47:00.747905+0000","last_peered":"2024-04-21T08:47:00.747905+0000","last_clean":"2024-04-21T08:47:00.747905+0000","last_became_active":"2024-04-21T08:44:34.496223+0000","last_became_peered":"2024-04-21T08:44:34.496223+0000","last_unstale":"2024-04-21T08:47:00.747905+0000","last_undegraded":"2024-04-21T08:47:00.747905+0000","last_fullsized":"2024-04-21T08:47:00.747905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:56.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718461+0000","last_change":"2024-04-21T08:44:38.517537+0000","last_active":"2024-04-21T08:47:00.718461+0000","last_peered":"2024-04-21T08:47:00.718461+0000","last_clean":"2024-04-21T08:47:00.718461+0000","last_became_active":"2024-04-21T08:44:38.517134+0000","last_became_peered":"2024-04-21T08:44:38.517134+0000","last_unstale":"2024-04-21T08:47:00.718461+0000","last_undegraded":"2024-04-21T08:47:00.718461+0000","last_fullsized":"2024-04-21T08:47:00.718461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:38.634418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'39","reported_seq":326,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.179097+0000","last_change":"2024-04-21T08:44:33.485582+0000","last_active":"2024-04-21T08:47:01.179097+0000","last_peered":"2024-04-21T08:47:01.179097+0000","last_clean":"2024-04-21T08:47:01.179097+0000","last_became_active":"2024-04-21T08:44:33.485249+0000","last_became_peered":"2024-04-21T08:44:33.485249+0000","last_unstale":"2024-04-21T08:47:01.179097+0000","last_undegraded":"2024-04-21T08:47:01.179097+0000","last_fullsized":"2024-04-21T08:47:01.179097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:16:10.571694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67037,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":254,"num_read_kb":193,"num_write":41,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721670+0000","last_change":"2024-04-21T08:44:34.491361+0000","last_active":"2024-04-21T08:47:00.721670+0000","last_peered":"2024-04-21T08:47:00.721670+0000","last_clean":"2024-04-21T08:47:00.721670+0000","last_became_active":"2024-04-21T08:44:34.491050+0000","last_became_peered":"2024-04-21T08:44:34.491050+0000","last_unstale":"2024-04-21T08:47:00.721670+0000","last_undegraded":"2024-04-21T08:47:00.721670+0000","last_fullsized":"2024-04-21T08:47:00.721670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:39.082448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729534+0000","last_change":"2024-04-21T08:44:38.528745+0000","last_active":"2024-04-21T08:47:00.729534+0000","last_peered":"2024-04-21T08:47:00.729534+0000","last_clean":"2024-04-21T08:47:00.729534+0000","last_became_active":"2024-04-21T08:44:38.528275+0000","last_became_peered":"2024-04-21T08:44:38.528275+0000","last_unstale":"2024-04-21T08:47:00.729534+0000","last_undegraded":"2024-04-21T08:47:00.729534+0000","last_fullsized":"2024-04-21T08:47:00.729534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:06:11.338420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626897+0000","last_change":"2024-04-21T08:44:33.482059+0000","last_active":"2024-04-21T08:47:00.626897+0000","last_peered":"2024-04-21T08:47:00.626897+0000","last_clean":"2024-04-21T08:47:00.626897+0000","last_became_active":"2024-04-21T08:44:33.481663+0000","last_became_peered":"2024-04-21T08:44:33.481663+0000","last_unstale":"2024-04-21T08:47:00.626897+0000","last_undegraded":"2024-04-21T08:47:00.626897+0000","last_fullsized":"2024-04-21T08:47:00.626897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:01.240452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874587+0000","last_change":"2024-04-21T08:44:34.491398+0000","last_active":"2024-04-21T08:47:00.874587+0000","last_peered":"2024-04-21T08:47:00.874587+0000","last_clean":"2024-04-21T08:47:00.874587+0000","last_became_active":"2024-04-21T08:44:34.491073+0000","last_became_peered":"2024-04-21T08:44:34.491073+0000","last_unstale":"2024-04-21T08:47:00.874587+0000","last_undegraded":"2024-04-21T08:47:00.874587+0000","last_fullsized":"2024-04-21T08:47:00.874587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:22.566063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747937+0000","last_change":"2024-04-21T08:44:38.515653+0000","last_active":"2024-04-21T08:47:00.747937+0000","last_peered":"2024-04-21T08:47:00.747937+0000","last_clean":"2024-04-21T08:47:00.747937+0000","last_became_active":"2024-04-21T08:44:38.514796+0000","last_became_peered":"2024-04-21T08:44:38.514796+0000","last_unstale":"2024-04-21T08:47:00.747937+0000","last_undegraded":"2024-04-21T08:47:00.747937+0000","last_fullsized":"2024-04-21T08:47:00.747937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:00:22.662136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729003+0000","last_change":"2024-04-21T08:44:33.487073+0000","last_active":"2024-04-21T08:47:00.729003+0000","last_peered":"2024-04-21T08:47:00.729003+0000","last_clean":"2024-04-21T08:47:00.729003+0000","last_became_active":"2024-04-21T08:44:33.486740+0000","last_became_peered":"2024-04-21T08:44:33.486740+0000","last_unstale":"2024-04-21T08:47:00.729003+0000","last_undegraded":"2024-04-21T08:47:00.729003+0000","last_fullsized":"2024-04-21T08:47:00.729003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:32.673074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748512+0000","last_change":"2024-04-21T08:44:34.496757+0000","last_active":"2024-04-21T08:47:00.748512+0000","last_peered":"2024-04-21T08:47:00.748512+0000","last_clean":"2024-04-21T08:47:00.748512+0000","last_became_active":"2024-04-21T08:44:34.496422+0000","last_became_peered":"2024-04-21T08:44:34.496422+0000","last_unstale":"2024-04-21T08:47:00.748512+0000","last_undegraded":"2024-04-21T08:47:00.748512+0000","last_fullsized":"2024-04-21T08:47:00.748512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:08.686810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718934+0000","last_change":"2024-04-21T08:44:38.520935+0000","last_active":"2024-04-21T08:47:00.718934+0000","last_peered":"2024-04-21T08:47:00.718934+0000","last_clean":"2024-04-21T08:47:00.718934+0000","last_became_active":"2024-04-21T08:44:38.520481+0000","last_became_peered":"2024-04-21T08:44:38.520481+0000","last_unstale":"2024-04-21T08:47:00.718934+0000","last_undegraded":"2024-04-21T08:47:00.718934+0000","last_fullsized":"2024-04-21T08:47:00.718934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:12.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'7","reported_seq":166,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.165119+0000","last_change":"2024-04-21T08:44:33.485797+0000","last_active":"2024-04-21T08:47:01.165119+0000","last_peered":"2024-04-21T08:47:01.165119+0000","last_clean":"2024-04-21T08:47:01.165119+0000","last_became_active":"2024-04-21T08:44:33.485496+0000","last_became_peered":"2024-04-21T08:44:33.485496+0000","last_unstale":"2024-04-21T08:47:01.165119+0000","last_undegraded":"2024-04-21T08:47:01.165119+0000","last_fullsized":"2024-04-21T08:47:01.165119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:11:34.604611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748282+0000","last_change":"2024-04-21T08:44:34.506453+0000","last_active":"2024-04-21T08:47:00.748282+0000","last_peered":"2024-04-21T08:47:00.748282+0000","last_clean":"2024-04-21T08:47:00.748282+0000","last_became_active":"2024-04-21T08:44:34.506121+0000","last_became_peered":"2024-04-21T08:44:34.506121+0000","last_unstale":"2024-04-21T08:47:00.748282+0000","last_undegraded":"2024-04-21T08:47:00.748282+0000","last_fullsized":"2024-04-21T08:47:00.748282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:07.927366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875250+0000","last_change":"2024-04-21T08:44:38.523718+0000","last_active":"2024-04-21T08:47:00.875250+0000","last_peered":"2024-04-21T08:47:00.875250+0000","last_clean":"2024-04-21T08:47:00.875250+0000","last_became_active":"2024-04-21T08:44:38.523003+0000","last_became_peered":"2024-04-21T08:44:38.523003+0000","last_unstale":"2024-04-21T08:47:00.875250+0000","last_undegraded":"2024-04-21T08:47:00.875250+0000","last_fullsized":"2024-04-21T08:47:00.875250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:20.837450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721393+0000","last_change":"2024-04-21T08:44:33.482029+0000","last_active":"2024-04-21T08:47:00.721393+0000","last_peered":"2024-04-21T08:47:00.721393+0000","last_clean":"2024-04-21T08:47:00.721393+0000","last_became_active":"2024-04-21T08:44:33.481713+0000","last_became_peered":"2024-04-21T08:44:33.481713+0000","last_unstale":"2024-04-21T08:47:00.721393+0000","last_undegraded":"2024-04-21T08:47:00.721393+0000","last_fullsized":"2024-04-21T08:47:00.721393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:46:34.717244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728921+0000","last_change":"2024-04-21T08:44:34.491944+0000","last_active":"2024-04-21T08:47:00.728921+0000","last_peered":"2024-04-21T08:47:00.728921+0000","last_clean":"2024-04-21T08:47:00.728921+0000","last_became_active":"2024-04-21T08:44:34.491638+0000","last_became_peered":"2024-04-21T08:44:34.491638+0000","last_unstale":"2024-04-21T08:47:00.728921+0000","last_undegraded":"2024-04-21T08:47:00.728921+0000","last_fullsized":"2024-04-21T08:47:00.728921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:08:07.925937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728793+0000","last_change":"2024-04-21T08:44:38.528971+0000","last_active":"2024-04-21T08:47:00.728793+0000","last_peered":"2024-04-21T08:47:00.728793+0000","last_clean":"2024-04-21T08:47:00.728793+0000","last_became_active":"2024-04-21T08:44:38.528453+0000","last_became_peered":"2024-04-21T08:44:38.528453+0000","last_unstale":"2024-04-21T08:47:00.728793+0000","last_undegraded":"2024-04-21T08:47:00.728793+0000","last_fullsized":"2024-04-21T08:47:00.728793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:50:31.237749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721764+0000","last_change":"2024-04-21T08:44:33.479262+0000","last_active":"2024-04-21T08:47:00.721764+0000","last_peered":"2024-04-21T08:47:00.721764+0000","last_clean":"2024-04-21T08:47:00.721764+0000","last_became_active":"2024-04-21T08:44:33.478907+0000","last_became_peered":"2024-04-21T08:44:33.478907+0000","last_unstale":"2024-04-21T08:47:00.721764+0000","last_undegraded":"2024-04-21T08:47:00.721764+0000","last_fullsized":"2024-04-21T08:47:00.721764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:33:43.879126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748024+0000","last_change":"2024-04-21T08:44:34.498503+0000","last_active":"2024-04-21T08:47:00.748024+0000","last_peered":"2024-04-21T08:47:00.748024+0000","last_clean":"2024-04-21T08:47:00.748024+0000","last_became_active":"2024-04-21T08:44:34.495140+0000","last_became_peered":"2024-04-21T08:44:34.495140+0000","last_unstale":"2024-04-21T08:47:00.748024+0000","last_undegraded":"2024-04-21T08:47:00.748024+0000","last_fullsized":"2024-04-21T08:47:00.748024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:24.518349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721372+0000","last_change":"2024-04-21T08:44:38.517717+0000","last_active":"2024-04-21T08:47:00.721372+0000","last_peered":"2024-04-21T08:47:00.721372+0000","last_clean":"2024-04-21T08:47:00.721372+0000","last_became_active":"2024-04-21T08:44:38.517272+0000","last_became_peered":"2024-04-21T08:44:38.517272+0000","last_unstale":"2024-04-21T08:47:00.721372+0000","last_undegraded":"2024-04-21T08:47:00.721372+0000","last_fullsized":"2024-04-21T08:47:00.721372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.664130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718792+0000","last_change":"2024-04-21T08:44:33.477498+0000","last_active":"2024-04-21T08:47:00.718792+0000","last_peered":"2024-04-21T08:47:00.718792+0000","last_clean":"2024-04-21T08:47:00.718792+0000","last_became_active":"2024-04-21T08:44:33.477139+0000","last_became_peered":"2024-04-21T08:44:33.477139+0000","last_unstale":"2024-04-21T08:47:00.718792+0000","last_undegraded":"2024-04-21T08:47:00.718792+0000","last_fullsized":"2024-04-21T08:47:00.718792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:26:59.390583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626701+0000","last_change":"2024-04-21T08:44:34.493586+0000","last_active":"2024-04-21T08:47:00.626701+0000","last_peered":"2024-04-21T08:47:00.626701+0000","last_clean":"2024-04-21T08:47:00.626701+0000","last_became_active":"2024-04-21T08:44:34.493258+0000","last_became_peered":"2024-04-21T08:44:34.493258+0000","last_unstale":"2024-04-21T08:47:00.626701+0000","last_undegraded":"2024-04-21T08:47:00.626701+0000","last_fullsized":"2024-04-21T08:47:00.626701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:11:18.783332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874853+0000","last_change":"2024-04-21T08:44:38.519783+0000","last_active":"2024-04-21T08:47:00.874853+0000","last_peered":"2024-04-21T08:47:00.874853+0000","last_clean":"2024-04-21T08:47:00.874853+0000","last_became_active":"2024-04-21T08:44:38.519397+0000","last_became_peered":"2024-04-21T08:44:38.519397+0000","last_unstale":"2024-04-21T08:47:00.874853+0000","last_undegraded":"2024-04-21T08:47:00.874853+0000","last_fullsized":"2024-04-21T08:47:00.874853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:08.740240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":160,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749428+0000","last_change":"2024-04-21T08:44:33.490098+0000","last_active":"2024-04-21T08:47:00.749428+0000","last_peered":"2024-04-21T08:47:00.749428+0000","last_clean":"2024-04-21T08:47:00.749428+0000","last_became_active":"2024-04-21T08:44:33.489691+0000","last_became_peered":"2024-04-21T08:44:33.489691+0000","last_unstale":"2024-04-21T08:47:00.749428+0000","last_undegraded":"2024-04-21T08:47:00.749428+0000","last_fullsized":"2024-04-21T08:47:00.749428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:44.965998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626543+0000","last_change":"2024-04-21T08:46:27.489580+0000","last_active":"2024-04-21T08:47:00.626543+0000","last_peered":"2024-04-21T08:47:00.626543+0000","last_clean":"2024-04-21T08:47:00.626543+0000","last_became_active":"2024-04-21T08:46:27.489086+0000","last_became_peered":"2024-04-21T08:46:27.489086+0000","last_unstale":"2024-04-21T08:47:00.626543+0000","last_undegraded":"2024-04-21T08:47:00.626543+0000","last_fullsized":"2024-04-21T08:47:00.626543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:38:50.618957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874908+0000","last_change":"2024-04-21T08:44:38.521010+0000","last_active":"2024-04-21T08:47:00.874908+0000","last_peered":"2024-04-21T08:47:00.874908+0000","last_clean":"2024-04-21T08:47:00.874908+0000","last_became_active":"2024-04-21T08:44:38.520605+0000","last_became_peered":"2024-04-21T08:44:38.520605+0000","last_unstale":"2024-04-21T08:47:00.874908+0000","last_undegraded":"2024-04-21T08:47:00.874908+0000","last_fullsized":"2024-04-21T08:47:00.874908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:50.378803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728846+0000","last_change":"2024-04-21T08:45:27.410137+0000","last_active":"2024-04-21T08:47:00.728846+0000","last_peered":"2024-04-21T08:47:00.728846+0000","last_clean":"2024-04-21T08:47:00.728846+0000","last_became_active":"2024-04-21T08:45:27.409728+0000","last_became_peered":"2024-04-21T08:45:27.409728+0000","last_unstale":"2024-04-21T08:47:00.728846+0000","last_undegraded":"2024-04-21T08:47:00.728846+0000","last_fullsized":"2024-04-21T08:47:00.728846+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:17.934902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":162,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.885117+0000","last_change":"2024-04-21T08:44:33.494031+0000","last_active":"2024-04-21T08:47:00.885117+0000","last_peered":"2024-04-21T08:47:00.885117+0000","last_clean":"2024-04-21T08:47:00.885117+0000","last_became_active":"2024-04-21T08:44:33.493580+0000","last_became_peered":"2024-04-21T08:44:33.493580+0000","last_unstale":"2024-04-21T08:47:00.885117+0000","last_undegraded":"2024-04-21T08:47:00.885117+0000","last_fullsized":"2024-04-21T08:47:00.885117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:39:48.230616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748658+0000","last_change":"2024-04-21T08:44:38.518067+0000","last_active":"2024-04-21T08:47:00.748658+0000","last_peered":"2024-04-21T08:47:00.748658+0000","last_clean":"2024-04-21T08:47:00.748658+0000","last_became_active":"2024-04-21T08:44:38.517594+0000","last_became_peered":"2024-04-21T08:44:38.517594+0000","last_unstale":"2024-04-21T08:47:00.748658+0000","last_undegraded":"2024-04-21T08:47:00.748658+0000","last_fullsized":"2024-04-21T08:47:00.748658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:35.261582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718443+0000","last_change":"2024-04-21T08:45:27.517719+0000","last_active":"2024-04-21T08:47:00.718443+0000","last_peered":"2024-04-21T08:47:00.718443+0000","last_clean":"2024-04-21T08:47:00.718443+0000","last_became_active":"2024-04-21T08:45:27.517429+0000","last_became_peered":"2024-04-21T08:45:27.517429+0000","last_unstale":"2024-04-21T08:47:00.718443+0000","last_undegraded":"2024-04-21T08:47:00.718443+0000","last_fullsized":"2024-04-21T08:47:00.718443+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:49:34.097759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748213+0000","last_change":"2024-04-21T08:44:33.485110+0000","last_active":"2024-04-21T08:47:00.748213+0000","last_peered":"2024-04-21T08:47:00.748213+0000","last_clean":"2024-04-21T08:47:00.748213+0000","last_became_active":"2024-04-21T08:44:33.484722+0000","last_became_peered":"2024-04-21T08:44:33.484722+0000","last_unstale":"2024-04-21T08:47:00.748213+0000","last_undegraded":"2024-04-21T08:47:00.748213+0000","last_fullsized":"2024-04-21T08:47:00.748213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:47:16.773560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":368,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748009+0000","last_change":"2024-04-21T08:43:17.984594+0000","last_active":"2024-04-21T08:47:00.748009+0000","last_peered":"2024-04-21T08:47:00.748009+0000","last_clean":"2024-04-21T08:47:00.748009+0000","last_became_active":"2024-04-21T08:43:17.984207+0000","last_became_peered":"2024-04-21T08:43:17.984207+0000","last_unstale":"2024-04-21T08:47:00.748009+0000","last_undegraded":"2024-04-21T08:47:00.748009+0000","last_fullsized":"2024-04-21T08:47:00.748009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_clean_scrub_stamp":"2024-04-21T08:39:50.910206+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:35.260499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721571+0000","last_change":"2024-04-21T08:45:27.518834+0000","last_active":"2024-04-21T08:47:00.721571+0000","last_peered":"2024-04-21T08:47:00.721571+0000","last_clean":"2024-04-21T08:47:00.721571+0000","last_became_active":"2024-04-21T08:45:27.518390+0000","last_became_peered":"2024-04-21T08:45:27.518390+0000","last_unstale":"2024-04-21T08:47:00.721571+0000","last_undegraded":"2024-04-21T08:47:00.721571+0000","last_fullsized":"2024-04-21T08:47:00.721571+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:58:32.673008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874296+0000","last_change":"2024-04-21T08:44:34.499975+0000","last_active":"2024-04-21T08:47:00.874296+0000","last_peered":"2024-04-21T08:47:00.874296+0000","last_clean":"2024-04-21T08:47:00.874296+0000","last_became_active":"2024-04-21T08:44:34.499646+0000","last_became_peered":"2024-04-21T08:44:34.499646+0000","last_unstale":"2024-04-21T08:47:00.874296+0000","last_undegraded":"2024-04-21T08:47:00.874296+0000","last_fullsized":"2024-04-21T08:47:00.874296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:19:38.082502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748114+0000","last_change":"2024-04-21T08:44:33.484449+0000","last_active":"2024-04-21T08:47:00.748114+0000","last_peered":"2024-04-21T08:47:00.748114+0000","last_clean":"2024-04-21T08:47:00.748114+0000","last_became_active":"2024-04-21T08:44:33.484155+0000","last_became_peered":"2024-04-21T08:44:33.484155+0000","last_unstale":"2024-04-21T08:47:00.748114+0000","last_undegraded":"2024-04-21T08:47:00.748114+0000","last_fullsized":"2024-04-21T08:47:00.748114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:03.480474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747766+0000","last_change":"2024-04-21T08:44:38.515283+0000","last_active":"2024-04-21T08:47:00.747766+0000","last_peered":"2024-04-21T08:47:00.747766+0000","last_clean":"2024-04-21T08:47:00.747766+0000","last_became_active":"2024-04-21T08:44:38.514764+0000","last_became_peered":"2024-04-21T08:44:38.514764+0000","last_unstale":"2024-04-21T08:47:00.747766+0000","last_undegraded":"2024-04-21T08:47:00.747766+0000","last_fullsized":"2024-04-21T08:47:00.747766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:17:36.378392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721569+0000","last_change":"2024-04-21T08:44:33.489198+0000","last_active":"2024-04-21T08:47:00.721569+0000","last_peered":"2024-04-21T08:47:00.721569+0000","last_clean":"2024-04-21T08:47:00.721569+0000","last_became_active":"2024-04-21T08:44:33.488794+0000","last_became_peered":"2024-04-21T08:44:33.488794+0000","last_unstale":"2024-04-21T08:47:00.721569+0000","last_undegraded":"2024-04-21T08:47:00.721569+0000","last_fullsized":"2024-04-21T08:47:00.721569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:24.948283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874977+0000","last_change":"2024-04-21T08:44:34.488811+0000","last_active":"2024-04-21T08:47:00.874977+0000","last_peered":"2024-04-21T08:47:00.874977+0000","last_clean":"2024-04-21T08:47:00.874977+0000","last_became_active":"2024-04-21T08:44:34.488472+0000","last_became_peered":"2024-04-21T08:44:34.488472+0000","last_unstale":"2024-04-21T08:47:00.874977+0000","last_undegraded":"2024-04-21T08:47:00.874977+0000","last_fullsized":"2024-04-21T08:47:00.874977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:44.779492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622045+0000","last_change":"2024-04-21T08:44:38.522711+0000","last_active":"2024-04-21T08:47:00.622045+0000","last_peered":"2024-04-21T08:47:00.622045+0000","last_clean":"2024-04-21T08:47:00.622045+0000","last_became_active":"2024-04-21T08:44:38.522322+0000","last_became_peered":"2024-04-21T08:44:38.522322+0000","last_unstale":"2024-04-21T08:47:00.622045+0000","last_undegraded":"2024-04-21T08:47:00.622045+0000","last_fullsized":"2024-04-21T08:47:00.622045+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:26:13.323283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'48","reported_seq":104,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.836065+0000","last_change":"2024-04-21T08:44:33.481815+0000","last_active":"2024-04-21T08:47:00.836065+0000","last_peered":"2024-04-21T08:47:00.836065+0000","last_clean":"2024-04-21T08:47:00.836065+0000","last_became_active":"2024-04-21T08:44:33.481469+0000","last_became_peered":"2024-04-21T08:44:33.481469+0000","last_unstale":"2024-04-21T08:47:00.836065+0000","last_undegraded":"2024-04-21T08:47:00.836065+0000","last_fullsized":"2024-04-21T08:47:00.836065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:13.970821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623173+0000","last_change":"2024-04-21T08:44:34.496996+0000","last_active":"2024-04-21T08:47:00.623173+0000","last_peered":"2024-04-21T08:47:00.623173+0000","last_clean":"2024-04-21T08:47:00.623173+0000","last_became_active":"2024-04-21T08:44:34.496676+0000","last_became_peered":"2024-04-21T08:44:34.496676+0000","last_unstale":"2024-04-21T08:47:00.623173+0000","last_undegraded":"2024-04-21T08:47:00.623173+0000","last_fullsized":"2024-04-21T08:47:00.623173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:43:08.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626592+0000","last_change":"2024-04-21T08:44:38.516240+0000","last_active":"2024-04-21T08:47:00.626592+0000","last_peered":"2024-04-21T08:47:00.626592+0000","last_clean":"2024-04-21T08:47:00.626592+0000","last_became_active":"2024-04-21T08:44:38.515801+0000","last_became_peered":"2024-04-21T08:44:38.515801+0000","last_unstale":"2024-04-21T08:47:00.626592+0000","last_undegraded":"2024-04-21T08:47:00.626592+0000","last_fullsized":"2024-04-21T08:47:00.626592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:46.520373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749302+0000","last_change":"2024-04-21T08:44:33.488952+0000","last_active":"2024-04-21T08:47:00.749302+0000","last_peered":"2024-04-21T08:47:00.749302+0000","last_clean":"2024-04-21T08:47:00.749302+0000","last_became_active":"2024-04-21T08:44:33.488617+0000","last_became_peered":"2024-04-21T08:44:33.488617+0000","last_unstale":"2024-04-21T08:47:00.749302+0000","last_undegraded":"2024-04-21T08:47:00.749302+0000","last_fullsized":"2024-04-21T08:47:00.749302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:37:24.679132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747751+0000","last_change":"2024-04-21T08:44:34.510568+0000","last_active":"2024-04-21T08:47:00.747751+0000","last_peered":"2024-04-21T08:47:00.747751+0000","last_clean":"2024-04-21T08:47:00.747751+0000","last_became_active":"2024-04-21T08:44:34.509900+0000","last_became_peered":"2024-04-21T08:44:34.509900+0000","last_unstale":"2024-04-21T08:47:00.747751+0000","last_undegraded":"2024-04-21T08:47:00.747751+0000","last_fullsized":"2024-04-21T08:47:00.747751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:08.426141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748835+0000","last_change":"2024-04-21T08:45:27.976874+0000","last_active":"2024-04-21T08:47:00.748835+0000","last_peered":"2024-04-21T08:47:00.748835+0000","last_clean":"2024-04-21T08:47:00.748835+0000","last_became_active":"2024-04-21T08:45:27.976387+0000","last_became_peered":"2024-04-21T08:45:27.976387+0000","last_unstale":"2024-04-21T08:47:00.748835+0000","last_undegraded":"2024-04-21T08:47:00.748835+0000","last_fullsized":"2024-04-21T08:47:00.748835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:38:18.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747268+0000","last_change":"2024-04-21T08:44:33.481803+0000","last_active":"2024-04-21T08:47:00.747268+0000","last_peered":"2024-04-21T08:47:00.747268+0000","last_clean":"2024-04-21T08:47:00.747268+0000","last_became_active":"2024-04-21T08:44:33.481256+0000","last_became_peered":"2024-04-21T08:44:33.481256+0000","last_unstale":"2024-04-21T08:47:00.747268+0000","last_undegraded":"2024-04-21T08:47:00.747268+0000","last_fullsized":"2024-04-21T08:47:00.747268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:17.804557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721258+0000","last_change":"2024-04-21T08:44:34.490206+0000","last_active":"2024-04-21T08:47:00.721258+0000","last_peered":"2024-04-21T08:47:00.721258+0000","last_clean":"2024-04-21T08:47:00.721258+0000","last_became_active":"2024-04-21T08:44:34.489797+0000","last_became_peered":"2024-04-21T08:44:34.489797+0000","last_unstale":"2024-04-21T08:47:00.721258+0000","last_undegraded":"2024-04-21T08:47:00.721258+0000","last_fullsized":"2024-04-21T08:47:00.721258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:09.055429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719147+0000","last_change":"2024-04-21T08:44:38.517385+0000","last_active":"2024-04-21T08:47:00.719147+0000","last_peered":"2024-04-21T08:47:00.719147+0000","last_clean":"2024-04-21T08:47:00.719147+0000","last_became_active":"2024-04-21T08:44:38.516964+0000","last_became_peered":"2024-04-21T08:44:38.516964+0000","last_unstale":"2024-04-21T08:47:00.719147+0000","last_undegraded":"2024-04-21T08:47:00.719147+0000","last_fullsized":"2024-04-21T08:47:00.719147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:33.364076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622039+0000","last_change":"2024-04-21T08:44:33.490586+0000","last_active":"2024-04-21T08:47:00.622039+0000","last_peered":"2024-04-21T08:47:00.622039+0000","last_clean":"2024-04-21T08:47:00.622039+0000","last_became_active":"2024-04-21T08:44:33.489281+0000","last_became_peered":"2024-04-21T08:44:33.489281+0000","last_unstale":"2024-04-21T08:47:00.622039+0000","last_undegraded":"2024-04-21T08:47:00.622039+0000","last_fullsized":"2024-04-21T08:47:00.622039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:41:29.856165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747602+0000","last_change":"2024-04-21T08:44:34.493394+0000","last_active":"2024-04-21T08:47:00.747602+0000","last_peered":"2024-04-21T08:47:00.747602+0000","last_clean":"2024-04-21T08:47:00.747602+0000","last_became_active":"2024-04-21T08:44:34.492367+0000","last_became_peered":"2024-04-21T08:44:34.492367+0000","last_unstale":"2024-04-21T08:47:00.747602+0000","last_undegraded":"2024-04-21T08:47:00.747602+0000","last_fullsized":"2024-04-21T08:47:00.747602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:13.111266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748165+0000","last_change":"2024-04-21T08:44:38.519113+0000","last_active":"2024-04-21T08:47:00.748165+0000","last_peered":"2024-04-21T08:47:00.748165+0000","last_clean":"2024-04-21T08:47:00.748165+0000","last_became_active":"2024-04-21T08:44:38.518469+0000","last_became_peered":"2024-04-21T08:44:38.518469+0000","last_unstale":"2024-04-21T08:47:00.748165+0000","last_undegraded":"2024-04-21T08:47:00.748165+0000","last_fullsized":"2024-04-21T08:47:00.748165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:39:22.200041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'5","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747723+0000","last_change":"2024-04-21T08:46:27.492549+0000","last_active":"2024-04-21T08:47:00.747723+0000","last_peered":"2024-04-21T08:47:00.747723+0000","last_clean":"2024-04-21T08:47:00.747723+0000","last_became_active":"2024-04-21T08:46:27.488054+0000","last_became_peered":"2024-04-21T08:46:27.488054+0000","last_unstale":"2024-04-21T08:47:00.747723+0000","last_undegraded":"2024-04-21T08:47:00.747723+0000","last_fullsized":"2024-04-21T08:47:00.747723+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:51:54.797532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622446+0000","last_change":"2024-04-21T08:44:34.502226+0000","last_active":"2024-04-21T08:47:00.622446+0000","last_peered":"2024-04-21T08:47:00.622446+0000","last_clean":"2024-04-21T08:47:00.622446+0000","last_became_active":"2024-04-21T08:44:34.501775+0000","last_became_peered":"2024-04-21T08:44:34.501775+0000","last_unstale":"2024-04-21T08:47:00.622446+0000","last_undegraded":"2024-04-21T08:47:00.622446+0000","last_fullsized":"2024-04-21T08:47:00.622446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:58.325267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748655+0000","last_change":"2024-04-21T08:44:38.522250+0000","last_active":"2024-04-21T08:47:00.748655+0000","last_peered":"2024-04-21T08:47:00.748655+0000","last_clean":"2024-04-21T08:47:00.748655+0000","last_became_active":"2024-04-21T08:44:38.520870+0000","last_became_peered":"2024-04-21T08:44:38.520870+0000","last_unstale":"2024-04-21T08:47:00.748655+0000","last_undegraded":"2024-04-21T08:47:00.748655+0000","last_fullsized":"2024-04-21T08:47:00.748655+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:02.391785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747761+0000","last_change":"2024-04-21T08:46:27.496215+0000","last_active":"2024-04-21T08:47:00.747761+0000","last_peered":"2024-04-21T08:47:00.747761+0000","last_clean":"2024-04-21T08:47:00.747761+0000","last_became_active":"2024-04-21T08:46:27.488287+0000","last_became_peered":"2024-04-21T08:46:27.488287+0000","last_unstale":"2024-04-21T08:47:00.747761+0000","last_undegraded":"2024-04-21T08:47:00.747761+0000","last_fullsized":"2024-04-21T08:47:00.747761+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:08:05.390563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749473+0000","last_change":"2024-04-21T08:44:34.496034+0000","last_active":"2024-04-21T08:47:00.749473+0000","last_peered":"2024-04-21T08:47:00.749473+0000","last_clean":"2024-04-21T08:47:00.749473+0000","last_became_active":"2024-04-21T08:44:34.495641+0000","last_became_peered":"2024-04-21T08:44:34.495641+0000","last_unstale":"2024-04-21T08:47:00.749473+0000","last_undegraded":"2024-04-21T08:47:00.749473+0000","last_fullsized":"2024-04-21T08:47:00.749473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:21:18.893326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747361+0000","last_change":"2024-04-21T08:44:38.517288+0000","last_active":"2024-04-21T08:47:00.747361+0000","last_peered":"2024-04-21T08:47:00.747361+0000","last_clean":"2024-04-21T08:47:00.747361+0000","last_became_active":"2024-04-21T08:44:38.516853+0000","last_became_peered":"2024-04-21T08:44:38.516853+0000","last_unstale":"2024-04-21T08:47:00.747361+0000","last_undegraded":"2024-04-21T08:47:00.747361+0000","last_fullsized":"2024-04-21T08:47:00.747361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:27:22.371703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'7","reported_seq":163,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.237801+0000","last_change":"2024-04-21T08:44:33.491578+0000","last_active":"2024-04-21T08:47:01.237801+0000","last_peered":"2024-04-21T08:47:01.237801+0000","last_clean":"2024-04-21T08:47:01.237801+0000","last_became_active":"2024-04-21T08:44:33.491238+0000","last_became_peered":"2024-04-21T08:44:33.491238+0000","last_unstale":"2024-04-21T08:47:01.237801+0000","last_undegraded":"2024-04-21T08:47:01.237801+0000","last_fullsized":"2024-04-21T08:47:01.237801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:50.399031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749074+0000","last_change":"2024-04-21T08:44:34.497632+0000","last_active":"2024-04-21T08:47:00.749074+0000","last_peered":"2024-04-21T08:47:00.749074+0000","last_clean":"2024-04-21T08:47:00.749074+0000","last_became_active":"2024-04-21T08:44:34.497318+0000","last_became_peered":"2024-04-21T08:44:34.497318+0000","last_unstale":"2024-04-21T08:47:00.749074+0000","last_undegraded":"2024-04-21T08:47:00.749074+0000","last_fullsized":"2024-04-21T08:47:00.749074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:11:33.918530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747370+0000","last_change":"2024-04-21T08:44:38.518414+0000","last_active":"2024-04-21T08:47:00.747370+0000","last_peered":"2024-04-21T08:47:00.747370+0000","last_clean":"2024-04-21T08:47:00.747370+0000","last_became_active":"2024-04-21T08:44:38.517953+0000","last_became_peered":"2024-04-21T08:44:38.517953+0000","last_unstale":"2024-04-21T08:47:00.747370+0000","last_undegraded":"2024-04-21T08:47:00.747370+0000","last_fullsized":"2024-04-21T08:47:00.747370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:48.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.833716+0000","last_change":"2024-04-21T08:44:33.493896+0000","last_active":"2024-04-21T08:47:00.833716+0000","last_peered":"2024-04-21T08:47:00.833716+0000","last_clean":"2024-04-21T08:47:00.833716+0000","last_became_active":"2024-04-21T08:44:33.493566+0000","last_became_peered":"2024-04-21T08:44:33.493566+0000","last_unstale":"2024-04-21T08:47:00.833716+0000","last_undegraded":"2024-04-21T08:47:00.833716+0000","last_fullsized":"2024-04-21T08:47:00.833716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:47:17.307690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748612+0000","last_change":"2024-04-21T08:44:34.500406+0000","last_active":"2024-04-21T08:47:00.748612+0000","last_peered":"2024-04-21T08:47:00.748612+0000","last_clean":"2024-04-21T08:47:00.748612+0000","last_became_active":"2024-04-21T08:44:34.500048+0000","last_became_peered":"2024-04-21T08:44:34.500048+0000","last_unstale":"2024-04-21T08:47:00.748612+0000","last_undegraded":"2024-04-21T08:47:00.748612+0000","last_fullsized":"2024-04-21T08:47:00.748612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:10:58.166155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728633+0000","last_change":"2024-04-21T08:44:38.514812+0000","last_active":"2024-04-21T08:47:00.728633+0000","last_peered":"2024-04-21T08:47:00.728633+0000","last_clean":"2024-04-21T08:47:00.728633+0000","last_became_active":"2024-04-21T08:44:38.514247+0000","last_became_peered":"2024-04-21T08:44:38.514247+0000","last_unstale":"2024-04-21T08:47:00.728633+0000","last_undegraded":"2024-04-21T08:47:00.728633+0000","last_fullsized":"2024-04-21T08:47:00.728633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:06:10.588453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875149+0000","last_change":"2024-04-21T08:44:33.493887+0000","last_active":"2024-04-21T08:47:00.875149+0000","last_peered":"2024-04-21T08:47:00.875149+0000","last_clean":"2024-04-21T08:47:00.875149+0000","last_became_active":"2024-04-21T08:44:33.493555+0000","last_became_peered":"2024-04-21T08:44:33.493555+0000","last_unstale":"2024-04-21T08:47:00.875149+0000","last_undegraded":"2024-04-21T08:47:00.875149+0000","last_fullsized":"2024-04-21T08:47:00.875149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:18.089102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749096+0000","last_change":"2024-04-21T08:44:34.503259+0000","last_active":"2024-04-21T08:47:00.749096+0000","last_peered":"2024-04-21T08:47:00.749096+0000","last_clean":"2024-04-21T08:47:00.749096+0000","last_became_active":"2024-04-21T08:44:34.502845+0000","last_became_peered":"2024-04-21T08:44:34.502845+0000","last_unstale":"2024-04-21T08:47:00.749096+0000","last_undegraded":"2024-04-21T08:47:00.749096+0000","last_fullsized":"2024-04-21T08:47:00.749096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:19.902178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747653+0000","last_change":"2024-04-21T08:44:34.497608+0000","last_active":"2024-04-21T08:47:00.747653+0000","last_peered":"2024-04-21T08:47:00.747653+0000","last_clean":"2024-04-21T08:47:00.747653+0000","last_became_active":"2024-04-21T08:44:34.497255+0000","last_became_peered":"2024-04-21T08:44:34.497255+0000","last_unstale":"2024-04-21T08:47:00.747653+0000","last_undegraded":"2024-04-21T08:47:00.747653+0000","last_fullsized":"2024-04-21T08:47:00.747653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:51:15.181837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626828+0000","last_change":"2024-04-21T08:44:34.495408+0000","last_active":"2024-04-21T08:47:00.626828+0000","last_peered":"2024-04-21T08:47:00.626828+0000","last_clean":"2024-04-21T08:47:00.626828+0000","last_became_active":"2024-04-21T08:44:34.495038+0000","last_became_peered":"2024-04-21T08:44:34.495038+0000","last_unstale":"2024-04-21T08:47:00.626828+0000","last_undegraded":"2024-04-21T08:47:00.626828+0000","last_fullsized":"2024-04-21T08:47:00.626828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:59.512245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626703+0000","last_change":"2024-04-21T08:44:34.493292+0000","last_active":"2024-04-21T08:47:00.626703+0000","last_peered":"2024-04-21T08:47:00.626703+0000","last_clean":"2024-04-21T08:47:00.626703+0000","last_became_active":"2024-04-21T08:44:34.492913+0000","last_became_peered":"2024-04-21T08:44:34.492913+0000","last_unstale":"2024-04-21T08:47:00.626703+0000","last_undegraded":"2024-04-21T08:47:00.626703+0000","last_fullsized":"2024-04-21T08:47:00.626703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:24.271859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747551+0000","last_change":"2024-04-21T08:44:34.510448+0000","last_active":"2024-04-21T08:47:00.747551+0000","last_peered":"2024-04-21T08:47:00.747551+0000","last_clean":"2024-04-21T08:47:00.747551+0000","last_became_active":"2024-04-21T08:44:34.509958+0000","last_became_peered":"2024-04-21T08:44:34.509958+0000","last_unstale":"2024-04-21T08:47:00.747551+0000","last_undegraded":"2024-04-21T08:47:00.747551+0000","last_fullsized":"2024-04-21T08:47:00.747551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:36.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875000+0000","last_change":"2024-04-21T08:44:34.497415+0000","last_active":"2024-04-21T08:47:00.875000+0000","last_peered":"2024-04-21T08:47:00.875000+0000","last_clean":"2024-04-21T08:47:00.875000+0000","last_became_active":"2024-04-21T08:44:34.497120+0000","last_became_peered":"2024-04-21T08:44:34.497120+0000","last_unstale":"2024-04-21T08:47:00.875000+0000","last_undegraded":"2024-04-21T08:47:00.875000+0000","last_fullsized":"2024-04-21T08:47:00.875000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:30.232338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747974+0000","last_change":"2024-04-21T08:44:34.500356+0000","last_active":"2024-04-21T08:47:00.747974+0000","last_peered":"2024-04-21T08:47:00.747974+0000","last_clean":"2024-04-21T08:47:00.747974+0000","last_became_active":"2024-04-21T08:44:34.500032+0000","last_became_peered":"2024-04-21T08:44:34.500032+0000","last_unstale":"2024-04-21T08:47:00.747974+0000","last_undegraded":"2024-04-21T08:47:00.747974+0000","last_fullsized":"2024-04-21T08:47:00.747974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:24.900221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627464+0000","last_change":"2024-04-21T08:44:34.493629+0000","last_active":"2024-04-21T08:47:00.627464+0000","last_peered":"2024-04-21T08:47:00.627464+0000","last_clean":"2024-04-21T08:47:00.627464+0000","last_became_active":"2024-04-21T08:44:34.493273+0000","last_became_peered":"2024-04-21T08:44:34.493273+0000","last_unstale":"2024-04-21T08:47:00.627464+0000","last_undegraded":"2024-04-21T08:47:00.627464+0000","last_fullsized":"2024-04-21T08:47:00.627464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:22:42.698957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623249+0000","last_change":"2024-04-21T08:44:34.493427+0000","last_active":"2024-04-21T08:47:00.623249+0000","last_peered":"2024-04-21T08:47:00.623249+0000","last_clean":"2024-04-21T08:47:00.623249+0000","last_became_active":"2024-04-21T08:44:34.493105+0000","last_became_peered":"2024-04-21T08:44:34.493105+0000","last_unstale":"2024-04-21T08:47:00.623249+0000","last_undegraded":"2024-04-21T08:47:00.623249+0000","last_fullsized":"2024-04-21T08:47:00.623249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:37.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874153+0000","last_change":"2024-04-21T08:45:27.515899+0000","last_active":"2024-04-21T08:47:00.874153+0000","last_peered":"2024-04-21T08:47:00.874153+0000","last_clean":"2024-04-21T08:47:00.874153+0000","last_became_active":"2024-04-21T08:45:27.515567+0000","last_became_peered":"2024-04-21T08:45:27.515567+0000","last_unstale":"2024-04-21T08:47:00.874153+0000","last_undegraded":"2024-04-21T08:47:00.874153+0000","last_fullsized":"2024-04-21T08:47:00.874153+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:16.645869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622080+0000","last_change":"2024-04-21T08:44:34.499420+0000","last_active":"2024-04-21T08:47:00.622080+0000","last_peered":"2024-04-21T08:47:00.622080+0000","last_clean":"2024-04-21T08:47:00.622080+0000","last_became_active":"2024-04-21T08:44:34.499079+0000","last_became_peered":"2024-04-21T08:44:34.499079+0000","last_unstale":"2024-04-21T08:47:00.622080+0000","last_undegraded":"2024-04-21T08:47:00.622080+0000","last_fullsized":"2024-04-21T08:47:00.622080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:45.351422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747540+0000","last_change":"2024-04-21T08:44:34.493420+0000","last_active":"2024-04-21T08:47:00.747540+0000","last_peered":"2024-04-21T08:47:00.747540+0000","last_clean":"2024-04-21T08:47:00.747540+0000","last_became_active":"2024-04-21T08:44:34.493098+0000","last_became_peered":"2024-04-21T08:44:34.493098+0000","last_unstale":"2024-04-21T08:47:00.747540+0000","last_undegraded":"2024-04-21T08:47:00.747540+0000","last_fullsized":"2024-04-21T08:47:00.747540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:22:34.316822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874666+0000","last_change":"2024-04-21T08:44:34.494760+0000","last_active":"2024-04-21T08:47:00.874666+0000","last_peered":"2024-04-21T08:47:00.874666+0000","last_clean":"2024-04-21T08:47:00.874666+0000","last_became_active":"2024-04-21T08:44:34.494265+0000","last_became_peered":"2024-04-21T08:44:34.494265+0000","last_unstale":"2024-04-21T08:47:00.874666+0000","last_undegraded":"2024-04-21T08:47:00.874666+0000","last_fullsized":"2024-04-21T08:47:00.874666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:16.035030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748002+0000","last_change":"2024-04-21T08:44:34.498785+0000","last_active":"2024-04-21T08:47:00.748002+0000","last_peered":"2024-04-21T08:47:00.748002+0000","last_clean":"2024-04-21T08:47:00.748002+0000","last_became_active":"2024-04-21T08:44:34.495159+0000","last_became_peered":"2024-04-21T08:44:34.495159+0000","last_unstale":"2024-04-21T08:47:00.748002+0000","last_undegraded":"2024-04-21T08:47:00.748002+0000","last_fullsized":"2024-04-21T08:47:00.748002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:03:20.095016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600929,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1346,"num_read_kb":788,"num_write":372,"num_write_kb":1466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1802787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1081344,"data_stored":531093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28888,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716264,"statfs":{"total":95995035648,"available":95965454336,"internally_reserved":0,"allocated":1916928,"data_stored":1316546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1503232,"data_stored":927553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1495040,"data_stored":923835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1699840,"data_stored":1128597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":525745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":593812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":527983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":595904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1050269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":529819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37196,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707956,"statfs":{"total":95995035648,"available":95956946944,"internally_reserved":0,"allocated":1843200,"data_stored":1254997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":201155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:47:05.686 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:47:05.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.819+0000 7f567072b640 1 -- 172.21.15.82:0/920326234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56681072d0 msgr2=0x7f566810eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.819+0000 7f567072b640 1 --2- 172.21.15.82:0/920326234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56681072d0 0x7f566810eea0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5660009ca0 tx=0x7f566002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 -- 172.21.15.82:0/920326234 shutdown_connections 2024-04-21T08:47:05.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 --2- 172.21.15.82:0/920326234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56681072d0 0x7f566810eea0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 --2- 172.21.15.82:0/920326234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5668102bb0 0x7f5668106bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 --2- 172.21.15.82:0/920326234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5668102200 0x7f56681025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 -- 172.21.15.82:0/920326234 >> 172.21.15.82:0/920326234 conn(0x7f56680793e0 msgr2=0x7f56680797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:05.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 -- 172.21.15.82:0/920326234 shutdown_connections 2024-04-21T08:47:05.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 -- 172.21.15.82:0/920326234 wait complete. 2024-04-21T08:47:05.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 Processor -- start 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 -- start start 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5668102200 0x7f5668198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56681072d0 0x7f5668198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f566819cf90 0x7f566819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5668112220 con 0x7f5668102200 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56681120a0 con 0x7f56681072d0 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f567072b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56681123a0 con 0x7f566819cf90 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f566e4c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5668102200 0x7f5668198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.820+0000 7f566e4c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5668102200 0x7f5668198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44374/0 (socket says 172.21.15.82:44374) 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f566e4c7640 1 -- 172.21.15.82:0/3743584458 learned_addr learned my addr 172.21.15.82:0/3743584458 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f566ecc8640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f566819cf90 0x7f566819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f566dcc6640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56681072d0 0x7f5668198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:05.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f566e4c7640 1 -- 172.21.15.82:0/3743584458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f566819cf90 msgr2=0x7f566819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f566e4c7640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f566819cf90 0x7f566819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f566e4c7640 1 -- 172.21.15.82:0/3743584458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56681072d0 msgr2=0x7f5668198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:05.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f566e4c7640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56681072d0 0x7f5668198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:05.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f566e4c7640 1 -- 172.21.15.82:0/3743584458 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566819da80 con 0x7f5668102200 2024-04-21T08:47:05.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f566ecc8640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f566819cf90 0x7f566819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:05.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f566e4c7640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5668102200 0x7f5668198720 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f566400d950 tx=0x7f566400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:05.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f565f7fe640 1 -- 172.21.15.82:0/3743584458 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664017030 con 0x7f5668102200 2024-04-21T08:47:05.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f565f7fe640 1 -- 172.21.15.82:0/3743584458 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5664004020 con 0x7f5668102200 2024-04-21T08:47:05.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f567072b640 1 -- 172.21.15.82:0/3743584458 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566819dd70 con 0x7f5668102200 2024-04-21T08:47:05.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f565f7fe640 1 -- 172.21.15.82:0/3743584458 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640047a0 con 0x7f5668102200 2024-04-21T08:47:05.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.821+0000 7f567072b640 1 -- 172.21.15.82:0/3743584458 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56681a5560 con 0x7f5668102200 2024-04-21T08:47:05.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.822+0000 7f567072b640 1 -- 172.21.15.82:0/3743584458 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5668068550 con 0x7f5668102200 2024-04-21T08:47:05.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.823+0000 7f565f7fe640 1 -- 172.21.15.82:0/3743584458 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5664009a30 con 0x7f5668102200 2024-04-21T08:47:05.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.824+0000 7f565f7fe640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f56540a3e40 0x7f56540a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:05.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.824+0000 7f566dcc6640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f56540a3e40 0x7f56540a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:05.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.824+0000 7f565f7fe640 1 -- 172.21.15.82:0/3743584458 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f56640d0980 con 0x7f5668102200 2024-04-21T08:47:05.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.825+0000 7f566dcc6640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f56540a3e40 0x7f56540a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f56681017e0 tx=0x7f565800c4a0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:05.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:05.829+0000 7f565f7fe640 1 -- 172.21.15.82:0/3743584458 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f566409ca10 con 0x7f5668102200 2024-04-21T08:47:05.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:47:05.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:47:05.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:47:05.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:47:05.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: Updating smithi106:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:47:05.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/155889491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:05.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: Updating smithi082:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: Updating smithi134:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1880972080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:47:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:47:05.978 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:05 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.003+0000 7f567072b640 1 -- 172.21.15.82:0/3743584458 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5668102bb0 con 0x7f56540a3e40 2024-04-21T08:47:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.012+0000 7f565f7fe640 1 -- 172.21.15.82:0/3743584458 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339428 (secure 0 0 0) 0x7f5668102bb0 con 0x7f56540a3e40 2024-04-21T08:47:06.015 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-21T08:47:06.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.016+0000 7f567072b640 1 -- 172.21.15.82:0/3743584458 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f56540a3e40 msgr2=0x7f56540a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.016+0000 7f567072b640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f56540a3e40 0x7f56540a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f56681017e0 tx=0x7f565800c4a0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.016+0000 7f567072b640 1 -- 172.21.15.82:0/3743584458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5668102200 msgr2=0x7f5668198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.016+0000 7f567072b640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5668102200 0x7f5668198720 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f566400d950 tx=0x7f566400de10 comp rx=0 tx=0).stop 2024-04-21T08:47:06.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.016+0000 7f567072b640 1 -- 172.21.15.82:0/3743584458 shutdown_connections 2024-04-21T08:47:06.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.016+0000 7f567072b640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f566819cf90 0x7f566819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.016+0000 7f567072b640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56681072d0 0x7f5668198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.016+0000 7f567072b640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f56540a3e40 0x7f56540a6300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.016+0000 7f567072b640 1 --2- 172.21.15.82:0/3743584458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5668102200 0x7f5668198720 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.016+0000 7f567072b640 1 -- 172.21.15.82:0/3743584458 >> 172.21.15.82:0/3743584458 conn(0x7f56680793e0 msgr2=0x7f56680ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.016+0000 7f567072b640 1 -- 172.21.15.82:0/3743584458 shutdown_connections 2024-04-21T08:47:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.016+0000 7f567072b640 1 -- 172.21.15.82:0/3743584458 wait complete. 2024-04-21T08:47:06.026 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:47:06.026 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:47:06.026 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:47:06.026 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:47:06.026 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: Updating smithi106:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:47:06.026 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/155889491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:06.026 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: Updating smithi082:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:47:06.026 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: Updating smithi134:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:47:06.026 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-21T08:47:06.026 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.026 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.027 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.027 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.027 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.027 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.027 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.027 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1880972080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:06.027 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:47:06.027 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:47:06.027 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:47:06.027 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:05 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.028 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-21T08:47:04.633865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191531,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1482,"num_read_kb":907,"num_write":555,"num_write_kb":4406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":939,"ondisk_log_size":939,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357300,"kb_used_data":16376,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124584524,"statfs":{"total":1151940427776,"available":1151574552576,"internally_reserved":0,"allocated":16769024,"data_stored":9906153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.985540"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748276+0000","last_change":"2024-04-21T08:44:34.494393+0000","last_active":"2024-04-21T08:47:00.748276+0000","last_peered":"2024-04-21T08:47:00.748276+0000","last_clean":"2024-04-21T08:47:00.748276+0000","last_became_active":"2024-04-21T08:44:34.493583+0000","last_became_peered":"2024-04-21T08:44:34.493583+0000","last_unstale":"2024-04-21T08:47:00.748276+0000","last_undegraded":"2024-04-21T08:47:00.748276+0000","last_fullsized":"2024-04-21T08:47:00.748276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:37.211676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747826+0000","last_change":"2024-04-21T08:44:34.496291+0000","last_active":"2024-04-21T08:47:00.747826+0000","last_peered":"2024-04-21T08:47:00.747826+0000","last_clean":"2024-04-21T08:47:00.747826+0000","last_became_active":"2024-04-21T08:44:34.495019+0000","last_became_peered":"2024-04-21T08:44:34.495019+0000","last_unstale":"2024-04-21T08:47:00.747826+0000","last_undegraded":"2024-04-21T08:47:00.747826+0000","last_fullsized":"2024-04-21T08:47:00.747826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:05.309847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747464+0000","last_change":"2024-04-21T08:44:34.497948+0000","last_active":"2024-04-21T08:47:00.747464+0000","last_peered":"2024-04-21T08:47:00.747464+0000","last_clean":"2024-04-21T08:47:00.747464+0000","last_became_active":"2024-04-21T08:44:34.497657+0000","last_became_peered":"2024-04-21T08:44:34.497657+0000","last_unstale":"2024-04-21T08:47:00.747464+0000","last_undegraded":"2024-04-21T08:47:00.747464+0000","last_fullsized":"2024-04-21T08:47:00.747464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:11.463715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748377+0000","last_change":"2024-04-21T08:44:34.492682+0000","last_active":"2024-04-21T08:47:00.748377+0000","last_peered":"2024-04-21T08:47:00.748377+0000","last_clean":"2024-04-21T08:47:00.748377+0000","last_became_active":"2024-04-21T08:44:34.492230+0000","last_became_peered":"2024-04-21T08:44:34.492230+0000","last_unstale":"2024-04-21T08:47:00.748377+0000","last_undegraded":"2024-04-21T08:47:00.748377+0000","last_fullsized":"2024-04-21T08:47:00.748377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:40.365608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721320+0000","last_change":"2024-04-21T08:44:34.492160+0000","last_active":"2024-04-21T08:47:00.721320+0000","last_peered":"2024-04-21T08:47:00.721320+0000","last_clean":"2024-04-21T08:47:00.721320+0000","last_became_active":"2024-04-21T08:44:34.491701+0000","last_became_peered":"2024-04-21T08:44:34.491701+0000","last_unstale":"2024-04-21T08:47:00.721320+0000","last_undegraded":"2024-04-21T08:47:00.721320+0000","last_fullsized":"2024-04-21T08:47:00.721320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:50.542661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626504+0000","last_change":"2024-04-21T08:44:34.497341+0000","last_active":"2024-04-21T08:47:00.626504+0000","last_peered":"2024-04-21T08:47:00.626504+0000","last_clean":"2024-04-21T08:47:00.626504+0000","last_became_active":"2024-04-21T08:44:34.495794+0000","last_became_peered":"2024-04-21T08:44:34.495794+0000","last_unstale":"2024-04-21T08:47:00.626504+0000","last_undegraded":"2024-04-21T08:47:00.626504+0000","last_fullsized":"2024-04-21T08:47:00.626504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:13.854753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748474+0000","last_change":"2024-04-21T08:44:34.501578+0000","last_active":"2024-04-21T08:47:00.748474+0000","last_peered":"2024-04-21T08:47:00.748474+0000","last_clean":"2024-04-21T08:47:00.748474+0000","last_became_active":"2024-04-21T08:44:34.500848+0000","last_became_peered":"2024-04-21T08:44:34.500848+0000","last_unstale":"2024-04-21T08:47:00.748474+0000","last_undegraded":"2024-04-21T08:47:00.748474+0000","last_fullsized":"2024-04-21T08:47:00.748474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:53:27.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.621951+0000","last_change":"2024-04-21T08:44:34.502975+0000","last_active":"2024-04-21T08:47:00.621951+0000","last_peered":"2024-04-21T08:47:00.621951+0000","last_clean":"2024-04-21T08:47:00.621951+0000","last_became_active":"2024-04-21T08:44:34.502676+0000","last_became_peered":"2024-04-21T08:44:34.502676+0000","last_unstale":"2024-04-21T08:47:00.621951+0000","last_undegraded":"2024-04-21T08:47:00.621951+0000","last_fullsized":"2024-04-21T08:47:00.621951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:32:13.881603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748249+0000","last_change":"2024-04-21T08:44:34.495703+0000","last_active":"2024-04-21T08:47:00.748249+0000","last_peered":"2024-04-21T08:47:00.748249+0000","last_clean":"2024-04-21T08:47:00.748249+0000","last_became_active":"2024-04-21T08:44:34.495380+0000","last_became_peered":"2024-04-21T08:44:34.495380+0000","last_unstale":"2024-04-21T08:47:00.748249+0000","last_undegraded":"2024-04-21T08:47:00.748249+0000","last_fullsized":"2024-04-21T08:47:00.748249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:45:28.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626928+0000","last_change":"2024-04-21T08:44:34.499361+0000","last_active":"2024-04-21T08:47:00.626928+0000","last_peered":"2024-04-21T08:47:00.626928+0000","last_clean":"2024-04-21T08:47:00.626928+0000","last_became_active":"2024-04-21T08:44:34.499044+0000","last_became_peered":"2024-04-21T08:44:34.499044+0000","last_unstale":"2024-04-21T08:47:00.626928+0000","last_undegraded":"2024-04-21T08:47:00.626928+0000","last_fullsized":"2024-04-21T08:47:00.626928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:12.118985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748894+0000","last_change":"2024-04-21T08:44:34.499631+0000","last_active":"2024-04-21T08:47:00.748894+0000","last_peered":"2024-04-21T08:47:00.748894+0000","last_clean":"2024-04-21T08:47:00.748894+0000","last_became_active":"2024-04-21T08:44:34.499316+0000","last_became_peered":"2024-04-21T08:44:34.499316+0000","last_unstale":"2024-04-21T08:47:00.748894+0000","last_undegraded":"2024-04-21T08:47:00.748894+0000","last_fullsized":"2024-04-21T08:47:00.748894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:26.123017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748326+0000","last_change":"2024-04-21T08:44:34.504857+0000","last_active":"2024-04-21T08:47:00.748326+0000","last_peered":"2024-04-21T08:47:00.748326+0000","last_clean":"2024-04-21T08:47:00.748326+0000","last_became_active":"2024-04-21T08:44:34.504564+0000","last_became_peered":"2024-04-21T08:44:34.504564+0000","last_unstale":"2024-04-21T08:47:00.748326+0000","last_undegraded":"2024-04-21T08:47:00.748326+0000","last_fullsized":"2024-04-21T08:47:00.748326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:32:05.216162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874684+0000","last_change":"2024-04-21T08:44:34.484074+0000","last_active":"2024-04-21T08:47:00.874684+0000","last_peered":"2024-04-21T08:47:00.874684+0000","last_clean":"2024-04-21T08:47:00.874684+0000","last_became_active":"2024-04-21T08:44:34.483309+0000","last_became_peered":"2024-04-21T08:44:34.483309+0000","last_unstale":"2024-04-21T08:47:00.874684+0000","last_undegraded":"2024-04-21T08:47:00.874684+0000","last_fullsized":"2024-04-21T08:47:00.874684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:39.686048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747340+0000","last_change":"2024-04-21T08:44:34.499159+0000","last_active":"2024-04-21T08:47:00.747340+0000","last_peered":"2024-04-21T08:47:00.747340+0000","last_clean":"2024-04-21T08:47:00.747340+0000","last_became_active":"2024-04-21T08:44:34.496609+0000","last_became_peered":"2024-04-21T08:44:34.496609+0000","last_unstale":"2024-04-21T08:47:00.747340+0000","last_undegraded":"2024-04-21T08:47:00.747340+0000","last_fullsized":"2024-04-21T08:47:00.747340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:11.716321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747233+0000","last_change":"2024-04-21T08:45:27.406543+0000","last_active":"2024-04-21T08:47:00.747233+0000","last_peered":"2024-04-21T08:47:00.747233+0000","last_clean":"2024-04-21T08:47:00.747233+0000","last_became_active":"2024-04-21T08:45:27.406190+0000","last_became_peered":"2024-04-21T08:45:27.406190+0000","last_unstale":"2024-04-21T08:47:00.747233+0000","last_undegraded":"2024-04-21T08:47:00.747233+0000","last_fullsized":"2024-04-21T08:47:00.747233+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:56:17.760163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747327+0000","last_change":"2024-04-21T08:44:34.494018+0000","last_active":"2024-04-21T08:47:00.747327+0000","last_peered":"2024-04-21T08:47:00.747327+0000","last_clean":"2024-04-21T08:47:00.747327+0000","last_became_active":"2024-04-21T08:44:34.493692+0000","last_became_peered":"2024-04-21T08:44:34.493692+0000","last_unstale":"2024-04-21T08:47:00.747327+0000","last_undegraded":"2024-04-21T08:47:00.747327+0000","last_fullsized":"2024-04-21T08:47:00.747327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:25:16.944160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748187+0000","last_change":"2024-04-21T08:44:34.493231+0000","last_active":"2024-04-21T08:47:00.748187+0000","last_peered":"2024-04-21T08:47:00.748187+0000","last_clean":"2024-04-21T08:47:00.748187+0000","last_became_active":"2024-04-21T08:44:34.492826+0000","last_became_peered":"2024-04-21T08:44:34.492826+0000","last_unstale":"2024-04-21T08:47:00.748187+0000","last_undegraded":"2024-04-21T08:47:00.748187+0000","last_fullsized":"2024-04-21T08:47:00.748187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:36.815012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719352+0000","last_change":"2024-04-21T08:44:34.495142+0000","last_active":"2024-04-21T08:47:00.719352+0000","last_peered":"2024-04-21T08:47:00.719352+0000","last_clean":"2024-04-21T08:47:00.719352+0000","last_became_active":"2024-04-21T08:44:34.494803+0000","last_became_peered":"2024-04-21T08:44:34.494803+0000","last_unstale":"2024-04-21T08:47:00.719352+0000","last_undegraded":"2024-04-21T08:47:00.719352+0000","last_fullsized":"2024-04-21T08:47:00.719352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:22.974053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627186+0000","last_change":"2024-04-21T08:44:34.491252+0000","last_active":"2024-04-21T08:47:00.627186+0000","last_peered":"2024-04-21T08:47:00.627186+0000","last_clean":"2024-04-21T08:47:00.627186+0000","last_became_active":"2024-04-21T08:44:34.490814+0000","last_became_peered":"2024-04-21T08:44:34.490814+0000","last_unstale":"2024-04-21T08:47:00.627186+0000","last_undegraded":"2024-04-21T08:47:00.627186+0000","last_fullsized":"2024-04-21T08:47:00.627186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:54:52.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626862+0000","last_change":"2024-04-21T08:44:38.518647+0000","last_active":"2024-04-21T08:47:00.626862+0000","last_peered":"2024-04-21T08:47:00.626862+0000","last_clean":"2024-04-21T08:47:00.626862+0000","last_became_active":"2024-04-21T08:44:38.518239+0000","last_became_peered":"2024-04-21T08:44:38.518239+0000","last_unstale":"2024-04-21T08:47:00.626862+0000","last_undegraded":"2024-04-21T08:47:00.626862+0000","last_fullsized":"2024-04-21T08:47:00.626862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:44.781542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":167,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719204+0000","last_change":"2024-04-21T08:44:33.484484+0000","last_active":"2024-04-21T08:47:00.719204+0000","last_peered":"2024-04-21T08:47:00.719204+0000","last_clean":"2024-04-21T08:47:00.719204+0000","last_became_active":"2024-04-21T08:44:33.484160+0000","last_became_peered":"2024-04-21T08:44:33.484160+0000","last_unstale":"2024-04-21T08:47:00.719204+0000","last_undegraded":"2024-04-21T08:47:00.719204+0000","last_fullsized":"2024-04-21T08:47:00.719204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:45.962598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718920+0000","last_change":"2024-04-21T08:46:27.484947+0000","last_active":"2024-04-21T08:47:00.718920+0000","last_peered":"2024-04-21T08:47:00.718920+0000","last_clean":"2024-04-21T08:47:00.718920+0000","last_became_active":"2024-04-21T08:46:27.484530+0000","last_became_peered":"2024-04-21T08:46:27.484530+0000","last_unstale":"2024-04-21T08:47:00.718920+0000","last_undegraded":"2024-04-21T08:47:00.718920+0000","last_fullsized":"2024-04-21T08:47:00.718920+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:59.248176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747712+0000","last_change":"2024-04-21T08:45:27.518006+0000","last_active":"2024-04-21T08:47:00.747712+0000","last_peered":"2024-04-21T08:47:00.747712+0000","last_clean":"2024-04-21T08:47:00.747712+0000","last_became_active":"2024-04-21T08:45:27.517409+0000","last_became_peered":"2024-04-21T08:45:27.517409+0000","last_unstale":"2024-04-21T08:47:00.747712+0000","last_undegraded":"2024-04-21T08:47:00.747712+0000","last_fullsized":"2024-04-21T08:47:00.747712+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:45:01.253587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623343+0000","last_change":"2024-04-21T08:44:33.490508+0000","last_active":"2024-04-21T08:47:00.623343+0000","last_peered":"2024-04-21T08:47:00.623343+0000","last_clean":"2024-04-21T08:47:00.623343+0000","last_became_active":"2024-04-21T08:44:33.490084+0000","last_became_peered":"2024-04-21T08:44:33.490084+0000","last_unstale":"2024-04-21T08:47:00.623343+0000","last_undegraded":"2024-04-21T08:47:00.623343+0000","last_fullsized":"2024-04-21T08:47:00.623343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:21.553435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747770+0000","last_change":"2024-04-21T08:44:34.491843+0000","last_active":"2024-04-21T08:47:00.747770+0000","last_peered":"2024-04-21T08:47:00.747770+0000","last_clean":"2024-04-21T08:47:00.747770+0000","last_became_active":"2024-04-21T08:44:34.490578+0000","last_became_peered":"2024-04-21T08:44:34.490578+0000","last_unstale":"2024-04-21T08:47:00.747770+0000","last_undegraded":"2024-04-21T08:47:00.747770+0000","last_fullsized":"2024-04-21T08:47:00.747770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:05:12.272330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748032+0000","last_change":"2024-04-21T08:44:38.520258+0000","last_active":"2024-04-21T08:47:00.748032+0000","last_peered":"2024-04-21T08:47:00.748032+0000","last_clean":"2024-04-21T08:47:00.748032+0000","last_became_active":"2024-04-21T08:44:38.519824+0000","last_became_peered":"2024-04-21T08:44:38.519824+0000","last_unstale":"2024-04-21T08:47:00.748032+0000","last_undegraded":"2024-04-21T08:47:00.748032+0000","last_fullsized":"2024-04-21T08:47:00.748032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:39.716745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721769+0000","last_change":"2024-04-21T08:44:33.484934+0000","last_active":"2024-04-21T08:47:00.721769+0000","last_peered":"2024-04-21T08:47:00.721769+0000","last_clean":"2024-04-21T08:47:00.721769+0000","last_became_active":"2024-04-21T08:44:33.484591+0000","last_became_peered":"2024-04-21T08:44:33.484591+0000","last_unstale":"2024-04-21T08:47:00.721769+0000","last_undegraded":"2024-04-21T08:47:00.721769+0000","last_fullsized":"2024-04-21T08:47:00.721769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:48.826611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747621+0000","last_change":"2024-04-21T08:44:34.501109+0000","last_active":"2024-04-21T08:47:00.747621+0000","last_peered":"2024-04-21T08:47:00.747621+0000","last_clean":"2024-04-21T08:47:00.747621+0000","last_became_active":"2024-04-21T08:44:34.500750+0000","last_became_peered":"2024-04-21T08:44:34.500750+0000","last_unstale":"2024-04-21T08:47:00.747621+0000","last_undegraded":"2024-04-21T08:47:00.747621+0000","last_fullsized":"2024-04-21T08:47:00.747621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:29:28.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747712+0000","last_change":"2024-04-21T08:44:38.519633+0000","last_active":"2024-04-21T08:47:00.747712+0000","last_peered":"2024-04-21T08:47:00.747712+0000","last_clean":"2024-04-21T08:47:00.747712+0000","last_became_active":"2024-04-21T08:44:38.519215+0000","last_became_peered":"2024-04-21T08:44:38.519215+0000","last_unstale":"2024-04-21T08:47:00.747712+0000","last_undegraded":"2024-04-21T08:47:00.747712+0000","last_fullsized":"2024-04-21T08:47:00.747712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:43:59.695844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721688+0000","last_change":"2024-04-21T08:44:33.483267+0000","last_active":"2024-04-21T08:47:00.721688+0000","last_peered":"2024-04-21T08:47:00.721688+0000","last_clean":"2024-04-21T08:47:00.721688+0000","last_became_active":"2024-04-21T08:44:33.482932+0000","last_became_peered":"2024-04-21T08:44:33.482932+0000","last_unstale":"2024-04-21T08:47:00.721688+0000","last_undegraded":"2024-04-21T08:47:00.721688+0000","last_fullsized":"2024-04-21T08:47:00.721688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:53.311550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729235+0000","last_change":"2024-04-21T08:44:34.497389+0000","last_active":"2024-04-21T08:47:00.729235+0000","last_peered":"2024-04-21T08:47:00.729235+0000","last_clean":"2024-04-21T08:47:00.729235+0000","last_became_active":"2024-04-21T08:44:34.497030+0000","last_became_peered":"2024-04-21T08:44:34.497030+0000","last_unstale":"2024-04-21T08:47:00.729235+0000","last_undegraded":"2024-04-21T08:47:00.729235+0000","last_fullsized":"2024-04-21T08:47:00.729235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:20.371888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626918+0000","last_change":"2024-04-21T08:44:38.518095+0000","last_active":"2024-04-21T08:47:00.626918+0000","last_peered":"2024-04-21T08:47:00.626918+0000","last_clean":"2024-04-21T08:47:00.626918+0000","last_became_active":"2024-04-21T08:44:38.517010+0000","last_became_peered":"2024-04-21T08:44:38.517010+0000","last_unstale":"2024-04-21T08:47:00.626918+0000","last_undegraded":"2024-04-21T08:47:00.626918+0000","last_fullsized":"2024-04-21T08:47:00.626918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:02.379512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'8","reported_seq":167,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.167577+0000","last_change":"2024-04-21T08:44:33.487999+0000","last_active":"2024-04-21T08:47:01.167577+0000","last_peered":"2024-04-21T08:47:01.167577+0000","last_clean":"2024-04-21T08:47:01.167577+0000","last_became_active":"2024-04-21T08:44:33.487666+0000","last_became_peered":"2024-04-21T08:44:33.487666+0000","last_unstale":"2024-04-21T08:47:01.167577+0000","last_undegraded":"2024-04-21T08:47:01.167577+0000","last_fullsized":"2024-04-21T08:47:01.167577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:01.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1712,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721477+0000","last_change":"2024-04-21T08:44:34.494213+0000","last_active":"2024-04-21T08:47:00.721477+0000","last_peered":"2024-04-21T08:47:00.721477+0000","last_clean":"2024-04-21T08:47:00.721477+0000","last_became_active":"2024-04-21T08:44:34.493853+0000","last_became_peered":"2024-04-21T08:44:34.493853+0000","last_unstale":"2024-04-21T08:47:00.721477+0000","last_undegraded":"2024-04-21T08:47:00.721477+0000","last_fullsized":"2024-04-21T08:47:00.721477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:42.499131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627012+0000","last_change":"2024-04-21T08:45:27.410579+0000","last_active":"2024-04-21T08:47:00.627012+0000","last_peered":"2024-04-21T08:47:00.627012+0000","last_clean":"2024-04-21T08:47:00.627012+0000","last_became_active":"2024-04-21T08:45:27.409929+0000","last_became_peered":"2024-04-21T08:45:27.409929+0000","last_unstale":"2024-04-21T08:47:00.627012+0000","last_undegraded":"2024-04-21T08:47:00.627012+0000","last_fullsized":"2024-04-21T08:47:00.627012+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:35.897482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626508+0000","last_change":"2024-04-21T08:44:33.488751+0000","last_active":"2024-04-21T08:47:00.626508+0000","last_peered":"2024-04-21T08:47:00.626508+0000","last_clean":"2024-04-21T08:47:00.626508+0000","last_became_active":"2024-04-21T08:44:33.488425+0000","last_became_peered":"2024-04-21T08:44:33.488425+0000","last_unstale":"2024-04-21T08:47:00.626508+0000","last_undegraded":"2024-04-21T08:47:00.626508+0000","last_fullsized":"2024-04-21T08:47:00.626508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:28:47.868814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729103+0000","last_change":"2024-04-21T08:44:34.492770+0000","last_active":"2024-04-21T08:47:00.729103+0000","last_peered":"2024-04-21T08:47:00.729103+0000","last_clean":"2024-04-21T08:47:00.729103+0000","last_became_active":"2024-04-21T08:44:34.491340+0000","last_became_peered":"2024-04-21T08:44:34.491340+0000","last_unstale":"2024-04-21T08:47:00.729103+0000","last_undegraded":"2024-04-21T08:47:00.729103+0000","last_fullsized":"2024-04-21T08:47:00.729103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:47:54.280990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748097+0000","last_change":"2024-04-21T08:44:38.518888+0000","last_active":"2024-04-21T08:47:00.748097+0000","last_peered":"2024-04-21T08:47:00.748097+0000","last_clean":"2024-04-21T08:47:00.748097+0000","last_became_active":"2024-04-21T08:44:38.518421+0000","last_became_peered":"2024-04-21T08:44:38.518421+0000","last_unstale":"2024-04-21T08:47:00.748097+0000","last_undegraded":"2024-04-21T08:47:00.748097+0000","last_fullsized":"2024-04-21T08:47:00.748097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:21.119364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874482+0000","last_change":"2024-04-21T08:44:33.484383+0000","last_active":"2024-04-21T08:47:00.874482+0000","last_peered":"2024-04-21T08:47:00.874482+0000","last_clean":"2024-04-21T08:47:00.874482+0000","last_became_active":"2024-04-21T08:44:33.484083+0000","last_became_peered":"2024-04-21T08:44:33.484083+0000","last_unstale":"2024-04-21T08:47:00.874482+0000","last_undegraded":"2024-04-21T08:47:00.874482+0000","last_fullsized":"2024-04-21T08:47:00.874482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:13:14.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622126+0000","last_change":"2024-04-21T08:44:34.495636+0000","last_active":"2024-04-21T08:47:00.622126+0000","last_peered":"2024-04-21T08:47:00.622126+0000","last_clean":"2024-04-21T08:47:00.622126+0000","last_became_active":"2024-04-21T08:44:34.494967+0000","last_became_peered":"2024-04-21T08:44:34.494967+0000","last_unstale":"2024-04-21T08:47:00.622126+0000","last_undegraded":"2024-04-21T08:47:00.622126+0000","last_fullsized":"2024-04-21T08:47:00.622126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:47:02.379208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747724+0000","last_change":"2024-04-21T08:44:38.515145+0000","last_active":"2024-04-21T08:47:00.747724+0000","last_peered":"2024-04-21T08:47:00.747724+0000","last_clean":"2024-04-21T08:47:00.747724+0000","last_became_active":"2024-04-21T08:44:38.514755+0000","last_became_peered":"2024-04-21T08:44:38.514755+0000","last_unstale":"2024-04-21T08:47:00.747724+0000","last_undegraded":"2024-04-21T08:47:00.747724+0000","last_fullsized":"2024-04-21T08:47:00.747724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:38:15.826293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728869+0000","last_change":"2024-04-21T08:44:33.484714+0000","last_active":"2024-04-21T08:47:00.728869+0000","last_peered":"2024-04-21T08:47:00.728869+0000","last_clean":"2024-04-21T08:47:00.728869+0000","last_became_active":"2024-04-21T08:44:33.484385+0000","last_became_peered":"2024-04-21T08:44:33.484385+0000","last_unstale":"2024-04-21T08:47:00.728869+0000","last_undegraded":"2024-04-21T08:47:00.728869+0000","last_fullsized":"2024-04-21T08:47:00.728869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:03.406860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718544+0000","last_change":"2024-04-21T08:44:34.490989+0000","last_active":"2024-04-21T08:47:00.718544+0000","last_peered":"2024-04-21T08:47:00.718544+0000","last_clean":"2024-04-21T08:47:00.718544+0000","last_became_active":"2024-04-21T08:44:34.490658+0000","last_became_peered":"2024-04-21T08:44:34.490658+0000","last_unstale":"2024-04-21T08:47:00.718544+0000","last_undegraded":"2024-04-21T08:47:00.718544+0000","last_fullsized":"2024-04-21T08:47:00.718544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:42:15.537913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622288+0000","last_change":"2024-04-21T08:44:38.519723+0000","last_active":"2024-04-21T08:47:00.622288+0000","last_peered":"2024-04-21T08:47:00.622288+0000","last_clean":"2024-04-21T08:47:00.622288+0000","last_became_active":"2024-04-21T08:44:38.519336+0000","last_became_peered":"2024-04-21T08:44:38.519336+0000","last_unstale":"2024-04-21T08:47:00.622288+0000","last_undegraded":"2024-04-21T08:47:00.622288+0000","last_fullsized":"2024-04-21T08:47:00.622288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:00.332153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'6","reported_seq":281,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.237217+0000","last_change":"2024-04-21T08:44:33.484404+0000","last_active":"2024-04-21T08:47:01.237217+0000","last_peered":"2024-04-21T08:47:01.237217+0000","last_clean":"2024-04-21T08:47:01.237217+0000","last_became_active":"2024-04-21T08:44:33.484088+0000","last_became_peered":"2024-04-21T08:44:33.484088+0000","last_unstale":"2024-04-21T08:47:01.237217+0000","last_undegraded":"2024-04-21T08:47:01.237217+0000","last_fullsized":"2024-04-21T08:47:01.237217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:41.948622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":242,"num_read_kb":242,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748214+0000","last_change":"2024-04-21T08:44:34.502598+0000","last_active":"2024-04-21T08:47:00.748214+0000","last_peered":"2024-04-21T08:47:00.748214+0000","last_clean":"2024-04-21T08:47:00.748214+0000","last_became_active":"2024-04-21T08:44:34.502057+0000","last_became_peered":"2024-04-21T08:44:34.502057+0000","last_unstale":"2024-04-21T08:47:00.748214+0000","last_undegraded":"2024-04-21T08:47:00.748214+0000","last_fullsized":"2024-04-21T08:47:00.748214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:24.052873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718327+0000","last_change":"2024-04-21T08:44:38.519699+0000","last_active":"2024-04-21T08:47:00.718327+0000","last_peered":"2024-04-21T08:47:00.718327+0000","last_clean":"2024-04-21T08:47:00.718327+0000","last_became_active":"2024-04-21T08:44:38.517655+0000","last_became_peered":"2024-04-21T08:44:38.517655+0000","last_unstale":"2024-04-21T08:47:00.718327+0000","last_undegraded":"2024-04-21T08:47:00.718327+0000","last_fullsized":"2024-04-21T08:47:00.718327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:12.784479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748806+0000","last_change":"2024-04-21T08:46:27.494747+0000","last_active":"2024-04-21T08:47:00.748806+0000","last_peered":"2024-04-21T08:47:00.748806+0000","last_clean":"2024-04-21T08:47:00.748806+0000","last_became_active":"2024-04-21T08:46:27.490469+0000","last_became_peered":"2024-04-21T08:46:27.490469+0000","last_unstale":"2024-04-21T08:47:00.748806+0000","last_undegraded":"2024-04-21T08:47:00.748806+0000","last_fullsized":"2024-04-21T08:47:00.748806+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:16:35.257267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747850+0000","last_change":"2024-04-21T08:45:27.518395+0000","last_active":"2024-04-21T08:47:00.747850+0000","last_peered":"2024-04-21T08:47:00.747850+0000","last_clean":"2024-04-21T08:47:00.747850+0000","last_became_active":"2024-04-21T08:45:27.517969+0000","last_became_peered":"2024-04-21T08:45:27.517969+0000","last_unstale":"2024-04-21T08:47:00.747850+0000","last_undegraded":"2024-04-21T08:47:00.747850+0000","last_fullsized":"2024-04-21T08:47:00.747850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:55.306556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748406+0000","last_change":"2024-04-21T08:44:38.517956+0000","last_active":"2024-04-21T08:47:00.748406+0000","last_peered":"2024-04-21T08:47:00.748406+0000","last_clean":"2024-04-21T08:47:00.748406+0000","last_became_active":"2024-04-21T08:44:38.517422+0000","last_became_peered":"2024-04-21T08:44:38.517422+0000","last_unstale":"2024-04-21T08:47:00.748406+0000","last_undegraded":"2024-04-21T08:47:00.748406+0000","last_fullsized":"2024-04-21T08:47:00.748406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:04.650784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729223+0000","last_change":"2024-04-21T08:44:33.489151+0000","last_active":"2024-04-21T08:47:00.729223+0000","last_peered":"2024-04-21T08:47:00.729223+0000","last_clean":"2024-04-21T08:47:00.729223+0000","last_became_active":"2024-04-21T08:44:33.488781+0000","last_became_peered":"2024-04-21T08:44:33.488781+0000","last_unstale":"2024-04-21T08:47:00.729223+0000","last_undegraded":"2024-04-21T08:47:00.729223+0000","last_fullsized":"2024-04-21T08:47:00.729223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:40.675100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747905+0000","last_change":"2024-04-21T08:44:34.496528+0000","last_active":"2024-04-21T08:47:00.747905+0000","last_peered":"2024-04-21T08:47:00.747905+0000","last_clean":"2024-04-21T08:47:00.747905+0000","last_became_active":"2024-04-21T08:44:34.496223+0000","last_became_peered":"2024-04-21T08:44:34.496223+0000","last_unstale":"2024-04-21T08:47:00.747905+0000","last_undegraded":"2024-04-21T08:47:00.747905+0000","last_fullsized":"2024-04-21T08:47:00.747905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:56.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718461+0000","last_change":"2024-04-21T08:44:38.517537+0000","last_active":"2024-04-21T08:47:00.718461+0000","last_peered":"2024-04-21T08:47:00.718461+0000","last_clean":"2024-04-21T08:47:00.718461+0000","last_became_active":"2024-04-21T08:44:38.517134+0000","last_became_peered":"2024-04-21T08:44:38.517134+0000","last_unstale":"2024-04-21T08:47:00.718461+0000","last_undegraded":"2024-04-21T08:47:00.718461+0000","last_fullsized":"2024-04-21T08:47:00.718461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:38.634418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'39","reported_seq":326,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.179097+0000","last_change":"2024-04-21T08:44:33.485582+0000","last_active":"2024-04-21T08:47:01.179097+0000","last_peered":"2024-04-21T08:47:01.179097+0000","last_clean":"2024-04-21T08:47:01.179097+0000","last_became_active":"2024-04-21T08:44:33.485249+0000","last_became_peered":"2024-04-21T08:44:33.485249+0000","last_unstale":"2024-04-21T08:47:01.179097+0000","last_undegraded":"2024-04-21T08:47:01.179097+0000","last_fullsized":"2024-04-21T08:47:01.179097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:16:10.571694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67037,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":254,"num_read_kb":193,"num_write":41,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721670+0000","last_change":"2024-04-21T08:44:34.491361+0000","last_active":"2024-04-21T08:47:00.721670+0000","last_peered":"2024-04-21T08:47:00.721670+0000","last_clean":"2024-04-21T08:47:00.721670+0000","last_became_active":"2024-04-21T08:44:34.491050+0000","last_became_peered":"2024-04-21T08:44:34.491050+0000","last_unstale":"2024-04-21T08:47:00.721670+0000","last_undegraded":"2024-04-21T08:47:00.721670+0000","last_fullsized":"2024-04-21T08:47:00.721670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:39.082448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729534+0000","last_change":"2024-04-21T08:44:38.528745+0000","last_active":"2024-04-21T08:47:00.729534+0000","last_peered":"2024-04-21T08:47:00.729534+0000","last_clean":"2024-04-21T08:47:00.729534+0000","last_became_active":"2024-04-21T08:44:38.528275+0000","last_became_peered":"2024-04-21T08:44:38.528275+0000","last_unstale":"2024-04-21T08:47:00.729534+0000","last_undegraded":"2024-04-21T08:47:00.729534+0000","last_fullsized":"2024-04-21T08:47:00.729534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:06:11.338420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626897+0000","last_change":"2024-04-21T08:44:33.482059+0000","last_active":"2024-04-21T08:47:00.626897+0000","last_peered":"2024-04-21T08:47:00.626897+0000","last_clean":"2024-04-21T08:47:00.626897+0000","last_became_active":"2024-04-21T08:44:33.481663+0000","last_became_peered":"2024-04-21T08:44:33.481663+0000","last_unstale":"2024-04-21T08:47:00.626897+0000","last_undegraded":"2024-04-21T08:47:00.626897+0000","last_fullsized":"2024-04-21T08:47:00.626897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:01.240452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874587+0000","last_change":"2024-04-21T08:44:34.491398+0000","last_active":"2024-04-21T08:47:00.874587+0000","last_peered":"2024-04-21T08:47:00.874587+0000","last_clean":"2024-04-21T08:47:00.874587+0000","last_became_active":"2024-04-21T08:44:34.491073+0000","last_became_peered":"2024-04-21T08:44:34.491073+0000","last_unstale":"2024-04-21T08:47:00.874587+0000","last_undegraded":"2024-04-21T08:47:00.874587+0000","last_fullsized":"2024-04-21T08:47:00.874587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:22.566063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747937+0000","last_change":"2024-04-21T08:44:38.515653+0000","last_active":"2024-04-21T08:47:00.747937+0000","last_peered":"2024-04-21T08:47:00.747937+0000","last_clean":"2024-04-21T08:47:00.747937+0000","last_became_active":"2024-04-21T08:44:38.514796+0000","last_became_peered":"2024-04-21T08:44:38.514796+0000","last_unstale":"2024-04-21T08:47:00.747937+0000","last_undegraded":"2024-04-21T08:47:00.747937+0000","last_fullsized":"2024-04-21T08:47:00.747937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:00:22.662136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729003+0000","last_change":"2024-04-21T08:44:33.487073+0000","last_active":"2024-04-21T08:47:00.729003+0000","last_peered":"2024-04-21T08:47:00.729003+0000","last_clean":"2024-04-21T08:47:00.729003+0000","last_became_active":"2024-04-21T08:44:33.486740+0000","last_became_peered":"2024-04-21T08:44:33.486740+0000","last_unstale":"2024-04-21T08:47:00.729003+0000","last_undegraded":"2024-04-21T08:47:00.729003+0000","last_fullsized":"2024-04-21T08:47:00.729003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:32.673074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748512+0000","last_change":"2024-04-21T08:44:34.496757+0000","last_active":"2024-04-21T08:47:00.748512+0000","last_peered":"2024-04-21T08:47:00.748512+0000","last_clean":"2024-04-21T08:47:00.748512+0000","last_became_active":"2024-04-21T08:44:34.496422+0000","last_became_peered":"2024-04-21T08:44:34.496422+0000","last_unstale":"2024-04-21T08:47:00.748512+0000","last_undegraded":"2024-04-21T08:47:00.748512+0000","last_fullsized":"2024-04-21T08:47:00.748512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:08.686810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718934+0000","last_change":"2024-04-21T08:44:38.520935+0000","last_active":"2024-04-21T08:47:00.718934+0000","last_peered":"2024-04-21T08:47:00.718934+0000","last_clean":"2024-04-21T08:47:00.718934+0000","last_became_active":"2024-04-21T08:44:38.520481+0000","last_became_peered":"2024-04-21T08:44:38.520481+0000","last_unstale":"2024-04-21T08:47:00.718934+0000","last_undegraded":"2024-04-21T08:47:00.718934+0000","last_fullsized":"2024-04-21T08:47:00.718934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:12.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'7","reported_seq":166,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.165119+0000","last_change":"2024-04-21T08:44:33.485797+0000","last_active":"2024-04-21T08:47:01.165119+0000","last_peered":"2024-04-21T08:47:01.165119+0000","last_clean":"2024-04-21T08:47:01.165119+0000","last_became_active":"2024-04-21T08:44:33.485496+0000","last_became_peered":"2024-04-21T08:44:33.485496+0000","last_unstale":"2024-04-21T08:47:01.165119+0000","last_undegraded":"2024-04-21T08:47:01.165119+0000","last_fullsized":"2024-04-21T08:47:01.165119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:11:34.604611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748282+0000","last_change":"2024-04-21T08:44:34.506453+0000","last_active":"2024-04-21T08:47:00.748282+0000","last_peered":"2024-04-21T08:47:00.748282+0000","last_clean":"2024-04-21T08:47:00.748282+0000","last_became_active":"2024-04-21T08:44:34.506121+0000","last_became_peered":"2024-04-21T08:44:34.506121+0000","last_unstale":"2024-04-21T08:47:00.748282+0000","last_undegraded":"2024-04-21T08:47:00.748282+0000","last_fullsized":"2024-04-21T08:47:00.748282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:07.927366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875250+0000","last_change":"2024-04-21T08:44:38.523718+0000","last_active":"2024-04-21T08:47:00.875250+0000","last_peered":"2024-04-21T08:47:00.875250+0000","last_clean":"2024-04-21T08:47:00.875250+0000","last_became_active":"2024-04-21T08:44:38.523003+0000","last_became_peered":"2024-04-21T08:44:38.523003+0000","last_unstale":"2024-04-21T08:47:00.875250+0000","last_undegraded":"2024-04-21T08:47:00.875250+0000","last_fullsized":"2024-04-21T08:47:00.875250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:20.837450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721393+0000","last_change":"2024-04-21T08:44:33.482029+0000","last_active":"2024-04-21T08:47:00.721393+0000","last_peered":"2024-04-21T08:47:00.721393+0000","last_clean":"2024-04-21T08:47:00.721393+0000","last_became_active":"2024-04-21T08:44:33.481713+0000","last_became_peered":"2024-04-21T08:44:33.481713+0000","last_unstale":"2024-04-21T08:47:00.721393+0000","last_undegraded":"2024-04-21T08:47:00.721393+0000","last_fullsized":"2024-04-21T08:47:00.721393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:46:34.717244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728921+0000","last_change":"2024-04-21T08:44:34.491944+0000","last_active":"2024-04-21T08:47:00.728921+0000","last_peered":"2024-04-21T08:47:00.728921+0000","last_clean":"2024-04-21T08:47:00.728921+0000","last_became_active":"2024-04-21T08:44:34.491638+0000","last_became_peered":"2024-04-21T08:44:34.491638+0000","last_unstale":"2024-04-21T08:47:00.728921+0000","last_undegraded":"2024-04-21T08:47:00.728921+0000","last_fullsized":"2024-04-21T08:47:00.728921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:08:07.925937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728793+0000","last_change":"2024-04-21T08:44:38.528971+0000","last_active":"2024-04-21T08:47:00.728793+0000","last_peered":"2024-04-21T08:47:00.728793+0000","last_clean":"2024-04-21T08:47:00.728793+0000","last_became_active":"2024-04-21T08:44:38.528453+0000","last_became_peered":"2024-04-21T08:44:38.528453+0000","last_unstale":"2024-04-21T08:47:00.728793+0000","last_undegraded":"2024-04-21T08:47:00.728793+0000","last_fullsized":"2024-04-21T08:47:00.728793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:50:31.237749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721764+0000","last_change":"2024-04-21T08:44:33.479262+0000","last_active":"2024-04-21T08:47:00.721764+0000","last_peered":"2024-04-21T08:47:00.721764+0000","last_clean":"2024-04-21T08:47:00.721764+0000","last_became_active":"2024-04-21T08:44:33.478907+0000","last_became_peered":"2024-04-21T08:44:33.478907+0000","last_unstale":"2024-04-21T08:47:00.721764+0000","last_undegraded":"2024-04-21T08:47:00.721764+0000","last_fullsized":"2024-04-21T08:47:00.721764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:33:43.879126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748024+0000","last_change":"2024-04-21T08:44:34.498503+0000","last_active":"2024-04-21T08:47:00.748024+0000","last_peered":"2024-04-21T08:47:00.748024+0000","last_clean":"2024-04-21T08:47:00.748024+0000","last_became_active":"2024-04-21T08:44:34.495140+0000","last_became_peered":"2024-04-21T08:44:34.495140+0000","last_unstale":"2024-04-21T08:47:00.748024+0000","last_undegraded":"2024-04-21T08:47:00.748024+0000","last_fullsized":"2024-04-21T08:47:00.748024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:24.518349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721372+0000","last_change":"2024-04-21T08:44:38.517717+0000","last_active":"2024-04-21T08:47:00.721372+0000","last_peered":"2024-04-21T08:47:00.721372+0000","last_clean":"2024-04-21T08:47:00.721372+0000","last_became_active":"2024-04-21T08:44:38.517272+0000","last_became_peered":"2024-04-21T08:44:38.517272+0000","last_unstale":"2024-04-21T08:47:00.721372+0000","last_undegraded":"2024-04-21T08:47:00.721372+0000","last_fullsized":"2024-04-21T08:47:00.721372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.664130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718792+0000","last_change":"2024-04-21T08:44:33.477498+0000","last_active":"2024-04-21T08:47:00.718792+0000","last_peered":"2024-04-21T08:47:00.718792+0000","last_clean":"2024-04-21T08:47:00.718792+0000","last_became_active":"2024-04-21T08:44:33.477139+0000","last_became_peered":"2024-04-21T08:44:33.477139+0000","last_unstale":"2024-04-21T08:47:00.718792+0000","last_undegraded":"2024-04-21T08:47:00.718792+0000","last_fullsized":"2024-04-21T08:47:00.718792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:26:59.390583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626701+0000","last_change":"2024-04-21T08:44:34.493586+0000","last_active":"2024-04-21T08:47:00.626701+0000","last_peered":"2024-04-21T08:47:00.626701+0000","last_clean":"2024-04-21T08:47:00.626701+0000","last_became_active":"2024-04-21T08:44:34.493258+0000","last_became_peered":"2024-04-21T08:44:34.493258+0000","last_unstale":"2024-04-21T08:47:00.626701+0000","last_undegraded":"2024-04-21T08:47:00.626701+0000","last_fullsized":"2024-04-21T08:47:00.626701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:11:18.783332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874853+0000","last_change":"2024-04-21T08:44:38.519783+0000","last_active":"2024-04-21T08:47:00.874853+0000","last_peered":"2024-04-21T08:47:00.874853+0000","last_clean":"2024-04-21T08:47:00.874853+0000","last_became_active":"2024-04-21T08:44:38.519397+0000","last_became_peered":"2024-04-21T08:44:38.519397+0000","last_unstale":"2024-04-21T08:47:00.874853+0000","last_undegraded":"2024-04-21T08:47:00.874853+0000","last_fullsized":"2024-04-21T08:47:00.874853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:08.740240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":160,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749428+0000","last_change":"2024-04-21T08:44:33.490098+0000","last_active":"2024-04-21T08:47:00.749428+0000","last_peered":"2024-04-21T08:47:00.749428+0000","last_clean":"2024-04-21T08:47:00.749428+0000","last_became_active":"2024-04-21T08:44:33.489691+0000","last_became_peered":"2024-04-21T08:44:33.489691+0000","last_unstale":"2024-04-21T08:47:00.749428+0000","last_undegraded":"2024-04-21T08:47:00.749428+0000","last_fullsized":"2024-04-21T08:47:00.749428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:44.965998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626543+0000","last_change":"2024-04-21T08:46:27.489580+0000","last_active":"2024-04-21T08:47:00.626543+0000","last_peered":"2024-04-21T08:47:00.626543+0000","last_clean":"2024-04-21T08:47:00.626543+0000","last_became_active":"2024-04-21T08:46:27.489086+0000","last_became_peered":"2024-04-21T08:46:27.489086+0000","last_unstale":"2024-04-21T08:47:00.626543+0000","last_undegraded":"2024-04-21T08:47:00.626543+0000","last_fullsized":"2024-04-21T08:47:00.626543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:38:50.618957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874908+0000","last_change":"2024-04-21T08:44:38.521010+0000","last_active":"2024-04-21T08:47:00.874908+0000","last_peered":"2024-04-21T08:47:00.874908+0000","last_clean":"2024-04-21T08:47:00.874908+0000","last_became_active":"2024-04-21T08:44:38.520605+0000","last_became_peered":"2024-04-21T08:44:38.520605+0000","last_unstale":"2024-04-21T08:47:00.874908+0000","last_undegraded":"2024-04-21T08:47:00.874908+0000","last_fullsized":"2024-04-21T08:47:00.874908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:50.378803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728846+0000","last_change":"2024-04-21T08:45:27.410137+0000","last_active":"2024-04-21T08:47:00.728846+0000","last_peered":"2024-04-21T08:47:00.728846+0000","last_clean":"2024-04-21T08:47:00.728846+0000","last_became_active":"2024-04-21T08:45:27.409728+0000","last_became_peered":"2024-04-21T08:45:27.409728+0000","last_unstale":"2024-04-21T08:47:00.728846+0000","last_undegraded":"2024-04-21T08:47:00.728846+0000","last_fullsized":"2024-04-21T08:47:00.728846+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:17.934902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":162,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.885117+0000","last_change":"2024-04-21T08:44:33.494031+0000","last_active":"2024-04-21T08:47:00.885117+0000","last_peered":"2024-04-21T08:47:00.885117+0000","last_clean":"2024-04-21T08:47:00.885117+0000","last_became_active":"2024-04-21T08:44:33.493580+0000","last_became_peered":"2024-04-21T08:44:33.493580+0000","last_unstale":"2024-04-21T08:47:00.885117+0000","last_undegraded":"2024-04-21T08:47:00.885117+0000","last_fullsized":"2024-04-21T08:47:00.885117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:39:48.230616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748658+0000","last_change":"2024-04-21T08:44:38.518067+0000","last_active":"2024-04-21T08:47:00.748658+0000","last_peered":"2024-04-21T08:47:00.748658+0000","last_clean":"2024-04-21T08:47:00.748658+0000","last_became_active":"2024-04-21T08:44:38.517594+0000","last_became_peered":"2024-04-21T08:44:38.517594+0000","last_unstale":"2024-04-21T08:47:00.748658+0000","last_undegraded":"2024-04-21T08:47:00.748658+0000","last_fullsized":"2024-04-21T08:47:00.748658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:35.261582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718443+0000","last_change":"2024-04-21T08:45:27.517719+0000","last_active":"2024-04-21T08:47:00.718443+0000","last_peered":"2024-04-21T08:47:00.718443+0000","last_clean":"2024-04-21T08:47:00.718443+0000","last_became_active":"2024-04-21T08:45:27.517429+0000","last_became_peered":"2024-04-21T08:45:27.517429+0000","last_unstale":"2024-04-21T08:47:00.718443+0000","last_undegraded":"2024-04-21T08:47:00.718443+0000","last_fullsized":"2024-04-21T08:47:00.718443+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:49:34.097759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748213+0000","last_change":"2024-04-21T08:44:33.485110+0000","last_active":"2024-04-21T08:47:00.748213+0000","last_peered":"2024-04-21T08:47:00.748213+0000","last_clean":"2024-04-21T08:47:00.748213+0000","last_became_active":"2024-04-21T08:44:33.484722+0000","last_became_peered":"2024-04-21T08:44:33.484722+0000","last_unstale":"2024-04-21T08:47:00.748213+0000","last_undegraded":"2024-04-21T08:47:00.748213+0000","last_fullsized":"2024-04-21T08:47:00.748213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:47:16.773560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":368,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748009+0000","last_change":"2024-04-21T08:43:17.984594+0000","last_active":"2024-04-21T08:47:00.748009+0000","last_peered":"2024-04-21T08:47:00.748009+0000","last_clean":"2024-04-21T08:47:00.748009+0000","last_became_active":"2024-04-21T08:43:17.984207+0000","last_became_peered":"2024-04-21T08:43:17.984207+0000","last_unstale":"2024-04-21T08:47:00.748009+0000","last_undegraded":"2024-04-21T08:47:00.748009+0000","last_fullsized":"2024-04-21T08:47:00.748009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_clean_scrub_stamp":"2024-04-21T08:39:50.910206+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:35.260499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721571+0000","last_change":"2024-04-21T08:45:27.518834+0000","last_active":"2024-04-21T08:47:00.721571+0000","last_peered":"2024-04-21T08:47:00.721571+0000","last_clean":"2024-04-21T08:47:00.721571+0000","last_became_active":"2024-04-21T08:45:27.518390+0000","last_became_peered":"2024-04-21T08:45:27.518390+0000","last_unstale":"2024-04-21T08:47:00.721571+0000","last_undegraded":"2024-04-21T08:47:00.721571+0000","last_fullsized":"2024-04-21T08:47:00.721571+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:58:32.673008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874296+0000","last_change":"2024-04-21T08:44:34.499975+0000","last_active":"2024-04-21T08:47:00.874296+0000","last_peered":"2024-04-21T08:47:00.874296+0000","last_clean":"2024-04-21T08:47:00.874296+0000","last_became_active":"2024-04-21T08:44:34.499646+0000","last_became_peered":"2024-04-21T08:44:34.499646+0000","last_unstale":"2024-04-21T08:47:00.874296+0000","last_undegraded":"2024-04-21T08:47:00.874296+0000","last_fullsized":"2024-04-21T08:47:00.874296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:19:38.082502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748114+0000","last_change":"2024-04-21T08:44:33.484449+0000","last_active":"2024-04-21T08:47:00.748114+0000","last_peered":"2024-04-21T08:47:00.748114+0000","last_clean":"2024-04-21T08:47:00.748114+0000","last_became_active":"2024-04-21T08:44:33.484155+0000","last_became_peered":"2024-04-21T08:44:33.484155+0000","last_unstale":"2024-04-21T08:47:00.748114+0000","last_undegraded":"2024-04-21T08:47:00.748114+0000","last_fullsized":"2024-04-21T08:47:00.748114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:03.480474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747766+0000","last_change":"2024-04-21T08:44:38.515283+0000","last_active":"2024-04-21T08:47:00.747766+0000","last_peered":"2024-04-21T08:47:00.747766+0000","last_clean":"2024-04-21T08:47:00.747766+0000","last_became_active":"2024-04-21T08:44:38.514764+0000","last_became_peered":"2024-04-21T08:44:38.514764+0000","last_unstale":"2024-04-21T08:47:00.747766+0000","last_undegraded":"2024-04-21T08:47:00.747766+0000","last_fullsized":"2024-04-21T08:47:00.747766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:17:36.378392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721569+0000","last_change":"2024-04-21T08:44:33.489198+0000","last_active":"2024-04-21T08:47:00.721569+0000","last_peered":"2024-04-21T08:47:00.721569+0000","last_clean":"2024-04-21T08:47:00.721569+0000","last_became_active":"2024-04-21T08:44:33.488794+0000","last_became_peered":"2024-04-21T08:44:33.488794+0000","last_unstale":"2024-04-21T08:47:00.721569+0000","last_undegraded":"2024-04-21T08:47:00.721569+0000","last_fullsized":"2024-04-21T08:47:00.721569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:24.948283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874977+0000","last_change":"2024-04-21T08:44:34.488811+0000","last_active":"2024-04-21T08:47:00.874977+0000","last_peered":"2024-04-21T08:47:00.874977+0000","last_clean":"2024-04-21T08:47:00.874977+0000","last_became_active":"2024-04-21T08:44:34.488472+0000","last_became_peered":"2024-04-21T08:44:34.488472+0000","last_unstale":"2024-04-21T08:47:00.874977+0000","last_undegraded":"2024-04-21T08:47:00.874977+0000","last_fullsized":"2024-04-21T08:47:00.874977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:44.779492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622045+0000","last_change":"2024-04-21T08:44:38.522711+0000","last_active":"2024-04-21T08:47:00.622045+0000","last_peered":"2024-04-21T08:47:00.622045+0000","last_clean":"2024-04-21T08:47:00.622045+0000","last_became_active":"2024-04-21T08:44:38.522322+0000","last_became_peered":"2024-04-21T08:44:38.522322+0000","last_unstale":"2024-04-21T08:47:00.622045+0000","last_undegraded":"2024-04-21T08:47:00.622045+0000","last_fullsized":"2024-04-21T08:47:00.622045+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:26:13.323283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'48","reported_seq":104,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.836065+0000","last_change":"2024-04-21T08:44:33.481815+0000","last_active":"2024-04-21T08:47:00.836065+0000","last_peered":"2024-04-21T08:47:00.836065+0000","last_clean":"2024-04-21T08:47:00.836065+0000","last_became_active":"2024-04-21T08:44:33.481469+0000","last_became_peered":"2024-04-21T08:44:33.481469+0000","last_unstale":"2024-04-21T08:47:00.836065+0000","last_undegraded":"2024-04-21T08:47:00.836065+0000","last_fullsized":"2024-04-21T08:47:00.836065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:13.970821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623173+0000","last_change":"2024-04-21T08:44:34.496996+0000","last_active":"2024-04-21T08:47:00.623173+0000","last_peered":"2024-04-21T08:47:00.623173+0000","last_clean":"2024-04-21T08:47:00.623173+0000","last_became_active":"2024-04-21T08:44:34.496676+0000","last_became_peered":"2024-04-21T08:44:34.496676+0000","last_unstale":"2024-04-21T08:47:00.623173+0000","last_undegraded":"2024-04-21T08:47:00.623173+0000","last_fullsized":"2024-04-21T08:47:00.623173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:43:08.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626592+0000","last_change":"2024-04-21T08:44:38.516240+0000","last_active":"2024-04-21T08:47:00.626592+0000","last_peered":"2024-04-21T08:47:00.626592+0000","last_clean":"2024-04-21T08:47:00.626592+0000","last_became_active":"2024-04-21T08:44:38.515801+0000","last_became_peered":"2024-04-21T08:44:38.515801+0000","last_unstale":"2024-04-21T08:47:00.626592+0000","last_undegraded":"2024-04-21T08:47:00.626592+0000","last_fullsized":"2024-04-21T08:47:00.626592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:46.520373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749302+0000","last_change":"2024-04-21T08:44:33.488952+0000","last_active":"2024-04-21T08:47:00.749302+0000","last_peered":"2024-04-21T08:47:00.749302+0000","last_clean":"2024-04-21T08:47:00.749302+0000","last_became_active":"2024-04-21T08:44:33.488617+0000","last_became_peered":"2024-04-21T08:44:33.488617+0000","last_unstale":"2024-04-21T08:47:00.749302+0000","last_undegraded":"2024-04-21T08:47:00.749302+0000","last_fullsized":"2024-04-21T08:47:00.749302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:37:24.679132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747751+0000","last_change":"2024-04-21T08:44:34.510568+0000","last_active":"2024-04-21T08:47:00.747751+0000","last_peered":"2024-04-21T08:47:00.747751+0000","last_clean":"2024-04-21T08:47:00.747751+0000","last_became_active":"2024-04-21T08:44:34.509900+0000","last_became_peered":"2024-04-21T08:44:34.509900+0000","last_unstale":"2024-04-21T08:47:00.747751+0000","last_undegraded":"2024-04-21T08:47:00.747751+0000","last_fullsized":"2024-04-21T08:47:00.747751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:08.426141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748835+0000","last_change":"2024-04-21T08:45:27.976874+0000","last_active":"2024-04-21T08:47:00.748835+0000","last_peered":"2024-04-21T08:47:00.748835+0000","last_clean":"2024-04-21T08:47:00.748835+0000","last_became_active":"2024-04-21T08:45:27.976387+0000","last_became_peered":"2024-04-21T08:45:27.976387+0000","last_unstale":"2024-04-21T08:47:00.748835+0000","last_undegraded":"2024-04-21T08:47:00.748835+0000","last_fullsized":"2024-04-21T08:47:00.748835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:38:18.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747268+0000","last_change":"2024-04-21T08:44:33.481803+0000","last_active":"2024-04-21T08:47:00.747268+0000","last_peered":"2024-04-21T08:47:00.747268+0000","last_clean":"2024-04-21T08:47:00.747268+0000","last_became_active":"2024-04-21T08:44:33.481256+0000","last_became_peered":"2024-04-21T08:44:33.481256+0000","last_unstale":"2024-04-21T08:47:00.747268+0000","last_undegraded":"2024-04-21T08:47:00.747268+0000","last_fullsized":"2024-04-21T08:47:00.747268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:17.804557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721258+0000","last_change":"2024-04-21T08:44:34.490206+0000","last_active":"2024-04-21T08:47:00.721258+0000","last_peered":"2024-04-21T08:47:00.721258+0000","last_clean":"2024-04-21T08:47:00.721258+0000","last_became_active":"2024-04-21T08:44:34.489797+0000","last_became_peered":"2024-04-21T08:44:34.489797+0000","last_unstale":"2024-04-21T08:47:00.721258+0000","last_undegraded":"2024-04-21T08:47:00.721258+0000","last_fullsized":"2024-04-21T08:47:00.721258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:09.055429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719147+0000","last_change":"2024-04-21T08:44:38.517385+0000","last_active":"2024-04-21T08:47:00.719147+0000","last_peered":"2024-04-21T08:47:00.719147+0000","last_clean":"2024-04-21T08:47:00.719147+0000","last_became_active":"2024-04-21T08:44:38.516964+0000","last_became_peered":"2024-04-21T08:44:38.516964+0000","last_unstale":"2024-04-21T08:47:00.719147+0000","last_undegraded":"2024-04-21T08:47:00.719147+0000","last_fullsized":"2024-04-21T08:47:00.719147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:33.364076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622039+0000","last_change":"2024-04-21T08:44:33.490586+0000","last_active":"2024-04-21T08:47:00.622039+0000","last_peered":"2024-04-21T08:47:00.622039+0000","last_clean":"2024-04-21T08:47:00.622039+0000","last_became_active":"2024-04-21T08:44:33.489281+0000","last_became_peered":"2024-04-21T08:44:33.489281+0000","last_unstale":"2024-04-21T08:47:00.622039+0000","last_undegraded":"2024-04-21T08:47:00.622039+0000","last_fullsized":"2024-04-21T08:47:00.622039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:41:29.856165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747602+0000","last_change":"2024-04-21T08:44:34.493394+0000","last_active":"2024-04-21T08:47:00.747602+0000","last_peered":"2024-04-21T08:47:00.747602+0000","last_clean":"2024-04-21T08:47:00.747602+0000","last_became_active":"2024-04-21T08:44:34.492367+0000","last_became_peered":"2024-04-21T08:44:34.492367+0000","last_unstale":"2024-04-21T08:47:00.747602+0000","last_undegraded":"2024-04-21T08:47:00.747602+0000","last_fullsized":"2024-04-21T08:47:00.747602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:13.111266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748165+0000","last_change":"2024-04-21T08:44:38.519113+0000","last_active":"2024-04-21T08:47:00.748165+0000","last_peered":"2024-04-21T08:47:00.748165+0000","last_clean":"2024-04-21T08:47:00.748165+0000","last_became_active":"2024-04-21T08:44:38.518469+0000","last_became_peered":"2024-04-21T08:44:38.518469+0000","last_unstale":"2024-04-21T08:47:00.748165+0000","last_undegraded":"2024-04-21T08:47:00.748165+0000","last_fullsized":"2024-04-21T08:47:00.748165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:39:22.200041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'5","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747723+0000","last_change":"2024-04-21T08:46:27.492549+0000","last_active":"2024-04-21T08:47:00.747723+0000","last_peered":"2024-04-21T08:47:00.747723+0000","last_clean":"2024-04-21T08:47:00.747723+0000","last_became_active":"2024-04-21T08:46:27.488054+0000","last_became_peered":"2024-04-21T08:46:27.488054+0000","last_unstale":"2024-04-21T08:47:00.747723+0000","last_undegraded":"2024-04-21T08:47:00.747723+0000","last_fullsized":"2024-04-21T08:47:00.747723+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:51:54.797532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622446+0000","last_change":"2024-04-21T08:44:34.502226+0000","last_active":"2024-04-21T08:47:00.622446+0000","last_peered":"2024-04-21T08:47:00.622446+0000","last_clean":"2024-04-21T08:47:00.622446+0000","last_became_active":"2024-04-21T08:44:34.501775+0000","last_became_peered":"2024-04-21T08:44:34.501775+0000","last_unstale":"2024-04-21T08:47:00.622446+0000","last_undegraded":"2024-04-21T08:47:00.622446+0000","last_fullsized":"2024-04-21T08:47:00.622446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:58.325267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748655+0000","last_change":"2024-04-21T08:44:38.522250+0000","last_active":"2024-04-21T08:47:00.748655+0000","last_peered":"2024-04-21T08:47:00.748655+0000","last_clean":"2024-04-21T08:47:00.748655+0000","last_became_active":"2024-04-21T08:44:38.520870+0000","last_became_peered":"2024-04-21T08:44:38.520870+0000","last_unstale":"2024-04-21T08:47:00.748655+0000","last_undegraded":"2024-04-21T08:47:00.748655+0000","last_fullsized":"2024-04-21T08:47:00.748655+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:02.391785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747761+0000","last_change":"2024-04-21T08:46:27.496215+0000","last_active":"2024-04-21T08:47:00.747761+0000","last_peered":"2024-04-21T08:47:00.747761+0000","last_clean":"2024-04-21T08:47:00.747761+0000","last_became_active":"2024-04-21T08:46:27.488287+0000","last_became_peered":"2024-04-21T08:46:27.488287+0000","last_unstale":"2024-04-21T08:47:00.747761+0000","last_undegraded":"2024-04-21T08:47:00.747761+0000","last_fullsized":"2024-04-21T08:47:00.747761+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:08:05.390563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749473+0000","last_change":"2024-04-21T08:44:34.496034+0000","last_active":"2024-04-21T08:47:00.749473+0000","last_peered":"2024-04-21T08:47:00.749473+0000","last_clean":"2024-04-21T08:47:00.749473+0000","last_became_active":"2024-04-21T08:44:34.495641+0000","last_became_peered":"2024-04-21T08:44:34.495641+0000","last_unstale":"2024-04-21T08:47:00.749473+0000","last_undegraded":"2024-04-21T08:47:00.749473+0000","last_fullsized":"2024-04-21T08:47:00.749473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:21:18.893326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747361+0000","last_change":"2024-04-21T08:44:38.517288+0000","last_active":"2024-04-21T08:47:00.747361+0000","last_peered":"2024-04-21T08:47:00.747361+0000","last_clean":"2024-04-21T08:47:00.747361+0000","last_became_active":"2024-04-21T08:44:38.516853+0000","last_became_peered":"2024-04-21T08:44:38.516853+0000","last_unstale":"2024-04-21T08:47:00.747361+0000","last_undegraded":"2024-04-21T08:47:00.747361+0000","last_fullsized":"2024-04-21T08:47:00.747361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:27:22.371703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'7","reported_seq":163,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.237801+0000","last_change":"2024-04-21T08:44:33.491578+0000","last_active":"2024-04-21T08:47:01.237801+0000","last_peered":"2024-04-21T08:47:01.237801+0000","last_clean":"2024-04-21T08:47:01.237801+0000","last_became_active":"2024-04-21T08:44:33.491238+0000","last_became_peered":"2024-04-21T08:44:33.491238+0000","last_unstale":"2024-04-21T08:47:01.237801+0000","last_undegraded":"2024-04-21T08:47:01.237801+0000","last_fullsized":"2024-04-21T08:47:01.237801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:50.399031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749074+0000","last_change":"2024-04-21T08:44:34.497632+0000","last_active":"2024-04-21T08:47:00.749074+0000","last_peered":"2024-04-21T08:47:00.749074+0000","last_clean":"2024-04-21T08:47:00.749074+0000","last_became_active":"2024-04-21T08:44:34.497318+0000","last_became_peered":"2024-04-21T08:44:34.497318+0000","last_unstale":"2024-04-21T08:47:00.749074+0000","last_undegraded":"2024-04-21T08:47:00.749074+0000","last_fullsized":"2024-04-21T08:47:00.749074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:11:33.918530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747370+0000","last_change":"2024-04-21T08:44:38.518414+0000","last_active":"2024-04-21T08:47:00.747370+0000","last_peered":"2024-04-21T08:47:00.747370+0000","last_clean":"2024-04-21T08:47:00.747370+0000","last_became_active":"2024-04-21T08:44:38.517953+0000","last_became_peered":"2024-04-21T08:44:38.517953+0000","last_unstale":"2024-04-21T08:47:00.747370+0000","last_undegraded":"2024-04-21T08:47:00.747370+0000","last_fullsized":"2024-04-21T08:47:00.747370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:48.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.833716+0000","last_change":"2024-04-21T08:44:33.493896+0000","last_active":"2024-04-21T08:47:00.833716+0000","last_peered":"2024-04-21T08:47:00.833716+0000","last_clean":"2024-04-21T08:47:00.833716+0000","last_became_active":"2024-04-21T08:44:33.493566+0000","last_became_peered":"2024-04-21T08:44:33.493566+0000","last_unstale":"2024-04-21T08:47:00.833716+0000","last_undegraded":"2024-04-21T08:47:00.833716+0000","last_fullsized":"2024-04-21T08:47:00.833716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:47:17.307690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748612+0000","last_change":"2024-04-21T08:44:34.500406+0000","last_active":"2024-04-21T08:47:00.748612+0000","last_peered":"2024-04-21T08:47:00.748612+0000","last_clean":"2024-04-21T08:47:00.748612+0000","last_became_active":"2024-04-21T08:44:34.500048+0000","last_became_peered":"2024-04-21T08:44:34.500048+0000","last_unstale":"2024-04-21T08:47:00.748612+0000","last_undegraded":"2024-04-21T08:47:00.748612+0000","last_fullsized":"2024-04-21T08:47:00.748612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:10:58.166155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728633+0000","last_change":"2024-04-21T08:44:38.514812+0000","last_active":"2024-04-21T08:47:00.728633+0000","last_peered":"2024-04-21T08:47:00.728633+0000","last_clean":"2024-04-21T08:47:00.728633+0000","last_became_active":"2024-04-21T08:44:38.514247+0000","last_became_peered":"2024-04-21T08:44:38.514247+0000","last_unstale":"2024-04-21T08:47:00.728633+0000","last_undegraded":"2024-04-21T08:47:00.728633+0000","last_fullsized":"2024-04-21T08:47:00.728633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:06:10.588453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875149+0000","last_change":"2024-04-21T08:44:33.493887+0000","last_active":"2024-04-21T08:47:00.875149+0000","last_peered":"2024-04-21T08:47:00.875149+0000","last_clean":"2024-04-21T08:47:00.875149+0000","last_became_active":"2024-04-21T08:44:33.493555+0000","last_became_peered":"2024-04-21T08:44:33.493555+0000","last_unstale":"2024-04-21T08:47:00.875149+0000","last_undegraded":"2024-04-21T08:47:00.875149+0000","last_fullsized":"2024-04-21T08:47:00.875149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:18.089102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749096+0000","last_change":"2024-04-21T08:44:34.503259+0000","last_active":"2024-04-21T08:47:00.749096+0000","last_peered":"2024-04-21T08:47:00.749096+0000","last_clean":"2024-04-21T08:47:00.749096+0000","last_became_active":"2024-04-21T08:44:34.502845+0000","last_became_peered":"2024-04-21T08:44:34.502845+0000","last_unstale":"2024-04-21T08:47:00.749096+0000","last_undegraded":"2024-04-21T08:47:00.749096+0000","last_fullsized":"2024-04-21T08:47:00.749096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:19.902178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747653+0000","last_change":"2024-04-21T08:44:34.497608+0000","last_active":"2024-04-21T08:47:00.747653+0000","last_peered":"2024-04-21T08:47:00.747653+0000","last_clean":"2024-04-21T08:47:00.747653+0000","last_became_active":"2024-04-21T08:44:34.497255+0000","last_became_peered":"2024-04-21T08:44:34.497255+0000","last_unstale":"2024-04-21T08:47:00.747653+0000","last_undegraded":"2024-04-21T08:47:00.747653+0000","last_fullsized":"2024-04-21T08:47:00.747653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:51:15.181837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626828+0000","last_change":"2024-04-21T08:44:34.495408+0000","last_active":"2024-04-21T08:47:00.626828+0000","last_peered":"2024-04-21T08:47:00.626828+0000","last_clean":"2024-04-21T08:47:00.626828+0000","last_became_active":"2024-04-21T08:44:34.495038+0000","last_became_peered":"2024-04-21T08:44:34.495038+0000","last_unstale":"2024-04-21T08:47:00.626828+0000","last_undegraded":"2024-04-21T08:47:00.626828+0000","last_fullsized":"2024-04-21T08:47:00.626828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:59.512245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626703+0000","last_change":"2024-04-21T08:44:34.493292+0000","last_active":"2024-04-21T08:47:00.626703+0000","last_peered":"2024-04-21T08:47:00.626703+0000","last_clean":"2024-04-21T08:47:00.626703+0000","last_became_active":"2024-04-21T08:44:34.492913+0000","last_became_peered":"2024-04-21T08:44:34.492913+0000","last_unstale":"2024-04-21T08:47:00.626703+0000","last_undegraded":"2024-04-21T08:47:00.626703+0000","last_fullsized":"2024-04-21T08:47:00.626703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:24.271859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747551+0000","last_change":"2024-04-21T08:44:34.510448+0000","last_active":"2024-04-21T08:47:00.747551+0000","last_peered":"2024-04-21T08:47:00.747551+0000","last_clean":"2024-04-21T08:47:00.747551+0000","last_became_active":"2024-04-21T08:44:34.509958+0000","last_became_peered":"2024-04-21T08:44:34.509958+0000","last_unstale":"2024-04-21T08:47:00.747551+0000","last_undegraded":"2024-04-21T08:47:00.747551+0000","last_fullsized":"2024-04-21T08:47:00.747551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:36.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875000+0000","last_change":"2024-04-21T08:44:34.497415+0000","last_active":"2024-04-21T08:47:00.875000+0000","last_peered":"2024-04-21T08:47:00.875000+0000","last_clean":"2024-04-21T08:47:00.875000+0000","last_became_active":"2024-04-21T08:44:34.497120+0000","last_became_peered":"2024-04-21T08:44:34.497120+0000","last_unstale":"2024-04-21T08:47:00.875000+0000","last_undegraded":"2024-04-21T08:47:00.875000+0000","last_fullsized":"2024-04-21T08:47:00.875000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:30.232338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747974+0000","last_change":"2024-04-21T08:44:34.500356+0000","last_active":"2024-04-21T08:47:00.747974+0000","last_peered":"2024-04-21T08:47:00.747974+0000","last_clean":"2024-04-21T08:47:00.747974+0000","last_became_active":"2024-04-21T08:44:34.500032+0000","last_became_peered":"2024-04-21T08:44:34.500032+0000","last_unstale":"2024-04-21T08:47:00.747974+0000","last_undegraded":"2024-04-21T08:47:00.747974+0000","last_fullsized":"2024-04-21T08:47:00.747974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:24.900221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627464+0000","last_change":"2024-04-21T08:44:34.493629+0000","last_active":"2024-04-21T08:47:00.627464+0000","last_peered":"2024-04-21T08:47:00.627464+0000","last_clean":"2024-04-21T08:47:00.627464+0000","last_became_active":"2024-04-21T08:44:34.493273+0000","last_became_peered":"2024-04-21T08:44:34.493273+0000","last_unstale":"2024-04-21T08:47:00.627464+0000","last_undegraded":"2024-04-21T08:47:00.627464+0000","last_fullsized":"2024-04-21T08:47:00.627464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:22:42.698957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623249+0000","last_change":"2024-04-21T08:44:34.493427+0000","last_active":"2024-04-21T08:47:00.623249+0000","last_peered":"2024-04-21T08:47:00.623249+0000","last_clean":"2024-04-21T08:47:00.623249+0000","last_became_active":"2024-04-21T08:44:34.493105+0000","last_became_peered":"2024-04-21T08:44:34.493105+0000","last_unstale":"2024-04-21T08:47:00.623249+0000","last_undegraded":"2024-04-21T08:47:00.623249+0000","last_fullsized":"2024-04-21T08:47:00.623249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:37.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874153+0000","last_change":"2024-04-21T08:45:27.515899+0000","last_active":"2024-04-21T08:47:00.874153+0000","last_peered":"2024-04-21T08:47:00.874153+0000","last_clean":"2024-04-21T08:47:00.874153+0000","last_became_active":"2024-04-21T08:45:27.515567+0000","last_became_peered":"2024-04-21T08:45:27.515567+0000","last_unstale":"2024-04-21T08:47:00.874153+0000","last_undegraded":"2024-04-21T08:47:00.874153+0000","last_fullsized":"2024-04-21T08:47:00.874153+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:16.645869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622080+0000","last_change":"2024-04-21T08:44:34.499420+0000","last_active":"2024-04-21T08:47:00.622080+0000","last_peered":"2024-04-21T08:47:00.622080+0000","last_clean":"2024-04-21T08:47:00.622080+0000","last_became_active":"2024-04-21T08:44:34.499079+0000","last_became_peered":"2024-04-21T08:44:34.499079+0000","last_unstale":"2024-04-21T08:47:00.622080+0000","last_undegraded":"2024-04-21T08:47:00.622080+0000","last_fullsized":"2024-04-21T08:47:00.622080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:45.351422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747540+0000","last_change":"2024-04-21T08:44:34.493420+0000","last_active":"2024-04-21T08:47:00.747540+0000","last_peered":"2024-04-21T08:47:00.747540+0000","last_clean":"2024-04-21T08:47:00.747540+0000","last_became_active":"2024-04-21T08:44:34.493098+0000","last_became_peered":"2024-04-21T08:44:34.493098+0000","last_unstale":"2024-04-21T08:47:00.747540+0000","last_undegraded":"2024-04-21T08:47:00.747540+0000","last_fullsized":"2024-04-21T08:47:00.747540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:22:34.316822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874666+0000","last_change":"2024-04-21T08:44:34.494760+0000","last_active":"2024-04-21T08:47:00.874666+0000","last_peered":"2024-04-21T08:47:00.874666+0000","last_clean":"2024-04-21T08:47:00.874666+0000","last_became_active":"2024-04-21T08:44:34.494265+0000","last_became_peered":"2024-04-21T08:44:34.494265+0000","last_unstale":"2024-04-21T08:47:00.874666+0000","last_undegraded":"2024-04-21T08:47:00.874666+0000","last_fullsized":"2024-04-21T08:47:00.874666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:16.035030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748002+0000","last_change":"2024-04-21T08:44:34.498785+0000","last_active":"2024-04-21T08:47:00.748002+0000","last_peered":"2024-04-21T08:47:00.748002+0000","last_clean":"2024-04-21T08:47:00.748002+0000","last_became_active":"2024-04-21T08:44:34.495159+0000","last_became_peered":"2024-04-21T08:44:34.495159+0000","last_unstale":"2024-04-21T08:47:00.748002+0000","last_undegraded":"2024-04-21T08:47:00.748002+0000","last_fullsized":"2024-04-21T08:47:00.748002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:03:20.095016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600929,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1346,"num_read_kb":788,"num_write":372,"num_write_kb":1466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1802787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1081344,"data_stored":531093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28888,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716264,"statfs":{"total":95995035648,"available":95965454336,"internally_reserved":0,"allocated":1916928,"data_stored":1316546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1503232,"data_stored":927553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1495040,"data_stored":923835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1699840,"data_stored":1128597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":525745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":593812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":527983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":595904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1050269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":529819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37196,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707956,"statfs":{"total":95995035648,"available":95956946944,"internally_reserved":0,"allocated":1843200,"data_stored":1254997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":201155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:47:06.033 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-21T08:47:06.033 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:47:06.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:47:06.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:47:06.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:47:06.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:47:06.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: Updating smithi106:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/155889491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: Updating smithi082:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: Updating smithi134:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1880972080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:47:06.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:47:06.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:05 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:47:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.167+0000 7f3427870640 1 -- 172.21.15.82:0/1943722663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34201021a0 msgr2=0x7f3420102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.167+0000 7f3427870640 1 --2- 172.21.15.82:0/1943722663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34201021a0 0x7f3420102580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f341c009900 tx=0x7f341c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.167+0000 7f3427870640 1 -- 172.21.15.82:0/1943722663 shutdown_connections 2024-04-21T08:47:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.167+0000 7f3427870640 1 --2- 172.21.15.82:0/1943722663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3420107270 0x7f342010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.167+0000 7f3427870640 1 --2- 172.21.15.82:0/1943722663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3420102b50 0x7f3420106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.167+0000 7f3427870640 1 --2- 172.21.15.82:0/1943722663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34201021a0 0x7f3420102580 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.167+0000 7f3427870640 1 -- 172.21.15.82:0/1943722663 >> 172.21.15.82:0/1943722663 conn(0x7f34200793a0 msgr2=0x7f34200797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.167+0000 7f3427870640 1 -- 172.21.15.82:0/1943722663 shutdown_connections 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.167+0000 7f3427870640 1 -- 172.21.15.82:0/1943722663 wait complete. 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f3427870640 1 Processor -- start 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f3427870640 1 -- start start 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f3427870640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34201021a0 0x7f3420074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f3427870640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3420107270 0x7f3420074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f3427870640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3420074cd0 0x7f34201a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f3427870640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f34201121d0 con 0x7f34201021a0 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f3427870640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3420112050 con 0x7f3420107270 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f3427870640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3420112350 con 0x7f3420074cd0 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f342560c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34201021a0 0x7f3420074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f342560c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34201021a0 0x7f3420074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44398/0 (socket says 172.21.15.82:44398) 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f342560c640 1 -- 172.21.15.82:0/1122473266 learned_addr learned my addr 172.21.15.82:0/1122473266 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f3424e0b640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3420107270 0x7f3420074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f3425e0d640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3420074cd0 0x7f34201a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f342560c640 1 -- 172.21.15.82:0/1122473266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3420074cd0 msgr2=0x7f34201a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f342560c640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3420074cd0 0x7f34201a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f342560c640 1 -- 172.21.15.82:0/1122473266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3420107270 msgr2=0x7f3420074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f342560c640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3420107270 0x7f3420074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.168+0000 7f342560c640 1 -- 172.21.15.82:0/1122473266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34201a6fc0 con 0x7f34201021a0 2024-04-21T08:47:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.169+0000 7f342560c640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34201021a0 0x7f3420074250 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f341c005940 tx=0x7f341c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.169+0000 7f34167fc640 1 -- 172.21.15.82:0/1122473266 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c043020 con 0x7f34201021a0 2024-04-21T08:47:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.169+0000 7f34167fc640 1 -- 172.21.15.82:0/1122473266 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f341c02caf0 con 0x7f34201021a0 2024-04-21T08:47:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.169+0000 7f34167fc640 1 -- 172.21.15.82:0/1122473266 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c003af0 con 0x7f34201021a0 2024-04-21T08:47:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.169+0000 7f3427870640 1 -- 172.21.15.82:0/1122473266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34201a7250 con 0x7f34201021a0 2024-04-21T08:47:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.169+0000 7f3427870640 1 -- 172.21.15.82:0/1122473266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34201a77f0 con 0x7f34201021a0 2024-04-21T08:47:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.170+0000 7f3427870640 1 -- 172.21.15.82:0/1122473266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3420069230 con 0x7f34201021a0 2024-04-21T08:47:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.171+0000 7f34167fc640 1 -- 172.21.15.82:0/1122473266 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f341c046020 con 0x7f34201021a0 2024-04-21T08:47:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.172+0000 7f34167fc640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f340c0a3ef0 0x7f340c0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.172+0000 7f3424e0b640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f340c0a3ef0 0x7f340c0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.172+0000 7f34167fc640 1 -- 172.21.15.82:0/1122473266 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f341c0f3430 con 0x7f34201021a0 2024-04-21T08:47:06.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.172+0000 7f3424e0b640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f340c0a3ef0 0x7f340c0a63b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3420075ce0 tx=0x7f341000b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:06.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.175+0000 7f34167fc640 1 -- 172.21.15.82:0/1122473266 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f341c04d020 con 0x7f34201021a0 2024-04-21T08:47:06.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.350+0000 7f3427870640 1 -- 172.21.15.82:0/1122473266 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f342010a510 con 0x7f340c0a3ef0 2024-04-21T08:47:06.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.359+0000 7f34167fc640 1 -- 172.21.15.82:0/1122473266 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339428 (secure 0 0 0) 0x7f3410003890 con 0x7f340c0a3ef0 2024-04-21T08:47:06.362 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:06.364 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-21T08:47:06.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.362+0000 7f3427870640 1 -- 172.21.15.82:0/1122473266 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f340c0a3ef0 msgr2=0x7f340c0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.362+0000 7f3427870640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f340c0a3ef0 0x7f340c0a63b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3420075ce0 tx=0x7f341000b390 comp rx=0 tx=0).stop 2024-04-21T08:47:06.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.362+0000 7f3427870640 1 -- 172.21.15.82:0/1122473266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34201021a0 msgr2=0x7f3420074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.362+0000 7f3427870640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34201021a0 0x7f3420074250 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f341c005940 tx=0x7f341c0045e0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.362+0000 7f3427870640 1 -- 172.21.15.82:0/1122473266 shutdown_connections 2024-04-21T08:47:06.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.362+0000 7f3427870640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3420074cd0 0x7f34201a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.362+0000 7f3427870640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3420107270 0x7f3420074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.362+0000 7f3427870640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f340c0a3ef0 0x7f340c0a63b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.362+0000 7f3427870640 1 --2- 172.21.15.82:0/1122473266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34201021a0 0x7f3420074250 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.362+0000 7f3427870640 1 -- 172.21.15.82:0/1122473266 >> 172.21.15.82:0/1122473266 conn(0x7f34200793a0 msgr2=0x7f34200fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:06.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.362+0000 7f3427870640 1 -- 172.21.15.82:0/1122473266 shutdown_connections 2024-04-21T08:47:06.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.362+0000 7f3427870640 1 -- 172.21.15.82:0/1122473266 wait complete. 2024-04-21T08:47:06.373 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-21T08:47:04.633865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191531,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1482,"num_read_kb":907,"num_write":555,"num_write_kb":4406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":939,"ondisk_log_size":939,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357300,"kb_used_data":16376,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124584524,"statfs":{"total":1151940427776,"available":1151574552576,"internally_reserved":0,"allocated":16769024,"data_stored":9906153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.985540"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748276+0000","last_change":"2024-04-21T08:44:34.494393+0000","last_active":"2024-04-21T08:47:00.748276+0000","last_peered":"2024-04-21T08:47:00.748276+0000","last_clean":"2024-04-21T08:47:00.748276+0000","last_became_active":"2024-04-21T08:44:34.493583+0000","last_became_peered":"2024-04-21T08:44:34.493583+0000","last_unstale":"2024-04-21T08:47:00.748276+0000","last_undegraded":"2024-04-21T08:47:00.748276+0000","last_fullsized":"2024-04-21T08:47:00.748276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:37.211676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747826+0000","last_change":"2024-04-21T08:44:34.496291+0000","last_active":"2024-04-21T08:47:00.747826+0000","last_peered":"2024-04-21T08:47:00.747826+0000","last_clean":"2024-04-21T08:47:00.747826+0000","last_became_active":"2024-04-21T08:44:34.495019+0000","last_became_peered":"2024-04-21T08:44:34.495019+0000","last_unstale":"2024-04-21T08:47:00.747826+0000","last_undegraded":"2024-04-21T08:47:00.747826+0000","last_fullsized":"2024-04-21T08:47:00.747826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:05.309847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747464+0000","last_change":"2024-04-21T08:44:34.497948+0000","last_active":"2024-04-21T08:47:00.747464+0000","last_peered":"2024-04-21T08:47:00.747464+0000","last_clean":"2024-04-21T08:47:00.747464+0000","last_became_active":"2024-04-21T08:44:34.497657+0000","last_became_peered":"2024-04-21T08:44:34.497657+0000","last_unstale":"2024-04-21T08:47:00.747464+0000","last_undegraded":"2024-04-21T08:47:00.747464+0000","last_fullsized":"2024-04-21T08:47:00.747464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:11.463715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748377+0000","last_change":"2024-04-21T08:44:34.492682+0000","last_active":"2024-04-21T08:47:00.748377+0000","last_peered":"2024-04-21T08:47:00.748377+0000","last_clean":"2024-04-21T08:47:00.748377+0000","last_became_active":"2024-04-21T08:44:34.492230+0000","last_became_peered":"2024-04-21T08:44:34.492230+0000","last_unstale":"2024-04-21T08:47:00.748377+0000","last_undegraded":"2024-04-21T08:47:00.748377+0000","last_fullsized":"2024-04-21T08:47:00.748377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:40.365608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721320+0000","last_change":"2024-04-21T08:44:34.492160+0000","last_active":"2024-04-21T08:47:00.721320+0000","last_peered":"2024-04-21T08:47:00.721320+0000","last_clean":"2024-04-21T08:47:00.721320+0000","last_became_active":"2024-04-21T08:44:34.491701+0000","last_became_peered":"2024-04-21T08:44:34.491701+0000","last_unstale":"2024-04-21T08:47:00.721320+0000","last_undegraded":"2024-04-21T08:47:00.721320+0000","last_fullsized":"2024-04-21T08:47:00.721320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:50.542661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626504+0000","last_change":"2024-04-21T08:44:34.497341+0000","last_active":"2024-04-21T08:47:00.626504+0000","last_peered":"2024-04-21T08:47:00.626504+0000","last_clean":"2024-04-21T08:47:00.626504+0000","last_became_active":"2024-04-21T08:44:34.495794+0000","last_became_peered":"2024-04-21T08:44:34.495794+0000","last_unstale":"2024-04-21T08:47:00.626504+0000","last_undegraded":"2024-04-21T08:47:00.626504+0000","last_fullsized":"2024-04-21T08:47:00.626504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:13.854753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748474+0000","last_change":"2024-04-21T08:44:34.501578+0000","last_active":"2024-04-21T08:47:00.748474+0000","last_peered":"2024-04-21T08:47:00.748474+0000","last_clean":"2024-04-21T08:47:00.748474+0000","last_became_active":"2024-04-21T08:44:34.500848+0000","last_became_peered":"2024-04-21T08:44:34.500848+0000","last_unstale":"2024-04-21T08:47:00.748474+0000","last_undegraded":"2024-04-21T08:47:00.748474+0000","last_fullsized":"2024-04-21T08:47:00.748474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:53:27.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.621951+0000","last_change":"2024-04-21T08:44:34.502975+0000","last_active":"2024-04-21T08:47:00.621951+0000","last_peered":"2024-04-21T08:47:00.621951+0000","last_clean":"2024-04-21T08:47:00.621951+0000","last_became_active":"2024-04-21T08:44:34.502676+0000","last_became_peered":"2024-04-21T08:44:34.502676+0000","last_unstale":"2024-04-21T08:47:00.621951+0000","last_undegraded":"2024-04-21T08:47:00.621951+0000","last_fullsized":"2024-04-21T08:47:00.621951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:32:13.881603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748249+0000","last_change":"2024-04-21T08:44:34.495703+0000","last_active":"2024-04-21T08:47:00.748249+0000","last_peered":"2024-04-21T08:47:00.748249+0000","last_clean":"2024-04-21T08:47:00.748249+0000","last_became_active":"2024-04-21T08:44:34.495380+0000","last_became_peered":"2024-04-21T08:44:34.495380+0000","last_unstale":"2024-04-21T08:47:00.748249+0000","last_undegraded":"2024-04-21T08:47:00.748249+0000","last_fullsized":"2024-04-21T08:47:00.748249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:45:28.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626928+0000","last_change":"2024-04-21T08:44:34.499361+0000","last_active":"2024-04-21T08:47:00.626928+0000","last_peered":"2024-04-21T08:47:00.626928+0000","last_clean":"2024-04-21T08:47:00.626928+0000","last_became_active":"2024-04-21T08:44:34.499044+0000","last_became_peered":"2024-04-21T08:44:34.499044+0000","last_unstale":"2024-04-21T08:47:00.626928+0000","last_undegraded":"2024-04-21T08:47:00.626928+0000","last_fullsized":"2024-04-21T08:47:00.626928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:12.118985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748894+0000","last_change":"2024-04-21T08:44:34.499631+0000","last_active":"2024-04-21T08:47:00.748894+0000","last_peered":"2024-04-21T08:47:00.748894+0000","last_clean":"2024-04-21T08:47:00.748894+0000","last_became_active":"2024-04-21T08:44:34.499316+0000","last_became_peered":"2024-04-21T08:44:34.499316+0000","last_unstale":"2024-04-21T08:47:00.748894+0000","last_undegraded":"2024-04-21T08:47:00.748894+0000","last_fullsized":"2024-04-21T08:47:00.748894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:26.123017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748326+0000","last_change":"2024-04-21T08:44:34.504857+0000","last_active":"2024-04-21T08:47:00.748326+0000","last_peered":"2024-04-21T08:47:00.748326+0000","last_clean":"2024-04-21T08:47:00.748326+0000","last_became_active":"2024-04-21T08:44:34.504564+0000","last_became_peered":"2024-04-21T08:44:34.504564+0000","last_unstale":"2024-04-21T08:47:00.748326+0000","last_undegraded":"2024-04-21T08:47:00.748326+0000","last_fullsized":"2024-04-21T08:47:00.748326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:32:05.216162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874684+0000","last_change":"2024-04-21T08:44:34.484074+0000","last_active":"2024-04-21T08:47:00.874684+0000","last_peered":"2024-04-21T08:47:00.874684+0000","last_clean":"2024-04-21T08:47:00.874684+0000","last_became_active":"2024-04-21T08:44:34.483309+0000","last_became_peered":"2024-04-21T08:44:34.483309+0000","last_unstale":"2024-04-21T08:47:00.874684+0000","last_undegraded":"2024-04-21T08:47:00.874684+0000","last_fullsized":"2024-04-21T08:47:00.874684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:39.686048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747340+0000","last_change":"2024-04-21T08:44:34.499159+0000","last_active":"2024-04-21T08:47:00.747340+0000","last_peered":"2024-04-21T08:47:00.747340+0000","last_clean":"2024-04-21T08:47:00.747340+0000","last_became_active":"2024-04-21T08:44:34.496609+0000","last_became_peered":"2024-04-21T08:44:34.496609+0000","last_unstale":"2024-04-21T08:47:00.747340+0000","last_undegraded":"2024-04-21T08:47:00.747340+0000","last_fullsized":"2024-04-21T08:47:00.747340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:11.716321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747233+0000","last_change":"2024-04-21T08:45:27.406543+0000","last_active":"2024-04-21T08:47:00.747233+0000","last_peered":"2024-04-21T08:47:00.747233+0000","last_clean":"2024-04-21T08:47:00.747233+0000","last_became_active":"2024-04-21T08:45:27.406190+0000","last_became_peered":"2024-04-21T08:45:27.406190+0000","last_unstale":"2024-04-21T08:47:00.747233+0000","last_undegraded":"2024-04-21T08:47:00.747233+0000","last_fullsized":"2024-04-21T08:47:00.747233+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:56:17.760163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747327+0000","last_change":"2024-04-21T08:44:34.494018+0000","last_active":"2024-04-21T08:47:00.747327+0000","last_peered":"2024-04-21T08:47:00.747327+0000","last_clean":"2024-04-21T08:47:00.747327+0000","last_became_active":"2024-04-21T08:44:34.493692+0000","last_became_peered":"2024-04-21T08:44:34.493692+0000","last_unstale":"2024-04-21T08:47:00.747327+0000","last_undegraded":"2024-04-21T08:47:00.747327+0000","last_fullsized":"2024-04-21T08:47:00.747327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:25:16.944160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748187+0000","last_change":"2024-04-21T08:44:34.493231+0000","last_active":"2024-04-21T08:47:00.748187+0000","last_peered":"2024-04-21T08:47:00.748187+0000","last_clean":"2024-04-21T08:47:00.748187+0000","last_became_active":"2024-04-21T08:44:34.492826+0000","last_became_peered":"2024-04-21T08:44:34.492826+0000","last_unstale":"2024-04-21T08:47:00.748187+0000","last_undegraded":"2024-04-21T08:47:00.748187+0000","last_fullsized":"2024-04-21T08:47:00.748187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:36.815012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719352+0000","last_change":"2024-04-21T08:44:34.495142+0000","last_active":"2024-04-21T08:47:00.719352+0000","last_peered":"2024-04-21T08:47:00.719352+0000","last_clean":"2024-04-21T08:47:00.719352+0000","last_became_active":"2024-04-21T08:44:34.494803+0000","last_became_peered":"2024-04-21T08:44:34.494803+0000","last_unstale":"2024-04-21T08:47:00.719352+0000","last_undegraded":"2024-04-21T08:47:00.719352+0000","last_fullsized":"2024-04-21T08:47:00.719352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:22.974053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627186+0000","last_change":"2024-04-21T08:44:34.491252+0000","last_active":"2024-04-21T08:47:00.627186+0000","last_peered":"2024-04-21T08:47:00.627186+0000","last_clean":"2024-04-21T08:47:00.627186+0000","last_became_active":"2024-04-21T08:44:34.490814+0000","last_became_peered":"2024-04-21T08:44:34.490814+0000","last_unstale":"2024-04-21T08:47:00.627186+0000","last_undegraded":"2024-04-21T08:47:00.627186+0000","last_fullsized":"2024-04-21T08:47:00.627186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:54:52.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626862+0000","last_change":"2024-04-21T08:44:38.518647+0000","last_active":"2024-04-21T08:47:00.626862+0000","last_peered":"2024-04-21T08:47:00.626862+0000","last_clean":"2024-04-21T08:47:00.626862+0000","last_became_active":"2024-04-21T08:44:38.518239+0000","last_became_peered":"2024-04-21T08:44:38.518239+0000","last_unstale":"2024-04-21T08:47:00.626862+0000","last_undegraded":"2024-04-21T08:47:00.626862+0000","last_fullsized":"2024-04-21T08:47:00.626862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:44.781542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":167,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719204+0000","last_change":"2024-04-21T08:44:33.484484+0000","last_active":"2024-04-21T08:47:00.719204+0000","last_peered":"2024-04-21T08:47:00.719204+0000","last_clean":"2024-04-21T08:47:00.719204+0000","last_became_active":"2024-04-21T08:44:33.484160+0000","last_became_peered":"2024-04-21T08:44:33.484160+0000","last_unstale":"2024-04-21T08:47:00.719204+0000","last_undegraded":"2024-04-21T08:47:00.719204+0000","last_fullsized":"2024-04-21T08:47:00.719204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:45.962598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718920+0000","last_change":"2024-04-21T08:46:27.484947+0000","last_active":"2024-04-21T08:47:00.718920+0000","last_peered":"2024-04-21T08:47:00.718920+0000","last_clean":"2024-04-21T08:47:00.718920+0000","last_became_active":"2024-04-21T08:46:27.484530+0000","last_became_peered":"2024-04-21T08:46:27.484530+0000","last_unstale":"2024-04-21T08:47:00.718920+0000","last_undegraded":"2024-04-21T08:47:00.718920+0000","last_fullsized":"2024-04-21T08:47:00.718920+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:59.248176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747712+0000","last_change":"2024-04-21T08:45:27.518006+0000","last_active":"2024-04-21T08:47:00.747712+0000","last_peered":"2024-04-21T08:47:00.747712+0000","last_clean":"2024-04-21T08:47:00.747712+0000","last_became_active":"2024-04-21T08:45:27.517409+0000","last_became_peered":"2024-04-21T08:45:27.517409+0000","last_unstale":"2024-04-21T08:47:00.747712+0000","last_undegraded":"2024-04-21T08:47:00.747712+0000","last_fullsized":"2024-04-21T08:47:00.747712+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:45:01.253587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623343+0000","last_change":"2024-04-21T08:44:33.490508+0000","last_active":"2024-04-21T08:47:00.623343+0000","last_peered":"2024-04-21T08:47:00.623343+0000","last_clean":"2024-04-21T08:47:00.623343+0000","last_became_active":"2024-04-21T08:44:33.490084+0000","last_became_peered":"2024-04-21T08:44:33.490084+0000","last_unstale":"2024-04-21T08:47:00.623343+0000","last_undegraded":"2024-04-21T08:47:00.623343+0000","last_fullsized":"2024-04-21T08:47:00.623343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:21.553435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747770+0000","last_change":"2024-04-21T08:44:34.491843+0000","last_active":"2024-04-21T08:47:00.747770+0000","last_peered":"2024-04-21T08:47:00.747770+0000","last_clean":"2024-04-21T08:47:00.747770+0000","last_became_active":"2024-04-21T08:44:34.490578+0000","last_became_peered":"2024-04-21T08:44:34.490578+0000","last_unstale":"2024-04-21T08:47:00.747770+0000","last_undegraded":"2024-04-21T08:47:00.747770+0000","last_fullsized":"2024-04-21T08:47:00.747770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:05:12.272330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748032+0000","last_change":"2024-04-21T08:44:38.520258+0000","last_active":"2024-04-21T08:47:00.748032+0000","last_peered":"2024-04-21T08:47:00.748032+0000","last_clean":"2024-04-21T08:47:00.748032+0000","last_became_active":"2024-04-21T08:44:38.519824+0000","last_became_peered":"2024-04-21T08:44:38.519824+0000","last_unstale":"2024-04-21T08:47:00.748032+0000","last_undegraded":"2024-04-21T08:47:00.748032+0000","last_fullsized":"2024-04-21T08:47:00.748032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:39.716745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721769+0000","last_change":"2024-04-21T08:44:33.484934+0000","last_active":"2024-04-21T08:47:00.721769+0000","last_peered":"2024-04-21T08:47:00.721769+0000","last_clean":"2024-04-21T08:47:00.721769+0000","last_became_active":"2024-04-21T08:44:33.484591+0000","last_became_peered":"2024-04-21T08:44:33.484591+0000","last_unstale":"2024-04-21T08:47:00.721769+0000","last_undegraded":"2024-04-21T08:47:00.721769+0000","last_fullsized":"2024-04-21T08:47:00.721769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:48.826611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747621+0000","last_change":"2024-04-21T08:44:34.501109+0000","last_active":"2024-04-21T08:47:00.747621+0000","last_peered":"2024-04-21T08:47:00.747621+0000","last_clean":"2024-04-21T08:47:00.747621+0000","last_became_active":"2024-04-21T08:44:34.500750+0000","last_became_peered":"2024-04-21T08:44:34.500750+0000","last_unstale":"2024-04-21T08:47:00.747621+0000","last_undegraded":"2024-04-21T08:47:00.747621+0000","last_fullsized":"2024-04-21T08:47:00.747621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:29:28.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747712+0000","last_change":"2024-04-21T08:44:38.519633+0000","last_active":"2024-04-21T08:47:00.747712+0000","last_peered":"2024-04-21T08:47:00.747712+0000","last_clean":"2024-04-21T08:47:00.747712+0000","last_became_active":"2024-04-21T08:44:38.519215+0000","last_became_peered":"2024-04-21T08:44:38.519215+0000","last_unstale":"2024-04-21T08:47:00.747712+0000","last_undegraded":"2024-04-21T08:47:00.747712+0000","last_fullsized":"2024-04-21T08:47:00.747712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:43:59.695844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721688+0000","last_change":"2024-04-21T08:44:33.483267+0000","last_active":"2024-04-21T08:47:00.721688+0000","last_peered":"2024-04-21T08:47:00.721688+0000","last_clean":"2024-04-21T08:47:00.721688+0000","last_became_active":"2024-04-21T08:44:33.482932+0000","last_became_peered":"2024-04-21T08:44:33.482932+0000","last_unstale":"2024-04-21T08:47:00.721688+0000","last_undegraded":"2024-04-21T08:47:00.721688+0000","last_fullsized":"2024-04-21T08:47:00.721688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:53.311550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729235+0000","last_change":"2024-04-21T08:44:34.497389+0000","last_active":"2024-04-21T08:47:00.729235+0000","last_peered":"2024-04-21T08:47:00.729235+0000","last_clean":"2024-04-21T08:47:00.729235+0000","last_became_active":"2024-04-21T08:44:34.497030+0000","last_became_peered":"2024-04-21T08:44:34.497030+0000","last_unstale":"2024-04-21T08:47:00.729235+0000","last_undegraded":"2024-04-21T08:47:00.729235+0000","last_fullsized":"2024-04-21T08:47:00.729235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:20.371888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626918+0000","last_change":"2024-04-21T08:44:38.518095+0000","last_active":"2024-04-21T08:47:00.626918+0000","last_peered":"2024-04-21T08:47:00.626918+0000","last_clean":"2024-04-21T08:47:00.626918+0000","last_became_active":"2024-04-21T08:44:38.517010+0000","last_became_peered":"2024-04-21T08:44:38.517010+0000","last_unstale":"2024-04-21T08:47:00.626918+0000","last_undegraded":"2024-04-21T08:47:00.626918+0000","last_fullsized":"2024-04-21T08:47:00.626918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:02.379512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'8","reported_seq":167,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.167577+0000","last_change":"2024-04-21T08:44:33.487999+0000","last_active":"2024-04-21T08:47:01.167577+0000","last_peered":"2024-04-21T08:47:01.167577+0000","last_clean":"2024-04-21T08:47:01.167577+0000","last_became_active":"2024-04-21T08:44:33.487666+0000","last_became_peered":"2024-04-21T08:44:33.487666+0000","last_unstale":"2024-04-21T08:47:01.167577+0000","last_undegraded":"2024-04-21T08:47:01.167577+0000","last_fullsized":"2024-04-21T08:47:01.167577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:01.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1712,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721477+0000","last_change":"2024-04-21T08:44:34.494213+0000","last_active":"2024-04-21T08:47:00.721477+0000","last_peered":"2024-04-21T08:47:00.721477+0000","last_clean":"2024-04-21T08:47:00.721477+0000","last_became_active":"2024-04-21T08:44:34.493853+0000","last_became_peered":"2024-04-21T08:44:34.493853+0000","last_unstale":"2024-04-21T08:47:00.721477+0000","last_undegraded":"2024-04-21T08:47:00.721477+0000","last_fullsized":"2024-04-21T08:47:00.721477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:42.499131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627012+0000","last_change":"2024-04-21T08:45:27.410579+0000","last_active":"2024-04-21T08:47:00.627012+0000","last_peered":"2024-04-21T08:47:00.627012+0000","last_clean":"2024-04-21T08:47:00.627012+0000","last_became_active":"2024-04-21T08:45:27.409929+0000","last_became_peered":"2024-04-21T08:45:27.409929+0000","last_unstale":"2024-04-21T08:47:00.627012+0000","last_undegraded":"2024-04-21T08:47:00.627012+0000","last_fullsized":"2024-04-21T08:47:00.627012+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:35.897482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626508+0000","last_change":"2024-04-21T08:44:33.488751+0000","last_active":"2024-04-21T08:47:00.626508+0000","last_peered":"2024-04-21T08:47:00.626508+0000","last_clean":"2024-04-21T08:47:00.626508+0000","last_became_active":"2024-04-21T08:44:33.488425+0000","last_became_peered":"2024-04-21T08:44:33.488425+0000","last_unstale":"2024-04-21T08:47:00.626508+0000","last_undegraded":"2024-04-21T08:47:00.626508+0000","last_fullsized":"2024-04-21T08:47:00.626508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:28:47.868814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729103+0000","last_change":"2024-04-21T08:44:34.492770+0000","last_active":"2024-04-21T08:47:00.729103+0000","last_peered":"2024-04-21T08:47:00.729103+0000","last_clean":"2024-04-21T08:47:00.729103+0000","last_became_active":"2024-04-21T08:44:34.491340+0000","last_became_peered":"2024-04-21T08:44:34.491340+0000","last_unstale":"2024-04-21T08:47:00.729103+0000","last_undegraded":"2024-04-21T08:47:00.729103+0000","last_fullsized":"2024-04-21T08:47:00.729103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:47:54.280990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748097+0000","last_change":"2024-04-21T08:44:38.518888+0000","last_active":"2024-04-21T08:47:00.748097+0000","last_peered":"2024-04-21T08:47:00.748097+0000","last_clean":"2024-04-21T08:47:00.748097+0000","last_became_active":"2024-04-21T08:44:38.518421+0000","last_became_peered":"2024-04-21T08:44:38.518421+0000","last_unstale":"2024-04-21T08:47:00.748097+0000","last_undegraded":"2024-04-21T08:47:00.748097+0000","last_fullsized":"2024-04-21T08:47:00.748097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:21.119364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874482+0000","last_change":"2024-04-21T08:44:33.484383+0000","last_active":"2024-04-21T08:47:00.874482+0000","last_peered":"2024-04-21T08:47:00.874482+0000","last_clean":"2024-04-21T08:47:00.874482+0000","last_became_active":"2024-04-21T08:44:33.484083+0000","last_became_peered":"2024-04-21T08:44:33.484083+0000","last_unstale":"2024-04-21T08:47:00.874482+0000","last_undegraded":"2024-04-21T08:47:00.874482+0000","last_fullsized":"2024-04-21T08:47:00.874482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:13:14.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622126+0000","last_change":"2024-04-21T08:44:34.495636+0000","last_active":"2024-04-21T08:47:00.622126+0000","last_peered":"2024-04-21T08:47:00.622126+0000","last_clean":"2024-04-21T08:47:00.622126+0000","last_became_active":"2024-04-21T08:44:34.494967+0000","last_became_peered":"2024-04-21T08:44:34.494967+0000","last_unstale":"2024-04-21T08:47:00.622126+0000","last_undegraded":"2024-04-21T08:47:00.622126+0000","last_fullsized":"2024-04-21T08:47:00.622126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:47:02.379208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747724+0000","last_change":"2024-04-21T08:44:38.515145+0000","last_active":"2024-04-21T08:47:00.747724+0000","last_peered":"2024-04-21T08:47:00.747724+0000","last_clean":"2024-04-21T08:47:00.747724+0000","last_became_active":"2024-04-21T08:44:38.514755+0000","last_became_peered":"2024-04-21T08:44:38.514755+0000","last_unstale":"2024-04-21T08:47:00.747724+0000","last_undegraded":"2024-04-21T08:47:00.747724+0000","last_fullsized":"2024-04-21T08:47:00.747724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:38:15.826293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728869+0000","last_change":"2024-04-21T08:44:33.484714+0000","last_active":"2024-04-21T08:47:00.728869+0000","last_peered":"2024-04-21T08:47:00.728869+0000","last_clean":"2024-04-21T08:47:00.728869+0000","last_became_active":"2024-04-21T08:44:33.484385+0000","last_became_peered":"2024-04-21T08:44:33.484385+0000","last_unstale":"2024-04-21T08:47:00.728869+0000","last_undegraded":"2024-04-21T08:47:00.728869+0000","last_fullsized":"2024-04-21T08:47:00.728869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:03.406860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718544+0000","last_change":"2024-04-21T08:44:34.490989+0000","last_active":"2024-04-21T08:47:00.718544+0000","last_peered":"2024-04-21T08:47:00.718544+0000","last_clean":"2024-04-21T08:47:00.718544+0000","last_became_active":"2024-04-21T08:44:34.490658+0000","last_became_peered":"2024-04-21T08:44:34.490658+0000","last_unstale":"2024-04-21T08:47:00.718544+0000","last_undegraded":"2024-04-21T08:47:00.718544+0000","last_fullsized":"2024-04-21T08:47:00.718544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:42:15.537913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622288+0000","last_change":"2024-04-21T08:44:38.519723+0000","last_active":"2024-04-21T08:47:00.622288+0000","last_peered":"2024-04-21T08:47:00.622288+0000","last_clean":"2024-04-21T08:47:00.622288+0000","last_became_active":"2024-04-21T08:44:38.519336+0000","last_became_peered":"2024-04-21T08:44:38.519336+0000","last_unstale":"2024-04-21T08:47:00.622288+0000","last_undegraded":"2024-04-21T08:47:00.622288+0000","last_fullsized":"2024-04-21T08:47:00.622288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:00.332153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'6","reported_seq":281,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.237217+0000","last_change":"2024-04-21T08:44:33.484404+0000","last_active":"2024-04-21T08:47:01.237217+0000","last_peered":"2024-04-21T08:47:01.237217+0000","last_clean":"2024-04-21T08:47:01.237217+0000","last_became_active":"2024-04-21T08:44:33.484088+0000","last_became_peered":"2024-04-21T08:44:33.484088+0000","last_unstale":"2024-04-21T08:47:01.237217+0000","last_undegraded":"2024-04-21T08:47:01.237217+0000","last_fullsized":"2024-04-21T08:47:01.237217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:41.948622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":242,"num_read_kb":242,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748214+0000","last_change":"2024-04-21T08:44:34.502598+0000","last_active":"2024-04-21T08:47:00.748214+0000","last_peered":"2024-04-21T08:47:00.748214+0000","last_clean":"2024-04-21T08:47:00.748214+0000","last_became_active":"2024-04-21T08:44:34.502057+0000","last_became_peered":"2024-04-21T08:44:34.502057+0000","last_unstale":"2024-04-21T08:47:00.748214+0000","last_undegraded":"2024-04-21T08:47:00.748214+0000","last_fullsized":"2024-04-21T08:47:00.748214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:24.052873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718327+0000","last_change":"2024-04-21T08:44:38.519699+0000","last_active":"2024-04-21T08:47:00.718327+0000","last_peered":"2024-04-21T08:47:00.718327+0000","last_clean":"2024-04-21T08:47:00.718327+0000","last_became_active":"2024-04-21T08:44:38.517655+0000","last_became_peered":"2024-04-21T08:44:38.517655+0000","last_unstale":"2024-04-21T08:47:00.718327+0000","last_undegraded":"2024-04-21T08:47:00.718327+0000","last_fullsized":"2024-04-21T08:47:00.718327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:12.784479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748806+0000","last_change":"2024-04-21T08:46:27.494747+0000","last_active":"2024-04-21T08:47:00.748806+0000","last_peered":"2024-04-21T08:47:00.748806+0000","last_clean":"2024-04-21T08:47:00.748806+0000","last_became_active":"2024-04-21T08:46:27.490469+0000","last_became_peered":"2024-04-21T08:46:27.490469+0000","last_unstale":"2024-04-21T08:47:00.748806+0000","last_undegraded":"2024-04-21T08:47:00.748806+0000","last_fullsized":"2024-04-21T08:47:00.748806+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:16:35.257267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747850+0000","last_change":"2024-04-21T08:45:27.518395+0000","last_active":"2024-04-21T08:47:00.747850+0000","last_peered":"2024-04-21T08:47:00.747850+0000","last_clean":"2024-04-21T08:47:00.747850+0000","last_became_active":"2024-04-21T08:45:27.517969+0000","last_became_peered":"2024-04-21T08:45:27.517969+0000","last_unstale":"2024-04-21T08:47:00.747850+0000","last_undegraded":"2024-04-21T08:47:00.747850+0000","last_fullsized":"2024-04-21T08:47:00.747850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:55.306556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748406+0000","last_change":"2024-04-21T08:44:38.517956+0000","last_active":"2024-04-21T08:47:00.748406+0000","last_peered":"2024-04-21T08:47:00.748406+0000","last_clean":"2024-04-21T08:47:00.748406+0000","last_became_active":"2024-04-21T08:44:38.517422+0000","last_became_peered":"2024-04-21T08:44:38.517422+0000","last_unstale":"2024-04-21T08:47:00.748406+0000","last_undegraded":"2024-04-21T08:47:00.748406+0000","last_fullsized":"2024-04-21T08:47:00.748406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:04.650784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729223+0000","last_change":"2024-04-21T08:44:33.489151+0000","last_active":"2024-04-21T08:47:00.729223+0000","last_peered":"2024-04-21T08:47:00.729223+0000","last_clean":"2024-04-21T08:47:00.729223+0000","last_became_active":"2024-04-21T08:44:33.488781+0000","last_became_peered":"2024-04-21T08:44:33.488781+0000","last_unstale":"2024-04-21T08:47:00.729223+0000","last_undegraded":"2024-04-21T08:47:00.729223+0000","last_fullsized":"2024-04-21T08:47:00.729223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:40.675100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747905+0000","last_change":"2024-04-21T08:44:34.496528+0000","last_active":"2024-04-21T08:47:00.747905+0000","last_peered":"2024-04-21T08:47:00.747905+0000","last_clean":"2024-04-21T08:47:00.747905+0000","last_became_active":"2024-04-21T08:44:34.496223+0000","last_became_peered":"2024-04-21T08:44:34.496223+0000","last_unstale":"2024-04-21T08:47:00.747905+0000","last_undegraded":"2024-04-21T08:47:00.747905+0000","last_fullsized":"2024-04-21T08:47:00.747905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:56.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718461+0000","last_change":"2024-04-21T08:44:38.517537+0000","last_active":"2024-04-21T08:47:00.718461+0000","last_peered":"2024-04-21T08:47:00.718461+0000","last_clean":"2024-04-21T08:47:00.718461+0000","last_became_active":"2024-04-21T08:44:38.517134+0000","last_became_peered":"2024-04-21T08:44:38.517134+0000","last_unstale":"2024-04-21T08:47:00.718461+0000","last_undegraded":"2024-04-21T08:47:00.718461+0000","last_fullsized":"2024-04-21T08:47:00.718461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:38.634418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'39","reported_seq":326,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.179097+0000","last_change":"2024-04-21T08:44:33.485582+0000","last_active":"2024-04-21T08:47:01.179097+0000","last_peered":"2024-04-21T08:47:01.179097+0000","last_clean":"2024-04-21T08:47:01.179097+0000","last_became_active":"2024-04-21T08:44:33.485249+0000","last_became_peered":"2024-04-21T08:44:33.485249+0000","last_unstale":"2024-04-21T08:47:01.179097+0000","last_undegraded":"2024-04-21T08:47:01.179097+0000","last_fullsized":"2024-04-21T08:47:01.179097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:16:10.571694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67037,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":254,"num_read_kb":193,"num_write":41,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721670+0000","last_change":"2024-04-21T08:44:34.491361+0000","last_active":"2024-04-21T08:47:00.721670+0000","last_peered":"2024-04-21T08:47:00.721670+0000","last_clean":"2024-04-21T08:47:00.721670+0000","last_became_active":"2024-04-21T08:44:34.491050+0000","last_became_peered":"2024-04-21T08:44:34.491050+0000","last_unstale":"2024-04-21T08:47:00.721670+0000","last_undegraded":"2024-04-21T08:47:00.721670+0000","last_fullsized":"2024-04-21T08:47:00.721670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:39.082448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729534+0000","last_change":"2024-04-21T08:44:38.528745+0000","last_active":"2024-04-21T08:47:00.729534+0000","last_peered":"2024-04-21T08:47:00.729534+0000","last_clean":"2024-04-21T08:47:00.729534+0000","last_became_active":"2024-04-21T08:44:38.528275+0000","last_became_peered":"2024-04-21T08:44:38.528275+0000","last_unstale":"2024-04-21T08:47:00.729534+0000","last_undegraded":"2024-04-21T08:47:00.729534+0000","last_fullsized":"2024-04-21T08:47:00.729534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:06:11.338420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626897+0000","last_change":"2024-04-21T08:44:33.482059+0000","last_active":"2024-04-21T08:47:00.626897+0000","last_peered":"2024-04-21T08:47:00.626897+0000","last_clean":"2024-04-21T08:47:00.626897+0000","last_became_active":"2024-04-21T08:44:33.481663+0000","last_became_peered":"2024-04-21T08:44:33.481663+0000","last_unstale":"2024-04-21T08:47:00.626897+0000","last_undegraded":"2024-04-21T08:47:00.626897+0000","last_fullsized":"2024-04-21T08:47:00.626897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:01.240452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874587+0000","last_change":"2024-04-21T08:44:34.491398+0000","last_active":"2024-04-21T08:47:00.874587+0000","last_peered":"2024-04-21T08:47:00.874587+0000","last_clean":"2024-04-21T08:47:00.874587+0000","last_became_active":"2024-04-21T08:44:34.491073+0000","last_became_peered":"2024-04-21T08:44:34.491073+0000","last_unstale":"2024-04-21T08:47:00.874587+0000","last_undegraded":"2024-04-21T08:47:00.874587+0000","last_fullsized":"2024-04-21T08:47:00.874587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:22.566063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747937+0000","last_change":"2024-04-21T08:44:38.515653+0000","last_active":"2024-04-21T08:47:00.747937+0000","last_peered":"2024-04-21T08:47:00.747937+0000","last_clean":"2024-04-21T08:47:00.747937+0000","last_became_active":"2024-04-21T08:44:38.514796+0000","last_became_peered":"2024-04-21T08:44:38.514796+0000","last_unstale":"2024-04-21T08:47:00.747937+0000","last_undegraded":"2024-04-21T08:47:00.747937+0000","last_fullsized":"2024-04-21T08:47:00.747937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:00:22.662136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729003+0000","last_change":"2024-04-21T08:44:33.487073+0000","last_active":"2024-04-21T08:47:00.729003+0000","last_peered":"2024-04-21T08:47:00.729003+0000","last_clean":"2024-04-21T08:47:00.729003+0000","last_became_active":"2024-04-21T08:44:33.486740+0000","last_became_peered":"2024-04-21T08:44:33.486740+0000","last_unstale":"2024-04-21T08:47:00.729003+0000","last_undegraded":"2024-04-21T08:47:00.729003+0000","last_fullsized":"2024-04-21T08:47:00.729003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:32.673074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748512+0000","last_change":"2024-04-21T08:44:34.496757+0000","last_active":"2024-04-21T08:47:00.748512+0000","last_peered":"2024-04-21T08:47:00.748512+0000","last_clean":"2024-04-21T08:47:00.748512+0000","last_became_active":"2024-04-21T08:44:34.496422+0000","last_became_peered":"2024-04-21T08:44:34.496422+0000","last_unstale":"2024-04-21T08:47:00.748512+0000","last_undegraded":"2024-04-21T08:47:00.748512+0000","last_fullsized":"2024-04-21T08:47:00.748512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:08.686810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718934+0000","last_change":"2024-04-21T08:44:38.520935+0000","last_active":"2024-04-21T08:47:00.718934+0000","last_peered":"2024-04-21T08:47:00.718934+0000","last_clean":"2024-04-21T08:47:00.718934+0000","last_became_active":"2024-04-21T08:44:38.520481+0000","last_became_peered":"2024-04-21T08:44:38.520481+0000","last_unstale":"2024-04-21T08:47:00.718934+0000","last_undegraded":"2024-04-21T08:47:00.718934+0000","last_fullsized":"2024-04-21T08:47:00.718934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:12.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'7","reported_seq":166,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.165119+0000","last_change":"2024-04-21T08:44:33.485797+0000","last_active":"2024-04-21T08:47:01.165119+0000","last_peered":"2024-04-21T08:47:01.165119+0000","last_clean":"2024-04-21T08:47:01.165119+0000","last_became_active":"2024-04-21T08:44:33.485496+0000","last_became_peered":"2024-04-21T08:44:33.485496+0000","last_unstale":"2024-04-21T08:47:01.165119+0000","last_undegraded":"2024-04-21T08:47:01.165119+0000","last_fullsized":"2024-04-21T08:47:01.165119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:11:34.604611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748282+0000","last_change":"2024-04-21T08:44:34.506453+0000","last_active":"2024-04-21T08:47:00.748282+0000","last_peered":"2024-04-21T08:47:00.748282+0000","last_clean":"2024-04-21T08:47:00.748282+0000","last_became_active":"2024-04-21T08:44:34.506121+0000","last_became_peered":"2024-04-21T08:44:34.506121+0000","last_unstale":"2024-04-21T08:47:00.748282+0000","last_undegraded":"2024-04-21T08:47:00.748282+0000","last_fullsized":"2024-04-21T08:47:00.748282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:07.927366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875250+0000","last_change":"2024-04-21T08:44:38.523718+0000","last_active":"2024-04-21T08:47:00.875250+0000","last_peered":"2024-04-21T08:47:00.875250+0000","last_clean":"2024-04-21T08:47:00.875250+0000","last_became_active":"2024-04-21T08:44:38.523003+0000","last_became_peered":"2024-04-21T08:44:38.523003+0000","last_unstale":"2024-04-21T08:47:00.875250+0000","last_undegraded":"2024-04-21T08:47:00.875250+0000","last_fullsized":"2024-04-21T08:47:00.875250+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:20.837450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721393+0000","last_change":"2024-04-21T08:44:33.482029+0000","last_active":"2024-04-21T08:47:00.721393+0000","last_peered":"2024-04-21T08:47:00.721393+0000","last_clean":"2024-04-21T08:47:00.721393+0000","last_became_active":"2024-04-21T08:44:33.481713+0000","last_became_peered":"2024-04-21T08:44:33.481713+0000","last_unstale":"2024-04-21T08:47:00.721393+0000","last_undegraded":"2024-04-21T08:47:00.721393+0000","last_fullsized":"2024-04-21T08:47:00.721393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:46:34.717244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728921+0000","last_change":"2024-04-21T08:44:34.491944+0000","last_active":"2024-04-21T08:47:00.728921+0000","last_peered":"2024-04-21T08:47:00.728921+0000","last_clean":"2024-04-21T08:47:00.728921+0000","last_became_active":"2024-04-21T08:44:34.491638+0000","last_became_peered":"2024-04-21T08:44:34.491638+0000","last_unstale":"2024-04-21T08:47:00.728921+0000","last_undegraded":"2024-04-21T08:47:00.728921+0000","last_fullsized":"2024-04-21T08:47:00.728921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:08:07.925937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728793+0000","last_change":"2024-04-21T08:44:38.528971+0000","last_active":"2024-04-21T08:47:00.728793+0000","last_peered":"2024-04-21T08:47:00.728793+0000","last_clean":"2024-04-21T08:47:00.728793+0000","last_became_active":"2024-04-21T08:44:38.528453+0000","last_became_peered":"2024-04-21T08:44:38.528453+0000","last_unstale":"2024-04-21T08:47:00.728793+0000","last_undegraded":"2024-04-21T08:47:00.728793+0000","last_fullsized":"2024-04-21T08:47:00.728793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:50:31.237749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721764+0000","last_change":"2024-04-21T08:44:33.479262+0000","last_active":"2024-04-21T08:47:00.721764+0000","last_peered":"2024-04-21T08:47:00.721764+0000","last_clean":"2024-04-21T08:47:00.721764+0000","last_became_active":"2024-04-21T08:44:33.478907+0000","last_became_peered":"2024-04-21T08:44:33.478907+0000","last_unstale":"2024-04-21T08:47:00.721764+0000","last_undegraded":"2024-04-21T08:47:00.721764+0000","last_fullsized":"2024-04-21T08:47:00.721764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:33:43.879126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748024+0000","last_change":"2024-04-21T08:44:34.498503+0000","last_active":"2024-04-21T08:47:00.748024+0000","last_peered":"2024-04-21T08:47:00.748024+0000","last_clean":"2024-04-21T08:47:00.748024+0000","last_became_active":"2024-04-21T08:44:34.495140+0000","last_became_peered":"2024-04-21T08:44:34.495140+0000","last_unstale":"2024-04-21T08:47:00.748024+0000","last_undegraded":"2024-04-21T08:47:00.748024+0000","last_fullsized":"2024-04-21T08:47:00.748024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:24.518349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721372+0000","last_change":"2024-04-21T08:44:38.517717+0000","last_active":"2024-04-21T08:47:00.721372+0000","last_peered":"2024-04-21T08:47:00.721372+0000","last_clean":"2024-04-21T08:47:00.721372+0000","last_became_active":"2024-04-21T08:44:38.517272+0000","last_became_peered":"2024-04-21T08:44:38.517272+0000","last_unstale":"2024-04-21T08:47:00.721372+0000","last_undegraded":"2024-04-21T08:47:00.721372+0000","last_fullsized":"2024-04-21T08:47:00.721372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.664130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718792+0000","last_change":"2024-04-21T08:44:33.477498+0000","last_active":"2024-04-21T08:47:00.718792+0000","last_peered":"2024-04-21T08:47:00.718792+0000","last_clean":"2024-04-21T08:47:00.718792+0000","last_became_active":"2024-04-21T08:44:33.477139+0000","last_became_peered":"2024-04-21T08:44:33.477139+0000","last_unstale":"2024-04-21T08:47:00.718792+0000","last_undegraded":"2024-04-21T08:47:00.718792+0000","last_fullsized":"2024-04-21T08:47:00.718792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:26:59.390583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626701+0000","last_change":"2024-04-21T08:44:34.493586+0000","last_active":"2024-04-21T08:47:00.626701+0000","last_peered":"2024-04-21T08:47:00.626701+0000","last_clean":"2024-04-21T08:47:00.626701+0000","last_became_active":"2024-04-21T08:44:34.493258+0000","last_became_peered":"2024-04-21T08:44:34.493258+0000","last_unstale":"2024-04-21T08:47:00.626701+0000","last_undegraded":"2024-04-21T08:47:00.626701+0000","last_fullsized":"2024-04-21T08:47:00.626701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:11:18.783332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874853+0000","last_change":"2024-04-21T08:44:38.519783+0000","last_active":"2024-04-21T08:47:00.874853+0000","last_peered":"2024-04-21T08:47:00.874853+0000","last_clean":"2024-04-21T08:47:00.874853+0000","last_became_active":"2024-04-21T08:44:38.519397+0000","last_became_peered":"2024-04-21T08:44:38.519397+0000","last_unstale":"2024-04-21T08:47:00.874853+0000","last_undegraded":"2024-04-21T08:47:00.874853+0000","last_fullsized":"2024-04-21T08:47:00.874853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:08.740240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":160,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749428+0000","last_change":"2024-04-21T08:44:33.490098+0000","last_active":"2024-04-21T08:47:00.749428+0000","last_peered":"2024-04-21T08:47:00.749428+0000","last_clean":"2024-04-21T08:47:00.749428+0000","last_became_active":"2024-04-21T08:44:33.489691+0000","last_became_peered":"2024-04-21T08:44:33.489691+0000","last_unstale":"2024-04-21T08:47:00.749428+0000","last_undegraded":"2024-04-21T08:47:00.749428+0000","last_fullsized":"2024-04-21T08:47:00.749428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:44.965998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626543+0000","last_change":"2024-04-21T08:46:27.489580+0000","last_active":"2024-04-21T08:47:00.626543+0000","last_peered":"2024-04-21T08:47:00.626543+0000","last_clean":"2024-04-21T08:47:00.626543+0000","last_became_active":"2024-04-21T08:46:27.489086+0000","last_became_peered":"2024-04-21T08:46:27.489086+0000","last_unstale":"2024-04-21T08:47:00.626543+0000","last_undegraded":"2024-04-21T08:47:00.626543+0000","last_fullsized":"2024-04-21T08:47:00.626543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:38:50.618957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874908+0000","last_change":"2024-04-21T08:44:38.521010+0000","last_active":"2024-04-21T08:47:00.874908+0000","last_peered":"2024-04-21T08:47:00.874908+0000","last_clean":"2024-04-21T08:47:00.874908+0000","last_became_active":"2024-04-21T08:44:38.520605+0000","last_became_peered":"2024-04-21T08:44:38.520605+0000","last_unstale":"2024-04-21T08:47:00.874908+0000","last_undegraded":"2024-04-21T08:47:00.874908+0000","last_fullsized":"2024-04-21T08:47:00.874908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:50.378803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728846+0000","last_change":"2024-04-21T08:45:27.410137+0000","last_active":"2024-04-21T08:47:00.728846+0000","last_peered":"2024-04-21T08:47:00.728846+0000","last_clean":"2024-04-21T08:47:00.728846+0000","last_became_active":"2024-04-21T08:45:27.409728+0000","last_became_peered":"2024-04-21T08:45:27.409728+0000","last_unstale":"2024-04-21T08:47:00.728846+0000","last_undegraded":"2024-04-21T08:47:00.728846+0000","last_fullsized":"2024-04-21T08:47:00.728846+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:17.934902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":162,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.885117+0000","last_change":"2024-04-21T08:44:33.494031+0000","last_active":"2024-04-21T08:47:00.885117+0000","last_peered":"2024-04-21T08:47:00.885117+0000","last_clean":"2024-04-21T08:47:00.885117+0000","last_became_active":"2024-04-21T08:44:33.493580+0000","last_became_peered":"2024-04-21T08:44:33.493580+0000","last_unstale":"2024-04-21T08:47:00.885117+0000","last_undegraded":"2024-04-21T08:47:00.885117+0000","last_fullsized":"2024-04-21T08:47:00.885117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:39:48.230616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748658+0000","last_change":"2024-04-21T08:44:38.518067+0000","last_active":"2024-04-21T08:47:00.748658+0000","last_peered":"2024-04-21T08:47:00.748658+0000","last_clean":"2024-04-21T08:47:00.748658+0000","last_became_active":"2024-04-21T08:44:38.517594+0000","last_became_peered":"2024-04-21T08:44:38.517594+0000","last_unstale":"2024-04-21T08:47:00.748658+0000","last_undegraded":"2024-04-21T08:47:00.748658+0000","last_fullsized":"2024-04-21T08:47:00.748658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:35.261582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718443+0000","last_change":"2024-04-21T08:45:27.517719+0000","last_active":"2024-04-21T08:47:00.718443+0000","last_peered":"2024-04-21T08:47:00.718443+0000","last_clean":"2024-04-21T08:47:00.718443+0000","last_became_active":"2024-04-21T08:45:27.517429+0000","last_became_peered":"2024-04-21T08:45:27.517429+0000","last_unstale":"2024-04-21T08:47:00.718443+0000","last_undegraded":"2024-04-21T08:47:00.718443+0000","last_fullsized":"2024-04-21T08:47:00.718443+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:49:34.097759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748213+0000","last_change":"2024-04-21T08:44:33.485110+0000","last_active":"2024-04-21T08:47:00.748213+0000","last_peered":"2024-04-21T08:47:00.748213+0000","last_clean":"2024-04-21T08:47:00.748213+0000","last_became_active":"2024-04-21T08:44:33.484722+0000","last_became_peered":"2024-04-21T08:44:33.484722+0000","last_unstale":"2024-04-21T08:47:00.748213+0000","last_undegraded":"2024-04-21T08:47:00.748213+0000","last_fullsized":"2024-04-21T08:47:00.748213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:47:16.773560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":368,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748009+0000","last_change":"2024-04-21T08:43:17.984594+0000","last_active":"2024-04-21T08:47:00.748009+0000","last_peered":"2024-04-21T08:47:00.748009+0000","last_clean":"2024-04-21T08:47:00.748009+0000","last_became_active":"2024-04-21T08:43:17.984207+0000","last_became_peered":"2024-04-21T08:43:17.984207+0000","last_unstale":"2024-04-21T08:47:00.748009+0000","last_undegraded":"2024-04-21T08:47:00.748009+0000","last_fullsized":"2024-04-21T08:47:00.748009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_clean_scrub_stamp":"2024-04-21T08:39:50.910206+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:35.260499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721571+0000","last_change":"2024-04-21T08:45:27.518834+0000","last_active":"2024-04-21T08:47:00.721571+0000","last_peered":"2024-04-21T08:47:00.721571+0000","last_clean":"2024-04-21T08:47:00.721571+0000","last_became_active":"2024-04-21T08:45:27.518390+0000","last_became_peered":"2024-04-21T08:45:27.518390+0000","last_unstale":"2024-04-21T08:47:00.721571+0000","last_undegraded":"2024-04-21T08:47:00.721571+0000","last_fullsized":"2024-04-21T08:47:00.721571+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:58:32.673008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874296+0000","last_change":"2024-04-21T08:44:34.499975+0000","last_active":"2024-04-21T08:47:00.874296+0000","last_peered":"2024-04-21T08:47:00.874296+0000","last_clean":"2024-04-21T08:47:00.874296+0000","last_became_active":"2024-04-21T08:44:34.499646+0000","last_became_peered":"2024-04-21T08:44:34.499646+0000","last_unstale":"2024-04-21T08:47:00.874296+0000","last_undegraded":"2024-04-21T08:47:00.874296+0000","last_fullsized":"2024-04-21T08:47:00.874296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:19:38.082502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748114+0000","last_change":"2024-04-21T08:44:33.484449+0000","last_active":"2024-04-21T08:47:00.748114+0000","last_peered":"2024-04-21T08:47:00.748114+0000","last_clean":"2024-04-21T08:47:00.748114+0000","last_became_active":"2024-04-21T08:44:33.484155+0000","last_became_peered":"2024-04-21T08:44:33.484155+0000","last_unstale":"2024-04-21T08:47:00.748114+0000","last_undegraded":"2024-04-21T08:47:00.748114+0000","last_fullsized":"2024-04-21T08:47:00.748114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:03.480474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747766+0000","last_change":"2024-04-21T08:44:38.515283+0000","last_active":"2024-04-21T08:47:00.747766+0000","last_peered":"2024-04-21T08:47:00.747766+0000","last_clean":"2024-04-21T08:47:00.747766+0000","last_became_active":"2024-04-21T08:44:38.514764+0000","last_became_peered":"2024-04-21T08:44:38.514764+0000","last_unstale":"2024-04-21T08:47:00.747766+0000","last_undegraded":"2024-04-21T08:47:00.747766+0000","last_fullsized":"2024-04-21T08:47:00.747766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:17:36.378392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721569+0000","last_change":"2024-04-21T08:44:33.489198+0000","last_active":"2024-04-21T08:47:00.721569+0000","last_peered":"2024-04-21T08:47:00.721569+0000","last_clean":"2024-04-21T08:47:00.721569+0000","last_became_active":"2024-04-21T08:44:33.488794+0000","last_became_peered":"2024-04-21T08:44:33.488794+0000","last_unstale":"2024-04-21T08:47:00.721569+0000","last_undegraded":"2024-04-21T08:47:00.721569+0000","last_fullsized":"2024-04-21T08:47:00.721569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:24.948283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874977+0000","last_change":"2024-04-21T08:44:34.488811+0000","last_active":"2024-04-21T08:47:00.874977+0000","last_peered":"2024-04-21T08:47:00.874977+0000","last_clean":"2024-04-21T08:47:00.874977+0000","last_became_active":"2024-04-21T08:44:34.488472+0000","last_became_peered":"2024-04-21T08:44:34.488472+0000","last_unstale":"2024-04-21T08:47:00.874977+0000","last_undegraded":"2024-04-21T08:47:00.874977+0000","last_fullsized":"2024-04-21T08:47:00.874977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:44.779492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622045+0000","last_change":"2024-04-21T08:44:38.522711+0000","last_active":"2024-04-21T08:47:00.622045+0000","last_peered":"2024-04-21T08:47:00.622045+0000","last_clean":"2024-04-21T08:47:00.622045+0000","last_became_active":"2024-04-21T08:44:38.522322+0000","last_became_peered":"2024-04-21T08:44:38.522322+0000","last_unstale":"2024-04-21T08:47:00.622045+0000","last_undegraded":"2024-04-21T08:47:00.622045+0000","last_fullsized":"2024-04-21T08:47:00.622045+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:26:13.323283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'48","reported_seq":104,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.836065+0000","last_change":"2024-04-21T08:44:33.481815+0000","last_active":"2024-04-21T08:47:00.836065+0000","last_peered":"2024-04-21T08:47:00.836065+0000","last_clean":"2024-04-21T08:47:00.836065+0000","last_became_active":"2024-04-21T08:44:33.481469+0000","last_became_peered":"2024-04-21T08:44:33.481469+0000","last_unstale":"2024-04-21T08:47:00.836065+0000","last_undegraded":"2024-04-21T08:47:00.836065+0000","last_fullsized":"2024-04-21T08:47:00.836065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:13.970821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623173+0000","last_change":"2024-04-21T08:44:34.496996+0000","last_active":"2024-04-21T08:47:00.623173+0000","last_peered":"2024-04-21T08:47:00.623173+0000","last_clean":"2024-04-21T08:47:00.623173+0000","last_became_active":"2024-04-21T08:44:34.496676+0000","last_became_peered":"2024-04-21T08:44:34.496676+0000","last_unstale":"2024-04-21T08:47:00.623173+0000","last_undegraded":"2024-04-21T08:47:00.623173+0000","last_fullsized":"2024-04-21T08:47:00.623173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:43:08.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626592+0000","last_change":"2024-04-21T08:44:38.516240+0000","last_active":"2024-04-21T08:47:00.626592+0000","last_peered":"2024-04-21T08:47:00.626592+0000","last_clean":"2024-04-21T08:47:00.626592+0000","last_became_active":"2024-04-21T08:44:38.515801+0000","last_became_peered":"2024-04-21T08:44:38.515801+0000","last_unstale":"2024-04-21T08:47:00.626592+0000","last_undegraded":"2024-04-21T08:47:00.626592+0000","last_fullsized":"2024-04-21T08:47:00.626592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:46.520373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749302+0000","last_change":"2024-04-21T08:44:33.488952+0000","last_active":"2024-04-21T08:47:00.749302+0000","last_peered":"2024-04-21T08:47:00.749302+0000","last_clean":"2024-04-21T08:47:00.749302+0000","last_became_active":"2024-04-21T08:44:33.488617+0000","last_became_peered":"2024-04-21T08:44:33.488617+0000","last_unstale":"2024-04-21T08:47:00.749302+0000","last_undegraded":"2024-04-21T08:47:00.749302+0000","last_fullsized":"2024-04-21T08:47:00.749302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:37:24.679132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747751+0000","last_change":"2024-04-21T08:44:34.510568+0000","last_active":"2024-04-21T08:47:00.747751+0000","last_peered":"2024-04-21T08:47:00.747751+0000","last_clean":"2024-04-21T08:47:00.747751+0000","last_became_active":"2024-04-21T08:44:34.509900+0000","last_became_peered":"2024-04-21T08:44:34.509900+0000","last_unstale":"2024-04-21T08:47:00.747751+0000","last_undegraded":"2024-04-21T08:47:00.747751+0000","last_fullsized":"2024-04-21T08:47:00.747751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:08.426141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748835+0000","last_change":"2024-04-21T08:45:27.976874+0000","last_active":"2024-04-21T08:47:00.748835+0000","last_peered":"2024-04-21T08:47:00.748835+0000","last_clean":"2024-04-21T08:47:00.748835+0000","last_became_active":"2024-04-21T08:45:27.976387+0000","last_became_peered":"2024-04-21T08:45:27.976387+0000","last_unstale":"2024-04-21T08:47:00.748835+0000","last_undegraded":"2024-04-21T08:47:00.748835+0000","last_fullsized":"2024-04-21T08:47:00.748835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:38:18.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747268+0000","last_change":"2024-04-21T08:44:33.481803+0000","last_active":"2024-04-21T08:47:00.747268+0000","last_peered":"2024-04-21T08:47:00.747268+0000","last_clean":"2024-04-21T08:47:00.747268+0000","last_became_active":"2024-04-21T08:44:33.481256+0000","last_became_peered":"2024-04-21T08:44:33.481256+0000","last_unstale":"2024-04-21T08:47:00.747268+0000","last_undegraded":"2024-04-21T08:47:00.747268+0000","last_fullsized":"2024-04-21T08:47:00.747268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:17.804557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721258+0000","last_change":"2024-04-21T08:44:34.490206+0000","last_active":"2024-04-21T08:47:00.721258+0000","last_peered":"2024-04-21T08:47:00.721258+0000","last_clean":"2024-04-21T08:47:00.721258+0000","last_became_active":"2024-04-21T08:44:34.489797+0000","last_became_peered":"2024-04-21T08:44:34.489797+0000","last_unstale":"2024-04-21T08:47:00.721258+0000","last_undegraded":"2024-04-21T08:47:00.721258+0000","last_fullsized":"2024-04-21T08:47:00.721258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:09.055429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719147+0000","last_change":"2024-04-21T08:44:38.517385+0000","last_active":"2024-04-21T08:47:00.719147+0000","last_peered":"2024-04-21T08:47:00.719147+0000","last_clean":"2024-04-21T08:47:00.719147+0000","last_became_active":"2024-04-21T08:44:38.516964+0000","last_became_peered":"2024-04-21T08:44:38.516964+0000","last_unstale":"2024-04-21T08:47:00.719147+0000","last_undegraded":"2024-04-21T08:47:00.719147+0000","last_fullsized":"2024-04-21T08:47:00.719147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:33.364076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622039+0000","last_change":"2024-04-21T08:44:33.490586+0000","last_active":"2024-04-21T08:47:00.622039+0000","last_peered":"2024-04-21T08:47:00.622039+0000","last_clean":"2024-04-21T08:47:00.622039+0000","last_became_active":"2024-04-21T08:44:33.489281+0000","last_became_peered":"2024-04-21T08:44:33.489281+0000","last_unstale":"2024-04-21T08:47:00.622039+0000","last_undegraded":"2024-04-21T08:47:00.622039+0000","last_fullsized":"2024-04-21T08:47:00.622039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:41:29.856165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747602+0000","last_change":"2024-04-21T08:44:34.493394+0000","last_active":"2024-04-21T08:47:00.747602+0000","last_peered":"2024-04-21T08:47:00.747602+0000","last_clean":"2024-04-21T08:47:00.747602+0000","last_became_active":"2024-04-21T08:44:34.492367+0000","last_became_peered":"2024-04-21T08:44:34.492367+0000","last_unstale":"2024-04-21T08:47:00.747602+0000","last_undegraded":"2024-04-21T08:47:00.747602+0000","last_fullsized":"2024-04-21T08:47:00.747602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:13.111266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748165+0000","last_change":"2024-04-21T08:44:38.519113+0000","last_active":"2024-04-21T08:47:00.748165+0000","last_peered":"2024-04-21T08:47:00.748165+0000","last_clean":"2024-04-21T08:47:00.748165+0000","last_became_active":"2024-04-21T08:44:38.518469+0000","last_became_peered":"2024-04-21T08:44:38.518469+0000","last_unstale":"2024-04-21T08:47:00.748165+0000","last_undegraded":"2024-04-21T08:47:00.748165+0000","last_fullsized":"2024-04-21T08:47:00.748165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:39:22.200041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'5","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747723+0000","last_change":"2024-04-21T08:46:27.492549+0000","last_active":"2024-04-21T08:47:00.747723+0000","last_peered":"2024-04-21T08:47:00.747723+0000","last_clean":"2024-04-21T08:47:00.747723+0000","last_became_active":"2024-04-21T08:46:27.488054+0000","last_became_peered":"2024-04-21T08:46:27.488054+0000","last_unstale":"2024-04-21T08:47:00.747723+0000","last_undegraded":"2024-04-21T08:47:00.747723+0000","last_fullsized":"2024-04-21T08:47:00.747723+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:51:54.797532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622446+0000","last_change":"2024-04-21T08:44:34.502226+0000","last_active":"2024-04-21T08:47:00.622446+0000","last_peered":"2024-04-21T08:47:00.622446+0000","last_clean":"2024-04-21T08:47:00.622446+0000","last_became_active":"2024-04-21T08:44:34.501775+0000","last_became_peered":"2024-04-21T08:44:34.501775+0000","last_unstale":"2024-04-21T08:47:00.622446+0000","last_undegraded":"2024-04-21T08:47:00.622446+0000","last_fullsized":"2024-04-21T08:47:00.622446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:58.325267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748655+0000","last_change":"2024-04-21T08:44:38.522250+0000","last_active":"2024-04-21T08:47:00.748655+0000","last_peered":"2024-04-21T08:47:00.748655+0000","last_clean":"2024-04-21T08:47:00.748655+0000","last_became_active":"2024-04-21T08:44:38.520870+0000","last_became_peered":"2024-04-21T08:44:38.520870+0000","last_unstale":"2024-04-21T08:47:00.748655+0000","last_undegraded":"2024-04-21T08:47:00.748655+0000","last_fullsized":"2024-04-21T08:47:00.748655+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:02.391785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747761+0000","last_change":"2024-04-21T08:46:27.496215+0000","last_active":"2024-04-21T08:47:00.747761+0000","last_peered":"2024-04-21T08:47:00.747761+0000","last_clean":"2024-04-21T08:47:00.747761+0000","last_became_active":"2024-04-21T08:46:27.488287+0000","last_became_peered":"2024-04-21T08:46:27.488287+0000","last_unstale":"2024-04-21T08:47:00.747761+0000","last_undegraded":"2024-04-21T08:47:00.747761+0000","last_fullsized":"2024-04-21T08:47:00.747761+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:08:05.390563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749473+0000","last_change":"2024-04-21T08:44:34.496034+0000","last_active":"2024-04-21T08:47:00.749473+0000","last_peered":"2024-04-21T08:47:00.749473+0000","last_clean":"2024-04-21T08:47:00.749473+0000","last_became_active":"2024-04-21T08:44:34.495641+0000","last_became_peered":"2024-04-21T08:44:34.495641+0000","last_unstale":"2024-04-21T08:47:00.749473+0000","last_undegraded":"2024-04-21T08:47:00.749473+0000","last_fullsized":"2024-04-21T08:47:00.749473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:21:18.893326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747361+0000","last_change":"2024-04-21T08:44:38.517288+0000","last_active":"2024-04-21T08:47:00.747361+0000","last_peered":"2024-04-21T08:47:00.747361+0000","last_clean":"2024-04-21T08:47:00.747361+0000","last_became_active":"2024-04-21T08:44:38.516853+0000","last_became_peered":"2024-04-21T08:44:38.516853+0000","last_unstale":"2024-04-21T08:47:00.747361+0000","last_undegraded":"2024-04-21T08:47:00.747361+0000","last_fullsized":"2024-04-21T08:47:00.747361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:27:22.371703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'7","reported_seq":163,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:01.237801+0000","last_change":"2024-04-21T08:44:33.491578+0000","last_active":"2024-04-21T08:47:01.237801+0000","last_peered":"2024-04-21T08:47:01.237801+0000","last_clean":"2024-04-21T08:47:01.237801+0000","last_became_active":"2024-04-21T08:44:33.491238+0000","last_became_peered":"2024-04-21T08:44:33.491238+0000","last_unstale":"2024-04-21T08:47:01.237801+0000","last_undegraded":"2024-04-21T08:47:01.237801+0000","last_fullsized":"2024-04-21T08:47:01.237801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:50.399031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749074+0000","last_change":"2024-04-21T08:44:34.497632+0000","last_active":"2024-04-21T08:47:00.749074+0000","last_peered":"2024-04-21T08:47:00.749074+0000","last_clean":"2024-04-21T08:47:00.749074+0000","last_became_active":"2024-04-21T08:44:34.497318+0000","last_became_peered":"2024-04-21T08:44:34.497318+0000","last_unstale":"2024-04-21T08:47:00.749074+0000","last_undegraded":"2024-04-21T08:47:00.749074+0000","last_fullsized":"2024-04-21T08:47:00.749074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:11:33.918530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747370+0000","last_change":"2024-04-21T08:44:38.518414+0000","last_active":"2024-04-21T08:47:00.747370+0000","last_peered":"2024-04-21T08:47:00.747370+0000","last_clean":"2024-04-21T08:47:00.747370+0000","last_became_active":"2024-04-21T08:44:38.517953+0000","last_became_peered":"2024-04-21T08:44:38.517953+0000","last_unstale":"2024-04-21T08:47:00.747370+0000","last_undegraded":"2024-04-21T08:47:00.747370+0000","last_fullsized":"2024-04-21T08:47:00.747370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:48.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.833716+0000","last_change":"2024-04-21T08:44:33.493896+0000","last_active":"2024-04-21T08:47:00.833716+0000","last_peered":"2024-04-21T08:47:00.833716+0000","last_clean":"2024-04-21T08:47:00.833716+0000","last_became_active":"2024-04-21T08:44:33.493566+0000","last_became_peered":"2024-04-21T08:44:33.493566+0000","last_unstale":"2024-04-21T08:47:00.833716+0000","last_undegraded":"2024-04-21T08:47:00.833716+0000","last_fullsized":"2024-04-21T08:47:00.833716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:47:17.307690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748612+0000","last_change":"2024-04-21T08:44:34.500406+0000","last_active":"2024-04-21T08:47:00.748612+0000","last_peered":"2024-04-21T08:47:00.748612+0000","last_clean":"2024-04-21T08:47:00.748612+0000","last_became_active":"2024-04-21T08:44:34.500048+0000","last_became_peered":"2024-04-21T08:44:34.500048+0000","last_unstale":"2024-04-21T08:47:00.748612+0000","last_undegraded":"2024-04-21T08:47:00.748612+0000","last_fullsized":"2024-04-21T08:47:00.748612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:10:58.166155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728633+0000","last_change":"2024-04-21T08:44:38.514812+0000","last_active":"2024-04-21T08:47:00.728633+0000","last_peered":"2024-04-21T08:47:00.728633+0000","last_clean":"2024-04-21T08:47:00.728633+0000","last_became_active":"2024-04-21T08:44:38.514247+0000","last_became_peered":"2024-04-21T08:44:38.514247+0000","last_unstale":"2024-04-21T08:47:00.728633+0000","last_undegraded":"2024-04-21T08:47:00.728633+0000","last_fullsized":"2024-04-21T08:47:00.728633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:06:10.588453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875149+0000","last_change":"2024-04-21T08:44:33.493887+0000","last_active":"2024-04-21T08:47:00.875149+0000","last_peered":"2024-04-21T08:47:00.875149+0000","last_clean":"2024-04-21T08:47:00.875149+0000","last_became_active":"2024-04-21T08:44:33.493555+0000","last_became_peered":"2024-04-21T08:44:33.493555+0000","last_unstale":"2024-04-21T08:47:00.875149+0000","last_undegraded":"2024-04-21T08:47:00.875149+0000","last_fullsized":"2024-04-21T08:47:00.875149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:18.089102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749096+0000","last_change":"2024-04-21T08:44:34.503259+0000","last_active":"2024-04-21T08:47:00.749096+0000","last_peered":"2024-04-21T08:47:00.749096+0000","last_clean":"2024-04-21T08:47:00.749096+0000","last_became_active":"2024-04-21T08:44:34.502845+0000","last_became_peered":"2024-04-21T08:44:34.502845+0000","last_unstale":"2024-04-21T08:47:00.749096+0000","last_undegraded":"2024-04-21T08:47:00.749096+0000","last_fullsized":"2024-04-21T08:47:00.749096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:19.902178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747653+0000","last_change":"2024-04-21T08:44:34.497608+0000","last_active":"2024-04-21T08:47:00.747653+0000","last_peered":"2024-04-21T08:47:00.747653+0000","last_clean":"2024-04-21T08:47:00.747653+0000","last_became_active":"2024-04-21T08:44:34.497255+0000","last_became_peered":"2024-04-21T08:44:34.497255+0000","last_unstale":"2024-04-21T08:47:00.747653+0000","last_undegraded":"2024-04-21T08:47:00.747653+0000","last_fullsized":"2024-04-21T08:47:00.747653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:51:15.181837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626828+0000","last_change":"2024-04-21T08:44:34.495408+0000","last_active":"2024-04-21T08:47:00.626828+0000","last_peered":"2024-04-21T08:47:00.626828+0000","last_clean":"2024-04-21T08:47:00.626828+0000","last_became_active":"2024-04-21T08:44:34.495038+0000","last_became_peered":"2024-04-21T08:44:34.495038+0000","last_unstale":"2024-04-21T08:47:00.626828+0000","last_undegraded":"2024-04-21T08:47:00.626828+0000","last_fullsized":"2024-04-21T08:47:00.626828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:59.512245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626703+0000","last_change":"2024-04-21T08:44:34.493292+0000","last_active":"2024-04-21T08:47:00.626703+0000","last_peered":"2024-04-21T08:47:00.626703+0000","last_clean":"2024-04-21T08:47:00.626703+0000","last_became_active":"2024-04-21T08:44:34.492913+0000","last_became_peered":"2024-04-21T08:44:34.492913+0000","last_unstale":"2024-04-21T08:47:00.626703+0000","last_undegraded":"2024-04-21T08:47:00.626703+0000","last_fullsized":"2024-04-21T08:47:00.626703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:24.271859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747551+0000","last_change":"2024-04-21T08:44:34.510448+0000","last_active":"2024-04-21T08:47:00.747551+0000","last_peered":"2024-04-21T08:47:00.747551+0000","last_clean":"2024-04-21T08:47:00.747551+0000","last_became_active":"2024-04-21T08:44:34.509958+0000","last_became_peered":"2024-04-21T08:44:34.509958+0000","last_unstale":"2024-04-21T08:47:00.747551+0000","last_undegraded":"2024-04-21T08:47:00.747551+0000","last_fullsized":"2024-04-21T08:47:00.747551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:36.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875000+0000","last_change":"2024-04-21T08:44:34.497415+0000","last_active":"2024-04-21T08:47:00.875000+0000","last_peered":"2024-04-21T08:47:00.875000+0000","last_clean":"2024-04-21T08:47:00.875000+0000","last_became_active":"2024-04-21T08:44:34.497120+0000","last_became_peered":"2024-04-21T08:44:34.497120+0000","last_unstale":"2024-04-21T08:47:00.875000+0000","last_undegraded":"2024-04-21T08:47:00.875000+0000","last_fullsized":"2024-04-21T08:47:00.875000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:30.232338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747974+0000","last_change":"2024-04-21T08:44:34.500356+0000","last_active":"2024-04-21T08:47:00.747974+0000","last_peered":"2024-04-21T08:47:00.747974+0000","last_clean":"2024-04-21T08:47:00.747974+0000","last_became_active":"2024-04-21T08:44:34.500032+0000","last_became_peered":"2024-04-21T08:44:34.500032+0000","last_unstale":"2024-04-21T08:47:00.747974+0000","last_undegraded":"2024-04-21T08:47:00.747974+0000","last_fullsized":"2024-04-21T08:47:00.747974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:24.900221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627464+0000","last_change":"2024-04-21T08:44:34.493629+0000","last_active":"2024-04-21T08:47:00.627464+0000","last_peered":"2024-04-21T08:47:00.627464+0000","last_clean":"2024-04-21T08:47:00.627464+0000","last_became_active":"2024-04-21T08:44:34.493273+0000","last_became_peered":"2024-04-21T08:44:34.493273+0000","last_unstale":"2024-04-21T08:47:00.627464+0000","last_undegraded":"2024-04-21T08:47:00.627464+0000","last_fullsized":"2024-04-21T08:47:00.627464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:22:42.698957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623249+0000","last_change":"2024-04-21T08:44:34.493427+0000","last_active":"2024-04-21T08:47:00.623249+0000","last_peered":"2024-04-21T08:47:00.623249+0000","last_clean":"2024-04-21T08:47:00.623249+0000","last_became_active":"2024-04-21T08:44:34.493105+0000","last_became_peered":"2024-04-21T08:44:34.493105+0000","last_unstale":"2024-04-21T08:47:00.623249+0000","last_undegraded":"2024-04-21T08:47:00.623249+0000","last_fullsized":"2024-04-21T08:47:00.623249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:37.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874153+0000","last_change":"2024-04-21T08:45:27.515899+0000","last_active":"2024-04-21T08:47:00.874153+0000","last_peered":"2024-04-21T08:47:00.874153+0000","last_clean":"2024-04-21T08:47:00.874153+0000","last_became_active":"2024-04-21T08:45:27.515567+0000","last_became_peered":"2024-04-21T08:45:27.515567+0000","last_unstale":"2024-04-21T08:47:00.874153+0000","last_undegraded":"2024-04-21T08:47:00.874153+0000","last_fullsized":"2024-04-21T08:47:00.874153+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:16.645869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622080+0000","last_change":"2024-04-21T08:44:34.499420+0000","last_active":"2024-04-21T08:47:00.622080+0000","last_peered":"2024-04-21T08:47:00.622080+0000","last_clean":"2024-04-21T08:47:00.622080+0000","last_became_active":"2024-04-21T08:44:34.499079+0000","last_became_peered":"2024-04-21T08:44:34.499079+0000","last_unstale":"2024-04-21T08:47:00.622080+0000","last_undegraded":"2024-04-21T08:47:00.622080+0000","last_fullsized":"2024-04-21T08:47:00.622080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:45.351422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747540+0000","last_change":"2024-04-21T08:44:34.493420+0000","last_active":"2024-04-21T08:47:00.747540+0000","last_peered":"2024-04-21T08:47:00.747540+0000","last_clean":"2024-04-21T08:47:00.747540+0000","last_became_active":"2024-04-21T08:44:34.493098+0000","last_became_peered":"2024-04-21T08:44:34.493098+0000","last_unstale":"2024-04-21T08:47:00.747540+0000","last_undegraded":"2024-04-21T08:47:00.747540+0000","last_fullsized":"2024-04-21T08:47:00.747540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:22:34.316822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874666+0000","last_change":"2024-04-21T08:44:34.494760+0000","last_active":"2024-04-21T08:47:00.874666+0000","last_peered":"2024-04-21T08:47:00.874666+0000","last_clean":"2024-04-21T08:47:00.874666+0000","last_became_active":"2024-04-21T08:44:34.494265+0000","last_became_peered":"2024-04-21T08:44:34.494265+0000","last_unstale":"2024-04-21T08:47:00.874666+0000","last_undegraded":"2024-04-21T08:47:00.874666+0000","last_fullsized":"2024-04-21T08:47:00.874666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:16.035030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748002+0000","last_change":"2024-04-21T08:44:34.498785+0000","last_active":"2024-04-21T08:47:00.748002+0000","last_peered":"2024-04-21T08:47:00.748002+0000","last_clean":"2024-04-21T08:47:00.748002+0000","last_became_active":"2024-04-21T08:44:34.495159+0000","last_became_peered":"2024-04-21T08:44:34.495159+0000","last_unstale":"2024-04-21T08:47:00.748002+0000","last_undegraded":"2024-04-21T08:47:00.748002+0000","last_fullsized":"2024-04-21T08:47:00.748002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:03:20.095016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600929,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1346,"num_read_kb":788,"num_write":372,"num_write_kb":1466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1802787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1081344,"data_stored":531093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28888,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716264,"statfs":{"total":95995035648,"available":95965454336,"internally_reserved":0,"allocated":1916928,"data_stored":1316546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1503232,"data_stored":927553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1495040,"data_stored":923835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1699840,"data_stored":1128597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":525745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":593812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":527983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":595904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1050269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":529819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37196,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707956,"statfs":{"total":95995035648,"available":95956946944,"internally_reserved":0,"allocated":1843200,"data_stored":1254997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":201155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:47:06.378 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:47:06.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.514+0000 7f02d366d640 1 -- 172.21.15.82:0/1410295034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02cc1011a0 msgr2=0x7f02cc101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.514+0000 7f02d366d640 1 --2- 172.21.15.82:0/1410295034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02cc1011a0 0x7f02cc101620 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009930 tx=0x7f02bc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.514+0000 7f02d366d640 1 -- 172.21.15.82:0/1410295034 shutdown_connections 2024-04-21T08:47:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.514+0000 7f02d366d640 1 --2- 172.21.15.82:0/1410295034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02cc101b60 0x7f02cc10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.514+0000 7f02d366d640 1 --2- 172.21.15.82:0/1410295034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02cc1011a0 0x7f02cc101620 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.514+0000 7f02d366d640 1 --2- 172.21.15.82:0/1410295034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02cc105e30 0x7f02cc106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.514+0000 7f02d366d640 1 -- 172.21.15.82:0/1410295034 >> 172.21.15.82:0/1410295034 conn(0x7f02cc077e30 msgr2=0x7f02cc078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.514+0000 7f02d366d640 1 -- 172.21.15.82:0/1410295034 shutdown_connections 2024-04-21T08:47:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.514+0000 7f02d366d640 1 -- 172.21.15.82:0/1410295034 wait complete. 2024-04-21T08:47:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d366d640 1 Processor -- start 2024-04-21T08:47:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d366d640 1 -- start start 2024-04-21T08:47:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d366d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02cc1011a0 0x7f02cc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d366d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02cc101b60 0x7f02cc198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:06.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d366d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02cc105e30 0x7f02cc19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:06.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d366d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc1110e0 con 0x7f02cc101b60 2024-04-21T08:47:06.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d366d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc110f60 con 0x7f02cc105e30 2024-04-21T08:47:06.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d366d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc111260 con 0x7f02cc1011a0 2024-04-21T08:47:06.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d0c08640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02cc101b60 0x7f02cc198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:06.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d0c08640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02cc101b60 0x7f02cc198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44428/0 (socket says 172.21.15.82:44428) 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d0c08640 1 -- 172.21.15.82:0/47230490 learned_addr learned my addr 172.21.15.82:0/47230490 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d1c0a640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02cc105e30 0x7f02cc19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d1409640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02cc1011a0 0x7f02cc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d0c08640 1 -- 172.21.15.82:0/47230490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02cc1011a0 msgr2=0x7f02cc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d0c08640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02cc1011a0 0x7f02cc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d0c08640 1 -- 172.21.15.82:0/47230490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02cc105e30 msgr2=0x7f02cc19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d0c08640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02cc105e30 0x7f02cc19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d0c08640 1 -- 172.21.15.82:0/47230490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02cc19d720 con 0x7f02cc101b60 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.515+0000 7f02d1c0a640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02cc105e30 0x7f02cc19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.516+0000 7f02d0c08640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02cc101b60 0x7f02cc198c70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009230 tx=0x7f02bc004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.516+0000 7f02c27fc640 1 -- 172.21.15.82:0/47230490 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc043020 con 0x7f02cc101b60 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.516+0000 7f02c27fc640 1 -- 172.21.15.82:0/47230490 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02bc004350 con 0x7f02cc101b60 2024-04-21T08:47:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.516+0000 7f02d366d640 1 -- 172.21.15.82:0/47230490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02cc19d9b0 con 0x7f02cc101b60 2024-04-21T08:47:06.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.516+0000 7f02c27fc640 1 -- 172.21.15.82:0/47230490 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc003b50 con 0x7f02cc101b60 2024-04-21T08:47:06.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.516+0000 7f02d366d640 1 -- 172.21.15.82:0/47230490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02cc1a52a0 con 0x7f02cc101b60 2024-04-21T08:47:06.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.517+0000 7f02d366d640 1 -- 172.21.15.82:0/47230490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02cc068de0 con 0x7f02cc101b60 2024-04-21T08:47:06.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.519+0000 7f02c27fc640 1 -- 172.21.15.82:0/47230490 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f02bc046020 con 0x7f02cc101b60 2024-04-21T08:47:06.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.519+0000 7f02c27fc640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f02b80a3e40 0x7f02b80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:06.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.519+0000 7f02d1409640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f02b80a3e40 0x7f02b80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:06.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.519+0000 7f02c27fc640 1 -- 172.21.15.82:0/47230490 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f02bc0f33e0 con 0x7f02cc101b60 2024-04-21T08:47:06.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.520+0000 7f02d1409640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f02b80a3e40 0x7f02b80a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f02c8005fd0 tx=0x7f02c8005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:06.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.523+0000 7f02c27fc640 1 -- 172.21.15.82:0/47230490 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02bc04d020 con 0x7f02cc101b60 2024-04-21T08:47:06.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.706+0000 7f02d366d640 1 -- 172.21.15.82:0/47230490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02cc10a360 con 0x7f02cc101b60 2024-04-21T08:47:06.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.708+0000 7f02c27fc640 1 -- 172.21.15.82:0/47230490 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25247 (secure 0 0 0) 0x7f02bc0bf6a0 con 0x7f02cc101b60 2024-04-21T08:47:06.708 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:06.708 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:47:00.617381+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2119539425":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6800/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/539366680":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/2275507980":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2883183315":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:47:06.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.710+0000 7f02d366d640 1 -- 172.21.15.82:0/47230490 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f02b80a3e40 msgr2=0x7f02b80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.710+0000 7f02d366d640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f02b80a3e40 0x7f02b80a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f02c8005fd0 tx=0x7f02c8005850 comp rx=0 tx=0).stop 2024-04-21T08:47:06.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.710+0000 7f02d366d640 1 -- 172.21.15.82:0/47230490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02cc101b60 msgr2=0x7f02cc198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.710+0000 7f02d366d640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02cc101b60 0x7f02cc198c70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009230 tx=0x7f02bc004bb0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.710+0000 7f02d366d640 1 -- 172.21.15.82:0/47230490 shutdown_connections 2024-04-21T08:47:06.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.710+0000 7f02d366d640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02cc105e30 0x7f02cc19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.710+0000 7f02d366d640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f02b80a3e40 0x7f02b80a6300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.710+0000 7f02d366d640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02cc101b60 0x7f02cc198c70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.710+0000 7f02d366d640 1 --2- 172.21.15.82:0/47230490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02cc1011a0 0x7f02cc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.710+0000 7f02d366d640 1 -- 172.21.15.82:0/47230490 >> 172.21.15.82:0/47230490 conn(0x7f02cc077e30 msgr2=0x7f02cc0693a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:06.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.710+0000 7f02d366d640 1 -- 172.21.15.82:0/47230490 shutdown_connections 2024-04-21T08:47:06.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.710+0000 7f02d366d640 1 -- 172.21.15.82:0/47230490 wait complete. 2024-04-21T08:47:06.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:47:06.721 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:47:06.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.823+0000 7f48d2a25640 1 -- 172.21.15.82:0/2942287672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48cc102b40 msgr2=0x7f48cc106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.823+0000 7f48d2a25640 1 --2- 172.21.15.82:0/2942287672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48cc102b40 0x7f48cc106b80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009900 tx=0x7f48c002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:06.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.823+0000 7f48d2a25640 1 -- 172.21.15.82:0/2942287672 shutdown_connections 2024-04-21T08:47:06.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.823+0000 7f48d2a25640 1 --2- 172.21.15.82:0/2942287672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48cc107260 0x7f48cc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.823+0000 7f48d2a25640 1 --2- 172.21.15.82:0/2942287672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48cc102b40 0x7f48cc106b80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.823+0000 7f48d2a25640 1 --2- 172.21.15.82:0/2942287672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc102190 0x7f48cc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.823+0000 7f48d2a25640 1 -- 172.21.15.82:0/2942287672 >> 172.21.15.82:0/2942287672 conn(0x7f48cc0793a0 msgr2=0x7f48cc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.823+0000 7f48d2a25640 1 -- 172.21.15.82:0/2942287672 shutdown_connections 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.823+0000 7f48d2a25640 1 -- 172.21.15.82:0/2942287672 wait complete. 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.824+0000 7f48d2a25640 1 Processor -- start 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.824+0000 7f48d2a25640 1 -- start start 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.824+0000 7f48d2a25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc102190 0x7f48cc074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.824+0000 7f48d2a25640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48cc107260 0x7f48cc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.824+0000 7f48d2a25640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48cc074cd0 0x7f48cc1a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.824+0000 7f48d2a25640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc112200 con 0x7f48cc074cd0 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.824+0000 7f48d2a25640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc112080 con 0x7f48cc102190 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.824+0000 7f48d2a25640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc112380 con 0x7f48cc107260 2024-04-21T08:47:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.824+0000 7f48d0fc2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48cc074cd0 0x7f48cc1a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.824+0000 7f48d0fc2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48cc074cd0 0x7f48cc1a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44444/0 (socket says 172.21.15.82:44444) 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.824+0000 7f48d0fc2640 1 -- 172.21.15.82:0/2271939800 learned_addr learned my addr 172.21.15.82:0/2271939800 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.825+0000 7f48cbfff640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc102190 0x7f48cc074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.825+0000 7f48d0fc2640 1 -- 172.21.15.82:0/2271939800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48cc107260 msgr2=0x7f48cc074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.825+0000 7f48d0fc2640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48cc107260 0x7f48cc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.825+0000 7f48d0fc2640 1 -- 172.21.15.82:0/2271939800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc102190 msgr2=0x7f48cc074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.825+0000 7f48d0fc2640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc102190 0x7f48cc074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.825+0000 7f48d0fc2640 1 -- 172.21.15.82:0/2271939800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc1a7010 con 0x7f48cc074cd0 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.825+0000 7f48d0fc2640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48cc074cd0 0x7f48cc1a6910 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f48c400b970 tx=0x7f48c400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.825+0000 7f48c97fa640 1 -- 172.21.15.82:0/2271939800 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4019070 con 0x7f48cc074cd0 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.825+0000 7f48c97fa640 1 -- 172.21.15.82:0/2271939800 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48c4014440 con 0x7f48cc074cd0 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.825+0000 7f48d2a25640 1 -- 172.21.15.82:0/2271939800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc1a7300 con 0x7f48cc074cd0 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.825+0000 7f48c97fa640 1 -- 172.21.15.82:0/2271939800 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40075d0 con 0x7f48cc074cd0 2024-04-21T08:47:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.825+0000 7f48d2a25640 1 -- 172.21.15.82:0/2271939800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc1a7850 con 0x7f48cc074cd0 2024-04-21T08:47:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.826+0000 7f48d2a25640 1 -- 172.21.15.82:0/2271939800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48cc0691c0 con 0x7f48cc074cd0 2024-04-21T08:47:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.827+0000 7f48c97fa640 1 -- 172.21.15.82:0/2271939800 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f48c401c070 con 0x7f48cc074cd0 2024-04-21T08:47:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.828+0000 7f48c97fa640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f48b40a3e70 0x7f48b40a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.828+0000 7f48cbfff640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f48b40a3e70 0x7f48b40a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.828+0000 7f48c97fa640 1 -- 172.21.15.82:0/2271939800 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f48c40d0330 con 0x7f48cc074cd0 2024-04-21T08:47:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.828+0000 7f48cbfff640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f48b40a3e70 0x7f48b40a6330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00ba40 tx=0x7f48bc005cc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:06.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:06.833+0000 7f48c97fa640 1 -- 172.21.15.82:0/2271939800 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48c409c4d0 con 0x7f48cc074cd0 2024-04-21T08:47:06.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:06 smithi134 ceph-mon[31772]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:06.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:06 smithi134 ceph-mon[31772]: from='client.15702 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:06.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:06 smithi134 ceph-mon[31772]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:07.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.008+0000 7f48d2a25640 1 -- 172.21.15.82:0/2271939800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f48cc102b40 con 0x7f48cc074cd0 2024-04-21T08:47:07.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.009+0000 7f48c97fa640 1 -- 172.21.15.82:0/2271939800 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f48c409bdd0 con 0x7f48cc074cd0 2024-04-21T08:47:07.008 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:47:07.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.010+0000 7f48d2a25640 1 -- 172.21.15.82:0/2271939800 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f48b40a3e70 msgr2=0x7f48b40a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.010+0000 7f48d2a25640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f48b40a3e70 0x7f48b40a6330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00ba40 tx=0x7f48bc005cc0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.010+0000 7f48d2a25640 1 -- 172.21.15.82:0/2271939800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48cc074cd0 msgr2=0x7f48cc1a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.010+0000 7f48d2a25640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48cc074cd0 0x7f48cc1a6910 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f48c400b970 tx=0x7f48c400be30 comp rx=0 tx=0).stop 2024-04-21T08:47:07.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.011+0000 7f48d2a25640 1 -- 172.21.15.82:0/2271939800 shutdown_connections 2024-04-21T08:47:07.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.011+0000 7f48d2a25640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f48b40a3e70 0x7f48b40a6330 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.011+0000 7f48d2a25640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48cc074cd0 0x7f48cc1a6910 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.011+0000 7f48d2a25640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48cc107260 0x7f48cc074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.011+0000 7f48d2a25640 1 --2- 172.21.15.82:0/2271939800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc102190 0x7f48cc074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.011+0000 7f48d2a25640 1 -- 172.21.15.82:0/2271939800 >> 172.21.15.82:0/2271939800 conn(0x7f48cc0793a0 msgr2=0x7f48cc0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:07.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.011+0000 7f48d2a25640 1 -- 172.21.15.82:0/2271939800 shutdown_connections 2024-04-21T08:47:07.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.011+0000 7f48d2a25640 1 -- 172.21.15.82:0/2271939800 wait complete. 2024-04-21T08:47:07.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:06 smithi082 ceph-mon[27967]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:07.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:06 smithi082 ceph-mon[27967]: from='client.15702 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:07.020 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:06 smithi082 ceph-mon[27967]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:07.021 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:47:07.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.123+0000 7f1809106640 1 -- 172.21.15.82:0/1062135398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18041072b0 msgr2=0x7f180410eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.123+0000 7f1809106640 1 --2- 172.21.15.82:0/1062135398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18041072b0 0x7f180410eec0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009ca0 tx=0x7f17fc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.123+0000 7f1809106640 1 -- 172.21.15.82:0/1062135398 shutdown_connections 2024-04-21T08:47:07.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.123+0000 7f1809106640 1 --2- 172.21.15.82:0/1062135398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18041072b0 0x7f180410eec0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.123+0000 7f1809106640 1 --2- 172.21.15.82:0/1062135398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18041014a0 0x7f1804106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.123+0000 7f1809106640 1 --2- 172.21.15.82:0/1062135398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1804100af0 0x7f1804100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.123+0000 7f1809106640 1 -- 172.21.15.82:0/1062135398 >> 172.21.15.82:0/1062135398 conn(0x7f18040760e0 msgr2=0x7f18040764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:07.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.123+0000 7f1809106640 1 -- 172.21.15.82:0/1062135398 shutdown_connections 2024-04-21T08:47:07.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.123+0000 7f1809106640 1 -- 172.21.15.82:0/1062135398 wait complete. 2024-04-21T08:47:07.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.123+0000 7f1809106640 1 Processor -- start 2024-04-21T08:47:07.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1809106640 1 -- start start 2024-04-21T08:47:07.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1809106640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1804100af0 0x7f1804198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:07.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1809106640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18041072b0 0x7f1804198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:07.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1809106640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f180419d000 0x7f180419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:07.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1809106640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1804112250 con 0x7f1804100af0 2024-04-21T08:47:07.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1809106640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18041120d0 con 0x7f180419d000 2024-04-21T08:47:07.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1809106640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18041123d0 con 0x7f18041072b0 2024-04-21T08:47:07.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1802d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1804100af0 0x7f1804198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:07.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1802d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1804100af0 0x7f1804198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44458/0 (socket says 172.21.15.82:44458) 2024-04-21T08:47:07.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1802d9d640 1 -- 172.21.15.82:0/138483993 learned_addr learned my addr 172.21.15.82:0/138483993 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:07.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f180359e640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f180419d000 0x7f180419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:07.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f180259c640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18041072b0 0x7f1804198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:07.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1802d9d640 1 -- 172.21.15.82:0/138483993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18041072b0 msgr2=0x7f1804198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1802d9d640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18041072b0 0x7f1804198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1802d9d640 1 -- 172.21.15.82:0/138483993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f180419d000 msgr2=0x7f180419d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1802d9d640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f180419d000 0x7f180419d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1802d9d640 1 -- 172.21.15.82:0/138483993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180419daf0 con 0x7f1804100af0 2024-04-21T08:47:07.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f180359e640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f180419d000 0x7f180419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:07.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f180259c640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18041072b0 0x7f1804198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:07.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.124+0000 7f1802d9d640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1804100af0 0x7f1804198790 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f17f800d770 tx=0x7f17f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:07.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.125+0000 7f17ebfff640 1 -- 172.21.15.82:0/138483993 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8017070 con 0x7f1804100af0 2024-04-21T08:47:07.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.125+0000 7f1809106640 1 -- 172.21.15.82:0/138483993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180419dde0 con 0x7f1804100af0 2024-04-21T08:47:07.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.125+0000 7f17ebfff640 1 -- 172.21.15.82:0/138483993 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f80054d0 con 0x7f1804100af0 2024-04-21T08:47:07.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.125+0000 7f17ebfff640 1 -- 172.21.15.82:0/138483993 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8004c30 con 0x7f1804100af0 2024-04-21T08:47:07.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.125+0000 7f1809106640 1 -- 172.21.15.82:0/138483993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18041a55d0 con 0x7f1804100af0 2024-04-21T08:47:07.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.126+0000 7f1809106640 1 -- 172.21.15.82:0/138483993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18040691c0 con 0x7f1804100af0 2024-04-21T08:47:07.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:06 smithi106 ceph-mon[31072]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:07.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:06 smithi106 ceph-mon[31072]: from='client.15702 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:07.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:06 smithi106 ceph-mon[31072]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:07.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.127+0000 7f17ebfff640 1 -- 172.21.15.82:0/138483993 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f17f8004e10 con 0x7f1804100af0 2024-04-21T08:47:07.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.127+0000 7f17ebfff640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f17ec0a3e40 0x7f17ec0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:07.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.127+0000 7f180259c640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f17ec0a3e40 0x7f17ec0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:07.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.127+0000 7f17ebfff640 1 -- 172.21.15.82:0/138483993 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f17f80d09f0 con 0x7f1804100af0 2024-04-21T08:47:07.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.128+0000 7f180259c640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f17ec0a3e40 0x7f17ec0a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f18041086e0 tx=0x7f17f4005d20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:07.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.132+0000 7f17ebfff640 1 -- 172.21.15.82:0/138483993 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17f809ca80 con 0x7f1804100af0 2024-04-21T08:47:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.309+0000 7f1809106640 1 -- 172.21.15.82:0/138483993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f180410a580 con 0x7f1804100af0 2024-04-21T08:47:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.310+0000 7f17ebfff640 1 -- 172.21.15.82:0/138483993 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f17f809c380 con 0x7f1804100af0 2024-04-21T08:47:07.309 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:47:07.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.311+0000 7f1809106640 1 -- 172.21.15.82:0/138483993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f17ec0a3e40 msgr2=0x7f17ec0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.311+0000 7f1809106640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f17ec0a3e40 0x7f17ec0a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f18041086e0 tx=0x7f17f4005d20 comp rx=0 tx=0).stop 2024-04-21T08:47:07.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.311+0000 7f1809106640 1 -- 172.21.15.82:0/138483993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1804100af0 msgr2=0x7f1804198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.311+0000 7f1809106640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1804100af0 0x7f1804198790 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f17f800d770 tx=0x7f17f800dc30 comp rx=0 tx=0).stop 2024-04-21T08:47:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.312+0000 7f1809106640 1 -- 172.21.15.82:0/138483993 shutdown_connections 2024-04-21T08:47:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.312+0000 7f1809106640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f180419d000 0x7f180419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.312+0000 7f1809106640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18041072b0 0x7f1804198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.312+0000 7f1809106640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f17ec0a3e40 0x7f17ec0a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.312+0000 7f1809106640 1 --2- 172.21.15.82:0/138483993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1804100af0 0x7f1804198790 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.312+0000 7f1809106640 1 -- 172.21.15.82:0/138483993 >> 172.21.15.82:0/138483993 conn(0x7f18040760e0 msgr2=0x7f18040fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.312+0000 7f1809106640 1 -- 172.21.15.82:0/138483993 shutdown_connections 2024-04-21T08:47:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.312+0000 7f1809106640 1 -- 172.21.15.82:0/138483993 wait complete. 2024-04-21T08:47:07.323 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:47:07.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.424+0000 7fa746eba640 1 -- 172.21.15.82:0/619250902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa74010e7e0 msgr2=0x7fa740110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.424+0000 7fa746eba640 1 --2- 172.21.15.82:0/619250902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa74010e7e0 0x7fa740110bd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa738009ca0 tx=0x7fa73802c180 comp rx=0 tx=0).stop 2024-04-21T08:47:07.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.424+0000 7fa746eba640 1 -- 172.21.15.82:0/619250902 shutdown_connections 2024-04-21T08:47:07.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.424+0000 7fa746eba640 1 --2- 172.21.15.82:0/619250902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa74010e7e0 0x7fa740110bd0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.424+0000 7fa746eba640 1 --2- 172.21.15.82:0/619250902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa740074c20 0x7fa74010e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.424+0000 7fa746eba640 1 --2- 172.21.15.82:0/619250902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa740074300 0x7fa7400746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.424+0000 7fa746eba640 1 -- 172.21.15.82:0/619250902 >> 172.21.15.82:0/619250902 conn(0x7fa7400fd990 msgr2=0x7fa7400ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:07.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.424+0000 7fa746eba640 1 -- 172.21.15.82:0/619250902 shutdown_connections 2024-04-21T08:47:07.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.424+0000 7fa746eba640 1 -- 172.21.15.82:0/619250902 wait complete. 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.424+0000 7fa746eba640 1 Processor -- start 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.424+0000 7fa746eba640 1 -- start start 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa746eba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa740074300 0x7fa7401a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa746eba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa740074c20 0x7fa7401a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa746eba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa74010e7e0 0x7fa74019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa746eba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa740114390 con 0x7fa74010e7e0 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa746eba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa740114210 con 0x7fa740074300 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa746eba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa740114510 con 0x7fa740074c20 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa745457640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa74010e7e0 0x7fa74019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa745457640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa74010e7e0 0x7fa74019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44464/0 (socket says 172.21.15.82:44464) 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa745457640 1 -- 172.21.15.82:0/3751225384 learned_addr learned my addr 172.21.15.82:0/3751225384 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa73ffff640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa740074c20 0x7fa7401a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa744c56640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa740074300 0x7fa7401a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:07.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa745457640 1 -- 172.21.15.82:0/3751225384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa740074c20 msgr2=0x7fa7401a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa745457640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa740074c20 0x7fa7401a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa745457640 1 -- 172.21.15.82:0/3751225384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa740074300 msgr2=0x7fa7401a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa745457640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa740074300 0x7fa7401a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa745457640 1 -- 172.21.15.82:0/3751225384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74019afd0 con 0x7fa74010e7e0 2024-04-21T08:47:07.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa73ffff640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa740074c20 0x7fa7401a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:07.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa745457640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa74010e7e0 0x7fa74019a710 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa73802c790 tx=0x7fa738003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:07.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa73dffb640 1 -- 172.21.15.82:0/3751225384 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738039040 con 0x7fa74010e7e0 2024-04-21T08:47:07.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa73dffb640 1 -- 172.21.15.82:0/3751225384 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa738044070 con 0x7fa74010e7e0 2024-04-21T08:47:07.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa746eba640 1 -- 172.21.15.82:0/3751225384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74019b260 con 0x7fa74010e7e0 2024-04-21T08:47:07.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.425+0000 7fa73dffb640 1 -- 172.21.15.82:0/3751225384 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738032850 con 0x7fa74010e7e0 2024-04-21T08:47:07.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.426+0000 7fa746eba640 1 -- 172.21.15.82:0/3751225384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7401a73f0 con 0x7fa74010e7e0 2024-04-21T08:47:07.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.427+0000 7fa746eba640 1 -- 172.21.15.82:0/3751225384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7400759b0 con 0x7fa74010e7e0 2024-04-21T08:47:07.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.428+0000 7fa73dffb640 1 -- 172.21.15.82:0/3751225384 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa738004240 con 0x7fa74010e7e0 2024-04-21T08:47:07.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.428+0000 7fa73dffb640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa7300a3e40 0x7fa7300a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:07.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.428+0000 7fa744c56640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa7300a3e40 0x7fa7300a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:07.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.428+0000 7fa73dffb640 1 -- 172.21.15.82:0/3751225384 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa7380f3330 con 0x7fa74010e7e0 2024-04-21T08:47:07.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.429+0000 7fa744c56640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa7300a3e40 0x7fa7300a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa734005e10 tx=0x7fa734005da0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:07.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.433+0000 7fa73dffb640 1 -- 172.21.15.82:0/3751225384 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7380bf4d0 con 0x7fa74010e7e0 2024-04-21T08:47:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.612+0000 7fa746eba640 1 -- 172.21.15.82:0/3751225384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa740109990 con 0x7fa74010e7e0 2024-04-21T08:47:07.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.612+0000 7fa73dffb640 1 -- 172.21.15.82:0/3751225384 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fa7380bedd0 con 0x7fa74010e7e0 2024-04-21T08:47:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:47:07.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.614+0000 7fa746eba640 1 -- 172.21.15.82:0/3751225384 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa7300a3e40 msgr2=0x7fa7300a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.614+0000 7fa746eba640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa7300a3e40 0x7fa7300a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa734005e10 tx=0x7fa734005da0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.614+0000 7fa746eba640 1 -- 172.21.15.82:0/3751225384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa74010e7e0 msgr2=0x7fa74019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.614+0000 7fa746eba640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa74010e7e0 0x7fa74019a710 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa73802c790 tx=0x7fa738003c20 comp rx=0 tx=0).stop 2024-04-21T08:47:07.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.614+0000 7fa746eba640 1 -- 172.21.15.82:0/3751225384 shutdown_connections 2024-04-21T08:47:07.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.614+0000 7fa746eba640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa7300a3e40 0x7fa7300a6300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.614+0000 7fa746eba640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa74010e7e0 0x7fa74019a710 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.614+0000 7fa746eba640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa740074c20 0x7fa7401a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.614+0000 7fa746eba640 1 --2- 172.21.15.82:0/3751225384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa740074300 0x7fa7401a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.614+0000 7fa746eba640 1 -- 172.21.15.82:0/3751225384 >> 172.21.15.82:0/3751225384 conn(0x7fa7400fd990 msgr2=0x7fa7400fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.614+0000 7fa746eba640 1 -- 172.21.15.82:0/3751225384 shutdown_connections 2024-04-21T08:47:07.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.614+0000 7fa746eba640 1 -- 172.21.15.82:0/3751225384 wait complete. 2024-04-21T08:47:07.625 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:47:07.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.728+0000 7f54ed670640 1 -- 172.21.15.82:0/639802167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54e81031d0 msgr2=0x7f54e81035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.728+0000 7f54ed670640 1 --2- 172.21.15.82:0/639802167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54e81031d0 0x7f54e81035b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009ca0 tx=0x7f54d802c230 comp rx=0 tx=0).stop 2024-04-21T08:47:07.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.729+0000 7f54ed670640 1 -- 172.21.15.82:0/639802167 shutdown_connections 2024-04-21T08:47:07.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.729+0000 7f54ed670640 1 --2- 172.21.15.82:0/639802167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e810c510 0x7f54e810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.729+0000 7f54ed670640 1 --2- 172.21.15.82:0/639802167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e8103af0 0x7f54e810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.729+0000 7f54ed670640 1 --2- 172.21.15.82:0/639802167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54e81031d0 0x7f54e81035b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.729+0000 7f54ed670640 1 -- 172.21.15.82:0/639802167 >> 172.21.15.82:0/639802167 conn(0x7f54e8077f50 msgr2=0x7f54e8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:07.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.729+0000 7f54ed670640 1 -- 172.21.15.82:0/639802167 shutdown_connections 2024-04-21T08:47:07.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.729+0000 7f54ed670640 1 -- 172.21.15.82:0/639802167 wait complete. 2024-04-21T08:47:07.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.729+0000 7f54ed670640 1 Processor -- start 2024-04-21T08:47:07.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.729+0000 7f54ed670640 1 -- start start 2024-04-21T08:47:07.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.729+0000 7f54ed670640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54e81031d0 0x7f54e8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:07.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54ed670640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e8103af0 0x7f54e8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:07.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54ed670640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e810c510 0x7f54e819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:07.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54ed670640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81121a0 con 0x7f54e81031d0 2024-04-21T08:47:07.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54ed670640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8112020 con 0x7f54e8103af0 2024-04-21T08:47:07.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54ed670640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8112320 con 0x7f54e810c510 2024-04-21T08:47:07.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54e81031d0 0x7f54e8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:07.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54e81031d0 0x7f54e8198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44480/0 (socket says 172.21.15.82:44480) 2024-04-21T08:47:07.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54e6ffd640 1 -- 172.21.15.82:0/484165039 learned_addr learned my addr 172.21.15.82:0/484165039 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:07.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54e67fc640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e8103af0 0x7f54e8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:07.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54e6ffd640 1 -- 172.21.15.82:0/484165039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e810c510 msgr2=0x7f54e819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54e6ffd640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e810c510 0x7f54e819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54e6ffd640 1 -- 172.21.15.82:0/484165039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e8103af0 msgr2=0x7f54e8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54e6ffd640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e8103af0 0x7f54e8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54e6ffd640 1 -- 172.21.15.82:0/484165039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e819d730 con 0x7f54e81031d0 2024-04-21T08:47:07.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54e67fc640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e8103af0 0x7f54e8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:07.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54e6ffd640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54e81031d0 0x7f54e8198740 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f54d8005e30 tx=0x7f54d8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:07.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54cffff640 1 -- 172.21.15.82:0/484165039 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8039040 con 0x7f54e81031d0 2024-04-21T08:47:07.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54ed670640 1 -- 172.21.15.82:0/484165039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e819d9c0 con 0x7f54e81031d0 2024-04-21T08:47:07.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54cffff640 1 -- 172.21.15.82:0/484165039 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d8044070 con 0x7f54e81031d0 2024-04-21T08:47:07.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54cffff640 1 -- 172.21.15.82:0/484165039 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d80329a0 con 0x7f54e81031d0 2024-04-21T08:47:07.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.730+0000 7f54ed670640 1 -- 172.21.15.82:0/484165039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e81a52b0 con 0x7f54e81031d0 2024-04-21T08:47:07.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.731+0000 7f54ed670640 1 -- 172.21.15.82:0/484165039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e8068d70 con 0x7f54e81031d0 2024-04-21T08:47:07.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.732+0000 7f54cffff640 1 -- 172.21.15.82:0/484165039 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f54d8032b40 con 0x7f54e81031d0 2024-04-21T08:47:07.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.733+0000 7f54cffff640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f54d00a3e40 0x7f54d00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:07.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.733+0000 7f54e67fc640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f54d00a3e40 0x7f54d00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:07.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.733+0000 7f54cffff640 1 -- 172.21.15.82:0/484165039 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f54d80f3b50 con 0x7f54e81031d0 2024-04-21T08:47:07.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.733+0000 7f54e67fc640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f54d00a3e40 0x7f54d00a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f54e810e450 tx=0x7f54dc005d30 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:07.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.736+0000 7f54cffff640 1 -- 172.21.15.82:0/484165039 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54d804e020 con 0x7f54e81031d0 2024-04-21T08:47:07.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.914+0000 7f54ed670640 1 -- 172.21.15.82:0/484165039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f54e8199a90 con 0x7f54e81031d0 2024-04-21T08:47:07.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.914+0000 7f54cffff640 1 -- 172.21.15.82:0/484165039 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f54d80bfcf0 con 0x7f54e81031d0 2024-04-21T08:47:07.914 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:47:07.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.916+0000 7f54ed670640 1 -- 172.21.15.82:0/484165039 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f54d00a3e40 msgr2=0x7f54d00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.916+0000 7f54ed670640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f54d00a3e40 0x7f54d00a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f54e810e450 tx=0x7f54dc005d30 comp rx=0 tx=0).stop 2024-04-21T08:47:07.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.916+0000 7f54ed670640 1 -- 172.21.15.82:0/484165039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54e81031d0 msgr2=0x7f54e8198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:07.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.916+0000 7f54ed670640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54e81031d0 0x7f54e8198740 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f54d8005e30 tx=0x7f54d8003c20 comp rx=0 tx=0).stop 2024-04-21T08:47:07.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.916+0000 7f54ed670640 1 -- 172.21.15.82:0/484165039 shutdown_connections 2024-04-21T08:47:07.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.916+0000 7f54ed670640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e810c510 0x7f54e819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.916+0000 7f54ed670640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54e8103af0 0x7f54e8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.916+0000 7f54ed670640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f54d00a3e40 0x7f54d00a6300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.916+0000 7f54ed670640 1 --2- 172.21.15.82:0/484165039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54e81031d0 0x7f54e8198740 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:07.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.916+0000 7f54ed670640 1 -- 172.21.15.82:0/484165039 >> 172.21.15.82:0/484165039 conn(0x7f54e8077f50 msgr2=0x7f54e80fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:07.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.917+0000 7f54ed670640 1 -- 172.21.15.82:0/484165039 shutdown_connections 2024-04-21T08:47:07.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:07.917+0000 7f54ed670640 1 -- 172.21.15.82:0/484165039 wait complete. 2024-04-21T08:47:07.926 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:07 smithi082 ceph-mon[27967]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:47:07.926 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:07 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/47230490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:07.926 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:07 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2271939800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:07.926 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:07 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/138483993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:07.926 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:07 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3751225384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:07.927 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:47:07.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:07 smithi134 ceph-mon[31772]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:47:07.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:07 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/47230490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:07.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:07 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2271939800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:07.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:07 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/138483993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:07.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:07 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3751225384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:08.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.026+0000 7fb27ac21640 1 -- 172.21.15.82:0/1007864107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2741011a0 msgr2=0x7fb274101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.026+0000 7fb27ac21640 1 --2- 172.21.15.82:0/1007864107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2741011a0 0x7fb274101620 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb268009930 tx=0x7fb26802bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.026+0000 7fb27ac21640 1 -- 172.21.15.82:0/1007864107 shutdown_connections 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.026+0000 7fb27ac21640 1 --2- 172.21.15.82:0/1007864107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb274101b60 0x7fb27410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.026+0000 7fb27ac21640 1 --2- 172.21.15.82:0/1007864107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2741011a0 0x7fb274101620 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.026+0000 7fb27ac21640 1 --2- 172.21.15.82:0/1007864107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb274105e30 0x7fb274106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.026+0000 7fb27ac21640 1 -- 172.21.15.82:0/1007864107 >> 172.21.15.82:0/1007864107 conn(0x7fb274077e30 msgr2=0x7fb274078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.026+0000 7fb27ac21640 1 -- 172.21.15.82:0/1007864107 shutdown_connections 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.026+0000 7fb27ac21640 1 -- 172.21.15.82:0/1007864107 wait complete. 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.027+0000 7fb27ac21640 1 Processor -- start 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.027+0000 7fb27ac21640 1 -- start start 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.027+0000 7fb27ac21640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2741011a0 0x7fb2741975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.027+0000 7fb27ac21640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb274101b60 0x7fb274197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.027+0000 7fb27ac21640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb274105e30 0x7fb27419bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.027+0000 7fb27ac21640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb2741110c0 con 0x7fb274101b60 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.027+0000 7fb27ac21640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb274110f40 con 0x7fb2741011a0 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.027+0000 7fb27ac21640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb274111240 con 0x7fb274105e30 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.027+0000 7fb273fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb274101b60 0x7fb274197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.027+0000 7fb273fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb274101b60 0x7fb274197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44494/0 (socket says 172.21.15.82:44494) 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.027+0000 7fb273fff640 1 -- 172.21.15.82:0/19987036 learned_addr learned my addr 172.21.15.82:0/19987036 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.027+0000 7fb2789bd640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2741011a0 0x7fb2741975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.028+0000 7fb273fff640 1 -- 172.21.15.82:0/19987036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb274105e30 msgr2=0x7fb27419bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.028+0000 7fb273fff640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb274105e30 0x7fb27419bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.028+0000 7fb273fff640 1 -- 172.21.15.82:0/19987036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2741011a0 msgr2=0x7fb2741975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.028+0000 7fb273fff640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2741011a0 0x7fb2741975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.028+0000 7fb273fff640 1 -- 172.21.15.82:0/19987036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27419c5d0 con 0x7fb274101b60 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.028+0000 7fb273fff640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb274101b60 0x7fb274197b20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb268009230 tx=0x7fb26802c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.028+0000 7fb271ffb640 1 -- 172.21.15.82:0/19987036 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268043020 con 0x7fb274101b60 2024-04-21T08:47:08.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.028+0000 7fb271ffb640 1 -- 172.21.15.82:0/19987036 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2680043d0 con 0x7fb274101b60 2024-04-21T08:47:08.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.028+0000 7fb27ac21640 1 -- 172.21.15.82:0/19987036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27419c860 con 0x7fb274101b60 2024-04-21T08:47:08.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.028+0000 7fb271ffb640 1 -- 172.21.15.82:0/19987036 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268003bc0 con 0x7fb274101b60 2024-04-21T08:47:08.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.028+0000 7fb27ac21640 1 -- 172.21.15.82:0/19987036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2741a4150 con 0x7fb274101b60 2024-04-21T08:47:08.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.029+0000 7fb27ac21640 1 -- 172.21.15.82:0/19987036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb274068de0 con 0x7fb274101b60 2024-04-21T08:47:08.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.031+0000 7fb271ffb640 1 -- 172.21.15.82:0/19987036 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fb268046020 con 0x7fb274101b60 2024-04-21T08:47:08.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.031+0000 7fb271ffb640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb25c0a3e40 0x7fb25c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.032+0000 7fb2789bd640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb25c0a3e40 0x7fb25c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.032+0000 7fb271ffb640 1 -- 172.21.15.82:0/19987036 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fb2680f3410 con 0x7fb274101b60 2024-04-21T08:47:08.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.032+0000 7fb2789bd640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb25c0a3e40 0x7fb25c0a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb264005fd0 tx=0x7fb264005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.035+0000 7fb271ffb640 1 -- 172.21.15.82:0/19987036 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb26804d020 con 0x7fb274101b60 2024-04-21T08:47:08.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:07 smithi106 ceph-mon[31072]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:47:08.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:07 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/47230490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:08.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:07 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2271939800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:08.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:07 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/138483993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:08.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:07 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3751225384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:08.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.253+0000 7fb27ac21640 1 -- 172.21.15.82:0/19987036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb274198930 con 0x7fb274101b60 2024-04-21T08:47:08.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.254+0000 7fb271ffb640 1 -- 172.21.15.82:0/19987036 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7fb26803e450 con 0x7fb274101b60 2024-04-21T08:47:08.254 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:08.254 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:46:59.334314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:47:08.255 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 28 2024-04-21T08:47:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.256+0000 7fb27ac21640 1 -- 172.21.15.82:0/19987036 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb25c0a3e40 msgr2=0x7fb25c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.256+0000 7fb27ac21640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb25c0a3e40 0x7fb25c0a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb264005fd0 tx=0x7fb264005850 comp rx=0 tx=0).stop 2024-04-21T08:47:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.256+0000 7fb27ac21640 1 -- 172.21.15.82:0/19987036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb274101b60 msgr2=0x7fb274197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.256+0000 7fb27ac21640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb274101b60 0x7fb274197b20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb268009230 tx=0x7fb26802c710 comp rx=0 tx=0).stop 2024-04-21T08:47:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.256+0000 7fb27ac21640 1 -- 172.21.15.82:0/19987036 shutdown_connections 2024-04-21T08:47:08.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.256+0000 7fb27ac21640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb274105e30 0x7fb27419bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.256+0000 7fb27ac21640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb25c0a3e40 0x7fb25c0a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.256+0000 7fb27ac21640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb274101b60 0x7fb274197b20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.256+0000 7fb27ac21640 1 --2- 172.21.15.82:0/19987036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2741011a0 0x7fb2741975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.256+0000 7fb27ac21640 1 -- 172.21.15.82:0/19987036 >> 172.21.15.82:0/19987036 conn(0x7fb274077e30 msgr2=0x7fb2740fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.256+0000 7fb27ac21640 1 -- 172.21.15.82:0/19987036 shutdown_connections 2024-04-21T08:47:08.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.256+0000 7fb27ac21640 1 -- 172.21.15.82:0/19987036 wait complete. 2024-04-21T08:47:08.267 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:47:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.365+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/2200704995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d80690e0 msgr2=0x7ff6d8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.365+0000 7ff6dcc9a640 1 --2- 172.21.15.82:0/2200704995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d80690e0 0x7ff6d8069560 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009930 tx=0x7ff6cc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.366+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/2200704995 shutdown_connections 2024-04-21T08:47:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.366+0000 7ff6dcc9a640 1 --2- 172.21.15.82:0/2200704995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d8069aa0 0x7ff6d810edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.366+0000 7ff6dcc9a640 1 --2- 172.21.15.82:0/2200704995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d80690e0 0x7ff6d8069560 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.366+0000 7ff6dcc9a640 1 --2- 172.21.15.82:0/2200704995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d8108030 0x7ff6d8108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.366+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/2200704995 >> 172.21.15.82:0/2200704995 conn(0x7ff6d8077e30 msgr2=0x7ff6d8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.366+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/2200704995 shutdown_connections 2024-04-21T08:47:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.366+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/2200704995 wait complete. 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.366+0000 7ff6dcc9a640 1 Processor -- start 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.366+0000 7ff6dcc9a640 1 -- start start 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6dcc9a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d80690e0 0x7ff6d81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6dcc9a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d8069aa0 0x7ff6d8198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6dcc9a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d8108030 0x7ff6d819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6dcc9a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d8112140 con 0x7ff6d8108030 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6dcc9a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d8111fc0 con 0x7ff6d8069aa0 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6dcc9a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d81122c0 con 0x7ff6d80690e0 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d8108030 0x7ff6d819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d8108030 0x7ff6d819cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44510/0 (socket says 172.21.15.82:44510) 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d6d9d640 1 -- 172.21.15.82:0/3154898638 learned_addr learned my addr 172.21.15.82:0/3154898638 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d659c640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d80690e0 0x7ff6d81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d5d9b640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d8069aa0 0x7ff6d8198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d6d9d640 1 -- 172.21.15.82:0/3154898638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d80690e0 msgr2=0x7ff6d81986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d6d9d640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d80690e0 0x7ff6d81986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d6d9d640 1 -- 172.21.15.82:0/3154898638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d8069aa0 msgr2=0x7ff6d8198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d6d9d640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d8069aa0 0x7ff6d8198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d6d9d640 1 -- 172.21.15.82:0/3154898638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d819d6b0 con 0x7ff6d8108030 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d659c640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d80690e0 0x7ff6d81986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d5d9b640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d8069aa0 0x7ff6d8198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6d6d9d640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d8108030 0x7ff6d819cf80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000eaa0 tx=0x7ff6d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6c77fe640 1 -- 172.21.15.82:0/3154898638 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000cdb0 con 0x7ff6d8108030 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6c77fe640 1 -- 172.21.15.82:0/3154898638 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d0005020 con 0x7ff6d8108030 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/3154898638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d819d9a0 con 0x7ff6d8108030 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6c77fe640 1 -- 172.21.15.82:0/3154898638 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d00116d0 con 0x7ff6d8108030 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.367+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/3154898638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d81a5230 con 0x7ff6d8108030 2024-04-21T08:47:08.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.368+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/3154898638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d8068de0 con 0x7ff6d8108030 2024-04-21T08:47:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.369+0000 7ff6c77fe640 1 -- 172.21.15.82:0/3154898638 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff6d00051d0 con 0x7ff6d8108030 2024-04-21T08:47:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.370+0000 7ff6c77fe640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6c00a3ec0 0x7ff6c00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.370+0000 7ff6d659c640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6c00a3ec0 0x7ff6c00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.370+0000 7ff6c77fe640 1 -- 172.21.15.82:0/3154898638 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff6d00d10a0 con 0x7ff6d8108030 2024-04-21T08:47:08.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.370+0000 7ff6d659c640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6c00a3ec0 0x7ff6c00a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80058c0 tx=0x7ff6c8005ec0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.374+0000 7ff6c77fe640 1 -- 172.21.15.82:0/3154898638 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6d0016030 con 0x7ff6d8108030 2024-04-21T08:47:08.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.554+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/3154898638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6d8109e90 con 0x7ff6d8108030 2024-04-21T08:47:08.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.555+0000 7ff6c77fe640 1 -- 172.21.15.82:0/3154898638 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25247 (secure 0 0 0) 0x7ff6d009d130 con 0x7ff6d8108030 2024-04-21T08:47:08.555 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:08.555 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:47:00.617381+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2119539425":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6800/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/539366680":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/2275507980":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2883183315":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:47:08.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.558+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/3154898638 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6c00a3ec0 msgr2=0x7ff6c00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.558+0000 7ff6dcc9a640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6c00a3ec0 0x7ff6c00a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80058c0 tx=0x7ff6c8005ec0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.558+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/3154898638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d8108030 msgr2=0x7ff6d819cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.558+0000 7ff6dcc9a640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d8108030 0x7ff6d819cf80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000eaa0 tx=0x7ff6d000ef60 comp rx=0 tx=0).stop 2024-04-21T08:47:08.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.558+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/3154898638 shutdown_connections 2024-04-21T08:47:08.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.558+0000 7ff6dcc9a640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6c00a3ec0 0x7ff6c00a6380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.558+0000 7ff6dcc9a640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6d8108030 0x7ff6d819cf80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.558+0000 7ff6dcc9a640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6d8069aa0 0x7ff6d8198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.558+0000 7ff6dcc9a640 1 --2- 172.21.15.82:0/3154898638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6d80690e0 0x7ff6d81986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.558+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/3154898638 >> 172.21.15.82:0/3154898638 conn(0x7ff6d8077e30 msgr2=0x7ff6d80fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.558+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/3154898638 shutdown_connections 2024-04-21T08:47:08.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.558+0000 7ff6dcc9a640 1 -- 172.21.15.82:0/3154898638 wait complete. 2024-04-21T08:47:08.570 DEBUG:tasks.fwd_scrub:Yielding 2024-04-21T08:47:08.570 INFO:teuthology.run_tasks:Running task exec... 2024-04-21T08:47:08.580 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T08:47:08.580 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-04-21T08:47:08.580 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-21T08:47:08.581 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-21T08:47:08.581 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:47:08.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:47:08.581 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:47:08.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.685+0000 7f7229b4b640 1 -- 172.21.15.82:0/3386651274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72241010f0 msgr2=0x7f72241014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.685+0000 7f7229b4b640 1 --2- 172.21.15.82:0/3386651274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72241010f0 0x7f72241014d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7214009900 tx=0x7f721402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:08.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.685+0000 7f7229b4b640 1 -- 172.21.15.82:0/3386651274 shutdown_connections 2024-04-21T08:47:08.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.685+0000 7f7229b4b640 1 --2- 172.21.15.82:0/3386651274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7224106170 0x7f722410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.685+0000 7f7229b4b640 1 --2- 172.21.15.82:0/3386651274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7224101aa0 0x7f7224105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.685+0000 7f7229b4b640 1 --2- 172.21.15.82:0/3386651274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72241010f0 0x7f72241014d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.685+0000 7f7229b4b640 1 -- 172.21.15.82:0/3386651274 >> 172.21.15.82:0/3386651274 conn(0x7f7224077fd0 msgr2=0x7f72240783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.685+0000 7f7229b4b640 1 -- 172.21.15.82:0/3386651274 shutdown_connections 2024-04-21T08:47:08.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.685+0000 7f7229b4b640 1 -- 172.21.15.82:0/3386651274 wait complete. 2024-04-21T08:47:08.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.685+0000 7f7229b4b640 1 Processor -- start 2024-04-21T08:47:08.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7229b4b640 1 -- start start 2024-04-21T08:47:08.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7229b4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72241010f0 0x7f7224199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7229b4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7224106170 0x7f7224199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7229b4b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f722419a060 0x7f72241a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7229b4b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7224111130 con 0x7f722419a060 2024-04-21T08:47:08.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7229b4b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7224110fb0 con 0x7f7224106170 2024-04-21T08:47:08.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7229b4b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f72241112b0 con 0x7f72241010f0 2024-04-21T08:47:08.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7223fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f722419a060 0x7f72241a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7223fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f722419a060 0x7f72241a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44518/0 (socket says 172.21.15.82:44518) 2024-04-21T08:47:08.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7223fff640 1 -- 172.21.15.82:0/17495936 learned_addr learned my addr 172.21.15.82:0/17495936 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f72237fe640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72241010f0 0x7f7224199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7222ffd640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7224106170 0x7f7224199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7223fff640 1 -- 172.21.15.82:0/17495936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72241010f0 msgr2=0x7f7224199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7223fff640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72241010f0 0x7f7224199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7223fff640 1 -- 172.21.15.82:0/17495936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7224106170 msgr2=0x7f7224199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.686+0000 7f7223fff640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7224106170 0x7f7224199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.687+0000 7f7223fff640 1 -- 172.21.15.82:0/17495936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72241a4de0 con 0x7f722419a060 2024-04-21T08:47:08.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.687+0000 7f7222ffd640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7224106170 0x7f7224199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:08.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.687+0000 7f7223fff640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f722419a060 0x7f72241a4610 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f721c00b970 tx=0x7f721c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.687+0000 7f7220ff9640 1 -- 172.21.15.82:0/17495936 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c019070 con 0x7f722419a060 2024-04-21T08:47:08.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.687+0000 7f7229b4b640 1 -- 172.21.15.82:0/17495936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72241a50d0 con 0x7f722419a060 2024-04-21T08:47:08.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.687+0000 7f7220ff9640 1 -- 172.21.15.82:0/17495936 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f721c014440 con 0x7f722419a060 2024-04-21T08:47:08.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.687+0000 7f7220ff9640 1 -- 172.21.15.82:0/17495936 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721c0075d0 con 0x7f722419a060 2024-04-21T08:47:08.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.687+0000 7f7229b4b640 1 -- 172.21.15.82:0/17495936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72241a5620 con 0x7f722419a060 2024-04-21T08:47:08.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.688+0000 7f7229b4b640 1 -- 172.21.15.82:0/17495936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f722404fce0 con 0x7f722419a060 2024-04-21T08:47:08.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.689+0000 7f0c018ba640 1 -- 172.21.15.82:0/3025895868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bfc101ac0 msgr2=0x7f0bfc105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.689+0000 7f0c018ba640 1 --2- 172.21.15.82:0/3025895868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bfc101ac0 0x7f0bfc105b00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009900 tx=0x7f0be402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:08.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.689+0000 7f0c018ba640 1 -- 172.21.15.82:0/3025895868 shutdown_connections 2024-04-21T08:47:08.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.689+0000 7f0c018ba640 1 --2- 172.21.15.82:0/3025895868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bfc1061e0 0x7f0bfc10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.689+0000 7f0c018ba640 1 --2- 172.21.15.82:0/3025895868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bfc101ac0 0x7f0bfc105b00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.689+0000 7f0c018ba640 1 --2- 172.21.15.82:0/3025895868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bfc101110 0x7f0bfc1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.689+0000 7f0c018ba640 1 -- 172.21.15.82:0/3025895868 >> 172.21.15.82:0/3025895868 conn(0x7f0bfc077f80 msgr2=0x7f0bfc078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.689+0000 7f0c018ba640 1 -- 172.21.15.82:0/3025895868 shutdown_connections 2024-04-21T08:47:08.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.689+0000 7f0c018ba640 1 -- 172.21.15.82:0/3025895868 wait complete. 2024-04-21T08:47:08.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0c018ba640 1 Processor -- start 2024-04-21T08:47:08.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f7220ff9640 1 -- 172.21.15.82:0/17495936 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f721c01c070 con 0x7f722419a060 2024-04-21T08:47:08.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0c018ba640 1 -- start start 2024-04-21T08:47:08.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0c018ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bfc101110 0x7f0bfc1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0c018ba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bfc1061e0 0x7f0bfc199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0c018ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bfc19a1a0 0x7f0bfc1a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0c018ba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc111150 con 0x7f0bfc1061e0 2024-04-21T08:47:08.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0c018ba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc110fd0 con 0x7f0bfc19a1a0 2024-04-21T08:47:08.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0c018ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1112d0 con 0x7f0bfc101110 2024-04-21T08:47:08.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bfc1061e0 0x7f0bfc199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bfc1061e0 0x7f0bfc199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44542/0 (socket says 172.21.15.82:44542) 2024-04-21T08:47:08.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bf3fff640 1 -- 172.21.15.82:0/3629859884 learned_addr learned my addr 172.21.15.82:0/3629859884 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bfc101110 0x7f0bfc1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bfb7fe640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bfc19a1a0 0x7f0bfc1a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bf3fff640 1 -- 172.21.15.82:0/3629859884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bfc101110 msgr2=0x7f0bfc1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bf3fff640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bfc101110 0x7f0bfc1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bf3fff640 1 -- 172.21.15.82:0/3629859884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bfc19a1a0 msgr2=0x7f0bfc1a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bf3fff640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bfc19a1a0 0x7f0bfc1a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bf3fff640 1 -- 172.21.15.82:0/3629859884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc1a4f20 con 0x7f0bfc1061e0 2024-04-21T08:47:08.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bfc101110 0x7f0bfc1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:08.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bfb7fe640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bfc19a1a0 0x7f0bfc1a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:08.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f0bf3fff640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bfc1061e0 0x7f0bfc199c10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0be4000c00 tx=0x7f0be402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.690+0000 7f7220ff9640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f720c0a3ee0 0x7f720c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.691+0000 7f72237fe640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f720c0a3ee0 0x7f720c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.691+0000 7f7220ff9640 1 -- 172.21.15.82:0/17495936 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f721c0d0750 con 0x7f722419a060 2024-04-21T08:47:08.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.691+0000 7f72237fe640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f720c0a3ee0 0x7f720c0a63a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7214006010 tx=0x7f7214005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.694+0000 7f0c008b8640 1 -- 172.21.15.82:0/3629859884 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4043020 con 0x7f0bfc1061e0 2024-04-21T08:47:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.694+0000 7f0c008b8640 1 -- 172.21.15.82:0/3629859884 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0be402caf0 con 0x7f0bfc1061e0 2024-04-21T08:47:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.694+0000 7f0c018ba640 1 -- 172.21.15.82:0/3629859884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bfc1a51b0 con 0x7f0bfc1061e0 2024-04-21T08:47:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.694+0000 7f0c008b8640 1 -- 172.21.15.82:0/3629859884 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4003b20 con 0x7f0bfc1061e0 2024-04-21T08:47:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.694+0000 7f0c018ba640 1 -- 172.21.15.82:0/3629859884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bfc1a5750 con 0x7f0bfc1061e0 2024-04-21T08:47:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.694+0000 7f7220ff9640 1 -- 172.21.15.82:0/17495936 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f721c09c2a0 con 0x7f722419a060 2024-04-21T08:47:08.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0c04589740 1 Processor -- start 2024-04-21T08:47:08.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0c04589740 1 -- start start 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0c04589740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76dac8e0 0x562e76daccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0c04589740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76db5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0c04589740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76db5e10 0x562e76db8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0c04589740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562e76d10f00 con 0x562e76dad290 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0c04589740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562e76cae590 con 0x562e76db5e10 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0c04589740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562e76d0cec0 con 0x562e76dac8e0 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0bfaffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76db5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0bfaffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76db5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44548/0 (socket says 172.21.15.82:44548) 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0bfaffd640 1 -- 172.21.15.82:0/3297952584 learned_addr learned my addr 172.21.15.82:0/3297952584 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0c008b8640 1 -- 172.21.15.82:0/3629859884 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f0be4046020 con 0x7f0bfc1061e0 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.697+0000 7f0bfb7fe640 1 --2- 172.21.15.82:0/3297952584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76dac8e0 0x562e76daccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.698+0000 7f0bfaffd640 1 -- 172.21.15.82:0/3297952584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76dac8e0 msgr2=0x562e76daccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.698+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3297952584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76dac8e0 0x562e76daccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.698+0000 7f0bfaffd640 1 -- 172.21.15.82:0/3297952584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76db5e10 msgr2=0x562e76db8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.698+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3297952584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76db5e10 0x562e76db8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.698+0000 7f0bfaffd640 1 -- 172.21.15.82:0/3297952584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e76aeb7e0 con 0x562e76dad290 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.698+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3297952584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76db5840 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009d50 tx=0x7f0bec0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb5ec60448ed638 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.698+0000 7f0bf1ffb640 1 -- 172.21.15.82:0/3297952584 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec012650 con 0x562e76dad290 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.698+0000 7f0bf1ffb640 1 -- 172.21.15.82:0/3297952584 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bec009e40 con 0x562e76dad290 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.698+0000 7f0bf1ffb640 1 -- 172.21.15.82:0/3297952584 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec00fc00 con 0x562e76dad290 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.698+0000 7f0c008b8640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0be80a3ec0 0x7f0be80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.698+0000 7f0bfb7fe640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0be80a3ec0 0x7f0be80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.698+0000 7f0c008b8640 1 -- 172.21.15.82:0/3629859884 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f0be40f2e70 con 0x7f0bfc1061e0 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.699+0000 7f0c04589740 1 -- 172.21.15.82:0/3297952584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 msgr2=0x562e76db5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.699+0000 7f0bfb7fe640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0be80a3ec0 0x7f0be80a6380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4005fd0 tx=0x7f0bf4009500 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.699+0000 7f0c04589740 1 --2- 172.21.15.82:0/3297952584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76db5840 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009d50 tx=0x7f0bec0117b0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.699+0000 7f0c04589740 1 -- 172.21.15.82:0/3297952584 shutdown_connections 2024-04-21T08:47:08.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.699+0000 7f0c04589740 1 --2- 172.21.15.82:0/3297952584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76db5e10 0x562e76db8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.699+0000 7f0c04589740 1 --2- 172.21.15.82:0/3297952584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76db5840 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.699+0000 7f0c04589740 1 --2- 172.21.15.82:0/3297952584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76dac8e0 0x562e76daccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.699+0000 7f0c04589740 1 -- 172.21.15.82:0/3297952584 >> 172.21.15.82:0/3297952584 conn(0x562e76da6fb0 msgr2=0x562e76da7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.699+0000 7f0c04589740 1 -- 172.21.15.82:0/3297952584 shutdown_connections 2024-04-21T08:47:08.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.699+0000 7f0c04589740 1 -- 172.21.15.82:0/3297952584 wait complete. 2024-04-21T08:47:08.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.699+0000 7f0c04589740 1 -- start start 2024-04-21T08:47:08.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.699+0000 7f0c04589740 1 Processor -- start 2024-04-21T08:47:08.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0c04589740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76db2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0c04589740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76dad290 0x562e76db32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0c04589740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76db5e10 0x562e76db3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0c04589740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562e76db8d80 con 0x562e76db5e10 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0c04589740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562e76cca580 con 0x562e76dac8e0 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0c04589740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562e76db8f00 con 0x562e76dad290 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0c04589740 10 client.0 _ll_register_callbacks cb 0x562e76cd0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bfb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76db5e10 0x562e76db3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bfb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76db5e10 0x562e76db3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44558/0 (socket says 172.21.15.82:44558) 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bfb7fe640 1 -- 172.21.15.82:0/3360026564 learned_addr learned my addr 172.21.15.82:0/3360026564 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76db2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bf3fff640 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76dad290 0x562e76db32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bfb7fe640 1 -- 172.21.15.82:0/3360026564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76dad290 msgr2=0x562e76db32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bfb7fe640 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76dad290 0x562e76db32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bfb7fe640 1 -- 172.21.15.82:0/3360026564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 msgr2=0x562e76db2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bfb7fe640 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76db2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bfb7fe640 1 -- 172.21.15.82:0/3360026564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e76b4bf70 con 0x562e76db5e10 2024-04-21T08:47:08.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bfb7fe640 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76db5e10 0x562e76db3830 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4012930 tx=0x7f0bf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e257d2375df8a334 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:08.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bdaffd640 1 -- 172.21.15.82:0/3360026564 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf400fce0 con 0x562e76db5e10 2024-04-21T08:47:08.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0bdaffd640 1 -- 172.21.15.82:0/3360026564 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f0bf401f420 con 0x562e76db5e10 2024-04-21T08:47:08.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.700+0000 7f0c04589740 1 -- 172.21.15.82:0/3360026564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e76d1e920 con 0x562e76db5e10 2024-04-21T08:47:08.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.701+0000 7f0bdaffd640 1 -- 172.21.15.82:0/3360026564 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0bf400fe80 con 0x562e76db5e10 2024-04-21T08:47:08.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.701+0000 7f0c04589740 10 client.15768 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:08.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.701+0000 7f0c04589740 1 -- 172.21.15.82:0/3360026564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e769e3170 con 0x562e76db5e10 2024-04-21T08:47:08.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.701+0000 7f0bdaffd640 1 -- 172.21.15.82:0/3360026564 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562e76d1e920 con 0x562e76db5e10 2024-04-21T08:47:08.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.701+0000 7f0bdaffd640 1 -- 172.21.15.82:0/3360026564 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0bf4027760 con 0x562e76db5e10 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.701+0000 7f0c04589740 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.701+0000 7f0c04589740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.701+0000 7f0c04589740 20 client.15768 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.701+0000 7f0c04589740 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e76dc63a0 0x562e76dc87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.701+0000 7f0c04589740 4 client.15768 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.701+0000 7f0c04589740 1 -- 172.21.15.82:0/3360026564 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e76d18790 con 0x562e76dc63a0 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.701+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e76dc63a0 0x562e76dc87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.702+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e76dc63a0 0x562e76dc87d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f121a177695c9c85 server_cookie=87b341d6f48463e3 in_seq=0 out_seq=0 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.702+0000 7f0bdaffd640 10 client.15768 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.702+0000 7f0bdaffd640 1 -- 172.21.15.82:0/3360026564 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e76d18790 con 0x562e76dc63a0 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.702+0000 7f0bdaffd640 10 client.15768 handle_command_reply: tid=0 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.702+0000 7f0c04589740 1 client.15768 shutdown 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.703+0000 7f0c04589740 1 -- 172.21.15.82:0/3360026564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76db5e10 msgr2=0x562e76db3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.703+0000 7f0c04589740 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76db5e10 0x562e76db3830 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4012930 tx=0x7f0bf4004a40 comp rx=0 tx=0).stop 2024-04-21T08:47:08.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.703+0000 7f0c04589740 1 -- 172.21.15.82:0/3360026564 shutdown_connections 2024-04-21T08:47:08.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.703+0000 7f0c04589740 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e76dc63a0 0x562e76dc87d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.703+0000 7f0c04589740 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76db5e10 0x562e76db3830 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.703+0000 7f0c04589740 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76dad290 0x562e76db32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.703+0000 7f0c04589740 1 --2- 172.21.15.82:0/3360026564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76db2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.703+0000 7f0c04589740 1 -- 172.21.15.82:0/3360026564 >> 172.21.15.82:0/3360026564 conn(0x562e76da6fb0 msgr2=0x562e76da8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.703+0000 7f0bdaffd640 0 client.15768 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:08.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.703+0000 7f0c04589740 1 -- 172.21.15.82:0/3360026564 shutdown_connections 2024-04-21T08:47:08.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.703+0000 7f0c04589740 1 -- 172.21.15.82:0/3360026564 wait complete. 2024-04-21T08:47:08.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.703+0000 7f0c04589740 20 client.15768 trim_cache size 0 max 16384 2024-04-21T08:47:08.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.739+0000 7f0c04589740 1 Processor -- start 2024-04-21T08:47:08.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.739+0000 7f0c04589740 1 -- start start 2024-04-21T08:47:08.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.739+0000 7f0c04589740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76db54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.739+0000 7f0c04589740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76da22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.739+0000 7f0c04589740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76db5e10 0x562e76dc1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.739+0000 7f0c04589740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1af440 con 0x562e76dad290 2024-04-21T08:47:08.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.739+0000 7f0c04589740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1b0f80 con 0x562e76dac8e0 2024-04-21T08:47:08.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.739+0000 7f0c04589740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1afb10 con 0x562e76db5e10 2024-04-21T08:47:08.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.739+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76da22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.739+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76da22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44572/0 (socket says 172.21.15.82:44572) 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.739+0000 7f0bf3fff640 1 -- 172.21.15.82:0/3071433895 learned_addr learned my addr 172.21.15.82:0/3071433895 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.740+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3071433895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76db54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.740+0000 7f0bf3fff640 1 -- 172.21.15.82:0/3071433895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76db5e10 msgr2=0x562e76dc1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.740+0000 7f0bf3fff640 1 --2- 172.21.15.82:0/3071433895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76db5e10 0x562e76dc1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.740+0000 7f0bf3fff640 1 -- 172.21.15.82:0/3071433895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 msgr2=0x562e76db54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.740+0000 7f0bf3fff640 1 --2- 172.21.15.82:0/3071433895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76db54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.740+0000 7f0bf3fff640 1 -- 172.21.15.82:0/3071433895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e76db1320 con 0x562e76dad290 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.740+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3071433895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76db54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.740+0000 7f0bf3fff640 1 --2- 172.21.15.82:0/3071433895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76da22d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0be40be5b0 tx=0x7f0be40df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f08e75c870eac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.740+0000 7f0bf17fa640 1 -- 172.21.15.82:0/3071433895 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be403c350 con 0x562e76dad290 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.740+0000 7f0bf17fa640 1 -- 172.21.15.82:0/3071433895 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0be40befb0 con 0x562e76dad290 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.740+0000 7f0bf17fa640 1 -- 172.21.15.82:0/3071433895 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4003cc0 con 0x562e76dad290 2024-04-21T08:47:08.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.741+0000 7f0c04589740 1 -- 172.21.15.82:0/3071433895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 msgr2=0x562e76da22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.741+0000 7f0c04589740 1 --2- 172.21.15.82:0/3071433895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76da22d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0be40be5b0 tx=0x7f0be40df040 comp rx=0 tx=0).stop 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.741+0000 7f0c04589740 1 -- 172.21.15.82:0/3071433895 shutdown_connections 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.741+0000 7f0c04589740 1 --2- 172.21.15.82:0/3071433895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76db5e10 0x562e76dc1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.741+0000 7f0c04589740 1 --2- 172.21.15.82:0/3071433895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76da22d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.741+0000 7f0c04589740 1 --2- 172.21.15.82:0/3071433895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76db54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.741+0000 7f0c04589740 1 -- 172.21.15.82:0/3071433895 >> 172.21.15.82:0/3071433895 conn(0x562e76da6fb0 msgr2=0x562e76da5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.741+0000 7f0c04589740 1 -- 172.21.15.82:0/3071433895 shutdown_connections 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.741+0000 7f0c04589740 1 -- 172.21.15.82:0/3071433895 wait complete. 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.741+0000 7f0c04589740 1 -- start start 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.741+0000 7f0c04589740 1 Processor -- start 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0c04589740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76dbe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0c04589740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76daf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0c04589740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76db5e10 0x562e76dafb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0c04589740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1b08b0 con 0x562e76dad290 2024-04-21T08:47:08.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0c04589740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1b01e0 con 0x562e76dac8e0 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0c04589740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc1ad420 con 0x562e76db5e10 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0c04589740 10 client.0 _ll_register_callbacks cb 0x7f0bf4009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76daf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76daf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44586/0 (socket says 172.21.15.82:44586) 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bf3fff640 1 -- 172.21.15.82:0/3742066931 learned_addr learned my addr 172.21.15.82:0/3742066931 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76dbe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bfb7fe640 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76db5e10 0x562e76dafb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bf3fff640 1 -- 172.21.15.82:0/3742066931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76db5e10 msgr2=0x562e76dafb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bf3fff640 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76db5e10 0x562e76dafb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bf3fff640 1 -- 172.21.15.82:0/3742066931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 msgr2=0x562e76dbe3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bf3fff640 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76dbe3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bf3fff640 1 -- 172.21.15.82:0/3742066931 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e76db0220 con 0x562e76dad290 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bfb7fe640 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76db5e10 0x562e76dafb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bf3fff640 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76daf640 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0be40bad10 tx=0x7f0be4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6028039c4593498 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bdaffd640 1 -- 172.21.15.82:0/3742066931 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be40a1020 con 0x562e76dad290 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0bdaffd640 1 -- 172.21.15.82:0/3742066931 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f0be40bfde0 con 0x562e76dad290 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.742+0000 7f0c04589740 1 -- 172.21.15.82:0/3742066931 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be40befb0 con 0x562e76dad290 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.743+0000 7f0bdaffd640 1 -- 172.21.15.82:0/3742066931 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0be40c7520 con 0x562e76dad290 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.743+0000 7f0c04589740 10 client.15774 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.743+0000 7f0c04589740 1 -- 172.21.15.82:0/3742066931 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e76e2c750 con 0x562e76dad290 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.743+0000 7f0bdaffd640 1 -- 172.21.15.82:0/3742066931 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0be40befb0 con 0x562e76dad290 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.743+0000 7f0bdaffd640 1 -- 172.21.15.82:0/3742066931 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0be40bdd00 con 0x562e76dad290 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.743+0000 7f0c04589740 10 client.15774 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.743+0000 7f0c04589740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.743+0000 7f0c04589740 20 client.15774 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.743+0000 7f0c04589740 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e76e2ccb0 0x562e76e2f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.743+0000 7f0c04589740 4 client.15774 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.743+0000 7f0c04589740 1 -- 172.21.15.82:0/3742066931 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562e76c94c50 con 0x562e76e2ccb0 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.743+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e76e2ccb0 0x562e76e2f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.744+0000 7f0bfaffd640 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e76e2ccb0 0x562e76e2f0d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44901bc48fdee13a server_cookie=a0ec4b1d6ec03fa3 in_seq=0 out_seq=0 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.744+0000 7f0bdaffd640 10 client.15774 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0bdaffd640 1 -- 172.21.15.82:0/3742066931 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562e76c94c50 con 0x562e76e2ccb0 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0bdaffd640 10 client.15774 handle_command_reply: tid=0 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0c04589740 1 client.15774 shutdown 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0c04589740 1 -- 172.21.15.82:0/3742066931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 msgr2=0x562e76daf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0c04589740 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76daf640 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0be40bad10 tx=0x7f0be4005bd0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0c04589740 1 -- 172.21.15.82:0/3742066931 shutdown_connections 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0c04589740 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e76e2ccb0 0x562e76e2f0d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0c04589740 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e76db5e10 0x562e76dafb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0c04589740 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e76dad290 0x562e76daf640 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0c04589740 1 --2- 172.21.15.82:0/3742066931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e76dac8e0 0x562e76dbe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0c04589740 1 -- 172.21.15.82:0/3742066931 >> 172.21.15.82:0/3742066931 conn(0x562e76da6fb0 msgr2=0x562e76da6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0bdaffd640 0 client.15774 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0c04589740 1 -- 172.21.15.82:0/3742066931 shutdown_connections 2024-04-21T08:47:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.745+0000 7f0c04589740 1 -- 172.21.15.82:0/3742066931 wait complete. 2024-04-21T08:47:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c04589740 20 client.15774 trim_cache size 0 max 16384 2024-04-21T08:47:08.752 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:47:08.752 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:47:08.752 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "66de9615-6102-4e2b-8b7b-20d9ab8139db", 2024-04-21T08:47:08.752 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:47:08.753 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:47:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c018ba640 1 -- 172.21.15.82:0/3629859884 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0be80a3ec0 msgr2=0x7f0be80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c018ba640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0be80a3ec0 0x7f0be80a6380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4005fd0 tx=0x7f0bf4009500 comp rx=0 tx=0).stop 2024-04-21T08:47:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c018ba640 1 -- 172.21.15.82:0/3629859884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bfc1061e0 msgr2=0x7f0bfc199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c018ba640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bfc1061e0 0x7f0bfc199c10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0be4000c00 tx=0x7f0be402c660 comp rx=0 tx=0).stop 2024-04-21T08:47:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c018ba640 1 -- 172.21.15.82:0/3629859884 shutdown_connections 2024-04-21T08:47:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c018ba640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bfc19a1a0 0x7f0bfc1a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c018ba640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0be80a3ec0 0x7f0be80a6380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c018ba640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bfc1061e0 0x7f0bfc199c10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c018ba640 1 --2- 172.21.15.82:0/3629859884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bfc101110 0x7f0bfc1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c018ba640 1 -- 172.21.15.82:0/3629859884 >> 172.21.15.82:0/3629859884 conn(0x7f0bfc077f80 msgr2=0x7f0bfc0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c018ba640 1 -- 172.21.15.82:0/3629859884 shutdown_connections 2024-04-21T08:47:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.746+0000 7f0c018ba640 1 -- 172.21.15.82:0/3629859884 wait complete. 2024-04-21T08:47:08.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:47:08.757 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:47:08.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.858+0000 7fd61658e640 1 -- 172.21.15.82:0/1434906874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd610101a30 msgr2=0x7fd610105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.858+0000 7fd61658e640 1 --2- 172.21.15.82:0/1434906874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd610101a30 0x7fd610105a70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd600009900 tx=0x7fd60002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:08.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:08 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/484165039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:08.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:08 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/19987036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:08.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:08 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3154898638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:08.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.858+0000 7fd61658e640 1 -- 172.21.15.82:0/1434906874 shutdown_connections 2024-04-21T08:47:08.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.858+0000 7fd61658e640 1 --2- 172.21.15.82:0/1434906874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd610106150 0x7fd61010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.858+0000 7fd61658e640 1 --2- 172.21.15.82:0/1434906874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd610101a30 0x7fd610105a70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.858+0000 7fd61658e640 1 --2- 172.21.15.82:0/1434906874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd610101080 0x7fd610101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.858+0000 7fd61658e640 1 -- 172.21.15.82:0/1434906874 >> 172.21.15.82:0/1434906874 conn(0x7fd610078000 msgr2=0x7fd610078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.858+0000 7fd61658e640 1 -- 172.21.15.82:0/1434906874 shutdown_connections 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.858+0000 7fd61658e640 1 -- 172.21.15.82:0/1434906874 wait complete. 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.859+0000 7fd61658e640 1 Processor -- start 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.859+0000 7fd61658e640 1 -- start start 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.859+0000 7fd61658e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd610101080 0x7fd61019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.859+0000 7fd61658e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd610106150 0x7fd61019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.859+0000 7fd61658e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61019f170 0x7fd61019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.859+0000 7fd61658e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd61010ff60 con 0x7fd610106150 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.859+0000 7fd61658e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd61010fde0 con 0x7fd61019f170 2024-04-21T08:47:08.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.859+0000 7fd61658e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6101100e0 con 0x7fd610101080 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd610106150 0x7fd61019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd610106150 0x7fd61019b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44608/0 (socket says 172.21.15.82:44608) 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60f7fe640 1 -- 172.21.15.82:0/151636690 learned_addr learned my addr 172.21.15.82:0/151636690 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60ffff640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd610101080 0x7fd61019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd614b2b640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61019f170 0x7fd61019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60f7fe640 1 -- 172.21.15.82:0/151636690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd610101080 msgr2=0x7fd61019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60f7fe640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd610101080 0x7fd61019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60f7fe640 1 -- 172.21.15.82:0/151636690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61019f170 msgr2=0x7fd61019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60f7fe640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61019f170 0x7fd61019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60f7fe640 1 -- 172.21.15.82:0/151636690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61019fc30 con 0x7fd610106150 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd614b2b640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61019f170 0x7fd61019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60f7fe640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd610106150 0x7fd61019b2b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd600005ec0 tx=0x7fd60002ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60d7fa640 1 -- 172.21.15.82:0/151636690 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600043020 con 0x7fd610106150 2024-04-21T08:47:08.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd61658e640 1 -- 172.21.15.82:0/151636690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61019fec0 con 0x7fd610106150 2024-04-21T08:47:08.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60d7fa640 1 -- 172.21.15.82:0/151636690 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6000052b0 con 0x7fd610106150 2024-04-21T08:47:08.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd60d7fa640 1 -- 172.21.15.82:0/151636690 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600003c60 con 0x7fd610106150 2024-04-21T08:47:08.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.860+0000 7fd61658e640 1 -- 172.21.15.82:0/151636690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6101a7710 con 0x7fd610106150 2024-04-21T08:47:08.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd60d7fa640 1 -- 172.21.15.82:0/151636690 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd600046020 con 0x7fd610106150 2024-04-21T08:47:08.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd619177740 1 Processor -- start 2024-04-21T08:47:08.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd619177740 1 -- start start 2024-04-21T08:47:08.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd619177740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4da580 0x55c43b4da960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd619177740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 0x55c43b4e3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd619177740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4e3a80 0x55c43b4e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd619177740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c43b43f1e0 con 0x55c43b4daea0 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd619177740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c43b3dc390 con 0x55c43b4e3a80 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd619177740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c43b43b1a0 con 0x55c43b4da580 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd60d7fa640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5f80a3e40 0x7fd5f80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd60ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 0x55c43b4e3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd60ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 0x55c43b4e3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44618/0 (socket says 172.21.15.82:44618) 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.863+0000 7fd60ffff640 1 -- 172.21.15.82:0/2921243248 learned_addr learned my addr 172.21.15.82:0/2921243248 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd614b2b640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5f80a3e40 0x7fd5f80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd60d7fa640 1 -- 172.21.15.82:0/151636690 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fd6000f30f0 con 0x7fd610106150 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd614b2b640 1 --2- 172.21.15.82:0/2921243248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4da580 0x55c43b4da960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd60f7fe640 1 --2- 172.21.15.82:0/2921243248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4e3a80 0x55c43b4e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd60ffff640 1 -- 172.21.15.82:0/2921243248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4da580 msgr2=0x55c43b4da960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd60ffff640 1 --2- 172.21.15.82:0/2921243248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4da580 0x55c43b4da960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd60ffff640 1 -- 172.21.15.82:0/2921243248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4e3a80 msgr2=0x55c43b4e5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd60ffff640 1 --2- 172.21.15.82:0/2921243248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4e3a80 0x55c43b4e5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd60ffff640 1 -- 172.21.15.82:0/2921243248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c43b21b7d0 con 0x55c43b4daea0 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd60ffff640 1 --2- 172.21.15.82:0/2921243248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 0x55c43b4e3540 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd60400ad00 tx=0x7fd60400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94761c4d5ad814f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd614b2b640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5f80a3e40 0x7fd5f80a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd608005fd0 tx=0x7fd60800a2f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd5f67fc640 1 -- 172.21.15.82:0/2921243248 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60401d070 con 0x55c43b4daea0 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd5f67fc640 1 -- 172.21.15.82:0/2921243248 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd604018af0 con 0x55c43b4daea0 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.864+0000 7fd5f67fc640 1 -- 172.21.15.82:0/2921243248 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604010b30 con 0x55c43b4daea0 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.865+0000 7fd619177740 1 -- 172.21.15.82:0/2921243248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 msgr2=0x55c43b4e3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.865+0000 7fd619177740 1 --2- 172.21.15.82:0/2921243248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 0x55c43b4e3540 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd60400ad00 tx=0x7fd60400f790 comp rx=0 tx=0).stop 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.865+0000 7fd619177740 1 -- 172.21.15.82:0/2921243248 shutdown_connections 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.865+0000 7fd619177740 1 --2- 172.21.15.82:0/2921243248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4e3a80 0x55c43b4e5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.865+0000 7fd619177740 1 --2- 172.21.15.82:0/2921243248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 0x55c43b4e3540 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.865+0000 7fd619177740 1 --2- 172.21.15.82:0/2921243248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4da580 0x55c43b4da960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.865+0000 7fd619177740 1 -- 172.21.15.82:0/2921243248 >> 172.21.15.82:0/2921243248 conn(0x55c43b4d4cb0 msgr2=0x55c43b4d5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.865+0000 7fd619177740 1 -- 172.21.15.82:0/2921243248 shutdown_connections 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.865+0000 7fd619177740 1 -- 172.21.15.82:0/2921243248 wait complete. 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.865+0000 7fd619177740 1 -- start start 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.865+0000 7fd619177740 1 Processor -- start 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd619177740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4da580 0x55c43b4e0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd619177740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4daea0 0x55c43b4e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd619177740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4e3a80 0x55c43b4e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd619177740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c43b4e6870 con 0x55c43b4e3a80 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd619177740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c43b3f82f0 con 0x55c43b4daea0 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd619177740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c43b4e69f0 con 0x55c43b4da580 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd619177740 10 client.0 _ll_register_callbacks cb 0x55c43b405580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd614b2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4e3a80 0x55c43b4e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd614b2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4e3a80 0x55c43b4e1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44630/0 (socket says 172.21.15.82:44630) 2024-04-21T08:47:08.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd614b2b640 1 -- 172.21.15.82:0/3275814935 learned_addr learned my addr 172.21.15.82:0/3275814935 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd60f7fe640 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4daea0 0x55c43b4e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd60ffff640 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4da580 0x55c43b4e0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd614b2b640 1 -- 172.21.15.82:0/3275814935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4da580 msgr2=0x55c43b4e0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd614b2b640 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4da580 0x55c43b4e0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd614b2b640 1 -- 172.21.15.82:0/3275814935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4daea0 msgr2=0x55c43b4e0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd614b2b640 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4daea0 0x55c43b4e0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd614b2b640 1 -- 172.21.15.82:0/3275814935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c43b1513f0 con 0x55c43b4e3a80 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd614b2b640 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4e3a80 0x55c43b4e1530 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd608010740 tx=0x7fd608004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e411a846f65d543 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd5ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd5ef7fe640 1 -- 172.21.15.82:0/3275814935 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608011470 con 0x55c43b4e3a80 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd5ef7fe640 1 -- 172.21.15.82:0/3275814935 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fd608011610 con 0x55c43b4e3a80 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd619177740 1 -- 172.21.15.82:0/3275814935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd604018af0 con 0x55c43b4e3a80 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd5ef7fe640 1 -- 172.21.15.82:0/3275814935 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd608011810 con 0x55c43b4e3a80 2024-04-21T08:47:08.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd619177740 10 client.15786 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.866+0000 7fd619177740 1 -- 172.21.15.82:0/3275814935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c43b1110f0 con 0x55c43b4e3a80 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.867+0000 7fd5ef7fe640 1 -- 172.21.15.82:0/3275814935 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd604018af0 con 0x55c43b4e3a80 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.867+0000 7fd5ef7fe640 1 -- 172.21.15.82:0/3275814935 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd608026a00 con 0x55c43b4e3a80 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.867+0000 7fd619177740 10 client.15786 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.867+0000 7fd619177740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.867+0000 7fd619177740 20 client.15786 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.867+0000 7fd619177740 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c43b4f3b70 0x55c43b4f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.867+0000 7fd619177740 4 client.15786 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.867+0000 7fd619177740 1 -- 172.21.15.82:0/3275814935 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c43b44cc00 con 0x55c43b4f3b70 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.867+0000 7fd60ffff640 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c43b4f3b70 0x55c43b4f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.868+0000 7fd60ffff640 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c43b4f3b70 0x55c43b4f5fd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8075e0c7f41b5f7a server_cookie=9cbfc6d1722f1963 in_seq=0 out_seq=0 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.868+0000 7fd5ef7fe640 10 client.15786 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:08.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.868+0000 7fd5ef7fe640 1 -- 172.21.15.82:0/3275814935 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c43b44cc00 con 0x55c43b4f3b70 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.868+0000 7fd5ef7fe640 10 client.15786 handle_command_reply: tid=0 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.868+0000 7fd619177740 1 client.15786 shutdown 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.868+0000 7fd619177740 1 -- 172.21.15.82:0/3275814935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4e3a80 msgr2=0x55c43b4e1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.868+0000 7fd619177740 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4e3a80 0x55c43b4e1530 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd608010740 tx=0x7fd608004a40 comp rx=0 tx=0).stop 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.869+0000 7fd619177740 1 -- 172.21.15.82:0/3275814935 shutdown_connections 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.869+0000 7fd619177740 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c43b4f3b70 0x55c43b4f5fd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.869+0000 7fd619177740 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4e3a80 0x55c43b4e1530 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.869+0000 7fd619177740 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4daea0 0x55c43b4e0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.869+0000 7fd5ef7fe640 0 client.15786 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.869+0000 7fd619177740 1 --2- 172.21.15.82:0/3275814935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4da580 0x55c43b4e0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.869+0000 7fd619177740 1 -- 172.21.15.82:0/3275814935 >> 172.21.15.82:0/3275814935 conn(0x55c43b4d4cb0 msgr2=0x55c43b4d6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.869+0000 7fd619177740 1 -- 172.21.15.82:0/3275814935 shutdown_connections 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.869+0000 7fd619177740 1 -- 172.21.15.82:0/3275814935 wait complete. 2024-04-21T08:47:08.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.869+0000 7fd619177740 20 client.15786 trim_cache size 0 max 16384 2024-04-21T08:47:08.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd619177740 1 Processor -- start 2024-04-21T08:47:08.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd619177740 1 -- start start 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd619177740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4da580 0x55c43b4e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd619177740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4daea0 0x55c43b4ef730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd619177740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4e3a80 0x55c43b4d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd619177740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd6101b1460 con 0x55c43b4da580 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd619177740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd6101b2fa0 con 0x55c43b4daea0 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd619177740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6101b1b30 con 0x55c43b4e3a80 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd60ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4da580 0x55c43b4e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd60ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4da580 0x55c43b4e31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44646/0 (socket says 172.21.15.82:44646) 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd60ffff640 1 -- 172.21.15.82:0/311730582 learned_addr learned my addr 172.21.15.82:0/311730582 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd614b2b640 1 --2- 172.21.15.82:0/311730582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4e3a80 0x55c43b4d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.886+0000 7fd60ffff640 1 -- 172.21.15.82:0/311730582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4e3a80 msgr2=0x55c43b4d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd60ffff640 1 --2- 172.21.15.82:0/311730582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4e3a80 0x55c43b4d3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd60ffff640 1 -- 172.21.15.82:0/311730582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4daea0 msgr2=0x55c43b4ef730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd60f7fe640 1 --2- 172.21.15.82:0/311730582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4daea0 0x55c43b4ef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd60ffff640 1 --2- 172.21.15.82:0/311730582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4daea0 0x55c43b4ef730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd60ffff640 1 -- 172.21.15.82:0/311730582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c43b4d5880 con 0x55c43b4da580 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd614b2b640 1 --2- 172.21.15.82:0/311730582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4e3a80 0x55c43b4d3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd60f7fe640 1 --2- 172.21.15.82:0/311730582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4daea0 0x55c43b4ef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd60ffff640 1 --2- 172.21.15.82:0/311730582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4da580 0x55c43b4e31e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd60400fc90 tx=0x7fd604010790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a83c56f785c758 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd5f5ffb640 1 -- 172.21.15.82:0/311730582 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60402f070 con 0x55c43b4da580 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd5f5ffb640 1 -- 172.21.15.82:0/311730582 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6040189f0 con 0x55c43b4da580 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd5f5ffb640 1 -- 172.21.15.82:0/311730582 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60401d040 con 0x55c43b4da580 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd619177740 1 -- 172.21.15.82:0/311730582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4da580 msgr2=0x55c43b4e31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.887+0000 7fd619177740 1 --2- 172.21.15.82:0/311730582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4da580 0x55c43b4e31e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd60400fc90 tx=0x7fd604010790 comp rx=0 tx=0).stop 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 -- 172.21.15.82:0/311730582 shutdown_connections 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 --2- 172.21.15.82:0/311730582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4e3a80 0x55c43b4d3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 --2- 172.21.15.82:0/311730582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4daea0 0x55c43b4ef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 --2- 172.21.15.82:0/311730582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4da580 0x55c43b4e31e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 -- 172.21.15.82:0/311730582 >> 172.21.15.82:0/311730582 conn(0x55c43b4d4cb0 msgr2=0x55c43b4d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 -- 172.21.15.82:0/311730582 shutdown_connections 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 -- 172.21.15.82:0/311730582 wait complete. 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 -- start start 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 Processor -- start 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4da580 0x55c43b4ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 0x55c43b4e7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4e3a80 0x55c43b4e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd6101b28d0 con 0x55c43b4daea0 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd6101b2200 con 0x55c43b4da580 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6101af440 con 0x55c43b4e3a80 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd619177740 10 client.0 _ll_register_callbacks cb 0x7fd60800ae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd60f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 0x55c43b4e7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd60f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 0x55c43b4e7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44658/0 (socket says 172.21.15.82:44658) 2024-04-21T08:47:08.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.888+0000 7fd60f7fe640 1 -- 172.21.15.82:0/2302378670 learned_addr learned my addr 172.21.15.82:0/2302378670 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd60ffff640 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4da580 0x55c43b4ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd60f7fe640 1 -- 172.21.15.82:0/2302378670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4e3a80 msgr2=0x55c43b4e7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd614b2b640 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4e3a80 0x55c43b4e7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd60f7fe640 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4e3a80 0x55c43b4e7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd60f7fe640 1 -- 172.21.15.82:0/2302378670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4da580 msgr2=0x55c43b4ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd60f7fe640 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4da580 0x55c43b4ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd60f7fe640 1 -- 172.21.15.82:0/2302378670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c43b4e82a0 con 0x55c43b4daea0 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd60ffff640 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4da580 0x55c43b4ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd60f7fe640 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 0x55c43b4e7460 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd6000beac0 tx=0x7fd60002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00429ba4291e546 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd5ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd5ef7fe640 1 -- 172.21.15.82:0/2302378670 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6000bfe60 con 0x55c43b4daea0 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd5ef7fe640 1 -- 172.21.15.82:0/2302378670 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fd600005460 con 0x55c43b4daea0 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd619177740 1 -- 172.21.15.82:0/2302378670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6040189f0 con 0x55c43b4daea0 2024-04-21T08:47:08.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd5ef7fe640 1 -- 172.21.15.82:0/2302378670 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6000bf230 con 0x55c43b4daea0 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd619177740 10 client.15792 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd619177740 1 -- 172.21.15.82:0/2302378670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c43b55a190 con 0x55c43b4daea0 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd5ef7fe640 1 -- 172.21.15.82:0/2302378670 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6040189f0 con 0x55c43b4daea0 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.889+0000 7fd5ef7fe640 1 -- 172.21.15.82:0/2302378670 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6000bed90 con 0x55c43b4daea0 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.890+0000 7fd619177740 10 client.15792 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.890+0000 7fd619177740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.890+0000 7fd619177740 20 client.15792 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.890+0000 7fd619177740 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c43b55a940 0x55c43b55cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.890+0000 7fd619177740 4 client.15792 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.890+0000 7fd619177740 1 -- 172.21.15.82:0/2302378670 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c43b446a70 con 0x55c43b55a940 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.890+0000 7fd60ffff640 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c43b55a940 0x55c43b55cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.890+0000 7fd5ef7fe640 10 client.15792 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.890+0000 7fd60ffff640 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c43b55a940 0x55c43b55cd60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=612bcdfa0112242f server_cookie=b0da6981e322c82d in_seq=0 out_seq=0 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.890+0000 7fd5ef7fe640 1 -- 172.21.15.82:0/2302378670 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c43b446a70 con 0x55c43b55a940 2024-04-21T08:47:08.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.890+0000 7fd5ef7fe640 10 client.15792 handle_command_reply: tid=0 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.890+0000 7fd619177740 1 client.15792 shutdown 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd619177740 1 -- 172.21.15.82:0/2302378670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 msgr2=0x55c43b4e7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd619177740 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 0x55c43b4e7460 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd6000beac0 tx=0x7fd60002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd619177740 1 -- 172.21.15.82:0/2302378670 shutdown_connections 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd619177740 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c43b55a940 0x55c43b55cd60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd619177740 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c43b4e3a80 0x55c43b4e7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd619177740 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c43b4daea0 0x55c43b4e7460 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd619177740 1 --2- 172.21.15.82:0/2302378670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c43b4da580 0x55c43b4ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd5ef7fe640 0 client.15792 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd619177740 1 -- 172.21.15.82:0/2302378670 >> 172.21.15.82:0/2302378670 conn(0x55c43b4d4cb0 msgr2=0x55c43b4d3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd619177740 1 -- 172.21.15.82:0/2302378670 shutdown_connections 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd619177740 1 -- 172.21.15.82:0/2302378670 wait complete. 2024-04-21T08:47:08.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd619177740 20 client.15792 trim_cache size 0 max 16384 2024-04-21T08:47:08.898 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:47:08.898 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:47:08.898 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:47:08.898 INFO:teuthology.orchestra.run.smithi082.stdout: "66de9615-6102-4e2b-8b7b-20d9ab8139db": { 2024-04-21T08:47:08.898 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:47:08.898 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "66de9615-6102-4e2b-8b7b-20d9ab8139db", 2024-04-21T08:47:08.898 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:47:08.898 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:47:08.898 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:47:08.898 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:47:08.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd61658e640 1 -- 172.21.15.82:0/151636690 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5f80a3e40 msgr2=0x7fd5f80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd61658e640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5f80a3e40 0x7fd5f80a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd608005fd0 tx=0x7fd60800a2f0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd61658e640 1 -- 172.21.15.82:0/151636690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd610106150 msgr2=0x7fd61019b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd61658e640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd610106150 0x7fd61019b2b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd600005ec0 tx=0x7fd60002ca10 comp rx=0 tx=0).stop 2024-04-21T08:47:08.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd61658e640 1 -- 172.21.15.82:0/151636690 shutdown_connections 2024-04-21T08:47:08.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd61658e640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61019f170 0x7fd61019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd61658e640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5f80a3e40 0x7fd5f80a6300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd61658e640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd610106150 0x7fd61019b2b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd61658e640 1 --2- 172.21.15.82:0/151636690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd610101080 0x7fd61019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd61658e640 1 -- 172.21.15.82:0/151636690 >> 172.21.15.82:0/151636690 conn(0x7fd610078000 msgr2=0x7fd6100fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd61658e640 1 -- 172.21.15.82:0/151636690 shutdown_connections 2024-04-21T08:47:08.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.891+0000 7fd61658e640 1 -- 172.21.15.82:0/151636690 wait complete. 2024-04-21T08:47:08.904 INFO:tasks.cephfs.filesystem:scrub status for tag:66de9615-6102-4e2b-8b7b-20d9ab8139db - {'path': '/', 'tag': '66de9615-6102-4e2b-8b7b-20d9ab8139db', 'options': 'recursive,force'} 2024-04-21T08:47:08.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.930+0000 7f7229b4b640 1 -- 172.21.15.82:0/17495936 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7224109410 con 0x7f720c0a3ee0 2024-04-21T08:47:08.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:08 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/484165039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:08.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:08 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/19987036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:08.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:08 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3154898638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:08.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.992+0000 7f7220ff9640 1 -- 172.21.15.82:0/17495936 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7214004270 con 0x7f720c0a3ee0 2024-04-21T08:47:08.992 INFO:teuthology.orchestra.run.smithi082.stdout:Schedule set for path / 2024-04-21T08:47:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.994+0000 7f7229b4b640 1 -- 172.21.15.82:0/17495936 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f720c0a3ee0 msgr2=0x7f720c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.994+0000 7f7229b4b640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f720c0a3ee0 0x7f720c0a63a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7214006010 tx=0x7f7214005e40 comp rx=0 tx=0).stop 2024-04-21T08:47:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.994+0000 7f7229b4b640 1 -- 172.21.15.82:0/17495936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f722419a060 msgr2=0x7f72241a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.994+0000 7f7229b4b640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f722419a060 0x7f72241a4610 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f721c00b970 tx=0x7f721c00be30 comp rx=0 tx=0).stop 2024-04-21T08:47:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.994+0000 7f7229b4b640 1 -- 172.21.15.82:0/17495936 shutdown_connections 2024-04-21T08:47:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.994+0000 7f7229b4b640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f720c0a3ee0 0x7f720c0a63a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.994+0000 7f7229b4b640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f722419a060 0x7f72241a4610 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.994+0000 7f7229b4b640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7224106170 0x7f7224199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.994+0000 7f7229b4b640 1 --2- 172.21.15.82:0/17495936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72241010f0 0x7f7224199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.994+0000 7f7229b4b640 1 -- 172.21.15.82:0/17495936 >> 172.21.15.82:0/17495936 conn(0x7f7224077fd0 msgr2=0x7f72240ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.994+0000 7f7229b4b640 1 -- 172.21.15.82:0/17495936 shutdown_connections 2024-04-21T08:47:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:08.995+0000 7f7229b4b640 1 -- 172.21.15.82:0/17495936 wait complete. 2024-04-21T08:47:09.005 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-21T08:47:09.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:08 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/484165039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:09.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:08 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/19987036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:09.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:08 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3154898638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.147+0000 7f0023c31640 1 -- 172.21.15.82:0/840182216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f001c102b80 msgr2=0x7f001c106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.147+0000 7f0023c31640 1 --2- 172.21.15.82:0/840182216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f001c102b80 0x7f001c106bc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0018009900 tx=0x7f001802bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.147+0000 7f0023c31640 1 -- 172.21.15.82:0/840182216 shutdown_connections 2024-04-21T08:47:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.147+0000 7f0023c31640 1 --2- 172.21.15.82:0/840182216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001c1072a0 0x7f001c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.147+0000 7f0023c31640 1 --2- 172.21.15.82:0/840182216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f001c102b80 0x7f001c106bc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.147+0000 7f0023c31640 1 --2- 172.21.15.82:0/840182216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c1021d0 0x7f001c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.147+0000 7f0023c31640 1 -- 172.21.15.82:0/840182216 >> 172.21.15.82:0/840182216 conn(0x7f001c0793e0 msgr2=0x7f001c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.147+0000 7f0023c31640 1 -- 172.21.15.82:0/840182216 shutdown_connections 2024-04-21T08:47:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.147+0000 7f0023c31640 1 -- 172.21.15.82:0/840182216 wait complete. 2024-04-21T08:47:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f0023c31640 1 Processor -- start 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f0023c31640 1 -- start start 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f0023c31640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001c1021d0 0x7f001c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f0023c31640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c1072a0 0x7f001c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f0023c31640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f001c19cef0 0x7f001c19d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f0023c31640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f001c112230 con 0x7f001c19cef0 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f0023c31640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1120b0 con 0x7f001c1021d0 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f0023c31640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1123b0 con 0x7f001c1072a0 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f00221ce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f001c19cef0 0x7f001c19d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f00221ce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f001c19cef0 0x7f001c19d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44682/0 (socket says 172.21.15.82:44682) 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f00221ce640 1 -- 172.21.15.82:0/1099525284 learned_addr learned my addr 172.21.15.82:0/1099525284 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f00211cc640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c1072a0 0x7f001c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f00219cd640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001c1021d0 0x7f001c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f00221ce640 1 -- 172.21.15.82:0/1099525284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c1072a0 msgr2=0x7f001c198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f00221ce640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c1072a0 0x7f001c198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f00221ce640 1 -- 172.21.15.82:0/1099525284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001c1021d0 msgr2=0x7f001c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f00221ce640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001c1021d0 0x7f001c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f00221ce640 1 -- 172.21.15.82:0/1099525284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c19d9e0 con 0x7f001c19cef0 2024-04-21T08:47:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f00211cc640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c1072a0 0x7f001c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.148+0000 7f00219cd640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001c1021d0 0x7f001c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.149+0000 7f00221ce640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f001c19cef0 0x7f001c19d3a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f001400cab0 tx=0x7f001400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.149+0000 7f0012ffd640 1 -- 172.21.15.82:0/1099525284 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001400ae20 con 0x7f001c19cef0 2024-04-21T08:47:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.149+0000 7f0012ffd640 1 -- 172.21.15.82:0/1099525284 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f001400f900 con 0x7f001c19cef0 2024-04-21T08:47:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.149+0000 7f0023c31640 1 -- 172.21.15.82:0/1099525284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001c19dcd0 con 0x7f001c19cef0 2024-04-21T08:47:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.149+0000 7f0012ffd640 1 -- 172.21.15.82:0/1099525284 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140189f0 con 0x7f001c19cef0 2024-04-21T08:47:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.149+0000 7f0023c31640 1 -- 172.21.15.82:0/1099525284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f001c1a5490 con 0x7f001c19cef0 2024-04-21T08:47:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.150+0000 7f0023c31640 1 -- 172.21.15.82:0/1099525284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f001c04fce0 con 0x7f001c19cef0 2024-04-21T08:47:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.151+0000 7f0012ffd640 1 -- 172.21.15.82:0/1099525284 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f001400fab0 con 0x7f001c19cef0 2024-04-21T08:47:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.151+0000 7f0012ffd640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f00080a3f60 0x7f00080a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.152+0000 7f00219cd640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f00080a3f60 0x7f00080a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.152+0000 7f0012ffd640 1 -- 172.21.15.82:0/1099525284 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f00140d15d0 con 0x7f001c19cef0 2024-04-21T08:47:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.152+0000 7f00219cd640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f00080a3f60 0x7f00080a6420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f000c005fd0 tx=0x7f000c005ea0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:09.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.156+0000 7f0012ffd640 1 -- 172.21.15.82:0/1099525284 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0014014030 con 0x7f001c19cef0 2024-04-21T08:47:09.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.358+0000 7f0023c31640 1 -- 172.21.15.82:0/1099525284 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f001c102b80 con 0x7f00080a3f60 2024-04-21T08:47:09.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.419+0000 7f0012ffd640 1 -- 172.21.15.82:0/1099525284 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f001c102b80 con 0x7f00080a3f60 2024-04-21T08:47:09.418 INFO:teuthology.orchestra.run.smithi082.stdout:Retention added to path / 2024-04-21T08:47:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.421+0000 7f0023c31640 1 -- 172.21.15.82:0/1099525284 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f00080a3f60 msgr2=0x7f00080a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.421+0000 7f0023c31640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f00080a3f60 0x7f00080a6420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f000c005fd0 tx=0x7f000c005ea0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.421+0000 7f0023c31640 1 -- 172.21.15.82:0/1099525284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f001c19cef0 msgr2=0x7f001c19d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.421+0000 7f0023c31640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f001c19cef0 0x7f001c19d3a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f001400cab0 tx=0x7f001400cf70 comp rx=0 tx=0).stop 2024-04-21T08:47:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.421+0000 7f0023c31640 1 -- 172.21.15.82:0/1099525284 shutdown_connections 2024-04-21T08:47:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.421+0000 7f0023c31640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f00080a3f60 0x7f00080a6420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.421+0000 7f0023c31640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f001c19cef0 0x7f001c19d3a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.421+0000 7f0023c31640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c1072a0 0x7f001c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.421+0000 7f0023c31640 1 --2- 172.21.15.82:0/1099525284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001c1021d0 0x7f001c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.421+0000 7f0023c31640 1 -- 172.21.15.82:0/1099525284 >> 172.21.15.82:0/1099525284 conn(0x7f001c0793e0 msgr2=0x7f001c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.421+0000 7f0023c31640 1 -- 172.21.15.82:0/1099525284 shutdown_connections 2024-04-21T08:47:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.421+0000 7f0023c31640 1 -- 172.21.15.82:0/1099525284 wait complete. 2024-04-21T08:47:09.431 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-21T08:47:09.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.530+0000 7febd5bbd640 1 -- 172.21.15.82:0/1311203680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febd006b3e0 msgr2=0x7febd0106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.530+0000 7febd5bbd640 1 --2- 172.21.15.82:0/1311203680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febd006b3e0 0x7febd0106bb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7febc4009900 tx=0x7febc402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:09.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.530+0000 7febd5bbd640 1 -- 172.21.15.82:0/1311203680 shutdown_connections 2024-04-21T08:47:09.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.530+0000 7febd5bbd640 1 --2- 172.21.15.82:0/1311203680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febd01072e0 0x7febd010ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.530+0000 7febd5bbd640 1 --2- 172.21.15.82:0/1311203680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febd006b3e0 0x7febd0106bb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.530+0000 7febd5bbd640 1 --2- 172.21.15.82:0/1311203680 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd006aa30 0x7febd006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.530+0000 7febd5bbd640 1 -- 172.21.15.82:0/1311203680 >> 172.21.15.82:0/1311203680 conn(0x7febd0077ba0 msgr2=0x7febd0077fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:09.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.530+0000 7febd5bbd640 1 -- 172.21.15.82:0/1311203680 shutdown_connections 2024-04-21T08:47:09.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.530+0000 7febd5bbd640 1 -- 172.21.15.82:0/1311203680 wait complete. 2024-04-21T08:47:09.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.531+0000 7febd5bbd640 1 Processor -- start 2024-04-21T08:47:09.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.531+0000 7febd5bbd640 1 -- start start 2024-04-21T08:47:09.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.531+0000 7febd5bbd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febd006aa30 0x7febd01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.531+0000 7febd5bbd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febd01072e0 0x7febd0198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.531+0000 7febd5bbd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd019cfb0 0x7febd019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.531+0000 7febd5bbd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7febd01121d0 con 0x7febd006aa30 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.531+0000 7febd5bbd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7febd0112050 con 0x7febd01072e0 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.531+0000 7febd5bbd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7febd0112350 con 0x7febd019cfb0 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.531+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febd006aa30 0x7febd01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.531+0000 7febcf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febd006aa30 0x7febd01986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44702/0 (socket says 172.21.15.82:44702) 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.531+0000 7febcf7fe640 1 -- 172.21.15.82:0/3389808525 learned_addr learned my addr 172.21.15.82:0/3389808525 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febceffd640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febd01072e0 0x7febd0198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febcffff640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd019cfb0 0x7febd019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febcf7fe640 1 -- 172.21.15.82:0/3389808525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd019cfb0 msgr2=0x7febd019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febcf7fe640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd019cfb0 0x7febd019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febcf7fe640 1 -- 172.21.15.82:0/3389808525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febd01072e0 msgr2=0x7febd0198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febcf7fe640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febd01072e0 0x7febd0198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febcf7fe640 1 -- 172.21.15.82:0/3389808525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd019daa0 con 0x7febd006aa30 2024-04-21T08:47:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febceffd640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febd01072e0 0x7febd0198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febcf7fe640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febd006aa30 0x7febd01986f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7febc000d770 tx=0x7febc000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febccff9640 1 -- 172.21.15.82:0/3389808525 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0017070 con 0x7febd006aa30 2024-04-21T08:47:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febccff9640 1 -- 172.21.15.82:0/3389808525 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc00054d0 con 0x7febd006aa30 2024-04-21T08:47:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febccff9640 1 -- 172.21.15.82:0/3389808525 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0004c30 con 0x7febd006aa30 2024-04-21T08:47:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febd5bbd640 1 -- 172.21.15.82:0/3389808525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd019dd90 con 0x7febd006aa30 2024-04-21T08:47:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.532+0000 7febd5bbd640 1 -- 172.21.15.82:0/3389808525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd01a5500 con 0x7febd006aa30 2024-04-21T08:47:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.533+0000 7febd5bbd640 1 -- 172.21.15.82:0/3389808525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd0103690 con 0x7febd006aa30 2024-04-21T08:47:09.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.534+0000 7febccff9640 1 -- 172.21.15.82:0/3389808525 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7febc0004e10 con 0x7febd006aa30 2024-04-21T08:47:09.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.534+0000 7febccff9640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7febb80a3ee0 0x7febb80a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.535+0000 7febceffd640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7febb80a3ee0 0x7febb80a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.535+0000 7febccff9640 1 -- 172.21.15.82:0/3389808525 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7febc00d09f0 con 0x7febd006aa30 2024-04-21T08:47:09.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.535+0000 7febceffd640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7febb80a3ee0 0x7febb80a63a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7febc4009230 tx=0x7febc4009610 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:09.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.539+0000 7febccff9640 1 -- 172.21.15.82:0/3389808525 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7febc009ca80 con 0x7febd006aa30 2024-04-21T08:47:09.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.740+0000 7febd5bbd640 1 -- 172.21.15.82:0/3389808525 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7febd006b3e0 con 0x7febb80a3ee0 2024-04-21T08:47:09.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.748+0000 7febccff9640 1 -- 172.21.15.82:0/3389808525 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7febd006b3e0 con 0x7febb80a3ee0 2024-04-21T08:47:09.748 INFO:teuthology.orchestra.run.smithi082.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-21T00:00:00", "created": "2024-04-21T08:47:08", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-21T08:47:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.750+0000 7febd5bbd640 1 -- 172.21.15.82:0/3389808525 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7febb80a3ee0 msgr2=0x7febb80a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.750+0000 7febd5bbd640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7febb80a3ee0 0x7febb80a63a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7febc4009230 tx=0x7febc4009610 comp rx=0 tx=0).stop 2024-04-21T08:47:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.750+0000 7febd5bbd640 1 -- 172.21.15.82:0/3389808525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febd006aa30 msgr2=0x7febd01986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.750+0000 7febd5bbd640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febd006aa30 0x7febd01986f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7febc000d770 tx=0x7febc000dc30 comp rx=0 tx=0).stop 2024-04-21T08:47:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.750+0000 7febd5bbd640 1 -- 172.21.15.82:0/3389808525 shutdown_connections 2024-04-21T08:47:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.750+0000 7febd5bbd640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd019cfb0 0x7febd019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.750+0000 7febd5bbd640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febd01072e0 0x7febd0198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.750+0000 7febd5bbd640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7febb80a3ee0 0x7febb80a63a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.750+0000 7febd5bbd640 1 --2- 172.21.15.82:0/3389808525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febd006aa30 0x7febd01986f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.750+0000 7febd5bbd640 1 -- 172.21.15.82:0/3389808525 >> 172.21.15.82:0/3389808525 conn(0x7febd0077ba0 msgr2=0x7febd00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.750+0000 7febd5bbd640 1 -- 172.21.15.82:0/3389808525 shutdown_connections 2024-04-21T08:47:09.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.751+0000 7febd5bbd640 1 -- 172.21.15.82:0/3389808525 wait complete. 2024-04-21T08:47:09.760 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:09 smithi082 ceph-mon[27967]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.1 KiB/s wr, 26 op/s 2024-04-21T08:47:09.760 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:09 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:47:09.760 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:09 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:09.760 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:09 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:47:09.760 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:09 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:09.760 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:09 smithi082 ceph-mon[27967]: from='client.15756 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:47:09.760 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:09 smithi082 ceph-mon[27967]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:47:09.761 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-21T08:47:09.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.861+0000 7fb8d77f4640 1 -- 172.21.15.82:0/3865934823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8d0103b00 msgr2=0x7fb8d010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.861+0000 7fb8d77f4640 1 --2- 172.21.15.82:0/3865934823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8d0103b00 0x7fb8d010bfe0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009900 tx=0x7fb8c402bd60 comp rx=0 tx=0).stop 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 -- 172.21.15.82:0/3865934823 shutdown_connections 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 --2- 172.21.15.82:0/3865934823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8d010c520 0x7fb8d010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 --2- 172.21.15.82:0/3865934823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8d0103b00 0x7fb8d010bfe0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 --2- 172.21.15.82:0/3865934823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d01031e0 0x7fb8d01035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 -- 172.21.15.82:0/3865934823 >> 172.21.15.82:0/3865934823 conn(0x7fb8d0077fd0 msgr2=0x7fb8d00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 -- 172.21.15.82:0/3865934823 shutdown_connections 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 -- 172.21.15.82:0/3865934823 wait complete. 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 Processor -- start 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 -- start start 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8d01031e0 0x7fb8d0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8d0103b00 0x7fb8d0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d010c520 0x7fb8d019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d01121b0 con 0x7fb8d01031e0 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0112030 con 0x7fb8d010c520 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d77f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0112330 con 0x7fb8d0103b00 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d5590640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8d01031e0 0x7fb8d0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d5590640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8d01031e0 0x7fb8d0198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44716/0 (socket says 172.21.15.82:44716) 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.862+0000 7fb8d5590640 1 -- 172.21.15.82:0/1015912621 learned_addr learned my addr 172.21.15.82:0/1015912621 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8d4d8f640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8d0103b00 0x7fb8d0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8d5590640 1 -- 172.21.15.82:0/1015912621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8d0103b00 msgr2=0x7fb8d0198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8d5590640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8d0103b00 0x7fb8d0198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8d5590640 1 -- 172.21.15.82:0/1015912621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d010c520 msgr2=0x7fb8d019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8d5d91640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d010c520 0x7fb8d019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8d5590640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d010c520 0x7fb8d019cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8d5590640 1 -- 172.21.15.82:0/1015912621 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d019d6b0 con 0x7fb8d01031e0 2024-04-21T08:47:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8d5590640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8d01031e0 0x7fb8d0198710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800d950 tx=0x7fb8c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8ce7fc640 1 -- 172.21.15.82:0/1015912621 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8017070 con 0x7fb8d01031e0 2024-04-21T08:47:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8ce7fc640 1 -- 172.21.15.82:0/1015912621 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8c8009c30 con 0x7fb8d01031e0 2024-04-21T08:47:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8d77f4640 1 -- 172.21.15.82:0/1015912621 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d019d9a0 con 0x7fb8d01031e0 2024-04-21T08:47:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8ce7fc640 1 -- 172.21.15.82:0/1015912621 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c800f620 con 0x7fb8d01031e0 2024-04-21T08:47:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.863+0000 7fb8d77f4640 1 -- 172.21.15.82:0/1015912621 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d01a5230 con 0x7fb8d01031e0 2024-04-21T08:47:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.864+0000 7fb8d77f4640 1 -- 172.21.15.82:0/1015912621 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d004fce0 con 0x7fb8d01031e0 2024-04-21T08:47:09.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.865+0000 7fb8ce7fc640 1 -- 172.21.15.82:0/1015912621 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fb8c8005030 con 0x7fb8d01031e0 2024-04-21T08:47:09.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.866+0000 7fb8ce7fc640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb8b80a3ee0 0x7fb8b80a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:09.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.866+0000 7fb8d4d8f640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb8b80a3ee0 0x7fb8b80a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:09.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.866+0000 7fb8ce7fc640 1 -- 172.21.15.82:0/1015912621 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fb8c80d0b40 con 0x7fb8d01031e0 2024-04-21T08:47:09.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.866+0000 7fb8d4d8f640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb8b80a3ee0 0x7fb8b80a63a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb8d010e310 tx=0x7fb8c402c7c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:09.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:09.869+0000 7fb8ce7fc640 1 -- 172.21.15.82:0/1015912621 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8c809cbd0 con 0x7fb8d01031e0 2024-04-21T08:47:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:09 smithi134 ceph-mon[31772]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.1 KiB/s wr, 26 op/s 2024-04-21T08:47:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:09 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:47:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:09 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:09 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:47:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:09 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:09 smithi134 ceph-mon[31772]: from='client.15756 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:47:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:09 smithi134 ceph-mon[31772]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:47:10.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.072+0000 7fb8d77f4640 1 -- 172.21.15.82:0/1015912621 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb8d0062390 con 0x7fb8b80a3ee0 2024-04-21T08:47:10.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.079+0000 7fb8ce7fc640 1 -- 172.21.15.82:0/1015912621 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb8d0062390 con 0x7fb8b80a3ee0 2024-04-21T08:47:10.079 INFO:teuthology.orchestra.run.smithi082.stdout:/ 1m 6m3h 2024-04-21T08:47:10.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.081+0000 7fb8d77f4640 1 -- 172.21.15.82:0/1015912621 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb8b80a3ee0 msgr2=0x7fb8b80a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.081+0000 7fb8d77f4640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb8b80a3ee0 0x7fb8b80a63a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb8d010e310 tx=0x7fb8c402c7c0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.081+0000 7fb8d77f4640 1 -- 172.21.15.82:0/1015912621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8d01031e0 msgr2=0x7fb8d0198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.081+0000 7fb8d77f4640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8d01031e0 0x7fb8d0198710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800d950 tx=0x7fb8c800de10 comp rx=0 tx=0).stop 2024-04-21T08:47:10.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.081+0000 7fb8d77f4640 1 -- 172.21.15.82:0/1015912621 shutdown_connections 2024-04-21T08:47:10.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.081+0000 7fb8d77f4640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d010c520 0x7fb8d019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.081+0000 7fb8d77f4640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8d0103b00 0x7fb8d0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.081+0000 7fb8d77f4640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb8b80a3ee0 0x7fb8b80a63a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.081+0000 7fb8d77f4640 1 --2- 172.21.15.82:0/1015912621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8d01031e0 0x7fb8d0198710 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.081+0000 7fb8d77f4640 1 -- 172.21.15.82:0/1015912621 >> 172.21.15.82:0/1015912621 conn(0x7fb8d0077fd0 msgr2=0x7fb8d00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:10.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.081+0000 7fb8d77f4640 1 -- 172.21.15.82:0/1015912621 shutdown_connections 2024-04-21T08:47:10.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.081+0000 7fb8d77f4640 1 -- 172.21.15.82:0/1015912621 wait complete. 2024-04-21T08:47:10.091 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-21T08:47:10.116 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-21T08:47:10.138 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-21T08:47:10.151 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-04-21T08:47:10.151 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-21T08:47:10.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:09 smithi106 ceph-mon[31072]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.1 KiB/s wr, 26 op/s 2024-04-21T08:47:10.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:09 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:47:10.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:09 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:10.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:09 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:47:10.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:09 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:10.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:09 smithi106 ceph-mon[31072]: from='client.15756 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:47:10.152 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:09 smithi106 ceph-mon[31072]: from='client.15798 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:47:10.158 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-21T08:47:10.170 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:47:10.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.269+0000 7f5730f9d640 1 -- 172.21.15.82:0/1087959554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f572c1005b0 msgr2=0x7f572c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.269+0000 7f5730f9d640 1 --2- 172.21.15.82:0/1087959554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f572c1005b0 0x7f572c100a30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5720009900 tx=0x7f572002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.269+0000 7f5730f9d640 1 -- 172.21.15.82:0/1087959554 shutdown_connections 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.269+0000 7f5730f9d640 1 --2- 172.21.15.82:0/1087959554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f572c100f70 0x7f572c10d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.269+0000 7f5730f9d640 1 --2- 172.21.15.82:0/1087959554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f572c1005b0 0x7f572c100a30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.269+0000 7f5730f9d640 1 --2- 172.21.15.82:0/1087959554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f572c106670 0x7f572c106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.269+0000 7f5730f9d640 1 -- 172.21.15.82:0/1087959554 >> 172.21.15.82:0/1087959554 conn(0x7f572c077450 msgr2=0x7f572c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.269+0000 7f5730f9d640 1 -- 172.21.15.82:0/1087959554 shutdown_connections 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.269+0000 7f5730f9d640 1 -- 172.21.15.82:0/1087959554 wait complete. 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.270+0000 7f5730f9d640 1 Processor -- start 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.270+0000 7f5730f9d640 1 -- start start 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.270+0000 7f5730f9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f572c1005b0 0x7f572c199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.270+0000 7f5730f9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f572c100f70 0x7f572c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.270+0000 7f5730f9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f572c106670 0x7f572c1a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.270+0000 7f5730f9d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f572c111070 con 0x7f572c100f70 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.270+0000 7f5730f9d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f572c110ef0 con 0x7f572c106670 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.270+0000 7f5730f9d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f572c1111f0 con 0x7f572c1005b0 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.270+0000 7f5729d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f572c100f70 0x7f572c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.270+0000 7f5729d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f572c100f70 0x7f572c199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44728/0 (socket says 172.21.15.82:44728) 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.270+0000 7f5729d9b640 1 -- 172.21.15.82:0/3869127127 learned_addr learned my addr 172.21.15.82:0/3869127127 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f572ad9d640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f572c106670 0x7f572c1a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f572a59c640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f572c1005b0 0x7f572c199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f5729d9b640 1 -- 172.21.15.82:0/3869127127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f572c1005b0 msgr2=0x7f572c199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f5729d9b640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f572c1005b0 0x7f572c199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f5729d9b640 1 -- 172.21.15.82:0/3869127127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f572c106670 msgr2=0x7f572c1a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f5729d9b640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f572c106670 0x7f572c1a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f5729d9b640 1 -- 172.21.15.82:0/3869127127 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572c1a4af0 con 0x7f572c100f70 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f572a59c640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f572c1005b0 0x7f572c199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f5729d9b640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f572c100f70 0x7f572c199a70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5720006010 tx=0x7f5720004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:10.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f571b7fe640 1 -- 172.21.15.82:0/3869127127 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720043020 con 0x7f572c100f70 2024-04-21T08:47:10.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f5730f9d640 1 -- 172.21.15.82:0/3869127127 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572c1a4cc0 con 0x7f572c100f70 2024-04-21T08:47:10.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f571b7fe640 1 -- 172.21.15.82:0/3869127127 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5720004440 con 0x7f572c100f70 2024-04-21T08:47:10.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f571b7fe640 1 -- 172.21.15.82:0/3869127127 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720003b50 con 0x7f572c100f70 2024-04-21T08:47:10.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.271+0000 7f5730f9d640 1 -- 172.21.15.82:0/3869127127 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f572c1a51d0 con 0x7f572c100f70 2024-04-21T08:47:10.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.272+0000 7f5730f9d640 1 -- 172.21.15.82:0/3869127127 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f572c069230 con 0x7f572c100f70 2024-04-21T08:47:10.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.273+0000 7f571b7fe640 1 -- 172.21.15.82:0/3869127127 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5720046020 con 0x7f572c100f70 2024-04-21T08:47:10.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.274+0000 7f571b7fe640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f57140a3e40 0x7f57140a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:10.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.274+0000 7f572a59c640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f57140a3e40 0x7f57140a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:10.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.274+0000 7f571b7fe640 1 -- 172.21.15.82:0/3869127127 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f57200f28f0 con 0x7f572c100f70 2024-04-21T08:47:10.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.274+0000 7f572a59c640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f57140a3e40 0x7f57140a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5724005fd0 tx=0x7f57240093c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:10.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.278+0000 7f571b7fe640 1 -- 172.21.15.82:0/3869127127 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f572004d070 con 0x7f572c100f70 2024-04-21T08:47:10.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.458+0000 7f5730f9d640 1 -- 172.21.15.82:0/3869127127 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f572c108430 con 0x7f572c100f70 2024-04-21T08:47:10.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.459+0000 7f571b7fe640 1 -- 172.21.15.82:0/3869127127 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25247 (secure 0 0 0) 0x7f57200a42b0 con 0x7f572c100f70 2024-04-21T08:47:10.459 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:10.459 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:47:00.617381+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2119539425":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6800/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/539366680":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/2275507980":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2883183315":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:47:10.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.461+0000 7f5730f9d640 1 -- 172.21.15.82:0/3869127127 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f57140a3e40 msgr2=0x7f57140a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.461+0000 7f5730f9d640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f57140a3e40 0x7f57140a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5724005fd0 tx=0x7f57240093c0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.461+0000 7f5730f9d640 1 -- 172.21.15.82:0/3869127127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f572c100f70 msgr2=0x7f572c199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.461+0000 7f5730f9d640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f572c100f70 0x7f572c199a70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5720006010 tx=0x7f5720004040 comp rx=0 tx=0).stop 2024-04-21T08:47:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.461+0000 7f5730f9d640 1 -- 172.21.15.82:0/3869127127 shutdown_connections 2024-04-21T08:47:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.461+0000 7f5730f9d640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f572c106670 0x7f572c1a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.461+0000 7f5730f9d640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f57140a3e40 0x7f57140a6300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.461+0000 7f5730f9d640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f572c100f70 0x7f572c199a70 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.461+0000 7f5730f9d640 1 --2- 172.21.15.82:0/3869127127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f572c1005b0 0x7f572c199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.461+0000 7f5730f9d640 1 -- 172.21.15.82:0/3869127127 >> 172.21.15.82:0/3869127127 conn(0x7f572c077450 msgr2=0x7f572c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.461+0000 7f5730f9d640 1 -- 172.21.15.82:0/3869127127 shutdown_connections 2024-04-21T08:47:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.461+0000 7f5730f9d640 1 -- 172.21.15.82:0/3869127127 wait complete. 2024-04-21T08:47:10.472 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:47:10.473 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:47:10.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.576+0000 7efd664fd640 1 -- 172.21.15.82:0/1947856468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd60107260 msgr2=0x7efd6010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.576+0000 7efd664fd640 1 --2- 172.21.15.82:0/1947856468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd60107260 0x7efd6010ee30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7efd58009ca0 tx=0x7efd5802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.576+0000 7efd664fd640 1 -- 172.21.15.82:0/1947856468 shutdown_connections 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.576+0000 7efd664fd640 1 --2- 172.21.15.82:0/1947856468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd60107260 0x7efd6010ee30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.576+0000 7efd664fd640 1 --2- 172.21.15.82:0/1947856468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd60102b40 0x7efd60106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.576+0000 7efd664fd640 1 --2- 172.21.15.82:0/1947856468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd60102190 0x7efd60102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.576+0000 7efd664fd640 1 -- 172.21.15.82:0/1947856468 >> 172.21.15.82:0/1947856468 conn(0x7efd600793a0 msgr2=0x7efd600797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.576+0000 7efd664fd640 1 -- 172.21.15.82:0/1947856468 shutdown_connections 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.576+0000 7efd664fd640 1 -- 172.21.15.82:0/1947856468 wait complete. 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd664fd640 1 Processor -- start 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd664fd640 1 -- start start 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd664fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd60102190 0x7efd60198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd664fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd60107260 0x7efd60198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd664fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6019cfa0 0x7efd6019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd664fd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efd601121d0 con 0x7efd60107260 2024-04-21T08:47:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd664fd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efd60112050 con 0x7efd60102190 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd664fd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd60112350 con 0x7efd6019cfa0 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd5f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd60107260 0x7efd60198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd5f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd60107260 0x7efd60198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44742/0 (socket says 172.21.15.82:44742) 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd5f7fe640 1 -- 172.21.15.82:0/3921312583 learned_addr learned my addr 172.21.15.82:0/3921312583 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd5f7fe640 1 -- 172.21.15.82:0/3921312583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6019cfa0 msgr2=0x7efd6019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd64a9a640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6019cfa0 0x7efd6019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd5ffff640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd60102190 0x7efd60198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd5f7fe640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6019cfa0 0x7efd6019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd5f7fe640 1 -- 172.21.15.82:0/3921312583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd60102190 msgr2=0x7efd60198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd5f7fe640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd60102190 0x7efd60198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd5f7fe640 1 -- 172.21.15.82:0/3921312583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6019da90 con 0x7efd60107260 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd64a9a640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6019cfa0 0x7efd6019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.577+0000 7efd5ffff640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd60102190 0x7efd60198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.578+0000 7efd5f7fe640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd60107260 0x7efd60198c70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efd5400c970 tx=0x7efd5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.578+0000 7efd5d7fa640 1 -- 172.21.15.82:0/3921312583 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54014070 con 0x7efd60107260 2024-04-21T08:47:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.578+0000 7efd5d7fa640 1 -- 172.21.15.82:0/3921312583 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd54004440 con 0x7efd60107260 2024-04-21T08:47:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.578+0000 7efd664fd640 1 -- 172.21.15.82:0/3921312583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6019dd80 con 0x7efd60107260 2024-04-21T08:47:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.578+0000 7efd5d7fa640 1 -- 172.21.15.82:0/3921312583 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54003e50 con 0x7efd60107260 2024-04-21T08:47:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.578+0000 7efd664fd640 1 -- 172.21.15.82:0/3921312583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd601a55c0 con 0x7efd60107260 2024-04-21T08:47:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.579+0000 7efd664fd640 1 -- 172.21.15.82:0/3921312583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd600691c0 con 0x7efd60107260 2024-04-21T08:47:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.580+0000 7efd5d7fa640 1 -- 172.21.15.82:0/3921312583 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7efd54017070 con 0x7efd60107260 2024-04-21T08:47:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.580+0000 7efd5d7fa640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efd480a4060 0x7efd480a6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.580+0000 7efd5d7fa640 1 -- 172.21.15.82:0/3921312583 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7efd540d09e0 con 0x7efd60107260 2024-04-21T08:47:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.580+0000 7efd5ffff640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efd480a4060 0x7efd480a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.581+0000 7efd5ffff640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efd480a4060 0x7efd480a6520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efd500058c0 tx=0x7efd50005d30 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:10.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.585+0000 7efd5d7fa640 1 -- 172.21.15.82:0/3921312583 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd5401e030 con 0x7efd60107260 2024-04-21T08:47:10.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.767+0000 7efd664fd640 1 -- 172.21.15.82:0/3921312583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd60102b40 con 0x7efd60107260 2024-04-21T08:47:10.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.767+0000 7efd5d7fa640 1 -- 172.21.15.82:0/3921312583 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7efd5400de20 con 0x7efd60107260 2024-04-21T08:47:10.767 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:47:10.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.769+0000 7efd664fd640 1 -- 172.21.15.82:0/3921312583 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efd480a4060 msgr2=0x7efd480a6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.769+0000 7efd664fd640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efd480a4060 0x7efd480a6520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efd500058c0 tx=0x7efd50005d30 comp rx=0 tx=0).stop 2024-04-21T08:47:10.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.769+0000 7efd664fd640 1 -- 172.21.15.82:0/3921312583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd60107260 msgr2=0x7efd60198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.769+0000 7efd664fd640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd60107260 0x7efd60198c70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efd5400c970 tx=0x7efd5400ce30 comp rx=0 tx=0).stop 2024-04-21T08:47:10.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.769+0000 7efd664fd640 1 -- 172.21.15.82:0/3921312583 shutdown_connections 2024-04-21T08:47:10.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.770+0000 7efd664fd640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6019cfa0 0x7efd6019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.770+0000 7efd664fd640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efd480a4060 0x7efd480a6520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.770+0000 7efd664fd640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd60107260 0x7efd60198c70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.770+0000 7efd664fd640 1 --2- 172.21.15.82:0/3921312583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd60102190 0x7efd60198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.770+0000 7efd664fd640 1 -- 172.21.15.82:0/3921312583 >> 172.21.15.82:0/3921312583 conn(0x7efd600793a0 msgr2=0x7efd600ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:10.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.770+0000 7efd664fd640 1 -- 172.21.15.82:0/3921312583 shutdown_connections 2024-04-21T08:47:10.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.770+0000 7efd664fd640 1 -- 172.21.15.82:0/3921312583 wait complete. 2024-04-21T08:47:10.779 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:10 smithi082 ceph-mon[27967]: from='client.15804 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:47:10.779 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:10 smithi082 ceph-mon[27967]: from='client.15810 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:47:10.779 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:10 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3869127127' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:10.780 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:47:10.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.881+0000 7f689d8c1640 1 -- 172.21.15.82:0/1431020746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898101a50 msgr2=0x7f6898105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.881+0000 7f689d8c1640 1 --2- 172.21.15.82:0/1431020746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898101a50 0x7f6898105a90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f688c009900 tx=0x7f688c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:10.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.881+0000 7f689d8c1640 1 -- 172.21.15.82:0/1431020746 shutdown_connections 2024-04-21T08:47:10.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.881+0000 7f689d8c1640 1 --2- 172.21.15.82:0/1431020746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6898106170 0x7f689810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.881+0000 7f689d8c1640 1 --2- 172.21.15.82:0/1431020746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898101a50 0x7f6898105a90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.881+0000 7f689d8c1640 1 --2- 172.21.15.82:0/1431020746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68981010a0 0x7f6898101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.881+0000 7f689d8c1640 1 -- 172.21.15.82:0/1431020746 >> 172.21.15.82:0/1431020746 conn(0x7f6898077eb0 msgr2=0x7f68980782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:10.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.881+0000 7f689d8c1640 1 -- 172.21.15.82:0/1431020746 shutdown_connections 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.881+0000 7f689d8c1640 1 -- 172.21.15.82:0/1431020746 wait complete. 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f689d8c1640 1 Processor -- start 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f689d8c1640 1 -- start start 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f689d8c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68981010a0 0x7f689819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f689d8c1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898106170 0x7f689819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f689d8c1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68981986a0 0x7f6898198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f689d8c1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6898111110 con 0x7f6898106170 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f689d8c1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6898110f90 con 0x7f68981986a0 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f689d8c1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6898111290 con 0x7f68981010a0 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f68967fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898106170 0x7f689819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f68967fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898106170 0x7f689819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44766/0 (socket says 172.21.15.82:44766) 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f68967fc640 1 -- 172.21.15.82:0/1154161364 learned_addr learned my addr 172.21.15.82:0/1154161364 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f68977fe640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68981986a0 0x7f6898198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f68967fc640 1 -- 172.21.15.82:0/1154161364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68981010a0 msgr2=0x7f689819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f68967fc640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68981010a0 0x7f689819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f68967fc640 1 -- 172.21.15.82:0/1154161364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68981986a0 msgr2=0x7f6898198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:10.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f68967fc640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68981986a0 0x7f6898198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:10.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.882+0000 7f68967fc640 1 -- 172.21.15.82:0/1154161364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68981992a0 con 0x7f6898106170 2024-04-21T08:47:10.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.883+0000 7f68967fc640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898106170 0x7f689819eb20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f688c005ec0 tx=0x7f688c02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:10.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.883+0000 7f689c8bf640 1 -- 172.21.15.82:0/1154161364 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c043020 con 0x7f6898106170 2024-04-21T08:47:10.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.883+0000 7f689c8bf640 1 -- 172.21.15.82:0/1154161364 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f688c0052b0 con 0x7f6898106170 2024-04-21T08:47:10.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.883+0000 7f689d8c1640 1 -- 172.21.15.82:0/1154161364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6898199530 con 0x7f6898106170 2024-04-21T08:47:10.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.883+0000 7f689c8bf640 1 -- 172.21.15.82:0/1154161364 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c003d40 con 0x7f6898106170 2024-04-21T08:47:10.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.883+0000 7f689d8c1640 1 -- 172.21.15.82:0/1154161364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68981a56d0 con 0x7f6898106170 2024-04-21T08:47:10.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.884+0000 7f689d8c1640 1 -- 172.21.15.82:0/1154161364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6898102570 con 0x7f6898106170 2024-04-21T08:47:10.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.885+0000 7f689c8bf640 1 -- 172.21.15.82:0/1154161364 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f688c046020 con 0x7f6898106170 2024-04-21T08:47:10.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.885+0000 7f689c8bf640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68800a3e40 0x7f68800a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:10.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.886+0000 7f6896ffd640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68800a3e40 0x7f68800a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:10.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.886+0000 7f689c8bf640 1 -- 172.21.15.82:0/1154161364 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f688c0f3710 con 0x7f6898106170 2024-04-21T08:47:10.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.886+0000 7f6896ffd640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68800a3e40 0x7f68800a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f689000ab90 tx=0x7f6890009250 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:10.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:10.889+0000 7f689c8bf640 1 -- 172.21.15.82:0/1154161364 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f688c04d020 con 0x7f6898106170 2024-04-21T08:47:10.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:10 smithi134 ceph-mon[31772]: from='client.15804 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:47:10.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:10 smithi134 ceph-mon[31772]: from='client.15810 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:47:10.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:10 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3869127127' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:11.029 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:10 smithi106 ceph-mon[31072]: from='client.15804 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:47:11.030 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:10 smithi106 ceph-mon[31072]: from='client.15810 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:47:11.030 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:10 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3869127127' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:11.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.067+0000 7f689d8c1640 1 -- 172.21.15.82:0/1154161364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f689810a3d0 con 0x7f6898106170 2024-04-21T08:47:11.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.067+0000 7f689c8bf640 1 -- 172.21.15.82:0/1154161364 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f688c0bf8b0 con 0x7f6898106170 2024-04-21T08:47:11.067 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:47:11.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.069+0000 7f689d8c1640 1 -- 172.21.15.82:0/1154161364 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68800a3e40 msgr2=0x7f68800a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.069+0000 7f689d8c1640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68800a3e40 0x7f68800a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f689000ab90 tx=0x7f6890009250 comp rx=0 tx=0).stop 2024-04-21T08:47:11.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.069+0000 7f689d8c1640 1 -- 172.21.15.82:0/1154161364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898106170 msgr2=0x7f689819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.069+0000 7f689d8c1640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898106170 0x7f689819eb20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f688c005ec0 tx=0x7f688c02ca10 comp rx=0 tx=0).stop 2024-04-21T08:47:11.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.069+0000 7f689d8c1640 1 -- 172.21.15.82:0/1154161364 shutdown_connections 2024-04-21T08:47:11.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.069+0000 7f689d8c1640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68981986a0 0x7f6898198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.069+0000 7f689d8c1640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68800a3e40 0x7f68800a6300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.069+0000 7f689d8c1640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6898106170 0x7f689819eb20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.069+0000 7f689d8c1640 1 --2- 172.21.15.82:0/1154161364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68981010a0 0x7f689819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.069+0000 7f689d8c1640 1 -- 172.21.15.82:0/1154161364 >> 172.21.15.82:0/1154161364 conn(0x7f6898077eb0 msgr2=0x7f689806aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:11.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.069+0000 7f689d8c1640 1 -- 172.21.15.82:0/1154161364 shutdown_connections 2024-04-21T08:47:11.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.069+0000 7f689d8c1640 1 -- 172.21.15.82:0/1154161364 wait complete. 2024-04-21T08:47:11.079 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:47:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.181+0000 7f8882fdb640 1 -- 172.21.15.82:0/977534549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c106170 msgr2=0x7f887c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.181+0000 7f8882fdb640 1 --2- 172.21.15.82:0/977534549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c106170 0x7f887c10dd40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8874009ca0 tx=0x7f887402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.181+0000 7f8882fdb640 1 -- 172.21.15.82:0/977534549 shutdown_connections 2024-04-21T08:47:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.181+0000 7f8882fdb640 1 --2- 172.21.15.82:0/977534549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c106170 0x7f887c10dd40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.181+0000 7f8882fdb640 1 --2- 172.21.15.82:0/977534549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f887c101a50 0x7f887c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.181+0000 7f8882fdb640 1 --2- 172.21.15.82:0/977534549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887c1010a0 0x7f887c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.181+0000 7f8882fdb640 1 -- 172.21.15.82:0/977534549 >> 172.21.15.82:0/977534549 conn(0x7f887c077eb0 msgr2=0x7f887c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.181+0000 7f8882fdb640 1 -- 172.21.15.82:0/977534549 shutdown_connections 2024-04-21T08:47:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.181+0000 7f8882fdb640 1 -- 172.21.15.82:0/977534549 wait complete. 2024-04-21T08:47:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f8882fdb640 1 Processor -- start 2024-04-21T08:47:11.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f8882fdb640 1 -- start start 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f8882fdb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887c1010a0 0x7f887c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f8882fdb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c106170 0x7f887c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f8882fdb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f887c199f80 0x7f887c1a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f8882fdb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f887c111130 con 0x7f887c106170 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f8882fdb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f887c110fb0 con 0x7f887c1010a0 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f8882fdb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f887c1112b0 con 0x7f887c199f80 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f887bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c106170 0x7f887c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f887bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c106170 0x7f887c1999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44788/0 (socket says 172.21.15.82:44788) 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f887bfff640 1 -- 172.21.15.82:0/2883449603 learned_addr learned my addr 172.21.15.82:0/2883449603 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f8880d77640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887c1010a0 0x7f887c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f8881578640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f887c199f80 0x7f887c1a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f887bfff640 1 -- 172.21.15.82:0/2883449603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f887c199f80 msgr2=0x7f887c1a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f887bfff640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f887c199f80 0x7f887c1a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f887bfff640 1 -- 172.21.15.82:0/2883449603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887c1010a0 msgr2=0x7f887c1994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f887bfff640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887c1010a0 0x7f887c1994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f887bfff640 1 -- 172.21.15.82:0/2883449603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887c1a4d30 con 0x7f887c106170 2024-04-21T08:47:11.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f887bfff640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c106170 0x7f887c1999f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f886800b450 tx=0x7f886800b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:11.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.182+0000 7f8879ffb640 1 -- 172.21.15.82:0/2883449603 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868015020 con 0x7f887c106170 2024-04-21T08:47:11.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.183+0000 7f8879ffb640 1 -- 172.21.15.82:0/2883449603 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8868011ce0 con 0x7f887c106170 2024-04-21T08:47:11.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.183+0000 7f8882fdb640 1 -- 172.21.15.82:0/2883449603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887c1a4fc0 con 0x7f887c106170 2024-04-21T08:47:11.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.183+0000 7f8879ffb640 1 -- 172.21.15.82:0/2883449603 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868010dc0 con 0x7f887c106170 2024-04-21T08:47:11.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.183+0000 7f8882fdb640 1 -- 172.21.15.82:0/2883449603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f887c1a54e0 con 0x7f887c106170 2024-04-21T08:47:11.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.184+0000 7f8882fdb640 1 -- 172.21.15.82:0/2883449603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f887c0691c0 con 0x7f887c106170 2024-04-21T08:47:11.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.185+0000 7f8879ffb640 1 -- 172.21.15.82:0/2883449603 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f8868018020 con 0x7f887c106170 2024-04-21T08:47:11.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.185+0000 7f8879ffb640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f886c0a3ec0 0x7f886c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:11.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.185+0000 7f8880d77640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f886c0a3ec0 0x7f886c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:11.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.185+0000 7f8879ffb640 1 -- 172.21.15.82:0/2883449603 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f88680d05d0 con 0x7f887c106170 2024-04-21T08:47:11.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.186+0000 7f8880d77640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f886c0a3ec0 0x7f886c0a6380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f887000abe0 tx=0x7f887000a2d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:11.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.190+0000 7f8879ffb640 1 -- 172.21.15.82:0/2883449603 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f886801f020 con 0x7f887c106170 2024-04-21T08:47:11.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.369+0000 7f8882fdb640 1 -- 172.21.15.82:0/2883449603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f887c101a50 con 0x7f887c106170 2024-04-21T08:47:11.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.370+0000 7f8879ffb640 1 -- 172.21.15.82:0/2883449603 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f886809c770 con 0x7f887c106170 2024-04-21T08:47:11.370 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:47:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.372+0000 7f8882fdb640 1 -- 172.21.15.82:0/2883449603 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f886c0a3ec0 msgr2=0x7f886c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.372+0000 7f8882fdb640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f886c0a3ec0 0x7f886c0a6380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f887000abe0 tx=0x7f887000a2d0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.372+0000 7f8882fdb640 1 -- 172.21.15.82:0/2883449603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c106170 msgr2=0x7f887c1999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.372+0000 7f8882fdb640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c106170 0x7f887c1999f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f886800b450 tx=0x7f886800b850 comp rx=0 tx=0).stop 2024-04-21T08:47:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.372+0000 7f8882fdb640 1 -- 172.21.15.82:0/2883449603 shutdown_connections 2024-04-21T08:47:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.372+0000 7f8882fdb640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f887c199f80 0x7f887c1a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.372+0000 7f8882fdb640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f886c0a3ec0 0x7f886c0a6380 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.372+0000 7f8882fdb640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c106170 0x7f887c1999f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.372+0000 7f8882fdb640 1 --2- 172.21.15.82:0/2883449603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f887c1010a0 0x7f887c1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.372+0000 7f8882fdb640 1 -- 172.21.15.82:0/2883449603 >> 172.21.15.82:0/2883449603 conn(0x7f887c077eb0 msgr2=0x7f887c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.372+0000 7f8882fdb640 1 -- 172.21.15.82:0/2883449603 shutdown_connections 2024-04-21T08:47:11.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.372+0000 7f8882fdb640 1 -- 172.21.15.82:0/2883449603 wait complete. 2024-04-21T08:47:11.382 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:47:11.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.484+0000 7fe9fc738640 1 -- 172.21.15.82:0/2498025004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f4103af0 msgr2=0x7fe9f410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.484+0000 7fe9fc738640 1 --2- 172.21.15.82:0/2498025004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f4103af0 0x7fe9f410bfd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009900 tx=0x7fe9e402bd60 comp rx=0 tx=0).stop 2024-04-21T08:47:11.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.484+0000 7fe9fc738640 1 -- 172.21.15.82:0/2498025004 shutdown_connections 2024-04-21T08:47:11.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.484+0000 7fe9fc738640 1 --2- 172.21.15.82:0/2498025004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9f410c510 0x7fe9f410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.484+0000 7fe9fc738640 1 --2- 172.21.15.82:0/2498025004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f4103af0 0x7fe9f410bfd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.484+0000 7fe9fc738640 1 --2- 172.21.15.82:0/2498025004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9f41031d0 0x7fe9f41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.484+0000 7fe9fc738640 1 -- 172.21.15.82:0/2498025004 >> 172.21.15.82:0/2498025004 conn(0x7fe9f4077f50 msgr2=0x7fe9f4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:11.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.484+0000 7fe9fc738640 1 -- 172.21.15.82:0/2498025004 shutdown_connections 2024-04-21T08:47:11.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.484+0000 7fe9fc738640 1 -- 172.21.15.82:0/2498025004 wait complete. 2024-04-21T08:47:11.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.484+0000 7fe9fc738640 1 Processor -- start 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9fc738640 1 -- start start 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9fc738640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9f41031d0 0x7fe9f4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9fc738640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f4103af0 0x7fe9f4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9fc738640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9f410c510 0x7fe9f419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9fc738640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f4112180 con 0x7fe9f4103af0 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9fc738640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f4112000 con 0x7fe9f410c510 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9fc738640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f4112300 con 0x7fe9f41031d0 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9f9cd3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f4103af0 0x7fe9f4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9f9cd3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f4103af0 0x7fe9f4198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44814/0 (socket says 172.21.15.82:44814) 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9f9cd3640 1 -- 172.21.15.82:0/3670977086 learned_addr learned my addr 172.21.15.82:0/3670977086 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9fa4d4640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9f41031d0 0x7fe9f4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9f9cd3640 1 -- 172.21.15.82:0/3670977086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9f41031d0 msgr2=0x7fe9f4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9f9cd3640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9f41031d0 0x7fe9f4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9facd5640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9f410c510 0x7fe9f419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:11.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9f9cd3640 1 -- 172.21.15.82:0/3670977086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9f410c510 msgr2=0x7fe9f419cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9f9cd3640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9f410c510 0x7fe9f419cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9f9cd3640 1 -- 172.21.15.82:0/3670977086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f419d710 con 0x7fe9f4103af0 2024-04-21T08:47:11.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9fa4d4640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9f41031d0 0x7fe9f4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:11.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9f9cd3640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f4103af0 0x7fe9f4198c60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4005940 tx=0x7fe9e40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:11.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9eb7fe640 1 -- 172.21.15.82:0/3670977086 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4043020 con 0x7fe9f4103af0 2024-04-21T08:47:11.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9eb7fe640 1 -- 172.21.15.82:0/3670977086 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9e402dc30 con 0x7fe9f4103af0 2024-04-21T08:47:11.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9eb7fe640 1 -- 172.21.15.82:0/3670977086 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4003a10 con 0x7fe9f4103af0 2024-04-21T08:47:11.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.485+0000 7fe9fc738640 1 -- 172.21.15.82:0/3670977086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f419d9a0 con 0x7fe9f4103af0 2024-04-21T08:47:11.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.486+0000 7fe9fc738640 1 -- 172.21.15.82:0/3670977086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f41a5290 con 0x7fe9f4103af0 2024-04-21T08:47:11.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.487+0000 7fe9fc738640 1 -- 172.21.15.82:0/3670977086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f4068d70 con 0x7fe9f4103af0 2024-04-21T08:47:11.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.488+0000 7fe9eb7fe640 1 -- 172.21.15.82:0/3670977086 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe9e4046020 con 0x7fe9f4103af0 2024-04-21T08:47:11.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.488+0000 7fe9eb7fe640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9e00a3e40 0x7fe9e00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:11.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.489+0000 7fe9fa4d4640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9e00a3e40 0x7fe9e00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:11.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.489+0000 7fe9eb7fe640 1 -- 172.21.15.82:0/3670977086 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe9e40f3580 con 0x7fe9f4103af0 2024-04-21T08:47:11.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.489+0000 7fe9fa4d4640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9e00a3e40 0x7fe9e00a6300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00058c0 tx=0x7fe9f0005d30 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:11.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.492+0000 7fe9eb7fe640 1 -- 172.21.15.82:0/3670977086 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9e404d020 con 0x7fe9f4103af0 2024-04-21T08:47:11.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.675+0000 7fe9fc738640 1 -- 172.21.15.82:0/3670977086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe9f41076c0 con 0x7fe9f4103af0 2024-04-21T08:47:11.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.675+0000 7fe9eb7fe640 1 -- 172.21.15.82:0/3670977086 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fe9e403e450 con 0x7fe9f4103af0 2024-04-21T08:47:11.675 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:47:11.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.677+0000 7fe9fc738640 1 -- 172.21.15.82:0/3670977086 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9e00a3e40 msgr2=0x7fe9e00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.677+0000 7fe9fc738640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9e00a3e40 0x7fe9e00a6300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00058c0 tx=0x7fe9f0005d30 comp rx=0 tx=0).stop 2024-04-21T08:47:11.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.677+0000 7fe9fc738640 1 -- 172.21.15.82:0/3670977086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f4103af0 msgr2=0x7fe9f4198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.677+0000 7fe9fc738640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f4103af0 0x7fe9f4198c60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4005940 tx=0x7fe9e40045e0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.677+0000 7fe9fc738640 1 -- 172.21.15.82:0/3670977086 shutdown_connections 2024-04-21T08:47:11.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.677+0000 7fe9fc738640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9f410c510 0x7fe9f419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.677+0000 7fe9fc738640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9e00a3e40 0x7fe9e00a6300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.677+0000 7fe9fc738640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f4103af0 0x7fe9f4198c60 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.677+0000 7fe9fc738640 1 --2- 172.21.15.82:0/3670977086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9f41031d0 0x7fe9f4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.677+0000 7fe9fc738640 1 -- 172.21.15.82:0/3670977086 >> 172.21.15.82:0/3670977086 conn(0x7fe9f4077f50 msgr2=0x7fe9f40fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:11.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.677+0000 7fe9fc738640 1 -- 172.21.15.82:0/3670977086 shutdown_connections 2024-04-21T08:47:11.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.677+0000 7fe9fc738640 1 -- 172.21.15.82:0/3670977086 wait complete. 2024-04-21T08:47:11.688 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:47:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.786+0000 7fe40a677640 1 -- 172.21.15.82:0/2855959557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe404101b60 msgr2=0x7fe40410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.786+0000 7fe40a677640 1 --2- 172.21.15.82:0/2855959557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe404101b60 0x7fe40410dc90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc009cd0 tx=0x7fe3fc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.786+0000 7fe40a677640 1 -- 172.21.15.82:0/2855959557 shutdown_connections 2024-04-21T08:47:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.786+0000 7fe40a677640 1 --2- 172.21.15.82:0/2855959557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe404101b60 0x7fe40410dc90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.786+0000 7fe40a677640 1 --2- 172.21.15.82:0/2855959557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4041011a0 0x7fe404101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.786+0000 7fe40a677640 1 --2- 172.21.15.82:0/2855959557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404105e30 0x7fe404106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.786+0000 7fe40a677640 1 -- 172.21.15.82:0/2855959557 >> 172.21.15.82:0/2855959557 conn(0x7fe404077e30 msgr2=0x7fe404078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.786+0000 7fe40a677640 1 -- 172.21.15.82:0/2855959557 shutdown_connections 2024-04-21T08:47:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.786+0000 7fe40a677640 1 -- 172.21.15.82:0/2855959557 wait complete. 2024-04-21T08:47:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.786+0000 7fe40a677640 1 Processor -- start 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.786+0000 7fe40a677640 1 -- start start 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe40a677640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4041011a0 0x7fe4041986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe40a677640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404101b60 0x7fe404198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe40a677640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe404105e30 0x7fe40419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe40a677640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe404111140 con 0x7fe4041011a0 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe40a677640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe404110fc0 con 0x7fe404105e30 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe40a677640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4041112c0 con 0x7fe404101b60 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe403fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4041011a0 0x7fe4041986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe403fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4041011a0 0x7fe4041986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44824/0 (socket says 172.21.15.82:44824) 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe403fff640 1 -- 172.21.15.82:0/3791743242 learned_addr learned my addr 172.21.15.82:0/3791743242 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe408c14640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe404105e30 0x7fe40419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe4037fe640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404101b60 0x7fe404198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe403fff640 1 -- 172.21.15.82:0/3791743242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404101b60 msgr2=0x7fe404198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe403fff640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404101b60 0x7fe404198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe403fff640 1 -- 172.21.15.82:0/3791743242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe404105e30 msgr2=0x7fe40419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe403fff640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe404105e30 0x7fe40419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe403fff640 1 -- 172.21.15.82:0/3791743242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40419d660 con 0x7fe4041011a0 2024-04-21T08:47:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe408c14640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe404105e30 0x7fe40419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe403fff640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4041011a0 0x7fe4041986c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00b4a0 tx=0x7fe3ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe4017fa640 1 -- 172.21.15.82:0/3791743242 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec014070 con 0x7fe4041011a0 2024-04-21T08:47:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe4017fa640 1 -- 172.21.15.82:0/3791743242 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3ec010ac0 con 0x7fe4041011a0 2024-04-21T08:47:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe40a677640 1 -- 172.21.15.82:0/3791743242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40419d950 con 0x7fe4041011a0 2024-04-21T08:47:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe4017fa640 1 -- 172.21.15.82:0/3791743242 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec017c30 con 0x7fe4041011a0 2024-04-21T08:47:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.787+0000 7fe40a677640 1 -- 172.21.15.82:0/3791743242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4041a51e0 con 0x7fe4041011a0 2024-04-21T08:47:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.788+0000 7fe40a677640 1 -- 172.21.15.82:0/3791743242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe404102570 con 0x7fe4041011a0 2024-04-21T08:47:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.789+0000 7fe4017fa640 1 -- 172.21.15.82:0/3791743242 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe3ec017070 con 0x7fe4041011a0 2024-04-21T08:47:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.790+0000 7fe4017fa640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe3f00a3e40 0x7fe3f00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.790+0000 7fe4037fe640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe3f00a3e40 0x7fe3f00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.790+0000 7fe4017fa640 1 -- 172.21.15.82:0/3791743242 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe3ec0d04f0 con 0x7fe4041011a0 2024-04-21T08:47:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.790+0000 7fe4037fe640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe3f00a3e40 0x7fe3f00a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe4040fe530 tx=0x7fe3f4005cc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:11.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.794+0000 7fe4017fa640 1 -- 172.21.15.82:0/3791743242 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3ec09c690 con 0x7fe4041011a0 2024-04-21T08:47:11.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.974+0000 7fe40a677640 1 -- 172.21.15.82:0/3791743242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe404068de0 con 0x7fe4041011a0 2024-04-21T08:47:11.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.976+0000 7fe4017fa640 1 -- 172.21.15.82:0/3791743242 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25247 (secure 0 0 0) 0x7fe3ec09bf90 con 0x7fe4041011a0 2024-04-21T08:47:11.976 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:11.976 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:47:00.617381+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2119539425":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6800/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/539366680":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/2275507980":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2883183315":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:47:11.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:11 smithi134 ceph-mon[31772]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 911 B/s wr, 20 op/s 2024-04-21T08:47:11.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:11 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3921312583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:11.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:11 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1154161364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:11.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:11 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2883449603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.978+0000 7fe40a677640 1 -- 172.21.15.82:0/3791743242 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe3f00a3e40 msgr2=0x7fe3f00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.978+0000 7fe40a677640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe3f00a3e40 0x7fe3f00a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe4040fe530 tx=0x7fe3f4005cc0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.978+0000 7fe40a677640 1 -- 172.21.15.82:0/3791743242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4041011a0 msgr2=0x7fe4041986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.978+0000 7fe40a677640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4041011a0 0x7fe4041986c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00b4a0 tx=0x7fe3ec00b960 comp rx=0 tx=0).stop 2024-04-21T08:47:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.978+0000 7fe40a677640 1 -- 172.21.15.82:0/3791743242 shutdown_connections 2024-04-21T08:47:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.978+0000 7fe40a677640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe404105e30 0x7fe40419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.978+0000 7fe40a677640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe404101b60 0x7fe404198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.978+0000 7fe40a677640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe3f00a3e40 0x7fe3f00a6300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.978+0000 7fe40a677640 1 --2- 172.21.15.82:0/3791743242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4041011a0 0x7fe4041986c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:11.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.978+0000 7fe40a677640 1 -- 172.21.15.82:0/3791743242 >> 172.21.15.82:0/3791743242 conn(0x7fe404077e30 msgr2=0x7fe404069360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:11.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.978+0000 7fe40a677640 1 -- 172.21.15.82:0/3791743242 shutdown_connections 2024-04-21T08:47:11.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:11.978+0000 7fe40a677640 1 -- 172.21.15.82:0/3791743242 wait complete. 2024-04-21T08:47:11.988 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:11 smithi082 ceph-mon[27967]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 911 B/s wr, 20 op/s 2024-04-21T08:47:11.988 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:11 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3921312583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:11.988 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:11 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1154161364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:11.988 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:11 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2883449603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:11.990 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:47:11.990 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:47:12.046 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:11 smithi106 ceph-mon[31072]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 911 B/s wr, 20 op/s 2024-04-21T08:47:12.046 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:11 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3921312583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:12.046 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:11 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1154161364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:12.046 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:11 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2883449603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.091+0000 7f61fadb2640 1 -- 172.21.15.82:0/3168287832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4102b40 msgr2=0x7f61f4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.091+0000 7f61fadb2640 1 --2- 172.21.15.82:0/3168287832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4102b40 0x7f61f4106b80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009900 tx=0x7f61e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.091+0000 7f61fadb2640 1 -- 172.21.15.82:0/3168287832 shutdown_connections 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.091+0000 7f61fadb2640 1 --2- 172.21.15.82:0/3168287832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f4107260 0x7f61f410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.091+0000 7f61fadb2640 1 --2- 172.21.15.82:0/3168287832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4102b40 0x7f61f4106b80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.091+0000 7f61fadb2640 1 --2- 172.21.15.82:0/3168287832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f4102190 0x7f61f4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.091+0000 7f61fadb2640 1 -- 172.21.15.82:0/3168287832 >> 172.21.15.82:0/3168287832 conn(0x7f61f40793a0 msgr2=0x7f61f40797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.091+0000 7f61fadb2640 1 -- 172.21.15.82:0/3168287832 shutdown_connections 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.091+0000 7f61fadb2640 1 -- 172.21.15.82:0/3168287832 wait complete. 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.091+0000 7f61fadb2640 1 Processor -- start 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.091+0000 7f61fadb2640 1 -- start start 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61fadb2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4102190 0x7f61f419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61fadb2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f4107260 0x7f61f419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61fadb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f4198620 0x7f61f4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61fadb2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f61f41121a0 con 0x7f61f4102190 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61fadb2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f61f4112020 con 0x7f61f4198620 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61fadb2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61f4112320 con 0x7f61f4107260 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f8b4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4102190 0x7f61f419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f8b4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4102190 0x7f61f419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44842/0 (socket says 172.21.15.82:44842) 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f8b4e640 1 -- 172.21.15.82:0/1178775428 learned_addr learned my addr 172.21.15.82:0/1178775428 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f934f640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f4198620 0x7f61f4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f3fff640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f4107260 0x7f61f419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f8b4e640 1 -- 172.21.15.82:0/1178775428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f4107260 msgr2=0x7f61f419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f8b4e640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f4107260 0x7f61f419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f8b4e640 1 -- 172.21.15.82:0/1178775428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f4198620 msgr2=0x7f61f4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f8b4e640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f4198620 0x7f61f4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f8b4e640 1 -- 172.21.15.82:0/1178775428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f4199140 con 0x7f61f4102190 2024-04-21T08:47:12.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f3fff640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f4107260 0x7f61f419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:12.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f8b4e640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4102190 0x7f61f419e420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f61e800e9f0 tx=0x7f61e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:12.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f1ffb640 1 -- 172.21.15.82:0/1178775428 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e800cd80 con 0x7f61f4102190 2024-04-21T08:47:12.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f1ffb640 1 -- 172.21.15.82:0/1178775428 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61e80108e0 con 0x7f61f4102190 2024-04-21T08:47:12.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61f1ffb640 1 -- 172.21.15.82:0/1178775428 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8018980 con 0x7f61f4102190 2024-04-21T08:47:12.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.092+0000 7f61fadb2640 1 -- 172.21.15.82:0/1178775428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f4199430 con 0x7f61f4102190 2024-04-21T08:47:12.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.093+0000 7f61fadb2640 1 -- 172.21.15.82:0/1178775428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f41a5630 con 0x7f61f4102190 2024-04-21T08:47:12.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.094+0000 7f61fadb2640 1 -- 172.21.15.82:0/1178775428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f40691c0 con 0x7f61f4102190 2024-04-21T08:47:12.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.095+0000 7f61f1ffb640 1 -- 172.21.15.82:0/1178775428 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f61e8010430 con 0x7f61f4102190 2024-04-21T08:47:12.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.095+0000 7f61f1ffb640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f61e40a3e40 0x7f61e40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.095+0000 7f61f3fff640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f61e40a3e40 0x7f61e40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.095+0000 7f61f1ffb640 1 -- 172.21.15.82:0/1178775428 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f61e80d1200 con 0x7f61f4102190 2024-04-21T08:47:12.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.096+0000 7f61f3fff640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f61e40a3e40 0x7f61e40a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f61e0006010 tx=0x7f61e0005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:12.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.100+0000 7f61f1ffb640 1 -- 172.21.15.82:0/1178775428 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f61e809d290 con 0x7f61f4102190 2024-04-21T08:47:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.276+0000 7f61fadb2640 1 -- 172.21.15.82:0/1178775428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61f4102b40 con 0x7f61f4102190 2024-04-21T08:47:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.277+0000 7f61f1ffb640 1 -- 172.21.15.82:0/1178775428 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f61e809cb90 con 0x7f61f4102190 2024-04-21T08:47:12.277 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:47:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.279+0000 7f61fadb2640 1 -- 172.21.15.82:0/1178775428 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f61e40a3e40 msgr2=0x7f61e40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.279+0000 7f61fadb2640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f61e40a3e40 0x7f61e40a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f61e0006010 tx=0x7f61e0005a20 comp rx=0 tx=0).stop 2024-04-21T08:47:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.279+0000 7f61fadb2640 1 -- 172.21.15.82:0/1178775428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4102190 msgr2=0x7f61f419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.279+0000 7f61fadb2640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4102190 0x7f61f419e420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f61e800e9f0 tx=0x7f61e800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.279+0000 7f61fadb2640 1 -- 172.21.15.82:0/1178775428 shutdown_connections 2024-04-21T08:47:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.279+0000 7f61fadb2640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61f4198620 0x7f61f4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.279+0000 7f61fadb2640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61f4107260 0x7f61f419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.279+0000 7f61fadb2640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f61e40a3e40 0x7f61e40a6300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.279+0000 7f61fadb2640 1 --2- 172.21.15.82:0/1178775428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4102190 0x7f61f419e420 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.279+0000 7f61fadb2640 1 -- 172.21.15.82:0/1178775428 >> 172.21.15.82:0/1178775428 conn(0x7f61f40793a0 msgr2=0x7f61f4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.279+0000 7f61fadb2640 1 -- 172.21.15.82:0/1178775428 shutdown_connections 2024-04-21T08:47:12.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.279+0000 7f61fadb2640 1 -- 172.21.15.82:0/1178775428 wait complete. 2024-04-21T08:47:12.289 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:47:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.390+0000 7fe74e9db640 1 -- 172.21.15.82:0/3236550180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe748106170 msgr2=0x7fe74810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.390+0000 7fe74e9db640 1 --2- 172.21.15.82:0/3236550180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe748106170 0x7fe74810dd40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe740009ca0 tx=0x7fe74002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.390+0000 7fe74e9db640 1 -- 172.21.15.82:0/3236550180 shutdown_connections 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.390+0000 7fe74e9db640 1 --2- 172.21.15.82:0/3236550180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe748106170 0x7fe74810dd40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.390+0000 7fe74e9db640 1 --2- 172.21.15.82:0/3236550180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe748101a50 0x7fe748105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.390+0000 7fe74e9db640 1 --2- 172.21.15.82:0/3236550180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7481010a0 0x7fe748101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.390+0000 7fe74e9db640 1 -- 172.21.15.82:0/3236550180 >> 172.21.15.82:0/3236550180 conn(0x7fe748077eb0 msgr2=0x7fe7480782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.390+0000 7fe74e9db640 1 -- 172.21.15.82:0/3236550180 shutdown_connections 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.390+0000 7fe74e9db640 1 -- 172.21.15.82:0/3236550180 wait complete. 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe74e9db640 1 Processor -- start 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe74e9db640 1 -- start start 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe74e9db640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7481010a0 0x7fe748198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe74e9db640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe748106170 0x7fe748198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe74e9db640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74819cfa0 0x7fe74819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe74e9db640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7481121f0 con 0x7fe748106170 2024-04-21T08:47:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe74e9db640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe748112070 con 0x7fe74819cfa0 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe74e9db640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe748112370 con 0x7fe7481010a0 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe7477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe748106170 0x7fe748198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe7477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe748106170 0x7fe748198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44854/0 (socket says 172.21.15.82:44854) 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe7477fe640 1 -- 172.21.15.82:0/628089190 learned_addr learned my addr 172.21.15.82:0/628089190 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe74cf78640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74819cfa0 0x7fe74819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe747fff640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7481010a0 0x7fe748198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe7477fe640 1 -- 172.21.15.82:0/628089190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7481010a0 msgr2=0x7fe748198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe7477fe640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7481010a0 0x7fe748198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe7477fe640 1 -- 172.21.15.82:0/628089190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74819cfa0 msgr2=0x7fe74819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe7477fe640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74819cfa0 0x7fe74819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.391+0000 7fe7477fe640 1 -- 172.21.15.82:0/628089190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74819da90 con 0x7fe748106170 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.392+0000 7fe7477fe640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe748106170 0x7fe748198c70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00e9c0 tx=0x7fe73c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.392+0000 7fe7457fa640 1 -- 172.21.15.82:0/628089190 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c00cd80 con 0x7fe748106170 2024-04-21T08:47:12.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.392+0000 7fe7457fa640 1 -- 172.21.15.82:0/628089190 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe73c0054d0 con 0x7fe748106170 2024-04-21T08:47:12.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.392+0000 7fe74e9db640 1 -- 172.21.15.82:0/628089190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74819dd80 con 0x7fe748106170 2024-04-21T08:47:12.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.392+0000 7fe7457fa640 1 -- 172.21.15.82:0/628089190 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c010430 con 0x7fe748106170 2024-04-21T08:47:12.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.392+0000 7fe74e9db640 1 -- 172.21.15.82:0/628089190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7481a5570 con 0x7fe748106170 2024-04-21T08:47:12.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.393+0000 7fe74e9db640 1 -- 172.21.15.82:0/628089190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7480691c0 con 0x7fe748106170 2024-04-21T08:47:12.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.394+0000 7fe7457fa640 1 -- 172.21.15.82:0/628089190 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe73c004fd0 con 0x7fe748106170 2024-04-21T08:47:12.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.395+0000 7fe7457fa640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7300a3e40 0x7fe7300a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.395+0000 7fe747fff640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7300a3e40 0x7fe7300a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.395+0000 7fe7457fa640 1 -- 172.21.15.82:0/628089190 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe73c0d1290 con 0x7fe748106170 2024-04-21T08:47:12.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.395+0000 7fe747fff640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7300a3e40 0x7fe7300a6300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe738005fd0 tx=0x7fe738005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:12.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.399+0000 7fe7457fa640 1 -- 172.21.15.82:0/628089190 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe73c09d320 con 0x7fe748106170 2024-04-21T08:47:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.575+0000 7fe74e9db640 1 -- 172.21.15.82:0/628089190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe748101a50 con 0x7fe748106170 2024-04-21T08:47:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.575+0000 7fe7457fa640 1 -- 172.21.15.82:0/628089190 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fe73c09cc20 con 0x7fe748106170 2024-04-21T08:47:12.575 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:47:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.577+0000 7fe74e9db640 1 -- 172.21.15.82:0/628089190 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7300a3e40 msgr2=0x7fe7300a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.577+0000 7fe74e9db640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7300a3e40 0x7fe7300a6300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe738005fd0 tx=0x7fe738005850 comp rx=0 tx=0).stop 2024-04-21T08:47:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.577+0000 7fe74e9db640 1 -- 172.21.15.82:0/628089190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe748106170 msgr2=0x7fe748198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.577+0000 7fe74e9db640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe748106170 0x7fe748198c70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00e9c0 tx=0x7fe73c00ee80 comp rx=0 tx=0).stop 2024-04-21T08:47:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.577+0000 7fe74e9db640 1 -- 172.21.15.82:0/628089190 shutdown_connections 2024-04-21T08:47:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.577+0000 7fe74e9db640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe74819cfa0 0x7fe74819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.577+0000 7fe74e9db640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7300a3e40 0x7fe7300a6300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.577+0000 7fe74e9db640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe748106170 0x7fe748198c70 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.577+0000 7fe74e9db640 1 --2- 172.21.15.82:0/628089190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7481010a0 0x7fe748198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.577+0000 7fe74e9db640 1 -- 172.21.15.82:0/628089190 >> 172.21.15.82:0/628089190 conn(0x7fe748077eb0 msgr2=0x7fe7480fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.577+0000 7fe74e9db640 1 -- 172.21.15.82:0/628089190 shutdown_connections 2024-04-21T08:47:12.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.577+0000 7fe74e9db640 1 -- 172.21.15.82:0/628089190 wait complete. 2024-04-21T08:47:12.588 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:47:12.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.690+0000 7f54f1746640 1 -- 172.21.15.82:0/2767806730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54ec103b00 msgr2=0x7f54ec10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.690+0000 7f54f1746640 1 --2- 172.21.15.82:0/2767806730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54ec103b00 0x7f54ec10bfe0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009900 tx=0x7f54e002bd60 comp rx=0 tx=0).stop 2024-04-21T08:47:12.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.690+0000 7f54f1746640 1 -- 172.21.15.82:0/2767806730 shutdown_connections 2024-04-21T08:47:12.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.690+0000 7f54f1746640 1 --2- 172.21.15.82:0/2767806730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54ec10c520 0x7f54ec10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.690+0000 7f54f1746640 1 --2- 172.21.15.82:0/2767806730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54ec103b00 0x7f54ec10bfe0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.690+0000 7f54f1746640 1 --2- 172.21.15.82:0/2767806730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54ec1031e0 0x7f54ec1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.690+0000 7f54f1746640 1 -- 172.21.15.82:0/2767806730 >> 172.21.15.82:0/2767806730 conn(0x7f54ec0771d0 msgr2=0x7f54ec0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.690+0000 7f54f1746640 1 -- 172.21.15.82:0/2767806730 shutdown_connections 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.690+0000 7f54f1746640 1 -- 172.21.15.82:0/2767806730 wait complete. 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54f1746640 1 Processor -- start 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54f1746640 1 -- start start 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54f1746640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54ec1031e0 0x7f54ec19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54f1746640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54ec103b00 0x7f54ec19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54f1746640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54ec10c520 0x7f54ec198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54f1746640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec1121b0 con 0x7f54ec1031e0 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54f1746640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec112030 con 0x7f54ec10c520 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54f1746640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec112330 con 0x7f54ec103b00 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54eaffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54ec1031e0 0x7f54ec19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54eaffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54ec1031e0 0x7f54ec19e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56280/0 (socket says 172.21.15.82:56280) 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54eaffd640 1 -- 172.21.15.82:0/1039426540 learned_addr learned my addr 172.21.15.82:0/1039426540 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54eb7fe640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54ec10c520 0x7f54ec198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54eaffd640 1 -- 172.21.15.82:0/1039426540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54ec103b00 msgr2=0x7f54ec19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.691+0000 7f54eaffd640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54ec103b00 0x7f54ec19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.692+0000 7f54eaffd640 1 -- 172.21.15.82:0/1039426540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54ec10c520 msgr2=0x7f54ec198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.692+0000 7f54eaffd640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54ec10c520 0x7f54ec198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.692+0000 7f54eaffd640 1 -- 172.21.15.82:0/1039426540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ec198d10 con 0x7f54ec1031e0 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.692+0000 7f54eb7fe640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54ec10c520 0x7f54ec198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.692+0000 7f54eaffd640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54ec1031e0 0x7f54ec19e3e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00d950 tx=0x7f54dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.692+0000 7f54d3fff640 1 -- 172.21.15.82:0/1039426540 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc017030 con 0x7f54ec1031e0 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.692+0000 7f54d3fff640 1 -- 172.21.15.82:0/1039426540 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54dc004020 con 0x7f54ec1031e0 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.692+0000 7f54f1746640 1 -- 172.21.15.82:0/1039426540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ec199000 con 0x7f54ec1031e0 2024-04-21T08:47:12.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.692+0000 7f54d3fff640 1 -- 172.21.15.82:0/1039426540 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0047a0 con 0x7f54ec1031e0 2024-04-21T08:47:12.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.692+0000 7f54f1746640 1 -- 172.21.15.82:0/1039426540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ec1a52a0 con 0x7f54ec1031e0 2024-04-21T08:47:12.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.693+0000 7f54f1746640 1 -- 172.21.15.82:0/1039426540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54ec0691c0 con 0x7f54ec1031e0 2024-04-21T08:47:12.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.694+0000 7f54d3fff640 1 -- 172.21.15.82:0/1039426540 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f54dc009a30 con 0x7f54ec1031e0 2024-04-21T08:47:12.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.694+0000 7f54d3fff640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f54d40a3e40 0x7f54d40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.695+0000 7f54ea7fc640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f54d40a3e40 0x7f54d40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.695+0000 7f54d3fff640 1 -- 172.21.15.82:0/1039426540 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f54dc0d0980 con 0x7f54ec1031e0 2024-04-21T08:47:12.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.695+0000 7f54ea7fc640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f54d40a3e40 0x7f54d40a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f54e00096c0 tx=0x7f54e0000bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:12.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.699+0000 7f54d3fff640 1 -- 172.21.15.82:0/1039426540 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54dc09ca10 con 0x7f54ec1031e0 2024-04-21T08:47:12.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:12 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3670977086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:12.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:12 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3791743242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:12.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:12 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1178775428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:12.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:12 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/628089190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:12.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.875+0000 7f54f1746640 1 -- 172.21.15.82:0/1039426540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f54ec02d5c0 con 0x7f54ec1031e0 2024-04-21T08:47:12.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.876+0000 7f54d3fff640 1 -- 172.21.15.82:0/1039426540 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f54dc09c310 con 0x7f54ec1031e0 2024-04-21T08:47:12.876 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:47:12.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.878+0000 7f54f1746640 1 -- 172.21.15.82:0/1039426540 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f54d40a3e40 msgr2=0x7f54d40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.878+0000 7f54f1746640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f54d40a3e40 0x7f54d40a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f54e00096c0 tx=0x7f54e0000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.878+0000 7f54f1746640 1 -- 172.21.15.82:0/1039426540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54ec1031e0 msgr2=0x7f54ec19e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.878+0000 7f54f1746640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54ec1031e0 0x7f54ec19e3e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00d950 tx=0x7f54dc00de10 comp rx=0 tx=0).stop 2024-04-21T08:47:12.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.878+0000 7f54f1746640 1 -- 172.21.15.82:0/1039426540 shutdown_connections 2024-04-21T08:47:12.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.878+0000 7f54f1746640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54ec10c520 0x7f54ec198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.878+0000 7f54f1746640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54ec103b00 0x7f54ec19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.878+0000 7f54f1746640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f54d40a3e40 0x7f54d40a6300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.878+0000 7f54f1746640 1 --2- 172.21.15.82:0/1039426540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54ec1031e0 0x7f54ec19e3e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.878+0000 7f54f1746640 1 -- 172.21.15.82:0/1039426540 >> 172.21.15.82:0/1039426540 conn(0x7f54ec0771d0 msgr2=0x7f54ec0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:12.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.878+0000 7f54f1746640 1 -- 172.21.15.82:0/1039426540 shutdown_connections 2024-04-21T08:47:12.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.878+0000 7f54f1746640 1 -- 172.21.15.82:0/1039426540 wait complete. 2024-04-21T08:47:12.888 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:47:12.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:12 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3670977086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:12.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:12 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3791743242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:12.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:12 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1178775428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:12.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:12 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/628089190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:12.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.990+0000 7fe16c66d640 1 -- 172.21.15.82:0/404443755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe164102b40 msgr2=0x7fe164106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.990+0000 7fe16c66d640 1 --2- 172.21.15.82:0/404443755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe164102b40 0x7fe164106b80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe154009900 tx=0x7fe15402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:12.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.990+0000 7fe16c66d640 1 -- 172.21.15.82:0/404443755 shutdown_connections 2024-04-21T08:47:12.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.990+0000 7fe16c66d640 1 --2- 172.21.15.82:0/404443755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe164107260 0x7fe16410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.990+0000 7fe16c66d640 1 --2- 172.21.15.82:0/404443755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe164102b40 0x7fe164106b80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.990+0000 7fe16c66d640 1 --2- 172.21.15.82:0/404443755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe164102190 0x7fe164102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.990+0000 7fe16c66d640 1 -- 172.21.15.82:0/404443755 >> 172.21.15.82:0/404443755 conn(0x7fe1640fd990 msgr2=0x7fe1640ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:12.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.990+0000 7fe16c66d640 1 -- 172.21.15.82:0/404443755 shutdown_connections 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.990+0000 7fe16c66d640 1 -- 172.21.15.82:0/404443755 wait complete. 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16c66d640 1 Processor -- start 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16c66d640 1 -- start start 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16c66d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe164102190 0x7fe16419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16c66d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe164107260 0x7fe16419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16c66d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe164198590 0x7fe164198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16c66d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641121f0 con 0x7fe164198590 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16c66d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe164112070 con 0x7fe164102190 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16c66d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe164112370 con 0x7fe164107260 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16ac0a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe164198590 0x7fe164198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16ac0a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe164198590 0x7fe164198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56298/0 (socket says 172.21.15.82:56298) 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16ac0a640 1 -- 172.21.15.82:0/537925038 learned_addr learned my addr 172.21.15.82:0/537925038 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe169c08640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe164107260 0x7fe16419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16a409640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe164102190 0x7fe16419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16ac0a640 1 -- 172.21.15.82:0/537925038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe164107260 msgr2=0x7fe16419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16ac0a640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe164107260 0x7fe16419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16ac0a640 1 -- 172.21.15.82:0/537925038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe164102190 msgr2=0x7fe16419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:12.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16ac0a640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe164102190 0x7fe16419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:12.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.991+0000 7fe16ac0a640 1 -- 172.21.15.82:0/537925038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1641990b0 con 0x7fe164198590 2024-04-21T08:47:12.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.992+0000 7fe16ac0a640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe164198590 0x7fe164198a40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00eaa0 tx=0x7fe15c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:12.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.992+0000 7fe15b7fe640 1 -- 172.21.15.82:0/537925038 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00cdb0 con 0x7fe164198590 2024-04-21T08:47:12.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.992+0000 7fe16c66d640 1 -- 172.21.15.82:0/537925038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1641993a0 con 0x7fe164198590 2024-04-21T08:47:12.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.992+0000 7fe15b7fe640 1 -- 172.21.15.82:0/537925038 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe15c005020 con 0x7fe164198590 2024-04-21T08:47:12.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.992+0000 7fe15b7fe640 1 -- 172.21.15.82:0/537925038 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0116d0 con 0x7fe164198590 2024-04-21T08:47:12.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.992+0000 7fe16c66d640 1 -- 172.21.15.82:0/537925038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1641a55f0 con 0x7fe164198590 2024-04-21T08:47:12.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.993+0000 7fe16c66d640 1 -- 172.21.15.82:0/537925038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1640691c0 con 0x7fe164198590 2024-04-21T08:47:12.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.994+0000 7fe15b7fe640 1 -- 172.21.15.82:0/537925038 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe15c0054a0 con 0x7fe164198590 2024-04-21T08:47:12.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.994+0000 7fe15b7fe640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe1500a3e40 0x7fe1500a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:12.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.994+0000 7fe16a409640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe1500a3e40 0x7fe1500a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:12.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.994+0000 7fe15b7fe640 1 -- 172.21.15.82:0/537925038 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe15c0d12c0 con 0x7fe164198590 2024-04-21T08:47:12.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.995+0000 7fe16a409640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe1500a3e40 0x7fe1500a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe160005d00 tx=0x7fe16000b410 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:12.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:12.998+0000 7fe15b7fe640 1 -- 172.21.15.82:0/537925038 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe15c016030 con 0x7fe164198590 2024-04-21T08:47:13.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:12 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3670977086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:13.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:12 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3791743242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:13.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:12 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1178775428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:13.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:12 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/628089190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:13.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.175+0000 7fe16c66d640 1 -- 172.21.15.82:0/537925038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe164102b40 con 0x7fe164198590 2024-04-21T08:47:13.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.175+0000 7fe15b7fe640 1 -- 172.21.15.82:0/537925038 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fe15c09cd00 con 0x7fe164198590 2024-04-21T08:47:13.175 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:47:13.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.177+0000 7fe16c66d640 1 -- 172.21.15.82:0/537925038 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe1500a3e40 msgr2=0x7fe1500a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.177+0000 7fe16c66d640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe1500a3e40 0x7fe1500a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe160005d00 tx=0x7fe16000b410 comp rx=0 tx=0).stop 2024-04-21T08:47:13.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.177+0000 7fe16c66d640 1 -- 172.21.15.82:0/537925038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe164198590 msgr2=0x7fe164198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.177+0000 7fe16c66d640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe164198590 0x7fe164198a40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00eaa0 tx=0x7fe15c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:47:13.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.177+0000 7fe16c66d640 1 -- 172.21.15.82:0/537925038 shutdown_connections 2024-04-21T08:47:13.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.177+0000 7fe16c66d640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe1500a3e40 0x7fe1500a6300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.177+0000 7fe16c66d640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe164198590 0x7fe164198a40 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.177+0000 7fe16c66d640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe164107260 0x7fe16419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.177+0000 7fe16c66d640 1 --2- 172.21.15.82:0/537925038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe164102190 0x7fe16419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.177+0000 7fe16c66d640 1 -- 172.21.15.82:0/537925038 >> 172.21.15.82:0/537925038 conn(0x7fe1640fd990 msgr2=0x7fe1640fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:13.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.178+0000 7fe16c66d640 1 -- 172.21.15.82:0/537925038 shutdown_connections 2024-04-21T08:47:13.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.178+0000 7fe16c66d640 1 -- 172.21.15.82:0/537925038 wait complete. 2024-04-21T08:47:13.188 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-21T08:47:13.188 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:47:13.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.286+0000 7f1cd6a11640 1 -- 172.21.15.82:0/559211248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd006b2a0 msgr2=0x7f1cd0106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.286+0000 7f1cd6a11640 1 --2- 172.21.15.82:0/559211248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd006b2a0 0x7f1cd0106bc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009900 tx=0x7f1cc002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:13.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.286+0000 7f1cd6a11640 1 -- 172.21.15.82:0/559211248 shutdown_connections 2024-04-21T08:47:13.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.286+0000 7f1cd6a11640 1 --2- 172.21.15.82:0/559211248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cd01072f0 0x7f1cd010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.286+0000 7f1cd6a11640 1 --2- 172.21.15.82:0/559211248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd006b2a0 0x7f1cd0106bc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.286+0000 7f1cd6a11640 1 --2- 172.21.15.82:0/559211248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd006a8f0 0x7f1cd006acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.286+0000 7f1cd6a11640 1 -- 172.21.15.82:0/559211248 >> 172.21.15.82:0/559211248 conn(0x7f1cd0077e30 msgr2=0x7f1cd0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:13.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.286+0000 7f1cd6a11640 1 -- 172.21.15.82:0/559211248 shutdown_connections 2024-04-21T08:47:13.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.286+0000 7f1cd6a11640 1 -- 172.21.15.82:0/559211248 wait complete. 2024-04-21T08:47:13.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.286+0000 7f1cd6a11640 1 Processor -- start 2024-04-21T08:47:13.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1cd6a11640 1 -- start start 2024-04-21T08:47:13.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1cd6a11640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cd006a8f0 0x7f1cd019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1cd6a11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd01072f0 0x7f1cd019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1cd6a11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd01985a0 0x7f1cd0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1cd6a11640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd01121c0 con 0x7f1cd01072f0 2024-04-21T08:47:13.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1cd6a11640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0112040 con 0x7f1cd006a8f0 2024-04-21T08:47:13.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1cd6a11640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0112340 con 0x7f1cd01985a0 2024-04-21T08:47:13.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1ccf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd01072f0 0x7f1cd019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1ccf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd01072f0 0x7f1cd019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56306/0 (socket says 172.21.15.82:56306) 2024-04-21T08:47:13.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1ccf7fe640 1 -- 172.21.15.82:0/3621719785 learned_addr learned my addr 172.21.15.82:0/3621719785 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:13.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1cd4fae640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd01985a0 0x7f1cd0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1ccffff640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cd006a8f0 0x7f1cd019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1ccf7fe640 1 -- 172.21.15.82:0/3621719785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd01985a0 msgr2=0x7f1cd0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1ccf7fe640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd01985a0 0x7f1cd0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1ccf7fe640 1 -- 172.21.15.82:0/3621719785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cd006a8f0 msgr2=0x7f1cd019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1ccf7fe640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cd006a8f0 0x7f1cd019e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1ccf7fe640 1 -- 172.21.15.82:0/3621719785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd01990c0 con 0x7f1cd01072f0 2024-04-21T08:47:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1cd4fae640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd01985a0 0x7f1cd0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.287+0000 7f1ccf7fe640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd01072f0 0x7f1cd019e930 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0036040 tx=0x7f1cc002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.288+0000 7f1ccd7fa640 1 -- 172.21.15.82:0/3621719785 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0043020 con 0x7f1cd01072f0 2024-04-21T08:47:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.288+0000 7f1ccd7fa640 1 -- 172.21.15.82:0/3621719785 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc002cbe0 con 0x7f1cd01072f0 2024-04-21T08:47:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.288+0000 7f1cd6a11640 1 -- 172.21.15.82:0/3621719785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd0199350 con 0x7f1cd01072f0 2024-04-21T08:47:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.288+0000 7f1ccd7fa640 1 -- 172.21.15.82:0/3621719785 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0003c30 con 0x7f1cd01072f0 2024-04-21T08:47:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.288+0000 7f1cd6a11640 1 -- 172.21.15.82:0/3621719785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd01a55a0 con 0x7f1cd01072f0 2024-04-21T08:47:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.289+0000 7f1cd6a11640 1 -- 172.21.15.82:0/3621719785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd0069230 con 0x7f1cd01072f0 2024-04-21T08:47:13.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.290+0000 7f1ccd7fa640 1 -- 172.21.15.82:0/3621719785 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f1cc0046020 con 0x7f1cd01072f0 2024-04-21T08:47:13.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.291+0000 7f1ccd7fa640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1cb80a3e40 0x7f1cb80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.291+0000 7f1ccffff640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1cb80a3e40 0x7f1cb80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.291+0000 7f1ccd7fa640 1 -- 172.21.15.82:0/3621719785 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1cc00f3070 con 0x7f1cd01072f0 2024-04-21T08:47:13.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.291+0000 7f1ccffff640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1cb80a3e40 0x7f1cb80a6300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4005fd0 tx=0x7f1cc4005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:13.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.295+0000 7f1ccd7fa640 1 -- 172.21.15.82:0/3621719785 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1cc004d020 con 0x7f1cd01072f0 2024-04-21T08:47:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.468+0000 7f1cd6a11640 1 -- 172.21.15.82:0/3621719785 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1cd006b2a0 con 0x7f1cb80a3e40 2024-04-21T08:47:13.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.478+0000 7f1ccd7fa640 1 -- 172.21.15.82:0/3621719785 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339453 (secure 0 0 0) 0x7f1cd006b2a0 con 0x7f1cb80a3e40 2024-04-21T08:47:13.481 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:13.483 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-21T08:47:13.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.481+0000 7f1cd6a11640 1 -- 172.21.15.82:0/3621719785 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1cb80a3e40 msgr2=0x7f1cb80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.481+0000 7f1cd6a11640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1cb80a3e40 0x7f1cb80a6300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4005fd0 tx=0x7f1cc4005850 comp rx=0 tx=0).stop 2024-04-21T08:47:13.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.481+0000 7f1cd6a11640 1 -- 172.21.15.82:0/3621719785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd01072f0 msgr2=0x7f1cd019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.481+0000 7f1cd6a11640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd01072f0 0x7f1cd019e930 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0036040 tx=0x7f1cc002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.481+0000 7f1cd6a11640 1 -- 172.21.15.82:0/3621719785 shutdown_connections 2024-04-21T08:47:13.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.481+0000 7f1cd6a11640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd01985a0 0x7f1cd0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.481+0000 7f1cd6a11640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1cb80a3e40 0x7f1cb80a6300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.481+0000 7f1cd6a11640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cd01072f0 0x7f1cd019e930 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.481+0000 7f1cd6a11640 1 --2- 172.21.15.82:0/3621719785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cd006a8f0 0x7f1cd019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.481+0000 7f1cd6a11640 1 -- 172.21.15.82:0/3621719785 >> 172.21.15.82:0/3621719785 conn(0x7f1cd0077e30 msgr2=0x7f1cd00ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:13.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.481+0000 7f1cd6a11640 1 -- 172.21.15.82:0/3621719785 shutdown_connections 2024-04-21T08:47:13.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.481+0000 7f1cd6a11640 1 -- 172.21.15.82:0/3621719785 wait complete. 2024-04-21T08:47:13.492 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-04-21T08:47:12.635896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1392247,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1943,"num_read_kb":1407,"num_write":708,"num_write_kb":5328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1035,"ondisk_log_size":1035,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358168,"kb_used_data":16952,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124583656,"statfs":{"total":1151940427776,"available":1151573663744,"internally_reserved":0,"allocated":17358848,"data_stored":10509083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200716,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":461,"num_read_kb":500,"num_write":153,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.987572"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748276+0000","last_change":"2024-04-21T08:44:34.494393+0000","last_active":"2024-04-21T08:47:00.748276+0000","last_peered":"2024-04-21T08:47:00.748276+0000","last_clean":"2024-04-21T08:47:00.748276+0000","last_became_active":"2024-04-21T08:44:34.493583+0000","last_became_peered":"2024-04-21T08:44:34.493583+0000","last_unstale":"2024-04-21T08:47:00.748276+0000","last_undegraded":"2024-04-21T08:47:00.748276+0000","last_fullsized":"2024-04-21T08:47:00.748276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:37.211676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747826+0000","last_change":"2024-04-21T08:44:34.496291+0000","last_active":"2024-04-21T08:47:00.747826+0000","last_peered":"2024-04-21T08:47:00.747826+0000","last_clean":"2024-04-21T08:47:00.747826+0000","last_became_active":"2024-04-21T08:44:34.495019+0000","last_became_peered":"2024-04-21T08:44:34.495019+0000","last_unstale":"2024-04-21T08:47:00.747826+0000","last_undegraded":"2024-04-21T08:47:00.747826+0000","last_fullsized":"2024-04-21T08:47:00.747826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:05.309847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747464+0000","last_change":"2024-04-21T08:44:34.497948+0000","last_active":"2024-04-21T08:47:00.747464+0000","last_peered":"2024-04-21T08:47:00.747464+0000","last_clean":"2024-04-21T08:47:00.747464+0000","last_became_active":"2024-04-21T08:44:34.497657+0000","last_became_peered":"2024-04-21T08:44:34.497657+0000","last_unstale":"2024-04-21T08:47:00.747464+0000","last_undegraded":"2024-04-21T08:47:00.747464+0000","last_fullsized":"2024-04-21T08:47:00.747464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:11.463715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748377+0000","last_change":"2024-04-21T08:44:34.492682+0000","last_active":"2024-04-21T08:47:00.748377+0000","last_peered":"2024-04-21T08:47:00.748377+0000","last_clean":"2024-04-21T08:47:00.748377+0000","last_became_active":"2024-04-21T08:44:34.492230+0000","last_became_peered":"2024-04-21T08:44:34.492230+0000","last_unstale":"2024-04-21T08:47:00.748377+0000","last_undegraded":"2024-04-21T08:47:00.748377+0000","last_fullsized":"2024-04-21T08:47:00.748377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:40.365608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721373+0000","last_change":"2024-04-21T08:44:34.492160+0000","last_active":"2024-04-21T08:47:00.721373+0000","last_peered":"2024-04-21T08:47:00.721373+0000","last_clean":"2024-04-21T08:47:00.721373+0000","last_became_active":"2024-04-21T08:44:34.491701+0000","last_became_peered":"2024-04-21T08:44:34.491701+0000","last_unstale":"2024-04-21T08:47:00.721373+0000","last_undegraded":"2024-04-21T08:47:00.721373+0000","last_fullsized":"2024-04-21T08:47:00.721373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:50.542661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626584+0000","last_change":"2024-04-21T08:44:34.497341+0000","last_active":"2024-04-21T08:47:00.626584+0000","last_peered":"2024-04-21T08:47:00.626584+0000","last_clean":"2024-04-21T08:47:00.626584+0000","last_became_active":"2024-04-21T08:44:34.495794+0000","last_became_peered":"2024-04-21T08:44:34.495794+0000","last_unstale":"2024-04-21T08:47:00.626584+0000","last_undegraded":"2024-04-21T08:47:00.626584+0000","last_fullsized":"2024-04-21T08:47:00.626584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:13.854753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748527+0000","last_change":"2024-04-21T08:44:34.501578+0000","last_active":"2024-04-21T08:47:00.748527+0000","last_peered":"2024-04-21T08:47:00.748527+0000","last_clean":"2024-04-21T08:47:00.748527+0000","last_became_active":"2024-04-21T08:44:34.500848+0000","last_became_peered":"2024-04-21T08:44:34.500848+0000","last_unstale":"2024-04-21T08:47:00.748527+0000","last_undegraded":"2024-04-21T08:47:00.748527+0000","last_fullsized":"2024-04-21T08:47:00.748527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:53:27.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.621951+0000","last_change":"2024-04-21T08:44:34.502975+0000","last_active":"2024-04-21T08:47:00.621951+0000","last_peered":"2024-04-21T08:47:00.621951+0000","last_clean":"2024-04-21T08:47:00.621951+0000","last_became_active":"2024-04-21T08:44:34.502676+0000","last_became_peered":"2024-04-21T08:44:34.502676+0000","last_unstale":"2024-04-21T08:47:00.621951+0000","last_undegraded":"2024-04-21T08:47:00.621951+0000","last_fullsized":"2024-04-21T08:47:00.621951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:32:13.881603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748293+0000","last_change":"2024-04-21T08:44:34.495703+0000","last_active":"2024-04-21T08:47:00.748293+0000","last_peered":"2024-04-21T08:47:00.748293+0000","last_clean":"2024-04-21T08:47:00.748293+0000","last_became_active":"2024-04-21T08:44:34.495380+0000","last_became_peered":"2024-04-21T08:44:34.495380+0000","last_unstale":"2024-04-21T08:47:00.748293+0000","last_undegraded":"2024-04-21T08:47:00.748293+0000","last_fullsized":"2024-04-21T08:47:00.748293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:45:28.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626928+0000","last_change":"2024-04-21T08:44:34.499361+0000","last_active":"2024-04-21T08:47:00.626928+0000","last_peered":"2024-04-21T08:47:00.626928+0000","last_clean":"2024-04-21T08:47:00.626928+0000","last_became_active":"2024-04-21T08:44:34.499044+0000","last_became_peered":"2024-04-21T08:44:34.499044+0000","last_unstale":"2024-04-21T08:47:00.626928+0000","last_undegraded":"2024-04-21T08:47:00.626928+0000","last_fullsized":"2024-04-21T08:47:00.626928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:12.118985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748894+0000","last_change":"2024-04-21T08:44:34.499631+0000","last_active":"2024-04-21T08:47:00.748894+0000","last_peered":"2024-04-21T08:47:00.748894+0000","last_clean":"2024-04-21T08:47:00.748894+0000","last_became_active":"2024-04-21T08:44:34.499316+0000","last_became_peered":"2024-04-21T08:44:34.499316+0000","last_unstale":"2024-04-21T08:47:00.748894+0000","last_undegraded":"2024-04-21T08:47:00.748894+0000","last_fullsized":"2024-04-21T08:47:00.748894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:26.123017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748326+0000","last_change":"2024-04-21T08:44:34.504857+0000","last_active":"2024-04-21T08:47:00.748326+0000","last_peered":"2024-04-21T08:47:00.748326+0000","last_clean":"2024-04-21T08:47:00.748326+0000","last_became_active":"2024-04-21T08:44:34.504564+0000","last_became_peered":"2024-04-21T08:44:34.504564+0000","last_unstale":"2024-04-21T08:47:00.748326+0000","last_undegraded":"2024-04-21T08:47:00.748326+0000","last_fullsized":"2024-04-21T08:47:00.748326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:32:05.216162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874684+0000","last_change":"2024-04-21T08:44:34.484074+0000","last_active":"2024-04-21T08:47:00.874684+0000","last_peered":"2024-04-21T08:47:00.874684+0000","last_clean":"2024-04-21T08:47:00.874684+0000","last_became_active":"2024-04-21T08:44:34.483309+0000","last_became_peered":"2024-04-21T08:44:34.483309+0000","last_unstale":"2024-04-21T08:47:00.874684+0000","last_undegraded":"2024-04-21T08:47:00.874684+0000","last_fullsized":"2024-04-21T08:47:00.874684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:39.686048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747340+0000","last_change":"2024-04-21T08:44:34.499159+0000","last_active":"2024-04-21T08:47:00.747340+0000","last_peered":"2024-04-21T08:47:00.747340+0000","last_clean":"2024-04-21T08:47:00.747340+0000","last_became_active":"2024-04-21T08:44:34.496609+0000","last_became_peered":"2024-04-21T08:44:34.496609+0000","last_unstale":"2024-04-21T08:47:00.747340+0000","last_undegraded":"2024-04-21T08:47:00.747340+0000","last_fullsized":"2024-04-21T08:47:00.747340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:11.716321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747233+0000","last_change":"2024-04-21T08:45:27.406543+0000","last_active":"2024-04-21T08:47:00.747233+0000","last_peered":"2024-04-21T08:47:00.747233+0000","last_clean":"2024-04-21T08:47:00.747233+0000","last_became_active":"2024-04-21T08:45:27.406190+0000","last_became_peered":"2024-04-21T08:45:27.406190+0000","last_unstale":"2024-04-21T08:47:00.747233+0000","last_undegraded":"2024-04-21T08:47:00.747233+0000","last_fullsized":"2024-04-21T08:47:00.747233+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:56:17.760163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747327+0000","last_change":"2024-04-21T08:44:34.494018+0000","last_active":"2024-04-21T08:47:00.747327+0000","last_peered":"2024-04-21T08:47:00.747327+0000","last_clean":"2024-04-21T08:47:00.747327+0000","last_became_active":"2024-04-21T08:44:34.493692+0000","last_became_peered":"2024-04-21T08:44:34.493692+0000","last_unstale":"2024-04-21T08:47:00.747327+0000","last_undegraded":"2024-04-21T08:47:00.747327+0000","last_fullsized":"2024-04-21T08:47:00.747327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:25:16.944160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748187+0000","last_change":"2024-04-21T08:44:34.493231+0000","last_active":"2024-04-21T08:47:00.748187+0000","last_peered":"2024-04-21T08:47:00.748187+0000","last_clean":"2024-04-21T08:47:00.748187+0000","last_became_active":"2024-04-21T08:44:34.492826+0000","last_became_peered":"2024-04-21T08:44:34.492826+0000","last_unstale":"2024-04-21T08:47:00.748187+0000","last_undegraded":"2024-04-21T08:47:00.748187+0000","last_fullsized":"2024-04-21T08:47:00.748187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:36.815012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719352+0000","last_change":"2024-04-21T08:44:34.495142+0000","last_active":"2024-04-21T08:47:00.719352+0000","last_peered":"2024-04-21T08:47:00.719352+0000","last_clean":"2024-04-21T08:47:00.719352+0000","last_became_active":"2024-04-21T08:44:34.494803+0000","last_became_peered":"2024-04-21T08:44:34.494803+0000","last_unstale":"2024-04-21T08:47:00.719352+0000","last_undegraded":"2024-04-21T08:47:00.719352+0000","last_fullsized":"2024-04-21T08:47:00.719352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:22.974053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627186+0000","last_change":"2024-04-21T08:44:34.491252+0000","last_active":"2024-04-21T08:47:00.627186+0000","last_peered":"2024-04-21T08:47:00.627186+0000","last_clean":"2024-04-21T08:47:00.627186+0000","last_became_active":"2024-04-21T08:44:34.490814+0000","last_became_peered":"2024-04-21T08:44:34.490814+0000","last_unstale":"2024-04-21T08:47:00.627186+0000","last_undegraded":"2024-04-21T08:47:00.627186+0000","last_fullsized":"2024-04-21T08:47:00.627186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:54:52.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626918+0000","last_change":"2024-04-21T08:44:38.518647+0000","last_active":"2024-04-21T08:47:00.626918+0000","last_peered":"2024-04-21T08:47:00.626918+0000","last_clean":"2024-04-21T08:47:00.626918+0000","last_became_active":"2024-04-21T08:44:38.518239+0000","last_became_peered":"2024-04-21T08:44:38.518239+0000","last_unstale":"2024-04-21T08:47:00.626918+0000","last_undegraded":"2024-04-21T08:47:00.626918+0000","last_fullsized":"2024-04-21T08:47:00.626918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:44.781542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":178,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.174380+0000","last_change":"2024-04-21T08:44:33.484484+0000","last_active":"2024-04-21T08:47:10.174380+0000","last_peered":"2024-04-21T08:47:10.174380+0000","last_clean":"2024-04-21T08:47:10.174380+0000","last_became_active":"2024-04-21T08:44:33.484160+0000","last_became_peered":"2024-04-21T08:44:33.484160+0000","last_unstale":"2024-04-21T08:47:10.174380+0000","last_undegraded":"2024-04-21T08:47:10.174380+0000","last_fullsized":"2024-04-21T08:47:10.174380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:45.962598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718920+0000","last_change":"2024-04-21T08:46:27.484947+0000","last_active":"2024-04-21T08:47:00.718920+0000","last_peered":"2024-04-21T08:47:00.718920+0000","last_clean":"2024-04-21T08:47:00.718920+0000","last_became_active":"2024-04-21T08:46:27.484530+0000","last_became_peered":"2024-04-21T08:46:27.484530+0000","last_unstale":"2024-04-21T08:47:00.718920+0000","last_undegraded":"2024-04-21T08:47:00.718920+0000","last_fullsized":"2024-04-21T08:47:00.718920+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:59.248176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747775+0000","last_change":"2024-04-21T08:45:27.518006+0000","last_active":"2024-04-21T08:47:00.747775+0000","last_peered":"2024-04-21T08:47:00.747775+0000","last_clean":"2024-04-21T08:47:00.747775+0000","last_became_active":"2024-04-21T08:45:27.517409+0000","last_became_peered":"2024-04-21T08:45:27.517409+0000","last_unstale":"2024-04-21T08:47:00.747775+0000","last_undegraded":"2024-04-21T08:47:00.747775+0000","last_fullsized":"2024-04-21T08:47:00.747775+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:45:01.253587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623343+0000","last_change":"2024-04-21T08:44:33.490508+0000","last_active":"2024-04-21T08:47:00.623343+0000","last_peered":"2024-04-21T08:47:00.623343+0000","last_clean":"2024-04-21T08:47:00.623343+0000","last_became_active":"2024-04-21T08:44:33.490084+0000","last_became_peered":"2024-04-21T08:44:33.490084+0000","last_unstale":"2024-04-21T08:47:00.623343+0000","last_undegraded":"2024-04-21T08:47:00.623343+0000","last_fullsized":"2024-04-21T08:47:00.623343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:21.553435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747770+0000","last_change":"2024-04-21T08:44:34.491843+0000","last_active":"2024-04-21T08:47:00.747770+0000","last_peered":"2024-04-21T08:47:00.747770+0000","last_clean":"2024-04-21T08:47:00.747770+0000","last_became_active":"2024-04-21T08:44:34.490578+0000","last_became_peered":"2024-04-21T08:44:34.490578+0000","last_unstale":"2024-04-21T08:47:00.747770+0000","last_undegraded":"2024-04-21T08:47:00.747770+0000","last_fullsized":"2024-04-21T08:47:00.747770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:05:12.272330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748084+0000","last_change":"2024-04-21T08:44:38.520258+0000","last_active":"2024-04-21T08:47:00.748084+0000","last_peered":"2024-04-21T08:47:00.748084+0000","last_clean":"2024-04-21T08:47:00.748084+0000","last_became_active":"2024-04-21T08:44:38.519824+0000","last_became_peered":"2024-04-21T08:44:38.519824+0000","last_unstale":"2024-04-21T08:47:00.748084+0000","last_undegraded":"2024-04-21T08:47:00.748084+0000","last_fullsized":"2024-04-21T08:47:00.748084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:39.716745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721810+0000","last_change":"2024-04-21T08:44:33.484934+0000","last_active":"2024-04-21T08:47:00.721810+0000","last_peered":"2024-04-21T08:47:00.721810+0000","last_clean":"2024-04-21T08:47:00.721810+0000","last_became_active":"2024-04-21T08:44:33.484591+0000","last_became_peered":"2024-04-21T08:44:33.484591+0000","last_unstale":"2024-04-21T08:47:00.721810+0000","last_undegraded":"2024-04-21T08:47:00.721810+0000","last_fullsized":"2024-04-21T08:47:00.721810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:48.826611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747675+0000","last_change":"2024-04-21T08:44:34.501109+0000","last_active":"2024-04-21T08:47:00.747675+0000","last_peered":"2024-04-21T08:47:00.747675+0000","last_clean":"2024-04-21T08:47:00.747675+0000","last_became_active":"2024-04-21T08:44:34.500750+0000","last_became_peered":"2024-04-21T08:44:34.500750+0000","last_unstale":"2024-04-21T08:47:00.747675+0000","last_undegraded":"2024-04-21T08:47:00.747675+0000","last_fullsized":"2024-04-21T08:47:00.747675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:29:28.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747855+0000","last_change":"2024-04-21T08:44:38.519633+0000","last_active":"2024-04-21T08:47:00.747855+0000","last_peered":"2024-04-21T08:47:00.747855+0000","last_clean":"2024-04-21T08:47:00.747855+0000","last_became_active":"2024-04-21T08:44:38.519215+0000","last_became_peered":"2024-04-21T08:44:38.519215+0000","last_unstale":"2024-04-21T08:47:00.747855+0000","last_undegraded":"2024-04-21T08:47:00.747855+0000","last_fullsized":"2024-04-21T08:47:00.747855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:43:59.695844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721745+0000","last_change":"2024-04-21T08:44:33.483267+0000","last_active":"2024-04-21T08:47:00.721745+0000","last_peered":"2024-04-21T08:47:00.721745+0000","last_clean":"2024-04-21T08:47:00.721745+0000","last_became_active":"2024-04-21T08:44:33.482932+0000","last_became_peered":"2024-04-21T08:44:33.482932+0000","last_unstale":"2024-04-21T08:47:00.721745+0000","last_undegraded":"2024-04-21T08:47:00.721745+0000","last_fullsized":"2024-04-21T08:47:00.721745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:53.311550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729284+0000","last_change":"2024-04-21T08:44:34.497389+0000","last_active":"2024-04-21T08:47:00.729284+0000","last_peered":"2024-04-21T08:47:00.729284+0000","last_clean":"2024-04-21T08:47:00.729284+0000","last_became_active":"2024-04-21T08:44:34.497030+0000","last_became_peered":"2024-04-21T08:44:34.497030+0000","last_unstale":"2024-04-21T08:47:00.729284+0000","last_undegraded":"2024-04-21T08:47:00.729284+0000","last_fullsized":"2024-04-21T08:47:00.729284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:20.371888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626966+0000","last_change":"2024-04-21T08:44:38.518095+0000","last_active":"2024-04-21T08:47:00.626966+0000","last_peered":"2024-04-21T08:47:00.626966+0000","last_clean":"2024-04-21T08:47:00.626966+0000","last_became_active":"2024-04-21T08:44:38.517010+0000","last_became_peered":"2024-04-21T08:44:38.517010+0000","last_unstale":"2024-04-21T08:47:00.626966+0000","last_undegraded":"2024-04-21T08:47:00.626966+0000","last_fullsized":"2024-04-21T08:47:00.626966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:02.379512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":180,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.182251+0000","last_change":"2024-04-21T08:44:33.487999+0000","last_active":"2024-04-21T08:47:10.182251+0000","last_peered":"2024-04-21T08:47:10.182251+0000","last_clean":"2024-04-21T08:47:10.182251+0000","last_became_active":"2024-04-21T08:44:33.487666+0000","last_became_peered":"2024-04-21T08:44:33.487666+0000","last_unstale":"2024-04-21T08:47:10.182251+0000","last_undegraded":"2024-04-21T08:47:10.182251+0000","last_fullsized":"2024-04-21T08:47:10.182251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:01.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2103,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721554+0000","last_change":"2024-04-21T08:44:34.494213+0000","last_active":"2024-04-21T08:47:00.721554+0000","last_peered":"2024-04-21T08:47:00.721554+0000","last_clean":"2024-04-21T08:47:00.721554+0000","last_became_active":"2024-04-21T08:44:34.493853+0000","last_became_peered":"2024-04-21T08:44:34.493853+0000","last_unstale":"2024-04-21T08:47:00.721554+0000","last_undegraded":"2024-04-21T08:47:00.721554+0000","last_fullsized":"2024-04-21T08:47:00.721554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:42.499131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627058+0000","last_change":"2024-04-21T08:45:27.410579+0000","last_active":"2024-04-21T08:47:00.627058+0000","last_peered":"2024-04-21T08:47:00.627058+0000","last_clean":"2024-04-21T08:47:00.627058+0000","last_became_active":"2024-04-21T08:45:27.409929+0000","last_became_peered":"2024-04-21T08:45:27.409929+0000","last_unstale":"2024-04-21T08:47:00.627058+0000","last_undegraded":"2024-04-21T08:47:00.627058+0000","last_fullsized":"2024-04-21T08:47:00.627058+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:35.897482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626508+0000","last_change":"2024-04-21T08:44:33.488751+0000","last_active":"2024-04-21T08:47:00.626508+0000","last_peered":"2024-04-21T08:47:00.626508+0000","last_clean":"2024-04-21T08:47:00.626508+0000","last_became_active":"2024-04-21T08:44:33.488425+0000","last_became_peered":"2024-04-21T08:44:33.488425+0000","last_unstale":"2024-04-21T08:47:00.626508+0000","last_undegraded":"2024-04-21T08:47:00.626508+0000","last_fullsized":"2024-04-21T08:47:00.626508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:28:47.868814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729150+0000","last_change":"2024-04-21T08:44:34.492770+0000","last_active":"2024-04-21T08:47:00.729150+0000","last_peered":"2024-04-21T08:47:00.729150+0000","last_clean":"2024-04-21T08:47:00.729150+0000","last_became_active":"2024-04-21T08:44:34.491340+0000","last_became_peered":"2024-04-21T08:44:34.491340+0000","last_unstale":"2024-04-21T08:47:00.729150+0000","last_undegraded":"2024-04-21T08:47:00.729150+0000","last_fullsized":"2024-04-21T08:47:00.729150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:47:54.280990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748097+0000","last_change":"2024-04-21T08:44:38.518888+0000","last_active":"2024-04-21T08:47:00.748097+0000","last_peered":"2024-04-21T08:47:00.748097+0000","last_clean":"2024-04-21T08:47:00.748097+0000","last_became_active":"2024-04-21T08:44:38.518421+0000","last_became_peered":"2024-04-21T08:44:38.518421+0000","last_unstale":"2024-04-21T08:47:00.748097+0000","last_undegraded":"2024-04-21T08:47:00.748097+0000","last_fullsized":"2024-04-21T08:47:00.748097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:21.119364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874482+0000","last_change":"2024-04-21T08:44:33.484383+0000","last_active":"2024-04-21T08:47:00.874482+0000","last_peered":"2024-04-21T08:47:00.874482+0000","last_clean":"2024-04-21T08:47:00.874482+0000","last_became_active":"2024-04-21T08:44:33.484083+0000","last_became_peered":"2024-04-21T08:44:33.484083+0000","last_unstale":"2024-04-21T08:47:00.874482+0000","last_undegraded":"2024-04-21T08:47:00.874482+0000","last_fullsized":"2024-04-21T08:47:00.874482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:13:14.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622126+0000","last_change":"2024-04-21T08:44:34.495636+0000","last_active":"2024-04-21T08:47:00.622126+0000","last_peered":"2024-04-21T08:47:00.622126+0000","last_clean":"2024-04-21T08:47:00.622126+0000","last_became_active":"2024-04-21T08:44:34.494967+0000","last_became_peered":"2024-04-21T08:44:34.494967+0000","last_unstale":"2024-04-21T08:47:00.622126+0000","last_undegraded":"2024-04-21T08:47:00.622126+0000","last_fullsized":"2024-04-21T08:47:00.622126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:47:02.379208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747724+0000","last_change":"2024-04-21T08:44:38.515145+0000","last_active":"2024-04-21T08:47:00.747724+0000","last_peered":"2024-04-21T08:47:00.747724+0000","last_clean":"2024-04-21T08:47:00.747724+0000","last_became_active":"2024-04-21T08:44:38.514755+0000","last_became_peered":"2024-04-21T08:44:38.514755+0000","last_unstale":"2024-04-21T08:47:00.747724+0000","last_undegraded":"2024-04-21T08:47:00.747724+0000","last_fullsized":"2024-04-21T08:47:00.747724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:38:15.826293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728869+0000","last_change":"2024-04-21T08:44:33.484714+0000","last_active":"2024-04-21T08:47:00.728869+0000","last_peered":"2024-04-21T08:47:00.728869+0000","last_clean":"2024-04-21T08:47:00.728869+0000","last_became_active":"2024-04-21T08:44:33.484385+0000","last_became_peered":"2024-04-21T08:44:33.484385+0000","last_unstale":"2024-04-21T08:47:00.728869+0000","last_undegraded":"2024-04-21T08:47:00.728869+0000","last_fullsized":"2024-04-21T08:47:00.728869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:03.406860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718544+0000","last_change":"2024-04-21T08:44:34.490989+0000","last_active":"2024-04-21T08:47:00.718544+0000","last_peered":"2024-04-21T08:47:00.718544+0000","last_clean":"2024-04-21T08:47:00.718544+0000","last_became_active":"2024-04-21T08:44:34.490658+0000","last_became_peered":"2024-04-21T08:44:34.490658+0000","last_unstale":"2024-04-21T08:47:00.718544+0000","last_undegraded":"2024-04-21T08:47:00.718544+0000","last_fullsized":"2024-04-21T08:47:00.718544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:42:15.537913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622288+0000","last_change":"2024-04-21T08:44:38.519723+0000","last_active":"2024-04-21T08:47:00.622288+0000","last_peered":"2024-04-21T08:47:00.622288+0000","last_clean":"2024-04-21T08:47:00.622288+0000","last_became_active":"2024-04-21T08:44:38.519336+0000","last_became_peered":"2024-04-21T08:44:38.519336+0000","last_unstale":"2024-04-21T08:47:00.622288+0000","last_undegraded":"2024-04-21T08:47:00.622288+0000","last_fullsized":"2024-04-21T08:47:00.622288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:00.332153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'6","reported_seq":301,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:11.183116+0000","last_change":"2024-04-21T08:44:33.484404+0000","last_active":"2024-04-21T08:47:11.183116+0000","last_peered":"2024-04-21T08:47:11.183116+0000","last_clean":"2024-04-21T08:47:11.183116+0000","last_became_active":"2024-04-21T08:44:33.484088+0000","last_became_peered":"2024-04-21T08:44:33.484088+0000","last_unstale":"2024-04-21T08:47:11.183116+0000","last_undegraded":"2024-04-21T08:47:11.183116+0000","last_fullsized":"2024-04-21T08:47:11.183116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:41.948622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":262,"num_read_kb":262,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748214+0000","last_change":"2024-04-21T08:44:34.502598+0000","last_active":"2024-04-21T08:47:00.748214+0000","last_peered":"2024-04-21T08:47:00.748214+0000","last_clean":"2024-04-21T08:47:00.748214+0000","last_became_active":"2024-04-21T08:44:34.502057+0000","last_became_peered":"2024-04-21T08:44:34.502057+0000","last_unstale":"2024-04-21T08:47:00.748214+0000","last_undegraded":"2024-04-21T08:47:00.748214+0000","last_fullsized":"2024-04-21T08:47:00.748214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:24.052873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718327+0000","last_change":"2024-04-21T08:44:38.519699+0000","last_active":"2024-04-21T08:47:00.718327+0000","last_peered":"2024-04-21T08:47:00.718327+0000","last_clean":"2024-04-21T08:47:00.718327+0000","last_became_active":"2024-04-21T08:44:38.517655+0000","last_became_peered":"2024-04-21T08:44:38.517655+0000","last_unstale":"2024-04-21T08:47:00.718327+0000","last_undegraded":"2024-04-21T08:47:00.718327+0000","last_fullsized":"2024-04-21T08:47:00.718327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:12.784479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748806+0000","last_change":"2024-04-21T08:46:27.494747+0000","last_active":"2024-04-21T08:47:00.748806+0000","last_peered":"2024-04-21T08:47:00.748806+0000","last_clean":"2024-04-21T08:47:00.748806+0000","last_became_active":"2024-04-21T08:46:27.490469+0000","last_became_peered":"2024-04-21T08:46:27.490469+0000","last_unstale":"2024-04-21T08:47:00.748806+0000","last_undegraded":"2024-04-21T08:47:00.748806+0000","last_fullsized":"2024-04-21T08:47:00.748806+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:16:35.257267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747850+0000","last_change":"2024-04-21T08:45:27.518395+0000","last_active":"2024-04-21T08:47:00.747850+0000","last_peered":"2024-04-21T08:47:00.747850+0000","last_clean":"2024-04-21T08:47:00.747850+0000","last_became_active":"2024-04-21T08:45:27.517969+0000","last_became_peered":"2024-04-21T08:45:27.517969+0000","last_unstale":"2024-04-21T08:47:00.747850+0000","last_undegraded":"2024-04-21T08:47:00.747850+0000","last_fullsized":"2024-04-21T08:47:00.747850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:55.306556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748406+0000","last_change":"2024-04-21T08:44:38.517956+0000","last_active":"2024-04-21T08:47:00.748406+0000","last_peered":"2024-04-21T08:47:00.748406+0000","last_clean":"2024-04-21T08:47:00.748406+0000","last_became_active":"2024-04-21T08:44:38.517422+0000","last_became_peered":"2024-04-21T08:44:38.517422+0000","last_unstale":"2024-04-21T08:47:00.748406+0000","last_undegraded":"2024-04-21T08:47:00.748406+0000","last_fullsized":"2024-04-21T08:47:00.748406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:04.650784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729223+0000","last_change":"2024-04-21T08:44:33.489151+0000","last_active":"2024-04-21T08:47:00.729223+0000","last_peered":"2024-04-21T08:47:00.729223+0000","last_clean":"2024-04-21T08:47:00.729223+0000","last_became_active":"2024-04-21T08:44:33.488781+0000","last_became_peered":"2024-04-21T08:44:33.488781+0000","last_unstale":"2024-04-21T08:47:00.729223+0000","last_undegraded":"2024-04-21T08:47:00.729223+0000","last_fullsized":"2024-04-21T08:47:00.729223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:40.675100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747905+0000","last_change":"2024-04-21T08:44:34.496528+0000","last_active":"2024-04-21T08:47:00.747905+0000","last_peered":"2024-04-21T08:47:00.747905+0000","last_clean":"2024-04-21T08:47:00.747905+0000","last_became_active":"2024-04-21T08:44:34.496223+0000","last_became_peered":"2024-04-21T08:44:34.496223+0000","last_unstale":"2024-04-21T08:47:00.747905+0000","last_undegraded":"2024-04-21T08:47:00.747905+0000","last_fullsized":"2024-04-21T08:47:00.747905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:56.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718461+0000","last_change":"2024-04-21T08:44:38.517537+0000","last_active":"2024-04-21T08:47:00.718461+0000","last_peered":"2024-04-21T08:47:00.718461+0000","last_clean":"2024-04-21T08:47:00.718461+0000","last_became_active":"2024-04-21T08:44:38.517134+0000","last_became_peered":"2024-04-21T08:44:38.517134+0000","last_unstale":"2024-04-21T08:47:00.718461+0000","last_undegraded":"2024-04-21T08:47:00.718461+0000","last_fullsized":"2024-04-21T08:47:00.718461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:38.634418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'41","reported_seq":351,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:11.193441+0000","last_change":"2024-04-21T08:44:33.485582+0000","last_active":"2024-04-21T08:47:11.193441+0000","last_peered":"2024-04-21T08:47:11.193441+0000","last_clean":"2024-04-21T08:47:11.193441+0000","last_became_active":"2024-04-21T08:44:33.485249+0000","last_became_peered":"2024-04-21T08:44:33.485249+0000","last_unstale":"2024-04-21T08:47:11.193441+0000","last_undegraded":"2024-04-21T08:47:11.193441+0000","last_fullsized":"2024-04-21T08:47:11.193441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:16:10.571694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69948,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":277,"num_read_kb":208,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721670+0000","last_change":"2024-04-21T08:44:34.491361+0000","last_active":"2024-04-21T08:47:00.721670+0000","last_peered":"2024-04-21T08:47:00.721670+0000","last_clean":"2024-04-21T08:47:00.721670+0000","last_became_active":"2024-04-21T08:44:34.491050+0000","last_became_peered":"2024-04-21T08:44:34.491050+0000","last_unstale":"2024-04-21T08:47:00.721670+0000","last_undegraded":"2024-04-21T08:47:00.721670+0000","last_fullsized":"2024-04-21T08:47:00.721670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:39.082448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729534+0000","last_change":"2024-04-21T08:44:38.528745+0000","last_active":"2024-04-21T08:47:00.729534+0000","last_peered":"2024-04-21T08:47:00.729534+0000","last_clean":"2024-04-21T08:47:00.729534+0000","last_became_active":"2024-04-21T08:44:38.528275+0000","last_became_peered":"2024-04-21T08:44:38.528275+0000","last_unstale":"2024-04-21T08:47:00.729534+0000","last_undegraded":"2024-04-21T08:47:00.729534+0000","last_fullsized":"2024-04-21T08:47:00.729534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:06:11.338420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626897+0000","last_change":"2024-04-21T08:44:33.482059+0000","last_active":"2024-04-21T08:47:00.626897+0000","last_peered":"2024-04-21T08:47:00.626897+0000","last_clean":"2024-04-21T08:47:00.626897+0000","last_became_active":"2024-04-21T08:44:33.481663+0000","last_became_peered":"2024-04-21T08:44:33.481663+0000","last_unstale":"2024-04-21T08:47:00.626897+0000","last_undegraded":"2024-04-21T08:47:00.626897+0000","last_fullsized":"2024-04-21T08:47:00.626897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:01.240452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874587+0000","last_change":"2024-04-21T08:44:34.491398+0000","last_active":"2024-04-21T08:47:00.874587+0000","last_peered":"2024-04-21T08:47:00.874587+0000","last_clean":"2024-04-21T08:47:00.874587+0000","last_became_active":"2024-04-21T08:44:34.491073+0000","last_became_peered":"2024-04-21T08:44:34.491073+0000","last_unstale":"2024-04-21T08:47:00.874587+0000","last_undegraded":"2024-04-21T08:47:00.874587+0000","last_fullsized":"2024-04-21T08:47:00.874587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:22.566063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747937+0000","last_change":"2024-04-21T08:44:38.515653+0000","last_active":"2024-04-21T08:47:00.747937+0000","last_peered":"2024-04-21T08:47:00.747937+0000","last_clean":"2024-04-21T08:47:00.747937+0000","last_became_active":"2024-04-21T08:44:38.514796+0000","last_became_peered":"2024-04-21T08:44:38.514796+0000","last_unstale":"2024-04-21T08:47:00.747937+0000","last_undegraded":"2024-04-21T08:47:00.747937+0000","last_fullsized":"2024-04-21T08:47:00.747937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:00:22.662136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729003+0000","last_change":"2024-04-21T08:44:33.487073+0000","last_active":"2024-04-21T08:47:00.729003+0000","last_peered":"2024-04-21T08:47:00.729003+0000","last_clean":"2024-04-21T08:47:00.729003+0000","last_became_active":"2024-04-21T08:44:33.486740+0000","last_became_peered":"2024-04-21T08:44:33.486740+0000","last_unstale":"2024-04-21T08:47:00.729003+0000","last_undegraded":"2024-04-21T08:47:00.729003+0000","last_fullsized":"2024-04-21T08:47:00.729003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:32.673074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748512+0000","last_change":"2024-04-21T08:44:34.496757+0000","last_active":"2024-04-21T08:47:00.748512+0000","last_peered":"2024-04-21T08:47:00.748512+0000","last_clean":"2024-04-21T08:47:00.748512+0000","last_became_active":"2024-04-21T08:44:34.496422+0000","last_became_peered":"2024-04-21T08:44:34.496422+0000","last_unstale":"2024-04-21T08:47:00.748512+0000","last_undegraded":"2024-04-21T08:47:00.748512+0000","last_fullsized":"2024-04-21T08:47:00.748512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:08.686810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718978+0000","last_change":"2024-04-21T08:44:38.520935+0000","last_active":"2024-04-21T08:47:00.718978+0000","last_peered":"2024-04-21T08:47:00.718978+0000","last_clean":"2024-04-21T08:47:00.718978+0000","last_became_active":"2024-04-21T08:44:38.520481+0000","last_became_peered":"2024-04-21T08:44:38.520481+0000","last_unstale":"2024-04-21T08:47:00.718978+0000","last_undegraded":"2024-04-21T08:47:00.718978+0000","last_fullsized":"2024-04-21T08:47:00.718978+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:12.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'8","reported_seq":178,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.175644+0000","last_change":"2024-04-21T08:44:33.485797+0000","last_active":"2024-04-21T08:47:10.175644+0000","last_peered":"2024-04-21T08:47:10.175644+0000","last_clean":"2024-04-21T08:47:10.175644+0000","last_became_active":"2024-04-21T08:44:33.485496+0000","last_became_peered":"2024-04-21T08:44:33.485496+0000","last_unstale":"2024-04-21T08:47:10.175644+0000","last_undegraded":"2024-04-21T08:47:10.175644+0000","last_fullsized":"2024-04-21T08:47:10.175644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:11:34.604611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1761,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748282+0000","last_change":"2024-04-21T08:44:34.506453+0000","last_active":"2024-04-21T08:47:00.748282+0000","last_peered":"2024-04-21T08:47:00.748282+0000","last_clean":"2024-04-21T08:47:00.748282+0000","last_became_active":"2024-04-21T08:44:34.506121+0000","last_became_peered":"2024-04-21T08:44:34.506121+0000","last_unstale":"2024-04-21T08:47:00.748282+0000","last_undegraded":"2024-04-21T08:47:00.748282+0000","last_fullsized":"2024-04-21T08:47:00.748282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:07.927366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875292+0000","last_change":"2024-04-21T08:44:38.523718+0000","last_active":"2024-04-21T08:47:00.875292+0000","last_peered":"2024-04-21T08:47:00.875292+0000","last_clean":"2024-04-21T08:47:00.875292+0000","last_became_active":"2024-04-21T08:44:38.523003+0000","last_became_peered":"2024-04-21T08:44:38.523003+0000","last_unstale":"2024-04-21T08:47:00.875292+0000","last_undegraded":"2024-04-21T08:47:00.875292+0000","last_fullsized":"2024-04-21T08:47:00.875292+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:20.837450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721393+0000","last_change":"2024-04-21T08:44:33.482029+0000","last_active":"2024-04-21T08:47:00.721393+0000","last_peered":"2024-04-21T08:47:00.721393+0000","last_clean":"2024-04-21T08:47:00.721393+0000","last_became_active":"2024-04-21T08:44:33.481713+0000","last_became_peered":"2024-04-21T08:44:33.481713+0000","last_unstale":"2024-04-21T08:47:00.721393+0000","last_undegraded":"2024-04-21T08:47:00.721393+0000","last_fullsized":"2024-04-21T08:47:00.721393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:46:34.717244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728921+0000","last_change":"2024-04-21T08:44:34.491944+0000","last_active":"2024-04-21T08:47:00.728921+0000","last_peered":"2024-04-21T08:47:00.728921+0000","last_clean":"2024-04-21T08:47:00.728921+0000","last_became_active":"2024-04-21T08:44:34.491638+0000","last_became_peered":"2024-04-21T08:44:34.491638+0000","last_unstale":"2024-04-21T08:47:00.728921+0000","last_undegraded":"2024-04-21T08:47:00.728921+0000","last_fullsized":"2024-04-21T08:47:00.728921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:08:07.925937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728793+0000","last_change":"2024-04-21T08:44:38.528971+0000","last_active":"2024-04-21T08:47:00.728793+0000","last_peered":"2024-04-21T08:47:00.728793+0000","last_clean":"2024-04-21T08:47:00.728793+0000","last_became_active":"2024-04-21T08:44:38.528453+0000","last_became_peered":"2024-04-21T08:44:38.528453+0000","last_unstale":"2024-04-21T08:47:00.728793+0000","last_undegraded":"2024-04-21T08:47:00.728793+0000","last_fullsized":"2024-04-21T08:47:00.728793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:50:31.237749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721764+0000","last_change":"2024-04-21T08:44:33.479262+0000","last_active":"2024-04-21T08:47:00.721764+0000","last_peered":"2024-04-21T08:47:00.721764+0000","last_clean":"2024-04-21T08:47:00.721764+0000","last_became_active":"2024-04-21T08:44:33.478907+0000","last_became_peered":"2024-04-21T08:44:33.478907+0000","last_unstale":"2024-04-21T08:47:00.721764+0000","last_undegraded":"2024-04-21T08:47:00.721764+0000","last_fullsized":"2024-04-21T08:47:00.721764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:33:43.879126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748024+0000","last_change":"2024-04-21T08:44:34.498503+0000","last_active":"2024-04-21T08:47:00.748024+0000","last_peered":"2024-04-21T08:47:00.748024+0000","last_clean":"2024-04-21T08:47:00.748024+0000","last_became_active":"2024-04-21T08:44:34.495140+0000","last_became_peered":"2024-04-21T08:44:34.495140+0000","last_unstale":"2024-04-21T08:47:00.748024+0000","last_undegraded":"2024-04-21T08:47:00.748024+0000","last_fullsized":"2024-04-21T08:47:00.748024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:24.518349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721372+0000","last_change":"2024-04-21T08:44:38.517717+0000","last_active":"2024-04-21T08:47:00.721372+0000","last_peered":"2024-04-21T08:47:00.721372+0000","last_clean":"2024-04-21T08:47:00.721372+0000","last_became_active":"2024-04-21T08:44:38.517272+0000","last_became_peered":"2024-04-21T08:44:38.517272+0000","last_unstale":"2024-04-21T08:47:00.721372+0000","last_undegraded":"2024-04-21T08:47:00.721372+0000","last_fullsized":"2024-04-21T08:47:00.721372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.664130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718792+0000","last_change":"2024-04-21T08:44:33.477498+0000","last_active":"2024-04-21T08:47:00.718792+0000","last_peered":"2024-04-21T08:47:00.718792+0000","last_clean":"2024-04-21T08:47:00.718792+0000","last_became_active":"2024-04-21T08:44:33.477139+0000","last_became_peered":"2024-04-21T08:44:33.477139+0000","last_unstale":"2024-04-21T08:47:00.718792+0000","last_undegraded":"2024-04-21T08:47:00.718792+0000","last_fullsized":"2024-04-21T08:47:00.718792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:26:59.390583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626701+0000","last_change":"2024-04-21T08:44:34.493586+0000","last_active":"2024-04-21T08:47:00.626701+0000","last_peered":"2024-04-21T08:47:00.626701+0000","last_clean":"2024-04-21T08:47:00.626701+0000","last_became_active":"2024-04-21T08:44:34.493258+0000","last_became_peered":"2024-04-21T08:44:34.493258+0000","last_unstale":"2024-04-21T08:47:00.626701+0000","last_undegraded":"2024-04-21T08:47:00.626701+0000","last_fullsized":"2024-04-21T08:47:00.626701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:11:18.783332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874853+0000","last_change":"2024-04-21T08:44:38.519783+0000","last_active":"2024-04-21T08:47:00.874853+0000","last_peered":"2024-04-21T08:47:00.874853+0000","last_clean":"2024-04-21T08:47:00.874853+0000","last_became_active":"2024-04-21T08:44:38.519397+0000","last_became_peered":"2024-04-21T08:44:38.519397+0000","last_unstale":"2024-04-21T08:47:00.874853+0000","last_undegraded":"2024-04-21T08:47:00.874853+0000","last_fullsized":"2024-04-21T08:47:00.874853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:08.740240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":170,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.175110+0000","last_change":"2024-04-21T08:44:33.490098+0000","last_active":"2024-04-21T08:47:10.175110+0000","last_peered":"2024-04-21T08:47:10.175110+0000","last_clean":"2024-04-21T08:47:10.175110+0000","last_became_active":"2024-04-21T08:44:33.489691+0000","last_became_peered":"2024-04-21T08:44:33.489691+0000","last_unstale":"2024-04-21T08:47:10.175110+0000","last_undegraded":"2024-04-21T08:47:10.175110+0000","last_fullsized":"2024-04-21T08:47:10.175110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:44.965998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626543+0000","last_change":"2024-04-21T08:46:27.489580+0000","last_active":"2024-04-21T08:47:00.626543+0000","last_peered":"2024-04-21T08:47:00.626543+0000","last_clean":"2024-04-21T08:47:00.626543+0000","last_became_active":"2024-04-21T08:46:27.489086+0000","last_became_peered":"2024-04-21T08:46:27.489086+0000","last_unstale":"2024-04-21T08:47:00.626543+0000","last_undegraded":"2024-04-21T08:47:00.626543+0000","last_fullsized":"2024-04-21T08:47:00.626543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:38:50.618957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874908+0000","last_change":"2024-04-21T08:44:38.521010+0000","last_active":"2024-04-21T08:47:00.874908+0000","last_peered":"2024-04-21T08:47:00.874908+0000","last_clean":"2024-04-21T08:47:00.874908+0000","last_became_active":"2024-04-21T08:44:38.520605+0000","last_became_peered":"2024-04-21T08:44:38.520605+0000","last_unstale":"2024-04-21T08:47:00.874908+0000","last_undegraded":"2024-04-21T08:47:00.874908+0000","last_fullsized":"2024-04-21T08:47:00.874908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:50.378803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728846+0000","last_change":"2024-04-21T08:45:27.410137+0000","last_active":"2024-04-21T08:47:00.728846+0000","last_peered":"2024-04-21T08:47:00.728846+0000","last_clean":"2024-04-21T08:47:00.728846+0000","last_became_active":"2024-04-21T08:45:27.409728+0000","last_became_peered":"2024-04-21T08:45:27.409728+0000","last_unstale":"2024-04-21T08:47:00.728846+0000","last_undegraded":"2024-04-21T08:47:00.728846+0000","last_fullsized":"2024-04-21T08:47:00.728846+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:17.934902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":172,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.175058+0000","last_change":"2024-04-21T08:44:33.494031+0000","last_active":"2024-04-21T08:47:10.175058+0000","last_peered":"2024-04-21T08:47:10.175058+0000","last_clean":"2024-04-21T08:47:10.175058+0000","last_became_active":"2024-04-21T08:44:33.493580+0000","last_became_peered":"2024-04-21T08:44:33.493580+0000","last_unstale":"2024-04-21T08:47:10.175058+0000","last_undegraded":"2024-04-21T08:47:10.175058+0000","last_fullsized":"2024-04-21T08:47:10.175058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:39:48.230616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748658+0000","last_change":"2024-04-21T08:44:38.518067+0000","last_active":"2024-04-21T08:47:00.748658+0000","last_peered":"2024-04-21T08:47:00.748658+0000","last_clean":"2024-04-21T08:47:00.748658+0000","last_became_active":"2024-04-21T08:44:38.517594+0000","last_became_peered":"2024-04-21T08:44:38.517594+0000","last_unstale":"2024-04-21T08:47:00.748658+0000","last_undegraded":"2024-04-21T08:47:00.748658+0000","last_fullsized":"2024-04-21T08:47:00.748658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:35.261582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718443+0000","last_change":"2024-04-21T08:45:27.517719+0000","last_active":"2024-04-21T08:47:00.718443+0000","last_peered":"2024-04-21T08:47:00.718443+0000","last_clean":"2024-04-21T08:47:00.718443+0000","last_became_active":"2024-04-21T08:45:27.517429+0000","last_became_peered":"2024-04-21T08:45:27.517429+0000","last_unstale":"2024-04-21T08:47:00.718443+0000","last_undegraded":"2024-04-21T08:47:00.718443+0000","last_fullsized":"2024-04-21T08:47:00.718443+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:49:34.097759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748213+0000","last_change":"2024-04-21T08:44:33.485110+0000","last_active":"2024-04-21T08:47:00.748213+0000","last_peered":"2024-04-21T08:47:00.748213+0000","last_clean":"2024-04-21T08:47:00.748213+0000","last_became_active":"2024-04-21T08:44:33.484722+0000","last_became_peered":"2024-04-21T08:44:33.484722+0000","last_unstale":"2024-04-21T08:47:00.748213+0000","last_undegraded":"2024-04-21T08:47:00.748213+0000","last_fullsized":"2024-04-21T08:47:00.748213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:47:16.773560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"94'148","reported_seq":412,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:02.744643+0000","last_change":"2024-04-21T08:43:17.984594+0000","last_active":"2024-04-21T08:47:02.744643+0000","last_peered":"2024-04-21T08:47:02.744643+0000","last_clean":"2024-04-21T08:47:02.744643+0000","last_became_active":"2024-04-21T08:43:17.984207+0000","last_became_peered":"2024-04-21T08:43:17.984207+0000","last_unstale":"2024-04-21T08:47:02.744643+0000","last_undegraded":"2024-04-21T08:47:02.744643+0000","last_fullsized":"2024-04-21T08:47:02.744643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_clean_scrub_stamp":"2024-04-21T08:39:50.910206+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:35.260499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721571+0000","last_change":"2024-04-21T08:45:27.518834+0000","last_active":"2024-04-21T08:47:00.721571+0000","last_peered":"2024-04-21T08:47:00.721571+0000","last_clean":"2024-04-21T08:47:00.721571+0000","last_became_active":"2024-04-21T08:45:27.518390+0000","last_became_peered":"2024-04-21T08:45:27.518390+0000","last_unstale":"2024-04-21T08:47:00.721571+0000","last_undegraded":"2024-04-21T08:47:00.721571+0000","last_fullsized":"2024-04-21T08:47:00.721571+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:58:32.673008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874296+0000","last_change":"2024-04-21T08:44:34.499975+0000","last_active":"2024-04-21T08:47:00.874296+0000","last_peered":"2024-04-21T08:47:00.874296+0000","last_clean":"2024-04-21T08:47:00.874296+0000","last_became_active":"2024-04-21T08:44:34.499646+0000","last_became_peered":"2024-04-21T08:44:34.499646+0000","last_unstale":"2024-04-21T08:47:00.874296+0000","last_undegraded":"2024-04-21T08:47:00.874296+0000","last_fullsized":"2024-04-21T08:47:00.874296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:19:38.082502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748114+0000","last_change":"2024-04-21T08:44:33.484449+0000","last_active":"2024-04-21T08:47:00.748114+0000","last_peered":"2024-04-21T08:47:00.748114+0000","last_clean":"2024-04-21T08:47:00.748114+0000","last_became_active":"2024-04-21T08:44:33.484155+0000","last_became_peered":"2024-04-21T08:44:33.484155+0000","last_unstale":"2024-04-21T08:47:00.748114+0000","last_undegraded":"2024-04-21T08:47:00.748114+0000","last_fullsized":"2024-04-21T08:47:00.748114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:03.480474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747766+0000","last_change":"2024-04-21T08:44:38.515283+0000","last_active":"2024-04-21T08:47:00.747766+0000","last_peered":"2024-04-21T08:47:00.747766+0000","last_clean":"2024-04-21T08:47:00.747766+0000","last_became_active":"2024-04-21T08:44:38.514764+0000","last_became_peered":"2024-04-21T08:44:38.514764+0000","last_unstale":"2024-04-21T08:47:00.747766+0000","last_undegraded":"2024-04-21T08:47:00.747766+0000","last_fullsized":"2024-04-21T08:47:00.747766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:17:36.378392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721569+0000","last_change":"2024-04-21T08:44:33.489198+0000","last_active":"2024-04-21T08:47:00.721569+0000","last_peered":"2024-04-21T08:47:00.721569+0000","last_clean":"2024-04-21T08:47:00.721569+0000","last_became_active":"2024-04-21T08:44:33.488794+0000","last_became_peered":"2024-04-21T08:44:33.488794+0000","last_unstale":"2024-04-21T08:47:00.721569+0000","last_undegraded":"2024-04-21T08:47:00.721569+0000","last_fullsized":"2024-04-21T08:47:00.721569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:24.948283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874977+0000","last_change":"2024-04-21T08:44:34.488811+0000","last_active":"2024-04-21T08:47:00.874977+0000","last_peered":"2024-04-21T08:47:00.874977+0000","last_clean":"2024-04-21T08:47:00.874977+0000","last_became_active":"2024-04-21T08:44:34.488472+0000","last_became_peered":"2024-04-21T08:44:34.488472+0000","last_unstale":"2024-04-21T08:47:00.874977+0000","last_undegraded":"2024-04-21T08:47:00.874977+0000","last_fullsized":"2024-04-21T08:47:00.874977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:44.779492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622045+0000","last_change":"2024-04-21T08:44:38.522711+0000","last_active":"2024-04-21T08:47:00.622045+0000","last_peered":"2024-04-21T08:47:00.622045+0000","last_clean":"2024-04-21T08:47:00.622045+0000","last_became_active":"2024-04-21T08:44:38.522322+0000","last_became_peered":"2024-04-21T08:44:38.522322+0000","last_unstale":"2024-04-21T08:47:00.622045+0000","last_undegraded":"2024-04-21T08:47:00.622045+0000","last_fullsized":"2024-04-21T08:47:00.622045+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:26:13.323283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'102","reported_seq":227,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.078157+0000","last_change":"2024-04-21T08:44:33.481815+0000","last_active":"2024-04-21T08:47:10.078157+0000","last_peered":"2024-04-21T08:47:10.078157+0000","last_clean":"2024-04-21T08:47:10.078157+0000","last_became_active":"2024-04-21T08:44:33.481469+0000","last_became_peered":"2024-04-21T08:44:33.481469+0000","last_unstale":"2024-04-21T08:47:10.078157+0000","last_undegraded":"2024-04-21T08:47:10.078157+0000","last_fullsized":"2024-04-21T08:47:10.078157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:13.970821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":185,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623173+0000","last_change":"2024-04-21T08:44:34.496996+0000","last_active":"2024-04-21T08:47:00.623173+0000","last_peered":"2024-04-21T08:47:00.623173+0000","last_clean":"2024-04-21T08:47:00.623173+0000","last_became_active":"2024-04-21T08:44:34.496676+0000","last_became_peered":"2024-04-21T08:44:34.496676+0000","last_unstale":"2024-04-21T08:47:00.623173+0000","last_undegraded":"2024-04-21T08:47:00.623173+0000","last_fullsized":"2024-04-21T08:47:00.623173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:43:08.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626592+0000","last_change":"2024-04-21T08:44:38.516240+0000","last_active":"2024-04-21T08:47:00.626592+0000","last_peered":"2024-04-21T08:47:00.626592+0000","last_clean":"2024-04-21T08:47:00.626592+0000","last_became_active":"2024-04-21T08:44:38.515801+0000","last_became_peered":"2024-04-21T08:44:38.515801+0000","last_unstale":"2024-04-21T08:47:00.626592+0000","last_undegraded":"2024-04-21T08:47:00.626592+0000","last_fullsized":"2024-04-21T08:47:00.626592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:46.520373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749302+0000","last_change":"2024-04-21T08:44:33.488952+0000","last_active":"2024-04-21T08:47:00.749302+0000","last_peered":"2024-04-21T08:47:00.749302+0000","last_clean":"2024-04-21T08:47:00.749302+0000","last_became_active":"2024-04-21T08:44:33.488617+0000","last_became_peered":"2024-04-21T08:44:33.488617+0000","last_unstale":"2024-04-21T08:47:00.749302+0000","last_undegraded":"2024-04-21T08:47:00.749302+0000","last_fullsized":"2024-04-21T08:47:00.749302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:37:24.679132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747751+0000","last_change":"2024-04-21T08:44:34.510568+0000","last_active":"2024-04-21T08:47:00.747751+0000","last_peered":"2024-04-21T08:47:00.747751+0000","last_clean":"2024-04-21T08:47:00.747751+0000","last_became_active":"2024-04-21T08:44:34.509900+0000","last_became_peered":"2024-04-21T08:44:34.509900+0000","last_unstale":"2024-04-21T08:47:00.747751+0000","last_undegraded":"2024-04-21T08:47:00.747751+0000","last_fullsized":"2024-04-21T08:47:00.747751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:08.426141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748835+0000","last_change":"2024-04-21T08:45:27.976874+0000","last_active":"2024-04-21T08:47:00.748835+0000","last_peered":"2024-04-21T08:47:00.748835+0000","last_clean":"2024-04-21T08:47:00.748835+0000","last_became_active":"2024-04-21T08:45:27.976387+0000","last_became_peered":"2024-04-21T08:45:27.976387+0000","last_unstale":"2024-04-21T08:47:00.748835+0000","last_undegraded":"2024-04-21T08:47:00.748835+0000","last_fullsized":"2024-04-21T08:47:00.748835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:38:18.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747268+0000","last_change":"2024-04-21T08:44:33.481803+0000","last_active":"2024-04-21T08:47:00.747268+0000","last_peered":"2024-04-21T08:47:00.747268+0000","last_clean":"2024-04-21T08:47:00.747268+0000","last_became_active":"2024-04-21T08:44:33.481256+0000","last_became_peered":"2024-04-21T08:44:33.481256+0000","last_unstale":"2024-04-21T08:47:00.747268+0000","last_undegraded":"2024-04-21T08:47:00.747268+0000","last_fullsized":"2024-04-21T08:47:00.747268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:17.804557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721258+0000","last_change":"2024-04-21T08:44:34.490206+0000","last_active":"2024-04-21T08:47:00.721258+0000","last_peered":"2024-04-21T08:47:00.721258+0000","last_clean":"2024-04-21T08:47:00.721258+0000","last_became_active":"2024-04-21T08:44:34.489797+0000","last_became_peered":"2024-04-21T08:44:34.489797+0000","last_unstale":"2024-04-21T08:47:00.721258+0000","last_undegraded":"2024-04-21T08:47:00.721258+0000","last_fullsized":"2024-04-21T08:47:00.721258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:09.055429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719147+0000","last_change":"2024-04-21T08:44:38.517385+0000","last_active":"2024-04-21T08:47:00.719147+0000","last_peered":"2024-04-21T08:47:00.719147+0000","last_clean":"2024-04-21T08:47:00.719147+0000","last_became_active":"2024-04-21T08:44:38.516964+0000","last_became_peered":"2024-04-21T08:44:38.516964+0000","last_unstale":"2024-04-21T08:47:00.719147+0000","last_undegraded":"2024-04-21T08:47:00.719147+0000","last_fullsized":"2024-04-21T08:47:00.719147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:33.364076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622039+0000","last_change":"2024-04-21T08:44:33.490586+0000","last_active":"2024-04-21T08:47:00.622039+0000","last_peered":"2024-04-21T08:47:00.622039+0000","last_clean":"2024-04-21T08:47:00.622039+0000","last_became_active":"2024-04-21T08:44:33.489281+0000","last_became_peered":"2024-04-21T08:44:33.489281+0000","last_unstale":"2024-04-21T08:47:00.622039+0000","last_undegraded":"2024-04-21T08:47:00.622039+0000","last_fullsized":"2024-04-21T08:47:00.622039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:41:29.856165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747602+0000","last_change":"2024-04-21T08:44:34.493394+0000","last_active":"2024-04-21T08:47:00.747602+0000","last_peered":"2024-04-21T08:47:00.747602+0000","last_clean":"2024-04-21T08:47:00.747602+0000","last_became_active":"2024-04-21T08:44:34.492367+0000","last_became_peered":"2024-04-21T08:44:34.492367+0000","last_unstale":"2024-04-21T08:47:00.747602+0000","last_undegraded":"2024-04-21T08:47:00.747602+0000","last_fullsized":"2024-04-21T08:47:00.747602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:13.111266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748165+0000","last_change":"2024-04-21T08:44:38.519113+0000","last_active":"2024-04-21T08:47:00.748165+0000","last_peered":"2024-04-21T08:47:00.748165+0000","last_clean":"2024-04-21T08:47:00.748165+0000","last_became_active":"2024-04-21T08:44:38.518469+0000","last_became_peered":"2024-04-21T08:44:38.518469+0000","last_unstale":"2024-04-21T08:47:00.748165+0000","last_undegraded":"2024-04-21T08:47:00.748165+0000","last_fullsized":"2024-04-21T08:47:00.748165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:39:22.200041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'5","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747723+0000","last_change":"2024-04-21T08:46:27.492549+0000","last_active":"2024-04-21T08:47:00.747723+0000","last_peered":"2024-04-21T08:47:00.747723+0000","last_clean":"2024-04-21T08:47:00.747723+0000","last_became_active":"2024-04-21T08:46:27.488054+0000","last_became_peered":"2024-04-21T08:46:27.488054+0000","last_unstale":"2024-04-21T08:47:00.747723+0000","last_undegraded":"2024-04-21T08:47:00.747723+0000","last_fullsized":"2024-04-21T08:47:00.747723+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:51:54.797532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622446+0000","last_change":"2024-04-21T08:44:34.502226+0000","last_active":"2024-04-21T08:47:00.622446+0000","last_peered":"2024-04-21T08:47:00.622446+0000","last_clean":"2024-04-21T08:47:00.622446+0000","last_became_active":"2024-04-21T08:44:34.501775+0000","last_became_peered":"2024-04-21T08:44:34.501775+0000","last_unstale":"2024-04-21T08:47:00.622446+0000","last_undegraded":"2024-04-21T08:47:00.622446+0000","last_fullsized":"2024-04-21T08:47:00.622446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:58.325267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748655+0000","last_change":"2024-04-21T08:44:38.522250+0000","last_active":"2024-04-21T08:47:00.748655+0000","last_peered":"2024-04-21T08:47:00.748655+0000","last_clean":"2024-04-21T08:47:00.748655+0000","last_became_active":"2024-04-21T08:44:38.520870+0000","last_became_peered":"2024-04-21T08:44:38.520870+0000","last_unstale":"2024-04-21T08:47:00.748655+0000","last_undegraded":"2024-04-21T08:47:00.748655+0000","last_fullsized":"2024-04-21T08:47:00.748655+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:02.391785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747761+0000","last_change":"2024-04-21T08:46:27.496215+0000","last_active":"2024-04-21T08:47:00.747761+0000","last_peered":"2024-04-21T08:47:00.747761+0000","last_clean":"2024-04-21T08:47:00.747761+0000","last_became_active":"2024-04-21T08:46:27.488287+0000","last_became_peered":"2024-04-21T08:46:27.488287+0000","last_unstale":"2024-04-21T08:47:00.747761+0000","last_undegraded":"2024-04-21T08:47:00.747761+0000","last_fullsized":"2024-04-21T08:47:00.747761+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:08:05.390563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749473+0000","last_change":"2024-04-21T08:44:34.496034+0000","last_active":"2024-04-21T08:47:00.749473+0000","last_peered":"2024-04-21T08:47:00.749473+0000","last_clean":"2024-04-21T08:47:00.749473+0000","last_became_active":"2024-04-21T08:44:34.495641+0000","last_became_peered":"2024-04-21T08:44:34.495641+0000","last_unstale":"2024-04-21T08:47:00.749473+0000","last_undegraded":"2024-04-21T08:47:00.749473+0000","last_fullsized":"2024-04-21T08:47:00.749473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:21:18.893326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747361+0000","last_change":"2024-04-21T08:44:38.517288+0000","last_active":"2024-04-21T08:47:00.747361+0000","last_peered":"2024-04-21T08:47:00.747361+0000","last_clean":"2024-04-21T08:47:00.747361+0000","last_became_active":"2024-04-21T08:44:38.516853+0000","last_became_peered":"2024-04-21T08:44:38.516853+0000","last_unstale":"2024-04-21T08:47:00.747361+0000","last_undegraded":"2024-04-21T08:47:00.747361+0000","last_fullsized":"2024-04-21T08:47:00.747361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:27:22.371703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":175,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.251660+0000","last_change":"2024-04-21T08:44:33.491578+0000","last_active":"2024-04-21T08:47:10.251660+0000","last_peered":"2024-04-21T08:47:10.251660+0000","last_clean":"2024-04-21T08:47:10.251660+0000","last_became_active":"2024-04-21T08:44:33.491238+0000","last_became_peered":"2024-04-21T08:44:33.491238+0000","last_unstale":"2024-04-21T08:47:10.251660+0000","last_undegraded":"2024-04-21T08:47:10.251660+0000","last_fullsized":"2024-04-21T08:47:10.251660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:50.399031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2343,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749074+0000","last_change":"2024-04-21T08:44:34.497632+0000","last_active":"2024-04-21T08:47:00.749074+0000","last_peered":"2024-04-21T08:47:00.749074+0000","last_clean":"2024-04-21T08:47:00.749074+0000","last_became_active":"2024-04-21T08:44:34.497318+0000","last_became_peered":"2024-04-21T08:44:34.497318+0000","last_unstale":"2024-04-21T08:47:00.749074+0000","last_undegraded":"2024-04-21T08:47:00.749074+0000","last_fullsized":"2024-04-21T08:47:00.749074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:11:33.918530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747370+0000","last_change":"2024-04-21T08:44:38.518414+0000","last_active":"2024-04-21T08:47:00.747370+0000","last_peered":"2024-04-21T08:47:00.747370+0000","last_clean":"2024-04-21T08:47:00.747370+0000","last_became_active":"2024-04-21T08:44:38.517953+0000","last_became_peered":"2024-04-21T08:44:38.517953+0000","last_unstale":"2024-04-21T08:47:00.747370+0000","last_undegraded":"2024-04-21T08:47:00.747370+0000","last_fullsized":"2024-04-21T08:47:00.747370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:48.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'67","reported_seq":132,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:09.745995+0000","last_change":"2024-04-21T08:44:33.493896+0000","last_active":"2024-04-21T08:47:09.745995+0000","last_peered":"2024-04-21T08:47:09.745995+0000","last_clean":"2024-04-21T08:47:09.745995+0000","last_became_active":"2024-04-21T08:44:33.493566+0000","last_became_peered":"2024-04-21T08:44:33.493566+0000","last_unstale":"2024-04-21T08:47:09.745995+0000","last_undegraded":"2024-04-21T08:47:09.745995+0000","last_fullsized":"2024-04-21T08:47:09.745995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:47:17.307690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748612+0000","last_change":"2024-04-21T08:44:34.500406+0000","last_active":"2024-04-21T08:47:00.748612+0000","last_peered":"2024-04-21T08:47:00.748612+0000","last_clean":"2024-04-21T08:47:00.748612+0000","last_became_active":"2024-04-21T08:44:34.500048+0000","last_became_peered":"2024-04-21T08:44:34.500048+0000","last_unstale":"2024-04-21T08:47:00.748612+0000","last_undegraded":"2024-04-21T08:47:00.748612+0000","last_fullsized":"2024-04-21T08:47:00.748612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:10:58.166155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728633+0000","last_change":"2024-04-21T08:44:38.514812+0000","last_active":"2024-04-21T08:47:00.728633+0000","last_peered":"2024-04-21T08:47:00.728633+0000","last_clean":"2024-04-21T08:47:00.728633+0000","last_became_active":"2024-04-21T08:44:38.514247+0000","last_became_peered":"2024-04-21T08:44:38.514247+0000","last_unstale":"2024-04-21T08:47:00.728633+0000","last_undegraded":"2024-04-21T08:47:00.728633+0000","last_fullsized":"2024-04-21T08:47:00.728633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:06:10.588453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875149+0000","last_change":"2024-04-21T08:44:33.493887+0000","last_active":"2024-04-21T08:47:00.875149+0000","last_peered":"2024-04-21T08:47:00.875149+0000","last_clean":"2024-04-21T08:47:00.875149+0000","last_became_active":"2024-04-21T08:44:33.493555+0000","last_became_peered":"2024-04-21T08:44:33.493555+0000","last_unstale":"2024-04-21T08:47:00.875149+0000","last_undegraded":"2024-04-21T08:47:00.875149+0000","last_fullsized":"2024-04-21T08:47:00.875149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:18.089102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749096+0000","last_change":"2024-04-21T08:44:34.503259+0000","last_active":"2024-04-21T08:47:00.749096+0000","last_peered":"2024-04-21T08:47:00.749096+0000","last_clean":"2024-04-21T08:47:00.749096+0000","last_became_active":"2024-04-21T08:44:34.502845+0000","last_became_peered":"2024-04-21T08:44:34.502845+0000","last_unstale":"2024-04-21T08:47:00.749096+0000","last_undegraded":"2024-04-21T08:47:00.749096+0000","last_fullsized":"2024-04-21T08:47:00.749096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:19.902178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747653+0000","last_change":"2024-04-21T08:44:34.497608+0000","last_active":"2024-04-21T08:47:00.747653+0000","last_peered":"2024-04-21T08:47:00.747653+0000","last_clean":"2024-04-21T08:47:00.747653+0000","last_became_active":"2024-04-21T08:44:34.497255+0000","last_became_peered":"2024-04-21T08:44:34.497255+0000","last_unstale":"2024-04-21T08:47:00.747653+0000","last_undegraded":"2024-04-21T08:47:00.747653+0000","last_fullsized":"2024-04-21T08:47:00.747653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:51:15.181837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626828+0000","last_change":"2024-04-21T08:44:34.495408+0000","last_active":"2024-04-21T08:47:00.626828+0000","last_peered":"2024-04-21T08:47:00.626828+0000","last_clean":"2024-04-21T08:47:00.626828+0000","last_became_active":"2024-04-21T08:44:34.495038+0000","last_became_peered":"2024-04-21T08:44:34.495038+0000","last_unstale":"2024-04-21T08:47:00.626828+0000","last_undegraded":"2024-04-21T08:47:00.626828+0000","last_fullsized":"2024-04-21T08:47:00.626828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:59.512245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626703+0000","last_change":"2024-04-21T08:44:34.493292+0000","last_active":"2024-04-21T08:47:00.626703+0000","last_peered":"2024-04-21T08:47:00.626703+0000","last_clean":"2024-04-21T08:47:00.626703+0000","last_became_active":"2024-04-21T08:44:34.492913+0000","last_became_peered":"2024-04-21T08:44:34.492913+0000","last_unstale":"2024-04-21T08:47:00.626703+0000","last_undegraded":"2024-04-21T08:47:00.626703+0000","last_fullsized":"2024-04-21T08:47:00.626703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:24.271859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747551+0000","last_change":"2024-04-21T08:44:34.510448+0000","last_active":"2024-04-21T08:47:00.747551+0000","last_peered":"2024-04-21T08:47:00.747551+0000","last_clean":"2024-04-21T08:47:00.747551+0000","last_became_active":"2024-04-21T08:44:34.509958+0000","last_became_peered":"2024-04-21T08:44:34.509958+0000","last_unstale":"2024-04-21T08:47:00.747551+0000","last_undegraded":"2024-04-21T08:47:00.747551+0000","last_fullsized":"2024-04-21T08:47:00.747551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:36.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875000+0000","last_change":"2024-04-21T08:44:34.497415+0000","last_active":"2024-04-21T08:47:00.875000+0000","last_peered":"2024-04-21T08:47:00.875000+0000","last_clean":"2024-04-21T08:47:00.875000+0000","last_became_active":"2024-04-21T08:44:34.497120+0000","last_became_peered":"2024-04-21T08:44:34.497120+0000","last_unstale":"2024-04-21T08:47:00.875000+0000","last_undegraded":"2024-04-21T08:47:00.875000+0000","last_fullsized":"2024-04-21T08:47:00.875000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:30.232338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747974+0000","last_change":"2024-04-21T08:44:34.500356+0000","last_active":"2024-04-21T08:47:00.747974+0000","last_peered":"2024-04-21T08:47:00.747974+0000","last_clean":"2024-04-21T08:47:00.747974+0000","last_became_active":"2024-04-21T08:44:34.500032+0000","last_became_peered":"2024-04-21T08:44:34.500032+0000","last_unstale":"2024-04-21T08:47:00.747974+0000","last_undegraded":"2024-04-21T08:47:00.747974+0000","last_fullsized":"2024-04-21T08:47:00.747974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:24.900221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627464+0000","last_change":"2024-04-21T08:44:34.493629+0000","last_active":"2024-04-21T08:47:00.627464+0000","last_peered":"2024-04-21T08:47:00.627464+0000","last_clean":"2024-04-21T08:47:00.627464+0000","last_became_active":"2024-04-21T08:44:34.493273+0000","last_became_peered":"2024-04-21T08:44:34.493273+0000","last_unstale":"2024-04-21T08:47:00.627464+0000","last_undegraded":"2024-04-21T08:47:00.627464+0000","last_fullsized":"2024-04-21T08:47:00.627464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:22:42.698957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623249+0000","last_change":"2024-04-21T08:44:34.493427+0000","last_active":"2024-04-21T08:47:00.623249+0000","last_peered":"2024-04-21T08:47:00.623249+0000","last_clean":"2024-04-21T08:47:00.623249+0000","last_became_active":"2024-04-21T08:44:34.493105+0000","last_became_peered":"2024-04-21T08:44:34.493105+0000","last_unstale":"2024-04-21T08:47:00.623249+0000","last_undegraded":"2024-04-21T08:47:00.623249+0000","last_fullsized":"2024-04-21T08:47:00.623249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:37.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874153+0000","last_change":"2024-04-21T08:45:27.515899+0000","last_active":"2024-04-21T08:47:00.874153+0000","last_peered":"2024-04-21T08:47:00.874153+0000","last_clean":"2024-04-21T08:47:00.874153+0000","last_became_active":"2024-04-21T08:45:27.515567+0000","last_became_peered":"2024-04-21T08:45:27.515567+0000","last_unstale":"2024-04-21T08:47:00.874153+0000","last_undegraded":"2024-04-21T08:47:00.874153+0000","last_fullsized":"2024-04-21T08:47:00.874153+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:16.645869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622080+0000","last_change":"2024-04-21T08:44:34.499420+0000","last_active":"2024-04-21T08:47:00.622080+0000","last_peered":"2024-04-21T08:47:00.622080+0000","last_clean":"2024-04-21T08:47:00.622080+0000","last_became_active":"2024-04-21T08:44:34.499079+0000","last_became_peered":"2024-04-21T08:44:34.499079+0000","last_unstale":"2024-04-21T08:47:00.622080+0000","last_undegraded":"2024-04-21T08:47:00.622080+0000","last_fullsized":"2024-04-21T08:47:00.622080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:45.351422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747540+0000","last_change":"2024-04-21T08:44:34.493420+0000","last_active":"2024-04-21T08:47:00.747540+0000","last_peered":"2024-04-21T08:47:00.747540+0000","last_clean":"2024-04-21T08:47:00.747540+0000","last_became_active":"2024-04-21T08:44:34.493098+0000","last_became_peered":"2024-04-21T08:44:34.493098+0000","last_unstale":"2024-04-21T08:47:00.747540+0000","last_undegraded":"2024-04-21T08:47:00.747540+0000","last_fullsized":"2024-04-21T08:47:00.747540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:22:34.316822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874666+0000","last_change":"2024-04-21T08:44:34.494760+0000","last_active":"2024-04-21T08:47:00.874666+0000","last_peered":"2024-04-21T08:47:00.874666+0000","last_clean":"2024-04-21T08:47:00.874666+0000","last_became_active":"2024-04-21T08:44:34.494265+0000","last_became_peered":"2024-04-21T08:44:34.494265+0000","last_unstale":"2024-04-21T08:47:00.874666+0000","last_undegraded":"2024-04-21T08:47:00.874666+0000","last_fullsized":"2024-04-21T08:47:00.874666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:16.035030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748002+0000","last_change":"2024-04-21T08:44:34.498785+0000","last_active":"2024-04-21T08:47:00.748002+0000","last_peered":"2024-04-21T08:47:00.748002+0000","last_clean":"2024-04-21T08:47:00.748002+0000","last_became_active":"2024-04-21T08:44:34.495159+0000","last_became_peered":"2024-04-21T08:44:34.495159+0000","last_unstale":"2024-04-21T08:47:00.748002+0000","last_undegraded":"2024-04-21T08:47:00.748002+0000","last_fullsized":"2024-04-21T08:47:00.748002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:03:20.095016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801645,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1706,"num_read_kb":1077,"num_write":505,"num_write_kb":2388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2404935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":385,"ondisk_log_size":385,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383725,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1081344,"data_stored":531484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645362,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29200,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715952,"statfs":{"total":95995035648,"available":95965134848,"internally_reserved":0,"allocated":2113536,"data_stored":1516480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1503232,"data_stored":927553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070524,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1495040,"data_stored":923835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332162,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1699840,"data_stored":1129379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":525745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":596723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":528374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":599597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28912,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716240,"statfs":{"total":95995035648,"available":95965429760,"internally_reserved":0,"allocated":1818624,"data_stored":1246901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":530992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757324,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37448,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707704,"statfs":{"total":95995035648,"available":95956688896,"internally_reserved":0,"allocated":2039808,"data_stored":1452020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":74574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":332533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":401089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:47:13.498 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:47:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.631+0000 7fa5941f4640 1 -- 172.21.15.82:0/40860876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa58c107260 msgr2=0x7fa58c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.631+0000 7fa5941f4640 1 --2- 172.21.15.82:0/40860876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa58c107260 0x7fa58c10ee30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa584009ca0 tx=0x7fa58402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.631+0000 7fa5941f4640 1 -- 172.21.15.82:0/40860876 shutdown_connections 2024-04-21T08:47:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.631+0000 7fa5941f4640 1 --2- 172.21.15.82:0/40860876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa58c107260 0x7fa58c10ee30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.631+0000 7fa5941f4640 1 --2- 172.21.15.82:0/40860876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa58c102b40 0x7fa58c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.631+0000 7fa5941f4640 1 --2- 172.21.15.82:0/40860876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa58c102190 0x7fa58c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.631+0000 7fa5941f4640 1 -- 172.21.15.82:0/40860876 >> 172.21.15.82:0/40860876 conn(0x7fa58c0793e0 msgr2=0x7fa58c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.631+0000 7fa5941f4640 1 -- 172.21.15.82:0/40860876 shutdown_connections 2024-04-21T08:47:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.631+0000 7fa5941f4640 1 -- 172.21.15.82:0/40860876 wait complete. 2024-04-21T08:47:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa5941f4640 1 Processor -- start 2024-04-21T08:47:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa5941f4640 1 -- start start 2024-04-21T08:47:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa5941f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa58c102190 0x7fa58c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa5941f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa58c107260 0x7fa58c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa5941f4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa58c19cfb0 0x7fa58c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa5941f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1121d0 con 0x7fa58c107260 2024-04-21T08:47:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa5941f4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c112050 con 0x7fa58c19cfb0 2024-04-21T08:47:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa5941f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c112350 con 0x7fa58c102190 2024-04-21T08:47:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa59178f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa58c107260 0x7fa58c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa59178f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa58c107260 0x7fa58c198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56320/0 (socket says 172.21.15.82:56320) 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa591f90640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa58c102190 0x7fa58c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa592791640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa58c19cfb0 0x7fa58c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa59178f640 1 -- 172.21.15.82:0/3363002107 learned_addr learned my addr 172.21.15.82:0/3363002107 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa59178f640 1 -- 172.21.15.82:0/3363002107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa58c102190 msgr2=0x7fa58c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa59178f640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa58c102190 0x7fa58c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa59178f640 1 -- 172.21.15.82:0/3363002107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa58c19cfb0 msgr2=0x7fa58c19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa59178f640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa58c19cfb0 0x7fa58c19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa59178f640 1 -- 172.21.15.82:0/3363002107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58c19daa0 con 0x7fa58c107260 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.632+0000 7fa591f90640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa58c102190 0x7fa58c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.633+0000 7fa59178f640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa58c107260 0x7fa58c198c30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00e9f0 tx=0x7fa57c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.633+0000 7fa582ffd640 1 -- 172.21.15.82:0/3363002107 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c00ccc0 con 0x7fa58c107260 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.633+0000 7fa582ffd640 1 -- 172.21.15.82:0/3363002107 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa57c0054d0 con 0x7fa58c107260 2024-04-21T08:47:13.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.633+0000 7fa5941f4640 1 -- 172.21.15.82:0/3363002107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58c19dd90 con 0x7fa58c107260 2024-04-21T08:47:13.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.633+0000 7fa582ffd640 1 -- 172.21.15.82:0/3363002107 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c010430 con 0x7fa58c107260 2024-04-21T08:47:13.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.633+0000 7fa5941f4640 1 -- 172.21.15.82:0/3363002107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58c1a5530 con 0x7fa58c107260 2024-04-21T08:47:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.634+0000 7fa5941f4640 1 -- 172.21.15.82:0/3363002107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa58c069230 con 0x7fa58c107260 2024-04-21T08:47:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.635+0000 7fa582ffd640 1 -- 172.21.15.82:0/3363002107 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa57c0046c0 con 0x7fa58c107260 2024-04-21T08:47:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.635+0000 7fa582ffd640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5780a3e40 0x7fa5780a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.636+0000 7fa591f90640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5780a3e40 0x7fa5780a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.636+0000 7fa582ffd640 1 -- 172.21.15.82:0/3363002107 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa57c0d1380 con 0x7fa58c107260 2024-04-21T08:47:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.636+0000 7fa591f90640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5780a3e40 0x7fa5780a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa588009fc0 tx=0x7fa588005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.639+0000 7fa582ffd640 1 -- 172.21.15.82:0/3363002107 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa57c053ff0 con 0x7fa58c107260 2024-04-21T08:47:13.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.817+0000 7fa5941f4640 1 -- 172.21.15.82:0/3363002107 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa58c102b40 con 0x7fa5780a3e40 2024-04-21T08:47:13.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.826+0000 7fa582ffd640 1 -- 172.21.15.82:0/3363002107 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339453 (secure 0 0 0) 0x7fa58c102b40 con 0x7fa5780a3e40 2024-04-21T08:47:13.829 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:13.833 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-21T08:47:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.829+0000 7fa5941f4640 1 -- 172.21.15.82:0/3363002107 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5780a3e40 msgr2=0x7fa5780a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.829+0000 7fa5941f4640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5780a3e40 0x7fa5780a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa588009fc0 tx=0x7fa588005f90 comp rx=0 tx=0).stop 2024-04-21T08:47:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.829+0000 7fa5941f4640 1 -- 172.21.15.82:0/3363002107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa58c107260 msgr2=0x7fa58c198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.829+0000 7fa5941f4640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa58c107260 0x7fa58c198c30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00e9f0 tx=0x7fa57c00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.830+0000 7fa5941f4640 1 -- 172.21.15.82:0/3363002107 shutdown_connections 2024-04-21T08:47:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.830+0000 7fa5941f4640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa58c19cfb0 0x7fa58c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.830+0000 7fa5941f4640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5780a3e40 0x7fa5780a6300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.830+0000 7fa5941f4640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa58c107260 0x7fa58c198c30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.830+0000 7fa5941f4640 1 --2- 172.21.15.82:0/3363002107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa58c102190 0x7fa58c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.830+0000 7fa5941f4640 1 -- 172.21.15.82:0/3363002107 >> 172.21.15.82:0/3363002107 conn(0x7fa58c0793e0 msgr2=0x7fa58c0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.830+0000 7fa5941f4640 1 -- 172.21.15.82:0/3363002107 shutdown_connections 2024-04-21T08:47:13.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.830+0000 7fa5941f4640 1 -- 172.21.15.82:0/3363002107 wait complete. 2024-04-21T08:47:13.839 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:13 smithi082 ceph-mon[27967]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-21T08:47:13.839 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:13 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1039426540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:13.839 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:13 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/537925038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:13.839 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:13 smithi082 ceph-mon[27967]: from='client.15876 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:13.840 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-04-21T08:47:12.635896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1392247,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1943,"num_read_kb":1407,"num_write":708,"num_write_kb":5328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1035,"ondisk_log_size":1035,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358168,"kb_used_data":16952,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124583656,"statfs":{"total":1151940427776,"available":1151573663744,"internally_reserved":0,"allocated":17358848,"data_stored":10509083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200716,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":461,"num_read_kb":500,"num_write":153,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.987572"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748276+0000","last_change":"2024-04-21T08:44:34.494393+0000","last_active":"2024-04-21T08:47:00.748276+0000","last_peered":"2024-04-21T08:47:00.748276+0000","last_clean":"2024-04-21T08:47:00.748276+0000","last_became_active":"2024-04-21T08:44:34.493583+0000","last_became_peered":"2024-04-21T08:44:34.493583+0000","last_unstale":"2024-04-21T08:47:00.748276+0000","last_undegraded":"2024-04-21T08:47:00.748276+0000","last_fullsized":"2024-04-21T08:47:00.748276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:37.211676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747826+0000","last_change":"2024-04-21T08:44:34.496291+0000","last_active":"2024-04-21T08:47:00.747826+0000","last_peered":"2024-04-21T08:47:00.747826+0000","last_clean":"2024-04-21T08:47:00.747826+0000","last_became_active":"2024-04-21T08:44:34.495019+0000","last_became_peered":"2024-04-21T08:44:34.495019+0000","last_unstale":"2024-04-21T08:47:00.747826+0000","last_undegraded":"2024-04-21T08:47:00.747826+0000","last_fullsized":"2024-04-21T08:47:00.747826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:05.309847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747464+0000","last_change":"2024-04-21T08:44:34.497948+0000","last_active":"2024-04-21T08:47:00.747464+0000","last_peered":"2024-04-21T08:47:00.747464+0000","last_clean":"2024-04-21T08:47:00.747464+0000","last_became_active":"2024-04-21T08:44:34.497657+0000","last_became_peered":"2024-04-21T08:44:34.497657+0000","last_unstale":"2024-04-21T08:47:00.747464+0000","last_undegraded":"2024-04-21T08:47:00.747464+0000","last_fullsized":"2024-04-21T08:47:00.747464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:11.463715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748377+0000","last_change":"2024-04-21T08:44:34.492682+0000","last_active":"2024-04-21T08:47:00.748377+0000","last_peered":"2024-04-21T08:47:00.748377+0000","last_clean":"2024-04-21T08:47:00.748377+0000","last_became_active":"2024-04-21T08:44:34.492230+0000","last_became_peered":"2024-04-21T08:44:34.492230+0000","last_unstale":"2024-04-21T08:47:00.748377+0000","last_undegraded":"2024-04-21T08:47:00.748377+0000","last_fullsized":"2024-04-21T08:47:00.748377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:40.365608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721373+0000","last_change":"2024-04-21T08:44:34.492160+0000","last_active":"2024-04-21T08:47:00.721373+0000","last_peered":"2024-04-21T08:47:00.721373+0000","last_clean":"2024-04-21T08:47:00.721373+0000","last_became_active":"2024-04-21T08:44:34.491701+0000","last_became_peered":"2024-04-21T08:44:34.491701+0000","last_unstale":"2024-04-21T08:47:00.721373+0000","last_undegraded":"2024-04-21T08:47:00.721373+0000","last_fullsized":"2024-04-21T08:47:00.721373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:50.542661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626584+0000","last_change":"2024-04-21T08:44:34.497341+0000","last_active":"2024-04-21T08:47:00.626584+0000","last_peered":"2024-04-21T08:47:00.626584+0000","last_clean":"2024-04-21T08:47:00.626584+0000","last_became_active":"2024-04-21T08:44:34.495794+0000","last_became_peered":"2024-04-21T08:44:34.495794+0000","last_unstale":"2024-04-21T08:47:00.626584+0000","last_undegraded":"2024-04-21T08:47:00.626584+0000","last_fullsized":"2024-04-21T08:47:00.626584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:13.854753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748527+0000","last_change":"2024-04-21T08:44:34.501578+0000","last_active":"2024-04-21T08:47:00.748527+0000","last_peered":"2024-04-21T08:47:00.748527+0000","last_clean":"2024-04-21T08:47:00.748527+0000","last_became_active":"2024-04-21T08:44:34.500848+0000","last_became_peered":"2024-04-21T08:44:34.500848+0000","last_unstale":"2024-04-21T08:47:00.748527+0000","last_undegraded":"2024-04-21T08:47:00.748527+0000","last_fullsized":"2024-04-21T08:47:00.748527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:53:27.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.621951+0000","last_change":"2024-04-21T08:44:34.502975+0000","last_active":"2024-04-21T08:47:00.621951+0000","last_peered":"2024-04-21T08:47:00.621951+0000","last_clean":"2024-04-21T08:47:00.621951+0000","last_became_active":"2024-04-21T08:44:34.502676+0000","last_became_peered":"2024-04-21T08:44:34.502676+0000","last_unstale":"2024-04-21T08:47:00.621951+0000","last_undegraded":"2024-04-21T08:47:00.621951+0000","last_fullsized":"2024-04-21T08:47:00.621951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:32:13.881603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748293+0000","last_change":"2024-04-21T08:44:34.495703+0000","last_active":"2024-04-21T08:47:00.748293+0000","last_peered":"2024-04-21T08:47:00.748293+0000","last_clean":"2024-04-21T08:47:00.748293+0000","last_became_active":"2024-04-21T08:44:34.495380+0000","last_became_peered":"2024-04-21T08:44:34.495380+0000","last_unstale":"2024-04-21T08:47:00.748293+0000","last_undegraded":"2024-04-21T08:47:00.748293+0000","last_fullsized":"2024-04-21T08:47:00.748293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:45:28.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626928+0000","last_change":"2024-04-21T08:44:34.499361+0000","last_active":"2024-04-21T08:47:00.626928+0000","last_peered":"2024-04-21T08:47:00.626928+0000","last_clean":"2024-04-21T08:47:00.626928+0000","last_became_active":"2024-04-21T08:44:34.499044+0000","last_became_peered":"2024-04-21T08:44:34.499044+0000","last_unstale":"2024-04-21T08:47:00.626928+0000","last_undegraded":"2024-04-21T08:47:00.626928+0000","last_fullsized":"2024-04-21T08:47:00.626928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:12.118985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748894+0000","last_change":"2024-04-21T08:44:34.499631+0000","last_active":"2024-04-21T08:47:00.748894+0000","last_peered":"2024-04-21T08:47:00.748894+0000","last_clean":"2024-04-21T08:47:00.748894+0000","last_became_active":"2024-04-21T08:44:34.499316+0000","last_became_peered":"2024-04-21T08:44:34.499316+0000","last_unstale":"2024-04-21T08:47:00.748894+0000","last_undegraded":"2024-04-21T08:47:00.748894+0000","last_fullsized":"2024-04-21T08:47:00.748894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:26.123017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748326+0000","last_change":"2024-04-21T08:44:34.504857+0000","last_active":"2024-04-21T08:47:00.748326+0000","last_peered":"2024-04-21T08:47:00.748326+0000","last_clean":"2024-04-21T08:47:00.748326+0000","last_became_active":"2024-04-21T08:44:34.504564+0000","last_became_peered":"2024-04-21T08:44:34.504564+0000","last_unstale":"2024-04-21T08:47:00.748326+0000","last_undegraded":"2024-04-21T08:47:00.748326+0000","last_fullsized":"2024-04-21T08:47:00.748326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:32:05.216162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874684+0000","last_change":"2024-04-21T08:44:34.484074+0000","last_active":"2024-04-21T08:47:00.874684+0000","last_peered":"2024-04-21T08:47:00.874684+0000","last_clean":"2024-04-21T08:47:00.874684+0000","last_became_active":"2024-04-21T08:44:34.483309+0000","last_became_peered":"2024-04-21T08:44:34.483309+0000","last_unstale":"2024-04-21T08:47:00.874684+0000","last_undegraded":"2024-04-21T08:47:00.874684+0000","last_fullsized":"2024-04-21T08:47:00.874684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:39.686048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747340+0000","last_change":"2024-04-21T08:44:34.499159+0000","last_active":"2024-04-21T08:47:00.747340+0000","last_peered":"2024-04-21T08:47:00.747340+0000","last_clean":"2024-04-21T08:47:00.747340+0000","last_became_active":"2024-04-21T08:44:34.496609+0000","last_became_peered":"2024-04-21T08:44:34.496609+0000","last_unstale":"2024-04-21T08:47:00.747340+0000","last_undegraded":"2024-04-21T08:47:00.747340+0000","last_fullsized":"2024-04-21T08:47:00.747340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:11.716321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747233+0000","last_change":"2024-04-21T08:45:27.406543+0000","last_active":"2024-04-21T08:47:00.747233+0000","last_peered":"2024-04-21T08:47:00.747233+0000","last_clean":"2024-04-21T08:47:00.747233+0000","last_became_active":"2024-04-21T08:45:27.406190+0000","last_became_peered":"2024-04-21T08:45:27.406190+0000","last_unstale":"2024-04-21T08:47:00.747233+0000","last_undegraded":"2024-04-21T08:47:00.747233+0000","last_fullsized":"2024-04-21T08:47:00.747233+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:56:17.760163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747327+0000","last_change":"2024-04-21T08:44:34.494018+0000","last_active":"2024-04-21T08:47:00.747327+0000","last_peered":"2024-04-21T08:47:00.747327+0000","last_clean":"2024-04-21T08:47:00.747327+0000","last_became_active":"2024-04-21T08:44:34.493692+0000","last_became_peered":"2024-04-21T08:44:34.493692+0000","last_unstale":"2024-04-21T08:47:00.747327+0000","last_undegraded":"2024-04-21T08:47:00.747327+0000","last_fullsized":"2024-04-21T08:47:00.747327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:25:16.944160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748187+0000","last_change":"2024-04-21T08:44:34.493231+0000","last_active":"2024-04-21T08:47:00.748187+0000","last_peered":"2024-04-21T08:47:00.748187+0000","last_clean":"2024-04-21T08:47:00.748187+0000","last_became_active":"2024-04-21T08:44:34.492826+0000","last_became_peered":"2024-04-21T08:44:34.492826+0000","last_unstale":"2024-04-21T08:47:00.748187+0000","last_undegraded":"2024-04-21T08:47:00.748187+0000","last_fullsized":"2024-04-21T08:47:00.748187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:36.815012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719352+0000","last_change":"2024-04-21T08:44:34.495142+0000","last_active":"2024-04-21T08:47:00.719352+0000","last_peered":"2024-04-21T08:47:00.719352+0000","last_clean":"2024-04-21T08:47:00.719352+0000","last_became_active":"2024-04-21T08:44:34.494803+0000","last_became_peered":"2024-04-21T08:44:34.494803+0000","last_unstale":"2024-04-21T08:47:00.719352+0000","last_undegraded":"2024-04-21T08:47:00.719352+0000","last_fullsized":"2024-04-21T08:47:00.719352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:22.974053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627186+0000","last_change":"2024-04-21T08:44:34.491252+0000","last_active":"2024-04-21T08:47:00.627186+0000","last_peered":"2024-04-21T08:47:00.627186+0000","last_clean":"2024-04-21T08:47:00.627186+0000","last_became_active":"2024-04-21T08:44:34.490814+0000","last_became_peered":"2024-04-21T08:44:34.490814+0000","last_unstale":"2024-04-21T08:47:00.627186+0000","last_undegraded":"2024-04-21T08:47:00.627186+0000","last_fullsized":"2024-04-21T08:47:00.627186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:54:52.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626918+0000","last_change":"2024-04-21T08:44:38.518647+0000","last_active":"2024-04-21T08:47:00.626918+0000","last_peered":"2024-04-21T08:47:00.626918+0000","last_clean":"2024-04-21T08:47:00.626918+0000","last_became_active":"2024-04-21T08:44:38.518239+0000","last_became_peered":"2024-04-21T08:44:38.518239+0000","last_unstale":"2024-04-21T08:47:00.626918+0000","last_undegraded":"2024-04-21T08:47:00.626918+0000","last_fullsized":"2024-04-21T08:47:00.626918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:44.781542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":178,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.174380+0000","last_change":"2024-04-21T08:44:33.484484+0000","last_active":"2024-04-21T08:47:10.174380+0000","last_peered":"2024-04-21T08:47:10.174380+0000","last_clean":"2024-04-21T08:47:10.174380+0000","last_became_active":"2024-04-21T08:44:33.484160+0000","last_became_peered":"2024-04-21T08:44:33.484160+0000","last_unstale":"2024-04-21T08:47:10.174380+0000","last_undegraded":"2024-04-21T08:47:10.174380+0000","last_fullsized":"2024-04-21T08:47:10.174380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:45.962598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718920+0000","last_change":"2024-04-21T08:46:27.484947+0000","last_active":"2024-04-21T08:47:00.718920+0000","last_peered":"2024-04-21T08:47:00.718920+0000","last_clean":"2024-04-21T08:47:00.718920+0000","last_became_active":"2024-04-21T08:46:27.484530+0000","last_became_peered":"2024-04-21T08:46:27.484530+0000","last_unstale":"2024-04-21T08:47:00.718920+0000","last_undegraded":"2024-04-21T08:47:00.718920+0000","last_fullsized":"2024-04-21T08:47:00.718920+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:59.248176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747775+0000","last_change":"2024-04-21T08:45:27.518006+0000","last_active":"2024-04-21T08:47:00.747775+0000","last_peered":"2024-04-21T08:47:00.747775+0000","last_clean":"2024-04-21T08:47:00.747775+0000","last_became_active":"2024-04-21T08:45:27.517409+0000","last_became_peered":"2024-04-21T08:45:27.517409+0000","last_unstale":"2024-04-21T08:47:00.747775+0000","last_undegraded":"2024-04-21T08:47:00.747775+0000","last_fullsized":"2024-04-21T08:47:00.747775+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:45:01.253587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623343+0000","last_change":"2024-04-21T08:44:33.490508+0000","last_active":"2024-04-21T08:47:00.623343+0000","last_peered":"2024-04-21T08:47:00.623343+0000","last_clean":"2024-04-21T08:47:00.623343+0000","last_became_active":"2024-04-21T08:44:33.490084+0000","last_became_peered":"2024-04-21T08:44:33.490084+0000","last_unstale":"2024-04-21T08:47:00.623343+0000","last_undegraded":"2024-04-21T08:47:00.623343+0000","last_fullsized":"2024-04-21T08:47:00.623343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:21.553435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747770+0000","last_change":"2024-04-21T08:44:34.491843+0000","last_active":"2024-04-21T08:47:00.747770+0000","last_peered":"2024-04-21T08:47:00.747770+0000","last_clean":"2024-04-21T08:47:00.747770+0000","last_became_active":"2024-04-21T08:44:34.490578+0000","last_became_peered":"2024-04-21T08:44:34.490578+0000","last_unstale":"2024-04-21T08:47:00.747770+0000","last_undegraded":"2024-04-21T08:47:00.747770+0000","last_fullsized":"2024-04-21T08:47:00.747770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:05:12.272330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748084+0000","last_change":"2024-04-21T08:44:38.520258+0000","last_active":"2024-04-21T08:47:00.748084+0000","last_peered":"2024-04-21T08:47:00.748084+0000","last_clean":"2024-04-21T08:47:00.748084+0000","last_became_active":"2024-04-21T08:44:38.519824+0000","last_became_peered":"2024-04-21T08:44:38.519824+0000","last_unstale":"2024-04-21T08:47:00.748084+0000","last_undegraded":"2024-04-21T08:47:00.748084+0000","last_fullsized":"2024-04-21T08:47:00.748084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:39.716745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721810+0000","last_change":"2024-04-21T08:44:33.484934+0000","last_active":"2024-04-21T08:47:00.721810+0000","last_peered":"2024-04-21T08:47:00.721810+0000","last_clean":"2024-04-21T08:47:00.721810+0000","last_became_active":"2024-04-21T08:44:33.484591+0000","last_became_peered":"2024-04-21T08:44:33.484591+0000","last_unstale":"2024-04-21T08:47:00.721810+0000","last_undegraded":"2024-04-21T08:47:00.721810+0000","last_fullsized":"2024-04-21T08:47:00.721810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:48.826611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747675+0000","last_change":"2024-04-21T08:44:34.501109+0000","last_active":"2024-04-21T08:47:00.747675+0000","last_peered":"2024-04-21T08:47:00.747675+0000","last_clean":"2024-04-21T08:47:00.747675+0000","last_became_active":"2024-04-21T08:44:34.500750+0000","last_became_peered":"2024-04-21T08:44:34.500750+0000","last_unstale":"2024-04-21T08:47:00.747675+0000","last_undegraded":"2024-04-21T08:47:00.747675+0000","last_fullsized":"2024-04-21T08:47:00.747675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:29:28.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747855+0000","last_change":"2024-04-21T08:44:38.519633+0000","last_active":"2024-04-21T08:47:00.747855+0000","last_peered":"2024-04-21T08:47:00.747855+0000","last_clean":"2024-04-21T08:47:00.747855+0000","last_became_active":"2024-04-21T08:44:38.519215+0000","last_became_peered":"2024-04-21T08:44:38.519215+0000","last_unstale":"2024-04-21T08:47:00.747855+0000","last_undegraded":"2024-04-21T08:47:00.747855+0000","last_fullsized":"2024-04-21T08:47:00.747855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:43:59.695844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721745+0000","last_change":"2024-04-21T08:44:33.483267+0000","last_active":"2024-04-21T08:47:00.721745+0000","last_peered":"2024-04-21T08:47:00.721745+0000","last_clean":"2024-04-21T08:47:00.721745+0000","last_became_active":"2024-04-21T08:44:33.482932+0000","last_became_peered":"2024-04-21T08:44:33.482932+0000","last_unstale":"2024-04-21T08:47:00.721745+0000","last_undegraded":"2024-04-21T08:47:00.721745+0000","last_fullsized":"2024-04-21T08:47:00.721745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:53.311550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729284+0000","last_change":"2024-04-21T08:44:34.497389+0000","last_active":"2024-04-21T08:47:00.729284+0000","last_peered":"2024-04-21T08:47:00.729284+0000","last_clean":"2024-04-21T08:47:00.729284+0000","last_became_active":"2024-04-21T08:44:34.497030+0000","last_became_peered":"2024-04-21T08:44:34.497030+0000","last_unstale":"2024-04-21T08:47:00.729284+0000","last_undegraded":"2024-04-21T08:47:00.729284+0000","last_fullsized":"2024-04-21T08:47:00.729284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:20.371888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626966+0000","last_change":"2024-04-21T08:44:38.518095+0000","last_active":"2024-04-21T08:47:00.626966+0000","last_peered":"2024-04-21T08:47:00.626966+0000","last_clean":"2024-04-21T08:47:00.626966+0000","last_became_active":"2024-04-21T08:44:38.517010+0000","last_became_peered":"2024-04-21T08:44:38.517010+0000","last_unstale":"2024-04-21T08:47:00.626966+0000","last_undegraded":"2024-04-21T08:47:00.626966+0000","last_fullsized":"2024-04-21T08:47:00.626966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:02.379512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":180,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.182251+0000","last_change":"2024-04-21T08:44:33.487999+0000","last_active":"2024-04-21T08:47:10.182251+0000","last_peered":"2024-04-21T08:47:10.182251+0000","last_clean":"2024-04-21T08:47:10.182251+0000","last_became_active":"2024-04-21T08:44:33.487666+0000","last_became_peered":"2024-04-21T08:44:33.487666+0000","last_unstale":"2024-04-21T08:47:10.182251+0000","last_undegraded":"2024-04-21T08:47:10.182251+0000","last_fullsized":"2024-04-21T08:47:10.182251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:01.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2103,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721554+0000","last_change":"2024-04-21T08:44:34.494213+0000","last_active":"2024-04-21T08:47:00.721554+0000","last_peered":"2024-04-21T08:47:00.721554+0000","last_clean":"2024-04-21T08:47:00.721554+0000","last_became_active":"2024-04-21T08:44:34.493853+0000","last_became_peered":"2024-04-21T08:44:34.493853+0000","last_unstale":"2024-04-21T08:47:00.721554+0000","last_undegraded":"2024-04-21T08:47:00.721554+0000","last_fullsized":"2024-04-21T08:47:00.721554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:42.499131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627058+0000","last_change":"2024-04-21T08:45:27.410579+0000","last_active":"2024-04-21T08:47:00.627058+0000","last_peered":"2024-04-21T08:47:00.627058+0000","last_clean":"2024-04-21T08:47:00.627058+0000","last_became_active":"2024-04-21T08:45:27.409929+0000","last_became_peered":"2024-04-21T08:45:27.409929+0000","last_unstale":"2024-04-21T08:47:00.627058+0000","last_undegraded":"2024-04-21T08:47:00.627058+0000","last_fullsized":"2024-04-21T08:47:00.627058+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:35.897482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626508+0000","last_change":"2024-04-21T08:44:33.488751+0000","last_active":"2024-04-21T08:47:00.626508+0000","last_peered":"2024-04-21T08:47:00.626508+0000","last_clean":"2024-04-21T08:47:00.626508+0000","last_became_active":"2024-04-21T08:44:33.488425+0000","last_became_peered":"2024-04-21T08:44:33.488425+0000","last_unstale":"2024-04-21T08:47:00.626508+0000","last_undegraded":"2024-04-21T08:47:00.626508+0000","last_fullsized":"2024-04-21T08:47:00.626508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:28:47.868814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729150+0000","last_change":"2024-04-21T08:44:34.492770+0000","last_active":"2024-04-21T08:47:00.729150+0000","last_peered":"2024-04-21T08:47:00.729150+0000","last_clean":"2024-04-21T08:47:00.729150+0000","last_became_active":"2024-04-21T08:44:34.491340+0000","last_became_peered":"2024-04-21T08:44:34.491340+0000","last_unstale":"2024-04-21T08:47:00.729150+0000","last_undegraded":"2024-04-21T08:47:00.729150+0000","last_fullsized":"2024-04-21T08:47:00.729150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:47:54.280990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748097+0000","last_change":"2024-04-21T08:44:38.518888+0000","last_active":"2024-04-21T08:47:00.748097+0000","last_peered":"2024-04-21T08:47:00.748097+0000","last_clean":"2024-04-21T08:47:00.748097+0000","last_became_active":"2024-04-21T08:44:38.518421+0000","last_became_peered":"2024-04-21T08:44:38.518421+0000","last_unstale":"2024-04-21T08:47:00.748097+0000","last_undegraded":"2024-04-21T08:47:00.748097+0000","last_fullsized":"2024-04-21T08:47:00.748097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:21.119364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874482+0000","last_change":"2024-04-21T08:44:33.484383+0000","last_active":"2024-04-21T08:47:00.874482+0000","last_peered":"2024-04-21T08:47:00.874482+0000","last_clean":"2024-04-21T08:47:00.874482+0000","last_became_active":"2024-04-21T08:44:33.484083+0000","last_became_peered":"2024-04-21T08:44:33.484083+0000","last_unstale":"2024-04-21T08:47:00.874482+0000","last_undegraded":"2024-04-21T08:47:00.874482+0000","last_fullsized":"2024-04-21T08:47:00.874482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:13:14.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622126+0000","last_change":"2024-04-21T08:44:34.495636+0000","last_active":"2024-04-21T08:47:00.622126+0000","last_peered":"2024-04-21T08:47:00.622126+0000","last_clean":"2024-04-21T08:47:00.622126+0000","last_became_active":"2024-04-21T08:44:34.494967+0000","last_became_peered":"2024-04-21T08:44:34.494967+0000","last_unstale":"2024-04-21T08:47:00.622126+0000","last_undegraded":"2024-04-21T08:47:00.622126+0000","last_fullsized":"2024-04-21T08:47:00.622126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:47:02.379208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747724+0000","last_change":"2024-04-21T08:44:38.515145+0000","last_active":"2024-04-21T08:47:00.747724+0000","last_peered":"2024-04-21T08:47:00.747724+0000","last_clean":"2024-04-21T08:47:00.747724+0000","last_became_active":"2024-04-21T08:44:38.514755+0000","last_became_peered":"2024-04-21T08:44:38.514755+0000","last_unstale":"2024-04-21T08:47:00.747724+0000","last_undegraded":"2024-04-21T08:47:00.747724+0000","last_fullsized":"2024-04-21T08:47:00.747724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:38:15.826293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728869+0000","last_change":"2024-04-21T08:44:33.484714+0000","last_active":"2024-04-21T08:47:00.728869+0000","last_peered":"2024-04-21T08:47:00.728869+0000","last_clean":"2024-04-21T08:47:00.728869+0000","last_became_active":"2024-04-21T08:44:33.484385+0000","last_became_peered":"2024-04-21T08:44:33.484385+0000","last_unstale":"2024-04-21T08:47:00.728869+0000","last_undegraded":"2024-04-21T08:47:00.728869+0000","last_fullsized":"2024-04-21T08:47:00.728869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:03.406860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718544+0000","last_change":"2024-04-21T08:44:34.490989+0000","last_active":"2024-04-21T08:47:00.718544+0000","last_peered":"2024-04-21T08:47:00.718544+0000","last_clean":"2024-04-21T08:47:00.718544+0000","last_became_active":"2024-04-21T08:44:34.490658+0000","last_became_peered":"2024-04-21T08:44:34.490658+0000","last_unstale":"2024-04-21T08:47:00.718544+0000","last_undegraded":"2024-04-21T08:47:00.718544+0000","last_fullsized":"2024-04-21T08:47:00.718544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:42:15.537913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622288+0000","last_change":"2024-04-21T08:44:38.519723+0000","last_active":"2024-04-21T08:47:00.622288+0000","last_peered":"2024-04-21T08:47:00.622288+0000","last_clean":"2024-04-21T08:47:00.622288+0000","last_became_active":"2024-04-21T08:44:38.519336+0000","last_became_peered":"2024-04-21T08:44:38.519336+0000","last_unstale":"2024-04-21T08:47:00.622288+0000","last_undegraded":"2024-04-21T08:47:00.622288+0000","last_fullsized":"2024-04-21T08:47:00.622288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:00.332153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'6","reported_seq":301,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:11.183116+0000","last_change":"2024-04-21T08:44:33.484404+0000","last_active":"2024-04-21T08:47:11.183116+0000","last_peered":"2024-04-21T08:47:11.183116+0000","last_clean":"2024-04-21T08:47:11.183116+0000","last_became_active":"2024-04-21T08:44:33.484088+0000","last_became_peered":"2024-04-21T08:44:33.484088+0000","last_unstale":"2024-04-21T08:47:11.183116+0000","last_undegraded":"2024-04-21T08:47:11.183116+0000","last_fullsized":"2024-04-21T08:47:11.183116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:41.948622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":262,"num_read_kb":262,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748214+0000","last_change":"2024-04-21T08:44:34.502598+0000","last_active":"2024-04-21T08:47:00.748214+0000","last_peered":"2024-04-21T08:47:00.748214+0000","last_clean":"2024-04-21T08:47:00.748214+0000","last_became_active":"2024-04-21T08:44:34.502057+0000","last_became_peered":"2024-04-21T08:44:34.502057+0000","last_unstale":"2024-04-21T08:47:00.748214+0000","last_undegraded":"2024-04-21T08:47:00.748214+0000","last_fullsized":"2024-04-21T08:47:00.748214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:24.052873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718327+0000","last_change":"2024-04-21T08:44:38.519699+0000","last_active":"2024-04-21T08:47:00.718327+0000","last_peered":"2024-04-21T08:47:00.718327+0000","last_clean":"2024-04-21T08:47:00.718327+0000","last_became_active":"2024-04-21T08:44:38.517655+0000","last_became_peered":"2024-04-21T08:44:38.517655+0000","last_unstale":"2024-04-21T08:47:00.718327+0000","last_undegraded":"2024-04-21T08:47:00.718327+0000","last_fullsized":"2024-04-21T08:47:00.718327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:12.784479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748806+0000","last_change":"2024-04-21T08:46:27.494747+0000","last_active":"2024-04-21T08:47:00.748806+0000","last_peered":"2024-04-21T08:47:00.748806+0000","last_clean":"2024-04-21T08:47:00.748806+0000","last_became_active":"2024-04-21T08:46:27.490469+0000","last_became_peered":"2024-04-21T08:46:27.490469+0000","last_unstale":"2024-04-21T08:47:00.748806+0000","last_undegraded":"2024-04-21T08:47:00.748806+0000","last_fullsized":"2024-04-21T08:47:00.748806+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:16:35.257267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747850+0000","last_change":"2024-04-21T08:45:27.518395+0000","last_active":"2024-04-21T08:47:00.747850+0000","last_peered":"2024-04-21T08:47:00.747850+0000","last_clean":"2024-04-21T08:47:00.747850+0000","last_became_active":"2024-04-21T08:45:27.517969+0000","last_became_peered":"2024-04-21T08:45:27.517969+0000","last_unstale":"2024-04-21T08:47:00.747850+0000","last_undegraded":"2024-04-21T08:47:00.747850+0000","last_fullsized":"2024-04-21T08:47:00.747850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:55.306556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748406+0000","last_change":"2024-04-21T08:44:38.517956+0000","last_active":"2024-04-21T08:47:00.748406+0000","last_peered":"2024-04-21T08:47:00.748406+0000","last_clean":"2024-04-21T08:47:00.748406+0000","last_became_active":"2024-04-21T08:44:38.517422+0000","last_became_peered":"2024-04-21T08:44:38.517422+0000","last_unstale":"2024-04-21T08:47:00.748406+0000","last_undegraded":"2024-04-21T08:47:00.748406+0000","last_fullsized":"2024-04-21T08:47:00.748406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:04.650784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729223+0000","last_change":"2024-04-21T08:44:33.489151+0000","last_active":"2024-04-21T08:47:00.729223+0000","last_peered":"2024-04-21T08:47:00.729223+0000","last_clean":"2024-04-21T08:47:00.729223+0000","last_became_active":"2024-04-21T08:44:33.488781+0000","last_became_peered":"2024-04-21T08:44:33.488781+0000","last_unstale":"2024-04-21T08:47:00.729223+0000","last_undegraded":"2024-04-21T08:47:00.729223+0000","last_fullsized":"2024-04-21T08:47:00.729223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:40.675100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747905+0000","last_change":"2024-04-21T08:44:34.496528+0000","last_active":"2024-04-21T08:47:00.747905+0000","last_peered":"2024-04-21T08:47:00.747905+0000","last_clean":"2024-04-21T08:47:00.747905+0000","last_became_active":"2024-04-21T08:44:34.496223+0000","last_became_peered":"2024-04-21T08:44:34.496223+0000","last_unstale":"2024-04-21T08:47:00.747905+0000","last_undegraded":"2024-04-21T08:47:00.747905+0000","last_fullsized":"2024-04-21T08:47:00.747905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:56.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718461+0000","last_change":"2024-04-21T08:44:38.517537+0000","last_active":"2024-04-21T08:47:00.718461+0000","last_peered":"2024-04-21T08:47:00.718461+0000","last_clean":"2024-04-21T08:47:00.718461+0000","last_became_active":"2024-04-21T08:44:38.517134+0000","last_became_peered":"2024-04-21T08:44:38.517134+0000","last_unstale":"2024-04-21T08:47:00.718461+0000","last_undegraded":"2024-04-21T08:47:00.718461+0000","last_fullsized":"2024-04-21T08:47:00.718461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:38.634418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'41","reported_seq":351,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:11.193441+0000","last_change":"2024-04-21T08:44:33.485582+0000","last_active":"2024-04-21T08:47:11.193441+0000","last_peered":"2024-04-21T08:47:11.193441+0000","last_clean":"2024-04-21T08:47:11.193441+0000","last_became_active":"2024-04-21T08:44:33.485249+0000","last_became_peered":"2024-04-21T08:44:33.485249+0000","last_unstale":"2024-04-21T08:47:11.193441+0000","last_undegraded":"2024-04-21T08:47:11.193441+0000","last_fullsized":"2024-04-21T08:47:11.193441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:16:10.571694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69948,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":277,"num_read_kb":208,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721670+0000","last_change":"2024-04-21T08:44:34.491361+0000","last_active":"2024-04-21T08:47:00.721670+0000","last_peered":"2024-04-21T08:47:00.721670+0000","last_clean":"2024-04-21T08:47:00.721670+0000","last_became_active":"2024-04-21T08:44:34.491050+0000","last_became_peered":"2024-04-21T08:44:34.491050+0000","last_unstale":"2024-04-21T08:47:00.721670+0000","last_undegraded":"2024-04-21T08:47:00.721670+0000","last_fullsized":"2024-04-21T08:47:00.721670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:39.082448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729534+0000","last_change":"2024-04-21T08:44:38.528745+0000","last_active":"2024-04-21T08:47:00.729534+0000","last_peered":"2024-04-21T08:47:00.729534+0000","last_clean":"2024-04-21T08:47:00.729534+0000","last_became_active":"2024-04-21T08:44:38.528275+0000","last_became_peered":"2024-04-21T08:44:38.528275+0000","last_unstale":"2024-04-21T08:47:00.729534+0000","last_undegraded":"2024-04-21T08:47:00.729534+0000","last_fullsized":"2024-04-21T08:47:00.729534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:06:11.338420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626897+0000","last_change":"2024-04-21T08:44:33.482059+0000","last_active":"2024-04-21T08:47:00.626897+0000","last_peered":"2024-04-21T08:47:00.626897+0000","last_clean":"2024-04-21T08:47:00.626897+0000","last_became_active":"2024-04-21T08:44:33.481663+0000","last_became_peered":"2024-04-21T08:44:33.481663+0000","last_unstale":"2024-04-21T08:47:00.626897+0000","last_undegraded":"2024-04-21T08:47:00.626897+0000","last_fullsized":"2024-04-21T08:47:00.626897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:01.240452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874587+0000","last_change":"2024-04-21T08:44:34.491398+0000","last_active":"2024-04-21T08:47:00.874587+0000","last_peered":"2024-04-21T08:47:00.874587+0000","last_clean":"2024-04-21T08:47:00.874587+0000","last_became_active":"2024-04-21T08:44:34.491073+0000","last_became_peered":"2024-04-21T08:44:34.491073+0000","last_unstale":"2024-04-21T08:47:00.874587+0000","last_undegraded":"2024-04-21T08:47:00.874587+0000","last_fullsized":"2024-04-21T08:47:00.874587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:22.566063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747937+0000","last_change":"2024-04-21T08:44:38.515653+0000","last_active":"2024-04-21T08:47:00.747937+0000","last_peered":"2024-04-21T08:47:00.747937+0000","last_clean":"2024-04-21T08:47:00.747937+0000","last_became_active":"2024-04-21T08:44:38.514796+0000","last_became_peered":"2024-04-21T08:44:38.514796+0000","last_unstale":"2024-04-21T08:47:00.747937+0000","last_undegraded":"2024-04-21T08:47:00.747937+0000","last_fullsized":"2024-04-21T08:47:00.747937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:00:22.662136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729003+0000","last_change":"2024-04-21T08:44:33.487073+0000","last_active":"2024-04-21T08:47:00.729003+0000","last_peered":"2024-04-21T08:47:00.729003+0000","last_clean":"2024-04-21T08:47:00.729003+0000","last_became_active":"2024-04-21T08:44:33.486740+0000","last_became_peered":"2024-04-21T08:44:33.486740+0000","last_unstale":"2024-04-21T08:47:00.729003+0000","last_undegraded":"2024-04-21T08:47:00.729003+0000","last_fullsized":"2024-04-21T08:47:00.729003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:32.673074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748512+0000","last_change":"2024-04-21T08:44:34.496757+0000","last_active":"2024-04-21T08:47:00.748512+0000","last_peered":"2024-04-21T08:47:00.748512+0000","last_clean":"2024-04-21T08:47:00.748512+0000","last_became_active":"2024-04-21T08:44:34.496422+0000","last_became_peered":"2024-04-21T08:44:34.496422+0000","last_unstale":"2024-04-21T08:47:00.748512+0000","last_undegraded":"2024-04-21T08:47:00.748512+0000","last_fullsized":"2024-04-21T08:47:00.748512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:08.686810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718978+0000","last_change":"2024-04-21T08:44:38.520935+0000","last_active":"2024-04-21T08:47:00.718978+0000","last_peered":"2024-04-21T08:47:00.718978+0000","last_clean":"2024-04-21T08:47:00.718978+0000","last_became_active":"2024-04-21T08:44:38.520481+0000","last_became_peered":"2024-04-21T08:44:38.520481+0000","last_unstale":"2024-04-21T08:47:00.718978+0000","last_undegraded":"2024-04-21T08:47:00.718978+0000","last_fullsized":"2024-04-21T08:47:00.718978+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:12.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'8","reported_seq":178,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.175644+0000","last_change":"2024-04-21T08:44:33.485797+0000","last_active":"2024-04-21T08:47:10.175644+0000","last_peered":"2024-04-21T08:47:10.175644+0000","last_clean":"2024-04-21T08:47:10.175644+0000","last_became_active":"2024-04-21T08:44:33.485496+0000","last_became_peered":"2024-04-21T08:44:33.485496+0000","last_unstale":"2024-04-21T08:47:10.175644+0000","last_undegraded":"2024-04-21T08:47:10.175644+0000","last_fullsized":"2024-04-21T08:47:10.175644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:11:34.604611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1761,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748282+0000","last_change":"2024-04-21T08:44:34.506453+0000","last_active":"2024-04-21T08:47:00.748282+0000","last_peered":"2024-04-21T08:47:00.748282+0000","last_clean":"2024-04-21T08:47:00.748282+0000","last_became_active":"2024-04-21T08:44:34.506121+0000","last_became_peered":"2024-04-21T08:44:34.506121+0000","last_unstale":"2024-04-21T08:47:00.748282+0000","last_undegraded":"2024-04-21T08:47:00.748282+0000","last_fullsized":"2024-04-21T08:47:00.748282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:07.927366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875292+0000","last_change":"2024-04-21T08:44:38.523718+0000","last_active":"2024-04-21T08:47:00.875292+0000","last_peered":"2024-04-21T08:47:00.875292+0000","last_clean":"2024-04-21T08:47:00.875292+0000","last_became_active":"2024-04-21T08:44:38.523003+0000","last_became_peered":"2024-04-21T08:44:38.523003+0000","last_unstale":"2024-04-21T08:47:00.875292+0000","last_undegraded":"2024-04-21T08:47:00.875292+0000","last_fullsized":"2024-04-21T08:47:00.875292+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:20.837450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721393+0000","last_change":"2024-04-21T08:44:33.482029+0000","last_active":"2024-04-21T08:47:00.721393+0000","last_peered":"2024-04-21T08:47:00.721393+0000","last_clean":"2024-04-21T08:47:00.721393+0000","last_became_active":"2024-04-21T08:44:33.481713+0000","last_became_peered":"2024-04-21T08:44:33.481713+0000","last_unstale":"2024-04-21T08:47:00.721393+0000","last_undegraded":"2024-04-21T08:47:00.721393+0000","last_fullsized":"2024-04-21T08:47:00.721393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:46:34.717244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728921+0000","last_change":"2024-04-21T08:44:34.491944+0000","last_active":"2024-04-21T08:47:00.728921+0000","last_peered":"2024-04-21T08:47:00.728921+0000","last_clean":"2024-04-21T08:47:00.728921+0000","last_became_active":"2024-04-21T08:44:34.491638+0000","last_became_peered":"2024-04-21T08:44:34.491638+0000","last_unstale":"2024-04-21T08:47:00.728921+0000","last_undegraded":"2024-04-21T08:47:00.728921+0000","last_fullsized":"2024-04-21T08:47:00.728921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:08:07.925937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728793+0000","last_change":"2024-04-21T08:44:38.528971+0000","last_active":"2024-04-21T08:47:00.728793+0000","last_peered":"2024-04-21T08:47:00.728793+0000","last_clean":"2024-04-21T08:47:00.728793+0000","last_became_active":"2024-04-21T08:44:38.528453+0000","last_became_peered":"2024-04-21T08:44:38.528453+0000","last_unstale":"2024-04-21T08:47:00.728793+0000","last_undegraded":"2024-04-21T08:47:00.728793+0000","last_fullsized":"2024-04-21T08:47:00.728793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:50:31.237749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721764+0000","last_change":"2024-04-21T08:44:33.479262+0000","last_active":"2024-04-21T08:47:00.721764+0000","last_peered":"2024-04-21T08:47:00.721764+0000","last_clean":"2024-04-21T08:47:00.721764+0000","last_became_active":"2024-04-21T08:44:33.478907+0000","last_became_peered":"2024-04-21T08:44:33.478907+0000","last_unstale":"2024-04-21T08:47:00.721764+0000","last_undegraded":"2024-04-21T08:47:00.721764+0000","last_fullsized":"2024-04-21T08:47:00.721764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:33:43.879126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748024+0000","last_change":"2024-04-21T08:44:34.498503+0000","last_active":"2024-04-21T08:47:00.748024+0000","last_peered":"2024-04-21T08:47:00.748024+0000","last_clean":"2024-04-21T08:47:00.748024+0000","last_became_active":"2024-04-21T08:44:34.495140+0000","last_became_peered":"2024-04-21T08:44:34.495140+0000","last_unstale":"2024-04-21T08:47:00.748024+0000","last_undegraded":"2024-04-21T08:47:00.748024+0000","last_fullsized":"2024-04-21T08:47:00.748024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:24.518349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721372+0000","last_change":"2024-04-21T08:44:38.517717+0000","last_active":"2024-04-21T08:47:00.721372+0000","last_peered":"2024-04-21T08:47:00.721372+0000","last_clean":"2024-04-21T08:47:00.721372+0000","last_became_active":"2024-04-21T08:44:38.517272+0000","last_became_peered":"2024-04-21T08:44:38.517272+0000","last_unstale":"2024-04-21T08:47:00.721372+0000","last_undegraded":"2024-04-21T08:47:00.721372+0000","last_fullsized":"2024-04-21T08:47:00.721372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.664130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718792+0000","last_change":"2024-04-21T08:44:33.477498+0000","last_active":"2024-04-21T08:47:00.718792+0000","last_peered":"2024-04-21T08:47:00.718792+0000","last_clean":"2024-04-21T08:47:00.718792+0000","last_became_active":"2024-04-21T08:44:33.477139+0000","last_became_peered":"2024-04-21T08:44:33.477139+0000","last_unstale":"2024-04-21T08:47:00.718792+0000","last_undegraded":"2024-04-21T08:47:00.718792+0000","last_fullsized":"2024-04-21T08:47:00.718792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:26:59.390583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626701+0000","last_change":"2024-04-21T08:44:34.493586+0000","last_active":"2024-04-21T08:47:00.626701+0000","last_peered":"2024-04-21T08:47:00.626701+0000","last_clean":"2024-04-21T08:47:00.626701+0000","last_became_active":"2024-04-21T08:44:34.493258+0000","last_became_peered":"2024-04-21T08:44:34.493258+0000","last_unstale":"2024-04-21T08:47:00.626701+0000","last_undegraded":"2024-04-21T08:47:00.626701+0000","last_fullsized":"2024-04-21T08:47:00.626701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:11:18.783332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874853+0000","last_change":"2024-04-21T08:44:38.519783+0000","last_active":"2024-04-21T08:47:00.874853+0000","last_peered":"2024-04-21T08:47:00.874853+0000","last_clean":"2024-04-21T08:47:00.874853+0000","last_became_active":"2024-04-21T08:44:38.519397+0000","last_became_peered":"2024-04-21T08:44:38.519397+0000","last_unstale":"2024-04-21T08:47:00.874853+0000","last_undegraded":"2024-04-21T08:47:00.874853+0000","last_fullsized":"2024-04-21T08:47:00.874853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:08.740240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":170,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.175110+0000","last_change":"2024-04-21T08:44:33.490098+0000","last_active":"2024-04-21T08:47:10.175110+0000","last_peered":"2024-04-21T08:47:10.175110+0000","last_clean":"2024-04-21T08:47:10.175110+0000","last_became_active":"2024-04-21T08:44:33.489691+0000","last_became_peered":"2024-04-21T08:44:33.489691+0000","last_unstale":"2024-04-21T08:47:10.175110+0000","last_undegraded":"2024-04-21T08:47:10.175110+0000","last_fullsized":"2024-04-21T08:47:10.175110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:44.965998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626543+0000","last_change":"2024-04-21T08:46:27.489580+0000","last_active":"2024-04-21T08:47:00.626543+0000","last_peered":"2024-04-21T08:47:00.626543+0000","last_clean":"2024-04-21T08:47:00.626543+0000","last_became_active":"2024-04-21T08:46:27.489086+0000","last_became_peered":"2024-04-21T08:46:27.489086+0000","last_unstale":"2024-04-21T08:47:00.626543+0000","last_undegraded":"2024-04-21T08:47:00.626543+0000","last_fullsized":"2024-04-21T08:47:00.626543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:38:50.618957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874908+0000","last_change":"2024-04-21T08:44:38.521010+0000","last_active":"2024-04-21T08:47:00.874908+0000","last_peered":"2024-04-21T08:47:00.874908+0000","last_clean":"2024-04-21T08:47:00.874908+0000","last_became_active":"2024-04-21T08:44:38.520605+0000","last_became_peered":"2024-04-21T08:44:38.520605+0000","last_unstale":"2024-04-21T08:47:00.874908+0000","last_undegraded":"2024-04-21T08:47:00.874908+0000","last_fullsized":"2024-04-21T08:47:00.874908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:50.378803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728846+0000","last_change":"2024-04-21T08:45:27.410137+0000","last_active":"2024-04-21T08:47:00.728846+0000","last_peered":"2024-04-21T08:47:00.728846+0000","last_clean":"2024-04-21T08:47:00.728846+0000","last_became_active":"2024-04-21T08:45:27.409728+0000","last_became_peered":"2024-04-21T08:45:27.409728+0000","last_unstale":"2024-04-21T08:47:00.728846+0000","last_undegraded":"2024-04-21T08:47:00.728846+0000","last_fullsized":"2024-04-21T08:47:00.728846+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:17.934902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":172,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.175058+0000","last_change":"2024-04-21T08:44:33.494031+0000","last_active":"2024-04-21T08:47:10.175058+0000","last_peered":"2024-04-21T08:47:10.175058+0000","last_clean":"2024-04-21T08:47:10.175058+0000","last_became_active":"2024-04-21T08:44:33.493580+0000","last_became_peered":"2024-04-21T08:44:33.493580+0000","last_unstale":"2024-04-21T08:47:10.175058+0000","last_undegraded":"2024-04-21T08:47:10.175058+0000","last_fullsized":"2024-04-21T08:47:10.175058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:39:48.230616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748658+0000","last_change":"2024-04-21T08:44:38.518067+0000","last_active":"2024-04-21T08:47:00.748658+0000","last_peered":"2024-04-21T08:47:00.748658+0000","last_clean":"2024-04-21T08:47:00.748658+0000","last_became_active":"2024-04-21T08:44:38.517594+0000","last_became_peered":"2024-04-21T08:44:38.517594+0000","last_unstale":"2024-04-21T08:47:00.748658+0000","last_undegraded":"2024-04-21T08:47:00.748658+0000","last_fullsized":"2024-04-21T08:47:00.748658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:35.261582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718443+0000","last_change":"2024-04-21T08:45:27.517719+0000","last_active":"2024-04-21T08:47:00.718443+0000","last_peered":"2024-04-21T08:47:00.718443+0000","last_clean":"2024-04-21T08:47:00.718443+0000","last_became_active":"2024-04-21T08:45:27.517429+0000","last_became_peered":"2024-04-21T08:45:27.517429+0000","last_unstale":"2024-04-21T08:47:00.718443+0000","last_undegraded":"2024-04-21T08:47:00.718443+0000","last_fullsized":"2024-04-21T08:47:00.718443+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:49:34.097759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748213+0000","last_change":"2024-04-21T08:44:33.485110+0000","last_active":"2024-04-21T08:47:00.748213+0000","last_peered":"2024-04-21T08:47:00.748213+0000","last_clean":"2024-04-21T08:47:00.748213+0000","last_became_active":"2024-04-21T08:44:33.484722+0000","last_became_peered":"2024-04-21T08:44:33.484722+0000","last_unstale":"2024-04-21T08:47:00.748213+0000","last_undegraded":"2024-04-21T08:47:00.748213+0000","last_fullsized":"2024-04-21T08:47:00.748213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:47:16.773560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"94'148","reported_seq":412,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:02.744643+0000","last_change":"2024-04-21T08:43:17.984594+0000","last_active":"2024-04-21T08:47:02.744643+0000","last_peered":"2024-04-21T08:47:02.744643+0000","last_clean":"2024-04-21T08:47:02.744643+0000","last_became_active":"2024-04-21T08:43:17.984207+0000","last_became_peered":"2024-04-21T08:43:17.984207+0000","last_unstale":"2024-04-21T08:47:02.744643+0000","last_undegraded":"2024-04-21T08:47:02.744643+0000","last_fullsized":"2024-04-21T08:47:02.744643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_clean_scrub_stamp":"2024-04-21T08:39:50.910206+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:35.260499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721571+0000","last_change":"2024-04-21T08:45:27.518834+0000","last_active":"2024-04-21T08:47:00.721571+0000","last_peered":"2024-04-21T08:47:00.721571+0000","last_clean":"2024-04-21T08:47:00.721571+0000","last_became_active":"2024-04-21T08:45:27.518390+0000","last_became_peered":"2024-04-21T08:45:27.518390+0000","last_unstale":"2024-04-21T08:47:00.721571+0000","last_undegraded":"2024-04-21T08:47:00.721571+0000","last_fullsized":"2024-04-21T08:47:00.721571+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:58:32.673008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874296+0000","last_change":"2024-04-21T08:44:34.499975+0000","last_active":"2024-04-21T08:47:00.874296+0000","last_peered":"2024-04-21T08:47:00.874296+0000","last_clean":"2024-04-21T08:47:00.874296+0000","last_became_active":"2024-04-21T08:44:34.499646+0000","last_became_peered":"2024-04-21T08:44:34.499646+0000","last_unstale":"2024-04-21T08:47:00.874296+0000","last_undegraded":"2024-04-21T08:47:00.874296+0000","last_fullsized":"2024-04-21T08:47:00.874296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:19:38.082502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748114+0000","last_change":"2024-04-21T08:44:33.484449+0000","last_active":"2024-04-21T08:47:00.748114+0000","last_peered":"2024-04-21T08:47:00.748114+0000","last_clean":"2024-04-21T08:47:00.748114+0000","last_became_active":"2024-04-21T08:44:33.484155+0000","last_became_peered":"2024-04-21T08:44:33.484155+0000","last_unstale":"2024-04-21T08:47:00.748114+0000","last_undegraded":"2024-04-21T08:47:00.748114+0000","last_fullsized":"2024-04-21T08:47:00.748114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:03.480474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747766+0000","last_change":"2024-04-21T08:44:38.515283+0000","last_active":"2024-04-21T08:47:00.747766+0000","last_peered":"2024-04-21T08:47:00.747766+0000","last_clean":"2024-04-21T08:47:00.747766+0000","last_became_active":"2024-04-21T08:44:38.514764+0000","last_became_peered":"2024-04-21T08:44:38.514764+0000","last_unstale":"2024-04-21T08:47:00.747766+0000","last_undegraded":"2024-04-21T08:47:00.747766+0000","last_fullsized":"2024-04-21T08:47:00.747766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:17:36.378392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721569+0000","last_change":"2024-04-21T08:44:33.489198+0000","last_active":"2024-04-21T08:47:00.721569+0000","last_peered":"2024-04-21T08:47:00.721569+0000","last_clean":"2024-04-21T08:47:00.721569+0000","last_became_active":"2024-04-21T08:44:33.488794+0000","last_became_peered":"2024-04-21T08:44:33.488794+0000","last_unstale":"2024-04-21T08:47:00.721569+0000","last_undegraded":"2024-04-21T08:47:00.721569+0000","last_fullsized":"2024-04-21T08:47:00.721569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:24.948283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874977+0000","last_change":"2024-04-21T08:44:34.488811+0000","last_active":"2024-04-21T08:47:00.874977+0000","last_peered":"2024-04-21T08:47:00.874977+0000","last_clean":"2024-04-21T08:47:00.874977+0000","last_became_active":"2024-04-21T08:44:34.488472+0000","last_became_peered":"2024-04-21T08:44:34.488472+0000","last_unstale":"2024-04-21T08:47:00.874977+0000","last_undegraded":"2024-04-21T08:47:00.874977+0000","last_fullsized":"2024-04-21T08:47:00.874977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:44.779492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622045+0000","last_change":"2024-04-21T08:44:38.522711+0000","last_active":"2024-04-21T08:47:00.622045+0000","last_peered":"2024-04-21T08:47:00.622045+0000","last_clean":"2024-04-21T08:47:00.622045+0000","last_became_active":"2024-04-21T08:44:38.522322+0000","last_became_peered":"2024-04-21T08:44:38.522322+0000","last_unstale":"2024-04-21T08:47:00.622045+0000","last_undegraded":"2024-04-21T08:47:00.622045+0000","last_fullsized":"2024-04-21T08:47:00.622045+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:26:13.323283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'102","reported_seq":227,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.078157+0000","last_change":"2024-04-21T08:44:33.481815+0000","last_active":"2024-04-21T08:47:10.078157+0000","last_peered":"2024-04-21T08:47:10.078157+0000","last_clean":"2024-04-21T08:47:10.078157+0000","last_became_active":"2024-04-21T08:44:33.481469+0000","last_became_peered":"2024-04-21T08:44:33.481469+0000","last_unstale":"2024-04-21T08:47:10.078157+0000","last_undegraded":"2024-04-21T08:47:10.078157+0000","last_fullsized":"2024-04-21T08:47:10.078157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:13.970821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":185,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623173+0000","last_change":"2024-04-21T08:44:34.496996+0000","last_active":"2024-04-21T08:47:00.623173+0000","last_peered":"2024-04-21T08:47:00.623173+0000","last_clean":"2024-04-21T08:47:00.623173+0000","last_became_active":"2024-04-21T08:44:34.496676+0000","last_became_peered":"2024-04-21T08:44:34.496676+0000","last_unstale":"2024-04-21T08:47:00.623173+0000","last_undegraded":"2024-04-21T08:47:00.623173+0000","last_fullsized":"2024-04-21T08:47:00.623173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:43:08.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626592+0000","last_change":"2024-04-21T08:44:38.516240+0000","last_active":"2024-04-21T08:47:00.626592+0000","last_peered":"2024-04-21T08:47:00.626592+0000","last_clean":"2024-04-21T08:47:00.626592+0000","last_became_active":"2024-04-21T08:44:38.515801+0000","last_became_peered":"2024-04-21T08:44:38.515801+0000","last_unstale":"2024-04-21T08:47:00.626592+0000","last_undegraded":"2024-04-21T08:47:00.626592+0000","last_fullsized":"2024-04-21T08:47:00.626592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:46.520373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749302+0000","last_change":"2024-04-21T08:44:33.488952+0000","last_active":"2024-04-21T08:47:00.749302+0000","last_peered":"2024-04-21T08:47:00.749302+0000","last_clean":"2024-04-21T08:47:00.749302+0000","last_became_active":"2024-04-21T08:44:33.488617+0000","last_became_peered":"2024-04-21T08:44:33.488617+0000","last_unstale":"2024-04-21T08:47:00.749302+0000","last_undegraded":"2024-04-21T08:47:00.749302+0000","last_fullsized":"2024-04-21T08:47:00.749302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:37:24.679132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747751+0000","last_change":"2024-04-21T08:44:34.510568+0000","last_active":"2024-04-21T08:47:00.747751+0000","last_peered":"2024-04-21T08:47:00.747751+0000","last_clean":"2024-04-21T08:47:00.747751+0000","last_became_active":"2024-04-21T08:44:34.509900+0000","last_became_peered":"2024-04-21T08:44:34.509900+0000","last_unstale":"2024-04-21T08:47:00.747751+0000","last_undegraded":"2024-04-21T08:47:00.747751+0000","last_fullsized":"2024-04-21T08:47:00.747751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:08.426141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748835+0000","last_change":"2024-04-21T08:45:27.976874+0000","last_active":"2024-04-21T08:47:00.748835+0000","last_peered":"2024-04-21T08:47:00.748835+0000","last_clean":"2024-04-21T08:47:00.748835+0000","last_became_active":"2024-04-21T08:45:27.976387+0000","last_became_peered":"2024-04-21T08:45:27.976387+0000","last_unstale":"2024-04-21T08:47:00.748835+0000","last_undegraded":"2024-04-21T08:47:00.748835+0000","last_fullsized":"2024-04-21T08:47:00.748835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:38:18.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747268+0000","last_change":"2024-04-21T08:44:33.481803+0000","last_active":"2024-04-21T08:47:00.747268+0000","last_peered":"2024-04-21T08:47:00.747268+0000","last_clean":"2024-04-21T08:47:00.747268+0000","last_became_active":"2024-04-21T08:44:33.481256+0000","last_became_peered":"2024-04-21T08:44:33.481256+0000","last_unstale":"2024-04-21T08:47:00.747268+0000","last_undegraded":"2024-04-21T08:47:00.747268+0000","last_fullsized":"2024-04-21T08:47:00.747268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:17.804557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721258+0000","last_change":"2024-04-21T08:44:34.490206+0000","last_active":"2024-04-21T08:47:00.721258+0000","last_peered":"2024-04-21T08:47:00.721258+0000","last_clean":"2024-04-21T08:47:00.721258+0000","last_became_active":"2024-04-21T08:44:34.489797+0000","last_became_peered":"2024-04-21T08:44:34.489797+0000","last_unstale":"2024-04-21T08:47:00.721258+0000","last_undegraded":"2024-04-21T08:47:00.721258+0000","last_fullsized":"2024-04-21T08:47:00.721258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:09.055429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719147+0000","last_change":"2024-04-21T08:44:38.517385+0000","last_active":"2024-04-21T08:47:00.719147+0000","last_peered":"2024-04-21T08:47:00.719147+0000","last_clean":"2024-04-21T08:47:00.719147+0000","last_became_active":"2024-04-21T08:44:38.516964+0000","last_became_peered":"2024-04-21T08:44:38.516964+0000","last_unstale":"2024-04-21T08:47:00.719147+0000","last_undegraded":"2024-04-21T08:47:00.719147+0000","last_fullsized":"2024-04-21T08:47:00.719147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:33.364076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622039+0000","last_change":"2024-04-21T08:44:33.490586+0000","last_active":"2024-04-21T08:47:00.622039+0000","last_peered":"2024-04-21T08:47:00.622039+0000","last_clean":"2024-04-21T08:47:00.622039+0000","last_became_active":"2024-04-21T08:44:33.489281+0000","last_became_peered":"2024-04-21T08:44:33.489281+0000","last_unstale":"2024-04-21T08:47:00.622039+0000","last_undegraded":"2024-04-21T08:47:00.622039+0000","last_fullsized":"2024-04-21T08:47:00.622039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:41:29.856165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747602+0000","last_change":"2024-04-21T08:44:34.493394+0000","last_active":"2024-04-21T08:47:00.747602+0000","last_peered":"2024-04-21T08:47:00.747602+0000","last_clean":"2024-04-21T08:47:00.747602+0000","last_became_active":"2024-04-21T08:44:34.492367+0000","last_became_peered":"2024-04-21T08:44:34.492367+0000","last_unstale":"2024-04-21T08:47:00.747602+0000","last_undegraded":"2024-04-21T08:47:00.747602+0000","last_fullsized":"2024-04-21T08:47:00.747602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:13.111266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748165+0000","last_change":"2024-04-21T08:44:38.519113+0000","last_active":"2024-04-21T08:47:00.748165+0000","last_peered":"2024-04-21T08:47:00.748165+0000","last_clean":"2024-04-21T08:47:00.748165+0000","last_became_active":"2024-04-21T08:44:38.518469+0000","last_became_peered":"2024-04-21T08:44:38.518469+0000","last_unstale":"2024-04-21T08:47:00.748165+0000","last_undegraded":"2024-04-21T08:47:00.748165+0000","last_fullsized":"2024-04-21T08:47:00.748165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:39:22.200041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'5","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747723+0000","last_change":"2024-04-21T08:46:27.492549+0000","last_active":"2024-04-21T08:47:00.747723+0000","last_peered":"2024-04-21T08:47:00.747723+0000","last_clean":"2024-04-21T08:47:00.747723+0000","last_became_active":"2024-04-21T08:46:27.488054+0000","last_became_peered":"2024-04-21T08:46:27.488054+0000","last_unstale":"2024-04-21T08:47:00.747723+0000","last_undegraded":"2024-04-21T08:47:00.747723+0000","last_fullsized":"2024-04-21T08:47:00.747723+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:51:54.797532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622446+0000","last_change":"2024-04-21T08:44:34.502226+0000","last_active":"2024-04-21T08:47:00.622446+0000","last_peered":"2024-04-21T08:47:00.622446+0000","last_clean":"2024-04-21T08:47:00.622446+0000","last_became_active":"2024-04-21T08:44:34.501775+0000","last_became_peered":"2024-04-21T08:44:34.501775+0000","last_unstale":"2024-04-21T08:47:00.622446+0000","last_undegraded":"2024-04-21T08:47:00.622446+0000","last_fullsized":"2024-04-21T08:47:00.622446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:58.325267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748655+0000","last_change":"2024-04-21T08:44:38.522250+0000","last_active":"2024-04-21T08:47:00.748655+0000","last_peered":"2024-04-21T08:47:00.748655+0000","last_clean":"2024-04-21T08:47:00.748655+0000","last_became_active":"2024-04-21T08:44:38.520870+0000","last_became_peered":"2024-04-21T08:44:38.520870+0000","last_unstale":"2024-04-21T08:47:00.748655+0000","last_undegraded":"2024-04-21T08:47:00.748655+0000","last_fullsized":"2024-04-21T08:47:00.748655+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:02.391785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747761+0000","last_change":"2024-04-21T08:46:27.496215+0000","last_active":"2024-04-21T08:47:00.747761+0000","last_peered":"2024-04-21T08:47:00.747761+0000","last_clean":"2024-04-21T08:47:00.747761+0000","last_became_active":"2024-04-21T08:46:27.488287+0000","last_became_peered":"2024-04-21T08:46:27.488287+0000","last_unstale":"2024-04-21T08:47:00.747761+0000","last_undegraded":"2024-04-21T08:47:00.747761+0000","last_fullsized":"2024-04-21T08:47:00.747761+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:08:05.390563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749473+0000","last_change":"2024-04-21T08:44:34.496034+0000","last_active":"2024-04-21T08:47:00.749473+0000","last_peered":"2024-04-21T08:47:00.749473+0000","last_clean":"2024-04-21T08:47:00.749473+0000","last_became_active":"2024-04-21T08:44:34.495641+0000","last_became_peered":"2024-04-21T08:44:34.495641+0000","last_unstale":"2024-04-21T08:47:00.749473+0000","last_undegraded":"2024-04-21T08:47:00.749473+0000","last_fullsized":"2024-04-21T08:47:00.749473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:21:18.893326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747361+0000","last_change":"2024-04-21T08:44:38.517288+0000","last_active":"2024-04-21T08:47:00.747361+0000","last_peered":"2024-04-21T08:47:00.747361+0000","last_clean":"2024-04-21T08:47:00.747361+0000","last_became_active":"2024-04-21T08:44:38.516853+0000","last_became_peered":"2024-04-21T08:44:38.516853+0000","last_unstale":"2024-04-21T08:47:00.747361+0000","last_undegraded":"2024-04-21T08:47:00.747361+0000","last_fullsized":"2024-04-21T08:47:00.747361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:27:22.371703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":175,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.251660+0000","last_change":"2024-04-21T08:44:33.491578+0000","last_active":"2024-04-21T08:47:10.251660+0000","last_peered":"2024-04-21T08:47:10.251660+0000","last_clean":"2024-04-21T08:47:10.251660+0000","last_became_active":"2024-04-21T08:44:33.491238+0000","last_became_peered":"2024-04-21T08:44:33.491238+0000","last_unstale":"2024-04-21T08:47:10.251660+0000","last_undegraded":"2024-04-21T08:47:10.251660+0000","last_fullsized":"2024-04-21T08:47:10.251660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:50.399031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2343,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749074+0000","last_change":"2024-04-21T08:44:34.497632+0000","last_active":"2024-04-21T08:47:00.749074+0000","last_peered":"2024-04-21T08:47:00.749074+0000","last_clean":"2024-04-21T08:47:00.749074+0000","last_became_active":"2024-04-21T08:44:34.497318+0000","last_became_peered":"2024-04-21T08:44:34.497318+0000","last_unstale":"2024-04-21T08:47:00.749074+0000","last_undegraded":"2024-04-21T08:47:00.749074+0000","last_fullsized":"2024-04-21T08:47:00.749074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:11:33.918530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747370+0000","last_change":"2024-04-21T08:44:38.518414+0000","last_active":"2024-04-21T08:47:00.747370+0000","last_peered":"2024-04-21T08:47:00.747370+0000","last_clean":"2024-04-21T08:47:00.747370+0000","last_became_active":"2024-04-21T08:44:38.517953+0000","last_became_peered":"2024-04-21T08:44:38.517953+0000","last_unstale":"2024-04-21T08:47:00.747370+0000","last_undegraded":"2024-04-21T08:47:00.747370+0000","last_fullsized":"2024-04-21T08:47:00.747370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:48.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'67","reported_seq":132,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:09.745995+0000","last_change":"2024-04-21T08:44:33.493896+0000","last_active":"2024-04-21T08:47:09.745995+0000","last_peered":"2024-04-21T08:47:09.745995+0000","last_clean":"2024-04-21T08:47:09.745995+0000","last_became_active":"2024-04-21T08:44:33.493566+0000","last_became_peered":"2024-04-21T08:44:33.493566+0000","last_unstale":"2024-04-21T08:47:09.745995+0000","last_undegraded":"2024-04-21T08:47:09.745995+0000","last_fullsized":"2024-04-21T08:47:09.745995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:47:17.307690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748612+0000","last_change":"2024-04-21T08:44:34.500406+0000","last_active":"2024-04-21T08:47:00.748612+0000","last_peered":"2024-04-21T08:47:00.748612+0000","last_clean":"2024-04-21T08:47:00.748612+0000","last_became_active":"2024-04-21T08:44:34.500048+0000","last_became_peered":"2024-04-21T08:44:34.500048+0000","last_unstale":"2024-04-21T08:47:00.748612+0000","last_undegraded":"2024-04-21T08:47:00.748612+0000","last_fullsized":"2024-04-21T08:47:00.748612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:10:58.166155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728633+0000","last_change":"2024-04-21T08:44:38.514812+0000","last_active":"2024-04-21T08:47:00.728633+0000","last_peered":"2024-04-21T08:47:00.728633+0000","last_clean":"2024-04-21T08:47:00.728633+0000","last_became_active":"2024-04-21T08:44:38.514247+0000","last_became_peered":"2024-04-21T08:44:38.514247+0000","last_unstale":"2024-04-21T08:47:00.728633+0000","last_undegraded":"2024-04-21T08:47:00.728633+0000","last_fullsized":"2024-04-21T08:47:00.728633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:06:10.588453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875149+0000","last_change":"2024-04-21T08:44:33.493887+0000","last_active":"2024-04-21T08:47:00.875149+0000","last_peered":"2024-04-21T08:47:00.875149+0000","last_clean":"2024-04-21T08:47:00.875149+0000","last_became_active":"2024-04-21T08:44:33.493555+0000","last_became_peered":"2024-04-21T08:44:33.493555+0000","last_unstale":"2024-04-21T08:47:00.875149+0000","last_undegraded":"2024-04-21T08:47:00.875149+0000","last_fullsized":"2024-04-21T08:47:00.875149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:18.089102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749096+0000","last_change":"2024-04-21T08:44:34.503259+0000","last_active":"2024-04-21T08:47:00.749096+0000","last_peered":"2024-04-21T08:47:00.749096+0000","last_clean":"2024-04-21T08:47:00.749096+0000","last_became_active":"2024-04-21T08:44:34.502845+0000","last_became_peered":"2024-04-21T08:44:34.502845+0000","last_unstale":"2024-04-21T08:47:00.749096+0000","last_undegraded":"2024-04-21T08:47:00.749096+0000","last_fullsized":"2024-04-21T08:47:00.749096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:19.902178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747653+0000","last_change":"2024-04-21T08:44:34.497608+0000","last_active":"2024-04-21T08:47:00.747653+0000","last_peered":"2024-04-21T08:47:00.747653+0000","last_clean":"2024-04-21T08:47:00.747653+0000","last_became_active":"2024-04-21T08:44:34.497255+0000","last_became_peered":"2024-04-21T08:44:34.497255+0000","last_unstale":"2024-04-21T08:47:00.747653+0000","last_undegraded":"2024-04-21T08:47:00.747653+0000","last_fullsized":"2024-04-21T08:47:00.747653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:51:15.181837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626828+0000","last_change":"2024-04-21T08:44:34.495408+0000","last_active":"2024-04-21T08:47:00.626828+0000","last_peered":"2024-04-21T08:47:00.626828+0000","last_clean":"2024-04-21T08:47:00.626828+0000","last_became_active":"2024-04-21T08:44:34.495038+0000","last_became_peered":"2024-04-21T08:44:34.495038+0000","last_unstale":"2024-04-21T08:47:00.626828+0000","last_undegraded":"2024-04-21T08:47:00.626828+0000","last_fullsized":"2024-04-21T08:47:00.626828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:59.512245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626703+0000","last_change":"2024-04-21T08:44:34.493292+0000","last_active":"2024-04-21T08:47:00.626703+0000","last_peered":"2024-04-21T08:47:00.626703+0000","last_clean":"2024-04-21T08:47:00.626703+0000","last_became_active":"2024-04-21T08:44:34.492913+0000","last_became_peered":"2024-04-21T08:44:34.492913+0000","last_unstale":"2024-04-21T08:47:00.626703+0000","last_undegraded":"2024-04-21T08:47:00.626703+0000","last_fullsized":"2024-04-21T08:47:00.626703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:24.271859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747551+0000","last_change":"2024-04-21T08:44:34.510448+0000","last_active":"2024-04-21T08:47:00.747551+0000","last_peered":"2024-04-21T08:47:00.747551+0000","last_clean":"2024-04-21T08:47:00.747551+0000","last_became_active":"2024-04-21T08:44:34.509958+0000","last_became_peered":"2024-04-21T08:44:34.509958+0000","last_unstale":"2024-04-21T08:47:00.747551+0000","last_undegraded":"2024-04-21T08:47:00.747551+0000","last_fullsized":"2024-04-21T08:47:00.747551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:36.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875000+0000","last_change":"2024-04-21T08:44:34.497415+0000","last_active":"2024-04-21T08:47:00.875000+0000","last_peered":"2024-04-21T08:47:00.875000+0000","last_clean":"2024-04-21T08:47:00.875000+0000","last_became_active":"2024-04-21T08:44:34.497120+0000","last_became_peered":"2024-04-21T08:44:34.497120+0000","last_unstale":"2024-04-21T08:47:00.875000+0000","last_undegraded":"2024-04-21T08:47:00.875000+0000","last_fullsized":"2024-04-21T08:47:00.875000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:30.232338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747974+0000","last_change":"2024-04-21T08:44:34.500356+0000","last_active":"2024-04-21T08:47:00.747974+0000","last_peered":"2024-04-21T08:47:00.747974+0000","last_clean":"2024-04-21T08:47:00.747974+0000","last_became_active":"2024-04-21T08:44:34.500032+0000","last_became_peered":"2024-04-21T08:44:34.500032+0000","last_unstale":"2024-04-21T08:47:00.747974+0000","last_undegraded":"2024-04-21T08:47:00.747974+0000","last_fullsized":"2024-04-21T08:47:00.747974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:24.900221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627464+0000","last_change":"2024-04-21T08:44:34.493629+0000","last_active":"2024-04-21T08:47:00.627464+0000","last_peered":"2024-04-21T08:47:00.627464+0000","last_clean":"2024-04-21T08:47:00.627464+0000","last_became_active":"2024-04-21T08:44:34.493273+0000","last_became_peered":"2024-04-21T08:44:34.493273+0000","last_unstale":"2024-04-21T08:47:00.627464+0000","last_undegraded":"2024-04-21T08:47:00.627464+0000","last_fullsized":"2024-04-21T08:47:00.627464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:22:42.698957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623249+0000","last_change":"2024-04-21T08:44:34.493427+0000","last_active":"2024-04-21T08:47:00.623249+0000","last_peered":"2024-04-21T08:47:00.623249+0000","last_clean":"2024-04-21T08:47:00.623249+0000","last_became_active":"2024-04-21T08:44:34.493105+0000","last_became_peered":"2024-04-21T08:44:34.493105+0000","last_unstale":"2024-04-21T08:47:00.623249+0000","last_undegraded":"2024-04-21T08:47:00.623249+0000","last_fullsized":"2024-04-21T08:47:00.623249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:37.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874153+0000","last_change":"2024-04-21T08:45:27.515899+0000","last_active":"2024-04-21T08:47:00.874153+0000","last_peered":"2024-04-21T08:47:00.874153+0000","last_clean":"2024-04-21T08:47:00.874153+0000","last_became_active":"2024-04-21T08:45:27.515567+0000","last_became_peered":"2024-04-21T08:45:27.515567+0000","last_unstale":"2024-04-21T08:47:00.874153+0000","last_undegraded":"2024-04-21T08:47:00.874153+0000","last_fullsized":"2024-04-21T08:47:00.874153+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:16.645869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622080+0000","last_change":"2024-04-21T08:44:34.499420+0000","last_active":"2024-04-21T08:47:00.622080+0000","last_peered":"2024-04-21T08:47:00.622080+0000","last_clean":"2024-04-21T08:47:00.622080+0000","last_became_active":"2024-04-21T08:44:34.499079+0000","last_became_peered":"2024-04-21T08:44:34.499079+0000","last_unstale":"2024-04-21T08:47:00.622080+0000","last_undegraded":"2024-04-21T08:47:00.622080+0000","last_fullsized":"2024-04-21T08:47:00.622080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:45.351422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747540+0000","last_change":"2024-04-21T08:44:34.493420+0000","last_active":"2024-04-21T08:47:00.747540+0000","last_peered":"2024-04-21T08:47:00.747540+0000","last_clean":"2024-04-21T08:47:00.747540+0000","last_became_active":"2024-04-21T08:44:34.493098+0000","last_became_peered":"2024-04-21T08:44:34.493098+0000","last_unstale":"2024-04-21T08:47:00.747540+0000","last_undegraded":"2024-04-21T08:47:00.747540+0000","last_fullsized":"2024-04-21T08:47:00.747540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:22:34.316822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874666+0000","last_change":"2024-04-21T08:44:34.494760+0000","last_active":"2024-04-21T08:47:00.874666+0000","last_peered":"2024-04-21T08:47:00.874666+0000","last_clean":"2024-04-21T08:47:00.874666+0000","last_became_active":"2024-04-21T08:44:34.494265+0000","last_became_peered":"2024-04-21T08:44:34.494265+0000","last_unstale":"2024-04-21T08:47:00.874666+0000","last_undegraded":"2024-04-21T08:47:00.874666+0000","last_fullsized":"2024-04-21T08:47:00.874666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:16.035030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748002+0000","last_change":"2024-04-21T08:44:34.498785+0000","last_active":"2024-04-21T08:47:00.748002+0000","last_peered":"2024-04-21T08:47:00.748002+0000","last_clean":"2024-04-21T08:47:00.748002+0000","last_became_active":"2024-04-21T08:44:34.495159+0000","last_became_peered":"2024-04-21T08:44:34.495159+0000","last_unstale":"2024-04-21T08:47:00.748002+0000","last_undegraded":"2024-04-21T08:47:00.748002+0000","last_fullsized":"2024-04-21T08:47:00.748002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:03:20.095016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801645,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1706,"num_read_kb":1077,"num_write":505,"num_write_kb":2388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2404935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":385,"ondisk_log_size":385,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383725,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1081344,"data_stored":531484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645362,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29200,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715952,"statfs":{"total":95995035648,"available":95965134848,"internally_reserved":0,"allocated":2113536,"data_stored":1516480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1503232,"data_stored":927553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070524,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1495040,"data_stored":923835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332162,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1699840,"data_stored":1129379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":525745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":596723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":528374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":599597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28912,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716240,"statfs":{"total":95995035648,"available":95965429760,"internally_reserved":0,"allocated":1818624,"data_stored":1246901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":530992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757324,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37448,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707704,"statfs":{"total":95995035648,"available":95956688896,"internally_reserved":0,"allocated":2039808,"data_stored":1452020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":74574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":332533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":401089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:47:13.845 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-21T08:47:13.845 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:47:13.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:13 smithi134 ceph-mon[31772]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-21T08:47:13.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:13 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1039426540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:13.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:13 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/537925038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:13.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:13 smithi134 ceph-mon[31772]: from='client.15876 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:13.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.979+0000 7f10d8411640 1 -- 172.21.15.82:0/974065205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d0102b40 msgr2=0x7f10d0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.979+0000 7f10d8411640 1 --2- 172.21.15.82:0/974065205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d0102b40 0x7f10d0106b80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009900 tx=0x7f10c002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:13.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.979+0000 7f10d8411640 1 -- 172.21.15.82:0/974065205 shutdown_connections 2024-04-21T08:47:13.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.979+0000 7f10d8411640 1 --2- 172.21.15.82:0/974065205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10d0107260 0x7f10d010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.979+0000 7f10d8411640 1 --2- 172.21.15.82:0/974065205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d0102b40 0x7f10d0106b80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.979+0000 7f10d8411640 1 --2- 172.21.15.82:0/974065205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10d0102190 0x7f10d0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.979+0000 7f10d8411640 1 -- 172.21.15.82:0/974065205 >> 172.21.15.82:0/974065205 conn(0x7f10d00793e0 msgr2=0x7f10d00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:13.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.979+0000 7f10d8411640 1 -- 172.21.15.82:0/974065205 shutdown_connections 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.979+0000 7f10d8411640 1 -- 172.21.15.82:0/974065205 wait complete. 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d8411640 1 Processor -- start 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d8411640 1 -- start start 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d8411640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d0102190 0x7f10d0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d8411640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10d0107260 0x7f10d0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d8411640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10d019d020 0x7f10d019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d8411640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f10d01121a0 con 0x7f10d0102190 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d8411640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f10d0112020 con 0x7f10d019d020 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d8411640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10d0112320 con 0x7f10d0107260 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d61ad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d0102190 0x7f10d0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d61ad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d0102190 0x7f10d0198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56352/0 (socket says 172.21.15.82:56352) 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d61ad640 1 -- 172.21.15.82:0/4012416348 learned_addr learned my addr 172.21.15.82:0/4012416348 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:13.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d69ae640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10d019d020 0x7f10d019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d59ac640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10d0107260 0x7f10d0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d61ad640 1 -- 172.21.15.82:0/4012416348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10d0107260 msgr2=0x7f10d0198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d61ad640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10d0107260 0x7f10d0198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d61ad640 1 -- 172.21.15.82:0/4012416348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10d019d020 msgr2=0x7f10d019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d61ad640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10d019d020 0x7f10d019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d61ad640 1 -- 172.21.15.82:0/4012416348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d019db10 con 0x7f10d0102190 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d59ac640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10d0107260 0x7f10d0198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d61ad640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d0102190 0x7f10d0198760 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f10c800d950 tx=0x7f10c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10c77fe640 1 -- 172.21.15.82:0/4012416348 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8017070 con 0x7f10d0102190 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10c77fe640 1 -- 172.21.15.82:0/4012416348 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10c8009c30 con 0x7f10d0102190 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.980+0000 7f10d8411640 1 -- 172.21.15.82:0/4012416348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d019de00 con 0x7f10d0102190 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.981+0000 7f10c77fe640 1 -- 172.21.15.82:0/4012416348 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c800f620 con 0x7f10d0102190 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.981+0000 7f10d8411640 1 -- 172.21.15.82:0/4012416348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d01a55f0 con 0x7f10d0102190 2024-04-21T08:47:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.981+0000 7f10d8411640 1 -- 172.21.15.82:0/4012416348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d0069230 con 0x7f10d0102190 2024-04-21T08:47:13.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.983+0000 7f10c77fe640 1 -- 172.21.15.82:0/4012416348 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f10c8010460 con 0x7f10d0102190 2024-04-21T08:47:13.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.983+0000 7f10c77fe640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f10bc0a3e40 0x7f10bc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.983+0000 7f10d59ac640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f10bc0a3e40 0x7f10bc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.983+0000 7f10c77fe640 1 -- 172.21.15.82:0/4012416348 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f10c80d0b60 con 0x7f10d0102190 2024-04-21T08:47:13.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.984+0000 7f10d59ac640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f10bc0a3e40 0x7f10bc0a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f10c0036040 tx=0x7f10c0000bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:13.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:13.988+0000 7f10c77fe640 1 -- 172.21.15.82:0/4012416348 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10c809cbf0 con 0x7f10d0102190 2024-04-21T08:47:14.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:13 smithi106 ceph-mon[31072]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-21T08:47:14.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:13 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1039426540' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:14.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:13 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/537925038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:14.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:13 smithi106 ceph-mon[31072]: from='client.15876 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:14.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.163+0000 7f10d8411640 1 -- 172.21.15.82:0/4012416348 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10d0102b40 con 0x7f10bc0a3e40 2024-04-21T08:47:14.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.172+0000 7f10c77fe640 1 -- 172.21.15.82:0/4012416348 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339453 (secure 0 0 0) 0x7f10d0102b40 con 0x7f10bc0a3e40 2024-04-21T08:47:14.178 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:14.179 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-21T08:47:14.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.176+0000 7f10d8411640 1 -- 172.21.15.82:0/4012416348 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f10bc0a3e40 msgr2=0x7f10bc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.176+0000 7f10d8411640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f10bc0a3e40 0x7f10bc0a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f10c0036040 tx=0x7f10c0000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.176+0000 7f10d8411640 1 -- 172.21.15.82:0/4012416348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d0102190 msgr2=0x7f10d0198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.176+0000 7f10d8411640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d0102190 0x7f10d0198760 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f10c800d950 tx=0x7f10c800de10 comp rx=0 tx=0).stop 2024-04-21T08:47:14.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.176+0000 7f10d8411640 1 -- 172.21.15.82:0/4012416348 shutdown_connections 2024-04-21T08:47:14.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.176+0000 7f10d8411640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10d019d020 0x7f10d019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.176+0000 7f10d8411640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10d0107260 0x7f10d0198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.176+0000 7f10d8411640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f10bc0a3e40 0x7f10bc0a6300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.176+0000 7f10d8411640 1 --2- 172.21.15.82:0/4012416348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d0102190 0x7f10d0198760 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.176+0000 7f10d8411640 1 -- 172.21.15.82:0/4012416348 >> 172.21.15.82:0/4012416348 conn(0x7f10d00793e0 msgr2=0x7f10d01000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.176+0000 7f10d8411640 1 -- 172.21.15.82:0/4012416348 shutdown_connections 2024-04-21T08:47:14.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.176+0000 7f10d8411640 1 -- 172.21.15.82:0/4012416348 wait complete. 2024-04-21T08:47:14.186 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-04-21T08:47:12.635896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1392247,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1943,"num_read_kb":1407,"num_write":708,"num_write_kb":5328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1035,"ondisk_log_size":1035,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358168,"kb_used_data":16952,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124583656,"statfs":{"total":1151940427776,"available":1151573663744,"internally_reserved":0,"allocated":17358848,"data_stored":10509083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200716,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":461,"num_read_kb":500,"num_write":153,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.987572"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748276+0000","last_change":"2024-04-21T08:44:34.494393+0000","last_active":"2024-04-21T08:47:00.748276+0000","last_peered":"2024-04-21T08:47:00.748276+0000","last_clean":"2024-04-21T08:47:00.748276+0000","last_became_active":"2024-04-21T08:44:34.493583+0000","last_became_peered":"2024-04-21T08:44:34.493583+0000","last_unstale":"2024-04-21T08:47:00.748276+0000","last_undegraded":"2024-04-21T08:47:00.748276+0000","last_fullsized":"2024-04-21T08:47:00.748276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:37.211676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747826+0000","last_change":"2024-04-21T08:44:34.496291+0000","last_active":"2024-04-21T08:47:00.747826+0000","last_peered":"2024-04-21T08:47:00.747826+0000","last_clean":"2024-04-21T08:47:00.747826+0000","last_became_active":"2024-04-21T08:44:34.495019+0000","last_became_peered":"2024-04-21T08:44:34.495019+0000","last_unstale":"2024-04-21T08:47:00.747826+0000","last_undegraded":"2024-04-21T08:47:00.747826+0000","last_fullsized":"2024-04-21T08:47:00.747826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:45:05.309847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747464+0000","last_change":"2024-04-21T08:44:34.497948+0000","last_active":"2024-04-21T08:47:00.747464+0000","last_peered":"2024-04-21T08:47:00.747464+0000","last_clean":"2024-04-21T08:47:00.747464+0000","last_became_active":"2024-04-21T08:44:34.497657+0000","last_became_peered":"2024-04-21T08:44:34.497657+0000","last_unstale":"2024-04-21T08:47:00.747464+0000","last_undegraded":"2024-04-21T08:47:00.747464+0000","last_fullsized":"2024-04-21T08:47:00.747464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:11.463715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748377+0000","last_change":"2024-04-21T08:44:34.492682+0000","last_active":"2024-04-21T08:47:00.748377+0000","last_peered":"2024-04-21T08:47:00.748377+0000","last_clean":"2024-04-21T08:47:00.748377+0000","last_became_active":"2024-04-21T08:44:34.492230+0000","last_became_peered":"2024-04-21T08:44:34.492230+0000","last_unstale":"2024-04-21T08:47:00.748377+0000","last_undegraded":"2024-04-21T08:47:00.748377+0000","last_fullsized":"2024-04-21T08:47:00.748377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:40.365608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721373+0000","last_change":"2024-04-21T08:44:34.492160+0000","last_active":"2024-04-21T08:47:00.721373+0000","last_peered":"2024-04-21T08:47:00.721373+0000","last_clean":"2024-04-21T08:47:00.721373+0000","last_became_active":"2024-04-21T08:44:34.491701+0000","last_became_peered":"2024-04-21T08:44:34.491701+0000","last_unstale":"2024-04-21T08:47:00.721373+0000","last_undegraded":"2024-04-21T08:47:00.721373+0000","last_fullsized":"2024-04-21T08:47:00.721373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:49:50.542661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626584+0000","last_change":"2024-04-21T08:44:34.497341+0000","last_active":"2024-04-21T08:47:00.626584+0000","last_peered":"2024-04-21T08:47:00.626584+0000","last_clean":"2024-04-21T08:47:00.626584+0000","last_became_active":"2024-04-21T08:44:34.495794+0000","last_became_peered":"2024-04-21T08:44:34.495794+0000","last_unstale":"2024-04-21T08:47:00.626584+0000","last_undegraded":"2024-04-21T08:47:00.626584+0000","last_fullsized":"2024-04-21T08:47:00.626584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:13.854753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748527+0000","last_change":"2024-04-21T08:44:34.501578+0000","last_active":"2024-04-21T08:47:00.748527+0000","last_peered":"2024-04-21T08:47:00.748527+0000","last_clean":"2024-04-21T08:47:00.748527+0000","last_became_active":"2024-04-21T08:44:34.500848+0000","last_became_peered":"2024-04-21T08:44:34.500848+0000","last_unstale":"2024-04-21T08:47:00.748527+0000","last_undegraded":"2024-04-21T08:47:00.748527+0000","last_fullsized":"2024-04-21T08:47:00.748527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:53:27.529595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.621951+0000","last_change":"2024-04-21T08:44:34.502975+0000","last_active":"2024-04-21T08:47:00.621951+0000","last_peered":"2024-04-21T08:47:00.621951+0000","last_clean":"2024-04-21T08:47:00.621951+0000","last_became_active":"2024-04-21T08:44:34.502676+0000","last_became_peered":"2024-04-21T08:44:34.502676+0000","last_unstale":"2024-04-21T08:47:00.621951+0000","last_undegraded":"2024-04-21T08:47:00.621951+0000","last_fullsized":"2024-04-21T08:47:00.621951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:32:13.881603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748293+0000","last_change":"2024-04-21T08:44:34.495703+0000","last_active":"2024-04-21T08:47:00.748293+0000","last_peered":"2024-04-21T08:47:00.748293+0000","last_clean":"2024-04-21T08:47:00.748293+0000","last_became_active":"2024-04-21T08:44:34.495380+0000","last_became_peered":"2024-04-21T08:44:34.495380+0000","last_unstale":"2024-04-21T08:47:00.748293+0000","last_undegraded":"2024-04-21T08:47:00.748293+0000","last_fullsized":"2024-04-21T08:47:00.748293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:45:28.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626928+0000","last_change":"2024-04-21T08:44:34.499361+0000","last_active":"2024-04-21T08:47:00.626928+0000","last_peered":"2024-04-21T08:47:00.626928+0000","last_clean":"2024-04-21T08:47:00.626928+0000","last_became_active":"2024-04-21T08:44:34.499044+0000","last_became_peered":"2024-04-21T08:44:34.499044+0000","last_unstale":"2024-04-21T08:47:00.626928+0000","last_undegraded":"2024-04-21T08:47:00.626928+0000","last_fullsized":"2024-04-21T08:47:00.626928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:55:12.118985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748894+0000","last_change":"2024-04-21T08:44:34.499631+0000","last_active":"2024-04-21T08:47:00.748894+0000","last_peered":"2024-04-21T08:47:00.748894+0000","last_clean":"2024-04-21T08:47:00.748894+0000","last_became_active":"2024-04-21T08:44:34.499316+0000","last_became_peered":"2024-04-21T08:44:34.499316+0000","last_unstale":"2024-04-21T08:47:00.748894+0000","last_undegraded":"2024-04-21T08:47:00.748894+0000","last_fullsized":"2024-04-21T08:47:00.748894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:26.123017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748326+0000","last_change":"2024-04-21T08:44:34.504857+0000","last_active":"2024-04-21T08:47:00.748326+0000","last_peered":"2024-04-21T08:47:00.748326+0000","last_clean":"2024-04-21T08:47:00.748326+0000","last_became_active":"2024-04-21T08:44:34.504564+0000","last_became_peered":"2024-04-21T08:44:34.504564+0000","last_unstale":"2024-04-21T08:47:00.748326+0000","last_undegraded":"2024-04-21T08:47:00.748326+0000","last_fullsized":"2024-04-21T08:47:00.748326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:32:05.216162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874684+0000","last_change":"2024-04-21T08:44:34.484074+0000","last_active":"2024-04-21T08:47:00.874684+0000","last_peered":"2024-04-21T08:47:00.874684+0000","last_clean":"2024-04-21T08:47:00.874684+0000","last_became_active":"2024-04-21T08:44:34.483309+0000","last_became_peered":"2024-04-21T08:44:34.483309+0000","last_unstale":"2024-04-21T08:47:00.874684+0000","last_undegraded":"2024-04-21T08:47:00.874684+0000","last_fullsized":"2024-04-21T08:47:00.874684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:12:39.686048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747340+0000","last_change":"2024-04-21T08:44:34.499159+0000","last_active":"2024-04-21T08:47:00.747340+0000","last_peered":"2024-04-21T08:47:00.747340+0000","last_clean":"2024-04-21T08:47:00.747340+0000","last_became_active":"2024-04-21T08:44:34.496609+0000","last_became_peered":"2024-04-21T08:44:34.496609+0000","last_unstale":"2024-04-21T08:47:00.747340+0000","last_undegraded":"2024-04-21T08:47:00.747340+0000","last_fullsized":"2024-04-21T08:47:00.747340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:11.716321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747233+0000","last_change":"2024-04-21T08:45:27.406543+0000","last_active":"2024-04-21T08:47:00.747233+0000","last_peered":"2024-04-21T08:47:00.747233+0000","last_clean":"2024-04-21T08:47:00.747233+0000","last_became_active":"2024-04-21T08:45:27.406190+0000","last_became_peered":"2024-04-21T08:45:27.406190+0000","last_unstale":"2024-04-21T08:47:00.747233+0000","last_undegraded":"2024-04-21T08:47:00.747233+0000","last_fullsized":"2024-04-21T08:47:00.747233+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:56:17.760163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747327+0000","last_change":"2024-04-21T08:44:34.494018+0000","last_active":"2024-04-21T08:47:00.747327+0000","last_peered":"2024-04-21T08:47:00.747327+0000","last_clean":"2024-04-21T08:47:00.747327+0000","last_became_active":"2024-04-21T08:44:34.493692+0000","last_became_peered":"2024-04-21T08:44:34.493692+0000","last_unstale":"2024-04-21T08:47:00.747327+0000","last_undegraded":"2024-04-21T08:47:00.747327+0000","last_fullsized":"2024-04-21T08:47:00.747327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:25:16.944160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748187+0000","last_change":"2024-04-21T08:44:34.493231+0000","last_active":"2024-04-21T08:47:00.748187+0000","last_peered":"2024-04-21T08:47:00.748187+0000","last_clean":"2024-04-21T08:47:00.748187+0000","last_became_active":"2024-04-21T08:44:34.492826+0000","last_became_peered":"2024-04-21T08:44:34.492826+0000","last_unstale":"2024-04-21T08:47:00.748187+0000","last_undegraded":"2024-04-21T08:47:00.748187+0000","last_fullsized":"2024-04-21T08:47:00.748187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:36.815012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719352+0000","last_change":"2024-04-21T08:44:34.495142+0000","last_active":"2024-04-21T08:47:00.719352+0000","last_peered":"2024-04-21T08:47:00.719352+0000","last_clean":"2024-04-21T08:47:00.719352+0000","last_became_active":"2024-04-21T08:44:34.494803+0000","last_became_peered":"2024-04-21T08:44:34.494803+0000","last_unstale":"2024-04-21T08:47:00.719352+0000","last_undegraded":"2024-04-21T08:47:00.719352+0000","last_fullsized":"2024-04-21T08:47:00.719352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:22.974053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627186+0000","last_change":"2024-04-21T08:44:34.491252+0000","last_active":"2024-04-21T08:47:00.627186+0000","last_peered":"2024-04-21T08:47:00.627186+0000","last_clean":"2024-04-21T08:47:00.627186+0000","last_became_active":"2024-04-21T08:44:34.490814+0000","last_became_peered":"2024-04-21T08:44:34.490814+0000","last_unstale":"2024-04-21T08:47:00.627186+0000","last_undegraded":"2024-04-21T08:47:00.627186+0000","last_fullsized":"2024-04-21T08:47:00.627186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:54:52.238191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626918+0000","last_change":"2024-04-21T08:44:38.518647+0000","last_active":"2024-04-21T08:47:00.626918+0000","last_peered":"2024-04-21T08:47:00.626918+0000","last_clean":"2024-04-21T08:47:00.626918+0000","last_became_active":"2024-04-21T08:44:38.518239+0000","last_became_peered":"2024-04-21T08:44:38.518239+0000","last_unstale":"2024-04-21T08:47:00.626918+0000","last_undegraded":"2024-04-21T08:47:00.626918+0000","last_fullsized":"2024-04-21T08:47:00.626918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:22:44.781542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":178,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.174380+0000","last_change":"2024-04-21T08:44:33.484484+0000","last_active":"2024-04-21T08:47:10.174380+0000","last_peered":"2024-04-21T08:47:10.174380+0000","last_clean":"2024-04-21T08:47:10.174380+0000","last_became_active":"2024-04-21T08:44:33.484160+0000","last_became_peered":"2024-04-21T08:44:33.484160+0000","last_unstale":"2024-04-21T08:47:10.174380+0000","last_undegraded":"2024-04-21T08:47:10.174380+0000","last_fullsized":"2024-04-21T08:47:10.174380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:45.962598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718920+0000","last_change":"2024-04-21T08:46:27.484947+0000","last_active":"2024-04-21T08:47:00.718920+0000","last_peered":"2024-04-21T08:47:00.718920+0000","last_clean":"2024-04-21T08:47:00.718920+0000","last_became_active":"2024-04-21T08:46:27.484530+0000","last_became_peered":"2024-04-21T08:46:27.484530+0000","last_unstale":"2024-04-21T08:47:00.718920+0000","last_undegraded":"2024-04-21T08:47:00.718920+0000","last_fullsized":"2024-04-21T08:47:00.718920+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:59.248176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747775+0000","last_change":"2024-04-21T08:45:27.518006+0000","last_active":"2024-04-21T08:47:00.747775+0000","last_peered":"2024-04-21T08:47:00.747775+0000","last_clean":"2024-04-21T08:47:00.747775+0000","last_became_active":"2024-04-21T08:45:27.517409+0000","last_became_peered":"2024-04-21T08:45:27.517409+0000","last_unstale":"2024-04-21T08:47:00.747775+0000","last_undegraded":"2024-04-21T08:47:00.747775+0000","last_fullsized":"2024-04-21T08:47:00.747775+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:45:01.253587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623343+0000","last_change":"2024-04-21T08:44:33.490508+0000","last_active":"2024-04-21T08:47:00.623343+0000","last_peered":"2024-04-21T08:47:00.623343+0000","last_clean":"2024-04-21T08:47:00.623343+0000","last_became_active":"2024-04-21T08:44:33.490084+0000","last_became_peered":"2024-04-21T08:44:33.490084+0000","last_unstale":"2024-04-21T08:47:00.623343+0000","last_undegraded":"2024-04-21T08:47:00.623343+0000","last_fullsized":"2024-04-21T08:47:00.623343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:21.553435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747770+0000","last_change":"2024-04-21T08:44:34.491843+0000","last_active":"2024-04-21T08:47:00.747770+0000","last_peered":"2024-04-21T08:47:00.747770+0000","last_clean":"2024-04-21T08:47:00.747770+0000","last_became_active":"2024-04-21T08:44:34.490578+0000","last_became_peered":"2024-04-21T08:44:34.490578+0000","last_unstale":"2024-04-21T08:47:00.747770+0000","last_undegraded":"2024-04-21T08:47:00.747770+0000","last_fullsized":"2024-04-21T08:47:00.747770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:05:12.272330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748084+0000","last_change":"2024-04-21T08:44:38.520258+0000","last_active":"2024-04-21T08:47:00.748084+0000","last_peered":"2024-04-21T08:47:00.748084+0000","last_clean":"2024-04-21T08:47:00.748084+0000","last_became_active":"2024-04-21T08:44:38.519824+0000","last_became_peered":"2024-04-21T08:44:38.519824+0000","last_unstale":"2024-04-21T08:47:00.748084+0000","last_undegraded":"2024-04-21T08:47:00.748084+0000","last_fullsized":"2024-04-21T08:47:00.748084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:39.716745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721810+0000","last_change":"2024-04-21T08:44:33.484934+0000","last_active":"2024-04-21T08:47:00.721810+0000","last_peered":"2024-04-21T08:47:00.721810+0000","last_clean":"2024-04-21T08:47:00.721810+0000","last_became_active":"2024-04-21T08:44:33.484591+0000","last_became_peered":"2024-04-21T08:44:33.484591+0000","last_unstale":"2024-04-21T08:47:00.721810+0000","last_undegraded":"2024-04-21T08:47:00.721810+0000","last_fullsized":"2024-04-21T08:47:00.721810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:50:48.826611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747675+0000","last_change":"2024-04-21T08:44:34.501109+0000","last_active":"2024-04-21T08:47:00.747675+0000","last_peered":"2024-04-21T08:47:00.747675+0000","last_clean":"2024-04-21T08:47:00.747675+0000","last_became_active":"2024-04-21T08:44:34.500750+0000","last_became_peered":"2024-04-21T08:44:34.500750+0000","last_unstale":"2024-04-21T08:47:00.747675+0000","last_undegraded":"2024-04-21T08:47:00.747675+0000","last_fullsized":"2024-04-21T08:47:00.747675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:29:28.792930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747855+0000","last_change":"2024-04-21T08:44:38.519633+0000","last_active":"2024-04-21T08:47:00.747855+0000","last_peered":"2024-04-21T08:47:00.747855+0000","last_clean":"2024-04-21T08:47:00.747855+0000","last_became_active":"2024-04-21T08:44:38.519215+0000","last_became_peered":"2024-04-21T08:44:38.519215+0000","last_unstale":"2024-04-21T08:47:00.747855+0000","last_undegraded":"2024-04-21T08:47:00.747855+0000","last_fullsized":"2024-04-21T08:47:00.747855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:43:59.695844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721745+0000","last_change":"2024-04-21T08:44:33.483267+0000","last_active":"2024-04-21T08:47:00.721745+0000","last_peered":"2024-04-21T08:47:00.721745+0000","last_clean":"2024-04-21T08:47:00.721745+0000","last_became_active":"2024-04-21T08:44:33.482932+0000","last_became_peered":"2024-04-21T08:44:33.482932+0000","last_unstale":"2024-04-21T08:47:00.721745+0000","last_undegraded":"2024-04-21T08:47:00.721745+0000","last_fullsized":"2024-04-21T08:47:00.721745+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:48:53.311550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729284+0000","last_change":"2024-04-21T08:44:34.497389+0000","last_active":"2024-04-21T08:47:00.729284+0000","last_peered":"2024-04-21T08:47:00.729284+0000","last_clean":"2024-04-21T08:47:00.729284+0000","last_became_active":"2024-04-21T08:44:34.497030+0000","last_became_peered":"2024-04-21T08:44:34.497030+0000","last_unstale":"2024-04-21T08:47:00.729284+0000","last_undegraded":"2024-04-21T08:47:00.729284+0000","last_fullsized":"2024-04-21T08:47:00.729284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:20.371888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626966+0000","last_change":"2024-04-21T08:44:38.518095+0000","last_active":"2024-04-21T08:47:00.626966+0000","last_peered":"2024-04-21T08:47:00.626966+0000","last_clean":"2024-04-21T08:47:00.626966+0000","last_became_active":"2024-04-21T08:44:38.517010+0000","last_became_peered":"2024-04-21T08:44:38.517010+0000","last_unstale":"2024-04-21T08:47:00.626966+0000","last_undegraded":"2024-04-21T08:47:00.626966+0000","last_fullsized":"2024-04-21T08:47:00.626966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:33:02.379512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":180,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.182251+0000","last_change":"2024-04-21T08:44:33.487999+0000","last_active":"2024-04-21T08:47:10.182251+0000","last_peered":"2024-04-21T08:47:10.182251+0000","last_clean":"2024-04-21T08:47:10.182251+0000","last_became_active":"2024-04-21T08:44:33.487666+0000","last_became_peered":"2024-04-21T08:44:33.487666+0000","last_unstale":"2024-04-21T08:47:10.182251+0000","last_undegraded":"2024-04-21T08:47:10.182251+0000","last_fullsized":"2024-04-21T08:47:10.182251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:52:01.054309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2103,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721554+0000","last_change":"2024-04-21T08:44:34.494213+0000","last_active":"2024-04-21T08:47:00.721554+0000","last_peered":"2024-04-21T08:47:00.721554+0000","last_clean":"2024-04-21T08:47:00.721554+0000","last_became_active":"2024-04-21T08:44:34.493853+0000","last_became_peered":"2024-04-21T08:44:34.493853+0000","last_unstale":"2024-04-21T08:47:00.721554+0000","last_undegraded":"2024-04-21T08:47:00.721554+0000","last_fullsized":"2024-04-21T08:47:00.721554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:42.499131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627058+0000","last_change":"2024-04-21T08:45:27.410579+0000","last_active":"2024-04-21T08:47:00.627058+0000","last_peered":"2024-04-21T08:47:00.627058+0000","last_clean":"2024-04-21T08:47:00.627058+0000","last_became_active":"2024-04-21T08:45:27.409929+0000","last_became_peered":"2024-04-21T08:45:27.409929+0000","last_unstale":"2024-04-21T08:47:00.627058+0000","last_undegraded":"2024-04-21T08:47:00.627058+0000","last_fullsized":"2024-04-21T08:47:00.627058+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:34:35.897482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626508+0000","last_change":"2024-04-21T08:44:33.488751+0000","last_active":"2024-04-21T08:47:00.626508+0000","last_peered":"2024-04-21T08:47:00.626508+0000","last_clean":"2024-04-21T08:47:00.626508+0000","last_became_active":"2024-04-21T08:44:33.488425+0000","last_became_peered":"2024-04-21T08:44:33.488425+0000","last_unstale":"2024-04-21T08:47:00.626508+0000","last_undegraded":"2024-04-21T08:47:00.626508+0000","last_fullsized":"2024-04-21T08:47:00.626508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:28:47.868814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729150+0000","last_change":"2024-04-21T08:44:34.492770+0000","last_active":"2024-04-21T08:47:00.729150+0000","last_peered":"2024-04-21T08:47:00.729150+0000","last_clean":"2024-04-21T08:47:00.729150+0000","last_became_active":"2024-04-21T08:44:34.491340+0000","last_became_peered":"2024-04-21T08:44:34.491340+0000","last_unstale":"2024-04-21T08:47:00.729150+0000","last_undegraded":"2024-04-21T08:47:00.729150+0000","last_fullsized":"2024-04-21T08:47:00.729150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:47:54.280990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748097+0000","last_change":"2024-04-21T08:44:38.518888+0000","last_active":"2024-04-21T08:47:00.748097+0000","last_peered":"2024-04-21T08:47:00.748097+0000","last_clean":"2024-04-21T08:47:00.748097+0000","last_became_active":"2024-04-21T08:44:38.518421+0000","last_became_peered":"2024-04-21T08:44:38.518421+0000","last_unstale":"2024-04-21T08:47:00.748097+0000","last_undegraded":"2024-04-21T08:47:00.748097+0000","last_fullsized":"2024-04-21T08:47:00.748097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:21.119364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874482+0000","last_change":"2024-04-21T08:44:33.484383+0000","last_active":"2024-04-21T08:47:00.874482+0000","last_peered":"2024-04-21T08:47:00.874482+0000","last_clean":"2024-04-21T08:47:00.874482+0000","last_became_active":"2024-04-21T08:44:33.484083+0000","last_became_peered":"2024-04-21T08:44:33.484083+0000","last_unstale":"2024-04-21T08:47:00.874482+0000","last_undegraded":"2024-04-21T08:47:00.874482+0000","last_fullsized":"2024-04-21T08:47:00.874482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:13:14.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622126+0000","last_change":"2024-04-21T08:44:34.495636+0000","last_active":"2024-04-21T08:47:00.622126+0000","last_peered":"2024-04-21T08:47:00.622126+0000","last_clean":"2024-04-21T08:47:00.622126+0000","last_became_active":"2024-04-21T08:44:34.494967+0000","last_became_peered":"2024-04-21T08:44:34.494967+0000","last_unstale":"2024-04-21T08:47:00.622126+0000","last_undegraded":"2024-04-21T08:47:00.622126+0000","last_fullsized":"2024-04-21T08:47:00.622126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:47:02.379208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747724+0000","last_change":"2024-04-21T08:44:38.515145+0000","last_active":"2024-04-21T08:47:00.747724+0000","last_peered":"2024-04-21T08:47:00.747724+0000","last_clean":"2024-04-21T08:47:00.747724+0000","last_became_active":"2024-04-21T08:44:38.514755+0000","last_became_peered":"2024-04-21T08:44:38.514755+0000","last_unstale":"2024-04-21T08:47:00.747724+0000","last_undegraded":"2024-04-21T08:47:00.747724+0000","last_fullsized":"2024-04-21T08:47:00.747724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:38:15.826293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728869+0000","last_change":"2024-04-21T08:44:33.484714+0000","last_active":"2024-04-21T08:47:00.728869+0000","last_peered":"2024-04-21T08:47:00.728869+0000","last_clean":"2024-04-21T08:47:00.728869+0000","last_became_active":"2024-04-21T08:44:33.484385+0000","last_became_peered":"2024-04-21T08:44:33.484385+0000","last_unstale":"2024-04-21T08:47:00.728869+0000","last_undegraded":"2024-04-21T08:47:00.728869+0000","last_fullsized":"2024-04-21T08:47:00.728869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:06:03.406860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718544+0000","last_change":"2024-04-21T08:44:34.490989+0000","last_active":"2024-04-21T08:47:00.718544+0000","last_peered":"2024-04-21T08:47:00.718544+0000","last_clean":"2024-04-21T08:47:00.718544+0000","last_became_active":"2024-04-21T08:44:34.490658+0000","last_became_peered":"2024-04-21T08:44:34.490658+0000","last_unstale":"2024-04-21T08:47:00.718544+0000","last_undegraded":"2024-04-21T08:47:00.718544+0000","last_fullsized":"2024-04-21T08:47:00.718544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:42:15.537913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622288+0000","last_change":"2024-04-21T08:44:38.519723+0000","last_active":"2024-04-21T08:47:00.622288+0000","last_peered":"2024-04-21T08:47:00.622288+0000","last_clean":"2024-04-21T08:47:00.622288+0000","last_became_active":"2024-04-21T08:44:38.519336+0000","last_became_peered":"2024-04-21T08:44:38.519336+0000","last_unstale":"2024-04-21T08:47:00.622288+0000","last_undegraded":"2024-04-21T08:47:00.622288+0000","last_fullsized":"2024-04-21T08:47:00.622288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:18:00.332153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'6","reported_seq":301,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:11.183116+0000","last_change":"2024-04-21T08:44:33.484404+0000","last_active":"2024-04-21T08:47:11.183116+0000","last_peered":"2024-04-21T08:47:11.183116+0000","last_clean":"2024-04-21T08:47:11.183116+0000","last_became_active":"2024-04-21T08:44:33.484088+0000","last_became_peered":"2024-04-21T08:44:33.484088+0000","last_unstale":"2024-04-21T08:47:11.183116+0000","last_undegraded":"2024-04-21T08:47:11.183116+0000","last_fullsized":"2024-04-21T08:47:11.183116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:41.948622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":262,"num_read_kb":262,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748214+0000","last_change":"2024-04-21T08:44:34.502598+0000","last_active":"2024-04-21T08:47:00.748214+0000","last_peered":"2024-04-21T08:47:00.748214+0000","last_clean":"2024-04-21T08:47:00.748214+0000","last_became_active":"2024-04-21T08:44:34.502057+0000","last_became_peered":"2024-04-21T08:44:34.502057+0000","last_unstale":"2024-04-21T08:47:00.748214+0000","last_undegraded":"2024-04-21T08:47:00.748214+0000","last_fullsized":"2024-04-21T08:47:00.748214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:24.052873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718327+0000","last_change":"2024-04-21T08:44:38.519699+0000","last_active":"2024-04-21T08:47:00.718327+0000","last_peered":"2024-04-21T08:47:00.718327+0000","last_clean":"2024-04-21T08:47:00.718327+0000","last_became_active":"2024-04-21T08:44:38.517655+0000","last_became_peered":"2024-04-21T08:44:38.517655+0000","last_unstale":"2024-04-21T08:47:00.718327+0000","last_undegraded":"2024-04-21T08:47:00.718327+0000","last_fullsized":"2024-04-21T08:47:00.718327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:12.784479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748806+0000","last_change":"2024-04-21T08:46:27.494747+0000","last_active":"2024-04-21T08:47:00.748806+0000","last_peered":"2024-04-21T08:47:00.748806+0000","last_clean":"2024-04-21T08:47:00.748806+0000","last_became_active":"2024-04-21T08:46:27.490469+0000","last_became_peered":"2024-04-21T08:46:27.490469+0000","last_unstale":"2024-04-21T08:47:00.748806+0000","last_undegraded":"2024-04-21T08:47:00.748806+0000","last_fullsized":"2024-04-21T08:47:00.748806+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:16:35.257267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747850+0000","last_change":"2024-04-21T08:45:27.518395+0000","last_active":"2024-04-21T08:47:00.747850+0000","last_peered":"2024-04-21T08:47:00.747850+0000","last_clean":"2024-04-21T08:47:00.747850+0000","last_became_active":"2024-04-21T08:45:27.517969+0000","last_became_peered":"2024-04-21T08:45:27.517969+0000","last_unstale":"2024-04-21T08:47:00.747850+0000","last_undegraded":"2024-04-21T08:47:00.747850+0000","last_fullsized":"2024-04-21T08:47:00.747850+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:54:55.306556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748406+0000","last_change":"2024-04-21T08:44:38.517956+0000","last_active":"2024-04-21T08:47:00.748406+0000","last_peered":"2024-04-21T08:47:00.748406+0000","last_clean":"2024-04-21T08:47:00.748406+0000","last_became_active":"2024-04-21T08:44:38.517422+0000","last_became_peered":"2024-04-21T08:44:38.517422+0000","last_unstale":"2024-04-21T08:47:00.748406+0000","last_undegraded":"2024-04-21T08:47:00.748406+0000","last_fullsized":"2024-04-21T08:47:00.748406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:04.650784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729223+0000","last_change":"2024-04-21T08:44:33.489151+0000","last_active":"2024-04-21T08:47:00.729223+0000","last_peered":"2024-04-21T08:47:00.729223+0000","last_clean":"2024-04-21T08:47:00.729223+0000","last_became_active":"2024-04-21T08:44:33.488781+0000","last_became_peered":"2024-04-21T08:44:33.488781+0000","last_unstale":"2024-04-21T08:47:00.729223+0000","last_undegraded":"2024-04-21T08:47:00.729223+0000","last_fullsized":"2024-04-21T08:47:00.729223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:40.675100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747905+0000","last_change":"2024-04-21T08:44:34.496528+0000","last_active":"2024-04-21T08:47:00.747905+0000","last_peered":"2024-04-21T08:47:00.747905+0000","last_clean":"2024-04-21T08:47:00.747905+0000","last_became_active":"2024-04-21T08:44:34.496223+0000","last_became_peered":"2024-04-21T08:44:34.496223+0000","last_unstale":"2024-04-21T08:47:00.747905+0000","last_undegraded":"2024-04-21T08:47:00.747905+0000","last_fullsized":"2024-04-21T08:47:00.747905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:44:56.647235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718461+0000","last_change":"2024-04-21T08:44:38.517537+0000","last_active":"2024-04-21T08:47:00.718461+0000","last_peered":"2024-04-21T08:47:00.718461+0000","last_clean":"2024-04-21T08:47:00.718461+0000","last_became_active":"2024-04-21T08:44:38.517134+0000","last_became_peered":"2024-04-21T08:44:38.517134+0000","last_unstale":"2024-04-21T08:47:00.718461+0000","last_undegraded":"2024-04-21T08:47:00.718461+0000","last_fullsized":"2024-04-21T08:47:00.718461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:38.634418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'41","reported_seq":351,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:11.193441+0000","last_change":"2024-04-21T08:44:33.485582+0000","last_active":"2024-04-21T08:47:11.193441+0000","last_peered":"2024-04-21T08:47:11.193441+0000","last_clean":"2024-04-21T08:47:11.193441+0000","last_became_active":"2024-04-21T08:44:33.485249+0000","last_became_peered":"2024-04-21T08:44:33.485249+0000","last_unstale":"2024-04-21T08:47:11.193441+0000","last_undegraded":"2024-04-21T08:47:11.193441+0000","last_fullsized":"2024-04-21T08:47:11.193441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:16:10.571694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69948,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":277,"num_read_kb":208,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721670+0000","last_change":"2024-04-21T08:44:34.491361+0000","last_active":"2024-04-21T08:47:00.721670+0000","last_peered":"2024-04-21T08:47:00.721670+0000","last_clean":"2024-04-21T08:47:00.721670+0000","last_became_active":"2024-04-21T08:44:34.491050+0000","last_became_peered":"2024-04-21T08:44:34.491050+0000","last_unstale":"2024-04-21T08:47:00.721670+0000","last_undegraded":"2024-04-21T08:47:00.721670+0000","last_fullsized":"2024-04-21T08:47:00.721670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:39.082448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729534+0000","last_change":"2024-04-21T08:44:38.528745+0000","last_active":"2024-04-21T08:47:00.729534+0000","last_peered":"2024-04-21T08:47:00.729534+0000","last_clean":"2024-04-21T08:47:00.729534+0000","last_became_active":"2024-04-21T08:44:38.528275+0000","last_became_peered":"2024-04-21T08:44:38.528275+0000","last_unstale":"2024-04-21T08:47:00.729534+0000","last_undegraded":"2024-04-21T08:47:00.729534+0000","last_fullsized":"2024-04-21T08:47:00.729534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:06:11.338420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626897+0000","last_change":"2024-04-21T08:44:33.482059+0000","last_active":"2024-04-21T08:47:00.626897+0000","last_peered":"2024-04-21T08:47:00.626897+0000","last_clean":"2024-04-21T08:47:00.626897+0000","last_became_active":"2024-04-21T08:44:33.481663+0000","last_became_peered":"2024-04-21T08:44:33.481663+0000","last_unstale":"2024-04-21T08:47:00.626897+0000","last_undegraded":"2024-04-21T08:47:00.626897+0000","last_fullsized":"2024-04-21T08:47:00.626897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:40:01.240452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874587+0000","last_change":"2024-04-21T08:44:34.491398+0000","last_active":"2024-04-21T08:47:00.874587+0000","last_peered":"2024-04-21T08:47:00.874587+0000","last_clean":"2024-04-21T08:47:00.874587+0000","last_became_active":"2024-04-21T08:44:34.491073+0000","last_became_peered":"2024-04-21T08:44:34.491073+0000","last_unstale":"2024-04-21T08:47:00.874587+0000","last_undegraded":"2024-04-21T08:47:00.874587+0000","last_fullsized":"2024-04-21T08:47:00.874587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:13:22.566063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747937+0000","last_change":"2024-04-21T08:44:38.515653+0000","last_active":"2024-04-21T08:47:00.747937+0000","last_peered":"2024-04-21T08:47:00.747937+0000","last_clean":"2024-04-21T08:47:00.747937+0000","last_became_active":"2024-04-21T08:44:38.514796+0000","last_became_peered":"2024-04-21T08:44:38.514796+0000","last_unstale":"2024-04-21T08:47:00.747937+0000","last_undegraded":"2024-04-21T08:47:00.747937+0000","last_fullsized":"2024-04-21T08:47:00.747937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:00:22.662136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.729003+0000","last_change":"2024-04-21T08:44:33.487073+0000","last_active":"2024-04-21T08:47:00.729003+0000","last_peered":"2024-04-21T08:47:00.729003+0000","last_clean":"2024-04-21T08:47:00.729003+0000","last_became_active":"2024-04-21T08:44:33.486740+0000","last_became_peered":"2024-04-21T08:44:33.486740+0000","last_unstale":"2024-04-21T08:47:00.729003+0000","last_undegraded":"2024-04-21T08:47:00.729003+0000","last_fullsized":"2024-04-21T08:47:00.729003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:33:32.673074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748512+0000","last_change":"2024-04-21T08:44:34.496757+0000","last_active":"2024-04-21T08:47:00.748512+0000","last_peered":"2024-04-21T08:47:00.748512+0000","last_clean":"2024-04-21T08:47:00.748512+0000","last_became_active":"2024-04-21T08:44:34.496422+0000","last_became_peered":"2024-04-21T08:44:34.496422+0000","last_unstale":"2024-04-21T08:47:00.748512+0000","last_undegraded":"2024-04-21T08:47:00.748512+0000","last_fullsized":"2024-04-21T08:47:00.748512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:39:08.686810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718978+0000","last_change":"2024-04-21T08:44:38.520935+0000","last_active":"2024-04-21T08:47:00.718978+0000","last_peered":"2024-04-21T08:47:00.718978+0000","last_clean":"2024-04-21T08:47:00.718978+0000","last_became_active":"2024-04-21T08:44:38.520481+0000","last_became_peered":"2024-04-21T08:44:38.520481+0000","last_unstale":"2024-04-21T08:47:00.718978+0000","last_undegraded":"2024-04-21T08:47:00.718978+0000","last_fullsized":"2024-04-21T08:47:00.718978+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:07:12.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'8","reported_seq":178,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.175644+0000","last_change":"2024-04-21T08:44:33.485797+0000","last_active":"2024-04-21T08:47:10.175644+0000","last_peered":"2024-04-21T08:47:10.175644+0000","last_clean":"2024-04-21T08:47:10.175644+0000","last_became_active":"2024-04-21T08:44:33.485496+0000","last_became_peered":"2024-04-21T08:44:33.485496+0000","last_unstale":"2024-04-21T08:47:10.175644+0000","last_undegraded":"2024-04-21T08:47:10.175644+0000","last_fullsized":"2024-04-21T08:47:10.175644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:11:34.604611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1761,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748282+0000","last_change":"2024-04-21T08:44:34.506453+0000","last_active":"2024-04-21T08:47:00.748282+0000","last_peered":"2024-04-21T08:47:00.748282+0000","last_clean":"2024-04-21T08:47:00.748282+0000","last_became_active":"2024-04-21T08:44:34.506121+0000","last_became_peered":"2024-04-21T08:44:34.506121+0000","last_unstale":"2024-04-21T08:47:00.748282+0000","last_undegraded":"2024-04-21T08:47:00.748282+0000","last_fullsized":"2024-04-21T08:47:00.748282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:18:07.927366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875292+0000","last_change":"2024-04-21T08:44:38.523718+0000","last_active":"2024-04-21T08:47:00.875292+0000","last_peered":"2024-04-21T08:47:00.875292+0000","last_clean":"2024-04-21T08:47:00.875292+0000","last_became_active":"2024-04-21T08:44:38.523003+0000","last_became_peered":"2024-04-21T08:44:38.523003+0000","last_unstale":"2024-04-21T08:47:00.875292+0000","last_undegraded":"2024-04-21T08:47:00.875292+0000","last_fullsized":"2024-04-21T08:47:00.875292+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:14:20.837450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721393+0000","last_change":"2024-04-21T08:44:33.482029+0000","last_active":"2024-04-21T08:47:00.721393+0000","last_peered":"2024-04-21T08:47:00.721393+0000","last_clean":"2024-04-21T08:47:00.721393+0000","last_became_active":"2024-04-21T08:44:33.481713+0000","last_became_peered":"2024-04-21T08:44:33.481713+0000","last_unstale":"2024-04-21T08:47:00.721393+0000","last_undegraded":"2024-04-21T08:47:00.721393+0000","last_fullsized":"2024-04-21T08:47:00.721393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:46:34.717244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728921+0000","last_change":"2024-04-21T08:44:34.491944+0000","last_active":"2024-04-21T08:47:00.728921+0000","last_peered":"2024-04-21T08:47:00.728921+0000","last_clean":"2024-04-21T08:47:00.728921+0000","last_became_active":"2024-04-21T08:44:34.491638+0000","last_became_peered":"2024-04-21T08:44:34.491638+0000","last_unstale":"2024-04-21T08:47:00.728921+0000","last_undegraded":"2024-04-21T08:47:00.728921+0000","last_fullsized":"2024-04-21T08:47:00.728921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:08:07.925937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728793+0000","last_change":"2024-04-21T08:44:38.528971+0000","last_active":"2024-04-21T08:47:00.728793+0000","last_peered":"2024-04-21T08:47:00.728793+0000","last_clean":"2024-04-21T08:47:00.728793+0000","last_became_active":"2024-04-21T08:44:38.528453+0000","last_became_peered":"2024-04-21T08:44:38.528453+0000","last_unstale":"2024-04-21T08:47:00.728793+0000","last_undegraded":"2024-04-21T08:47:00.728793+0000","last_fullsized":"2024-04-21T08:47:00.728793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:50:31.237749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721764+0000","last_change":"2024-04-21T08:44:33.479262+0000","last_active":"2024-04-21T08:47:00.721764+0000","last_peered":"2024-04-21T08:47:00.721764+0000","last_clean":"2024-04-21T08:47:00.721764+0000","last_became_active":"2024-04-21T08:44:33.478907+0000","last_became_peered":"2024-04-21T08:44:33.478907+0000","last_unstale":"2024-04-21T08:47:00.721764+0000","last_undegraded":"2024-04-21T08:47:00.721764+0000","last_fullsized":"2024-04-21T08:47:00.721764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:33:43.879126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748024+0000","last_change":"2024-04-21T08:44:34.498503+0000","last_active":"2024-04-21T08:47:00.748024+0000","last_peered":"2024-04-21T08:47:00.748024+0000","last_clean":"2024-04-21T08:47:00.748024+0000","last_became_active":"2024-04-21T08:44:34.495140+0000","last_became_peered":"2024-04-21T08:44:34.495140+0000","last_unstale":"2024-04-21T08:47:00.748024+0000","last_undegraded":"2024-04-21T08:47:00.748024+0000","last_fullsized":"2024-04-21T08:47:00.748024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:38:24.518349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721372+0000","last_change":"2024-04-21T08:44:38.517717+0000","last_active":"2024-04-21T08:47:00.721372+0000","last_peered":"2024-04-21T08:47:00.721372+0000","last_clean":"2024-04-21T08:47:00.721372+0000","last_became_active":"2024-04-21T08:44:38.517272+0000","last_became_peered":"2024-04-21T08:44:38.517272+0000","last_unstale":"2024-04-21T08:47:00.721372+0000","last_undegraded":"2024-04-21T08:47:00.721372+0000","last_fullsized":"2024-04-21T08:47:00.721372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.664130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718792+0000","last_change":"2024-04-21T08:44:33.477498+0000","last_active":"2024-04-21T08:47:00.718792+0000","last_peered":"2024-04-21T08:47:00.718792+0000","last_clean":"2024-04-21T08:47:00.718792+0000","last_became_active":"2024-04-21T08:44:33.477139+0000","last_became_peered":"2024-04-21T08:44:33.477139+0000","last_unstale":"2024-04-21T08:47:00.718792+0000","last_undegraded":"2024-04-21T08:47:00.718792+0000","last_fullsized":"2024-04-21T08:47:00.718792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:26:59.390583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626701+0000","last_change":"2024-04-21T08:44:34.493586+0000","last_active":"2024-04-21T08:47:00.626701+0000","last_peered":"2024-04-21T08:47:00.626701+0000","last_clean":"2024-04-21T08:47:00.626701+0000","last_became_active":"2024-04-21T08:44:34.493258+0000","last_became_peered":"2024-04-21T08:44:34.493258+0000","last_unstale":"2024-04-21T08:47:00.626701+0000","last_undegraded":"2024-04-21T08:47:00.626701+0000","last_fullsized":"2024-04-21T08:47:00.626701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:11:18.783332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874853+0000","last_change":"2024-04-21T08:44:38.519783+0000","last_active":"2024-04-21T08:47:00.874853+0000","last_peered":"2024-04-21T08:47:00.874853+0000","last_clean":"2024-04-21T08:47:00.874853+0000","last_became_active":"2024-04-21T08:44:38.519397+0000","last_became_peered":"2024-04-21T08:44:38.519397+0000","last_unstale":"2024-04-21T08:47:00.874853+0000","last_undegraded":"2024-04-21T08:47:00.874853+0000","last_fullsized":"2024-04-21T08:47:00.874853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:31:08.740240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":170,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.175110+0000","last_change":"2024-04-21T08:44:33.490098+0000","last_active":"2024-04-21T08:47:10.175110+0000","last_peered":"2024-04-21T08:47:10.175110+0000","last_clean":"2024-04-21T08:47:10.175110+0000","last_became_active":"2024-04-21T08:44:33.489691+0000","last_became_peered":"2024-04-21T08:44:33.489691+0000","last_unstale":"2024-04-21T08:47:10.175110+0000","last_undegraded":"2024-04-21T08:47:10.175110+0000","last_fullsized":"2024-04-21T08:47:10.175110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:44.965998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626543+0000","last_change":"2024-04-21T08:46:27.489580+0000","last_active":"2024-04-21T08:47:00.626543+0000","last_peered":"2024-04-21T08:47:00.626543+0000","last_clean":"2024-04-21T08:47:00.626543+0000","last_became_active":"2024-04-21T08:46:27.489086+0000","last_became_peered":"2024-04-21T08:46:27.489086+0000","last_unstale":"2024-04-21T08:47:00.626543+0000","last_undegraded":"2024-04-21T08:47:00.626543+0000","last_fullsized":"2024-04-21T08:47:00.626543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:38:50.618957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874908+0000","last_change":"2024-04-21T08:44:38.521010+0000","last_active":"2024-04-21T08:47:00.874908+0000","last_peered":"2024-04-21T08:47:00.874908+0000","last_clean":"2024-04-21T08:47:00.874908+0000","last_became_active":"2024-04-21T08:44:38.520605+0000","last_became_peered":"2024-04-21T08:44:38.520605+0000","last_unstale":"2024-04-21T08:47:00.874908+0000","last_undegraded":"2024-04-21T08:47:00.874908+0000","last_fullsized":"2024-04-21T08:47:00.874908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:50.378803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728846+0000","last_change":"2024-04-21T08:45:27.410137+0000","last_active":"2024-04-21T08:47:00.728846+0000","last_peered":"2024-04-21T08:47:00.728846+0000","last_clean":"2024-04-21T08:47:00.728846+0000","last_became_active":"2024-04-21T08:45:27.409728+0000","last_became_peered":"2024-04-21T08:45:27.409728+0000","last_unstale":"2024-04-21T08:47:00.728846+0000","last_undegraded":"2024-04-21T08:47:00.728846+0000","last_fullsized":"2024-04-21T08:47:00.728846+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:48:17.934902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":172,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.175058+0000","last_change":"2024-04-21T08:44:33.494031+0000","last_active":"2024-04-21T08:47:10.175058+0000","last_peered":"2024-04-21T08:47:10.175058+0000","last_clean":"2024-04-21T08:47:10.175058+0000","last_became_active":"2024-04-21T08:44:33.493580+0000","last_became_peered":"2024-04-21T08:44:33.493580+0000","last_unstale":"2024-04-21T08:47:10.175058+0000","last_undegraded":"2024-04-21T08:47:10.175058+0000","last_fullsized":"2024-04-21T08:47:10.175058+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:39:48.230616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748658+0000","last_change":"2024-04-21T08:44:38.518067+0000","last_active":"2024-04-21T08:47:00.748658+0000","last_peered":"2024-04-21T08:47:00.748658+0000","last_clean":"2024-04-21T08:47:00.748658+0000","last_became_active":"2024-04-21T08:44:38.517594+0000","last_became_peered":"2024-04-21T08:44:38.517594+0000","last_unstale":"2024-04-21T08:47:00.748658+0000","last_undegraded":"2024-04-21T08:47:00.748658+0000","last_fullsized":"2024-04-21T08:47:00.748658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:26:35.261582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.718443+0000","last_change":"2024-04-21T08:45:27.517719+0000","last_active":"2024-04-21T08:47:00.718443+0000","last_peered":"2024-04-21T08:47:00.718443+0000","last_clean":"2024-04-21T08:47:00.718443+0000","last_became_active":"2024-04-21T08:45:27.517429+0000","last_became_peered":"2024-04-21T08:45:27.517429+0000","last_unstale":"2024-04-21T08:47:00.718443+0000","last_undegraded":"2024-04-21T08:47:00.718443+0000","last_fullsized":"2024-04-21T08:47:00.718443+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:49:34.097759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748213+0000","last_change":"2024-04-21T08:44:33.485110+0000","last_active":"2024-04-21T08:47:00.748213+0000","last_peered":"2024-04-21T08:47:00.748213+0000","last_clean":"2024-04-21T08:47:00.748213+0000","last_became_active":"2024-04-21T08:44:33.484722+0000","last_became_peered":"2024-04-21T08:44:33.484722+0000","last_unstale":"2024-04-21T08:47:00.748213+0000","last_undegraded":"2024-04-21T08:47:00.748213+0000","last_fullsized":"2024-04-21T08:47:00.748213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:47:16.773560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"94'148","reported_seq":412,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:02.744643+0000","last_change":"2024-04-21T08:43:17.984594+0000","last_active":"2024-04-21T08:47:02.744643+0000","last_peered":"2024-04-21T08:47:02.744643+0000","last_clean":"2024-04-21T08:47:02.744643+0000","last_became_active":"2024-04-21T08:43:17.984207+0000","last_became_peered":"2024-04-21T08:43:17.984207+0000","last_unstale":"2024-04-21T08:47:02.744643+0000","last_undegraded":"2024-04-21T08:47:02.744643+0000","last_fullsized":"2024-04-21T08:47:02.744643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:39:50.910206+0000","last_clean_scrub_stamp":"2024-04-21T08:39:50.910206+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:41:35.260499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721571+0000","last_change":"2024-04-21T08:45:27.518834+0000","last_active":"2024-04-21T08:47:00.721571+0000","last_peered":"2024-04-21T08:47:00.721571+0000","last_clean":"2024-04-21T08:47:00.721571+0000","last_became_active":"2024-04-21T08:45:27.518390+0000","last_became_peered":"2024-04-21T08:45:27.518390+0000","last_unstale":"2024-04-21T08:47:00.721571+0000","last_undegraded":"2024-04-21T08:47:00.721571+0000","last_fullsized":"2024-04-21T08:47:00.721571+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:58:32.673008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874296+0000","last_change":"2024-04-21T08:44:34.499975+0000","last_active":"2024-04-21T08:47:00.874296+0000","last_peered":"2024-04-21T08:47:00.874296+0000","last_clean":"2024-04-21T08:47:00.874296+0000","last_became_active":"2024-04-21T08:44:34.499646+0000","last_became_peered":"2024-04-21T08:44:34.499646+0000","last_unstale":"2024-04-21T08:47:00.874296+0000","last_undegraded":"2024-04-21T08:47:00.874296+0000","last_fullsized":"2024-04-21T08:47:00.874296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:19:38.082502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748114+0000","last_change":"2024-04-21T08:44:33.484449+0000","last_active":"2024-04-21T08:47:00.748114+0000","last_peered":"2024-04-21T08:47:00.748114+0000","last_clean":"2024-04-21T08:47:00.748114+0000","last_became_active":"2024-04-21T08:44:33.484155+0000","last_became_peered":"2024-04-21T08:44:33.484155+0000","last_unstale":"2024-04-21T08:47:00.748114+0000","last_undegraded":"2024-04-21T08:47:00.748114+0000","last_fullsized":"2024-04-21T08:47:00.748114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:02:03.480474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747766+0000","last_change":"2024-04-21T08:44:38.515283+0000","last_active":"2024-04-21T08:47:00.747766+0000","last_peered":"2024-04-21T08:47:00.747766+0000","last_clean":"2024-04-21T08:47:00.747766+0000","last_became_active":"2024-04-21T08:44:38.514764+0000","last_became_peered":"2024-04-21T08:44:38.514764+0000","last_unstale":"2024-04-21T08:47:00.747766+0000","last_undegraded":"2024-04-21T08:47:00.747766+0000","last_fullsized":"2024-04-21T08:47:00.747766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:17:36.378392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721569+0000","last_change":"2024-04-21T08:44:33.489198+0000","last_active":"2024-04-21T08:47:00.721569+0000","last_peered":"2024-04-21T08:47:00.721569+0000","last_clean":"2024-04-21T08:47:00.721569+0000","last_became_active":"2024-04-21T08:44:33.488794+0000","last_became_peered":"2024-04-21T08:44:33.488794+0000","last_unstale":"2024-04-21T08:47:00.721569+0000","last_undegraded":"2024-04-21T08:47:00.721569+0000","last_fullsized":"2024-04-21T08:47:00.721569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:24.948283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874977+0000","last_change":"2024-04-21T08:44:34.488811+0000","last_active":"2024-04-21T08:47:00.874977+0000","last_peered":"2024-04-21T08:47:00.874977+0000","last_clean":"2024-04-21T08:47:00.874977+0000","last_became_active":"2024-04-21T08:44:34.488472+0000","last_became_peered":"2024-04-21T08:44:34.488472+0000","last_unstale":"2024-04-21T08:47:00.874977+0000","last_undegraded":"2024-04-21T08:47:00.874977+0000","last_fullsized":"2024-04-21T08:47:00.874977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:52:44.779492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622045+0000","last_change":"2024-04-21T08:44:38.522711+0000","last_active":"2024-04-21T08:47:00.622045+0000","last_peered":"2024-04-21T08:47:00.622045+0000","last_clean":"2024-04-21T08:47:00.622045+0000","last_became_active":"2024-04-21T08:44:38.522322+0000","last_became_peered":"2024-04-21T08:44:38.522322+0000","last_unstale":"2024-04-21T08:47:00.622045+0000","last_undegraded":"2024-04-21T08:47:00.622045+0000","last_fullsized":"2024-04-21T08:47:00.622045+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:26:13.323283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'102","reported_seq":227,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.078157+0000","last_change":"2024-04-21T08:44:33.481815+0000","last_active":"2024-04-21T08:47:10.078157+0000","last_peered":"2024-04-21T08:47:10.078157+0000","last_clean":"2024-04-21T08:47:10.078157+0000","last_became_active":"2024-04-21T08:44:33.481469+0000","last_became_peered":"2024-04-21T08:44:33.481469+0000","last_unstale":"2024-04-21T08:47:10.078157+0000","last_undegraded":"2024-04-21T08:47:10.078157+0000","last_fullsized":"2024-04-21T08:47:10.078157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:13.970821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":185,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623173+0000","last_change":"2024-04-21T08:44:34.496996+0000","last_active":"2024-04-21T08:47:00.623173+0000","last_peered":"2024-04-21T08:47:00.623173+0000","last_clean":"2024-04-21T08:47:00.623173+0000","last_became_active":"2024-04-21T08:44:34.496676+0000","last_became_peered":"2024-04-21T08:44:34.496676+0000","last_unstale":"2024-04-21T08:47:00.623173+0000","last_undegraded":"2024-04-21T08:47:00.623173+0000","last_fullsized":"2024-04-21T08:47:00.623173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:43:08.262656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626592+0000","last_change":"2024-04-21T08:44:38.516240+0000","last_active":"2024-04-21T08:47:00.626592+0000","last_peered":"2024-04-21T08:47:00.626592+0000","last_clean":"2024-04-21T08:47:00.626592+0000","last_became_active":"2024-04-21T08:44:38.515801+0000","last_became_peered":"2024-04-21T08:44:38.515801+0000","last_unstale":"2024-04-21T08:47:00.626592+0000","last_undegraded":"2024-04-21T08:47:00.626592+0000","last_fullsized":"2024-04-21T08:47:00.626592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:39:46.520373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749302+0000","last_change":"2024-04-21T08:44:33.488952+0000","last_active":"2024-04-21T08:47:00.749302+0000","last_peered":"2024-04-21T08:47:00.749302+0000","last_clean":"2024-04-21T08:47:00.749302+0000","last_became_active":"2024-04-21T08:44:33.488617+0000","last_became_peered":"2024-04-21T08:44:33.488617+0000","last_unstale":"2024-04-21T08:47:00.749302+0000","last_undegraded":"2024-04-21T08:47:00.749302+0000","last_fullsized":"2024-04-21T08:47:00.749302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:37:24.679132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747751+0000","last_change":"2024-04-21T08:44:34.510568+0000","last_active":"2024-04-21T08:47:00.747751+0000","last_peered":"2024-04-21T08:47:00.747751+0000","last_clean":"2024-04-21T08:47:00.747751+0000","last_became_active":"2024-04-21T08:44:34.509900+0000","last_became_peered":"2024-04-21T08:44:34.509900+0000","last_unstale":"2024-04-21T08:47:00.747751+0000","last_undegraded":"2024-04-21T08:47:00.747751+0000","last_fullsized":"2024-04-21T08:47:00.747751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:08.426141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748835+0000","last_change":"2024-04-21T08:45:27.976874+0000","last_active":"2024-04-21T08:47:00.748835+0000","last_peered":"2024-04-21T08:47:00.748835+0000","last_clean":"2024-04-21T08:47:00.748835+0000","last_became_active":"2024-04-21T08:45:27.976387+0000","last_became_peered":"2024-04-21T08:45:27.976387+0000","last_unstale":"2024-04-21T08:47:00.748835+0000","last_undegraded":"2024-04-21T08:47:00.748835+0000","last_fullsized":"2024-04-21T08:47:00.748835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:38:18.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747268+0000","last_change":"2024-04-21T08:44:33.481803+0000","last_active":"2024-04-21T08:47:00.747268+0000","last_peered":"2024-04-21T08:47:00.747268+0000","last_clean":"2024-04-21T08:47:00.747268+0000","last_became_active":"2024-04-21T08:44:33.481256+0000","last_became_peered":"2024-04-21T08:44:33.481256+0000","last_unstale":"2024-04-21T08:47:00.747268+0000","last_undegraded":"2024-04-21T08:47:00.747268+0000","last_fullsized":"2024-04-21T08:47:00.747268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:24:17.804557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.721258+0000","last_change":"2024-04-21T08:44:34.490206+0000","last_active":"2024-04-21T08:47:00.721258+0000","last_peered":"2024-04-21T08:47:00.721258+0000","last_clean":"2024-04-21T08:47:00.721258+0000","last_became_active":"2024-04-21T08:44:34.489797+0000","last_became_peered":"2024-04-21T08:44:34.489797+0000","last_unstale":"2024-04-21T08:47:00.721258+0000","last_undegraded":"2024-04-21T08:47:00.721258+0000","last_fullsized":"2024-04-21T08:47:00.721258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:30:09.055429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.719147+0000","last_change":"2024-04-21T08:44:38.517385+0000","last_active":"2024-04-21T08:47:00.719147+0000","last_peered":"2024-04-21T08:47:00.719147+0000","last_clean":"2024-04-21T08:47:00.719147+0000","last_became_active":"2024-04-21T08:44:38.516964+0000","last_became_peered":"2024-04-21T08:44:38.516964+0000","last_unstale":"2024-04-21T08:47:00.719147+0000","last_undegraded":"2024-04-21T08:47:00.719147+0000","last_fullsized":"2024-04-21T08:47:00.719147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:37:33.364076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622039+0000","last_change":"2024-04-21T08:44:33.490586+0000","last_active":"2024-04-21T08:47:00.622039+0000","last_peered":"2024-04-21T08:47:00.622039+0000","last_clean":"2024-04-21T08:47:00.622039+0000","last_became_active":"2024-04-21T08:44:33.489281+0000","last_became_peered":"2024-04-21T08:44:33.489281+0000","last_unstale":"2024-04-21T08:47:00.622039+0000","last_undegraded":"2024-04-21T08:47:00.622039+0000","last_fullsized":"2024-04-21T08:47:00.622039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:41:29.856165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747602+0000","last_change":"2024-04-21T08:44:34.493394+0000","last_active":"2024-04-21T08:47:00.747602+0000","last_peered":"2024-04-21T08:47:00.747602+0000","last_clean":"2024-04-21T08:47:00.747602+0000","last_became_active":"2024-04-21T08:44:34.492367+0000","last_became_peered":"2024-04-21T08:44:34.492367+0000","last_unstale":"2024-04-21T08:47:00.747602+0000","last_undegraded":"2024-04-21T08:47:00.747602+0000","last_fullsized":"2024-04-21T08:47:00.747602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:13.111266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748165+0000","last_change":"2024-04-21T08:44:38.519113+0000","last_active":"2024-04-21T08:47:00.748165+0000","last_peered":"2024-04-21T08:47:00.748165+0000","last_clean":"2024-04-21T08:47:00.748165+0000","last_became_active":"2024-04-21T08:44:38.518469+0000","last_became_peered":"2024-04-21T08:44:38.518469+0000","last_unstale":"2024-04-21T08:47:00.748165+0000","last_undegraded":"2024-04-21T08:47:00.748165+0000","last_fullsized":"2024-04-21T08:47:00.748165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:39:22.200041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'5","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747723+0000","last_change":"2024-04-21T08:46:27.492549+0000","last_active":"2024-04-21T08:47:00.747723+0000","last_peered":"2024-04-21T08:47:00.747723+0000","last_clean":"2024-04-21T08:47:00.747723+0000","last_became_active":"2024-04-21T08:46:27.488054+0000","last_became_peered":"2024-04-21T08:46:27.488054+0000","last_unstale":"2024-04-21T08:47:00.747723+0000","last_undegraded":"2024-04-21T08:47:00.747723+0000","last_fullsized":"2024-04-21T08:47:00.747723+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:51:54.797532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622446+0000","last_change":"2024-04-21T08:44:34.502226+0000","last_active":"2024-04-21T08:47:00.622446+0000","last_peered":"2024-04-21T08:47:00.622446+0000","last_clean":"2024-04-21T08:47:00.622446+0000","last_became_active":"2024-04-21T08:44:34.501775+0000","last_became_peered":"2024-04-21T08:44:34.501775+0000","last_unstale":"2024-04-21T08:47:00.622446+0000","last_undegraded":"2024-04-21T08:47:00.622446+0000","last_fullsized":"2024-04-21T08:47:00.622446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:58.325267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748655+0000","last_change":"2024-04-21T08:44:38.522250+0000","last_active":"2024-04-21T08:47:00.748655+0000","last_peered":"2024-04-21T08:47:00.748655+0000","last_clean":"2024-04-21T08:47:00.748655+0000","last_became_active":"2024-04-21T08:44:38.520870+0000","last_became_peered":"2024-04-21T08:44:38.520870+0000","last_unstale":"2024-04-21T08:47:00.748655+0000","last_undegraded":"2024-04-21T08:47:00.748655+0000","last_fullsized":"2024-04-21T08:47:00.748655+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:47:02.391785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747761+0000","last_change":"2024-04-21T08:46:27.496215+0000","last_active":"2024-04-21T08:47:00.747761+0000","last_peered":"2024-04-21T08:47:00.747761+0000","last_clean":"2024-04-21T08:47:00.747761+0000","last_became_active":"2024-04-21T08:46:27.488287+0000","last_became_peered":"2024-04-21T08:46:27.488287+0000","last_unstale":"2024-04-21T08:47:00.747761+0000","last_undegraded":"2024-04-21T08:47:00.747761+0000","last_fullsized":"2024-04-21T08:47:00.747761+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:08:05.390563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749473+0000","last_change":"2024-04-21T08:44:34.496034+0000","last_active":"2024-04-21T08:47:00.749473+0000","last_peered":"2024-04-21T08:47:00.749473+0000","last_clean":"2024-04-21T08:47:00.749473+0000","last_became_active":"2024-04-21T08:44:34.495641+0000","last_became_peered":"2024-04-21T08:44:34.495641+0000","last_unstale":"2024-04-21T08:47:00.749473+0000","last_undegraded":"2024-04-21T08:47:00.749473+0000","last_fullsized":"2024-04-21T08:47:00.749473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:21:18.893326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747361+0000","last_change":"2024-04-21T08:44:38.517288+0000","last_active":"2024-04-21T08:47:00.747361+0000","last_peered":"2024-04-21T08:47:00.747361+0000","last_clean":"2024-04-21T08:47:00.747361+0000","last_became_active":"2024-04-21T08:44:38.516853+0000","last_became_peered":"2024-04-21T08:44:38.516853+0000","last_unstale":"2024-04-21T08:47:00.747361+0000","last_undegraded":"2024-04-21T08:47:00.747361+0000","last_fullsized":"2024-04-21T08:47:00.747361+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:27:22.371703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":175,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:10.251660+0000","last_change":"2024-04-21T08:44:33.491578+0000","last_active":"2024-04-21T08:47:10.251660+0000","last_peered":"2024-04-21T08:47:10.251660+0000","last_clean":"2024-04-21T08:47:10.251660+0000","last_became_active":"2024-04-21T08:44:33.491238+0000","last_became_peered":"2024-04-21T08:44:33.491238+0000","last_unstale":"2024-04-21T08:47:10.251660+0000","last_undegraded":"2024-04-21T08:47:10.251660+0000","last_fullsized":"2024-04-21T08:47:10.251660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:50.399031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2343,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749074+0000","last_change":"2024-04-21T08:44:34.497632+0000","last_active":"2024-04-21T08:47:00.749074+0000","last_peered":"2024-04-21T08:47:00.749074+0000","last_clean":"2024-04-21T08:47:00.749074+0000","last_became_active":"2024-04-21T08:44:34.497318+0000","last_became_peered":"2024-04-21T08:44:34.497318+0000","last_unstale":"2024-04-21T08:47:00.749074+0000","last_undegraded":"2024-04-21T08:47:00.749074+0000","last_fullsized":"2024-04-21T08:47:00.749074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:11:33.918530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747370+0000","last_change":"2024-04-21T08:44:38.518414+0000","last_active":"2024-04-21T08:47:00.747370+0000","last_peered":"2024-04-21T08:47:00.747370+0000","last_clean":"2024-04-21T08:47:00.747370+0000","last_became_active":"2024-04-21T08:44:38.517953+0000","last_became_peered":"2024-04-21T08:44:38.517953+0000","last_unstale":"2024-04-21T08:47:00.747370+0000","last_undegraded":"2024-04-21T08:47:00.747370+0000","last_fullsized":"2024-04-21T08:47:00.747370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:48.088929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'67","reported_seq":132,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:09.745995+0000","last_change":"2024-04-21T08:44:33.493896+0000","last_active":"2024-04-21T08:47:09.745995+0000","last_peered":"2024-04-21T08:47:09.745995+0000","last_clean":"2024-04-21T08:47:09.745995+0000","last_became_active":"2024-04-21T08:44:33.493566+0000","last_became_peered":"2024-04-21T08:44:33.493566+0000","last_unstale":"2024-04-21T08:47:09.745995+0000","last_undegraded":"2024-04-21T08:47:09.745995+0000","last_fullsized":"2024-04-21T08:47:09.745995+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:47:17.307690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748612+0000","last_change":"2024-04-21T08:44:34.500406+0000","last_active":"2024-04-21T08:47:00.748612+0000","last_peered":"2024-04-21T08:47:00.748612+0000","last_clean":"2024-04-21T08:47:00.748612+0000","last_became_active":"2024-04-21T08:44:34.500048+0000","last_became_peered":"2024-04-21T08:44:34.500048+0000","last_unstale":"2024-04-21T08:47:00.748612+0000","last_undegraded":"2024-04-21T08:47:00.748612+0000","last_fullsized":"2024-04-21T08:47:00.748612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:10:58.166155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.728633+0000","last_change":"2024-04-21T08:44:38.514812+0000","last_active":"2024-04-21T08:47:00.728633+0000","last_peered":"2024-04-21T08:47:00.728633+0000","last_clean":"2024-04-21T08:47:00.728633+0000","last_became_active":"2024-04-21T08:44:38.514247+0000","last_became_peered":"2024-04-21T08:44:38.514247+0000","last_unstale":"2024-04-21T08:47:00.728633+0000","last_undegraded":"2024-04-21T08:47:00.728633+0000","last_fullsized":"2024-04-21T08:47:00.728633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:37.494305+0000","last_clean_scrub_stamp":"2024-04-21T08:44:37.494305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:06:10.588453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875149+0000","last_change":"2024-04-21T08:44:33.493887+0000","last_active":"2024-04-21T08:47:00.875149+0000","last_peered":"2024-04-21T08:47:00.875149+0000","last_clean":"2024-04-21T08:47:00.875149+0000","last_became_active":"2024-04-21T08:44:33.493555+0000","last_became_peered":"2024-04-21T08:44:33.493555+0000","last_unstale":"2024-04-21T08:47:00.875149+0000","last_undegraded":"2024-04-21T08:47:00.875149+0000","last_fullsized":"2024-04-21T08:47:00.875149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:32.459024+0000","last_clean_scrub_stamp":"2024-04-21T08:44:32.459024+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:36:18.089102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.749096+0000","last_change":"2024-04-21T08:44:34.503259+0000","last_active":"2024-04-21T08:47:00.749096+0000","last_peered":"2024-04-21T08:47:00.749096+0000","last_clean":"2024-04-21T08:47:00.749096+0000","last_became_active":"2024-04-21T08:44:34.502845+0000","last_became_peered":"2024-04-21T08:44:34.502845+0000","last_unstale":"2024-04-21T08:47:00.749096+0000","last_undegraded":"2024-04-21T08:47:00.749096+0000","last_fullsized":"2024-04-21T08:47:00.749096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:19.902178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747653+0000","last_change":"2024-04-21T08:44:34.497608+0000","last_active":"2024-04-21T08:47:00.747653+0000","last_peered":"2024-04-21T08:47:00.747653+0000","last_clean":"2024-04-21T08:47:00.747653+0000","last_became_active":"2024-04-21T08:44:34.497255+0000","last_became_peered":"2024-04-21T08:44:34.497255+0000","last_unstale":"2024-04-21T08:47:00.747653+0000","last_undegraded":"2024-04-21T08:47:00.747653+0000","last_fullsized":"2024-04-21T08:47:00.747653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:51:15.181837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626828+0000","last_change":"2024-04-21T08:44:34.495408+0000","last_active":"2024-04-21T08:47:00.626828+0000","last_peered":"2024-04-21T08:47:00.626828+0000","last_clean":"2024-04-21T08:47:00.626828+0000","last_became_active":"2024-04-21T08:44:34.495038+0000","last_became_peered":"2024-04-21T08:44:34.495038+0000","last_unstale":"2024-04-21T08:47:00.626828+0000","last_undegraded":"2024-04-21T08:47:00.626828+0000","last_fullsized":"2024-04-21T08:47:00.626828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:19:59.512245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.626703+0000","last_change":"2024-04-21T08:44:34.493292+0000","last_active":"2024-04-21T08:47:00.626703+0000","last_peered":"2024-04-21T08:47:00.626703+0000","last_clean":"2024-04-21T08:47:00.626703+0000","last_became_active":"2024-04-21T08:44:34.492913+0000","last_became_peered":"2024-04-21T08:44:34.492913+0000","last_unstale":"2024-04-21T08:47:00.626703+0000","last_undegraded":"2024-04-21T08:47:00.626703+0000","last_fullsized":"2024-04-21T08:47:00.626703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:21:24.271859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747551+0000","last_change":"2024-04-21T08:44:34.510448+0000","last_active":"2024-04-21T08:47:00.747551+0000","last_peered":"2024-04-21T08:47:00.747551+0000","last_clean":"2024-04-21T08:47:00.747551+0000","last_became_active":"2024-04-21T08:44:34.509958+0000","last_became_peered":"2024-04-21T08:44:34.509958+0000","last_unstale":"2024-04-21T08:47:00.747551+0000","last_undegraded":"2024-04-21T08:47:00.747551+0000","last_fullsized":"2024-04-21T08:47:00.747551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:47:36.797324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.875000+0000","last_change":"2024-04-21T08:44:34.497415+0000","last_active":"2024-04-21T08:47:00.875000+0000","last_peered":"2024-04-21T08:47:00.875000+0000","last_clean":"2024-04-21T08:47:00.875000+0000","last_became_active":"2024-04-21T08:44:34.497120+0000","last_became_peered":"2024-04-21T08:44:34.497120+0000","last_unstale":"2024-04-21T08:47:00.875000+0000","last_undegraded":"2024-04-21T08:47:00.875000+0000","last_fullsized":"2024-04-21T08:47:00.875000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:11:30.232338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747974+0000","last_change":"2024-04-21T08:44:34.500356+0000","last_active":"2024-04-21T08:47:00.747974+0000","last_peered":"2024-04-21T08:47:00.747974+0000","last_clean":"2024-04-21T08:47:00.747974+0000","last_became_active":"2024-04-21T08:44:34.500032+0000","last_became_peered":"2024-04-21T08:44:34.500032+0000","last_unstale":"2024-04-21T08:47:00.747974+0000","last_undegraded":"2024-04-21T08:47:00.747974+0000","last_fullsized":"2024-04-21T08:47:00.747974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:57:24.900221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.627464+0000","last_change":"2024-04-21T08:44:34.493629+0000","last_active":"2024-04-21T08:47:00.627464+0000","last_peered":"2024-04-21T08:47:00.627464+0000","last_clean":"2024-04-21T08:47:00.627464+0000","last_became_active":"2024-04-21T08:44:34.493273+0000","last_became_peered":"2024-04-21T08:44:34.493273+0000","last_unstale":"2024-04-21T08:47:00.627464+0000","last_undegraded":"2024-04-21T08:47:00.627464+0000","last_fullsized":"2024-04-21T08:47:00.627464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:22:42.698957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.623249+0000","last_change":"2024-04-21T08:44:34.493427+0000","last_active":"2024-04-21T08:47:00.623249+0000","last_peered":"2024-04-21T08:47:00.623249+0000","last_clean":"2024-04-21T08:47:00.623249+0000","last_became_active":"2024-04-21T08:44:34.493105+0000","last_became_peered":"2024-04-21T08:44:34.493105+0000","last_unstale":"2024-04-21T08:47:00.623249+0000","last_undegraded":"2024-04-21T08:47:00.623249+0000","last_fullsized":"2024-04-21T08:47:00.623249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:48:37.867487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874153+0000","last_change":"2024-04-21T08:45:27.515899+0000","last_active":"2024-04-21T08:47:00.874153+0000","last_peered":"2024-04-21T08:47:00.874153+0000","last_clean":"2024-04-21T08:47:00.874153+0000","last_became_active":"2024-04-21T08:45:27.515567+0000","last_became_peered":"2024-04-21T08:45:27.515567+0000","last_unstale":"2024-04-21T08:47:00.874153+0000","last_undegraded":"2024-04-21T08:47:00.874153+0000","last_fullsized":"2024-04-21T08:47:00.874153+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:48:16.645869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.622080+0000","last_change":"2024-04-21T08:44:34.499420+0000","last_active":"2024-04-21T08:47:00.622080+0000","last_peered":"2024-04-21T08:47:00.622080+0000","last_clean":"2024-04-21T08:47:00.622080+0000","last_became_active":"2024-04-21T08:44:34.499079+0000","last_became_peered":"2024-04-21T08:44:34.499079+0000","last_unstale":"2024-04-21T08:47:00.622080+0000","last_undegraded":"2024-04-21T08:47:00.622080+0000","last_fullsized":"2024-04-21T08:47:00.622080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:45.351422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.747540+0000","last_change":"2024-04-21T08:44:34.493420+0000","last_active":"2024-04-21T08:47:00.747540+0000","last_peered":"2024-04-21T08:47:00.747540+0000","last_clean":"2024-04-21T08:47:00.747540+0000","last_became_active":"2024-04-21T08:44:34.493098+0000","last_became_peered":"2024-04-21T08:44:34.493098+0000","last_unstale":"2024-04-21T08:47:00.747540+0000","last_undegraded":"2024-04-21T08:47:00.747540+0000","last_fullsized":"2024-04-21T08:47:00.747540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:22:34.316822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.874666+0000","last_change":"2024-04-21T08:44:34.494760+0000","last_active":"2024-04-21T08:47:00.874666+0000","last_peered":"2024-04-21T08:47:00.874666+0000","last_clean":"2024-04-21T08:47:00.874666+0000","last_became_active":"2024-04-21T08:44:34.494265+0000","last_became_peered":"2024-04-21T08:44:34.494265+0000","last_unstale":"2024-04-21T08:47:00.874666+0000","last_undegraded":"2024-04-21T08:47:00.874666+0000","last_fullsized":"2024-04-21T08:47:00.874666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:16.035030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:47:00.748002+0000","last_change":"2024-04-21T08:44:34.498785+0000","last_active":"2024-04-21T08:47:00.748002+0000","last_peered":"2024-04-21T08:47:00.748002+0000","last_clean":"2024-04-21T08:47:00.748002+0000","last_became_active":"2024-04-21T08:44:34.495159+0000","last_became_peered":"2024-04-21T08:44:34.495159+0000","last_unstale":"2024-04-21T08:47:00.748002+0000","last_undegraded":"2024-04-21T08:47:00.748002+0000","last_fullsized":"2024-04-21T08:47:00.748002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:44:33.464128+0000","last_clean_scrub_stamp":"2024-04-21T08:44:33.464128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:03:20.095016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801645,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1706,"num_read_kb":1077,"num_write":505,"num_write_kb":2388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2404935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":385,"ondisk_log_size":385,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383725,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1081344,"data_stored":531484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645362,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29200,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715952,"statfs":{"total":95995035648,"available":95965134848,"internally_reserved":0,"allocated":2113536,"data_stored":1516480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1503232,"data_stored":927553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070524,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1495040,"data_stored":923835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332162,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1699840,"data_stored":1129379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":525745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":596723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":528374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":599597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28912,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716240,"statfs":{"total":95995035648,"available":95965429760,"internally_reserved":0,"allocated":1818624,"data_stored":1246901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":530992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757324,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37448,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707704,"statfs":{"total":95995035648,"available":95956688896,"internally_reserved":0,"allocated":2039808,"data_stored":1452020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":74574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":332533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":401089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:47:14.191 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:47:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.325+0000 7f126d025640 1 -- 172.21.15.82:0/1326247302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1268074220 msgr2=0x7f1268074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.325+0000 7f126d025640 1 --2- 172.21.15.82:0/1326247302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1268074220 0x7f1268074600 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1250009900 tx=0x7f125002bcd0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.325+0000 7f126d025640 1 -- 172.21.15.82:0/1326247302 shutdown_connections 2024-04-21T08:47:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.325+0000 7f126d025640 1 --2- 172.21.15.82:0/1326247302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f126810e830 0x7f1268110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.325+0000 7f126d025640 1 --2- 172.21.15.82:0/1326247302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1268074b40 0x7f126810e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.325+0000 7f126d025640 1 --2- 172.21.15.82:0/1326247302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1268074220 0x7f1268074600 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.325+0000 7f126d025640 1 -- 172.21.15.82:0/1326247302 >> 172.21.15.82:0/1326247302 conn(0x7f12680793e0 msgr2=0x7f12680797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.325+0000 7f126d025640 1 -- 172.21.15.82:0/1326247302 shutdown_connections 2024-04-21T08:47:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.325+0000 7f126d025640 1 -- 172.21.15.82:0/1326247302 wait complete. 2024-04-21T08:47:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.325+0000 7f126d025640 1 Processor -- start 2024-04-21T08:47:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126d025640 1 -- start start 2024-04-21T08:47:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126d025640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1268074220 0x7f12681a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126d025640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1268074b40 0x7f12681a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126d025640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f126810e830 0x7f126819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126d025640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f12681143d0 con 0x7f126810e830 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126d025640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1268114250 con 0x7f1268074220 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126d025640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1268114550 con 0x7f1268074b40 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f126810e830 0x7f126819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f126810e830 0x7f126819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56368/0 (socket says 172.21.15.82:56368) 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126759e640 1 -- 172.21.15.82:0/868381409 learned_addr learned my addr 172.21.15.82:0/868381409 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126659c640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1268074b40 0x7f12681a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f1266d9d640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1268074220 0x7f12681a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126759e640 1 -- 172.21.15.82:0/868381409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1268074b40 msgr2=0x7f12681a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126759e640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1268074b40 0x7f12681a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126759e640 1 -- 172.21.15.82:0/868381409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1268074220 msgr2=0x7f12681a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126759e640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1268074220 0x7f12681a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126759e640 1 -- 172.21.15.82:0/868381409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126819afd0 con 0x7f126810e830 2024-04-21T08:47:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f1266d9d640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1268074220 0x7f12681a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f126759e640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f126810e830 0x7f126819a710 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f126000ccb0 tx=0x7f126000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.326+0000 7f125ffff640 1 -- 172.21.15.82:0/868381409 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126000f450 con 0x7f126810e830 2024-04-21T08:47:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.327+0000 7f126d025640 1 -- 172.21.15.82:0/868381409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126819b2c0 con 0x7f126810e830 2024-04-21T08:47:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.327+0000 7f125ffff640 1 -- 172.21.15.82:0/868381409 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f126000faa0 con 0x7f126810e830 2024-04-21T08:47:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.327+0000 7f125ffff640 1 -- 172.21.15.82:0/868381409 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12600189d0 con 0x7f126810e830 2024-04-21T08:47:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.327+0000 7f126d025640 1 -- 172.21.15.82:0/868381409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12681a7580 con 0x7f126810e830 2024-04-21T08:47:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.327+0000 7f126d025640 1 -- 172.21.15.82:0/868381409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1268075960 con 0x7f126810e830 2024-04-21T08:47:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.329+0000 7f125ffff640 1 -- 172.21.15.82:0/868381409 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f126000fc50 con 0x7f126810e830 2024-04-21T08:47:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.329+0000 7f125ffff640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f12540a3e40 0x7f12540a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.329+0000 7f1266d9d640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f12540a3e40 0x7f12540a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.329+0000 7f125ffff640 1 -- 172.21.15.82:0/868381409 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f12600d1280 con 0x7f126810e830 2024-04-21T08:47:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.330+0000 7f1266d9d640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f12540a3e40 0x7f12540a6300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1250005ab0 tx=0x7f1250005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.333+0000 7f125ffff640 1 -- 172.21.15.82:0/868381409 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1260014030 con 0x7f126810e830 2024-04-21T08:47:14.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.551+0000 7f126d025640 1 -- 172.21.15.82:0/868381409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1268069230 con 0x7f126810e830 2024-04-21T08:47:14.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.552+0000 7f125ffff640 1 -- 172.21.15.82:0/868381409 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f126009d310 con 0x7f126810e830 2024-04-21T08:47:14.551 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:14.551 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:46:59.334314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:47:14.552 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 28 2024-04-21T08:47:14.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.553+0000 7f126d025640 1 -- 172.21.15.82:0/868381409 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f12540a3e40 msgr2=0x7f12540a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.553+0000 7f126d025640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f12540a3e40 0x7f12540a6300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1250005ab0 tx=0x7f1250005a20 comp rx=0 tx=0).stop 2024-04-21T08:47:14.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.553+0000 7f126d025640 1 -- 172.21.15.82:0/868381409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f126810e830 msgr2=0x7f126819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.553+0000 7f126d025640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f126810e830 0x7f126819a710 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f126000ccb0 tx=0x7f126000a620 comp rx=0 tx=0).stop 2024-04-21T08:47:14.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.553+0000 7f126d025640 1 -- 172.21.15.82:0/868381409 shutdown_connections 2024-04-21T08:47:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.553+0000 7f126d025640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f12540a3e40 0x7f12540a6300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.553+0000 7f126d025640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f126810e830 0x7f126819a710 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.553+0000 7f126d025640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1268074b40 0x7f12681a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.553+0000 7f126d025640 1 --2- 172.21.15.82:0/868381409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1268074220 0x7f12681a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.553+0000 7f126d025640 1 -- 172.21.15.82:0/868381409 >> 172.21.15.82:0/868381409 conn(0x7f12680793e0 msgr2=0x7f12680fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.554+0000 7f126d025640 1 -- 172.21.15.82:0/868381409 shutdown_connections 2024-04-21T08:47:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.554+0000 7f126d025640 1 -- 172.21.15.82:0/868381409 wait complete. 2024-04-21T08:47:14.565 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:47:14.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.664+0000 7f6c2497e640 1 -- 172.21.15.82:0/2251885234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c201031d0 msgr2=0x7f6c201035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.664+0000 7f6c2497e640 1 --2- 172.21.15.82:0/2251885234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c201031d0 0x7f6c201035b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009900 tx=0x7f6c1002bcd0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.664+0000 7f6c2497e640 1 -- 172.21.15.82:0/2251885234 shutdown_connections 2024-04-21T08:47:14.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.664+0000 7f6c2497e640 1 --2- 172.21.15.82:0/2251885234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c2010c5a0 0x7f6c2010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.664+0000 7f6c2497e640 1 --2- 172.21.15.82:0/2251885234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c20103af0 0x7f6c2010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.664+0000 7f6c2497e640 1 --2- 172.21.15.82:0/2251885234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c201031d0 0x7f6c201035b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.664+0000 7f6c2497e640 1 -- 172.21.15.82:0/2251885234 >> 172.21.15.82:0/2251885234 conn(0x7f6c200771d0 msgr2=0x7f6c200775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.664+0000 7f6c2497e640 1 -- 172.21.15.82:0/2251885234 shutdown_connections 2024-04-21T08:47:14.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.664+0000 7f6c2497e640 1 -- 172.21.15.82:0/2251885234 wait complete. 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.664+0000 7f6c2497e640 1 Processor -- start 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c2497e640 1 -- start start 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c2497e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c201031d0 0x7f6c2019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c2497e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c20103af0 0x7f6c2019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c2497e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c2010c5a0 0x7f6c20198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c2497e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6c20112260 con 0x7f6c2010c5a0 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c2497e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201120e0 con 0x7f6c201031d0 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c2497e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201123e0 con 0x7f6c20103af0 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c1ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c2010c5a0 0x7f6c20198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c1ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c2010c5a0 0x7f6c20198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56392/0 (socket says 172.21.15.82:56392) 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c1ed9d640 1 -- 172.21.15.82:0/1011785315 learned_addr learned my addr 172.21.15.82:0/1011785315 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c17fff640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c20103af0 0x7f6c2019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c1e59c640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c201031d0 0x7f6c2019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c1ed9d640 1 -- 172.21.15.82:0/1011785315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c20103af0 msgr2=0x7f6c2019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c1ed9d640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c20103af0 0x7f6c2019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c1ed9d640 1 -- 172.21.15.82:0/1011785315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c201031d0 msgr2=0x7f6c2019e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c1ed9d640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c201031d0 0x7f6c2019e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c1ed9d640 1 -- 172.21.15.82:0/1011785315 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c20198da0 con 0x7f6c2010c5a0 2024-04-21T08:47:14.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c1e59c640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c201031d0 0x7f6c2019e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:14.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c1ed9d640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c2010c5a0 0x7f6c20198620 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800e9d0 tx=0x7f6c1800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.665+0000 7f6c177fe640 1 -- 172.21.15.82:0/1011785315 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18004d20 con 0x7f6c2010c5a0 2024-04-21T08:47:14.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.666+0000 7f6c2497e640 1 -- 172.21.15.82:0/1011785315 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c20199090 con 0x7f6c2010c5a0 2024-04-21T08:47:14.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.666+0000 7f6c177fe640 1 -- 172.21.15.82:0/1011785315 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c18005020 con 0x7f6c2010c5a0 2024-04-21T08:47:14.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.666+0000 7f6c177fe640 1 -- 172.21.15.82:0/1011785315 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c180116b0 con 0x7f6c2010c5a0 2024-04-21T08:47:14.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.666+0000 7f6c2497e640 1 -- 172.21.15.82:0/1011785315 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c201a5330 con 0x7f6c2010c5a0 2024-04-21T08:47:14.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.667+0000 7f6c2497e640 1 -- 172.21.15.82:0/1011785315 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c20069230 con 0x7f6c2010c5a0 2024-04-21T08:47:14.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.668+0000 7f6c177fe640 1 -- 172.21.15.82:0/1011785315 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6c180051d0 con 0x7f6c2010c5a0 2024-04-21T08:47:14.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.668+0000 7f6c177fe640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c0c0a3e40 0x7f6c0c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.668+0000 7f6c1e59c640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c0c0a3e40 0x7f6c0c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.668+0000 7f6c177fe640 1 -- 172.21.15.82:0/1011785315 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6c180d1390 con 0x7f6c2010c5a0 2024-04-21T08:47:14.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.669+0000 7f6c1e59c640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c0c0a3e40 0x7f6c0c0a6300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c10036040 tx=0x7f6c10000bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.673+0000 7f6c177fe640 1 -- 172.21.15.82:0/1011785315 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c18016030 con 0x7f6c2010c5a0 2024-04-21T08:47:14.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.852+0000 7f6c2497e640 1 -- 172.21.15.82:0/1011785315 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c201076c0 con 0x7f6c2010c5a0 2024-04-21T08:47:14.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.854+0000 7f6c177fe640 1 -- 172.21.15.82:0/1011785315 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25247 (secure 0 0 0) 0x7f6c1809d420 con 0x7f6c2010c5a0 2024-04-21T08:47:14.854 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:14.854 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:47:00.617381+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2119539425":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6800/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/539366680":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/2275507980":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2883183315":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:47:14.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.856+0000 7f6c2497e640 1 -- 172.21.15.82:0/1011785315 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c0c0a3e40 msgr2=0x7f6c0c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.856+0000 7f6c2497e640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c0c0a3e40 0x7f6c0c0a6300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c10036040 tx=0x7f6c10000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.856+0000 7f6c2497e640 1 -- 172.21.15.82:0/1011785315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c2010c5a0 msgr2=0x7f6c20198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.856+0000 7f6c2497e640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c2010c5a0 0x7f6c20198620 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800e9d0 tx=0x7f6c1800edd0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.856+0000 7f6c2497e640 1 -- 172.21.15.82:0/1011785315 shutdown_connections 2024-04-21T08:47:14.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.856+0000 7f6c2497e640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c0c0a3e40 0x7f6c0c0a6300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.856+0000 7f6c2497e640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c2010c5a0 0x7f6c20198620 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.856+0000 7f6c2497e640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c20103af0 0x7f6c2019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.856+0000 7f6c2497e640 1 --2- 172.21.15.82:0/1011785315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c201031d0 0x7f6c2019e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.856+0000 7f6c2497e640 1 -- 172.21.15.82:0/1011785315 >> 172.21.15.82:0/1011785315 conn(0x7f6c200771d0 msgr2=0x7f6c200ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.856+0000 7f6c2497e640 1 -- 172.21.15.82:0/1011785315 shutdown_connections 2024-04-21T08:47:14.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.856+0000 7f6c2497e640 1 -- 172.21.15.82:0/1011785315 wait complete. 2024-04-21T08:47:14.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:14 smithi082 ceph-mon[27967]: from='client.15882 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:14.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:14 smithi082 ceph-mon[27967]: from='client.15888 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:14.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:14 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/868381409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:14.867 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:47:14.867 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:47:14.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.968+0000 7f37585d5640 1 -- 172.21.15.82:0/748333788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37501010a0 msgr2=0x7f3750101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.968+0000 7f37585d5640 1 --2- 172.21.15.82:0/748333788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37501010a0 0x7f3750101480 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f374c009900 tx=0x7f374c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:14.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.968+0000 7f37585d5640 1 -- 172.21.15.82:0/748333788 shutdown_connections 2024-04-21T08:47:14.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.968+0000 7f37585d5640 1 --2- 172.21.15.82:0/748333788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3750106170 0x7f375010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.968+0000 7f37585d5640 1 --2- 172.21.15.82:0/748333788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3750101a50 0x7f3750105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.968+0000 7f37585d5640 1 --2- 172.21.15.82:0/748333788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37501010a0 0x7f3750101480 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.968+0000 7f37585d5640 1 -- 172.21.15.82:0/748333788 >> 172.21.15.82:0/748333788 conn(0x7f3750077eb0 msgr2=0x7f37500782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.968+0000 7f37585d5640 1 -- 172.21.15.82:0/748333788 shutdown_connections 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.968+0000 7f37585d5640 1 -- 172.21.15.82:0/748333788 wait complete. 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f37585d5640 1 Processor -- start 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f37585d5640 1 -- start start 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f37585d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37501010a0 0x7f3750197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f37585d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3750106170 0x7f3750197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f37585d5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375019be50 0x7f375019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f37585d5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f37501110b0 con 0x7f375019be50 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f37585d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3750110f30 con 0x7f37501010a0 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f37585d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3750111230 con 0x7f3750106170 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f3756b72640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375019be50 0x7f375019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f3756b72640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375019be50 0x7f375019c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56410/0 (socket says 172.21.15.82:56410) 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f3756b72640 1 -- 172.21.15.82:0/930700261 learned_addr learned my addr 172.21.15.82:0/930700261 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f3755b70640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3750106170 0x7f3750197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f3756371640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37501010a0 0x7f3750197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f3756b72640 1 -- 172.21.15.82:0/930700261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3750106170 msgr2=0x7f3750197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f3756b72640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3750106170 0x7f3750197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f3756b72640 1 -- 172.21.15.82:0/930700261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37501010a0 msgr2=0x7f3750197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f3756b72640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37501010a0 0x7f3750197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f3756b72640 1 -- 172.21.15.82:0/930700261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375019c940 con 0x7f375019be50 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f3755b70640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3750106170 0x7f3750197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.969+0000 7f3756b72640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375019be50 0x7f375019c300 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f374800eaa0 tx=0x7f374800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.970+0000 7f37477fe640 1 -- 172.21.15.82:0/930700261 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374800cdb0 con 0x7f375019be50 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.970+0000 7f37477fe640 1 -- 172.21.15.82:0/930700261 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3748005020 con 0x7f375019be50 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.970+0000 7f37585d5640 1 -- 172.21.15.82:0/930700261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375019cc30 con 0x7f375019be50 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.970+0000 7f37477fe640 1 -- 172.21.15.82:0/930700261 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37480116d0 con 0x7f375019be50 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.970+0000 7f37585d5640 1 -- 172.21.15.82:0/930700261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37501a43d0 con 0x7f375019be50 2024-04-21T08:47:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.970+0000 7f37585d5640 1 -- 172.21.15.82:0/930700261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37500691c0 con 0x7f375019be50 2024-04-21T08:47:14.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.972+0000 7f37477fe640 1 -- 172.21.15.82:0/930700261 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f37480051d0 con 0x7f375019be50 2024-04-21T08:47:14.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.972+0000 7f37477fe640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f373c0a3e40 0x7f373c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:14.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.972+0000 7f3756371640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f373c0a3e40 0x7f373c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:14.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.972+0000 7f37477fe640 1 -- 172.21.15.82:0/930700261 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f37480d1490 con 0x7f375019be50 2024-04-21T08:47:14.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.973+0000 7f3756371640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f373c0a3e40 0x7f373c0a6300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f374c000c00 tx=0x7f374c004040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:14.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:14.976+0000 7f37477fe640 1 -- 172.21.15.82:0/930700261 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3748016030 con 0x7f375019be50 2024-04-21T08:47:14.975 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:14 smithi134 ceph-mon[31772]: from='client.15882 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:14.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:14 smithi134 ceph-mon[31772]: from='client.15888 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:14.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:14 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/868381409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:15.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:14 smithi106 ceph-mon[31072]: from='client.15882 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:15.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:14 smithi106 ceph-mon[31072]: from='client.15888 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:47:15.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:14 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/868381409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:15.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.154+0000 7f37585d5640 1 -- 172.21.15.82:0/930700261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3750101a50 con 0x7f375019be50 2024-04-21T08:47:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.154+0000 7f37477fe640 1 -- 172.21.15.82:0/930700261 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f374809d520 con 0x7f375019be50 2024-04-21T08:47:15.154 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:47:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.156+0000 7f37585d5640 1 -- 172.21.15.82:0/930700261 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f373c0a3e40 msgr2=0x7f373c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.156+0000 7f37585d5640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f373c0a3e40 0x7f373c0a6300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f374c000c00 tx=0x7f374c004040 comp rx=0 tx=0).stop 2024-04-21T08:47:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.156+0000 7f37585d5640 1 -- 172.21.15.82:0/930700261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375019be50 msgr2=0x7f375019c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.156+0000 7f37585d5640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375019be50 0x7f375019c300 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f374800eaa0 tx=0x7f374800ef60 comp rx=0 tx=0).stop 2024-04-21T08:47:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.156+0000 7f37585d5640 1 -- 172.21.15.82:0/930700261 shutdown_connections 2024-04-21T08:47:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.156+0000 7f37585d5640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f373c0a3e40 0x7f373c0a6300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.156+0000 7f37585d5640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375019be50 0x7f375019c300 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.156+0000 7f37585d5640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3750106170 0x7f3750197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.156+0000 7f37585d5640 1 --2- 172.21.15.82:0/930700261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37501010a0 0x7f3750197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.156+0000 7f37585d5640 1 -- 172.21.15.82:0/930700261 >> 172.21.15.82:0/930700261 conn(0x7f3750077eb0 msgr2=0x7f3750100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.156+0000 7f37585d5640 1 -- 172.21.15.82:0/930700261 shutdown_connections 2024-04-21T08:47:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.156+0000 7f37585d5640 1 -- 172.21.15.82:0/930700261 wait complete. 2024-04-21T08:47:15.167 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:47:15.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.268+0000 7f88c5b50640 1 -- 172.21.15.82:0/4145403962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c01010a0 msgr2=0x7f88c0101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.268+0000 7f88c5b50640 1 --2- 172.21.15.82:0/4145403962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c01010a0 0x7f88c0101480 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009900 tx=0x7f88b402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:15.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.268+0000 7f88c5b50640 1 -- 172.21.15.82:0/4145403962 shutdown_connections 2024-04-21T08:47:15.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.268+0000 7f88c5b50640 1 --2- 172.21.15.82:0/4145403962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c0106170 0x7f88c010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.268+0000 7f88c5b50640 1 --2- 172.21.15.82:0/4145403962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c0101a50 0x7f88c0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.268+0000 7f88c5b50640 1 --2- 172.21.15.82:0/4145403962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c01010a0 0x7f88c0101480 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.268+0000 7f88c5b50640 1 -- 172.21.15.82:0/4145403962 >> 172.21.15.82:0/4145403962 conn(0x7f88c00771d0 msgr2=0x7f88c00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.268+0000 7f88c5b50640 1 -- 172.21.15.82:0/4145403962 shutdown_connections 2024-04-21T08:47:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.268+0000 7f88c5b50640 1 -- 172.21.15.82:0/4145403962 wait complete. 2024-04-21T08:47:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88c5b50640 1 Processor -- start 2024-04-21T08:47:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88c5b50640 1 -- start start 2024-04-21T08:47:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88c5b50640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c01010a0 0x7f88c0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88c5b50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c0106170 0x7f88c0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88c5b50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c019cff0 0x7f88c019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88c5b50640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f88c01110b0 con 0x7f88c019cff0 2024-04-21T08:47:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88c5b50640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f88c0110f30 con 0x7f88c01010a0 2024-04-21T08:47:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88c5b50640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88c0111230 con 0x7f88c0106170 2024-04-21T08:47:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c019cff0 0x7f88c019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c019cff0 0x7f88c019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56428/0 (socket says 172.21.15.82:56428) 2024-04-21T08:47:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88bffff640 1 -- 172.21.15.82:0/2771831662 learned_addr learned my addr 172.21.15.82:0/2771831662 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88bf7fe640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c01010a0 0x7f88c0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88bffff640 1 -- 172.21.15.82:0/2771831662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c0106170 msgr2=0x7f88c0198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88bffff640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c0106170 0x7f88c0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88bffff640 1 -- 172.21.15.82:0/2771831662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c01010a0 msgr2=0x7f88c0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88bffff640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c01010a0 0x7f88c0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88bffff640 1 -- 172.21.15.82:0/2771831662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c019dae0 con 0x7f88c019cff0 2024-04-21T08:47:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88bf7fe640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c01010a0 0x7f88c0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.269+0000 7f88bffff640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c019cff0 0x7f88c019d4a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f88b800cce0 tx=0x7f88b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.270+0000 7f88bcff9640 1 -- 172.21.15.82:0/2771831662 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b800f450 con 0x7f88c019cff0 2024-04-21T08:47:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.270+0000 7f88bcff9640 1 -- 172.21.15.82:0/2771831662 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b800faa0 con 0x7f88c019cff0 2024-04-21T08:47:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.270+0000 7f88bcff9640 1 -- 172.21.15.82:0/2771831662 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8018b70 con 0x7f88c019cff0 2024-04-21T08:47:15.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.270+0000 7f88c5b50640 1 -- 172.21.15.82:0/2771831662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c019ddd0 con 0x7f88c019cff0 2024-04-21T08:47:15.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.270+0000 7f88c5b50640 1 -- 172.21.15.82:0/2771831662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c01a5570 con 0x7f88c019cff0 2024-04-21T08:47:15.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.271+0000 7f88c5b50640 1 -- 172.21.15.82:0/2771831662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c00691c0 con 0x7f88c019cff0 2024-04-21T08:47:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.272+0000 7f88bcff9640 1 -- 172.21.15.82:0/2771831662 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f88b8018d10 con 0x7f88c019cff0 2024-04-21T08:47:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.272+0000 7f88bcff9640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f88a80a3e40 0x7f88a80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.272+0000 7f88bf7fe640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f88a80a3e40 0x7f88a80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.272+0000 7f88bcff9640 1 -- 172.21.15.82:0/2771831662 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f88b80d1300 con 0x7f88c019cff0 2024-04-21T08:47:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.273+0000 7f88bf7fe640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f88a80a3e40 0x7f88a80a6300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f88b4036040 tx=0x7f88b4000bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.277+0000 7f88bcff9640 1 -- 172.21.15.82:0/2771831662 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88b8014030 con 0x7f88c019cff0 2024-04-21T08:47:15.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.453+0000 7f88c5b50640 1 -- 172.21.15.82:0/2771831662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f88c010a3b0 con 0x7f88c019cff0 2024-04-21T08:47:15.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.453+0000 7f88bcff9640 1 -- 172.21.15.82:0/2771831662 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f88b809d390 con 0x7f88c019cff0 2024-04-21T08:47:15.453 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:47:15.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.455+0000 7f88c5b50640 1 -- 172.21.15.82:0/2771831662 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f88a80a3e40 msgr2=0x7f88a80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.455+0000 7f88c5b50640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f88a80a3e40 0x7f88a80a6300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f88b4036040 tx=0x7f88b4000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.455+0000 7f88c5b50640 1 -- 172.21.15.82:0/2771831662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c019cff0 msgr2=0x7f88c019d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.455+0000 7f88c5b50640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c019cff0 0x7f88c019d4a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f88b800cce0 tx=0x7f88b800a620 comp rx=0 tx=0).stop 2024-04-21T08:47:15.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.455+0000 7f88c5b50640 1 -- 172.21.15.82:0/2771831662 shutdown_connections 2024-04-21T08:47:15.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.455+0000 7f88c5b50640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f88a80a3e40 0x7f88a80a6300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.455+0000 7f88c5b50640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88c019cff0 0x7f88c019d4a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.455+0000 7f88c5b50640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c0106170 0x7f88c0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.455+0000 7f88c5b50640 1 --2- 172.21.15.82:0/2771831662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c01010a0 0x7f88c0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.455+0000 7f88c5b50640 1 -- 172.21.15.82:0/2771831662 >> 172.21.15.82:0/2771831662 conn(0x7f88c00771d0 msgr2=0x7f88c006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.455+0000 7f88c5b50640 1 -- 172.21.15.82:0/2771831662 shutdown_connections 2024-04-21T08:47:15.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.455+0000 7f88c5b50640 1 -- 172.21.15.82:0/2771831662 wait complete. 2024-04-21T08:47:15.466 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:47:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 -- 172.21.15.82:0/3189865341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca4101ad0 msgr2=0x7f7ca410dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 --2- 172.21.15.82:0/3189865341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca4101ad0 0x7f7ca410dcf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009ca0 tx=0x7f7c9c02c140 comp rx=0 tx=0).stop 2024-04-21T08:47:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 -- 172.21.15.82:0/3189865341 shutdown_connections 2024-04-21T08:47:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 --2- 172.21.15.82:0/3189865341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca4101ad0 0x7f7ca410dcf0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 --2- 172.21.15.82:0/3189865341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca4101130 0x7f7ca4101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 --2- 172.21.15.82:0/3189865341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ca4105ea0 0x7f7ca4106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 -- 172.21.15.82:0/3189865341 >> 172.21.15.82:0/3189865341 conn(0x7f7ca4077e30 msgr2=0x7f7ca4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 -- 172.21.15.82:0/3189865341 shutdown_connections 2024-04-21T08:47:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 -- 172.21.15.82:0/3189865341 wait complete. 2024-04-21T08:47:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 Processor -- start 2024-04-21T08:47:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 -- start start 2024-04-21T08:47:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ca4101130 0x7f7ca4197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca4101ad0 0x7f7ca4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca4105ea0 0x7f7ca4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca41111d0 con 0x7f7ca4101ad0 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca4111050 con 0x7f7ca4101130 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.567+0000 7f7cab674640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca4111350 con 0x7f7ca4105ea0 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7ca8c0f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca4101ad0 0x7f7ca4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7ca8c0f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca4101ad0 0x7f7ca4197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56456/0 (socket says 172.21.15.82:56456) 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7ca8c0f640 1 -- 172.21.15.82:0/1503736685 learned_addr learned my addr 172.21.15.82:0/1503736685 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7ca9410640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ca4101130 0x7f7ca4197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7ca9c11640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca4105ea0 0x7f7ca4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7ca8c0f640 1 -- 172.21.15.82:0/1503736685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca4105ea0 msgr2=0x7f7ca4197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7ca8c0f640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca4105ea0 0x7f7ca4197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7ca8c0f640 1 -- 172.21.15.82:0/1503736685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ca4101130 msgr2=0x7f7ca4197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7ca8c0f640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ca4101130 0x7f7ca4197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7ca8c0f640 1 -- 172.21.15.82:0/1503736685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca41a6c70 con 0x7f7ca4101ad0 2024-04-21T08:47:15.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7ca9410640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ca4101130 0x7f7ca4197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:15.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7ca8c0f640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca4101ad0 0x7f7ca4197940 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000d950 tx=0x7f7ca000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7c9a7fc640 1 -- 172.21.15.82:0/1503736685 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0017070 con 0x7f7ca4101ad0 2024-04-21T08:47:15.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7c9a7fc640 1 -- 172.21.15.82:0/1503736685 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ca0009c30 con 0x7f7ca4101ad0 2024-04-21T08:47:15.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7cab674640 1 -- 172.21.15.82:0/1503736685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca41a6f60 con 0x7f7ca4101ad0 2024-04-21T08:47:15.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7c9a7fc640 1 -- 172.21.15.82:0/1503736685 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca000f620 con 0x7f7ca4101ad0 2024-04-21T08:47:15.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.568+0000 7f7cab674640 1 -- 172.21.15.82:0/1503736685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca41a7420 con 0x7f7ca4101ad0 2024-04-21T08:47:15.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.569+0000 7f7cab674640 1 -- 172.21.15.82:0/1503736685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ca40691c0 con 0x7f7ca4101ad0 2024-04-21T08:47:15.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.570+0000 7f7c9a7fc640 1 -- 172.21.15.82:0/1503736685 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f7ca0010460 con 0x7f7ca4101ad0 2024-04-21T08:47:15.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.571+0000 7f7c9a7fc640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7c900a3e40 0x7f7c900a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.571+0000 7f7ca9410640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7c900a3e40 0x7f7c900a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.571+0000 7f7c9a7fc640 1 -- 172.21.15.82:0/1503736685 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f7ca00d0ba0 con 0x7f7ca4101ad0 2024-04-21T08:47:15.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.571+0000 7f7ca9410640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7c900a3e40 0x7f7c900a6300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c940058c0 tx=0x7f7c94005ec0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.575+0000 7f7c9a7fc640 1 -- 172.21.15.82:0/1503736685 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ca009cbd0 con 0x7f7ca4101ad0 2024-04-21T08:47:15.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.754+0000 7f7cab674640 1 -- 172.21.15.82:0/1503736685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7ca4109410 con 0x7f7ca4101ad0 2024-04-21T08:47:15.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.754+0000 7f7c9a7fc640 1 -- 172.21.15.82:0/1503736685 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f7ca0009de0 con 0x7f7ca4101ad0 2024-04-21T08:47:15.754 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:47:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.756+0000 7f7cab674640 1 -- 172.21.15.82:0/1503736685 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7c900a3e40 msgr2=0x7f7c900a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.756+0000 7f7cab674640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7c900a3e40 0x7f7c900a6300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c940058c0 tx=0x7f7c94005ec0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.756+0000 7f7cab674640 1 -- 172.21.15.82:0/1503736685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca4101ad0 msgr2=0x7f7ca4197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.756+0000 7f7cab674640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca4101ad0 0x7f7ca4197940 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000d950 tx=0x7f7ca000de10 comp rx=0 tx=0).stop 2024-04-21T08:47:15.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.756+0000 7f7cab674640 1 -- 172.21.15.82:0/1503736685 shutdown_connections 2024-04-21T08:47:15.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.756+0000 7f7cab674640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ca4105ea0 0x7f7ca4197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.756+0000 7f7cab674640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7c900a3e40 0x7f7c900a6300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.756+0000 7f7cab674640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca4101ad0 0x7f7ca4197940 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.757+0000 7f7cab674640 1 --2- 172.21.15.82:0/1503736685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7ca4101130 0x7f7ca4197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.757+0000 7f7cab674640 1 -- 172.21.15.82:0/1503736685 >> 172.21.15.82:0/1503736685 conn(0x7f7ca4077e30 msgr2=0x7f7ca40fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.757+0000 7f7cab674640 1 -- 172.21.15.82:0/1503736685 shutdown_connections 2024-04-21T08:47:15.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.757+0000 7f7cab674640 1 -- 172.21.15.82:0/1503736685 wait complete. 2024-04-21T08:47:15.766 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:15 smithi082 ceph-mon[27967]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-21T08:47:15.766 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:15 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1011785315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:15.766 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:15 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/930700261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:15.766 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:15 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2771831662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:15.767 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:47:15.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.869+0000 7fec0dc25640 1 -- 172.21.15.82:0/2076992770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec081011a0 msgr2=0x7fec08101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.869+0000 7fec0dc25640 1 --2- 172.21.15.82:0/2076992770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec081011a0 0x7fec08101600 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7febfc009930 tx=0x7febfc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:15.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.869+0000 7fec0dc25640 1 -- 172.21.15.82:0/2076992770 shutdown_connections 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.869+0000 7fec0dc25640 1 --2- 172.21.15.82:0/2076992770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec08101b40 0x7fec0810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.869+0000 7fec0dc25640 1 --2- 172.21.15.82:0/2076992770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec081011a0 0x7fec08101600 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.869+0000 7fec0dc25640 1 --2- 172.21.15.82:0/2076992770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec08107030 0x7fec08107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.869+0000 7fec0dc25640 1 -- 172.21.15.82:0/2076992770 >> 172.21.15.82:0/2076992770 conn(0x7fec08076c50 msgr2=0x7fec08077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.869+0000 7fec0dc25640 1 -- 172.21.15.82:0/2076992770 shutdown_connections 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.869+0000 7fec0dc25640 1 -- 172.21.15.82:0/2076992770 wait complete. 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec0dc25640 1 Processor -- start 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec0dc25640 1 -- start start 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec0dc25640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec081011a0 0x7fec08199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec0dc25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec08101b40 0x7fec08199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec0dc25640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec08107030 0x7fec081a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec0dc25640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fec081110c0 con 0x7fec08107030 2024-04-21T08:47:15.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec0dc25640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec08110f40 con 0x7fec08101b40 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec0dc25640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec08111240 con 0x7fec081011a0 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec07fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec08107030 0x7fec081a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec07fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec08107030 0x7fec081a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56480/0 (socket says 172.21.15.82:56480) 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec07fff640 1 -- 172.21.15.82:0/2150373215 learned_addr learned my addr 172.21.15.82:0/2150373215 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec06ffd640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec08101b40 0x7fec08199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec077fe640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec081011a0 0x7fec08199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec07fff640 1 -- 172.21.15.82:0/2150373215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec081011a0 msgr2=0x7fec08199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec07fff640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec081011a0 0x7fec08199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec07fff640 1 -- 172.21.15.82:0/2150373215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec08101b40 msgr2=0x7fec08199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec07fff640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec08101b40 0x7fec08199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec07fff640 1 -- 172.21.15.82:0/2150373215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec081a4b20 con 0x7fec08107030 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.870+0000 7fec077fe640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec081011a0 0x7fec08199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:15.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.871+0000 7fec06ffd640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec08101b40 0x7fec08199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:15.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.871+0000 7fec07fff640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec08107030 0x7fec081a45e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fec0000cce0 tx=0x7fec0000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.871+0000 7fec04ff9640 1 -- 172.21.15.82:0/2150373215 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0000f450 con 0x7fec08107030 2024-04-21T08:47:15.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.871+0000 7fec0dc25640 1 -- 172.21.15.82:0/2150373215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec081a4cf0 con 0x7fec08107030 2024-04-21T08:47:15.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.871+0000 7fec04ff9640 1 -- 172.21.15.82:0/2150373215 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec0000faa0 con 0x7fec08107030 2024-04-21T08:47:15.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.871+0000 7fec04ff9640 1 -- 172.21.15.82:0/2150373215 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00018b70 con 0x7fec08107030 2024-04-21T08:47:15.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.871+0000 7fec0dc25640 1 -- 172.21.15.82:0/2150373215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec081a51c0 con 0x7fec08107030 2024-04-21T08:47:15.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.872+0000 7fec0dc25640 1 -- 172.21.15.82:0/2150373215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec08102590 con 0x7fec08107030 2024-04-21T08:47:15.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.873+0000 7fec04ff9640 1 -- 172.21.15.82:0/2150373215 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fec00018d10 con 0x7fec08107030 2024-04-21T08:47:15.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.874+0000 7fec04ff9640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7febf00a3e40 0x7febf00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:15.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.874+0000 7fec077fe640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7febf00a3e40 0x7febf00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:15.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.874+0000 7fec04ff9640 1 -- 172.21.15.82:0/2150373215 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fec000d1480 con 0x7fec08107030 2024-04-21T08:47:15.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.874+0000 7fec077fe640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7febf00a3e40 0x7febf00a6300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7febf8009ab0 tx=0x7febf80094b0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:15.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:15.877+0000 7fec04ff9640 1 -- 172.21.15.82:0/2150373215 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec00014030 con 0x7fec08107030 2024-04-21T08:47:15.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:15 smithi134 ceph-mon[31772]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-21T08:47:15.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:15 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1011785315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:15.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:15 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/930700261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:15.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:15 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2771831662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:16.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.054+0000 7fec0dc25640 1 -- 172.21.15.82:0/2150373215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fec08109340 con 0x7fec08107030 2024-04-21T08:47:16.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.054+0000 7fec04ff9640 1 -- 172.21.15.82:0/2150373215 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fec0009d540 con 0x7fec08107030 2024-04-21T08:47:16.054 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:47:16.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.056+0000 7fec0dc25640 1 -- 172.21.15.82:0/2150373215 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7febf00a3e40 msgr2=0x7febf00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.056+0000 7fec0dc25640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7febf00a3e40 0x7febf00a6300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7febf8009ab0 tx=0x7febf80094b0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.056+0000 7fec0dc25640 1 -- 172.21.15.82:0/2150373215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec08107030 msgr2=0x7fec081a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.056+0000 7fec0dc25640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec08107030 0x7fec081a45e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fec0000cce0 tx=0x7fec0000a620 comp rx=0 tx=0).stop 2024-04-21T08:47:16.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.056+0000 7fec0dc25640 1 -- 172.21.15.82:0/2150373215 shutdown_connections 2024-04-21T08:47:16.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.056+0000 7fec0dc25640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7febf00a3e40 0x7febf00a6300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.056+0000 7fec0dc25640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec08107030 0x7fec081a45e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.056+0000 7fec0dc25640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec08101b40 0x7fec08199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.056+0000 7fec0dc25640 1 --2- 172.21.15.82:0/2150373215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec081011a0 0x7fec08199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.056+0000 7fec0dc25640 1 -- 172.21.15.82:0/2150373215 >> 172.21.15.82:0/2150373215 conn(0x7fec08076c50 msgr2=0x7fec080ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.056+0000 7fec0dc25640 1 -- 172.21.15.82:0/2150373215 shutdown_connections 2024-04-21T08:47:16.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.056+0000 7fec0dc25640 1 -- 172.21.15.82:0/2150373215 wait complete. 2024-04-21T08:47:16.067 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:47:16.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:15 smithi106 ceph-mon[31072]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-21T08:47:16.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:15 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1011785315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:16.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:15 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/930700261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:47:16.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:15 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2771831662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:47:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.164+0000 7fd5175d5640 1 -- 172.21.15.82:0/1394043902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5101004c0 msgr2=0x7fd510100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.164+0000 7fd5175d5640 1 --2- 172.21.15.82:0/1394043902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5101004c0 0x7fd510100940 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd500009930 tx=0x7fd50002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.164+0000 7fd5175d5640 1 -- 172.21.15.82:0/1394043902 shutdown_connections 2024-04-21T08:47:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.164+0000 7fd5175d5640 1 --2- 172.21.15.82:0/1394043902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510100e80 0x7fd51010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.164+0000 7fd5175d5640 1 --2- 172.21.15.82:0/1394043902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5101004c0 0x7fd510100940 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.164+0000 7fd5175d5640 1 --2- 172.21.15.82:0/1394043902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd510106f50 0x7fd510107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.164+0000 7fd5175d5640 1 -- 172.21.15.82:0/1394043902 >> 172.21.15.82:0/1394043902 conn(0x7fd510077450 msgr2=0x7fd5100fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.164+0000 7fd5175d5640 1 -- 172.21.15.82:0/1394043902 shutdown_connections 2024-04-21T08:47:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.164+0000 7fd5175d5640 1 -- 172.21.15.82:0/1394043902 wait complete. 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd5175d5640 1 Processor -- start 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd5175d5640 1 -- start start 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd5175d5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5101004c0 0x7fd51019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd5175d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd510100e80 0x7fd51019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd5175d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510106f50 0x7fd510198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd5175d5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd510112280 con 0x7fd5101004c0 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd5175d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd510112100 con 0x7fd510100e80 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd5175d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd510112400 con 0x7fd510106f50 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd515371640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5101004c0 0x7fd51019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd515371640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5101004c0 0x7fd51019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56490/0 (socket says 172.21.15.82:56490) 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd515371640 1 -- 172.21.15.82:0/2606015824 learned_addr learned my addr 172.21.15.82:0/2606015824 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd515b72640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510106f50 0x7fd510198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd515371640 1 -- 172.21.15.82:0/2606015824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510106f50 msgr2=0x7fd510198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd515371640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510106f50 0x7fd510198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd515371640 1 -- 172.21.15.82:0/2606015824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd510100e80 msgr2=0x7fd51019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd515371640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd510100e80 0x7fd51019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd515371640 1 -- 172.21.15.82:0/2606015824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd510198d80 con 0x7fd5101004c0 2024-04-21T08:47:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.165+0000 7fd515371640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5101004c0 0x7fd51019e400 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd50800cce0 tx=0x7fd50800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.166+0000 7fd5067fc640 1 -- 172.21.15.82:0/2606015824 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508004b20 con 0x7fd5101004c0 2024-04-21T08:47:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.166+0000 7fd5067fc640 1 -- 172.21.15.82:0/2606015824 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd508004cc0 con 0x7fd5101004c0 2024-04-21T08:47:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.166+0000 7fd5175d5640 1 -- 172.21.15.82:0/2606015824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd510199070 con 0x7fd5101004c0 2024-04-21T08:47:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.166+0000 7fd5067fc640 1 -- 172.21.15.82:0/2606015824 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50800f6b0 con 0x7fd5101004c0 2024-04-21T08:47:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.166+0000 7fd5175d5640 1 -- 172.21.15.82:0/2606015824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5101a52c0 con 0x7fd5101004c0 2024-04-21T08:47:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.166+0000 7fd5175d5640 1 -- 172.21.15.82:0/2606015824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd510069230 con 0x7fd5101004c0 2024-04-21T08:47:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.168+0000 7fd5067fc640 1 -- 172.21.15.82:0/2606015824 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd50800f850 con 0x7fd5101004c0 2024-04-21T08:47:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.168+0000 7fd5067fc640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd4fc0a3e40 0x7fd4fc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.168+0000 7fd514b70640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd4fc0a3e40 0x7fd4fc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.168+0000 7fd5067fc640 1 -- 172.21.15.82:0/2606015824 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fd5080d13a0 con 0x7fd5101004c0 2024-04-21T08:47:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.169+0000 7fd514b70640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd4fc0a3e40 0x7fd4fc0a6300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd500036040 tx=0x7fd500000bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.172+0000 7fd5067fc640 1 -- 172.21.15.82:0/2606015824 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd508014030 con 0x7fd5101004c0 2024-04-21T08:47:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.389+0000 7fd5175d5640 1 -- 172.21.15.82:0/2606015824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd510199ae0 con 0x7fd5101004c0 2024-04-21T08:47:16.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.390+0000 7fd5067fc640 1 -- 172.21.15.82:0/2606015824 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7fd50809d430 con 0x7fd5101004c0 2024-04-21T08:47:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:16.390 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:46:59.334314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:47:16.390 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 28 2024-04-21T08:47:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.392+0000 7fd5175d5640 1 -- 172.21.15.82:0/2606015824 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd4fc0a3e40 msgr2=0x7fd4fc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.392+0000 7fd5175d5640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd4fc0a3e40 0x7fd4fc0a6300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd500036040 tx=0x7fd500000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.392+0000 7fd5175d5640 1 -- 172.21.15.82:0/2606015824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5101004c0 msgr2=0x7fd51019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.392+0000 7fd5175d5640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5101004c0 0x7fd51019e400 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd50800cce0 tx=0x7fd50800a620 comp rx=0 tx=0).stop 2024-04-21T08:47:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.392+0000 7fd5175d5640 1 -- 172.21.15.82:0/2606015824 shutdown_connections 2024-04-21T08:47:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.392+0000 7fd5175d5640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd510106f50 0x7fd510198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.392+0000 7fd5175d5640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd510100e80 0x7fd51019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.392+0000 7fd5175d5640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd4fc0a3e40 0x7fd4fc0a6300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.392+0000 7fd5175d5640 1 --2- 172.21.15.82:0/2606015824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5101004c0 0x7fd51019e400 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.392+0000 7fd5175d5640 1 -- 172.21.15.82:0/2606015824 >> 172.21.15.82:0/2606015824 conn(0x7fd510077450 msgr2=0x7fd510104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.392+0000 7fd5175d5640 1 -- 172.21.15.82:0/2606015824 shutdown_connections 2024-04-21T08:47:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.392+0000 7fd5175d5640 1 -- 172.21.15.82:0/2606015824 wait complete. 2024-04-21T08:47:16.402 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:47:16.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.500+0000 7f4d71c9a640 1 -- 172.21.15.82:0/2432997149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d6c105e30 msgr2=0x7f4d6c106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.500+0000 7f4d71c9a640 1 --2- 172.21.15.82:0/2432997149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d6c105e30 0x7f4d6c106210 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009930 tx=0x7f4d5c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:16.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.500+0000 7f4d71c9a640 1 -- 172.21.15.82:0/2432997149 shutdown_connections 2024-04-21T08:47:16.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 --2- 172.21.15.82:0/2432997149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d6c101b60 0x7f4d6c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 --2- 172.21.15.82:0/2432997149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d6c1011a0 0x7f4d6c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 --2- 172.21.15.82:0/2432997149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d6c105e30 0x7f4d6c106210 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 -- 172.21.15.82:0/2432997149 >> 172.21.15.82:0/2432997149 conn(0x7f4d6c077e30 msgr2=0x7f4d6c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 -- 172.21.15.82:0/2432997149 shutdown_connections 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 -- 172.21.15.82:0/2432997149 wait complete. 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 Processor -- start 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 -- start start 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d6c1011a0 0x7f4d6c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d6c101b60 0x7f4d6c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d6c105e30 0x7f4d6c1a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1110c0 con 0x7f4d6c1011a0 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c110f40 con 0x7f4d6c105e30 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.501+0000 7f4d71c9a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c111240 con 0x7f4d6c101b60 2024-04-21T08:47:16.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d6b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d6c1011a0 0x7f4d6c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d6b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d6c1011a0 0x7f4d6c1995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56508/0 (socket says 172.21.15.82:56508) 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d6b7fe640 1 -- 172.21.15.82:0/817128288 learned_addr learned my addr 172.21.15.82:0/817128288 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d62ffd640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d6c101b60 0x7f4d6c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d6bfff640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d6c105e30 0x7f4d6c1a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d6b7fe640 1 -- 172.21.15.82:0/817128288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d6c101b60 msgr2=0x7f4d6c199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d6b7fe640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d6c101b60 0x7f4d6c199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d6b7fe640 1 -- 172.21.15.82:0/817128288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d6c105e30 msgr2=0x7f4d6c1a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d6b7fe640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d6c105e30 0x7f4d6c1a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d6b7fe640 1 -- 172.21.15.82:0/817128288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c1a4b80 con 0x7f4d6c1011a0 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d6b7fe640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d6c1011a0 0x7f4d6c1995c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009230 tx=0x7f4d5c02c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d697fa640 1 -- 172.21.15.82:0/817128288 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c043020 con 0x7f4d6c1011a0 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d697fa640 1 -- 172.21.15.82:0/817128288 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d5c0043d0 con 0x7f4d6c1011a0 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d71c9a640 1 -- 172.21.15.82:0/817128288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6c1a4d50 con 0x7f4d6c1011a0 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d697fa640 1 -- 172.21.15.82:0/817128288 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c003bc0 con 0x7f4d6c1011a0 2024-04-21T08:47:16.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.502+0000 7f4d71c9a640 1 -- 172.21.15.82:0/817128288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6c1a51c0 con 0x7f4d6c1011a0 2024-04-21T08:47:16.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.503+0000 7f4d71c9a640 1 -- 172.21.15.82:0/817128288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d6c068de0 con 0x7f4d6c1011a0 2024-04-21T08:47:16.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.504+0000 7f4d697fa640 1 -- 172.21.15.82:0/817128288 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4d5c046020 con 0x7f4d6c1011a0 2024-04-21T08:47:16.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.505+0000 7f4d697fa640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4d580a3e40 0x7f4d580a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.505+0000 7f4d62ffd640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4d580a3e40 0x7f4d580a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.505+0000 7f4d697fa640 1 -- 172.21.15.82:0/817128288 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f4d5c0f2fc0 con 0x7f4d6c1011a0 2024-04-21T08:47:16.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.505+0000 7f4d62ffd640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4d580a3e40 0x7f4d580a6300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d54005fd0 tx=0x7f4d54005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.509+0000 7f4d697fa640 1 -- 172.21.15.82:0/817128288 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d5c04d020 con 0x7f4d6c1011a0 2024-04-21T08:47:16.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.689+0000 7f4d71c9a640 1 -- 172.21.15.82:0/817128288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d6c02d5c0 con 0x7f4d6c1011a0 2024-04-21T08:47:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.691+0000 7f4d697fa640 1 -- 172.21.15.82:0/817128288 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25247 (secure 0 0 0) 0x7f4d5c03c350 con 0x7f4d6c1011a0 2024-04-21T08:47:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:16.694 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:47:00.617381+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2119539425":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6800/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/539366680":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/2275507980":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2883183315":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:47:16.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.693+0000 7f4d71c9a640 1 -- 172.21.15.82:0/817128288 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4d580a3e40 msgr2=0x7f4d580a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.693+0000 7f4d71c9a640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4d580a3e40 0x7f4d580a6300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d54005fd0 tx=0x7f4d54005850 comp rx=0 tx=0).stop 2024-04-21T08:47:16.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.693+0000 7f4d71c9a640 1 -- 172.21.15.82:0/817128288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d6c1011a0 msgr2=0x7f4d6c1995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.693+0000 7f4d71c9a640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d6c1011a0 0x7f4d6c1995c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009230 tx=0x7f4d5c02c710 comp rx=0 tx=0).stop 2024-04-21T08:47:16.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.693+0000 7f4d71c9a640 1 -- 172.21.15.82:0/817128288 shutdown_connections 2024-04-21T08:47:16.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.693+0000 7f4d71c9a640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d6c105e30 0x7f4d6c1a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.693+0000 7f4d71c9a640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d6c101b60 0x7f4d6c199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.693+0000 7f4d71c9a640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4d580a3e40 0x7f4d580a6300 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.693+0000 7f4d71c9a640 1 --2- 172.21.15.82:0/817128288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d6c1011a0 0x7f4d6c1995c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.693+0000 7f4d71c9a640 1 -- 172.21.15.82:0/817128288 >> 172.21.15.82:0/817128288 conn(0x7f4d6c077e30 msgr2=0x7f4d6c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.693+0000 7f4d71c9a640 1 -- 172.21.15.82:0/817128288 shutdown_connections 2024-04-21T08:47:16.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.693+0000 7f4d71c9a640 1 -- 172.21.15.82:0/817128288 wait complete. 2024-04-21T08:47:16.704 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 664552 2024-04-21T08:47:16.704 DEBUG:tasks.quiescer:Yielding 2024-04-21T08:47:16.704 INFO:teuthology.run_tasks:Running task workunit... 2024-04-21T08:47:16.719 INFO:tasks.workunit:Pulling workunits from ref 4f6bc5273643aaf5660909e85400647a8951c10b 2024-04-21T08:47:16.719 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-21T08:47:16.719 INFO:tasks.workunit:timeout=3h 2024-04-21T08:47:16.719 INFO:tasks.workunit:cleanup=True 2024-04-21T08:47:16.719 DEBUG:teuthology.orchestra.run.smithi082:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-21T08:47:16.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:47:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-21T08:47:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-21T08:47:16.735 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-21T08:47:16.735 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:47:16.735 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-21T08:47:16.735 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-21 08:45:02.053994502 +0000 2024-04-21T08:47:16.735 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-21 08:45:02.053994502 +0000 2024-04-21T08:47:16.735 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-21 08:46:58.609805166 +0000 2024-04-21T08:47:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-21 08:45:02.053994502 +0000 2024-04-21T08:47:16.735 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-21T08:47:16.735 DEBUG:teuthology.orchestra.run.smithi082:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-21T08:47:16.806 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/leonid-s-usov/ceph-ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 4f6bc5273643aaf5660909e85400647a8951c10b 2024-04-21T08:47:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.820+0000 7f81bf9e6640 1 -- 172.21.15.82:0/3212701783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81b8101b60 msgr2=0x7f81b810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.820+0000 7f81bf9e6640 1 --2- 172.21.15.82:0/3212701783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81b8101b60 0x7f81b810dc90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009cd0 tx=0x7f81b002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.820+0000 7f81bf9e6640 1 -- 172.21.15.82:0/3212701783 shutdown_connections 2024-04-21T08:47:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.820+0000 7f81bf9e6640 1 --2- 172.21.15.82:0/3212701783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81b8101b60 0x7f81b810dc90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.820+0000 7f81bf9e6640 1 --2- 172.21.15.82:0/3212701783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81b81011a0 0x7f81b8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.820+0000 7f81bf9e6640 1 --2- 172.21.15.82:0/3212701783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b8105e30 0x7f81b8106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.820+0000 7f81bf9e6640 1 -- 172.21.15.82:0/3212701783 >> 172.21.15.82:0/3212701783 conn(0x7f81b8077e30 msgr2=0x7f81b8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bf9e6640 1 -- 172.21.15.82:0/3212701783 shutdown_connections 2024-04-21T08:47:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bf9e6640 1 -- 172.21.15.82:0/3212701783 wait complete. 2024-04-21T08:47:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bf9e6640 1 Processor -- start 2024-04-21T08:47:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bf9e6640 1 -- start start 2024-04-21T08:47:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bf9e6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81b81011a0 0x7f81b8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bf9e6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81b8101b60 0x7f81b8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bf9e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b8105e30 0x7f81b819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bf9e6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f81b81110e0 con 0x7f81b8101b60 2024-04-21T08:47:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bf9e6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f81b8110f60 con 0x7f81b81011a0 2024-04-21T08:47:16.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bf9e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f81b8111260 con 0x7f81b8105e30 2024-04-21T08:47:16.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bcf81640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81b8101b60 0x7f81b8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bcf81640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81b8101b60 0x7f81b8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56520/0 (socket says 172.21.15.82:56520) 2024-04-21T08:47:16.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.821+0000 7f81bcf81640 1 -- 172.21.15.82:0/2789025179 learned_addr learned my addr 172.21.15.82:0/2789025179 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:16.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81bdf83640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b8105e30 0x7f81b819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81bd782640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81b81011a0 0x7f81b8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81bcf81640 1 -- 172.21.15.82:0/2789025179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b8105e30 msgr2=0x7f81b819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81bcf81640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b8105e30 0x7f81b819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81bcf81640 1 -- 172.21.15.82:0/2789025179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81b81011a0 msgr2=0x7f81b8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81bcf81640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81b81011a0 0x7f81b8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81bcf81640 1 -- 172.21.15.82:0/2789025179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b819d6b0 con 0x7f81b8101b60 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81bd782640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81b81011a0 0x7f81b8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81bcf81640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81b8101b60 0x7f81b8198c50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f81b400c890 tx=0x7f81b400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81ae7fc640 1 -- 172.21.15.82:0/2789025179 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4004220 con 0x7f81b8101b60 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81ae7fc640 1 -- 172.21.15.82:0/2789025179 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b40043c0 con 0x7f81b8101b60 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81bf9e6640 1 -- 172.21.15.82:0/2789025179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b819d9a0 con 0x7f81b8101b60 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81ae7fc640 1 -- 172.21.15.82:0/2789025179 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4004a00 con 0x7f81b8101b60 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.822+0000 7f81bf9e6640 1 -- 172.21.15.82:0/2789025179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b81a5230 con 0x7f81b8101b60 2024-04-21T08:47:16.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.823+0000 7f81bf9e6640 1 -- 172.21.15.82:0/2789025179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b8068de0 con 0x7f81b8101b60 2024-04-21T08:47:16.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.824+0000 7f81ae7fc640 1 -- 172.21.15.82:0/2789025179 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f81b40079b0 con 0x7f81b8101b60 2024-04-21T08:47:16.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.825+0000 7f81ae7fc640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f81a40a3e40 0x7f81a40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:16.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.825+0000 7f81bd782640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f81a40a3e40 0x7f81a40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:16.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.825+0000 7f81ae7fc640 1 -- 172.21.15.82:0/2789025179 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f81b40d0be0 con 0x7f81b8101b60 2024-04-21T08:47:16.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.826+0000 7f81bd782640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f81a40a3e40 0x7f81a40a6300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f81a80058c0 tx=0x7f81a8005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:16.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:16.829+0000 7f81ae7fc640 1 -- 172.21.15.82:0/2789025179 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81b4014030 con 0x7f81b8101b60 2024-04-21T08:47:16.835 INFO:tasks.workunit.client.0.smithi082.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-21T08:47:16.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:16 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1503736685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:16.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:16 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2150373215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:16.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:16 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2606015824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:17.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.049+0000 7f81bf9e6640 1 -- 172.21.15.82:0/2789025179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81b81a54c0 con 0x7f81b8101b60 2024-04-21T08:47:17.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.049+0000 7f81ae7fc640 1 -- 172.21.15.82:0/2789025179 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f81b409cc50 con 0x7f81b8101b60 2024-04-21T08:47:17.049 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:17.049 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:46:59.334314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:47:17.049 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 28 2024-04-21T08:47:17.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.051+0000 7f81bf9e6640 1 -- 172.21.15.82:0/2789025179 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f81a40a3e40 msgr2=0x7f81a40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.051+0000 7f81bf9e6640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f81a40a3e40 0x7f81a40a6300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f81a80058c0 tx=0x7f81a8005850 comp rx=0 tx=0).stop 2024-04-21T08:47:17.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.051+0000 7f81bf9e6640 1 -- 172.21.15.82:0/2789025179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81b8101b60 msgr2=0x7f81b8198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.051+0000 7f81bf9e6640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81b8101b60 0x7f81b8198c50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f81b400c890 tx=0x7f81b400cd50 comp rx=0 tx=0).stop 2024-04-21T08:47:17.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.051+0000 7f81bf9e6640 1 -- 172.21.15.82:0/2789025179 shutdown_connections 2024-04-21T08:47:17.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.051+0000 7f81bf9e6640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81b8105e30 0x7f81b819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.051+0000 7f81bf9e6640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f81a40a3e40 0x7f81a40a6300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.051+0000 7f81bf9e6640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81b8101b60 0x7f81b8198c50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.051+0000 7f81bf9e6640 1 --2- 172.21.15.82:0/2789025179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81b81011a0 0x7f81b8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.051+0000 7f81bf9e6640 1 -- 172.21.15.82:0/2789025179 >> 172.21.15.82:0/2789025179 conn(0x7f81b8077e30 msgr2=0x7f81b8104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.051+0000 7f81bf9e6640 1 -- 172.21.15.82:0/2789025179 shutdown_connections 2024-04-21T08:47:17.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.051+0000 7f81bf9e6640 1 -- 172.21.15.82:0/2789025179 wait complete. 2024-04-21T08:47:17.061 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:16 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1503736685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:17.061 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:16 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2150373215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:17.061 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:16 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2606015824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:17.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T08:44:33.819176+0000', 'modified': '2024-04-21T08:46:59.334314+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24509, 'mds_1': 24481, 'mds_2': 24476, 'mds_3': 24409, 'mds_4': 24407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6839/3697606919', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 3697606919}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 3697606919}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6841/1327149046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6840', 'nonce': 1327149046}, {'type': 'v1', 'addr': '172.21.15.82:6841', 'nonce': 1327149046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.106:6833/2583766346', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6832', 'nonce': 2583766346}, {'type': 'v1', 'addr': '172.21.15.106:6833', 'nonce': 2583766346}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'l', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.134:6833/2942732132', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2942732132}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2942732132}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6835/3901395826', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6834', 'nonce': 3901395826}, {'type': 'v1', 'addr': '172.21.15.106:6835', 'nonce': 3901395826}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/2962604291', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 2962604291}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 2962604291}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/1870695477', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 1870695477}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 1870695477}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.106:6837/3731760515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 3731760515}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 3731760515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.134:6839/394037083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 394037083}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 394037083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/3796875133', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 3796875133}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 3796875133}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24407, 24409, 24476, 24481, 24509]} 2024-04-21T08:47:17.063 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-21T08:47:17.063 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-21T08:47:17.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:16 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1503736685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:47:17.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:16 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2150373215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:47:17.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:16 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2606015824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:17.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 -- 172.21.15.82:0/2775429569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1208106160 msgr2=0x7f120810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 --2- 172.21.15.82:0/2775429569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1208106160 0x7f120810dd30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1200009ca0 tx=0x7f120002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 -- 172.21.15.82:0/2775429569 shutdown_connections 2024-04-21T08:47:17.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 --2- 172.21.15.82:0/2775429569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1208106160 0x7f120810dd30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 --2- 172.21.15.82:0/2775429569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1208101a40 0x7f1208105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 --2- 172.21.15.82:0/2775429569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1208101090 0x7f1208101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 -- 172.21.15.82:0/2775429569 >> 172.21.15.82:0/2775429569 conn(0x7f1208077f50 msgr2=0x7f1208078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 -- 172.21.15.82:0/2775429569 shutdown_connections 2024-04-21T08:47:17.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 -- 172.21.15.82:0/2775429569 wait complete. 2024-04-21T08:47:17.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 Processor -- start 2024-04-21T08:47:17.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 -- start start 2024-04-21T08:47:17.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1208101090 0x7f1208198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1208106160 0x7f1208198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120819cfd0 0x7f120819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.203+0000 7f120fd11640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f12081110d0 con 0x7f1208106160 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120fd11640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1208110f50 con 0x7f1208101090 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120fd11640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1208111250 con 0x7f120819cfd0 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120d2ac640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1208106160 0x7f1208198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120d2ac640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1208106160 0x7f1208198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56542/0 (socket says 172.21.15.82:56542) 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120d2ac640 1 -- 172.21.15.82:0/960035223 learned_addr learned my addr 172.21.15.82:0/960035223 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120daad640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1208101090 0x7f1208198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120e2ae640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120819cfd0 0x7f120819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120d2ac640 1 -- 172.21.15.82:0/960035223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120819cfd0 msgr2=0x7f120819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120d2ac640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120819cfd0 0x7f120819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120d2ac640 1 -- 172.21.15.82:0/960035223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1208101090 msgr2=0x7f1208198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120d2ac640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1208101090 0x7f1208198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120d2ac640 1 -- 172.21.15.82:0/960035223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120819dac0 con 0x7f1208106160 2024-04-21T08:47:17.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120daad640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1208101090 0x7f1208198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:17.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120e2ae640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120819cfd0 0x7f120819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:17.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120d2ac640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1208106160 0x7f1208198c50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e9c0 tx=0x7f11fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f11faffd640 1 -- 172.21.15.82:0/960035223 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00cd80 con 0x7f1208106160 2024-04-21T08:47:17.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120fd11640 1 -- 172.21.15.82:0/960035223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120819ddb0 con 0x7f1208106160 2024-04-21T08:47:17.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f11faffd640 1 -- 172.21.15.82:0/960035223 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11fc0054d0 con 0x7f1208106160 2024-04-21T08:47:17.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f11faffd640 1 -- 172.21.15.82:0/960035223 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc010430 con 0x7f1208106160 2024-04-21T08:47:17.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.204+0000 7f120fd11640 1 -- 172.21.15.82:0/960035223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12081a55f0 con 0x7f1208106160 2024-04-21T08:47:17.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.206+0000 7f11faffd640 1 -- 172.21.15.82:0/960035223 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f11fc004fd0 con 0x7f1208106160 2024-04-21T08:47:17.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f12129d7740 1 Processor -- start 2024-04-21T08:47:17.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f12129d7740 1 -- start start 2024-04-21T08:47:17.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f12129d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f3690 0x5571947f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f12129d7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f4040 0x5571947fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f12129d7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947fd140 0x5571947fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f12129d7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557194757b10 con 0x5571947f4040 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f12129d7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5571946f5240 con 0x5571947fd140 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f12129d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557194753ad0 con 0x5571947f3690 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f11faffd640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f11f40a3e40 0x7f11f40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f120daad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f4040 0x5571947fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f120daad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f4040 0x5571947fcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56552/0 (socket says 172.21.15.82:56552) 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f120daad640 1 -- 172.21.15.82:0/1834297057 learned_addr learned my addr 172.21.15.82:0/1834297057 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f120e2ae640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f11f40a3e40 0x7f11f40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f120e2ae640 1 --2- 172.21.15.82:0/1834297057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f3690 0x5571947f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.207+0000 7f11faffd640 1 -- 172.21.15.82:0/960035223 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f11fc0d1320 con 0x7f1208106160 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.208+0000 7f120d2ac640 1 --2- 172.21.15.82:0/1834297057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947fd140 0x5571947fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.208+0000 7f120daad640 1 -- 172.21.15.82:0/1834297057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f3690 msgr2=0x5571947f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.208+0000 7f120daad640 1 --2- 172.21.15.82:0/1834297057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f3690 0x5571947f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.208+0000 7f120daad640 1 -- 172.21.15.82:0/1834297057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947fd140 msgr2=0x5571947fd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.208+0000 7f120daad640 1 --2- 172.21.15.82:0/1834297057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947fd140 0x5571947fd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.208+0000 7f120daad640 1 -- 172.21.15.82:0/1834297057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55719446a2d0 con 0x5571947f4040 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.208+0000 7f120daad640 1 --2- 172.21.15.82:0/1834297057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f4040 0x5571947fcb70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1204009d40 tx=0x7f12040117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd6d08818cf962c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.208+0000 7f120e2ae640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f11f40a3e40 0x7f11f40a6300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1200006010 tx=0x7f120002e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.208+0000 7f11f3fff640 1 -- 172.21.15.82:0/1834297057 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204012650 con 0x5571947f4040 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.208+0000 7f11f3fff640 1 -- 172.21.15.82:0/1834297057 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1204009e30 con 0x5571947f4040 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.208+0000 7f11f3fff640 1 -- 172.21.15.82:0/1834297057 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120400fbb0 con 0x5571947f4040 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.209+0000 7f12129d7740 1 -- 172.21.15.82:0/1834297057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f4040 msgr2=0x5571947fcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.209+0000 7f12129d7740 1 --2- 172.21.15.82:0/1834297057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f4040 0x5571947fcb70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1204009d40 tx=0x7f12040117b0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.209+0000 7f12129d7740 1 -- 172.21.15.82:0/1834297057 shutdown_connections 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.209+0000 7f12129d7740 1 --2- 172.21.15.82:0/1834297057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947fd140 0x5571947fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.209+0000 7f12129d7740 1 --2- 172.21.15.82:0/1834297057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f4040 0x5571947fcb70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.209+0000 7f12129d7740 1 --2- 172.21.15.82:0/1834297057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f3690 0x5571947f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.209+0000 7f12129d7740 1 -- 172.21.15.82:0/1834297057 >> 172.21.15.82:0/1834297057 conn(0x5571947eda60 msgr2=0x5571947ede40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.209+0000 7f12129d7740 1 -- 172.21.15.82:0/1834297057 shutdown_connections 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.209+0000 7f12129d7740 1 -- 172.21.15.82:0/1834297057 wait complete. 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.209+0000 7f12129d7740 1 -- start start 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.209+0000 7f12129d7740 1 Processor -- start 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f12129d7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947f3690 0x557194800bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f12129d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f4040 0x557194801110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f12129d7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947fd140 0x557194801650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f12129d7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557194804080 con 0x5571947fd140 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f12129d7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5571946e0190 con 0x5571947f3690 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f12129d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557194804200 con 0x5571947f4040 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f12129d7740 10 client.0 _ll_register_callbacks cb 0x5571946db8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f120e2ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947fd140 0x557194801650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f120e2ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947fd140 0x557194801650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56560/0 (socket says 172.21.15.82:56560) 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f120e2ae640 1 -- 172.21.15.82:0/3101455343 learned_addr learned my addr 172.21.15.82:0/3101455343 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f120daad640 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947f3690 0x557194800bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f120d2ac640 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f4040 0x557194801110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f120e2ae640 1 -- 172.21.15.82:0/3101455343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f4040 msgr2=0x557194801110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f120e2ae640 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f4040 0x557194801110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f120e2ae640 1 -- 172.21.15.82:0/3101455343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947f3690 msgr2=0x557194800bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f120e2ae640 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947f3690 0x557194800bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f120e2ae640 1 -- 172.21.15.82:0/3101455343 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571945346b0 con 0x5571947fd140 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f120e2ae640 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947fd140 0x557194801650 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1200009380 tx=0x7f120002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b7c5df5d61fdfd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f11f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f11f17fa640 1 -- 172.21.15.82:0/3101455343 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200049040 con 0x5571947fd140 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f11f17fa640 1 -- 172.21.15.82:0/3101455343 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1200004040 con 0x5571947fd140 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f12129d7740 1 -- 172.21.15.82:0/3101455343 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1204009e30 con 0x5571947fd140 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f11f17fa640 1 -- 172.21.15.82:0/3101455343 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1200004240 con 0x5571947fd140 2024-04-21T08:47:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f12129d7740 10 client.15954 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.210+0000 7f12129d7740 1 -- 172.21.15.82:0/3101455343 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557194810250 con 0x5571947fd140 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.211+0000 7f11f17fa640 1 -- 172.21.15.82:0/3101455343 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1204009e30 con 0x5571947fd140 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.211+0000 7f11f17fa640 1 -- 172.21.15.82:0/3101455343 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f12000349b0 con 0x5571947fd140 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.211+0000 7f12129d7740 10 client.15954 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.211+0000 7f12129d7740 10 client.15954 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.211+0000 7f12129d7740 20 client.15954 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.211+0000 7f12129d7740 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5571948128f0 0x557194813d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.211+0000 7f12129d7740 4 client.15954 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.211+0000 7f12129d7740 1 -- 172.21.15.82:0/3101455343 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557194765530 con 0x5571948128f0 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.211+0000 7f120daad640 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5571948128f0 0x557194813d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.212+0000 7f11f17fa640 10 client.15954 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.212+0000 7f120daad640 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5571948128f0 0x557194813d60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9e4c85daead8594 server_cookie=b011d1c68381f8ac in_seq=0 out_seq=0 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.212+0000 7f11f17fa640 1 -- 172.21.15.82:0/3101455343 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557194765530 con 0x5571948128f0 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.212+0000 7f11f17fa640 10 client.15954 handle_command_reply: tid=0 2024-04-21T08:47:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.212+0000 7f12129d7740 1 client.15954 shutdown 2024-04-21T08:47:17.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.212+0000 7f12129d7740 1 -- 172.21.15.82:0/3101455343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947fd140 msgr2=0x557194801650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.212+0000 7f12129d7740 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947fd140 0x557194801650 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1200009380 tx=0x7f120002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.212+0000 7f12129d7740 1 -- 172.21.15.82:0/3101455343 shutdown_connections 2024-04-21T08:47:17.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.213+0000 7f12129d7740 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5571948128f0 0x557194813d60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.213+0000 7f12129d7740 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947fd140 0x557194801650 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.213+0000 7f12129d7740 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f4040 0x557194801110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.213+0000 7f12129d7740 1 --2- 172.21.15.82:0/3101455343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947f3690 0x557194800bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.213+0000 7f11f17fa640 0 client.15954 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:47:17.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.213+0000 7f12129d7740 1 -- 172.21.15.82:0/3101455343 >> 172.21.15.82:0/3101455343 conn(0x5571947eda60 msgr2=0x5571947efbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.213+0000 7f12129d7740 1 -- 172.21.15.82:0/3101455343 shutdown_connections 2024-04-21T08:47:17.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.213+0000 7f12129d7740 1 -- 172.21.15.82:0/3101455343 wait complete. 2024-04-21T08:47:17.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.213+0000 7f12129d7740 20 client.15954 trim_cache size 0 max 16384 2024-04-21T08:47:17.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f12129d7740 1 Processor -- start 2024-04-21T08:47:17.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f12129d7740 1 -- start start 2024-04-21T08:47:17.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f12129d7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 0x557194803300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f12129d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f4040 0x5571947fe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f12129d7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947fd140 0x5571947ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f12129d7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5571948120a0 con 0x5571947f3690 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f12129d7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557194812770 con 0x5571947fd140 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f12129d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557194812e40 con 0x5571947f4040 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f120daad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 0x557194803300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f120daad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 0x557194803300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56564/0 (socket says 172.21.15.82:56564) 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f120daad640 1 -- 172.21.15.82:0/1452084886 learned_addr learned my addr 172.21.15.82:0/1452084886 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f120e2ae640 1 --2- 172.21.15.82:0/1452084886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947fd140 0x5571947ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f120daad640 1 -- 172.21.15.82:0/1452084886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f4040 msgr2=0x5571947fe530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f120daad640 1 --2- 172.21.15.82:0/1452084886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f4040 0x5571947fe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f120daad640 1 -- 172.21.15.82:0/1452084886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947fd140 msgr2=0x5571947ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f120daad640 1 --2- 172.21.15.82:0/1452084886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947fd140 0x5571947ff4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f120daad640 1 -- 172.21.15.82:0/1452084886 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571947fbd10 con 0x5571947f3690 2024-04-21T08:47:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f120e2ae640 1 --2- 172.21.15.82:0/1452084886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947fd140 0x5571947ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f120daad640 1 --2- 172.21.15.82:0/1452084886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 0x557194803300 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f120400f7e0 tx=0x7f120400f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d61896ca78542c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.231+0000 7f11f37fe640 1 -- 172.21.15.82:0/1452084886 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204009e30 con 0x5571947f3690 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f11f37fe640 1 -- 172.21.15.82:0/1452084886 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f120400fe20 con 0x5571947f3690 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f11f37fe640 1 -- 172.21.15.82:0/1452084886 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204004660 con 0x5571947f3690 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f12129d7740 1 -- 172.21.15.82:0/1452084886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 msgr2=0x557194803300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f12129d7740 1 --2- 172.21.15.82:0/1452084886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 0x557194803300 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f120400f7e0 tx=0x7f120400f770 comp rx=0 tx=0).stop 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f12129d7740 1 -- 172.21.15.82:0/1452084886 shutdown_connections 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f12129d7740 1 --2- 172.21.15.82:0/1452084886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947fd140 0x5571947ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f12129d7740 1 --2- 172.21.15.82:0/1452084886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947f4040 0x5571947fe530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f12129d7740 1 --2- 172.21.15.82:0/1452084886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 0x557194803300 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f12129d7740 1 -- 172.21.15.82:0/1452084886 >> 172.21.15.82:0/1452084886 conn(0x5571947eda60 msgr2=0x5571947e8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f12129d7740 1 -- 172.21.15.82:0/1452084886 shutdown_connections 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f12129d7740 1 -- 172.21.15.82:0/1452084886 wait complete. 2024-04-21T08:47:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f12129d7740 1 -- start start 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.232+0000 7f12129d7740 1 Processor -- start 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f12129d7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 0x5571947fd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f12129d7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947f4040 0x5571947fde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f12129d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947fd140 0x5571947fe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f12129d7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557194813be0 con 0x5571947f3690 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f12129d7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557194813510 con 0x5571947f4040 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f12129d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5571948142b0 con 0x5571947fd140 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f12129d7740 10 client.0 _ll_register_callbacks cb 0x7f1200046040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f120daad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 0x5571947fd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f120daad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 0x5571947fd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56578/0 (socket says 172.21.15.82:56578) 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f120daad640 1 -- 172.21.15.82:0/1255236139 learned_addr learned my addr 172.21.15.82:0/1255236139 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f120daad640 1 -- 172.21.15.82:0/1255236139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947fd140 msgr2=0x5571947fe3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f120daad640 1 --2- 172.21.15.82:0/1255236139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947fd140 0x5571947fe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f120d2ac640 1 --2- 172.21.15.82:0/1255236139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947f4040 0x5571947fde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f120daad640 1 -- 172.21.15.82:0/1255236139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947f4040 msgr2=0x5571947fde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f120daad640 1 --2- 172.21.15.82:0/1255236139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947f4040 0x5571947fde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f120daad640 1 -- 172.21.15.82:0/1255236139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55719481a010 con 0x5571947f3690 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f120daad640 1 --2- 172.21.15.82:0/1255236139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 0x5571947fd950 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1204009d10 tx=0x7f12040095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113fa97c1021c530 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f11f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.233+0000 7f11f17fa640 1 -- 172.21.15.82:0/1255236139 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120403e060 con 0x5571947f3690 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f11f17fa640 1 -- 172.21.15.82:0/1255236139 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1204005620 con 0x5571947f3690 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f12129d7740 1 -- 172.21.15.82:0/1255236139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120400fe20 con 0x5571947f3690 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f11f17fa640 1 -- 172.21.15.82:0/1255236139 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f120400f970 con 0x5571947f3690 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f12129d7740 10 client.15960 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f12129d7740 1 -- 172.21.15.82:0/1255236139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557194878ab0 con 0x5571947f3690 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f11f17fa640 1 -- 172.21.15.82:0/1255236139 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f120400fe20 con 0x5571947f3690 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f11f17fa640 1 -- 172.21.15.82:0/1255236139 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f120402e070 con 0x5571947f3690 2024-04-21T08:47:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f12129d7740 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f12129d7740 10 client.15960 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:47:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f12129d7740 20 client.15960 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f12129d7740 1 --2- 172.21.15.82:0/1255236139 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55719487cf90 0x55719487bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f12129d7740 4 client.15960 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:47:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f12129d7740 1 -- 172.21.15.82:0/1255236139 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "status"}) -- 0x55719475f3a0 con 0x55719487cf90 2024-04-21T08:47:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.234+0000 7f120d2ac640 1 --2- 172.21.15.82:0/1255236139 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55719487cf90 0x55719487bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f120d2ac640 1 --2- 172.21.15.82:0/1255236139 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55719487cf90 0x55719487bca0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30eec59b542466f1 server_cookie=e4da6734e9f56f8e in_seq=0 out_seq=0 2024-04-21T08:47:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f11f17fa640 10 client.15960 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:47:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f11f17fa640 1 -- 172.21.15.82:0/1255236139 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55719475f3a0 con 0x55719487cf90 2024-04-21T08:47:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f11f17fa640 10 client.15960 handle_command_reply: tid=0 2024-04-21T08:47:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f12129d7740 1 client.15960 shutdown 2024-04-21T08:47:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f12129d7740 1 -- 172.21.15.82:0/1255236139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 msgr2=0x5571947fd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f12129d7740 1 --2- 172.21.15.82:0/1255236139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 0x5571947fd950 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1204009d10 tx=0x7f12040095b0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f12129d7740 1 -- 172.21.15.82:0/1255236139 shutdown_connections 2024-04-21T08:47:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f12129d7740 1 --2- 172.21.15.82:0/1255236139 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55719487cf90 0x55719487bca0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f12129d7740 1 --2- 172.21.15.82:0/1255236139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571947fd140 0x5571947fe3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f12129d7740 1 --2- 172.21.15.82:0/1255236139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571947f4040 0x5571947fde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f11f17fa640 0 client.15960 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:47:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f12129d7740 1 --2- 172.21.15.82:0/1255236139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571947f3690 0x5571947fd950 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f12129d7740 1 -- 172.21.15.82:0/1255236139 >> 172.21.15.82:0/1255236139 conn(0x5571947eda60 msgr2=0x5571947ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f12129d7740 1 -- 172.21.15.82:0/1255236139 shutdown_connections 2024-04-21T08:47:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.235+0000 7f12129d7740 1 -- 172.21.15.82:0/1255236139 wait complete. 2024-04-21T08:47:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f12129d7740 20 client.15960 trim_cache size 0 max 16384 2024-04-21T08:47:17.241 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:47:17.242 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:47:17.242 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 4, 2024-04-21T08:47:17.242 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24407, 2024-04-21T08:47:17.242 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:47:17.242 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:47:17.242 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:47:17.242 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 136.23311071200001, 2024-04-21T08:47:17.242 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 28, 2024-04-21T08:47:17.242 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 94, 2024-04-21T08:47:17.242 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 93, 2024-04-21T08:47:17.242 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 197.40826863699999 2024-04-21T08:47:17.242 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:47:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f120fd11640 1 -- 172.21.15.82:0/960035223 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f11f40a3e40 msgr2=0x7f11f40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f120fd11640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f11f40a3e40 0x7f11f40a6300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1200006010 tx=0x7f120002e040 comp rx=0 tx=0).stop 2024-04-21T08:47:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f120fd11640 1 -- 172.21.15.82:0/960035223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1208106160 msgr2=0x7f1208198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f120fd11640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1208106160 0x7f1208198c50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e9c0 tx=0x7f11fc00ee80 comp rx=0 tx=0).stop 2024-04-21T08:47:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f120fd11640 1 -- 172.21.15.82:0/960035223 shutdown_connections 2024-04-21T08:47:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f120fd11640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120819cfd0 0x7f120819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f120fd11640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f11f40a3e40 0x7f11f40a6300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f120fd11640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1208106160 0x7f1208198c50 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f120fd11640 1 --2- 172.21.15.82:0/960035223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1208101090 0x7f1208198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f120fd11640 1 -- 172.21.15.82:0/960035223 >> 172.21.15.82:0/960035223 conn(0x7f1208077f50 msgr2=0x7f12080693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f120fd11640 1 -- 172.21.15.82:0/960035223 shutdown_connections 2024-04-21T08:47:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.236+0000 7f120fd11640 1 -- 172.21.15.82:0/960035223 wait complete. 2024-04-21T08:47:17.246 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-21T08:47:17.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.346+0000 7fcac2985640 1 -- 172.21.15.82:0/876185614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcabc106170 msgr2=0x7fcabc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.346+0000 7fcac2985640 1 --2- 172.21.15.82:0/876185614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcabc106170 0x7fcabc10dd40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009ca0 tx=0x7fcab402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.346+0000 7fcac2985640 1 -- 172.21.15.82:0/876185614 shutdown_connections 2024-04-21T08:47:17.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.346+0000 7fcac2985640 1 --2- 172.21.15.82:0/876185614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcabc106170 0x7fcabc10dd40 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.346+0000 7fcac2985640 1 --2- 172.21.15.82:0/876185614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcabc101a50 0x7fcabc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.346+0000 7fcac2985640 1 --2- 172.21.15.82:0/876185614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc1010a0 0x7fcabc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.346+0000 7fcac2985640 1 -- 172.21.15.82:0/876185614 >> 172.21.15.82:0/876185614 conn(0x7fcabc077eb0 msgr2=0x7fcabc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.346+0000 7fcac2985640 1 -- 172.21.15.82:0/876185614 shutdown_connections 2024-04-21T08:47:17.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.346+0000 7fcac2985640 1 -- 172.21.15.82:0/876185614 wait complete. 2024-04-21T08:47:17.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.347+0000 7fcac2985640 1 Processor -- start 2024-04-21T08:47:17.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.347+0000 7fcac2985640 1 -- start start 2024-04-21T08:47:17.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.347+0000 7fcac2985640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcabc1010a0 0x7fcabc1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.347+0000 7fcac2985640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcabc106170 0x7fcabc197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.347+0000 7fcac2985640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc19bea0 0x7fcabc19c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.347+0000 7fcac2985640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc1110e0 con 0x7fcabc1010a0 2024-04-21T08:47:17.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.347+0000 7fcac2985640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc110f60 con 0x7fcabc19bea0 2024-04-21T08:47:17.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.347+0000 7fcac2985640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc111260 con 0x7fcabc106170 2024-04-21T08:47:17.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.347+0000 7fcabbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcabc1010a0 0x7fcabc1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.347+0000 7fcabbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcabc1010a0 0x7fcabc1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56610/0 (socket says 172.21.15.82:56610) 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.347+0000 7fcabbfff640 1 -- 172.21.15.82:0/1624890966 learned_addr learned my addr 172.21.15.82:0/1624890966 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcac0f22640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc19bea0 0x7fcabc19c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcabbfff640 1 -- 172.21.15.82:0/1624890966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcabc106170 msgr2=0x7fcabc197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcabbfff640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcabc106170 0x7fcabc197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcabbfff640 1 -- 172.21.15.82:0/1624890966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc19bea0 msgr2=0x7fcabc19c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcabbfff640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc19bea0 0x7fcabc19c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcabbfff640 1 -- 172.21.15.82:0/1624890966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc19c990 con 0x7fcabc1010a0 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcac0f22640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc19bea0 0x7fcabc19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcabbfff640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcabc1010a0 0x7fcabc1975e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00e9f0 tx=0x7fcaac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcab97fa640 1 -- 172.21.15.82:0/1624890966 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00cd80 con 0x7fcabc1010a0 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcab97fa640 1 -- 172.21.15.82:0/1624890966 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaac0108e0 con 0x7fcabc1010a0 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcac2985640 1 -- 172.21.15.82:0/1624890966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc19cc80 con 0x7fcabc1010a0 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcab97fa640 1 -- 172.21.15.82:0/1624890966 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac018980 con 0x7fcabc1010a0 2024-04-21T08:47:17.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.348+0000 7fcac2985640 1 -- 172.21.15.82:0/1624890966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1a44c0 con 0x7fcabc1010a0 2024-04-21T08:47:17.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.350+0000 7fcab97fa640 1 -- 172.21.15.82:0/1624890966 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fcaac010430 con 0x7fcabc1010a0 2024-04-21T08:47:17.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.350+0000 7fcac5571740 1 Processor -- start 2024-04-21T08:47:17.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.350+0000 7fcac5571740 1 -- start start 2024-04-21T08:47:17.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcac5571740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 0x55ea582d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcac5571740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 0x55ea582dc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcac5571740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582dcef0 0x55ea582df300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcac5571740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ea58235b10 con 0x55ea582d2000 2024-04-21T08:47:17.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcac5571740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ea581d3240 con 0x55ea582dcef0 2024-04-21T08:47:17.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcac5571740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ea58231ad0 con 0x55ea582d1650 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcab97fa640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcaa40a3ec0 0x7fcaa40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcabbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 0x55ea582dc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcac0f22640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcaa40a3ec0 0x7fcaa40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcabbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 0x55ea582dc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56626/0 (socket says 172.21.15.82:56626) 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcabbfff640 1 -- 172.21.15.82:0/3293923273 learned_addr learned my addr 172.21.15.82:0/3293923273 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcabb7fe640 1 --2- 172.21.15.82:0/3293923273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582dcef0 0x55ea582df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcac0f22640 1 --2- 172.21.15.82:0/3293923273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 0x55ea582d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcab97fa640 1 -- 172.21.15.82:0/1624890966 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fcaac0d12a0 con 0x7fcabc1010a0 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcabbfff640 1 -- 172.21.15.82:0/3293923273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 msgr2=0x55ea582d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcabbfff640 1 --2- 172.21.15.82:0/3293923273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 0x55ea582d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcabbfff640 1 -- 172.21.15.82:0/3293923273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582dcef0 msgr2=0x55ea582df300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcabbfff640 1 --2- 172.21.15.82:0/3293923273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582dcef0 0x55ea582df300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcac0f22640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcaa40a3ec0 0x7fcaa40a6380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcab4004af0 tx=0x7fcab4032d90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcabbfff640 1 -- 172.21.15.82:0/3293923273 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea57f482d0 con 0x55ea582d2000 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcabb7fe640 1 --2- 172.21.15.82:0/3293923273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582dcef0 0x55ea582df300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.351+0000 7fcabbfff640 1 --2- 172.21.15.82:0/3293923273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 0x55ea582dc920 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0097b0 tx=0x7fcaac01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208a2460188c486d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.352+0000 7fcaa27fc640 1 -- 172.21.15.82:0/3293923273 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac087070 con 0x55ea582d2000 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.352+0000 7fcaa27fc640 1 -- 172.21.15.82:0/3293923273 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaac0c7030 con 0x55ea582d2000 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.352+0000 7fcaa27fc640 1 -- 172.21.15.82:0/3293923273 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac0a0fe0 con 0x55ea582d2000 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.352+0000 7fcac5571740 1 -- 172.21.15.82:0/3293923273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 msgr2=0x55ea582dc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.352+0000 7fcac5571740 1 --2- 172.21.15.82:0/3293923273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 0x55ea582dc920 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fcaac0097b0 tx=0x7fcaac01f000 comp rx=0 tx=0).stop 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.352+0000 7fcac5571740 1 -- 172.21.15.82:0/3293923273 shutdown_connections 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.352+0000 7fcac5571740 1 --2- 172.21.15.82:0/3293923273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582dcef0 0x55ea582df300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.352+0000 7fcac5571740 1 --2- 172.21.15.82:0/3293923273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 0x55ea582dc920 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.352+0000 7fcac5571740 1 --2- 172.21.15.82:0/3293923273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 0x55ea582d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.352+0000 7fcac5571740 1 -- 172.21.15.82:0/3293923273 >> 172.21.15.82:0/3293923273 conn(0x55ea582cba60 msgr2=0x55ea582cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.352+0000 7fcac5571740 1 -- 172.21.15.82:0/3293923273 shutdown_connections 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.352+0000 7fcac5571740 1 -- 172.21.15.82:0/3293923273 wait complete. 2024-04-21T08:47:17.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcac5571740 1 -- start start 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcac5571740 1 Processor -- start 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcac5571740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 0x55ea582d9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcac5571740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 0x55ea582da320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcac5571740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582dcef0 0x55ea582da860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcac5571740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ea582dfd60 con 0x55ea582d2000 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcac5571740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ea581be190 con 0x55ea582dcef0 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcac5571740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ea582dfee0 con 0x55ea582d1650 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcac5571740 10 client.0 _ll_register_callbacks cb 0x55ea581b98b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcabb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 0x55ea582da320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcabb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 0x55ea582da320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56628/0 (socket says 172.21.15.82:56628) 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcabb7fe640 1 -- 172.21.15.82:0/4205765351 learned_addr learned my addr 172.21.15.82:0/4205765351 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcabbfff640 1 --2- 172.21.15.82:0/4205765351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 0x55ea582d9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcabb7fe640 1 -- 172.21.15.82:0/4205765351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 msgr2=0x55ea582d9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcabb7fe640 1 --2- 172.21.15.82:0/4205765351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 0x55ea582d9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcabb7fe640 1 -- 172.21.15.82:0/4205765351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582dcef0 msgr2=0x55ea582da860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcabb7fe640 1 --2- 172.21.15.82:0/4205765351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582dcef0 0x55ea582da860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.353+0000 7fcabb7fe640 1 -- 172.21.15.82:0/4205765351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea580126b0 con 0x55ea582d2000 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fcabb7fe640 1 --2- 172.21.15.82:0/4205765351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 0x55ea582da320 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcab00038a0 tx=0x7fcab0003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a91f2cc7d6f8394 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fca9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fca9b7fe640 1 -- 172.21.15.82:0/4205765351 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab001a470 con 0x55ea582d2000 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fca9b7fe640 1 -- 172.21.15.82:0/4205765351 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fcab0011440 con 0x55ea582d2000 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fcac5571740 1 -- 172.21.15.82:0/4205765351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaac0c7030 con 0x55ea582d2000 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fca9b7fe640 1 -- 172.21.15.82:0/4205765351 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcab0011640 con 0x55ea582d2000 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fcac5571740 10 client.15972 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fcac5571740 1 -- 172.21.15.82:0/4205765351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea582ec090 con 0x55ea582d2000 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fca9b7fe640 1 -- 172.21.15.82:0/4205765351 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaac0c7030 con 0x55ea582d2000 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fca9b7fe640 1 -- 172.21.15.82:0/4205765351 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcab001aa50 con 0x55ea582d2000 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fcac5571740 10 client.15972 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fcac5571740 10 client.15972 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fcac5571740 20 client.15972 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fcac5571740 1 --2- 172.21.15.82:0/4205765351 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ea582ee780 0x55ea582efbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fcac5571740 4 client.15972 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.354+0000 7fcac5571740 1 -- 172.21.15.82:0/4205765351 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea58243530 con 0x55ea582ee780 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.355+0000 7fcabbfff640 1 --2- 172.21.15.82:0/4205765351 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ea582ee780 0x55ea582efbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.355+0000 7fca9b7fe640 10 client.15972 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.355+0000 7fcabbfff640 1 --2- 172.21.15.82:0/4205765351 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ea582ee780 0x55ea582efbf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=df774eeba5e309ee server_cookie=9b338f0a3f1c2347 in_seq=0 out_seq=0 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fca9b7fe640 1 -- 172.21.15.82:0/4205765351 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea58243530 con 0x55ea582ee780 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fca9b7fe640 10 client.15972 handle_command_reply: tid=0 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fcac5571740 1 client.15972 shutdown 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fcac5571740 1 -- 172.21.15.82:0/4205765351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 msgr2=0x55ea582da320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fcac5571740 1 --2- 172.21.15.82:0/4205765351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 0x55ea582da320 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcab00038a0 tx=0x7fcab0003d60 comp rx=0 tx=0).stop 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fcac5571740 1 -- 172.21.15.82:0/4205765351 shutdown_connections 2024-04-21T08:47:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fcac5571740 1 --2- 172.21.15.82:0/4205765351 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ea582ee780 0x55ea582efbf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fcac5571740 1 --2- 172.21.15.82:0/4205765351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582dcef0 0x55ea582da860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fca9b7fe640 0 client.15972 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:47:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fcac5571740 1 --2- 172.21.15.82:0/4205765351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d2000 0x55ea582da320 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fcac5571740 1 --2- 172.21.15.82:0/4205765351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 0x55ea582d9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fcac5571740 1 -- 172.21.15.82:0/4205765351 >> 172.21.15.82:0/4205765351 conn(0x55ea582cba60 msgr2=0x55ea582cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fcac5571740 1 -- 172.21.15.82:0/4205765351 shutdown_connections 2024-04-21T08:47:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fcac5571740 1 -- 172.21.15.82:0/4205765351 wait complete. 2024-04-21T08:47:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.356+0000 7fcac5571740 20 client.15972 trim_cache size 0 max 16384 2024-04-21T08:47:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.374+0000 7fcac5571740 1 Processor -- start 2024-04-21T08:47:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.374+0000 7fcac5571740 1 -- start start 2024-04-21T08:47:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.374+0000 7fcac5571740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d1650 0x55ea582dc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.374+0000 7fcac5571740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582d2000 0x55ea582da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.374+0000 7fcac5571740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582dcef0 0x55ea582d86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.374+0000 7fcac5571740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ea582edf30 con 0x55ea582d1650 2024-04-21T08:47:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.374+0000 7fcac5571740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ea582ee600 con 0x55ea582d2000 2024-04-21T08:47:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.374+0000 7fcac5571740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ea582eecd0 con 0x55ea582dcef0 2024-04-21T08:47:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.374+0000 7fcabbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d1650 0x55ea582dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.374+0000 7fcabbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d1650 0x55ea582dc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56642/0 (socket says 172.21.15.82:56642) 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.374+0000 7fcabbfff640 1 -- 172.21.15.82:0/3881369400 learned_addr learned my addr 172.21.15.82:0/3881369400 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcabb7fe640 1 --2- 172.21.15.82:0/3881369400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582d2000 0x55ea582da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcabbfff640 1 -- 172.21.15.82:0/3881369400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582dcef0 msgr2=0x55ea582d86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcabbfff640 1 --2- 172.21.15.82:0/3881369400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582dcef0 0x55ea582d86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcabbfff640 1 -- 172.21.15.82:0/3881369400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582d2000 msgr2=0x55ea582da680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcabbfff640 1 --2- 172.21.15.82:0/3881369400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582d2000 0x55ea582da680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcabbfff640 1 -- 172.21.15.82:0/3881369400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea582dff30 con 0x55ea582d1650 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcabb7fe640 1 --2- 172.21.15.82:0/3881369400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582d2000 0x55ea582da680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcabbfff640 1 --2- 172.21.15.82:0/3881369400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d1650 0x55ea582dc510 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fcaac09cbc0 tx=0x7fcaac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9502f714974d4dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcaa1ffb640 1 -- 172.21.15.82:0/3881369400 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac087040 con 0x55ea582d1650 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcaa1ffb640 1 -- 172.21.15.82:0/3881369400 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaac09d330 con 0x55ea582d1650 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcaa1ffb640 1 -- 172.21.15.82:0/3881369400 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac09af30 con 0x55ea582d1650 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcac5571740 1 -- 172.21.15.82:0/3881369400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d1650 msgr2=0x55ea582dc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.375+0000 7fcac5571740 1 --2- 172.21.15.82:0/3881369400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d1650 0x55ea582dc510 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fcaac09cbc0 tx=0x7fcaac000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 -- 172.21.15.82:0/3881369400 shutdown_connections 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 --2- 172.21.15.82:0/3881369400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582dcef0 0x55ea582d86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 --2- 172.21.15.82:0/3881369400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582d2000 0x55ea582da680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 --2- 172.21.15.82:0/3881369400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582d1650 0x55ea582dc510 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 -- 172.21.15.82:0/3881369400 >> 172.21.15.82:0/3881369400 conn(0x55ea582cba60 msgr2=0x55ea582cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 -- 172.21.15.82:0/3881369400 shutdown_connections 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 -- 172.21.15.82:0/3881369400 wait complete. 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 -- start start 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 Processor -- start 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 0x55ea582f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582d2000 0x55ea582d67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582dcef0 0x55ea582d6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ea582efa70 con 0x55ea582dcef0 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ea582ef3a0 con 0x55ea582d2000 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ea582f0140 con 0x55ea582d1650 2024-04-21T08:47:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac5571740 10 client.0 _ll_register_callbacks cb 0x7fcab0004140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac0f22640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582dcef0 0x55ea582d6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac0f22640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582dcef0 0x55ea582d6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56650/0 (socket says 172.21.15.82:56650) 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.376+0000 7fcac0f22640 1 -- 172.21.15.82:0/1618535210 learned_addr learned my addr 172.21.15.82:0/1618535210 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcac0f22640 1 -- 172.21.15.82:0/1618535210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 msgr2=0x55ea582f81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcabbfff640 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 0x55ea582f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcabb7fe640 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582d2000 0x55ea582d67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcac0f22640 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 0x55ea582f81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcac0f22640 1 -- 172.21.15.82:0/1618535210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582d2000 msgr2=0x55ea582d67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcac0f22640 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582d2000 0x55ea582d67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcac0f22640 1 -- 172.21.15.82:0/1618535210 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea582d7470 con 0x55ea582dcef0 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcabb7fe640 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582d2000 0x55ea582d67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcac0f22640 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582dcef0 0x55ea582d6d30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009ca0 tx=0x7fcab40328e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5f432866e87a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fca9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fca9b7fe640 1 -- 172.21.15.82:0/1618535210 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab404e030 con 0x55ea582dcef0 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fca9b7fe640 1 -- 172.21.15.82:0/1618535210 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fcab4004250 con 0x55ea582dcef0 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcac5571740 1 -- 172.21.15.82:0/1618535210 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaac09d330 con 0x55ea582dcef0 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fca9b7fe640 1 -- 172.21.15.82:0/1618535210 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcab405b050 con 0x55ea582dcef0 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcac5571740 10 client.15978 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcac5571740 1 -- 172.21.15.82:0/1618535210 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea58354730 con 0x55ea582dcef0 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fca9b7fe640 1 -- 172.21.15.82:0/1618535210 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaac09d330 con 0x55ea582dcef0 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fca9b7fe640 1 -- 172.21.15.82:0/1618535210 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcab4049040 con 0x55ea582dcef0 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.377+0000 7fcac5571740 10 client.15978 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.378+0000 7fcac5571740 10 client.15978 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.378+0000 7fcac5571740 20 client.15978 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.378+0000 7fcac5571740 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ea58359220 0x55ea58357f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.378+0000 7fcac5571740 4 client.15978 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.378+0000 7fcac5571740 1 -- 172.21.15.82:0/1618535210 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "status"}) -- 0x55ea5823d3a0 con 0x55ea58359220 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.378+0000 7fcabbfff640 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ea58359220 0x55ea58357f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.378+0000 7fca9b7fe640 10 client.15978 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.378+0000 7fcabbfff640 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ea58359220 0x55ea58357f30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6047a01dfba50c07 server_cookie=6b190b714619d7c3 in_seq=0 out_seq=0 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.378+0000 7fca9b7fe640 1 -- 172.21.15.82:0/1618535210 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ea5823d3a0 con 0x55ea58359220 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.378+0000 7fca9b7fe640 10 client.15978 handle_command_reply: tid=0 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.378+0000 7fcac5571740 1 client.15978 shutdown 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac5571740 1 -- 172.21.15.82:0/1618535210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582dcef0 msgr2=0x55ea582d6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac5571740 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582dcef0 0x55ea582d6d30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009ca0 tx=0x7fcab40328e0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac5571740 1 -- 172.21.15.82:0/1618535210 shutdown_connections 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac5571740 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ea58359220 0x55ea58357f30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac5571740 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea582dcef0 0x55ea582d6d30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac5571740 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ea582d2000 0x55ea582d67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fca9b7fe640 0 client.15978 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac5571740 1 --2- 172.21.15.82:0/1618535210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea582d1650 0x55ea582f81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac5571740 1 -- 172.21.15.82:0/1618535210 >> 172.21.15.82:0/1618535210 conn(0x55ea582cba60 msgr2=0x55ea582d8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac5571740 1 -- 172.21.15.82:0/1618535210 shutdown_connections 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac5571740 1 -- 172.21.15.82:0/1618535210 wait complete. 2024-04-21T08:47:17.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac5571740 20 client.15978 trim_cache size 0 max 16384 2024-04-21T08:47:17.385 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:47:17.385 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:47:17.385 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 3, 2024-04-21T08:47:17.385 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24409, 2024-04-21T08:47:17.385 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:47:17.385 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:47:17.385 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:47:17.385 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 137.38945514599999, 2024-04-21T08:47:17.385 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 28, 2024-04-21T08:47:17.385 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 94, 2024-04-21T08:47:17.385 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 93, 2024-04-21T08:47:17.386 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 200.42663112100001 2024-04-21T08:47:17.386 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:47:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac2985640 1 -- 172.21.15.82:0/1624890966 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcaa40a3ec0 msgr2=0x7fcaa40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac2985640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcaa40a3ec0 0x7fcaa40a6380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcab4004af0 tx=0x7fcab4032d90 comp rx=0 tx=0).stop 2024-04-21T08:47:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac2985640 1 -- 172.21.15.82:0/1624890966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcabc1010a0 msgr2=0x7fcabc1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac2985640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcabc1010a0 0x7fcabc1975e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00e9f0 tx=0x7fcaac00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac2985640 1 -- 172.21.15.82:0/1624890966 shutdown_connections 2024-04-21T08:47:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac2985640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc19bea0 0x7fcabc19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac2985640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcabc106170 0x7fcabc197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac2985640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcaa40a3ec0 0x7fcaa40a6380 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac2985640 1 --2- 172.21.15.82:0/1624890966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcabc1010a0 0x7fcabc1975e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac2985640 1 -- 172.21.15.82:0/1624890966 >> 172.21.15.82:0/1624890966 conn(0x7fcabc077eb0 msgr2=0x7fcabc0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac2985640 1 -- 172.21.15.82:0/1624890966 shutdown_connections 2024-04-21T08:47:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.379+0000 7fcac2985640 1 -- 172.21.15.82:0/1624890966 wait complete. 2024-04-21T08:47:17.389 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-21T08:47:17.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.489+0000 7f67edcba640 1 -- 172.21.15.82:0/775147612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e806a880 msgr2=0x7f67e806ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.489+0000 7f67edcba640 1 --2- 172.21.15.82:0/775147612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e806a880 0x7f67e806ac60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009ca0 tx=0x7f67d802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.489+0000 7f67edcba640 1 -- 172.21.15.82:0/775147612 shutdown_connections 2024-04-21T08:47:17.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.489+0000 7f67edcba640 1 --2- 172.21.15.82:0/775147612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e8107270 0x7f67e810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.489+0000 7f67edcba640 1 --2- 172.21.15.82:0/775147612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e806b230 0x7f67e8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.489+0000 7f67edcba640 1 --2- 172.21.15.82:0/775147612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e806a880 0x7f67e806ac60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.490+0000 7f67edcba640 1 -- 172.21.15.82:0/775147612 >> 172.21.15.82:0/775147612 conn(0x7f67e8077eb0 msgr2=0x7f67e80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.490+0000 7f67edcba640 1 -- 172.21.15.82:0/775147612 shutdown_connections 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.490+0000 7f67edcba640 1 -- 172.21.15.82:0/775147612 wait complete. 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.490+0000 7f67edcba640 1 Processor -- start 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.490+0000 7f67edcba640 1 -- start start 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.490+0000 7f67edcba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e806a880 0x7f67e819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.490+0000 7f67edcba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e8107270 0x7f67e819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.490+0000 7f67edcba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e8198630 0x7f67e8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.490+0000 7f67edcba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f67e81122b0 con 0x7f67e8198630 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.490+0000 7f67edcba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f67e8112130 con 0x7f67e806a880 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.490+0000 7f67edcba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67e8112430 con 0x7f67e8107270 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e8198630 0x7f67e8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e8198630 0x7f67e8198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56666/0 (socket says 172.21.15.82:56666) 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e7fff640 1 -- 172.21.15.82:0/3345602109 learned_addr learned my addr 172.21.15.82:0/3345602109 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e6ffd640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e8107270 0x7f67e819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e77fe640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e806a880 0x7f67e819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e7fff640 1 -- 172.21.15.82:0/3345602109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e8107270 msgr2=0x7f67e819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e7fff640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e8107270 0x7f67e819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e7fff640 1 -- 172.21.15.82:0/3345602109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e806a880 msgr2=0x7f67e819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e7fff640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e806a880 0x7f67e819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e7fff640 1 -- 172.21.15.82:0/3345602109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e8199230 con 0x7f67e8198630 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e77fe640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e806a880 0x7f67e819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e7fff640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e8198630 0x7f67e8198ab0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009fd0 tx=0x7f67e000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e4ff9640 1 -- 172.21.15.82:0/3345602109 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0015020 con 0x7f67e8198630 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e4ff9640 1 -- 172.21.15.82:0/3345602109 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e0011ce0 con 0x7f67e8198630 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67edcba640 1 -- 172.21.15.82:0/3345602109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e81a5310 con 0x7f67e8198630 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67e4ff9640 1 -- 172.21.15.82:0/3345602109 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0010dc0 con 0x7f67e8198630 2024-04-21T08:47:17.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.491+0000 7f67edcba640 1 -- 172.21.15.82:0/3345602109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e81a5710 con 0x7f67e8198630 2024-04-21T08:47:17.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.493+0000 7f67e4ff9640 1 -- 172.21.15.82:0/3345602109 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f67e0018020 con 0x7f67e8198630 2024-04-21T08:47:17.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.493+0000 7f67f0985740 1 Processor -- start 2024-04-21T08:47:17.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67f0985740 1 -- start start 2024-04-21T08:47:17.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67f0985740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 0x559be83adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67f0985740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83ae160 0x559be83b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67f0985740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83b6ce0 0x559be83b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67f0985740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559be8311b10 con 0x559be83ad7b0 2024-04-21T08:47:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67f0985740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559be82af240 con 0x559be83b6ce0 2024-04-21T08:47:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67f0985740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559be830dad0 con 0x559be83ae160 2024-04-21T08:47:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 0x559be83adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 0x559be83adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56668/0 (socket says 172.21.15.82:56668) 2024-04-21T08:47:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67e6ffd640 1 -- 172.21.15.82:0/1414407658 learned_addr learned my addr 172.21.15.82:0/1414407658 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67e6ffd640 1 -- 172.21.15.82:0/1414407658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83ae160 msgr2=0x559be83b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67e6ffd640 1 --2- 172.21.15.82:0/1414407658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83ae160 0x559be83b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67e6ffd640 1 -- 172.21.15.82:0/1414407658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83b6ce0 msgr2=0x559be83b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67e6ffd640 1 --2- 172.21.15.82:0/1414407658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83b6ce0 0x559be83b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67e6ffd640 1 -- 172.21.15.82:0/1414407658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be80242d0 con 0x559be83ad7b0 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67e4ff9640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f67d00a3e40 0x7f67d00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67e6ffd640 1 --2- 172.21.15.82:0/1414407658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 0x559be83adb90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009ab0 tx=0x7f67dc009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13832a09c2706cc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67e77fe640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f67d00a3e40 0x7f67d00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67e4ff9640 1 -- 172.21.15.82:0/3345602109 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f67e00d05f0 con 0x7f67e8198630 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.494+0000 7f67cdffb640 1 -- 172.21.15.82:0/1414407658 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc018020 con 0x559be83ad7b0 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.495+0000 7f67cdffb640 1 -- 172.21.15.82:0/1414407658 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67dc010dc0 con 0x559be83ad7b0 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.495+0000 7f67cdffb640 1 -- 172.21.15.82:0/1414407658 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc0136c0 con 0x559be83ad7b0 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.495+0000 7f67f0985740 1 -- 172.21.15.82:0/1414407658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 msgr2=0x559be83adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.495+0000 7f67e77fe640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f67d00a3e40 0x7f67d00a6300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f67d8000f80 tx=0x7f67d8005ef0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.495+0000 7f67f0985740 1 --2- 172.21.15.82:0/1414407658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 0x559be83adb90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009ab0 tx=0x7f67dc009f70 comp rx=0 tx=0).stop 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.495+0000 7f67f0985740 1 -- 172.21.15.82:0/1414407658 shutdown_connections 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.495+0000 7f67f0985740 1 --2- 172.21.15.82:0/1414407658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83b6ce0 0x559be83b9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.495+0000 7f67f0985740 1 --2- 172.21.15.82:0/1414407658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83ae160 0x559be83b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.495+0000 7f67f0985740 1 --2- 172.21.15.82:0/1414407658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 0x559be83adb90 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.495+0000 7f67f0985740 1 -- 172.21.15.82:0/1414407658 >> 172.21.15.82:0/1414407658 conn(0x559be83a7a60 msgr2=0x559be83a7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.495+0000 7f67f0985740 1 -- 172.21.15.82:0/1414407658 shutdown_connections 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.495+0000 7f67f0985740 1 -- 172.21.15.82:0/1414407658 wait complete. 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67f0985740 1 -- start start 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67f0985740 1 Processor -- start 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67f0985740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 0x559be83c1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67f0985740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ae160 0x559be83c1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67f0985740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83b6ce0 0x559be83bb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67f0985740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559be83b9ca0 con 0x559be83ad7b0 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67f0985740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559be829a190 con 0x559be83ae160 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67f0985740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559be83b9e20 con 0x559be83b6ce0 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67f0985740 10 client.0 _ll_register_callbacks cb 0x559be82958b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67e77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 0x559be83c1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67e77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 0x559be83c1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56684/0 (socket says 172.21.15.82:56684) 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67e77fe640 1 -- 172.21.15.82:0/3445810255 learned_addr learned my addr 172.21.15.82:0/3445810255 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67e7fff640 1 --2- 172.21.15.82:0/3445810255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83b6ce0 0x559be83bb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67e77fe640 1 -- 172.21.15.82:0/3445810255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83b6ce0 msgr2=0x559be83bb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67e77fe640 1 --2- 172.21.15.82:0/3445810255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83b6ce0 0x559be83bb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67e77fe640 1 -- 172.21.15.82:0/3445810255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ae160 msgr2=0x559be83c1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67e77fe640 1 --2- 172.21.15.82:0/3445810255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ae160 0x559be83c1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.496+0000 7f67e77fe640 1 -- 172.21.15.82:0/3445810255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be80ee6b0 con 0x559be83ad7b0 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67e77fe640 1 --2- 172.21.15.82:0/3445810255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 0x559be83c1010 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f67d8005fa0 tx=0x7f67d802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd995e1f89a66c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67c6ffd640 1 -- 172.21.15.82:0/3445810255 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8039040 con 0x559be83ad7b0 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67c6ffd640 1 -- 172.21.15.82:0/3445810255 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f67d803f670 con 0x559be83ad7b0 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67f0985740 1 -- 172.21.15.82:0/3445810255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67dc010dc0 con 0x559be83ad7b0 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67c6ffd640 1 -- 172.21.15.82:0/3445810255 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f67d8058050 con 0x559be83ad7b0 2024-04-21T08:47:17.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67f0985740 10 client.15990 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67f0985740 1 -- 172.21.15.82:0/3445810255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559be83ca290 con 0x559be83ad7b0 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67c6ffd640 1 -- 172.21.15.82:0/3445810255 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67dc010dc0 con 0x559be83ad7b0 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67c6ffd640 1 -- 172.21.15.82:0/3445810255 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f67d8053030 con 0x559be83ad7b0 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67f0985740 10 client.15990 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67f0985740 10 client.15990 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67f0985740 20 client.15990 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67f0985740 1 --2- 172.21.15.82:0/3445810255 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x559be83cc9d0 0x559be83cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67f0985740 4 client.15990 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.497+0000 7f67f0985740 1 -- 172.21.15.82:0/3445810255 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559be831f530 con 0x559be83cc9d0 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.498+0000 7f67e6ffd640 1 --2- 172.21.15.82:0/3445810255 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x559be83cc9d0 0x559be83cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.498+0000 7f67c6ffd640 10 client.15990 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.498+0000 7f67e6ffd640 1 --2- 172.21.15.82:0/3445810255 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x559be83cc9d0 0x559be83cde40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ca9fc6ec5502424 server_cookie=dfa46b09c85084c8 in_seq=0 out_seq=0 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67c6ffd640 1 -- 172.21.15.82:0/3445810255 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559be831f530 con 0x559be83cc9d0 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67c6ffd640 10 client.15990 handle_command_reply: tid=0 2024-04-21T08:47:17.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67f0985740 1 client.15990 shutdown 2024-04-21T08:47:17.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67f0985740 1 -- 172.21.15.82:0/3445810255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 msgr2=0x559be83c1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67f0985740 1 --2- 172.21.15.82:0/3445810255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 0x559be83c1010 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f67d8005fa0 tx=0x7f67d802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67f0985740 1 -- 172.21.15.82:0/3445810255 shutdown_connections 2024-04-21T08:47:17.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67f0985740 1 --2- 172.21.15.82:0/3445810255 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x559be83cc9d0 0x559be83cde40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67f0985740 1 --2- 172.21.15.82:0/3445810255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83b6ce0 0x559be83bb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67f0985740 1 --2- 172.21.15.82:0/3445810255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ae160 0x559be83c1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67c6ffd640 0 client.15990 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:47:17.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67f0985740 1 --2- 172.21.15.82:0/3445810255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ad7b0 0x559be83c1010 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67f0985740 1 -- 172.21.15.82:0/3445810255 >> 172.21.15.82:0/3445810255 conn(0x559be83a7a60 msgr2=0x559be83a9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67f0985740 1 -- 172.21.15.82:0/3445810255 shutdown_connections 2024-04-21T08:47:17.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67f0985740 1 -- 172.21.15.82:0/3445810255 wait complete. 2024-04-21T08:47:17.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.499+0000 7f67f0985740 20 client.15990 trim_cache size 0 max 16384 2024-04-21T08:47:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.517+0000 7f67f0985740 1 Processor -- start 2024-04-21T08:47:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.517+0000 7f67f0985740 1 -- start start 2024-04-21T08:47:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.517+0000 7f67f0985740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83ad7b0 0x559be83b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.517+0000 7f67f0985740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ae160 0x559be83b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.517+0000 7f67f0985740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83b6ce0 0x559be83d6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.517+0000 7f67f0985740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559be83cc180 con 0x559be83b6ce0 2024-04-21T08:47:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.517+0000 7f67f0985740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559be83cc850 con 0x559be83ae160 2024-04-21T08:47:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.517+0000 7f67f0985740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559be83ccf20 con 0x559be83ad7b0 2024-04-21T08:47:17.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.517+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83b6ce0 0x559be83d6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83b6ce0 0x559be83d6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56690/0 (socket says 172.21.15.82:56690) 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67e7fff640 1 -- 172.21.15.82:0/453900320 learned_addr learned my addr 172.21.15.82:0/453900320 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67e6ffd640 1 --2- 172.21.15.82:0/453900320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ae160 0x559be83b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67e7fff640 1 -- 172.21.15.82:0/453900320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83ad7b0 msgr2=0x559be83b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67e7fff640 1 --2- 172.21.15.82:0/453900320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83ad7b0 0x559be83b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67e7fff640 1 -- 172.21.15.82:0/453900320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ae160 msgr2=0x559be83b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67e7fff640 1 --2- 172.21.15.82:0/453900320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ae160 0x559be83b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67e7fff640 1 -- 172.21.15.82:0/453900320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be83b6500 con 0x559be83b6ce0 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67e6ffd640 1 --2- 172.21.15.82:0/453900320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ae160 0x559be83b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67e7fff640 1 --2- 172.21.15.82:0/453900320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83b6ce0 0x559be83d6120 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f67e009c020 tx=0x7f67e0098460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a5bee068977c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67cd7fa640 1 -- 172.21.15.82:0/453900320 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e00a0440 con 0x559be83b6ce0 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67cd7fa640 1 -- 172.21.15.82:0/453900320 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e00a4d00 con 0x559be83b6ce0 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.518+0000 7f67cd7fa640 1 -- 172.21.15.82:0/453900320 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0098700 con 0x559be83b6ce0 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 -- 172.21.15.82:0/453900320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83b6ce0 msgr2=0x559be83d6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 --2- 172.21.15.82:0/453900320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83b6ce0 0x559be83d6120 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f67e009c020 tx=0x7f67e0098460 comp rx=0 tx=0).stop 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 -- 172.21.15.82:0/453900320 shutdown_connections 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 --2- 172.21.15.82:0/453900320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83b6ce0 0x559be83d6120 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 --2- 172.21.15.82:0/453900320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ae160 0x559be83b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 --2- 172.21.15.82:0/453900320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83ad7b0 0x559be83b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 -- 172.21.15.82:0/453900320 >> 172.21.15.82:0/453900320 conn(0x559be83a7a60 msgr2=0x559be83a2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 -- 172.21.15.82:0/453900320 shutdown_connections 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 -- 172.21.15.82:0/453900320 wait complete. 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 -- start start 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 Processor -- start 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ad7b0 0x559be83b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ae160 0x559be83c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83b6ce0 0x559be83c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559be83cdcc0 con 0x559be83ae160 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559be83cd5f0 con 0x559be83ad7b0 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559be83ce390 con 0x559be83b6ce0 2024-04-21T08:47:17.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.519+0000 7f67f0985740 10 client.0 _ll_register_callbacks cb 0x7f67d8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ae160 0x559be83c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ae160 0x559be83c3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56698/0 (socket says 172.21.15.82:56698) 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67e6ffd640 1 -- 172.21.15.82:0/2060626911 learned_addr learned my addr 172.21.15.82:0/2060626911 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67e77fe640 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ad7b0 0x559be83b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67e6ffd640 1 -- 172.21.15.82:0/2060626911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83b6ce0 msgr2=0x559be83c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67e6ffd640 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83b6ce0 0x559be83c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67e6ffd640 1 -- 172.21.15.82:0/2060626911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ad7b0 msgr2=0x559be83b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67e6ffd640 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ad7b0 0x559be83b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67e6ffd640 1 -- 172.21.15.82:0/2060626911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be83c4b50 con 0x559be83ae160 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67e77fe640 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ad7b0 0x559be83b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67e6ffd640 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ae160 0x559be83c3d40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009ab0 tx=0x7f67dc014e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a7e247e95391f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67c6ffd640 1 -- 172.21.15.82:0/2060626911 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc018040 con 0x559be83ae160 2024-04-21T08:47:17.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67c6ffd640 1 -- 172.21.15.82:0/2060626911 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f67dc028dd0 con 0x559be83ae160 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67f0985740 1 -- 172.21.15.82:0/2060626911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e00a4d00 con 0x559be83ae160 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67c6ffd640 1 -- 172.21.15.82:0/2060626911 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f67dc009460 con 0x559be83ae160 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67f0985740 10 client.15996 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67f0985740 1 -- 172.21.15.82:0/2060626911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559be84329f0 con 0x559be83ae160 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.520+0000 7f67c6ffd640 1 -- 172.21.15.82:0/2060626911 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e00a4d00 con 0x559be83ae160 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.521+0000 7f67c6ffd640 1 -- 172.21.15.82:0/2060626911 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f67dc014560 con 0x559be83ae160 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.521+0000 7f67f0985740 10 client.15996 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.521+0000 7f67f0985740 10 client.15996 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.521+0000 7f67f0985740 20 client.15996 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.521+0000 7f67f0985740 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x559be84371f0 0x559be8435f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.521+0000 7f67f0985740 4 client.15996 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.521+0000 7f67f0985740 1 -- 172.21.15.82:0/2060626911 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "status"}) -- 0x559be83193a0 con 0x559be84371f0 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.521+0000 7f67e77fe640 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x559be84371f0 0x559be8435f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.521+0000 7f67c6ffd640 10 client.15996 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:47:17.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.521+0000 7f67e77fe640 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x559be84371f0 0x559be8435f00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac7a742d84db9bc3 server_cookie=5b020f11dc4ff527 in_seq=0 out_seq=0 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67c6ffd640 1 -- 172.21.15.82:0/2060626911 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559be83193a0 con 0x559be84371f0 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67c6ffd640 10 client.15996 handle_command_reply: tid=0 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67f0985740 1 client.15996 shutdown 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67f0985740 1 -- 172.21.15.82:0/2060626911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ae160 msgr2=0x559be83c3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67f0985740 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ae160 0x559be83c3d40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009ab0 tx=0x7f67dc014e80 comp rx=0 tx=0).stop 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67f0985740 1 -- 172.21.15.82:0/2060626911 shutdown_connections 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67f0985740 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x559be84371f0 0x559be8435f00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67f0985740 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559be83b6ce0 0x559be83c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67f0985740 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559be83ae160 0x559be83c3d40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67c6ffd640 0 client.15996 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67f0985740 1 --2- 172.21.15.82:0/2060626911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559be83ad7b0 0x559be83b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67f0985740 1 -- 172.21.15.82:0/2060626911 >> 172.21.15.82:0/2060626911 conn(0x559be83a7a60 msgr2=0x559be83b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67f0985740 1 -- 172.21.15.82:0/2060626911 shutdown_connections 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67f0985740 1 -- 172.21.15.82:0/2060626911 wait complete. 2024-04-21T08:47:17.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67f0985740 20 client.15996 trim_cache size 0 max 16384 2024-04-21T08:47:17.528 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:47:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:47:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 2, 2024-04-21T08:47:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24476, 2024-04-21T08:47:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:47:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:47:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:47:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 138.07053965200001, 2024-04-21T08:47:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 28, 2024-04-21T08:47:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 94, 2024-04-21T08:47:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 93, 2024-04-21T08:47:17.529 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 180.39749192299999 2024-04-21T08:47:17.529 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:47:17.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67edcba640 1 -- 172.21.15.82:0/3345602109 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f67d00a3e40 msgr2=0x7f67d00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67edcba640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f67d00a3e40 0x7f67d00a6300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f67d8000f80 tx=0x7f67d8005ef0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67edcba640 1 -- 172.21.15.82:0/3345602109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e8198630 msgr2=0x7f67e8198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67edcba640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e8198630 0x7f67e8198ab0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009fd0 tx=0x7f67e000b820 comp rx=0 tx=0).stop 2024-04-21T08:47:17.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67edcba640 1 -- 172.21.15.82:0/3345602109 shutdown_connections 2024-04-21T08:47:17.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67edcba640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f67d00a3e40 0x7f67d00a6300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67edcba640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e8198630 0x7f67e8198ab0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67edcba640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e8107270 0x7f67e819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67edcba640 1 --2- 172.21.15.82:0/3345602109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67e806a880 0x7f67e819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.522+0000 7f67edcba640 1 -- 172.21.15.82:0/3345602109 >> 172.21.15.82:0/3345602109 conn(0x7f67e8077eb0 msgr2=0x7f67e81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.523+0000 7f67edcba640 1 -- 172.21.15.82:0/3345602109 shutdown_connections 2024-04-21T08:47:17.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.523+0000 7f67edcba640 1 -- 172.21.15.82:0/3345602109 wait complete. 2024-04-21T08:47:17.533 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T08:47:17.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.632+0000 7f338c84a640 1 -- 172.21.15.82:0/319531996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3388101090 msgr2=0x7f3388101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.632+0000 7f338c84a640 1 --2- 172.21.15.82:0/319531996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3388101090 0x7f3388101470 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3378009ca0 tx=0x7f337802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.633+0000 7f338c84a640 1 -- 172.21.15.82:0/319531996 shutdown_connections 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.633+0000 7f338c84a640 1 --2- 172.21.15.82:0/319531996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3388106160 0x7f338810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.633+0000 7f338c84a640 1 --2- 172.21.15.82:0/319531996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3388101a40 0x7f3388105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.633+0000 7f338c84a640 1 --2- 172.21.15.82:0/319531996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3388101090 0x7f3388101470 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.633+0000 7f338c84a640 1 -- 172.21.15.82:0/319531996 >> 172.21.15.82:0/319531996 conn(0x7f3388077f50 msgr2=0x7f3388078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.633+0000 7f338c84a640 1 -- 172.21.15.82:0/319531996 shutdown_connections 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.633+0000 7f338c84a640 1 -- 172.21.15.82:0/319531996 wait complete. 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.633+0000 7f338c84a640 1 Processor -- start 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.633+0000 7f338c84a640 1 -- start start 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f338c84a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3388101090 0x7f3388197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f338c84a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3388106160 0x7f3388197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f338c84a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f338819bf50 0x7f338819c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f338c84a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f33881110d0 con 0x7f338819bf50 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f338c84a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3388110f50 con 0x7f3388101090 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f338c84a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3388111250 con 0x7f3388106160 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f3386d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f338819bf50 0x7f338819c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f3386d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f338819bf50 0x7f338819c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56708/0 (socket says 172.21.15.82:56708) 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f3386d9d640 1 -- 172.21.15.82:0/2648166993 learned_addr learned my addr 172.21.15.82:0/2648166993 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f338659c640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3388101090 0x7f3388197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f3385d9b640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3388106160 0x7f3388197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f3386d9d640 1 -- 172.21.15.82:0/2648166993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3388106160 msgr2=0x7f3388197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f3386d9d640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3388106160 0x7f3388197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f3386d9d640 1 -- 172.21.15.82:0/2648166993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3388101090 msgr2=0x7f3388197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f3386d9d640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3388101090 0x7f3388197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f3386d9d640 1 -- 172.21.15.82:0/2648166993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338819ca40 con 0x7f338819bf50 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f3386d9d640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f338819bf50 0x7f338819c400 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3380009860 tx=0x7f3380009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f337f7fe640 1 -- 172.21.15.82:0/2648166993 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380005020 con 0x7f338819bf50 2024-04-21T08:47:17.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f337f7fe640 1 -- 172.21.15.82:0/2648166993 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33800047a0 con 0x7f338819bf50 2024-04-21T08:47:17.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f338c84a640 1 -- 172.21.15.82:0/2648166993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338819cd30 con 0x7f338819bf50 2024-04-21T08:47:17.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f337f7fe640 1 -- 172.21.15.82:0/2648166993 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380007a80 con 0x7f338819bf50 2024-04-21T08:47:17.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.634+0000 7f338c84a640 1 -- 172.21.15.82:0/2648166993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3388069400 con 0x7f338819bf50 2024-04-21T08:47:17.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.637+0000 7f337f7fe640 1 -- 172.21.15.82:0/2648166993 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3380004950 con 0x7f338819bf50 2024-04-21T08:47:17.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.637+0000 7f338f43f740 1 Processor -- start 2024-04-21T08:47:17.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.637+0000 7f338f43f740 1 -- start start 2024-04-21T08:47:17.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.637+0000 7f338f43f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559318917b0 0x555931891b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.637+0000 7f338f43f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x55593189a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.637+0000 7f338f43f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 0x55593189d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.637+0000 7f338f43f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5559317f5b10 con 0x55593189ace0 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.637+0000 7f338f43f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555931793240 con 0x555931892160 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.637+0000 7f338f43f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5559317f1ad0 con 0x5559318917b0 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.637+0000 7f337f7fe640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f33740a3e40 0x7f33740a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f3385d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 0x55593189d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f3385d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 0x55593189d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56720/0 (socket says 172.21.15.82:56720) 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f3385d9b640 1 -- 172.21.15.82:0/2732002090 learned_addr learned my addr 172.21.15.82:0/2732002090 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f3385d9b640 1 --2- 172.21.15.82:0/2732002090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559318917b0 0x555931891b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f3386d9d640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f33740a3e40 0x7f33740a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f338659c640 1 --2- 172.21.15.82:0/2732002090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x55593189a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f337f7fe640 1 -- 172.21.15.82:0/2648166993 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f33800d2170 con 0x7f338819bf50 2024-04-21T08:47:17.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f3385d9b640 1 -- 172.21.15.82:0/2732002090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559318917b0 msgr2=0x555931891b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f3385d9b640 1 --2- 172.21.15.82:0/2732002090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559318917b0 0x555931891b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f3385d9b640 1 -- 172.21.15.82:0/2732002090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 msgr2=0x55593189a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f3385d9b640 1 --2- 172.21.15.82:0/2732002090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x55593189a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f3385d9b640 1 -- 172.21.15.82:0/2732002090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559315082d0 con 0x55593189ace0 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f338659c640 1 --2- 172.21.15.82:0/2732002090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x55593189a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f3385d9b640 1 --2- 172.21.15.82:0/2732002090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 0x55593189d130 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3370012770 tx=0x7f3370004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3b374a2a6c9fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f3386d9d640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f33740a3e40 0x7f33740a6300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f33800cf3e0 tx=0x7f33800be040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f337cff9640 1 -- 172.21.15.82:0/2732002090 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370012cf0 con 0x55593189ace0 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f337cff9640 1 -- 172.21.15.82:0/2732002090 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3370010770 con 0x55593189ace0 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.638+0000 7f337cff9640 1 -- 172.21.15.82:0/2732002090 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700044e0 con 0x55593189ace0 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.639+0000 7f338f43f740 1 -- 172.21.15.82:0/2732002090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 msgr2=0x55593189d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.639+0000 7f338f43f740 1 --2- 172.21.15.82:0/2732002090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 0x55593189d130 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3370012770 tx=0x7f3370004730 comp rx=0 tx=0).stop 2024-04-21T08:47:17.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.639+0000 7f338f43f740 1 -- 172.21.15.82:0/2732002090 shutdown_connections 2024-04-21T08:47:17.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.639+0000 7f338f43f740 1 --2- 172.21.15.82:0/2732002090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 0x55593189d130 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.639+0000 7f338f43f740 1 --2- 172.21.15.82:0/2732002090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x55593189a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.639+0000 7f338f43f740 1 --2- 172.21.15.82:0/2732002090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559318917b0 0x555931891b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.639+0000 7f338f43f740 1 -- 172.21.15.82:0/2732002090 >> 172.21.15.82:0/2732002090 conn(0x55593188ba60 msgr2=0x55593188be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.639+0000 7f338f43f740 1 -- 172.21.15.82:0/2732002090 shutdown_connections 2024-04-21T08:47:17.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.639+0000 7f338f43f740 1 -- 172.21.15.82:0/2732002090 wait complete. 2024-04-21T08:47:17.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.639+0000 7f338f43f740 1 -- start start 2024-04-21T08:47:17.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.639+0000 7f338f43f740 1 Processor -- start 2024-04-21T08:47:17.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f338f43f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 0x555931897cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f338f43f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x555931898210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f338f43f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55593189ace0 0x555931898750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f338f43f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55593189dc50 con 0x5559318917b0 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f338f43f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55593177e190 con 0x555931892160 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f338f43f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55593189ddd0 con 0x55593189ace0 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f338f43f740 10 client.0 _ll_register_callbacks cb 0x5559317798b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f3385d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 0x555931897cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f3385d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 0x555931897cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56724/0 (socket says 172.21.15.82:56724) 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f3385d9b640 1 -- 172.21.15.82:0/4214444289 learned_addr learned my addr 172.21.15.82:0/4214444289 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f338659c640 1 --2- 172.21.15.82:0/4214444289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x555931898210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f3385d9b640 1 -- 172.21.15.82:0/4214444289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55593189ace0 msgr2=0x555931898750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f3385d9b640 1 --2- 172.21.15.82:0/4214444289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55593189ace0 0x555931898750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f3385d9b640 1 -- 172.21.15.82:0/4214444289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 msgr2=0x555931898210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f3385d9b640 1 --2- 172.21.15.82:0/4214444289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x555931898210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f3385d9b640 1 -- 172.21.15.82:0/4214444289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559315d26b0 con 0x5559318917b0 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f3385d9b640 1 --2- 172.21.15.82:0/4214444289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 0x555931897cd0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f337000b7e0 tx=0x7f3370005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d639f2bb94048fd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f336dffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f336dffb640 1 -- 172.21.15.82:0/4214444289 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370010b20 con 0x5559318917b0 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f336dffb640 1 -- 172.21.15.82:0/4214444289 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f3370010cc0 con 0x5559318917b0 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.640+0000 7f338f43f740 1 -- 172.21.15.82:0/4214444289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3370010770 con 0x5559318917b0 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.641+0000 7f336dffb640 1 -- 172.21.15.82:0/4214444289 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33700035f0 con 0x5559318917b0 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.641+0000 7f338f43f740 10 client.16008 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.641+0000 7f338f43f740 1 -- 172.21.15.82:0/4214444289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5559318a9f40 con 0x5559318917b0 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.641+0000 7f336dffb640 1 -- 172.21.15.82:0/4214444289 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3370010770 con 0x5559318917b0 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.641+0000 7f336dffb640 1 -- 172.21.15.82:0/4214444289 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f337001f620 con 0x5559318917b0 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.641+0000 7f338f43f740 10 client.16008 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.641+0000 7f338f43f740 10 client.16008 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.641+0000 7f338f43f740 20 client.16008 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.641+0000 7f338f43f740 1 --2- 172.21.15.82:0/4214444289 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5559318ac240 0x5559318ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.641+0000 7f338f43f740 4 client.16008 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.641+0000 7f338f43f740 1 -- 172.21.15.82:0/4214444289 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555931803530 con 0x5559318ac240 2024-04-21T08:47:17.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.641+0000 7f338659c640 1 --2- 172.21.15.82:0/4214444289 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5559318ac240 0x5559318ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f336dffb640 10 client.16008 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f338659c640 1 --2- 172.21.15.82:0/4214444289 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5559318ac240 0x5559318ad6b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e843d091d76f935 server_cookie=61db82fa1d7b76e3 in_seq=0 out_seq=0 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f336dffb640 1 -- 172.21.15.82:0/4214444289 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555931803530 con 0x5559318ac240 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f336dffb640 10 client.16008 handle_command_reply: tid=0 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f338f43f740 1 client.16008 shutdown 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f338f43f740 1 -- 172.21.15.82:0/4214444289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 msgr2=0x555931897cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f338f43f740 1 --2- 172.21.15.82:0/4214444289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 0x555931897cd0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f337000b7e0 tx=0x7f3370005c90 comp rx=0 tx=0).stop 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f338f43f740 1 -- 172.21.15.82:0/4214444289 shutdown_connections 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f338f43f740 1 --2- 172.21.15.82:0/4214444289 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5559318ac240 0x5559318ad6b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f338f43f740 1 --2- 172.21.15.82:0/4214444289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55593189ace0 0x555931898750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f338f43f740 1 --2- 172.21.15.82:0/4214444289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x555931898210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f336dffb640 0 client.16008 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.642+0000 7f338f43f740 1 --2- 172.21.15.82:0/4214444289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 0x555931897cd0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.643+0000 7f338f43f740 1 -- 172.21.15.82:0/4214444289 >> 172.21.15.82:0/4214444289 conn(0x55593188ba60 msgr2=0x55593188dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.643+0000 7f338f43f740 1 -- 172.21.15.82:0/4214444289 shutdown_connections 2024-04-21T08:47:17.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.643+0000 7f338f43f740 1 -- 172.21.15.82:0/4214444289 wait complete. 2024-04-21T08:47:17.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.643+0000 7f338f43f740 20 client.16008 trim_cache size 0 max 16384 2024-04-21T08:47:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.660+0000 7f338f43f740 1 Processor -- start 2024-04-21T08:47:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f338f43f740 1 -- start start 2024-04-21T08:47:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f338f43f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 0x55593189a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f338f43f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x5559318abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f338f43f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55593189ace0 0x5559318b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f338f43f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5559318ab9f0 con 0x5559318917b0 2024-04-21T08:47:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f338f43f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5559318ac0c0 con 0x555931892160 2024-04-21T08:47:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f338f43f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5559318ac790 con 0x55593189ace0 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f3385d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 0x55593189a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f3385d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 0x55593189a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56734/0 (socket says 172.21.15.82:56734) 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f3385d9b640 1 -- 172.21.15.82:0/1280000684 learned_addr learned my addr 172.21.15.82:0/1280000684 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f3385d9b640 1 -- 172.21.15.82:0/1280000684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55593189ace0 msgr2=0x5559318b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f3385d9b640 1 --2- 172.21.15.82:0/1280000684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55593189ace0 0x5559318b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f3385d9b640 1 -- 172.21.15.82:0/1280000684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 msgr2=0x5559318abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f3385d9b640 1 --2- 172.21.15.82:0/1280000684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x5559318abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f3385d9b640 1 -- 172.21.15.82:0/1280000684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555931891b90 con 0x5559318917b0 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f3385d9b640 1 --2- 172.21.15.82:0/1280000684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 0x55593189a400 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f337000b620 tx=0x7f337000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdfc18b76bc2d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f336ffff640 1 -- 172.21.15.82:0/1280000684 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700120b0 con 0x5559318917b0 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f336ffff640 1 -- 172.21.15.82:0/1280000684 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3370016040 con 0x5559318917b0 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.661+0000 7f336ffff640 1 -- 172.21.15.82:0/1280000684 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370003c50 con 0x5559318917b0 2024-04-21T08:47:17.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.662+0000 7f338f43f740 1 -- 172.21.15.82:0/1280000684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 msgr2=0x55593189a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.662+0000 7f338f43f740 1 --2- 172.21.15.82:0/1280000684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 0x55593189a400 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f337000b620 tx=0x7f337000b690 comp rx=0 tx=0).stop 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.662+0000 7f338f43f740 1 -- 172.21.15.82:0/1280000684 shutdown_connections 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.662+0000 7f338f43f740 1 --2- 172.21.15.82:0/1280000684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55593189ace0 0x5559318b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.662+0000 7f338f43f740 1 --2- 172.21.15.82:0/1280000684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x5559318abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.662+0000 7f338f43f740 1 --2- 172.21.15.82:0/1280000684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559318917b0 0x55593189a400 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.662+0000 7f338f43f740 1 -- 172.21.15.82:0/1280000684 >> 172.21.15.82:0/1280000684 conn(0x55593188ba60 msgr2=0x555931886800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.662+0000 7f338f43f740 1 -- 172.21.15.82:0/1280000684 shutdown_connections 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.662+0000 7f338f43f740 1 -- 172.21.15.82:0/1280000684 wait complete. 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.662+0000 7f338f43f740 1 -- start start 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.662+0000 7f338f43f740 1 Processor -- start 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f338f43f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559318917b0 0x5559318b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f338f43f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x5559318b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f338f43f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 0x5559318b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f338f43f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5559318ad530 con 0x55593189ace0 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f338f43f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5559318ace60 con 0x555931892160 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f338f43f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5559318adc00 con 0x5559318917b0 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f338f43f740 10 client.0 _ll_register_callbacks cb 0x7f337000be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f3385d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 0x5559318b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f3385d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 0x5559318b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56736/0 (socket says 172.21.15.82:56736) 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f3385d9b640 1 -- 172.21.15.82:0/1476673522 learned_addr learned my addr 172.21.15.82:0/1476673522 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f3385d9b640 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559318917b0 0x5559318b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f338659c640 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x5559318b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f3385d9b640 1 -- 172.21.15.82:0/1476673522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559318917b0 msgr2=0x5559318b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f3385d9b640 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559318917b0 0x5559318b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f3385d9b640 1 -- 172.21.15.82:0/1476673522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 msgr2=0x5559318b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f3385d9b640 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x5559318b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f3385d9b640 1 -- 172.21.15.82:0/1476673522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559318b4d80 con 0x55593189ace0 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f3385d9b640 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 0x5559318b4580 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f337000c010 tx=0x7f3370014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a46673c9f2f71d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f336dffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f336dffb640 1 -- 172.21.15.82:0/1476673522 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700038d0 con 0x55593189ace0 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f336dffb640 1 -- 172.21.15.82:0/1476673522 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f3370003a70 con 0x55593189ace0 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f338f43f740 1 -- 172.21.15.82:0/1476673522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3370016040 con 0x55593189ace0 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f336dffb640 1 -- 172.21.15.82:0/1476673522 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f337001f460 con 0x55593189ace0 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f338f43f740 10 client.16014 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.663+0000 7f338f43f740 1 -- 172.21.15.82:0/1476673522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555931917200 con 0x55593189ace0 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.664+0000 7f336dffb640 1 -- 172.21.15.82:0/1476673522 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3370016040 con 0x55593189ace0 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.664+0000 7f336dffb640 1 -- 172.21.15.82:0/1476673522 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3370027540 con 0x55593189ace0 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.664+0000 7f338f43f740 10 client.16014 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.664+0000 7f338f43f740 10 client.16014 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.664+0000 7f338f43f740 20 client.16014 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.664+0000 7f338f43f740 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55593191bb50 0x55593191a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.664+0000 7f338f43f740 4 client.16014 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.664+0000 7f338f43f740 1 -- 172.21.15.82:0/1476673522 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "status"}) -- 0x5559317fd3a0 con 0x55593191bb50 2024-04-21T08:47:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.664+0000 7f338659c640 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55593191bb50 0x55593191a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.664+0000 7f338659c640 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55593191bb50 0x55593191a860 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbc35469baab726e server_cookie=cb843c39ff0e7803 in_seq=0 out_seq=0 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.664+0000 7f336dffb640 10 client.16014 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f336dffb640 1 -- 172.21.15.82:0/1476673522 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5559317fd3a0 con 0x55593191bb50 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f336dffb640 10 client.16014 handle_command_reply: tid=0 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338f43f740 1 client.16014 shutdown 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338f43f740 1 -- 172.21.15.82:0/1476673522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 msgr2=0x5559318b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338f43f740 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 0x5559318b4580 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f337000c010 tx=0x7f3370014040 comp rx=0 tx=0).stop 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338f43f740 1 -- 172.21.15.82:0/1476673522 shutdown_connections 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338f43f740 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55593191bb50 0x55593191a860 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338f43f740 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55593189ace0 0x5559318b4580 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f336dffb640 0 client.16014 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338f43f740 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555931892160 0x5559318b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338f43f740 1 --2- 172.21.15.82:0/1476673522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559318917b0 0x5559318b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338f43f740 1 -- 172.21.15.82:0/1476673522 >> 172.21.15.82:0/1476673522 conn(0x55593188ba60 msgr2=0x55593188a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338f43f740 1 -- 172.21.15.82:0/1476673522 shutdown_connections 2024-04-21T08:47:17.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338f43f740 1 -- 172.21.15.82:0/1476673522 wait complete. 2024-04-21T08:47:17.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338f43f740 20 client.16014 trim_cache size 0 max 16384 2024-04-21T08:47:17.671 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:47:17.671 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:47:17.671 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 1, 2024-04-21T08:47:17.671 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24481, 2024-04-21T08:47:17.671 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:47:17.671 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:47:17.671 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:47:17.671 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 139.22174086300001, 2024-04-21T08:47:17.672 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 28, 2024-04-21T08:47:17.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 94, 2024-04-21T08:47:17.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 93, 2024-04-21T08:47:17.672 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 177.756405255 2024-04-21T08:47:17.672 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:47:17.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338c84a640 1 -- 172.21.15.82:0/2648166993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f33740a3e40 msgr2=0x7f33740a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338c84a640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f33740a3e40 0x7f33740a6300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f33800cf3e0 tx=0x7f33800be040 comp rx=0 tx=0).stop 2024-04-21T08:47:17.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338c84a640 1 -- 172.21.15.82:0/2648166993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f338819bf50 msgr2=0x7f338819c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338c84a640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f338819bf50 0x7f338819c400 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3380009860 tx=0x7f3380009d20 comp rx=0 tx=0).stop 2024-04-21T08:47:17.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338c84a640 1 -- 172.21.15.82:0/2648166993 shutdown_connections 2024-04-21T08:47:17.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338c84a640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f33740a3e40 0x7f33740a6300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338c84a640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f338819bf50 0x7f338819c400 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338c84a640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3388106160 0x7f3388197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338c84a640 1 --2- 172.21.15.82:0/2648166993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3388101090 0x7f3388197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338c84a640 1 -- 172.21.15.82:0/2648166993 >> 172.21.15.82:0/2648166993 conn(0x7f3388077f50 msgr2=0x7f33880ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338c84a640 1 -- 172.21.15.82:0/2648166993 shutdown_connections 2024-04-21T08:47:17.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.665+0000 7f338c84a640 1 -- 172.21.15.82:0/2648166993 wait complete. 2024-04-21T08:47:17.675 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T08:47:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.774+0000 7f3783b23640 1 -- 172.21.15.82:0/3631291266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c107270 msgr2=0x7f377c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.774+0000 7f3783b23640 1 --2- 172.21.15.82:0/3631291266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c107270 0x7f377c10ee40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3774009ca0 tx=0x7f377402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.774+0000 7f3783b23640 1 -- 172.21.15.82:0/3631291266 shutdown_connections 2024-04-21T08:47:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.774+0000 7f3783b23640 1 --2- 172.21.15.82:0/3631291266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c107270 0x7f377c10ee40 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.774+0000 7f3783b23640 1 --2- 172.21.15.82:0/3631291266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c102b50 0x7f377c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.774+0000 7f3783b23640 1 --2- 172.21.15.82:0/3631291266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f377c1021a0 0x7f377c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.774+0000 7f3783b23640 1 -- 172.21.15.82:0/3631291266 >> 172.21.15.82:0/3631291266 conn(0x7f377c0fd9c0 msgr2=0x7f377c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.774+0000 7f3783b23640 1 -- 172.21.15.82:0/3631291266 shutdown_connections 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.774+0000 7f3783b23640 1 -- 172.21.15.82:0/3631291266 wait complete. 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.775+0000 7f3783b23640 1 Processor -- start 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.775+0000 7f3783b23640 1 -- start start 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.775+0000 7f3783b23640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c1021a0 0x7f377c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.775+0000 7f3783b23640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c107270 0x7f377c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.775+0000 7f3783b23640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f377c19d010 0x7f377c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.775+0000 7f3783b23640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1121c0 con 0x7f377c1021a0 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.775+0000 7f3783b23640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f377c112040 con 0x7f377c107270 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.775+0000 7f3783b23640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f377c112340 con 0x7f377c19d010 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.775+0000 7f37818bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c1021a0 0x7f377c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.775+0000 7f37818bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c1021a0 0x7f377c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56768/0 (socket says 172.21.15.82:56768) 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.775+0000 7f37818bf640 1 -- 172.21.15.82:0/1590647263 learned_addr learned my addr 172.21.15.82:0/1590647263 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.775+0000 7f37820c0640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f377c19d010 0x7f377c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.776+0000 7f37810be640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c107270 0x7f377c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.776+0000 7f37818bf640 1 -- 172.21.15.82:0/1590647263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f377c19d010 msgr2=0x7f377c19d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.776+0000 7f37818bf640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f377c19d010 0x7f377c19d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.776+0000 7f37818bf640 1 -- 172.21.15.82:0/1590647263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c107270 msgr2=0x7f377c198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.776+0000 7f37818bf640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c107270 0x7f377c198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.776+0000 7f37818bf640 1 -- 172.21.15.82:0/1590647263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c19db00 con 0x7f377c1021a0 2024-04-21T08:47:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.776+0000 7f37818bf640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c1021a0 0x7f377c1987a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f377800d950 tx=0x7f377800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.776+0000 7f3772ffd640 1 -- 172.21.15.82:0/1590647263 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778017030 con 0x7f377c1021a0 2024-04-21T08:47:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.776+0000 7f3772ffd640 1 -- 172.21.15.82:0/1590647263 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3778004020 con 0x7f377c1021a0 2024-04-21T08:47:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.776+0000 7f3783b23640 1 -- 172.21.15.82:0/1590647263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377c19ddf0 con 0x7f377c1021a0 2024-04-21T08:47:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.776+0000 7f3772ffd640 1 -- 172.21.15.82:0/1590647263 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780047a0 con 0x7f377c1021a0 2024-04-21T08:47:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.776+0000 7f3783b23640 1 -- 172.21.15.82:0/1590647263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377c1a5630 con 0x7f377c1021a0 2024-04-21T08:47:17.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.778+0000 7f3772ffd640 1 -- 172.21.15.82:0/1590647263 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3778009a30 con 0x7f377c1021a0 2024-04-21T08:47:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37867ee740 1 Processor -- start 2024-04-21T08:47:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37867ee740 1 -- start start 2024-04-21T08:47:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37867ee740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc00fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37867ee740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc09b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37867ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc0c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f3772ffd640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f37680a3ec0 0x7f37680a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37867ee740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5589bbb60b10 con 0x5589bbc01570 2024-04-21T08:47:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37867ee740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5589bbafe240 con 0x5589bbc00bc0 2024-04-21T08:47:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37867ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5589bbb5cad0 con 0x5589bbc0a0f0 2024-04-21T08:47:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37818bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc09b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37820c0640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f37680a3ec0 0x7f37680a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37818bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc09b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56778/0 (socket says 172.21.15.82:56778) 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37818bf640 1 -- 172.21.15.82:0/397494578 learned_addr learned my addr 172.21.15.82:0/397494578 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f3772ffd640 1 -- 172.21.15.82:0/1590647263 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f37780d0860 con 0x7f377c1021a0 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37820c0640 1 --2- 172.21.15.82:0/397494578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc00fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.779+0000 7f37810be640 1 --2- 172.21.15.82:0/397494578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc0c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.780+0000 7f37818bf640 1 -- 172.21.15.82:0/397494578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 msgr2=0x5589bbc0c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.780+0000 7f37818bf640 1 --2- 172.21.15.82:0/397494578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc0c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.780+0000 7f37818bf640 1 -- 172.21.15.82:0/397494578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 msgr2=0x5589bbc00fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.780+0000 7f37818bf640 1 --2- 172.21.15.82:0/397494578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc00fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.780+0000 7f37818bf640 1 -- 172.21.15.82:0/397494578 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589bb8732d0 con 0x5589bbc01570 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.780+0000 7f37810be640 1 --2- 172.21.15.82:0/397494578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc0c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.780+0000 7f37818bf640 1 --2- 172.21.15.82:0/397494578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc09b20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f377809bef0 tx=0x7f377809bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745522b2b15f6051 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.780+0000 7f37820c0640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f37680a3ec0 0x7f37680a6380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3774004af0 tx=0x7f3774032400 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.780+0000 7f3767fff640 1 -- 172.21.15.82:0/397494578 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780056a0 con 0x5589bbc01570 2024-04-21T08:47:17.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.780+0000 7f3767fff640 1 -- 172.21.15.82:0/397494578 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3778014040 con 0x5589bbc01570 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.780+0000 7f3767fff640 1 -- 172.21.15.82:0/397494578 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780a4e60 con 0x5589bbc01570 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 -- 172.21.15.82:0/397494578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 msgr2=0x5589bbc09b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 --2- 172.21.15.82:0/397494578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc09b20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f377809bef0 tx=0x7f377809bd30 comp rx=0 tx=0).stop 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 -- 172.21.15.82:0/397494578 shutdown_connections 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 --2- 172.21.15.82:0/397494578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc0c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 --2- 172.21.15.82:0/397494578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc09b20 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 --2- 172.21.15.82:0/397494578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc00fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 -- 172.21.15.82:0/397494578 >> 172.21.15.82:0/397494578 conn(0x5589bbbfae70 msgr2=0x5589bbbfb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 -- 172.21.15.82:0/397494578 shutdown_connections 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 -- 172.21.15.82:0/397494578 wait complete. 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 -- start start 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 Processor -- start 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc07b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.781+0000 7f37867ee740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5589bbc0d000 con 0x5589bbc01570 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37867ee740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5589bbae9190 con 0x5589bbc00bc0 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37867ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5589bbc0d180 con 0x5589bbc0a0f0 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37867ee740 10 client.0 _ll_register_callbacks cb 0x5589bbae48b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37810be640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37810be640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc07620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56794/0 (socket says 172.21.15.82:56794) 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37810be640 1 -- 172.21.15.82:0/2462813359 learned_addr learned my addr 172.21.15.82:0/2462813359 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37818bf640 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37820c0640 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc07b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37810be640 1 -- 172.21.15.82:0/2462813359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 msgr2=0x5589bbc07b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37810be640 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc07b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37810be640 1 -- 172.21.15.82:0/2462813359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 msgr2=0x5589bbc070e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37810be640 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc070e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37810be640 1 -- 172.21.15.82:0/2462813359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589bb93d6b0 con 0x5589bbc01570 2024-04-21T08:47:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37810be640 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc07620 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f376c009fd0 tx=0x7f376c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae58b083c7f9522f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37657fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37657fa640 1 -- 172.21.15.82:0/2462813359 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c004220 con 0x5589bbc01570 2024-04-21T08:47:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37657fa640 1 -- 172.21.15.82:0/2462813359 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f376c0043c0 con 0x5589bbc01570 2024-04-21T08:47:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37867ee740 1 -- 172.21.15.82:0/2462813359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3778014040 con 0x5589bbc01570 2024-04-21T08:47:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37657fa640 1 -- 172.21.15.82:0/2462813359 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f376c012720 con 0x5589bbc01570 2024-04-21T08:47:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37867ee740 10 client.16026 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.782+0000 7f37867ee740 1 -- 172.21.15.82:0/2462813359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5589bbc19340 con 0x5589bbc01570 2024-04-21T08:47:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.783+0000 7f37657fa640 1 -- 172.21.15.82:0/2462813359 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3778014040 con 0x5589bbc01570 2024-04-21T08:47:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.783+0000 7f37657fa640 1 -- 172.21.15.82:0/2462813359 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f376c003bc0 con 0x5589bbc01570 2024-04-21T08:47:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.783+0000 7f37867ee740 10 client.16026 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.783+0000 7f37867ee740 10 client.16026 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:47:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.783+0000 7f37867ee740 20 client.16026 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.783+0000 7f37867ee740 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5589bbc1b9f0 0x5589bbc1ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.783+0000 7f37867ee740 4 client.16026 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.783+0000 7f37867ee740 1 -- 172.21.15.82:0/2462813359 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589bbb6e530 con 0x5589bbc1b9f0 2024-04-21T08:47:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.783+0000 7f37818bf640 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5589bbc1b9f0 0x5589bbc1ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.783+0000 7f37657fa640 10 client.16026 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.783+0000 7f37818bf640 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5589bbc1b9f0 0x5589bbc1ce60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f05ae489fd586a4 server_cookie=dc7510c8a17127ba in_seq=0 out_seq=0 2024-04-21T08:47:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37657fa640 1 -- 172.21.15.82:0/2462813359 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5589bbb6e530 con 0x5589bbc1b9f0 2024-04-21T08:47:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37657fa640 10 client.16026 handle_command_reply: tid=0 2024-04-21T08:47:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37867ee740 1 client.16026 shutdown 2024-04-21T08:47:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37867ee740 1 -- 172.21.15.82:0/2462813359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 msgr2=0x5589bbc07620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37867ee740 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc07620 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f376c009fd0 tx=0x7f376c011870 comp rx=0 tx=0).stop 2024-04-21T08:47:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37867ee740 1 -- 172.21.15.82:0/2462813359 shutdown_connections 2024-04-21T08:47:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37867ee740 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5589bbc1b9f0 0x5589bbc1ce60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37867ee740 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc07b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37657fa640 0 client.16026 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37867ee740 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc07620 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37867ee740 1 --2- 172.21.15.82:0/2462813359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37867ee740 1 -- 172.21.15.82:0/2462813359 >> 172.21.15.82:0/2462813359 conn(0x5589bbbfae70 msgr2=0x5589bbbfcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37867ee740 1 -- 172.21.15.82:0/2462813359 shutdown_connections 2024-04-21T08:47:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.784+0000 7f37867ee740 1 -- 172.21.15.82:0/2462813359 wait complete. 2024-04-21T08:47:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.785+0000 7f37867ee740 20 client.16026 trim_cache size 0 max 16384 2024-04-21T08:47:17.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.802+0000 7f37867ee740 1 Processor -- start 2024-04-21T08:47:17.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.802+0000 7f37867ee740 1 -- start start 2024-04-21T08:47:17.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37867ee740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc00bc0 0x5589bbc16a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37867ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc01570 0x5589bbc09810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37867ee740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc0a0f0 0x5589bbc07980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37867ee740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5589bbc1b1a0 con 0x5589bbc00bc0 2024-04-21T08:47:17.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37867ee740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5589bbc1b870 con 0x5589bbc0a0f0 2024-04-21T08:47:17.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37867ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5589bbc1bf40 con 0x5589bbc01570 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37818bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc00bc0 0x5589bbc16a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37818bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc00bc0 0x5589bbc16a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56800/0 (socket says 172.21.15.82:56800) 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37818bf640 1 -- 172.21.15.82:0/2265493305 learned_addr learned my addr 172.21.15.82:0/2265493305 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37810be640 1 --2- 172.21.15.82:0/2265493305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc01570 0x5589bbc09810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37820c0640 1 --2- 172.21.15.82:0/2265493305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc0a0f0 0x5589bbc07980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37818bf640 1 -- 172.21.15.82:0/2265493305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc01570 msgr2=0x5589bbc09810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37818bf640 1 --2- 172.21.15.82:0/2265493305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc01570 0x5589bbc09810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37818bf640 1 -- 172.21.15.82:0/2265493305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc0a0f0 msgr2=0x5589bbc07980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37818bf640 1 --2- 172.21.15.82:0/2265493305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc0a0f0 0x5589bbc07980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37818bf640 1 -- 172.21.15.82:0/2265493305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589bbc02cb0 con 0x5589bbc00bc0 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37818bf640 1 --2- 172.21.15.82:0/2265493305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc00bc0 0x5589bbc16a00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3778097ed0 tx=0x7f37780d0bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17883197febf77ba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37677fe640 1 -- 172.21.15.82:0/2265493305 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780ba070 con 0x5589bbc00bc0 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37677fe640 1 -- 172.21.15.82:0/2265493305 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3778014030 con 0x5589bbc00bc0 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.803+0000 7f37677fe640 1 -- 172.21.15.82:0/2265493305 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377806b070 con 0x5589bbc00bc0 2024-04-21T08:47:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.804+0000 7f37867ee740 1 -- 172.21.15.82:0/2265493305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc00bc0 msgr2=0x5589bbc16a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.804+0000 7f37867ee740 1 --2- 172.21.15.82:0/2265493305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc00bc0 0x5589bbc16a00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3778097ed0 tx=0x7f37780d0bd0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.804+0000 7f37867ee740 1 -- 172.21.15.82:0/2265493305 shutdown_connections 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.804+0000 7f37867ee740 1 --2- 172.21.15.82:0/2265493305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc0a0f0 0x5589bbc07980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.804+0000 7f37867ee740 1 --2- 172.21.15.82:0/2265493305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc01570 0x5589bbc09810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.804+0000 7f37867ee740 1 --2- 172.21.15.82:0/2265493305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc00bc0 0x5589bbc16a00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.804+0000 7f37867ee740 1 -- 172.21.15.82:0/2265493305 >> 172.21.15.82:0/2265493305 conn(0x5589bbbfae70 msgr2=0x5589bbc059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.804+0000 7f37867ee740 1 -- 172.21.15.82:0/2265493305 shutdown_connections 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.804+0000 7f37867ee740 1 -- 172.21.15.82:0/2265493305 wait complete. 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.804+0000 7f37867ee740 1 -- start start 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.804+0000 7f37867ee740 1 Processor -- start 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37867ee740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc25360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37867ee740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc1e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37867ee740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc1ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37867ee740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5589bbc1cce0 con 0x5589bbc01570 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37867ee740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5589bbc1c610 con 0x5589bbc00bc0 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37867ee740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5589bbc1d3b0 con 0x5589bbc0a0f0 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37867ee740 10 client.0 _ll_register_callbacks cb 0x7f376c009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37810be640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc1e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37810be640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc1e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56810/0 (socket says 172.21.15.82:56810) 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37810be640 1 -- 172.21.15.82:0/328441577 learned_addr learned my addr 172.21.15.82:0/328441577 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37818bf640 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc25360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37820c0640 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc1ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37810be640 1 -- 172.21.15.82:0/328441577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 msgr2=0x5589bbc1ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37810be640 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc1ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37810be640 1 -- 172.21.15.82:0/328441577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 msgr2=0x5589bbc25360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37810be640 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc25360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37810be640 1 -- 172.21.15.82:0/328441577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589bbc1f260 con 0x5589bbc01570 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37818bf640 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc25360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37810be640 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc1e6e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5589bbc06940 tx=0x7f376c005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5732524274a52ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37657fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37657fa640 1 -- 172.21.15.82:0/328441577 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c005450 con 0x5589bbc01570 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37657fa640 1 -- 172.21.15.82:0/328441577 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f376c0055f0 con 0x5589bbc01570 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.805+0000 7f37867ee740 1 -- 172.21.15.82:0/328441577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3778014030 con 0x5589bbc01570 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.806+0000 7f37657fa640 1 -- 172.21.15.82:0/328441577 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f376c01a6c0 con 0x5589bbc01570 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.806+0000 7f37867ee740 10 client.16032 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.806+0000 7f37867ee740 1 -- 172.21.15.82:0/328441577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5589bbc819a0 con 0x5589bbc01570 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.806+0000 7f37657fa640 1 -- 172.21.15.82:0/328441577 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3778014030 con 0x5589bbc01570 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.806+0000 7f37657fa640 1 -- 172.21.15.82:0/328441577 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f376c0049b0 con 0x5589bbc01570 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.806+0000 7f37867ee740 10 client.16032 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.806+0000 7f37867ee740 10 client.16032 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.806+0000 7f37867ee740 20 client.16032 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.806+0000 7f37867ee740 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5589bbc864b0 0x5589bbc851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.806+0000 7f37867ee740 4 client.16032 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.806+0000 7f37867ee740 1 -- 172.21.15.82:0/328441577 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "status"}) -- 0x5589bbb683a0 con 0x5589bbc864b0 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.806+0000 7f37818bf640 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5589bbc864b0 0x5589bbc851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37657fa640 10 client.16032 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37818bf640 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5589bbc864b0 0x5589bbc851c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=298fe9f3a851e8d1 server_cookie=705bcc4774135eaa in_seq=0 out_seq=0 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37657fa640 1 -- 172.21.15.82:0/328441577 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5589bbb683a0 con 0x5589bbc864b0 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37657fa640 10 client.16032 handle_command_reply: tid=0 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37867ee740 1 client.16032 shutdown 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37867ee740 1 -- 172.21.15.82:0/328441577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 msgr2=0x5589bbc1e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37867ee740 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc1e6e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5589bbc06940 tx=0x7f376c005b30 comp rx=0 tx=0).stop 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37867ee740 1 -- 172.21.15.82:0/328441577 shutdown_connections 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37867ee740 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5589bbc864b0 0x5589bbc851c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37867ee740 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5589bbc0a0f0 0x5589bbc1ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37657fa640 0 client.16032 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37867ee740 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589bbc01570 0x5589bbc1e6e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37867ee740 1 --2- 172.21.15.82:0/328441577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5589bbc00bc0 0x5589bbc25360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37867ee740 1 -- 172.21.15.82:0/328441577 >> 172.21.15.82:0/328441577 conn(0x5589bbbfae70 msgr2=0x5589bbc15400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37867ee740 1 -- 172.21.15.82:0/328441577 shutdown_connections 2024-04-21T08:47:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37867ee740 1 -- 172.21.15.82:0/328441577 wait complete. 2024-04-21T08:47:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.807+0000 7f37867ee740 20 client.16032 trim_cache size 0 max 16384 2024-04-21T08:47:17.814 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:47:17.814 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "24d05dee-ffba-11ee-bc93-c7b262605968", 2024-04-21T08:47:17.814 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-21T08:47:17.814 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24509, 2024-04-21T08:47:17.814 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-21T08:47:17.815 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-21T08:47:17.815 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-21T08:47:17.815 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 163.32492845199999, 2024-04-21T08:47:17.815 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 28, 2024-04-21T08:47:17.815 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 94, 2024-04-21T08:47:17.815 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 93, 2024-04-21T08:47:17.815 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 169.39247644599999 2024-04-21T08:47:17.815 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:47:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.808+0000 7f3783b23640 1 -- 172.21.15.82:0/1590647263 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f37680a3ec0 msgr2=0x7f37680a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.808+0000 7f3783b23640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f37680a3ec0 0x7f37680a6380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3774004af0 tx=0x7f3774032400 comp rx=0 tx=0).stop 2024-04-21T08:47:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.808+0000 7f3783b23640 1 -- 172.21.15.82:0/1590647263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c1021a0 msgr2=0x7f377c1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.808+0000 7f3783b23640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c1021a0 0x7f377c1987a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f377800d950 tx=0x7f377800de10 comp rx=0 tx=0).stop 2024-04-21T08:47:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.808+0000 7f3783b23640 1 -- 172.21.15.82:0/1590647263 shutdown_connections 2024-04-21T08:47:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.808+0000 7f3783b23640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f377c19d010 0x7f377c19d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.808+0000 7f3783b23640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c107270 0x7f377c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.808+0000 7f3783b23640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f37680a3ec0 0x7f37680a6380 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.808+0000 7f3783b23640 1 --2- 172.21.15.82:0/1590647263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c1021a0 0x7f377c1987a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.808+0000 7f3783b23640 1 -- 172.21.15.82:0/1590647263 >> 172.21.15.82:0/1590647263 conn(0x7f377c0fd9c0 msgr2=0x7f377c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.808+0000 7f3783b23640 1 -- 172.21.15.82:0/1590647263 shutdown_connections 2024-04-21T08:47:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:17.808+0000 7f3783b23640 1 -- 172.21.15.82:0/1590647263 wait complete. 2024-04-21T08:47:17.818 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 120 sec 2024-04-21T08:47:17.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:17 smithi134 ceph-mon[31772]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-21T08:47:17.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:17 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/817128288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:17.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:17 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2789025179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:18.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:17 smithi082 ceph-mon[27967]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-21T08:47:18.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:17 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/817128288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:18.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:17 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2789025179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:18.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:17 smithi106 ceph-mon[31072]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-21T08:47:18.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:17 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/817128288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:47:18.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:17 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2789025179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:19.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:19 smithi134 ceph-mon[31772]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-04-21T08:47:20.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:19 smithi082 ceph-mon[27967]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-04-21T08:47:20.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:19 smithi106 ceph-mon[31072]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-04-21T08:47:21.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:21 smithi134 ceph-mon[31772]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-04-21T08:47:22.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:21 smithi082 ceph-mon[27967]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-04-21T08:47:22.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:21 smithi106 ceph-mon[31072]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-04-21T08:47:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.290+0000 7fa8291f4640 1 -- 172.21.15.82:0/444000040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82410c520 msgr2=0x7fa82410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.290+0000 7fa8291f4640 1 --2- 172.21.15.82:0/444000040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82410c520 0x7fa82410e9c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009900 tx=0x7fa81c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:47:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.290+0000 7fa8291f4640 1 -- 172.21.15.82:0/444000040 shutdown_connections 2024-04-21T08:47:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.290+0000 7fa8291f4640 1 --2- 172.21.15.82:0/444000040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82410c520 0x7fa82410e9c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.290+0000 7fa8291f4640 1 --2- 172.21.15.82:0/444000040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa824103b00 0x7fa82410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.290+0000 7fa8291f4640 1 --2- 172.21.15.82:0/444000040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8241031e0 0x7fa8241035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.290+0000 7fa8291f4640 1 -- 172.21.15.82:0/444000040 >> 172.21.15.82:0/444000040 conn(0x7fa824077fd0 msgr2=0x7fa8240783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.290+0000 7fa8291f4640 1 -- 172.21.15.82:0/444000040 shutdown_connections 2024-04-21T08:47:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.290+0000 7fa8291f4640 1 -- 172.21.15.82:0/444000040 wait complete. 2024-04-21T08:47:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.291+0000 7fa8291f4640 1 Processor -- start 2024-04-21T08:47:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.291+0000 7fa8291f4640 1 -- start start 2024-04-21T08:47:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.291+0000 7fa8291f4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8241031e0 0x7fa8241986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa8291f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa824103b00 0x7fa824198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa8291f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82410c520 0x7fa82419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa8291f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241121b0 con 0x7fa82410c520 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa8291f4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa824112030 con 0x7fa8241031e0 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa8291f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa824112330 con 0x7fa824103b00 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa82359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82410c520 0x7fa82419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa82359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82410c520 0x7fa82419cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50616/0 (socket says 172.21.15.82:50616) 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa82359e640 1 -- 172.21.15.82:0/2125086227 learned_addr learned my addr 172.21.15.82:0/2125086227 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa822d9d640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8241031e0 0x7fa8241986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa81bfff640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa824103b00 0x7fa824198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa82359e640 1 -- 172.21.15.82:0/2125086227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa824103b00 msgr2=0x7fa824198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa82359e640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa824103b00 0x7fa824198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa82359e640 1 -- 172.21.15.82:0/2125086227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8241031e0 msgr2=0x7fa8241986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa82359e640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8241031e0 0x7fa8241986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa82359e640 1 -- 172.21.15.82:0/2125086227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82419d690 con 0x7fa82410c520 2024-04-21T08:47:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa82359e640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82410c520 0x7fa82419cf60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009230 tx=0x7fa81c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa820d99640 1 -- 172.21.15.82:0/2125086227 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c043020 con 0x7fa82410c520 2024-04-21T08:47:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa820d99640 1 -- 172.21.15.82:0/2125086227 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa81c02dcb0 con 0x7fa82410c520 2024-04-21T08:47:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa820d99640 1 -- 172.21.15.82:0/2125086227 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c003a90 con 0x7fa82410c520 2024-04-21T08:47:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa8291f4640 1 -- 172.21.15.82:0/2125086227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82419d920 con 0x7fa82410c520 2024-04-21T08:47:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.292+0000 7fa8291f4640 1 -- 172.21.15.82:0/2125086227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8241a5210 con 0x7fa82410c520 2024-04-21T08:47:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.295+0000 7fa820d99640 1 -- 172.21.15.82:0/2125086227 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa81c046020 con 0x7fa82410c520 2024-04-21T08:47:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.295+0000 7fa82bde5740 1 Processor -- start 2024-04-21T08:47:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.295+0000 7fa82bde5740 1 -- start start 2024-04-21T08:47:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa82bde5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c2718d0 0x55ca2c270620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa82bde5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c270b60 0x55ca2c270fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa82bde5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c27a0e0 0x55ca2c27c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa82bde5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ca2c1d58d0 con 0x55ca2c2718d0 2024-04-21T08:47:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa82bde5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ca2c172e80 con 0x55ca2c270b60 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa82bde5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca2c1d1890 con 0x55ca2c27a0e0 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa820d99640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa8100a3f60 0x7fa8100a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c2718d0 0x55ca2c270620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c2718d0 0x55ca2c270620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50628/0 (socket says 172.21.15.82:50628) 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa81bfff640 1 -- 172.21.15.82:0/4253921239 learned_addr learned my addr 172.21.15.82:0/4253921239 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa822d9d640 1 --2- 172.21.15.82:0/4253921239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c270b60 0x55ca2c270fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa82359e640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa8100a3f60 0x7fa8100a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa820d99640 1 -- 172.21.15.82:0/2125086227 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa81c0f2ef0 con 0x7fa82410c520 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa81bfff640 1 --2- 172.21.15.82:0/4253921239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c27a0e0 0x55ca2c27c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa81bfff640 1 -- 172.21.15.82:0/4253921239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c27a0e0 msgr2=0x55ca2c27c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa81bfff640 1 --2- 172.21.15.82:0/4253921239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c27a0e0 0x55ca2c27c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa81bfff640 1 -- 172.21.15.82:0/4253921239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c270b60 msgr2=0x55ca2c270fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa81bfff640 1 --2- 172.21.15.82:0/4253921239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c270b60 0x55ca2c270fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa81bfff640 1 -- 172.21.15.82:0/4253921239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca2c194320 con 0x55ca2c2718d0 2024-04-21T08:47:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa81bfff640 1 --2- 172.21.15.82:0/4253921239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c2718d0 0x55ca2c270620 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00bf80 tx=0x7fa80c00b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dc9622a0995d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.296+0000 7fa82359e640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa8100a3f60 0x7fa8100a6420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0f0270 tx=0x7fa81c0a9040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.297+0000 7fa8197fa640 1 -- 172.21.15.82:0/4253921239 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c019030 con 0x55ca2c2718d0 2024-04-21T08:47:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.297+0000 7fa8197fa640 1 -- 172.21.15.82:0/4253921239 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa80c005bf0 con 0x55ca2c2718d0 2024-04-21T08:47:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.297+0000 7fa8197fa640 1 -- 172.21.15.82:0/4253921239 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c01d600 con 0x55ca2c2718d0 2024-04-21T08:47:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.297+0000 7fa82bde5740 1 -- 172.21.15.82:0/4253921239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c2718d0 msgr2=0x55ca2c270620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.297+0000 7fa82bde5740 1 --2- 172.21.15.82:0/4253921239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c2718d0 0x55ca2c270620 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00bf80 tx=0x7fa80c00b610 comp rx=0 tx=0).stop 2024-04-21T08:47:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.297+0000 7fa82bde5740 1 -- 172.21.15.82:0/4253921239 shutdown_connections 2024-04-21T08:47:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.297+0000 7fa82bde5740 1 --2- 172.21.15.82:0/4253921239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c27a0e0 0x55ca2c27c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.297+0000 7fa82bde5740 1 --2- 172.21.15.82:0/4253921239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c270b60 0x55ca2c270fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.297+0000 7fa82bde5740 1 --2- 172.21.15.82:0/4253921239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c2718d0 0x55ca2c270620 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.297+0000 7fa82bde5740 1 -- 172.21.15.82:0/4253921239 >> 172.21.15.82:0/4253921239 conn(0x55ca2c237c90 msgr2=0x55ca2c26b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa82bde5740 1 -- 172.21.15.82:0/4253921239 shutdown_connections 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa82bde5740 1 -- 172.21.15.82:0/4253921239 wait complete. 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa82bde5740 1 -- start start 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa82bde5740 1 Processor -- start 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa82bde5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c270b60 0x55ca2c279a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa82bde5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c2718d0 0x55ca2c276e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa82bde5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c277380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa82bde5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ca2c27ce10 con 0x55ca2c27a0e0 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa82bde5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ca2c15deb0 con 0x55ca2c270b60 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa82bde5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca2c27cf90 con 0x55ca2c2718d0 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa82bde5740 10 client.0 _ll_register_callbacks cb 0x55ca2c18ab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c277380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c277380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50630/0 (socket says 172.21.15.82:50630) 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa81bfff640 1 -- 172.21.15.82:0/257856911 learned_addr learned my addr 172.21.15.82:0/257856911 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.298+0000 7fa81bfff640 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c270b60 0x55ca2c279a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa822d9d640 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c2718d0 0x55ca2c276e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa81bfff640 1 -- 172.21.15.82:0/257856911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c2718d0 msgr2=0x55ca2c276e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa81bfff640 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c2718d0 0x55ca2c276e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa81bfff640 1 -- 172.21.15.82:0/257856911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c270b60 msgr2=0x55ca2c279a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa81bfff640 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c270b60 0x55ca2c279a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa81bfff640 1 -- 172.21.15.82:0/257856911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca2bee8210 con 0x55ca2c27a0e0 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa81bfff640 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c277380 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00b460 tx=0x7fa80c00b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1328101f0a772f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa80a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa80a7fc640 1 -- 172.21.15.82:0/257856911 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c019040 con 0x55ca2c27a0e0 2024-04-21T08:47:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa80a7fc640 1 -- 172.21.15.82:0/257856911 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa80c0119f0 con 0x55ca2c27a0e0 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa82bde5740 1 -- 172.21.15.82:0/257856911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ca2c1e9880 con 0x55ca2c27a0e0 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa80a7fc640 1 -- 172.21.15.82:0/257856911 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa80c034050 con 0x55ca2c27a0e0 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa82bde5740 10 client.16044 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa82bde5740 1 -- 172.21.15.82:0/257856911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca2bfb4620 con 0x55ca2c27a0e0 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa80a7fc640 1 -- 172.21.15.82:0/257856911 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ca2c1e9880 con 0x55ca2c27a0e0 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.299+0000 7fa80a7fc640 1 -- 172.21.15.82:0/257856911 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa80c01d3c0 con 0x55ca2c27a0e0 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.300+0000 7fa82bde5740 10 client.16044 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.300+0000 7fa82bde5740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.300+0000 7fa82bde5740 20 client.16044 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.300+0000 7fa82bde5740 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ca2c28a050 0x55ca2c28c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.300+0000 7fa82bde5740 4 client.16044 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.300+0000 7fa82bde5740 1 -- 172.21.15.82:0/257856911 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca2c1753d0 con 0x55ca2c28a050 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.300+0000 7fa822d9d640 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ca2c28a050 0x55ca2c28c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.300+0000 7fa80a7fc640 10 client.16044 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.300+0000 7fa822d9d640 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ca2c28a050 0x55ca2c28c440 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f9f76395d9953dd server_cookie=8ddf6e9a58326be in_seq=0 out_seq=0 2024-04-21T08:47:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa80a7fc640 1 -- 172.21.15.82:0/257856911 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca2c1753d0 con 0x55ca2c28a050 2024-04-21T08:47:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa80a7fc640 10 client.16044 handle_command_reply: tid=0 2024-04-21T08:47:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa82bde5740 1 client.16044 shutdown 2024-04-21T08:47:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa82bde5740 1 -- 172.21.15.82:0/257856911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 msgr2=0x55ca2c277380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa82bde5740 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c277380 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00b460 tx=0x7fa80c00b740 comp rx=0 tx=0).stop 2024-04-21T08:47:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa82bde5740 1 -- 172.21.15.82:0/257856911 shutdown_connections 2024-04-21T08:47:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa82bde5740 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ca2c28a050 0x55ca2c28c440 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa82bde5740 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c277380 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa82bde5740 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c2718d0 0x55ca2c276e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa82bde5740 1 --2- 172.21.15.82:0/257856911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c270b60 0x55ca2c279a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa82bde5740 1 -- 172.21.15.82:0/257856911 >> 172.21.15.82:0/257856911 conn(0x55ca2c237c90 msgr2=0x55ca2c26d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa80a7fc640 0 client.16044 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa82bde5740 1 -- 172.21.15.82:0/257856911 shutdown_connections 2024-04-21T08:47:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa82bde5740 1 -- 172.21.15.82:0/257856911 wait complete. 2024-04-21T08:47:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.301+0000 7fa82bde5740 20 client.16044 trim_cache size 0 max 16384 2024-04-21T08:47:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.318+0000 7fa82bde5740 1 Processor -- start 2024-04-21T08:47:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.318+0000 7fa82bde5740 1 -- start start 2024-04-21T08:47:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa82bde5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c270b60 0x55ca2c285d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa82bde5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c2718d0 0x55ca2c279530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa82bde5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c279a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa82bde5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241af3b0 con 0x55ca2c27a0e0 2024-04-21T08:47:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa82bde5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b0ef0 con 0x55ca2c2718d0 2024-04-21T08:47:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa82bde5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241afa80 con 0x55ca2c270b60 2024-04-21T08:47:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c279a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c279a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50638/0 (socket says 172.21.15.82:50638) 2024-04-21T08:47:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa81bfff640 1 -- 172.21.15.82:0/1156085802 learned_addr learned my addr 172.21.15.82:0/1156085802 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa822d9d640 1 --2- 172.21.15.82:0/1156085802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c2718d0 0x55ca2c279530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa81bfff640 1 --2- 172.21.15.82:0/1156085802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c270b60 0x55ca2c285d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa81bfff640 1 -- 172.21.15.82:0/1156085802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c270b60 msgr2=0x55ca2c285d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa81bfff640 1 --2- 172.21.15.82:0/1156085802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c270b60 0x55ca2c285d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa81bfff640 1 -- 172.21.15.82:0/1156085802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c2718d0 msgr2=0x55ca2c279530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa81bfff640 1 --2- 172.21.15.82:0/1156085802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c2718d0 0x55ca2c279530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa81bfff640 1 -- 172.21.15.82:0/1156085802 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca2c26fe30 con 0x55ca2c27a0e0 2024-04-21T08:47:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.319+0000 7fa81bfff640 1 --2- 172.21.15.82:0/1156085802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c279a70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa80c013fd0 tx=0x7fa80c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5926fad4b7c8608 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.320+0000 7fa818ff9640 1 -- 172.21.15.82:0/1156085802 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c019070 con 0x55ca2c27a0e0 2024-04-21T08:47:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.320+0000 7fa818ff9640 1 -- 172.21.15.82:0/1156085802 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa80c01de20 con 0x55ca2c27a0e0 2024-04-21T08:47:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.320+0000 7fa818ff9640 1 -- 172.21.15.82:0/1156085802 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c01d980 con 0x55ca2c27a0e0 2024-04-21T08:47:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.320+0000 7fa82bde5740 1 -- 172.21.15.82:0/1156085802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 msgr2=0x55ca2c279a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.320+0000 7fa82bde5740 1 --2- 172.21.15.82:0/1156085802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c279a70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa80c013fd0 tx=0x7fa80c015040 comp rx=0 tx=0).stop 2024-04-21T08:47:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.320+0000 7fa82bde5740 1 -- 172.21.15.82:0/1156085802 shutdown_connections 2024-04-21T08:47:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.320+0000 7fa82bde5740 1 --2- 172.21.15.82:0/1156085802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c279a70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.320+0000 7fa82bde5740 1 --2- 172.21.15.82:0/1156085802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c2718d0 0x55ca2c279530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.320+0000 7fa82bde5740 1 --2- 172.21.15.82:0/1156085802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c270b60 0x55ca2c285d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.320+0000 7fa82bde5740 1 -- 172.21.15.82:0/1156085802 >> 172.21.15.82:0/1156085802 conn(0x55ca2c237c90 msgr2=0x55ca2c272060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa82bde5740 1 -- 172.21.15.82:0/1156085802 shutdown_connections 2024-04-21T08:47:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa82bde5740 1 -- 172.21.15.82:0/1156085802 wait complete. 2024-04-21T08:47:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa82bde5740 1 -- start start 2024-04-21T08:47:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa82bde5740 1 Processor -- start 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa82bde5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c270b60 0x55ca2c2827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa82bde5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c2718d0 0x55ca2c27d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa82bde5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c27d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa82bde5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b0820 con 0x55ca2c27a0e0 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa82bde5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241b0150 con 0x55ca2c2718d0 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa82bde5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241ad390 con 0x55ca2c270b60 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa82bde5740 10 client.0 _ll_register_callbacks cb 0x7fa814009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c27d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c27d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50650/0 (socket says 172.21.15.82:50650) 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa81bfff640 1 -- 172.21.15.82:0/701881156 learned_addr learned my addr 172.21.15.82:0/701881156 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa81bfff640 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c270b60 0x55ca2c2827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.321+0000 7fa822d9d640 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c2718d0 0x55ca2c27d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa81bfff640 1 -- 172.21.15.82:0/701881156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c270b60 msgr2=0x55ca2c2827a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa81bfff640 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c270b60 0x55ca2c2827a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa81bfff640 1 -- 172.21.15.82:0/701881156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c2718d0 msgr2=0x55ca2c27d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa81bfff640 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c2718d0 0x55ca2c27d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa81bfff640 1 -- 172.21.15.82:0/701881156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca2c2ee410 con 0x55ca2c27a0e0 2024-04-21T08:47:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa81bfff640 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c27d950 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa80c011780 tx=0x7fa80c013ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af8b2664f2ec4eb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa80a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa80a7fc640 1 -- 172.21.15.82:0/701881156 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c019070 con 0x55ca2c27a0e0 2024-04-21T08:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa82bde5740 1 -- 172.21.15.82:0/701881156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa80c01de20 con 0x55ca2c27a0e0 2024-04-21T08:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa80a7fc640 1 -- 172.21.15.82:0/701881156 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa80c028040 con 0x55ca2c27a0e0 2024-04-21T08:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa80a7fc640 1 -- 172.21.15.82:0/701881156 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa80c01d980 con 0x55ca2c27a0e0 2024-04-21T08:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa82bde5740 10 client.16050 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa82bde5740 1 -- 172.21.15.82:0/701881156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca2c2ef5a0 con 0x55ca2c27a0e0 2024-04-21T08:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.322+0000 7fa80a7fc640 1 -- 172.21.15.82:0/701881156 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa80c01de20 con 0x55ca2c27a0e0 2024-04-21T08:47:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.323+0000 7fa80a7fc640 1 -- 172.21.15.82:0/701881156 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa80c0051c0 con 0x55ca2c27a0e0 2024-04-21T08:47:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.323+0000 7fa82bde5740 10 client.16050 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.323+0000 7fa82bde5740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.323+0000 7fa82bde5740 20 client.16050 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.323+0000 7fa82bde5740 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ca2c2efb00 0x55ca2c2f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.323+0000 7fa82bde5740 4 client.16050 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:47:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.323+0000 7fa82bde5740 1 -- 172.21.15.82:0/701881156 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca2c1e32f0 con 0x55ca2c2efb00 2024-04-21T08:47:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.323+0000 7fa822d9d640 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ca2c2efb00 0x55ca2c2f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.323+0000 7fa80a7fc640 10 client.16050 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.323+0000 7fa822d9d640 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ca2c2efb00 0x55ca2c2f1ef0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d24af48d272a52 server_cookie=ee04b6d2b6ee56a1 in_seq=0 out_seq=0 2024-04-21T08:47:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa80a7fc640 1 -- 172.21.15.82:0/701881156 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca2c1e32f0 con 0x55ca2c2efb00 2024-04-21T08:47:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa80a7fc640 10 client.16050 handle_command_reply: tid=0 2024-04-21T08:47:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa82bde5740 1 client.16050 shutdown 2024-04-21T08:47:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa82bde5740 1 -- 172.21.15.82:0/701881156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 msgr2=0x55ca2c27d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa82bde5740 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c27d950 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa80c011780 tx=0x7fa80c013ee0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa82bde5740 1 -- 172.21.15.82:0/701881156 shutdown_connections 2024-04-21T08:47:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa82bde5740 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ca2c2efb00 0x55ca2c2f1ef0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa82bde5740 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ca2c27a0e0 0x55ca2c27d950 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa82bde5740 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ca2c2718d0 0x55ca2c27d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa80a7fc640 0 client.16050 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa82bde5740 1 --2- 172.21.15.82:0/701881156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca2c270b60 0x55ca2c2827a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa82bde5740 1 -- 172.21.15.82:0/701881156 >> 172.21.15.82:0/701881156 conn(0x55ca2c237c90 msgr2=0x55ca2c2756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa82bde5740 1 -- 172.21.15.82:0/701881156 shutdown_connections 2024-04-21T08:47:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa82bde5740 1 -- 172.21.15.82:0/701881156 wait complete. 2024-04-21T08:47:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.334+0000 7fa82bde5740 20 client.16050 trim_cache size 0 max 16384 2024-04-21T08:47:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:47:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:47:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:47:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:47:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.335+0000 7fa8291f4640 1 -- 172.21.15.82:0/2125086227 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa8100a3f60 msgr2=0x7fa8100a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.335+0000 7fa8291f4640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa8100a3f60 0x7fa8100a6420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0f0270 tx=0x7fa81c0a9040 comp rx=0 tx=0).stop 2024-04-21T08:47:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.335+0000 7fa8291f4640 1 -- 172.21.15.82:0/2125086227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82410c520 msgr2=0x7fa82419cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.335+0000 7fa8291f4640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82410c520 0x7fa82419cf60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009230 tx=0x7fa81c0045e0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.335+0000 7fa8291f4640 1 -- 172.21.15.82:0/2125086227 shutdown_connections 2024-04-21T08:47:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.335+0000 7fa8291f4640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa8100a3f60 0x7fa8100a6420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.335+0000 7fa8291f4640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82410c520 0x7fa82419cf60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.335+0000 7fa8291f4640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa824103b00 0x7fa824198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.335+0000 7fa8291f4640 1 --2- 172.21.15.82:0/2125086227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8241031e0 0x7fa8241986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.335+0000 7fa8291f4640 1 -- 172.21.15.82:0/2125086227 >> 172.21.15.82:0/2125086227 conn(0x7fa824077fd0 msgr2=0x7fa8240fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.335+0000 7fa8291f4640 1 -- 172.21.15.82:0/2125086227 shutdown_connections 2024-04-21T08:47:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:23.335+0000 7fa8291f4640 1 -- 172.21.15.82:0/2125086227 wait complete. 2024-04-21T08:47:23.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:23 smithi134 ceph-mon[31772]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-21T08:47:23.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:23 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:47:24.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:23 smithi082 ceph-mon[27967]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-21T08:47:24.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:23 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:47:24.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:23 smithi106 ceph-mon[31072]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-21T08:47:24.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:23 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:47:25.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:25 smithi134 ceph-mon[31772]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-21T08:47:26.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:25 smithi082 ceph-mon[27967]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-21T08:47:26.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:25 smithi106 ceph-mon[31072]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-21T08:47:27.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:27 smithi134 ceph-mon[31772]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-21T08:47:28.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:27 smithi082 ceph-mon[27967]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-21T08:47:28.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:27 smithi106 ceph-mon[31072]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-21T08:47:29.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:29 smithi134 ceph-mon[31772]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 17 op/s 2024-04-21T08:47:30.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:29 smithi082 ceph-mon[27967]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 17 op/s 2024-04-21T08:47:30.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:29 smithi106 ceph-mon[31072]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 17 op/s 2024-04-21T08:47:31.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:31 smithi134 ceph-mon[31772]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-21T08:47:32.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:31 smithi082 ceph-mon[27967]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-21T08:47:32.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:31 smithi106 ceph-mon[31072]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-21T08:47:33.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:33 smithi134 ceph-mon[31772]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 17 op/s 2024-04-21T08:47:34.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:33 smithi082 ceph-mon[27967]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 17 op/s 2024-04-21T08:47:34.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:33 smithi106 ceph-mon[31072]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 17 op/s 2024-04-21T08:47:35.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:35 smithi134 ceph-mon[31772]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-21T08:47:36.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:35 smithi082 ceph-mon[27967]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-21T08:47:36.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:35 smithi106 ceph-mon[31072]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-21T08:47:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.460+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/82538541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a01021a0 msgr2=0x7ff9a0102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.460+0000 7ff9a4ed2640 1 --2- 172.21.15.82:0/82538541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a01021a0 0x7ff9a0102580 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff990009900 tx=0x7ff99002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.460+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/82538541 shutdown_connections 2024-04-21T08:47:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.460+0000 7ff9a4ed2640 1 --2- 172.21.15.82:0/82538541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a0107270 0x7ff9a010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.460+0000 7ff9a4ed2640 1 --2- 172.21.15.82:0/82538541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9a0102b50 0x7ff9a0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.460+0000 7ff9a4ed2640 1 --2- 172.21.15.82:0/82538541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a01021a0 0x7ff9a0102580 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.460+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/82538541 >> 172.21.15.82:0/82538541 conn(0x7ff9a00793e0 msgr2=0x7ff9a00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.460+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/82538541 shutdown_connections 2024-04-21T08:47:36.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.460+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/82538541 wait complete. 2024-04-21T08:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.461+0000 7ff9a4ed2640 1 Processor -- start 2024-04-21T08:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.461+0000 7ff9a4ed2640 1 -- start start 2024-04-21T08:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.461+0000 7ff9a4ed2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9a01021a0 0x7ff9a0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.461+0000 7ff9a4ed2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a0107270 0x7ff9a0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.461+0000 7ff9a4ed2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a019cfa0 0x7ff9a019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.461+0000 7ff9a4ed2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0112230 con 0x7ff9a019cfa0 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.461+0000 7ff9a4ed2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01120b0 con 0x7ff9a01021a0 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.461+0000 7ff9a4ed2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01123b0 con 0x7ff9a0107270 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.462+0000 7ff99ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a019cfa0 0x7ff9a019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.462+0000 7ff99ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a019cfa0 0x7ff9a019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56540/0 (socket says 172.21.15.82:56540) 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.462+0000 7ff99ed9d640 1 -- 172.21.15.82:0/4157171495 learned_addr learned my addr 172.21.15.82:0/4157171495 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.462+0000 7ff99e59c640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9a01021a0 0x7ff9a0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.462+0000 7ff99dd9b640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a0107270 0x7ff9a0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.462+0000 7ff99ed9d640 1 -- 172.21.15.82:0/4157171495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a0107270 msgr2=0x7ff9a0198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.462+0000 7ff99ed9d640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a0107270 0x7ff9a0198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.462+0000 7ff99ed9d640 1 -- 172.21.15.82:0/4157171495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9a01021a0 msgr2=0x7ff9a0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.462+0000 7ff99ed9d640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9a01021a0 0x7ff9a0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.462+0000 7ff99ed9d640 1 -- 172.21.15.82:0/4157171495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a019da90 con 0x7ff9a019cfa0 2024-04-21T08:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.462+0000 7ff99e59c640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9a01021a0 0x7ff9a0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.462+0000 7ff99ed9d640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a019cfa0 0x7ff9a019d450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff99800ea60 tx=0x7ff99800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.463+0000 7ff98f7fe640 1 -- 172.21.15.82:0/4157171495 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99800ce40 con 0x7ff9a019cfa0 2024-04-21T08:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.463+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/4157171495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a019dd80 con 0x7ff9a019cfa0 2024-04-21T08:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.463+0000 7ff98f7fe640 1 -- 172.21.15.82:0/4157171495 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff998005020 con 0x7ff9a019cfa0 2024-04-21T08:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.463+0000 7ff98f7fe640 1 -- 172.21.15.82:0/4157171495 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998010670 con 0x7ff9a019cfa0 2024-04-21T08:47:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.463+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/4157171495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a00744f0 con 0x7ff9a019cfa0 2024-04-21T08:47:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.465+0000 7ff98f7fe640 1 -- 172.21.15.82:0/4157171495 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff9980054b0 con 0x7ff9a019cfa0 2024-04-21T08:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.466+0000 7ff98f7fe640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff9880a3ee0 0x7ff9880a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.466+0000 7ff98f7fe640 1 -- 172.21.15.82:0/4157171495 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff9980d1340 con 0x7ff9a019cfa0 2024-04-21T08:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.466+0000 7ff99e59c640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff9880a3ee0 0x7ff9880a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.466+0000 7ff99e59c640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff9880a3ee0 0x7ff9880a63a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff990005ab0 tx=0x7ff990005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.467+0000 7ff9a7b9f740 1 Processor -- start 2024-04-21T08:47:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.467+0000 7ff9a7b9f740 1 -- start start 2024-04-21T08:47:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.467+0000 7ff9a7b9f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153b8d0 0x55d0d153a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.467+0000 7ff9a7b9f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d153ab60 0x55d0d153afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.467+0000 7ff9a7b9f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 0x55d0d15464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 0x55d0d15464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 0x55d0d15464d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56548/0 (socket says 172.21.15.82:56548) 2024-04-21T08:47:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99ed9d640 1 -- 172.21.15.82:0/2988687968 learned_addr learned my addr 172.21.15.82:0/2988687968 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2988687968 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d0d149f8d0 con 0x55d0d15440e0 2024-04-21T08:47:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2988687968 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d0d143ce80 con 0x55d0d153b8d0 2024-04-21T08:47:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2988687968 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0d149b890 con 0x55d0d153ab60 2024-04-21T08:47:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99e59c640 1 --2- 172.21.15.82:0/2988687968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153b8d0 0x55d0d153a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99dd9b640 1 --2- 172.21.15.82:0/2988687968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d153ab60 0x55d0d153afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99e59c640 1 -- 172.21.15.82:0/2988687968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d153ab60 msgr2=0x55d0d153afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99e59c640 1 --2- 172.21.15.82:0/2988687968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d153ab60 0x55d0d153afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99e59c640 1 -- 172.21.15.82:0/2988687968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 msgr2=0x55d0d15464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99e59c640 1 --2- 172.21.15.82:0/2988687968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 0x55d0d15464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99e59c640 1 -- 172.21.15.82:0/2988687968 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0d145e320 con 0x55d0d153b8d0 2024-04-21T08:47:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99ed9d640 1 --2- 172.21.15.82:0/2988687968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 0x55d0d15464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99dd9b640 1 --2- 172.21.15.82:0/2988687968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d153ab60 0x55d0d153afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.468+0000 7ff99e59c640 1 --2- 172.21.15.82:0/2988687968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153b8d0 0x55d0d153a620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff99002c170 tx=0x7ff99003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7766cd9e1f816d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.469+0000 7ff98cff9640 1 -- 172.21.15.82:0/2988687968 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99002c7e0 con 0x55d0d153b8d0 2024-04-21T08:47:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.469+0000 7ff98cff9640 1 -- 172.21.15.82:0/2988687968 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9900035f0 con 0x55d0d153b8d0 2024-04-21T08:47:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.469+0000 7ff98cff9640 1 -- 172.21.15.82:0/2988687968 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99003f440 con 0x55d0d153b8d0 2024-04-21T08:47:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.469+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2988687968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153b8d0 msgr2=0x55d0d153a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.469+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/2988687968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153b8d0 0x55d0d153a620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff99002c170 tx=0x7ff99003f000 comp rx=0 tx=0).stop 2024-04-21T08:47:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.470+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2988687968 shutdown_connections 2024-04-21T08:47:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.470+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/2988687968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 0x55d0d15464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.470+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/2988687968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d153ab60 0x55d0d153afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.470+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/2988687968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153b8d0 0x55d0d153a620 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.470+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2988687968 >> 172.21.15.82:0/2988687968 conn(0x55d0d1501c90 msgr2=0x55d0d1535850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.470+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2988687968 shutdown_connections 2024-04-21T08:47:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.470+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2988687968 wait complete. 2024-04-21T08:47:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.470+0000 7ff9a7b9f740 1 -- start start 2024-04-21T08:47:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.470+0000 7ff9a7b9f740 1 Processor -- start 2024-04-21T08:47:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff9a7b9f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153ab60 0x55d0d1543b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff9a7b9f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153b8d0 0x55d0d1540ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff9a7b9f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d15440e0 0x55d0d1541620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff9a7b9f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d0d1546db0 con 0x55d0d153ab60 2024-04-21T08:47:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff99e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153ab60 0x55d0d1543b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff9a7b9f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d0d1427eb0 con 0x55d0d153b8d0 2024-04-21T08:47:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff9a7b9f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0d1546f30 con 0x55d0d15440e0 2024-04-21T08:47:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff99e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153ab60 0x55d0d1543b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56566/0 (socket says 172.21.15.82:56566) 2024-04-21T08:47:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff99e59c640 1 -- 172.21.15.82:0/2005310120 learned_addr learned my addr 172.21.15.82:0/2005310120 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff9a7b9f740 10 client.0 _ll_register_callbacks cb 0x55d0d1454b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff99dd9b640 1 --2- 172.21.15.82:0/2005310120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153b8d0 0x55d0d1540ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff99e59c640 1 -- 172.21.15.82:0/2005310120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d15440e0 msgr2=0x55d0d1541620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff99e59c640 1 --2- 172.21.15.82:0/2005310120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d15440e0 0x55d0d1541620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff99e59c640 1 -- 172.21.15.82:0/2005310120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153b8d0 msgr2=0x55d0d1540ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff99e59c640 1 --2- 172.21.15.82:0/2005310120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153b8d0 0x55d0d1540ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.471+0000 7ff99e59c640 1 -- 172.21.15.82:0/2005310120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0d11b2210 con 0x55d0d153ab60 2024-04-21T08:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.472+0000 7ff99e59c640 1 --2- 172.21.15.82:0/2005310120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153ab60 0x55d0d1543b50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff990009900 tx=0x7ff9900039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81edcc003e5e025 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.472+0000 7ff985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.472+0000 7ff985ffb640 1 -- 172.21.15.82:0/2005310120 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990059060 con 0x55d0d153ab60 2024-04-21T08:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.472+0000 7ff985ffb640 1 -- 172.21.15.82:0/2005310120 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff99003f440 con 0x55d0d153ab60 2024-04-21T08:47:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.472+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2005310120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9900035f0 con 0x55d0d153ab60 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.472+0000 7ff985ffb640 1 -- 172.21.15.82:0/2005310120 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff990060050 con 0x55d0d153ab60 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.472+0000 7ff9a7b9f740 10 client.16062 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.472+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2005310120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0d127e620 con 0x55d0d153ab60 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.472+0000 7ff985ffb640 1 -- 172.21.15.82:0/2005310120 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9900035f0 con 0x55d0d153ab60 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.473+0000 7ff985ffb640 1 -- 172.21.15.82:0/2005310120 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff990034540 con 0x55d0d153ab60 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.473+0000 7ff9a7b9f740 10 client.16062 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.473+0000 7ff9a7b9f740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.473+0000 7ff9a7b9f740 20 client.16062 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.473+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/2005310120 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0d1553f50 0x55d0d1556340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.473+0000 7ff9a7b9f740 4 client.16062 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.473+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2005310120 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0d14b3880 con 0x55d0d1553f50 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.474+0000 7ff99dd9b640 1 --2- 172.21.15.82:0/2005310120 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0d1553f50 0x55d0d1556340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.474+0000 7ff985ffb640 10 client.16062 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.474+0000 7ff99dd9b640 1 --2- 172.21.15.82:0/2005310120 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0d1553f50 0x55d0d1556340 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4fed06cb6a561 server_cookie=9519325dfe110da7 in_seq=0 out_seq=0 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff985ffb640 1 -- 172.21.15.82:0/2005310120 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0d14b3880 con 0x55d0d1553f50 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff985ffb640 10 client.16062 handle_command_reply: tid=0 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff9a7b9f740 1 client.16062 shutdown 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2005310120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153ab60 msgr2=0x55d0d1543b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/2005310120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153ab60 0x55d0d1543b50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff990009900 tx=0x7ff9900039d0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2005310120 shutdown_connections 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/2005310120 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0d1553f50 0x55d0d1556340 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff985ffb640 0 client.16062 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/2005310120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d15440e0 0x55d0d1541620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/2005310120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153b8d0 0x55d0d1540ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/2005310120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153ab60 0x55d0d1543b50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2005310120 >> 172.21.15.82:0/2005310120 conn(0x55d0d1501c90 msgr2=0x55d0d15375a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2005310120 shutdown_connections 2024-04-21T08:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/2005310120 wait complete. 2024-04-21T08:47:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.475+0000 7ff9a7b9f740 20 client.16062 trim_cache size 0 max 16384 2024-04-21T08:47:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.504+0000 7ff9a7b9f740 1 Processor -- start 2024-04-21T08:47:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.504+0000 7ff9a7b9f740 1 -- start start 2024-04-21T08:47:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.504+0000 7ff9a7b9f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153ab60 0x55d0d154fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.504+0000 7ff9a7b9f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153b8d0 0x55d0d153fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.504+0000 7ff9a7b9f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d15440e0 0x55d0d1543610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.504+0000 7ff9a7b9f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01b15b0 con 0x55d0d153b8d0 2024-04-21T08:47:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.504+0000 7ff9a7b9f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01b30f0 con 0x55d0d153ab60 2024-04-21T08:47:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.504+0000 7ff9a7b9f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01b1c80 con 0x55d0d15440e0 2024-04-21T08:47:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff99dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153b8d0 0x55d0d153fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff99dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153b8d0 0x55d0d153fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56576/0 (socket says 172.21.15.82:56576) 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff99dd9b640 1 -- 172.21.15.82:0/73709141 learned_addr learned my addr 172.21.15.82:0/73709141 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff99e59c640 1 --2- 172.21.15.82:0/73709141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153ab60 0x55d0d154fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff99ed9d640 1 --2- 172.21.15.82:0/73709141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d15440e0 0x55d0d1543610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff99dd9b640 1 -- 172.21.15.82:0/73709141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d15440e0 msgr2=0x55d0d1543610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff99dd9b640 1 --2- 172.21.15.82:0/73709141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d15440e0 0x55d0d1543610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff99dd9b640 1 -- 172.21.15.82:0/73709141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153ab60 msgr2=0x55d0d154fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff99dd9b640 1 --2- 172.21.15.82:0/73709141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153ab60 0x55d0d154fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff99dd9b640 1 -- 172.21.15.82:0/73709141 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0d152fce0 con 0x55d0d153b8d0 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff99ed9d640 1 --2- 172.21.15.82:0/73709141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d15440e0 0x55d0d1543610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff99dd9b640 1 --2- 172.21.15.82:0/73709141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153b8d0 0x55d0d153fec0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff994005b60 tx=0x7ff99400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad629df8155a50ba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff987fff640 1 -- 172.21.15.82:0/73709141 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994022070 con 0x55d0d153b8d0 2024-04-21T08:47:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff987fff640 1 -- 172.21.15.82:0/73709141 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff99401c430 con 0x55d0d153b8d0 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.505+0000 7ff987fff640 1 -- 172.21.15.82:0/73709141 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940039e0 con 0x55d0d153b8d0 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.506+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/73709141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153b8d0 msgr2=0x55d0d153fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.506+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/73709141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153b8d0 0x55d0d153fec0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff994005b60 tx=0x7ff99400f7f0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.506+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/73709141 shutdown_connections 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.506+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/73709141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d15440e0 0x55d0d1543610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.506+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/73709141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d153b8d0 0x55d0d153fec0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.506+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/73709141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153ab60 0x55d0d154fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.506+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/73709141 >> 172.21.15.82:0/73709141 conn(0x55d0d1501c90 msgr2=0x55d0d153f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.506+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/73709141 shutdown_connections 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.506+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/73709141 wait complete. 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.506+0000 7ff9a7b9f740 1 -- start start 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff9a7b9f740 1 Processor -- start 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff9a7b9f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153ab60 0x55d0d154c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff9a7b9f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d153b8d0 0x55d0d154c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff9a7b9f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 0x55d0d1545290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff9a7b9f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01b2a20 con 0x55d0d15440e0 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff9a7b9f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01b2350 con 0x55d0d153ab60 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff9a7b9f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01af590 con 0x55d0d153b8d0 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff9a7b9f740 10 client.0 _ll_register_callbacks cb 0x7ff998098f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff99e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153ab60 0x55d0d154c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff99ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 0x55d0d1545290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff99e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153ab60 0x55d0d154c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:37202/0 (socket says 172.21.15.82:37202) 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff99e59c640 1 -- 172.21.15.82:0/1455931457 learned_addr learned my addr 172.21.15.82:0/1455931457 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff99ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 0x55d0d1545290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56586/0 (socket says 172.21.15.82:56586) 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff99ed9d640 1 -- 172.21.15.82:0/1455931457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d153b8d0 msgr2=0x55d0d154c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff99ed9d640 1 --2- 172.21.15.82:0/1455931457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d153b8d0 0x55d0d154c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff99ed9d640 1 -- 172.21.15.82:0/1455931457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153ab60 msgr2=0x55d0d154c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff99ed9d640 1 --2- 172.21.15.82:0/1455931457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153ab60 0x55d0d154c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff99ed9d640 1 -- 172.21.15.82:0/1455931457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0d1546940 con 0x55d0d15440e0 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.507+0000 7ff99e59c640 1 --2- 172.21.15.82:0/1455931457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153ab60 0x55d0d154c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff99ed9d640 1 --2- 172.21.15.82:0/1455931457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 0x55d0d1545290 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c760 tx=0x7ff99809d510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410b630720f87e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff985ffb640 1 -- 172.21.15.82:0/1455931457 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9980d1150 con 0x55d0d15440e0 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff985ffb640 1 -- 172.21.15.82:0/1455931457 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff9980a54b0 con 0x55d0d15440e0 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/1455931457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff99401c430 con 0x55d0d15440e0 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff985ffb640 1 -- 172.21.15.82:0/1455931457 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9980a56b0 con 0x55d0d15440e0 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff9a7b9f740 10 client.16068 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/1455931457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0d1539e30 con 0x55d0d15440e0 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff985ffb640 1 -- 172.21.15.82:0/1455931457 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff99401c430 con 0x55d0d15440e0 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff985ffb640 1 -- 172.21.15.82:0/1455931457 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff9980ce830 con 0x55d0d15440e0 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff9a7b9f740 10 client.16068 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff9a7b9f740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff9a7b9f740 20 client.16068 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/1455931457 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0d1553f50 0x55d0d15bcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff9a7b9f740 4 client.16068 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:47:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.508+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/1455931457 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0d143f3d0 con 0x55d0d1553f50 2024-04-21T08:47:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.509+0000 7ff99e59c640 1 --2- 172.21.15.82:0/1455931457 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0d1553f50 0x55d0d15bcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.509+0000 7ff99e59c640 1 --2- 172.21.15.82:0/1455931457 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0d1553f50 0x55d0d15bcca0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba0acb954d58a2c server_cookie=94908fc76b5f0a1a in_seq=0 out_seq=0 2024-04-21T08:47:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.509+0000 7ff985ffb640 10 client.16068 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff985ffb640 1 -- 172.21.15.82:0/1455931457 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0d143f3d0 con 0x55d0d1553f50 2024-04-21T08:47:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff985ffb640 10 client.16068 handle_command_reply: tid=0 2024-04-21T08:47:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff9a7b9f740 1 client.16068 shutdown 2024-04-21T08:47:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/1455931457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 msgr2=0x55d0d1545290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/1455931457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 0x55d0d1545290 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c760 tx=0x7ff99809d510 comp rx=0 tx=0).stop 2024-04-21T08:47:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/1455931457 shutdown_connections 2024-04-21T08:47:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/1455931457 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0d1553f50 0x55d0d15bcca0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/1455931457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d15440e0 0x55d0d1545290 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/1455931457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0d153b8d0 0x55d0d154c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff985ffb640 0 client.16068 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff9a7b9f740 1 --2- 172.21.15.82:0/1455931457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0d153ab60 0x55d0d154c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/1455931457 >> 172.21.15.82:0/1455931457 conn(0x55d0d1501c90 msgr2=0x55d0d1540040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/1455931457 shutdown_connections 2024-04-21T08:47:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff9a7b9f740 1 -- 172.21.15.82:0/1455931457 wait complete. 2024-04-21T08:47:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.513+0000 7ff9a7b9f740 20 client.16068 trim_cache size 0 max 16384 2024-04-21T08:47:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:47:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:47:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:47:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:47:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.514+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/4157171495 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff9880a3ee0 msgr2=0x7ff9880a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.514+0000 7ff9a4ed2640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff9880a3ee0 0x7ff9880a63a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff990005ab0 tx=0x7ff990005a20 comp rx=0 tx=0).stop 2024-04-21T08:47:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.514+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/4157171495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a019cfa0 msgr2=0x7ff9a019d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.514+0000 7ff9a4ed2640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a019cfa0 0x7ff9a019d450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff99800ea60 tx=0x7ff99800ef20 comp rx=0 tx=0).stop 2024-04-21T08:47:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.514+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/4157171495 shutdown_connections 2024-04-21T08:47:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.514+0000 7ff9a4ed2640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff9880a3ee0 0x7ff9880a63a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.514+0000 7ff9a4ed2640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9a019cfa0 0x7ff9a019d450 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.514+0000 7ff9a4ed2640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a0107270 0x7ff9a0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.514+0000 7ff9a4ed2640 1 --2- 172.21.15.82:0/4157171495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9a01021a0 0x7ff9a0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.514+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/4157171495 >> 172.21.15.82:0/4157171495 conn(0x7ff9a00793e0 msgr2=0x7ff9a00ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.514+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/4157171495 shutdown_connections 2024-04-21T08:47:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:36.514+0000 7ff9a4ed2640 1 -- 172.21.15.82:0/4157171495 wait complete. 2024-04-21T08:47:37.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:37 smithi134 ceph-mon[31772]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-21T08:47:38.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:37 smithi082 ceph-mon[27967]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-21T08:47:38.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:37 smithi106 ceph-mon[31072]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-04-21T08:47:38.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:47:38.905 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:47:39.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.012+0000 7f1362c3f640 1 -- 172.21.15.82:0/1836198373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135c10e8f0 msgr2=0x7f135c110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.012+0000 7f1362c3f640 1 --2- 172.21.15.82:0/1836198373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135c10e8f0 0x7f135c110ce0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1354009ca0 tx=0x7f135402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.012+0000 7f1362c3f640 1 -- 172.21.15.82:0/1836198373 shutdown_connections 2024-04-21T08:47:39.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.012+0000 7f1362c3f640 1 --2- 172.21.15.82:0/1836198373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135c10e8f0 0x7f135c110ce0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.012+0000 7f1362c3f640 1 --2- 172.21.15.82:0/1836198373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f135c101660 0x7f135c10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.012+0000 7f1362c3f640 1 --2- 172.21.15.82:0/1836198373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f135c100d40 0x7f135c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.012+0000 7f1362c3f640 1 -- 172.21.15.82:0/1836198373 >> 172.21.15.82:0/1836198373 conn(0x7f135c078000 msgr2=0x7f135c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.012+0000 7f1362c3f640 1 -- 172.21.15.82:0/1836198373 shutdown_connections 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.012+0000 7f1362c3f640 1 -- 172.21.15.82:0/1836198373 wait complete. 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.013+0000 7f1362c3f640 1 Processor -- start 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.013+0000 7f1362c3f640 1 -- start start 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.013+0000 7f1362c3f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f135c100d40 0x7f135c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.013+0000 7f1362c3f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135c101660 0x7f135c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.013+0000 7f1362c3f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f135c10e8f0 0x7f135c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.013+0000 7f1362c3f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1143b0 con 0x7f135c101660 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.013+0000 7f1362c3f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f135c114230 con 0x7f135c10e8f0 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.013+0000 7f1362c3f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f135c114530 con 0x7f135c100d40 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.013+0000 7f135bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135c101660 0x7f135c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.013+0000 7f135bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135c101660 0x7f135c1a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56604/0 (socket says 172.21.15.82:56604) 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f135bfff640 1 -- 172.21.15.82:0/3578107825 learned_addr learned my addr 172.21.15.82:0/3578107825 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f13609db640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f135c100d40 0x7f135c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f13611dc640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f135c10e8f0 0x7f135c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f135bfff640 1 -- 172.21.15.82:0/3578107825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f135c100d40 msgr2=0x7f135c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f135bfff640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f135c100d40 0x7f135c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f135bfff640 1 -- 172.21.15.82:0/3578107825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f135c10e8f0 msgr2=0x7f135c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f135bfff640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f135c10e8f0 0x7f135c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f135bfff640 1 -- 172.21.15.82:0/3578107825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c19aff0 con 0x7f135c101660 2024-04-21T08:47:39.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f13611dc640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f135c10e8f0 0x7f135c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:39.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f135bfff640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135c101660 0x7f135c1a0bb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f134800d950 tx=0x7f134800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f1359ffb640 1 -- 172.21.15.82:0/3578107825 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348017070 con 0x7f135c101660 2024-04-21T08:47:39.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f1359ffb640 1 -- 172.21.15.82:0/3578107825 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1348009c30 con 0x7f135c101660 2024-04-21T08:47:39.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f1362c3f640 1 -- 172.21.15.82:0/3578107825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135c19b2e0 con 0x7f135c101660 2024-04-21T08:47:39.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.014+0000 7f1359ffb640 1 -- 172.21.15.82:0/3578107825 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134800f620 con 0x7f135c101660 2024-04-21T08:47:39.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.015+0000 7f1362c3f640 1 -- 172.21.15.82:0/3578107825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135c1a75f0 con 0x7f135c101660 2024-04-21T08:47:39.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.017+0000 7f1359ffb640 1 -- 172.21.15.82:0/3578107825 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f1348010460 con 0x7f135c101660 2024-04-21T08:47:39.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.017+0000 7f1359ffb640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f134c0a3e40 0x7f134c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.018+0000 7f13609db640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f134c0a3e40 0x7f134c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.018+0000 7f1359ffb640 1 -- 172.21.15.82:0/3578107825 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f13480d0390 con 0x7f135c101660 2024-04-21T08:47:39.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.018+0000 7f13609db640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f134c0a3e40 0x7f134c0a6300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1350009bd0 tx=0x7f1350005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f136582f740 1 Processor -- start 2024-04-21T08:47:39.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f136582f740 1 -- start start 2024-04-21T08:47:39.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f136582f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb74a580 0x562abb74a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f136582f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb753540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f136582f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb753a80 0x562abb755e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f136582f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562abb6af1e0 con 0x562abb74aea0 2024-04-21T08:47:39.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f136582f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562abb64c390 con 0x562abb74a580 2024-04-21T08:47:39.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f136582f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562abb6ab1a0 con 0x562abb753a80 2024-04-21T08:47:39.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f135bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb753540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f135bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb753540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56612/0 (socket says 172.21.15.82:56612) 2024-04-21T08:47:39.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f135bfff640 1 -- 172.21.15.82:0/3942380616 learned_addr learned my addr 172.21.15.82:0/3942380616 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f13611dc640 1 --2- 172.21.15.82:0/3942380616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb753a80 0x562abb755e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f13609db640 1 --2- 172.21.15.82:0/3942380616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb74a580 0x562abb74a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f135bfff640 1 -- 172.21.15.82:0/3942380616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb753a80 msgr2=0x562abb755e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f135bfff640 1 --2- 172.21.15.82:0/3942380616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb753a80 0x562abb755e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f135bfff640 1 -- 172.21.15.82:0/3942380616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb74a580 msgr2=0x562abb74a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f135bfff640 1 --2- 172.21.15.82:0/3942380616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb74a580 0x562abb74a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f135bfff640 1 -- 172.21.15.82:0/3942380616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abb48b7d0 con 0x562abb74aea0 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.019+0000 7f13609db640 1 --2- 172.21.15.82:0/3942380616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb74a580 0x562abb74a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.020+0000 7f135bfff640 1 --2- 172.21.15.82:0/3942380616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb753540 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f13480986f0 tx=0x7f134809c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836990c5019ab8b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.020+0000 7f1346ffd640 1 -- 172.21.15.82:0/3942380616 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348020050 con 0x562abb74aea0 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.020+0000 7f1346ffd640 1 -- 172.21.15.82:0/3942380616 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13480a00d0 con 0x562abb74aea0 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.020+0000 7f1346ffd640 1 -- 172.21.15.82:0/3942380616 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480bc0b0 con 0x562abb74aea0 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.021+0000 7f136582f740 1 -- 172.21.15.82:0/3942380616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 msgr2=0x562abb753540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.021+0000 7f136582f740 1 --2- 172.21.15.82:0/3942380616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb753540 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f13480986f0 tx=0x7f134809c560 comp rx=0 tx=0).stop 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.021+0000 7f136582f740 1 -- 172.21.15.82:0/3942380616 shutdown_connections 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.021+0000 7f136582f740 1 --2- 172.21.15.82:0/3942380616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb753a80 0x562abb755e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.021+0000 7f136582f740 1 --2- 172.21.15.82:0/3942380616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb753540 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.021+0000 7f136582f740 1 --2- 172.21.15.82:0/3942380616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb74a580 0x562abb74a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.021+0000 7f136582f740 1 -- 172.21.15.82:0/3942380616 >> 172.21.15.82:0/3942380616 conn(0x562abb744cb0 msgr2=0x562abb745090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.021+0000 7f136582f740 1 -- 172.21.15.82:0/3942380616 shutdown_connections 2024-04-21T08:47:39.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.021+0000 7f136582f740 1 -- 172.21.15.82:0/3942380616 wait complete. 2024-04-21T08:47:39.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.021+0000 7f136582f740 1 -- start start 2024-04-21T08:47:39.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.021+0000 7f136582f740 1 Processor -- start 2024-04-21T08:47:39.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f136582f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb74a580 0x562abb750a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f136582f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb750fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f135bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb750fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f135bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb750fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56614/0 (socket says 172.21.15.82:56614) 2024-04-21T08:47:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f135bfff640 1 -- 172.21.15.82:0/598925273 learned_addr learned my addr 172.21.15.82:0/598925273 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f136582f740 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb753a80 0x562abb7514e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f136582f740 1 -- 172.21.15.82:0/598925273 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562abb756810 con 0x562abb74aea0 2024-04-21T08:47:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f136582f740 1 -- 172.21.15.82:0/598925273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562abb6682f0 con 0x562abb753a80 2024-04-21T08:47:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f13609db640 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb74a580 0x562abb750a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f136582f740 1 -- 172.21.15.82:0/598925273 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562abb756990 con 0x562abb74a580 2024-04-21T08:47:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f136582f740 10 client.0 _ll_register_callbacks cb 0x562abb675580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f13611dc640 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb753a80 0x562abb7514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f13609db640 1 -- 172.21.15.82:0/598925273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb753a80 msgr2=0x562abb7514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.022+0000 7f13609db640 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb753a80 0x562abb7514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.023+0000 7f13609db640 1 -- 172.21.15.82:0/598925273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 msgr2=0x562abb750fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.023+0000 7f13609db640 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb750fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.023+0000 7f13609db640 1 -- 172.21.15.82:0/598925273 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abb3c13f0 con 0x562abb74a580 2024-04-21T08:47:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.023+0000 7f135bfff640 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb750fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.023+0000 7f13609db640 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb74a580 0x562abb750a60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1350003970 tx=0x7f1350004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=707c677bacebd7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.023+0000 7f133bfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:47:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.023+0000 7f133bfff640 1 -- 172.21.15.82:0/598925273 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135001dd40 con 0x562abb74a580 2024-04-21T08:47:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.023+0000 7f136582f740 1 -- 172.21.15.82:0/598925273 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13480a00d0 con 0x562abb74a580 2024-04-21T08:47:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.023+0000 7f133bfff640 1 -- 172.21.15.82:0/598925273 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1350005020 con 0x562abb74a580 2024-04-21T08:47:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.023+0000 7f133bfff640 1 -- 172.21.15.82:0/598925273 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1350005220 con 0x562abb74a580 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.023+0000 7f136582f740 10 client.25309 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.023+0000 7f136582f740 1 -- 172.21.15.82:0/598925273 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562abb3810f0 con 0x562abb74a580 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.024+0000 7f133bfff640 1 -- 172.21.15.82:0/598925273 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13480a00d0 con 0x562abb74a580 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.024+0000 7f133bfff640 1 -- 172.21.15.82:0/598925273 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f135001e420 con 0x562abb74a580 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.024+0000 7f136582f740 10 client.25309 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.024+0000 7f136582f740 10 client.25309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.024+0000 7f136582f740 20 client.25309 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.024+0000 7f136582f740 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562abb7639c0 0x562abb765e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.024+0000 7f136582f740 4 client.25309 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.024+0000 7f136582f740 1 -- 172.21.15.82:0/598925273 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562abb6bcc00 con 0x562abb7639c0 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.024+0000 7f135bfff640 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562abb7639c0 0x562abb765e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.025+0000 7f133bfff640 10 client.25309 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.025+0000 7f135bfff640 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562abb7639c0 0x562abb765e30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f1398d502e094b1 server_cookie=6984e234802bf528 in_seq=0 out_seq=0 2024-04-21T08:47:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.025+0000 7f133bfff640 1 -- 172.21.15.82:0/598925273 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562abb6bcc00 con 0x562abb7639c0 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.025+0000 7f133bfff640 10 client.25309 handle_command_reply: tid=0 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.025+0000 7f136582f740 1 client.25309 shutdown 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.026+0000 7f136582f740 1 -- 172.21.15.82:0/598925273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb74a580 msgr2=0x562abb750a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.026+0000 7f136582f740 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb74a580 0x562abb750a60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1350003970 tx=0x7f1350004730 comp rx=0 tx=0).stop 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.026+0000 7f136582f740 1 -- 172.21.15.82:0/598925273 shutdown_connections 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.026+0000 7f136582f740 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562abb7639c0 0x562abb765e30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.026+0000 7f136582f740 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb753a80 0x562abb7514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.026+0000 7f133bfff640 0 client.25309 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.026+0000 7f136582f740 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb750fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.026+0000 7f136582f740 1 --2- 172.21.15.82:0/598925273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb74a580 0x562abb750a60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.026+0000 7f136582f740 1 -- 172.21.15.82:0/598925273 >> 172.21.15.82:0/598925273 conn(0x562abb744cb0 msgr2=0x562abb746a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.026+0000 7f136582f740 1 -- 172.21.15.82:0/598925273 shutdown_connections 2024-04-21T08:47:39.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.026+0000 7f136582f740 1 -- 172.21.15.82:0/598925273 wait complete. 2024-04-21T08:47:39.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.026+0000 7f136582f740 20 client.25309 trim_cache size 0 max 16384 2024-04-21T08:47:39.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.060+0000 7f136582f740 1 Processor -- start 2024-04-21T08:47:39.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.060+0000 7f136582f740 1 -- start start 2024-04-21T08:47:39.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.060+0000 7f136582f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb74a580 0x562abb753190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.060+0000 7f136582f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb75f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.060+0000 7f136582f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb753a80 0x562abb76b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.060+0000 7f136582f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1b1670 con 0x562abb74aea0 2024-04-21T08:47:39.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.060+0000 7f136582f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1b31b0 con 0x562abb753a80 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.060+0000 7f136582f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1b1d40 con 0x562abb74a580 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.060+0000 7f135bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb75f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.060+0000 7f135bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb75f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56630/0 (socket says 172.21.15.82:56630) 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.060+0000 7f135bfff640 1 -- 172.21.15.82:0/735043501 learned_addr learned my addr 172.21.15.82:0/735043501 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.061+0000 7f13609db640 1 --2- 172.21.15.82:0/735043501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb74a580 0x562abb753190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.061+0000 7f13611dc640 1 --2- 172.21.15.82:0/735043501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb753a80 0x562abb76b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.061+0000 7f135bfff640 1 -- 172.21.15.82:0/735043501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb74a580 msgr2=0x562abb753190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.061+0000 7f135bfff640 1 --2- 172.21.15.82:0/735043501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb74a580 0x562abb753190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.061+0000 7f135bfff640 1 -- 172.21.15.82:0/735043501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb753a80 msgr2=0x562abb76b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.061+0000 7f135bfff640 1 --2- 172.21.15.82:0/735043501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb753a80 0x562abb76b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.061+0000 7f135bfff640 1 -- 172.21.15.82:0/735043501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abb755e70 con 0x562abb74aea0 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.061+0000 7f135bfff640 1 --2- 172.21.15.82:0/735043501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb75f700 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f134809c5d0 tx=0x7f13480bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a551decf2eb10b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.061+0000 7f13467fc640 1 -- 172.21.15.82:0/735043501 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480b5070 con 0x562abb74aea0 2024-04-21T08:47:39.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.061+0000 7f13467fc640 1 -- 172.21.15.82:0/735043501 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13480a00d0 con 0x562abb74aea0 2024-04-21T08:47:39.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.061+0000 7f13467fc640 1 -- 172.21.15.82:0/735043501 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348020040 con 0x562abb74aea0 2024-04-21T08:47:39.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.062+0000 7f136582f740 1 -- 172.21.15.82:0/735043501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 msgr2=0x562abb75f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.062+0000 7f136582f740 1 --2- 172.21.15.82:0/735043501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb75f700 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f134809c5d0 tx=0x7f13480bc040 comp rx=0 tx=0).stop 2024-04-21T08:47:39.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.062+0000 7f136582f740 1 -- 172.21.15.82:0/735043501 shutdown_connections 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.062+0000 7f136582f740 1 --2- 172.21.15.82:0/735043501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb753a80 0x562abb76b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.062+0000 7f136582f740 1 --2- 172.21.15.82:0/735043501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74aea0 0x562abb75f700 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.062+0000 7f136582f740 1 --2- 172.21.15.82:0/735043501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb74a580 0x562abb753190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.062+0000 7f136582f740 1 -- 172.21.15.82:0/735043501 >> 172.21.15.82:0/735043501 conn(0x562abb744cb0 msgr2=0x562abb74f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.062+0000 7f136582f740 1 -- 172.21.15.82:0/735043501 shutdown_connections 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.062+0000 7f136582f740 1 -- 172.21.15.82:0/735043501 wait complete. 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.062+0000 7f136582f740 1 -- start start 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.062+0000 7f136582f740 1 Processor -- start 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.063+0000 7f136582f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74a580 0x562abb75bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.063+0000 7f136582f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb74aea0 0x562abb75c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.063+0000 7f136582f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb753a80 0x562abb74ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.063+0000 7f136582f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1b2ae0 con 0x562abb74a580 2024-04-21T08:47:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.063+0000 7f136582f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1b2410 con 0x562abb74aea0 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.063+0000 7f136582f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1af650 con 0x562abb753a80 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.063+0000 7f13609db640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74a580 0x562abb75bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.063+0000 7f136582f740 10 client.0 _ll_register_callbacks cb 0x7f1350009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.063+0000 7f13609db640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74a580 0x562abb75bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56646/0 (socket says 172.21.15.82:56646) 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.063+0000 7f13609db640 1 -- 172.21.15.82:0/2434503990 learned_addr learned my addr 172.21.15.82:0/2434503990 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.063+0000 7f135bfff640 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb74aea0 0x562abb75c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.063+0000 7f13611dc640 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb753a80 0x562abb74ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f13609db640 1 -- 172.21.15.82:0/2434503990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb753a80 msgr2=0x562abb74ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f13609db640 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb753a80 0x562abb74ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f13609db640 1 -- 172.21.15.82:0/2434503990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb74aea0 msgr2=0x562abb75c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f13609db640 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb74aea0 0x562abb75c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f13609db640 1 -- 172.21.15.82:0/2434503990 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abb760230 con 0x562abb74a580 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f13609db640 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74a580 0x562abb75bc10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1350003940 tx=0x7f135001d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49283310b7fbbc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f133bfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f133bfff640 1 -- 172.21.15.82:0/2434503990 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350039070 con 0x562abb74a580 2024-04-21T08:47:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f133bfff640 1 -- 172.21.15.82:0/2434503990 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f13500055d0 con 0x562abb74a580 2024-04-21T08:47:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f136582f740 1 -- 172.21.15.82:0/2434503990 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13480a00d0 con 0x562abb74a580 2024-04-21T08:47:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f133bfff640 1 -- 172.21.15.82:0/2434503990 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f135001db40 con 0x562abb74a580 2024-04-21T08:47:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f136582f740 10 client.16086 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.064+0000 7f136582f740 1 -- 172.21.15.82:0/2434503990 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562abb7c9800 con 0x562abb74a580 2024-04-21T08:47:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.065+0000 7f133bfff640 1 -- 172.21.15.82:0/2434503990 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13480a00d0 con 0x562abb74a580 2024-04-21T08:47:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.065+0000 7f133bfff640 1 -- 172.21.15.82:0/2434503990 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1350004c60 con 0x562abb74a580 2024-04-21T08:47:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.065+0000 7f136582f740 10 client.16086 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.065+0000 7f136582f740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.065+0000 7f136582f740 20 client.16086 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.065+0000 7f136582f740 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562abb7639c0 0x562abb7cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.065+0000 7f136582f740 4 client.16086 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.065+0000 7f136582f740 1 -- 172.21.15.82:0/2434503990 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562abb6b6a70 con 0x562abb7639c0 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.065+0000 7f135bfff640 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562abb7639c0 0x562abb7cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f133bfff640 10 client.16086 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f135bfff640 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562abb7639c0 0x562abb7cbfc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afcd450e5baf2336 server_cookie=1eacc8c0c8ad2be8 in_seq=0 out_seq=0 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f133bfff640 1 -- 172.21.15.82:0/2434503990 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562abb6b6a70 con 0x562abb7639c0 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f133bfff640 10 client.16086 handle_command_reply: tid=0 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f136582f740 1 client.16086 shutdown 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f136582f740 1 -- 172.21.15.82:0/2434503990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74a580 msgr2=0x562abb75bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f136582f740 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74a580 0x562abb75bc10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1350003940 tx=0x7f135001d8c0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f136582f740 1 -- 172.21.15.82:0/2434503990 shutdown_connections 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f136582f740 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562abb7639c0 0x562abb7cbfc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f133bfff640 0 client.16086 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f136582f740 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562abb753a80 0x562abb74ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f136582f740 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562abb74aea0 0x562abb75c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f136582f740 1 --2- 172.21.15.82:0/2434503990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562abb74a580 0x562abb75bc10 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f136582f740 1 -- 172.21.15.82:0/2434503990 >> 172.21.15.82:0/2434503990 conn(0x562abb744cb0 msgr2=0x562abb7434e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f136582f740 1 -- 172.21.15.82:0/2434503990 shutdown_connections 2024-04-21T08:47:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f136582f740 1 -- 172.21.15.82:0/2434503990 wait complete. 2024-04-21T08:47:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.066+0000 7f136582f740 20 client.16086 trim_cache size 0 max 16384 2024-04-21T08:47:39.072 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:47:39.072 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:47:39.073 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:47:39.073 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:47:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.067+0000 7f1362c3f640 1 -- 172.21.15.82:0/3578107825 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f134c0a3e40 msgr2=0x7f134c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.067+0000 7f1362c3f640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f134c0a3e40 0x7f134c0a6300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1350009bd0 tx=0x7f1350005f90 comp rx=0 tx=0).stop 2024-04-21T08:47:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.067+0000 7f1362c3f640 1 -- 172.21.15.82:0/3578107825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135c101660 msgr2=0x7f135c1a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.067+0000 7f1362c3f640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135c101660 0x7f135c1a0bb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f134800d950 tx=0x7f134800de10 comp rx=0 tx=0).stop 2024-04-21T08:47:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.067+0000 7f1362c3f640 1 -- 172.21.15.82:0/3578107825 shutdown_connections 2024-04-21T08:47:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.067+0000 7f1362c3f640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f135c10e8f0 0x7f135c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.067+0000 7f1362c3f640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f134c0a3e40 0x7f134c0a6300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.067+0000 7f1362c3f640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135c101660 0x7f135c1a0bb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.067+0000 7f1362c3f640 1 --2- 172.21.15.82:0/3578107825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f135c100d40 0x7f135c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.067+0000 7f1362c3f640 1 -- 172.21.15.82:0/3578107825 >> 172.21.15.82:0/3578107825 conn(0x7f135c078000 msgr2=0x7f135c10fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.067+0000 7f1362c3f640 1 -- 172.21.15.82:0/3578107825 shutdown_connections 2024-04-21T08:47:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.067+0000 7f1362c3f640 1 -- 172.21.15.82:0/3578107825 wait complete. 2024-04-21T08:47:39.079 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:47:39.079 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:47:39.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.177+0000 7fb9b249a640 1 -- 172.21.15.82:0/1197851433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9ac069aa0 msgr2=0x7fb9ac10ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.177+0000 7fb9b249a640 1 --2- 172.21.15.82:0/1197851433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9ac069aa0 0x7fb9ac10ede0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009ca0 tx=0x7fb9a402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.178+0000 7fb9b249a640 1 -- 172.21.15.82:0/1197851433 shutdown_connections 2024-04-21T08:47:39.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.178+0000 7fb9b249a640 1 --2- 172.21.15.82:0/1197851433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9ac069aa0 0x7fb9ac10ede0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.178+0000 7fb9b249a640 1 --2- 172.21.15.82:0/1197851433 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9ac0690e0 0x7fb9ac069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.178+0000 7fb9b249a640 1 --2- 172.21.15.82:0/1197851433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac106f50 0x7fb9ac107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.178+0000 7fb9b249a640 1 -- 172.21.15.82:0/1197851433 >> 172.21.15.82:0/1197851433 conn(0x7fb9ac077e30 msgr2=0x7fb9ac078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.178+0000 7fb9b249a640 1 -- 172.21.15.82:0/1197851433 shutdown_connections 2024-04-21T08:47:39.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.178+0000 7fb9b249a640 1 -- 172.21.15.82:0/1197851433 wait complete. 2024-04-21T08:47:39.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.178+0000 7fb9b249a640 1 Processor -- start 2024-04-21T08:47:39.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.178+0000 7fb9b249a640 1 -- start start 2024-04-21T08:47:39.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b249a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9ac0690e0 0x7fb9ac198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b249a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac069aa0 0x7fb9ac198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b249a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9ac106f50 0x7fb9ac19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b249a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac112150 con 0x7fb9ac106f50 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b249a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac111fd0 con 0x7fb9ac0690e0 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b249a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac1122d0 con 0x7fb9ac069aa0 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b0a37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9ac106f50 0x7fb9ac19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b0a37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9ac106f50 0x7fb9ac19ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56676/0 (socket says 172.21.15.82:56676) 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b0a37640 1 -- 172.21.15.82:0/993289825 learned_addr learned my addr 172.21.15.82:0/993289825 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9abfff640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9ac0690e0 0x7fb9ac198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9ab7fe640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac069aa0 0x7fb9ac198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b0a37640 1 -- 172.21.15.82:0/993289825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac069aa0 msgr2=0x7fb9ac198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b0a37640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac069aa0 0x7fb9ac198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b0a37640 1 -- 172.21.15.82:0/993289825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9ac0690e0 msgr2=0x7fb9ac198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b0a37640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9ac0690e0 0x7fb9ac198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.179+0000 7fb9b0a37640 1 -- 172.21.15.82:0/993289825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac19d540 con 0x7fb9ac106f50 2024-04-21T08:47:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.180+0000 7fb9b0a37640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9ac106f50 0x7fb9ac19ce10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb9a402c700 tx=0x7fb9a40346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.180+0000 7fb9a97fa640 1 -- 172.21.15.82:0/993289825 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4041020 con 0x7fb9ac106f50 2024-04-21T08:47:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.180+0000 7fb9b249a640 1 -- 172.21.15.82:0/993289825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ac19d7d0 con 0x7fb9ac106f50 2024-04-21T08:47:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.180+0000 7fb9a97fa640 1 -- 172.21.15.82:0/993289825 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9a4032960 con 0x7fb9ac106f50 2024-04-21T08:47:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.180+0000 7fb9a97fa640 1 -- 172.21.15.82:0/993289825 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4039040 con 0x7fb9ac106f50 2024-04-21T08:47:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.180+0000 7fb9b249a640 1 -- 172.21.15.82:0/993289825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ac1a50c0 con 0x7fb9ac106f50 2024-04-21T08:47:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.181+0000 7fb9b249a640 1 -- 172.21.15.82:0/993289825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9ac068de0 con 0x7fb9ac106f50 2024-04-21T08:47:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.183+0000 7fb9a97fa640 1 -- 172.21.15.82:0/993289825 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fb9a4044020 con 0x7fb9ac106f50 2024-04-21T08:47:39.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.184+0000 7fb9a97fa640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb9940a3e40 0x7fb9940a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.184+0000 7fb9abfff640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb9940a3e40 0x7fb9940a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.184+0000 7fb9a97fa640 1 -- 172.21.15.82:0/993289825 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fb9a40f3020 con 0x7fb9ac106f50 2024-04-21T08:47:39.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.184+0000 7fb9abfff640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb9940a3e40 0x7fb9940a6300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb99c005fd0 tx=0x7fb99c00a400 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.191+0000 7fb9a97fa640 1 -- 172.21.15.82:0/993289825 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9a404b020 con 0x7fb9ac106f50 2024-04-21T08:47:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.549+0000 7fb9b249a640 1 -- 172.21.15.82:0/993289825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9ac10a4d0 con 0x7fb9ac106f50 2024-04-21T08:47:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.550+0000 7fb9a97fa640 1 -- 172.21.15.82:0/993289825 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7fb9a40bf2e0 con 0x7fb9ac106f50 2024-04-21T08:47:39.551 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:47:39.551 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:46:59.334314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:47:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 28 2024-04-21T08:47:39.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.552+0000 7fb9b249a640 1 -- 172.21.15.82:0/993289825 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb9940a3e40 msgr2=0x7fb9940a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.552+0000 7fb9b249a640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb9940a3e40 0x7fb9940a6300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb99c005fd0 tx=0x7fb99c00a400 comp rx=0 tx=0).stop 2024-04-21T08:47:39.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.552+0000 7fb9b249a640 1 -- 172.21.15.82:0/993289825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9ac106f50 msgr2=0x7fb9ac19ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.552+0000 7fb9b249a640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9ac106f50 0x7fb9ac19ce10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb9a402c700 tx=0x7fb9a40346d0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.553+0000 7fb9b249a640 1 -- 172.21.15.82:0/993289825 shutdown_connections 2024-04-21T08:47:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.553+0000 7fb9b249a640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb9940a3e40 0x7fb9940a6300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.553+0000 7fb9b249a640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9ac106f50 0x7fb9ac19ce10 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.553+0000 7fb9b249a640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9ac069aa0 0x7fb9ac198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.553+0000 7fb9b249a640 1 --2- 172.21.15.82:0/993289825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9ac0690e0 0x7fb9ac198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.553+0000 7fb9b249a640 1 -- 172.21.15.82:0/993289825 >> 172.21.15.82:0/993289825 conn(0x7fb9ac077e30 msgr2=0x7fb9ac0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.553+0000 7fb9b249a640 1 -- 172.21.15.82:0/993289825 shutdown_connections 2024-04-21T08:47:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.553+0000 7fb9b249a640 1 -- 172.21.15.82:0/993289825 wait complete. 2024-04-21T08:47:39.565 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:47:39.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.666+0000 7fc96a2ce640 1 -- 172.21.15.82:0/437820827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc964101a40 msgr2=0x7fc964105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.666+0000 7fc96a2ce640 1 --2- 172.21.15.82:0/437820827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc964101a40 0x7fc964105a80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc958009900 tx=0x7fc95802bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:39.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.667+0000 7fc96a2ce640 1 -- 172.21.15.82:0/437820827 shutdown_connections 2024-04-21T08:47:39.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.667+0000 7fc96a2ce640 1 --2- 172.21.15.82:0/437820827 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc964106160 0x7fc96410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.667+0000 7fc96a2ce640 1 --2- 172.21.15.82:0/437820827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc964101a40 0x7fc964105a80 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.667+0000 7fc96a2ce640 1 --2- 172.21.15.82:0/437820827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc964101090 0x7fc964101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.667+0000 7fc96a2ce640 1 -- 172.21.15.82:0/437820827 >> 172.21.15.82:0/437820827 conn(0x7fc964077f50 msgr2=0x7fc964078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.667+0000 7fc96a2ce640 1 -- 172.21.15.82:0/437820827 shutdown_connections 2024-04-21T08:47:39.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.667+0000 7fc96a2ce640 1 -- 172.21.15.82:0/437820827 wait complete. 2024-04-21T08:47:39.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.667+0000 7fc96a2ce640 1 Processor -- start 2024-04-21T08:47:39.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.667+0000 7fc96a2ce640 1 -- start start 2024-04-21T08:47:39.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc96a2ce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc964101090 0x7fc964197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc96a2ce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc964106160 0x7fc964197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc96a2ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc96419be80 0x7fc96419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc96a2ce640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641110d0 con 0x7fc964101090 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc96a2ce640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc964110f50 con 0x7fc964106160 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc96a2ce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc964111250 con 0x7fc96419be80 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc963fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc964101090 0x7fc964197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc963fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc964101090 0x7fc964197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56692/0 (socket says 172.21.15.82:56692) 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc963fff640 1 -- 172.21.15.82:0/3042483527 learned_addr learned my addr 172.21.15.82:0/3042483527 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc9637fe640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc964106160 0x7fc964197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc96886b640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc96419be80 0x7fc96419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc963fff640 1 -- 172.21.15.82:0/3042483527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc96419be80 msgr2=0x7fc96419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc963fff640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc96419be80 0x7fc96419c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc963fff640 1 -- 172.21.15.82:0/3042483527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc964106160 msgr2=0x7fc964197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc963fff640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc964106160 0x7fc964197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.668+0000 7fc963fff640 1 -- 172.21.15.82:0/3042483527 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96419c970 con 0x7fc964101090 2024-04-21T08:47:39.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.669+0000 7fc963fff640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc964101090 0x7fc964197610 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc95400c920 tx=0x7fc95400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.669+0000 7fc9617fa640 1 -- 172.21.15.82:0/3042483527 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954007c70 con 0x7fc964101090 2024-04-21T08:47:39.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.669+0000 7fc9617fa640 1 -- 172.21.15.82:0/3042483527 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc954007e10 con 0x7fc964101090 2024-04-21T08:47:39.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.669+0000 7fc96a2ce640 1 -- 172.21.15.82:0/3042483527 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96419cc60 con 0x7fc964101090 2024-04-21T08:47:39.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.669+0000 7fc9617fa640 1 -- 172.21.15.82:0/3042483527 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954018820 con 0x7fc964101090 2024-04-21T08:47:39.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.669+0000 7fc96a2ce640 1 -- 172.21.15.82:0/3042483527 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9640693d0 con 0x7fc964101090 2024-04-21T08:47:39.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.672+0000 7fc9617fa640 1 -- 172.21.15.82:0/3042483527 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc95400f450 con 0x7fc964101090 2024-04-21T08:47:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.672+0000 7fc96cf95740 1 Processor -- start 2024-04-21T08:47:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.672+0000 7fc96cf95740 1 -- start start 2024-04-21T08:47:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96cf95740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96cf95740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e29e80 0x55a672e32430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96cf95740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e32a00 0x55a672e34e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96cf95740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a672d8e1f0 con 0x55a672e294d0 2024-04-21T08:47:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96cf95740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a672d2b4b0 con 0x55a672e32a00 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96cf95740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a672d8a1b0 con 0x55a672e29e80 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96886b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96886b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e298b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56696/0 (socket says 172.21.15.82:56696) 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96886b640 1 -- 172.21.15.82:0/3336363726 learned_addr learned my addr 172.21.15.82:0/3336363726 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc963fff640 1 --2- 172.21.15.82:0/3336363726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e29e80 0x55a672e32430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc9617fa640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc94c0a3e40 0x7fc94c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc9637fe640 1 --2- 172.21.15.82:0/3336363726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e32a00 0x55a672e34e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96886b640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc94c0a3e40 0x7fc94c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96886b640 1 -- 172.21.15.82:0/3336363726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e29e80 msgr2=0x55a672e32430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96886b640 1 --2- 172.21.15.82:0/3336363726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e29e80 0x55a672e32430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96886b640 1 -- 172.21.15.82:0/3336363726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e32a00 msgr2=0x55a672e34e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96886b640 1 --2- 172.21.15.82:0/3336363726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e32a00 0x55a672e34e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc96886b640 1 -- 172.21.15.82:0/3336363726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a672d1d1b0 con 0x55a672e294d0 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc9637fe640 1 --2- 172.21.15.82:0/3336363726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e32a00 0x55a672e34e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.673+0000 7fc9617fa640 1 -- 172.21.15.82:0/3042483527 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc9540d0f90 con 0x7fc964101090 2024-04-21T08:47:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.674+0000 7fc96886b640 1 --2- 172.21.15.82:0/3336363726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e298b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc95c013840 tx=0x7fc95c013d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fc3c76b5dacc1e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.674+0000 7fc94a7fc640 1 -- 172.21.15.82:0/3336363726 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c013250 con 0x55a672e294d0 2024-04-21T08:47:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.674+0000 7fc94a7fc640 1 -- 172.21.15.82:0/3336363726 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc95c003e20 con 0x55a672e294d0 2024-04-21T08:47:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.674+0000 7fc94a7fc640 1 -- 172.21.15.82:0/3336363726 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c0047d0 con 0x55a672e294d0 2024-04-21T08:47:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.674+0000 7fc96cf95740 1 -- 172.21.15.82:0/3336363726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 msgr2=0x55a672e298b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.674+0000 7fc96cf95740 1 --2- 172.21.15.82:0/3336363726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e298b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc95c013840 tx=0x7fc95c013d00 comp rx=0 tx=0).stop 2024-04-21T08:47:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.674+0000 7fc96886b640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc94c0a3e40 0x7fc94c0a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc95c020000 tx=0x7fc95c01e000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.675+0000 7fc96cf95740 1 -- 172.21.15.82:0/3336363726 shutdown_connections 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.675+0000 7fc96cf95740 1 --2- 172.21.15.82:0/3336363726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e32a00 0x55a672e34e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.675+0000 7fc96cf95740 1 --2- 172.21.15.82:0/3336363726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e29e80 0x55a672e32430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.675+0000 7fc96cf95740 1 --2- 172.21.15.82:0/3336363726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e298b0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.675+0000 7fc96cf95740 1 -- 172.21.15.82:0/3336363726 >> 172.21.15.82:0/3336363726 conn(0x55a672e23ba0 msgr2=0x55a672e23f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.675+0000 7fc96cf95740 1 -- 172.21.15.82:0/3336363726 shutdown_connections 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.675+0000 7fc96cf95740 1 -- 172.21.15.82:0/3336363726 wait complete. 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.675+0000 7fc96cf95740 1 -- start start 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.675+0000 7fc96cf95740 1 Processor -- start 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc96cf95740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e32360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc96cf95740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e29e80 0x55a672e2f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc96cf95740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e32a00 0x55a672e2fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc96cf95740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a672e35970 con 0x55a672e294d0 2024-04-21T08:47:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc963fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e32360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc963fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e32360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56712/0 (socket says 172.21.15.82:56712) 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc963fff640 1 -- 172.21.15.82:0/3661222948 learned_addr learned my addr 172.21.15.82:0/3661222948 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc96cf95740 1 -- 172.21.15.82:0/3661222948 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a672d47500 con 0x55a672e32a00 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc96cf95740 1 -- 172.21.15.82:0/3661222948 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a672e35af0 con 0x55a672e29e80 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc96886b640 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e32a00 0x55a672e2fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc96cf95740 10 client.0 _ll_register_callbacks cb 0x55a672d15b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc9637fe640 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e29e80 0x55a672e2f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc963fff640 1 -- 172.21.15.82:0/3661222948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e29e80 msgr2=0x55a672e2f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc963fff640 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e29e80 0x55a672e2f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc963fff640 1 -- 172.21.15.82:0/3661222948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e32a00 msgr2=0x55a672e2fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc963fff640 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e32a00 0x55a672e2fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc963fff640 1 -- 172.21.15.82:0/3661222948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a672aa03f0 con 0x55a672e294d0 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc9637fe640 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e29e80 0x55a672e2f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc96886b640 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e32a00 0x55a672e2fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:39.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc963fff640 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e32360 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc95409c620 tx=0x7fc95400fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388ffdb888757e26 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.676+0000 7fc9437fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc9437fe640 1 -- 172.21.15.82:0/3661222948 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954085e60 con 0x55a672e294d0 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc9437fe640 1 -- 172.21.15.82:0/3661222948 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc9540d3110 con 0x55a672e294d0 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc96cf95740 1 -- 172.21.15.82:0/3661222948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc95c003e20 con 0x55a672e294d0 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc9437fe640 1 -- 172.21.15.82:0/3661222948 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc95409c920 con 0x55a672e294d0 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc96cf95740 10 client.16104 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc96cf95740 1 -- 172.21.15.82:0/3661222948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a672b6a7d0 con 0x55a672e294d0 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc9437fe640 1 -- 172.21.15.82:0/3661222948 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9540a0cd0 con 0x55a672e294d0 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc9437fe640 1 -- 172.21.15.82:0/3661222948 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc9540a5190 con 0x55a672e294d0 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc96cf95740 10 client.16104 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc96cf95740 10 client.16104 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc96cf95740 20 client.16104 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc96cf95740 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a672e42cb0 0x55a672e450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc96cf95740 4 client.16104 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:39.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.677+0000 7fc96cf95740 1 -- 172.21.15.82:0/3661222948 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc95c003e20 con 0x55a672e42cb0 2024-04-21T08:47:39.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.678+0000 7fc9637fe640 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a672e42cb0 0x55a672e450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.678+0000 7fc9437fe640 10 client.16104 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:47:39.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.678+0000 7fc9637fe640 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a672e42cb0 0x55a672e450a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8683dec15d1d7c4f server_cookie=7ab9cbfc8d18f29e in_seq=0 out_seq=0 2024-04-21T08:47:39.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.679+0000 7fc9437fe640 1 -- 172.21.15.82:0/3661222948 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fc95c003e20 con 0x55a672e42cb0 2024-04-21T08:47:39.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.679+0000 7fc9437fe640 10 client.16104 handle_command_reply: tid=0 2024-04-21T08:47:39.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.679+0000 7fc96cf95740 1 client.16104 shutdown 2024-04-21T08:47:39.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.679+0000 7fc96cf95740 1 -- 172.21.15.82:0/3661222948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 msgr2=0x55a672e32360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.679+0000 7fc96cf95740 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e32360 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc95409c620 tx=0x7fc95400fe60 comp rx=0 tx=0).stop 2024-04-21T08:47:39.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.679+0000 7fc96cf95740 1 -- 172.21.15.82:0/3661222948 shutdown_connections 2024-04-21T08:47:39.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.679+0000 7fc96cf95740 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a672e42cb0 0x55a672e450a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.679+0000 7fc96cf95740 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e32a00 0x55a672e2fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.679+0000 7fc96cf95740 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e29e80 0x55a672e2f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.679+0000 7fc9437fe640 0 client.16104 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:47:39.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.679+0000 7fc96cf95740 1 --2- 172.21.15.82:0/3661222948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e32360 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.679+0000 7fc96cf95740 1 -- 172.21.15.82:0/3661222948 >> 172.21.15.82:0/3661222948 conn(0x55a672e23ba0 msgr2=0x55a672e258f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.680+0000 7fc96cf95740 1 -- 172.21.15.82:0/3661222948 shutdown_connections 2024-04-21T08:47:39.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.680+0000 7fc96cf95740 1 -- 172.21.15.82:0/3661222948 wait complete. 2024-04-21T08:47:39.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.680+0000 7fc96cf95740 20 client.16104 trim_cache size 0 max 16384 2024-04-21T08:47:39.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.705+0000 7fc96cf95740 1 Processor -- start 2024-04-21T08:47:39.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.705+0000 7fc96cf95740 1 -- start start 2024-04-21T08:47:39.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.705+0000 7fc96cf95740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e3e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.705+0000 7fc96cf95740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e29e80 0x55a672e31e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.705+0000 7fc963fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e3e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.705+0000 7fc963fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e3e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56726/0 (socket says 172.21.15.82:56726) 2024-04-21T08:47:39.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.705+0000 7fc963fff640 1 -- 172.21.15.82:0/2542890190 learned_addr learned my addr 172.21.15.82:0/2542890190 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.705+0000 7fc96cf95740 1 --2- 172.21.15.82:0/2542890190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e32a00 0x55a672e32360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.706+0000 7fc96cf95740 1 -- 172.21.15.82:0/2542890190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641b1550 con 0x55a672e294d0 2024-04-21T08:47:39.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.706+0000 7fc96cf95740 1 -- 172.21.15.82:0/2542890190 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641b3090 con 0x55a672e29e80 2024-04-21T08:47:39.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.706+0000 7fc96cf95740 1 -- 172.21.15.82:0/2542890190 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641b1c20 con 0x55a672e32a00 2024-04-21T08:47:39.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.706+0000 7fc9637fe640 1 --2- 172.21.15.82:0/2542890190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e29e80 0x55a672e31e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.706+0000 7fc963fff640 1 -- 172.21.15.82:0/2542890190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e32a00 msgr2=0x55a672e32360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.706+0000 7fc96886b640 1 --2- 172.21.15.82:0/2542890190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e32a00 0x55a672e32360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.706+0000 7fc963fff640 1 --2- 172.21.15.82:0/2542890190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e32a00 0x55a672e32360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.706+0000 7fc963fff640 1 -- 172.21.15.82:0/2542890190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e29e80 msgr2=0x55a672e31e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.706+0000 7fc963fff640 1 --2- 172.21.15.82:0/2542890190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e29e80 0x55a672e31e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.706+0000 7fc963fff640 1 -- 172.21.15.82:0/2542890190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a672e30070 con 0x55a672e294d0 2024-04-21T08:47:39.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.706+0000 7fc963fff640 1 --2- 172.21.15.82:0/2542890190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e3e830 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc95409c8b0 tx=0x7fc954098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9749b6d573e080f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.706+0000 7fc949ffb640 1 -- 172.21.15.82:0/2542890190 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954085e60 con 0x55a672e294d0 2024-04-21T08:47:39.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.707+0000 7fc949ffb640 1 -- 172.21.15.82:0/2542890190 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9540ce0e0 con 0x55a672e294d0 2024-04-21T08:47:39.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.707+0000 7fc949ffb640 1 -- 172.21.15.82:0/2542890190 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9540981d0 con 0x55a672e294d0 2024-04-21T08:47:39.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.707+0000 7fc96cf95740 1 -- 172.21.15.82:0/2542890190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 msgr2=0x55a672e3e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.707+0000 7fc96cf95740 1 --2- 172.21.15.82:0/2542890190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e3e830 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc95409c8b0 tx=0x7fc954098040 comp rx=0 tx=0).stop 2024-04-21T08:47:39.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.708+0000 7fc96cf95740 1 -- 172.21.15.82:0/2542890190 shutdown_connections 2024-04-21T08:47:39.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.708+0000 7fc96cf95740 1 --2- 172.21.15.82:0/2542890190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e32a00 0x55a672e32360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.708+0000 7fc96cf95740 1 --2- 172.21.15.82:0/2542890190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e29e80 0x55a672e31e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.708+0000 7fc96cf95740 1 --2- 172.21.15.82:0/2542890190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e294d0 0x55a672e3e830 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.708+0000 7fc96cf95740 1 -- 172.21.15.82:0/2542890190 >> 172.21.15.82:0/2542890190 conn(0x55a672e23ba0 msgr2=0x55a672e22c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.708+0000 7fc96cf95740 1 -- 172.21.15.82:0/2542890190 shutdown_connections 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.708+0000 7fc96cf95740 1 -- 172.21.15.82:0/2542890190 wait complete. 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.708+0000 7fc96cf95740 1 -- start start 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.708+0000 7fc96cf95740 1 Processor -- start 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.709+0000 7fc96cf95740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e294d0 0x55a672e3b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.709+0000 7fc96cf95740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e29e80 0x55a672e2c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.709+0000 7fc96cf95740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e32a00 0x55a672e2c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.709+0000 7fc96cf95740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641b29c0 con 0x55a672e32a00 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.709+0000 7fc96cf95740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641b22f0 con 0x55a672e29e80 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.709+0000 7fc96cf95740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641af530 con 0x55a672e294d0 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.709+0000 7fc96cf95740 10 client.0 _ll_register_callbacks cb 0x7fc95409c460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.709+0000 7fc96886b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e32a00 0x55a672e2c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.709+0000 7fc96886b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e32a00 0x55a672e2c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56738/0 (socket says 172.21.15.82:56738) 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.709+0000 7fc96886b640 1 -- 172.21.15.82:0/2216851681 learned_addr learned my addr 172.21.15.82:0/2216851681 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.709+0000 7fc963fff640 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e294d0 0x55a672e3b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.709+0000 7fc9637fe640 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e29e80 0x55a672e2c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.710+0000 7fc96886b640 1 -- 172.21.15.82:0/2216851681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e294d0 msgr2=0x55a672e3b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.710+0000 7fc96886b640 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e294d0 0x55a672e3b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.710+0000 7fc96886b640 1 -- 172.21.15.82:0/2216851681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e29e80 msgr2=0x55a672e2c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.710+0000 7fc96886b640 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e29e80 0x55a672e2c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.710+0000 7fc96886b640 1 -- 172.21.15.82:0/2216851681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a672e2cd20 con 0x55a672e32a00 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.710+0000 7fc9637fe640 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e29e80 0x55a672e2c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.710+0000 7fc96886b640 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e32a00 0x55a672e2c5e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc95c011750 tx=0x7fc95c004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1389d3812d127bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.710+0000 7fc9437fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.710+0000 7fc9437fe640 1 -- 172.21.15.82:0/2216851681 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c013050 con 0x55a672e32a00 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.710+0000 7fc9437fe640 1 -- 172.21.15.82:0/2216851681 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc95c0131f0 con 0x55a672e32a00 2024-04-21T08:47:39.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.710+0000 7fc96cf95740 1 -- 172.21.15.82:0/2216851681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9540ce0e0 con 0x55a672e32a00 2024-04-21T08:47:39.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.711+0000 7fc9437fe640 1 -- 172.21.15.82:0/2216851681 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc95c0133f0 con 0x55a672e32a00 2024-04-21T08:47:39.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.711+0000 7fc96cf95740 10 client.16110 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:39.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.711+0000 7fc96cf95740 1 -- 172.21.15.82:0/2216851681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a672ea90f0 con 0x55a672e32a00 2024-04-21T08:47:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.711+0000 7fc9437fe640 1 -- 172.21.15.82:0/2216851681 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9540ce0e0 con 0x55a672e32a00 2024-04-21T08:47:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.711+0000 7fc9437fe640 1 -- 172.21.15.82:0/2216851681 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc95c01da20 con 0x55a672e32a00 2024-04-21T08:47:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.712+0000 7fc96cf95740 10 client.16110 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.712+0000 7fc96cf95740 10 client.16110 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:47:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.712+0000 7fc96cf95740 20 client.16110 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.712+0000 7fc96cf95740 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a672ea96a0 0x55a672eabac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.712+0000 7fc96cf95740 4 client.16110 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.712+0000 7fc96cf95740 1 -- 172.21.15.82:0/2216851681 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a672d2d930 con 0x55a672ea96a0 2024-04-21T08:47:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.712+0000 7fc963fff640 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a672ea96a0 0x55a672eabac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc9437fe640 10 client.16110 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:47:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc963fff640 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a672ea96a0 0x55a672eabac0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=54056b0b0f58eef6 server_cookie=ec91b37af0f9975a in_seq=0 out_seq=0 2024-04-21T08:47:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc9437fe640 1 -- 172.21.15.82:0/2216851681 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a672d2d930 con 0x55a672ea96a0 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc9437fe640 10 client.16110 handle_command_reply: tid=0 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc96cf95740 1 client.16110 shutdown 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc96cf95740 1 -- 172.21.15.82:0/2216851681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e32a00 msgr2=0x55a672e2c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc96cf95740 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e32a00 0x55a672e2c5e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc95c011750 tx=0x7fc95c004940 comp rx=0 tx=0).stop 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc96cf95740 1 -- 172.21.15.82:0/2216851681 shutdown_connections 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc96cf95740 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a672ea96a0 0x55a672eabac0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc96cf95740 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a672e32a00 0x55a672e2c5e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc96cf95740 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a672e29e80 0x55a672e2c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc96cf95740 1 --2- 172.21.15.82:0/2216851681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a672e294d0 0x55a672e3b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc96cf95740 1 -- 172.21.15.82:0/2216851681 >> 172.21.15.82:0/2216851681 conn(0x55a672e23ba0 msgr2=0x55a672e24e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.713+0000 7fc9437fe640 0 client.16110 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.714+0000 7fc96cf95740 1 -- 172.21.15.82:0/2216851681 shutdown_connections 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.714+0000 7fc96cf95740 1 -- 172.21.15.82:0/2216851681 wait complete. 2024-04-21T08:47:39.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.714+0000 7fc96cf95740 20 client.16110 trim_cache size 0 max 16384 2024-04-21T08:47:39.718 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:47:39.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.714+0000 7fc96a2ce640 1 -- 172.21.15.82:0/3042483527 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc94c0a3e40 msgr2=0x7fc94c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.714+0000 7fc96a2ce640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc94c0a3e40 0x7fc94c0a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc95c020000 tx=0x7fc95c01e000 comp rx=0 tx=0).stop 2024-04-21T08:47:39.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.714+0000 7fc96a2ce640 1 -- 172.21.15.82:0/3042483527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc964101090 msgr2=0x7fc964197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.714+0000 7fc96a2ce640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc964101090 0x7fc964197610 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc95400c920 tx=0x7fc95400cde0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.715+0000 7fc96a2ce640 1 -- 172.21.15.82:0/3042483527 shutdown_connections 2024-04-21T08:47:39.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.715+0000 7fc96a2ce640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc96419be80 0x7fc96419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.715+0000 7fc96a2ce640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc964106160 0x7fc964197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.715+0000 7fc96a2ce640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc94c0a3e40 0x7fc94c0a6300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.715+0000 7fc96a2ce640 1 --2- 172.21.15.82:0/3042483527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc964101090 0x7fc964197610 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.715+0000 7fc96a2ce640 1 -- 172.21.15.82:0/3042483527 >> 172.21.15.82:0/3042483527 conn(0x7fc964077f50 msgr2=0x7fc9640fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.715+0000 7fc96a2ce640 1 -- 172.21.15.82:0/3042483527 shutdown_connections 2024-04-21T08:47:39.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.715+0000 7fc96a2ce640 1 -- 172.21.15.82:0/3042483527 wait complete. 2024-04-21T08:47:39.729 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:47:39.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:39 smithi082 ceph-mon[27967]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-04-21T08:47:39.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:39 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/993289825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:39.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.862+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/1702246986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbe4101080 msgr2=0x7fbbe4106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.862+0000 7fbbeb1ce640 1 --2- 172.21.15.82:0/1702246986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbe4101080 0x7fbbe4106bb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009900 tx=0x7fbbd002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:39.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.862+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/1702246986 shutdown_connections 2024-04-21T08:47:39.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.862+0000 7fbbeb1ce640 1 --2- 172.21.15.82:0/1702246986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe41072e0 0x7fbbe410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.862+0000 7fbbeb1ce640 1 --2- 172.21.15.82:0/1702246986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbe4101080 0x7fbbe4106bb0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.862+0000 7fbbeb1ce640 1 --2- 172.21.15.82:0/1702246986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe41006d0 0x7fbbe4100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.862+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/1702246986 >> 172.21.15.82:0/1702246986 conn(0x7fbbe40773f0 msgr2=0x7fbbe40fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.862+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/1702246986 shutdown_connections 2024-04-21T08:47:39.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.862+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/1702246986 wait complete. 2024-04-21T08:47:39.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.863+0000 7fbbeb1ce640 1 Processor -- start 2024-04-21T08:47:39.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.863+0000 7fbbeb1ce640 1 -- start start 2024-04-21T08:47:39.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.863+0000 7fbbeb1ce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbe41006d0 0x7fbbe4101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.863+0000 7fbbeb1ce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe41072e0 0x7fbbe4101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.863+0000 7fbbeb1ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe41020c0 0x7fbbe41a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.863+0000 7fbbeb1ce640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4112190 con 0x7fbbe41006d0 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.863+0000 7fbbeb1ce640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4112010 con 0x7fbbe41072e0 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.863+0000 7fbbeb1ce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4112310 con 0x7fbbe41020c0 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.863+0000 7fbbe8f6a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbe41006d0 0x7fbbe4101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbe8f6a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbe41006d0 0x7fbbe4101640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56756/0 (socket says 172.21.15.82:56756) 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbe8f6a640 1 -- 172.21.15.82:0/4247667648 learned_addr learned my addr 172.21.15.82:0/4247667648 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbe976b640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe41020c0 0x7fbbe41a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbe3fff640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe41072e0 0x7fbbe4101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbe8f6a640 1 -- 172.21.15.82:0/4247667648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe41020c0 msgr2=0x7fbbe41a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe41020c0 0x7fbbe41a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbe8f6a640 1 -- 172.21.15.82:0/4247667648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe41072e0 msgr2=0x7fbbe4101b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe41072e0 0x7fbbe4101b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbe8f6a640 1 -- 172.21.15.82:0/4247667648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe41a7020 con 0x7fbbe41006d0 2024-04-21T08:47:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbe41006d0 0x7fbbe4101640 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800d950 tx=0x7fbbd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbe1ffb640 1 -- 172.21.15.82:0/4247667648 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8017030 con 0x7fbbe41006d0 2024-04-21T08:47:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/4247667648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe41a7310 con 0x7fbbe41006d0 2024-04-21T08:47:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.864+0000 7fbbe1ffb640 1 -- 172.21.15.82:0/4247667648 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd8004020 con 0x7fbbe41006d0 2024-04-21T08:47:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.865+0000 7fbbe1ffb640 1 -- 172.21.15.82:0/4247667648 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8005640 con 0x7fbbe41006d0 2024-04-21T08:47:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.865+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/4247667648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe41a7880 con 0x7fbbe41006d0 2024-04-21T08:47:39.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.868+0000 7fbbe1ffb640 1 -- 172.21.15.82:0/4247667648 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fbbd8009a30 con 0x7fbbe41006d0 2024-04-21T08:47:39.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbeddc1740 1 Processor -- start 2024-04-21T08:47:39.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbeddc1740 1 -- start start 2024-04-21T08:47:39.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbeddc1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed77534d0 0x559ed77538b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbeddc1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed7753e80 0x559ed775c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbeddc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed775ca00 0x559ed775ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbeddc1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559ed76b81f0 con 0x559ed7753e80 2024-04-21T08:47:39.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbeddc1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559ed76554b0 con 0x559ed77534d0 2024-04-21T08:47:39.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbeddc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559ed76b41b0 con 0x559ed775ca00 2024-04-21T08:47:39.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbe1ffb640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbbd40a4090 0x7fbbd40a6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbe8f6a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed7753e80 0x559ed775c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbe976b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed77534d0 0x559ed77538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed775ca00 0x559ed775ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbe976b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed77534d0 0x559ed77538b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:37364/0 (socket says 172.21.15.82:37364) 2024-04-21T08:47:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.869+0000 7fbbe976b640 1 -- 172.21.15.82:0/2735602911 learned_addr learned my addr 172.21.15.82:0/2735602911 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.870+0000 7fbbe976b640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbbd40a4090 0x7fbbd40a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.870+0000 7fbbe1ffb640 1 -- 172.21.15.82:0/4247667648 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fbbd80d02d0 con 0x7fbbe41006d0 2024-04-21T08:47:39.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.870+0000 7fbbe8f6a640 1 -- 172.21.15.82:0/2735602911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed775ca00 msgr2=0x559ed775ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.870+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/2735602911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed775ca00 0x559ed775ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.870+0000 7fbbe8f6a640 1 -- 172.21.15.82:0/2735602911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed77534d0 msgr2=0x559ed77538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.870+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/2735602911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed77534d0 0x559ed77538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.870+0000 7fbbe8f6a640 1 -- 172.21.15.82:0/2735602911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed76471b0 con 0x559ed7753e80 2024-04-21T08:47:39.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.870+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/2735602911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed7753e80 0x559ed775c430 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80cd480 tx=0x7fbbd80045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37120c32279f635 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.870+0000 7fbbceffd640 1 -- 172.21.15.82:0/2735602911 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8014070 con 0x559ed7753e80 2024-04-21T08:47:39.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.870+0000 7fbbe976b640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbbd40a4090 0x7fbbd40a6550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc0139c0 tx=0x7fbbdc009bf0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.870+0000 7fbbceffd640 1 -- 172.21.15.82:0/2735602911 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd809bc00 con 0x559ed7753e80 2024-04-21T08:47:39.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.870+0000 7fbbceffd640 1 -- 172.21.15.82:0/2735602911 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80bc0c0 con 0x559ed7753e80 2024-04-21T08:47:39.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.871+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2735602911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed7753e80 msgr2=0x559ed775c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.871+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/2735602911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed7753e80 0x559ed775c430 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80cd480 tx=0x7fbbd80045a0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.871+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2735602911 shutdown_connections 2024-04-21T08:47:39.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.871+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/2735602911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed775ca00 0x559ed775ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.871+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/2735602911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed7753e80 0x559ed775c430 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.871+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/2735602911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed77534d0 0x559ed77538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.871+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2735602911 >> 172.21.15.82:0/2735602911 conn(0x559ed774dba0 msgr2=0x559ed774df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.871+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2735602911 shutdown_connections 2024-04-21T08:47:39.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.871+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2735602911 wait complete. 2024-04-21T08:47:39.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.871+0000 7fbbeddc1740 1 -- start start 2024-04-21T08:47:39.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbeddc1740 1 Processor -- start 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbeddc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed77534d0 0x559ed775c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbeddc1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed7753e80 0x559ed7759680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbeddc1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 0x559ed7759c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbeddc1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559ed775f910 con 0x559ed775ca00 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbeddc1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559ed7671500 con 0x559ed7753e80 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbeddc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559ed775fa90 con 0x559ed77534d0 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbeddc1740 10 client.0 _ll_register_callbacks cb 0x559ed763fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbe976b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 0x559ed7759c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbe976b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 0x559ed7759c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56770/0 (socket says 172.21.15.82:56770) 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbe976b640 1 -- 172.21.15.82:0/2975019262 learned_addr learned my addr 172.21.15.82:0/2975019262 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed77534d0 0x559ed775c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.872+0000 7fbbe3fff640 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed7753e80 0x559ed7759680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbe976b640 1 -- 172.21.15.82:0/2975019262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed77534d0 msgr2=0x559ed775c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbe976b640 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed77534d0 0x559ed775c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbe976b640 1 -- 172.21.15.82:0/2975019262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed7753e80 msgr2=0x559ed7759680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbe976b640 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed7753e80 0x559ed7759680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbe976b640 1 -- 172.21.15.82:0/2975019262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed73ca3f0 con 0x559ed775ca00 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed77534d0 0x559ed775c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbe3fff640 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed7753e80 0x559ed7759680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbe976b640 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 0x559ed7759c00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00fca0 tx=0x7fbbdc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e102338fab090eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbc3fff640 1 -- 172.21.15.82:0/2975019262 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc010cc0 con 0x559ed775ca00 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbc3fff640 1 -- 172.21.15.82:0/2975019262 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fbbdc01e410 con 0x559ed775ca00 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2975019262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd809bc00 con 0x559ed775ca00 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbc3fff640 1 -- 172.21.15.82:0/2975019262 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbbdc010e60 con 0x559ed775ca00 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbeddc1740 10 client.16122 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:39.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2975019262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559ed74947d0 con 0x559ed775ca00 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.873+0000 7fbbc3fff640 1 -- 172.21.15.82:0/2975019262 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd809bc00 con 0x559ed775ca00 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.874+0000 7fbbc3fff640 1 -- 172.21.15.82:0/2975019262 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbbdc026070 con 0x559ed775ca00 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.874+0000 7fbbeddc1740 10 client.16122 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.874+0000 7fbbeddc1740 10 client.16122 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.874+0000 7fbbeddc1740 20 client.16122 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.874+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x559ed776cbe0 0x559ed776f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.874+0000 7fbbeddc1740 4 client.16122 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.874+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2975019262 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ed7657930 con 0x559ed776cbe0 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.874+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x559ed776cbe0 0x559ed776f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.875+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x559ed776cbe0 0x559ed776f010 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d6aaddb07cd8d334 server_cookie=6c6945e9353627cc in_seq=0 out_seq=0 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.875+0000 7fbbc3fff640 10 client.16122 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.875+0000 7fbbc3fff640 1 -- 172.21.15.82:0/2975019262 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ed7657930 con 0x559ed776cbe0 2024-04-21T08:47:39.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.875+0000 7fbbc3fff640 10 client.16122 handle_command_reply: tid=0 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.875+0000 7fbbeddc1740 1 client.16122 shutdown 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.875+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2975019262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 msgr2=0x559ed7759c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.875+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 0x559ed7759c00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00fca0 tx=0x7fbbdc005bd0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.875+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2975019262 shutdown_connections 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.875+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x559ed776cbe0 0x559ed776f010 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.876+0000 7fbbc3fff640 0 client.16122 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.876+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 0x559ed7759c00 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.876+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed7753e80 0x559ed7759680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.876+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/2975019262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed77534d0 0x559ed775c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.876+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2975019262 >> 172.21.15.82:0/2975019262 conn(0x559ed774dba0 msgr2=0x559ed774f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.876+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2975019262 shutdown_connections 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.876+0000 7fbbeddc1740 1 -- 172.21.15.82:0/2975019262 wait complete. 2024-04-21T08:47:39.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.876+0000 7fbbeddc1740 20 client.16122 trim_cache size 0 max 16384 2024-04-21T08:47:39.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.901+0000 7fbbeddc1740 1 Processor -- start 2024-04-21T08:47:39.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.901+0000 7fbbeddc1740 1 -- start start 2024-04-21T08:47:39.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbeddc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed77534d0 0x559ed77687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbeddc1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed7753e80 0x559ed775bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbeddc1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 0x559ed775c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbeddc1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41b1620 con 0x559ed775ca00 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbeddc1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41b3160 con 0x559ed7753e80 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbeddc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41b1cf0 con 0x559ed77534d0 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe976b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 0x559ed775c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed7753e80 0x559ed775bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe976b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 0x559ed775c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56776/0 (socket says 172.21.15.82:56776) 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed7753e80 0x559ed775bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:37392/0 (socket says 172.21.15.82:37392) 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe976b640 1 -- 172.21.15.82:0/292182529 learned_addr learned my addr 172.21.15.82:0/292182529 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/292182529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed77534d0 0x559ed77687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe976b640 1 -- 172.21.15.82:0/292182529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed77534d0 msgr2=0x559ed77687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe976b640 1 --2- 172.21.15.82:0/292182529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed77534d0 0x559ed77687d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe976b640 1 -- 172.21.15.82:0/292182529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed7753e80 msgr2=0x559ed775bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe976b640 1 --2- 172.21.15.82:0/292182529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed7753e80 0x559ed775bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe976b640 1 -- 172.21.15.82:0/292182529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed7755270 con 0x559ed775ca00 2024-04-21T08:47:39.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/292182529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed77534d0 0x559ed77687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:39.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.902+0000 7fbbe976b640 1 --2- 172.21.15.82:0/292182529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 0x559ed775c2f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00fc70 tx=0x7fbbdc013fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c67b661c84f3d1e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.903+0000 7fbbce7fc640 1 -- 172.21.15.82:0/292182529 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc00fe50 con 0x559ed775ca00 2024-04-21T08:47:39.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.903+0000 7fbbce7fc640 1 -- 172.21.15.82:0/292182529 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc010ac0 con 0x559ed775ca00 2024-04-21T08:47:39.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.903+0000 7fbbce7fc640 1 -- 172.21.15.82:0/292182529 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc028e40 con 0x559ed775ca00 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.903+0000 7fbbeddc1740 1 -- 172.21.15.82:0/292182529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 msgr2=0x559ed775c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.903+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/292182529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 0x559ed775c2f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00fc70 tx=0x7fbbdc013fa0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.903+0000 7fbbeddc1740 1 -- 172.21.15.82:0/292182529 shutdown_connections 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.903+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/292182529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed775ca00 0x559ed775c2f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.903+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/292182529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed7753e80 0x559ed775bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.903+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/292182529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed77534d0 0x559ed77687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.903+0000 7fbbeddc1740 1 -- 172.21.15.82:0/292182529 >> 172.21.15.82:0/292182529 conn(0x559ed774dba0 msgr2=0x559ed7759640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.903+0000 7fbbeddc1740 1 -- 172.21.15.82:0/292182529 shutdown_connections 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.903+0000 7fbbeddc1740 1 -- 172.21.15.82:0/292182529 wait complete. 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbeddc1740 1 -- start start 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbeddc1740 1 Processor -- start 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbeddc1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed77534d0 0x559ed7764ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbeddc1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed7753e80 0x559ed7755ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbeddc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed775ca00 0x559ed7756430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbeddc1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41b2a90 con 0x559ed7753e80 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbeddc1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41b23c0 con 0x559ed77534d0 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbeddc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe41af600 con 0x559ed775ca00 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbeddc1740 10 client.0 _ll_register_callbacks cb 0x7fbbdc013620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed7753e80 0x559ed7755ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbe8f6a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed77534d0 0x559ed7764ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed7753e80 0x559ed7755ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56786/0 (socket says 172.21.15.82:56786) 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.904+0000 7fbbe3fff640 1 -- 172.21.15.82:0/401102763 learned_addr learned my addr 172.21.15.82:0/401102763 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbe8f6a640 1 -- 172.21.15.82:0/401102763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed775ca00 msgr2=0x559ed7756430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/401102763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed775ca00 0x559ed7756430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbe8f6a640 1 -- 172.21.15.82:0/401102763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed7753e80 msgr2=0x559ed7755ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/401102763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed7753e80 0x559ed7755ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbe8f6a640 1 -- 172.21.15.82:0/401102763 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed7756a70 con 0x559ed77534d0 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbe8f6a640 1 --2- 172.21.15.82:0/401102763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed77534d0 0x559ed7764ed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbbd809bb40 tx=0x7fbbd800b530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d69c6fda167df3f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbc3fff640 1 -- 172.21.15.82:0/401102763 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80b4070 con 0x559ed77534d0 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbeddc1740 1 -- 172.21.15.82:0/401102763 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbdc010ac0 con 0x559ed77534d0 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbc3fff640 1 -- 172.21.15.82:0/401102763 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fbbd8009d20 con 0x559ed77534d0 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbc3fff640 1 -- 172.21.15.82:0/401102763 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbbd8014040 con 0x559ed77534d0 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbeddc1740 10 client.25484 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.905+0000 7fbbeddc1740 1 -- 172.21.15.82:0/401102763 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559ed77d3020 con 0x559ed77534d0 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.906+0000 7fbbc3fff640 1 -- 172.21.15.82:0/401102763 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc010ac0 con 0x559ed77534d0 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.906+0000 7fbbc3fff640 1 -- 172.21.15.82:0/401102763 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbbd8098270 con 0x559ed77534d0 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.906+0000 7fbbeddc1740 10 client.25484 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.906+0000 7fbbeddc1740 10 client.25484 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.906+0000 7fbbeddc1740 20 client.25484 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:39.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.906+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/401102763 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x559ed776cbe0 0x559ed77d52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.906+0000 7fbbeddc1740 4 client.25484 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.906+0000 7fbbeddc1740 1 -- 172.21.15.82:0/401102763 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ed76c5c10 con 0x559ed776cbe0 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.906+0000 7fbbe3fff640 1 --2- 172.21.15.82:0/401102763 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x559ed776cbe0 0x559ed77d52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbe3fff640 1 --2- 172.21.15.82:0/401102763 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x559ed776cbe0 0x559ed77d52c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aacc2819578b6042 server_cookie=bd7983f273a80b24 in_seq=0 out_seq=0 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbc3fff640 10 client.25484 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbc3fff640 1 -- 172.21.15.82:0/401102763 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ed76c5c10 con 0x559ed776cbe0 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbc3fff640 10 client.25484 handle_command_reply: tid=0 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbeddc1740 1 client.25484 shutdown 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbeddc1740 1 -- 172.21.15.82:0/401102763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed77534d0 msgr2=0x559ed7764ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/401102763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed77534d0 0x559ed7764ed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbbd809bb40 tx=0x7fbbd800b530 comp rx=0 tx=0).stop 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbeddc1740 1 -- 172.21.15.82:0/401102763 shutdown_connections 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/401102763 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x559ed776cbe0 0x559ed77d52c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/401102763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559ed775ca00 0x559ed7756430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/401102763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ed7753e80 0x559ed7755ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbc3fff640 0 client.25484 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:47:39.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbeddc1740 1 --2- 172.21.15.82:0/401102763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ed77534d0 0x559ed7764ed0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbeddc1740 1 -- 172.21.15.82:0/401102763 >> 172.21.15.82:0/401102763 conn(0x559ed774dba0 msgr2=0x559ed774c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbeddc1740 1 -- 172.21.15.82:0/401102763 shutdown_connections 2024-04-21T08:47:39.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbeddc1740 1 -- 172.21.15.82:0/401102763 wait complete. 2024-04-21T08:47:39.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.907+0000 7fbbeddc1740 20 client.25484 trim_cache size 0 max 16384 2024-04-21T08:47:39.913 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:47:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.908+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/4247667648 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbbd40a4090 msgr2=0x7fbbd40a6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.908+0000 7fbbeb1ce640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbbd40a4090 0x7fbbd40a6550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc0139c0 tx=0x7fbbdc009bf0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.908+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/4247667648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbe41006d0 msgr2=0x7fbbe4101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.908+0000 7fbbeb1ce640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbe41006d0 0x7fbbe4101640 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800d950 tx=0x7fbbd800de10 comp rx=0 tx=0).stop 2024-04-21T08:47:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.908+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/4247667648 shutdown_connections 2024-04-21T08:47:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.908+0000 7fbbeb1ce640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe41020c0 0x7fbbe41a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.908+0000 7fbbeb1ce640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe41072e0 0x7fbbe4101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.908+0000 7fbbeb1ce640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbbd40a4090 0x7fbbd40a6550 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.908+0000 7fbbeb1ce640 1 --2- 172.21.15.82:0/4247667648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbe41006d0 0x7fbbe4101640 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.908+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/4247667648 >> 172.21.15.82:0/4247667648 conn(0x7fbbe40773f0 msgr2=0x7fbbe4105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.908+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/4247667648 shutdown_connections 2024-04-21T08:47:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:39.908+0000 7fbbeb1ce640 1 -- 172.21.15.82:0/4247667648 wait complete. 2024-04-21T08:47:39.920 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:47:39.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:39 smithi134 ceph-mon[31772]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-04-21T08:47:39.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:39 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/993289825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.024+0000 7f6722f15640 1 -- 172.21.15.82:0/1113901184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671c1031d0 msgr2=0x7f671c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.024+0000 7f671b7fe640 1 -- 172.21.15.82:0/1113901184 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710005220 con 0x7f671c1031d0 2024-04-21T08:47:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.024+0000 7f6722f15640 1 --2- 172.21.15.82:0/1113901184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671c1031d0 0x7f671c1035b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6710009900 tx=0x7f671002bd60 comp rx=0 tx=0).stop 2024-04-21T08:47:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 -- 172.21.15.82:0/1113901184 shutdown_connections 2024-04-21T08:47:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 --2- 172.21.15.82:0/1113901184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c10c510 0x7f671c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 --2- 172.21.15.82:0/1113901184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f671c103af0 0x7f671c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 --2- 172.21.15.82:0/1113901184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671c1031d0 0x7f671c1035b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 -- 172.21.15.82:0/1113901184 >> 172.21.15.82:0/1113901184 conn(0x7f671c077f50 msgr2=0x7f671c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 -- 172.21.15.82:0/1113901184 shutdown_connections 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 -- 172.21.15.82:0/1113901184 wait complete. 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 Processor -- start 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 -- start start 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671c1031d0 0x7f671c100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f671c103af0 0x7f671c101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c10c510 0x7f671c1018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.025+0000 7f6722f15640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1121f0 con 0x7f671c1031d0 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f6720cb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671c1031d0 0x7f671c100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f6722f15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f671c112070 con 0x7f671c10c510 2024-04-21T08:47:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f6720cb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671c1031d0 0x7f671c100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56804/0 (socket says 172.21.15.82:56804) 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f671bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f671c103af0 0x7f671c101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f6720cb1640 1 -- 172.21.15.82:0/969104572 learned_addr learned my addr 172.21.15.82:0/969104572 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f67214b2640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c10c510 0x7f671c1018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f6722f15640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f671c112370 con 0x7f671c103af0 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f671bfff640 1 -- 172.21.15.82:0/969104572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c10c510 msgr2=0x7f671c1018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f671bfff640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c10c510 0x7f671c1018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f671bfff640 1 -- 172.21.15.82:0/969104572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671c1031d0 msgr2=0x7f671c100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f671bfff640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671c1031d0 0x7f671c100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f671bfff640 1 -- 172.21.15.82:0/969104572 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c1a6cd0 con 0x7f671c103af0 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f6720cb1640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671c1031d0 0x7f671c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.026+0000 7f671bfff640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f671c103af0 0x7f671c101360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f670800e9f0 tx=0x7f670800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.027+0000 7f6719ffb640 1 -- 172.21.15.82:0/969104572 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670800ccc0 con 0x7f671c103af0 2024-04-21T08:47:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.027+0000 7f6719ffb640 1 -- 172.21.15.82:0/969104572 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6708005020 con 0x7f671c103af0 2024-04-21T08:47:40.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.027+0000 7f6719ffb640 1 -- 172.21.15.82:0/969104572 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708004a00 con 0x7f671c103af0 2024-04-21T08:47:40.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.027+0000 7f6722f15640 1 -- 172.21.15.82:0/969104572 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c1a6fc0 con 0x7f671c103af0 2024-04-21T08:47:40.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.027+0000 7f6722f15640 1 -- 172.21.15.82:0/969104572 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c1a73d0 con 0x7f671c103af0 2024-04-21T08:47:40.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.029+0000 7f6719ffb640 1 -- 172.21.15.82:0/969104572 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6708004ba0 con 0x7f671c103af0 2024-04-21T08:47:40.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.030+0000 7f6719ffb640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f670c0a3e80 0x7f670c0a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.030+0000 7f6719ffb640 1 -- 172.21.15.82:0/969104572 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f67080d0e00 con 0x7f671c103af0 2024-04-21T08:47:40.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.030+0000 7f6725bdd740 1 Processor -- start 2024-04-21T08:47:40.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.030+0000 7f6720cb1640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f670c0a3e80 0x7f670c0a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.030+0000 7f6725bdd740 1 -- start start 2024-04-21T08:47:40.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6725bdd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc388524d0 0x55dc388528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6725bdd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc3885b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6725bdd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 0x55dc3885de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6725bdd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55dc387b71f0 con 0x55dc3885ba00 2024-04-21T08:47:40.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6725bdd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55dc387544b0 con 0x55dc388524d0 2024-04-21T08:47:40.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6725bdd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dc387b31b0 con 0x55dc38852e80 2024-04-21T08:47:40.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f67214b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 0x55dc3885de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6720cb1640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f670c0a3e80 0x7f670c0a6340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f67100096c0 tx=0x7f671002c280 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6720cb1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc388524d0 0x55dc388528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f67214b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 0x55dc3885de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56816/0 (socket says 172.21.15.82:56816) 2024-04-21T08:47:40.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f67214b2640 1 -- 172.21.15.82:0/2458704057 learned_addr learned my addr 172.21.15.82:0/2458704057 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f671bfff640 1 --2- 172.21.15.82:0/2458704057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc3885b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6720cb1640 1 -- 172.21.15.82:0/2458704057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 msgr2=0x55dc3885b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6720cb1640 1 --2- 172.21.15.82:0/2458704057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc3885b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6720cb1640 1 -- 172.21.15.82:0/2458704057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 msgr2=0x55dc3885de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6720cb1640 1 --2- 172.21.15.82:0/2458704057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 0x55dc3885de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f6720cb1640 1 -- 172.21.15.82:0/2458704057 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc387461b0 con 0x55dc388524d0 2024-04-21T08:47:40.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f671bfff640 1 --2- 172.21.15.82:0/2458704057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc3885b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:40.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.031+0000 7f67214b2640 1 --2- 172.21.15.82:0/2458704057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 0x55dc3885de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:40.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.032+0000 7f6720cb1640 1 --2- 172.21.15.82:0/2458704057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc388524d0 0x55dc388528b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6710009230 tx=0x7f671003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4036e24b25ab1325 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.032+0000 7f6706ffd640 1 -- 172.21.15.82:0/2458704057 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671002c940 con 0x55dc388524d0 2024-04-21T08:47:40.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.032+0000 7f6706ffd640 1 -- 172.21.15.82:0/2458704057 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6710051070 con 0x55dc388524d0 2024-04-21T08:47:40.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.033+0000 7f6725bdd740 1 -- 172.21.15.82:0/2458704057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc388524d0 msgr2=0x55dc388528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.033+0000 7f6725bdd740 1 --2- 172.21.15.82:0/2458704057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc388524d0 0x55dc388528b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6710009230 tx=0x7f671003f000 comp rx=0 tx=0).stop 2024-04-21T08:47:40.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.033+0000 7f6725bdd740 1 -- 172.21.15.82:0/2458704057 shutdown_connections 2024-04-21T08:47:40.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.033+0000 7f6725bdd740 1 --2- 172.21.15.82:0/2458704057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 0x55dc3885de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.033+0000 7f6725bdd740 1 --2- 172.21.15.82:0/2458704057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc3885b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.033+0000 7f6725bdd740 1 --2- 172.21.15.82:0/2458704057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc388524d0 0x55dc388528b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.033+0000 7f6725bdd740 1 -- 172.21.15.82:0/2458704057 >> 172.21.15.82:0/2458704057 conn(0x55dc3884cba0 msgr2=0x55dc3884cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.033+0000 7f6725bdd740 1 -- 172.21.15.82:0/2458704057 shutdown_connections 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.033+0000 7f6725bdd740 1 -- 172.21.15.82:0/2458704057 wait complete. 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.033+0000 7f6725bdd740 1 -- start start 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.033+0000 7f6725bdd740 1 Processor -- start 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.034+0000 7f6725bdd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 0x55dc3885b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.034+0000 7f6725bdd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc38858540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.034+0000 7f6725bdd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc3885ba00 0x55dc38858a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.034+0000 7f6725bdd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55dc3885e9b0 con 0x55dc388524d0 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.034+0000 7f6725bdd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55dc38770500 con 0x55dc3885ba00 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.034+0000 7f6725bdd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dc3885eb30 con 0x55dc38852e80 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.034+0000 7f6720cb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 0x55dc3885b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.034+0000 7f6720cb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 0x55dc3885b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56822/0 (socket says 172.21.15.82:56822) 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.034+0000 7f6720cb1640 1 -- 172.21.15.82:0/1370284427 learned_addr learned my addr 172.21.15.82:0/1370284427 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.034+0000 7f6725bdd740 10 client.0 _ll_register_callbacks cb 0x55dc3873eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f67214b2640 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc3885ba00 0x55dc38858a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f671bfff640 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc38858540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f6720cb1640 1 -- 172.21.15.82:0/1370284427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 msgr2=0x55dc38858540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f6720cb1640 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc38858540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f6720cb1640 1 -- 172.21.15.82:0/1370284427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc3885ba00 msgr2=0x55dc38858a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f6720cb1640 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc3885ba00 0x55dc38858a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f6720cb1640 1 -- 172.21.15.82:0/1370284427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc384c93f0 con 0x55dc388524d0 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f671bfff640 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc38858540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f67214b2640 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc3885ba00 0x55dc38858a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f6720cb1640 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 0x55dc3885b1b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f671002bda0 tx=0x7f671002cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce204a181b667a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f66fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:40.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f66fbfff640 1 -- 172.21.15.82:0/1370284427 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671003f8a0 con 0x55dc388524d0 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f66fbfff640 1 -- 172.21.15.82:0/1370284427 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f671003fa40 con 0x55dc388524d0 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.035+0000 7f6725bdd740 1 -- 172.21.15.82:0/1370284427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6710051070 con 0x55dc388524d0 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.036+0000 7f66fbfff640 1 -- 172.21.15.82:0/1370284427 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6710064050 con 0x55dc388524d0 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.036+0000 7f6725bdd740 10 client.16140 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.036+0000 7f6725bdd740 1 -- 172.21.15.82:0/1370284427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc385937d0 con 0x55dc388524d0 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.036+0000 7f66fbfff640 1 -- 172.21.15.82:0/1370284427 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6710051070 con 0x55dc388524d0 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.036+0000 7f66fbfff640 1 -- 172.21.15.82:0/1370284427 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f671004cbe0 con 0x55dc388524d0 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.036+0000 7f6725bdd740 10 client.16140 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.036+0000 7f6725bdd740 10 client.16140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.036+0000 7f6725bdd740 20 client.16140 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.036+0000 7f6725bdd740 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55dc3886bbb0 0x55dc3886dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.036+0000 7f6725bdd740 4 client.16140 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.036+0000 7f6725bdd740 1 -- 172.21.15.82:0/1370284427 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc38756930 con 0x55dc3886bbb0 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.037+0000 7f671bfff640 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55dc3886bbb0 0x55dc3886dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.037+0000 7f66fbfff640 10 client.16140 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.037+0000 7f671bfff640 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55dc3886bbb0 0x55dc3886dfc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2f68bef44e2746f server_cookie=12f05b197818b6f3 in_seq=0 out_seq=0 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f66fbfff640 1 -- 172.21.15.82:0/1370284427 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc38756930 con 0x55dc3886bbb0 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f66fbfff640 10 client.16140 handle_command_reply: tid=0 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f6725bdd740 1 client.16140 shutdown 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f6725bdd740 1 -- 172.21.15.82:0/1370284427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 msgr2=0x55dc3885b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f6725bdd740 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 0x55dc3885b1b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f671002bda0 tx=0x7f671002cbf0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f6725bdd740 1 -- 172.21.15.82:0/1370284427 shutdown_connections 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f6725bdd740 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55dc3886bbb0 0x55dc3886dfc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f6725bdd740 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc3885ba00 0x55dc38858a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f6725bdd740 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc38858540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f66fbfff640 0 client.16140 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:47:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f6725bdd740 1 --2- 172.21.15.82:0/1370284427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 0x55dc3885b1b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f6725bdd740 1 -- 172.21.15.82:0/1370284427 >> 172.21.15.82:0/1370284427 conn(0x55dc3884cba0 msgr2=0x55dc3884e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f6725bdd740 1 -- 172.21.15.82:0/1370284427 shutdown_connections 2024-04-21T08:47:40.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f6725bdd740 1 -- 172.21.15.82:0/1370284427 wait complete. 2024-04-21T08:47:40.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.038+0000 7f6725bdd740 20 client.16140 trim_cache size 0 max 16384 2024-04-21T08:47:40.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f6725bdd740 1 Processor -- start 2024-04-21T08:47:40.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f6725bdd740 1 -- start start 2024-04-21T08:47:40.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f6725bdd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc388524d0 0x55dc388678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f6725bdd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc3885ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f6725bdd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 0x55dc3885b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f6725bdd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1b1540 con 0x55dc3885ba00 2024-04-21T08:47:40.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f6725bdd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1b3080 con 0x55dc388524d0 2024-04-21T08:47:40.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f6725bdd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1b1c10 con 0x55dc38852e80 2024-04-21T08:47:40.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f67214b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 0x55dc3885b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f67214b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 0x55dc3885b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56832/0 (socket says 172.21.15.82:56832) 2024-04-21T08:47:40.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f67214b2640 1 -- 172.21.15.82:0/3103336877 learned_addr learned my addr 172.21.15.82:0/3103336877 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f6720cb1640 1 --2- 172.21.15.82:0/3103336877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc388524d0 0x55dc388678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f671bfff640 1 --2- 172.21.15.82:0/3103336877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc3885ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f67214b2640 1 -- 172.21.15.82:0/3103336877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 msgr2=0x55dc3885ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f67214b2640 1 --2- 172.21.15.82:0/3103336877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc3885ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f67214b2640 1 -- 172.21.15.82:0/3103336877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc388524d0 msgr2=0x55dc388678c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f67214b2640 1 --2- 172.21.15.82:0/3103336877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc388524d0 0x55dc388678c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.065+0000 7f67214b2640 1 -- 172.21.15.82:0/3103336877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc38852c20 con 0x55dc3885ba00 2024-04-21T08:47:40.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.066+0000 7f67214b2640 1 --2- 172.21.15.82:0/3103336877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 0x55dc3885b1b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6714013910 tx=0x7f6714013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9b3eab9ff49f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.066+0000 7f67067fc640 1 -- 172.21.15.82:0/3103336877 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714011980 con 0x55dc3885ba00 2024-04-21T08:47:40.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.066+0000 7f67067fc640 1 -- 172.21.15.82:0/3103336877 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6714003970 con 0x55dc3885ba00 2024-04-21T08:47:40.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.066+0000 7f67067fc640 1 -- 172.21.15.82:0/3103336877 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67140047b0 con 0x55dc3885ba00 2024-04-21T08:47:40.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 -- 172.21.15.82:0/3103336877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 msgr2=0x55dc3885b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 --2- 172.21.15.82:0/3103336877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 0x55dc3885b1b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6714013910 tx=0x7f6714013dd0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 -- 172.21.15.82:0/3103336877 shutdown_connections 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 --2- 172.21.15.82:0/3103336877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc3885ba00 0x55dc3885b1b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 --2- 172.21.15.82:0/3103336877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc3885ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 --2- 172.21.15.82:0/3103336877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc388524d0 0x55dc388678c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 -- 172.21.15.82:0/3103336877 >> 172.21.15.82:0/3103336877 conn(0x55dc3884cba0 msgr2=0x55dc3884e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 -- 172.21.15.82:0/3103336877 shutdown_connections 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 -- 172.21.15.82:0/3103336877 wait complete. 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 -- start start 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 Processor -- start 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 0x55dc38864310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.067+0000 7f6725bdd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc38855c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6725bdd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc3885ba00 0x55dc388547b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6725bdd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1b29b0 con 0x55dc388524d0 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6725bdd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1b22e0 con 0x55dc3885ba00 2024-04-21T08:47:40.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6725bdd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1af520 con 0x55dc38852e80 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6720cb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 0x55dc38864310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6725bdd740 10 client.0 _ll_register_callbacks cb 0x7f67100042f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6720cb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 0x55dc38864310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56838/0 (socket says 172.21.15.82:56838) 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6720cb1640 1 -- 172.21.15.82:0/1054915637 learned_addr learned my addr 172.21.15.82:0/1054915637 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f671bfff640 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc38855c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f67214b2640 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc3885ba00 0x55dc388547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6720cb1640 1 -- 172.21.15.82:0/1054915637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 msgr2=0x55dc38855c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6720cb1640 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc38855c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6720cb1640 1 -- 172.21.15.82:0/1054915637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc3885ba00 msgr2=0x55dc388547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6720cb1640 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc3885ba00 0x55dc388547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6720cb1640 1 -- 172.21.15.82:0/1054915637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc38854cf0 con 0x55dc388524d0 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6720cb1640 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 0x55dc38864310 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f67100059d0 tx=0x7f671002cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f326c4c32009065 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f66fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:40.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f66fbfff640 1 -- 172.21.15.82:0/1054915637 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710034b20 con 0x55dc388524d0 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f66fbfff640 1 -- 172.21.15.82:0/1054915637 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6710034cc0 con 0x55dc388524d0 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.068+0000 7f6725bdd740 1 -- 172.21.15.82:0/1054915637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6714003970 con 0x55dc388524d0 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.069+0000 7f66fbfff640 1 -- 172.21.15.82:0/1054915637 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f671003f5e0 con 0x55dc388524d0 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.069+0000 7f6725bdd740 10 client.16146 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.069+0000 7f6725bdd740 1 -- 172.21.15.82:0/1054915637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc388d16f0 con 0x55dc388524d0 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.069+0000 7f66fbfff640 1 -- 172.21.15.82:0/1054915637 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6714003970 con 0x55dc388524d0 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.069+0000 7f66fbfff640 1 -- 172.21.15.82:0/1054915637 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f671004c440 con 0x55dc388524d0 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.069+0000 7f6725bdd740 10 client.16146 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.069+0000 7f6725bdd740 10 client.16146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.069+0000 7f6725bdd740 20 client.16146 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.069+0000 7f6725bdd740 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55dc3886bbb0 0x55dc388d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.069+0000 7f6725bdd740 4 client.16146 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:40.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.069+0000 7f6725bdd740 1 -- 172.21.15.82:0/1054915637 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc387c4c10 con 0x55dc3886bbb0 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.069+0000 7f671bfff640 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55dc3886bbb0 0x55dc388d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f66fbfff640 10 client.16146 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f671bfff640 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55dc3886bbb0 0x55dc388d3c30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c32aec8861754ffe server_cookie=78d8228f882b302e in_seq=0 out_seq=0 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f66fbfff640 1 -- 172.21.15.82:0/1054915637 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc387c4c10 con 0x55dc3886bbb0 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f66fbfff640 10 client.16146 handle_command_reply: tid=0 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f6725bdd740 1 client.16146 shutdown 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f6725bdd740 1 -- 172.21.15.82:0/1054915637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 msgr2=0x55dc38864310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f6725bdd740 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 0x55dc38864310 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f67100059d0 tx=0x7f671002cce0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f6725bdd740 1 -- 172.21.15.82:0/1054915637 shutdown_connections 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f6725bdd740 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55dc3886bbb0 0x55dc388d3c30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f66fbfff640 0 client.16146 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f6725bdd740 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc3885ba00 0x55dc388547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f6725bdd740 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dc38852e80 0x55dc38855c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f6725bdd740 1 --2- 172.21.15.82:0/1054915637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc388524d0 0x55dc38864310 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f6725bdd740 1 -- 172.21.15.82:0/1054915637 >> 172.21.15.82:0/1054915637 conn(0x55dc3884cba0 msgr2=0x55dc38853260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f6725bdd740 1 -- 172.21.15.82:0/1054915637 shutdown_connections 2024-04-21T08:47:40.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f6725bdd740 1 -- 172.21.15.82:0/1054915637 wait complete. 2024-04-21T08:47:40.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.070+0000 7f6725bdd740 20 client.16146 trim_cache size 0 max 16384 2024-04-21T08:47:40.076 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:47:40.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.071+0000 7f6722f15640 1 -- 172.21.15.82:0/969104572 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f670c0a3e80 msgr2=0x7f670c0a6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.071+0000 7f6722f15640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f670c0a3e80 0x7f670c0a6340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f67100096c0 tx=0x7f671002c280 comp rx=0 tx=0).stop 2024-04-21T08:47:40.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.071+0000 7f6722f15640 1 -- 172.21.15.82:0/969104572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f671c103af0 msgr2=0x7f671c101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.071+0000 7f6722f15640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f671c103af0 0x7f671c101360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f670800e9f0 tx=0x7f670800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.071+0000 7f6722f15640 1 -- 172.21.15.82:0/969104572 shutdown_connections 2024-04-21T08:47:40.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.071+0000 7f6722f15640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c10c510 0x7f671c1018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.071+0000 7f6722f15640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f671c103af0 0x7f671c101360 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.071+0000 7f6722f15640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f670c0a3e80 0x7f670c0a6340 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.071+0000 7f6722f15640 1 --2- 172.21.15.82:0/969104572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671c1031d0 0x7f671c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.071+0000 7f6722f15640 1 -- 172.21.15.82:0/969104572 >> 172.21.15.82:0/969104572 conn(0x7f671c077f50 msgr2=0x7f671c0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.071+0000 7f6722f15640 1 -- 172.21.15.82:0/969104572 shutdown_connections 2024-04-21T08:47:40.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.071+0000 7f6722f15640 1 -- 172.21.15.82:0/969104572 wait complete. 2024-04-21T08:47:40.085 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:47:40.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:39 smithi106 ceph-mon[31072]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-04-21T08:47:40.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:39 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/993289825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:40.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.188+0000 7efde6322640 1 -- 172.21.15.82:0/4273959388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde0101130 msgr2=0x7efde0101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.188+0000 7efde6322640 1 --2- 172.21.15.82:0/4273959388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde0101130 0x7efde0101590 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009900 tx=0x7efdd402bc80 comp rx=0 tx=0).stop 2024-04-21T08:47:40.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.188+0000 7efde6322640 1 -- 172.21.15.82:0/4273959388 shutdown_connections 2024-04-21T08:47:40.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.188+0000 7efde6322640 1 --2- 172.21.15.82:0/4273959388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0101ad0 0x7efde010dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.188+0000 7efde6322640 1 --2- 172.21.15.82:0/4273959388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde0101130 0x7efde0101590 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.188+0000 7efde6322640 1 --2- 172.21.15.82:0/4273959388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efde0105ea0 0x7efde0106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.188+0000 7efde6322640 1 -- 172.21.15.82:0/4273959388 >> 172.21.15.82:0/4273959388 conn(0x7efde0077e30 msgr2=0x7efde0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.188+0000 7efde6322640 1 -- 172.21.15.82:0/4273959388 shutdown_connections 2024-04-21T08:47:40.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.188+0000 7efde6322640 1 -- 172.21.15.82:0/4273959388 wait complete. 2024-04-21T08:47:40.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.189+0000 7efde6322640 1 Processor -- start 2024-04-21T08:47:40.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.189+0000 7efde6322640 1 -- start start 2024-04-21T08:47:40.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.189+0000 7efde6322640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde0101ad0 0x7efde01a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.189+0000 7efde6322640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0105ea0 0x7efde01a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.189+0000 7efde6322640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efde019aae0 0x7efde019af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.189+0000 7efde6322640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efde0110130 con 0x7efde0101ad0 2024-04-21T08:47:40.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.189+0000 7efde6322640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efde010ffb0 con 0x7efde019aae0 2024-04-21T08:47:40.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.189+0000 7efde6322640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efde01102b0 con 0x7efde0105ea0 2024-04-21T08:47:40.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde0101ad0 0x7efde01a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde0101ad0 0x7efde01a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56862/0 (socket says 172.21.15.82:56862) 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddffff640 1 -- 172.21.15.82:0/277003430 learned_addr learned my addr 172.21.15.82:0/277003430 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efde48bf640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efde019aae0 0x7efde019af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddf7fe640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0105ea0 0x7efde01a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddffff640 1 -- 172.21.15.82:0/277003430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0105ea0 msgr2=0x7efde01a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddffff640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0105ea0 0x7efde01a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddffff640 1 -- 172.21.15.82:0/277003430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efde019aae0 msgr2=0x7efde019af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddffff640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efde019aae0 0x7efde019af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddffff640 1 -- 172.21.15.82:0/277003430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde019b600 con 0x7efde0101ad0 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddffff640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde0101ad0 0x7efde01a0930 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efdd000e9f0 tx=0x7efdd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddd7fa640 1 -- 172.21.15.82:0/277003430 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd000cd80 con 0x7efde0101ad0 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efde6322640 1 -- 172.21.15.82:0/277003430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde019b8f0 con 0x7efde0101ad0 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddd7fa640 1 -- 172.21.15.82:0/277003430 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdd00108e0 con 0x7efde0101ad0 2024-04-21T08:47:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efddd7fa640 1 -- 172.21.15.82:0/277003430 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0018980 con 0x7efde0101ad0 2024-04-21T08:47:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.190+0000 7efde6322640 1 -- 172.21.15.82:0/277003430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde01a7b90 con 0x7efde0101ad0 2024-04-21T08:47:40.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.193+0000 7efddd7fa640 1 -- 172.21.15.82:0/277003430 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7efdd0010430 con 0x7efde0101ad0 2024-04-21T08:47:40.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.194+0000 7efde8fea740 1 Processor -- start 2024-04-21T08:47:40.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.194+0000 7efde8fea740 1 -- start start 2024-04-21T08:47:40.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.194+0000 7efddd7fa640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efdc80a3e40 0x7efdc80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.194+0000 7efde8fea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce8d4d0 0x55f6bce8d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.194+0000 7efde8fea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8de80 0x55f6bce96430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.194+0000 7efde8fea740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce96a00 0x55f6bce98e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.194+0000 7efde8fea740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f6bcdf21f0 con 0x55f6bce96a00 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.194+0000 7efde8fea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f6bcd8f4b0 con 0x55f6bce8de80 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.194+0000 7efde8fea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f6bcdee1b0 con 0x55f6bce8d4d0 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.194+0000 7efddd7fa640 1 -- 172.21.15.82:0/277003430 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7efdd00d1150 con 0x7efde0101ad0 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.194+0000 7efde48bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce96a00 0x55f6bce98e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efde48bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce96a00 0x55f6bce98e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56866/0 (socket says 172.21.15.82:56866) 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efde48bf640 1 -- 172.21.15.82:0/874229147 learned_addr learned my addr 172.21.15.82:0/874229147 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efddffff640 1 --2- 172.21.15.82:0/874229147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce8d4d0 0x55f6bce8d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efddf7fe640 1 --2- 172.21.15.82:0/874229147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8de80 0x55f6bce96430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efddffff640 1 -- 172.21.15.82:0/874229147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8de80 msgr2=0x55f6bce96430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efddffff640 1 --2- 172.21.15.82:0/874229147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8de80 0x55f6bce96430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efddffff640 1 -- 172.21.15.82:0/874229147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce96a00 msgr2=0x55f6bce98e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efddffff640 1 --2- 172.21.15.82:0/874229147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce96a00 0x55f6bce98e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efddffff640 1 -- 172.21.15.82:0/874229147 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6bcd811b0 con 0x55f6bce8d4d0 2024-04-21T08:47:40.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efde48bf640 1 --2- 172.21.15.82:0/874229147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce96a00 0x55f6bce98e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:40.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efde48bf640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efdc80a3e40 0x7efdc80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efddffff640 1 --2- 172.21.15.82:0/874229147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce8d4d0 0x55f6bce8d8b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ce360 tx=0x7efdd004afb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8af524e99cfc76a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efdc67fc640 1 -- 172.21.15.82:0/874229147 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd009cb90 con 0x55f6bce8d4d0 2024-04-21T08:47:40.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efde48bf640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efdc80a3e40 0x7efdc80a6300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efdd800bfd0 tx=0x7efdd800b730 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.195+0000 7efdc67fc640 1 -- 172.21.15.82:0/874229147 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdd004d020 con 0x55f6bce8d4d0 2024-04-21T08:47:40.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.196+0000 7efdc67fc640 1 -- 172.21.15.82:0/874229147 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0052350 con 0x55f6bce8d4d0 2024-04-21T08:47:40.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.196+0000 7efde8fea740 1 -- 172.21.15.82:0/874229147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce8d4d0 msgr2=0x55f6bce8d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.196+0000 7efde8fea740 1 --2- 172.21.15.82:0/874229147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce8d4d0 0x55f6bce8d8b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ce360 tx=0x7efdd004afb0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.198+0000 7efde8fea740 1 -- 172.21.15.82:0/874229147 shutdown_connections 2024-04-21T08:47:40.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.198+0000 7efde8fea740 1 --2- 172.21.15.82:0/874229147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce96a00 0x55f6bce98e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.198+0000 7efde8fea740 1 --2- 172.21.15.82:0/874229147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8de80 0x55f6bce96430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.198+0000 7efde8fea740 1 --2- 172.21.15.82:0/874229147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce8d4d0 0x55f6bce8d8b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.198+0000 7efde8fea740 1 -- 172.21.15.82:0/874229147 >> 172.21.15.82:0/874229147 conn(0x55f6bce87ba0 msgr2=0x55f6bce87f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.198+0000 7efde8fea740 1 -- 172.21.15.82:0/874229147 shutdown_connections 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.198+0000 7efde8fea740 1 -- 172.21.15.82:0/874229147 wait complete. 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.198+0000 7efde8fea740 1 -- start start 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.198+0000 7efde8fea740 1 Processor -- start 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efde8fea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8d4d0 0x55f6bce96290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efde8fea740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 0x55f6bce93620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efde8fea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 0x55f6bce93b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efde8fea740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f6bce99970 con 0x55f6bce8de80 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efde8fea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f6bcdab500 con 0x55f6bce8d4d0 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efde8fea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f6bce99af0 con 0x55f6bce96a00 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efde8fea740 10 client.0 _ll_register_callbacks cb 0x55f6bcd79b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efddffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8d4d0 0x55f6bce96290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efddf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 0x55f6bce93620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efddffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8d4d0 0x55f6bce96290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:37488/0 (socket says 172.21.15.82:37488) 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efddffff640 1 -- 172.21.15.82:0/1578767367 learned_addr learned my addr 172.21.15.82:0/1578767367 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efde48bf640 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 0x55f6bce93b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efddffff640 1 -- 172.21.15.82:0/1578767367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 msgr2=0x55f6bce93b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efddffff640 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 0x55f6bce93b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efddffff640 1 -- 172.21.15.82:0/1578767367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 msgr2=0x55f6bce93620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efddffff640 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 0x55f6bce93620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efddffff640 1 -- 172.21.15.82:0/1578767367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6bcb043f0 con 0x55f6bce8d4d0 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.199+0000 7efddf7fe640 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 0x55f6bce93620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.200+0000 7efddffff640 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8d4d0 0x55f6bce96290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ce470 tx=0x7efdd0018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edcf6bb56216c909 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.200+0000 7efdbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.200+0000 7efdbf7fe640 1 -- 172.21.15.82:0/1578767367 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd009cb90 con 0x55f6bce8d4d0 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.200+0000 7efdbf7fe640 1 -- 172.21.15.82:0/1578767367 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7efdd00522b0 con 0x55f6bce8d4d0 2024-04-21T08:47:40.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.200+0000 7efde8fea740 1 -- 172.21.15.82:0/1578767367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd004d020 con 0x55f6bce8d4d0 2024-04-21T08:47:40.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.200+0000 7efdbf7fe640 1 -- 172.21.15.82:0/1578767367 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efdd00a5750 con 0x55f6bce8d4d0 2024-04-21T08:47:40.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.200+0000 7efde8fea740 10 client.25514 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:40.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.200+0000 7efde8fea740 1 -- 172.21.15.82:0/1578767367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6bcbce7d0 con 0x55f6bce8d4d0 2024-04-21T08:47:40.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.200+0000 7efdbf7fe640 1 -- 172.21.15.82:0/1578767367 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdd004d020 con 0x55f6bce8d4d0 2024-04-21T08:47:40.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.201+0000 7efdbf7fe640 1 -- 172.21.15.82:0/1578767367 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efdd0018c40 con 0x55f6bce8d4d0 2024-04-21T08:47:40.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.201+0000 7efde8fea740 10 client.25514 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:40.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.201+0000 7efde8fea740 10 client.25514 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:47:40.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.201+0000 7efde8fea740 20 client.25514 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:40.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.201+0000 7efde8fea740 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55f6bcea6a10 0x55f6bcea8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.201+0000 7efde8fea740 4 client.25514 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:40.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.201+0000 7efde8fea740 1 -- 172.21.15.82:0/1578767367 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6bcd91930 con 0x55f6bcea6a10 2024-04-21T08:47:40.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.202+0000 7efddf7fe640 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55f6bcea6a10 0x55f6bcea8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.202+0000 7efdbf7fe640 10 client.25514 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:47:40.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.202+0000 7efddf7fe640 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55f6bcea6a10 0x55f6bcea8e00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50d4d2374518840e server_cookie=50b3ed62667d5f5a in_seq=0 out_seq=0 2024-04-21T08:47:40.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efdbf7fe640 1 -- 172.21.15.82:0/1578767367 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6bcd91930 con 0x55f6bcea6a10 2024-04-21T08:47:40.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efdbf7fe640 10 client.25514 handle_command_reply: tid=0 2024-04-21T08:47:40.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efde8fea740 1 client.25514 shutdown 2024-04-21T08:47:40.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efde8fea740 1 -- 172.21.15.82:0/1578767367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8d4d0 msgr2=0x55f6bce96290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efde8fea740 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8d4d0 0x55f6bce96290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ce470 tx=0x7efdd0018420 comp rx=0 tx=0).stop 2024-04-21T08:47:40.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efde8fea740 1 -- 172.21.15.82:0/1578767367 shutdown_connections 2024-04-21T08:47:40.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efde8fea740 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55f6bcea6a10 0x55f6bcea8e00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efde8fea740 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 0x55f6bce93b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efde8fea740 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 0x55f6bce93620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efde8fea740 1 --2- 172.21.15.82:0/1578767367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8d4d0 0x55f6bce96290 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efde8fea740 1 -- 172.21.15.82:0/1578767367 >> 172.21.15.82:0/1578767367 conn(0x55f6bce87ba0 msgr2=0x55f6bce898f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efdbf7fe640 0 client.25514 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:47:40.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efde8fea740 1 -- 172.21.15.82:0/1578767367 shutdown_connections 2024-04-21T08:47:40.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efde8fea740 1 -- 172.21.15.82:0/1578767367 wait complete. 2024-04-21T08:47:40.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.203+0000 7efde8fea740 20 client.25514 trim_cache size 0 max 16384 2024-04-21T08:47:40.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.227+0000 7efde8fea740 1 Processor -- start 2024-04-21T08:47:40.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.227+0000 7efde8fea740 1 -- start start 2024-04-21T08:47:40.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efde8fea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8d4d0 0x55f6bce86b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efde8fea740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 0x55f6bce95d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efde8fea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 0x55f6bce96290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efde8fea740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efde01b1a90 con 0x55f6bce8de80 2024-04-21T08:47:40.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efde8fea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efde01b35d0 con 0x55f6bce8d4d0 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efde8fea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efde01b2160 con 0x55f6bce96a00 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efddf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 0x55f6bce95d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efddf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 0x55f6bce95d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56890/0 (socket says 172.21.15.82:56890) 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efddf7fe640 1 -- 172.21.15.82:0/1077668773 learned_addr learned my addr 172.21.15.82:0/1077668773 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efde48bf640 1 --2- 172.21.15.82:0/1077668773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 0x55f6bce96290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efddffff640 1 --2- 172.21.15.82:0/1077668773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8d4d0 0x55f6bce86b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efddf7fe640 1 -- 172.21.15.82:0/1077668773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 msgr2=0x55f6bce96290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efddf7fe640 1 --2- 172.21.15.82:0/1077668773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 0x55f6bce96290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efddf7fe640 1 -- 172.21.15.82:0/1077668773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8d4d0 msgr2=0x55f6bce86b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efddf7fe640 1 --2- 172.21.15.82:0/1077668773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8d4d0 0x55f6bce86b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efddf7fe640 1 -- 172.21.15.82:0/1077668773 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6bce98df0 con 0x55f6bce8de80 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.228+0000 7efddf7fe640 1 --2- 172.21.15.82:0/1077668773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 0x55f6bce95d50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efde0079260 tx=0x7efdd4005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa55160646e02ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.229+0000 7efdc5ffb640 1 -- 172.21.15.82:0/1077668773 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4004040 con 0x55f6bce8de80 2024-04-21T08:47:40.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.229+0000 7efdc5ffb640 1 -- 172.21.15.82:0/1077668773 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdd40054f0 con 0x55f6bce8de80 2024-04-21T08:47:40.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.229+0000 7efdc5ffb640 1 -- 172.21.15.82:0/1077668773 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4004b60 con 0x55f6bce8de80 2024-04-21T08:47:40.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.229+0000 7efde8fea740 1 -- 172.21.15.82:0/1077668773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 msgr2=0x55f6bce95d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.229+0000 7efde8fea740 1 --2- 172.21.15.82:0/1077668773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 0x55f6bce95d50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efde0079260 tx=0x7efdd4005e70 comp rx=0 tx=0).stop 2024-04-21T08:47:40.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.229+0000 7efde8fea740 1 -- 172.21.15.82:0/1077668773 shutdown_connections 2024-04-21T08:47:40.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.229+0000 7efde8fea740 1 --2- 172.21.15.82:0/1077668773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 0x55f6bce96290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.229+0000 7efde8fea740 1 --2- 172.21.15.82:0/1077668773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8de80 0x55f6bce95d50 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.229+0000 7efde8fea740 1 --2- 172.21.15.82:0/1077668773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8d4d0 0x55f6bce86b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.229+0000 7efde8fea740 1 -- 172.21.15.82:0/1077668773 >> 172.21.15.82:0/1077668773 conn(0x55f6bce87ba0 msgr2=0x55f6bcea25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.229+0000 7efde8fea740 1 -- 172.21.15.82:0/1077668773 shutdown_connections 2024-04-21T08:47:40.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efde8fea740 1 -- 172.21.15.82:0/1077668773 wait complete. 2024-04-21T08:47:40.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efde8fea740 1 -- start start 2024-04-21T08:47:40.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efde8fea740 1 Processor -- start 2024-04-21T08:47:40.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efde8fea740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8d4d0 0x55f6bceaa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efde8fea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8de80 0x55f6bceaab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efde8fea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 0x55f6bceab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efde8fea740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efde01b2f00 con 0x55f6bce8d4d0 2024-04-21T08:47:40.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efde8fea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efde01b2830 con 0x55f6bce8de80 2024-04-21T08:47:40.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efde8fea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efde01afb20 con 0x55f6bce96a00 2024-04-21T08:47:40.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efde8fea740 10 client.0 _ll_register_callbacks cb 0x7efdd0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:40.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efddffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8d4d0 0x55f6bceaa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efddffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8d4d0 0x55f6bceaa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56898/0 (socket says 172.21.15.82:56898) 2024-04-21T08:47:40.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.230+0000 7efddffff640 1 -- 172.21.15.82:0/3525616734 learned_addr learned my addr 172.21.15.82:0/3525616734 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efddf7fe640 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8de80 0x55f6bceaab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efde48bf640 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 0x55f6bceab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efddffff640 1 -- 172.21.15.82:0/3525616734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 msgr2=0x55f6bceab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efddffff640 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 0x55f6bceab060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efddffff640 1 -- 172.21.15.82:0/3525616734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8de80 msgr2=0x55f6bceaab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efddffff640 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8de80 0x55f6bceaab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efddffff640 1 -- 172.21.15.82:0/3525616734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6bcf0ad20 con 0x55f6bce8d4d0 2024-04-21T08:47:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efddffff640 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8d4d0 0x55f6bceaa5e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efdd009ae40 tx=0x7efdd001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1425e3273631363d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efdbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efdbf7fe640 1 -- 172.21.15.82:0/3525616734 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd009cb90 con 0x55f6bce8d4d0 2024-04-21T08:47:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efdbf7fe640 1 -- 172.21.15.82:0/3525616734 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7efdd009baf0 con 0x55f6bce8d4d0 2024-04-21T08:47:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efde8fea740 1 -- 172.21.15.82:0/3525616734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd40054f0 con 0x55f6bce8d4d0 2024-04-21T08:47:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efdbf7fe640 1 -- 172.21.15.82:0/3525616734 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efdd00a9050 con 0x55f6bce8d4d0 2024-04-21T08:47:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efde8fea740 10 client.16164 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.231+0000 7efde8fea740 1 -- 172.21.15.82:0/3525616734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6bcf0c020 con 0x55f6bce8d4d0 2024-04-21T08:47:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.232+0000 7efdbf7fe640 1 -- 172.21.15.82:0/3525616734 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdd004d030 con 0x55f6bce8d4d0 2024-04-21T08:47:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.232+0000 7efdbf7fe640 1 -- 172.21.15.82:0/3525616734 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efdd00a2140 con 0x55f6bce8d4d0 2024-04-21T08:47:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.232+0000 7efde8fea740 10 client.16164 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.232+0000 7efde8fea740 10 client.16164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:47:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.232+0000 7efde8fea740 20 client.16164 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.232+0000 7efde8fea740 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55f6bcf0c500 0x55f6bcf0e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.232+0000 7efde8fea740 4 client.16164 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.232+0000 7efde8fea740 1 -- 172.21.15.82:0/3525616734 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6bcdffc10 con 0x55f6bcf0c500 2024-04-21T08:47:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.232+0000 7efddf7fe640 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55f6bcf0c500 0x55f6bcf0e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efdbf7fe640 10 client.16164 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:47:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efddf7fe640 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55f6bcf0c500 0x55f6bcf0e8f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8c577c2155d2eba server_cookie=2bd600732e17aa81 in_seq=0 out_seq=0 2024-04-21T08:47:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efdbf7fe640 1 -- 172.21.15.82:0/3525616734 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6bcdffc10 con 0x55f6bcf0c500 2024-04-21T08:47:40.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efdbf7fe640 10 client.16164 handle_command_reply: tid=0 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efde8fea740 1 client.16164 shutdown 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efde8fea740 1 -- 172.21.15.82:0/3525616734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8d4d0 msgr2=0x55f6bceaa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efde8fea740 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8d4d0 0x55f6bceaa5e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efdd009ae40 tx=0x7efdd001f000 comp rx=0 tx=0).stop 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efde8fea740 1 -- 172.21.15.82:0/3525616734 shutdown_connections 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efde8fea740 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55f6bcf0c500 0x55f6bcf0e8f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efde8fea740 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f6bce96a00 0x55f6bceab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efdbf7fe640 0 client.16164 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efde8fea740 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f6bce8de80 0x55f6bceaab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efde8fea740 1 --2- 172.21.15.82:0/3525616734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bce8d4d0 0x55f6bceaa5e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efde8fea740 1 -- 172.21.15.82:0/3525616734 >> 172.21.15.82:0/3525616734 conn(0x55f6bce87ba0 msgr2=0x55f6bce89820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efde8fea740 1 -- 172.21.15.82:0/3525616734 shutdown_connections 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.233+0000 7efde8fea740 1 -- 172.21.15.82:0/3525616734 wait complete. 2024-04-21T08:47:40.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde8fea740 20 client.16164 trim_cache size 0 max 16384 2024-04-21T08:47:40.242 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:47:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde6322640 1 -- 172.21.15.82:0/277003430 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efdc80a3e40 msgr2=0x7efdc80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde6322640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efdc80a3e40 0x7efdc80a6300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efdd800bfd0 tx=0x7efdd800b730 comp rx=0 tx=0).stop 2024-04-21T08:47:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde6322640 1 -- 172.21.15.82:0/277003430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde0101ad0 msgr2=0x7efde01a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde6322640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde0101ad0 0x7efde01a0930 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efdd000e9f0 tx=0x7efdd000eeb0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde6322640 1 -- 172.21.15.82:0/277003430 shutdown_connections 2024-04-21T08:47:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde6322640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efde019aae0 0x7efde019af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde6322640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0105ea0 0x7efde01a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde6322640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efdc80a3e40 0x7efdc80a6300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde6322640 1 --2- 172.21.15.82:0/277003430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde0101ad0 0x7efde01a0930 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde6322640 1 -- 172.21.15.82:0/277003430 >> 172.21.15.82:0/277003430 conn(0x7efde0077e30 msgr2=0x7efde00ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde6322640 1 -- 172.21.15.82:0/277003430 shutdown_connections 2024-04-21T08:47:40.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.234+0000 7efde6322640 1 -- 172.21.15.82:0/277003430 wait complete. 2024-04-21T08:47:40.249 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:47:40.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.351+0000 7ff263643640 1 -- 172.21.15.82:0/297273718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff25c101a40 msgr2=0x7ff25c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.351+0000 7ff263643640 1 --2- 172.21.15.82:0/297273718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff25c101a40 0x7ff25c105a80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009900 tx=0x7ff24c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:40.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.351+0000 7ff263643640 1 -- 172.21.15.82:0/297273718 shutdown_connections 2024-04-21T08:47:40.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.351+0000 7ff263643640 1 --2- 172.21.15.82:0/297273718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff25c106160 0x7ff25c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.351+0000 7ff263643640 1 --2- 172.21.15.82:0/297273718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff25c101a40 0x7ff25c105a80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.351+0000 7ff263643640 1 --2- 172.21.15.82:0/297273718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff25c101090 0x7ff25c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.351+0000 7ff263643640 1 -- 172.21.15.82:0/297273718 >> 172.21.15.82:0/297273718 conn(0x7ff25c077f50 msgr2=0x7ff25c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.352+0000 7ff263643640 1 -- 172.21.15.82:0/297273718 shutdown_connections 2024-04-21T08:47:40.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.352+0000 7ff263643640 1 -- 172.21.15.82:0/297273718 wait complete. 2024-04-21T08:47:40.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.352+0000 7ff263643640 1 Processor -- start 2024-04-21T08:47:40.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.352+0000 7ff263643640 1 -- start start 2024-04-21T08:47:40.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.352+0000 7ff263643640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff25c101090 0x7ff25c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff2613df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff25c101090 0x7ff25c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff263643640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff25c106160 0x7ff25c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff263643640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff25c19a770 0x7ff25c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff2613df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff25c101090 0x7ff25c1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56926/0 (socket says 172.21.15.82:56926) 2024-04-21T08:47:40.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff263643640 1 -- 172.21.15.82:0/2064455501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c10ffc0 con 0x7ff25c101090 2024-04-21T08:47:40.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff2613df640 1 -- 172.21.15.82:0/2064455501 learned_addr learned my addr 172.21.15.82:0/2064455501 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff263643640 1 -- 172.21.15.82:0/2064455501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c10fe40 con 0x7ff25c19a770 2024-04-21T08:47:40.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff263643640 1 -- 172.21.15.82:0/2064455501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c110140 con 0x7ff25c106160 2024-04-21T08:47:40.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff260bde640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff25c106160 0x7ff25c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff2613df640 1 -- 172.21.15.82:0/2064455501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff25c106160 msgr2=0x7ff25c1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff2613df640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff25c106160 0x7ff25c1a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff261be0640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff25c19a770 0x7ff25c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff2613df640 1 -- 172.21.15.82:0/2064455501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff25c19a770 msgr2=0x7ff25c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff2613df640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff25c19a770 0x7ff25c19abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff2613df640 1 -- 172.21.15.82:0/2064455501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c19b370 con 0x7ff25c101090 2024-04-21T08:47:40.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff2613df640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff25c101090 0x7ff25c1a0660 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff25800e9f0 tx=0x7ff25800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff2527fc640 1 -- 172.21.15.82:0/2064455501 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25800cd80 con 0x7ff25c101090 2024-04-21T08:47:40.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff263643640 1 -- 172.21.15.82:0/2064455501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25c1a7400 con 0x7ff25c101090 2024-04-21T08:47:40.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff263643640 1 -- 172.21.15.82:0/2064455501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25c1a7800 con 0x7ff25c101090 2024-04-21T08:47:40.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff2527fc640 1 -- 172.21.15.82:0/2064455501 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2580054d0 con 0x7ff25c101090 2024-04-21T08:47:40.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.353+0000 7ff2527fc640 1 -- 172.21.15.82:0/2064455501 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258010690 con 0x7ff25c101090 2024-04-21T08:47:40.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.356+0000 7ff266239740 1 Processor -- start 2024-04-21T08:47:40.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.356+0000 7ff266239740 1 -- start start 2024-04-21T08:47:40.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.357+0000 7ff266239740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6ef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.357+0000 7ff266239740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.357+0000 7ff266239740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f8a00 0x55bb4d6fae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.357+0000 7ff266239740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bb4d6541f0 con 0x55bb4d6f8a00 2024-04-21T08:47:40.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.357+0000 7ff266239740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bb4d5f14b0 con 0x55bb4d6efe80 2024-04-21T08:47:40.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.357+0000 7ff266239740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bb4d6501b0 con 0x55bb4d6ef4d0 2024-04-21T08:47:40.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.357+0000 7ff260bde640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f8a00 0x55bb4d6fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.357+0000 7ff2527fc640 1 -- 172.21.15.82:0/2064455501 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff258010830 con 0x7ff25c101090 2024-04-21T08:47:40.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.357+0000 7ff260bde640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f8a00 0x55bb4d6fae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56938/0 (socket says 172.21.15.82:56938) 2024-04-21T08:47:40.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.357+0000 7ff260bde640 1 -- 172.21.15.82:0/4091036113 learned_addr learned my addr 172.21.15.82:0/4091036113 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.357+0000 7ff261be0640 1 --2- 172.21.15.82:0/4091036113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6ef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff2613df640 1 --2- 172.21.15.82:0/4091036113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff260bde640 1 -- 172.21.15.82:0/4091036113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 msgr2=0x55bb4d6ef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff260bde640 1 --2- 172.21.15.82:0/4091036113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6ef8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff260bde640 1 -- 172.21.15.82:0/4091036113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 msgr2=0x55bb4d6f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff260bde640 1 --2- 172.21.15.82:0/4091036113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff260bde640 1 -- 172.21.15.82:0/4091036113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4d5e31b0 con 0x55bb4d6f8a00 2024-04-21T08:47:40.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff2527fc640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff2480a3f20 0x7ff2480a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff261be0640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff2480a3f20 0x7ff2480a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff2527fc640 1 -- 172.21.15.82:0/2064455501 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff2580d1310 con 0x7ff25c101090 2024-04-21T08:47:40.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff260bde640 1 --2- 172.21.15.82:0/4091036113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f8a00 0x55bb4d6fae50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff24c006010 tx=0x7ff24c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b69fe5189cc232 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff2477fe640 1 -- 172.21.15.82:0/4091036113 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c043020 con 0x55bb4d6f8a00 2024-04-21T08:47:40.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff2477fe640 1 -- 172.21.15.82:0/4091036113 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24c02c7e0 con 0x55bb4d6f8a00 2024-04-21T08:47:40.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.358+0000 7ff2477fe640 1 -- 172.21.15.82:0/4091036113 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c005510 con 0x55bb4d6f8a00 2024-04-21T08:47:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.359+0000 7ff266239740 1 -- 172.21.15.82:0/4091036113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f8a00 msgr2=0x55bb4d6fae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.359+0000 7ff261be0640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff2480a3f20 0x7ff2480a63e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff254005cc0 tx=0x7ff25400b580 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.359+0000 7ff266239740 1 --2- 172.21.15.82:0/4091036113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f8a00 0x55bb4d6fae50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff24c006010 tx=0x7ff24c005dd0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.359+0000 7ff266239740 1 -- 172.21.15.82:0/4091036113 shutdown_connections 2024-04-21T08:47:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.359+0000 7ff266239740 1 --2- 172.21.15.82:0/4091036113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f8a00 0x55bb4d6fae50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.359+0000 7ff266239740 1 --2- 172.21.15.82:0/4091036113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.359+0000 7ff266239740 1 --2- 172.21.15.82:0/4091036113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6ef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.359+0000 7ff266239740 1 -- 172.21.15.82:0/4091036113 >> 172.21.15.82:0/4091036113 conn(0x55bb4d6e9ba0 msgr2=0x55bb4d6e9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.359+0000 7ff266239740 1 -- 172.21.15.82:0/4091036113 shutdown_connections 2024-04-21T08:47:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.359+0000 7ff266239740 1 -- 172.21.15.82:0/4091036113 wait complete. 2024-04-21T08:47:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff266239740 1 -- start start 2024-04-21T08:47:40.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff266239740 1 Processor -- start 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff266239740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff266239740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff266239740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 0x55bb4d704d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff266239740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bb4d6fb910 con 0x55bb4d6f5d50 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff266239740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bb4d60d500 con 0x55bb4d6efe80 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff266239740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bb4d6fba90 con 0x55bb4d6ef4d0 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff266239740 10 client.0 _ll_register_callbacks cb 0x55bb4d5dbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff261be0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 0x55bb4d704d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff261be0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 0x55bb4d704d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56950/0 (socket says 172.21.15.82:56950) 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff261be0640 1 -- 172.21.15.82:0/463833569 learned_addr learned my addr 172.21.15.82:0/463833569 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff260bde640 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.360+0000 7ff2613df640 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff261be0640 1 -- 172.21.15.82:0/463833569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 msgr2=0x55bb4d6f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff261be0640 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff261be0640 1 -- 172.21.15.82:0/463833569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 msgr2=0x55bb4d6f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff261be0640 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff261be0640 1 -- 172.21.15.82:0/463833569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4d3663f0 con 0x55bb4d6f5d50 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff261be0640 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 0x55bb4d704d80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff25400ecf0 tx=0x7ff254004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1d3c9daa79c6cf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff244ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff244ff9640 1 -- 172.21.15.82:0/463833569 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25401d420 con 0x55bb4d6f5d50 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff244ff9640 1 -- 172.21.15.82:0/463833569 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff25401d5c0 con 0x55bb4d6f5d50 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff266239740 1 -- 172.21.15.82:0/463833569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff24c02c7e0 con 0x55bb4d6f5d50 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff244ff9640 1 -- 172.21.15.82:0/463833569 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff25401d7c0 con 0x55bb4d6f5d50 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff266239740 10 client.16176 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.361+0000 7ff266239740 1 -- 172.21.15.82:0/463833569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb4d4307d0 con 0x55bb4d6f5d50 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.362+0000 7ff244ff9640 1 -- 172.21.15.82:0/463833569 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24c02c7e0 con 0x55bb4d6f5d50 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.362+0000 7ff244ff9640 1 -- 172.21.15.82:0/463833569 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff2540259d0 con 0x55bb4d6f5d50 2024-04-21T08:47:40.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.362+0000 7ff266239740 10 client.16176 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:40.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.362+0000 7ff266239740 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:40.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.362+0000 7ff266239740 20 client.16176 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:40.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.362+0000 7ff266239740 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55bb4d709530 0x55bb4d70b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.362+0000 7ff266239740 4 client.16176 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:40.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.362+0000 7ff266239740 1 -- 172.21.15.82:0/463833569 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb4d5f3930 con 0x55bb4d709530 2024-04-21T08:47:40.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.362+0000 7ff2613df640 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55bb4d709530 0x55bb4d70b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.363+0000 7ff244ff9640 10 client.16176 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:40.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.363+0000 7ff2613df640 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55bb4d709530 0x55bb4d70b9b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b2cd89aedce9d64 server_cookie=e596ef581a1ef513 in_seq=0 out_seq=0 2024-04-21T08:47:40.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.363+0000 7ff244ff9640 1 -- 172.21.15.82:0/463833569 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb4d5f3930 con 0x55bb4d709530 2024-04-21T08:47:40.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.363+0000 7ff244ff9640 10 client.16176 handle_command_reply: tid=0 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.363+0000 7ff266239740 1 client.16176 shutdown 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.363+0000 7ff266239740 1 -- 172.21.15.82:0/463833569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 msgr2=0x55bb4d704d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.363+0000 7ff266239740 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 0x55bb4d704d80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff25400ecf0 tx=0x7ff254004a40 comp rx=0 tx=0).stop 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.364+0000 7ff266239740 1 -- 172.21.15.82:0/463833569 shutdown_connections 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.364+0000 7ff266239740 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55bb4d709530 0x55bb4d70b9b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.364+0000 7ff266239740 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 0x55bb4d704d80 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.364+0000 7ff266239740 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.364+0000 7ff244ff9640 0 client.16176 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.364+0000 7ff266239740 1 --2- 172.21.15.82:0/463833569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.364+0000 7ff266239740 1 -- 172.21.15.82:0/463833569 >> 172.21.15.82:0/463833569 conn(0x55bb4d6e9ba0 msgr2=0x55bb4d6eb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.364+0000 7ff266239740 1 -- 172.21.15.82:0/463833569 shutdown_connections 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.364+0000 7ff266239740 1 -- 172.21.15.82:0/463833569 wait complete. 2024-04-21T08:47:40.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.364+0000 7ff266239740 20 client.16176 trim_cache size 0 max 16384 2024-04-21T08:47:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff266239740 1 Processor -- start 2024-04-21T08:47:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff266239740 1 -- start start 2024-04-21T08:47:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff266239740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff266239740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff266239740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 0x55bb4d7112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff266239740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff24000ff70 con 0x55bb4d6f5d50 2024-04-21T08:47:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff266239740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff240011ab0 con 0x55bb4d6efe80 2024-04-21T08:47:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff266239740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff240010640 con 0x55bb4d6ef4d0 2024-04-21T08:47:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff261be0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 0x55bb4d7112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff261be0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 0x55bb4d7112c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56964/0 (socket says 172.21.15.82:56964) 2024-04-21T08:47:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff261be0640 1 -- 172.21.15.82:0/2324224158 learned_addr learned my addr 172.21.15.82:0/2324224158 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff2613df640 1 --2- 172.21.15.82:0/2324224158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.390+0000 7ff260bde640 1 --2- 172.21.15.82:0/2324224158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.391+0000 7ff261be0640 1 -- 172.21.15.82:0/2324224158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 msgr2=0x55bb4d6f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.391+0000 7ff261be0640 1 --2- 172.21.15.82:0/2324224158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.391+0000 7ff261be0640 1 -- 172.21.15.82:0/2324224158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 msgr2=0x55bb4d6f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.391+0000 7ff261be0640 1 --2- 172.21.15.82:0/2324224158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.391+0000 7ff261be0640 1 -- 172.21.15.82:0/2324224158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb4d6f4660 con 0x55bb4d6f5d50 2024-04-21T08:47:40.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.391+0000 7ff260bde640 1 --2- 172.21.15.82:0/2324224158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:40.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.391+0000 7ff2613df640 1 --2- 172.21.15.82:0/2324224158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:40.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.391+0000 7ff261be0640 1 --2- 172.21.15.82:0/2324224158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 0x55bb4d7112c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff25400bc00 tx=0x7ff25400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6327d3200a8e1534 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.391+0000 7ff246ffd640 1 -- 172.21.15.82:0/2324224158 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25401d620 con 0x55bb4d6f5d50 2024-04-21T08:47:40.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.391+0000 7ff246ffd640 1 -- 172.21.15.82:0/2324224158 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff254004ab0 con 0x55bb4d6f5d50 2024-04-21T08:47:40.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.391+0000 7ff246ffd640 1 -- 172.21.15.82:0/2324224158 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25401b880 con 0x55bb4d6f5d50 2024-04-21T08:47:40.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.392+0000 7ff266239740 1 -- 172.21.15.82:0/2324224158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 msgr2=0x55bb4d7112c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.392+0000 7ff266239740 1 --2- 172.21.15.82:0/2324224158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 0x55bb4d7112c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff25400bc00 tx=0x7ff25400ec80 comp rx=0 tx=0).stop 2024-04-21T08:47:40.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.392+0000 7ff266239740 1 -- 172.21.15.82:0/2324224158 shutdown_connections 2024-04-21T08:47:40.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.392+0000 7ff266239740 1 --2- 172.21.15.82:0/2324224158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6f5d50 0x55bb4d7112c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.392+0000 7ff266239740 1 --2- 172.21.15.82:0/2324224158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.392+0000 7ff266239740 1 --2- 172.21.15.82:0/2324224158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.392+0000 7ff266239740 1 -- 172.21.15.82:0/2324224158 >> 172.21.15.82:0/2324224158 conn(0x55bb4d6e9ba0 msgr2=0x55bb4d6f0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.392+0000 7ff266239740 1 -- 172.21.15.82:0/2324224158 shutdown_connections 2024-04-21T08:47:40.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.392+0000 7ff266239740 1 -- 172.21.15.82:0/2324224158 wait complete. 2024-04-21T08:47:40.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff266239740 1 -- start start 2024-04-21T08:47:40.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff266239740 1 Processor -- start 2024-04-21T08:47:40.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff266239740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff266239740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff266239740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6f5d50 0x55bb4d6f3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff266239740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff2400113e0 con 0x55bb4d6efe80 2024-04-21T08:47:40.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff266239740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff240010d10 con 0x55bb4d6f5d50 2024-04-21T08:47:40.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff266239740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff24000df50 con 0x55bb4d6ef4d0 2024-04-21T08:47:40.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff266239740 10 client.0 _ll_register_callbacks cb 0x7ff25400e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:40.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff260bde640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff260bde640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56980/0 (socket says 172.21.15.82:56980) 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff260bde640 1 -- 172.21.15.82:0/435974815 learned_addr learned my addr 172.21.15.82:0/435974815 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff2613df640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.393+0000 7ff261be0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6f5d50 0x55bb4d6f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff260bde640 1 -- 172.21.15.82:0/435974815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 msgr2=0x55bb4d6f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff260bde640 1 --2- 172.21.15.82:0/435974815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff260bde640 1 -- 172.21.15.82:0/435974815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6f5d50 msgr2=0x55bb4d6f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff260bde640 1 --2- 172.21.15.82:0/435974815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6f5d50 0x55bb4d6f3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff260bde640 1 -- 172.21.15.82:0/435974815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb4d76ed80 con 0x55bb4d6efe80 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff2613df640 1 --2- 172.21.15.82:0/435974815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff260bde640 1 --2- 172.21.15.82:0/435974815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f2b10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff24c005940 tx=0x7ff24c02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd06492a7b454894 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff244ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff244ff9640 1 -- 172.21.15.82:0/435974815 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c043070 con 0x55bb4d6efe80 2024-04-21T08:47:40.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff244ff9640 1 -- 172.21.15.82:0/435974815 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff24c005230 con 0x55bb4d6efe80 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff266239740 1 -- 172.21.15.82:0/435974815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff254004ab0 con 0x55bb4d6efe80 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff244ff9640 1 -- 172.21.15.82:0/435974815 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff24c02cc90 con 0x55bb4d6efe80 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff266239740 10 client.16182 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.394+0000 7ff266239740 1 -- 172.21.15.82:0/435974815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb4d770060 con 0x55bb4d6efe80 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.395+0000 7ff244ff9640 1 -- 172.21.15.82:0/435974815 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24c005430 con 0x55bb4d6efe80 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.395+0000 7ff244ff9640 1 -- 172.21.15.82:0/435974815 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff24c0035f0 con 0x55bb4d6efe80 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.395+0000 7ff266239740 10 client.16182 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.395+0000 7ff266239740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.395+0000 7ff266239740 20 client.16182 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.395+0000 7ff266239740 1 --2- 172.21.15.82:0/435974815 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55bb4d770340 0x55bb4d772730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.395+0000 7ff266239740 4 client.16182 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.395+0000 7ff266239740 1 -- 172.21.15.82:0/435974815 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff254004ab0 con 0x55bb4d770340 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.395+0000 7ff2613df640 1 --2- 172.21.15.82:0/435974815 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55bb4d770340 0x55bb4d772730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:40.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.395+0000 7ff244ff9640 10 client.16182 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff2613df640 1 --2- 172.21.15.82:0/435974815 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55bb4d770340 0x55bb4d772730 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f7c9e407f3d344 server_cookie=9f14df962a7513ba in_seq=0 out_seq=0 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff244ff9640 1 -- 172.21.15.82:0/435974815 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff254004ab0 con 0x55bb4d770340 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff244ff9640 10 client.16182 handle_command_reply: tid=0 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff266239740 1 client.16182 shutdown 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff266239740 1 -- 172.21.15.82:0/435974815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6efe80 msgr2=0x55bb4d6f2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff266239740 1 --2- 172.21.15.82:0/435974815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f2b10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff24c005940 tx=0x7ff24c02ca10 comp rx=0 tx=0).stop 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff266239740 1 -- 172.21.15.82:0/435974815 shutdown_connections 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff266239740 1 --2- 172.21.15.82:0/435974815 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55bb4d770340 0x55bb4d772730 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff266239740 1 --2- 172.21.15.82:0/435974815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb4d6f5d50 0x55bb4d6f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff244ff9640 0 client.16182 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff266239740 1 --2- 172.21.15.82:0/435974815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb4d6efe80 0x55bb4d6f2b10 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff266239740 1 --2- 172.21.15.82:0/435974815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bb4d6ef4d0 0x55bb4d6f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff266239740 1 -- 172.21.15.82:0/435974815 >> 172.21.15.82:0/435974815 conn(0x55bb4d6e9ba0 msgr2=0x55bb4d6e8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff266239740 1 -- 172.21.15.82:0/435974815 shutdown_connections 2024-04-21T08:47:40.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff266239740 1 -- 172.21.15.82:0/435974815 wait complete. 2024-04-21T08:47:40.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.396+0000 7ff266239740 20 client.16182 trim_cache size 0 max 16384 2024-04-21T08:47:40.403 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:47:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.397+0000 7ff263643640 1 -- 172.21.15.82:0/2064455501 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff2480a3f20 msgr2=0x7ff2480a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.397+0000 7ff263643640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff2480a3f20 0x7ff2480a63e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff254005cc0 tx=0x7ff25400b580 comp rx=0 tx=0).stop 2024-04-21T08:47:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.397+0000 7ff263643640 1 -- 172.21.15.82:0/2064455501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff25c101090 msgr2=0x7ff25c1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.397+0000 7ff263643640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff25c101090 0x7ff25c1a0660 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff25800e9f0 tx=0x7ff25800edf0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.397+0000 7ff263643640 1 -- 172.21.15.82:0/2064455501 shutdown_connections 2024-04-21T08:47:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.397+0000 7ff263643640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff25c19a770 0x7ff25c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.397+0000 7ff263643640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff25c106160 0x7ff25c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.397+0000 7ff263643640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff2480a3f20 0x7ff2480a63e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.397+0000 7ff263643640 1 --2- 172.21.15.82:0/2064455501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff25c101090 0x7ff25c1a0660 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.397+0000 7ff263643640 1 -- 172.21.15.82:0/2064455501 >> 172.21.15.82:0/2064455501 conn(0x7ff25c077f50 msgr2=0x7ff25c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.397+0000 7ff263643640 1 -- 172.21.15.82:0/2064455501 shutdown_connections 2024-04-21T08:47:40.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:40.397+0000 7ff263643640 1 -- 172.21.15.82:0/2064455501 wait complete. 2024-04-21T08:47:41.412 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:47:41.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:47:41.412 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:47:41.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.520+0000 7fea91ac0640 1 -- 172.21.15.82:0/3924308712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea8c1021a0 msgr2=0x7fea8c102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.520+0000 7fea91ac0640 1 --2- 172.21.15.82:0/3924308712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea8c1021a0 0x7fea8c102580 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009900 tx=0x7fea7c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:41.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.520+0000 7fea91ac0640 1 -- 172.21.15.82:0/3924308712 shutdown_connections 2024-04-21T08:47:41.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.520+0000 7fea91ac0640 1 --2- 172.21.15.82:0/3924308712 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea8c107270 0x7fea8c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.520+0000 7fea91ac0640 1 --2- 172.21.15.82:0/3924308712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea8c102b50 0x7fea8c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.520+0000 7fea91ac0640 1 --2- 172.21.15.82:0/3924308712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea8c1021a0 0x7fea8c102580 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.520+0000 7fea91ac0640 1 -- 172.21.15.82:0/3924308712 >> 172.21.15.82:0/3924308712 conn(0x7fea8c079350 msgr2=0x7fea8c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:41.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.520+0000 7fea91ac0640 1 -- 172.21.15.82:0/3924308712 shutdown_connections 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.520+0000 7fea91ac0640 1 -- 172.21.15.82:0/3924308712 wait complete. 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea91ac0640 1 Processor -- start 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea91ac0640 1 -- start start 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea91ac0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea8c1021a0 0x7fea8c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea91ac0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea8c107270 0x7fea8c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea91ac0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea8c198690 0x7fea8c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea91ac0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c112230 con 0x7fea8c1021a0 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea91ac0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1120b0 con 0x7fea8c107270 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea91ac0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1123b0 con 0x7fea8c198690 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea8b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea8c1021a0 0x7fea8c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea8b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea8c1021a0 0x7fea8c19e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56998/0 (socket says 172.21.15.82:56998) 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea8b7fe640 1 -- 172.21.15.82:0/3556817025 learned_addr learned my addr 172.21.15.82:0/3556817025 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea8affd640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea8c107270 0x7fea8c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea8b7fe640 1 -- 172.21.15.82:0/3556817025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea8c198690 msgr2=0x7fea8c198b10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:47:41.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea8c198690 0x7fea8c198b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea8b7fe640 1 -- 172.21.15.82:0/3556817025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea8c107270 msgr2=0x7fea8c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea8c107270 0x7fea8c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.521+0000 7fea8b7fe640 1 -- 172.21.15.82:0/3556817025 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c199290 con 0x7fea8c1021a0 2024-04-21T08:47:41.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.522+0000 7fea8affd640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea8c107270 0x7fea8c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:41.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.522+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea8c1021a0 0x7fea8c19e580 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fea7c005940 tx=0x7fea7c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:41.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.522+0000 7fea88ff9640 1 -- 172.21.15.82:0/3556817025 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c043020 con 0x7fea8c1021a0 2024-04-21T08:47:41.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.522+0000 7fea91ac0640 1 -- 172.21.15.82:0/3556817025 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8c199520 con 0x7fea8c1021a0 2024-04-21T08:47:41.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.522+0000 7fea91ac0640 1 -- 172.21.15.82:0/3556817025 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8c1a56c0 con 0x7fea8c1021a0 2024-04-21T08:47:41.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.522+0000 7fea88ff9640 1 -- 172.21.15.82:0/3556817025 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea7c02caf0 con 0x7fea8c1021a0 2024-04-21T08:47:41.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.522+0000 7fea88ff9640 1 -- 172.21.15.82:0/3556817025 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c003ac0 con 0x7fea8c1021a0 2024-04-21T08:47:41.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.524+0000 7fea88ff9640 1 -- 172.21.15.82:0/3556817025 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fea7c046020 con 0x7fea8c1021a0 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.525+0000 7fea88ff9640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fea740a8450 0x7fea740aa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.525+0000 7fea88ff9640 1 -- 172.21.15.82:0/3556817025 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fea7c003860 con 0x7fea8c1021a0 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.525+0000 7fea8affd640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fea740a8450 0x7fea740aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.525+0000 7fea94781740 1 Processor -- start 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.525+0000 7fea94781740 1 -- start start 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.525+0000 7fea94781740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.525+0000 7fea94781740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b3290 0x5633985bb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.525+0000 7fea94781740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985bbe10 0x5633985be260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea94781740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563398516f00 con 0x5633985b28e0 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea94781740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5633984b4590 con 0x5633985b3290 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea94781740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563398512ec0 con 0x5633985bbe10 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea8affd640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fea740a8450 0x7fea740aa910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fea8c199e90 tx=0x7fea80008040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:41.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea8b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea8b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57014/0 (socket says 172.21.15.82:57014) 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea8b7fe640 1 -- 172.21.15.82:0/3773724630 learned_addr learned my addr 172.21.15.82:0/3773724630 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea8bfff640 1 --2- 172.21.15.82:0/3773724630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985bbe10 0x5633985be260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea8affd640 1 --2- 172.21.15.82:0/3773724630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b3290 0x5633985bb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea8b7fe640 1 -- 172.21.15.82:0/3773724630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985bbe10 msgr2=0x5633985be260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/3773724630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985bbe10 0x5633985be260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea8b7fe640 1 -- 172.21.15.82:0/3773724630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b3290 msgr2=0x5633985bb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/3773724630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b3290 0x5633985bb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea8b7fe640 1 -- 172.21.15.82:0/3773724630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633982f17e0 con 0x5633985b28e0 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/3773724630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985b2cc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fea7c000c00 tx=0x7fea7c0c3710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44873339fe4cc9a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea71ffb640 1 -- 172.21.15.82:0/3773724630 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0bac70 con 0x5633985b28e0 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.526+0000 7fea71ffb640 1 -- 172.21.15.82:0/3773724630 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea7c0be940 con 0x5633985b28e0 2024-04-21T08:47:41.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.527+0000 7fea71ffb640 1 -- 172.21.15.82:0/3773724630 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0bae10 con 0x5633985b28e0 2024-04-21T08:47:41.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.527+0000 7fea94781740 1 -- 172.21.15.82:0/3773724630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 msgr2=0x5633985b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.527+0000 7fea94781740 1 --2- 172.21.15.82:0/3773724630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985b2cc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fea7c000c00 tx=0x7fea7c0c3710 comp rx=0 tx=0).stop 2024-04-21T08:47:41.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.527+0000 7fea94781740 1 -- 172.21.15.82:0/3773724630 shutdown_connections 2024-04-21T08:47:41.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.527+0000 7fea94781740 1 --2- 172.21.15.82:0/3773724630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985bbe10 0x5633985be260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.527+0000 7fea94781740 1 --2- 172.21.15.82:0/3773724630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b3290 0x5633985bb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.527+0000 7fea94781740 1 --2- 172.21.15.82:0/3773724630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985b2cc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.528+0000 7fea94781740 1 -- 172.21.15.82:0/3773724630 >> 172.21.15.82:0/3773724630 conn(0x5633985acfb0 msgr2=0x5633985ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:41.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.528+0000 7fea94781740 1 -- 172.21.15.82:0/3773724630 shutdown_connections 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.528+0000 7fea94781740 1 -- 172.21.15.82:0/3773724630 wait complete. 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.528+0000 7fea94781740 1 -- start start 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.528+0000 7fea94781740 1 Processor -- start 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea94781740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea94781740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b3290 0x5633985b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea94781740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985bbe10 0x5633985b9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea94781740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563398516f00 con 0x5633985b28e0 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea94781740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5633984d0580 con 0x5633985b3290 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea94781740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5633985beb40 con 0x5633985bbe10 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea8b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea8b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985b8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57030/0 (socket says 172.21.15.82:57030) 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea8b7fe640 1 -- 172.21.15.82:0/1416623220 learned_addr learned my addr 172.21.15.82:0/1416623220 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea94781740 10 client.0 _ll_register_callbacks cb 0x5633984d6650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:41.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea8affd640 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b3290 0x5633985b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea8b7fe640 1 -- 172.21.15.82:0/1416623220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985bbe10 msgr2=0x5633985b9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea8bfff640 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985bbe10 0x5633985b9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985bbe10 0x5633985b9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea8b7fe640 1 -- 172.21.15.82:0/1416623220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b3290 msgr2=0x5633985b92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b3290 0x5633985b92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea8b7fe640 1 -- 172.21.15.82:0/1416623220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563398351f70 con 0x5633985b28e0 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea8affd640 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b3290 0x5633985b92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985b8d90 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0bb150 tx=0x7fea7c0be240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7daa5d89811e2f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea6affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea6affd640 1 -- 172.21.15.82:0/1416623220 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0bafb0 con 0x5633985b28e0 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea94781740 1 -- 172.21.15.82:0/1416623220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea7c0be940 con 0x5633985b28e0 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.529+0000 7fea6affd640 1 -- 172.21.15.82:0/1416623220 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fea7c0efea0 con 0x5633985b28e0 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.530+0000 7fea6affd640 1 -- 172.21.15.82:0/1416623220 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea7c0ce050 con 0x5633985b28e0 2024-04-21T08:47:41.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.530+0000 7fea94781740 10 client.16194 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.530+0000 7fea94781740 1 -- 172.21.15.82:0/1416623220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5633981e9170 con 0x5633985b28e0 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.530+0000 7fea6affd640 1 -- 172.21.15.82:0/1416623220 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea7c0be940 con 0x5633985b28e0 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.530+0000 7fea6affd640 1 -- 172.21.15.82:0/1416623220 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fea7c0d2ce0 con 0x5633985b28e0 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.530+0000 7fea94781740 10 client.16194 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.530+0000 7fea94781740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.530+0000 7fea94781740 20 client.16194 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.530+0000 7fea94781740 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5633985cbdc0 0x5633985ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.530+0000 7fea94781740 4 client.16194 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.530+0000 7fea94781740 1 -- 172.21.15.82:0/1416623220 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563398524920 con 0x5633985cbdc0 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.530+0000 7fea8affd640 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5633985cbdc0 0x5633985ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.531+0000 7fea6affd640 10 client.16194 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.531+0000 7fea8affd640 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5633985cbdc0 0x5633985ce1b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f5f034c7028e482 server_cookie=ae36efe278a4fedd in_seq=0 out_seq=0 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea6affd640 1 -- 172.21.15.82:0/1416623220 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563398524920 con 0x5633985cbdc0 2024-04-21T08:47:41.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea6affd640 10 client.16194 handle_command_reply: tid=0 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea94781740 1 client.16194 shutdown 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea94781740 1 -- 172.21.15.82:0/1416623220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 msgr2=0x5633985b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea94781740 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985b8d90 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0bb150 tx=0x7fea7c0be240 comp rx=0 tx=0).stop 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea94781740 1 -- 172.21.15.82:0/1416623220 shutdown_connections 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea94781740 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5633985cbdc0 0x5633985ce1b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea94781740 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985bbe10 0x5633985b9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea94781740 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b3290 0x5633985b92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea94781740 1 --2- 172.21.15.82:0/1416623220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985b8d90 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea94781740 1 -- 172.21.15.82:0/1416623220 >> 172.21.15.82:0/1416623220 conn(0x5633985acfb0 msgr2=0x5633985aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea6affd640 0 client.16194 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea94781740 1 -- 172.21.15.82:0/1416623220 shutdown_connections 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea94781740 1 -- 172.21.15.82:0/1416623220 wait complete. 2024-04-21T08:47:41.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.532+0000 7fea94781740 20 client.16194 trim_cache size 0 max 16384 2024-04-21T08:47:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea94781740 1 Processor -- start 2024-04-21T08:47:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea94781740 1 -- start start 2024-04-21T08:47:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea94781740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985bb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea94781740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985b3290 0x5633985c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea94781740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985bbe10 0x5633985b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea94781740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1ad180 con 0x5633985b28e0 2024-04-21T08:47:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea94781740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1aecc0 con 0x5633985bbe10 2024-04-21T08:47:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea94781740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1ad850 con 0x5633985b3290 2024-04-21T08:47:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea8b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985bb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea8b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985bb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57038/0 (socket says 172.21.15.82:57038) 2024-04-21T08:47:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea8b7fe640 1 -- 172.21.15.82:0/3767107768 learned_addr learned my addr 172.21.15.82:0/3767107768 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea8affd640 1 --2- 172.21.15.82:0/3767107768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985b3290 0x5633985c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.566+0000 7fea8bfff640 1 --2- 172.21.15.82:0/3767107768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985bbe10 0x5633985b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.567+0000 7fea8b7fe640 1 -- 172.21.15.82:0/3767107768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985b3290 msgr2=0x5633985c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.567+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/3767107768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985b3290 0x5633985c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.567+0000 7fea8b7fe640 1 -- 172.21.15.82:0/3767107768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985bbe10 msgr2=0x5633985b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.567+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/3767107768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985bbe10 0x5633985b7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.567+0000 7fea8b7fe640 1 -- 172.21.15.82:0/3767107768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633985b7320 con 0x5633985b28e0 2024-04-21T08:47:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.567+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/3767107768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985bb4c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0dd2f0 tx=0x7fea7c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7de1055c8908b3f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.567+0000 7fea717fa640 1 -- 172.21.15.82:0/3767107768 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0c3040 con 0x5633985b28e0 2024-04-21T08:47:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.567+0000 7fea717fa640 1 -- 172.21.15.82:0/3767107768 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea7c0be940 con 0x5633985b28e0 2024-04-21T08:47:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.567+0000 7fea717fa640 1 -- 172.21.15.82:0/3767107768 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0bb180 con 0x5633985b28e0 2024-04-21T08:47:41.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.568+0000 7fea94781740 1 -- 172.21.15.82:0/3767107768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 msgr2=0x5633985bb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.568+0000 7fea94781740 1 --2- 172.21.15.82:0/3767107768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985bb4c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0dd2f0 tx=0x7fea7c000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.568+0000 7fea94781740 1 -- 172.21.15.82:0/3767107768 shutdown_connections 2024-04-21T08:47:41.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.568+0000 7fea94781740 1 --2- 172.21.15.82:0/3767107768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985bbe10 0x5633985b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.568+0000 7fea94781740 1 --2- 172.21.15.82:0/3767107768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985b3290 0x5633985c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.568+0000 7fea94781740 1 --2- 172.21.15.82:0/3767107768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985b28e0 0x5633985bb4c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.568+0000 7fea94781740 1 -- 172.21.15.82:0/3767107768 >> 172.21.15.82:0/3767107768 conn(0x5633985acfb0 msgr2=0x5633985ab780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.568+0000 7fea94781740 1 -- 172.21.15.82:0/3767107768 shutdown_connections 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.568+0000 7fea94781740 1 -- 172.21.15.82:0/3767107768 wait complete. 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.569+0000 7fea94781740 1 -- start start 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.569+0000 7fea94781740 1 Processor -- start 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea94781740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b28e0 0x5633985b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea94781740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985b3290 0x5633985b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea94781740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985bbe10 0x5633985b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea94781740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1ae5f0 con 0x5633985bbe10 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea94781740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1adf20 con 0x5633985b28e0 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea94781740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1ab160 con 0x5633985b3290 2024-04-21T08:47:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea94781740 10 client.0 _ll_register_callbacks cb 0x7fea80009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea8bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985bbe10 0x5633985b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea8bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985bbe10 0x5633985b6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57046/0 (socket says 172.21.15.82:57046) 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea8bfff640 1 -- 172.21.15.82:0/3732777351 learned_addr learned my addr 172.21.15.82:0/3732777351 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b28e0 0x5633985b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea8affd640 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985b3290 0x5633985b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea8bfff640 1 -- 172.21.15.82:0/3732777351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985b3290 msgr2=0x5633985b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea8bfff640 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985b3290 0x5633985b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea8bfff640 1 -- 172.21.15.82:0/3732777351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b28e0 msgr2=0x5633985b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea8bfff640 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b28e0 0x5633985b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea8bfff640 1 -- 172.21.15.82:0/3732777351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563398631350 con 0x5633985bbe10 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea6affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.570+0000 7fea8bfff640 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985bbe10 0x5633985b6600 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fea84003f60 tx=0x7fea84003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c6f1710f105db1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea6affd640 1 -- 172.21.15.82:0/3732777351 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84004b20 con 0x5633985bbe10 2024-04-21T08:47:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea6affd640 1 -- 172.21.15.82:0/3732777351 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fea84004cc0 con 0x5633985bbe10 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea94781740 1 -- 172.21.15.82:0/3732777351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea7c0be940 con 0x5633985bbe10 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea6affd640 1 -- 172.21.15.82:0/3732777351 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea84026050 con 0x5633985bbe10 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea94781740 10 client.16200 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea94781740 1 -- 172.21.15.82:0/3732777351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5633986326f0 con 0x5633985bbe10 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea6affd640 1 -- 172.21.15.82:0/3732777351 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea84012cb0 con 0x5633985bbe10 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea6affd640 1 -- 172.21.15.82:0/3732777351 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fea84005630 con 0x5633985bbe10 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea94781740 10 client.16200 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea94781740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea94781740 20 client.16200 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea94781740 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563398632ef0 0x563398635310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea94781740 4 client.16200 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.571+0000 7fea94781740 1 -- 172.21.15.82:0/3732777351 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fea7c0be940 con 0x563398632ef0 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.572+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563398632ef0 0x563398635310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.572+0000 7fea8b7fe640 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563398632ef0 0x563398635310 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59d58c1fbe7043ff server_cookie=7f52f2c7c03850a9 in_seq=0 out_seq=0 2024-04-21T08:47:41.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.572+0000 7fea6affd640 10 client.16200 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.573+0000 7fea6affd640 1 -- 172.21.15.82:0/3732777351 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fea7c0be940 con 0x563398632ef0 2024-04-21T08:47:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.573+0000 7fea6affd640 10 client.16200 handle_command_reply: tid=0 2024-04-21T08:47:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.573+0000 7fea94781740 1 client.16200 shutdown 2024-04-21T08:47:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.574+0000 7fea94781740 1 -- 172.21.15.82:0/3732777351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985bbe10 msgr2=0x5633985b6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.574+0000 7fea94781740 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985bbe10 0x5633985b6600 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fea84003f60 tx=0x7fea84003f90 comp rx=0 tx=0).stop 2024-04-21T08:47:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.574+0000 7fea94781740 1 -- 172.21.15.82:0/3732777351 shutdown_connections 2024-04-21T08:47:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.574+0000 7fea94781740 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563398632ef0 0x563398635310 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.574+0000 7fea94781740 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633985bbe10 0x5633985b6600 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.574+0000 7fea6affd640 0 client.16200 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.574+0000 7fea94781740 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633985b3290 0x5633985b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.574+0000 7fea94781740 1 --2- 172.21.15.82:0/3732777351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633985b28e0 0x5633985b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.574+0000 7fea94781740 1 -- 172.21.15.82:0/3732777351 >> 172.21.15.82:0/3732777351 conn(0x5633985acfb0 msgr2=0x5633985b95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.574+0000 7fea94781740 1 -- 172.21.15.82:0/3732777351 shutdown_connections 2024-04-21T08:47:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.574+0000 7fea94781740 1 -- 172.21.15.82:0/3732777351 wait complete. 2024-04-21T08:47:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.574+0000 7fea94781740 20 client.16200 trim_cache size 0 max 16384 2024-04-21T08:47:41.579 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:47:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:47:41.579 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "63da8f5e-20c1-48b2-a9ac-532aed493f06", 2024-04-21T08:47:41.580 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:47:41.580 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:47:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.575+0000 7fea91ac0640 1 -- 172.21.15.82:0/3556817025 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fea740a8450 msgr2=0x7fea740aa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.575+0000 7fea91ac0640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fea740a8450 0x7fea740aa910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fea8c199e90 tx=0x7fea80008040 comp rx=0 tx=0).stop 2024-04-21T08:47:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.575+0000 7fea91ac0640 1 -- 172.21.15.82:0/3556817025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea8c1021a0 msgr2=0x7fea8c19e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.575+0000 7fea91ac0640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea8c1021a0 0x7fea8c19e580 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fea7c005940 tx=0x7fea7c0045e0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.575+0000 7fea91ac0640 1 -- 172.21.15.82:0/3556817025 shutdown_connections 2024-04-21T08:47:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.575+0000 7fea91ac0640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea8c198690 0x7fea8c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.575+0000 7fea91ac0640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea8c107270 0x7fea8c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.575+0000 7fea91ac0640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fea740a8450 0x7fea740aa910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.575+0000 7fea91ac0640 1 --2- 172.21.15.82:0/3556817025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea8c1021a0 0x7fea8c19e580 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.575+0000 7fea91ac0640 1 -- 172.21.15.82:0/3556817025 >> 172.21.15.82:0/3556817025 conn(0x7fea8c079350 msgr2=0x7fea8c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.575+0000 7fea91ac0640 1 -- 172.21.15.82:0/3556817025 shutdown_connections 2024-04-21T08:47:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.575+0000 7fea91ac0640 1 -- 172.21.15.82:0/3556817025 wait complete. 2024-04-21T08:47:41.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:47:41.587 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:47:41.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.696+0000 7f2925f0d640 1 -- 172.21.15.82:0/2098287759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2920106150 msgr2=0x7f292010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.696+0000 7f2925f0d640 1 --2- 172.21.15.82:0/2098287759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2920106150 0x7f292010dd20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2918009ca0 tx=0x7f291802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.696+0000 7f2925f0d640 1 -- 172.21.15.82:0/2098287759 shutdown_connections 2024-04-21T08:47:41.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.696+0000 7f2925f0d640 1 --2- 172.21.15.82:0/2098287759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2920106150 0x7f292010dd20 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.696+0000 7f2925f0d640 1 --2- 172.21.15.82:0/2098287759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2920101a30 0x7f2920105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.696+0000 7f2925f0d640 1 --2- 172.21.15.82:0/2098287759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2920101080 0x7f2920101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.696+0000 7f2925f0d640 1 -- 172.21.15.82:0/2098287759 >> 172.21.15.82:0/2098287759 conn(0x7f2920078000 msgr2=0x7f2920078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:41.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.696+0000 7f2925f0d640 1 -- 172.21.15.82:0/2098287759 shutdown_connections 2024-04-21T08:47:41.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.696+0000 7f2925f0d640 1 -- 172.21.15.82:0/2098287759 wait complete. 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.696+0000 7f2925f0d640 1 Processor -- start 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f2925f0d640 1 -- start start 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f2925f0d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2920101080 0x7f29201986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f2925f0d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2920106150 0x7f2920198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f2925f0d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f292019cfb0 0x7f292019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f2925f0d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29201110c0 con 0x7f2920101080 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f291f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2920101080 0x7f29201986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f291f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2920101080 0x7f29201986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57070/0 (socket says 172.21.15.82:57070) 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f291f7fe640 1 -- 172.21.15.82:0/3524098002 learned_addr learned my addr 172.21.15.82:0/3524098002 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f2925f0d640 1 -- 172.21.15.82:0/3524098002 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2920110f40 con 0x7f292019cfb0 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f2925f0d640 1 -- 172.21.15.82:0/3524098002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2920111240 con 0x7f2920106150 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f291ffff640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f292019cfb0 0x7f292019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f291f7fe640 1 -- 172.21.15.82:0/3524098002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2920106150 msgr2=0x7f2920198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f291f7fe640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2920106150 0x7f2920198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f291f7fe640 1 -- 172.21.15.82:0/3524098002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f292019cfb0 msgr2=0x7f292019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f291f7fe640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f292019cfb0 0x7f292019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f291f7fe640 1 -- 172.21.15.82:0/3524098002 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292019daa0 con 0x7f2920101080 2024-04-21T08:47:41.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.697+0000 7f291ffff640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f292019cfb0 0x7f292019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:41.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.698+0000 7f291f7fe640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2920101080 0x7f29201986f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f291000d950 tx=0x7f291000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:41.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.698+0000 7f291cff9640 1 -- 172.21.15.82:0/3524098002 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910017070 con 0x7f2920101080 2024-04-21T08:47:41.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.698+0000 7f2925f0d640 1 -- 172.21.15.82:0/3524098002 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292019dd90 con 0x7f2920101080 2024-04-21T08:47:41.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.698+0000 7f2925f0d640 1 -- 172.21.15.82:0/3524098002 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29201a55d0 con 0x7f2920101080 2024-04-21T08:47:41.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.698+0000 7f291cff9640 1 -- 172.21.15.82:0/3524098002 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2910009c30 con 0x7f2920101080 2024-04-21T08:47:41.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.698+0000 7f291cff9640 1 -- 172.21.15.82:0/3524098002 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291000f620 con 0x7f2920101080 2024-04-21T08:47:41.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.700+0000 7f291cff9640 1 -- 172.21.15.82:0/3524098002 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f291000f7c0 con 0x7f2920101080 2024-04-21T08:47:41.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.701+0000 7f2928bd0740 1 Processor -- start 2024-04-21T08:47:41.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.701+0000 7f291cff9640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29080a3ec0 0x7f29080a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.701+0000 7f2928bd0740 1 -- start start 2024-04-21T08:47:41.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.701+0000 7f291cff9640 1 -- 172.21.15.82:0/3524098002 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f29100d09d0 con 0x7f2920101080 2024-04-21T08:47:41.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.701+0000 7f2928bd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb54bff0 0x5582eb54c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.701+0000 7f291ffff640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29080a3ec0 0x7f29080a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.701+0000 7f2928bd0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 0x5582eb54ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.701+0000 7f2928bd0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb55e1b0 0x5582eb5605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.701+0000 7f2928bd0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5582eb4b11e0 con 0x5582eb54c9a0 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.701+0000 7f2928bd0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5582eb44e390 con 0x5582eb55e1b0 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.701+0000 7f2928bd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5582eb4ad1a0 con 0x5582eb54bff0 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.701+0000 7f291effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 0x5582eb54ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f291effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 0x5582eb54ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57082/0 (socket says 172.21.15.82:57082) 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f291effd640 1 -- 172.21.15.82:0/310683010 learned_addr learned my addr 172.21.15.82:0/310683010 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f291f7fe640 1 --2- 172.21.15.82:0/310683010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb54bff0 0x5582eb54c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f291ffff640 1 --2- 172.21.15.82:0/310683010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb55e1b0 0x5582eb5605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f291effd640 1 -- 172.21.15.82:0/310683010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb54bff0 msgr2=0x5582eb54c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f291effd640 1 --2- 172.21.15.82:0/310683010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb54bff0 0x5582eb54c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f291effd640 1 -- 172.21.15.82:0/310683010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb55e1b0 msgr2=0x5582eb5605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f291effd640 1 --2- 172.21.15.82:0/310683010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb55e1b0 0x5582eb5605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f291effd640 1 -- 172.21.15.82:0/310683010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582eb28d7d0 con 0x5582eb54c9a0 2024-04-21T08:47:41.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f291ffff640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29080a3ec0 0x7f29080a6380 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2918006010 tx=0x7f291802cca0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:41.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f291effd640 1 --2- 172.21.15.82:0/310683010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 0x5582eb54ce20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f291400ba20 tx=0x7f291400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebff37c3d1754b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:41.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f2905ffb640 1 -- 172.21.15.82:0/310683010 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914016070 con 0x5582eb54c9a0 2024-04-21T08:47:41.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f2905ffb640 1 -- 172.21.15.82:0/310683010 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29140054d0 con 0x5582eb54c9a0 2024-04-21T08:47:41.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.702+0000 7f2905ffb640 1 -- 172.21.15.82:0/310683010 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914004c70 con 0x5582eb54c9a0 2024-04-21T08:47:41.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.703+0000 7f2928bd0740 1 -- 172.21.15.82:0/310683010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 msgr2=0x5582eb54ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.703+0000 7f2928bd0740 1 --2- 172.21.15.82:0/310683010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 0x5582eb54ce20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f291400ba20 tx=0x7f291400bee0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.703+0000 7f2928bd0740 1 -- 172.21.15.82:0/310683010 shutdown_connections 2024-04-21T08:47:41.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.703+0000 7f2928bd0740 1 --2- 172.21.15.82:0/310683010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb55e1b0 0x5582eb5605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.703+0000 7f2928bd0740 1 --2- 172.21.15.82:0/310683010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 0x5582eb54ce20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.703+0000 7f2928bd0740 1 --2- 172.21.15.82:0/310683010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb54bff0 0x5582eb54c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.703+0000 7f2928bd0740 1 -- 172.21.15.82:0/310683010 >> 172.21.15.82:0/310683010 conn(0x5582eb546cb0 msgr2=0x5582eb547090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:41.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.703+0000 7f2928bd0740 1 -- 172.21.15.82:0/310683010 shutdown_connections 2024-04-21T08:47:41.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.703+0000 7f2928bd0740 1 -- 172.21.15.82:0/310683010 wait complete. 2024-04-21T08:47:41.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.703+0000 7f2928bd0740 1 -- start start 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f2928bd0740 1 Processor -- start 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f2928bd0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb54bff0 0x5582eb54e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f2928bd0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 0x5582eb54e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f2928bd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb55e1b0 0x5582eb54eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f2928bd0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5582eb561060 con 0x5582eb54c9a0 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f2928bd0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5582eb46a2f0 con 0x5582eb54bff0 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f2928bd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5582eb5611e0 con 0x5582eb55e1b0 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f2928bd0740 10 client.0 _ll_register_callbacks cb 0x5582eb477580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f291effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 0x5582eb54e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f291f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb54bff0 0x5582eb54e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f291effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 0x5582eb54e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57086/0 (socket says 172.21.15.82:57086) 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f291f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb54bff0 0x5582eb54e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:37676/0 (socket says 172.21.15.82:37676) 2024-04-21T08:47:41.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.704+0000 7f291effd640 1 -- 172.21.15.82:0/249038113 learned_addr learned my addr 172.21.15.82:0/249038113 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f291ffff640 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb55e1b0 0x5582eb54eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f291effd640 1 -- 172.21.15.82:0/249038113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb55e1b0 msgr2=0x5582eb54eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f291effd640 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb55e1b0 0x5582eb54eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f291effd640 1 -- 172.21.15.82:0/249038113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb54bff0 msgr2=0x5582eb54e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f291effd640 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb54bff0 0x5582eb54e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f291effd640 1 -- 172.21.15.82:0/249038113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582eb1c33f0 con 0x5582eb54c9a0 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f291ffff640 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb55e1b0 0x5582eb54eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f291f7fe640 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb54bff0 0x5582eb54e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f291effd640 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 0x5582eb54e9a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f29140048b0 tx=0x7f29140052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb23ca45490444 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f28feffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f28feffd640 1 -- 172.21.15.82:0/249038113 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914016070 con 0x5582eb54c9a0 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f28feffd640 1 -- 172.21.15.82:0/249038113 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f29140049f0 con 0x5582eb54c9a0 2024-04-21T08:47:41.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f2928bd0740 1 -- 172.21.15.82:0/249038113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29140054d0 con 0x5582eb54c9a0 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f28feffd640 1 -- 172.21.15.82:0/249038113 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f291402d050 con 0x5582eb54c9a0 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f2928bd0740 10 client.16212 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f2928bd0740 1 -- 172.21.15.82:0/249038113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582eb1830f0 con 0x5582eb54c9a0 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.705+0000 7f28feffd640 1 -- 172.21.15.82:0/249038113 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291400de50 con 0x5582eb54c9a0 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.706+0000 7f28feffd640 1 -- 172.21.15.82:0/249038113 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2914022020 con 0x5582eb54c9a0 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.706+0000 7f2928bd0740 10 client.16212 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.706+0000 7f2928bd0740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.706+0000 7f2928bd0740 20 client.16212 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.706+0000 7f2928bd0740 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5582eb56e020 0x5582eb570410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.706+0000 7f2928bd0740 4 client.16212 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.706+0000 7f2928bd0740 1 -- 172.21.15.82:0/249038113 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f29140054d0 con 0x5582eb56e020 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.706+0000 7f291f7fe640 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5582eb56e020 0x5582eb570410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.707+0000 7f28feffd640 10 client.16212 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.707+0000 7f291f7fe640 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5582eb56e020 0x5582eb570410 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f5e89e2b1e4cf83 server_cookie=47664642f985e834 in_seq=0 out_seq=0 2024-04-21T08:47:41.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.707+0000 7f28feffd640 1 -- 172.21.15.82:0/249038113 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f29140054d0 con 0x5582eb56e020 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.707+0000 7f28feffd640 10 client.16212 handle_command_reply: tid=0 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.707+0000 7f2928bd0740 1 client.16212 shutdown 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.708+0000 7f2928bd0740 1 -- 172.21.15.82:0/249038113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 msgr2=0x5582eb54e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.708+0000 7f2928bd0740 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 0x5582eb54e9a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f29140048b0 tx=0x7f29140052d0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.708+0000 7f2928bd0740 1 -- 172.21.15.82:0/249038113 shutdown_connections 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.708+0000 7f2928bd0740 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5582eb56e020 0x5582eb570410 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.708+0000 7f2928bd0740 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb55e1b0 0x5582eb54eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.708+0000 7f28feffd640 0 client.16212 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.708+0000 7f2928bd0740 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54c9a0 0x5582eb54e9a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.708+0000 7f2928bd0740 1 --2- 172.21.15.82:0/249038113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb54bff0 0x5582eb54e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.708+0000 7f2928bd0740 1 -- 172.21.15.82:0/249038113 >> 172.21.15.82:0/249038113 conn(0x5582eb546cb0 msgr2=0x5582eb548a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.708+0000 7f2928bd0740 1 -- 172.21.15.82:0/249038113 shutdown_connections 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.708+0000 7f2928bd0740 1 -- 172.21.15.82:0/249038113 wait complete. 2024-04-21T08:47:41.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.708+0000 7f2928bd0740 20 client.16212 trim_cache size 0 max 16384 2024-04-21T08:47:41.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:41 smithi082 ceph-mon[27967]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T08:47:41.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.726+0000 7f2928bd0740 1 Processor -- start 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.726+0000 7f2928bd0740 1 -- start start 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.726+0000 7f2928bd0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb546cb0 0x5582eb550b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.726+0000 7f2928bd0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb54bff0 0x5582eb569fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.726+0000 7f2928bd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb55e1b0 0x5582eb570410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.726+0000 7f2928bd0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f290c00ff70 con 0x5582eb546cb0 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.726+0000 7f2928bd0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f290c011ab0 con 0x5582eb54bff0 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.726+0000 7f2928bd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f290c010640 con 0x5582eb55e1b0 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.726+0000 7f291f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb546cb0 0x5582eb550b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.726+0000 7f291f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb546cb0 0x5582eb550b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57100/0 (socket says 172.21.15.82:57100) 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.726+0000 7f291f7fe640 1 -- 172.21.15.82:0/2019885309 learned_addr learned my addr 172.21.15.82:0/2019885309 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.726+0000 7f291ffff640 1 --2- 172.21.15.82:0/2019885309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb55e1b0 0x5582eb570410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.727+0000 7f291effd640 1 --2- 172.21.15.82:0/2019885309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb54bff0 0x5582eb569fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.727+0000 7f291f7fe640 1 -- 172.21.15.82:0/2019885309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb55e1b0 msgr2=0x5582eb570410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.727+0000 7f291f7fe640 1 --2- 172.21.15.82:0/2019885309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb55e1b0 0x5582eb570410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.727+0000 7f291f7fe640 1 -- 172.21.15.82:0/2019885309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb54bff0 msgr2=0x5582eb569fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.727+0000 7f291f7fe640 1 --2- 172.21.15.82:0/2019885309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb54bff0 0x5582eb569fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.727+0000 7f291f7fe640 1 -- 172.21.15.82:0/2019885309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582eb55d610 con 0x5582eb546cb0 2024-04-21T08:47:41.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.727+0000 7f291f7fe640 1 --2- 172.21.15.82:0/2019885309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb546cb0 0x5582eb550b90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f29100ce360 tx=0x7f2910005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48daa932f3f8f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:41.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.727+0000 7f29057fa640 1 -- 172.21.15.82:0/2019885309 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910020040 con 0x5582eb546cb0 2024-04-21T08:47:41.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.727+0000 7f29057fa640 1 -- 172.21.15.82:0/2019885309 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2910098b50 con 0x5582eb546cb0 2024-04-21T08:47:41.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.727+0000 7f29057fa640 1 -- 172.21.15.82:0/2019885309 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29100a18a0 con 0x5582eb546cb0 2024-04-21T08:47:41.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.728+0000 7f2928bd0740 1 -- 172.21.15.82:0/2019885309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb546cb0 msgr2=0x5582eb550b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.728+0000 7f2928bd0740 1 --2- 172.21.15.82:0/2019885309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb546cb0 0x5582eb550b90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f29100ce360 tx=0x7f2910005880 comp rx=0 tx=0).stop 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.728+0000 7f2928bd0740 1 -- 172.21.15.82:0/2019885309 shutdown_connections 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.728+0000 7f2928bd0740 1 --2- 172.21.15.82:0/2019885309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb55e1b0 0x5582eb570410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.728+0000 7f2928bd0740 1 --2- 172.21.15.82:0/2019885309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb54bff0 0x5582eb569fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.728+0000 7f2928bd0740 1 --2- 172.21.15.82:0/2019885309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb546cb0 0x5582eb550b90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.728+0000 7f2928bd0740 1 -- 172.21.15.82:0/2019885309 >> 172.21.15.82:0/2019885309 conn(0x5582eb55dd00 msgr2=0x5582eb54d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.728+0000 7f2928bd0740 1 -- 172.21.15.82:0/2019885309 shutdown_connections 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.728+0000 7f2928bd0740 1 -- 172.21.15.82:0/2019885309 wait complete. 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.728+0000 7f2928bd0740 1 -- start start 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f2928bd0740 1 Processor -- start 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f2928bd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb546cb0 0x5582eb55c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f2928bd0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54bff0 0x5582eb55c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f2928bd0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb55e1b0 0x5582eb5d34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f2928bd0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f290c0113e0 con 0x5582eb54bff0 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f2928bd0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f290c010d10 con 0x5582eb55e1b0 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f2928bd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f290c00df50 con 0x5582eb546cb0 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f2928bd0740 10 client.0 _ll_register_callbacks cb 0x7f291000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f291effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54bff0 0x5582eb55c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f291effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54bff0 0x5582eb55c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57116/0 (socket says 172.21.15.82:57116) 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f291effd640 1 -- 172.21.15.82:0/2372789432 learned_addr learned my addr 172.21.15.82:0/2372789432 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f291f7fe640 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb546cb0 0x5582eb55c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f291ffff640 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb55e1b0 0x5582eb5d34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f291effd640 1 -- 172.21.15.82:0/2372789432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb546cb0 msgr2=0x5582eb55c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f291effd640 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb546cb0 0x5582eb55c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f291effd640 1 -- 172.21.15.82:0/2372789432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb55e1b0 msgr2=0x5582eb5d34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f291effd640 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb55e1b0 0x5582eb5d34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f291effd640 1 -- 172.21.15.82:0/2372789432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582eb5d39e0 con 0x5582eb54bff0 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f291effd640 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54bff0 0x5582eb55c9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2914003870 tx=0x7f2914005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b897a6aaf63365d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.729+0000 7f28feffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.730+0000 7f28feffd640 1 -- 172.21.15.82:0/2372789432 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914016050 con 0x5582eb54bff0 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.730+0000 7f28feffd640 1 -- 172.21.15.82:0/2372789432 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f291400dc50 con 0x5582eb54bff0 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.730+0000 7f2928bd0740 1 -- 172.21.15.82:0/2372789432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2910098b50 con 0x5582eb54bff0 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.730+0000 7f28feffd640 1 -- 172.21.15.82:0/2372789432 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f291403e050 con 0x5582eb54bff0 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.730+0000 7f2928bd0740 10 client.16218 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.730+0000 7f2928bd0740 1 -- 172.21.15.82:0/2372789432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582eb5d4ca0 con 0x5582eb54bff0 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.730+0000 7f28feffd640 1 -- 172.21.15.82:0/2372789432 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2914005e90 con 0x5582eb54bff0 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.730+0000 7f28feffd640 1 -- 172.21.15.82:0/2372789432 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2914039030 con 0x5582eb54bff0 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.730+0000 7f2928bd0740 10 client.16218 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.730+0000 7f2928bd0740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.730+0000 7f2928bd0740 20 client.16218 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.730+0000 7f2928bd0740 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5582eb56e020 0x5582eb5d7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.731+0000 7f2928bd0740 4 client.16218 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.731+0000 7f2928bd0740 1 -- 172.21.15.82:0/2372789432 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f2910098b50 con 0x5582eb56e020 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.731+0000 7f291f7fe640 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5582eb56e020 0x5582eb5d7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.731+0000 7f291f7fe640 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5582eb56e020 0x5582eb5d7450 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58a657f83f1af8dd server_cookie=b9fa752a6c5d69be in_seq=0 out_seq=0 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.731+0000 7f28feffd640 10 client.16218 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.731+0000 7f28feffd640 1 -- 172.21.15.82:0/2372789432 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f2910098b50 con 0x5582eb56e020 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.731+0000 7f28feffd640 10 client.16218 handle_command_reply: tid=0 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.731+0000 7f2928bd0740 1 client.16218 shutdown 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2928bd0740 1 -- 172.21.15.82:0/2372789432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54bff0 msgr2=0x5582eb55c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2928bd0740 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54bff0 0x5582eb55c9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2914003870 tx=0x7f2914005a70 comp rx=0 tx=0).stop 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2928bd0740 1 -- 172.21.15.82:0/2372789432 shutdown_connections 2024-04-21T08:47:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2928bd0740 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5582eb56e020 0x5582eb5d7450 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2928bd0740 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5582eb55e1b0 0x5582eb5d34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2928bd0740 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582eb54bff0 0x5582eb55c9c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2928bd0740 1 --2- 172.21.15.82:0/2372789432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5582eb546cb0 0x5582eb55c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2928bd0740 1 -- 172.21.15.82:0/2372789432 >> 172.21.15.82:0/2372789432 conn(0x5582eb55dd00 msgr2=0x5582eb545d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:41.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f28feffd640 0 client.16218 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:41.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2928bd0740 1 -- 172.21.15.82:0/2372789432 shutdown_connections 2024-04-21T08:47:41.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2928bd0740 1 -- 172.21.15.82:0/2372789432 wait complete. 2024-04-21T08:47:41.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2928bd0740 20 client.16218 trim_cache size 0 max 16384 2024-04-21T08:47:41.738 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:47:41.738 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:47:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:47:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "63da8f5e-20c1-48b2-a9ac-532aed493f06": { 2024-04-21T08:47:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:47:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "63da8f5e-20c1-48b2-a9ac-532aed493f06", 2024-04-21T08:47:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:47:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:47:41.739 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:47:41.739 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:47:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2925f0d640 1 -- 172.21.15.82:0/3524098002 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29080a3ec0 msgr2=0x7f29080a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2925f0d640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29080a3ec0 0x7f29080a6380 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2918006010 tx=0x7f291802cca0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2925f0d640 1 -- 172.21.15.82:0/3524098002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2920101080 msgr2=0x7f29201986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2925f0d640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2920101080 0x7f29201986f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f291000d950 tx=0x7f291000de10 comp rx=0 tx=0).stop 2024-04-21T08:47:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2925f0d640 1 -- 172.21.15.82:0/3524098002 shutdown_connections 2024-04-21T08:47:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2925f0d640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f292019cfb0 0x7f292019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2925f0d640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2920106150 0x7f2920198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2925f0d640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29080a3ec0 0x7f29080a6380 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2925f0d640 1 --2- 172.21.15.82:0/3524098002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2920101080 0x7f29201986f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.732+0000 7f2925f0d640 1 -- 172.21.15.82:0/3524098002 >> 172.21.15.82:0/3524098002 conn(0x7f2920078000 msgr2=0x7f2920104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.733+0000 7f2925f0d640 1 -- 172.21.15.82:0/3524098002 shutdown_connections 2024-04-21T08:47:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:47:41.733+0000 7f2925f0d640 1 -- 172.21.15.82:0/3524098002 wait complete. 2024-04-21T08:47:41.747 INFO:tasks.cephfs.filesystem:scrub status for tag:63da8f5e-20c1-48b2-a9ac-532aed493f06 - {'path': '/', 'tag': '63da8f5e-20c1-48b2-a9ac-532aed493f06', 'options': 'recursive,force'} 2024-04-21T08:47:41.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:41 smithi134 ceph-mon[31772]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T08:47:42.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:41 smithi106 ceph-mon[31072]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T08:47:42.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:42 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:47:42.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:42 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:42.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:42 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:47:42.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:42 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:42 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:47:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:42 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:42 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:47:43.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:42 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:42 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:47:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:42 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:42 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:47:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:42 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:43.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:43 smithi134 ceph-mon[31772]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:47:43.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:43 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:47:44.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:43 smithi082 ceph-mon[27967]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:47:44.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:43 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:47:44.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:43 smithi106 ceph-mon[31072]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:47:44.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:43 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:47:45.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:45 smithi134 ceph-mon[31772]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:47:46.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:45 smithi082 ceph-mon[27967]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:47:46.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:45 smithi106 ceph-mon[31072]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:47:48.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:47 smithi082 ceph-mon[27967]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-21T08:47:48.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:47 smithi106 ceph-mon[31072]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-21T08:47:48.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:47 smithi134 ceph-mon[31772]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-21T08:47:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.641+0000 7fc1a9100640 1 -- 172.21.15.82:0/2883449454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a4106170 msgr2=0x7fc1a410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.641+0000 7fc1a9100640 1 --2- 172.21.15.82:0/2883449454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a4106170 0x7fc1a410dd40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009ca0 tx=0x7fc19c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.641+0000 7fc1a9100640 1 -- 172.21.15.82:0/2883449454 shutdown_connections 2024-04-21T08:47:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.641+0000 7fc1a9100640 1 --2- 172.21.15.82:0/2883449454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a4106170 0x7fc1a410dd40 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.641+0000 7fc1a9100640 1 --2- 172.21.15.82:0/2883449454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a4101aa0 0x7fc1a4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.641+0000 7fc1a9100640 1 --2- 172.21.15.82:0/2883449454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a41010f0 0x7fc1a41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.641+0000 7fc1a9100640 1 -- 172.21.15.82:0/2883449454 >> 172.21.15.82:0/2883449454 conn(0x7fc1a4077fd0 msgr2=0x7fc1a40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.641+0000 7fc1a9100640 1 -- 172.21.15.82:0/2883449454 shutdown_connections 2024-04-21T08:47:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.641+0000 7fc1a9100640 1 -- 172.21.15.82:0/2883449454 wait complete. 2024-04-21T08:47:49.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a9100640 1 Processor -- start 2024-04-21T08:47:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a9100640 1 -- start start 2024-04-21T08:47:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a9100640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a41010f0 0x7fc1a41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a9100640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a4106170 0x7fc1a4198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a9100640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a419cf70 0x7fc1a419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a9100640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41121f0 con 0x7fc1a41010f0 2024-04-21T08:47:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a9100640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4112070 con 0x7fc1a4106170 2024-04-21T08:47:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a9100640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4112370 con 0x7fc1a419cf70 2024-04-21T08:47:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a41010f0 0x7fc1a41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a41010f0 0x7fc1a41986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55760/0 (socket says 172.21.15.82:55760) 2024-04-21T08:47:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/309856409 learned_addr learned my addr 172.21.15.82:0/309856409 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a359e640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a419cf70 0x7fc1a419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.642+0000 7fc1a259c640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a4106170 0x7fc1a4198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.643+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/309856409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a419cf70 msgr2=0x7fc1a419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.643+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a419cf70 0x7fc1a419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.643+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/309856409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a4106170 msgr2=0x7fc1a4198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.643+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a4106170 0x7fc1a4198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.643+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/309856409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a419da60 con 0x7fc1a41010f0 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.643+0000 7fc1a359e640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a419cf70 0x7fc1a419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.643+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a41010f0 0x7fc1a41986b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc19400d770 tx=0x7fc19400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.643+0000 7fc18bfff640 1 -- 172.21.15.82:0/309856409 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194017070 con 0x7fc1a41010f0 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.643+0000 7fc1a9100640 1 -- 172.21.15.82:0/309856409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a419dd50 con 0x7fc1a41010f0 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.643+0000 7fc18bfff640 1 -- 172.21.15.82:0/309856409 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1940054d0 con 0x7fc1a41010f0 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.643+0000 7fc18bfff640 1 -- 172.21.15.82:0/309856409 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194004c30 con 0x7fc1a41010f0 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.643+0000 7fc1a9100640 1 -- 172.21.15.82:0/309856409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a410e8d0 con 0x7fc1a41010f0 2024-04-21T08:47:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.646+0000 7fc1abdcd740 1 Processor -- start 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1abdcd740 1 -- start start 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc18bfff640 1 -- 172.21.15.82:0/309856409 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc194004e10 con 0x7fc1a41010f0 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1abdcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75c0e0 0x5634bf75ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1abdcd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf75b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1abdcd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf7648f0 0x5634bf766ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1abdcd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5634bf6b78d0 con 0x5634bf75b370 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1abdcd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5634bf654e80 con 0x5634bf7648f0 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1abdcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5634bf6b3890 con 0x5634bf75c0e0 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1a2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf75b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1a359e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75c0e0 0x5634bf75ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1a2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf75b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55770/0 (socket says 172.21.15.82:55770) 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/3474646639 learned_addr learned my addr 172.21.15.82:0/3474646639 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1a259c640 1 --2- 172.21.15.82:0/3474646639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf7648f0 0x5634bf766ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/3474646639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75c0e0 msgr2=0x5634bf75ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/3474646639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75c0e0 0x5634bf75ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.647+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/3474646639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf7648f0 msgr2=0x5634bf766ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.648+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/3474646639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf7648f0 0x5634bf766ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.648+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/3474646639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634bf676320 con 0x5634bf75b370 2024-04-21T08:47:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.648+0000 7fc18bfff640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc18c0a3ee0 0x7fc18c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.648+0000 7fc1a259c640 1 --2- 172.21.15.82:0/3474646639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf7648f0 0x5634bf766ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.648+0000 7fc1a359e640 1 --2- 172.21.15.82:0/3474646639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75c0e0 0x5634bf75ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.648+0000 7fc18bfff640 1 -- 172.21.15.82:0/309856409 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc1940d0890 con 0x7fc1a41010f0 2024-04-21T08:47:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.648+0000 7fc1a359e640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc18c0a3ee0 0x7fc18c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.648+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/3474646639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf75b7f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc1940cdbb0 tx=0x7fc194022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04bd7e5256948cf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.648+0000 7fc1897fa640 1 -- 172.21.15.82:0/3474646639 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19409c0a0 con 0x5634bf75b370 2024-04-21T08:47:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.649+0000 7fc1897fa640 1 -- 172.21.15.82:0/3474646639 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc194005680 con 0x5634bf75b370 2024-04-21T08:47:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.649+0000 7fc1a359e640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc18c0a3ee0 0x7fc18c0a63a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc19c004af0 tx=0x7fc19c005c50 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.649+0000 7fc1897fa640 1 -- 172.21.15.82:0/3474646639 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194016170 con 0x5634bf75b370 2024-04-21T08:47:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.649+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3474646639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 msgr2=0x5634bf75b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.649+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/3474646639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf75b7f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc1940cdbb0 tx=0x7fc194022000 comp rx=0 tx=0).stop 2024-04-21T08:47:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3474646639 shutdown_connections 2024-04-21T08:47:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/3474646639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf7648f0 0x5634bf766ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/3474646639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf75b7f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/3474646639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75c0e0 0x5634bf75ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3474646639 >> 172.21.15.82:0/3474646639 conn(0x5634bf719c90 msgr2=0x5634bf756060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3474646639 shutdown_connections 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3474646639 wait complete. 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 -- start start 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 Processor -- start 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf7642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75c0e0 0x5634bf761640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf7648f0 0x5634bf761c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5634bf7675c0 con 0x5634bf75b370 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5634bf63feb0 con 0x5634bf7648f0 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.650+0000 7fc1abdcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5634bf767740 con 0x5634bf75c0e0 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1a2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf7642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1abdcd740 10 client.0 _ll_register_callbacks cb 0x5634bf66cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1a2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf7642b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55776/0 (socket says 172.21.15.82:55776) 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/328700331 learned_addr learned my addr 172.21.15.82:0/328700331 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1a259c640 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75c0e0 0x5634bf761640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1a359e640 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf7648f0 0x5634bf761c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/328700331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75c0e0 msgr2=0x5634bf761640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75c0e0 0x5634bf761640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/328700331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf7648f0 msgr2=0x5634bf761c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf7648f0 0x5634bf761c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/328700331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634bf3ca210 con 0x5634bf75b370 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1a359e640 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf7648f0 0x5634bf761c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf7642b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc1940ce220 tx=0x7fc1940cda70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1633725abec4a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc186ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc186ff5640 1 -- 172.21.15.82:0/328700331 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194020050 con 0x5634bf75b370 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc186ff5640 1 -- 172.21.15.82:0/328700331 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc194098940 con 0x5634bf75b370 2024-04-21T08:47:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1abdcd740 1 -- 172.21.15.82:0/328700331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc194005680 con 0x5634bf75b370 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc186ff5640 1 -- 172.21.15.82:0/328700331 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc19409bc30 con 0x5634bf75b370 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1abdcd740 10 client.16230 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.651+0000 7fc1abdcd740 1 -- 172.21.15.82:0/328700331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634bf496620 con 0x5634bf75b370 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.652+0000 7fc186ff5640 1 -- 172.21.15.82:0/328700331 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc194005680 con 0x5634bf75b370 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.652+0000 7fc186ff5640 1 -- 172.21.15.82:0/328700331 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc194005100 con 0x5634bf75b370 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.652+0000 7fc1abdcd740 10 client.16230 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.652+0000 7fc1abdcd740 10 client.16230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.652+0000 7fc1abdcd740 20 client.16230 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.652+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5634bf7747a0 0x5634bf776b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.652+0000 7fc1abdcd740 4 client.16230 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.652+0000 7fc1abdcd740 1 -- 172.21.15.82:0/328700331 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634bf6cb880 con 0x5634bf7747a0 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.652+0000 7fc1a259c640 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5634bf7747a0 0x5634bf776b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc186ff5640 10 client.16230 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc1a259c640 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5634bf7747a0 0x5634bf776b90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51db1c94c8f7dca2 server_cookie=2cccc984be51bf0d in_seq=0 out_seq=0 2024-04-21T08:47:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc186ff5640 1 -- 172.21.15.82:0/328700331 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634bf6cb880 con 0x5634bf7747a0 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc186ff5640 10 client.16230 handle_command_reply: tid=0 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc1abdcd740 1 client.16230 shutdown 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc1abdcd740 1 -- 172.21.15.82:0/328700331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 msgr2=0x5634bf7642b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf7642b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc1940ce220 tx=0x7fc1940cda70 comp rx=0 tx=0).stop 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc1abdcd740 1 -- 172.21.15.82:0/328700331 shutdown_connections 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5634bf7747a0 0x5634bf776b90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc186ff5640 0 client.16230 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf7648f0 0x5634bf761c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75c0e0 0x5634bf761640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/328700331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf7642b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.653+0000 7fc1abdcd740 1 -- 172.21.15.82:0/328700331 >> 172.21.15.82:0/328700331 conn(0x5634bf719c90 msgr2=0x5634bf757db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.654+0000 7fc1abdcd740 1 -- 172.21.15.82:0/328700331 shutdown_connections 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.654+0000 7fc1abdcd740 1 -- 172.21.15.82:0/328700331 wait complete. 2024-04-21T08:47:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.654+0000 7fc1abdcd740 20 client.16230 trim_cache size 0 max 16384 2024-04-21T08:47:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1abdcd740 1 Processor -- start 2024-04-21T08:47:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1abdcd740 1 -- start start 2024-04-21T08:47:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1abdcd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf770510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1abdcd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf75c0e0 0x5634bf760660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1abdcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf7648f0 0x5634bf763d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1abdcd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41ad0f0 con 0x5634bf75b370 2024-04-21T08:47:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1abdcd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41aec30 con 0x5634bf75c0e0 2024-04-21T08:47:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1abdcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41ad7c0 con 0x5634bf7648f0 2024-04-21T08:47:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1a2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf770510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1a2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf770510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55778/0 (socket says 172.21.15.82:55778) 2024-04-21T08:47:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/2098600781 learned_addr learned my addr 172.21.15.82:0/2098600781 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1a259c640 1 --2- 172.21.15.82:0/2098600781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf75c0e0 0x5634bf760660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/2098600781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf7648f0 msgr2=0x5634bf763d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/2098600781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf7648f0 0x5634bf763d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/2098600781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf75c0e0 msgr2=0x5634bf760660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/2098600781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf75c0e0 0x5634bf760660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1a2d9d640 1 -- 172.21.15.82:0/2098600781 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634bf7625b0 con 0x5634bf75b370 2024-04-21T08:47:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.669+0000 7fc1a259c640 1 --2- 172.21.15.82:0/2098600781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf75c0e0 0x5634bf760660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.670+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/2098600781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf770510 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc194098ca0 tx=0x7fc1940b0b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47fa0c55c2d0476 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.670+0000 7fc188ff9640 1 -- 172.21.15.82:0/2098600781 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194020070 con 0x5634bf75b370 2024-04-21T08:47:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.670+0000 7fc188ff9640 1 -- 172.21.15.82:0/2098600781 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc194098940 con 0x5634bf75b370 2024-04-21T08:47:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.670+0000 7fc188ff9640 1 -- 172.21.15.82:0/2098600781 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19409be10 con 0x5634bf75b370 2024-04-21T08:47:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.671+0000 7fc1abdcd740 1 -- 172.21.15.82:0/2098600781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 msgr2=0x5634bf770510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.671+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/2098600781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf770510 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc194098ca0 tx=0x7fc1940b0b90 comp rx=0 tx=0).stop 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.671+0000 7fc1abdcd740 1 -- 172.21.15.82:0/2098600781 shutdown_connections 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.671+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/2098600781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf7648f0 0x5634bf763d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.671+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/2098600781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf75c0e0 0x5634bf760660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.671+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/2098600781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf75b370 0x5634bf770510 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.671+0000 7fc1abdcd740 1 -- 172.21.15.82:0/2098600781 >> 172.21.15.82:0/2098600781 conn(0x5634bf719c90 msgr2=0x5634bf75ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.671+0000 7fc1abdcd740 1 -- 172.21.15.82:0/2098600781 shutdown_connections 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.671+0000 7fc1abdcd740 1 -- 172.21.15.82:0/2098600781 wait complete. 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.671+0000 7fc1abdcd740 1 -- start start 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.671+0000 7fc1abdcd740 1 Processor -- start 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1abdcd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75b370 0x5634bf76ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1abdcd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf75c0e0 0x5634bf76cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1abdcd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf7648f0 0x5634bf75eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1abdcd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41ae560 con 0x5634bf7648f0 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1abdcd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41ade90 con 0x5634bf75c0e0 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1abdcd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41ab0d0 con 0x5634bf75b370 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1a359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf7648f0 0x5634bf75eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1a259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf75c0e0 0x5634bf76cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1a359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf7648f0 0x5634bf75eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55784/0 (socket says 172.21.15.82:55784) 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1a359e640 1 -- 172.21.15.82:0/3667427164 learned_addr learned my addr 172.21.15.82:0/3667427164 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1abdcd740 10 client.0 _ll_register_callbacks cb 0x7fc19c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75b370 0x5634bf76ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1a259c640 1 -- 172.21.15.82:0/3667427164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75b370 msgr2=0x5634bf76ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1a259c640 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75b370 0x5634bf76ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1a259c640 1 -- 172.21.15.82:0/3667427164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf7648f0 msgr2=0x5634bf75eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1a259c640 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf7648f0 0x5634bf75eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1a259c640 1 -- 172.21.15.82:0/3667427164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634bf75f2e0 con 0x5634bf75c0e0 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.672+0000 7fc1a359e640 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf7648f0 0x5634bf75eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.673+0000 7fc1a259c640 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf75c0e0 0x5634bf76cf60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x5634bf760ea0 tx=0x7fc19800bd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa1c4fb248fb33c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.673+0000 7fc186ff5640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.673+0000 7fc186ff5640 1 -- 172.21.15.82:0/3667427164 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19801d3b0 con 0x5634bf75c0e0 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.673+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3667427164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc194098940 con 0x5634bf75c0e0 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.673+0000 7fc186ff5640 1 -- 172.21.15.82:0/3667427164 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc19801d550 con 0x5634bf75c0e0 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.673+0000 7fc186ff5640 1 -- 172.21.15.82:0/3667427164 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc19802d050 con 0x5634bf75c0e0 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.673+0000 7fc1abdcd740 10 client.25583 fetch_fsmap learned FSMap version 28 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.673+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3667427164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634bf765b40 con 0x5634bf75c0e0 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.673+0000 7fc186ff5640 1 -- 172.21.15.82:0/3667427164 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc194098940 con 0x5634bf75c0e0 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.673+0000 7fc186ff5640 1 -- 172.21.15.82:0/3667427164 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc198011440 con 0x5634bf75c0e0 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.674+0000 7fc1abdcd740 10 client.25583 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:47:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.674+0000 7fc1abdcd740 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.674+0000 7fc1abdcd740 20 client.25583 populate_metadata read hostname 'smithi082' 2024-04-21T08:47:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.674+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5634bf7747a0 0x5634bf7dceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.674+0000 7fc1abdcd740 4 client.25583 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:47:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.674+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3667427164 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634bf6573d0 con 0x5634bf7747a0 2024-04-21T08:47:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.674+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5634bf7747a0 0x5634bf7dceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.674+0000 7fc1a2d9d640 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5634bf7747a0 0x5634bf7dceb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8bc59f52882be6 server_cookie=9144e4903ee263d3 in_seq=0 out_seq=0 2024-04-21T08:47:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.674+0000 7fc186ff5640 10 client.25583 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc186ff5640 1 -- 172.21.15.82:0/3667427164 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634bf6573d0 con 0x5634bf7747a0 2024-04-21T08:47:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc186ff5640 10 client.25583 handle_command_reply: tid=0 2024-04-21T08:47:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc1abdcd740 1 client.25583 shutdown 2024-04-21T08:47:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3667427164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf75c0e0 msgr2=0x5634bf76cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf75c0e0 0x5634bf76cf60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x5634bf760ea0 tx=0x7fc19800bd00 comp rx=0 tx=0).stop 2024-04-21T08:47:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3667427164 shutdown_connections 2024-04-21T08:47:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5634bf7747a0 0x5634bf7dceb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5634bf7648f0 0x5634bf75eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634bf75c0e0 0x5634bf76cf60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc186ff5640 0 client.25583 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:47:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc1abdcd740 1 --2- 172.21.15.82:0/3667427164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634bf75b370 0x5634bf76ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3667427164 >> 172.21.15.82:0/3667427164 conn(0x5634bf719c90 msgr2=0x5634bf74c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3667427164 shutdown_connections 2024-04-21T08:47:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc1abdcd740 1 -- 172.21.15.82:0/3667427164 wait complete. 2024-04-21T08:47:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.682+0000 7fc1abdcd740 20 client.25583 trim_cache size 0 max 16384 2024-04-21T08:47:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:47:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:47:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:47:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:47:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.683+0000 7fc1a9100640 1 -- 172.21.15.82:0/309856409 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc18c0a3ee0 msgr2=0x7fc18c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.683+0000 7fc1a9100640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc18c0a3ee0 0x7fc18c0a63a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc19c004af0 tx=0x7fc19c005c50 comp rx=0 tx=0).stop 2024-04-21T08:47:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.683+0000 7fc1a9100640 1 -- 172.21.15.82:0/309856409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a41010f0 msgr2=0x7fc1a41986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.683+0000 7fc1a9100640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a41010f0 0x7fc1a41986b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc19400d770 tx=0x7fc19400dc30 comp rx=0 tx=0).stop 2024-04-21T08:47:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.683+0000 7fc1a9100640 1 -- 172.21.15.82:0/309856409 shutdown_connections 2024-04-21T08:47:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.683+0000 7fc1a9100640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a419cf70 0x7fc1a419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.683+0000 7fc1a9100640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1a4106170 0x7fc1a4198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.683+0000 7fc1a9100640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc18c0a3ee0 0x7fc18c0a63a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.683+0000 7fc1a9100640 1 --2- 172.21.15.82:0/309856409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a41010f0 0x7fc1a41986b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.683+0000 7fc1a9100640 1 -- 172.21.15.82:0/309856409 >> 172.21.15.82:0/309856409 conn(0x7fc1a4077fd0 msgr2=0x7fc1a40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.683+0000 7fc1a9100640 1 -- 172.21.15.82:0/309856409 shutdown_connections 2024-04-21T08:47:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:47:49.683+0000 7fc1a9100640 1 -- 172.21.15.82:0/309856409 wait complete. 2024-04-21T08:47:50.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:49 smithi082 ceph-mon[27967]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:47:50.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:49 smithi106 ceph-mon[31072]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:47:50.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:49 smithi134 ceph-mon[31772]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:47:52.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:51 smithi082 ceph-mon[27967]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-04-21T08:47:52.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:51 smithi106 ceph-mon[31072]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-04-21T08:47:52.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:51 smithi134 ceph-mon[31772]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-04-21T08:47:54.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:53 smithi082 ceph-mon[27967]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-04-21T08:47:54.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:53 smithi106 ceph-mon[31072]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-04-21T08:47:54.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:53 smithi134 ceph-mon[31772]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-04-21T08:47:54.953 INFO:tasks.workunit.client.0.smithi082.stderr:Note: switching to '4f6bc5273643aaf5660909e85400647a8951c10b'. 2024-04-21T08:47:54.954 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-21T08:47:54.954 INFO:tasks.workunit.client.0.smithi082.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-21T08:47:54.954 INFO:tasks.workunit.client.0.smithi082.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-21T08:47:54.954 INFO:tasks.workunit.client.0.smithi082.stderr:state without impacting any branches by switching back to a branch. 2024-04-21T08:47:54.954 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-21T08:47:54.954 INFO:tasks.workunit.client.0.smithi082.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-21T08:47:54.954 INFO:tasks.workunit.client.0.smithi082.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-21T08:47:54.954 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-21T08:47:54.954 INFO:tasks.workunit.client.0.smithi082.stderr: git switch -c 2024-04-21T08:47:54.954 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-21T08:47:54.954 INFO:tasks.workunit.client.0.smithi082.stderr:Or undo this operation with: 2024-04-21T08:47:54.955 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-21T08:47:54.955 INFO:tasks.workunit.client.0.smithi082.stderr: git switch - 2024-04-21T08:47:54.955 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-21T08:47:54.955 INFO:tasks.workunit.client.0.smithi082.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-21T08:47:54.955 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-21T08:47:54.955 INFO:tasks.workunit.client.0.smithi082.stderr:HEAD is now at 4f6bc527364 use ubuntu for fs/workload tests 2024-04-21T08:47:54.957 DEBUG:teuthology.orchestra.run.smithi082:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-21T08:47:54.993 INFO:tasks.workunit.client.0.smithi082.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-21T08:47:54.995 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-21T08:47:54.996 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-21T08:47:55.608 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-21T08:47:55.661 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-21T08:47:55.701 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-21T08:47:55.703 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-21T08:47:55.703 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-21T08:47:55.742 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-21T08:47:55.745 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:47:55.745 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-21T08:47:55.801 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-04-21T08:47:55.802 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-04-21T08:47:55.802 DEBUG:teuthology.orchestra.run.smithi082:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4f6bc5273643aaf5660909e85400647a8951c10b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-04-21T08:47:55.862 INFO:tasks.workunit.client.0.smithi082.stdout:getting blogbench 2024-04-21T08:47:55.862 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'getting blogbench' 2024-04-21T08:47:55.862 INFO:tasks.workunit.client.0.smithi082.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-04-21T08:47:55.879 INFO:tasks.workunit.client.0.smithi082.stderr:--2024-04-21 08:47:55-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-04-21T08:47:55.880 INFO:tasks.workunit.client.0.smithi082.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-04-21T08:47:55.902 INFO:tasks.workunit.client.0.smithi082.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-04-21T08:47:55.922 INFO:tasks.workunit.client.0.smithi082.stderr:HTTP request sent, awaiting response... 200 OK 2024-04-21T08:47:55.922 INFO:tasks.workunit.client.0.smithi082.stderr:Length: 103355 (101K) [application/octet-stream] 2024-04-21T08:47:55.923 INFO:tasks.workunit.client.0.smithi082.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-04-21T08:47:55.925 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-21T08:47:55.963 INFO:tasks.workunit.client.0.smithi082.stderr: 0K .......... .......... .......... .......... .......... 49% 1.24M 0s 2024-04-21T08:47:55.983 INFO:tasks.workunit.client.0.smithi082.stderr: 50K .......... .......... .......... .......... .......... 99% 2.43M 0s 2024-04-21T08:47:55.983 INFO:tasks.workunit.client.0.smithi082.stderr: 100K 100% 1.74T=0.06s 2024-04-21T08:47:55.983 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-21T08:47:55.983 INFO:tasks.workunit.client.0.smithi082.stderr:2024-04-21 08:47:55 (1.66 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-04-21T08:47:55.983 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-21T08:47:55.983 INFO:tasks.workunit.client.0.smithi082.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-04-21T08:47:56.015 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/ 2024-04-21T08:47:56.016 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/m4/ 2024-04-21T08:47:56.017 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-04-21T08:47:56.018 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/man/ 2024-04-21T08:47:56.019 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/man/blogbench.8 2024-04-21T08:47:56.020 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/man/Makefile.am 2024-04-21T08:47:56.021 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/man/Makefile.in 2024-04-21T08:47:56.022 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/ 2024-04-21T08:47:56.023 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/mysnprintf.c 2024-04-21T08:47:56.024 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/mysnprintf.h 2024-04-21T08:47:56.025 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/blogbench.c 2024-04-21T08:47:56.026 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/blogbench.h 2024-04-21T08:47:56.027 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/fakesnprintf.c 2024-04-21T08:47:56.028 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/fakesnprintf.h 2024-04-21T08:47:56.029 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/blogbench_p.h 2024-04-21T08:47:56.030 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-04-21T08:47:56.031 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-04-21T08:47:56.032 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/Makefile.am 2024-04-21T08:47:56.033 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/Makefile.in 2024-04-21T08:47:56.034 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/process.c 2024-04-21T08:47:56.035 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/globals.h 2024-04-21T08:47:56.036 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/commenter.c 2024-04-21T08:47:56.037 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/rewriter.c 2024-04-21T08:47:56.038 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/reader.c 2024-04-21T08:47:56.039 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/writer.c 2024-04-21T08:47:56.040 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/helpers.c 2024-04-21T08:47:56.042 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/NEWS 2024-04-21T08:47:56.043 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/compile 2024-04-21T08:47:56.044 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/depcomp 2024-04-21T08:47:56.045 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/aclocal.m4 2024-04-21T08:47:56.046 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/README 2024-04-21T08:47:56.047 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/configure 2024-04-21T08:47:56.050 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/configure.ac 2024-04-21T08:47:56.051 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/config.guess 2024-04-21T08:47:56.052 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/install-sh 2024-04-21T08:47:56.053 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/config.sub 2024-04-21T08:47:56.054 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/missing 2024-04-21T08:47:56.055 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/mkinstalldirs 2024-04-21T08:47:56.056 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/Makefile.am 2024-04-21T08:47:56.058 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/Makefile.in 2024-04-21T08:47:56.059 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/config.h.in 2024-04-21T08:47:56.060 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/AUTHORS 2024-04-21T08:47:56.061 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/INSTALL 2024-04-21T08:47:56.062 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/ChangeLog 2024-04-21T08:47:56.063 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/COPYING 2024-04-21T08:47:56.064 INFO:tasks.workunit.client.0.smithi082.stderr:+ cd blogbench-1.0/ 2024-04-21T08:47:56.064 INFO:tasks.workunit.client.0.smithi082.stdout:making blogbench 2024-04-21T08:47:56.065 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'making blogbench' 2024-04-21T08:47:56.065 INFO:tasks.workunit.client.0.smithi082.stderr:+ ./configure 2024-04-21T08:47:56.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:55 smithi082 ceph-mon[27967]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-21T08:47:56.127 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:55 smithi106 ceph-mon[31072]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-21T08:47:56.176 INFO:tasks.workunit.client.0.smithi082.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-21T08:47:56.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:55 smithi134 ceph-mon[31772]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-21T08:47:57.184 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether build environment is sane... yes 2024-04-21T08:47:57.194 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gawk... gawk 2024-04-21T08:47:57.214 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether make sets $(MAKE)... yes 2024-04-21T08:47:57.223 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gcc... gcc 2024-04-21T08:47:57.269 INFO:tasks.workunit.client.0.smithi082.stdout:checking for C compiler default output file name... a.out 2024-04-21T08:47:57.270 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether the C compiler works... yes 2024-04-21T08:47:57.273 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether we are cross compiling... no 2024-04-21T08:47:57.307 INFO:tasks.workunit.client.0.smithi082.stdout:checking for suffix of executables... 2024-04-21T08:47:57.341 INFO:tasks.workunit.client.0.smithi082.stdout:checking for suffix of object files... o 2024-04-21T08:47:57.374 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether we are using the GNU C compiler... yes 2024-04-21T08:47:57.406 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether gcc accepts -g... yes 2024-04-21T08:47:57.477 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gcc option to accept ANSI C... none needed 2024-04-21T08:47:57.529 INFO:tasks.workunit.client.0.smithi082.stdout:checking for style of include used by make... GNU 2024-04-21T08:47:57.622 INFO:tasks.workunit.client.0.smithi082.stdout:checking dependency style of gcc... gcc3 2024-04-21T08:47:57.622 INFO:tasks.workunit.client.0.smithi082.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-21T08:47:57.677 INFO:tasks.workunit.client.0.smithi082.stdout:checking how to run the C preprocessor... gcc -E 2024-04-21T08:47:57.718 INFO:tasks.workunit.client.0.smithi082.stdout:checking for egrep... grep -E 2024-04-21T08:47:57.726 INFO:tasks.workunit.client.0.smithi082.stdout:checking for AIX... no 2024-04-21T08:47:57.798 INFO:tasks.workunit.client.0.smithi082.stdout:checking for library containing strerror... none required 2024-04-21T08:47:57.943 INFO:tasks.workunit.client.0.smithi082.stdout:checking for ANSI C header files... yes 2024-04-21T08:47:57.979 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-21T08:47:58.024 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/types.h... yes 2024-04-21T08:47:58.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:57 smithi082 ceph-mon[27967]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-21T08:47:58.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:57 smithi106 ceph-mon[31072]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-21T08:47:58.175 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/stat.h... yes 2024-04-21T08:47:58.222 INFO:tasks.workunit.client.0.smithi082.stdout:checking for stdlib.h... yes 2024-04-21T08:47:58.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:57 smithi134 ceph-mon[31772]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-21T08:47:58.269 INFO:tasks.workunit.client.0.smithi082.stdout:checking for string.h... yes 2024-04-21T08:47:58.319 INFO:tasks.workunit.client.0.smithi082.stdout:checking for memory.h... yes 2024-04-21T08:47:58.367 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strings.h... yes 2024-04-21T08:47:58.414 INFO:tasks.workunit.client.0.smithi082.stdout:checking for inttypes.h... yes 2024-04-21T08:47:58.463 INFO:tasks.workunit.client.0.smithi082.stdout:checking for stdint.h... yes 2024-04-21T08:47:58.513 INFO:tasks.workunit.client.0.smithi082.stdout:checking for unistd.h... yes 2024-04-21T08:47:58.527 INFO:tasks.workunit.client.0.smithi082.stdout:checking for memory.h... (cached) yes 2024-04-21T08:47:58.531 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strings.h... (cached) yes 2024-04-21T08:47:58.536 INFO:tasks.workunit.client.0.smithi082.stdout:checking for unistd.h... (cached) yes 2024-04-21T08:47:58.583 INFO:tasks.workunit.client.0.smithi082.stdout:checking locale.h usability... yes 2024-04-21T08:47:58.605 INFO:tasks.workunit.client.0.smithi082.stdout:checking locale.h presence... yes 2024-04-21T08:47:58.606 INFO:tasks.workunit.client.0.smithi082.stdout:checking for locale.h... yes 2024-04-21T08:47:58.622 INFO:tasks.workunit.client.0.smithi082.stdout:checking for stdlib.h... (cached) yes 2024-04-21T08:47:58.626 INFO:tasks.workunit.client.0.smithi082.stdout:checking for string.h... (cached) yes 2024-04-21T08:47:58.675 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/socket.h usability... yes 2024-04-21T08:47:58.697 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/socket.h presence... yes 2024-04-21T08:47:58.697 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/socket.h... yes 2024-04-21T08:48:00.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:59 smithi082 ceph-mon[27967]: pgmap v32: 129 pgs: 129 active+clean; 1.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-21T08:48:00.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:47:59 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:48:00.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:59 smithi106 ceph-mon[31072]: pgmap v32: 129 pgs: 129 active+clean; 1.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-21T08:48:00.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:47:59 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:48:00.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:59 smithi134 ceph-mon[31772]: pgmap v32: 129 pgs: 129 active+clean; 1.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-21T08:48:00.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:47:59 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:48:00.476 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/time.h usability... yes 2024-04-21T08:48:00.503 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/time.h presence... yes 2024-04-21T08:48:00.503 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/time.h... yes 2024-04-21T08:48:00.559 INFO:tasks.workunit.client.0.smithi082.stdout:checking fcntl.h usability... yes 2024-04-21T08:48:00.579 INFO:tasks.workunit.client.0.smithi082.stdout:checking fcntl.h presence... yes 2024-04-21T08:48:00.579 INFO:tasks.workunit.client.0.smithi082.stdout:checking for fcntl.h... yes 2024-04-21T08:48:00.648 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/fcntl.h usability... yes 2024-04-21T08:48:00.668 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/fcntl.h presence... yes 2024-04-21T08:48:00.669 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/fcntl.h... yes 2024-04-21T08:48:00.755 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/param.h usability... yes 2024-04-21T08:48:00.781 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/param.h presence... yes 2024-04-21T08:48:00.782 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/param.h... yes 2024-04-21T08:48:00.834 INFO:tasks.workunit.client.0.smithi082.stdout:checking ioctl.h usability... no 2024-04-21T08:48:00.855 INFO:tasks.workunit.client.0.smithi082.stdout:checking ioctl.h presence... no 2024-04-21T08:48:00.855 INFO:tasks.workunit.client.0.smithi082.stdout:checking for ioctl.h... no 2024-04-21T08:48:00.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:48:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:48:00.431+0000 7fe0b3584700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:48:00.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:48:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:48:00.431+0000 7fe0b3584700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:48:00.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:48:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:48:00.431+0000 7fe0b3584700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:48:00.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:48:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:48:00.431+0000 7fe0b3584700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:48:00.859 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:48:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:48:00.431+0000 7fe0b3584700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:48:00.906 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/ioctl.h usability... yes 2024-04-21T08:48:00.926 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/ioctl.h presence... yes 2024-04-21T08:48:00.926 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/ioctl.h... yes 2024-04-21T08:48:00.981 INFO:tasks.workunit.client.0.smithi082.stdout:checking return type of signal handlers... void 2024-04-21T08:48:01.067 INFO:tasks.workunit.client.0.smithi082.stdout:checking for size_t... yes 2024-04-21T08:48:01.104 INFO:tasks.workunit.client.0.smithi082.stdout:checking for dev_t... yes 2024-04-21T08:48:01.148 INFO:tasks.workunit.client.0.smithi082.stdout:checking for short... yes 2024-04-21T08:48:01.242 INFO:tasks.workunit.client.0.smithi082.stdout:checking size of short... 2 2024-04-21T08:48:01.291 INFO:tasks.workunit.client.0.smithi082.stdout:checking for int... yes 2024-04-21T08:48:01.360 INFO:tasks.workunit.client.0.smithi082.stdout:checking size of int... 4 2024-04-21T08:48:01.405 INFO:tasks.workunit.client.0.smithi082.stdout:checking for long... yes 2024-04-21T08:48:01.473 INFO:tasks.workunit.client.0.smithi082.stdout:checking size of long... 8 2024-04-21T08:48:01.517 INFO:tasks.workunit.client.0.smithi082.stdout:checking for long long... yes 2024-04-21T08:48:01.584 INFO:tasks.workunit.client.0.smithi082.stdout:checking size of long long... 8 2024-04-21T08:48:01.626 INFO:tasks.workunit.client.0.smithi082.stdout:checking for ssize_t... yes 2024-04-21T08:48:01.666 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sig_atomic_t... yes 2024-04-21T08:48:01.714 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether gcc needs -traditional... no 2024-04-21T08:48:01.747 INFO:tasks.workunit.client.0.smithi082.stdout:checking for an ANSI C-conforming const... yes 2024-04-21T08:48:01.772 INFO:tasks.workunit.client.0.smithi082.stdout:checking for inline... inline 2024-04-21T08:48:01.832 INFO:tasks.workunit.client.0.smithi082.stdout:checking for working alloca.h... yes 2024-04-21T08:48:01.879 INFO:tasks.workunit.client.0.smithi082.stdout:checking for alloca... yes 2024-04-21T08:48:01.944 INFO:tasks.workunit.client.0.smithi082.stdout:checking for working memcmp... yes 2024-04-21T08:48:01.998 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strftime... yes 2024-04-21T08:48:02.068 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-04-21T08:48:02.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:01 smithi082 ceph-mon[27967]: pgmap v33: 129 pgs: 129 active+clean; 1.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-21T08:48:02.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:01 smithi082 ceph-mon[27967]: mgrmap e25: x(active, since 60s), standbys: z, y 2024-04-21T08:48:02.129 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether stat accepts an empty string... no 2024-04-21T08:48:02.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:01 smithi106 ceph-mon[31072]: pgmap v33: 129 pgs: 129 active+clean; 1.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-21T08:48:02.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:01 smithi106 ceph-mon[31072]: mgrmap e25: x(active, since 60s), standbys: z, y 2024-04-21T08:48:02.182 INFO:tasks.workunit.client.0.smithi082.stdout:checking for vprintf... yes 2024-04-21T08:48:02.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:01 smithi134 ceph-mon[31772]: pgmap v33: 129 pgs: 129 active+clean; 1.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-21T08:48:02.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:01 smithi134 ceph-mon[31772]: mgrmap e25: x(active, since 60s), standbys: z, y 2024-04-21T08:48:02.238 INFO:tasks.workunit.client.0.smithi082.stdout:checking for _doprnt... no 2024-04-21T08:48:02.260 INFO:tasks.workunit.client.0.smithi082.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-21T08:48:02.271 INFO:tasks.workunit.client.0.smithi082.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-21T08:48:02.320 INFO:tasks.workunit.client.0.smithi082.stdout:checking for the pthreads library -lpthreads... no 2024-04-21T08:48:02.372 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether pthreads work without any flags... yes 2024-04-21T08:48:02.418 INFO:tasks.workunit.client.0.smithi082.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-04-21T08:48:02.418 INFO:tasks.workunit.client.0.smithi082.stdout:checking if more special flags are required for pthreads... no 2024-04-21T08:48:02.419 INFO:tasks.workunit.client.0.smithi082.stdout:checking for cc_r... gcc 2024-04-21T08:48:02.500 INFO:tasks.workunit.client.0.smithi082.stdout:checking for setrlimit... yes 2024-04-21T08:48:02.560 INFO:tasks.workunit.client.0.smithi082.stdout:checking for setproctitle... no 2024-04-21T08:48:02.620 INFO:tasks.workunit.client.0.smithi082.stdout:checking for getopt_long... yes 2024-04-21T08:48:02.683 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strtoull... yes 2024-04-21T08:48:02.746 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strtoq... yes 2024-04-21T08:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.776+0000 7f3be55e2640 1 -- 172.21.15.82:0/3911678672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3be0102b50 msgr2=0x7f3be0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.776+0000 7f3be55e2640 1 --2- 172.21.15.82:0/3911678672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3be0102b50 0x7f3be0106b90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009900 tx=0x7f3bd402bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.776+0000 7f3be55e2640 1 -- 172.21.15.82:0/3911678672 shutdown_connections 2024-04-21T08:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.776+0000 7f3be55e2640 1 --2- 172.21.15.82:0/3911678672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3be0107270 0x7f3be010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.776+0000 7f3be55e2640 1 --2- 172.21.15.82:0/3911678672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3be0102b50 0x7f3be0106b90 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.776+0000 7f3be55e2640 1 --2- 172.21.15.82:0/3911678672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3be01021a0 0x7f3be0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.776+0000 7f3be55e2640 1 -- 172.21.15.82:0/3911678672 >> 172.21.15.82:0/3911678672 conn(0x7f3be00fda00 msgr2=0x7f3be00ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.776+0000 7f3be55e2640 1 -- 172.21.15.82:0/3911678672 shutdown_connections 2024-04-21T08:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.776+0000 7f3be55e2640 1 -- 172.21.15.82:0/3911678672 wait complete. 2024-04-21T08:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3be55e2640 1 Processor -- start 2024-04-21T08:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3be55e2640 1 -- start start 2024-04-21T08:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3be55e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3be01021a0 0x7f3be0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3be55e2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3be0107270 0x7f3be0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3be55e2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3be019cfa0 0x7f3be019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3be55e2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0112210 con 0x7f3be0107270 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3be55e2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0112090 con 0x7f3be019cfa0 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3be55e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0112390 con 0x7f3be01021a0 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3bde7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3be0107270 0x7f3be0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3bde7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3be0107270 0x7f3be0198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40948/0 (socket says 172.21.15.82:40948) 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3bde7fc640 1 -- 172.21.15.82:0/1576326559 learned_addr learned my addr 172.21.15.82:0/1576326559 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3be01021a0 0x7f3be0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3be019cfa0 0x7f3be019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3bde7fc640 1 -- 172.21.15.82:0/1576326559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3be01021a0 msgr2=0x7f3be0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3bde7fc640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3be01021a0 0x7f3be0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3bde7fc640 1 -- 172.21.15.82:0/1576326559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3be019cfa0 msgr2=0x7f3be019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3bde7fc640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3be019cfa0 0x7f3be019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3bde7fc640 1 -- 172.21.15.82:0/1576326559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be019da90 con 0x7f3be0107270 2024-04-21T08:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.777+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3be019cfa0 0x7f3be019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.778+0000 7f3bde7fc640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3be0107270 0x7f3be0198c70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4005ec0 tx=0x7f3bd402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.778+0000 7f3bc7fff640 1 -- 172.21.15.82:0/1576326559 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4043020 con 0x7f3be0107270 2024-04-21T08:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.778+0000 7f3be55e2640 1 -- 172.21.15.82:0/1576326559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be019dd20 con 0x7f3be0107270 2024-04-21T08:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.778+0000 7f3bc7fff640 1 -- 172.21.15.82:0/1576326559 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd40052b0 con 0x7f3be0107270 2024-04-21T08:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.778+0000 7f3bc7fff640 1 -- 172.21.15.82:0/1576326559 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4003d40 con 0x7f3be0107270 2024-04-21T08:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.778+0000 7f3be55e2640 1 -- 172.21.15.82:0/1576326559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be01a5570 con 0x7f3be0107270 2024-04-21T08:48:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.780+0000 7f3bc7fff640 1 -- 172.21.15.82:0/1576326559 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f3bd4046020 con 0x7f3be0107270 2024-04-21T08:48:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.780+0000 7f3be81cb740 1 Processor -- start 2024-04-21T08:48:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.780+0000 7f3be81cb740 1 -- start start 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3be81cb740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c088d0 0x564004c07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3be81cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c07b60 0x564004c07fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bc7fff640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3bc80a3ff0 0x7f3bc80a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3be81cb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c110e0 0x564004c134d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3be81cb740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564004b6c8d0 con 0x564004c088d0 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3be81cb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564004b09e80 con 0x564004c110e0 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3be81cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564004b68890 con 0x564004c07b60 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bdf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c088d0 0x564004c07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bde7fc640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3bc80a3ff0 0x7f3bc80a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bdf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c088d0 0x564004c07620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40952/0 (socket says 172.21.15.82:40952) 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bdf7fe640 1 -- 172.21.15.82:0/2811961483 learned_addr learned my addr 172.21.15.82:0/2811961483 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/2811961483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c07b60 0x564004c07fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/2811961483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c110e0 0x564004c134d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bc7fff640 1 -- 172.21.15.82:0/1576326559 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f3bd40f3110 con 0x7f3be0107270 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bdf7fe640 1 -- 172.21.15.82:0/2811961483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c07b60 msgr2=0x564004c07fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/2811961483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c07b60 0x564004c07fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bdf7fe640 1 -- 172.21.15.82:0/2811961483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c110e0 msgr2=0x564004c134d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/2811961483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c110e0 0x564004c134d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bdf7fe640 1 -- 172.21.15.82:0/2811961483 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564004b2b320 con 0x564004c088d0 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/2811961483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c088d0 0x564004c07620 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8012b10 tx=0x7f3bd8008650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a95579b1b0148f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bde7fc640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3bc80a3ff0 0x7f3bc80a64b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40f03d0 tx=0x7f3bd40891c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.781+0000 7f3bc57fa640 1 -- 172.21.15.82:0/2811961483 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8004d20 con 0x564004c088d0 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.782+0000 7f3bc57fa640 1 -- 172.21.15.82:0/2811961483 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd8005020 con 0x564004c088d0 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.782+0000 7f3bc57fa640 1 -- 172.21.15.82:0/2811961483 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800e6d0 con 0x564004c088d0 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.782+0000 7f3be81cb740 1 -- 172.21.15.82:0/2811961483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c088d0 msgr2=0x564004c07620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.782+0000 7f3be81cb740 1 --2- 172.21.15.82:0/2811961483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c088d0 0x564004c07620 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8012b10 tx=0x7f3bd8008650 comp rx=0 tx=0).stop 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.782+0000 7f3be81cb740 1 -- 172.21.15.82:0/2811961483 shutdown_connections 2024-04-21T08:48:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.782+0000 7f3be81cb740 1 --2- 172.21.15.82:0/2811961483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c110e0 0x564004c134d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.782+0000 7f3be81cb740 1 --2- 172.21.15.82:0/2811961483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c07b60 0x564004c07fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.782+0000 7f3be81cb740 1 --2- 172.21.15.82:0/2811961483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c088d0 0x564004c07620 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.782+0000 7f3be81cb740 1 -- 172.21.15.82:0/2811961483 >> 172.21.15.82:0/2811961483 conn(0x564004bcec90 msgr2=0x564004c02850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.782+0000 7f3be81cb740 1 -- 172.21.15.82:0/2811961483 shutdown_connections 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.782+0000 7f3be81cb740 1 -- 172.21.15.82:0/2811961483 wait complete. 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3be81cb740 1 -- start start 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3be81cb740 1 Processor -- start 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3be81cb740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 0x564004c109c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3be81cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c088d0 0x564004c0ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3be81cb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c110e0 0x564004c0e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3be81cb740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564004c13db0 con 0x564004c07b60 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3be81cb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564004af4eb0 con 0x564004c110e0 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3be81cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564004c13f30 con 0x564004c088d0 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3be81cb740 10 client.0 _ll_register_callbacks cb 0x564004b21b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3bdf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 0x564004c109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3bdf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 0x564004c109c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40960/0 (socket says 172.21.15.82:40960) 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3bdf7fe640 1 -- 172.21.15.82:0/108619523 learned_addr learned my addr 172.21.15.82:0/108619523 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c110e0 0x564004c0e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c088d0 0x564004c0ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3bdf7fe640 1 -- 172.21.15.82:0/108619523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c088d0 msgr2=0x564004c0ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c088d0 0x564004c0ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3bdf7fe640 1 -- 172.21.15.82:0/108619523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c110e0 msgr2=0x564004c0e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c110e0 0x564004c0e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3bdf7fe640 1 -- 172.21.15.82:0/108619523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56400487f210 con 0x564004c07b60 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.783+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c088d0 0x564004c0ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 0x564004c109c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80088a0 tx=0x7f3bd8008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a364dd47dbfd98 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3bc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3bc2ff5640 1 -- 172.21.15.82:0/108619523 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800ba20 con 0x564004c07b60 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3bc2ff5640 1 -- 172.21.15.82:0/108619523 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f3bd800bbc0 con 0x564004c07b60 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3be81cb740 1 -- 172.21.15.82:0/108619523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd8005020 con 0x564004c07b60 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3bc2ff5640 1 -- 172.21.15.82:0/108619523 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3bd8032050 con 0x564004c07b60 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3be81cb740 10 client.16254 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3be81cb740 1 -- 172.21.15.82:0/108619523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56400494b620 con 0x564004c07b60 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3bc2ff5640 1 -- 172.21.15.82:0/108619523 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd8005020 con 0x564004c07b60 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3bc2ff5640 1 -- 172.21.15.82:0/108619523 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f3bd800e710 con 0x564004c07b60 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3be81cb740 10 client.16254 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3be81cb740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3be81cb740 20 client.16254 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3be81cb740 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564004c20f50 0x564004c23340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3be81cb740 4 client.16254 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.784+0000 7f3be81cb740 1 -- 172.21.15.82:0/108619523 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564004b80880 con 0x564004c20f50 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.785+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564004c20f50 0x564004c23340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.785+0000 7f3bc2ff5640 10 client.16254 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.785+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564004c20f50 0x564004c23340 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918639f1c0773f1 server_cookie=c9bf3068270c957e in_seq=0 out_seq=0 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3bc2ff5640 1 -- 172.21.15.82:0/108619523 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564004b80880 con 0x564004c20f50 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3bc2ff5640 10 client.16254 handle_command_reply: tid=0 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3be81cb740 1 client.16254 shutdown 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3be81cb740 1 -- 172.21.15.82:0/108619523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 msgr2=0x564004c109c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3be81cb740 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 0x564004c109c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80088a0 tx=0x7f3bd8008ec0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3be81cb740 1 -- 172.21.15.82:0/108619523 shutdown_connections 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3be81cb740 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564004c20f50 0x564004c23340 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3be81cb740 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c110e0 0x564004c0e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3be81cb740 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c088d0 0x564004c0ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3be81cb740 1 --2- 172.21.15.82:0/108619523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 0x564004c109c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3be81cb740 1 -- 172.21.15.82:0/108619523 >> 172.21.15.82:0/108619523 conn(0x564004bcec90 msgr2=0x564004c045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3bc2ff5640 0 client.16254 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3be81cb740 1 -- 172.21.15.82:0/108619523 shutdown_connections 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3be81cb740 1 -- 172.21.15.82:0/108619523 wait complete. 2024-04-21T08:48:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.786+0000 7f3be81cb740 20 client.16254 trim_cache size 0 max 16384 2024-04-21T08:48:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3be81cb740 1 Processor -- start 2024-04-21T08:48:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3be81cb740 1 -- start start 2024-04-21T08:48:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3be81cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004bcec90 0x564004c1cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3be81cb740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 0x564004c0dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3be81cb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c088d0 0x564004c104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3be81cb740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01af270 con 0x564004c07b60 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3be81cb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01b0de0 con 0x564004c088d0 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3be81cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01af940 con 0x564004bcec90 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3bdeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 0x564004c0dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3bdeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 0x564004c0dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40964/0 (socket says 172.21.15.82:40964) 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3bdeffd640 1 -- 172.21.15.82:0/3294028162 learned_addr learned my addr 172.21.15.82:0/3294028162 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/3294028162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c088d0 0x564004c104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/3294028162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004bcec90 0x564004c1cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3bdeffd640 1 -- 172.21.15.82:0/3294028162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004bcec90 msgr2=0x564004c1cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/3294028162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004bcec90 0x564004c1cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3bdeffd640 1 -- 172.21.15.82:0/3294028162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c088d0 msgr2=0x564004c104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/3294028162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c088d0 0x564004c104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3bdeffd640 1 -- 172.21.15.82:0/3294028162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564004c06e30 con 0x564004c07b60 2024-04-21T08:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.801+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/3294028162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 0x564004c0dd80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000b7e0 tx=0x7f3bd000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299e7a1620493a59 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.802+0000 7f3bc4ff9640 1 -- 172.21.15.82:0/3294028162 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0022070 con 0x564004c07b60 2024-04-21T08:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.802+0000 7f3bc4ff9640 1 -- 172.21.15.82:0/3294028162 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd00143b0 con 0x564004c07b60 2024-04-21T08:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.802+0000 7f3bc4ff9640 1 -- 172.21.15.82:0/3294028162 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00255c0 con 0x564004c07b60 2024-04-21T08:48:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.802+0000 7f3be81cb740 1 -- 172.21.15.82:0/3294028162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 msgr2=0x564004c0dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.802+0000 7f3be81cb740 1 --2- 172.21.15.82:0/3294028162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 0x564004c0dd80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000b7e0 tx=0x7f3bd000b8d0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.802+0000 7f3be81cb740 1 -- 172.21.15.82:0/3294028162 shutdown_connections 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.802+0000 7f3be81cb740 1 --2- 172.21.15.82:0/3294028162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c088d0 0x564004c104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.802+0000 7f3be81cb740 1 --2- 172.21.15.82:0/3294028162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004c07b60 0x564004c0dd80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.802+0000 7f3be81cb740 1 --2- 172.21.15.82:0/3294028162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004bcec90 0x564004c1cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3be81cb740 1 -- 172.21.15.82:0/3294028162 >> 172.21.15.82:0/3294028162 conn(0x564004c0e460 msgr2=0x564004c0c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3be81cb740 1 -- 172.21.15.82:0/3294028162 shutdown_connections 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3be81cb740 1 -- 172.21.15.82:0/3294028162 wait complete. 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3be81cb740 1 -- start start 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3be81cb740 1 Processor -- start 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3be81cb740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004bcec90 0x564004c191f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3be81cb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c07b60 0x564004c19730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3be81cb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c088d0 0x564004c0a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3be81cb740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01b06e0 con 0x564004bcec90 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3be81cb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01b0010 con 0x564004c088d0 2024-04-21T08:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3be81cb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01ad250 con 0x564004c07b60 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3be81cb740 10 client.0 _ll_register_callbacks cb 0x7f3bd80124d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3bdf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004bcec90 0x564004c191f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3bdf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004bcec90 0x564004c191f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40980/0 (socket says 172.21.15.82:40980) 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.803+0000 7f3bdf7fe640 1 -- 172.21.15.82:0/716063988 learned_addr learned my addr 172.21.15.82:0/716063988 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c088d0 0x564004c0a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c07b60 0x564004c19730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bdf7fe640 1 -- 172.21.15.82:0/716063988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c07b60 msgr2=0x564004c19730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c07b60 0x564004c19730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bdf7fe640 1 -- 172.21.15.82:0/716063988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c088d0 msgr2=0x564004c0a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c088d0 0x564004c0a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bdf7fe640 1 -- 172.21.15.82:0/716063988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564004c0add0 con 0x564004bcec90 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c07b60 0x564004c19730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bdf7fe640 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004bcec90 0x564004c191f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8008870 tx=0x7f3bd8012750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f473ea91fad661 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bc2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bc2ff5640 1 -- 172.21.15.82:0/716063988 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800bc30 con 0x564004bcec90 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bc2ff5640 1 -- 172.21.15.82:0/716063988 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f3bd800bdd0 con 0x564004bcec90 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3be81cb740 1 -- 172.21.15.82:0/716063988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd00143b0 con 0x564004bcec90 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bc2ff5640 1 -- 172.21.15.82:0/716063988 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3bd8044050 con 0x564004bcec90 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3be81cb740 10 client.16260 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3be81cb740 1 -- 172.21.15.82:0/716063988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564004c86f40 con 0x564004bcec90 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bc2ff5640 1 -- 172.21.15.82:0/716063988 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd00143b0 con 0x564004bcec90 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.804+0000 7f3bc2ff5640 1 -- 172.21.15.82:0/716063988 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f3bd801d7b0 con 0x564004bcec90 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.805+0000 7f3be81cb740 10 client.16260 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.805+0000 7f3be81cb740 10 client.16260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.805+0000 7f3be81cb740 20 client.16260 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.805+0000 7f3be81cb740 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564004c110e0 0x564004c89610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.805+0000 7f3be81cb740 4 client.16260 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.805+0000 7f3be81cb740 1 -- 172.21.15.82:0/716063988 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564004b0c3d0 con 0x564004c110e0 2024-04-21T08:48:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.805+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564004c110e0 0x564004c89610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.805+0000 7f3bdeffd640 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564004c110e0 0x564004c89610 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=874085d21d237afb server_cookie=8df52c797ea364c7 in_seq=0 out_seq=0 2024-04-21T08:48:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.805+0000 7f3bc2ff5640 10 client.16260 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:02.813 INFO:tasks.workunit.client.0.smithi082.stdout:checking for snprintf... yes 2024-04-21T08:48:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3bc2ff5640 1 -- 172.21.15.82:0/716063988 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564004b0c3d0 con 0x564004c110e0 2024-04-21T08:48:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3bc2ff5640 10 client.16260 handle_command_reply: tid=0 2024-04-21T08:48:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3be81cb740 1 client.16260 shutdown 2024-04-21T08:48:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3be81cb740 1 -- 172.21.15.82:0/716063988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004bcec90 msgr2=0x564004c191f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3be81cb740 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004bcec90 0x564004c191f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8008870 tx=0x7f3bd8012750 comp rx=0 tx=0).stop 2024-04-21T08:48:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3be81cb740 1 -- 172.21.15.82:0/716063988 shutdown_connections 2024-04-21T08:48:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3be81cb740 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564004c110e0 0x564004c89610 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3be81cb740 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564004c088d0 0x564004c0a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3bc2ff5640 0 client.16260 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3be81cb740 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564004c07b60 0x564004c19730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3be81cb740 1 --2- 172.21.15.82:0/716063988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564004bcec90 0x564004c191f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3be81cb740 1 -- 172.21.15.82:0/716063988 >> 172.21.15.82:0/716063988 conn(0x564004c0e460 msgr2=0x564004c03c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3be81cb740 1 -- 172.21.15.82:0/716063988 shutdown_connections 2024-04-21T08:48:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3be81cb740 1 -- 172.21.15.82:0/716063988 wait complete. 2024-04-21T08:48:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.846+0000 7f3be81cb740 20 client.16260 trim_cache size 0 max 16384 2024-04-21T08:48:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:48:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:48:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:48:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:48:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.847+0000 7f3be55e2640 1 -- 172.21.15.82:0/1576326559 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3bc80a3ff0 msgr2=0x7f3bc80a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.847+0000 7f3be55e2640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3bc80a3ff0 0x7f3bc80a64b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40f03d0 tx=0x7f3bd40891c0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.847+0000 7f3be55e2640 1 -- 172.21.15.82:0/1576326559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3be0107270 msgr2=0x7f3be0198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.847+0000 7f3be55e2640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3be0107270 0x7f3be0198c70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4005ec0 tx=0x7f3bd402ca10 comp rx=0 tx=0).stop 2024-04-21T08:48:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.847+0000 7f3be55e2640 1 -- 172.21.15.82:0/1576326559 shutdown_connections 2024-04-21T08:48:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.847+0000 7f3be55e2640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3be019cfa0 0x7f3be019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.847+0000 7f3be55e2640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3bc80a3ff0 0x7f3bc80a64b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.847+0000 7f3be55e2640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3be0107270 0x7f3be0198c70 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.847+0000 7f3be55e2640 1 --2- 172.21.15.82:0/1576326559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3be01021a0 0x7f3be0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.847+0000 7f3be55e2640 1 -- 172.21.15.82:0/1576326559 >> 172.21.15.82:0/1576326559 conn(0x7f3be00fda00 msgr2=0x7f3be00ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.847+0000 7f3be55e2640 1 -- 172.21.15.82:0/1576326559 shutdown_connections 2024-04-21T08:48:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:02.847+0000 7f3be55e2640 1 -- 172.21.15.82:0/1576326559 wait complete. 2024-04-21T08:48:02.888 INFO:tasks.workunit.client.0.smithi082.stdout:checking for vsnprintf... yes 2024-04-21T08:48:02.949 INFO:tasks.workunit.client.0.smithi082.stdout:checking for vfprintf... yes 2024-04-21T08:48:03.011 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gethostname... yes 2024-04-21T08:48:03.071 INFO:tasks.workunit.client.0.smithi082.stdout:checking for setlocale... yes 2024-04-21T08:48:03.130 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sync... yes 2024-04-21T08:48:03.192 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether snprintf is C99 conformant... done 2024-04-21T08:48:03.203 INFO:tasks.workunit.client.0.smithi082.stdout:configure: creating ./config.status 2024-04-21T08:48:03.352 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: creating Makefile 2024-04-21T08:48:03.368 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: creating src/Makefile 2024-04-21T08:48:03.386 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: creating man/Makefile 2024-04-21T08:48:03.397 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: creating config.h 2024-04-21T08:48:03.452 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: executing depfiles commands 2024-04-21T08:48:03.576 INFO:tasks.workunit.client.0.smithi082.stderr:+ make 2024-04-21T08:48:03.585 INFO:tasks.workunit.client.0.smithi082.stdout:make all-recursive 2024-04-21T08:48:03.588 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-04-21T08:48:03.590 INFO:tasks.workunit.client.0.smithi082.stdout:Making all in src 2024-04-21T08:48:03.597 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-04-21T08:48:03.597 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-04-21T08:48:03.597 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-04-21T08:48:03.683 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-04-21T08:48:03.683 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-04-21T08:48:03.748 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-04-21T08:48:03.748 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-04-21T08:48:03.800 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-04-21T08:48:03.800 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-04-21T08:48:03.849 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-04-21T08:48:03.849 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-04-21T08:48:03.900 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-04-21T08:48:03.900 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-04-21T08:48:03.950 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-04-21T08:48:03.950 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-04-21T08:48:04.027 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-04-21T08:48:04.027 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-04-21T08:48:04.064 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-04-21T08:48:04.064 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-04-21T08:48:04.100 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-04-21T08:48:04.101 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-04-21T08:48:04.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:03 smithi082 ceph-mon[27967]: pgmap v34: 129 pgs: 129 active+clean; 4.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 269 KiB/s wr, 47 op/s 2024-04-21T08:48:04.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:03 smithi106 ceph-mon[31072]: pgmap v34: 129 pgs: 129 active+clean; 4.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 269 KiB/s wr, 47 op/s 2024-04-21T08:48:04.140 INFO:tasks.workunit.client.0.smithi082.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-04-21T08:48:04.158 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-04-21T08:48:04.159 INFO:tasks.workunit.client.0.smithi082.stdout:Making all in man 2024-04-21T08:48:04.180 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-04-21T08:48:04.180 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Nothing to be done for 'all'. 2024-04-21T08:48:04.180 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-04-21T08:48:04.181 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-04-21T08:48:04.182 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-04-21T08:48:04.183 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-04-21T08:48:04.183 INFO:tasks.workunit.client.0.smithi082.stderr:+ cd src 2024-04-21T08:48:04.183 INFO:tasks.workunit.client.0.smithi082.stderr:+ mkdir blogtest_in 2024-04-21T08:48:04.185 INFO:tasks.workunit.client.0.smithi082.stdout:running blogbench 2024-04-21T08:48:04.185 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'running blogbench' 2024-04-21T08:48:04.185 INFO:tasks.workunit.client.0.smithi082.stderr:+ ./blogbench -d blogtest_in 2024-04-21T08:48:04.185 INFO:tasks.workunit.client.0.smithi082.stdout: 2024-04-21T08:48:04.186 INFO:tasks.workunit.client.0.smithi082.stdout:Frequency = 10 secs 2024-04-21T08:48:04.186 INFO:tasks.workunit.client.0.smithi082.stdout:Scratch dir = [blogtest_in] 2024-04-21T08:48:04.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:03 smithi134 ceph-mon[31772]: pgmap v34: 129 pgs: 129 active+clean; 4.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 269 KiB/s wr, 47 op/s 2024-04-21T08:48:06.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:05 smithi082 ceph-mon[27967]: pgmap v35: 129 pgs: 129 active+clean; 4.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 267 KiB/s wr, 41 op/s 2024-04-21T08:48:06.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:05 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:48:06.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:05 smithi106 ceph-mon[31072]: pgmap v35: 129 pgs: 129 active+clean; 4.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 267 KiB/s wr, 41 op/s 2024-04-21T08:48:06.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:05 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:48:06.220 INFO:tasks.workunit.client.0.smithi082.stdout:Spawning 3 writers... 2024-04-21T08:48:06.220 INFO:tasks.workunit.client.0.smithi082.stdout:Spawning 1 rewriters... 2024-04-21T08:48:06.220 INFO:tasks.workunit.client.0.smithi082.stdout:Spawning 5 commenters... 2024-04-21T08:48:06.220 INFO:tasks.workunit.client.0.smithi082.stdout:Spawning 100 readers... 2024-04-21T08:48:06.221 INFO:tasks.workunit.client.0.smithi082.stdout:Benchmarking for 30 iterations. 2024-04-21T08:48:06.221 INFO:tasks.workunit.client.0.smithi082.stdout:The test will run during 5 minutes. 2024-04-21T08:48:06.221 INFO:tasks.workunit.client.0.smithi082.stdout: 2024-04-21T08:48:06.221 INFO:tasks.workunit.client.0.smithi082.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-04-21T08:48:06.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:05 smithi134 ceph-mon[31772]: pgmap v35: 129 pgs: 129 active+clean; 4.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 267 KiB/s wr, 41 op/s 2024-04-21T08:48:06.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:05 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:48:07.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:07 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:07 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:07 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:07 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:07 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:07 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:07 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:48:07.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:07 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:48:07.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:07 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:07 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:07 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:07 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:07 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:07 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:07 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:07 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:48:07.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:07 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:48:07.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:07 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:07 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:07 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:07 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:07 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:07 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:07 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:07 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:48:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:07 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:48:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:07 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:48:08.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:08 smithi082 ceph-mon[27967]: pgmap v36: 129 pgs: 129 active+clean; 6.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 519 KiB/s wr, 77 op/s 2024-04-21T08:48:08.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:08 smithi106 ceph-mon[31072]: pgmap v36: 129 pgs: 129 active+clean; 6.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 519 KiB/s wr, 77 op/s 2024-04-21T08:48:08.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:08 smithi134 ceph-mon[31772]: pgmap v36: 129 pgs: 129 active+clean; 6.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 519 KiB/s wr, 77 op/s 2024-04-21T08:48:10.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:10 smithi082 ceph-mon[27967]: pgmap v37: 129 pgs: 129 active+clean; 7.2 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 564 KiB/s wr, 97 op/s 2024-04-21T08:48:10.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:10 smithi106 ceph-mon[31072]: pgmap v37: 129 pgs: 129 active+clean; 7.2 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 564 KiB/s wr, 97 op/s 2024-04-21T08:48:10.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:10 smithi134 ceph-mon[31772]: pgmap v37: 129 pgs: 129 active+clean; 7.2 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 564 KiB/s wr, 97 op/s 2024-04-21T08:48:11.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:11.749 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:11.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.885+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/386642410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c48101a40 msgr2=0x7f6c48105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:11.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.885+0000 7f6c4fc6d640 1 --2- 172.21.15.82:0/386642410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c48101a40 0x7f6c48105a80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009900 tx=0x7f6c3802bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:11.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.885+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/386642410 shutdown_connections 2024-04-21T08:48:11.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.885+0000 7f6c4fc6d640 1 --2- 172.21.15.82:0/386642410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c48106160 0x7f6c4810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.885+0000 7f6c4fc6d640 1 --2- 172.21.15.82:0/386642410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c48101a40 0x7f6c48105a80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.885+0000 7f6c4fc6d640 1 --2- 172.21.15.82:0/386642410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c48101090 0x7f6c48101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.885+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/386642410 >> 172.21.15.82:0/386642410 conn(0x7f6c480771d0 msgr2=0x7f6c480775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:11.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.885+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/386642410 shutdown_connections 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.885+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/386642410 wait complete. 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.885+0000 7f6c4fc6d640 1 Processor -- start 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4fc6d640 1 -- start start 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4fc6d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c48101090 0x7f6c48198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4fc6d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c48106160 0x7f6c48198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4fc6d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c4819cff0 0x7f6c4819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4fc6d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6c48111080 con 0x7f6c4819cff0 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4fc6d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c48110f00 con 0x7f6c48106160 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4fc6d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c48111200 con 0x7f6c48101090 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4e20a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c4819cff0 0x7f6c4819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4e20a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c4819cff0 0x7f6c4819d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41000/0 (socket says 172.21.15.82:41000) 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4e20a640 1 -- 172.21.15.82:0/485891628 learned_addr learned my addr 172.21.15.82:0/485891628 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4d208640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c48106160 0x7f6c48198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4da09640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c48101090 0x7f6c48198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4e20a640 1 -- 172.21.15.82:0/485891628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c48101090 msgr2=0x7f6c48198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4e20a640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c48101090 0x7f6c48198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4e20a640 1 -- 172.21.15.82:0/485891628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c48106160 msgr2=0x7f6c48198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4e20a640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c48106160 0x7f6c48198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4e20a640 1 -- 172.21.15.82:0/485891628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4819dae0 con 0x7f6c4819cff0 2024-04-21T08:48:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c4e20a640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c4819cff0 0x7f6c4819d4a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400e970 tx=0x7f6c4400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c3effd640 1 -- 172.21.15.82:0/485891628 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400ccc0 con 0x7f6c4819cff0 2024-04-21T08:48:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.886+0000 7f6c3effd640 1 -- 172.21.15.82:0/485891628 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c4400ce60 con 0x7f6c4819cff0 2024-04-21T08:48:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.887+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/485891628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4819ddd0 con 0x7f6c4819cff0 2024-04-21T08:48:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.887+0000 7f6c3effd640 1 -- 172.21.15.82:0/485891628 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44010430 con 0x7f6c4819cff0 2024-04-21T08:48:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.887+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/485891628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c481a5540 con 0x7f6c4819cff0 2024-04-21T08:48:11.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.889+0000 7f6c3effd640 1 -- 172.21.15.82:0/485891628 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f6c44004ff0 con 0x7f6c4819cff0 2024-04-21T08:48:11.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.889+0000 7f6c52865740 1 Processor -- start 2024-04-21T08:48:11.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.889+0000 7f6c52865740 1 -- start start 2024-04-21T08:48:11.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.889+0000 7f6c52865740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x565204365960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.889+0000 7f6c52865740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565204365ea0 0x56520436e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.889+0000 7f6c3effd640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c340a3ed0 0x7f6c340a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.889+0000 7f6c52865740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 0x565204370e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.889+0000 7f6c52865740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5652042ca1e0 con 0x56520436ea80 2024-04-21T08:48:11.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.889+0000 7f6c52865740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x565204267390 con 0x565204365ea0 2024-04-21T08:48:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.889+0000 7f6c52865740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5652042c61a0 con 0x565204365580 2024-04-21T08:48:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4da09640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c340a3ed0 0x7f6c340a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4e20a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 0x565204370e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4d208640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x565204365960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4e20a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 0x565204370e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41008/0 (socket says 172.21.15.82:41008) 2024-04-21T08:48:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4e20a640 1 -- 172.21.15.82:0/3728899933 learned_addr learned my addr 172.21.15.82:0/3728899933 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4d208640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x565204365960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:59176/0 (socket says 172.21.15.82:59176) 2024-04-21T08:48:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4d208640 1 --2- 172.21.15.82:0/3728899933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565204365ea0 0x56520436e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c3effd640 1 -- 172.21.15.82:0/485891628 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6c440d1230 con 0x7f6c4819cff0 2024-04-21T08:48:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4d208640 1 -- 172.21.15.82:0/3728899933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565204365ea0 msgr2=0x56520436e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4d208640 1 --2- 172.21.15.82:0/3728899933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565204365ea0 0x56520436e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4d208640 1 -- 172.21.15.82:0/3728899933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 msgr2=0x565204370e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4d208640 1 --2- 172.21.15.82:0/3728899933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 0x565204370e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4d208640 1 -- 172.21.15.82:0/3728899933 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652040a67d0 con 0x565204365580 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4da09640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c340a3ed0 0x7f6c340a6390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c40005fd0 tx=0x7f6c40009680 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c4d208640 1 --2- 172.21.15.82:0/3728899933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x565204365960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c38036040 tx=0x7f6c38005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ae714c8066eb70f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c33fff640 1 -- 172.21.15.82:0/3728899933 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3804a070 con 0x565204365580 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c33fff640 1 -- 172.21.15.82:0/3728899933 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3802cc90 con 0x565204365580 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.890+0000 7f6c33fff640 1 -- 172.21.15.82:0/3728899933 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38003e90 con 0x565204365580 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.891+0000 7f6c52865740 1 -- 172.21.15.82:0/3728899933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 msgr2=0x565204365960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.891+0000 7f6c52865740 1 --2- 172.21.15.82:0/3728899933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x565204365960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c38036040 tx=0x7f6c38005e80 comp rx=0 tx=0).stop 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.891+0000 7f6c52865740 1 -- 172.21.15.82:0/3728899933 shutdown_connections 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.891+0000 7f6c52865740 1 --2- 172.21.15.82:0/3728899933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 0x565204370e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.891+0000 7f6c52865740 1 --2- 172.21.15.82:0/3728899933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565204365ea0 0x56520436e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.891+0000 7f6c52865740 1 --2- 172.21.15.82:0/3728899933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x565204365960 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.891+0000 7f6c52865740 1 -- 172.21.15.82:0/3728899933 >> 172.21.15.82:0/3728899933 conn(0x56520435fcb0 msgr2=0x565204360090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.891+0000 7f6c52865740 1 -- 172.21.15.82:0/3728899933 shutdown_connections 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.891+0000 7f6c52865740 1 -- 172.21.15.82:0/3728899933 wait complete. 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.891+0000 7f6c52865740 1 -- start start 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c52865740 1 Processor -- start 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c52865740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x56520436bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c52865740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 0x56520436c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c52865740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56520436ea80 0x56520436c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c52865740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x565204371810 con 0x565204365ea0 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c52865740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5652042832f0 con 0x56520436ea80 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c52865740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565204371990 con 0x565204365580 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c52865740 10 client.0 _ll_register_callbacks cb 0x565204290580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c4d208640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 0x56520436c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c4d208640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 0x56520436c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41016/0 (socket says 172.21.15.82:41016) 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c4d208640 1 -- 172.21.15.82:0/2663392872 learned_addr learned my addr 172.21.15.82:0/2663392872 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c4e20a640 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56520436ea80 0x56520436c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c4da09640 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x56520436bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c4d208640 1 -- 172.21.15.82:0/2663392872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 msgr2=0x56520436bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c4d208640 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x56520436bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c4d208640 1 -- 172.21.15.82:0/2663392872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56520436ea80 msgr2=0x56520436c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c4d208640 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56520436ea80 0x56520436c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.892+0000 7f6c4d208640 1 -- 172.21.15.82:0/2663392872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565203fdc3f0 con 0x565204365ea0 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c4d208640 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 0x56520436c0d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6c3802c170 tx=0x7f6c38009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd1f6fb0616a46d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c317fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c317fa640 1 -- 172.21.15.82:0/2663392872 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3804a040 con 0x565204365ea0 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c317fa640 1 -- 172.21.15.82:0/2663392872 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6c38003e70 con 0x565204365ea0 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c52865740 1 -- 172.21.15.82:0/2663392872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c3802cc90 con 0x565204365ea0 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c317fa640 1 -- 172.21.15.82:0/2663392872 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c38057050 con 0x565204365ea0 2024-04-21T08:48:11.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c52865740 10 client.16272 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c52865740 1 -- 172.21.15.82:0/2663392872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565203f9c0f0 con 0x565204365ea0 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c317fa640 1 -- 172.21.15.82:0/2663392872 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3802cc90 con 0x565204365ea0 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c317fa640 1 -- 172.21.15.82:0/2663392872 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f6c380048c0 con 0x565204365ea0 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c52865740 10 client.16272 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c52865740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c52865740 20 client.16272 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c52865740 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56520437e980 0x565204380d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c52865740 4 client.16272 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c52865740 1 -- 172.21.15.82:0/2663392872 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652042d7c00 con 0x56520437e980 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.893+0000 7f6c4da09640 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56520437e980 0x565204380d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.894+0000 7f6c4da09640 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56520437e980 0x565204380d70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c69b6dcd7221421d server_cookie=2755478a0242f789 in_seq=0 out_seq=0 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.894+0000 7f6c317fa640 10 client.16272 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c317fa640 1 -- 172.21.15.82:0/2663392872 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652042d7c00 con 0x56520437e980 2024-04-21T08:48:11.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c317fa640 10 client.16272 handle_command_reply: tid=0 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c52865740 1 client.16272 shutdown 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c52865740 1 -- 172.21.15.82:0/2663392872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 msgr2=0x56520436c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c52865740 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 0x56520436c0d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6c3802c170 tx=0x7f6c38009680 comp rx=0 tx=0).stop 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c52865740 1 -- 172.21.15.82:0/2663392872 shutdown_connections 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c52865740 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56520437e980 0x565204380d70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c52865740 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56520436ea80 0x56520436c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c317fa640 0 client.16272 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c52865740 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 0x56520436c0d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c52865740 1 --2- 172.21.15.82:0/2663392872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x56520436bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c52865740 1 -- 172.21.15.82:0/2663392872 >> 172.21.15.82:0/2663392872 conn(0x56520435fcb0 msgr2=0x565204361a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c52865740 1 -- 172.21.15.82:0/2663392872 shutdown_connections 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c52865740 1 -- 172.21.15.82:0/2663392872 wait complete. 2024-04-21T08:48:11.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.895+0000 7f6c52865740 20 client.16272 trim_cache size 0 max 16384 2024-04-21T08:48:11.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c52865740 1 Processor -- start 2024-04-21T08:48:11.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c52865740 1 -- start start 2024-04-21T08:48:11.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c52865740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x56520436e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c52865740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 0x56520436aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c52865740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56520436ea80 0x565204383d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c52865740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481af370 con 0x565204365ea0 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c52865740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481b0ee0 con 0x56520436ea80 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c52865740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481afa40 con 0x565204365580 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c4d208640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 0x56520436aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c4d208640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 0x56520436aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41022/0 (socket says 172.21.15.82:41022) 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c4d208640 1 -- 172.21.15.82:0/3438208041 learned_addr learned my addr 172.21.15.82:0/3438208041 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c4e20a640 1 --2- 172.21.15.82:0/3438208041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56520436ea80 0x565204383d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c4da09640 1 --2- 172.21.15.82:0/3438208041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x56520436e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c4d208640 1 -- 172.21.15.82:0/3438208041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 msgr2=0x56520436e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c4d208640 1 --2- 172.21.15.82:0/3438208041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x56520436e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c4d208640 1 -- 172.21.15.82:0/3438208041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56520436ea80 msgr2=0x565204383d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c4d208640 1 --2- 172.21.15.82:0/3438208041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56520436ea80 0x565204383d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c4d208640 1 -- 172.21.15.82:0/3438208041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56520435f7e0 con 0x565204365ea0 2024-04-21T08:48:11.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c4da09640 1 --2- 172.21.15.82:0/3438208041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x56520436e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.913+0000 7f6c4d208640 1 --2- 172.21.15.82:0/3438208041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 0x56520436aad0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6c38004c60 tx=0x7f6c3802c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b027cf1d4d2808 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.914+0000 7f6c337fe640 1 -- 172.21.15.82:0/3438208041 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38005d50 con 0x565204365ea0 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.914+0000 7f6c337fe640 1 -- 172.21.15.82:0/3438208041 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3802caf0 con 0x565204365ea0 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.914+0000 7f6c337fe640 1 -- 172.21.15.82:0/3438208041 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38003e40 con 0x565204365ea0 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.914+0000 7f6c52865740 1 -- 172.21.15.82:0/3438208041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 msgr2=0x56520436aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.914+0000 7f6c52865740 1 --2- 172.21.15.82:0/3438208041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 0x56520436aad0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6c38004c60 tx=0x7f6c3802c610 comp rx=0 tx=0).stop 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.914+0000 7f6c52865740 1 -- 172.21.15.82:0/3438208041 shutdown_connections 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.914+0000 7f6c52865740 1 --2- 172.21.15.82:0/3438208041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56520436ea80 0x565204383d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.914+0000 7f6c52865740 1 --2- 172.21.15.82:0/3438208041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565204365ea0 0x56520436aad0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.914+0000 7f6c52865740 1 --2- 172.21.15.82:0/3438208041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365580 0x56520436e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.914+0000 7f6c52865740 1 -- 172.21.15.82:0/3438208041 >> 172.21.15.82:0/3438208041 conn(0x56520435fcb0 msgr2=0x56520437a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.914+0000 7f6c52865740 1 -- 172.21.15.82:0/3438208041 shutdown_connections 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.914+0000 7f6c52865740 1 -- 172.21.15.82:0/3438208041 wait complete. 2024-04-21T08:48:11.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c52865740 1 -- start start 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c52865740 1 Processor -- start 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c52865740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565204365580 0x5652043819c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c52865740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365ea0 0x565204381f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c52865740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 0x565204382440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c52865740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481b07e0 con 0x56520436ea80 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c52865740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481b0110 con 0x565204365580 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c52865740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481ad350 con 0x565204365ea0 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c52865740 10 client.0 _ll_register_callbacks cb 0x7f6c38048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c4e20a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 0x565204382440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c4da09640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565204365580 0x5652043819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c4e20a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 0x565204382440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41038/0 (socket says 172.21.15.82:41038) 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c4e20a640 1 -- 172.21.15.82:0/128262926 learned_addr learned my addr 172.21.15.82:0/128262926 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c4d208640 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365ea0 0x565204381f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c4e20a640 1 -- 172.21.15.82:0/128262926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365ea0 msgr2=0x565204381f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c4e20a640 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365ea0 0x565204381f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c4e20a640 1 -- 172.21.15.82:0/128262926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565204365580 msgr2=0x5652043819c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c4e20a640 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565204365580 0x5652043819c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c4e20a640 1 -- 172.21.15.82:0/128262926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652043e3d60 con 0x56520436ea80 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.915+0000 7f6c4da09640 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565204365580 0x5652043819c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c4e20a640 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 0x565204382440 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6c44099030 tx=0x7f6c4407b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be948a4219b2cf67 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c317fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c317fa640 1 -- 172.21.15.82:0/128262926 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4409a220 con 0x56520436ea80 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c317fa640 1 -- 172.21.15.82:0/128262926 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6c4409ac70 con 0x56520436ea80 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c52865740 1 -- 172.21.15.82:0/128262926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c3802caf0 con 0x56520436ea80 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c317fa640 1 -- 172.21.15.82:0/128262926 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c440055f0 con 0x56520436ea80 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c52865740 10 client.16278 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c52865740 1 -- 172.21.15.82:0/128262926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5652043e5020 con 0x56520436ea80 2024-04-21T08:48:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c317fa640 1 -- 172.21.15.82:0/128262926 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3802caf0 con 0x56520436ea80 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c317fa640 1 -- 172.21.15.82:0/128262926 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f6c4409a620 con 0x56520436ea80 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c52865740 10 client.16278 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c52865740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c52865740 20 client.16278 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c52865740 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5652043e57a0 0x5652043e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c52865740 4 client.16278 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c52865740 1 -- 172.21.15.82:0/128262926 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652042d1a70 con 0x5652043e57a0 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.916+0000 7f6c4da09640 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5652043e57a0 0x5652043e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c317fa640 10 client.16278 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c4da09640 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5652043e57a0 0x5652043e7bc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c033acb2805298bd server_cookie=85706f809b915110 in_seq=0 out_seq=0 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c317fa640 1 -- 172.21.15.82:0/128262926 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5652042d1a70 con 0x5652043e57a0 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c317fa640 10 client.16278 handle_command_reply: tid=0 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c52865740 1 client.16278 shutdown 2024-04-21T08:48:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c52865740 1 -- 172.21.15.82:0/128262926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 msgr2=0x565204382440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c52865740 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 0x565204382440 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6c44099030 tx=0x7f6c4407b040 comp rx=0 tx=0).stop 2024-04-21T08:48:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c52865740 1 -- 172.21.15.82:0/128262926 shutdown_connections 2024-04-21T08:48:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c52865740 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5652043e57a0 0x5652043e7bc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c52865740 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56520436ea80 0x565204382440 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c52865740 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565204365ea0 0x565204381f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c52865740 1 --2- 172.21.15.82:0/128262926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565204365580 0x5652043819c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c317fa640 0 client.16278 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c52865740 1 -- 172.21.15.82:0/128262926 >> 172.21.15.82:0/128262926 conn(0x56520435fcb0 msgr2=0x565204379be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c52865740 1 -- 172.21.15.82:0/128262926 shutdown_connections 2024-04-21T08:48:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c52865740 1 -- 172.21.15.82:0/128262926 wait complete. 2024-04-21T08:48:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.917+0000 7f6c52865740 20 client.16278 trim_cache size 0 max 16384 2024-04-21T08:48:11.925 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:48:11.925 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:48:11.925 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:48:11.925 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:48:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.918+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/485891628 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c340a3ed0 msgr2=0x7f6c340a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.918+0000 7f6c4fc6d640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c340a3ed0 0x7f6c340a6390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c40005fd0 tx=0x7f6c40009680 comp rx=0 tx=0).stop 2024-04-21T08:48:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.918+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/485891628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c4819cff0 msgr2=0x7f6c4819d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.918+0000 7f6c4fc6d640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c4819cff0 0x7f6c4819d4a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400e970 tx=0x7f6c4400ed70 comp rx=0 tx=0).stop 2024-04-21T08:48:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.918+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/485891628 shutdown_connections 2024-04-21T08:48:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.918+0000 7f6c4fc6d640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c340a3ed0 0x7f6c340a6390 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.918+0000 7f6c4fc6d640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c4819cff0 0x7f6c4819d4a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.918+0000 7f6c4fc6d640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c48106160 0x7f6c48198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.918+0000 7f6c4fc6d640 1 --2- 172.21.15.82:0/485891628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c48101090 0x7f6c48198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.918+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/485891628 >> 172.21.15.82:0/485891628 conn(0x7f6c480771d0 msgr2=0x7f6c480fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.918+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/485891628 shutdown_connections 2024-04-21T08:48:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:11.918+0000 7f6c4fc6d640 1 -- 172.21.15.82:0/485891628 wait complete. 2024-04-21T08:48:11.928 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:48:11.928 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:48:12.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.032+0000 7fecc58ba640 1 -- 172.21.15.82:0/1855958554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0101a60 msgr2=0x7fecc0105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.032+0000 7fecc58ba640 1 --2- 172.21.15.82:0/1855958554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0101a60 0x7fecc0105aa0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7feca8009900 tx=0x7feca802bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:12.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.032+0000 7fecc58ba640 1 -- 172.21.15.82:0/1855958554 shutdown_connections 2024-04-21T08:48:12.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.032+0000 7fecc58ba640 1 --2- 172.21.15.82:0/1855958554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecc0106180 0x7fecc010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.032+0000 7fecc58ba640 1 --2- 172.21.15.82:0/1855958554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc0101a60 0x7fecc0105aa0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.032+0000 7fecc58ba640 1 --2- 172.21.15.82:0/1855958554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecc01010b0 0x7fecc0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.032+0000 7fecc58ba640 1 -- 172.21.15.82:0/1855958554 >> 172.21.15.82:0/1855958554 conn(0x7fecc00771d0 msgr2=0x7fecc00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.032+0000 7fecc58ba640 1 -- 172.21.15.82:0/1855958554 shutdown_connections 2024-04-21T08:48:12.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.032+0000 7fecc58ba640 1 -- 172.21.15.82:0/1855958554 wait complete. 2024-04-21T08:48:12.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecc58ba640 1 Processor -- start 2024-04-21T08:48:12.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecc58ba640 1 -- start start 2024-04-21T08:48:12.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecc58ba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc01010b0 0x7fecc0197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecc58ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecc0106180 0x7fecc0197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecc58ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecc019bee0 0x7fecc019c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecc58ba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01110c0 con 0x7fecc01010b0 2024-04-21T08:48:12.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecc58ba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0110f40 con 0x7fecc0106180 2024-04-21T08:48:12.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecc58ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0111240 con 0x7fecc019bee0 2024-04-21T08:48:12.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecbeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc01010b0 0x7fecc0197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecbeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc01010b0 0x7fecc0197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41054/0 (socket says 172.21.15.82:41054) 2024-04-21T08:48:12.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecbeffd640 1 -- 172.21.15.82:0/1330891724 learned_addr learned my addr 172.21.15.82:0/1330891724 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecb67fc640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecc0106180 0x7fecc0197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecbf7fe640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecc019bee0 0x7fecc019c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecbeffd640 1 -- 172.21.15.82:0/1330891724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecc019bee0 msgr2=0x7fecc019c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecbeffd640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecc019bee0 0x7fecc019c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecbeffd640 1 -- 172.21.15.82:0/1330891724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecc0106180 msgr2=0x7fecc0197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecbeffd640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecc0106180 0x7fecc0197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecbeffd640 1 -- 172.21.15.82:0/1330891724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc019c9d0 con 0x7fecc01010b0 2024-04-21T08:48:12.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.033+0000 7fecbf7fe640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecc019bee0 0x7fecc019c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:12.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.034+0000 7fecbeffd640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc01010b0 0x7fecc0197620 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fecb000d770 tx=0x7fecb000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.034+0000 7fecc48b8640 1 -- 172.21.15.82:0/1330891724 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0017070 con 0x7fecc01010b0 2024-04-21T08:48:12.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.034+0000 7fecc48b8640 1 -- 172.21.15.82:0/1330891724 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb00054d0 con 0x7fecc01010b0 2024-04-21T08:48:12.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.034+0000 7fecc58ba640 1 -- 172.21.15.82:0/1330891724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc019ccc0 con 0x7fecc01010b0 2024-04-21T08:48:12.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.034+0000 7fecc48b8640 1 -- 172.21.15.82:0/1330891724 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0004c30 con 0x7fecc01010b0 2024-04-21T08:48:12.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.034+0000 7fecc58ba640 1 -- 172.21.15.82:0/1330891724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc006a930 con 0x7fecc01010b0 2024-04-21T08:48:12.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.035+0000 7fecc58ba640 1 -- 172.21.15.82:0/1330891724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc0069230 con 0x7fecc01010b0 2024-04-21T08:48:12.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.036+0000 7fecc48b8640 1 -- 172.21.15.82:0/1330891724 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fecb0004e10 con 0x7fecc01010b0 2024-04-21T08:48:12.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.036+0000 7fecc48b8640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fecac0a3f50 0x7fecac0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.037+0000 7fecb67fc640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fecac0a3f50 0x7fecac0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.037+0000 7fecc48b8640 1 -- 172.21.15.82:0/1330891724 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fecb00d08f0 con 0x7fecc01010b0 2024-04-21T08:48:12.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.037+0000 7fecb67fc640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fecac0a3f50 0x7fecac0a6410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feca8009230 tx=0x7feca8005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.041+0000 7fecc48b8640 1 -- 172.21.15.82:0/1330891724 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fecb009c990 con 0x7fecc01010b0 2024-04-21T08:48:12.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:12 smithi082 ceph-mon[27967]: pgmap v38: 129 pgs: 129 active+clean; 7.2 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 553 KiB/s wr, 93 op/s 2024-04-21T08:48:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.276+0000 7fecc58ba640 1 -- 172.21.15.82:0/1330891724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecc0101a60 con 0x7fecc01010b0 2024-04-21T08:48:12.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.277+0000 7fecc48b8640 1 -- 172.21.15.82:0/1330891724 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11945 (secure 0 0 0) 0x7fecb009c290 con 0x7fecc01010b0 2024-04-21T08:48:12.277 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:48:12.277 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:47:58.422767+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24481,24476,24407,24409]},"id":1}]} 2024-04-21T08:48:12.277 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 29 2024-04-21T08:48:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.279+0000 7fecc58ba640 1 -- 172.21.15.82:0/1330891724 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fecac0a3f50 msgr2=0x7fecac0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.279+0000 7fecc58ba640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fecac0a3f50 0x7fecac0a6410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feca8009230 tx=0x7feca8005fd0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.279+0000 7fecc58ba640 1 -- 172.21.15.82:0/1330891724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc01010b0 msgr2=0x7fecc0197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.279+0000 7fecc58ba640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc01010b0 0x7fecc0197620 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fecb000d770 tx=0x7fecb000dc30 comp rx=0 tx=0).stop 2024-04-21T08:48:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.279+0000 7fecc58ba640 1 -- 172.21.15.82:0/1330891724 shutdown_connections 2024-04-21T08:48:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.279+0000 7fecc58ba640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fecc019bee0 0x7fecc019c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.279+0000 7fecc58ba640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fecc0106180 0x7fecc0197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.279+0000 7fecc58ba640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fecac0a3f50 0x7fecac0a6410 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.279+0000 7fecc58ba640 1 --2- 172.21.15.82:0/1330891724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecc01010b0 0x7fecc0197620 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.279+0000 7fecc58ba640 1 -- 172.21.15.82:0/1330891724 >> 172.21.15.82:0/1330891724 conn(0x7fecc00771d0 msgr2=0x7fecc00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.279+0000 7fecc58ba640 1 -- 172.21.15.82:0/1330891724 shutdown_connections 2024-04-21T08:48:12.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.279+0000 7fecc58ba640 1 -- 172.21.15.82:0/1330891724 wait complete. 2024-04-21T08:48:12.290 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:48:12.294 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:12 smithi134 ceph-mon[31772]: pgmap v38: 129 pgs: 129 active+clean; 7.2 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 553 KiB/s wr, 93 op/s 2024-04-21T08:48:12.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:12 smithi106 ceph-mon[31072]: pgmap v38: 129 pgs: 129 active+clean; 7.2 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 553 KiB/s wr, 93 op/s 2024-04-21T08:48:12.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.410+0000 7f5097750640 1 -- 172.21.15.82:0/115347306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5090101090 msgr2=0x7f5090101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.410+0000 7f5097750640 1 --2- 172.21.15.82:0/115347306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5090101090 0x7f5090101470 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f508c009900 tx=0x7f508c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:12.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.410+0000 7f5097750640 1 -- 172.21.15.82:0/115347306 shutdown_connections 2024-04-21T08:48:12.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.410+0000 7f5097750640 1 --2- 172.21.15.82:0/115347306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5090106160 0x7f509010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.410+0000 7f5097750640 1 --2- 172.21.15.82:0/115347306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5090101a40 0x7f5090105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.410+0000 7f5097750640 1 --2- 172.21.15.82:0/115347306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5090101090 0x7f5090101470 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.410+0000 7f5097750640 1 -- 172.21.15.82:0/115347306 >> 172.21.15.82:0/115347306 conn(0x7f5090077f50 msgr2=0x7f5090078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.410+0000 7f5097750640 1 -- 172.21.15.82:0/115347306 shutdown_connections 2024-04-21T08:48:12.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.410+0000 7f5097750640 1 -- 172.21.15.82:0/115347306 wait complete. 2024-04-21T08:48:12.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5097750640 1 Processor -- start 2024-04-21T08:48:12.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5097750640 1 -- start start 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5097750640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5090101090 0x7f5090197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5097750640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5090106160 0x7f5090197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5097750640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509019bf40 0x7f509019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5097750640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f50901110c0 con 0x7f509019bf40 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5097750640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5090110f40 con 0x7f5090101090 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5097750640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5090111240 con 0x7f5090106160 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5095ced640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509019bf40 0x7f509019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5095ced640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509019bf40 0x7f509019c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41060/0 (socket says 172.21.15.82:41060) 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5094ceb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5090106160 0x7f5090197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f50954ec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5090101090 0x7f5090197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5095ced640 1 -- 172.21.15.82:0/1200129341 learned_addr learned my addr 172.21.15.82:0/1200129341 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5095ced640 1 -- 172.21.15.82:0/1200129341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5090106160 msgr2=0x7f5090197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5095ced640 1 --2- 172.21.15.82:0/1200129341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5090106160 0x7f5090197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5095ced640 1 -- 172.21.15.82:0/1200129341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5090101090 msgr2=0x7f5090197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5095ced640 1 --2- 172.21.15.82:0/1200129341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5090101090 0x7f5090197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.411+0000 7f5095ced640 1 -- 172.21.15.82:0/1200129341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509019ca30 con 0x7f509019bf40 2024-04-21T08:48:12.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.412+0000 7f5095ced640 1 --2- 172.21.15.82:0/1200129341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509019bf40 0x7f509019c3f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f508800eb10 tx=0x7f508800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.412+0000 7f50867fc640 1 -- 172.21.15.82:0/1200129341 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088004b20 con 0x7f509019bf40 2024-04-21T08:48:12.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.412+0000 7f50867fc640 1 -- 172.21.15.82:0/1200129341 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5088004cc0 con 0x7f509019bf40 2024-04-21T08:48:12.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.412+0000 7f5097750640 1 -- 172.21.15.82:0/1200129341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509019cd20 con 0x7f509019bf40 2024-04-21T08:48:12.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.412+0000 7f50867fc640 1 -- 172.21.15.82:0/1200129341 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088010670 con 0x7f509019bf40 2024-04-21T08:48:12.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.412+0000 7f5097750640 1 -- 172.21.15.82:0/1200129341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50901a44c0 con 0x7f509019bf40 2024-04-21T08:48:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.414+0000 7f50867fc640 1 -- 172.21.15.82:0/1200129341 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f5088005020 con 0x7f509019bf40 2024-04-21T08:48:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.414+0000 7f509a41b740 1 Processor -- start 2024-04-21T08:48:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.414+0000 7f509a41b740 1 -- start start 2024-04-21T08:48:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.414+0000 7f50867fc640 1 --2- 172.21.15.82:0/1200129341 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f507c0a3ed0 0x7f507c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f5094ceb640 1 --2- 172.21.15.82:0/1200129341 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f507c0a3ed0 0x7f507c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f50867fc640 1 -- 172.21.15.82:0/1200129341 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f50880d13a0 con 0x7f509019bf40 2024-04-21T08:48:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f509a41b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 0x55b4b2ff2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f509a41b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ffd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f509a41b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ffdc10 0x55b4b3000020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f509a41b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b4b2f571f0 con 0x55b4b2ff2250 2024-04-21T08:48:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f509a41b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b4b2ef44b0 con 0x55b4b2ffdc10 2024-04-21T08:48:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f509a41b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b4b2f531b0 con 0x55b4b2ff2c00 2024-04-21T08:48:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f50954ec640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 0x55b4b2ff2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f50954ec640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 0x55b4b2ff2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41072/0 (socket says 172.21.15.82:41072) 2024-04-21T08:48:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f50954ec640 1 -- 172.21.15.82:0/4182238886 learned_addr learned my addr 172.21.15.82:0/4182238886 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f5095ced640 1 --2- 172.21.15.82:0/4182238886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ffdc10 0x55b4b3000020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f5094ceb640 1 --2- 172.21.15.82:0/4182238886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ffd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f5094ceb640 1 --2- 172.21.15.82:0/1200129341 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f507c0a3ed0 0x7f507c0a6390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5090100a70 tx=0x7f5080011440 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f50954ec640 1 -- 172.21.15.82:0/4182238886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 msgr2=0x55b4b2ffd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f50954ec640 1 --2- 172.21.15.82:0/4182238886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ffd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f50954ec640 1 -- 172.21.15.82:0/4182238886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ffdc10 msgr2=0x55b4b3000020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f50954ec640 1 --2- 172.21.15.82:0/4182238886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ffdc10 0x55b4b3000020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f50954ec640 1 -- 172.21.15.82:0/4182238886 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4b2ee61b0 con 0x55b4b2ff2250 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f5095ced640 1 --2- 172.21.15.82:0/4182238886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ffdc10 0x55b4b3000020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.415+0000 7f50954ec640 1 --2- 172.21.15.82:0/4182238886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 0x55b4b2ff2630 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f508c009900 tx=0x7f508c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecce4b260443397 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.416+0000 7f507b7fe640 1 -- 172.21.15.82:0/4182238886 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c045020 con 0x55b4b2ff2250 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.416+0000 7f507b7fe640 1 -- 172.21.15.82:0/4182238886 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f508c02c7e0 con 0x55b4b2ff2250 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.416+0000 7f507b7fe640 1 -- 172.21.15.82:0/4182238886 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c005500 con 0x55b4b2ff2250 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.416+0000 7f509a41b740 1 -- 172.21.15.82:0/4182238886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 msgr2=0x55b4b2ff2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.416+0000 7f509a41b740 1 --2- 172.21.15.82:0/4182238886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 0x55b4b2ff2630 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f508c009900 tx=0x7f508c0095f0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.416+0000 7f509a41b740 1 -- 172.21.15.82:0/4182238886 shutdown_connections 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.416+0000 7f509a41b740 1 --2- 172.21.15.82:0/4182238886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ffdc10 0x55b4b3000020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.416+0000 7f509a41b740 1 --2- 172.21.15.82:0/4182238886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ffd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.416+0000 7f509a41b740 1 --2- 172.21.15.82:0/4182238886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 0x55b4b2ff2630 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.416+0000 7f509a41b740 1 -- 172.21.15.82:0/4182238886 >> 172.21.15.82:0/4182238886 conn(0x55b4b2fecba0 msgr2=0x55b4b2fecf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.416+0000 7f509a41b740 1 -- 172.21.15.82:0/4182238886 shutdown_connections 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.416+0000 7f509a41b740 1 -- 172.21.15.82:0/4182238886 wait complete. 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f509a41b740 1 -- start start 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f509a41b740 1 Processor -- start 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f509a41b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 0x55b4b2ffd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f509a41b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ffa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f509a41b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ffada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f509a41b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b4b3000ae0 con 0x55b4b2ff2250 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f509a41b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b4b2f10500 con 0x55b4b2ffdc10 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f509a41b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b4b3000c60 con 0x55b4b2ff2c00 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f509a41b740 10 client.0 _ll_register_callbacks cb 0x55b4b2edeb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f50954ec640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 0x55b4b2ffd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f50954ec640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 0x55b4b2ffd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41082/0 (socket says 172.21.15.82:41082) 2024-04-21T08:48:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f50954ec640 1 -- 172.21.15.82:0/3503680802 learned_addr learned my addr 172.21.15.82:0/3503680802 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f5095ced640 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ffada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f5094ceb640 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ffa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.417+0000 7f50954ec640 1 -- 172.21.15.82:0/3503680802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 msgr2=0x55b4b2ffa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f50954ec640 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ffa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f50954ec640 1 -- 172.21.15.82:0/3503680802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ffdc10 msgr2=0x55b4b2ffada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f50954ec640 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ffada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f50954ec640 1 -- 172.21.15.82:0/3503680802 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4b2c693f0 con 0x55b4b2ff2250 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f5095ced640 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ffada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f5094ceb640 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ffa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f50954ec640 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 0x55b4b2ffd460 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f508c009430 tx=0x7f508c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836345f8c782b5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f5078ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f5078ff9640 1 -- 172.21.15.82:0/3503680802 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c045070 con 0x55b4b2ff2250 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f5078ff9640 1 -- 172.21.15.82:0/3503680802 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f508c004ad0 con 0x55b4b2ff2250 2024-04-21T08:48:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f509a41b740 1 -- 172.21.15.82:0/3503680802 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f508c02c7e0 con 0x55b4b2ff2250 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f5078ff9640 1 -- 172.21.15.82:0/3503680802 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f508c05a050 con 0x55b4b2ff2250 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f509a41b740 10 client.16296 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f509a41b740 1 -- 172.21.15.82:0/3503680802 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4b2d337d0 con 0x55b4b2ff2250 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f5078ff9640 1 -- 172.21.15.82:0/3503680802 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f508c02c7e0 con 0x55b4b2ff2250 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f5078ff9640 1 -- 172.21.15.82:0/3503680802 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f508c041c30 con 0x55b4b2ff2250 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f509a41b740 10 client.16296 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f509a41b740 10 client.16296 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f509a41b740 20 client.16296 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f509a41b740 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55b4b300dbf0 0x55b4b300ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f509a41b740 4 client.16296 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.418+0000 7f509a41b740 1 -- 172.21.15.82:0/3503680802 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4b2ef6930 con 0x55b4b300dbf0 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.419+0000 7f5094ceb640 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55b4b300dbf0 0x55b4b300ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.419+0000 7f5094ceb640 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55b4b300dbf0 0x55b4b300ffe0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b80b2c4d6ae940e0 server_cookie=41d5002386a57ee3 in_seq=0 out_seq=0 2024-04-21T08:48:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.419+0000 7f5078ff9640 10 client.16296 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:48:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f5078ff9640 1 -- 172.21.15.82:0/3503680802 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4b2ef6930 con 0x55b4b300dbf0 2024-04-21T08:48:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f5078ff9640 10 client.16296 handle_command_reply: tid=0 2024-04-21T08:48:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f509a41b740 1 client.16296 shutdown 2024-04-21T08:48:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f509a41b740 1 -- 172.21.15.82:0/3503680802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 msgr2=0x55b4b2ffd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f509a41b740 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 0x55b4b2ffd460 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f508c009430 tx=0x7f508c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f509a41b740 1 -- 172.21.15.82:0/3503680802 shutdown_connections 2024-04-21T08:48:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f509a41b740 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55b4b300dbf0 0x55b4b300ffe0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f509a41b740 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ffada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f5078ff9640 0 client.16296 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:48:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f509a41b740 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ffa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f509a41b740 1 --2- 172.21.15.82:0/3503680802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ff2250 0x55b4b2ffd460 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f509a41b740 1 -- 172.21.15.82:0/3503680802 >> 172.21.15.82:0/3503680802 conn(0x55b4b2fecba0 msgr2=0x55b4b2fee8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f509a41b740 1 -- 172.21.15.82:0/3503680802 shutdown_connections 2024-04-21T08:48:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f509a41b740 1 -- 172.21.15.82:0/3503680802 wait complete. 2024-04-21T08:48:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.420+0000 7f509a41b740 20 client.16296 trim_cache size 0 max 16384 2024-04-21T08:48:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.433+0000 7f509a41b740 1 Processor -- start 2024-04-21T08:48:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.433+0000 7f509a41b740 1 -- start start 2024-04-21T08:48:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.433+0000 7f509a41b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ff2250 0x55b4b30099e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.433+0000 7f509a41b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2feba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f509a41b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ffcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f509a41b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f50901ad0e0 con 0x55b4b2ffdc10 2024-04-21T08:48:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f509a41b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f50901aec50 con 0x55b4b2ff2250 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f509a41b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f50901ad7b0 con 0x55b4b2ff2c00 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f5095ced640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ffcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f50954ec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ff2250 0x55b4b30099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f5095ced640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ffcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41084/0 (socket says 172.21.15.82:41084) 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f5095ced640 1 -- 172.21.15.82:0/1267142034 learned_addr learned my addr 172.21.15.82:0/1267142034 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f5094ceb640 1 --2- 172.21.15.82:0/1267142034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2feba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f5095ced640 1 -- 172.21.15.82:0/1267142034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 msgr2=0x55b4b2feba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f5095ced640 1 --2- 172.21.15.82:0/1267142034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2feba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f5095ced640 1 -- 172.21.15.82:0/1267142034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ff2250 msgr2=0x55b4b30099e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f5095ced640 1 --2- 172.21.15.82:0/1267142034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ff2250 0x55b4b30099e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f5095ced640 1 -- 172.21.15.82:0/1267142034 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4b3000c50 con 0x55b4b2ffdc10 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f50954ec640 1 --2- 172.21.15.82:0/1267142034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ff2250 0x55b4b30099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f5095ced640 1 --2- 172.21.15.82:0/1267142034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ffcf90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f50880996c0 tx=0x7f50880bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3572a5781dd0f6cf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f507affd640 1 -- 172.21.15.82:0/1267142034 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50880b5070 con 0x55b4b2ffdc10 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f507affd640 1 -- 172.21.15.82:0/1267142034 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50880a10f0 con 0x55b4b2ffdc10 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.434+0000 7f507affd640 1 -- 172.21.15.82:0/1267142034 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508809d200 con 0x55b4b2ffdc10 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.435+0000 7f509a41b740 1 -- 172.21.15.82:0/1267142034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 msgr2=0x55b4b2ffcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.435+0000 7f509a41b740 1 --2- 172.21.15.82:0/1267142034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ffcf90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f50880996c0 tx=0x7f50880bd040 comp rx=0 tx=0).stop 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.435+0000 7f509a41b740 1 -- 172.21.15.82:0/1267142034 shutdown_connections 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.435+0000 7f509a41b740 1 --2- 172.21.15.82:0/1267142034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ffcf90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.435+0000 7f509a41b740 1 --2- 172.21.15.82:0/1267142034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2c00 0x55b4b2feba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.435+0000 7f509a41b740 1 --2- 172.21.15.82:0/1267142034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ff2250 0x55b4b30099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.435+0000 7f509a41b740 1 -- 172.21.15.82:0/1267142034 >> 172.21.15.82:0/1267142034 conn(0x55b4b2fecba0 msgr2=0x55b4b2feb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.435+0000 7f509a41b740 1 -- 172.21.15.82:0/1267142034 shutdown_connections 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.435+0000 7f509a41b740 1 -- 172.21.15.82:0/1267142034 wait complete. 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.435+0000 7f509a41b740 1 -- start start 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.435+0000 7f509a41b740 1 Processor -- start 2024-04-21T08:48:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f509a41b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2250 0x55b4b30060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f509a41b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ff70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f509a41b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ff75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f509a41b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f50901ae550 con 0x55b4b2ffdc10 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f509a41b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f50901ade80 con 0x55b4b2ff2c00 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f509a41b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f50901ab0c0 con 0x55b4b2ff2250 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f509a41b740 10 client.0 _ll_register_callbacks cb 0x7f508c02ccc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5095ced640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ff75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5095ced640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ff75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41092/0 (socket says 172.21.15.82:41092) 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5095ced640 1 -- 172.21.15.82:0/620548137 learned_addr learned my addr 172.21.15.82:0/620548137 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5094ceb640 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ff70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f50954ec640 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2250 0x55b4b30060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5095ced640 1 -- 172.21.15.82:0/620548137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2250 msgr2=0x55b4b30060e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5095ced640 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2250 0x55b4b30060e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5095ced640 1 -- 172.21.15.82:0/620548137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ff2c00 msgr2=0x55b4b2ff70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5095ced640 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ff70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5095ced640 1 -- 172.21.15.82:0/620548137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4b2ff7b60 con 0x55b4b2ffdc10 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5095ced640 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ff75f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f50880ce630 tx=0x7f508800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f171982cebbca69 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5078ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5078ff9640 1 -- 172.21.15.82:0/620548137 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088080060 con 0x55b4b2ffdc10 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5078ff9640 1 -- 172.21.15.82:0/620548137 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f50880a5730 con 0x55b4b2ffdc10 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f509a41b740 1 -- 172.21.15.82:0/620548137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50880a10f0 con 0x55b4b2ffdc10 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f5078ff9640 1 -- 172.21.15.82:0/620548137 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f50880bb070 con 0x55b4b2ffdc10 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f509a41b740 10 client.16302 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.436+0000 7f509a41b740 1 -- 172.21.15.82:0/620548137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4b3073bd0 con 0x55b4b2ffdc10 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f5078ff9640 1 -- 172.21.15.82:0/620548137 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50880a10f0 con 0x55b4b2ffdc10 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f5078ff9640 1 -- 172.21.15.82:0/620548137 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f508809c190 con 0x55b4b2ffdc10 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f509a41b740 10 client.16302 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f509a41b740 10 client.16302 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f509a41b740 20 client.16302 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f509a41b740 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55b4b3074290 0x55b4b30766b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f509a41b740 4 client.16302 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f509a41b740 1 -- 172.21.15.82:0/620548137 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4b2f64c10 con 0x55b4b3074290 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f50954ec640 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55b4b3074290 0x55b4b30766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f5078ff9640 10 client.16302 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f50954ec640 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55b4b3074290 0x55b4b30766b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=268a601b732e8548 server_cookie=11851b8a2d91e783 in_seq=0 out_seq=0 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f5078ff9640 1 -- 172.21.15.82:0/620548137 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4b2f64c10 con 0x55b4b3074290 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f5078ff9640 10 client.16302 handle_command_reply: tid=0 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.437+0000 7f509a41b740 1 client.16302 shutdown 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f509a41b740 1 -- 172.21.15.82:0/620548137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 msgr2=0x55b4b2ff75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f509a41b740 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ff75f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f50880ce630 tx=0x7f508800efd0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f509a41b740 1 -- 172.21.15.82:0/620548137 shutdown_connections 2024-04-21T08:48:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f509a41b740 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55b4b3074290 0x55b4b30766b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f509a41b740 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b4b2ffdc10 0x55b4b2ff75f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f509a41b740 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b4b2ff2c00 0x55b4b2ff70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f509a41b740 1 --2- 172.21.15.82:0/620548137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b4b2ff2250 0x55b4b30060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5078ff9640 0 client.16302 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:48:12.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f509a41b740 1 -- 172.21.15.82:0/620548137 >> 172.21.15.82:0/620548137 conn(0x55b4b2fecba0 msgr2=0x55b4b2febc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f509a41b740 1 -- 172.21.15.82:0/620548137 shutdown_connections 2024-04-21T08:48:12.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f509a41b740 1 -- 172.21.15.82:0/620548137 wait complete. 2024-04-21T08:48:12.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f509a41b740 20 client.16302 trim_cache size 0 max 16384 2024-04-21T08:48:12.445 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:48:12.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5097750640 1 -- 172.21.15.82:0/1200129341 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f507c0a3ed0 msgr2=0x7f507c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5097750640 1 --2- 172.21.15.82:0/1200129341 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f507c0a3ed0 0x7f507c0a6390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5090100a70 tx=0x7f5080011440 comp rx=0 tx=0).stop 2024-04-21T08:48:12.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5097750640 1 -- 172.21.15.82:0/1200129341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509019bf40 msgr2=0x7f509019c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5097750640 1 --2- 172.21.15.82:0/1200129341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509019bf40 0x7f509019c3f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f508800eb10 tx=0x7f508800c2d0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5097750640 1 -- 172.21.15.82:0/1200129341 shutdown_connections 2024-04-21T08:48:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5097750640 1 --2- 172.21.15.82:0/1200129341 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f507c0a3ed0 0x7f507c0a6390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5097750640 1 --2- 172.21.15.82:0/1200129341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509019bf40 0x7f509019c3f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5097750640 1 --2- 172.21.15.82:0/1200129341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5090106160 0x7f5090197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5097750640 1 --2- 172.21.15.82:0/1200129341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5090101090 0x7f5090197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5097750640 1 -- 172.21.15.82:0/1200129341 >> 172.21.15.82:0/1200129341 conn(0x7f5090077f50 msgr2=0x7f50900ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5097750640 1 -- 172.21.15.82:0/1200129341 shutdown_connections 2024-04-21T08:48:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.438+0000 7f5097750640 1 -- 172.21.15.82:0/1200129341 wait complete. 2024-04-21T08:48:12.449 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:48:12.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.558+0000 7f519134a640 1 -- 172.21.15.82:0/3480273596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c101f50 msgr2=0x7f518c105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.558+0000 7f519134a640 1 --2- 172.21.15.82:0/3480273596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c101f50 0x7f518c105a70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5180009900 tx=0x7f518002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:12.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.558+0000 7f519134a640 1 -- 172.21.15.82:0/3480273596 shutdown_connections 2024-04-21T08:48:12.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.558+0000 7f519134a640 1 --2- 172.21.15.82:0/3480273596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f518c106150 0x7f518c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.558+0000 7f519134a640 1 --2- 172.21.15.82:0/3480273596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c101f50 0x7f518c105a70 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.558+0000 7f519134a640 1 --2- 172.21.15.82:0/3480273596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f518c1015a0 0x7f518c101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.558+0000 7f519134a640 1 -- 172.21.15.82:0/3480273596 >> 172.21.15.82:0/3480273596 conn(0x7f518c0769d0 msgr2=0x7f518c076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.558+0000 7f519134a640 1 -- 172.21.15.82:0/3480273596 shutdown_connections 2024-04-21T08:48:12.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.558+0000 7f519134a640 1 -- 172.21.15.82:0/3480273596 wait complete. 2024-04-21T08:48:12.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.559+0000 7f519134a640 1 Processor -- start 2024-04-21T08:48:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.559+0000 7f519134a640 1 -- start start 2024-04-21T08:48:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f519134a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f518c1015a0 0x7f518c102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f519134a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f518c106150 0x7f518c103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f519134a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c103590 0x7f518c1a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f519134a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f518c111070 con 0x7f518c103590 2024-04-21T08:48:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f519134a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f518c110ef0 con 0x7f518c106150 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f519134a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1111f0 con 0x7f518c1015a0 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f518b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c103590 0x7f518c1a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f518b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c103590 0x7f518c1a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41108/0 (socket says 172.21.15.82:41108) 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f518b7fe640 1 -- 172.21.15.82:0/3395459832 learned_addr learned my addr 172.21.15.82:0/3395459832 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f518a7fc640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f518c106150 0x7f518c103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f518affd640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f518c1015a0 0x7f518c102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f518b7fe640 1 -- 172.21.15.82:0/3395459832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f518c1015a0 msgr2=0x7f518c102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f518b7fe640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f518c1015a0 0x7f518c102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f518b7fe640 1 -- 172.21.15.82:0/3395459832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f518c106150 msgr2=0x7f518c103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f518b7fe640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f518c106150 0x7f518c103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f518b7fe640 1 -- 172.21.15.82:0/3395459832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c1a7010 con 0x7f518c103590 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f518a7fc640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f518c106150 0x7f518c103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f518b7fe640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c103590 0x7f518c1a6910 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f518400eb10 tx=0x7f518400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.560+0000 7f5173fff640 1 -- 172.21.15.82:0/3395459832 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184004b20 con 0x7f518c103590 2024-04-21T08:48:12.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.561+0000 7f519134a640 1 -- 172.21.15.82:0/3395459832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c1a7300 con 0x7f518c103590 2024-04-21T08:48:12.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.561+0000 7f5173fff640 1 -- 172.21.15.82:0/3395459832 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5184004cc0 con 0x7f518c103590 2024-04-21T08:48:12.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.561+0000 7f5173fff640 1 -- 172.21.15.82:0/3395459832 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184010670 con 0x7f518c103590 2024-04-21T08:48:12.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.561+0000 7f519134a640 1 -- 172.21.15.82:0/3395459832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c1a77d0 con 0x7f518c103590 2024-04-21T08:48:12.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f5173fff640 1 -- 172.21.15.82:0/3395459832 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f5184010810 con 0x7f518c103590 2024-04-21T08:48:12.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f519400f740 1 Processor -- start 2024-04-21T08:48:12.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f519400f740 1 -- start start 2024-04-21T08:48:12.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f519400f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba74d0 0x556361ba78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f519400f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361bb0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f519400f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361bb2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f519400f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556361b0c1f0 con 0x556361ba7e80 2024-04-21T08:48:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f519400f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556361aa94b0 con 0x556361bb0a00 2024-04-21T08:48:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f519400f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556361b081b0 con 0x556361ba74d0 2024-04-21T08:48:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f518affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361bb0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f5173fff640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f51740a3f00 0x7f51740a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f518affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361bb0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41122/0 (socket says 172.21.15.82:41122) 2024-04-21T08:48:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f518a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361bb2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.563+0000 7f518affd640 1 -- 172.21.15.82:0/396888571 learned_addr learned my addr 172.21.15.82:0/396888571 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f518a7fc640 1 --2- 172.21.15.82:0/396888571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba74d0 0x556361ba78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f518b7fe640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f51740a3f00 0x7f51740a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f5173fff640 1 -- 172.21.15.82:0/3395459832 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f51840d13b0 con 0x7f518c103590 2024-04-21T08:48:12.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f518a7fc640 1 -- 172.21.15.82:0/396888571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba74d0 msgr2=0x556361ba78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f518a7fc640 1 --2- 172.21.15.82:0/396888571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba74d0 0x556361ba78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f518a7fc640 1 -- 172.21.15.82:0/396888571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 msgr2=0x556361bb0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f518a7fc640 1 --2- 172.21.15.82:0/396888571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361bb0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f518a7fc640 1 -- 172.21.15.82:0/396888571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556361a9b1b0 con 0x556361bb0a00 2024-04-21T08:48:12.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f518affd640 1 --2- 172.21.15.82:0/396888571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361bb0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:12.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f518a7fc640 1 --2- 172.21.15.82:0/396888571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361bb2e50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5180036040 tx=0x7f5180005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a95e839054fe7e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f518b7fe640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f51740a3f00 0x7f51740a63c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f51840ce560 tx=0x7f518406e790 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f51717fa640 1 -- 172.21.15.82:0/396888571 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51800044b0 con 0x556361bb0a00 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f51717fa640 1 -- 172.21.15.82:0/396888571 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f518002cc90 con 0x556361bb0a00 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.564+0000 7f51717fa640 1 -- 172.21.15.82:0/396888571 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180003e30 con 0x556361bb0a00 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.565+0000 7f519400f740 1 -- 172.21.15.82:0/396888571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 msgr2=0x556361bb2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.565+0000 7f519400f740 1 --2- 172.21.15.82:0/396888571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361bb2e50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5180036040 tx=0x7f5180005b50 comp rx=0 tx=0).stop 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.565+0000 7f519400f740 1 -- 172.21.15.82:0/396888571 shutdown_connections 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.565+0000 7f519400f740 1 --2- 172.21.15.82:0/396888571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361bb2e50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.565+0000 7f519400f740 1 --2- 172.21.15.82:0/396888571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361bb0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.565+0000 7f519400f740 1 --2- 172.21.15.82:0/396888571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba74d0 0x556361ba78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.565+0000 7f519400f740 1 -- 172.21.15.82:0/396888571 >> 172.21.15.82:0/396888571 conn(0x556361ba1ba0 msgr2=0x556361ba1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.565+0000 7f519400f740 1 -- 172.21.15.82:0/396888571 shutdown_connections 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.565+0000 7f519400f740 1 -- 172.21.15.82:0/396888571 wait complete. 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.565+0000 7f519400f740 1 -- start start 2024-04-21T08:48:12.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.565+0000 7f519400f740 1 Processor -- start 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f519400f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba74d0 0x556361bb0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f519400f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba7e80 0x556361bad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f519400f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361badd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f519400f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556361bb3910 con 0x556361ba74d0 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f519400f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556361ac5500 con 0x556361bb0a00 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f519400f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556361bb3a90 con 0x556361ba7e80 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f519400f740 10 client.0 _ll_register_callbacks cb 0x556361a93b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f518a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba74d0 0x556361bb0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f518a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba74d0 0x556361bb0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41138/0 (socket says 172.21.15.82:41138) 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f518a7fc640 1 -- 172.21.15.82:0/117619142 learned_addr learned my addr 172.21.15.82:0/117619142 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f518affd640 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba7e80 0x556361bad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f518a7fc640 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361badd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f518affd640 1 -- 172.21.15.82:0/117619142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 msgr2=0x556361badd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f518affd640 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361badd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f518affd640 1 -- 172.21.15.82:0/117619142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba74d0 msgr2=0x556361bb0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f518affd640 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba74d0 0x556361bb0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f518affd640 1 -- 172.21.15.82:0/117619142 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55636181e3f0 con 0x556361ba7e80 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f518a7fc640 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba74d0 0x556361bb0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f518affd640 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba7e80 0x556361bad6f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x556361ba2c10 tx=0x7f517c012b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e80175adcfbb8d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.566+0000 7f516eff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f516eff5640 1 -- 172.21.15.82:0/117619142 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c004050 con 0x556361ba7e80 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f516eff5640 1 -- 172.21.15.82:0/117619142 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f517c0041f0 con 0x556361ba7e80 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f519400f740 1 -- 172.21.15.82:0/117619142 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f518002cc90 con 0x556361ba7e80 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f516eff5640 1 -- 172.21.15.82:0/117619142 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f517c0043f0 con 0x556361ba7e80 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f519400f740 10 client.25483 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f519400f740 1 -- 172.21.15.82:0/117619142 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5563618e87d0 con 0x556361ba7e80 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f516eff5640 1 -- 172.21.15.82:0/117619142 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f518002cc90 con 0x556361ba7e80 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f516eff5640 1 -- 172.21.15.82:0/117619142 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f517c003ad0 con 0x556361ba7e80 2024-04-21T08:48:12.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f519400f740 10 client.25483 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f519400f740 10 client.25483 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f519400f740 20 client.25483 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f519400f740 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x556361bc0c50 0x556361bc3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f519400f740 4 client.25483 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.567+0000 7f519400f740 1 -- 172.21.15.82:0/117619142 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556361aab930 con 0x556361bc0c50 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.568+0000 7f518a7fc640 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x556361bc0c50 0x556361bc3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.568+0000 7f516eff5640 10 client.25483 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.568+0000 7f518a7fc640 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x556361bc0c50 0x556361bc3040 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dafcee653b1e4af5 server_cookie=362adef01879fc3c in_seq=0 out_seq=0 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.568+0000 7f516eff5640 1 -- 172.21.15.82:0/117619142 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556361aab930 con 0x556361bc0c50 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.568+0000 7f516eff5640 10 client.25483 handle_command_reply: tid=0 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f519400f740 1 client.25483 shutdown 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f519400f740 1 -- 172.21.15.82:0/117619142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba7e80 msgr2=0x556361bad6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f519400f740 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba7e80 0x556361bad6f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x556361ba2c10 tx=0x7f517c012b80 comp rx=0 tx=0).stop 2024-04-21T08:48:12.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f519400f740 1 -- 172.21.15.82:0/117619142 shutdown_connections 2024-04-21T08:48:12.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f519400f740 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x556361bc0c50 0x556361bc3040 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f519400f740 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361badd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f516eff5640 0 client.25483 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:48:12.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f519400f740 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba7e80 0x556361bad6f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f519400f740 1 --2- 172.21.15.82:0/117619142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba74d0 0x556361bb0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f519400f740 1 -- 172.21.15.82:0/117619142 >> 172.21.15.82:0/117619142 conn(0x556361ba1ba0 msgr2=0x556361ba38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f519400f740 1 -- 172.21.15.82:0/117619142 shutdown_connections 2024-04-21T08:48:12.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f519400f740 1 -- 172.21.15.82:0/117619142 wait complete. 2024-04-21T08:48:12.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.569+0000 7f519400f740 20 client.25483 trim_cache size 0 max 16384 2024-04-21T08:48:12.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f519400f740 1 Processor -- start 2024-04-21T08:48:12.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f519400f740 1 -- start start 2024-04-21T08:48:12.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f519400f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba74d0 0x556361bbc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f519400f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361ba8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f519400f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361bafe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f519400f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b1630 con 0x556361ba7e80 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f519400f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b31a0 con 0x556361bb0a00 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f519400f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b1d00 con 0x556361ba74d0 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f518affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361ba8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f518affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361ba8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41140/0 (socket says 172.21.15.82:41140) 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f518affd640 1 -- 172.21.15.82:0/281664264 learned_addr learned my addr 172.21.15.82:0/281664264 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f518a7fc640 1 --2- 172.21.15.82:0/281664264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba74d0 0x556361bbc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f518a7fc640 1 --2- 172.21.15.82:0/281664264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361bafe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f518affd640 1 -- 172.21.15.82:0/281664264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba74d0 msgr2=0x556361bbc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f518affd640 1 --2- 172.21.15.82:0/281664264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba74d0 0x556361bbc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f518affd640 1 -- 172.21.15.82:0/281664264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 msgr2=0x556361bafe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f518affd640 1 --2- 172.21.15.82:0/281664264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361bafe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f518affd640 1 -- 172.21.15.82:0/281664264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556361bac6a0 con 0x556361ba7e80 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.583+0000 7f518affd640 1 --2- 172.21.15.82:0/281664264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361ba8c60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f517c000c00 tx=0x7f517c012320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d687440d3eb2ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.584+0000 7f5170ff9640 1 -- 172.21.15.82:0/281664264 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c004000 con 0x556361ba7e80 2024-04-21T08:48:12.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.584+0000 7f5170ff9640 1 -- 172.21.15.82:0/281664264 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517c0041a0 con 0x556361ba7e80 2024-04-21T08:48:12.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.584+0000 7f5170ff9640 1 -- 172.21.15.82:0/281664264 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c005040 con 0x556361ba7e80 2024-04-21T08:48:12.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.584+0000 7f519400f740 1 -- 172.21.15.82:0/281664264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 msgr2=0x556361ba8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.584+0000 7f519400f740 1 --2- 172.21.15.82:0/281664264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361ba8c60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f517c000c00 tx=0x7f517c012320 comp rx=0 tx=0).stop 2024-04-21T08:48:12.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.584+0000 7f519400f740 1 -- 172.21.15.82:0/281664264 shutdown_connections 2024-04-21T08:48:12.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.584+0000 7f519400f740 1 --2- 172.21.15.82:0/281664264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361bb0a00 0x556361bafe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.584+0000 7f519400f740 1 --2- 172.21.15.82:0/281664264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361ba8c60 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.584+0000 7f519400f740 1 --2- 172.21.15.82:0/281664264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361ba74d0 0x556361bbc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.584+0000 7f519400f740 1 -- 172.21.15.82:0/281664264 >> 172.21.15.82:0/281664264 conn(0x556361ba1ba0 msgr2=0x556361ba1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.584+0000 7f519400f740 1 -- 172.21.15.82:0/281664264 shutdown_connections 2024-04-21T08:48:12.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.584+0000 7f519400f740 1 -- 172.21.15.82:0/281664264 wait complete. 2024-04-21T08:48:12.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f519400f740 1 -- start start 2024-04-21T08:48:12.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f519400f740 1 Processor -- start 2024-04-21T08:48:12.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f519400f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361ba74d0 0x556361bb9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f519400f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361baa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f519400f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361bb0a00 0x556361baa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f519400f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b2aa0 con 0x556361ba7e80 2024-04-21T08:48:12.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f519400f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b23d0 con 0x556361ba74d0 2024-04-21T08:48:12.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f519400f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1af610 con 0x556361bb0a00 2024-04-21T08:48:12.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f519400f740 10 client.0 _ll_register_callbacks cb 0x7f517c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:12.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f518affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361baa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f518affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361baa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41144/0 (socket says 172.21.15.82:41144) 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f518a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361ba74d0 0x556361bb9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f518affd640 1 -- 172.21.15.82:0/943467434 learned_addr learned my addr 172.21.15.82:0/943467434 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f518a7fc640 1 --2- 172.21.15.82:0/943467434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361bb0a00 0x556361baa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f518affd640 1 -- 172.21.15.82:0/943467434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361bb0a00 msgr2=0x556361baa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f518affd640 1 --2- 172.21.15.82:0/943467434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361bb0a00 0x556361baa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f518affd640 1 -- 172.21.15.82:0/943467434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361ba74d0 msgr2=0x556361bb9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f518affd640 1 --2- 172.21.15.82:0/943467434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361ba74d0 0x556361bb9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f518affd640 1 -- 172.21.15.82:0/943467434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556361bae070 con 0x556361ba7e80 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f518affd640 1 --2- 172.21.15.82:0/943467434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361baa120 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f517c012bc0 tx=0x7f517c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2235289f174ebbf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.585+0000 7f516eff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f516eff5640 1 -- 172.21.15.82:0/943467434 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c035020 con 0x556361ba7e80 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f516eff5640 1 -- 172.21.15.82:0/943467434 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f517c003c20 con 0x556361ba7e80 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f519400f740 1 -- 172.21.15.82:0/943467434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f517c0041a0 con 0x556361ba7e80 2024-04-21T08:48:12.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f516eff5640 1 -- 172.21.15.82:0/943467434 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f517c00f8f0 con 0x556361ba7e80 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f519400f740 10 client.16320 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f519400f740 1 -- 172.21.15.82:0/943467434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556361c27090 con 0x556361ba7e80 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f516eff5640 1 -- 172.21.15.82:0/943467434 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517c0041a0 con 0x556361ba7e80 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f516eff5640 1 -- 172.21.15.82:0/943467434 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f517c029420 con 0x556361ba7e80 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f519400f740 10 client.16320 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f519400f740 10 client.16320 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f519400f740 20 client.16320 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f519400f740 1 --2- 172.21.15.82:0/943467434 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x556361c276f0 0x556361c29b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f519400f740 4 client.16320 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f519400f740 1 -- 172.21.15.82:0/943467434 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556361b19c10 con 0x556361c276f0 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.586+0000 7f518a7fc640 1 --2- 172.21.15.82:0/943467434 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x556361c276f0 0x556361c29b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f518a7fc640 1 --2- 172.21.15.82:0/943467434 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x556361c276f0 0x556361c29b10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7dcc647df4834fb4 server_cookie=df55c7e7741ab4fe in_seq=0 out_seq=0 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f516eff5640 10 client.16320 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f516eff5640 1 -- 172.21.15.82:0/943467434 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556361b19c10 con 0x556361c276f0 2024-04-21T08:48:12.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f516eff5640 10 client.16320 handle_command_reply: tid=0 2024-04-21T08:48:12.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f519400f740 1 client.16320 shutdown 2024-04-21T08:48:12.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f519400f740 1 -- 172.21.15.82:0/943467434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 msgr2=0x556361baa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f519400f740 1 --2- 172.21.15.82:0/943467434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361baa120 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f517c012bc0 tx=0x7f517c0057b0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f519400f740 1 -- 172.21.15.82:0/943467434 shutdown_connections 2024-04-21T08:48:12.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f519400f740 1 --2- 172.21.15.82:0/943467434 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x556361c276f0 0x556361c29b10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f519400f740 1 --2- 172.21.15.82:0/943467434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556361bb0a00 0x556361baa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f519400f740 1 --2- 172.21.15.82:0/943467434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556361ba7e80 0x556361baa120 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f516eff5640 0 client.16320 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:48:12.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f519400f740 1 --2- 172.21.15.82:0/943467434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556361ba74d0 0x556361bb9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f519400f740 1 -- 172.21.15.82:0/943467434 >> 172.21.15.82:0/943467434 conn(0x556361ba1ba0 msgr2=0x556361ba0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f519400f740 1 -- 172.21.15.82:0/943467434 shutdown_connections 2024-04-21T08:48:12.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f519400f740 1 -- 172.21.15.82:0/943467434 wait complete. 2024-04-21T08:48:12.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.587+0000 7f519400f740 20 client.16320 trim_cache size 0 max 16384 2024-04-21T08:48:12.595 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:48:12.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.588+0000 7f519134a640 1 -- 172.21.15.82:0/3395459832 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f51740a3f00 msgr2=0x7f51740a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.588+0000 7f519134a640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f51740a3f00 0x7f51740a63c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f51840ce560 tx=0x7f518406e790 comp rx=0 tx=0).stop 2024-04-21T08:48:12.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.588+0000 7f519134a640 1 -- 172.21.15.82:0/3395459832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c103590 msgr2=0x7f518c1a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.588+0000 7f519134a640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c103590 0x7f518c1a6910 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f518400eb10 tx=0x7f518400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.588+0000 7f519134a640 1 -- 172.21.15.82:0/3395459832 shutdown_connections 2024-04-21T08:48:12.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.588+0000 7f519134a640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f51740a3f00 0x7f51740a63c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.588+0000 7f519134a640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c103590 0x7f518c1a6910 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.588+0000 7f519134a640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f518c106150 0x7f518c103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.588+0000 7f519134a640 1 --2- 172.21.15.82:0/3395459832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f518c1015a0 0x7f518c102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.588+0000 7f519134a640 1 -- 172.21.15.82:0/3395459832 >> 172.21.15.82:0/3395459832 conn(0x7f518c0769d0 msgr2=0x7f518c06a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.588+0000 7f519134a640 1 -- 172.21.15.82:0/3395459832 shutdown_connections 2024-04-21T08:48:12.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.588+0000 7f519134a640 1 -- 172.21.15.82:0/3395459832 wait complete. 2024-04-21T08:48:12.598 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:48:12.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.708+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/2159794953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0101090 msgr2=0x7fc5a0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.708+0000 7fc5a5ec1640 1 --2- 172.21.15.82:0/2159794953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0101090 0x7fc5a0101470 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc590009900 tx=0x7fc59002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:12.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.708+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/2159794953 shutdown_connections 2024-04-21T08:48:12.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.708+0000 7fc5a5ec1640 1 --2- 172.21.15.82:0/2159794953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5a0106160 0x7fc5a010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.708+0000 7fc5a5ec1640 1 --2- 172.21.15.82:0/2159794953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5a0101a40 0x7fc5a0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.708+0000 7fc5a5ec1640 1 --2- 172.21.15.82:0/2159794953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0101090 0x7fc5a0101470 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.708+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/2159794953 >> 172.21.15.82:0/2159794953 conn(0x7fc5a0077f50 msgr2=0x7fc5a0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.708+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/2159794953 shutdown_connections 2024-04-21T08:48:12.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.708+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/2159794953 wait complete. 2024-04-21T08:48:12.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.708+0000 7fc5a5ec1640 1 Processor -- start 2024-04-21T08:48:12.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc5a5ec1640 1 -- start start 2024-04-21T08:48:12.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc5a5ec1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5a0101090 0x7fc5a019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc5a5ec1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0106160 0x7fc5a019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc5a5ec1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5a0198590 0x7fc5a0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc5a5ec1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01110d0 con 0x7fc5a0106160 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc5a5ec1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a0110f50 con 0x7fc5a0101090 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc5a5ec1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a0111250 con 0x7fc5a0198590 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc59effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0106160 0x7fc5a019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc59effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0106160 0x7fc5a019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54568/0 (socket says 172.21.15.82:54568) 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc59effd640 1 -- 172.21.15.82:0/1772878763 learned_addr learned my addr 172.21.15.82:0/1772878763 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc59f7fe640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5a0101090 0x7fc5a019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc59effd640 1 -- 172.21.15.82:0/1772878763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5a0198590 msgr2=0x7fc5a0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc59effd640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5a0198590 0x7fc5a0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc59effd640 1 -- 172.21.15.82:0/1772878763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5a0101090 msgr2=0x7fc5a019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc59effd640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5a0101090 0x7fc5a019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc59effd640 1 -- 172.21.15.82:0/1772878763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a01990b0 con 0x7fc5a0106160 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc59effd640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0106160 0x7fc5a019e920 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc59400e9c0 tx=0x7fc59400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.709+0000 7fc59cff9640 1 -- 172.21.15.82:0/1772878763 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59400cd80 con 0x7fc5a0106160 2024-04-21T08:48:12.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.710+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/1772878763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a01993a0 con 0x7fc5a0106160 2024-04-21T08:48:12.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.710+0000 7fc59cff9640 1 -- 172.21.15.82:0/1772878763 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5940054d0 con 0x7fc5a0106160 2024-04-21T08:48:12.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.710+0000 7fc59cff9640 1 -- 172.21.15.82:0/1772878763 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594010430 con 0x7fc5a0106160 2024-04-21T08:48:12.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.710+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/1772878763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a01a5640 con 0x7fc5a0106160 2024-04-21T08:48:12.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.712+0000 7fc59cff9640 1 -- 172.21.15.82:0/1772878763 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fc594004fd0 con 0x7fc5a0106160 2024-04-21T08:48:12.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.712+0000 7fc5a8b91740 1 Processor -- start 2024-04-21T08:48:12.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.712+0000 7fc5a8b91740 1 -- start start 2024-04-21T08:48:12.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59cff9640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc5880a3ed0 0x7fc5880a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc5a8b91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 0x55deaf58c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc5a8b91740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf595820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc5a8b91740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf595e40 0x55deaf5962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc5a8b91740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55deaf4f11f0 con 0x55deaf595e40 2024-04-21T08:48:12.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc5a8b91740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55deaf48e4b0 con 0x55deaf58cce0 2024-04-21T08:48:12.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc5a8b91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55deaf4ed1b0 con 0x55deaf58c330 2024-04-21T08:48:12.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59ffff640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc5880a3ed0 0x7fc5880a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf595820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf595e40 0x55deaf5962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59cff9640 1 -- 172.21.15.82:0/1772878763 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc5940d1350 con 0x7fc5a0106160 2024-04-21T08:48:12.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 0x55deaf58c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf595e40 0x55deaf5962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54570/0 (socket says 172.21.15.82:54570) 2024-04-21T08:48:12.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 0x55deaf58c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:48810/0 (socket says 172.21.15.82:48810) 2024-04-21T08:48:12.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59ffff640 1 -- 172.21.15.82:0/2161810848 learned_addr learned my addr 172.21.15.82:0/2161810848 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59effd640 1 -- 172.21.15.82:0/2161810848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 msgr2=0x55deaf58c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59effd640 1 --2- 172.21.15.82:0/2161810848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 0x55deaf58c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59effd640 1 -- 172.21.15.82:0/2161810848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf595e40 msgr2=0x55deaf5962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59effd640 1 --2- 172.21.15.82:0/2161810848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf595e40 0x55deaf5962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59effd640 1 -- 172.21.15.82:0/2161810848 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deaf4801b0 con 0x55deaf58cce0 2024-04-21T08:48:12.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59f7fe640 1 --2- 172.21.15.82:0/2161810848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 0x55deaf58c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:12.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59ffff640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc5880a3ed0 0x7fc5880a6390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc598013b10 tx=0x7fc598009e10 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc59effd640 1 --2- 172.21.15.82:0/2161810848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf595820 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc5940ce4a0 tx=0x7fc59409cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28b78290acdc5c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.713+0000 7fc585ffb640 1 -- 172.21.15.82:0/2161810848 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59407a4f0 con 0x55deaf58cce0 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.714+0000 7fc585ffb640 1 -- 172.21.15.82:0/2161810848 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5940046c0 con 0x55deaf58cce0 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.714+0000 7fc585ffb640 1 -- 172.21.15.82:0/2161810848 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59407a130 con 0x55deaf58cce0 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.714+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2161810848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 msgr2=0x55deaf595820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.714+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2161810848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf595820 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc5940ce4a0 tx=0x7fc59409cb80 comp rx=0 tx=0).stop 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.714+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2161810848 shutdown_connections 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.714+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2161810848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf595e40 0x55deaf5962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.714+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2161810848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf595820 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.714+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2161810848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 0x55deaf58c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.714+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2161810848 >> 172.21.15.82:0/2161810848 conn(0x55deaf586ba0 msgr2=0x55deaf586f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.714+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2161810848 shutdown_connections 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.714+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2161810848 wait complete. 2024-04-21T08:48:12.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.715+0000 7fc5a8b91740 1 -- start start 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.715+0000 7fc5a8b91740 1 Processor -- start 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.715+0000 7fc5a8b91740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 0x55deaf59c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.715+0000 7fc5a8b91740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf5995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.715+0000 7fc5a8b91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf595e40 0x55deaf599b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.715+0000 7fc5a8b91740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55deaf59cd60 con 0x55deaf58c330 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.715+0000 7fc5a8b91740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55deaf4aa500 con 0x55deaf58cce0 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.715+0000 7fc5a8b91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55deaf59cee0 con 0x55deaf595e40 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.715+0000 7fc5a8b91740 10 client.0 _ll_register_callbacks cb 0x55deaf478b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.715+0000 7fc59f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 0x55deaf59c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc59f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 0x55deaf59c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54586/0 (socket says 172.21.15.82:54586) 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc59f7fe640 1 -- 172.21.15.82:0/3804707054 learned_addr learned my addr 172.21.15.82:0/3804707054 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc59effd640 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf5995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc59ffff640 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf595e40 0x55deaf599b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc59f7fe640 1 -- 172.21.15.82:0/3804707054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf595e40 msgr2=0x55deaf599b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc59f7fe640 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf595e40 0x55deaf599b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc59f7fe640 1 -- 172.21.15.82:0/3804707054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 msgr2=0x55deaf5995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc59f7fe640 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf5995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc59f7fe640 1 -- 172.21.15.82:0/3804707054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deaf2033f0 con 0x55deaf58c330 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc59ffff640 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf595e40 0x55deaf599b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc59f7fe640 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 0x55deaf59c260 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc590009900 tx=0x7fc59002c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e205c1be47d1f4e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc57effd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc57effd640 1 -- 172.21.15.82:0/3804707054 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59002c610 con 0x55deaf58c330 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc57effd640 1 -- 172.21.15.82:0/3804707054 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc590005030 con 0x55deaf58c330 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc5a8b91740 1 -- 172.21.15.82:0/3804707054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55deaf490930 con 0x55deaf58c330 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc57effd640 1 -- 172.21.15.82:0/3804707054 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc590005230 con 0x55deaf58c330 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc5a8b91740 10 client.16332 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.716+0000 7fc5a8b91740 1 -- 172.21.15.82:0/3804707054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55deaf2cd7d0 con 0x55deaf58c330 2024-04-21T08:48:12.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.717+0000 7fc57effd640 1 -- 172.21.15.82:0/3804707054 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55deaf490930 con 0x55deaf58c330 2024-04-21T08:48:12.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.717+0000 7fc57effd640 1 -- 172.21.15.82:0/3804707054 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fc590003680 con 0x55deaf58c330 2024-04-21T08:48:12.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.717+0000 7fc5a8b91740 10 client.16332 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:12.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.717+0000 7fc5a8b91740 10 client.16332 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:48:12.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.717+0000 7fc5a8b91740 20 client.16332 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:12.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.717+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55deaf5aa070 0x55deaf5ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.717+0000 7fc5a8b91740 4 client.16332 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:12.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.717+0000 7fc5a8b91740 1 -- 172.21.15.82:0/3804707054 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deaf4fec10 con 0x55deaf5aa070 2024-04-21T08:48:12.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.717+0000 7fc59effd640 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55deaf5aa070 0x55deaf5ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.717+0000 7fc57effd640 10 client.16332 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:48:12.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.717+0000 7fc59effd640 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55deaf5aa070 0x55deaf5ac4a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56e13b6b5e848f4a server_cookie=182d164e0f72b6ff in_seq=0 out_seq=0 2024-04-21T08:48:12.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.719+0000 7fc57effd640 1 -- 172.21.15.82:0/3804707054 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55deaf4fec10 con 0x55deaf5aa070 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.719+0000 7fc57effd640 10 client.16332 handle_command_reply: tid=0 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.719+0000 7fc5a8b91740 1 client.16332 shutdown 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.720+0000 7fc5a8b91740 1 -- 172.21.15.82:0/3804707054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 msgr2=0x55deaf59c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.720+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 0x55deaf59c260 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc590009900 tx=0x7fc59002c9f0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.720+0000 7fc5a8b91740 1 -- 172.21.15.82:0/3804707054 shutdown_connections 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.720+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55deaf5aa070 0x55deaf5ac4a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.720+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf595e40 0x55deaf599b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.720+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf5995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.720+0000 7fc57effd640 0 client.16332 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.720+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/3804707054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 0x55deaf59c260 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.720+0000 7fc5a8b91740 1 -- 172.21.15.82:0/3804707054 >> 172.21.15.82:0/3804707054 conn(0x55deaf586ba0 msgr2=0x55deaf5888f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.720+0000 7fc5a8b91740 1 -- 172.21.15.82:0/3804707054 shutdown_connections 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.720+0000 7fc5a8b91740 1 -- 172.21.15.82:0/3804707054 wait complete. 2024-04-21T08:48:12.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.720+0000 7fc5a8b91740 20 client.16332 trim_cache size 0 max 16384 2024-04-21T08:48:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.748+0000 7fc5a8b91740 1 Processor -- start 2024-04-21T08:48:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.748+0000 7fc5a8b91740 1 -- start start 2024-04-21T08:48:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.748+0000 7fc5a8b91740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 0x55deaf597230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.748+0000 7fc5a8b91740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf5a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.748+0000 7fc5a8b91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf595e40 0x55deaf594800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.748+0000 7fc5a8b91740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01ad2b0 con 0x55deaf58c330 2024-04-21T08:48:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.748+0000 7fc5a8b91740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01aee20 con 0x55deaf58cce0 2024-04-21T08:48:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.748+0000 7fc5a8b91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01ad980 con 0x55deaf595e40 2024-04-21T08:48:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.749+0000 7fc59f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 0x55deaf597230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.749+0000 7fc59f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 0x55deaf597230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54600/0 (socket says 172.21.15.82:54600) 2024-04-21T08:48:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.749+0000 7fc59f7fe640 1 -- 172.21.15.82:0/2327286311 learned_addr learned my addr 172.21.15.82:0/2327286311 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.749+0000 7fc59effd640 1 --2- 172.21.15.82:0/2327286311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf5a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.749+0000 7fc59ffff640 1 --2- 172.21.15.82:0/2327286311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf595e40 0x55deaf594800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.749+0000 7fc59f7fe640 1 -- 172.21.15.82:0/2327286311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf595e40 msgr2=0x55deaf594800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.749+0000 7fc59f7fe640 1 --2- 172.21.15.82:0/2327286311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf595e40 0x55deaf594800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.749+0000 7fc59f7fe640 1 -- 172.21.15.82:0/2327286311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 msgr2=0x55deaf5a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.749+0000 7fc59f7fe640 1 --2- 172.21.15.82:0/2327286311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf5a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.749+0000 7fc59f7fe640 1 -- 172.21.15.82:0/2327286311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deaf599ee0 con 0x55deaf58c330 2024-04-21T08:48:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.749+0000 7fc59ffff640 1 --2- 172.21.15.82:0/2327286311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf595e40 0x55deaf594800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.749+0000 7fc59f7fe640 1 --2- 172.21.15.82:0/2327286311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 0x55deaf597230 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc590003b20 tx=0x7fc59002cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999238b2374a242b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.750+0000 7fc5857fa640 1 -- 172.21.15.82:0/2327286311 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59005a070 con 0x55deaf58c330 2024-04-21T08:48:12.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.750+0000 7fc5857fa640 1 -- 172.21.15.82:0/2327286311 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5900053d0 con 0x55deaf58c330 2024-04-21T08:48:12.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.750+0000 7fc5857fa640 1 -- 172.21.15.82:0/2327286311 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590040bb0 con 0x55deaf58c330 2024-04-21T08:48:12.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.752+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2327286311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 msgr2=0x55deaf597230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.752+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2327286311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 0x55deaf597230 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc590003b20 tx=0x7fc59002cd70 comp rx=0 tx=0).stop 2024-04-21T08:48:12.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.752+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2327286311 shutdown_connections 2024-04-21T08:48:12.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.752+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2327286311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf595e40 0x55deaf594800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.752+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2327286311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf58cce0 0x55deaf5a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.752+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2327286311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58c330 0x55deaf597230 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.752+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2327286311 >> 172.21.15.82:0/2327286311 conn(0x55deaf586ba0 msgr2=0x55deaf595380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.753+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2327286311 shutdown_connections 2024-04-21T08:48:12.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.753+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2327286311 wait complete. 2024-04-21T08:48:12.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.753+0000 7fc5a8b91740 1 -- start start 2024-04-21T08:48:12.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.755+0000 7fc5a8b91740 1 Processor -- start 2024-04-21T08:48:12.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.755+0000 7fc5a8b91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 0x55deaf5a2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.755+0000 7fc5a8b91740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58cce0 0x55deaf59ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.755+0000 7fc5a8b91740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf595e40 0x55deaf59b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.756+0000 7fc59f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 0x55deaf5a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.756+0000 7fc59f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 0x55deaf5a2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:48842/0 (socket says 172.21.15.82:48842) 2024-04-21T08:48:12.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.756+0000 7fc59f7fe640 1 -- 172.21.15.82:0/2903472328 learned_addr learned my addr 172.21.15.82:0/2903472328 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.756+0000 7fc59ffff640 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf595e40 0x55deaf59b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.756+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2903472328 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01ae720 con 0x55deaf58cce0 2024-04-21T08:48:12.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.756+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2903472328 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01ae050 con 0x55deaf595e40 2024-04-21T08:48:12.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.756+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2903472328 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01ab290 con 0x55deaf58c330 2024-04-21T08:48:12.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.756+0000 7fc59effd640 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58cce0 0x55deaf59ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.756+0000 7fc5a8b91740 10 client.0 _ll_register_callbacks cb 0x7fc590004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:12.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.757+0000 7fc59f7fe640 1 -- 172.21.15.82:0/2903472328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf595e40 msgr2=0x55deaf59b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.757+0000 7fc59f7fe640 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf595e40 0x55deaf59b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.757+0000 7fc59f7fe640 1 -- 172.21.15.82:0/2903472328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58cce0 msgr2=0x55deaf59ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.757+0000 7fc59f7fe640 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58cce0 0x55deaf59ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.757+0000 7fc59f7fe640 1 -- 172.21.15.82:0/2903472328 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deaf59b790 con 0x55deaf58c330 2024-04-21T08:48:12.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.757+0000 7fc59f7fe640 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 0x55deaf5a2360 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc590058040 tx=0x7fc59002c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c6b7feeb98aa6e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.757+0000 7fc57effd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:48:12.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.758+0000 7fc57effd640 1 -- 172.21.15.82:0/2903472328 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59005a070 con 0x55deaf58c330 2024-04-21T08:48:12.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.758+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2903472328 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5900053d0 con 0x55deaf58c330 2024-04-21T08:48:12.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.758+0000 7fc57effd640 1 -- 172.21.15.82:0/2903472328 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc5900048c0 con 0x55deaf58c330 2024-04-21T08:48:12.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.758+0000 7fc57effd640 1 -- 172.21.15.82:0/2903472328 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc590005850 con 0x55deaf58c330 2024-04-21T08:48:12.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.758+0000 7fc5a8b91740 10 client.25501 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:12.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.758+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2903472328 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55deaf591180 con 0x55deaf58c330 2024-04-21T08:48:12.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.758+0000 7fc57effd640 1 -- 172.21.15.82:0/2903472328 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5900053d0 con 0x55deaf58c330 2024-04-21T08:48:12.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.758+0000 7fc57effd640 1 -- 172.21.15.82:0/2903472328 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fc590040b10 con 0x55deaf58c330 2024-04-21T08:48:12.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.759+0000 7fc5a8b91740 10 client.25501 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:12.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.759+0000 7fc5a8b91740 10 client.25501 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:48:12.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.759+0000 7fc5a8b91740 20 client.25501 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:12.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.759+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55deaf614dd0 0x55deaf6171c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.759+0000 7fc5a8b91740 4 client.25501 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:12.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.759+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2903472328 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55deaf4f8a80 con 0x55deaf614dd0 2024-04-21T08:48:12.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.759+0000 7fc59effd640 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55deaf614dd0 0x55deaf6171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.761+0000 7fc57effd640 10 client.25501 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:48:12.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.761+0000 7fc59effd640 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55deaf614dd0 0x55deaf6171c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e2898997dd30975 server_cookie=3d96a85f65b64c94 in_seq=0 out_seq=0 2024-04-21T08:48:12.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.765+0000 7fc57effd640 1 -- 172.21.15.82:0/2903472328 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55deaf4f8a80 con 0x55deaf614dd0 2024-04-21T08:48:12.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.765+0000 7fc57effd640 10 client.25501 handle_command_reply: tid=0 2024-04-21T08:48:12.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.765+0000 7fc5a8b91740 1 client.25501 shutdown 2024-04-21T08:48:12.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.766+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2903472328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 msgr2=0x55deaf5a2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.766+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 0x55deaf5a2360 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc590058040 tx=0x7fc59002c9f0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.766+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2903472328 shutdown_connections 2024-04-21T08:48:12.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.766+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55deaf614dd0 0x55deaf6171c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.766+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55deaf595e40 0x55deaf59b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.766+0000 7fc57effd640 0 client.25501 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:48:12.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.766+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55deaf58cce0 0x55deaf59ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.766+0000 7fc5a8b91740 1 --2- 172.21.15.82:0/2903472328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55deaf58c330 0x55deaf5a2360 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.766+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2903472328 >> 172.21.15.82:0/2903472328 conn(0x55deaf586ba0 msgr2=0x55deaf585370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.766+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2903472328 shutdown_connections 2024-04-21T08:48:12.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.766+0000 7fc5a8b91740 1 -- 172.21.15.82:0/2903472328 wait complete. 2024-04-21T08:48:12.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.766+0000 7fc5a8b91740 20 client.25501 trim_cache size 0 max 16384 2024-04-21T08:48:12.767 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:48:12.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.767+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/1772878763 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc5880a3ed0 msgr2=0x7fc5880a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.767+0000 7fc5a5ec1640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc5880a3ed0 0x7fc5880a6390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc598013b10 tx=0x7fc598009e10 comp rx=0 tx=0).stop 2024-04-21T08:48:12.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.767+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/1772878763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0106160 msgr2=0x7fc5a019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.767+0000 7fc5a5ec1640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0106160 0x7fc5a019e920 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc59400e9c0 tx=0x7fc59400ee80 comp rx=0 tx=0).stop 2024-04-21T08:48:12.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.768+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/1772878763 shutdown_connections 2024-04-21T08:48:12.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.768+0000 7fc5a5ec1640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5a0198590 0x7fc5a0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.768+0000 7fc5a5ec1640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc5880a3ed0 0x7fc5880a6390 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.768+0000 7fc5a5ec1640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a0106160 0x7fc5a019e920 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.768+0000 7fc5a5ec1640 1 --2- 172.21.15.82:0/1772878763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5a0101090 0x7fc5a019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.768+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/1772878763 >> 172.21.15.82:0/1772878763 conn(0x7fc5a0077f50 msgr2=0x7fc5a0069540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.768+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/1772878763 shutdown_connections 2024-04-21T08:48:12.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.768+0000 7fc5a5ec1640 1 -- 172.21.15.82:0/1772878763 wait complete. 2024-04-21T08:48:12.791 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:48:12.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.932+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/3931333265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a01021a0 msgr2=0x7fe2a0102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.932+0000 7fe2a6fb2640 1 --2- 172.21.15.82:0/3931333265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a01021a0 0x7fe2a0102580 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe294009900 tx=0x7fe29402bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:12.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.932+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/3931333265 shutdown_connections 2024-04-21T08:48:12.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.932+0000 7fe2a6fb2640 1 --2- 172.21.15.82:0/3931333265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2a0107270 0x7fe2a010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.932+0000 7fe2a6fb2640 1 --2- 172.21.15.82:0/3931333265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2a0102b50 0x7fe2a0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.932+0000 7fe2a6fb2640 1 --2- 172.21.15.82:0/3931333265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a01021a0 0x7fe2a0102580 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.932+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/3931333265 >> 172.21.15.82:0/3931333265 conn(0x7fe2a00fd9c0 msgr2=0x7fe2a00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.932+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/3931333265 shutdown_connections 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.932+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/3931333265 wait complete. 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.932+0000 7fe2a6fb2640 1 Processor -- start 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a6fb2640 1 -- start start 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a6fb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2a01021a0 0x7fe2a019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a6fb2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2a0107270 0x7fe2a019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a6fb2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0198600 0x7fe2a0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a6fb2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a0112220 con 0x7fe2a0198600 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a6fb2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01120a0 con 0x7fe2a01021a0 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a6fb2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01123a0 con 0x7fe2a0107270 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a554f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0198600 0x7fe2a0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a554f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0198600 0x7fe2a0198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54628/0 (socket says 172.21.15.82:54628) 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a554f640 1 -- 172.21.15.82:0/735108700 learned_addr learned my addr 172.21.15.82:0/735108700 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a4d4e640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2a01021a0 0x7fe2a019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe29ffff640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2a0107270 0x7fe2a019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a554f640 1 -- 172.21.15.82:0/735108700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2a0107270 msgr2=0x7fe2a019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a554f640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2a0107270 0x7fe2a019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a554f640 1 -- 172.21.15.82:0/735108700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2a01021a0 msgr2=0x7fe2a019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a554f640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2a01021a0 0x7fe2a019e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a554f640 1 -- 172.21.15.82:0/735108700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a0199120 con 0x7fe2a0198600 2024-04-21T08:48:12.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a554f640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0198600 0x7fe2a0198ab0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe29800eb10 tx=0x7fe29800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe29dffb640 1 -- 172.21.15.82:0/735108700 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298004b20 con 0x7fe2a0198600 2024-04-21T08:48:12.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/735108700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a0199410 con 0x7fe2a0198600 2024-04-21T08:48:12.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe29dffb640 1 -- 172.21.15.82:0/735108700 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe298004cc0 con 0x7fe2a0198600 2024-04-21T08:48:12.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.933+0000 7fe29dffb640 1 -- 172.21.15.82:0/735108700 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298010670 con 0x7fe2a0198600 2024-04-21T08:48:12.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.934+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/735108700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a01a5610 con 0x7fe2a0198600 2024-04-21T08:48:12.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.936+0000 7fe29dffb640 1 -- 172.21.15.82:0/735108700 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe298005020 con 0x7fe2a0198600 2024-04-21T08:48:12.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.936+0000 7fe2a9ba2740 1 Processor -- start 2024-04-21T08:48:12.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.936+0000 7fe29dffb640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe2900a3ed0 0x7fe2900a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.936+0000 7fe2a9ba2740 1 -- start start 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.936+0000 7fe29ffff640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe2900a3ed0 0x7fe2900a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.936+0000 7fe2a9ba2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 0x55c05bdcb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.936+0000 7fe2a9ba2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdcbd20 0x55c05bdd8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.936+0000 7fe2a9ba2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdd8e10 0x55c05bddb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.936+0000 7fe2a9ba2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c05bd301f0 con 0x55c05bdcb370 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.936+0000 7fe2a9ba2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c05bccd4b0 con 0x55c05bdcbd20 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.936+0000 7fe2a9ba2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c05bd2c1b0 con 0x55c05bdd8e10 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe29dffb640 1 -- 172.21.15.82:0/735108700 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe2980d13a0 con 0x7fe2a0198600 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe2a4d4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 0x55c05bdcb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe2a554f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdd8e10 0x55c05bddb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe2a4d4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 0x55c05bdcb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54640/0 (socket says 172.21.15.82:54640) 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe2a4d4e640 1 -- 172.21.15.82:0/2852920967 learned_addr learned my addr 172.21.15.82:0/2852920967 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe29ffff640 1 --2- 172.21.15.82:0/2852920967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdcbd20 0x55c05bdd8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe2a554f640 1 -- 172.21.15.82:0/2852920967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdcbd20 msgr2=0x55c05bdd8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe2a554f640 1 --2- 172.21.15.82:0/2852920967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdcbd20 0x55c05bdd8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe2a554f640 1 -- 172.21.15.82:0/2852920967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 msgr2=0x55c05bdcb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe2a554f640 1 --2- 172.21.15.82:0/2852920967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 0x55c05bdcb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe2a554f640 1 -- 172.21.15.82:0/2852920967 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c05bcbf1b0 con 0x55c05bdd8e10 2024-04-21T08:48:12.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe29ffff640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe2900a3ed0 0x7fe2900a6390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0108850 tx=0x7fe28c020df0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe2a554f640 1 --2- 172.21.15.82:0/2852920967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdd8e10 0x55c05bddb220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2980ce4f0 tx=0x7fe29809cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba4f214d58ed1533 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe28affd640 1 -- 172.21.15.82:0/2852920967 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29809cd40 con 0x55c05bdd8e10 2024-04-21T08:48:12.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe28affd640 1 -- 172.21.15.82:0/2852920967 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2980a10f0 con 0x55c05bdd8e10 2024-04-21T08:48:12.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.937+0000 7fe28affd640 1 -- 172.21.15.82:0/2852920967 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29809cfa0 con 0x55c05bdd8e10 2024-04-21T08:48:12.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.938+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/2852920967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdd8e10 msgr2=0x55c05bddb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.938+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/2852920967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdd8e10 0x55c05bddb220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2980ce4f0 tx=0x7fe29809cbd0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.938+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/2852920967 shutdown_connections 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.938+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/2852920967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdd8e10 0x55c05bddb220 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.938+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/2852920967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdcbd20 0x55c05bdd8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.938+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/2852920967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 0x55c05bdcb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.938+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/2852920967 >> 172.21.15.82:0/2852920967 conn(0x55c05bdc5ba0 msgr2=0x55c05bdc5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.938+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/2852920967 shutdown_connections 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.938+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/2852920967 wait complete. 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.938+0000 7fe2a9ba2740 1 -- start start 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.938+0000 7fe2a9ba2740 1 Processor -- start 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a9ba2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcb370 0x55c05bdd1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a9ba2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdcbd20 0x55c05bdd2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a9ba2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdd8e10 0x55c05bdd26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a9ba2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c05bdcc610 con 0x55c05bdd8e10 2024-04-21T08:48:12.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a9ba2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c05bce9500 con 0x55c05bdcbd20 2024-04-21T08:48:12.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a9ba2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c05bdcc790 con 0x55c05bdcb370 2024-04-21T08:48:12.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a9ba2740 10 client.0 _ll_register_callbacks cb 0x55c05bcb7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:12.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a554f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdd8e10 0x55c05bdd26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a554f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdd8e10 0x55c05bdd26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54652/0 (socket says 172.21.15.82:54652) 2024-04-21T08:48:12.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a554f640 1 -- 172.21.15.82:0/4055579235 learned_addr learned my addr 172.21.15.82:0/4055579235 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe29ffff640 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdcbd20 0x55c05bdd2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a4d4e640 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcb370 0x55c05bdd1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a554f640 1 -- 172.21.15.82:0/4055579235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcb370 msgr2=0x55c05bdd1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a554f640 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcb370 0x55c05bdd1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a554f640 1 -- 172.21.15.82:0/4055579235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdcbd20 msgr2=0x55c05bdd2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a554f640 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdcbd20 0x55c05bdd2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a554f640 1 -- 172.21.15.82:0/4055579235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c05ba423f0 con 0x55c05bdd8e10 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a554f640 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdd8e10 0x55c05bdd26d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe2980a5480 tx=0x7fe298004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae5b81b339e5c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe27ffff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe27ffff640 1 -- 172.21.15.82:0/4055579235 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298056050 con 0x55c05bdd8e10 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe27ffff640 1 -- 172.21.15.82:0/4055579235 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe2980a5520 con 0x55c05bdd8e10 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.939+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/4055579235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2980a10f0 con 0x55c05bdd8e10 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.940+0000 7fe27ffff640 1 -- 172.21.15.82:0/4055579235 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2980a59b0 con 0x55c05bdd8e10 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.940+0000 7fe2a9ba2740 10 client.16350 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.940+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/4055579235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c05bb0c7d0 con 0x55c05bdd8e10 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.940+0000 7fe27ffff640 1 -- 172.21.15.82:0/4055579235 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2980a10f0 con 0x55c05bdd8e10 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.940+0000 7fe27ffff640 1 -- 172.21.15.82:0/4055579235 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe2980bd160 con 0x55c05bdd8e10 2024-04-21T08:48:12.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.940+0000 7fe2a9ba2740 10 client.16350 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.940+0000 7fe2a9ba2740 10 client.16350 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.940+0000 7fe2a9ba2740 20 client.16350 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.940+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c05bde4a20 0x55c05bde6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.940+0000 7fe2a9ba2740 4 client.16350 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.940+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/4055579235 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c05bccf930 con 0x55c05bde4a20 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.940+0000 7fe2a4d4e640 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c05bde4a20 0x55c05bde6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.941+0000 7fe2a4d4e640 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c05bde4a20 0x55c05bde6e10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd44c0d4848fddf8 server_cookie=e4bfb9cabb8210cf in_seq=0 out_seq=0 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.941+0000 7fe27ffff640 10 client.16350 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.941+0000 7fe27ffff640 1 -- 172.21.15.82:0/4055579235 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c05bccf930 con 0x55c05bde4a20 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.941+0000 7fe27ffff640 10 client.16350 handle_command_reply: tid=0 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.941+0000 7fe2a9ba2740 1 client.16350 shutdown 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.942+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/4055579235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdd8e10 msgr2=0x55c05bdd26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.942+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdd8e10 0x55c05bdd26d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe2980a5480 tx=0x7fe298004a40 comp rx=0 tx=0).stop 2024-04-21T08:48:12.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.942+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/4055579235 shutdown_connections 2024-04-21T08:48:12.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.942+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c05bde4a20 0x55c05bde6e10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.942+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdd8e10 0x55c05bdd26d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.942+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdcbd20 0x55c05bdd2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.942+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/4055579235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcb370 0x55c05bdd1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.942+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/4055579235 >> 172.21.15.82:0/4055579235 conn(0x55c05bdc5ba0 msgr2=0x55c05bdc78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.942+0000 7fe27ffff640 0 client.16350 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:48:12.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.942+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/4055579235 shutdown_connections 2024-04-21T08:48:12.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.942+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/4055579235 wait complete. 2024-04-21T08:48:12.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.942+0000 7fe2a9ba2740 20 client.16350 trim_cache size 0 max 16384 2024-04-21T08:48:12.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe2a9ba2740 1 Processor -- start 2024-04-21T08:48:12.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe2a9ba2740 1 -- start start 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe2a9ba2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcb370 0x55c05bdd8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe2a9ba2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcbd20 0x55c05bdcc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe2a9ba2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdd8e10 0x55c05bdec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe2a9ba2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01ad240 con 0x55c05bdcbd20 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe2a9ba2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01aedb0 con 0x55c05bdd8e10 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe2a9ba2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01ad910 con 0x55c05bdcb370 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe29ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcbd20 0x55c05bdcc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe29ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcbd20 0x55c05bdcc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54668/0 (socket says 172.21.15.82:54668) 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe29ffff640 1 -- 172.21.15.82:0/2160453766 learned_addr learned my addr 172.21.15.82:0/2160453766 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe2a554f640 1 --2- 172.21.15.82:0/2160453766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdd8e10 0x55c05bdec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe2a4d4e640 1 --2- 172.21.15.82:0/2160453766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcb370 0x55c05bdd8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe29ffff640 1 -- 172.21.15.82:0/2160453766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcb370 msgr2=0x55c05bdd8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe29ffff640 1 --2- 172.21.15.82:0/2160453766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcb370 0x55c05bdd8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe29ffff640 1 -- 172.21.15.82:0/2160453766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdd8e10 msgr2=0x55c05bdec3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe29ffff640 1 --2- 172.21.15.82:0/2160453766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdd8e10 0x55c05bdec3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.957+0000 7fe29ffff640 1 -- 172.21.15.82:0/2160453766 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c05bdca790 con 0x55c05bdcbd20 2024-04-21T08:48:12.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.958+0000 7fe29ffff640 1 --2- 172.21.15.82:0/2160453766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcbd20 0x55c05bdcc9e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x55c05bdcc380 tx=0x7fe28c027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a24cc3ba34dfa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.958+0000 7fe28a7fc640 1 -- 172.21.15.82:0/2160453766 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c01eaf0 con 0x55c05bdcbd20 2024-04-21T08:48:12.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.958+0000 7fe28a7fc640 1 -- 172.21.15.82:0/2160453766 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe28c0054d0 con 0x55c05bdcbd20 2024-04-21T08:48:12.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.958+0000 7fe28a7fc640 1 -- 172.21.15.82:0/2160453766 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c004d30 con 0x55c05bdcbd20 2024-04-21T08:48:12.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.958+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/2160453766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcbd20 msgr2=0x55c05bdcc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.958+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/2160453766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcbd20 0x55c05bdcc9e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x55c05bdcc380 tx=0x7fe28c027000 comp rx=0 tx=0).stop 2024-04-21T08:48:12.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.958+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/2160453766 shutdown_connections 2024-04-21T08:48:12.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.958+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/2160453766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdd8e10 0x55c05bdec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.958+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/2160453766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcbd20 0x55c05bdcc9e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.958+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/2160453766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcb370 0x55c05bdd8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.958+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/2160453766 >> 172.21.15.82:0/2160453766 conn(0x55c05bdc5ba0 msgr2=0x55c05bdc4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/2160453766 shutdown_connections 2024-04-21T08:48:12.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/2160453766 wait complete. 2024-04-21T08:48:12.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a9ba2740 1 -- start start 2024-04-21T08:48:12.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a9ba2740 1 Processor -- start 2024-04-21T08:48:12.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a9ba2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 0x55c05bddbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a9ba2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcbd20 0x55c05bddc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a9ba2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdd8e10 0x55c05be48c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a9ba2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01ae6b0 con 0x55c05bdcb370 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a9ba2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01adfe0 con 0x55c05bdd8e10 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a9ba2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01ab220 con 0x55c05bdcbd20 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a9ba2740 10 client.0 _ll_register_callbacks cb 0x7fe298098fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a4d4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 0x55c05bddbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a4d4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 0x55c05bddbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54680/0 (socket says 172.21.15.82:54680) 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a4d4e640 1 -- 172.21.15.82:0/837195861 learned_addr learned my addr 172.21.15.82:0/837195861 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe29ffff640 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcbd20 0x55c05bddc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.959+0000 7fe2a554f640 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdd8e10 0x55c05be48c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe2a4d4e640 1 -- 172.21.15.82:0/837195861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcbd20 msgr2=0x55c05bddc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe2a4d4e640 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcbd20 0x55c05bddc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe2a4d4e640 1 -- 172.21.15.82:0/837195861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdd8e10 msgr2=0x55c05be48c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:12.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe2a4d4e640 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdd8e10 0x55c05be48c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe2a4d4e640 1 -- 172.21.15.82:0/837195861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c05be49150 con 0x55c05bdcb370 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe2a554f640 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdd8e10 0x55c05be48c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe2a4d4e640 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 0x55c05bddbf10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe294036040 tx=0x7fe294004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbe7352a21444fd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe27ffff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe27ffff640 1 -- 172.21.15.82:0/837195861 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29402c7e0 con 0x55c05bdcb370 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe27ffff640 1 -- 172.21.15.82:0/837195861 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe29402c980 con 0x55c05bdcb370 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/837195861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe28c0054d0 con 0x55c05bdcb370 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe27ffff640 1 -- 172.21.15.82:0/837195861 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe294059050 con 0x55c05bdcb370 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe2a9ba2740 10 client.16356 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/837195861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c05be4a310 con 0x55c05bdcb370 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.960+0000 7fe27ffff640 1 -- 172.21.15.82:0/837195861 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe28c0054d0 con 0x55c05bdcb370 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe27ffff640 1 -- 172.21.15.82:0/837195861 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe294005280 con 0x55c05bdcb370 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 10 client.16356 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:12.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 10 client.16356 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 20 client.16356 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c05be4aab0 0x55c05be4ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 4 client.16356 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/837195861 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c05bd3dc10 con 0x55c05be4aab0 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe29ffff640 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c05be4aab0 0x55c05be4ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe29ffff640 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c05be4aab0 0x55c05be4ced0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=769e8eb56cdde2f server_cookie=c1debfa88679f78f in_seq=0 out_seq=0 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe27ffff640 10 client.16356 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe27ffff640 1 -- 172.21.15.82:0/837195861 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c05bd3dc10 con 0x55c05be4aab0 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe27ffff640 10 client.16356 handle_command_reply: tid=0 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 1 client.16356 shutdown 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/837195861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 msgr2=0x55c05bddbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 0x55c05bddbf10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe294036040 tx=0x7fe294004500 comp rx=0 tx=0).stop 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/837195861 shutdown_connections 2024-04-21T08:48:12.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c05be4aab0 0x55c05be4ced0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c05bdd8e10 0x55c05be48c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c05bdcbd20 0x55c05bddc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe27ffff640 0 client.16356 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:48:12.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 1 --2- 172.21.15.82:0/837195861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c05bdcb370 0x55c05bddbf10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.961+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/837195861 >> 172.21.15.82:0/837195861 conn(0x55c05bdc5ba0 msgr2=0x55c05bdc4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/837195861 shutdown_connections 2024-04-21T08:48:12.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a9ba2740 1 -- 172.21.15.82:0/837195861 wait complete. 2024-04-21T08:48:12.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a9ba2740 20 client.16356 trim_cache size 0 max 16384 2024-04-21T08:48:12.970 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:48:12.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/735108700 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe2900a3ed0 msgr2=0x7fe2900a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a6fb2640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe2900a3ed0 0x7fe2900a6390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0108850 tx=0x7fe28c020df0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/735108700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0198600 msgr2=0x7fe2a0198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:12.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a6fb2640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0198600 0x7fe2a0198ab0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe29800eb10 tx=0x7fe29800c2d0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/735108700 shutdown_connections 2024-04-21T08:48:12.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a6fb2640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe2900a3ed0 0x7fe2900a6390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a6fb2640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a0198600 0x7fe2a0198ab0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a6fb2640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2a0107270 0x7fe2a019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a6fb2640 1 --2- 172.21.15.82:0/735108700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2a01021a0 0x7fe2a019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:12.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/735108700 >> 172.21.15.82:0/735108700 conn(0x7fe2a00fd9c0 msgr2=0x7fe2a00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:12.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/735108700 shutdown_connections 2024-04-21T08:48:12.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:12.962+0000 7fe2a6fb2640 1 -- 172.21.15.82:0/735108700 wait complete. 2024-04-21T08:48:12.977 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:48:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.132+0000 7f1ee1a81640 1 -- 172.21.15.82:0/1371927791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1edc1031d0 msgr2=0x7f1edc1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.132+0000 7f1ee1a81640 1 --2- 172.21.15.82:0/1371927791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1edc1031d0 0x7f1edc1035b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009900 tx=0x7f1ecc02bd60 comp rx=0 tx=0).stop 2024-04-21T08:48:13.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 -- 172.21.15.82:0/1371927791 shutdown_connections 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 --2- 172.21.15.82:0/1371927791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1edc10c510 0x7f1edc10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 --2- 172.21.15.82:0/1371927791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1edc103af0 0x7f1edc10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 --2- 172.21.15.82:0/1371927791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1edc1031d0 0x7f1edc1035b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 -- 172.21.15.82:0/1371927791 >> 172.21.15.82:0/1371927791 conn(0x7f1edc077f50 msgr2=0x7f1edc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 -- 172.21.15.82:0/1371927791 shutdown_connections 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 -- 172.21.15.82:0/1371927791 wait complete. 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 Processor -- start 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 -- start start 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1edc1031d0 0x7f1edc198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1edc103af0 0x7f1edc198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1edc10c510 0x7f1edc19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc1121a0 con 0x7f1edc1031d0 2024-04-21T08:48:13.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc112020 con 0x7f1edc10c510 2024-04-21T08:48:13.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.133+0000 7f1ee1a81640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc112320 con 0x7f1edc103af0 2024-04-21T08:48:13.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1edb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1edc1031d0 0x7f1edc198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1edb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1edc1031d0 0x7f1edc198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54696/0 (socket says 172.21.15.82:54696) 2024-04-21T08:48:13.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1edb7fe640 1 -- 172.21.15.82:0/2026035831 learned_addr learned my addr 172.21.15.82:0/2026035831 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:13.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1edbfff640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1edc10c510 0x7f1edc19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1edaffd640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1edc103af0 0x7f1edc198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1edb7fe640 1 -- 172.21.15.82:0/2026035831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1edc103af0 msgr2=0x7f1edc198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:13.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1edc103af0 0x7f1edc198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1edb7fe640 1 -- 172.21.15.82:0/2026035831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1edc10c510 msgr2=0x7f1edc19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:13.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1edc10c510 0x7f1edc19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1edb7fe640 1 -- 172.21.15.82:0/2026035831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1edc19d730 con 0x7f1edc1031d0 2024-04-21T08:48:13.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1edc1031d0 0x7f1edc198740 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc005940 tx=0x7f1ecc02c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1ed8ff9640 1 -- 172.21.15.82:0/2026035831 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc043020 con 0x7f1edc1031d0 2024-04-21T08:48:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1ed8ff9640 1 -- 172.21.15.82:0/2026035831 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ecc02dbb0 con 0x7f1edc1031d0 2024-04-21T08:48:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1ee1a81640 1 -- 172.21.15.82:0/2026035831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1edc19d9c0 con 0x7f1edc1031d0 2024-04-21T08:48:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1ed8ff9640 1 -- 172.21.15.82:0/2026035831 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc003a10 con 0x7f1edc1031d0 2024-04-21T08:48:13.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.134+0000 7f1ee1a81640 1 -- 172.21.15.82:0/2026035831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1edc1a52b0 con 0x7f1edc1031d0 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.136+0000 7f1ed8ff9640 1 -- 172.21.15.82:0/2026035831 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f1ecc046020 con 0x7f1edc1031d0 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.137+0000 7f1ed8ff9640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1ec40a3ed0 0x7f1ec40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.137+0000 7f1edaffd640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1ec40a3ed0 0x7f1ec40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.137+0000 7f1ed8ff9640 1 -- 172.21.15.82:0/2026035831 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1ecc0f2d50 con 0x7f1edc1031d0 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.137+0000 7f1edaffd640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1ec40a3ed0 0x7f1ec40a6390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1edc10e450 tx=0x7f1ed0005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1ee4678740 1 Processor -- start 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1ee4678740 1 -- start start 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1ee4678740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b0b4d0 0x5649e5b0b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1ee4678740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 0x5649e5b14430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1ee4678740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b14a00 0x5649e5b16e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1ee4678740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5649e5a701f0 con 0x5649e5b0be80 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1ee4678740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5649e5a0d4b0 con 0x5649e5b0b4d0 2024-04-21T08:48:13.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1ee4678740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5649e5a6c1b0 con 0x5649e5b14a00 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1edaffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 0x5649e5b14430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1edaffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 0x5649e5b14430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54708/0 (socket says 172.21.15.82:54708) 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1edaffd640 1 -- 172.21.15.82:0/566614133 learned_addr learned my addr 172.21.15.82:0/566614133 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/566614133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b0b4d0 0x5649e5b0b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1edbfff640 1 --2- 172.21.15.82:0/566614133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b14a00 0x5649e5b16e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1edaffd640 1 -- 172.21.15.82:0/566614133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b14a00 msgr2=0x5649e5b16e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1edaffd640 1 --2- 172.21.15.82:0/566614133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b14a00 0x5649e5b16e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1edaffd640 1 -- 172.21.15.82:0/566614133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b0b4d0 msgr2=0x5649e5b0b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1edaffd640 1 --2- 172.21.15.82:0/566614133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b0b4d0 0x5649e5b0b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1edaffd640 1 -- 172.21.15.82:0/566614133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649e59ff1b0 con 0x5649e5b0be80 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.138+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/566614133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b0b4d0 0x5649e5b0b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.139+0000 7f1edaffd640 1 --2- 172.21.15.82:0/566614133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 0x5649e5b14430 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000e630 tx=0x7f1ed0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fffe722f35e7ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.139+0000 7f1ec1ffb640 1 -- 172.21.15.82:0/566614133 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0004020 con 0x5649e5b0be80 2024-04-21T08:48:13.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.139+0000 7f1ec1ffb640 1 -- 172.21.15.82:0/566614133 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ed00041c0 con 0x5649e5b0be80 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.139+0000 7f1ec1ffb640 1 -- 172.21.15.82:0/566614133 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed001c670 con 0x5649e5b0be80 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.139+0000 7f1ee4678740 1 -- 172.21.15.82:0/566614133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 msgr2=0x5649e5b14430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.139+0000 7f1ee4678740 1 --2- 172.21.15.82:0/566614133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 0x5649e5b14430 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000e630 tx=0x7f1ed0014000 comp rx=0 tx=0).stop 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.139+0000 7f1ee4678740 1 -- 172.21.15.82:0/566614133 shutdown_connections 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.139+0000 7f1ee4678740 1 --2- 172.21.15.82:0/566614133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b14a00 0x5649e5b16e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.139+0000 7f1ee4678740 1 --2- 172.21.15.82:0/566614133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 0x5649e5b14430 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.139+0000 7f1ee4678740 1 --2- 172.21.15.82:0/566614133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b0b4d0 0x5649e5b0b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.139+0000 7f1ee4678740 1 -- 172.21.15.82:0/566614133 >> 172.21.15.82:0/566614133 conn(0x5649e5b05ba0 msgr2=0x5649e5b05f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1ee4678740 1 -- 172.21.15.82:0/566614133 shutdown_connections 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1ee4678740 1 -- 172.21.15.82:0/566614133 wait complete. 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1ee4678740 1 -- start start 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1ee4678740 1 Processor -- start 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1ee4678740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0b4d0 0x5649e5b14380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1ee4678740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 0x5649e5b11710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1ee4678740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b14a00 0x5649e5b11d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1ee4678740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5649e5b17970 con 0x5649e5b0be80 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1ee4678740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5649e5a29500 con 0x5649e5b14a00 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1ee4678740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5649e5b17af0 con 0x5649e5b0b4d0 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1ee4678740 10 client.0 _ll_register_callbacks cb 0x5649e59f7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1edaffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 0x5649e5b11710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1edaffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 0x5649e5b11710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54712/0 (socket says 172.21.15.82:54712) 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1edaffd640 1 -- 172.21.15.82:0/258767502 learned_addr learned my addr 172.21.15.82:0/258767502 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.140+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0b4d0 0x5649e5b14380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1edbfff640 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b14a00 0x5649e5b11d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1edaffd640 1 -- 172.21.15.82:0/258767502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0b4d0 msgr2=0x5649e5b14380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1edaffd640 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0b4d0 0x5649e5b14380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1edaffd640 1 -- 172.21.15.82:0/258767502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b14a00 msgr2=0x5649e5b11d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:13.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1edaffd640 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b14a00 0x5649e5b11d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1edaffd640 1 -- 172.21.15.82:0/258767502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649e57823f0 con 0x5649e5b0be80 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1edaffd640 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 0x5649e5b11710 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000e600 tx=0x7f1ed0004030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e171fc5a7f2f8c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1ebaffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1ebaffd640 1 -- 172.21.15.82:0/258767502 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0004370 con 0x5649e5b0be80 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1ebaffd640 1 -- 172.21.15.82:0/258767502 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1ed000ed60 con 0x5649e5b0be80 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1ee4678740 1 -- 172.21.15.82:0/258767502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ed00041c0 con 0x5649e5b0be80 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1ebaffd640 1 -- 172.21.15.82:0/258767502 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1ed001c410 con 0x5649e5b0be80 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1ee4678740 10 client.16368 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1ee4678740 1 -- 172.21.15.82:0/258767502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5649e584c7d0 con 0x5649e5b0be80 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.141+0000 7f1ebaffd640 1 -- 172.21.15.82:0/258767502 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ed00041c0 con 0x5649e5b0be80 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.142+0000 7f1ebaffd640 1 -- 172.21.15.82:0/258767502 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f1ed0026480 con 0x5649e5b0be80 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.142+0000 7f1ee4678740 10 client.16368 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.142+0000 7f1ee4678740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:13.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.142+0000 7f1ee4678740 20 client.16368 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.142+0000 7f1ee4678740 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5649e5b24b50 0x5649e5b26f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.142+0000 7f1ee4678740 4 client.16368 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.142+0000 7f1ee4678740 1 -- 172.21.15.82:0/258767502 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649e5a0f930 con 0x5649e5b24b50 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.142+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5649e5b24b50 0x5649e5b26f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.142+0000 7f1ebaffd640 10 client.16368 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.142+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5649e5b24b50 0x5649e5b26f40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eb69eaff3ffa8af server_cookie=836b057440bc41e0 in_seq=0 out_seq=0 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ebaffd640 1 -- 172.21.15.82:0/258767502 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649e5a0f930 con 0x5649e5b24b50 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ebaffd640 10 client.16368 handle_command_reply: tid=0 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ee4678740 1 client.16368 shutdown 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ee4678740 1 -- 172.21.15.82:0/258767502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 msgr2=0x5649e5b11710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ee4678740 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 0x5649e5b11710 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000e600 tx=0x7f1ed0004030 comp rx=0 tx=0).stop 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ee4678740 1 -- 172.21.15.82:0/258767502 shutdown_connections 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ee4678740 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5649e5b24b50 0x5649e5b26f40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ee4678740 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b14a00 0x5649e5b11d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ebaffd640 0 client.16368 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:13.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ee4678740 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0be80 0x5649e5b11710 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ee4678740 1 --2- 172.21.15.82:0/258767502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0b4d0 0x5649e5b14380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ee4678740 1 -- 172.21.15.82:0/258767502 >> 172.21.15.82:0/258767502 conn(0x5649e5b05ba0 msgr2=0x5649e5b078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:13.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ee4678740 1 -- 172.21.15.82:0/258767502 shutdown_connections 2024-04-21T08:48:13.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.143+0000 7f1ee4678740 1 -- 172.21.15.82:0/258767502 wait complete. 2024-04-21T08:48:13.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.144+0000 7f1ee4678740 20 client.16368 trim_cache size 0 max 16384 2024-04-21T08:48:13.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1ee4678740 1 Processor -- start 2024-04-21T08:48:13.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1ee4678740 1 -- start start 2024-04-21T08:48:13.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1ee4678740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b0b4d0 0x5649e5b20860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1ee4678740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0be80 0x5649e5b13e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1ee4678740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b14a00 0x5649e5b14380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1ee4678740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc1af400 con 0x5649e5b14a00 2024-04-21T08:48:13.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1ee4678740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc1b0fd0 con 0x5649e5b0b4d0 2024-04-21T08:48:13.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1ee4678740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc1afad0 con 0x5649e5b0be80 2024-04-21T08:48:13.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1edbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b14a00 0x5649e5b14380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1edb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b0b4d0 0x5649e5b20860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1edbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b14a00 0x5649e5b14380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54716/0 (socket says 172.21.15.82:54716) 2024-04-21T08:48:13.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1edaffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0be80 0x5649e5b13e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.157+0000 7f1edbfff640 1 -- 172.21.15.82:0/1931542822 learned_addr learned my addr 172.21.15.82:0/1931542822 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.158+0000 7f1edb7fe640 1 -- 172.21.15.82:0/1931542822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0be80 msgr2=0x5649e5b13e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.158+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/1931542822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0be80 0x5649e5b13e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.158+0000 7f1edb7fe640 1 -- 172.21.15.82:0/1931542822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b14a00 msgr2=0x5649e5b14380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.158+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/1931542822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b14a00 0x5649e5b14380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.158+0000 7f1edb7fe640 1 -- 172.21.15.82:0/1931542822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649e5b0b0a0 con 0x5649e5b0b4d0 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.158+0000 7f1edbfff640 1 --2- 172.21.15.82:0/1931542822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b14a00 0x5649e5b14380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.158+0000 7f1edaffd640 1 --2- 172.21.15.82:0/1931542822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0be80 0x5649e5b13e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.158+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/1931542822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b0b4d0 0x5649e5b20860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0effb0 tx=0x7f1ecc04a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c9b09965ab98a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.158+0000 7f1ec17fa640 1 -- 172.21.15.82:0/1931542822 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc02dd60 con 0x5649e5b0b4d0 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.158+0000 7f1ec17fa640 1 -- 172.21.15.82:0/1931542822 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ecc0bcc20 con 0x5649e5b0b4d0 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.158+0000 7f1ec17fa640 1 -- 172.21.15.82:0/1931542822 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc003bb0 con 0x5649e5b0b4d0 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.159+0000 7f1ee4678740 1 -- 172.21.15.82:0/1931542822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b0b4d0 msgr2=0x5649e5b20860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:13.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.159+0000 7f1ee4678740 1 --2- 172.21.15.82:0/1931542822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b0b4d0 0x5649e5b20860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0effb0 tx=0x7f1ecc04a040 comp rx=0 tx=0).stop 2024-04-21T08:48:13.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.159+0000 7f1ee4678740 1 -- 172.21.15.82:0/1931542822 shutdown_connections 2024-04-21T08:48:13.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.159+0000 7f1ee4678740 1 --2- 172.21.15.82:0/1931542822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b14a00 0x5649e5b14380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.159+0000 7f1ee4678740 1 --2- 172.21.15.82:0/1931542822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0be80 0x5649e5b13e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.159+0000 7f1ee4678740 1 --2- 172.21.15.82:0/1931542822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b0b4d0 0x5649e5b20860 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.159+0000 7f1ee4678740 1 -- 172.21.15.82:0/1931542822 >> 172.21.15.82:0/1931542822 conn(0x5649e5b05ba0 msgr2=0x5649e5b0cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:13.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.159+0000 7f1ee4678740 1 -- 172.21.15.82:0/1931542822 shutdown_connections 2024-04-21T08:48:13.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.159+0000 7f1ee4678740 1 -- 172.21.15.82:0/1931542822 wait complete. 2024-04-21T08:48:13.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.159+0000 7f1ee4678740 1 -- start start 2024-04-21T08:48:13.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.159+0000 7f1ee4678740 1 Processor -- start 2024-04-21T08:48:13.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ee4678740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0b4d0 0x5649e5b1cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ee4678740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0be80 0x5649e5b1d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ee4678740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b14a00 0x5649e5b19d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ee4678740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc1b08d0 con 0x5649e5b0b4d0 2024-04-21T08:48:13.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ee4678740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc1b0200 con 0x5649e5b14a00 2024-04-21T08:48:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ee4678740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc1ad3e0 con 0x5649e5b0be80 2024-04-21T08:48:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ee4678740 10 client.0 _ll_register_callbacks cb 0x7f1ed000e500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1edb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0b4d0 0x5649e5b1cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1edb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0b4d0 0x5649e5b1cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54728/0 (socket says 172.21.15.82:54728) 2024-04-21T08:48:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1edb7fe640 1 -- 172.21.15.82:0/3950517520 learned_addr learned my addr 172.21.15.82:0/3950517520 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1edaffd640 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0be80 0x5649e5b1d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1edbfff640 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b14a00 0x5649e5b19d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1edb7fe640 1 -- 172.21.15.82:0/3950517520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0be80 msgr2=0x5649e5b1d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0be80 0x5649e5b1d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1edb7fe640 1 -- 172.21.15.82:0/3950517520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b14a00 msgr2=0x5649e5b19d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b14a00 0x5649e5b19d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1edb7fe640 1 -- 172.21.15.82:0/3950517520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649e5b1a300 con 0x5649e5b0b4d0 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1edb7fe640 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0b4d0 0x5649e5b1cd70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc000c00 tx=0x7f1ecc0bacf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781caa4ce37064b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ebaffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ebaffd640 1 -- 172.21.15.82:0/3950517520 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc003bb0 con 0x5649e5b0b4d0 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ebaffd640 1 -- 172.21.15.82:0/3950517520 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1ecc003d50 con 0x5649e5b0b4d0 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ee4678740 1 -- 172.21.15.82:0/3950517520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ecc0bcc20 con 0x5649e5b0b4d0 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ebaffd640 1 -- 172.21.15.82:0/3950517520 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1ecc0bdd70 con 0x5649e5b0b4d0 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ee4678740 10 client.16374 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.160+0000 7f1ee4678740 1 -- 172.21.15.82:0/3950517520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5649e5b8b680 con 0x5649e5b0b4d0 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1ebaffd640 1 -- 172.21.15.82:0/3950517520 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ecc0bcc20 con 0x5649e5b0b4d0 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1ebaffd640 1 -- 172.21.15.82:0/3950517520 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f1ecc0bd7e0 con 0x5649e5b0b4d0 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1ee4678740 10 client.16374 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1ee4678740 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1ee4678740 20 client.16374 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1ee4678740 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5649e5b24b50 0x5649e5b8dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1ee4678740 4 client.16374 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1ee4678740 1 -- 172.21.15.82:0/3950517520 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649e5a7dc10 con 0x5649e5b24b50 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1edaffd640 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5649e5b24b50 0x5649e5b8dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1edaffd640 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5649e5b24b50 0x5649e5b8dde0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc04a46c597e6ea0 server_cookie=691d44dadcb3ac5f in_seq=0 out_seq=0 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1ebaffd640 10 client.16374 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1ebaffd640 1 -- 172.21.15.82:0/3950517520 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649e5a7dc10 con 0x5649e5b24b50 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.161+0000 7f1ebaffd640 10 client.16374 handle_command_reply: tid=0 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee4678740 1 client.16374 shutdown 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee4678740 1 -- 172.21.15.82:0/3950517520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0b4d0 msgr2=0x5649e5b1cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee4678740 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0b4d0 0x5649e5b1cd70 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc000c00 tx=0x7f1ecc0bacf0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee4678740 1 -- 172.21.15.82:0/3950517520 shutdown_connections 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee4678740 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5649e5b24b50 0x5649e5b8dde0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee4678740 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5649e5b14a00 0x5649e5b19d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee4678740 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649e5b0be80 0x5649e5b1d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee4678740 1 --2- 172.21.15.82:0/3950517520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649e5b0b4d0 0x5649e5b1cd70 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ebaffd640 0 client.16374 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:13.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee4678740 1 -- 172.21.15.82:0/3950517520 >> 172.21.15.82:0/3950517520 conn(0x5649e5b05ba0 msgr2=0x5649e5b04370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:13.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee4678740 1 -- 172.21.15.82:0/3950517520 shutdown_connections 2024-04-21T08:48:13.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee4678740 1 -- 172.21.15.82:0/3950517520 wait complete. 2024-04-21T08:48:13.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee4678740 20 client.16374 trim_cache size 0 max 16384 2024-04-21T08:48:13.169 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:48:13.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee1a81640 1 -- 172.21.15.82:0/2026035831 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1ec40a3ed0 msgr2=0x7f1ec40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:13.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee1a81640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1ec40a3ed0 0x7f1ec40a6390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1edc10e450 tx=0x7f1ed0005f90 comp rx=0 tx=0).stop 2024-04-21T08:48:13.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee1a81640 1 -- 172.21.15.82:0/2026035831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1edc1031d0 msgr2=0x7f1edc198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:13.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee1a81640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1edc1031d0 0x7f1edc198740 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc005940 tx=0x7f1ecc02c720 comp rx=0 tx=0).stop 2024-04-21T08:48:13.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee1a81640 1 -- 172.21.15.82:0/2026035831 shutdown_connections 2024-04-21T08:48:13.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee1a81640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1edc10c510 0x7f1edc19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee1a81640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1edc103af0 0x7f1edc198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee1a81640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1ec40a3ed0 0x7f1ec40a6390 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee1a81640 1 --2- 172.21.15.82:0/2026035831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1edc1031d0 0x7f1edc198740 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:13.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee1a81640 1 -- 172.21.15.82:0/2026035831 >> 172.21.15.82:0/2026035831 conn(0x7f1edc077f50 msgr2=0x7f1edc0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:13.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.162+0000 7f1ee1a81640 1 -- 172.21.15.82:0/2026035831 shutdown_connections 2024-04-21T08:48:13.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:13.163+0000 7f1ee1a81640 1 -- 172.21.15.82:0/2026035831 wait complete. 2024-04-21T08:48:13.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:13 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1330891724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:13.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:13 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1330891724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:13.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:13 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1330891724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:14.173 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:48:14.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:48:14.174 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:48:14.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.289+0000 7f7be7e43640 1 -- 172.21.15.82:0/1227423655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7be0101a40 msgr2=0x7f7be0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.289+0000 7f7be7e43640 1 --2- 172.21.15.82:0/1227423655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7be0101a40 0x7f7be0105a80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009900 tx=0x7f7bd002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:14.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.289+0000 7f7be7e43640 1 -- 172.21.15.82:0/1227423655 shutdown_connections 2024-04-21T08:48:14.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.289+0000 7f7be7e43640 1 --2- 172.21.15.82:0/1227423655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be0106160 0x7f7be010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.289+0000 7f7be7e43640 1 --2- 172.21.15.82:0/1227423655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7be0101a40 0x7f7be0105a80 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.289+0000 7f7be7e43640 1 --2- 172.21.15.82:0/1227423655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7be0101090 0x7f7be0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.289+0000 7f7be7e43640 1 -- 172.21.15.82:0/1227423655 >> 172.21.15.82:0/1227423655 conn(0x7f7be0077f50 msgr2=0x7f7be0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:14.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.289+0000 7f7be7e43640 1 -- 172.21.15.82:0/1227423655 shutdown_connections 2024-04-21T08:48:14.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.289+0000 7f7be7e43640 1 -- 172.21.15.82:0/1227423655 wait complete. 2024-04-21T08:48:14.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be7e43640 1 Processor -- start 2024-04-21T08:48:14.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be7e43640 1 -- start start 2024-04-21T08:48:14.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be7e43640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7be0101090 0x7f7be019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be7e43640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be0106160 0x7f7be019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be7e43640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7be01985b0 0x7f7be0198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be7e43640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01110f0 con 0x7f7be01985b0 2024-04-21T08:48:14.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be7e43640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7be0110f70 con 0x7f7be0106160 2024-04-21T08:48:14.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be7e43640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7be0111270 con 0x7f7be0101090 2024-04-21T08:48:14.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be63e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7be01985b0 0x7f7be0198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be53de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be0106160 0x7f7be019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be63e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7be01985b0 0x7f7be0198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54750/0 (socket says 172.21.15.82:54750) 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be63e0640 1 -- 172.21.15.82:0/3743414069 learned_addr learned my addr 172.21.15.82:0/3743414069 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be5bdf640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7be0101090 0x7f7be019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be63e0640 1 -- 172.21.15.82:0/3743414069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7be0101090 msgr2=0x7f7be019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be63e0640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7be0101090 0x7f7be019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be63e0640 1 -- 172.21.15.82:0/3743414069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be0106160 msgr2=0x7f7be019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be63e0640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be0106160 0x7f7be019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be63e0640 1 -- 172.21.15.82:0/3743414069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be01990d0 con 0x7f7be01985b0 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.290+0000 7f7be63e0640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7be01985b0 0x7f7be0198a60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800eb10 tx=0x7f7bd800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.291+0000 7f7bd6ffd640 1 -- 172.21.15.82:0/3743414069 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8004b20 con 0x7f7be01985b0 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.291+0000 7f7bd6ffd640 1 -- 172.21.15.82:0/3743414069 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bd8004cc0 con 0x7f7be01985b0 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.291+0000 7f7be7e43640 1 -- 172.21.15.82:0/3743414069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be01993c0 con 0x7f7be01985b0 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.291+0000 7f7bd6ffd640 1 -- 172.21.15.82:0/3743414069 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8010670 con 0x7f7be01985b0 2024-04-21T08:48:14.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.291+0000 7f7be7e43640 1 -- 172.21.15.82:0/3743414069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be01a55c0 con 0x7f7be01985b0 2024-04-21T08:48:14.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.293+0000 7f7bd6ffd640 1 -- 172.21.15.82:0/3743414069 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f7bd8005020 con 0x7f7be01985b0 2024-04-21T08:48:14.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7beaa3a740 1 Processor -- start 2024-04-21T08:48:14.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7bd6ffd640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7bcc0a3ed0 0x7f7bcc0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7beaa3a740 1 -- start start 2024-04-21T08:48:14.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7be53de640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7bcc0a3ed0 0x7f7bcc0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7bd6ffd640 1 -- 172.21.15.82:0/3743414069 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f7bd80d13a0 con 0x7f7be01985b0 2024-04-21T08:48:14.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7beaa3a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce0bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7beaa3a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce0c290 0x5556bce14840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7beaa3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce14e10 0x5556bce17260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7beaa3a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5556bcd6ff00 con 0x5556bce0c290 2024-04-21T08:48:14.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7beaa3a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5556bcd0d590 con 0x5556bce0b8e0 2024-04-21T08:48:14.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7beaa3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5556bcd6bec0 con 0x5556bce14e10 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7be5bdf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce0bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7be5bdf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce0bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:45444/0 (socket says 172.21.15.82:45444) 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7be5bdf640 1 -- 172.21.15.82:0/1213273846 learned_addr learned my addr 172.21.15.82:0/1213273846 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7be53de640 1 --2- 172.21.15.82:0/1213273846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce0c290 0x5556bce14840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.294+0000 7f7be53de640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7bcc0a3ed0 0x7f7bcc0a6390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7bd002c240 tx=0x7f7bd002c6d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.295+0000 7f7be63e0640 1 --2- 172.21.15.82:0/1213273846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce14e10 0x5556bce17260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.295+0000 7f7be5bdf640 1 -- 172.21.15.82:0/1213273846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce14e10 msgr2=0x5556bce17260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.295+0000 7f7be5bdf640 1 --2- 172.21.15.82:0/1213273846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce14e10 0x5556bce17260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.295+0000 7f7be5bdf640 1 -- 172.21.15.82:0/1213273846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce0c290 msgr2=0x5556bce14840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.295+0000 7f7be5bdf640 1 --2- 172.21.15.82:0/1213273846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce0c290 0x5556bce14840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.295+0000 7f7be5bdf640 1 -- 172.21.15.82:0/1213273846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556bcb4a7e0 con 0x5556bce0b8e0 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.295+0000 7f7be63e0640 1 --2- 172.21.15.82:0/1213273846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce14e10 0x5556bce17260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.295+0000 7f7be5bdf640 1 --2- 172.21.15.82:0/1213273846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce0bcc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc009d20 tx=0x7f7bdc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf9fa102138ba3f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.295+0000 7f7bcbfff640 1 -- 172.21.15.82:0/1213273846 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc01e070 con 0x5556bce0b8e0 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.295+0000 7f7bcbfff640 1 -- 172.21.15.82:0/1213273846 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bdc019ae0 con 0x5556bce0b8e0 2024-04-21T08:48:14.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.295+0000 7f7bcbfff640 1 -- 172.21.15.82:0/1213273846 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00ea80 con 0x5556bce0b8e0 2024-04-21T08:48:14.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.296+0000 7f7beaa3a740 1 -- 172.21.15.82:0/1213273846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 msgr2=0x5556bce0bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.296+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/1213273846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce0bcc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc009d20 tx=0x7f7bdc011810 comp rx=0 tx=0).stop 2024-04-21T08:48:14.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.296+0000 7f7beaa3a740 1 -- 172.21.15.82:0/1213273846 shutdown_connections 2024-04-21T08:48:14.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.296+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/1213273846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce14e10 0x5556bce17260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.296+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/1213273846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce0c290 0x5556bce14840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.296+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/1213273846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce0bcc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.296+0000 7f7beaa3a740 1 -- 172.21.15.82:0/1213273846 >> 172.21.15.82:0/1213273846 conn(0x5556bce05fb0 msgr2=0x5556bce06390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:14.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.296+0000 7f7beaa3a740 1 -- 172.21.15.82:0/1213273846 shutdown_connections 2024-04-21T08:48:14.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.296+0000 7f7beaa3a740 1 -- 172.21.15.82:0/1213273846 wait complete. 2024-04-21T08:48:14.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.296+0000 7f7beaa3a740 1 -- start start 2024-04-21T08:48:14.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.296+0000 7f7beaa3a740 1 Processor -- start 2024-04-21T08:48:14.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7beaa3a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce11e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7beaa3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce0c290 0x5556bce12360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7beaa3a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 0x5556bce128a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7beaa3a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5556bce17d80 con 0x5556bce14e10 2024-04-21T08:48:14.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7beaa3a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5556bcd29580 con 0x5556bce0b8e0 2024-04-21T08:48:14.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7beaa3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5556bce17f00 con 0x5556bce0c290 2024-04-21T08:48:14.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7beaa3a740 10 client.0 _ll_register_callbacks cb 0x5556bcd2f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:14.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7be63e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 0x5556bce128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7be63e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 0x5556bce128a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54778/0 (socket says 172.21.15.82:54778) 2024-04-21T08:48:14.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7be5bdf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce11e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7be63e0640 1 -- 172.21.15.82:0/2112173209 learned_addr learned my addr 172.21.15.82:0/2112173209 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:14.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7be53de640 1 --2- 172.21.15.82:0/2112173209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce0c290 0x5556bce12360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7be63e0640 1 -- 172.21.15.82:0/2112173209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce0c290 msgr2=0x5556bce12360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7be63e0640 1 --2- 172.21.15.82:0/2112173209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce0c290 0x5556bce12360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7be63e0640 1 -- 172.21.15.82:0/2112173209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 msgr2=0x5556bce11e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7be63e0640 1 --2- 172.21.15.82:0/2112173209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce11e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7be63e0640 1 -- 172.21.15.82:0/2112173209 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556bcbaaf70 con 0x5556bce14e10 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7be63e0640 1 --2- 172.21.15.82:0/2112173209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 0x5556bce128a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8099710 tx=0x7f7bd809c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43b191ed6f3ec9e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7bc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7bc97fa640 1 -- 172.21.15.82:0/2112173209 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80a10f0 con 0x5556bce14e10 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7bc97fa640 1 -- 172.21.15.82:0/2112173209 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f7bd809cfa0 con 0x5556bce14e10 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.297+0000 7f7beaa3a740 1 -- 172.21.15.82:0/2112173209 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bdc019ae0 con 0x5556bce14e10 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.298+0000 7f7bc97fa640 1 -- 172.21.15.82:0/2112173209 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7bd80a59b0 con 0x5556bce14e10 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.298+0000 7f7beaa3a740 10 client.16386 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.298+0000 7f7beaa3a740 1 -- 172.21.15.82:0/2112173209 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556bca42170 con 0x5556bce14e10 2024-04-21T08:48:14.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.298+0000 7f7bc97fa640 1 -- 172.21.15.82:0/2112173209 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bdc019ae0 con 0x5556bce14e10 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.298+0000 7f7bc97fa640 1 -- 172.21.15.82:0/2112173209 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f7bd80ce800 con 0x5556bce14e10 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.298+0000 7f7beaa3a740 10 client.16386 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.298+0000 7f7beaa3a740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.298+0000 7f7beaa3a740 20 client.16386 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.298+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/2112173209 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5556bce25340 0x5556bce27790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.298+0000 7f7beaa3a740 4 client.16386 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.298+0000 7f7beaa3a740 1 -- 172.21.15.82:0/2112173209 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556bcd7d920 con 0x5556bce25340 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.298+0000 7f7be5bdf640 1 --2- 172.21.15.82:0/2112173209 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5556bce25340 0x5556bce27790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.299+0000 7f7bc97fa640 10 client.16386 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.299+0000 7f7be5bdf640 1 --2- 172.21.15.82:0/2112173209 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5556bce25340 0x5556bce27790 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f1844bf6857b02 server_cookie=5dd77b2f64545116 in_seq=0 out_seq=0 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.299+0000 7f7bc97fa640 1 -- 172.21.15.82:0/2112173209 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556bcd7d920 con 0x5556bce25340 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.299+0000 7f7bc97fa640 10 client.16386 handle_command_reply: tid=0 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.299+0000 7f7beaa3a740 1 client.16386 shutdown 2024-04-21T08:48:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.299+0000 7f7beaa3a740 1 -- 172.21.15.82:0/2112173209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 msgr2=0x5556bce128a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.299+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/2112173209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 0x5556bce128a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8099710 tx=0x7f7bd809c760 comp rx=0 tx=0).stop 2024-04-21T08:48:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.300+0000 7f7beaa3a740 1 -- 172.21.15.82:0/2112173209 shutdown_connections 2024-04-21T08:48:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.300+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/2112173209 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5556bce25340 0x5556bce27790 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.300+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/2112173209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 0x5556bce128a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.300+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/2112173209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce0c290 0x5556bce12360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.300+0000 7f7bc97fa640 0 client.16386 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.300+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/2112173209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce11e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.300+0000 7f7beaa3a740 1 -- 172.21.15.82:0/2112173209 >> 172.21.15.82:0/2112173209 conn(0x5556bce05fb0 msgr2=0x5556bce07d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.300+0000 7f7beaa3a740 1 -- 172.21.15.82:0/2112173209 shutdown_connections 2024-04-21T08:48:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.300+0000 7f7beaa3a740 1 -- 172.21.15.82:0/2112173209 wait complete. 2024-04-21T08:48:14.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.300+0000 7f7beaa3a740 20 client.16386 trim_cache size 0 max 16384 2024-04-21T08:48:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.318+0000 7f7beaa3a740 1 Processor -- start 2024-04-21T08:48:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.318+0000 7f7beaa3a740 1 -- start start 2024-04-21T08:48:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.318+0000 7f7beaa3a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce0b8e0 0x5556bce14550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.318+0000 7f7beaa3a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0c290 0x5556bce20c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7beaa3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce14e10 0x5556bce124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7beaa3a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01af2a0 con 0x5556bce0b8e0 2024-04-21T08:48:14.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7beaa3a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01b0e70 con 0x5556bce0c290 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7beaa3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01af970 con 0x5556bce14e10 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7be5bdf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce0b8e0 0x5556bce14550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7be5bdf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce0b8e0 0x5556bce14550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54780/0 (socket says 172.21.15.82:54780) 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7be5bdf640 1 -- 172.21.15.82:0/3791501915 learned_addr learned my addr 172.21.15.82:0/3791501915 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7be63e0640 1 --2- 172.21.15.82:0/3791501915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce14e10 0x5556bce124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7be53de640 1 --2- 172.21.15.82:0/3791501915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0c290 0x5556bce20c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7be5bdf640 1 -- 172.21.15.82:0/3791501915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce14e10 msgr2=0x5556bce124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7be5bdf640 1 --2- 172.21.15.82:0/3791501915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce14e10 0x5556bce124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7be5bdf640 1 -- 172.21.15.82:0/3791501915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0c290 msgr2=0x5556bce20c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7be5bdf640 1 --2- 172.21.15.82:0/3791501915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0c290 0x5556bce20c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7be5bdf640 1 -- 172.21.15.82:0/3791501915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556bce0cbd0 con 0x5556bce0b8e0 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7be53de640 1 --2- 172.21.15.82:0/3791501915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0c290 0x5556bce20c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:14.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7be5bdf640 1 --2- 172.21.15.82:0/3791501915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce0b8e0 0x5556bce14550 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc013010 tx=0x7f7bdc00e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490aac20132b72c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:14.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7bcb7fe640 1 -- 172.21.15.82:0/3791501915 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc029e60 con 0x5556bce0b8e0 2024-04-21T08:48:14.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7bcb7fe640 1 -- 172.21.15.82:0/3791501915 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bdc01f040 con 0x5556bce0b8e0 2024-04-21T08:48:14.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.319+0000 7f7bcb7fe640 1 -- 172.21.15.82:0/3791501915 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc01e040 con 0x5556bce0b8e0 2024-04-21T08:48:14.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.320+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3791501915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce0b8e0 msgr2=0x5556bce14550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.320+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/3791501915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce0b8e0 0x5556bce14550 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc013010 tx=0x7f7bdc00e510 comp rx=0 tx=0).stop 2024-04-21T08:48:14.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.320+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3791501915 shutdown_connections 2024-04-21T08:48:14.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.320+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/3791501915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce14e10 0x5556bce124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.320+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/3791501915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0c290 0x5556bce20c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.320+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/3791501915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce0b8e0 0x5556bce14550 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.320+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3791501915 >> 172.21.15.82:0/3791501915 conn(0x5556bce05fb0 msgr2=0x5556bce11de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.320+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3791501915 shutdown_connections 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.320+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3791501915 wait complete. 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.320+0000 7f7beaa3a740 1 -- start start 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7beaa3a740 1 Processor -- start 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7beaa3a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce18b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7beaa3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce0c290 0x5556bce190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7beaa3a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 0x5556bce19600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7beaa3a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01b0770 con 0x5556bce14e10 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7beaa3a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01b00a0 con 0x5556bce0b8e0 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7beaa3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7be01ad280 con 0x5556bce0c290 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7beaa3a740 10 client.0 _ll_register_callbacks cb 0x7f7bd809cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7be63e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 0x5556bce19600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7be5bdf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce18b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7be63e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 0x5556bce19600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54796/0 (socket says 172.21.15.82:54796) 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7be5bdf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce18b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:45462/0 (socket says 172.21.15.82:45462) 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7be63e0640 1 -- 172.21.15.82:0/3763638301 learned_addr learned my addr 172.21.15.82:0/3763638301 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7be53de640 1 --2- 172.21.15.82:0/3763638301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce0c290 0x5556bce190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7be63e0640 1 -- 172.21.15.82:0/3763638301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce0c290 msgr2=0x5556bce190c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7be63e0640 1 --2- 172.21.15.82:0/3763638301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce0c290 0x5556bce190c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7be63e0640 1 -- 172.21.15.82:0/3763638301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 msgr2=0x5556bce18b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7be63e0640 1 --2- 172.21.15.82:0/3763638301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce18b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.321+0000 7f7be63e0640 1 -- 172.21.15.82:0/3763638301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556bce8ece0 con 0x5556bce14e10 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7be63e0640 1 --2- 172.21.15.82:0/3763638301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 0x5556bce19600 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7bd80ce5e0 tx=0x7f7bd800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b581d37de9363 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7bc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7bc97fa640 1 -- 172.21.15.82:0/3763638301 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd809d1a0 con 0x5556bce14e10 2024-04-21T08:48:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7bc97fa640 1 -- 172.21.15.82:0/3763638301 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f7bd807bb60 con 0x5556bce14e10 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3763638301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bdc01f040 con 0x5556bce14e10 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7bc97fa640 1 -- 172.21.15.82:0/3763638301 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7bd80a5730 con 0x5556bce14e10 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7beaa3a740 10 client.16392 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3763638301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5556bce8ffd0 con 0x5556bce14e10 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7bc97fa640 1 -- 172.21.15.82:0/3763638301 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bdc01f040 con 0x5556bce14e10 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7bc97fa640 1 -- 172.21.15.82:0/3763638301 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f7bd809c320 con 0x5556bce14e10 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7beaa3a740 10 client.16392 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7beaa3a740 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7beaa3a740 20 client.16392 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/3763638301 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5556bce90670 0x5556bce92a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7beaa3a740 4 client.16392 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.322+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3763638301 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5556bcd77790 con 0x5556bce90670 2024-04-21T08:48:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.323+0000 7f7be5bdf640 1 --2- 172.21.15.82:0/3763638301 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5556bce90670 0x5556bce92a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.323+0000 7f7be5bdf640 1 --2- 172.21.15.82:0/3763638301 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5556bce90670 0x5556bce92a90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af3d25f4f237478 server_cookie=39b774248f76fe8d in_seq=0 out_seq=0 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.323+0000 7f7bc97fa640 10 client.16392 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7bc97fa640 1 -- 172.21.15.82:0/3763638301 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5556bcd77790 con 0x5556bce90670 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7bc97fa640 10 client.16392 handle_command_reply: tid=0 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7beaa3a740 1 client.16392 shutdown 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3763638301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 msgr2=0x5556bce19600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/3763638301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 0x5556bce19600 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7bd80ce5e0 tx=0x7f7bd800efd0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3763638301 shutdown_connections 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/3763638301 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5556bce90670 0x5556bce92a90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/3763638301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556bce14e10 0x5556bce19600 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/3763638301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556bce0c290 0x5556bce190c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7bc97fa640 0 client.16392 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7beaa3a740 1 --2- 172.21.15.82:0/3763638301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5556bce0b8e0 0x5556bce18b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3763638301 >> 172.21.15.82:0/3763638301 conn(0x5556bce05fb0 msgr2=0x5556bce07bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3763638301 shutdown_connections 2024-04-21T08:48:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7beaa3a740 1 -- 172.21.15.82:0/3763638301 wait complete. 2024-04-21T08:48:14.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.324+0000 7f7beaa3a740 20 client.16392 trim_cache size 0 max 16384 2024-04-21T08:48:14.330 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:48:14.330 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:48:14.330 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "7b359986-e2f0-4a50-92c6-6faa7d8ba8c7", 2024-04-21T08:48:14.330 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:48:14.330 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:48:14.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.325+0000 7f7be7e43640 1 -- 172.21.15.82:0/3743414069 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7bcc0a3ed0 msgr2=0x7f7bcc0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.325+0000 7f7be7e43640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7bcc0a3ed0 0x7f7bcc0a6390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7bd002c240 tx=0x7f7bd002c6d0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.325+0000 7f7be7e43640 1 -- 172.21.15.82:0/3743414069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7be01985b0 msgr2=0x7f7be0198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.325+0000 7f7be7e43640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7be01985b0 0x7f7be0198a60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800eb10 tx=0x7f7bd800c2d0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.325+0000 7f7be7e43640 1 -- 172.21.15.82:0/3743414069 shutdown_connections 2024-04-21T08:48:14.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.325+0000 7f7be7e43640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7bcc0a3ed0 0x7f7bcc0a6390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.325+0000 7f7be7e43640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7be01985b0 0x7f7be0198a60 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.325+0000 7f7be7e43640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7be0106160 0x7f7be019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.325+0000 7f7be7e43640 1 --2- 172.21.15.82:0/3743414069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7be0101090 0x7f7be019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.325+0000 7f7be7e43640 1 -- 172.21.15.82:0/3743414069 >> 172.21.15.82:0/3743414069 conn(0x7f7be0077f50 msgr2=0x7f7be0100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:14.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.325+0000 7f7be7e43640 1 -- 172.21.15.82:0/3743414069 shutdown_connections 2024-04-21T08:48:14.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.325+0000 7f7be7e43640 1 -- 172.21.15.82:0/3743414069 wait complete. 2024-04-21T08:48:14.335 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:14 smithi082 ceph-mon[27967]: pgmap v39: 129 pgs: 129 active+clean; 43 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 3.7 MiB/s wr, 250 op/s 2024-04-21T08:48:14.335 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:14 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:48:14.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:14.336 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:14.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:14 smithi106 ceph-mon[31072]: pgmap v39: 129 pgs: 129 active+clean; 43 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 3.7 MiB/s wr, 250 op/s 2024-04-21T08:48:14.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:14 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:48:14.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.453+0000 7fd551df4640 1 -- 172.21.15.82:0/3665588109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54c101a30 msgr2=0x7fd54c105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.453+0000 7fd551df4640 1 --2- 172.21.15.82:0/3665588109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54c101a30 0x7fd54c105a70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd540009900 tx=0x7fd54002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:14.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.453+0000 7fd551df4640 1 -- 172.21.15.82:0/3665588109 shutdown_connections 2024-04-21T08:48:14.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.453+0000 7fd551df4640 1 --2- 172.21.15.82:0/3665588109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd54c106150 0x7fd54c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.453+0000 7fd551df4640 1 --2- 172.21.15.82:0/3665588109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54c101a30 0x7fd54c105a70 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.453+0000 7fd551df4640 1 --2- 172.21.15.82:0/3665588109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd54c101080 0x7fd54c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.453+0000 7fd551df4640 1 -- 172.21.15.82:0/3665588109 >> 172.21.15.82:0/3665588109 conn(0x7fd54c078000 msgr2=0x7fd54c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:14.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.453+0000 7fd551df4640 1 -- 172.21.15.82:0/3665588109 shutdown_connections 2024-04-21T08:48:14.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.453+0000 7fd551df4640 1 -- 172.21.15.82:0/3665588109 wait complete. 2024-04-21T08:48:14.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.453+0000 7fd551df4640 1 Processor -- start 2024-04-21T08:48:14.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd551df4640 1 -- start start 2024-04-21T08:48:14.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd551df4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54c101080 0x7fd54c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd551df4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd54c106150 0x7fd54c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd551df4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd54c19a080 0x7fd54c1a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd551df4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1110f0 con 0x7fd54c101080 2024-04-21T08:48:14.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd551df4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c110f70 con 0x7fd54c106150 2024-04-21T08:48:14.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd551df4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c111270 con 0x7fd54c19a080 2024-04-21T08:48:14.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd54b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54c101080 0x7fd54c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd54b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54c101080 0x7fd54c1995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54802/0 (socket says 172.21.15.82:54802) 2024-04-21T08:48:14.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd54b7fe640 1 -- 172.21.15.82:0/2293434606 learned_addr learned my addr 172.21.15.82:0/2293434606 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:14.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd54affd640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd54c106150 0x7fd54c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd54b7fe640 1 -- 172.21.15.82:0/2293434606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd54c19a080 msgr2=0x7fd54c1a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd54c19a080 0x7fd54c1a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd54b7fe640 1 -- 172.21.15.82:0/2293434606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd54c106150 msgr2=0x7fd54c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd54c106150 0x7fd54c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd54b7fe640 1 -- 172.21.15.82:0/2293434606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c1a4e00 con 0x7fd54c101080 2024-04-21T08:48:14.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd54affd640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd54c106150 0x7fd54c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:14.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.454+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54c101080 0x7fd54c1995b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00e9f0 tx=0x7fd53c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:14.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.455+0000 7fd548ff9640 1 -- 172.21.15.82:0/2293434606 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00cd80 con 0x7fd54c101080 2024-04-21T08:48:14.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.455+0000 7fd548ff9640 1 -- 172.21.15.82:0/2293434606 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd53c0108e0 con 0x7fd54c101080 2024-04-21T08:48:14.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.455+0000 7fd551df4640 1 -- 172.21.15.82:0/2293434606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c1a50f0 con 0x7fd54c101080 2024-04-21T08:48:14.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.455+0000 7fd548ff9640 1 -- 172.21.15.82:0/2293434606 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c018980 con 0x7fd54c101080 2024-04-21T08:48:14.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.455+0000 7fd551df4640 1 -- 172.21.15.82:0/2293434606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c1a5660 con 0x7fd54c101080 2024-04-21T08:48:14.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.457+0000 7fd548ff9640 1 -- 172.21.15.82:0/2293434606 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fd53c010430 con 0x7fd54c101080 2024-04-21T08:48:14.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd5549e9740 1 Processor -- start 2024-04-21T08:48:14.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd548ff9640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5340a3f50 0x7fd5340a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd5549e9740 1 -- start start 2024-04-21T08:48:14.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd54affd640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5340a3f50 0x7fd5340a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd548ff9640 1 -- 172.21.15.82:0/2293434606 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fd53c0d11e0 con 0x7fd54c101080 2024-04-21T08:48:14.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd5549e9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430d72b0 0x5596430d7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd5549e9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd5549e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430e4e90 0x5596430e7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd5549e9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55964303c1e0 con 0x5596430d7bd0 2024-04-21T08:48:14.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd5549e9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559642fd9390 con 0x5596430d72b0 2024-04-21T08:48:14.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd5549e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5596430381a0 con 0x5596430e4e90 2024-04-21T08:48:14.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd54affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd54b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430d72b0 0x5596430d7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd54affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54804/0 (socket says 172.21.15.82:54804) 2024-04-21T08:48:14.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd54affd640 1 -- 172.21.15.82:0/4246289430 learned_addr learned my addr 172.21.15.82:0/4246289430 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:14.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd54bfff640 1 --2- 172.21.15.82:0/4246289430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430e4e90 0x5596430e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd54b7fe640 1 -- 172.21.15.82:0/4246289430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430e4e90 msgr2=0x5596430e7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/4246289430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430e4e90 0x5596430e7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd54b7fe640 1 -- 172.21.15.82:0/4246289430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 msgr2=0x5596430e4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.458+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/4246289430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.459+0000 7fd54b7fe640 1 -- 172.21.15.82:0/4246289430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559642e187d0 con 0x5596430d72b0 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.459+0000 7fd54affd640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5340a3f50 0x7fd5340a6410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd540005ec0 tx=0x7fd540004430 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.459+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/4246289430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430d72b0 0x5596430d7690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0ce330 tx=0x7fd53c0107e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fdf2ffc2c12d94e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.459+0000 7fd531ffb640 1 -- 172.21.15.82:0/4246289430 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0a0f30 con 0x5596430d72b0 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.459+0000 7fd531ffb640 1 -- 172.21.15.82:0/4246289430 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd53c0a57f0 con 0x5596430d72b0 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.459+0000 7fd531ffb640 1 -- 172.21.15.82:0/4246289430 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c018b20 con 0x5596430d72b0 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.460+0000 7fd5549e9740 1 -- 172.21.15.82:0/4246289430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430d72b0 msgr2=0x5596430d7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.460+0000 7fd5549e9740 1 --2- 172.21.15.82:0/4246289430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430d72b0 0x5596430d7690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0ce330 tx=0x7fd53c0107e0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.460+0000 7fd5549e9740 1 -- 172.21.15.82:0/4246289430 shutdown_connections 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.460+0000 7fd5549e9740 1 --2- 172.21.15.82:0/4246289430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430e4e90 0x5596430e7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.460+0000 7fd5549e9740 1 --2- 172.21.15.82:0/4246289430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.460+0000 7fd5549e9740 1 --2- 172.21.15.82:0/4246289430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430d72b0 0x5596430d7690 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.460+0000 7fd5549e9740 1 -- 172.21.15.82:0/4246289430 >> 172.21.15.82:0/4246289430 conn(0x5596430d1cb0 msgr2=0x5596430d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.460+0000 7fd5549e9740 1 -- 172.21.15.82:0/4246289430 shutdown_connections 2024-04-21T08:48:14.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.460+0000 7fd5549e9740 1 -- 172.21.15.82:0/4246289430 wait complete. 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.460+0000 7fd5549e9740 1 -- start start 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.460+0000 7fd5549e9740 1 Processor -- start 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd5549e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430d72b0 0x5596430e1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd5549e9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd5549e9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430e4e90 0x5596430e29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd5549e9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5596430e7bf0 con 0x5596430d7bd0 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd5549e9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559642ff52f0 con 0x5596430e4e90 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd5549e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5596430e7d70 con 0x5596430d72b0 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd5549e9740 10 client.0 _ll_register_callbacks cb 0x559643002580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd54affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd54affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54808/0 (socket says 172.21.15.82:54808) 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd54affd640 1 -- 172.21.15.82:0/4007403566 learned_addr learned my addr 172.21.15.82:0/4007403566 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd54bfff640 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430e4e90 0x5596430e29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430d72b0 0x5596430e1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd54affd640 1 -- 172.21.15.82:0/4007403566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430d72b0 msgr2=0x5596430e1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd54affd640 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430d72b0 0x5596430e1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd54affd640 1 -- 172.21.15.82:0/4007403566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430e4e90 msgr2=0x5596430e29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd54affd640 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430e4e90 0x5596430e29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd54affd640 1 -- 172.21.15.82:0/4007403566 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559642d4e3f0 con 0x5596430d7bd0 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd54affd640 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e24a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd540036040 tx=0x7fd54002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf40db9d19243776 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd52affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd52affd640 1 -- 172.21.15.82:0/4007403566 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54002cc60 con 0x5596430d7bd0 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd52affd640 1 -- 172.21.15.82:0/4007403566 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fd540003c80 con 0x5596430d7bd0 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.461+0000 7fd5549e9740 1 -- 172.21.15.82:0/4007403566 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd53c0a57f0 con 0x5596430d7bd0 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.462+0000 7fd52affd640 1 -- 172.21.15.82:0/4007403566 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd540003e80 con 0x5596430d7bd0 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.462+0000 7fd5549e9740 10 client.16404 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.462+0000 7fd5549e9740 1 -- 172.21.15.82:0/4007403566 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559642d0e0f0 con 0x5596430d7bd0 2024-04-21T08:48:14.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.462+0000 7fd52affd640 1 -- 172.21.15.82:0/4007403566 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd53c0a57f0 con 0x5596430d7bd0 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.462+0000 7fd52affd640 1 -- 172.21.15.82:0/4007403566 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fd540034440 con 0x5596430d7bd0 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.462+0000 7fd5549e9740 10 client.16404 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.462+0000 7fd5549e9740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.462+0000 7fd5549e9740 20 client.16404 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.462+0000 7fd5549e9740 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5596430f4eb0 0x5596430f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.462+0000 7fd5549e9740 4 client.16404 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.462+0000 7fd5549e9740 1 -- 172.21.15.82:0/4007403566 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559643049c00 con 0x5596430f4eb0 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.462+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5596430f4eb0 0x5596430f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.463+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5596430f4eb0 0x5596430f72f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d289b28b525419a server_cookie=b0f58f18cf37eaef in_seq=0 out_seq=0 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.463+0000 7fd52affd640 10 client.16404 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.463+0000 7fd52affd640 1 -- 172.21.15.82:0/4007403566 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559643049c00 con 0x5596430f4eb0 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.463+0000 7fd52affd640 10 client.16404 handle_command_reply: tid=0 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd5549e9740 1 client.16404 shutdown 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd5549e9740 1 -- 172.21.15.82:0/4007403566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 msgr2=0x5596430e24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd5549e9740 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e24a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd540036040 tx=0x7fd54002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:14.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd5549e9740 1 -- 172.21.15.82:0/4007403566 shutdown_connections 2024-04-21T08:48:14.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd5549e9740 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5596430f4eb0 0x5596430f72f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd5549e9740 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430e4e90 0x5596430e29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd52affd640 0 client.16404 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:14.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd5549e9740 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e24a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd5549e9740 1 --2- 172.21.15.82:0/4007403566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430d72b0 0x5596430e1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd5549e9740 1 -- 172.21.15.82:0/4007403566 >> 172.21.15.82:0/4007403566 conn(0x5596430d1cb0 msgr2=0x5596430d3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:14.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd5549e9740 1 -- 172.21.15.82:0/4007403566 shutdown_connections 2024-04-21T08:48:14.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd5549e9740 1 -- 172.21.15.82:0/4007403566 wait complete. 2024-04-21T08:48:14.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.464+0000 7fd5549e9740 20 client.16404 trim_cache size 0 max 16384 2024-04-21T08:48:14.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:14 smithi134 ceph-mon[31772]: pgmap v39: 129 pgs: 129 active+clean; 43 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 3.7 MiB/s wr, 250 op/s 2024-04-21T08:48:14.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:14 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:48:14.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.484+0000 7fd5549e9740 1 Processor -- start 2024-04-21T08:48:14.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.484+0000 7fd5549e9740 1 -- start start 2024-04-21T08:48:14.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.484+0000 7fd5549e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430d72b0 0x5596430e4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd5549e9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd5549e9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430e4e90 0x5596430e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd5549e9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1ad2d0 con 0x5596430d7bd0 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd5549e9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1aeea0 con 0x5596430e4e90 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd5549e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1ad9a0 con 0x5596430d72b0 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd54affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd54affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430f0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54816/0 (socket says 172.21.15.82:54816) 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd54affd640 1 -- 172.21.15.82:0/3501544704 learned_addr learned my addr 172.21.15.82:0/3501544704 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/3501544704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430d72b0 0x5596430e4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd54bfff640 1 --2- 172.21.15.82:0/3501544704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430e4e90 0x5596430e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd54affd640 1 -- 172.21.15.82:0/3501544704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430d72b0 msgr2=0x5596430e4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd54affd640 1 --2- 172.21.15.82:0/3501544704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430d72b0 0x5596430e4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd54affd640 1 -- 172.21.15.82:0/3501544704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430e4e90 msgr2=0x5596430e0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd54affd640 1 --2- 172.21.15.82:0/3501544704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430e4e90 0x5596430e0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd54affd640 1 -- 172.21.15.82:0/3501544704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596430e75f0 con 0x5596430d7bd0 2024-04-21T08:48:14.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd54affd640 1 --2- 172.21.15.82:0/3501544704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430f0af0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd540034830 tx=0x7fd540009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e0d3df2d6b1e15 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd5317fa640 1 -- 172.21.15.82:0/3501544704 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54002c860 con 0x5596430d7bd0 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd5317fa640 1 -- 172.21.15.82:0/3501544704 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd540034e50 con 0x5596430d7bd0 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.485+0000 7fd5317fa640 1 -- 172.21.15.82:0/3501544704 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400428f0 con 0x5596430d7bd0 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.486+0000 7fd5549e9740 1 -- 172.21.15.82:0/3501544704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 msgr2=0x5596430f0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.486+0000 7fd5549e9740 1 --2- 172.21.15.82:0/3501544704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430f0af0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd540034830 tx=0x7fd540009250 comp rx=0 tx=0).stop 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.486+0000 7fd5549e9740 1 -- 172.21.15.82:0/3501544704 shutdown_connections 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.486+0000 7fd5549e9740 1 --2- 172.21.15.82:0/3501544704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430e4e90 0x5596430e0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.486+0000 7fd5549e9740 1 --2- 172.21.15.82:0/3501544704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430f0af0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.486+0000 7fd5549e9740 1 --2- 172.21.15.82:0/3501544704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430d72b0 0x5596430e4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.486+0000 7fd5549e9740 1 -- 172.21.15.82:0/3501544704 >> 172.21.15.82:0/3501544704 conn(0x5596430d1cb0 msgr2=0x5596430d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.486+0000 7fd5549e9740 1 -- 172.21.15.82:0/3501544704 shutdown_connections 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.486+0000 7fd5549e9740 1 -- 172.21.15.82:0/3501544704 wait complete. 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.486+0000 7fd5549e9740 1 -- start start 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd5549e9740 1 Processor -- start 2024-04-21T08:48:14.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd5549e9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430d72b0 0x5596430e8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd5549e9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd5549e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430e4e90 0x5596430e96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd5549e9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1ae7a0 con 0x5596430d7bd0 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd5549e9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1ae0d0 con 0x5596430d72b0 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd5549e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c1ab2b0 con 0x5596430e4e90 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd5549e9740 10 client.0 _ll_register_callbacks cb 0x7fd540009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd54affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd54b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430d72b0 0x5596430e8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd54affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54830/0 (socket says 172.21.15.82:54830) 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd54affd640 1 -- 172.21.15.82:0/2628114037 learned_addr learned my addr 172.21.15.82:0/2628114037 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd54bfff640 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430e4e90 0x5596430e96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd54b7fe640 1 -- 172.21.15.82:0/2628114037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430e4e90 msgr2=0x5596430e96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430e4e90 0x5596430e96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd54b7fe640 1 -- 172.21.15.82:0/2628114037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 msgr2=0x5596430e91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:14.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd54b7fe640 1 -- 172.21.15.82:0/2628114037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55964315a630 con 0x5596430d72b0 2024-04-21T08:48:14.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd54bfff640 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430e4e90 0x5596430e96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:14.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.487+0000 7fd54affd640 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:14.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd54b7fe640 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430d72b0 0x5596430e8c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd53c09ae30 tx=0x7fd53c09cd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9abc0b7ef479f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:14.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd52affd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:48:14.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd52affd640 1 -- 172.21.15.82:0/2628114037 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0a0f30 con 0x5596430d72b0 2024-04-21T08:48:14.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd5549e9740 1 -- 172.21.15.82:0/2628114037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd540034e50 con 0x5596430d72b0 2024-04-21T08:48:14.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd52affd640 1 -- 172.21.15.82:0/2628114037 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fd53c010a90 con 0x5596430d72b0 2024-04-21T08:48:14.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd52affd640 1 -- 172.21.15.82:0/2628114037 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd53c0a57f0 con 0x5596430d72b0 2024-04-21T08:48:14.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd5549e9740 10 client.25748 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd5549e9740 1 -- 172.21.15.82:0/2628114037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55964315b940 con 0x5596430d72b0 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd52affd640 1 -- 172.21.15.82:0/2628114037 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd540034e50 con 0x5596430d72b0 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd52affd640 1 -- 172.21.15.82:0/2628114037 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fd53c0ce640 con 0x5596430d72b0 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd5549e9740 10 client.25748 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd5549e9740 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd5549e9740 20 client.25748 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd5549e9740 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55964315bc80 0x55964315e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd5549e9740 4 client.25748 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.488+0000 7fd5549e9740 1 -- 172.21.15.82:0/2628114037 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559643043a70 con 0x55964315bc80 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd54affd640 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55964315bc80 0x55964315e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd52affd640 10 client.25748 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd54affd640 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55964315bc80 0x55964315e070 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16217e8e8971d90b server_cookie=2fbee4f2fc573b84 in_seq=0 out_seq=0 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd52affd640 1 -- 172.21.15.82:0/2628114037 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559643043a70 con 0x55964315bc80 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd52affd640 10 client.25748 handle_command_reply: tid=0 2024-04-21T08:48:14.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd5549e9740 1 client.25748 shutdown 2024-04-21T08:48:14.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd5549e9740 1 -- 172.21.15.82:0/2628114037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430d72b0 msgr2=0x5596430e8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd5549e9740 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430d72b0 0x5596430e8c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd53c09ae30 tx=0x7fd53c09cd00 comp rx=0 tx=0).stop 2024-04-21T08:48:14.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd5549e9740 1 -- 172.21.15.82:0/2628114037 shutdown_connections 2024-04-21T08:48:14.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd5549e9740 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55964315bc80 0x55964315e070 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd5549e9740 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596430e4e90 0x5596430e96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd5549e9740 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596430d7bd0 0x5596430e91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd52affd640 0 client.25748 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:14.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd5549e9740 1 --2- 172.21.15.82:0/2628114037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5596430d72b0 0x5596430e8c70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.489+0000 7fd5549e9740 1 -- 172.21.15.82:0/2628114037 >> 172.21.15.82:0/2628114037 conn(0x5596430d1cb0 msgr2=0x5596430d30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:14.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd5549e9740 1 -- 172.21.15.82:0/2628114037 shutdown_connections 2024-04-21T08:48:14.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd5549e9740 1 -- 172.21.15.82:0/2628114037 wait complete. 2024-04-21T08:48:14.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd5549e9740 20 client.25748 trim_cache size 0 max 16384 2024-04-21T08:48:14.497 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:48:14.497 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:48:14.497 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:48:14.497 INFO:teuthology.orchestra.run.smithi082.stdout: "7b359986-e2f0-4a50-92c6-6faa7d8ba8c7": { 2024-04-21T08:48:14.497 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:48:14.497 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "7b359986-e2f0-4a50-92c6-6faa7d8ba8c7", 2024-04-21T08:48:14.497 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:48:14.497 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:48:14.497 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:48:14.498 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:48:14.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd551df4640 1 -- 172.21.15.82:0/2293434606 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5340a3f50 msgr2=0x7fd5340a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd551df4640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5340a3f50 0x7fd5340a6410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd540005ec0 tx=0x7fd540004430 comp rx=0 tx=0).stop 2024-04-21T08:48:14.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd551df4640 1 -- 172.21.15.82:0/2293434606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54c101080 msgr2=0x7fd54c1995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:14.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd551df4640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54c101080 0x7fd54c1995b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00e9f0 tx=0x7fd53c00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd551df4640 1 -- 172.21.15.82:0/2293434606 shutdown_connections 2024-04-21T08:48:14.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd551df4640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd54c19a080 0x7fd54c1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd551df4640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd54c106150 0x7fd54c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd551df4640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5340a3f50 0x7fd5340a6410 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd551df4640 1 --2- 172.21.15.82:0/2293434606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54c101080 0x7fd54c1995b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:14.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd551df4640 1 -- 172.21.15.82:0/2293434606 >> 172.21.15.82:0/2293434606 conn(0x7fd54c078000 msgr2=0x7fd54c0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:14.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd551df4640 1 -- 172.21.15.82:0/2293434606 shutdown_connections 2024-04-21T08:48:14.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:14.490+0000 7fd551df4640 1 -- 172.21.15.82:0/2293434606 wait complete. 2024-04-21T08:48:14.502 INFO:tasks.cephfs.filesystem:scrub status for tag:7b359986-e2f0-4a50-92c6-6faa7d8ba8c7 - {'path': '/', 'tag': '7b359986-e2f0-4a50-92c6-6faa7d8ba8c7', 'options': 'recursive,force'} 2024-04-21T08:48:15.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:15 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:48:15.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:15 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:15.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:15 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:15.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:15 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:15.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:15 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:15.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:15 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:15.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:15 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:15.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:15 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:15.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:15 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:48:15.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:15 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:15.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:15 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:15.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:15 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:15.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:15 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:15.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:15 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:15.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:15 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:15.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:15 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:15 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:48:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:15 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:15 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:15 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:15 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:15 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:15 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:15.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:15 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.942+0000 7f6038db2640 1 -- 172.21.15.82:0/1063579216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6034103b40 msgr2=0x7f603410c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.942+0000 7f6038db2640 1 --2- 172.21.15.82:0/1063579216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6034103b40 0x7f603410c020 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f601c009900 tx=0x7f601c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.942+0000 7f6038db2640 1 -- 172.21.15.82:0/1063579216 shutdown_connections 2024-04-21T08:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.942+0000 7f6038db2640 1 --2- 172.21.15.82:0/1063579216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f603410c560 0x7f603410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.942+0000 7f6038db2640 1 --2- 172.21.15.82:0/1063579216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6034103b40 0x7f603410c020 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.942+0000 7f6038db2640 1 --2- 172.21.15.82:0/1063579216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6034103220 0x7f6034103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.942+0000 7f6038db2640 1 -- 172.21.15.82:0/1063579216 >> 172.21.15.82:0/1063579216 conn(0x7f6034077fd0 msgr2=0x7f60340783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.942+0000 7f6038db2640 1 -- 172.21.15.82:0/1063579216 shutdown_connections 2024-04-21T08:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.942+0000 7f6038db2640 1 -- 172.21.15.82:0/1063579216 wait complete. 2024-04-21T08:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.942+0000 7f6038db2640 1 Processor -- start 2024-04-21T08:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f6038db2640 1 -- start start 2024-04-21T08:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f6038db2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6034103220 0x7f6034198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f6038db2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6034103b40 0x7f6034198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f6038db2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f603410c560 0x7f603419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f6038db2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6034112200 con 0x7f6034103b40 2024-04-21T08:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f6038db2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6034112080 con 0x7f603410c560 2024-04-21T08:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f6038db2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6034112380 con 0x7f6034103220 2024-04-21T08:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f602bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6034103b40 0x7f6034198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f602bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6034103b40 0x7f6034198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54852/0 (socket says 172.21.15.82:54852) 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f602bfff640 1 -- 172.21.15.82:0/1351986177 learned_addr learned my addr 172.21.15.82:0/1351986177 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f6032d9d640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f603410c560 0x7f603419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f603259c640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6034103220 0x7f6034198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f602bfff640 1 -- 172.21.15.82:0/1351986177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6034103220 msgr2=0x7f6034198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f602bfff640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6034103220 0x7f6034198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f602bfff640 1 -- 172.21.15.82:0/1351986177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f603410c560 msgr2=0x7f603419cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f602bfff640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f603410c560 0x7f603419cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f602bfff640 1 -- 172.21.15.82:0/1351986177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603419d6f0 con 0x7f6034103b40 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f602bfff640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6034103b40 0x7f6034198c40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f601c000c00 tx=0x7f601c02c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f602b7fe640 1 -- 172.21.15.82:0/1351986177 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c043020 con 0x7f6034103b40 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f602b7fe640 1 -- 172.21.15.82:0/1351986177 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f601c02dbe0 con 0x7f6034103b40 2024-04-21T08:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.943+0000 7f6038db2640 1 -- 172.21.15.82:0/1351986177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603419d980 con 0x7f6034103b40 2024-04-21T08:48:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.944+0000 7f602b7fe640 1 -- 172.21.15.82:0/1351986177 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c003950 con 0x7f6034103b40 2024-04-21T08:48:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.944+0000 7f6038db2640 1 -- 172.21.15.82:0/1351986177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60341a5270 con 0x7f6034103b40 2024-04-21T08:48:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.946+0000 7f602b7fe640 1 -- 172.21.15.82:0/1351986177 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f601c046020 con 0x7f6034103b40 2024-04-21T08:48:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f603b9a0740 1 Processor -- start 2024-04-21T08:48:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f603b9a0740 1 -- start start 2024-04-21T08:48:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f602b7fe640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f60200a3f70 0x7f60200a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f603b9a0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3c8d0 0x55794cf3b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f603b9a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3bb60 0x55794cf3bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f602b7fe640 1 -- 172.21.15.82:0/1351986177 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f601c0f2c70 con 0x7f6034103b40 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f603b9a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf494f0 0x55794cf4b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f603b9a0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55794cea08d0 con 0x55794cf3c8d0 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f603b9a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55794ce3de80 con 0x55794cf3bb60 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f603b9a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55794ce9c890 con 0x55794cf494f0 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f6032d9d640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f60200a3f70 0x7f60200a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f603259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3c8d0 0x55794cf3b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f602bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3bb60 0x55794cf3bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f603259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3c8d0 0x55794cf3b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54860/0 (socket says 172.21.15.82:54860) 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f603259c640 1 -- 172.21.15.82:0/4050412391 learned_addr learned my addr 172.21.15.82:0/4050412391 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f602bfff640 1 -- 172.21.15.82:0/4050412391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf494f0 msgr2=0x55794cf4b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f602bfff640 1 --2- 172.21.15.82:0/4050412391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf494f0 0x55794cf4b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f602bfff640 1 -- 172.21.15.82:0/4050412391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3c8d0 msgr2=0x55794cf3b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f602bfff640 1 --2- 172.21.15.82:0/4050412391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3c8d0 0x55794cf3b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.947+0000 7f602bfff640 1 -- 172.21.15.82:0/4050412391 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794ce5f320 con 0x55794cf3bb60 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.948+0000 7f6032d9d640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f60200a3f70 0x7f60200a6430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f602c005fd0 tx=0x7f602c011810 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.948+0000 7f602bfff640 1 --2- 172.21.15.82:0/4050412391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3bb60 0x55794cf3bfe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f601c0efed0 tx=0x7f601c005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a439c8c58a73cf3c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.948+0000 7f6028ff9640 1 -- 172.21.15.82:0/4050412391 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c0beeb0 con 0x55794cf3bb60 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.948+0000 7f6028ff9640 1 -- 172.21.15.82:0/4050412391 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f601c0be7b0 con 0x55794cf3bb60 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.948+0000 7f6028ff9640 1 -- 172.21.15.82:0/4050412391 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c0c6f00 con 0x55794cf3bb60 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 -- 172.21.15.82:0/4050412391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3bb60 msgr2=0x55794cf3bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 --2- 172.21.15.82:0/4050412391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3bb60 0x55794cf3bfe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f601c0efed0 tx=0x7f601c005e20 comp rx=0 tx=0).stop 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 -- 172.21.15.82:0/4050412391 shutdown_connections 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 --2- 172.21.15.82:0/4050412391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf494f0 0x55794cf4b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 --2- 172.21.15.82:0/4050412391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3bb60 0x55794cf3bfe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 --2- 172.21.15.82:0/4050412391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3c8d0 0x55794cf3b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 -- 172.21.15.82:0/4050412391 >> 172.21.15.82:0/4050412391 conn(0x55794cf02c90 msgr2=0x55794cf36850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 -- 172.21.15.82:0/4050412391 shutdown_connections 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 -- 172.21.15.82:0/4050412391 wait complete. 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 -- start start 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 Processor -- start 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3bb60 0x55794cf48f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3c8d0 0x55794cf462b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.949+0000 7f603b9a0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf494f0 0x55794cf469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f603b9a0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55794cf4bfe0 con 0x55794cf494f0 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f603b9a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55794ce28eb0 con 0x55794cf3c8d0 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f603b9a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55794cf4c160 con 0x55794cf3bb60 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f603259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3bb60 0x55794cf48f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f602bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3c8d0 0x55794cf462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f602bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3c8d0 0x55794cf462b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:45538/0 (socket says 172.21.15.82:45538) 2024-04-21T08:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f603259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3bb60 0x55794cf48f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:49074/0 (socket says 172.21.15.82:49074) 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f602bfff640 1 -- 172.21.15.82:0/1587688094 learned_addr learned my addr 172.21.15.82:0/1587688094 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f603b9a0740 10 client.0 _ll_register_callbacks cb 0x55794ce55b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f6032d9d640 1 --2- 172.21.15.82:0/1587688094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf494f0 0x55794cf469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f602bfff640 1 -- 172.21.15.82:0/1587688094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3bb60 msgr2=0x55794cf48f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f602bfff640 1 --2- 172.21.15.82:0/1587688094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3bb60 0x55794cf48f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f602bfff640 1 -- 172.21.15.82:0/1587688094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf494f0 msgr2=0x55794cf469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f602bfff640 1 --2- 172.21.15.82:0/1587688094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf494f0 0x55794cf469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f602bfff640 1 -- 172.21.15.82:0/1587688094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794cbb3210 con 0x55794cf3c8d0 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f602bfff640 1 --2- 172.21.15.82:0/1587688094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3c8d0 0x55794cf462b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f601c0be740 tx=0x7f601c0bab80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36207c86c51dced8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f6019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f6019ffb640 1 -- 172.21.15.82:0/1587688094 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c0c2b60 con 0x55794cf3c8d0 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f6019ffb640 1 -- 172.21.15.82:0/1587688094 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f601c0c6f00 con 0x55794cf3c8d0 2024-04-21T08:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f603b9a0740 1 -- 172.21.15.82:0/1587688094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f601c0be7b0 con 0x55794cf3c8d0 2024-04-21T08:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f6019ffb640 1 -- 172.21.15.82:0/1587688094 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f601c0c7390 con 0x55794cf3c8d0 2024-04-21T08:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f603b9a0740 10 client.25760 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.950+0000 7f603b9a0740 1 -- 172.21.15.82:0/1587688094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55794cc7f620 con 0x55794cf3c8d0 2024-04-21T08:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.951+0000 7f6019ffb640 1 -- 172.21.15.82:0/1587688094 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f601c0be7b0 con 0x55794cf3c8d0 2024-04-21T08:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.951+0000 7f6019ffb640 1 -- 172.21.15.82:0/1587688094 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f601c0f00d0 con 0x55794cf3c8d0 2024-04-21T08:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.951+0000 7f603b9a0740 10 client.25760 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.951+0000 7f603b9a0740 10 client.25760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.951+0000 7f603b9a0740 20 client.25760 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.951+0000 7f603b9a0740 1 --2- 172.21.15.82:0/1587688094 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55794cf59090 0x55794cf5b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.951+0000 7f603b9a0740 4 client.25760 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.951+0000 7f603b9a0740 1 -- 172.21.15.82:0/1587688094 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55794ceb4880 con 0x55794cf59090 2024-04-21T08:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.951+0000 7f603259c640 1 --2- 172.21.15.82:0/1587688094 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55794cf59090 0x55794cf5b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.951+0000 7f6019ffb640 10 client.25760 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.951+0000 7f603259c640 1 --2- 172.21.15.82:0/1587688094 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55794cf59090 0x55794cf5b480 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f986e79da9889fc3 server_cookie=8923b1eb685f48ca in_seq=0 out_seq=0 2024-04-21T08:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f6019ffb640 1 -- 172.21.15.82:0/1587688094 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55794ceb4880 con 0x55794cf59090 2024-04-21T08:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f6019ffb640 10 client.25760 handle_command_reply: tid=0 2024-04-21T08:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f603b9a0740 1 client.25760 shutdown 2024-04-21T08:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f603b9a0740 1 -- 172.21.15.82:0/1587688094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3c8d0 msgr2=0x55794cf462b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f603b9a0740 1 --2- 172.21.15.82:0/1587688094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3c8d0 0x55794cf462b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f601c0be740 tx=0x7f601c0bab80 comp rx=0 tx=0).stop 2024-04-21T08:48:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f603b9a0740 1 -- 172.21.15.82:0/1587688094 shutdown_connections 2024-04-21T08:48:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f603b9a0740 1 --2- 172.21.15.82:0/1587688094 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55794cf59090 0x55794cf5b480 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f603b9a0740 1 --2- 172.21.15.82:0/1587688094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf494f0 0x55794cf469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f6019ffb640 0 client.25760 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f603b9a0740 1 --2- 172.21.15.82:0/1587688094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3c8d0 0x55794cf462b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f603b9a0740 1 --2- 172.21.15.82:0/1587688094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3bb60 0x55794cf48f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f603b9a0740 1 -- 172.21.15.82:0/1587688094 >> 172.21.15.82:0/1587688094 conn(0x55794cf02c90 msgr2=0x55794cf385a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f603b9a0740 1 -- 172.21.15.82:0/1587688094 shutdown_connections 2024-04-21T08:48:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.952+0000 7f603b9a0740 1 -- 172.21.15.82:0/1587688094 wait complete. 2024-04-21T08:48:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.953+0000 7f603b9a0740 20 client.25760 trim_cache size 0 max 16384 2024-04-21T08:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603b9a0740 1 Processor -- start 2024-04-21T08:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603b9a0740 1 -- start start 2024-04-21T08:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603b9a0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3bb60 0x55794cf54e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603b9a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3c8d0 0x55794cf46250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603b9a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf494f0 0x55794cf60cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603b9a0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f60341ad030 con 0x55794cf3bb60 2024-04-21T08:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603b9a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f60341aec00 con 0x55794cf494f0 2024-04-21T08:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603b9a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60341ad700 con 0x55794cf3c8d0 2024-04-21T08:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3bb60 0x55794cf54e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3bb60 0x55794cf54e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54870/0 (socket says 172.21.15.82:54870) 2024-04-21T08:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603259c640 1 -- 172.21.15.82:0/3695956554 learned_addr learned my addr 172.21.15.82:0/3695956554 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f602bfff640 1 --2- 172.21.15.82:0/3695956554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3c8d0 0x55794cf46250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f6032d9d640 1 --2- 172.21.15.82:0/3695956554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf494f0 0x55794cf60cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603259c640 1 -- 172.21.15.82:0/3695956554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3c8d0 msgr2=0x55794cf46250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603259c640 1 --2- 172.21.15.82:0/3695956554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3c8d0 0x55794cf46250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603259c640 1 -- 172.21.15.82:0/3695956554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf494f0 msgr2=0x55794cf60cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603259c640 1 --2- 172.21.15.82:0/3695956554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf494f0 0x55794cf60cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.968+0000 7f603259c640 1 -- 172.21.15.82:0/3695956554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794cf5ba60 con 0x55794cf3bb60 2024-04-21T08:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.969+0000 7f603259c640 1 --2- 172.21.15.82:0/3695956554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3bb60 0x55794cf54e10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f6024009d70 tx=0x7f6024005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da536301c7cdb95 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.969+0000 7f601bfff640 1 -- 172.21.15.82:0/3695956554 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60240200f0 con 0x55794cf3bb60 2024-04-21T08:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.969+0000 7f601bfff640 1 -- 172.21.15.82:0/3695956554 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6024025070 con 0x55794cf3bb60 2024-04-21T08:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.969+0000 7f601bfff640 1 -- 172.21.15.82:0/3695956554 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60240109f0 con 0x55794cf3bb60 2024-04-21T08:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.970+0000 7f603b9a0740 1 -- 172.21.15.82:0/3695956554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3bb60 msgr2=0x55794cf54e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.970+0000 7f603b9a0740 1 --2- 172.21.15.82:0/3695956554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3bb60 0x55794cf54e10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f6024009d70 tx=0x7f6024005f90 comp rx=0 tx=0).stop 2024-04-21T08:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.970+0000 7f603b9a0740 1 -- 172.21.15.82:0/3695956554 shutdown_connections 2024-04-21T08:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.970+0000 7f603b9a0740 1 --2- 172.21.15.82:0/3695956554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf494f0 0x55794cf60cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.970+0000 7f603b9a0740 1 --2- 172.21.15.82:0/3695956554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3c8d0 0x55794cf46250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.970+0000 7f603b9a0740 1 --2- 172.21.15.82:0/3695956554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf3bb60 0x55794cf54e10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.970+0000 7f603b9a0740 1 -- 172.21.15.82:0/3695956554 >> 172.21.15.82:0/3695956554 conn(0x55794cf02c90 msgr2=0x55794cf45b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.970+0000 7f603b9a0740 1 -- 172.21.15.82:0/3695956554 shutdown_connections 2024-04-21T08:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.970+0000 7f603b9a0740 1 -- 172.21.15.82:0/3695956554 wait complete. 2024-04-21T08:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.971+0000 7f603b9a0740 1 -- start start 2024-04-21T08:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.971+0000 7f603b9a0740 1 Processor -- start 2024-04-21T08:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.971+0000 7f603b9a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3bb60 0x55794cf43f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.971+0000 7f603b9a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3c8d0 0x55794cf51320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.971+0000 7f603b9a0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf494f0 0x55794cf4a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.971+0000 7f603b9a0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f60341ae500 con 0x55794cf494f0 2024-04-21T08:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.971+0000 7f603b9a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f60341ade30 con 0x55794cf3bb60 2024-04-21T08:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.971+0000 7f603b9a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60341ab010 con 0x55794cf3c8d0 2024-04-21T08:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.971+0000 7f603b9a0740 10 client.0 _ll_register_callbacks cb 0x7f60240097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f602bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3c8d0 0x55794cf51320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f6032d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf494f0 0x55794cf4a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f602bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3c8d0 0x55794cf51320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:49086/0 (socket says 172.21.15.82:49086) 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f602bfff640 1 -- 172.21.15.82:0/484343931 learned_addr learned my addr 172.21.15.82:0/484343931 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f603259c640 1 --2- 172.21.15.82:0/484343931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3bb60 0x55794cf43f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f602bfff640 1 -- 172.21.15.82:0/484343931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3bb60 msgr2=0x55794cf43f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f602bfff640 1 --2- 172.21.15.82:0/484343931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3bb60 0x55794cf43f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f602bfff640 1 -- 172.21.15.82:0/484343931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf494f0 msgr2=0x55794cf4a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f602bfff640 1 --2- 172.21.15.82:0/484343931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf494f0 0x55794cf4a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f602bfff640 1 -- 172.21.15.82:0/484343931 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794cf3ae30 con 0x55794cf3c8d0 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f602bfff640 1 --2- 172.21.15.82:0/484343931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3c8d0 0x55794cf51320 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f601c0a9320 tx=0x7f601c02bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b3a38e949f56495 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f6019ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f6019ffb640 1 -- 172.21.15.82:0/484343931 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c0be110 con 0x55794cf3c8d0 2024-04-21T08:48:15.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f6019ffb640 1 -- 172.21.15.82:0/484343931 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f601c0f0010 con 0x55794cf3c8d0 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f603b9a0740 1 -- 172.21.15.82:0/484343931 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6024025070 con 0x55794cf3c8d0 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f6019ffb640 1 -- 172.21.15.82:0/484343931 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f601c0bcb90 con 0x55794cf3c8d0 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.972+0000 7f603b9a0740 10 client.25576 fetch_fsmap learned FSMap version 30 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.973+0000 7f603b9a0740 1 -- 172.21.15.82:0/484343931 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55794cf3ae30 con 0x55794cf3c8d0 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.973+0000 7f6019ffb640 1 -- 172.21.15.82:0/484343931 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6024025070 con 0x55794cf3c8d0 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.973+0000 7f6019ffb640 1 -- 172.21.15.82:0/484343931 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f601c003af0 con 0x55794cf3c8d0 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.973+0000 7f603b9a0740 10 client.25576 fetch_fsmap finished waiting for FSMap version 30 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.973+0000 7f603b9a0740 10 client.25576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.973+0000 7f603b9a0740 20 client.25576 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.973+0000 7f603b9a0740 1 --2- 172.21.15.82:0/484343931 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55794cfbfd00 0x55794cfc2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.973+0000 7f603b9a0740 4 client.25576 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.973+0000 7f603b9a0740 1 -- 172.21.15.82:0/484343931 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55794ce403d0 con 0x55794cfbfd00 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.973+0000 7f603259c640 1 --2- 172.21.15.82:0/484343931 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55794cfbfd00 0x55794cfc2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.974+0000 7f6019ffb640 10 client.25576 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:15.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.974+0000 7f603259c640 1 --2- 172.21.15.82:0/484343931 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55794cfbfd00 0x55794cfc2120 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcc889e4b08ada08 server_cookie=f446f73c6d0b39b1 in_seq=0 out_seq=0 2024-04-21T08:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.982+0000 7f6019ffb640 1 -- 172.21.15.82:0/484343931 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55794ce403d0 con 0x55794cfbfd00 2024-04-21T08:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.982+0000 7f6019ffb640 10 client.25576 handle_command_reply: tid=0 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.982+0000 7f603b9a0740 1 client.25576 shutdown 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.982+0000 7f603b9a0740 1 -- 172.21.15.82:0/484343931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3c8d0 msgr2=0x55794cf51320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.982+0000 7f603b9a0740 1 --2- 172.21.15.82:0/484343931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3c8d0 0x55794cf51320 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f601c0a9320 tx=0x7f601c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.982+0000 7f603b9a0740 1 -- 172.21.15.82:0/484343931 shutdown_connections 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.982+0000 7f603b9a0740 1 --2- 172.21.15.82:0/484343931 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55794cfbfd00 0x55794cfc2120 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.982+0000 7f603b9a0740 1 --2- 172.21.15.82:0/484343931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55794cf494f0 0x55794cf4a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.982+0000 7f6019ffb640 0 client.25576 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.982+0000 7f603b9a0740 1 --2- 172.21.15.82:0/484343931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55794cf3c8d0 0x55794cf51320 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.982+0000 7f603b9a0740 1 --2- 172.21.15.82:0/484343931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794cf3bb60 0x55794cf43f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.982+0000 7f603b9a0740 1 -- 172.21.15.82:0/484343931 >> 172.21.15.82:0/484343931 conn(0x55794cf02c90 msgr2=0x55794cf37c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.983+0000 7f603b9a0740 1 -- 172.21.15.82:0/484343931 shutdown_connections 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.983+0000 7f603b9a0740 1 -- 172.21.15.82:0/484343931 wait complete. 2024-04-21T08:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.983+0000 7f603b9a0740 20 client.25576 trim_cache size 0 max 16384 2024-04-21T08:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:48:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.983+0000 7f6038db2640 1 -- 172.21.15.82:0/1351986177 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f60200a3f70 msgr2=0x7f60200a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.983+0000 7f6038db2640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f60200a3f70 0x7f60200a6430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f602c005fd0 tx=0x7f602c011810 comp rx=0 tx=0).stop 2024-04-21T08:48:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.983+0000 7f6038db2640 1 -- 172.21.15.82:0/1351986177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6034103b40 msgr2=0x7f6034198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.983+0000 7f6038db2640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6034103b40 0x7f6034198c40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f601c000c00 tx=0x7f601c02c6d0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.983+0000 7f6038db2640 1 -- 172.21.15.82:0/1351986177 shutdown_connections 2024-04-21T08:48:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.983+0000 7f6038db2640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f603410c560 0x7f603419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.983+0000 7f6038db2640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f60200a3f70 0x7f60200a6430 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.983+0000 7f6038db2640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6034103b40 0x7f6034198c40 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.984+0000 7f6038db2640 1 --2- 172.21.15.82:0/1351986177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6034103220 0x7f6034198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.984+0000 7f6038db2640 1 -- 172.21.15.82:0/1351986177 >> 172.21.15.82:0/1351986177 conn(0x7f6034077fd0 msgr2=0x7f60340fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.984+0000 7f6038db2640 1 -- 172.21.15.82:0/1351986177 shutdown_connections 2024-04-21T08:48:15.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:15.984+0000 7f6038db2640 1 -- 172.21.15.82:0/1351986177 wait complete. 2024-04-21T08:48:16.221 INFO:tasks.workunit.client.0.smithi082.stdout: 16 277562 733 130630 695 59069 1106 2024-04-21T08:48:16.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:16 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:16.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:16 smithi082 ceph-mon[27967]: pgmap v40: 129 pgs: 129 active+clean; 43 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 3.5 MiB/s wr, 213 op/s 2024-04-21T08:48:16.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:16 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:16.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:16 smithi106 ceph-mon[31072]: pgmap v40: 129 pgs: 129 active+clean; 43 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 3.5 MiB/s wr, 213 op/s 2024-04-21T08:48:16.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:16 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:16.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:16 smithi134 ceph-mon[31772]: pgmap v40: 129 pgs: 129 active+clean; 43 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 3.5 MiB/s wr, 213 op/s 2024-04-21T08:48:18.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:18 smithi082 ceph-mon[27967]: pgmap v41: 129 pgs: 129 active+clean; 51 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 333 op/s 2024-04-21T08:48:18.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:18 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:18.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:18 smithi106 ceph-mon[31072]: pgmap v41: 129 pgs: 129 active+clean; 51 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 333 op/s 2024-04-21T08:48:18.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:18 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:18.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:18 smithi134 ceph-mon[31772]: pgmap v41: 129 pgs: 129 active+clean; 51 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 333 op/s 2024-04-21T08:48:18.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:18 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:19.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:19 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:48:19.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:19 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:48:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:19 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:48:20.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:20 smithi082 ceph-mon[27967]: pgmap v42: 129 pgs: 129 active+clean; 50 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-04-21T08:48:20.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:20 smithi106 ceph-mon[31072]: pgmap v42: 129 pgs: 129 active+clean; 50 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-04-21T08:48:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:20 smithi134 ceph-mon[31772]: pgmap v42: 129 pgs: 129 active+clean; 50 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-04-21T08:48:22.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:22 smithi082 ceph-mon[27967]: pgmap v43: 129 pgs: 129 active+clean; 50 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 307 op/s 2024-04-21T08:48:22.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:22 smithi106 ceph-mon[31072]: pgmap v43: 129 pgs: 129 active+clean; 50 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 307 op/s 2024-04-21T08:48:22.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:22 smithi134 ceph-mon[31772]: pgmap v43: 129 pgs: 129 active+clean; 50 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 307 op/s 2024-04-21T08:48:24.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:24 smithi082 ceph-mon[27967]: pgmap v44: 129 pgs: 129 active+clean; 57 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 406 op/s 2024-04-21T08:48:24.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:24 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:48:24.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:24 smithi106 ceph-mon[31072]: pgmap v44: 129 pgs: 129 active+clean; 57 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 406 op/s 2024-04-21T08:48:24.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:24 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:48:24.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:24 smithi134 ceph-mon[31772]: pgmap v44: 129 pgs: 129 active+clean; 57 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 406 op/s 2024-04-21T08:48:24.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:24 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:48:26.222 INFO:tasks.workunit.client.0.smithi082.stdout: 28 571947 661 305529 505 228344 1507 2024-04-21T08:48:26.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:26 smithi082 ceph-mon[27967]: pgmap v45: 129 pgs: 129 active+clean; 57 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-04-21T08:48:26.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:26 smithi106 ceph-mon[31072]: pgmap v45: 129 pgs: 129 active+clean; 57 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-04-21T08:48:26.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:26 smithi134 ceph-mon[31772]: pgmap v45: 129 pgs: 129 active+clean; 57 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-04-21T08:48:28.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:28 smithi082 ceph-mon[27967]: pgmap v46: 129 pgs: 129 active+clean; 57 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 257 op/s 2024-04-21T08:48:28.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:28 smithi106 ceph-mon[31072]: pgmap v46: 129 pgs: 129 active+clean; 57 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 257 op/s 2024-04-21T08:48:28.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:28 smithi134 ceph-mon[31772]: pgmap v46: 129 pgs: 129 active+clean; 57 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 257 op/s 2024-04-21T08:48:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.084+0000 7f6af5340640 1 -- 172.21.15.82:0/1115848239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af010c520 msgr2=0x7f6af010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.084+0000 7f6af5340640 1 --2- 172.21.15.82:0/1115848239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af010c520 0x7f6af010e9c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009ca0 tx=0x7f6ae802c230 comp rx=0 tx=0).stop 2024-04-21T08:48:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.085+0000 7f6af5340640 1 -- 172.21.15.82:0/1115848239 shutdown_connections 2024-04-21T08:48:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.085+0000 7f6af5340640 1 --2- 172.21.15.82:0/1115848239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af010c520 0x7f6af010e9c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.085+0000 7f6af5340640 1 --2- 172.21.15.82:0/1115848239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6af0103b00 0x7f6af010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.085+0000 7f6af5340640 1 --2- 172.21.15.82:0/1115848239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6af01031e0 0x7f6af01035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.085+0000 7f6af5340640 1 -- 172.21.15.82:0/1115848239 >> 172.21.15.82:0/1115848239 conn(0x7f6af0077fd0 msgr2=0x7f6af00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.085+0000 7f6af5340640 1 -- 172.21.15.82:0/1115848239 shutdown_connections 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.085+0000 7f6af5340640 1 -- 172.21.15.82:0/1115848239 wait complete. 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.085+0000 7f6af5340640 1 Processor -- start 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6af5340640 1 -- start start 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6af5340640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6af01031e0 0x7f6af01a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6af5340640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6af0103b00 0x7f6af01a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6af5340640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af010c520 0x7f6af019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6af5340640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01121b0 con 0x7f6af010c520 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6af5340640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0112030 con 0x7f6af0103b00 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6af5340640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0112330 con 0x7f6af01031e0 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6aef7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af010c520 0x7f6af019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6af0103b00 0x7f6af01a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6aef7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af010c520 0x7f6af019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59854/0 (socket says 172.21.15.82:59854) 2024-04-21T08:48:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6aef7fe640 1 -- 172.21.15.82:0/1276254308 learned_addr learned my addr 172.21.15.82:0/1276254308 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6aeeffd640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6af01031e0 0x7f6af01a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6aef7fe640 1 -- 172.21.15.82:0/1276254308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6af01031e0 msgr2=0x7f6af01a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6aef7fe640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6af01031e0 0x7f6af01a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6aef7fe640 1 -- 172.21.15.82:0/1276254308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6af0103b00 msgr2=0x7f6af01a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6aef7fe640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6af0103b00 0x7f6af01a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6aef7fe640 1 -- 172.21.15.82:0/1276254308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af019af20 con 0x7f6af010c520 2024-04-21T08:48:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.086+0000 7f6aef7fe640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af010c520 0x7f6af019a660 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8005e30 tx=0x7f6ae8035d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.090+0000 7f6ad7fff640 1 -- 172.21.15.82:0/1276254308 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8039040 con 0x7f6af010c520 2024-04-21T08:48:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.090+0000 7f6ad7fff640 1 -- 172.21.15.82:0/1276254308 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae8044070 con 0x7f6af010c520 2024-04-21T08:48:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.090+0000 7f6ad7fff640 1 -- 172.21.15.82:0/1276254308 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae80328a0 con 0x7f6af010c520 2024-04-21T08:48:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.090+0000 7f6af5340640 1 -- 172.21.15.82:0/1276254308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af019b1b0 con 0x7f6af010c520 2024-04-21T08:48:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.090+0000 7f6af5340640 1 -- 172.21.15.82:0/1276254308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af01a7340 con 0x7f6af010c520 2024-04-21T08:48:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.093+0000 7f6ad7fff640 1 -- 172.21.15.82:0/1276254308 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f6ae8003d00 con 0x7f6af010c520 2024-04-21T08:48:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.093+0000 7f6af800d740 1 Processor -- start 2024-04-21T08:48:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.093+0000 7f6af800d740 1 -- start start 2024-04-21T08:48:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6af800d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 0x559f8a166620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6af800d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a166b60 0x559f8a166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6af800d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a1724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6af800d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559f8a0cb8d0 con 0x559f8a1700e0 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6af800d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559f8a068e80 con 0x559f8a166b60 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6af800d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f8a0c7890 con 0x559f8a1678d0 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a1724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a1724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59856/0 (socket says 172.21.15.82:59856) 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6aee7fc640 1 -- 172.21.15.82:0/545691778 learned_addr learned my addr 172.21.15.82:0/545691778 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6ad7fff640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6ad80a3ff0 0x7f6ad80a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6aeeffd640 1 --2- 172.21.15.82:0/545691778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a166b60 0x559f8a166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/545691778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 0x559f8a166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6aef7fe640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6ad80a3ff0 0x7f6ad80a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6aee7fc640 1 -- 172.21.15.82:0/545691778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 msgr2=0x559f8a166620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/545691778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 0x559f8a166620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6aee7fc640 1 -- 172.21.15.82:0/545691778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a166b60 msgr2=0x559f8a166fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/545691778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a166b60 0x559f8a166fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6aee7fc640 1 -- 172.21.15.82:0/545691778 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f8a08a320 con 0x559f8a1700e0 2024-04-21T08:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.094+0000 7f6ad7fff640 1 -- 172.21.15.82:0/1276254308 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6ae80f3020 con 0x7f6af010c520 2024-04-21T08:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.095+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/545691778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a1724d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009f10 tx=0x7f6ae4009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63f3a60e92a415a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.095+0000 7f6aef7fe640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6ad80a3ff0 0x7f6ad80a64b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80f02e0 tx=0x7f6ae803ac20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.096+0000 7f6ad57fa640 1 -- 172.21.15.82:0/545691778 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4004030 con 0x559f8a1700e0 2024-04-21T08:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.096+0000 7f6ad57fa640 1 -- 172.21.15.82:0/545691778 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae40041d0 con 0x559f8a1700e0 2024-04-21T08:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.096+0000 7f6ad57fa640 1 -- 172.21.15.82:0/545691778 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae400f740 con 0x559f8a1700e0 2024-04-21T08:48:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 -- 172.21.15.82:0/545691778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 msgr2=0x559f8a1724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 --2- 172.21.15.82:0/545691778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a1724d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009f10 tx=0x7f6ae4009770 comp rx=0 tx=0).stop 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 -- 172.21.15.82:0/545691778 shutdown_connections 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 --2- 172.21.15.82:0/545691778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a1724d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 --2- 172.21.15.82:0/545691778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a166b60 0x559f8a166fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 --2- 172.21.15.82:0/545691778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 0x559f8a166620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 -- 172.21.15.82:0/545691778 >> 172.21.15.82:0/545691778 conn(0x559f8a12dc90 msgr2=0x559f8a161850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 -- 172.21.15.82:0/545691778 shutdown_connections 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 -- 172.21.15.82:0/545691778 wait complete. 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 -- start start 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 Processor -- start 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a166b60 0x559f8a16fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a1678d0 0x559f8a16ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a16d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559f8a172e10 con 0x559f8a1700e0 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559f8a053eb0 con 0x559f8a1678d0 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f8a172f90 con 0x559f8a166b60 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.097+0000 7f6af800d740 10 client.0 _ll_register_callbacks cb 0x559f8a080b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a16d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a16d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59870/0 (socket says 172.21.15.82:59870) 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6aee7fc640 1 -- 172.21.15.82:0/2579640285 learned_addr learned my addr 172.21.15.82:0/2579640285 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a166b60 0x559f8a16fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6aeeffd640 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a1678d0 0x559f8a16ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6aee7fc640 1 -- 172.21.15.82:0/2579640285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a166b60 msgr2=0x559f8a16fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a166b60 0x559f8a16fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6aee7fc640 1 -- 172.21.15.82:0/2579640285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a1678d0 msgr2=0x559f8a16ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a1678d0 0x559f8a16ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6aee7fc640 1 -- 172.21.15.82:0/2579640285 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f89dde210 con 0x559f8a1700e0 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a16d460 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400f420 tx=0x7f6ae4003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd3628cf68b719c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6ad2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6ad2ff5640 1 -- 172.21.15.82:0/2579640285 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4003b30 con 0x559f8a1700e0 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6ad2ff5640 1 -- 172.21.15.82:0/2579640285 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6ae4005050 con 0x559f8a1700e0 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6af800d740 1 -- 172.21.15.82:0/2579640285 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae40041d0 con 0x559f8a1700e0 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6ad2ff5640 1 -- 172.21.15.82:0/2579640285 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ae4030050 con 0x559f8a1700e0 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6af800d740 10 client.16440 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.098+0000 7f6af800d740 1 -- 172.21.15.82:0/2579640285 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f89eaa620 con 0x559f8a1700e0 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.099+0000 7f6ad2ff5640 1 -- 172.21.15.82:0/2579640285 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae40041d0 con 0x559f8a1700e0 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.099+0000 7f6ad2ff5640 1 -- 172.21.15.82:0/2579640285 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f6ae4004d00 con 0x559f8a1700e0 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.099+0000 7f6af800d740 10 client.16440 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.099+0000 7f6af800d740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.099+0000 7f6af800d740 20 client.16440 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.099+0000 7f6af800d740 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f8a17fff0 0x559f8a1823e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.099+0000 7f6af800d740 4 client.16440 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.099+0000 7f6af800d740 1 -- 172.21.15.82:0/2579640285 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f8a0df880 con 0x559f8a17fff0 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.099+0000 7f6aeeffd640 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f8a17fff0 0x559f8a1823e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.100+0000 7f6ad2ff5640 10 client.16440 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.100+0000 7f6aeeffd640 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f8a17fff0 0x559f8a1823e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6326f9f21b2cc3e7 server_cookie=a490523b6bceac0b in_seq=0 out_seq=0 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.100+0000 7f6ad2ff5640 1 -- 172.21.15.82:0/2579640285 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f8a0df880 con 0x559f8a17fff0 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.100+0000 7f6ad2ff5640 10 client.16440 handle_command_reply: tid=0 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.100+0000 7f6af800d740 1 client.16440 shutdown 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.101+0000 7f6af800d740 1 -- 172.21.15.82:0/2579640285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 msgr2=0x559f8a16d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.101+0000 7f6af800d740 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a16d460 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400f420 tx=0x7f6ae4003f20 comp rx=0 tx=0).stop 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.101+0000 7f6af800d740 1 -- 172.21.15.82:0/2579640285 shutdown_connections 2024-04-21T08:48:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.101+0000 7f6af800d740 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f8a17fff0 0x559f8a1823e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.101+0000 7f6af800d740 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a16d460 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.101+0000 7f6af800d740 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a1678d0 0x559f8a16ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.101+0000 7f6ad2ff5640 0 client.16440 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.101+0000 7f6af800d740 1 --2- 172.21.15.82:0/2579640285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a166b60 0x559f8a16fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.101+0000 7f6af800d740 1 -- 172.21.15.82:0/2579640285 >> 172.21.15.82:0/2579640285 conn(0x559f8a12dc90 msgr2=0x559f8a1635a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.101+0000 7f6af800d740 1 -- 172.21.15.82:0/2579640285 shutdown_connections 2024-04-21T08:48:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.101+0000 7f6af800d740 1 -- 172.21.15.82:0/2579640285 wait complete. 2024-04-21T08:48:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.101+0000 7f6af800d740 20 client.16440 trim_cache size 0 max 16384 2024-04-21T08:48:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.118+0000 7f6af800d740 1 Processor -- start 2024-04-21T08:48:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.118+0000 7f6af800d740 1 -- start start 2024-04-21T08:48:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.118+0000 7f6af800d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a166b60 0x559f8a17bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.118+0000 7f6af800d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 0x559f8a16be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.118+0000 7f6af800d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a1700e0 0x559f8a16f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.118+0000 7f6af800d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b1570 con 0x559f8a166b60 2024-04-21T08:48:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.118+0000 7f6af800d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b31a0 con 0x559f8a1700e0 2024-04-21T08:48:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.118+0000 7f6af800d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b1c40 con 0x559f8a1678d0 2024-04-21T08:48:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.119+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a166b60 0x559f8a17bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.119+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a166b60 0x559f8a17bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59878/0 (socket says 172.21.15.82:59878) 2024-04-21T08:48:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.119+0000 7f6aee7fc640 1 -- 172.21.15.82:0/2783463635 learned_addr learned my addr 172.21.15.82:0/2783463635 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.119+0000 7f6aeeffd640 1 --2- 172.21.15.82:0/2783463635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 0x559f8a16be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.119+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/2783463635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a1700e0 0x559f8a16f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.119+0000 7f6aee7fc640 1 -- 172.21.15.82:0/2783463635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 msgr2=0x559f8a16be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.119+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/2783463635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 0x559f8a16be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.119+0000 7f6aee7fc640 1 -- 172.21.15.82:0/2783463635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a1700e0 msgr2=0x559f8a16f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.119+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/2783463635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a1700e0 0x559f8a16f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.119+0000 7f6aee7fc640 1 -- 172.21.15.82:0/2783463635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f8a165e30 con 0x559f8a166b60 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.119+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/2783463635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a166b60 0x559f8a17bd40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009250 tx=0x7f6ae4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca549cc9c8f118c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.120+0000 7f6ad4ff9640 1 -- 172.21.15.82:0/2783463635 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4003f90 con 0x559f8a166b60 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.120+0000 7f6ad4ff9640 1 -- 172.21.15.82:0/2783463635 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae4003900 con 0x559f8a166b60 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.120+0000 7f6ad4ff9640 1 -- 172.21.15.82:0/2783463635 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4005600 con 0x559f8a166b60 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.120+0000 7f6af800d740 1 -- 172.21.15.82:0/2783463635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a166b60 msgr2=0x559f8a17bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.120+0000 7f6af800d740 1 --2- 172.21.15.82:0/2783463635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a166b60 0x559f8a17bd40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009250 tx=0x7f6ae4005f90 comp rx=0 tx=0).stop 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.120+0000 7f6af800d740 1 -- 172.21.15.82:0/2783463635 shutdown_connections 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.120+0000 7f6af800d740 1 --2- 172.21.15.82:0/2783463635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a1700e0 0x559f8a16f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.120+0000 7f6af800d740 1 --2- 172.21.15.82:0/2783463635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 0x559f8a16be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.120+0000 7f6af800d740 1 --2- 172.21.15.82:0/2783463635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a166b60 0x559f8a17bd40 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.120+0000 7f6af800d740 1 -- 172.21.15.82:0/2783463635 >> 172.21.15.82:0/2783463635 conn(0x559f8a12dc90 msgr2=0x559f8a16b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.120+0000 7f6af800d740 1 -- 172.21.15.82:0/2783463635 shutdown_connections 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.120+0000 7f6af800d740 1 -- 172.21.15.82:0/2783463635 wait complete. 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6af800d740 1 -- start start 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6af800d740 1 Processor -- start 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6af800d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a166b60 0x559f8a170580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6af800d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 0x559f8a170ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6af800d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a177250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6af800d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b2aa0 con 0x559f8a1700e0 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6af800d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b23d0 con 0x559f8a166b60 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6af800d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01af550 con 0x559f8a1678d0 2024-04-21T08:48:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6af800d740 10 client.0 _ll_register_callbacks cb 0x7f6ae0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a177250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6aee7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a177250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59884/0 (socket says 172.21.15.82:59884) 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6aee7fc640 1 -- 172.21.15.82:0/2643815922 learned_addr learned my addr 172.21.15.82:0/2643815922 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a166b60 0x559f8a170580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6aeeffd640 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 0x559f8a170ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6aee7fc640 1 -- 172.21.15.82:0/2643815922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 msgr2=0x559f8a170ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 0x559f8a170ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6aee7fc640 1 -- 172.21.15.82:0/2643815922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a166b60 msgr2=0x559f8a170580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a166b60 0x559f8a170580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.121+0000 7f6aee7fc640 1 -- 172.21.15.82:0/2643815922 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8a177790 con 0x559f8a1700e0 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6aee7fc640 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a177250 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4011de0 tx=0x7f6ae4020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3e843e11c1b969 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6ad2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6ad2ff5640 1 -- 172.21.15.82:0/2643815922 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40041a0 con 0x559f8a1700e0 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6ad2ff5640 1 -- 172.21.15.82:0/2643815922 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6ae4003ab0 con 0x559f8a1700e0 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6af800d740 1 -- 172.21.15.82:0/2643815922 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae4003900 con 0x559f8a1700e0 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6ad2ff5640 1 -- 172.21.15.82:0/2643815922 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ae4047050 con 0x559f8a1700e0 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6af800d740 10 client.16446 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6af800d740 1 -- 172.21.15.82:0/2643815922 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f8a1789d0 con 0x559f8a1700e0 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6ad2ff5640 1 -- 172.21.15.82:0/2643815922 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae4003900 con 0x559f8a1700e0 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6ad2ff5640 1 -- 172.21.15.82:0/2643815922 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f6ae401a070 con 0x559f8a1700e0 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6af800d740 10 client.16446 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6af800d740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6af800d740 20 client.16446 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6af800d740 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f8a1e6610 0x559f8a1e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6af800d740 4 client.16446 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6af800d740 1 -- 172.21.15.82:0/2643815922 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f8a06b3d0 con 0x559f8a1e6610 2024-04-21T08:48:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.122+0000 7f6aeeffd640 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f8a1e6610 0x559f8a1e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.123+0000 7f6ad2ff5640 10 client.16446 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.123+0000 7f6aeeffd640 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f8a1e6610 0x559f8a1e8a30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711701bf768d3c1a server_cookie=9cf96da103358019 in_seq=0 out_seq=0 2024-04-21T08:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.257+0000 7f6ad2ff5640 1 -- 172.21.15.82:0/2643815922 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f8a06b3d0 con 0x559f8a1e6610 2024-04-21T08:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.257+0000 7f6ad2ff5640 10 client.16446 handle_command_reply: tid=0 2024-04-21T08:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.257+0000 7f6af800d740 1 client.16446 shutdown 2024-04-21T08:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.258+0000 7f6af800d740 1 -- 172.21.15.82:0/2643815922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 msgr2=0x559f8a177250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.258+0000 7f6af800d740 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a177250 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4011de0 tx=0x7f6ae4020040 comp rx=0 tx=0).stop 2024-04-21T08:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.258+0000 7f6af800d740 1 -- 172.21.15.82:0/2643815922 shutdown_connections 2024-04-21T08:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.258+0000 7f6af800d740 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f8a1e6610 0x559f8a1e8a30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.258+0000 7f6af800d740 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f8a1700e0 0x559f8a177250 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.258+0000 7f6af800d740 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f8a1678d0 0x559f8a170ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.258+0000 7f6ad2ff5640 0 client.16446 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.258+0000 7f6af800d740 1 --2- 172.21.15.82:0/2643815922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f8a166b60 0x559f8a170580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.258+0000 7f6af800d740 1 -- 172.21.15.82:0/2643815922 >> 172.21.15.82:0/2643815922 conn(0x559f8a12dc90 msgr2=0x559f8a162bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.258+0000 7f6af800d740 1 -- 172.21.15.82:0/2643815922 shutdown_connections 2024-04-21T08:48:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.258+0000 7f6af800d740 1 -- 172.21.15.82:0/2643815922 wait complete. 2024-04-21T08:48:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.258+0000 7f6af800d740 20 client.16446 trim_cache size 0 max 16384 2024-04-21T08:48:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:48:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:48:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:48:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:48:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.259+0000 7f6af5340640 1 -- 172.21.15.82:0/1276254308 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6ad80a3ff0 msgr2=0x7f6ad80a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.259+0000 7f6af5340640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6ad80a3ff0 0x7f6ad80a64b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80f02e0 tx=0x7f6ae803ac20 comp rx=0 tx=0).stop 2024-04-21T08:48:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.259+0000 7f6af5340640 1 -- 172.21.15.82:0/1276254308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af010c520 msgr2=0x7f6af019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.259+0000 7f6af5340640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af010c520 0x7f6af019a660 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8005e30 tx=0x7f6ae8035d50 comp rx=0 tx=0).stop 2024-04-21T08:48:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.259+0000 7f6af5340640 1 -- 172.21.15.82:0/1276254308 shutdown_connections 2024-04-21T08:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.259+0000 7f6af5340640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6ad80a3ff0 0x7f6ad80a64b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.259+0000 7f6af5340640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af010c520 0x7f6af019a660 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.259+0000 7f6af5340640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6af0103b00 0x7f6af01a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.259+0000 7f6af5340640 1 --2- 172.21.15.82:0/1276254308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6af01031e0 0x7f6af01a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.259+0000 7f6af5340640 1 -- 172.21.15.82:0/1276254308 >> 172.21.15.82:0/1276254308 conn(0x7f6af0077fd0 msgr2=0x7f6af0108390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.260+0000 7f6af5340640 1 -- 172.21.15.82:0/1276254308 shutdown_connections 2024-04-21T08:48:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:29.260+0000 7f6af5340640 1 -- 172.21.15.82:0/1276254308 wait complete. 2024-04-21T08:48:29.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:29 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:48:29.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:29 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:48:29.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:29 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:48:29.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:29 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:48:29.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:29 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:48:29.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:29 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:48:29.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:29 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:48:29.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:29 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:48:29.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:29 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:48:30.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:30 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:48:30.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:30 smithi082 ceph-mon[27967]: pgmap v47: 129 pgs: 129 active+clean; 67 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.7 MiB/s wr, 342 op/s 2024-04-21T08:48:30.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:30 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:48:30.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:30 smithi106 ceph-mon[31072]: pgmap v47: 129 pgs: 129 active+clean; 67 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.7 MiB/s wr, 342 op/s 2024-04-21T08:48:30.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:30 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:48:30.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:30 smithi134 ceph-mon[31772]: pgmap v47: 129 pgs: 129 active+clean; 67 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.7 MiB/s wr, 342 op/s 2024-04-21T08:48:32.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:32 smithi082 ceph-mon[27967]: pgmap v48: 129 pgs: 129 active+clean; 67 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.6 MiB/s wr, 310 op/s 2024-04-21T08:48:32.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:32 smithi106 ceph-mon[31072]: pgmap v48: 129 pgs: 129 active+clean; 67 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.6 MiB/s wr, 310 op/s 2024-04-21T08:48:32.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:32 smithi134 ceph-mon[31772]: pgmap v48: 129 pgs: 129 active+clean; 67 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.6 MiB/s wr, 310 op/s 2024-04-21T08:48:34.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:34 smithi082 ceph-mon[27967]: pgmap v49: 129 pgs: 129 active+clean; 109 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.8 MiB/s wr, 840 op/s 2024-04-21T08:48:34.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:34 smithi082 ceph-mon[27967]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-21T08:48:34.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:34 smithi106 ceph-mon[31072]: pgmap v49: 129 pgs: 129 active+clean; 109 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.8 MiB/s wr, 840 op/s 2024-04-21T08:48:34.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:34 smithi106 ceph-mon[31072]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-21T08:48:34.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:34 smithi134 ceph-mon[31772]: pgmap v49: 129 pgs: 129 active+clean; 109 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.8 MiB/s wr, 840 op/s 2024-04-21T08:48:34.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:34 smithi134 ceph-mon[31772]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-21T08:48:36.222 INFO:tasks.workunit.client.0.smithi082.stdout: 32 422424 416 252973 279 225585 1090 2024-04-21T08:48:36.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:36 smithi082 ceph-mon[27967]: pgmap v50: 129 pgs: 129 active+clean; 109 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 741 op/s 2024-04-21T08:48:36.377 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:36 smithi106 ceph-mon[31072]: pgmap v50: 129 pgs: 129 active+clean; 109 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 741 op/s 2024-04-21T08:48:36.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:36 smithi134 ceph-mon[31772]: pgmap v50: 129 pgs: 129 active+clean; 109 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 741 op/s 2024-04-21T08:48:38.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:38 smithi106 ceph-mon[31072]: pgmap v51: 129 pgs: 129 active+clean; 110 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 752 op/s 2024-04-21T08:48:38.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:38 smithi134 ceph-mon[31772]: pgmap v51: 129 pgs: 129 active+clean; 110 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 752 op/s 2024-04-21T08:48:38.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:38 smithi082 ceph-mon[27967]: pgmap v51: 129 pgs: 129 active+clean; 110 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 752 op/s 2024-04-21T08:48:40.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:40 smithi106 ceph-mon[31072]: pgmap v52: 129 pgs: 129 active+clean; 128 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 889 op/s 2024-04-21T08:48:40.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:40 smithi134 ceph-mon[31772]: pgmap v52: 129 pgs: 129 active+clean; 128 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 889 op/s 2024-04-21T08:48:40.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:40 smithi082 ceph-mon[27967]: pgmap v52: 129 pgs: 129 active+clean; 128 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 889 op/s 2024-04-21T08:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.360+0000 7f0a295fb640 1 -- 172.21.15.82:0/3653916771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a24102000 msgr2=0x7f0a241023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.360+0000 7f0a295fb640 1 --2- 172.21.15.82:0/3653916771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a24102000 0x7f0a241023e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0a14009900 tx=0x7f0a1402bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.360+0000 7f0a295fb640 1 -- 172.21.15.82:0/3653916771 shutdown_connections 2024-04-21T08:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.360+0000 7f0a295fb640 1 --2- 172.21.15.82:0/3653916771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a241070d0 0x7f0a2410eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.360+0000 7f0a295fb640 1 --2- 172.21.15.82:0/3653916771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a241029b0 0x7f0a241069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.360+0000 7f0a295fb640 1 --2- 172.21.15.82:0/3653916771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a24102000 0x7f0a241023e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.360+0000 7f0a295fb640 1 -- 172.21.15.82:0/3653916771 >> 172.21.15.82:0/3653916771 conn(0x7f0a240fd900 msgr2=0x7f0a240ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.360+0000 7f0a295fb640 1 -- 172.21.15.82:0/3653916771 shutdown_connections 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.360+0000 7f0a295fb640 1 -- 172.21.15.82:0/3653916771 wait complete. 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a295fb640 1 Processor -- start 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a295fb640 1 -- start start 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a295fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a24102000 0x7f0a241a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a295fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a241070d0 0x7f0a241a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a295fb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a2419a660 0x7f0a2419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a295fb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24112040 con 0x7f0a2419a660 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a295fb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24111ec0 con 0x7f0a24102000 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a295fb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241121c0 con 0x7f0a241070d0 2024-04-21T08:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a22ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a24102000 0x7f0a241a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a2419a660 0x7f0a2419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a22ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a24102000 0x7f0a241a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:58980/0 (socket says 172.21.15.82:58980) 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a22ffd640 1 -- 172.21.15.82:0/6161461 learned_addr learned my addr 172.21.15.82:0/6161461 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a241070d0 0x7f0a241a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a237fe640 1 -- 172.21.15.82:0/6161461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a241070d0 msgr2=0x7f0a241a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a237fe640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a241070d0 0x7f0a241a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a237fe640 1 -- 172.21.15.82:0/6161461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a24102000 msgr2=0x7f0a241a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a237fe640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a24102000 0x7f0a241a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a237fe640 1 -- 172.21.15.82:0/6161461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2419b260 con 0x7f0a2419a660 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a241070d0 0x7f0a241a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.361+0000 7f0a22ffd640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a24102000 0x7f0a241a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.362+0000 7f0a237fe640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a2419a660 0x7f0a2419aae0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009fd0 tx=0x7f0a1c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.362+0000 7f0a20ff9640 1 -- 172.21.15.82:0/6161461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c00ce20 con 0x7f0a2419a660 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.362+0000 7f0a20ff9640 1 -- 172.21.15.82:0/6161461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a1c005020 con 0x7f0a2419a660 2024-04-21T08:48:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.362+0000 7f0a295fb640 1 -- 172.21.15.82:0/6161461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a241a7340 con 0x7f0a2419a660 2024-04-21T08:48:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.362+0000 7f0a20ff9640 1 -- 172.21.15.82:0/6161461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c010690 con 0x7f0a2419a660 2024-04-21T08:48:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.362+0000 7f0a295fb640 1 -- 172.21.15.82:0/6161461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a241a77a0 con 0x7f0a2419a660 2024-04-21T08:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.365+0000 7f0a20ff9640 1 -- 172.21.15.82:0/6161461 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0a1c0054d0 con 0x7f0a2419a660 2024-04-21T08:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.365+0000 7f0a2c1e1740 1 Processor -- start 2024-04-21T08:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.365+0000 7f0a20ff9640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0a100a3d90 0x7f0a100a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.365+0000 7f0a2c1e1740 1 -- start start 2024-04-21T08:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.365+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0a100a3d90 0x7f0a100a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a20ff9640 1 -- 172.21.15.82:0/6161461 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f0a1c0d1270 con 0x7f0a2419a660 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a2c1e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f967ac0 0x55853f966810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a2c1e1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f966d50 0x55853f9671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a2c1e1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 0x55853f9726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a2c1e1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55853f8cb8d0 con 0x55853f9702d0 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a2c1e1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55853f868e80 con 0x55853f966d50 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a2c1e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55853f8c7890 con 0x55853f967ac0 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 0x55853f9726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 0x55853f9726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58530/0 (socket says 172.21.15.82:58530) 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a237fe640 1 -- 172.21.15.82:0/3288206632 learned_addr learned my addr 172.21.15.82:0/3288206632 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/3288206632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f966d50 0x55853f9671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a22ffd640 1 --2- 172.21.15.82:0/3288206632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f967ac0 0x55853f966810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0a100a3d90 0x7f0a100a6250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0a2419beb0 tx=0x7f0a0c009f30 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a237fe640 1 -- 172.21.15.82:0/3288206632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f967ac0 msgr2=0x55853f966810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a237fe640 1 --2- 172.21.15.82:0/3288206632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f967ac0 0x55853f966810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a237fe640 1 -- 172.21.15.82:0/3288206632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f966d50 msgr2=0x55853f9671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a237fe640 1 --2- 172.21.15.82:0/3288206632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f966d50 0x55853f9671d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a237fe640 1 -- 172.21.15.82:0/3288206632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55853f88a320 con 0x55853f9702d0 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/3288206632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f966d50 0x55853f9671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a237fe640 1 --2- 172.21.15.82:0/3288206632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 0x55853f9726c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c099820 tx=0x7f0a1c09cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a3782d49c1c37d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.366+0000 7f0a197fa640 1 -- 172.21.15.82:0/3288206632 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c0b5020 con 0x55853f9702d0 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.367+0000 7f0a197fa640 1 -- 172.21.15.82:0/3288206632 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a1c09f070 con 0x55853f9702d0 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.367+0000 7f0a197fa640 1 -- 172.21.15.82:0/3288206632 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c0b63d0 con 0x55853f9702d0 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.367+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/3288206632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 msgr2=0x55853f9726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.367+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/3288206632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 0x55853f9726c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c099820 tx=0x7f0a1c09cf10 comp rx=0 tx=0).stop 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.367+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/3288206632 shutdown_connections 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.367+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/3288206632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 0x55853f9726c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.367+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/3288206632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f966d50 0x55853f9671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.367+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/3288206632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f967ac0 0x55853f966810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.367+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/3288206632 >> 172.21.15.82:0/3288206632 conn(0x55853f92dc90 msgr2=0x55853f961a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.367+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/3288206632 shutdown_connections 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.367+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/3288206632 wait complete. 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a2c1e1740 1 -- start start 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a2c1e1740 1 Processor -- start 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a2c1e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 0x55853f96fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a2c1e1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 0x55853f96cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a2c1e1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f9702d0 0x55853f96d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a2c1e1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55853f973000 con 0x55853f967ac0 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a2c1e1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55853f853eb0 con 0x55853f9702d0 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a2c1e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55853f973180 con 0x55853f966d50 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a2c1e1740 10 client.0 _ll_register_callbacks cb 0x55853f880b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 0x55853f96cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 0x55853f96cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58538/0 (socket says 172.21.15.82:58538) 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a1bfff640 1 -- 172.21.15.82:0/225473781 learned_addr learned my addr 172.21.15.82:0/225473781 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a22ffd640 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 0x55853f96fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a237fe640 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f9702d0 0x55853f96d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a1bfff640 1 -- 172.21.15.82:0/225473781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 msgr2=0x55853f96fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 0x55853f96fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a1bfff640 1 -- 172.21.15.82:0/225473781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f9702d0 msgr2=0x55853f96d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f9702d0 0x55853f96d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.368+0000 7f0a1bfff640 1 -- 172.21.15.82:0/225473781 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55853f5de210 con 0x55853f967ac0 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 0x55853f96cfc0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0042c0 tx=0x7f0a0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c361b8e97be248e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a0a7fc640 1 -- 172.21.15.82:0/225473781 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c012640 con 0x55853f967ac0 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a0a7fc640 1 -- 172.21.15.82:0/225473781 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f0a0c0127e0 con 0x55853f967ac0 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/225473781 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a1c09f070 con 0x55853f967ac0 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a0a7fc640 1 -- 172.21.15.82:0/225473781 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0a0c02f050 con 0x55853f967ac0 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a2c1e1740 10 client.16458 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/225473781 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55853f6aa620 con 0x55853f967ac0 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a0a7fc640 1 -- 172.21.15.82:0/225473781 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a1c09f070 con 0x55853f967ac0 2024-04-21T08:48:42.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a0a7fc640 1 -- 172.21.15.82:0/225473781 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f0a0c00f040 con 0x55853f967ac0 2024-04-21T08:48:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a2c1e1740 10 client.16458 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a2c1e1740 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a2c1e1740 20 client.16458 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55853f980300 0x55853f982710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a2c1e1740 4 client.16458 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/225473781 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55853f8df880 con 0x55853f980300 2024-04-21T08:48:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.369+0000 7f0a22ffd640 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55853f980300 0x55853f982710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.370+0000 7f0a22ffd640 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55853f980300 0x55853f982710 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b651d874993d2e server_cookie=3b21e95c72388a31 in_seq=0 out_seq=0 2024-04-21T08:48:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.370+0000 7f0a0a7fc640 10 client.16458 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.371+0000 7f0a0a7fc640 1 -- 172.21.15.82:0/225473781 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55853f8df880 con 0x55853f980300 2024-04-21T08:48:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.371+0000 7f0a0a7fc640 10 client.16458 handle_command_reply: tid=0 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.371+0000 7f0a2c1e1740 1 client.16458 shutdown 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.371+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/225473781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 msgr2=0x55853f96cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.371+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 0x55853f96cfc0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0042c0 tx=0x7f0a0c004730 comp rx=0 tx=0).stop 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.371+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/225473781 shutdown_connections 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.371+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55853f980300 0x55853f982710 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.371+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f9702d0 0x55853f96d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.371+0000 7f0a0a7fc640 0 client.16458 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.371+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 0x55853f96cfc0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.372+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/225473781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 0x55853f96fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.372+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/225473781 >> 172.21.15.82:0/225473781 conn(0x55853f92dc90 msgr2=0x55853f963790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.372+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/225473781 shutdown_connections 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.372+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/225473781 wait complete. 2024-04-21T08:48:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.372+0000 7f0a2c1e1740 20 client.16458 trim_cache size 0 max 16384 2024-04-21T08:48:42.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:42 smithi106 ceph-mon[31072]: pgmap v53: 129 pgs: 129 active+clean; 128 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.0 MiB/s wr, 684 op/s 2024-04-21T08:48:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.404+0000 7f0a2c1e1740 1 Processor -- start 2024-04-21T08:48:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.404+0000 7f0a2c1e1740 1 -- start start 2024-04-21T08:48:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.404+0000 7f0a2c1e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 0x55853f960960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.404+0000 7f0a2c1e1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f967ac0 0x55853f96f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.404+0000 7f0a2c1e1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 0x55853f96fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.404+0000 7f0a2c1e1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241b1510 con 0x55853f9702d0 2024-04-21T08:48:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.404+0000 7f0a2c1e1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241b3140 con 0x55853f967ac0 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a2c1e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241b1be0 con 0x55853f966d50 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 0x55853f96fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 0x55853f96fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58544/0 (socket says 172.21.15.82:58544) 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a237fe640 1 -- 172.21.15.82:0/4036150880 learned_addr learned my addr 172.21.15.82:0/4036150880 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/4036150880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f967ac0 0x55853f96f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a22ffd640 1 --2- 172.21.15.82:0/4036150880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 0x55853f960960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a237fe640 1 -- 172.21.15.82:0/4036150880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 msgr2=0x55853f960960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a237fe640 1 --2- 172.21.15.82:0/4036150880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 0x55853f960960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a237fe640 1 -- 172.21.15.82:0/4036150880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f967ac0 msgr2=0x55853f96f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a237fe640 1 --2- 172.21.15.82:0/4036150880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f967ac0 0x55853f96f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a237fe640 1 -- 172.21.15.82:0/4036150880 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55853f95d300 con 0x55853f9702d0 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a237fe640 1 --2- 172.21.15.82:0/4036150880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 0x55853f96fc30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00c760 tx=0x7f0a1c009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1082ad452470b788 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a18ff9640 1 -- 172.21.15.82:0/4036150880 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c0ba070 con 0x55853f9702d0 2024-04-21T08:48:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a18ff9640 1 -- 172.21.15.82:0/4036150880 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a1c09f040 con 0x55853f9702d0 2024-04-21T08:48:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.405+0000 7f0a18ff9640 1 -- 172.21.15.82:0/4036150880 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c0b5040 con 0x55853f9702d0 2024-04-21T08:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.406+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/4036150880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 msgr2=0x55853f96fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.406+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/4036150880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 0x55853f96fc30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00c760 tx=0x7f0a1c009590 comp rx=0 tx=0).stop 2024-04-21T08:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.406+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/4036150880 shutdown_connections 2024-04-21T08:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.406+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/4036150880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f9702d0 0x55853f96fc30 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.406+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/4036150880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f967ac0 0x55853f96f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.406+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/4036150880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 0x55853f960960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.406+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/4036150880 >> 172.21.15.82:0/4036150880 conn(0x55853f92dc90 msgr2=0x55853f960250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.406+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/4036150880 shutdown_connections 2024-04-21T08:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.406+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/4036150880 wait complete. 2024-04-21T08:48:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.406+0000 7f0a2c1e1740 1 -- start start 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a2c1e1740 1 Processor -- start 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a2c1e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 0x55853f96a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a2c1e1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 0x55853f96a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a2c1e1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f9702d0 0x55853f9e5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a2c1e1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241b2a40 con 0x55853f967ac0 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a2c1e1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241b2370 con 0x55853f9702d0 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a2c1e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241af4f0 con 0x55853f966d50 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a2c1e1740 10 client.0 _ll_register_callbacks cb 0x7f0a0c010ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 0x55853f96a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 0x55853f96a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58558/0 (socket says 172.21.15.82:58558) 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a1bfff640 1 -- 172.21.15.82:0/1762932976 learned_addr learned my addr 172.21.15.82:0/1762932976 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a237fe640 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f9702d0 0x55853f9e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a22ffd640 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 0x55853f96a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a1bfff640 1 -- 172.21.15.82:0/1762932976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 msgr2=0x55853f96a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 0x55853f96a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a1bfff640 1 -- 172.21.15.82:0/1762932976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f9702d0 msgr2=0x55853f9e5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f9702d0 0x55853f9e5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a1bfff640 1 -- 172.21.15.82:0/1762932976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55853f9e5dd0 con 0x55853f967ac0 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 0x55853f96a870 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55853f96e590 tx=0x7f0a0c010920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc0501d02976625 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.407+0000 7f0a0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a0a7fc640 1 -- 172.21.15.82:0/1762932976 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c01f630 con 0x55853f967ac0 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a0a7fc640 1 -- 172.21.15.82:0/1762932976 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f0a0c01f7d0 con 0x55853f967ac0 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/1762932976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a1c09f040 con 0x55853f967ac0 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a0a7fc640 1 -- 172.21.15.82:0/1762932976 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0a0c01f9d0 con 0x55853f967ac0 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a2c1e1740 10 client.16464 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/1762932976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55853f9e6fe0 con 0x55853f967ac0 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a0a7fc640 1 -- 172.21.15.82:0/1762932976 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a1c09f040 con 0x55853f967ac0 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a0a7fc640 1 -- 172.21.15.82:0/1762932976 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f0a0c012440 con 0x55853f967ac0 2024-04-21T08:48:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a2c1e1740 10 client.16464 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a2c1e1740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a2c1e1740 20 client.16464 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55853f980300 0x55853f9e9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a2c1e1740 4 client.16464 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.408+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/1762932976 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55853f86b3d0 con 0x55853f980300 2024-04-21T08:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.409+0000 7f0a22ffd640 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55853f980300 0x55853f9e9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.409+0000 7f0a22ffd640 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55853f980300 0x55853f9e9760 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ff4d214c44744a server_cookie=770e6b99cb3caae2 in_seq=0 out_seq=0 2024-04-21T08:48:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.409+0000 7f0a0a7fc640 10 client.16464 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:42.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:42 smithi134 ceph-mon[31772]: pgmap v53: 129 pgs: 129 active+clean; 128 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.0 MiB/s wr, 684 op/s 2024-04-21T08:48:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a0a7fc640 1 -- 172.21.15.82:0/1762932976 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55853f86b3d0 con 0x55853f980300 2024-04-21T08:48:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a0a7fc640 10 client.16464 handle_command_reply: tid=0 2024-04-21T08:48:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a2c1e1740 1 client.16464 shutdown 2024-04-21T08:48:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/1762932976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 msgr2=0x55853f96a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 0x55853f96a870 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55853f96e590 tx=0x7f0a0c010920 comp rx=0 tx=0).stop 2024-04-21T08:48:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/1762932976 shutdown_connections 2024-04-21T08:48:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55853f980300 0x55853f9e9760 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55853f9702d0 0x55853f9e5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a0a7fc640 0 client.16464 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55853f967ac0 0x55853f96a870 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a2c1e1740 1 --2- 172.21.15.82:0/1762932976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55853f966d50 0x55853f96a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/1762932976 >> 172.21.15.82:0/1762932976 conn(0x55853f92dc90 msgr2=0x55853f963660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/1762932976 shutdown_connections 2024-04-21T08:48:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a2c1e1740 1 -- 172.21.15.82:0/1762932976 wait complete. 2024-04-21T08:48:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.585+0000 7f0a2c1e1740 20 client.16464 trim_cache size 0 max 16384 2024-04-21T08:48:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:48:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:48:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:48:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:48:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.586+0000 7f0a295fb640 1 -- 172.21.15.82:0/6161461 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0a100a3d90 msgr2=0x7f0a100a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.586+0000 7f0a295fb640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0a100a3d90 0x7f0a100a6250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0a2419beb0 tx=0x7f0a0c009f30 comp rx=0 tx=0).stop 2024-04-21T08:48:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.586+0000 7f0a295fb640 1 -- 172.21.15.82:0/6161461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a2419a660 msgr2=0x7f0a2419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.586+0000 7f0a295fb640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a2419a660 0x7f0a2419aae0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009fd0 tx=0x7f0a1c00ef10 comp rx=0 tx=0).stop 2024-04-21T08:48:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.586+0000 7f0a295fb640 1 -- 172.21.15.82:0/6161461 shutdown_connections 2024-04-21T08:48:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.586+0000 7f0a295fb640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0a100a3d90 0x7f0a100a6250 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.586+0000 7f0a295fb640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a2419a660 0x7f0a2419aae0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.586+0000 7f0a295fb640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a241070d0 0x7f0a241a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.586+0000 7f0a295fb640 1 --2- 172.21.15.82:0/6161461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a24102000 0x7f0a241a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.586+0000 7f0a295fb640 1 -- 172.21.15.82:0/6161461 >> 172.21.15.82:0/6161461 conn(0x7f0a240fd900 msgr2=0x7f0a240ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.586+0000 7f0a295fb640 1 -- 172.21.15.82:0/6161461 shutdown_connections 2024-04-21T08:48:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:42.586+0000 7f0a295fb640 1 -- 172.21.15.82:0/6161461 wait complete. 2024-04-21T08:48:42.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:42 smithi082 ceph-mon[27967]: pgmap v53: 129 pgs: 129 active+clean; 128 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.0 MiB/s wr, 684 op/s 2024-04-21T08:48:44.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:44 smithi106 ceph-mon[31072]: pgmap v54: 129 pgs: 129 active+clean; 135 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.9 MiB/s wr, 802 op/s 2024-04-21T08:48:44.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:44 smithi134 ceph-mon[31772]: pgmap v54: 129 pgs: 129 active+clean; 135 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.9 MiB/s wr, 802 op/s 2024-04-21T08:48:44.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:44.504 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:44.525 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:44 smithi082 ceph-mon[27967]: pgmap v54: 129 pgs: 129 active+clean; 135 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.9 MiB/s wr, 802 op/s 2024-04-21T08:48:44.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.622+0000 7f3968afd640 1 -- 172.21.15.82:0/2252533490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3964101a30 msgr2=0x7f3964105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:44.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.622+0000 7f3968afd640 1 --2- 172.21.15.82:0/2252533490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3964101a30 0x7f3964105a70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f394c009900 tx=0x7f394c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:44.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.622+0000 7f3968afd640 1 -- 172.21.15.82:0/2252533490 shutdown_connections 2024-04-21T08:48:44.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.622+0000 7f3968afd640 1 --2- 172.21.15.82:0/2252533490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3964106150 0x7f396410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.622+0000 7f3968afd640 1 --2- 172.21.15.82:0/2252533490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3964101a30 0x7f3964105a70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.622+0000 7f3968afd640 1 --2- 172.21.15.82:0/2252533490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3964101080 0x7f3964101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.622+0000 7f3968afd640 1 -- 172.21.15.82:0/2252533490 >> 172.21.15.82:0/2252533490 conn(0x7f3964078000 msgr2=0x7f3964078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:44.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.622+0000 7f3968afd640 1 -- 172.21.15.82:0/2252533490 shutdown_connections 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.622+0000 7f3968afd640 1 -- 172.21.15.82:0/2252533490 wait complete. 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f3968afd640 1 Processor -- start 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f3968afd640 1 -- start start 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f3968afd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3964101080 0x7f39641986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f3968afd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3964106150 0x7f3964198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f3968afd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f396419cfb0 0x7f396419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f3968afd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f39641121d0 con 0x7f3964101080 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f3968afd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3964112050 con 0x7f3964106150 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f3968afd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3964112350 con 0x7f396419cfb0 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f396259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3964101080 0x7f39641986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f396259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3964101080 0x7f39641986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34512/0 (socket says 172.21.15.82:34512) 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f396259c640 1 -- 172.21.15.82:0/284483234 learned_addr learned my addr 172.21.15.82:0/284483234 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f395bfff640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3964106150 0x7f3964198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f3962d9d640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f396419cfb0 0x7f396419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f396259c640 1 -- 172.21.15.82:0/284483234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f396419cfb0 msgr2=0x7f396419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f396259c640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f396419cfb0 0x7f396419d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f396259c640 1 -- 172.21.15.82:0/284483234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3964106150 msgr2=0x7f3964198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f396259c640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3964106150 0x7f3964198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f396259c640 1 -- 172.21.15.82:0/284483234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396419daa0 con 0x7f3964101080 2024-04-21T08:48:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f396259c640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3964101080 0x7f39641986f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f395400e9f0 tx=0x7f395400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f395b7fe640 1 -- 172.21.15.82:0/284483234 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395400cd80 con 0x7f3964101080 2024-04-21T08:48:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f3968afd640 1 -- 172.21.15.82:0/284483234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396419dd90 con 0x7f3964101080 2024-04-21T08:48:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f395b7fe640 1 -- 172.21.15.82:0/284483234 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39540108e0 con 0x7f3964101080 2024-04-21T08:48:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.623+0000 7f395b7fe640 1 -- 172.21.15.82:0/284483234 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954018980 con 0x7f3964101080 2024-04-21T08:48:44.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.624+0000 7f3968afd640 1 -- 172.21.15.82:0/284483234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39641a55a0 con 0x7f3964101080 2024-04-21T08:48:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.626+0000 7f395b7fe640 1 -- 172.21.15.82:0/284483234 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3954010430 con 0x7f3964101080 2024-04-21T08:48:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.626+0000 7f396b7ca740 1 Processor -- start 2024-04-21T08:48:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.626+0000 7f396b7ca740 1 -- start start 2024-04-21T08:48:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f396b7ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b4e580 0x5608e3b4e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f396b7ca740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b57540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f396b7ca740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b57a80 0x5608e3b59e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f396b7ca740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5608e3ab31e0 con 0x5608e3b4eea0 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f396b7ca740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5608e3a50390 con 0x5608e3b57a80 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f396b7ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5608e3aaf1a0 con 0x5608e3b4e580 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f395b7fe640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f39500a3d20 0x7f39500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f396259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b57540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f396259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b57540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34524/0 (socket says 172.21.15.82:34524) 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f396259c640 1 -- 172.21.15.82:0/2830009670 learned_addr learned my addr 172.21.15.82:0/2830009670 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f395bfff640 1 --2- 172.21.15.82:0/2830009670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b57a80 0x5608e3b59e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f3962d9d640 1 --2- 172.21.15.82:0/2830009670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b4e580 0x5608e3b4e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f3962d9d640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f39500a3d20 0x7f39500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f395bfff640 1 -- 172.21.15.82:0/2830009670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b4e580 msgr2=0x5608e3b4e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f395bfff640 1 --2- 172.21.15.82:0/2830009670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b4e580 0x5608e3b4e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f395bfff640 1 -- 172.21.15.82:0/2830009670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 msgr2=0x5608e3b57540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:44.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f395bfff640 1 --2- 172.21.15.82:0/2830009670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b57540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f395bfff640 1 -- 172.21.15.82:0/2830009670 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e388f7d0 con 0x5608e3b57a80 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f395b7fe640 1 -- 172.21.15.82:0/284483234 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f39540d0f50 con 0x7f3964101080 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f3962d9d640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f39500a3d20 0x7f39500a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f395c005c40 tx=0x7f395c005bd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.627+0000 7f395bfff640 1 --2- 172.21.15.82:0/2830009670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b57a80 0x5608e3b59e70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f394c036040 tx=0x7f394c005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7a6fcbda99d648f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.628+0000 7f3958ff9640 1 -- 172.21.15.82:0/2830009670 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c045020 con 0x5608e3b57a80 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.628+0000 7f3958ff9640 1 -- 172.21.15.82:0/2830009670 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394c02c7e0 con 0x5608e3b57a80 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.628+0000 7f3958ff9640 1 -- 172.21.15.82:0/2830009670 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c0055c0 con 0x5608e3b57a80 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.628+0000 7f396b7ca740 1 -- 172.21.15.82:0/2830009670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b57a80 msgr2=0x5608e3b59e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.628+0000 7f396b7ca740 1 --2- 172.21.15.82:0/2830009670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b57a80 0x5608e3b59e70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f394c036040 tx=0x7f394c005bf0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.628+0000 7f396b7ca740 1 -- 172.21.15.82:0/2830009670 shutdown_connections 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.628+0000 7f396b7ca740 1 --2- 172.21.15.82:0/2830009670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b57a80 0x5608e3b59e70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.628+0000 7f396b7ca740 1 --2- 172.21.15.82:0/2830009670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b57540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.628+0000 7f396b7ca740 1 --2- 172.21.15.82:0/2830009670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b4e580 0x5608e3b4e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.628+0000 7f396b7ca740 1 -- 172.21.15.82:0/2830009670 >> 172.21.15.82:0/2830009670 conn(0x5608e3b48cb0 msgr2=0x5608e3b49090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.628+0000 7f396b7ca740 1 -- 172.21.15.82:0/2830009670 shutdown_connections 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.628+0000 7f396b7ca740 1 -- 172.21.15.82:0/2830009670 wait complete. 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f396b7ca740 1 -- start start 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f396b7ca740 1 Processor -- start 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f396b7ca740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b4e580 0x5608e3b54a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f396b7ca740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b54fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f396b7ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b57a80 0x5608e3b55500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f396b7ca740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5608e3b5a810 con 0x5608e3b4eea0 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f396b7ca740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5608e3a6c2f0 con 0x5608e3b4e580 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f396b7ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5608e3b5a990 con 0x5608e3b57a80 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f396b7ca740 10 client.0 _ll_register_callbacks cb 0x5608e3a79580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f395bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b54fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f396259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b4e580 0x5608e3b54a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f395bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b54fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34538/0 (socket says 172.21.15.82:34538) 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f395bfff640 1 -- 172.21.15.82:0/2681334543 learned_addr learned my addr 172.21.15.82:0/2681334543 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f3962d9d640 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b57a80 0x5608e3b55500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f395bfff640 1 -- 172.21.15.82:0/2681334543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b57a80 msgr2=0x5608e3b55500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f395bfff640 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b57a80 0x5608e3b55500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f395bfff640 1 -- 172.21.15.82:0/2681334543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b4e580 msgr2=0x5608e3b54a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f395bfff640 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b4e580 0x5608e3b54a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f395bfff640 1 -- 172.21.15.82:0/2681334543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e37c53f0 con 0x5608e3b4eea0 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.629+0000 7f3962d9d640 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b57a80 0x5608e3b55500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f395bfff640 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b54fc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f394c02c170 tx=0x7f394c0054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af17919fac577765 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f3949ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f3949ffb640 1 -- 172.21.15.82:0/2681334543 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c0035f0 con 0x5608e3b4eea0 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f3949ffb640 1 -- 172.21.15.82:0/2681334543 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f394c003790 con 0x5608e3b4eea0 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f396b7ca740 1 -- 172.21.15.82:0/2681334543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f394c02c7e0 con 0x5608e3b4eea0 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f3949ffb640 1 -- 172.21.15.82:0/2681334543 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f394c05b050 con 0x5608e3b4eea0 2024-04-21T08:48:44.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f396b7ca740 10 client.16476 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f396b7ca740 1 -- 172.21.15.82:0/2681334543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5608e37850f0 con 0x5608e3b4eea0 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f3949ffb640 1 -- 172.21.15.82:0/2681334543 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394c02c7e0 con 0x5608e3b4eea0 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f3949ffb640 1 -- 172.21.15.82:0/2681334543 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f394c004b50 con 0x5608e3b4eea0 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f396b7ca740 10 client.16476 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f396b7ca740 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f396b7ca740 20 client.16476 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f396b7ca740 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5608e3b67860 0x5608e3b69c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f396b7ca740 4 client.16476 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.630+0000 7f396b7ca740 1 -- 172.21.15.82:0/2681334543 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608e3ac0c00 con 0x5608e3b67860 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.631+0000 7f396259c640 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5608e3b67860 0x5608e3b69c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.631+0000 7f3949ffb640 10 client.16476 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.631+0000 7f396259c640 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5608e3b67860 0x5608e3b69c50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c5de502bf562136 server_cookie=925b85bd23e6c14d in_seq=0 out_seq=0 2024-04-21T08:48:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f3949ffb640 1 -- 172.21.15.82:0/2681334543 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608e3ac0c00 con 0x5608e3b67860 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f3949ffb640 10 client.16476 handle_command_reply: tid=0 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f396b7ca740 1 client.16476 shutdown 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f396b7ca740 1 -- 172.21.15.82:0/2681334543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 msgr2=0x5608e3b54fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f396b7ca740 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b54fc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f394c02c170 tx=0x7f394c0054f0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f396b7ca740 1 -- 172.21.15.82:0/2681334543 shutdown_connections 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f396b7ca740 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5608e3b67860 0x5608e3b69c50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f396b7ca740 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b57a80 0x5608e3b55500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f396b7ca740 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b54fc0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f3949ffb640 0 client.16476 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f396b7ca740 1 --2- 172.21.15.82:0/2681334543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b4e580 0x5608e3b54a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f396b7ca740 1 -- 172.21.15.82:0/2681334543 >> 172.21.15.82:0/2681334543 conn(0x5608e3b48cb0 msgr2=0x5608e3b4aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f396b7ca740 1 -- 172.21.15.82:0/2681334543 shutdown_connections 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f396b7ca740 1 -- 172.21.15.82:0/2681334543 wait complete. 2024-04-21T08:48:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.632+0000 7f396b7ca740 20 client.16476 trim_cache size 0 max 16384 2024-04-21T08:48:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.651+0000 7f396b7ca740 1 Processor -- start 2024-04-21T08:48:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.651+0000 7f396b7ca740 1 -- start start 2024-04-21T08:48:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.651+0000 7f396b7ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b4e580 0x5608e3b571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.651+0000 7f396b7ca740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.651+0000 7f396b7ca740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b57a80 0x5608e3b49090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.651+0000 7f396b7ca740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f39641af2d0 con 0x5608e3b4eea0 2024-04-21T08:48:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.651+0000 7f396b7ca740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f39641b0f00 con 0x5608e3b57a80 2024-04-21T08:48:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.651+0000 7f396b7ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f39641af9a0 con 0x5608e3b4e580 2024-04-21T08:48:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.651+0000 7f395bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.651+0000 7f395bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b636d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34548/0 (socket says 172.21.15.82:34548) 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.651+0000 7f395bfff640 1 -- 172.21.15.82:0/4266942471 learned_addr learned my addr 172.21.15.82:0/4266942471 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.652+0000 7f3962d9d640 1 --2- 172.21.15.82:0/4266942471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b57a80 0x5608e3b49090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.652+0000 7f396259c640 1 --2- 172.21.15.82:0/4266942471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b4e580 0x5608e3b571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.652+0000 7f395bfff640 1 -- 172.21.15.82:0/4266942471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b4e580 msgr2=0x5608e3b571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.652+0000 7f395bfff640 1 --2- 172.21.15.82:0/4266942471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b4e580 0x5608e3b571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.652+0000 7f395bfff640 1 -- 172.21.15.82:0/4266942471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b57a80 msgr2=0x5608e3b49090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.652+0000 7f395bfff640 1 --2- 172.21.15.82:0/4266942471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b57a80 0x5608e3b49090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.652+0000 7f395bfff640 1 -- 172.21.15.82:0/4266942471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e3b52fd0 con 0x5608e3b4eea0 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.652+0000 7f396259c640 1 --2- 172.21.15.82:0/4266942471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b4e580 0x5608e3b571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.652+0000 7f395bfff640 1 --2- 172.21.15.82:0/4266942471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b636d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f394c009900 tx=0x7f394c009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389fe367af96b172 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.652+0000 7f394bfff640 1 -- 172.21.15.82:0/4266942471 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c045040 con 0x5608e3b4eea0 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.652+0000 7f394bfff640 1 -- 172.21.15.82:0/4266942471 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394c02c7e0 con 0x5608e3b4eea0 2024-04-21T08:48:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.652+0000 7f394bfff640 1 -- 172.21.15.82:0/4266942471 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c0037f0 con 0x5608e3b4eea0 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.653+0000 7f396b7ca740 1 -- 172.21.15.82:0/4266942471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 msgr2=0x5608e3b636d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.653+0000 7f396b7ca740 1 --2- 172.21.15.82:0/4266942471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b636d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f394c009900 tx=0x7f394c009630 comp rx=0 tx=0).stop 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.653+0000 7f396b7ca740 1 -- 172.21.15.82:0/4266942471 shutdown_connections 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.653+0000 7f396b7ca740 1 --2- 172.21.15.82:0/4266942471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b57a80 0x5608e3b49090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.653+0000 7f396b7ca740 1 --2- 172.21.15.82:0/4266942471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b636d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.653+0000 7f396b7ca740 1 --2- 172.21.15.82:0/4266942471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b4e580 0x5608e3b571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.653+0000 7f396b7ca740 1 -- 172.21.15.82:0/4266942471 >> 172.21.15.82:0/4266942471 conn(0x5608e3b48cb0 msgr2=0x5608e3b474e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.653+0000 7f396b7ca740 1 -- 172.21.15.82:0/4266942471 shutdown_connections 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.653+0000 7f396b7ca740 1 -- 172.21.15.82:0/4266942471 wait complete. 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.653+0000 7f396b7ca740 1 -- start start 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.653+0000 7f396b7ca740 1 Processor -- start 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f396b7ca740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b4e580 0x5608e3b60000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f396b7ca740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b5b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f396b7ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b57a80 0x5608e3b5b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f396b7ca740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f39641b0800 con 0x5608e3b4eea0 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f396b7ca740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f39641b0130 con 0x5608e3b4e580 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f396b7ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f39641ad2b0 con 0x5608e3b57a80 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f396b7ca740 10 client.0 _ll_register_callbacks cb 0x7f395409c900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f395bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b5b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f395bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b5b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34558/0 (socket says 172.21.15.82:34558) 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f395bfff640 1 -- 172.21.15.82:0/3459349993 learned_addr learned my addr 172.21.15.82:0/3459349993 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f396259c640 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b4e580 0x5608e3b60000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f3962d9d640 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b57a80 0x5608e3b5b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f395bfff640 1 -- 172.21.15.82:0/3459349993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b57a80 msgr2=0x5608e3b5b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f395bfff640 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b57a80 0x5608e3b5b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f395bfff640 1 -- 172.21.15.82:0/3459349993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b4e580 msgr2=0x5608e3b60000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f395bfff640 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b4e580 0x5608e3b60000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f395bfff640 1 -- 172.21.15.82:0/3459349993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e3b5c150 con 0x5608e3b4eea0 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f395bfff640 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b5b370 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f394c0096c0 tx=0x7f394c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca884b3849e956f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f3949ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f3949ffb640 1 -- 172.21.15.82:0/3459349993 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c063070 con 0x5608e3b4eea0 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f3949ffb640 1 -- 172.21.15.82:0/3459349993 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f394c004ad0 con 0x5608e3b4eea0 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.654+0000 7f396b7ca740 1 -- 172.21.15.82:0/3459349993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f394c02c7e0 con 0x5608e3b4eea0 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.655+0000 7f3949ffb640 1 -- 172.21.15.82:0/3459349993 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f394c06e050 con 0x5608e3b4eea0 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.655+0000 7f396b7ca740 10 client.16482 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.655+0000 7f396b7ca740 1 -- 172.21.15.82:0/3459349993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5608e3bd2040 con 0x5608e3b4eea0 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.655+0000 7f3949ffb640 1 -- 172.21.15.82:0/3459349993 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394c02c7e0 con 0x5608e3b4eea0 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.655+0000 7f3949ffb640 1 -- 172.21.15.82:0/3459349993 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f394c041510 con 0x5608e3b4eea0 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.655+0000 7f396b7ca740 10 client.16482 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.655+0000 7f396b7ca740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.655+0000 7f396b7ca740 20 client.16482 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.655+0000 7f396b7ca740 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5608e3bd26a0 0x5608e3bd4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.655+0000 7f396b7ca740 4 client.16482 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.655+0000 7f396b7ca740 1 -- 172.21.15.82:0/3459349993 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608e3abaa70 con 0x5608e3bd26a0 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.655+0000 7f396259c640 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5608e3bd26a0 0x5608e3bd4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396259c640 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5608e3bd26a0 0x5608e3bd4ac0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb8aa434c842e50 server_cookie=c8a3def3da786b2e in_seq=0 out_seq=0 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f3949ffb640 10 client.16482 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f3949ffb640 1 -- 172.21.15.82:0/3459349993 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5608e3abaa70 con 0x5608e3bd26a0 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f3949ffb640 10 client.16482 handle_command_reply: tid=0 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396b7ca740 1 client.16482 shutdown 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396b7ca740 1 -- 172.21.15.82:0/3459349993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 msgr2=0x5608e3b5b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396b7ca740 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b5b370 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f394c0096c0 tx=0x7f394c005b50 comp rx=0 tx=0).stop 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396b7ca740 1 -- 172.21.15.82:0/3459349993 shutdown_connections 2024-04-21T08:48:44.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396b7ca740 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5608e3bd26a0 0x5608e3bd4ac0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396b7ca740 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608e3b57a80 0x5608e3b5b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f3949ffb640 0 client.16482 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:44.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396b7ca740 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5608e3b4eea0 0x5608e3b5b370 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396b7ca740 1 --2- 172.21.15.82:0/3459349993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5608e3b4e580 0x5608e3b60000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396b7ca740 1 -- 172.21.15.82:0/3459349993 >> 172.21.15.82:0/3459349993 conn(0x5608e3b48cb0 msgr2=0x5608e3b474e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:44.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396b7ca740 1 -- 172.21.15.82:0/3459349993 shutdown_connections 2024-04-21T08:48:44.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396b7ca740 1 -- 172.21.15.82:0/3459349993 wait complete. 2024-04-21T08:48:44.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.656+0000 7f396b7ca740 20 client.16482 trim_cache size 0 max 16384 2024-04-21T08:48:44.662 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:48:44.663 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:48:44.663 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:48:44.663 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:48:44.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.657+0000 7f3968afd640 1 -- 172.21.15.82:0/284483234 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f39500a3d20 msgr2=0x7f39500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:44.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.657+0000 7f3968afd640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f39500a3d20 0x7f39500a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f395c005c40 tx=0x7f395c005bd0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.657+0000 7f3968afd640 1 -- 172.21.15.82:0/284483234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3964101080 msgr2=0x7f39641986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:44.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.657+0000 7f3968afd640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3964101080 0x7f39641986f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f395400e9f0 tx=0x7f395400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.657+0000 7f3968afd640 1 -- 172.21.15.82:0/284483234 shutdown_connections 2024-04-21T08:48:44.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.657+0000 7f3968afd640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f396419cfb0 0x7f396419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.657+0000 7f3968afd640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3964106150 0x7f3964198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.657+0000 7f3968afd640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f39500a3d20 0x7f39500a61e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.657+0000 7f3968afd640 1 --2- 172.21.15.82:0/284483234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3964101080 0x7f39641986f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.657+0000 7f3968afd640 1 -- 172.21.15.82:0/284483234 >> 172.21.15.82:0/284483234 conn(0x7f3964078000 msgr2=0x7f39640fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:44.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.657+0000 7f3968afd640 1 -- 172.21.15.82:0/284483234 shutdown_connections 2024-04-21T08:48:44.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.657+0000 7f3968afd640 1 -- 172.21.15.82:0/284483234 wait complete. 2024-04-21T08:48:44.668 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:48:44.668 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:48:44.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.846+0000 7f0d508c1640 1 -- 172.21.15.82:0/210659700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c0690e0 msgr2=0x7f0d4c069540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:44.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.846+0000 7f0d508c1640 1 --2- 172.21.15.82:0/210659700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c0690e0 0x7f0d4c069540 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009900 tx=0x7f0d4002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:44.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.846+0000 7f0d508c1640 1 -- 172.21.15.82:0/210659700 shutdown_connections 2024-04-21T08:48:44.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.846+0000 7f0d508c1640 1 --2- 172.21.15.82:0/210659700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d4c069a80 0x7f0d4c10eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.846+0000 7f0d508c1640 1 --2- 172.21.15.82:0/210659700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c0690e0 0x7f0d4c069540 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.846+0000 7f0d508c1640 1 --2- 172.21.15.82:0/210659700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d4c107fe0 0x7f0d4c1083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.846+0000 7f0d508c1640 1 -- 172.21.15.82:0/210659700 >> 172.21.15.82:0/210659700 conn(0x7f0d4c077e30 msgr2=0x7f0d4c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:44.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.846+0000 7f0d508c1640 1 -- 172.21.15.82:0/210659700 shutdown_connections 2024-04-21T08:48:44.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.846+0000 7f0d508c1640 1 -- 172.21.15.82:0/210659700 wait complete. 2024-04-21T08:48:44.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.847+0000 7f0d508c1640 1 Processor -- start 2024-04-21T08:48:44.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.847+0000 7f0d508c1640 1 -- start start 2024-04-21T08:48:44.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.847+0000 7f0d508c1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c0690e0 0x7f0d4c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.847+0000 7f0d508c1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d4c069a80 0x7f0d4c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.847+0000 7f0d508c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d4c107fe0 0x7f0d4c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.847+0000 7f0d508c1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c112160 con 0x7f0d4c0690e0 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.847+0000 7f0d508c1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c111fe0 con 0x7f0d4c069a80 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.847+0000 7f0d508c1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1122e0 con 0x7f0d4c107fe0 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.847+0000 7f0d4a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c0690e0 0x7f0d4c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.847+0000 7f0d4a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c0690e0 0x7f0d4c198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34568/0 (socket says 172.21.15.82:34568) 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d4a59c640 1 -- 172.21.15.82:0/4253732172 learned_addr learned my addr 172.21.15.82:0/4253732172 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d49d9b640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d4c069a80 0x7f0d4c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d4ad9d640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d4c107fe0 0x7f0d4c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d4a59c640 1 -- 172.21.15.82:0/4253732172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d4c107fe0 msgr2=0x7f0d4c19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d4a59c640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d4c107fe0 0x7f0d4c19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d4a59c640 1 -- 172.21.15.82:0/4253732172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d4c069a80 msgr2=0x7f0d4c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d4a59c640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d4c069a80 0x7f0d4c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d4a59c640 1 -- 172.21.15.82:0/4253732172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c19d710 con 0x7f0d4c0690e0 2024-04-21T08:48:44.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d49d9b640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d4c069a80 0x7f0d4c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:44.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d4a59c640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c0690e0 0x7f0d4c198720 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00d950 tx=0x7f0d3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:44.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d3b7fe640 1 -- 172.21.15.82:0/4253732172 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c017030 con 0x7f0d4c0690e0 2024-04-21T08:48:44.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d508c1640 1 -- 172.21.15.82:0/4253732172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c19da00 con 0x7f0d4c0690e0 2024-04-21T08:48:44.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d3b7fe640 1 -- 172.21.15.82:0/4253732172 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3c004020 con 0x7f0d4c0690e0 2024-04-21T08:48:44.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d3b7fe640 1 -- 172.21.15.82:0/4253732172 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c0047a0 con 0x7f0d4c0690e0 2024-04-21T08:48:44.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.848+0000 7f0d508c1640 1 -- 172.21.15.82:0/4253732172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c1011a0 con 0x7f0d4c0690e0 2024-04-21T08:48:44.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.850+0000 7f0d508c1640 1 -- 172.21.15.82:0/4253732172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4c068de0 con 0x7f0d4c0690e0 2024-04-21T08:48:44.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.851+0000 7f0d3b7fe640 1 -- 172.21.15.82:0/4253732172 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0d3c009a30 con 0x7f0d4c0690e0 2024-04-21T08:48:44.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.852+0000 7f0d3b7fe640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0d340a3d20 0x7f0d340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:44.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.852+0000 7f0d3b7fe640 1 -- 172.21.15.82:0/4253732172 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f0d3c0d0bf0 con 0x7f0d4c0690e0 2024-04-21T08:48:44.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.852+0000 7f0d49d9b640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0d340a3d20 0x7f0d340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:44.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.853+0000 7f0d49d9b640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0d340a3d20 0x7f0d340a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d40006010 tx=0x7f0d4002c6c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:44.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:44.857+0000 7f0d3b7fe640 1 -- 172.21.15.82:0/4253732172 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d3c09ced0 con 0x7f0d4c0690e0 2024-04-21T08:48:45.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.273+0000 7f0d508c1640 1 -- 172.21.15.82:0/4253732172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d4c199a70 con 0x7f0d4c0690e0 2024-04-21T08:48:45.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.273+0000 7f0d3b7fe640 1 -- 172.21.15.82:0/4253732172 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11952 (secure 0 0 0) 0x7f0d3c09c7d0 con 0x7f0d4c0690e0 2024-04-21T08:48:45.273 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:48:45.273 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:48:22.138089+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:48:45.274 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 32 2024-04-21T08:48:45.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.276+0000 7f0d508c1640 1 -- 172.21.15.82:0/4253732172 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0d340a3d20 msgr2=0x7f0d340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.276+0000 7f0d508c1640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0d340a3d20 0x7f0d340a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d40006010 tx=0x7f0d4002c6c0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.276+0000 7f0d508c1640 1 -- 172.21.15.82:0/4253732172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c0690e0 msgr2=0x7f0d4c198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.276+0000 7f0d508c1640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c0690e0 0x7f0d4c198720 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00d950 tx=0x7f0d3c00de10 comp rx=0 tx=0).stop 2024-04-21T08:48:45.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.276+0000 7f0d508c1640 1 -- 172.21.15.82:0/4253732172 shutdown_connections 2024-04-21T08:48:45.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.276+0000 7f0d508c1640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d4c107fe0 0x7f0d4c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.276+0000 7f0d508c1640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d4c069a80 0x7f0d4c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.276+0000 7f0d508c1640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0d340a3d20 0x7f0d340a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.276+0000 7f0d508c1640 1 --2- 172.21.15.82:0/4253732172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c0690e0 0x7f0d4c198720 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.276+0000 7f0d508c1640 1 -- 172.21.15.82:0/4253732172 >> 172.21.15.82:0/4253732172 conn(0x7f0d4c077e30 msgr2=0x7f0d4c0fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.276+0000 7f0d508c1640 1 -- 172.21.15.82:0/4253732172 shutdown_connections 2024-04-21T08:48:45.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.276+0000 7f0d508c1640 1 -- 172.21.15.82:0/4253732172 wait complete. 2024-04-21T08:48:45.287 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:48:45.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.414+0000 7f568a1b1640 1 -- 172.21.15.82:0/2370331166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5684106170 msgr2=0x7f568410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.415+0000 7f568a1b1640 1 --2- 172.21.15.82:0/2370331166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5684106170 0x7f568410dd40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f567c009ca0 tx=0x7f567c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.415+0000 7f568a1b1640 1 -- 172.21.15.82:0/2370331166 shutdown_connections 2024-04-21T08:48:45.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.415+0000 7f568a1b1640 1 --2- 172.21.15.82:0/2370331166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5684106170 0x7f568410dd40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.415+0000 7f568a1b1640 1 --2- 172.21.15.82:0/2370331166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5684101a50 0x7f5684105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.415+0000 7f568a1b1640 1 --2- 172.21.15.82:0/2370331166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56841010a0 0x7f5684101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.415+0000 7f568a1b1640 1 -- 172.21.15.82:0/2370331166 >> 172.21.15.82:0/2370331166 conn(0x7f56840771d0 msgr2=0x7f56840775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.415+0000 7f568a1b1640 1 -- 172.21.15.82:0/2370331166 shutdown_connections 2024-04-21T08:48:45.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.415+0000 7f568a1b1640 1 -- 172.21.15.82:0/2370331166 wait complete. 2024-04-21T08:48:45.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.415+0000 7f568a1b1640 1 Processor -- start 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f568a1b1640 1 -- start start 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f568a1b1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56841010a0 0x7f5684197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f568a1b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5684106170 0x7f5684197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f568a1b1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568419bee0 0x7f568419c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f568a1b1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f56841110e0 con 0x7f568419bee0 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f568a1b1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5684110f60 con 0x7f5684106170 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f568a1b1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5684111260 con 0x7f56841010a0 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f5683fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568419bee0 0x7f568419c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f5683fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568419bee0 0x7f568419c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34592/0 (socket says 172.21.15.82:34592) 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f5683fff640 1 -- 172.21.15.82:0/3995880502 learned_addr learned my addr 172.21.15.82:0/3995880502 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f5682ffd640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5684106170 0x7f5684197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f56837fe640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56841010a0 0x7f5684197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f5683fff640 1 -- 172.21.15.82:0/3995880502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56841010a0 msgr2=0x7f5684197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f5683fff640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56841010a0 0x7f5684197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f5683fff640 1 -- 172.21.15.82:0/3995880502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5684106170 msgr2=0x7f5684197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f5683fff640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5684106170 0x7f5684197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f5683fff640 1 -- 172.21.15.82:0/3995880502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568419c9d0 con 0x7f568419bee0 2024-04-21T08:48:45.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.416+0000 7f5683fff640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568419bee0 0x7f568419c390 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f567c005e30 tx=0x7f567c0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.417+0000 7f5680ff9640 1 -- 172.21.15.82:0/3995880502 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c041020 con 0x7f568419bee0 2024-04-21T08:48:45.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.417+0000 7f568a1b1640 1 -- 172.21.15.82:0/3995880502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568419cc60 con 0x7f568419bee0 2024-04-21T08:48:45.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.417+0000 7f568a1b1640 1 -- 172.21.15.82:0/3995880502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568406a7e0 con 0x7f568419bee0 2024-04-21T08:48:45.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.417+0000 7f5680ff9640 1 -- 172.21.15.82:0/3995880502 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f567c032970 con 0x7f568419bee0 2024-04-21T08:48:45.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.417+0000 7f5680ff9640 1 -- 172.21.15.82:0/3995880502 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c039040 con 0x7f568419bee0 2024-04-21T08:48:45.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.419+0000 7f5680ff9640 1 -- 172.21.15.82:0/3995880502 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f567c044020 con 0x7f568419bee0 2024-04-21T08:48:45.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.419+0000 7f568cd9b740 1 Processor -- start 2024-04-21T08:48:45.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.419+0000 7f568cd9b740 1 -- start start 2024-04-21T08:48:45.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f568cd9b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83c54d0 0x5593f83c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f568cd9b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f568cd9b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83cea00 0x5593f83d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f568cd9b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5593f832a1f0 con 0x5593f83c5e80 2024-04-21T08:48:45.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f568cd9b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5593f82c74b0 con 0x5593f83cea00 2024-04-21T08:48:45.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f568cd9b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5593f83261b0 con 0x5593f83c54d0 2024-04-21T08:48:45.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f5680ff9640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f566c0a3da0 0x7f566c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f56837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f56837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34606/0 (socket says 172.21.15.82:34606) 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f56837fe640 1 -- 172.21.15.82:0/729609182 learned_addr learned my addr 172.21.15.82:0/729609182 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f5682ffd640 1 --2- 172.21.15.82:0/729609182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83cea00 0x5593f83d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f5682ffd640 1 --2- 172.21.15.82:0/729609182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83c54d0 0x5593f83c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f5683fff640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f566c0a3da0 0x7f566c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f5680ff9640 1 -- 172.21.15.82:0/3995880502 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f567c0f2500 con 0x7f568419bee0 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f56837fe640 1 -- 172.21.15.82:0/729609182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83c54d0 msgr2=0x5593f83c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f56837fe640 1 --2- 172.21.15.82:0/729609182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83c54d0 0x5593f83c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f56837fe640 1 -- 172.21.15.82:0/729609182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83cea00 msgr2=0x5593f83d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f56837fe640 1 --2- 172.21.15.82:0/729609182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83cea00 0x5593f83d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f56837fe640 1 -- 172.21.15.82:0/729609182 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593f82b91b0 con 0x5593f83c5e80 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f5682ffd640 1 --2- 172.21.15.82:0/729609182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83cea00 0x5593f83d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f56837fe640 1 --2- 172.21.15.82:0/729609182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83ce430 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5674004640 tx=0x7f5674005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee4cdfdc38f566c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f5683fff640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f566c0a3da0 0x7f566c0a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f567c0efa50 tx=0x7f567c04d000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.420+0000 7f5669ffb640 1 -- 172.21.15.82:0/729609182 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674009e00 con 0x5593f83c5e80 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.421+0000 7f5669ffb640 1 -- 172.21.15.82:0/729609182 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5674020070 con 0x5593f83c5e80 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.421+0000 7f5669ffb640 1 -- 172.21.15.82:0/729609182 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567401b860 con 0x5593f83c5e80 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.421+0000 7f568cd9b740 1 -- 172.21.15.82:0/729609182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 msgr2=0x5593f83ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.421+0000 7f568cd9b740 1 --2- 172.21.15.82:0/729609182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83ce430 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5674004640 tx=0x7f5674005f90 comp rx=0 tx=0).stop 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.421+0000 7f568cd9b740 1 -- 172.21.15.82:0/729609182 shutdown_connections 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.421+0000 7f568cd9b740 1 --2- 172.21.15.82:0/729609182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83cea00 0x5593f83d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.421+0000 7f568cd9b740 1 --2- 172.21.15.82:0/729609182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83ce430 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.421+0000 7f568cd9b740 1 --2- 172.21.15.82:0/729609182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83c54d0 0x5593f83c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.421+0000 7f568cd9b740 1 -- 172.21.15.82:0/729609182 >> 172.21.15.82:0/729609182 conn(0x5593f83bfba0 msgr2=0x5593f83bff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.421+0000 7f568cd9b740 1 -- 172.21.15.82:0/729609182 shutdown_connections 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.421+0000 7f568cd9b740 1 -- 172.21.15.82:0/729609182 wait complete. 2024-04-21T08:48:45.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f568cd9b740 1 -- start start 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f568cd9b740 1 Processor -- start 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f568cd9b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83c54d0 0x5593f83ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f568cd9b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f568cd9b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83cea00 0x5593f83cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f568cd9b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5593f83d1970 con 0x5593f83c5e80 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f568cd9b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5593f82e3500 con 0x5593f83c54d0 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f568cd9b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5593f83d1af0 con 0x5593f83cea00 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f568cd9b740 10 client.0 _ll_register_callbacks cb 0x5593f82b1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f56837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f5682ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83c54d0 0x5593f83ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f5682ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83c54d0 0x5593f83ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:41986/0 (socket says 172.21.15.82:41986) 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f5682ffd640 1 -- 172.21.15.82:0/3141962136 learned_addr learned my addr 172.21.15.82:0/3141962136 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f56837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34612/0 (socket says 172.21.15.82:34612) 2024-04-21T08:48:45.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.422+0000 7f5682ffd640 1 --2- 172.21.15.82:0/3141962136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83cea00 0x5593f83cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f56837fe640 1 -- 172.21.15.82:0/3141962136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83cea00 msgr2=0x5593f83cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f56837fe640 1 --2- 172.21.15.82:0/3141962136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83cea00 0x5593f83cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f56837fe640 1 -- 172.21.15.82:0/3141962136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83c54d0 msgr2=0x5593f83ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f56837fe640 1 --2- 172.21.15.82:0/3141962136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83c54d0 0x5593f83ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f56837fe640 1 -- 172.21.15.82:0/3141962136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593f803c3f0 con 0x5593f83c5e80 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f56837fe640 1 --2- 172.21.15.82:0/3141962136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83cb5e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x5593f83c6280 tx=0x7f5674009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bf51ac441c4f28 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f5662ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f5662ffd640 1 -- 172.21.15.82:0/3141962136 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674005480 con 0x5593f83c5e80 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f5662ffd640 1 -- 172.21.15.82:0/3141962136 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f56740046c0 con 0x5593f83c5e80 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f568cd9b740 1 -- 172.21.15.82:0/3141962136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5674020070 con 0x5593f83c5e80 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f5662ffd640 1 -- 172.21.15.82:0/3141962136 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5674037050 con 0x5593f83c5e80 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f568cd9b740 10 client.16500 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:45.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f568cd9b740 1 -- 172.21.15.82:0/3141962136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5593f81067d0 con 0x5593f83c5e80 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f5662ffd640 1 -- 172.21.15.82:0/3141962136 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5674020070 con 0x5593f83c5e80 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f5662ffd640 1 -- 172.21.15.82:0/3141962136 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f567401b860 con 0x5593f83c5e80 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f568cd9b740 10 client.16500 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f568cd9b740 10 client.16500 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f568cd9b740 20 client.16500 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f568cd9b740 1 --2- 172.21.15.82:0/3141962136 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5593f83deaa0 0x5593f83e0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f568cd9b740 4 client.16500 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.423+0000 7f568cd9b740 1 -- 172.21.15.82:0/3141962136 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593f82c9930 con 0x5593f83deaa0 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.424+0000 7f5682ffd640 1 --2- 172.21.15.82:0/3141962136 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5593f83deaa0 0x5593f83e0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.424+0000 7f5662ffd640 10 client.16500 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.424+0000 7f5682ffd640 1 --2- 172.21.15.82:0/3141962136 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5593f83deaa0 0x5593f83e0e90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8b40ffc003ab87e2 server_cookie=fb7112c60b4e9349 in_seq=0 out_seq=0 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f5662ffd640 1 -- 172.21.15.82:0/3141962136 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593f82c9930 con 0x5593f83deaa0 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f5662ffd640 10 client.16500 handle_command_reply: tid=0 2024-04-21T08:48:45.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f568cd9b740 1 client.16500 shutdown 2024-04-21T08:48:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f568cd9b740 1 -- 172.21.15.82:0/3141962136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 msgr2=0x5593f83cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f568cd9b740 1 --2- 172.21.15.82:0/3141962136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83cb5e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x5593f83c6280 tx=0x7f5674009eb0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f568cd9b740 1 -- 172.21.15.82:0/3141962136 shutdown_connections 2024-04-21T08:48:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f568cd9b740 1 --2- 172.21.15.82:0/3141962136 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5593f83deaa0 0x5593f83e0e90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f568cd9b740 1 --2- 172.21.15.82:0/3141962136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83cea00 0x5593f83cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f568cd9b740 1 --2- 172.21.15.82:0/3141962136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83cb5e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f5662ffd640 0 client.16500 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:48:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f568cd9b740 1 --2- 172.21.15.82:0/3141962136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83c54d0 0x5593f83ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f568cd9b740 1 -- 172.21.15.82:0/3141962136 >> 172.21.15.82:0/3141962136 conn(0x5593f83bfba0 msgr2=0x5593f83c18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f568cd9b740 1 -- 172.21.15.82:0/3141962136 shutdown_connections 2024-04-21T08:48:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f568cd9b740 1 -- 172.21.15.82:0/3141962136 wait complete. 2024-04-21T08:48:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.425+0000 7f568cd9b740 20 client.16500 trim_cache size 0 max 16384 2024-04-21T08:48:45.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.439+0000 7f568cd9b740 1 Processor -- start 2024-04-21T08:48:45.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.439+0000 7f568cd9b740 1 -- start start 2024-04-21T08:48:45.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.439+0000 7f568cd9b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83c54d0 0x5593f83da8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.439+0000 7f568cd9b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.439+0000 7f568cd9b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83cea00 0x5593f83cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.439+0000 7f568cd9b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f567000ff70 con 0x5593f83c5e80 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.439+0000 7f568cd9b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5670011ba0 con 0x5593f83cea00 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.439+0000 7f568cd9b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5670010640 con 0x5593f83c54d0 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.439+0000 7f56837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.439+0000 7f56837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83c16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34622/0 (socket says 172.21.15.82:34622) 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.439+0000 7f56837fe640 1 -- 172.21.15.82:0/1638089734 learned_addr learned my addr 172.21.15.82:0/1638089734 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.439+0000 7f5682ffd640 1 --2- 172.21.15.82:0/1638089734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83cea00 0x5593f83cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.440+0000 7f5682ffd640 1 --2- 172.21.15.82:0/1638089734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83c54d0 0x5593f83da8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.440+0000 7f56837fe640 1 -- 172.21.15.82:0/1638089734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83c54d0 msgr2=0x5593f83da8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.440+0000 7f56837fe640 1 --2- 172.21.15.82:0/1638089734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83c54d0 0x5593f83da8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.440+0000 7f56837fe640 1 -- 172.21.15.82:0/1638089734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83cea00 msgr2=0x5593f83cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.440+0000 7f56837fe640 1 --2- 172.21.15.82:0/1638089734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83cea00 0x5593f83cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.440+0000 7f56837fe640 1 -- 172.21.15.82:0/1638089734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593f83c5c20 con 0x5593f83c5e80 2024-04-21T08:48:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.440+0000 7f5682ffd640 1 --2- 172.21.15.82:0/1638089734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83c54d0 0x5593f83da8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.440+0000 7f56837fe640 1 --2- 172.21.15.82:0/1638089734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83c16b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5674004560 tx=0x7f5674009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6f614276f8ec9b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.440+0000 7f56697fa640 1 -- 172.21.15.82:0/1638089734 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674033070 con 0x5593f83c5e80 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.440+0000 7f56697fa640 1 -- 172.21.15.82:0/1638089734 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5674020070 con 0x5593f83c5e80 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.440+0000 7f56697fa640 1 -- 172.21.15.82:0/1638089734 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674004ae0 con 0x5593f83c5e80 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.441+0000 7f568cd9b740 1 -- 172.21.15.82:0/1638089734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 msgr2=0x5593f83c16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.441+0000 7f568cd9b740 1 --2- 172.21.15.82:0/1638089734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83c16b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5674004560 tx=0x7f5674009310 comp rx=0 tx=0).stop 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.441+0000 7f568cd9b740 1 -- 172.21.15.82:0/1638089734 shutdown_connections 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.441+0000 7f568cd9b740 1 --2- 172.21.15.82:0/1638089734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83cea00 0x5593f83cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.441+0000 7f568cd9b740 1 --2- 172.21.15.82:0/1638089734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c5e80 0x5593f83c16b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.441+0000 7f568cd9b740 1 --2- 172.21.15.82:0/1638089734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83c54d0 0x5593f83da8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.441+0000 7f568cd9b740 1 -- 172.21.15.82:0/1638089734 >> 172.21.15.82:0/1638089734 conn(0x5593f83bfba0 msgr2=0x5593f83c0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.441+0000 7f568cd9b740 1 -- 172.21.15.82:0/1638089734 shutdown_connections 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.441+0000 7f568cd9b740 1 -- 172.21.15.82:0/1638089734 wait complete. 2024-04-21T08:48:45.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.441+0000 7f568cd9b740 1 -- start start 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.441+0000 7f568cd9b740 1 Processor -- start 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f568cd9b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c54d0 0x5593f83d6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f568cd9b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83c5e80 0x5593f83c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f568cd9b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83cea00 0x5593f83c8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f568cd9b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f56700114a0 con 0x5593f83c54d0 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f568cd9b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5670010dd0 con 0x5593f83c5e80 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f568cd9b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f567000df50 con 0x5593f83cea00 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f568cd9b740 10 client.0 _ll_register_callbacks cb 0x7f5674004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5682ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c54d0 0x5593f83d6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5682ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c54d0 0x5593f83d6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34638/0 (socket says 172.21.15.82:34638) 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5682ffd640 1 -- 172.21.15.82:0/3895267012 learned_addr learned my addr 172.21.15.82:0/3895267012 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f56837fe640 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83c5e80 0x5593f83c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5682ffd640 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83cea00 0x5593f83c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5682ffd640 1 -- 172.21.15.82:0/3895267012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83cea00 msgr2=0x5593f83c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5682ffd640 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83cea00 0x5593f83c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5682ffd640 1 -- 172.21.15.82:0/3895267012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83c5e80 msgr2=0x5593f83c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5682ffd640 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83c5e80 0x5593f83c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5682ffd640 1 -- 172.21.15.82:0/3895267012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593f83c8870 con 0x5593f83c54d0 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f56837fe640 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83c5e80 0x5593f83c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5682ffd640 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c54d0 0x5593f83d6fc0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x5593f83e1540 tx=0x7f5678012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8d6a60ea52bcb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5662ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5662ffd640 1 -- 172.21.15.82:0/3895267012 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567801a040 con 0x5593f83c54d0 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f5662ffd640 1 -- 172.21.15.82:0/3895267012 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f56780258b0 con 0x5593f83c54d0 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.442+0000 7f568cd9b740 1 -- 172.21.15.82:0/3895267012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5674020070 con 0x5593f83c54d0 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.443+0000 7f5662ffd640 1 -- 172.21.15.82:0/3895267012 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5678034950 con 0x5593f83c54d0 2024-04-21T08:48:45.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.443+0000 7f568cd9b740 10 client.16506 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.443+0000 7f568cd9b740 1 -- 172.21.15.82:0/3895267012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5593f8444a80 con 0x5593f83c54d0 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.443+0000 7f5662ffd640 1 -- 172.21.15.82:0/3895267012 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5674020070 con 0x5593f83c54d0 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.443+0000 7f5662ffd640 1 -- 172.21.15.82:0/3895267012 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f5678004c20 con 0x5593f83c54d0 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.443+0000 7f568cd9b740 10 client.16506 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.443+0000 7f568cd9b740 10 client.16506 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.443+0000 7f568cd9b740 20 client.16506 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.443+0000 7f568cd9b740 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5593f84451e0 0x5593f8447600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.443+0000 7f568cd9b740 4 client.16506 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.443+0000 7f568cd9b740 1 -- 172.21.15.82:0/3895267012 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593f83cbeb0 con 0x5593f84451e0 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.443+0000 7f56837fe640 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5593f84451e0 0x5593f8447600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f5662ffd640 10 client.16506 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f56837fe640 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5593f84451e0 0x5593f8447600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3eb5c716256cd3a1 server_cookie=98efc5c7e786ec3b in_seq=0 out_seq=0 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f5662ffd640 1 -- 172.21.15.82:0/3895267012 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593f83cbeb0 con 0x5593f84451e0 2024-04-21T08:48:45.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f5662ffd640 10 client.16506 handle_command_reply: tid=0 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f568cd9b740 1 client.16506 shutdown 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f568cd9b740 1 -- 172.21.15.82:0/3895267012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c54d0 msgr2=0x5593f83d6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f568cd9b740 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c54d0 0x5593f83d6fc0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x5593f83e1540 tx=0x7f5678012040 comp rx=0 tx=0).stop 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f568cd9b740 1 -- 172.21.15.82:0/3895267012 shutdown_connections 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f568cd9b740 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5593f84451e0 0x5593f8447600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f568cd9b740 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593f83cea00 0x5593f83c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f568cd9b740 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593f83c5e80 0x5593f83c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f568cd9b740 1 --2- 172.21.15.82:0/3895267012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593f83c54d0 0x5593f83d6fc0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f5662ffd640 0 client.16506 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f568cd9b740 1 -- 172.21.15.82:0/3895267012 >> 172.21.15.82:0/3895267012 conn(0x5593f83bfba0 msgr2=0x5593f83bec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f568cd9b740 1 -- 172.21.15.82:0/3895267012 shutdown_connections 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f568cd9b740 1 -- 172.21.15.82:0/3895267012 wait complete. 2024-04-21T08:48:45.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.444+0000 7f568cd9b740 20 client.16506 trim_cache size 0 max 16384 2024-04-21T08:48:45.451 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:48:45.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.445+0000 7f568a1b1640 1 -- 172.21.15.82:0/3995880502 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f566c0a3da0 msgr2=0x7f566c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.445+0000 7f568a1b1640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f566c0a3da0 0x7f566c0a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f567c0efa50 tx=0x7f567c04d000 comp rx=0 tx=0).stop 2024-04-21T08:48:45.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.445+0000 7f568a1b1640 1 -- 172.21.15.82:0/3995880502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568419bee0 msgr2=0x7f568419c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.445+0000 7f568a1b1640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568419bee0 0x7f568419c390 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f567c005e30 tx=0x7f567c0346d0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.445+0000 7f568a1b1640 1 -- 172.21.15.82:0/3995880502 shutdown_connections 2024-04-21T08:48:45.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.445+0000 7f568a1b1640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f566c0a3da0 0x7f566c0a6260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.445+0000 7f568a1b1640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568419bee0 0x7f568419c390 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.445+0000 7f568a1b1640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5684106170 0x7f5684197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.445+0000 7f568a1b1640 1 --2- 172.21.15.82:0/3995880502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56841010a0 0x7f5684197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.445+0000 7f568a1b1640 1 -- 172.21.15.82:0/3995880502 >> 172.21.15.82:0/3995880502 conn(0x7f56840771d0 msgr2=0x7f56840fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.445+0000 7f568a1b1640 1 -- 172.21.15.82:0/3995880502 shutdown_connections 2024-04-21T08:48:45.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.445+0000 7f568a1b1640 1 -- 172.21.15.82:0/3995880502 wait complete. 2024-04-21T08:48:45.456 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:48:45.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.566+0000 7f96a6ef0640 1 -- 172.21.15.82:0/3575341241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96a0106110 msgr2=0x7f96a010dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.566+0000 7f96a6ef0640 1 --2- 172.21.15.82:0/3575341241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96a0106110 0x7f96a010dc50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9698009ca0 tx=0x7f969802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.566+0000 7f96a6ef0640 1 -- 172.21.15.82:0/3575341241 shutdown_connections 2024-04-21T08:48:45.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.566+0000 7f96a6ef0640 1 --2- 172.21.15.82:0/3575341241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96a0106110 0x7f96a010dc50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.566+0000 7f96a6ef0640 1 --2- 172.21.15.82:0/3575341241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a0101c50 0x7f96a01059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.566+0000 7f96a6ef0640 1 --2- 172.21.15.82:0/3575341241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96a01012a0 0x7f96a0101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.566+0000 7f96a6ef0640 1 -- 172.21.15.82:0/3575341241 >> 172.21.15.82:0/3575341241 conn(0x7f96a0077c00 msgr2=0x7f96a0078010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.566+0000 7f96a6ef0640 1 -- 172.21.15.82:0/3575341241 shutdown_connections 2024-04-21T08:48:45.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.566+0000 7f96a6ef0640 1 -- 172.21.15.82:0/3575341241 wait complete. 2024-04-21T08:48:45.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a6ef0640 1 Processor -- start 2024-04-21T08:48:45.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a6ef0640 1 -- start start 2024-04-21T08:48:45.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a6ef0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96a01012a0 0x7f96a01993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a6ef0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96a0106110 0x7f96a0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a6ef0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a0199ec0 0x7f96a01a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a6ef0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0111010 con 0x7f96a0106110 2024-04-21T08:48:45.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a6ef0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0110e90 con 0x7f96a0199ec0 2024-04-21T08:48:45.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a6ef0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0111190 con 0x7f96a01012a0 2024-04-21T08:48:45.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a56ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96a0106110 0x7f96a0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a56ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96a0106110 0x7f96a0199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34660/0 (socket says 172.21.15.82:34660) 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a56ed640 1 -- 172.21.15.82:0/2257727730 learned_addr learned my addr 172.21.15.82:0/2257727730 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a66ef640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a0199ec0 0x7f96a01a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a5eee640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96a01012a0 0x7f96a01993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a56ed640 1 -- 172.21.15.82:0/2257727730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96a01012a0 msgr2=0x7f96a01993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.567+0000 7f96a56ed640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96a01012a0 0x7f96a01993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.568+0000 7f96a56ed640 1 -- 172.21.15.82:0/2257727730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a0199ec0 msgr2=0x7f96a01a4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.568+0000 7f96a56ed640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a0199ec0 0x7f96a01a4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.568+0000 7f96a56ed640 1 -- 172.21.15.82:0/2257727730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a01a4c40 con 0x7f96a0106110 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.568+0000 7f96a66ef640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a0199ec0 0x7f96a01a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.568+0000 7f96a5eee640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96a01012a0 0x7f96a01993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.568+0000 7f96a56ed640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96a0106110 0x7f96a0199930 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f969000e940 tx=0x7f969000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.568+0000 7f9696ffd640 1 -- 172.21.15.82:0/2257727730 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969000cd80 con 0x7f96a0106110 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.568+0000 7f9696ffd640 1 -- 172.21.15.82:0/2257727730 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96900035f0 con 0x7f96a0106110 2024-04-21T08:48:45.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.568+0000 7f96a6ef0640 1 -- 172.21.15.82:0/2257727730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a01a4f30 con 0x7f96a0106110 2024-04-21T08:48:45.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.568+0000 7f9696ffd640 1 -- 172.21.15.82:0/2257727730 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690010690 con 0x7f96a0106110 2024-04-21T08:48:45.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.568+0000 7f96a6ef0640 1 -- 172.21.15.82:0/2257727730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a01a5440 con 0x7f96a0106110 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.571+0000 7f96aae27740 1 Processor -- start 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.571+0000 7f96aae27740 1 -- start start 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.571+0000 7f96aae27740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 0x5606fc38cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.571+0000 7f96aae27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38d080 0x5606fc395630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.571+0000 7f9696ffd640 1 -- 172.21.15.82:0/2257727730 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9690004020 con 0x7f96a0106110 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.571+0000 7f96aae27740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc395c00 0x5606fc398050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.571+0000 7f96aae27740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5606fc2f11f0 con 0x5606fc395c00 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.571+0000 7f96aae27740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5606fc28e4b0 con 0x5606fc38c6d0 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.571+0000 7f96aae27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5606fc2ed1b0 con 0x5606fc38d080 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.571+0000 7f96a66ef640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 0x5606fc38cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f96a66ef640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 0x5606fc38cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42028/0 (socket says 172.21.15.82:42028) 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f96a5eee640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38d080 0x5606fc395630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f96a56ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc395c00 0x5606fc398050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f96a66ef640 1 -- 172.21.15.82:0/3414604908 learned_addr learned my addr 172.21.15.82:0/3414604908 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f96a66ef640 1 -- 172.21.15.82:0/3414604908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38d080 msgr2=0x5606fc395630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f96a66ef640 1 --2- 172.21.15.82:0/3414604908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38d080 0x5606fc395630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f96a66ef640 1 -- 172.21.15.82:0/3414604908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc395c00 msgr2=0x5606fc398050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f96a66ef640 1 --2- 172.21.15.82:0/3414604908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc395c00 0x5606fc398050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f96a66ef640 1 -- 172.21.15.82:0/3414604908 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606fc2801b0 con 0x5606fc38c6d0 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f96a5eee640 1 --2- 172.21.15.82:0/3414604908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38d080 0x5606fc395630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f96a66ef640 1 --2- 172.21.15.82:0/3414604908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 0x5606fc38cab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f969802c700 tx=0x7f96980323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e197c5c2e0156b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f9696ffd640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f968c0a3da0 0x7f968c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f968bfff640 1 -- 172.21.15.82:0/3414604908 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698043070 con 0x5606fc38c6d0 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f96a5eee640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f968c0a3da0 0x7f968c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f968bfff640 1 -- 172.21.15.82:0/3414604908 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9698032970 con 0x5606fc38c6d0 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f968bfff640 1 -- 172.21.15.82:0/3414604908 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698039040 con 0x5606fc38c6d0 2024-04-21T08:48:45.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.572+0000 7f9696ffd640 1 -- 172.21.15.82:0/2257727730 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f96900d0d00 con 0x7f96a0106110 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.573+0000 7f96aae27740 1 -- 172.21.15.82:0/3414604908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 msgr2=0x5606fc38cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.573+0000 7f96a5eee640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f968c0a3da0 0x7f968c0a6260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f969c009fd0 tx=0x7f969c005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.573+0000 7f96aae27740 1 --2- 172.21.15.82:0/3414604908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 0x5606fc38cab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f969802c700 tx=0x7f96980323d0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.573+0000 7f96aae27740 1 -- 172.21.15.82:0/3414604908 shutdown_connections 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.573+0000 7f96aae27740 1 --2- 172.21.15.82:0/3414604908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc395c00 0x5606fc398050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.573+0000 7f96aae27740 1 --2- 172.21.15.82:0/3414604908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38d080 0x5606fc395630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.573+0000 7f96aae27740 1 --2- 172.21.15.82:0/3414604908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 0x5606fc38cab0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.573+0000 7f96aae27740 1 -- 172.21.15.82:0/3414604908 >> 172.21.15.82:0/3414604908 conn(0x5606fc386da0 msgr2=0x5606fc387180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.573+0000 7f96aae27740 1 -- 172.21.15.82:0/3414604908 shutdown_connections 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.573+0000 7f96aae27740 1 -- 172.21.15.82:0/3414604908 wait complete. 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.573+0000 7f96aae27740 1 -- start start 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96aae27740 1 Processor -- start 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96aae27740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 0x5606fc395450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96aae27740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38d080 0x5606fc392850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96aae27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc395c00 0x5606fc392d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96aae27740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5606fc398b10 con 0x5606fc38d080 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96aae27740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5606fc2aa500 con 0x5606fc38c6d0 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96aae27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5606fc398c90 con 0x5606fc395c00 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96aae27740 10 client.0 _ll_register_callbacks cb 0x5606fc278b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96a56ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38d080 0x5606fc392850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96a5eee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 0x5606fc395450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96a66ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc395c00 0x5606fc392d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96a56ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38d080 0x5606fc392850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34678/0 (socket says 172.21.15.82:34678) 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96a66ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc395c00 0x5606fc392d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:50988/0 (socket says 172.21.15.82:50988) 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96a56ed640 1 -- 172.21.15.82:0/2982520942 learned_addr learned my addr 172.21.15.82:0/2982520942 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96a5eee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 0x5606fc395450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42042/0 (socket says 172.21.15.82:42042) 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96a66ef640 1 -- 172.21.15.82:0/2982520942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 msgr2=0x5606fc395450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96a66ef640 1 --2- 172.21.15.82:0/2982520942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 0x5606fc395450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96a66ef640 1 -- 172.21.15.82:0/2982520942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38d080 msgr2=0x5606fc392850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.574+0000 7f96a66ef640 1 --2- 172.21.15.82:0/2982520942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38d080 0x5606fc392850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96a66ef640 1 -- 172.21.15.82:0/2982520942 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606fc0033f0 con 0x5606fc395c00 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96a56ed640 1 --2- 172.21.15.82:0/2982520942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38d080 0x5606fc392850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96a66ef640 1 --2- 172.21.15.82:0/2982520942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc395c00 0x5606fc392d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f969802c630 tx=0x7f9698009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f97e872ddde797d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96897fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96897fa640 1 -- 172.21.15.82:0/2982520942 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969804c030 con 0x5606fc395c00 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96aae27740 1 -- 172.21.15.82:0/2982520942 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9698032970 con 0x5606fc395c00 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96897fa640 1 -- 172.21.15.82:0/2982520942 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f9698003fa0 con 0x5606fc395c00 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96897fa640 1 -- 172.21.15.82:0/2982520942 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9698059050 con 0x5606fc395c00 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96aae27740 10 client.25657 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96aae27740 1 -- 172.21.15.82:0/2982520942 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5606fc0cd7d0 con 0x5606fc395c00 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96897fa640 1 -- 172.21.15.82:0/2982520942 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9698032970 con 0x5606fc395c00 2024-04-21T08:48:45.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96897fa640 1 -- 172.21.15.82:0/2982520942 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f9698048810 con 0x5606fc395c00 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96aae27740 10 client.25657 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96aae27740 10 client.25657 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96aae27740 20 client.25657 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96aae27740 1 --2- 172.21.15.82:0/2982520942 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5606fc3a5c40 0x5606fc3a8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96aae27740 4 client.25657 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.575+0000 7f96aae27740 1 -- 172.21.15.82:0/2982520942 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606fc290930 con 0x5606fc3a5c40 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.576+0000 7f96a5eee640 1 --2- 172.21.15.82:0/2982520942 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5606fc3a5c40 0x5606fc3a8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.576+0000 7f96897fa640 10 client.25657 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.576+0000 7f96a5eee640 1 --2- 172.21.15.82:0/2982520942 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5606fc3a5c40 0x5606fc3a8030 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d3823a8ec7b928a server_cookie=4b0ede9ece38c8e7 in_seq=0 out_seq=0 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.577+0000 7f96897fa640 1 -- 172.21.15.82:0/2982520942 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606fc290930 con 0x5606fc3a5c40 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.577+0000 7f96897fa640 10 client.25657 handle_command_reply: tid=0 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.577+0000 7f96aae27740 1 client.25657 shutdown 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.577+0000 7f96aae27740 1 -- 172.21.15.82:0/2982520942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc395c00 msgr2=0x5606fc392d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.577+0000 7f96aae27740 1 --2- 172.21.15.82:0/2982520942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc395c00 0x5606fc392d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f969802c630 tx=0x7f9698009800 comp rx=0 tx=0).stop 2024-04-21T08:48:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.577+0000 7f96aae27740 1 -- 172.21.15.82:0/2982520942 shutdown_connections 2024-04-21T08:48:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.577+0000 7f96aae27740 1 --2- 172.21.15.82:0/2982520942 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5606fc3a5c40 0x5606fc3a8030 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.577+0000 7f96aae27740 1 --2- 172.21.15.82:0/2982520942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc395c00 0x5606fc392d90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.577+0000 7f96897fa640 0 client.25657 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:48:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.577+0000 7f96aae27740 1 --2- 172.21.15.82:0/2982520942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38d080 0x5606fc392850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.577+0000 7f96aae27740 1 --2- 172.21.15.82:0/2982520942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38c6d0 0x5606fc395450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.577+0000 7f96aae27740 1 -- 172.21.15.82:0/2982520942 >> 172.21.15.82:0/2982520942 conn(0x5606fc386da0 msgr2=0x5606fc388af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.578+0000 7f96aae27740 1 -- 172.21.15.82:0/2982520942 shutdown_connections 2024-04-21T08:48:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.578+0000 7f96aae27740 1 -- 172.21.15.82:0/2982520942 wait complete. 2024-04-21T08:48:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.578+0000 7f96aae27740 20 client.25657 trim_cache size 0 max 16384 2024-04-21T08:48:45.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96aae27740 1 Processor -- start 2024-04-21T08:48:45.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96aae27740 1 -- start start 2024-04-21T08:48:45.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96aae27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38c6d0 0x5606fc392130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96aae27740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38d080 0x5606fc385d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96aae27740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc395c00 0x5606fc394f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96aae27740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01af160 con 0x5606fc395c00 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96aae27740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01b0d90 con 0x5606fc38d080 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96aae27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01af830 con 0x5606fc38c6d0 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96a66ef640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc395c00 0x5606fc394f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96a66ef640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc395c00 0x5606fc394f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34694/0 (socket says 172.21.15.82:34694) 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96a66ef640 1 -- 172.21.15.82:0/1621069908 learned_addr learned my addr 172.21.15.82:0/1621069908 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96a5eee640 1 --2- 172.21.15.82:0/1621069908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38c6d0 0x5606fc392130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.593+0000 7f96a56ed640 1 --2- 172.21.15.82:0/1621069908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38d080 0x5606fc385d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.594+0000 7f96a66ef640 1 -- 172.21.15.82:0/1621069908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38c6d0 msgr2=0x5606fc392130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.594+0000 7f96a66ef640 1 --2- 172.21.15.82:0/1621069908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38c6d0 0x5606fc392130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.594+0000 7f96a66ef640 1 -- 172.21.15.82:0/1621069908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38d080 msgr2=0x5606fc385d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.594+0000 7f96a66ef640 1 --2- 172.21.15.82:0/1621069908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38d080 0x5606fc385d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.594+0000 7f96a66ef640 1 -- 172.21.15.82:0/1621069908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606fc3ade70 con 0x5606fc395c00 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.594+0000 7f96a66ef640 1 --2- 172.21.15.82:0/1621069908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc395c00 0x5606fc394f80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9698048c60 tx=0x7f96980323d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3cf27da3b4f211 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.594+0000 7f968b7fe640 1 -- 172.21.15.82:0/1621069908 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969804c020 con 0x5606fc395c00 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.594+0000 7f968b7fe640 1 -- 172.21.15.82:0/1621069908 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9698048e50 con 0x5606fc395c00 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.594+0000 7f968b7fe640 1 -- 172.21.15.82:0/1621069908 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698039070 con 0x5606fc395c00 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.595+0000 7f96aae27740 1 -- 172.21.15.82:0/1621069908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc395c00 msgr2=0x5606fc394f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.595+0000 7f96aae27740 1 --2- 172.21.15.82:0/1621069908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc395c00 0x5606fc394f80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9698048c60 tx=0x7f96980323d0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.595+0000 7f96aae27740 1 -- 172.21.15.82:0/1621069908 shutdown_connections 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.595+0000 7f96aae27740 1 --2- 172.21.15.82:0/1621069908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc395c00 0x5606fc394f80 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.595+0000 7f96aae27740 1 --2- 172.21.15.82:0/1621069908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc38d080 0x5606fc385d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.595+0000 7f96aae27740 1 --2- 172.21.15.82:0/1621069908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38c6d0 0x5606fc392130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.595+0000 7f96aae27740 1 -- 172.21.15.82:0/1621069908 >> 172.21.15.82:0/1621069908 conn(0x5606fc386da0 msgr2=0x5606fc38de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.595+0000 7f96aae27740 1 -- 172.21.15.82:0/1621069908 shutdown_connections 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.595+0000 7f96aae27740 1 -- 172.21.15.82:0/1621069908 wait complete. 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.595+0000 7f96aae27740 1 -- start start 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.595+0000 7f96aae27740 1 Processor -- start 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96aae27740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38c6d0 0x5606fc3a8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96aae27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38d080 0x5606fc3a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96aae27740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc395c00 0x5606fc40e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96aae27740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01b0690 con 0x5606fc38c6d0 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96aae27740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01affc0 con 0x5606fc395c00 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96aae27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01ad140 con 0x5606fc38d080 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96aae27740 10 client.0 _ll_register_callbacks cb 0x7f9690005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96a5eee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38c6d0 0x5606fc3a8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96a5eee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38c6d0 0x5606fc3a8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34710/0 (socket says 172.21.15.82:34710) 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96a5eee640 1 -- 172.21.15.82:0/2998842032 learned_addr learned my addr 172.21.15.82:0/2998842032 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96a66ef640 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc395c00 0x5606fc40e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96a56ed640 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38d080 0x5606fc3a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96a5eee640 1 -- 172.21.15.82:0/2998842032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38d080 msgr2=0x5606fc3a86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96a5eee640 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38d080 0x5606fc3a86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96a5eee640 1 -- 172.21.15.82:0/2998842032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc395c00 msgr2=0x5606fc40e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96a5eee640 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc395c00 0x5606fc40e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96a5eee640 1 -- 172.21.15.82:0/2998842032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606fc40e760 con 0x5606fc38c6d0 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96a5eee640 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38c6d0 0x5606fc3a8160 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x5606fc38d480 tx=0x7f969c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c853cf44cb5cf9e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96897fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96897fa640 1 -- 172.21.15.82:0/2998842032 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c00fd60 con 0x5606fc38c6d0 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96897fa640 1 -- 172.21.15.82:0/2998842032 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f969c02b380 con 0x5606fc38c6d0 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96aae27740 1 -- 172.21.15.82:0/2998842032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9698048e50 con 0x5606fc38c6d0 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96897fa640 1 -- 172.21.15.82:0/2998842032 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f969c03b050 con 0x5606fc38c6d0 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96aae27740 10 client.16524 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.596+0000 7f96aae27740 1 -- 172.21.15.82:0/2998842032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5606fc40fa20 con 0x5606fc38c6d0 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.597+0000 7f96897fa640 1 -- 172.21.15.82:0/2998842032 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9698048e50 con 0x5606fc38c6d0 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.597+0000 7f96897fa640 1 -- 172.21.15.82:0/2998842032 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f969c0047a0 con 0x5606fc38c6d0 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.597+0000 7f96aae27740 10 client.16524 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.597+0000 7f96aae27740 10 client.16524 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:48:45.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.597+0000 7f96aae27740 20 client.16524 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.597+0000 7f96aae27740 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5606fc410140 0x5606fc412580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.597+0000 7f96aae27740 4 client.16524 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.597+0000 7f96aae27740 1 -- 172.21.15.82:0/2998842032 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606fc2fec10 con 0x5606fc410140 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.597+0000 7f96a56ed640 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5606fc410140 0x5606fc412580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.597+0000 7f96897fa640 10 client.16524 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.597+0000 7f96a56ed640 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5606fc410140 0x5606fc412580 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9e73f1ce2242ff45 server_cookie=e567568c312c65ea in_seq=0 out_seq=0 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96897fa640 1 -- 172.21.15.82:0/2998842032 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606fc2fec10 con 0x5606fc410140 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96897fa640 10 client.16524 handle_command_reply: tid=0 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96aae27740 1 client.16524 shutdown 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96aae27740 1 -- 172.21.15.82:0/2998842032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38c6d0 msgr2=0x5606fc3a8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96aae27740 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38c6d0 0x5606fc3a8160 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x5606fc38d480 tx=0x7f969c0046c0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96aae27740 1 -- 172.21.15.82:0/2998842032 shutdown_connections 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96aae27740 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5606fc410140 0x5606fc412580 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96aae27740 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5606fc395c00 0x5606fc40e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96aae27740 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5606fc38d080 0x5606fc3a86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96897fa640 0 client.16524 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:48:45.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96aae27740 1 --2- 172.21.15.82:0/2998842032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fc38c6d0 0x5606fc3a8160 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96aae27740 1 -- 172.21.15.82:0/2998842032 >> 172.21.15.82:0/2998842032 conn(0x5606fc386da0 msgr2=0x5606fc3a17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96aae27740 1 -- 172.21.15.82:0/2998842032 shutdown_connections 2024-04-21T08:48:45.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96aae27740 1 -- 172.21.15.82:0/2998842032 wait complete. 2024-04-21T08:48:45.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.598+0000 7f96aae27740 20 client.16524 trim_cache size 0 max 16384 2024-04-21T08:48:45.604 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:48:45.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.599+0000 7f96a6ef0640 1 -- 172.21.15.82:0/2257727730 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f968c0a3da0 msgr2=0x7f968c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.599+0000 7f96a6ef0640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f968c0a3da0 0x7f968c0a6260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f969c009fd0 tx=0x7f969c005f90 comp rx=0 tx=0).stop 2024-04-21T08:48:45.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.599+0000 7f96a6ef0640 1 -- 172.21.15.82:0/2257727730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96a0106110 msgr2=0x7f96a0199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.599+0000 7f96a6ef0640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96a0106110 0x7f96a0199930 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f969000e940 tx=0x7f969000ee00 comp rx=0 tx=0).stop 2024-04-21T08:48:45.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.599+0000 7f96a6ef0640 1 -- 172.21.15.82:0/2257727730 shutdown_connections 2024-04-21T08:48:45.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.599+0000 7f96a6ef0640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96a0199ec0 0x7f96a01a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.599+0000 7f96a6ef0640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f968c0a3da0 0x7f968c0a6260 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.599+0000 7f96a6ef0640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96a0106110 0x7f96a0199930 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.599+0000 7f96a6ef0640 1 --2- 172.21.15.82:0/2257727730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96a01012a0 0x7f96a01993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.599+0000 7f96a6ef0640 1 -- 172.21.15.82:0/2257727730 >> 172.21.15.82:0/2257727730 conn(0x7f96a0077c00 msgr2=0x7f96a00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.599+0000 7f96a6ef0640 1 -- 172.21.15.82:0/2257727730 shutdown_connections 2024-04-21T08:48:45.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.599+0000 7f96a6ef0640 1 -- 172.21.15.82:0/2257727730 wait complete. 2024-04-21T08:48:45.609 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:48:45.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.717+0000 7fe95168e640 1 -- 172.21.15.82:0/3973103251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c102b30 msgr2=0x7fe94c106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.717+0000 7fe95168e640 1 --2- 172.21.15.82:0/3973103251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c102b30 0x7fe94c106b70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe940009900 tx=0x7fe94002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:45.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.717+0000 7fe95168e640 1 -- 172.21.15.82:0/3973103251 shutdown_connections 2024-04-21T08:48:45.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.717+0000 7fe95168e640 1 --2- 172.21.15.82:0/3973103251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe94c107250 0x7fe94c10ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.717+0000 7fe95168e640 1 --2- 172.21.15.82:0/3973103251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c102b30 0x7fe94c106b70 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.717+0000 7fe95168e640 1 --2- 172.21.15.82:0/3973103251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe94c102180 0x7fe94c102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.717+0000 7fe95168e640 1 -- 172.21.15.82:0/3973103251 >> 172.21.15.82:0/3973103251 conn(0x7fe94c0fd9c0 msgr2=0x7fe94c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.717+0000 7fe95168e640 1 -- 172.21.15.82:0/3973103251 shutdown_connections 2024-04-21T08:48:45.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.717+0000 7fe95168e640 1 -- 172.21.15.82:0/3973103251 wait complete. 2024-04-21T08:48:45.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe95168e640 1 Processor -- start 2024-04-21T08:48:45.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe95168e640 1 -- start start 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe95168e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe94c102180 0x7fe94c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe95168e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe94c107250 0x7fe94c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe95168e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c19d010 0x7fe94c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe95168e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c112190 con 0x7fe94c19d010 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe95168e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c112010 con 0x7fe94c102180 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe95168e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c112310 con 0x7fe94c107250 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c19d010 0x7fe94c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe94affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe94c102180 0x7fe94c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe94affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe94c102180 0x7fe94c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42078/0 (socket says 172.21.15.82:42078) 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe94affd640 1 -- 172.21.15.82:0/3810345587 learned_addr learned my addr 172.21.15.82:0/3810345587 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c19d010 0x7fe94c19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34740/0 (socket says 172.21.15.82:34740) 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe94a7fc640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe94c107250 0x7fe94c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe94affd640 1 -- 172.21.15.82:0/3810345587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe94c107250 msgr2=0x7fe94c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe94affd640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe94c107250 0x7fe94c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe94affd640 1 -- 172.21.15.82:0/3810345587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c19d010 msgr2=0x7fe94c19d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe94affd640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c19d010 0x7fe94c19d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.718+0000 7fe94affd640 1 -- 172.21.15.82:0/3810345587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94c19db00 con 0x7fe94c102180 2024-04-21T08:48:45.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.719+0000 7fe94affd640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe94c102180 0x7fe94c198750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00e9f0 tx=0x7fe93c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.719+0000 7fe933fff640 1 -- 172.21.15.82:0/3810345587 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c00cd80 con 0x7fe94c102180 2024-04-21T08:48:45.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.719+0000 7fe933fff640 1 -- 172.21.15.82:0/3810345587 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe93c0108e0 con 0x7fe94c102180 2024-04-21T08:48:45.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.719+0000 7fe95168e640 1 -- 172.21.15.82:0/3810345587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94c19ddf0 con 0x7fe94c102180 2024-04-21T08:48:45.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.719+0000 7fe933fff640 1 -- 172.21.15.82:0/3810345587 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c018980 con 0x7fe94c102180 2024-04-21T08:48:45.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.719+0000 7fe95168e640 1 -- 172.21.15.82:0/3810345587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe94c1a55b0 con 0x7fe94c102180 2024-04-21T08:48:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.721+0000 7fe95427d740 1 Processor -- start 2024-04-21T08:48:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.721+0000 7fe95427d740 1 -- start start 2024-04-21T08:48:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.721+0000 7fe95427d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624662414d0 0x5624662418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.721+0000 7fe95427d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 0x56246624a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.721+0000 7fe95427d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56246624aa00 0x56246624ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.721+0000 7fe95427d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5624661a61f0 con 0x562466241e80 2024-04-21T08:48:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.721+0000 7fe95427d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5624661434b0 con 0x56246624aa00 2024-04-21T08:48:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.721+0000 7fe95427d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5624661a21b0 con 0x5624662414d0 2024-04-21T08:48:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.721+0000 7fe933fff640 1 -- 172.21.15.82:0/3810345587 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe93c010430 con 0x7fe94c102180 2024-04-21T08:48:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56246624aa00 0x56246624ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 0x56246624a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56246624aa00 0x56246624ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42086/0 (socket says 172.21.15.82:42086) 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624662414d0 0x5624662418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 0x56246624a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34742/0 (socket says 172.21.15.82:34742) 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94a7fc640 1 -- 172.21.15.82:0/42458184 learned_addr learned my addr 172.21.15.82:0/42458184 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94a7fc640 1 -- 172.21.15.82:0/42458184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624662414d0 msgr2=0x5624662418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94a7fc640 1 --2- 172.21.15.82:0/42458184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624662414d0 0x5624662418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94a7fc640 1 -- 172.21.15.82:0/42458184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 msgr2=0x56246624a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94a7fc640 1 --2- 172.21.15.82:0/42458184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 0x56246624a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94a7fc640 1 -- 172.21.15.82:0/42458184 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624661351b0 con 0x56246624aa00 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94affd640 1 --2- 172.21.15.82:0/42458184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 0x56246624a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe933fff640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9340a3d20 0x7fe9340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94a7fc640 1 --2- 172.21.15.82:0/42458184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56246624aa00 0x56246624ce50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe940009900 tx=0x7fe9400095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83a419b8869b2b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe94b7fe640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9340a3d20 0x7fe9340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe9317fa640 1 -- 172.21.15.82:0/42458184 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940045020 con 0x56246624aa00 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe9317fa640 1 -- 172.21.15.82:0/42458184 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94002c7e0 con 0x56246624aa00 2024-04-21T08:48:45.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe9317fa640 1 -- 172.21.15.82:0/42458184 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9400054a0 con 0x56246624aa00 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.722+0000 7fe933fff640 1 -- 172.21.15.82:0/3810345587 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe93c0d0f00 con 0x7fe94c102180 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.723+0000 7fe95427d740 1 -- 172.21.15.82:0/42458184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56246624aa00 msgr2=0x56246624ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.723+0000 7fe94b7fe640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9340a3d20 0x7fe9340a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe944005fd0 tx=0x7fe944009500 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.723+0000 7fe95427d740 1 --2- 172.21.15.82:0/42458184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56246624aa00 0x56246624ce50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe940009900 tx=0x7fe9400095f0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.723+0000 7fe95427d740 1 -- 172.21.15.82:0/42458184 shutdown_connections 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.723+0000 7fe95427d740 1 --2- 172.21.15.82:0/42458184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56246624aa00 0x56246624ce50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.723+0000 7fe95427d740 1 --2- 172.21.15.82:0/42458184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 0x56246624a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.723+0000 7fe95427d740 1 --2- 172.21.15.82:0/42458184 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5624662414d0 0x5624662418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.723+0000 7fe95427d740 1 -- 172.21.15.82:0/42458184 >> 172.21.15.82:0/42458184 conn(0x56246623bba0 msgr2=0x56246623bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.723+0000 7fe95427d740 1 -- 172.21.15.82:0/42458184 shutdown_connections 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.723+0000 7fe95427d740 1 -- 172.21.15.82:0/42458184 wait complete. 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.723+0000 7fe95427d740 1 -- start start 2024-04-21T08:48:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.723+0000 7fe95427d740 1 Processor -- start 2024-04-21T08:48:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe95427d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624662414d0 0x56246624a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe95427d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562466241e80 0x562466247660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe95427d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56246624aa00 0x562466247bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe95427d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56246624d910 con 0x5624662414d0 2024-04-21T08:48:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe95427d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56246615f500 con 0x56246624aa00 2024-04-21T08:48:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe95427d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56246624da90 con 0x562466241e80 2024-04-21T08:48:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe95427d740 10 client.0 _ll_register_callbacks cb 0x56246612db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe94affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624662414d0 0x56246624a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe94affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624662414d0 0x56246624a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34756/0 (socket says 172.21.15.82:34756) 2024-04-21T08:48:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe94affd640 1 -- 172.21.15.82:0/478161930 learned_addr learned my addr 172.21.15.82:0/478161930 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe94b7fe640 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56246624aa00 0x562466247bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe94a7fc640 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562466241e80 0x562466247660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe94affd640 1 -- 172.21.15.82:0/478161930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562466241e80 msgr2=0x562466247660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe94affd640 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562466241e80 0x562466247660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe94affd640 1 -- 172.21.15.82:0/478161930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56246624aa00 msgr2=0x562466247bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe94affd640 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56246624aa00 0x562466247bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe94affd640 1 -- 172.21.15.82:0/478161930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562465eb83f0 con 0x5624662414d0 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe94affd640 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624662414d0 0x56246624a2d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c6a0 tx=0x7fe93c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6d85ca7721058d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe92eff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe92eff5640 1 -- 172.21.15.82:0/478161930 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c0ce6b0 con 0x5624662414d0 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe92eff5640 1 -- 172.21.15.82:0/478161930 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe93c0d4b40 con 0x5624662414d0 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.724+0000 7fe95427d740 1 -- 172.21.15.82:0/478161930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe94002c7e0 con 0x5624662414d0 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.725+0000 7fe92eff5640 1 -- 172.21.15.82:0/478161930 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe93c0bd0e0 con 0x5624662414d0 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.725+0000 7fe95427d740 10 client.16536 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.725+0000 7fe95427d740 1 -- 172.21.15.82:0/478161930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562465f827d0 con 0x5624662414d0 2024-04-21T08:48:45.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.725+0000 7fe92eff5640 1 -- 172.21.15.82:0/478161930 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94002c7e0 con 0x5624662414d0 2024-04-21T08:48:45.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.725+0000 7fe92eff5640 1 -- 172.21.15.82:0/478161930 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fe93c0d2070 con 0x5624662414d0 2024-04-21T08:48:45.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.725+0000 7fe95427d740 10 client.16536 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:45.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.725+0000 7fe95427d740 10 client.16536 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:48:45.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.725+0000 7fe95427d740 20 client.16536 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:45.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.725+0000 7fe95427d740 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56246625aaf0 0x56246625cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.725+0000 7fe95427d740 4 client.16536 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:45.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.725+0000 7fe95427d740 1 -- 172.21.15.82:0/478161930 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562466145930 con 0x56246625aaf0 2024-04-21T08:48:45.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.725+0000 7fe94a7fc640 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56246625aaf0 0x56246625cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.726+0000 7fe94a7fc640 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56246625aaf0 0x56246625cfb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d59253299ce3980 server_cookie=25a18c770b94294c in_seq=0 out_seq=0 2024-04-21T08:48:45.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.726+0000 7fe92eff5640 10 client.16536 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:48:45.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe92eff5640 1 -- 172.21.15.82:0/478161930 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562466145930 con 0x56246625aaf0 2024-04-21T08:48:45.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe92eff5640 10 client.16536 handle_command_reply: tid=0 2024-04-21T08:48:45.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe95427d740 1 client.16536 shutdown 2024-04-21T08:48:45.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe95427d740 1 -- 172.21.15.82:0/478161930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624662414d0 msgr2=0x56246624a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe95427d740 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624662414d0 0x56246624a2d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c6a0 tx=0x7fe93c018420 comp rx=0 tx=0).stop 2024-04-21T08:48:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe95427d740 1 -- 172.21.15.82:0/478161930 shutdown_connections 2024-04-21T08:48:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe95427d740 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56246625aaf0 0x56246625cfb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe95427d740 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56246624aa00 0x562466247bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe95427d740 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562466241e80 0x562466247660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe95427d740 1 --2- 172.21.15.82:0/478161930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5624662414d0 0x56246624a2d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe92eff5640 0 client.16536 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:48:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe95427d740 1 -- 172.21.15.82:0/478161930 >> 172.21.15.82:0/478161930 conn(0x56246623bba0 msgr2=0x56246623d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe95427d740 1 -- 172.21.15.82:0/478161930 shutdown_connections 2024-04-21T08:48:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe95427d740 1 -- 172.21.15.82:0/478161930 wait complete. 2024-04-21T08:48:45.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.730+0000 7fe95427d740 20 client.16536 trim_cache size 0 max 16384 2024-04-21T08:48:45.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.744+0000 7fe95427d740 1 Processor -- start 2024-04-21T08:48:45.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.744+0000 7fe95427d740 1 -- start start 2024-04-21T08:48:45.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.744+0000 7fe95427d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5624662414d0 0x5624662567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.744+0000 7fe95427d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562466241e80 0x562466249d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.744+0000 7fe95427d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56246624aa00 0x56246624a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.744+0000 7fe95427d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1af300 con 0x56246624aa00 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.744+0000 7fe95427d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1b0f30 con 0x5624662414d0 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.744+0000 7fe95427d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1af9d0 con 0x562466241e80 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.744+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56246624aa00 0x56246624a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.744+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56246624aa00 0x56246624a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34772/0 (socket says 172.21.15.82:34772) 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe94b7fe640 1 -- 172.21.15.82:0/1193564884 learned_addr learned my addr 172.21.15.82:0/1193564884 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe94affd640 1 --2- 172.21.15.82:0/1193564884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5624662414d0 0x5624662567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe94a7fc640 1 --2- 172.21.15.82:0/1193564884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562466241e80 0x562466249d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe94b7fe640 1 -- 172.21.15.82:0/1193564884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562466241e80 msgr2=0x562466249d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe94b7fe640 1 --2- 172.21.15.82:0/1193564884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562466241e80 0x562466249d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe94b7fe640 1 -- 172.21.15.82:0/1193564884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5624662414d0 msgr2=0x5624662567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe94b7fe640 1 --2- 172.21.15.82:0/1193564884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5624662414d0 0x5624662567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe94b7fe640 1 -- 172.21.15.82:0/1193564884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562466241be0 con 0x56246624aa00 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe94a7fc640 1 --2- 172.21.15.82:0/1193564884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562466241e80 0x562466249d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:45.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe94b7fe640 1 --2- 172.21.15.82:0/1193564884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56246624aa00 0x56246624a2d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe944003e60 tx=0x7fe94401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4912575d846d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe930ff9640 1 -- 172.21.15.82:0/1193564884 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944004b20 con 0x56246624aa00 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe930ff9640 1 -- 172.21.15.82:0/1193564884 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe944004cc0 con 0x56246624aa00 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.745+0000 7fe930ff9640 1 -- 172.21.15.82:0/1193564884 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9440053a0 con 0x56246624aa00 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 -- 172.21.15.82:0/1193564884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56246624aa00 msgr2=0x56246624a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 --2- 172.21.15.82:0/1193564884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56246624aa00 0x56246624a2d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe944003e60 tx=0x7fe94401a000 comp rx=0 tx=0).stop 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 -- 172.21.15.82:0/1193564884 shutdown_connections 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 --2- 172.21.15.82:0/1193564884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56246624aa00 0x56246624a2d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 --2- 172.21.15.82:0/1193564884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562466241e80 0x562466249d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 --2- 172.21.15.82:0/1193564884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5624662414d0 0x5624662567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 -- 172.21.15.82:0/1193564884 >> 172.21.15.82:0/1193564884 conn(0x56246623bba0 msgr2=0x562466242c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 -- 172.21.15.82:0/1193564884 shutdown_connections 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 -- 172.21.15.82:0/1193564884 wait complete. 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 -- start start 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 Processor -- start 2024-04-21T08:48:45.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5624662414d0 0x56246624e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 0x56246624ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246624aa00 0x56246624f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1b0830 con 0x562466241e80 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1b0160 con 0x5624662414d0 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1ad2e0 con 0x56246624aa00 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe95427d740 10 client.0 _ll_register_callbacks cb 0x7fe93c098dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe94a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 0x56246624ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe94a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 0x56246624ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34786/0 (socket says 172.21.15.82:34786) 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe94a7fc640 1 -- 172.21.15.82:0/862037798 learned_addr learned my addr 172.21.15.82:0/862037798 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.746+0000 7fe94affd640 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5624662414d0 0x56246624e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe94b7fe640 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246624aa00 0x56246624f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe94a7fc640 1 -- 172.21.15.82:0/862037798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246624aa00 msgr2=0x56246624f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe94a7fc640 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246624aa00 0x56246624f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe94a7fc640 1 -- 172.21.15.82:0/862037798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5624662414d0 msgr2=0x56246624e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe94a7fc640 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5624662414d0 0x56246624e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe94a7fc640 1 -- 172.21.15.82:0/862037798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624662c00f0 con 0x562466241e80 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe94a7fc640 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 0x56246624ebe0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe940009900 tx=0x7fe940005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bfa159d6f82aa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe92eff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe92eff5640 1 -- 172.21.15.82:0/862037798 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940045040 con 0x562466241e80 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe92eff5640 1 -- 172.21.15.82:0/862037798 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe940004450 con 0x562466241e80 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe95427d740 1 -- 172.21.15.82:0/862037798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe944004cc0 con 0x562466241e80 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe92eff5640 1 -- 172.21.15.82:0/862037798 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe94005e050 con 0x562466241e80 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe95427d740 10 client.16542 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe95427d740 1 -- 172.21.15.82:0/862037798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5624662c13c0 con 0x562466241e80 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe92eff5640 1 -- 172.21.15.82:0/862037798 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe944004cc0 con 0x562466241e80 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe92eff5640 1 -- 172.21.15.82:0/862037798 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fe940003960 con 0x562466241e80 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.747+0000 7fe95427d740 10 client.16542 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.748+0000 7fe95427d740 10 client.16542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.748+0000 7fe95427d740 20 client.16542 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.748+0000 7fe95427d740 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5624662c18c0 0x5624662c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.748+0000 7fe95427d740 4 client.16542 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:45.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.748+0000 7fe95427d740 1 -- 172.21.15.82:0/862037798 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624661b3c10 con 0x5624662c18c0 2024-04-21T08:48:45.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.748+0000 7fe94affd640 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5624662c18c0 0x5624662c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.748+0000 7fe92eff5640 10 client.16542 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:48:45.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.748+0000 7fe94affd640 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5624662c18c0 0x5624662c3ce0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c74a8106bba6bb4 server_cookie=6e778d579be45b59 in_seq=0 out_seq=0 2024-04-21T08:48:45.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.752+0000 7fe92eff5640 1 -- 172.21.15.82:0/862037798 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624661b3c10 con 0x5624662c18c0 2024-04-21T08:48:45.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.752+0000 7fe92eff5640 10 client.16542 handle_command_reply: tid=0 2024-04-21T08:48:45.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.752+0000 7fe95427d740 1 client.16542 shutdown 2024-04-21T08:48:45.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.752+0000 7fe95427d740 1 -- 172.21.15.82:0/862037798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 msgr2=0x56246624ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.752+0000 7fe95427d740 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 0x56246624ebe0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe940009900 tx=0x7fe940005b50 comp rx=0 tx=0).stop 2024-04-21T08:48:45.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95427d740 1 -- 172.21.15.82:0/862037798 shutdown_connections 2024-04-21T08:48:45.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95427d740 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5624662c18c0 0x5624662c3ce0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95427d740 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56246624aa00 0x56246624f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95427d740 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562466241e80 0x56246624ebe0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe92eff5640 0 client.16542 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:48:45.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95427d740 1 --2- 172.21.15.82:0/862037798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5624662414d0 0x56246624e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95427d740 1 -- 172.21.15.82:0/862037798 >> 172.21.15.82:0/862037798 conn(0x56246623bba0 msgr2=0x56246623cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95427d740 1 -- 172.21.15.82:0/862037798 shutdown_connections 2024-04-21T08:48:45.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95427d740 1 -- 172.21.15.82:0/862037798 wait complete. 2024-04-21T08:48:45.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95427d740 20 client.16542 trim_cache size 0 max 16384 2024-04-21T08:48:45.755 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:48:45.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95168e640 1 -- 172.21.15.82:0/3810345587 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9340a3d20 msgr2=0x7fe9340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95168e640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9340a3d20 0x7fe9340a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe944005fd0 tx=0x7fe944009500 comp rx=0 tx=0).stop 2024-04-21T08:48:45.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95168e640 1 -- 172.21.15.82:0/3810345587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe94c102180 msgr2=0x7fe94c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95168e640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe94c102180 0x7fe94c198750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00e9f0 tx=0x7fe93c00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95168e640 1 -- 172.21.15.82:0/3810345587 shutdown_connections 2024-04-21T08:48:45.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95168e640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9340a3d20 0x7fe9340a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95168e640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c19d010 0x7fe94c19d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95168e640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe94c107250 0x7fe94c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95168e640 1 --2- 172.21.15.82:0/3810345587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe94c102180 0x7fe94c198750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95168e640 1 -- 172.21.15.82:0/3810345587 >> 172.21.15.82:0/3810345587 conn(0x7fe94c0fd9c0 msgr2=0x7fe94c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95168e640 1 -- 172.21.15.82:0/3810345587 shutdown_connections 2024-04-21T08:48:45.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.753+0000 7fe95168e640 1 -- 172.21.15.82:0/3810345587 wait complete. 2024-04-21T08:48:45.764 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:48:45.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.872+0000 7f67020d2640 1 -- 172.21.15.82:0/251800216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66fc06a360 msgr2=0x7f66fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.872+0000 7f67020d2640 1 --2- 172.21.15.82:0/251800216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66fc06a360 0x7f66fc06a7a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009930 tx=0x7f66f002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:45.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.872+0000 7f67020d2640 1 -- 172.21.15.82:0/251800216 shutdown_connections 2024-04-21T08:48:45.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.872+0000 7f67020d2640 1 --2- 172.21.15.82:0/251800216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66fc06ace0 0x7f66fc10ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.872+0000 7f67020d2640 1 --2- 172.21.15.82:0/251800216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66fc06a360 0x7f66fc06a7a0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.872+0000 7f67020d2640 1 --2- 172.21.15.82:0/251800216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66fc107fd0 0x7f66fc1083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.872+0000 7f67020d2640 1 -- 172.21.15.82:0/251800216 >> 172.21.15.82:0/251800216 conn(0x7f66fc077ef0 msgr2=0x7f66fc078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.872+0000 7f67020d2640 1 -- 172.21.15.82:0/251800216 shutdown_connections 2024-04-21T08:48:45.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.872+0000 7f67020d2640 1 -- 172.21.15.82:0/251800216 wait complete. 2024-04-21T08:48:45.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.873+0000 7f67020d2640 1 Processor -- start 2024-04-21T08:48:45.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.873+0000 7f67020d2640 1 -- start start 2024-04-21T08:48:45.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.873+0000 7f67020d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66fc06a360 0x7f66fc19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.873+0000 7f67020d2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66fc06ace0 0x7f66fc19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.873+0000 7f67020d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66fc107fd0 0x7f66fc198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.873+0000 7f67020d2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1121f0 con 0x7f66fc06ace0 2024-04-21T08:48:45.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.873+0000 7f67020d2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc112070 con 0x7f66fc107fd0 2024-04-21T08:48:45.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.873+0000 7f67020d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc112370 con 0x7f66fc06a360 2024-04-21T08:48:45.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.873+0000 7f66faffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66fc06ace0 0x7f66fc19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.873+0000 7f66fbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66fc107fd0 0x7f66fc198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66fc06a360 0x7f66fc19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66faffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66fc06ace0 0x7f66fc19e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34802/0 (socket says 172.21.15.82:34802) 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66faffd640 1 -- 172.21.15.82:0/1730500378 learned_addr learned my addr 172.21.15.82:0/1730500378 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66faffd640 1 -- 172.21.15.82:0/1730500378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66fc06a360 msgr2=0x7f66fc19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66faffd640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66fc06a360 0x7f66fc19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66faffd640 1 -- 172.21.15.82:0/1730500378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66fc107fd0 msgr2=0x7f66fc198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66faffd640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66fc107fd0 0x7f66fc198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66faffd640 1 -- 172.21.15.82:0/1730500378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66fc198d10 con 0x7f66fc06ace0 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66fbfff640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66fc107fd0 0x7f66fc198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66fc06a360 0x7f66fc19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66faffd640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66fc06ace0 0x7f66fc19e920 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f66f0000c00 tx=0x7f66f002c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66f8ff9640 1 -- 172.21.15.82:0/1730500378 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0043020 con 0x7f66fc06ace0 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f67020d2640 1 -- 172.21.15.82:0/1730500378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66fc198fa0 con 0x7f66fc06ace0 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66f8ff9640 1 -- 172.21.15.82:0/1730500378 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f002caf0 con 0x7f66fc06ace0 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f66f8ff9640 1 -- 172.21.15.82:0/1730500378 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0003b20 con 0x7f66fc06ace0 2024-04-21T08:48:45.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.874+0000 7f67020d2640 1 -- 172.21.15.82:0/1730500378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66fc1a52a0 con 0x7f66fc06ace0 2024-04-21T08:48:45.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.878+0000 7f6704da1740 1 Processor -- start 2024-04-21T08:48:45.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.878+0000 7f66f8ff9640 1 -- 172.21.15.82:0/1730500378 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66f0046020 con 0x7f66fc06ace0 2024-04-21T08:48:45.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.878+0000 7f6704da1740 1 -- start start 2024-04-21T08:48:45.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.878+0000 7f6704da1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734c330 0x55d1c734c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.878+0000 7f6704da1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c7355820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.878+0000 7f6704da1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c73562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.878+0000 7f6704da1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d1c72b11f0 con 0x55d1c7355e40 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.878+0000 7f6704da1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d1c724e4b0 con 0x55d1c734c330 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.878+0000 7f6704da1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d1c72ad1b0 con 0x55d1c734cce0 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c7355820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66faffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c73562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66fbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734c330 0x55d1c734c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c7355820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:51078/0 (socket says 172.21.15.82:51078) 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66fb7fe640 1 -- 172.21.15.82:0/864098964 learned_addr learned my addr 172.21.15.82:0/864098964 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66fb7fe640 1 -- 172.21.15.82:0/864098964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734c330 msgr2=0x55d1c734c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/864098964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734c330 0x55d1c734c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66fb7fe640 1 -- 172.21.15.82:0/864098964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 msgr2=0x55d1c73562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/864098964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c73562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66fb7fe640 1 -- 172.21.15.82:0/864098964 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1c72401b0 con 0x55d1c734cce0 2024-04-21T08:48:45.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66f8ff9640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66e40a3d20 0x7f66e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/864098964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c7355820 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0114f0 tx=0x7f66ec0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70539c804142ba1c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66fbfff640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66e40a3d20 0x7f66e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66f8ff9640 1 -- 172.21.15.82:0/1730500378 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f66f00f2a70 con 0x7f66fc06ace0 2024-04-21T08:48:45.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.879+0000 7f66e1ffb640 1 -- 172.21.15.82:0/864098964 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec01e070 con 0x55d1c734cce0 2024-04-21T08:48:45.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.880+0000 7f66e1ffb640 1 -- 172.21.15.82:0/864098964 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec00fa70 con 0x55d1c734cce0 2024-04-21T08:48:45.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.880+0000 7f66e1ffb640 1 -- 172.21.15.82:0/864098964 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec019a80 con 0x55d1c734cce0 2024-04-21T08:48:45.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.881+0000 7f6704da1740 1 -- 172.21.15.82:0/864098964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 msgr2=0x55d1c7355820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.881+0000 7f66fbfff640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66e40a3d20 0x7f66e40a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009fd0 tx=0x7f66f4009500 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.881+0000 7f6704da1740 1 --2- 172.21.15.82:0/864098964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c7355820 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0114f0 tx=0x7f66ec0119b0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.881+0000 7f6704da1740 1 -- 172.21.15.82:0/864098964 shutdown_connections 2024-04-21T08:48:45.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.881+0000 7f6704da1740 1 --2- 172.21.15.82:0/864098964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c73562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.881+0000 7f6704da1740 1 --2- 172.21.15.82:0/864098964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c7355820 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.881+0000 7f6704da1740 1 --2- 172.21.15.82:0/864098964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734c330 0x55d1c734c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.881+0000 7f6704da1740 1 -- 172.21.15.82:0/864098964 >> 172.21.15.82:0/864098964 conn(0x55d1c7346ba0 msgr2=0x55d1c7346f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.881+0000 7f6704da1740 1 -- 172.21.15.82:0/864098964 shutdown_connections 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.881+0000 7f6704da1740 1 -- 172.21.15.82:0/864098964 wait complete. 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.881+0000 7f6704da1740 1 -- start start 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.882+0000 7f6704da1740 1 Processor -- start 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.882+0000 7f6704da1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734c330 0x55d1c734ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.882+0000 7f6704da1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c734f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.882+0000 7f6704da1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c734f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.882+0000 7f6704da1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d1c734d610 con 0x55d1c7355e40 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.882+0000 7f6704da1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d1c726a500 con 0x55d1c734c330 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.882+0000 7f6704da1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d1c734d790 con 0x55d1c734cce0 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.882+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734c330 0x55d1c734ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f66faffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c734f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734c330 0x55d1c734ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42134/0 (socket says 172.21.15.82:42134) 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f66fb7fe640 1 -- 172.21.15.82:0/3097221493 learned_addr learned my addr 172.21.15.82:0/3097221493 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f66fbfff640 1 --2- 172.21.15.82:0/3097221493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c734f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f6704da1740 10 client.0 _ll_register_callbacks cb 0x55d1c7238b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f66fb7fe640 1 -- 172.21.15.82:0/3097221493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 msgr2=0x55d1c734f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/3097221493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c734f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f66fb7fe640 1 -- 172.21.15.82:0/3097221493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 msgr2=0x55d1c734f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/3097221493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c734f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f66fb7fe640 1 -- 172.21.15.82:0/3097221493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c6fc33f0 con 0x55d1c734c330 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/3097221493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734c330 0x55d1c734ec90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0114c0 tx=0x7f66ec019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c44227d69587f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f66daffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f66daffd640 1 -- 172.21.15.82:0/3097221493 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec01e040 con 0x55d1c734c330 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.883+0000 7f6704da1740 1 -- 172.21.15.82:0/3097221493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ec00fa70 con 0x55d1c734c330 2024-04-21T08:48:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.884+0000 7f66daffd640 1 -- 172.21.15.82:0/3097221493 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f66ec00f700 con 0x55d1c734c330 2024-04-21T08:48:45.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.884+0000 7f66daffd640 1 -- 172.21.15.82:0/3097221493 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f66ec02f050 con 0x55d1c734c330 2024-04-21T08:48:45.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.884+0000 7f6704da1740 10 client.25892 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:45.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.884+0000 7f6704da1740 1 -- 172.21.15.82:0/3097221493 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1c708d7d0 con 0x55d1c734c330 2024-04-21T08:48:45.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.884+0000 7f66daffd640 1 -- 172.21.15.82:0/3097221493 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec00fa70 con 0x55d1c734c330 2024-04-21T08:48:45.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.884+0000 7f66daffd640 1 -- 172.21.15.82:0/3097221493 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f66ec02a030 con 0x55d1c734c330 2024-04-21T08:48:45.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.884+0000 7f6704da1740 10 client.25892 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:45.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.884+0000 7f6704da1740 10 client.25892 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:48:45.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.884+0000 7f6704da1740 20 client.25892 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:45.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.884+0000 7f6704da1740 1 --2- 172.21.15.82:0/3097221493 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55d1c7365ab0 0x55d1c7367ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.884+0000 7f6704da1740 4 client.25892 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:45.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.884+0000 7f6704da1740 1 -- 172.21.15.82:0/3097221493 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1c7250930 con 0x55d1c7365ab0 2024-04-21T08:48:45.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.885+0000 7f66faffd640 1 --2- 172.21.15.82:0/3097221493 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55d1c7365ab0 0x55d1c7367ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.885+0000 7f66faffd640 1 --2- 172.21.15.82:0/3097221493 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55d1c7365ab0 0x55d1c7367ea0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e92ca70b4071b26 server_cookie=d1d8a9fb079ffb4b in_seq=0 out_seq=0 2024-04-21T08:48:45.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.885+0000 7f66daffd640 10 client.25892 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:48:45.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f66daffd640 1 -- 172.21.15.82:0/3097221493 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1c7250930 con 0x55d1c7365ab0 2024-04-21T08:48:45.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f66daffd640 10 client.25892 handle_command_reply: tid=0 2024-04-21T08:48:45.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f6704da1740 1 client.25892 shutdown 2024-04-21T08:48:45.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f6704da1740 1 -- 172.21.15.82:0/3097221493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734c330 msgr2=0x55d1c734ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f6704da1740 1 --2- 172.21.15.82:0/3097221493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734c330 0x55d1c734ec90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0114c0 tx=0x7f66ec019640 comp rx=0 tx=0).stop 2024-04-21T08:48:45.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f6704da1740 1 -- 172.21.15.82:0/3097221493 shutdown_connections 2024-04-21T08:48:45.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f6704da1740 1 --2- 172.21.15.82:0/3097221493 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55d1c7365ab0 0x55d1c7367ea0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f6704da1740 1 --2- 172.21.15.82:0/3097221493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c734f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f6704da1740 1 --2- 172.21.15.82:0/3097221493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c734f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f6704da1740 1 --2- 172.21.15.82:0/3097221493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734c330 0x55d1c734ec90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f6704da1740 1 -- 172.21.15.82:0/3097221493 >> 172.21.15.82:0/3097221493 conn(0x55d1c7346ba0 msgr2=0x55d1c73488f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f66daffd640 0 client.25892 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:48:45.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f6704da1740 1 -- 172.21.15.82:0/3097221493 shutdown_connections 2024-04-21T08:48:45.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f6704da1740 1 -- 172.21.15.82:0/3097221493 wait complete. 2024-04-21T08:48:45.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.886+0000 7f6704da1740 20 client.25892 trim_cache size 0 max 16384 2024-04-21T08:48:45.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f6704da1740 1 Processor -- start 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f6704da1740 1 -- start start 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f6704da1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c734c330 0x55d1c7355330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f6704da1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c734f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f6704da1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c7355e40 0x55d1c736d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f6704da1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1ad230 con 0x55d1c734c330 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f6704da1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1aee60 con 0x55d1c7355e40 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f6704da1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1ad900 con 0x55d1c734cce0 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c734c330 0x55d1c7355330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c734c330 0x55d1c7355330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34818/0 (socket says 172.21.15.82:34818) 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f66fbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c7355e40 0x55d1c736d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f66fb7fe640 1 -- 172.21.15.82:0/81522131 learned_addr learned my addr 172.21.15.82:0/81522131 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f66faffd640 1 --2- 172.21.15.82:0/81522131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c734f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f66fb7fe640 1 -- 172.21.15.82:0/81522131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 msgr2=0x55d1c734f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/81522131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c734f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f66fb7fe640 1 -- 172.21.15.82:0/81522131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c7355e40 msgr2=0x55d1c736d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/81522131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c7355e40 0x55d1c736d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f66fb7fe640 1 -- 172.21.15.82:0/81522131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1c734b560 con 0x55d1c734c330 2024-04-21T08:48:45.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.901+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/81522131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c734c330 0x55d1c7355330 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f66ec005720 tx=0x7f66ec0059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343dce22ba5a3801 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.902+0000 7f66e17fa640 1 -- 172.21.15.82:0/81522131 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec01e040 con 0x55d1c734c330 2024-04-21T08:48:45.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.902+0000 7f66e17fa640 1 -- 172.21.15.82:0/81522131 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec019980 con 0x55d1c734c330 2024-04-21T08:48:45.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.902+0000 7f66e17fa640 1 -- 172.21.15.82:0/81522131 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec003830 con 0x55d1c734c330 2024-04-21T08:48:45.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.903+0000 7f6704da1740 1 -- 172.21.15.82:0/81522131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c734c330 msgr2=0x55d1c7355330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.903+0000 7f6704da1740 1 --2- 172.21.15.82:0/81522131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c734c330 0x55d1c7355330 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f66ec005720 tx=0x7f66ec0059b0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.903+0000 7f6704da1740 1 -- 172.21.15.82:0/81522131 shutdown_connections 2024-04-21T08:48:45.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.903+0000 7f6704da1740 1 --2- 172.21.15.82:0/81522131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c7355e40 0x55d1c736d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.903+0000 7f6704da1740 1 --2- 172.21.15.82:0/81522131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734cce0 0x55d1c734f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.903+0000 7f6704da1740 1 --2- 172.21.15.82:0/81522131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c734c330 0x55d1c7355330 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.903+0000 7f6704da1740 1 -- 172.21.15.82:0/81522131 >> 172.21.15.82:0/81522131 conn(0x55d1c7346ba0 msgr2=0x55d1c7345c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.903+0000 7f6704da1740 1 -- 172.21.15.82:0/81522131 shutdown_connections 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.903+0000 7f6704da1740 1 -- 172.21.15.82:0/81522131 wait complete. 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.903+0000 7f6704da1740 1 -- start start 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.903+0000 7f6704da1740 1 Processor -- start 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f6704da1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734c330 0x55d1c7361b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f6704da1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734cce0 0x55d1c73620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f6704da1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c735a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f6704da1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1ae760 con 0x55d1c7355e40 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f6704da1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1ae090 con 0x55d1c734cce0 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f6704da1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1ab210 con 0x55d1c734c330 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f6704da1740 10 client.0 _ll_register_callbacks cb 0x7f66f4009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66faffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734cce0 0x55d1c73620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734c330 0x55d1c7361b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66faffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734cce0 0x55d1c73620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42152/0 (socket says 172.21.15.82:42152) 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66fb7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734c330 0x55d1c7361b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:51108/0 (socket says 172.21.15.82:51108) 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66faffd640 1 -- 172.21.15.82:0/3182154575 learned_addr learned my addr 172.21.15.82:0/3182154575 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:45.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66fbfff640 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c735a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66fbfff640 1 -- 172.21.15.82:0/3182154575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734c330 msgr2=0x55d1c7361b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66fbfff640 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734c330 0x55d1c7361b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66fbfff640 1 -- 172.21.15.82:0/3182154575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734cce0 msgr2=0x55d1c73620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66fbfff640 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734cce0 0x55d1c73620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66fbfff640 1 -- 172.21.15.82:0/3182154575 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c7362970 con 0x55d1c7355e40 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66faffd640 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734cce0 0x55d1c73620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734c330 0x55d1c7361b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66fbfff640 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c735a160 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f66f40127c0 tx=0x7f66f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f079d5bcdb9b4870 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.904+0000 7f66daffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f66daffd640 1 -- 172.21.15.82:0/3182154575 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4004770 con 0x55d1c7355e40 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f66daffd640 1 -- 172.21.15.82:0/3182154575 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f66f4018030 con 0x55d1c7355e40 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f6704da1740 1 -- 172.21.15.82:0/3182154575 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ec019980 con 0x55d1c7355e40 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f66daffd640 1 -- 172.21.15.82:0/3182154575 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f66f4037050 con 0x55d1c7355e40 2024-04-21T08:48:45.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f6704da1740 10 client.16560 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f6704da1740 1 -- 172.21.15.82:0/3182154575 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d1c73cb580 con 0x55d1c7355e40 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f66daffd640 1 -- 172.21.15.82:0/3182154575 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec019980 con 0x55d1c7355e40 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f66daffd640 1 -- 172.21.15.82:0/3182154575 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f66f40055b0 con 0x55d1c7355e40 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f6704da1740 10 client.16560 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f6704da1740 10 client.16560 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f6704da1740 20 client.16560 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f6704da1740 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55d1c73cbd20 0x55d1c73ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f6704da1740 4 client.16560 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.905+0000 7f6704da1740 1 -- 172.21.15.82:0/3182154575 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1c72bec10 con 0x55d1c73cbd20 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.906+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55d1c73cbd20 0x55d1c73ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.906+0000 7f66daffd640 10 client.16560 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.906+0000 7f66fb7fe640 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55d1c73cbd20 0x55d1c73ce140 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=421c056672de0be4 server_cookie=976fb89e71c6e032 in_seq=0 out_seq=0 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.906+0000 7f66daffd640 1 -- 172.21.15.82:0/3182154575 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1c72bec10 con 0x55d1c73cbd20 2024-04-21T08:48:45.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.906+0000 7f66daffd640 10 client.16560 handle_command_reply: tid=0 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.906+0000 7f6704da1740 1 client.16560 shutdown 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f6704da1740 1 -- 172.21.15.82:0/3182154575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 msgr2=0x55d1c735a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f6704da1740 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c735a160 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f66f40127c0 tx=0x7f66f4004a40 comp rx=0 tx=0).stop 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f6704da1740 1 -- 172.21.15.82:0/3182154575 shutdown_connections 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f6704da1740 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55d1c73cbd20 0x55d1c73ce140 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f6704da1740 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1c7355e40 0x55d1c735a160 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f6704da1740 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d1c734cce0 0x55d1c73620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f66daffd640 0 client.16560 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f6704da1740 1 --2- 172.21.15.82:0/3182154575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c734c330 0x55d1c7361b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f6704da1740 1 -- 172.21.15.82:0/3182154575 >> 172.21.15.82:0/3182154575 conn(0x55d1c7346ba0 msgr2=0x55d1c7352130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f6704da1740 1 -- 172.21.15.82:0/3182154575 shutdown_connections 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f6704da1740 1 -- 172.21.15.82:0/3182154575 wait complete. 2024-04-21T08:48:45.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f6704da1740 20 client.16560 trim_cache size 0 max 16384 2024-04-21T08:48:45.912 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:48:45.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f67020d2640 1 -- 172.21.15.82:0/1730500378 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66e40a3d20 msgr2=0x7f66e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f67020d2640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66e40a3d20 0x7f66e40a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009fd0 tx=0x7f66f4009500 comp rx=0 tx=0).stop 2024-04-21T08:48:45.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f67020d2640 1 -- 172.21.15.82:0/1730500378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66fc06ace0 msgr2=0x7f66fc19e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f67020d2640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66fc06ace0 0x7f66fc19e920 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f66f0000c00 tx=0x7f66f002c660 comp rx=0 tx=0).stop 2024-04-21T08:48:45.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f67020d2640 1 -- 172.21.15.82:0/1730500378 shutdown_connections 2024-04-21T08:48:45.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f67020d2640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66fc107fd0 0x7f66fc198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f67020d2640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66e40a3d20 0x7f66e40a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f67020d2640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66fc06ace0 0x7f66fc19e920 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f67020d2640 1 --2- 172.21.15.82:0/1730500378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66fc06a360 0x7f66fc19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.907+0000 7f67020d2640 1 -- 172.21.15.82:0/1730500378 >> 172.21.15.82:0/1730500378 conn(0x7f66fc077ef0 msgr2=0x7f66fc0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.908+0000 7f67020d2640 1 -- 172.21.15.82:0/1730500378 shutdown_connections 2024-04-21T08:48:45.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:45.908+0000 7f67020d2640 1 -- 172.21.15.82:0/1730500378 wait complete. 2024-04-21T08:48:45.919 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:48:46.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.035+0000 7fe7b78e0640 1 -- 172.21.15.82:0/1121070424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0106160 msgr2=0x7fe7b010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:46.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.035+0000 7fe7b78e0640 1 --2- 172.21.15.82:0/1121070424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0106160 0x7fe7b010dd30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009ca0 tx=0x7fe7a802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.035+0000 7fe7b78e0640 1 -- 172.21.15.82:0/1121070424 shutdown_connections 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.035+0000 7fe7b78e0640 1 --2- 172.21.15.82:0/1121070424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b0106160 0x7fe7b010dd30 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.035+0000 7fe7b78e0640 1 --2- 172.21.15.82:0/1121070424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b0101a40 0x7fe7b0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.035+0000 7fe7b78e0640 1 --2- 172.21.15.82:0/1121070424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b0101090 0x7fe7b0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.035+0000 7fe7b78e0640 1 -- 172.21.15.82:0/1121070424 >> 172.21.15.82:0/1121070424 conn(0x7fe7b0077f50 msgr2=0x7fe7b0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.035+0000 7fe7b78e0640 1 -- 172.21.15.82:0/1121070424 shutdown_connections 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.035+0000 7fe7b78e0640 1 -- 172.21.15.82:0/1121070424 wait complete. 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.036+0000 7fe7b78e0640 1 Processor -- start 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.036+0000 7fe7b78e0640 1 -- start start 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.036+0000 7fe7b78e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b0101090 0x7fe7b0102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.036+0000 7fe7b78e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b0106160 0x7fe7b0103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.036+0000 7fe7b78e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b01035a0 0x7fe7b01a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.036+0000 7fe7b78e0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01110d0 con 0x7fe7b01035a0 2024-04-21T08:48:46.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.036+0000 7fe7b78e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b0110f50 con 0x7fe7b0106160 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.036+0000 7fe7b78e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b0111250 con 0x7fe7b0101090 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.036+0000 7fe7b5e7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b01035a0 0x7fe7b01a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.036+0000 7fe7b4e7b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b0106160 0x7fe7b0103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.036+0000 7fe7b5e7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b01035a0 0x7fe7b01a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34834/0 (socket says 172.21.15.82:34834) 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.036+0000 7fe7b5e7d640 1 -- 172.21.15.82:0/2473505824 learned_addr learned my addr 172.21.15.82:0/2473505824 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.037+0000 7fe7b567c640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b0101090 0x7fe7b0102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.037+0000 7fe7b5e7d640 1 -- 172.21.15.82:0/2473505824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b0101090 msgr2=0x7fe7b0102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.037+0000 7fe7b5e7d640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b0101090 0x7fe7b0102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.037+0000 7fe7b5e7d640 1 -- 172.21.15.82:0/2473505824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b0106160 msgr2=0x7fe7b0103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.037+0000 7fe7b5e7d640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b0106160 0x7fe7b0103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.037+0000 7fe7b5e7d640 1 -- 172.21.15.82:0/2473505824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b01a6fe0 con 0x7fe7b01035a0 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.037+0000 7fe7b5e7d640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b01035a0 0x7fe7b01a68e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8005e30 tx=0x7fe7a80346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:46.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.037+0000 7fe7a67fc640 1 -- 172.21.15.82:0/2473505824 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8041020 con 0x7fe7b01035a0 2024-04-21T08:48:46.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.037+0000 7fe7a67fc640 1 -- 172.21.15.82:0/2473505824 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a8032970 con 0x7fe7b01035a0 2024-04-21T08:48:46.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.037+0000 7fe7b78e0640 1 -- 172.21.15.82:0/2473505824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b01a7270 con 0x7fe7b01035a0 2024-04-21T08:48:46.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.037+0000 7fe7a67fc640 1 -- 172.21.15.82:0/2473505824 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8039040 con 0x7fe7b01035a0 2024-04-21T08:48:46.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.037+0000 7fe7b78e0640 1 -- 172.21.15.82:0/2473505824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b01a77b0 con 0x7fe7b01035a0 2024-04-21T08:48:46.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.039+0000 7fe7a67fc640 1 -- 172.21.15.82:0/2473505824 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe7a8044020 con 0x7fe7b01035a0 2024-04-21T08:48:46.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7ba5ac740 1 Processor -- start 2024-04-21T08:48:46.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7ba5ac740 1 -- start start 2024-04-21T08:48:46.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7a67fc640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe79c0a3dd0 0x7fe79c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7ba5ac740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c802f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7ba5ac740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802fc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7ba5ac740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802fee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7ba5ac740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561c802581f0 con 0x561c802f34d0 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7ba5ac740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561c801f54b0 con 0x561c802fca00 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7ba5ac740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c802541b0 con 0x561c802f3e80 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7b4e7b640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe79c0a3dd0 0x7fe79c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7b567c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c802f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7b567c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c802f38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34836/0 (socket says 172.21.15.82:34836) 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7a67fc640 1 -- 172.21.15.82:0/2473505824 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe7a80f25e0 con 0x7fe7b01035a0 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7b567c640 1 -- 172.21.15.82:0/1899388616 learned_addr learned my addr 172.21.15.82:0/1899388616 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7b5e7d640 1 --2- 172.21.15.82:0/1899388616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.040+0000 7fe7b4e7b640 1 --2- 172.21.15.82:0/1899388616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.041+0000 7fe7b567c640 1 -- 172.21.15.82:0/1899388616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 msgr2=0x561c802fc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.041+0000 7fe7b567c640 1 --2- 172.21.15.82:0/1899388616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802fc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.041+0000 7fe7b567c640 1 -- 172.21.15.82:0/1899388616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 msgr2=0x561c802fee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.041+0000 7fe7b567c640 1 --2- 172.21.15.82:0/1899388616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802fee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.041+0000 7fe7b567c640 1 -- 172.21.15.82:0/1899388616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c801e71b0 con 0x561c802f34d0 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.041+0000 7fe7b4e7b640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe79c0a3dd0 0x7fe79c0a6290 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe7b01045b0 tx=0x7fe7a0009f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.041+0000 7fe7b567c640 1 --2- 172.21.15.82:0/1899388616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c802f38b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009d20 tx=0x7fe7ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7704c0c1d6068c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.041+0000 7fe79b7fe640 1 -- 172.21.15.82:0/1899388616 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac01e070 con 0x561c802f34d0 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.041+0000 7fe79b7fe640 1 -- 172.21.15.82:0/1899388616 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7ac009d50 con 0x561c802f34d0 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.041+0000 7fe79b7fe640 1 -- 172.21.15.82:0/1899388616 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac018730 con 0x561c802f34d0 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.042+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/1899388616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 msgr2=0x561c802f38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.042+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/1899388616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c802f38b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009d20 tx=0x7fe7ac011810 comp rx=0 tx=0).stop 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.042+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/1899388616 shutdown_connections 2024-04-21T08:48:46.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.042+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/1899388616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.042+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/1899388616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.042+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/1899388616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c802f38b0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.042+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/1899388616 >> 172.21.15.82:0/1899388616 conn(0x561c802edba0 msgr2=0x561c802edf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.042+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/1899388616 shutdown_connections 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.042+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/1899388616 wait complete. 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7ba5ac740 1 -- start start 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7ba5ac740 1 Processor -- start 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7ba5ac740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f34d0 0x561c802fc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7ba5ac740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802f3e80 0x561c802f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7ba5ac740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802fca00 0x561c802f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7ba5ac740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561c802ff970 con 0x561c802fca00 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7ba5ac740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561c80211500 con 0x561c802f3e80 2024-04-21T08:48:46.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7ba5ac740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c802ffaf0 con 0x561c802f34d0 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7ba5ac740 10 client.0 _ll_register_callbacks cb 0x561c801dfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7b4e7b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802f3e80 0x561c802f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7b567c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f34d0 0x561c802fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7b4e7b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802f3e80 0x561c802f96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42196/0 (socket says 172.21.15.82:42196) 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7b5e7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802fca00 0x561c802f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.043+0000 7fe7b4e7b640 1 -- 172.21.15.82:0/3549595150 learned_addr learned my addr 172.21.15.82:0/3549595150 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe7b5e7d640 1 -- 172.21.15.82:0/3549595150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f34d0 msgr2=0x561c802fc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe7b5e7d640 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f34d0 0x561c802fc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe7b5e7d640 1 -- 172.21.15.82:0/3549595150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802f3e80 msgr2=0x561c802f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe7b5e7d640 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802f3e80 0x561c802f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe7b5e7d640 1 -- 172.21.15.82:0/3549595150 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c7ff6a3f0 con 0x561c802fca00 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe7b4e7b640 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802f3e80 0x561c802f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe7b567c640 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f34d0 0x561c802fc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:46.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe7b5e7d640 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802fca00 0x561c802f9cd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80efa80 tx=0x7fe7a80be160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d036f4a65152ed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe798ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe798ff9640 1 -- 172.21.15.82:0/3549595150 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80348f0 con 0x561c802fca00 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe798ff9640 1 -- 172.21.15.82:0/3549595150 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe7a80a9310 con 0x561c802fca00 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/3549595150 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7ac009d50 con 0x561c802fca00 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe798ff9640 1 -- 172.21.15.82:0/3549595150 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe7a8003e30 con 0x561c802fca00 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe7ba5ac740 10 client.16572 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.044+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/3549595150 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c800347d0 con 0x561c802fca00 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.045+0000 7fe798ff9640 1 -- 172.21.15.82:0/3549595150 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7ac009d50 con 0x561c802fca00 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.045+0000 7fe798ff9640 1 -- 172.21.15.82:0/3549595150 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fe7a803c030 con 0x561c802fca00 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.045+0000 7fe7ba5ac740 10 client.16572 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.045+0000 7fe7ba5ac740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.045+0000 7fe7ba5ac740 20 client.16572 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.045+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x561c8030cc70 0x561c8030f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.045+0000 7fe7ba5ac740 4 client.16572 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:46.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.045+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/3549595150 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c801f7930 con 0x561c8030cc70 2024-04-21T08:48:46.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.045+0000 7fe7b567c640 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x561c8030cc70 0x561c8030f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.045+0000 7fe798ff9640 10 client.16572 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:46.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.045+0000 7fe7b567c640 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x561c8030cc70 0x561c8030f0d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d86a055a4f664a23 server_cookie=17f54718704870cf in_seq=0 out_seq=0 2024-04-21T08:48:46.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.046+0000 7fe798ff9640 1 -- 172.21.15.82:0/3549595150 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c801f7930 con 0x561c8030cc70 2024-04-21T08:48:46.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.046+0000 7fe798ff9640 10 client.16572 handle_command_reply: tid=0 2024-04-21T08:48:46.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.046+0000 7fe7ba5ac740 1 client.16572 shutdown 2024-04-21T08:48:46.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.046+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/3549595150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802fca00 msgr2=0x561c802f9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:46.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.046+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802fca00 0x561c802f9cd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80efa80 tx=0x7fe7a80be160 comp rx=0 tx=0).stop 2024-04-21T08:48:46.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.046+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/3549595150 shutdown_connections 2024-04-21T08:48:46.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.046+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x561c8030cc70 0x561c8030f0d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.046+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802fca00 0x561c802f9cd0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.046+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802f3e80 0x561c802f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.046+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/3549595150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f34d0 0x561c802fc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.046+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/3549595150 >> 172.21.15.82:0/3549595150 conn(0x561c802edba0 msgr2=0x561c802ef8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:46.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.046+0000 7fe798ff9640 0 client.16572 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:46.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.047+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/3549595150 shutdown_connections 2024-04-21T08:48:46.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.047+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/3549595150 wait complete. 2024-04-21T08:48:46.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.047+0000 7fe7ba5ac740 20 client.16572 trim_cache size 0 max 16384 2024-04-21T08:48:46.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7ba5ac740 1 Processor -- start 2024-04-21T08:48:46.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7ba5ac740 1 -- start start 2024-04-21T08:48:46.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7ba5ac740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c80308830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7ba5ac740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7ba5ac740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7ba5ac740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01b1500 con 0x561c802f34d0 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7ba5ac740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01b3130 con 0x561c802fca00 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7ba5ac740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01b1bd0 con 0x561c802f3e80 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7b567c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c80308830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7b4e7b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7b567c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c80308830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34850/0 (socket says 172.21.15.82:34850) 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7b567c640 1 -- 172.21.15.82:0/1430264383 learned_addr learned my addr 172.21.15.82:0/1430264383 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.071+0000 7fe7b5e7d640 1 --2- 172.21.15.82:0/1430264383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.072+0000 7fe7b567c640 1 -- 172.21.15.82:0/1430264383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 msgr2=0x561c802f86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.072+0000 7fe7b567c640 1 --2- 172.21.15.82:0/1430264383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802f86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.072+0000 7fe7b567c640 1 -- 172.21.15.82:0/1430264383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 msgr2=0x561c802fbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.072+0000 7fe7b567c640 1 --2- 172.21.15.82:0/1430264383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802fbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.072+0000 7fe7b567c640 1 -- 172.21.15.82:0/1430264383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c802f3c20 con 0x561c802f34d0 2024-04-21T08:48:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.072+0000 7fe7b4e7b640 1 --2- 172.21.15.82:0/1430264383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:46.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.072+0000 7fe7b567c640 1 --2- 172.21.15.82:0/1430264383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c80308830 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac012fd0 tx=0x7fe7ac009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d074da720d98859 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:46.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.072+0000 7fe79affd640 1 -- 172.21.15.82:0/1430264383 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac02e070 con 0x561c802f34d0 2024-04-21T08:48:46.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.072+0000 7fe79affd640 1 -- 172.21.15.82:0/1430264383 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7ac0054f0 con 0x561c802f34d0 2024-04-21T08:48:46.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.072+0000 7fe79affd640 1 -- 172.21.15.82:0/1430264383 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac01e040 con 0x561c802f34d0 2024-04-21T08:48:46.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.073+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/1430264383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 msgr2=0x561c80308830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:46.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.073+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/1430264383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c80308830 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac012fd0 tx=0x7fe7ac009c70 comp rx=0 tx=0).stop 2024-04-21T08:48:46.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.073+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/1430264383 shutdown_connections 2024-04-21T08:48:46.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.073+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/1430264383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802fbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.073+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/1430264383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.073+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/1430264383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c80308830 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.073+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/1430264383 >> 172.21.15.82:0/1430264383 conn(0x561c802edba0 msgr2=0x561c802f7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:46.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.073+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/1430264383 shutdown_connections 2024-04-21T08:48:46.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.073+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/1430264383 wait complete. 2024-04-21T08:48:46.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.073+0000 7fe7ba5ac740 1 -- start start 2024-04-21T08:48:46.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.073+0000 7fe7ba5ac740 1 Processor -- start 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7ba5ac740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c80304f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7ba5ac740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802f5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7ba5ac740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7ba5ac740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01b2a30 con 0x561c802f34d0 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7ba5ac740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01b2360 con 0x561c802fca00 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7ba5ac740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b01af4e0 con 0x561c802f3e80 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7ba5ac740 10 client.0 _ll_register_callbacks cb 0x7fe7a80ba570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7b567c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c80304f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7b5e7d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7b5e7d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42212/0 (socket says 172.21.15.82:42212) 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7b567c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c80304f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34864/0 (socket says 172.21.15.82:34864) 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7b5e7d640 1 -- 172.21.15.82:0/922526249 learned_addr learned my addr 172.21.15.82:0/922526249 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7b4e7b640 1 --2- 172.21.15.82:0/922526249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7b5e7d640 1 -- 172.21.15.82:0/922526249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 msgr2=0x561c802f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:46.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7b5e7d640 1 --2- 172.21.15.82:0/922526249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7b5e7d640 1 -- 172.21.15.82:0/922526249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 msgr2=0x561c80304f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7b5e7d640 1 --2- 172.21.15.82:0/922526249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c80304f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.074+0000 7fe7b5e7d640 1 -- 172.21.15.82:0/922526249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c802f64b0 con 0x561c802fca00 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe7b5e7d640 1 --2- 172.21.15.82:0/922526249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802f5e70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8032460 tx=0x7fe7a80be1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0883dc47f00ceb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe798ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe798ff9640 1 -- 172.21.15.82:0/922526249 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80bdc30 con 0x561c802fca00 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/922526249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7ac0054f0 con 0x561c802fca00 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe798ff9640 1 -- 172.21.15.82:0/922526249 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe7a80324c0 con 0x561c802fca00 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe798ff9640 1 -- 172.21.15.82:0/922526249 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe7a80348f0 con 0x561c802fca00 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe7ba5ac740 10 client.25916 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/922526249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c80372250 con 0x561c802fca00 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe798ff9640 1 -- 172.21.15.82:0/922526249 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7ac0054f0 con 0x561c802fca00 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe798ff9640 1 -- 172.21.15.82:0/922526249 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fe7a80bd720 con 0x561c802fca00 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe7ba5ac740 10 client.25916 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:46.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe7ba5ac740 10 client.25916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe7ba5ac740 20 client.25916 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/922526249 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x561c80372670 0x561c80374a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe7ba5ac740 4 client.25916 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.075+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/922526249 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c80265c10 con 0x561c80372670 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7b567c640 1 --2- 172.21.15.82:0/922526249 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x561c80372670 0x561c80374a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7b567c640 1 --2- 172.21.15.82:0/922526249 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x561c80372670 0x561c80374a90 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e20e2310941175e server_cookie=9fa76684cfdc20c1 in_seq=0 out_seq=0 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe798ff9640 10 client.25916 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe798ff9640 1 -- 172.21.15.82:0/922526249 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c80265c10 con 0x561c80372670 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe798ff9640 10 client.25916 handle_command_reply: tid=0 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7ba5ac740 1 client.25916 shutdown 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/922526249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 msgr2=0x561c802f5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/922526249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802f5e70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8032460 tx=0x7fe7a80be1d0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/922526249 shutdown_connections 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/922526249 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x561c80372670 0x561c80374a90 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/922526249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c802fca00 0x561c802f5e70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/922526249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c802f3e80 0x561c802f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7ba5ac740 1 --2- 172.21.15.82:0/922526249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c802f34d0 0x561c80304f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe798ff9640 0 client.25916 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/922526249 >> 172.21.15.82:0/922526249 conn(0x561c802edba0 msgr2=0x561c802ec370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/922526249 shutdown_connections 2024-04-21T08:48:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.076+0000 7fe7ba5ac740 1 -- 172.21.15.82:0/922526249 wait complete. 2024-04-21T08:48:46.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7ba5ac740 20 client.25916 trim_cache size 0 max 16384 2024-04-21T08:48:46.082 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:48:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7b78e0640 1 -- 172.21.15.82:0/2473505824 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe79c0a3dd0 msgr2=0x7fe79c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7b78e0640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe79c0a3dd0 0x7fe79c0a6290 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe7b01045b0 tx=0x7fe7a0009f90 comp rx=0 tx=0).stop 2024-04-21T08:48:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7b78e0640 1 -- 172.21.15.82:0/2473505824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b01035a0 msgr2=0x7fe7b01a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7b78e0640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b01035a0 0x7fe7b01a68e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8005e30 tx=0x7fe7a80346d0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7b78e0640 1 -- 172.21.15.82:0/2473505824 shutdown_connections 2024-04-21T08:48:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7b78e0640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe79c0a3dd0 0x7fe79c0a6290 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7b78e0640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b01035a0 0x7fe7b01a68e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7b78e0640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b0106160 0x7fe7b0103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7b78e0640 1 --2- 172.21.15.82:0/2473505824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7b0101090 0x7fe7b0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:46.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7b78e0640 1 -- 172.21.15.82:0/2473505824 >> 172.21.15.82:0/2473505824 conn(0x7fe7b0077f50 msgr2=0x7fe7b0069450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:46.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7b78e0640 1 -- 172.21.15.82:0/2473505824 shutdown_connections 2024-04-21T08:48:46.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:46.077+0000 7fe7b78e0640 1 -- 172.21.15.82:0/2473505824 wait complete. 2024-04-21T08:48:46.222 INFO:tasks.workunit.client.0.smithi082.stdout: 41 627416 595 395638 505 349421 1462 2024-04-21T08:48:46.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:46 smithi082 ceph-mon[27967]: pgmap v55: 129 pgs: 129 active+clean; 135 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 272 op/s 2024-04-21T08:48:46.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:46 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4253732172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:46.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:46 smithi106 ceph-mon[31072]: pgmap v55: 129 pgs: 129 active+clean; 135 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 272 op/s 2024-04-21T08:48:46.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:46 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/4253732172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:46.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:46 smithi134 ceph-mon[31772]: pgmap v55: 129 pgs: 129 active+clean; 135 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 272 op/s 2024-04-21T08:48:46.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:46 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/4253732172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:47.091 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:48:47.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:48:47.092 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:48:47.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.336+0000 7f83d0ce4640 1 -- 172.21.15.82:0/2559056254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83cc1107a0 msgr2=0x7f83cc10e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.336+0000 7f83d0ce4640 1 --2- 172.21.15.82:0/2559056254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83cc1107a0 0x7f83cc10e290 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009cd0 tx=0x7f83bc02c0e0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.337+0000 7f83d0ce4640 1 -- 172.21.15.82:0/2559056254 shutdown_connections 2024-04-21T08:48:47.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.337+0000 7f83d0ce4640 1 --2- 172.21.15.82:0/2559056254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc11b730 0x7f83cc11db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.337+0000 7f83d0ce4640 1 --2- 172.21.15.82:0/2559056254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc10e7d0 0x7f83cc10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.337+0000 7f83d0ce4640 1 --2- 172.21.15.82:0/2559056254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83cc1107a0 0x7f83cc10e290 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.337+0000 7f83d0ce4640 1 -- 172.21.15.82:0/2559056254 >> 172.21.15.82:0/2559056254 conn(0x7f83cc06d060 msgr2=0x7f83cc06d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:47.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.338+0000 7f83d0ce4640 1 -- 172.21.15.82:0/2559056254 shutdown_connections 2024-04-21T08:48:47.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.338+0000 7f83d0ce4640 1 -- 172.21.15.82:0/2559056254 wait complete. 2024-04-21T08:48:47.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.339+0000 7f83d0ce4640 1 Processor -- start 2024-04-21T08:48:47.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.342+0000 7f83d0ce4640 1 -- start start 2024-04-21T08:48:47.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.342+0000 7f83d0ce4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc10e7d0 0x7f83cc1a7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.342+0000 7f83d0ce4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83cc1107a0 0x7f83cc1a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.342+0000 7f83d0ce4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc11b730 0x7f83cc1abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.342+0000 7f83d0ce4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc121020 con 0x7f83cc11b730 2024-04-21T08:48:47.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.342+0000 7f83d0ce4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc120ea0 con 0x7f83cc1107a0 2024-04-21T08:48:47.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.342+0000 7f83d0ce4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1211a0 con 0x7f83cc10e7d0 2024-04-21T08:48:47.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.343+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83cc1107a0 0x7f83cc1a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.343+0000 7f83ca59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc10e7d0 0x7f83cc1a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.343+0000 7f83ca59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc10e7d0 0x7f83cc1a7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:51168/0 (socket says 172.21.15.82:51168) 2024-04-21T08:48:47.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.343+0000 7f83ca59c640 1 -- 172.21.15.82:0/1734467029 learned_addr learned my addr 172.21.15.82:0/1734467029 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:47.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.345+0000 7f83cad9d640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc11b730 0x7f83cc1abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.345+0000 7f83ca59c640 1 -- 172.21.15.82:0/1734467029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83cc1107a0 msgr2=0x7f83cc1a7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.345+0000 7f83ca59c640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83cc1107a0 0x7f83cc1a7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.345+0000 7f83ca59c640 1 -- 172.21.15.82:0/1734467029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc11b730 msgr2=0x7f83cc1abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.345+0000 7f83ca59c640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc11b730 0x7f83cc1abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.346+0000 7f83ca59c640 1 -- 172.21.15.82:0/1734467029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83cc1ac630 con 0x7f83cc10e7d0 2024-04-21T08:48:47.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.347+0000 7f83ca59c640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc10e7d0 0x7f83cc1a7640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009ca0 tx=0x7f83bc035ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:47.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.351+0000 7f83bb7fe640 1 -- 172.21.15.82:0/1734467029 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc039040 con 0x7f83cc10e7d0 2024-04-21T08:48:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.351+0000 7f83bb7fe640 1 -- 172.21.15.82:0/1734467029 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83bc044070 con 0x7f83cc10e7d0 2024-04-21T08:48:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.351+0000 7f83bb7fe640 1 -- 172.21.15.82:0/1734467029 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc0042a0 con 0x7f83cc10e7d0 2024-04-21T08:48:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.351+0000 7f83d0ce4640 1 -- 172.21.15.82:0/1734467029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83cc1ac8c0 con 0x7f83cc10e7d0 2024-04-21T08:48:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.351+0000 7f83d0ce4640 1 -- 172.21.15.82:0/1734467029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83cc1b41b0 con 0x7f83cc10e7d0 2024-04-21T08:48:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.358+0000 7f83d39b3740 1 Processor -- start 2024-04-21T08:48:47.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.362+0000 7f83d39b3740 1 -- start start 2024-04-21T08:48:47.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.362+0000 7f83d39b3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3edacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.362+0000 7f83d39b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3edb290 0x55d5c3ee3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.362+0000 7f83d39b3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.362+0000 7f83d39b3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d5c3e3ef00 con 0x55d5c3eda8e0 2024-04-21T08:48:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.362+0000 7f83d39b3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d5c3ddc590 con 0x55d5c3ee3e10 2024-04-21T08:48:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.362+0000 7f83d39b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d5c3e3aec0 con 0x55d5c3edb290 2024-04-21T08:48:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.363+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.363+0000 7f83cad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3edacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.364+0000 7f83cad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3edacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34878/0 (socket says 172.21.15.82:34878) 2024-04-21T08:48:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.364+0000 7f83cad9d640 1 -- 172.21.15.82:0/1903917841 learned_addr learned my addr 172.21.15.82:0/1903917841 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.364+0000 7f83bb7fe640 1 -- 172.21.15.82:0/1734467029 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f83bc004440 con 0x7f83cc10e7d0 2024-04-21T08:48:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.364+0000 7f83ca59c640 1 --2- 172.21.15.82:0/1903917841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3edb290 0x55d5c3ee3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.365+0000 7f83bb7fe640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f83b40a3f20 0x7f83b40a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.366+0000 7f83cad9d640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f83b40a3f20 0x7f83b40a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.366+0000 7f83cad9d640 1 -- 172.21.15.82:0/1903917841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3edb290 msgr2=0x55d5c3ee3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.366+0000 7f83cad9d640 1 --2- 172.21.15.82:0/1903917841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3edb290 0x55d5c3ee3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.366+0000 7f83cad9d640 1 -- 172.21.15.82:0/1903917841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3ee3e10 msgr2=0x55d5c3ee6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.366+0000 7f83cad9d640 1 --2- 172.21.15.82:0/1903917841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.366+0000 7f83cad9d640 1 -- 172.21.15.82:0/1903917841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5c3c197e0 con 0x55d5c3eda8e0 2024-04-21T08:48:47.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.368+0000 7f83bb7fe640 1 -- 172.21.15.82:0/1734467029 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f83bc0f23c0 con 0x7f83cc10e7d0 2024-04-21T08:48:47.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.370+0000 7f83cad9d640 1 --2- 172.21.15.82:0/1903917841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3edacc0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f83c0015840 tx=0x7f83c0015d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f314cee5ea399e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:47.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.371+0000 7f83cad9d640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f83b40a3f20 0x7f83b40a63e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83c000f000 tx=0x7f83c001f000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:47.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.372+0000 7f83b8ff9640 1 -- 172.21.15.82:0/1903917841 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0009d60 con 0x55d5c3eda8e0 2024-04-21T08:48:47.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.373+0000 7f83b8ff9640 1 -- 172.21.15.82:0/1903917841 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c00044d0 con 0x55d5c3eda8e0 2024-04-21T08:48:47.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.373+0000 7f83b8ff9640 1 -- 172.21.15.82:0/1903917841 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0003da0 con 0x55d5c3eda8e0 2024-04-21T08:48:47.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.374+0000 7f83d39b3740 1 -- 172.21.15.82:0/1903917841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 msgr2=0x55d5c3edacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.374+0000 7f83d39b3740 1 --2- 172.21.15.82:0/1903917841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3edacc0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f83c0015840 tx=0x7f83c0015d00 comp rx=0 tx=0).stop 2024-04-21T08:48:47.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.378+0000 7f83d39b3740 1 -- 172.21.15.82:0/1903917841 shutdown_connections 2024-04-21T08:48:47.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.378+0000 7f83d39b3740 1 --2- 172.21.15.82:0/1903917841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.378+0000 7f83d39b3740 1 --2- 172.21.15.82:0/1903917841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3edb290 0x55d5c3ee3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.378+0000 7f83d39b3740 1 --2- 172.21.15.82:0/1903917841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3edacc0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.378+0000 7f83d39b3740 1 -- 172.21.15.82:0/1903917841 >> 172.21.15.82:0/1903917841 conn(0x55d5c3ed4fb0 msgr2=0x55d5c3ed5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:47.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.381+0000 7f83d39b3740 1 -- 172.21.15.82:0/1903917841 shutdown_connections 2024-04-21T08:48:47.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.382+0000 7f83d39b3740 1 -- 172.21.15.82:0/1903917841 wait complete. 2024-04-21T08:48:47.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.382+0000 7f83d39b3740 1 -- start start 2024-04-21T08:48:47.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.382+0000 7f83d39b3740 1 Processor -- start 2024-04-21T08:48:47.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.383+0000 7f83d39b3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3eda8e0 0x55d5c3ee0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.383+0000 7f83d39b3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3edb290 0x55d5c3ee1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.383+0000 7f83d39b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.383+0000 7f83d39b3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d5c3ee6d20 con 0x55d5c3edb290 2024-04-21T08:48:47.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.383+0000 7f83d39b3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d5c3df8580 con 0x55d5c3eda8e0 2024-04-21T08:48:47.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.383+0000 7f83d39b3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d5c3ee6ea0 con 0x55d5c3ee3e10 2024-04-21T08:48:47.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.383+0000 7f83d39b3740 10 client.0 _ll_register_callbacks cb 0x55d5c3dfe650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:47.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.384+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3edb290 0x55d5c3ee1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.384+0000 7f83ca59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3eda8e0 0x55d5c3ee0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.384+0000 7f83ca59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3eda8e0 0x55d5c3ee0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42260/0 (socket says 172.21.15.82:42260) 2024-04-21T08:48:47.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.384+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3edb290 0x55d5c3ee1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34882/0 (socket says 172.21.15.82:34882) 2024-04-21T08:48:47.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.384+0000 7f83ca59c640 1 -- 172.21.15.82:0/2465408102 learned_addr learned my addr 172.21.15.82:0/2465408102 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:47.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.384+0000 7f83cad9d640 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.385+0000 7f83ca59c640 1 -- 172.21.15.82:0/2465408102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3ee3e10 msgr2=0x55d5c3ee1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.385+0000 7f83ca59c640 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.385+0000 7f83ca59c640 1 -- 172.21.15.82:0/2465408102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3edb290 msgr2=0x55d5c3ee1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.385+0000 7f83ca59c640 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3edb290 0x55d5c3ee1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.385+0000 7f83ca59c640 1 -- 172.21.15.82:0/2465408102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5c3c79f70 con 0x55d5c3eda8e0 2024-04-21T08:48:47.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.386+0000 7f83cad9d640 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:47.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.391+0000 7f83ca59c640 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3eda8e0 0x55d5c3ee0f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f83bc005e30 tx=0x7f83bc032a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27337bd62d40f95e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:47.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.392+0000 7f83b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:48:47.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.393+0000 7f83b1ffb640 1 -- 172.21.15.82:0/2465408102 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc0bde20 con 0x55d5c3eda8e0 2024-04-21T08:48:47.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.393+0000 7f83d39b3740 1 -- 172.21.15.82:0/2465408102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c00044d0 con 0x55d5c3eda8e0 2024-04-21T08:48:47.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.393+0000 7f83b1ffb640 1 -- 172.21.15.82:0/2465408102 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f83bc0be2b0 con 0x55d5c3eda8e0 2024-04-21T08:48:47.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.395+0000 7f83b1ffb640 1 -- 172.21.15.82:0/2465408102 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f83bc0bd9b0 con 0x55d5c3eda8e0 2024-04-21T08:48:47.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.398+0000 7f83d39b3740 10 client.25928 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:47.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.398+0000 7f83d39b3740 1 -- 172.21.15.82:0/2465408102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d5c3b11170 con 0x55d5c3eda8e0 2024-04-21T08:48:47.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.401+0000 7f83b1ffb640 1 -- 172.21.15.82:0/2465408102 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c00044d0 con 0x55d5c3eda8e0 2024-04-21T08:48:47.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.401+0000 7f83b1ffb640 1 -- 172.21.15.82:0/2465408102 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f83bc03c030 con 0x55d5c3eda8e0 2024-04-21T08:48:47.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.401+0000 7f83d39b3740 10 client.25928 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:47.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.401+0000 7f83d39b3740 10 client.25928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:47.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.401+0000 7f83d39b3740 20 client.25928 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:47.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.401+0000 7f83d39b3740 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d5c3ef4590 0x55d5c3ef69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.401+0000 7f83d39b3740 4 client.25928 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:47.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.401+0000 7f83d39b3740 1 -- 172.21.15.82:0/2465408102 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5c3e4c920 con 0x55d5c3ef4590 2024-04-21T08:48:47.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.402+0000 7f83c9d9b640 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d5c3ef4590 0x55d5c3ef69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.412+0000 7f83c9d9b640 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d5c3ef4590 0x55d5c3ef69d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afff7680997ec45d server_cookie=41ffc585f603b362 in_seq=0 out_seq=0 2024-04-21T08:48:47.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.412+0000 7f83b1ffb640 10 client.25928 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:47.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.420+0000 7f83b1ffb640 1 -- 172.21.15.82:0/2465408102 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5c3e4c920 con 0x55d5c3ef4590 2024-04-21T08:48:47.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.420+0000 7f83b1ffb640 10 client.25928 handle_command_reply: tid=0 2024-04-21T08:48:47.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.420+0000 7f83d39b3740 1 client.25928 shutdown 2024-04-21T08:48:47.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.421+0000 7f83d39b3740 1 -- 172.21.15.82:0/2465408102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3eda8e0 msgr2=0x55d5c3ee0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.421+0000 7f83d39b3740 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3eda8e0 0x55d5c3ee0f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f83bc005e30 tx=0x7f83bc032a90 comp rx=0 tx=0).stop 2024-04-21T08:48:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.425+0000 7f83d39b3740 1 -- 172.21.15.82:0/2465408102 shutdown_connections 2024-04-21T08:48:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.425+0000 7f83d39b3740 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d5c3ef4590 0x55d5c3ef69d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.425+0000 7f83d39b3740 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.425+0000 7f83d39b3740 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3edb290 0x55d5c3ee1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.425+0000 7f83d39b3740 1 --2- 172.21.15.82:0/2465408102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3eda8e0 0x55d5c3ee0f00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.425+0000 7f83d39b3740 1 -- 172.21.15.82:0/2465408102 >> 172.21.15.82:0/2465408102 conn(0x55d5c3ed4fb0 msgr2=0x55d5c3ed6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:47.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.425+0000 7f83b1ffb640 0 client.25928 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.428+0000 7f83d39b3740 1 -- 172.21.15.82:0/2465408102 shutdown_connections 2024-04-21T08:48:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.428+0000 7f83d39b3740 1 -- 172.21.15.82:0/2465408102 wait complete. 2024-04-21T08:48:47.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.429+0000 7f83d39b3740 20 client.25928 trim_cache size 0 max 16384 2024-04-21T08:48:47.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.468+0000 7f83d39b3740 1 Processor -- start 2024-04-21T08:48:47.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.469+0000 7f83d39b3740 1 -- start start 2024-04-21T08:48:47.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.469+0000 7f83d39b3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3eefbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.470+0000 7f83d39b3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3edb290 0x55d5c3ee17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.470+0000 7f83d39b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3ee3e10 0x55d5c3efc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.470+0000 7f83d39b3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1be200 con 0x55d5c3eda8e0 2024-04-21T08:48:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.470+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3edb290 0x55d5c3ee17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.470+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3edb290 0x55d5c3ee17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42276/0 (socket says 172.21.15.82:42276) 2024-04-21T08:48:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.470+0000 7f83c9d9b640 1 -- 172.21.15.82:0/4267349317 learned_addr learned my addr 172.21.15.82:0/4267349317 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.470+0000 7f83d39b3740 1 -- 172.21.15.82:0/4267349317 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1bfe30 con 0x55d5c3edb290 2024-04-21T08:48:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.470+0000 7f83cad9d640 1 --2- 172.21.15.82:0/4267349317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3ee3e10 0x55d5c3efc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.470+0000 7f83d39b3740 1 -- 172.21.15.82:0/4267349317 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1be8d0 con 0x55d5c3ee3e10 2024-04-21T08:48:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.471+0000 7f83ca59c640 1 --2- 172.21.15.82:0/4267349317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3eefbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.471+0000 7f83c9d9b640 1 -- 172.21.15.82:0/4267349317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3ee3e10 msgr2=0x55d5c3efc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.471+0000 7f83c9d9b640 1 --2- 172.21.15.82:0/4267349317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3ee3e10 0x55d5c3efc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.471+0000 7f83c9d9b640 1 -- 172.21.15.82:0/4267349317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 msgr2=0x55d5c3eefbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.471+0000 7f83c9d9b640 1 --2- 172.21.15.82:0/4267349317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3eefbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.471+0000 7f83c9d9b640 1 -- 172.21.15.82:0/4267349317 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5c3ef7050 con 0x55d5c3edb290 2024-04-21T08:48:47.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.472+0000 7f83c9d9b640 1 --2- 172.21.15.82:0/4267349317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3edb290 0x55d5c3ee17a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009330 tx=0x7f83c4008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9a350597b56761a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:47.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.473+0000 7f83b3fff640 1 -- 172.21.15.82:0/4267349317 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4009d90 con 0x55d5c3edb290 2024-04-21T08:48:47.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.473+0000 7f83b3fff640 1 -- 172.21.15.82:0/4267349317 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c4021030 con 0x55d5c3edb290 2024-04-21T08:48:47.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.473+0000 7f83b3fff640 1 -- 172.21.15.82:0/4267349317 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40253d0 con 0x55d5c3edb290 2024-04-21T08:48:47.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.474+0000 7f83d39b3740 1 -- 172.21.15.82:0/4267349317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3edb290 msgr2=0x55d5c3ee17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.474+0000 7f83d39b3740 1 --2- 172.21.15.82:0/4267349317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3edb290 0x55d5c3ee17a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009330 tx=0x7f83c4008040 comp rx=0 tx=0).stop 2024-04-21T08:48:47.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.475+0000 7f83d39b3740 1 -- 172.21.15.82:0/4267349317 shutdown_connections 2024-04-21T08:48:47.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.475+0000 7f83d39b3740 1 --2- 172.21.15.82:0/4267349317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3ee3e10 0x55d5c3efc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.475+0000 7f83d39b3740 1 --2- 172.21.15.82:0/4267349317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3edb290 0x55d5c3ee17a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.475+0000 7f83d39b3740 1 --2- 172.21.15.82:0/4267349317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3eefbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.475+0000 7f83d39b3740 1 -- 172.21.15.82:0/4267349317 >> 172.21.15.82:0/4267349317 conn(0x55d5c3ed4fb0 msgr2=0x55d5c3ed6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:47.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.478+0000 7f83d39b3740 1 -- 172.21.15.82:0/4267349317 shutdown_connections 2024-04-21T08:48:47.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.478+0000 7f83d39b3740 1 -- 172.21.15.82:0/4267349317 wait complete. 2024-04-21T08:48:47.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.479+0000 7f83d39b3740 1 -- start start 2024-04-21T08:48:47.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.479+0000 7f83d39b3740 1 Processor -- start 2024-04-21T08:48:47.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.479+0000 7f83d39b3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3eec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.480+0000 7f83d39b3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3edb290 0x55d5c3eec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.480+0000 7f83d39b3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.480+0000 7f83d39b3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1bf730 con 0x55d5c3eda8e0 2024-04-21T08:48:47.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.480+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3edb290 0x55d5c3eec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.480+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3edb290 0x55d5c3eec5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:51192/0 (socket says 172.21.15.82:51192) 2024-04-21T08:48:47.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.480+0000 7f83c9d9b640 1 -- 172.21.15.82:0/198632498 learned_addr learned my addr 172.21.15.82:0/198632498 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:47.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.480+0000 7f83d39b3740 1 -- 172.21.15.82:0/198632498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1bf060 con 0x55d5c3ee3e10 2024-04-21T08:48:47.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.480+0000 7f83d39b3740 1 -- 172.21.15.82:0/198632498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1bc1e0 con 0x55d5c3edb290 2024-04-21T08:48:47.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.480+0000 7f83d39b3740 10 client.0 _ll_register_callbacks cb 0x7f83bc009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:47.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.481+0000 7f83ca59c640 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3eec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.481+0000 7f83cad9d640 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.482+0000 7f83c9d9b640 1 -- 172.21.15.82:0/198632498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3ee3e10 msgr2=0x55d5c3ee8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.482+0000 7f83c9d9b640 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.482+0000 7f83c9d9b640 1 -- 172.21.15.82:0/198632498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 msgr2=0x55d5c3eec0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.482+0000 7f83c9d9b640 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3eec0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.482+0000 7f83c9d9b640 1 -- 172.21.15.82:0/198632498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5c3ee8e30 con 0x55d5c3edb290 2024-04-21T08:48:47.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.483+0000 7f83c9d9b640 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3edb290 0x55d5c3eec5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83c4008010 tx=0x7f83c4009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2524b96d87814b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:47.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.484+0000 7f83b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:48:47.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.485+0000 7f83b1ffb640 1 -- 172.21.15.82:0/198632498 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4017d00 con 0x55d5c3edb290 2024-04-21T08:48:47.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.485+0000 7f83b1ffb640 1 -- 172.21.15.82:0/198632498 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f83c401d050 con 0x55d5c3edb290 2024-04-21T08:48:47.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.485+0000 7f83d39b3740 1 -- 172.21.15.82:0/198632498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c4021030 con 0x55d5c3edb290 2024-04-21T08:48:47.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.485+0000 7f83b1ffb640 1 -- 172.21.15.82:0/198632498 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f83c4036050 con 0x55d5c3edb290 2024-04-21T08:48:47.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.487+0000 7f83d39b3740 10 client.25735 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:47.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.487+0000 7f83d39b3740 1 -- 172.21.15.82:0/198632498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d5c3f5b780 con 0x55d5c3edb290 2024-04-21T08:48:47.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.488+0000 7f83b1ffb640 1 -- 172.21.15.82:0/198632498 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c4021030 con 0x55d5c3edb290 2024-04-21T08:48:47.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.488+0000 7f83b1ffb640 1 -- 172.21.15.82:0/198632498 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f83c40317a0 con 0x55d5c3edb290 2024-04-21T08:48:47.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.488+0000 7f83d39b3740 10 client.25735 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:47.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.488+0000 7f83d39b3740 10 client.25735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:47.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.489+0000 7f83d39b3740 20 client.25735 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:47.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.489+0000 7f83d39b3740 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d5c3f5bef0 0x55d5c3f5e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.489+0000 7f83d39b3740 4 client.25735 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:48:47.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.489+0000 7f83d39b3740 1 -- 172.21.15.82:0/198632498 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d5c3e46790 con 0x55d5c3f5bef0 2024-04-21T08:48:47.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.490+0000 7f83ca59c640 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d5c3f5bef0 0x55d5c3f5e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.491+0000 7f83ca59c640 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d5c3f5bef0 0x55d5c3f5e310 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16284aaa85db2d2a server_cookie=a2f2ec71693a69f in_seq=0 out_seq=0 2024-04-21T08:48:47.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.491+0000 7f83b1ffb640 10 client.25735 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:47.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.493+0000 7f83b1ffb640 1 -- 172.21.15.82:0/198632498 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d5c3e46790 con 0x55d5c3f5bef0 2024-04-21T08:48:47.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.493+0000 7f83b1ffb640 10 client.25735 handle_command_reply: tid=0 2024-04-21T08:48:47.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.494+0000 7f83d39b3740 1 client.25735 shutdown 2024-04-21T08:48:47.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.495+0000 7f83d39b3740 1 -- 172.21.15.82:0/198632498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3edb290 msgr2=0x55d5c3eec5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.495+0000 7f83d39b3740 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3edb290 0x55d5c3eec5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83c4008010 tx=0x7f83c4009e80 comp rx=0 tx=0).stop 2024-04-21T08:48:47.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.496+0000 7f83d39b3740 1 -- 172.21.15.82:0/198632498 shutdown_connections 2024-04-21T08:48:47.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.496+0000 7f83d39b3740 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d5c3f5bef0 0x55d5c3f5e310 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.496+0000 7f83d39b3740 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d5c3ee3e10 0x55d5c3ee8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.496+0000 7f83d39b3740 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5c3edb290 0x55d5c3eec5f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.496+0000 7f83d39b3740 1 --2- 172.21.15.82:0/198632498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d5c3eda8e0 0x55d5c3eec0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.496+0000 7f83d39b3740 1 -- 172.21.15.82:0/198632498 >> 172.21.15.82:0/198632498 conn(0x55d5c3ed4fb0 msgr2=0x55d5c3ed3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:47.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.496+0000 7f83b1ffb640 0 client.25735 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:47.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.498+0000 7f83d39b3740 1 -- 172.21.15.82:0/198632498 shutdown_connections 2024-04-21T08:48:47.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.498+0000 7f83d39b3740 1 -- 172.21.15.82:0/198632498 wait complete. 2024-04-21T08:48:47.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.499+0000 7f83d39b3740 20 client.25735 trim_cache size 0 max 16384 2024-04-21T08:48:47.499 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:48:47.499 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:48:47.499 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "ef297256-3ab5-4dd5-ba70-6899b5410f31", 2024-04-21T08:48:47.499 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:48:47.499 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:48:47.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.502+0000 7f83d0ce4640 1 -- 172.21.15.82:0/1734467029 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f83b40a3f20 msgr2=0x7f83b40a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.502+0000 7f83d0ce4640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f83b40a3f20 0x7f83b40a63e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83c000f000 tx=0x7f83c001f000 comp rx=0 tx=0).stop 2024-04-21T08:48:47.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.502+0000 7f83d0ce4640 1 -- 172.21.15.82:0/1734467029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc10e7d0 msgr2=0x7f83cc1a7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.502+0000 7f83d0ce4640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc10e7d0 0x7f83cc1a7640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009ca0 tx=0x7f83bc035ce0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.503+0000 7f83d0ce4640 1 -- 172.21.15.82:0/1734467029 shutdown_connections 2024-04-21T08:48:47.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.503+0000 7f83d0ce4640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f83b40a3f20 0x7f83b40a63e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.503+0000 7f83d0ce4640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc11b730 0x7f83cc1abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.503+0000 7f83d0ce4640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83cc1107a0 0x7f83cc1a7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.503+0000 7f83d0ce4640 1 --2- 172.21.15.82:0/1734467029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc10e7d0 0x7f83cc1a7640 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.503+0000 7f83d0ce4640 1 -- 172.21.15.82:0/1734467029 >> 172.21.15.82:0/1734467029 conn(0x7f83cc06d060 msgr2=0x7f83cc072a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:47.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.503+0000 7f83d0ce4640 1 -- 172.21.15.82:0/1734467029 shutdown_connections 2024-04-21T08:48:47.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.504+0000 7f83d0ce4640 1 -- 172.21.15.82:0/1734467029 wait complete. 2024-04-21T08:48:47.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:47.523 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:47.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.717+0000 7fea2ffff640 1 -- 172.21.15.82:0/4269872463 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28034d80 con 0x7fea38108f50 2024-04-21T08:48:47.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.718+0000 7fea3f50e640 1 -- 172.21.15.82:0/4269872463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea38108f50 msgr2=0x7fea381093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.718+0000 7fea3f50e640 1 --2- 172.21.15.82:0/4269872463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea38108f50 0x7fea381093d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fea28009860 tx=0x7fea2802bb70 comp rx=0 tx=0).stop 2024-04-21T08:48:47.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.718+0000 7fea3f50e640 1 -- 172.21.15.82:0/4269872463 shutdown_connections 2024-04-21T08:48:47.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.718+0000 7fea3f50e640 1 --2- 172.21.15.82:0/4269872463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea38109910 0x7fea381157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.718+0000 7fea3f50e640 1 --2- 172.21.15.82:0/4269872463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea38108f50 0x7fea381093d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.718+0000 7fea3f50e640 1 --2- 172.21.15.82:0/4269872463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea3810e930 0x7fea3810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.718+0000 7fea3f50e640 1 -- 172.21.15.82:0/4269872463 >> 172.21.15.82:0/4269872463 conn(0x7fea3806d4d0 msgr2=0x7fea3806d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:47.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.719+0000 7fea3f50e640 1 -- 172.21.15.82:0/4269872463 shutdown_connections 2024-04-21T08:48:47.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.719+0000 7fea3f50e640 1 -- 172.21.15.82:0/4269872463 wait complete. 2024-04-21T08:48:47.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.720+0000 7fea3f50e640 1 Processor -- start 2024-04-21T08:48:47.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.720+0000 7fea3f50e640 1 -- start start 2024-04-21T08:48:47.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.720+0000 7fea3f50e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea38108f50 0x7fea38115780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3f50e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea38109910 0x7fea38112870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3f50e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3810e930 0x7fea38112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3f50e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fea38118530 con 0x7fea38109910 2024-04-21T08:48:47.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3f50e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fea381183b0 con 0x7fea38108f50 2024-04-21T08:48:47.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3f50e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea381186b0 con 0x7fea3810e930 2024-04-21T08:48:47.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3daab640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3810e930 0x7fea38112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3daab640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3810e930 0x7fea38112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:51222/0 (socket says 172.21.15.82:51222) 2024-04-21T08:48:47.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3daab640 1 -- 172.21.15.82:0/2908476967 learned_addr learned my addr 172.21.15.82:0/2908476967 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:47.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3d2aa640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea38108f50 0x7fea38115780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3caa9640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea38109910 0x7fea38112870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3daab640 1 -- 172.21.15.82:0/2908476967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea38108f50 msgr2=0x7fea38115780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3daab640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea38108f50 0x7fea38115780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3daab640 1 -- 172.21.15.82:0/2908476967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea38109910 msgr2=0x7fea38112870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3daab640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea38109910 0x7fea38112870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.721+0000 7fea3daab640 1 -- 172.21.15.82:0/2908476967 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea381b3970 con 0x7fea3810e930 2024-04-21T08:48:47.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.722+0000 7fea3daab640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3810e930 0x7fea38112fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fea34007860 tx=0x7fea34007d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:47.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.722+0000 7fea2e7fc640 1 -- 172.21.15.82:0/2908476967 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34017030 con 0x7fea3810e930 2024-04-21T08:48:47.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.722+0000 7fea2e7fc640 1 -- 172.21.15.82:0/2908476967 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea3400dd60 con 0x7fea3810e930 2024-04-21T08:48:47.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.723+0000 7fea3f50e640 1 -- 172.21.15.82:0/2908476967 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea381b3ba0 con 0x7fea3810e930 2024-04-21T08:48:47.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.724+0000 7fea3f50e640 1 -- 172.21.15.82:0/2908476967 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea381b40c0 con 0x7fea3810e930 2024-04-21T08:48:47.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.724+0000 7fea2e7fc640 1 -- 172.21.15.82:0/2908476967 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34004a00 con 0x7fea3810e930 2024-04-21T08:48:47.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.727+0000 7fea2e7fc640 1 -- 172.21.15.82:0/2908476967 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fea3401a070 con 0x7fea3810e930 2024-04-21T08:48:47.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.731+0000 7fea421dc740 1 Processor -- start 2024-04-21T08:48:47.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.731+0000 7fea2e7fc640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fea240a3c80 0x7fea240a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.732+0000 7fea2e7fc640 1 -- 172.21.15.82:0/2908476967 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fea340d04d0 con 0x7fea3810e930 2024-04-21T08:48:47.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.732+0000 7fea421dc740 1 -- start start 2024-04-21T08:48:47.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.732+0000 7fea3caa9640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fea240a3c80 0x7fea240a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.732+0000 7fea421dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058580 0x55fda8058960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.732+0000 7fea421dc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058ea0 0x55fda8061540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.732+0000 7fea421dc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8061a80 0x55fda8063e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.732+0000 7fea421dc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55fda7fbd1e0 con 0x55fda8061a80 2024-04-21T08:48:47.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.732+0000 7fea421dc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fda7f5a390 con 0x55fda8058ea0 2024-04-21T08:48:47.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.732+0000 7fea421dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fda7fb91a0 con 0x55fda8058580 2024-04-21T08:48:47.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.732+0000 7fea3caa9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058ea0 0x55fda8061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.732+0000 7fea3caa9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058ea0 0x55fda8061540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42304/0 (socket says 172.21.15.82:42304) 2024-04-21T08:48:47.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.732+0000 7fea3caa9640 1 -- 172.21.15.82:0/1432004574 learned_addr learned my addr 172.21.15.82:0/1432004574 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:47.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.733+0000 7fea3caa9640 1 -- 172.21.15.82:0/1432004574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058580 msgr2=0x55fda8058960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:48:47.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.733+0000 7fea3caa9640 1 --2- 172.21.15.82:0/1432004574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058580 0x55fda8058960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.733+0000 7fea3caa9640 1 -- 172.21.15.82:0/1432004574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8061a80 msgr2=0x55fda8063e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.733+0000 7fea3caa9640 1 --2- 172.21.15.82:0/1432004574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8061a80 0x55fda8063e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.733+0000 7fea3caa9640 1 -- 172.21.15.82:0/1432004574 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fda7d997d0 con 0x55fda8058ea0 2024-04-21T08:48:47.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.733+0000 7fea3caa9640 1 --2- 172.21.15.82:0/1432004574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058ea0 0x55fda8061540 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea28009780 tx=0x7fea28034e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d54ed853019c1e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:47.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.734+0000 7fea3caa9640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fea240a3c80 0x7fea240a6140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fea28040000 tx=0x7fea2803d000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:47.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.734+0000 7fea237fe640 1 -- 172.21.15.82:0/1432004574 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28039040 con 0x55fda8058ea0 2024-04-21T08:48:47.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.734+0000 7fea237fe640 1 -- 172.21.15.82:0/1432004574 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea280044d0 con 0x55fda8058ea0 2024-04-21T08:48:47.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.735+0000 7fea421dc740 1 -- 172.21.15.82:0/1432004574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058ea0 msgr2=0x55fda8061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.735+0000 7fea421dc740 1 --2- 172.21.15.82:0/1432004574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058ea0 0x55fda8061540 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea28009780 tx=0x7fea28034e10 comp rx=0 tx=0).stop 2024-04-21T08:48:47.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.735+0000 7fea421dc740 1 -- 172.21.15.82:0/1432004574 shutdown_connections 2024-04-21T08:48:47.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.735+0000 7fea421dc740 1 --2- 172.21.15.82:0/1432004574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8061a80 0x55fda8063e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.735+0000 7fea421dc740 1 --2- 172.21.15.82:0/1432004574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058ea0 0x55fda8061540 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.735+0000 7fea421dc740 1 --2- 172.21.15.82:0/1432004574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058580 0x55fda8058960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.735+0000 7fea421dc740 1 -- 172.21.15.82:0/1432004574 >> 172.21.15.82:0/1432004574 conn(0x55fda8052cb0 msgr2=0x55fda8053090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:47.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.737+0000 7fea421dc740 1 -- 172.21.15.82:0/1432004574 shutdown_connections 2024-04-21T08:48:47.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.737+0000 7fea421dc740 1 -- 172.21.15.82:0/1432004574 wait complete. 2024-04-21T08:48:47.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.738+0000 7fea421dc740 1 -- start start 2024-04-21T08:48:47.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.738+0000 7fea421dc740 1 Processor -- start 2024-04-21T08:48:47.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.738+0000 7fea421dc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058580 0x55fda805eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.738+0000 7fea421dc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8058ea0 0x55fda805f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.738+0000 7fea421dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8061a80 0x55fda805f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.738+0000 7fea421dc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55fda8064860 con 0x55fda8058ea0 2024-04-21T08:48:47.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.738+0000 7fea421dc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fda7f762f0 con 0x55fda8058580 2024-04-21T08:48:47.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.738+0000 7fea421dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fda80649e0 con 0x55fda8061a80 2024-04-21T08:48:47.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.738+0000 7fea421dc740 10 client.0 _ll_register_callbacks cb 0x55fda7f83580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:47.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.739+0000 7fea3d2aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058580 0x55fda805eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.739+0000 7fea3d2aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058580 0x55fda805eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42306/0 (socket says 172.21.15.82:42306) 2024-04-21T08:48:47.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.739+0000 7fea3d2aa640 1 -- 172.21.15.82:0/3272545770 learned_addr learned my addr 172.21.15.82:0/3272545770 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:47.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.739+0000 7fea3daab640 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8061a80 0x55fda805f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.739+0000 7fea3caa9640 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8058ea0 0x55fda805f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.740+0000 7fea3daab640 1 -- 172.21.15.82:0/3272545770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058580 msgr2=0x55fda805eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.740+0000 7fea3daab640 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058580 0x55fda805eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.740+0000 7fea3daab640 1 -- 172.21.15.82:0/3272545770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8058ea0 msgr2=0x55fda805f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.740+0000 7fea3daab640 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8058ea0 0x55fda805f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.740+0000 7fea3daab640 1 -- 172.21.15.82:0/3272545770 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fda7ccf3f0 con 0x55fda8061a80 2024-04-21T08:48:47.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.740+0000 7fea3daab640 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8061a80 0x55fda805f5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea340cd900 tx=0x7fea3409bba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc66536dd2493cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:47.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.741+0000 7fea20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:48:47.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.741+0000 7fea20ff9640 1 -- 172.21.15.82:0/3272545770 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea340a0500 con 0x55fda8061a80 2024-04-21T08:48:47.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.741+0000 7fea20ff9640 1 -- 172.21.15.82:0/3272545770 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fea340d0150 con 0x55fda8061a80 2024-04-21T08:48:47.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.742+0000 7fea421dc740 1 -- 172.21.15.82:0/3272545770 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea280044d0 con 0x55fda8061a80 2024-04-21T08:48:47.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.744+0000 7fea20ff9640 1 -- 172.21.15.82:0/3272545770 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea340a4dc0 con 0x55fda8061a80 2024-04-21T08:48:47.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.744+0000 7fea421dc740 10 client.25744 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:47.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.744+0000 7fea421dc740 1 -- 172.21.15.82:0/3272545770 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fda7c8f0f0 con 0x55fda8061a80 2024-04-21T08:48:47.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.745+0000 7fea20ff9640 1 -- 172.21.15.82:0/3272545770 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea280044d0 con 0x55fda8061a80 2024-04-21T08:48:47.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.745+0000 7fea20ff9640 1 -- 172.21.15.82:0/3272545770 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fea34024060 con 0x55fda8061a80 2024-04-21T08:48:47.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.745+0000 7fea421dc740 10 client.25744 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:47.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.745+0000 7fea421dc740 10 client.25744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:47.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.745+0000 7fea421dc740 20 client.25744 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:47.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.745+0000 7fea421dc740 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fda8071b10 0x55fda8073f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.745+0000 7fea421dc740 4 client.25744 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:47.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.745+0000 7fea421dc740 1 -- 172.21.15.82:0/3272545770 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fda7fcac00 con 0x55fda8071b10 2024-04-21T08:48:47.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.747+0000 7fea3d2aa640 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fda8071b10 0x55fda8073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.748+0000 7fea3d2aa640 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fda8071b10 0x55fda8073f00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128de320f83ddf39 server_cookie=9dbb3161e263d9aa in_seq=0 out_seq=0 2024-04-21T08:48:47.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.748+0000 7fea20ff9640 10 client.25744 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:47.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.749+0000 7fea20ff9640 1 -- 172.21.15.82:0/3272545770 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fda7fcac00 con 0x55fda8071b10 2024-04-21T08:48:47.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.749+0000 7fea20ff9640 10 client.25744 handle_command_reply: tid=0 2024-04-21T08:48:47.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.749+0000 7fea421dc740 1 client.25744 shutdown 2024-04-21T08:48:47.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.749+0000 7fea421dc740 1 -- 172.21.15.82:0/3272545770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8061a80 msgr2=0x55fda805f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.749+0000 7fea421dc740 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8061a80 0x55fda805f5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea340cd900 tx=0x7fea3409bba0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.749+0000 7fea421dc740 1 -- 172.21.15.82:0/3272545770 shutdown_connections 2024-04-21T08:48:47.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.749+0000 7fea421dc740 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fda8071b10 0x55fda8073f00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.749+0000 7fea421dc740 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8061a80 0x55fda805f5a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.750+0000 7fea421dc740 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8058ea0 0x55fda805f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.750+0000 7fea20ff9640 0 client.25744 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:47.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.750+0000 7fea421dc740 1 --2- 172.21.15.82:0/3272545770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058580 0x55fda805eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.750+0000 7fea421dc740 1 -- 172.21.15.82:0/3272545770 >> 172.21.15.82:0/3272545770 conn(0x55fda8052cb0 msgr2=0x55fda8054a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:47.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.750+0000 7fea421dc740 1 -- 172.21.15.82:0/3272545770 shutdown_connections 2024-04-21T08:48:47.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.750+0000 7fea421dc740 1 -- 172.21.15.82:0/3272545770 wait complete. 2024-04-21T08:48:47.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.750+0000 7fea421dc740 20 client.25744 trim_cache size 0 max 16384 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.788+0000 7fea421dc740 1 Processor -- start 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.788+0000 7fea421dc740 1 -- start start 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.788+0000 7fea421dc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8058580 0x55fda8061250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.788+0000 7fea421dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058ea0 0x55fda8054970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.788+0000 7fea421dc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8061a80 0x55fda805f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.788+0000 7fea421dc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fea381be020 con 0x55fda8058580 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.788+0000 7fea421dc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fea381bfc50 con 0x55fda8061a80 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.788+0000 7fea421dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea381be6f0 con 0x55fda8058ea0 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.788+0000 7fea3daab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8061a80 0x55fda805f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.788+0000 7fea3daab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8061a80 0x55fda805f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42318/0 (socket says 172.21.15.82:42318) 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.788+0000 7fea3daab640 1 -- 172.21.15.82:0/134082954 learned_addr learned my addr 172.21.15.82:0/134082954 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.789+0000 7fea3d2aa640 1 --2- 172.21.15.82:0/134082954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8058580 0x55fda8061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.789+0000 7fea3caa9640 1 --2- 172.21.15.82:0/134082954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058ea0 0x55fda8054970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.789+0000 7fea3caa9640 1 -- 172.21.15.82:0/134082954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8061a80 msgr2=0x55fda805f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.789+0000 7fea3caa9640 1 --2- 172.21.15.82:0/134082954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8061a80 0x55fda805f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.790+0000 7fea3caa9640 1 -- 172.21.15.82:0/134082954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8058580 msgr2=0x55fda8061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.790+0000 7fea3caa9640 1 --2- 172.21.15.82:0/134082954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8058580 0x55fda8061250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.790+0000 7fea3caa9640 1 -- 172.21.15.82:0/134082954 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fda8058cd0 con 0x55fda8058ea0 2024-04-21T08:48:47.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.790+0000 7fea3caa9640 1 --2- 172.21.15.82:0/134082954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058ea0 0x55fda8054970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fea2802bbb0 tx=0x7fea28003770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85b2eceeb4f8dd20 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:47.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.791+0000 7fea22ffd640 1 -- 172.21.15.82:0/134082954 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28059020 con 0x55fda8058ea0 2024-04-21T08:48:47.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.791+0000 7fea22ffd640 1 -- 172.21.15.82:0/134082954 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea280044d0 con 0x55fda8058ea0 2024-04-21T08:48:47.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.791+0000 7fea22ffd640 1 -- 172.21.15.82:0/134082954 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28051040 con 0x55fda8058ea0 2024-04-21T08:48:47.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.792+0000 7fea421dc740 1 -- 172.21.15.82:0/134082954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058ea0 msgr2=0x55fda8054970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.792+0000 7fea421dc740 1 --2- 172.21.15.82:0/134082954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058ea0 0x55fda8054970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fea2802bbb0 tx=0x7fea28003770 comp rx=0 tx=0).stop 2024-04-21T08:48:47.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.793+0000 7fea421dc740 1 -- 172.21.15.82:0/134082954 shutdown_connections 2024-04-21T08:48:47.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.793+0000 7fea421dc740 1 --2- 172.21.15.82:0/134082954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8061a80 0x55fda805f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.793+0000 7fea421dc740 1 --2- 172.21.15.82:0/134082954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058ea0 0x55fda8054970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.793+0000 7fea421dc740 1 --2- 172.21.15.82:0/134082954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8058580 0x55fda8061250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.793+0000 7fea421dc740 1 -- 172.21.15.82:0/134082954 >> 172.21.15.82:0/134082954 conn(0x55fda8052cb0 msgr2=0x55fda80514e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:47.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.796+0000 7fea421dc740 1 -- 172.21.15.82:0/134082954 shutdown_connections 2024-04-21T08:48:47.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.798+0000 7fea421dc740 1 -- 172.21.15.82:0/134082954 wait complete. 2024-04-21T08:48:47.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.799+0000 7fea421dc740 1 -- start start 2024-04-21T08:48:47.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.799+0000 7fea421dc740 1 Processor -- start 2024-04-21T08:48:47.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.802+0000 7fea421dc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058580 0x55fda8065d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.802+0000 7fea421dc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058ea0 0x55fda80662d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.802+0000 7fea421dc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8061a80 0x55fda80d6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.802+0000 7fea421dc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fea381bf550 con 0x55fda8061a80 2024-04-21T08:48:47.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.802+0000 7fea421dc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fea381bee80 con 0x55fda8058580 2024-04-21T08:48:47.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.802+0000 7fea421dc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea381bc000 con 0x55fda8058ea0 2024-04-21T08:48:47.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.802+0000 7fea421dc740 10 client.0 _ll_register_callbacks cb 0x7fea340046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:47.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.802+0000 7fea3d2aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058580 0x55fda8065d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.802+0000 7fea3d2aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058580 0x55fda8065d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:42322/0 (socket says 172.21.15.82:42322) 2024-04-21T08:48:47.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.802+0000 7fea3d2aa640 1 -- 172.21.15.82:0/267146727 learned_addr learned my addr 172.21.15.82:0/267146727 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:47.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.802+0000 7fea3caa9640 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058ea0 0x55fda80662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.802+0000 7fea3daab640 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8061a80 0x55fda80d6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.803+0000 7fea3daab640 1 -- 172.21.15.82:0/267146727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058ea0 msgr2=0x55fda80662d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.803+0000 7fea3daab640 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058ea0 0x55fda80662d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.803+0000 7fea3daab640 1 -- 172.21.15.82:0/267146727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058580 msgr2=0x55fda8065d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:47.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.803+0000 7fea3daab640 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058580 0x55fda8065d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.803+0000 7fea3daab640 1 -- 172.21.15.82:0/267146727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fda80d7510 con 0x55fda8061a80 2024-04-21T08:48:47.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.803+0000 7fea3daab640 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8061a80 0x55fda80d6fd0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fea34098b20 tx=0x7fea340d0380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4686db00650d9cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:47.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.803+0000 7fea20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:47.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.804+0000 7fea20ff9640 1 -- 172.21.15.82:0/267146727 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea340a0500 con 0x55fda8061a80 2024-04-21T08:48:47.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.804+0000 7fea20ff9640 1 -- 172.21.15.82:0/267146727 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fea3400d8b0 con 0x55fda8061a80 2024-04-21T08:48:47.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.805+0000 7fea421dc740 1 -- 172.21.15.82:0/267146727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea280044d0 con 0x55fda8061a80 2024-04-21T08:48:47.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.805+0000 7fea20ff9640 1 -- 172.21.15.82:0/267146727 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea340983f0 con 0x55fda8061a80 2024-04-21T08:48:47.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.806+0000 7fea421dc740 10 client.16608 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:47.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.806+0000 7fea421dc740 1 -- 172.21.15.82:0/267146727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fda80d7510 con 0x55fda8061a80 2024-04-21T08:48:47.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.807+0000 7fea20ff9640 1 -- 172.21.15.82:0/267146727 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea280044d0 con 0x55fda8061a80 2024-04-21T08:48:47.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.807+0000 7fea20ff9640 1 -- 172.21.15.82:0/267146727 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fea340a1690 con 0x55fda8061a80 2024-04-21T08:48:47.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.807+0000 7fea421dc740 10 client.16608 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:47.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.807+0000 7fea421dc740 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:47.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.807+0000 7fea421dc740 20 client.16608 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:47.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.807+0000 7fea421dc740 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fda80d8b40 0x55fda80daf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:47.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.807+0000 7fea421dc740 4 client.16608 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:47.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.807+0000 7fea421dc740 1 -- 172.21.15.82:0/267146727 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fda7fc4a70 con 0x55fda80d8b40 2024-04-21T08:48:47.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.807+0000 7fea3d2aa640 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fda80d8b40 0x55fda80daf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:47.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.809+0000 7fea3d2aa640 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fda80d8b40 0x55fda80daf60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=131d0137de025772 server_cookie=8f9a4cb177db5ea7 in_seq=0 out_seq=0 2024-04-21T08:48:47.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.810+0000 7fea20ff9640 10 client.16608 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:47.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.810+0000 7fea20ff9640 1 -- 172.21.15.82:0/267146727 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55fda7fc4a70 con 0x55fda80d8b40 2024-04-21T08:48:47.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.810+0000 7fea20ff9640 10 client.16608 handle_command_reply: tid=0 2024-04-21T08:48:47.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.810+0000 7fea421dc740 1 client.16608 shutdown 2024-04-21T08:48:47.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.837+0000 7fea421dc740 1 -- 172.21.15.82:0/267146727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8061a80 msgr2=0x55fda80d6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.837+0000 7fea421dc740 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8061a80 0x55fda80d6fd0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fea34098b20 tx=0x7fea340d0380 comp rx=0 tx=0).stop 2024-04-21T08:48:47.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.837+0000 7fea421dc740 1 -- 172.21.15.82:0/267146727 shutdown_connections 2024-04-21T08:48:47.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.837+0000 7fea421dc740 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fda80d8b40 0x55fda80daf60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.837+0000 7fea421dc740 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fda8061a80 0x55fda80d6fd0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.837+0000 7fea421dc740 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fda8058ea0 0x55fda80662d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.837+0000 7fea421dc740 1 --2- 172.21.15.82:0/267146727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fda8058580 0x55fda8065d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.837+0000 7fea421dc740 1 -- 172.21.15.82:0/267146727 >> 172.21.15.82:0/267146727 conn(0x55fda8052cb0 msgr2=0x55fda8051d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:47.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.846+0000 7fea20ff9640 0 client.16608 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:47.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.846+0000 7fea421dc740 1 -- 172.21.15.82:0/267146727 shutdown_connections 2024-04-21T08:48:47.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.846+0000 7fea421dc740 1 -- 172.21.15.82:0/267146727 wait complete. 2024-04-21T08:48:47.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.852+0000 7fea421dc740 20 client.16608 trim_cache size 0 max 16384 2024-04-21T08:48:47.858 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:48:47.858 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (0 inodes in the stack)", 2024-04-21T08:48:47.858 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:48:47.858 INFO:teuthology.orchestra.run.smithi082.stdout: "ef297256-3ab5-4dd5-ba70-6899b5410f31": { 2024-04-21T08:48:47.858 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:48:47.858 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "ef297256-3ab5-4dd5-ba70-6899b5410f31", 2024-04-21T08:48:47.858 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:48:47.858 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:48:47.859 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:48:47.859 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:48:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.859+0000 7fea3f50e640 1 -- 172.21.15.82:0/2908476967 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fea240a3c80 msgr2=0x7fea240a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.859+0000 7fea3f50e640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fea240a3c80 0x7fea240a6140 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fea28040000 tx=0x7fea2803d000 comp rx=0 tx=0).stop 2024-04-21T08:48:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.859+0000 7fea3f50e640 1 -- 172.21.15.82:0/2908476967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3810e930 msgr2=0x7fea38112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.859+0000 7fea3f50e640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3810e930 0x7fea38112fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fea34007860 tx=0x7fea34007d20 comp rx=0 tx=0).stop 2024-04-21T08:48:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.860+0000 7fea3f50e640 1 -- 172.21.15.82:0/2908476967 shutdown_connections 2024-04-21T08:48:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.860+0000 7fea3f50e640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3810e930 0x7fea38112fe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.860+0000 7fea3f50e640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fea240a3c80 0x7fea240a6140 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.860+0000 7fea3f50e640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea38109910 0x7fea38112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.860+0000 7fea3f50e640 1 --2- 172.21.15.82:0/2908476967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea38108f50 0x7fea38115780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.860+0000 7fea3f50e640 1 -- 172.21.15.82:0/2908476967 >> 172.21.15.82:0/2908476967 conn(0x7fea3806d4d0 msgr2=0x7fea380727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:47.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.861+0000 7fea3f50e640 1 -- 172.21.15.82:0/2908476967 shutdown_connections 2024-04-21T08:48:47.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:48:47.861+0000 7fea3f50e640 1 -- 172.21.15.82:0/2908476967 wait complete. 2024-04-21T08:48:47.886 INFO:tasks.cephfs.filesystem:scrub status for tag:ef297256-3ab5-4dd5-ba70-6899b5410f31 - {'path': '/', 'tag': 'ef297256-3ab5-4dd5-ba70-6899b5410f31', 'options': 'recursive,force'} 2024-04-21T08:48:48.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:48 smithi082 ceph-mon[27967]: pgmap v56: 129 pgs: 129 active+clean; 135 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-04-21T08:48:48.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:48 smithi106 ceph-mon[31072]: pgmap v56: 129 pgs: 129 active+clean; 135 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-04-21T08:48:48.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:48 smithi134 ceph-mon[31772]: pgmap v56: 129 pgs: 129 active+clean; 135 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 305 op/s 2024-04-21T08:48:49.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:48:49.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:49.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:49.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:49.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:49 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:48:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:49 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:48:49.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:49.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:49.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:49.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:49.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:49.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:49.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:49.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:49.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:49.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:49.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:49.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:49 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:50.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:50 smithi134 ceph-mon[31772]: pgmap v57: 129 pgs: 129 active+clean; 141 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.4 MiB/s wr, 469 op/s 2024-04-21T08:48:50.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:50 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:48:50.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:50 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:50.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:50 smithi082 ceph-mon[27967]: pgmap v57: 129 pgs: 129 active+clean; 141 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.4 MiB/s wr, 469 op/s 2024-04-21T08:48:50.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:50 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:48:50.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:50 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:50.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:50 smithi106 ceph-mon[31072]: pgmap v57: 129 pgs: 129 active+clean; 141 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.4 MiB/s wr, 469 op/s 2024-04-21T08:48:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:50 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:48:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:50 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:52.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:52 smithi134 ceph-mon[31772]: pgmap v58: 129 pgs: 129 active+clean; 141 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 325 op/s 2024-04-21T08:48:52.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:52 smithi082 ceph-mon[27967]: pgmap v58: 129 pgs: 129 active+clean; 141 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 325 op/s 2024-04-21T08:48:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:52 smithi106 ceph-mon[31072]: pgmap v58: 129 pgs: 129 active+clean; 141 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 325 op/s 2024-04-21T08:48:54.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:54 smithi134 ceph-mon[31772]: pgmap v59: 129 pgs: 129 active+clean; 180 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 712 op/s 2024-04-21T08:48:54.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:54 smithi082 ceph-mon[27967]: pgmap v59: 129 pgs: 129 active+clean; 180 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 712 op/s 2024-04-21T08:48:54.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:54 smithi106 ceph-mon[31072]: pgmap v59: 129 pgs: 129 active+clean; 180 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 712 op/s 2024-04-21T08:48:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.689+0000 7f1979ceb640 1 -- 172.21.15.82:0/3277096661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f197410c5d0 msgr2=0x7f197410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.689+0000 7f1979ceb640 1 --2- 172.21.15.82:0/3277096661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f197410c5d0 0x7f197410ea00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f196c009ca0 tx=0x7f196c02c180 comp rx=0 tx=0).stop 2024-04-21T08:48:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.689+0000 7f1979ceb640 1 -- 172.21.15.82:0/3277096661 shutdown_connections 2024-04-21T08:48:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.689+0000 7f1979ceb640 1 --2- 172.21.15.82:0/3277096661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f197410c5d0 0x7f197410ea00 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.689+0000 7f1979ceb640 1 --2- 172.21.15.82:0/3277096661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1974103b20 0x7f197410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.689+0000 7f1979ceb640 1 --2- 172.21.15.82:0/3277096661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1974103200 0x7f19741035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.689+0000 7f1979ceb640 1 -- 172.21.15.82:0/3277096661 >> 172.21.15.82:0/3277096661 conn(0x7f19740771d0 msgr2=0x7f19740775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.689+0000 7f1979ceb640 1 -- 172.21.15.82:0/3277096661 shutdown_connections 2024-04-21T08:48:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.689+0000 7f1979ceb640 1 -- 172.21.15.82:0/3277096661 wait complete. 2024-04-21T08:48:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.689+0000 7f1979ceb640 1 Processor -- start 2024-04-21T08:48:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1979ceb640 1 -- start start 2024-04-21T08:48:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1979ceb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1974103200 0x7f1974100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1979ceb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1974103b20 0x7f1974101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1979ceb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f197410c5d0 0x7f19741018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1979ceb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f19741121d0 con 0x7f197410c5d0 2024-04-21T08:48:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1979ceb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1974112050 con 0x7f1974103b20 2024-04-21T08:48:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1979ceb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1974112350 con 0x7f1974103200 2024-04-21T08:48:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1972ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1974103b20 0x7f1974101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1973fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f197410c5d0 0x7f19741018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1972ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1974103b20 0x7f1974101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:49970/0 (socket says 172.21.15.82:49970) 2024-04-21T08:48:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1972ffd640 1 -- 172.21.15.82:0/2908401547 learned_addr learned my addr 172.21.15.82:0/2908401547 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f19737fe640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1974103200 0x7f1974100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1972ffd640 1 -- 172.21.15.82:0/2908401547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1974103200 msgr2=0x7f1974100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1972ffd640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1974103200 0x7f1974100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1972ffd640 1 -- 172.21.15.82:0/2908401547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f197410c5d0 msgr2=0x7f19741018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1972ffd640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f197410c5d0 0x7f19741018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1972ffd640 1 -- 172.21.15.82:0/2908401547 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19741aafd0 con 0x7f1974103b20 2024-04-21T08:48:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f1973fff640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f197410c5d0 0x7f19741018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.690+0000 7f19737fe640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1974103200 0x7f1974100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.691+0000 7f1972ffd640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1974103b20 0x7f1974101390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f196800e9f0 tx=0x7f196800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.691+0000 7f1970ff9640 1 -- 172.21.15.82:0/2908401547 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196800ccc0 con 0x7f1974103b20 2024-04-21T08:48:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.691+0000 7f1979ceb640 1 -- 172.21.15.82:0/2908401547 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19741ab200 con 0x7f1974103b20 2024-04-21T08:48:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.691+0000 7f1970ff9640 1 -- 172.21.15.82:0/2908401547 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19680108e0 con 0x7f1974103b20 2024-04-21T08:48:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.691+0000 7f1970ff9640 1 -- 172.21.15.82:0/2908401547 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968018880 con 0x7f1974103b20 2024-04-21T08:48:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.691+0000 7f1979ceb640 1 -- 172.21.15.82:0/2908401547 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19741ab750 con 0x7f1974103b20 2024-04-21T08:48:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.693+0000 7f1970ff9640 1 -- 172.21.15.82:0/2908401547 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1968010430 con 0x7f1974103b20 2024-04-21T08:48:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f197c9b9740 1 Processor -- start 2024-04-21T08:48:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f197c9b9740 1 -- start start 2024-04-21T08:48:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f1970ff9640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f195c0a3dc0 0x7f195c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f197c9b9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c1f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f1970ff9640 1 -- 172.21.15.82:0/2908401547 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f19680d0ef0 con 0x7f1974103b20 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f197c9b9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c1ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f1973fff640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f195c0a3dc0 0x7f195c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f197c9b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c290e0 0x55e3c9c2b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f197c9b9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e3c9b848d0 con 0x55e3c9c208d0 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f197c9b9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e3c9b21e80 con 0x55e3c9c1fb60 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f197c9b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e3c9b80890 con 0x55e3c9c290e0 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f19737fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c1f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f19737fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c1f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49582/0 (socket says 172.21.15.82:49582) 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f19737fe640 1 -- 172.21.15.82:0/1713025849 learned_addr learned my addr 172.21.15.82:0/1713025849 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.694+0000 7f1972ffd640 1 --2- 172.21.15.82:0/1713025849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c1ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.695+0000 7f1973fff640 1 --2- 172.21.15.82:0/1713025849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c290e0 0x55e3c9c2b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.695+0000 7f19737fe640 1 -- 172.21.15.82:0/1713025849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c290e0 msgr2=0x55e3c9c2b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.695+0000 7f19737fe640 1 --2- 172.21.15.82:0/1713025849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c290e0 0x55e3c9c2b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.695+0000 7f19737fe640 1 -- 172.21.15.82:0/1713025849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c1fb60 msgr2=0x55e3c9c1ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.695+0000 7f19737fe640 1 --2- 172.21.15.82:0/1713025849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c1ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.695+0000 7f19737fe640 1 -- 172.21.15.82:0/1713025849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3c9b43320 con 0x55e3c9c208d0 2024-04-21T08:48:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.695+0000 7f1973fff640 1 --2- 172.21.15.82:0/1713025849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c290e0 0x55e3c9c2b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.695+0000 7f1973fff640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f195c0a3dc0 0x7f195c0a6280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f196c005cc0 tx=0x7f196c0324a0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.695+0000 7f19737fe640 1 --2- 172.21.15.82:0/1713025849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c1f620 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f196400f4c0 tx=0x7f196400f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3394946370e28778 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.695+0000 7f1959ffb640 1 -- 172.21.15.82:0/1713025849 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964005220 con 0x55e3c9c208d0 2024-04-21T08:48:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.695+0000 7f1959ffb640 1 -- 172.21.15.82:0/1713025849 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19640047a0 con 0x55e3c9c208d0 2024-04-21T08:48:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.695+0000 7f1959ffb640 1 -- 172.21.15.82:0/1713025849 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964010ab0 con 0x55e3c9c208d0 2024-04-21T08:48:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.696+0000 7f197c9b9740 1 -- 172.21.15.82:0/1713025849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 msgr2=0x55e3c9c1f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.696+0000 7f197c9b9740 1 --2- 172.21.15.82:0/1713025849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c1f620 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f196400f4c0 tx=0x7f196400f980 comp rx=0 tx=0).stop 2024-04-21T08:48:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.696+0000 7f197c9b9740 1 -- 172.21.15.82:0/1713025849 shutdown_connections 2024-04-21T08:48:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.696+0000 7f197c9b9740 1 --2- 172.21.15.82:0/1713025849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c290e0 0x55e3c9c2b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.696+0000 7f197c9b9740 1 --2- 172.21.15.82:0/1713025849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c1ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.696+0000 7f197c9b9740 1 --2- 172.21.15.82:0/1713025849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c1f620 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.696+0000 7f197c9b9740 1 -- 172.21.15.82:0/1713025849 >> 172.21.15.82:0/1713025849 conn(0x55e3c9be6c90 msgr2=0x55e3c9c1a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.696+0000 7f197c9b9740 1 -- 172.21.15.82:0/1713025849 shutdown_connections 2024-04-21T08:48:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.696+0000 7f197c9b9740 1 -- 172.21.15.82:0/1713025849 wait complete. 2024-04-21T08:48:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.696+0000 7f197c9b9740 1 -- start start 2024-04-21T08:48:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f197c9b9740 1 Processor -- start 2024-04-21T08:48:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f197c9b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c28a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f197c9b9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c25dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f197c9b9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c290e0 0x55e3c9c26330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f197c9b9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e3c9c2be10 con 0x55e3c9c208d0 2024-04-21T08:48:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f197c9b9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e3c9b0ceb0 con 0x55e3c9c290e0 2024-04-21T08:48:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f197c9b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e3c9c2bf90 con 0x55e3c9c1fb60 2024-04-21T08:48:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f197c9b9740 10 client.0 _ll_register_callbacks cb 0x55e3c9b39b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f1972ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c25dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f1972ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c25dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49586/0 (socket says 172.21.15.82:49586) 2024-04-21T08:48:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f1973fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c290e0 0x55e3c9c26330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f1972ffd640 1 -- 172.21.15.82:0/2568083754 learned_addr learned my addr 172.21.15.82:0/2568083754 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f19737fe640 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c28a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f1972ffd640 1 -- 172.21.15.82:0/2568083754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 msgr2=0x55e3c9c28a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f1972ffd640 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c28a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f1972ffd640 1 -- 172.21.15.82:0/2568083754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c290e0 msgr2=0x55e3c9c26330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f1972ffd640 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c290e0 0x55e3c9c26330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f1972ffd640 1 -- 172.21.15.82:0/2568083754 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3c9897210 con 0x55e3c9c208d0 2024-04-21T08:48:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.697+0000 7f19737fe640 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c28a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f1972ffd640 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c25dd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f196809c7d0 tx=0x7f196809d290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedb898ae2daa3bb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f1952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f1952ffd640 1 -- 172.21.15.82:0/2568083754 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19680a5740 con 0x55e3c9c208d0 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f1952ffd640 1 -- 172.21.15.82:0/2568083754 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f196809add0 con 0x55e3c9c208d0 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f197c9b9740 1 -- 172.21.15.82:0/2568083754 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19640047a0 con 0x55e3c9c208d0 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f1952ffd640 1 -- 172.21.15.82:0/2568083754 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f19680b6020 con 0x55e3c9c208d0 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f197c9b9740 10 client.16620 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f197c9b9740 1 -- 172.21.15.82:0/2568083754 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e3c9963620 con 0x55e3c9c208d0 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f1952ffd640 1 -- 172.21.15.82:0/2568083754 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19640047a0 con 0x55e3c9c208d0 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f1952ffd640 1 -- 172.21.15.82:0/2568083754 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f1968098940 con 0x55e3c9c208d0 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f197c9b9740 10 client.16620 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f197c9b9740 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f197c9b9740 20 client.16620 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f197c9b9740 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e3c9c39070 0x55e3c9c3b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f197c9b9740 4 client.16620 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.698+0000 7f197c9b9740 1 -- 172.21.15.82:0/2568083754 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3c9b98880 con 0x55e3c9c39070 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.699+0000 7f19737fe640 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e3c9c39070 0x55e3c9c3b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.699+0000 7f19737fe640 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e3c9c39070 0x55e3c9c3b460 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7776428c0c3ea340 server_cookie=c7396b73789886bb in_seq=0 out_seq=0 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.699+0000 7f1952ffd640 10 client.16620 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f1952ffd640 1 -- 172.21.15.82:0/2568083754 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3c9b98880 con 0x55e3c9c39070 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f1952ffd640 10 client.16620 handle_command_reply: tid=0 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f197c9b9740 1 client.16620 shutdown 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f197c9b9740 1 -- 172.21.15.82:0/2568083754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 msgr2=0x55e3c9c25dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f197c9b9740 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c25dd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f196809c7d0 tx=0x7f196809d290 comp rx=0 tx=0).stop 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f197c9b9740 1 -- 172.21.15.82:0/2568083754 shutdown_connections 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f197c9b9740 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e3c9c39070 0x55e3c9c3b460 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f197c9b9740 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c290e0 0x55e3c9c26330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f197c9b9740 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c25dd0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f1952ffd640 0 client.16620 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f197c9b9740 1 --2- 172.21.15.82:0/2568083754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c28a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f197c9b9740 1 -- 172.21.15.82:0/2568083754 >> 172.21.15.82:0/2568083754 conn(0x55e3c9be6c90 msgr2=0x55e3c9c1c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f197c9b9740 1 -- 172.21.15.82:0/2568083754 shutdown_connections 2024-04-21T08:48:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f197c9b9740 1 -- 172.21.15.82:0/2568083754 wait complete. 2024-04-21T08:48:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.700+0000 7f197c9b9740 20 client.16620 trim_cache size 0 max 16384 2024-04-21T08:48:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f197c9b9740 1 Processor -- start 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f197c9b9740 1 -- start start 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f197c9b9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c34d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f197c9b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c208d0 0x55e3c9c215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f197c9b9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c290e0 0x55e3c9c28500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f197c9b9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f19741b58c0 con 0x55e3c9c290e0 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f197c9b9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19741b74f0 con 0x55e3c9c1fb60 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f197c9b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19741b5f90 con 0x55e3c9c208d0 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f1973fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c290e0 0x55e3c9c28500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f1973fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c290e0 0x55e3c9c28500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49596/0 (socket says 172.21.15.82:49596) 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f1973fff640 1 -- 172.21.15.82:0/1689051604 learned_addr learned my addr 172.21.15.82:0/1689051604 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f1972ffd640 1 --2- 172.21.15.82:0/1689051604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c208d0 0x55e3c9c215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f19737fe640 1 --2- 172.21.15.82:0/1689051604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c34d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f1973fff640 1 -- 172.21.15.82:0/1689051604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c208d0 msgr2=0x55e3c9c215d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f1973fff640 1 --2- 172.21.15.82:0/1689051604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c208d0 0x55e3c9c215d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f1973fff640 1 -- 172.21.15.82:0/1689051604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c1fb60 msgr2=0x55e3c9c34d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f1973fff640 1 --2- 172.21.15.82:0/1689051604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c34d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f1973fff640 1 -- 172.21.15.82:0/1689051604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3c9c15f30 con 0x55e3c9c290e0 2024-04-21T08:48:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.719+0000 7f1972ffd640 1 --2- 172.21.15.82:0/1689051604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c208d0 0x55e3c9c215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.720+0000 7f1973fff640 1 --2- 172.21.15.82:0/1689051604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c290e0 0x55e3c9c28500 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f196c009ca0 tx=0x7f196c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2dca0a67a296cd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.720+0000 7f19597fa640 1 -- 172.21.15.82:0/1689051604 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c048040 con 0x55e3c9c290e0 2024-04-21T08:48:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.720+0000 7f19597fa640 1 -- 172.21.15.82:0/1689051604 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f196c052070 con 0x55e3c9c290e0 2024-04-21T08:48:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.720+0000 7f19597fa640 1 -- 172.21.15.82:0/1689051604 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c039040 con 0x55e3c9c290e0 2024-04-21T08:48:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.720+0000 7f197c9b9740 1 -- 172.21.15.82:0/1689051604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c290e0 msgr2=0x55e3c9c28500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.720+0000 7f197c9b9740 1 --2- 172.21.15.82:0/1689051604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c290e0 0x55e3c9c28500 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f196c009ca0 tx=0x7f196c040000 comp rx=0 tx=0).stop 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 -- 172.21.15.82:0/1689051604 shutdown_connections 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 --2- 172.21.15.82:0/1689051604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c290e0 0x55e3c9c28500 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 --2- 172.21.15.82:0/1689051604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c208d0 0x55e3c9c215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 --2- 172.21.15.82:0/1689051604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c34d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 -- 172.21.15.82:0/1689051604 >> 172.21.15.82:0/1689051604 conn(0x55e3c9be6c90 msgr2=0x55e3c9c20ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 -- 172.21.15.82:0/1689051604 shutdown_connections 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 -- 172.21.15.82:0/1689051604 wait complete. 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 -- start start 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 Processor -- start 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c2cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c2d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c290e0 0x55e3c9c2d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f19741b6df0 con 0x55e3c9c208d0 2024-04-21T08:48:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19741b6720 con 0x55e3c9c290e0 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.721+0000 7f197c9b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19741b38a0 con 0x55e3c9c1fb60 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f197c9b9740 10 client.0 _ll_register_callbacks cb 0x7f196809c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f1972ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c2d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f19737fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c2cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f1973fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c290e0 0x55e3c9c2d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f19737fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c2cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:40502/0 (socket says 172.21.15.82:40502) 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f19737fe640 1 -- 172.21.15.82:0/2253212042 learned_addr learned my addr 172.21.15.82:0/2253212042 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f1973fff640 1 -- 172.21.15.82:0/2253212042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 msgr2=0x55e3c9c2cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f1973fff640 1 --2- 172.21.15.82:0/2253212042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c2cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f1973fff640 1 -- 172.21.15.82:0/2253212042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 msgr2=0x55e3c9c2d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f1973fff640 1 --2- 172.21.15.82:0/2253212042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c2d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f1973fff640 1 -- 172.21.15.82:0/2253212042 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3c9c266a0 con 0x55e3c9c290e0 2024-04-21T08:48:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f19737fe640 1 --2- 172.21.15.82:0/2253212042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c2cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f1973fff640 1 --2- 172.21.15.82:0/2253212042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c290e0 0x55e3c9c2d7d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f196c02c790 tx=0x7f196c04db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9c8d62819e0ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f1952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f1952ffd640 1 -- 172.21.15.82:0/2253212042 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c05a020 con 0x55e3c9c290e0 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f197c9b9740 1 -- 172.21.15.82:0/2253212042 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f196c052070 con 0x55e3c9c290e0 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.722+0000 7f1952ffd640 1 -- 172.21.15.82:0/2253212042 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f196c04dca0 con 0x55e3c9c290e0 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.723+0000 7f1952ffd640 1 -- 172.21.15.82:0/2253212042 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f196c065050 con 0x55e3c9c290e0 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.723+0000 7f197c9b9740 10 client.25970 fetch_fsmap learned FSMap version 32 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.723+0000 7f197c9b9740 1 -- 172.21.15.82:0/2253212042 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e3c9c9f780 con 0x55e3c9c290e0 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.723+0000 7f1952ffd640 1 -- 172.21.15.82:0/2253212042 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f196c052070 con 0x55e3c9c290e0 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.723+0000 7f1952ffd640 1 -- 172.21.15.82:0/2253212042 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f196c03f040 con 0x55e3c9c290e0 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.723+0000 7f197c9b9740 10 client.25970 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.723+0000 7f197c9b9740 10 client.25970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.723+0000 7f197c9b9740 20 client.25970 populate_metadata read hostname 'smithi082' 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.723+0000 7f197c9b9740 1 --2- 172.21.15.82:0/2253212042 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e3c9c9fec0 0x55e3c9ca22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.723+0000 7f197c9b9740 4 client.25970 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.723+0000 7f197c9b9740 1 -- 172.21.15.82:0/2253212042 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3c9b243d0 con 0x55e3c9c9fec0 2024-04-21T08:48:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.723+0000 7f19737fe640 1 --2- 172.21.15.82:0/2253212042 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e3c9c9fec0 0x55e3c9ca22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.724+0000 7f1952ffd640 10 client.25970 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:55.724+0000 7f19737fe640 1 --2- 172.21.15.82:0/2253212042 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e3c9c9fec0 0x55e3c9ca22e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d795801b51c88813 server_cookie=d8d80ddd3d929ba in_seq=0 out_seq=0 2024-04-21T08:48:56.222 INFO:tasks.workunit.client.0.smithi082.stdout: 49 1039603 329 690177 330 538494 948 2024-04-21T08:48:56.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:56 smithi134 ceph-mon[31772]: pgmap v60: 129 pgs: 129 active+clean; 180 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 595 op/s 2024-04-21T08:48:56.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:56 smithi082 ceph-mon[27967]: pgmap v60: 129 pgs: 129 active+clean; 180 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 595 op/s 2024-04-21T08:48:56.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:56 smithi106 ceph-mon[31072]: pgmap v60: 129 pgs: 129 active+clean; 180 MiB data, 848 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 595 op/s 2024-04-21T08:48:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f1952ffd640 1 -- 172.21.15.82:0/2253212042 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3c9b243d0 con 0x55e3c9c9fec0 2024-04-21T08:48:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f1952ffd640 10 client.25970 handle_command_reply: tid=0 2024-04-21T08:48:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f197c9b9740 1 client.25970 shutdown 2024-04-21T08:48:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f197c9b9740 1 -- 172.21.15.82:0/2253212042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c290e0 msgr2=0x55e3c9c2d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f197c9b9740 1 --2- 172.21.15.82:0/2253212042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c290e0 0x55e3c9c2d7d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f196c02c790 tx=0x7f196c04db30 comp rx=0 tx=0).stop 2024-04-21T08:48:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f197c9b9740 1 -- 172.21.15.82:0/2253212042 shutdown_connections 2024-04-21T08:48:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f197c9b9740 1 --2- 172.21.15.82:0/2253212042 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e3c9c9fec0 0x55e3c9ca22e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f197c9b9740 1 --2- 172.21.15.82:0/2253212042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3c9c290e0 0x55e3c9c2d7d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f1952ffd640 0 client.25970 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:48:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f197c9b9740 1 --2- 172.21.15.82:0/2253212042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3c9c208d0 0x55e3c9c2d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f197c9b9740 1 --2- 172.21.15.82:0/2253212042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3c9c1fb60 0x55e3c9c2cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f197c9b9740 1 -- 172.21.15.82:0/2253212042 >> 172.21.15.82:0/2253212042 conn(0x55e3c9be6c90 msgr2=0x55e3c9c19040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f197c9b9740 1 -- 172.21.15.82:0/2253212042 shutdown_connections 2024-04-21T08:48:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f197c9b9740 1 -- 172.21.15.82:0/2253212042 wait complete. 2024-04-21T08:48:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.427+0000 7f197c9b9740 20 client.25970 trim_cache size 0 max 16384 2024-04-21T08:48:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:48:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:48:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:48:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:48:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.428+0000 7f1979ceb640 1 -- 172.21.15.82:0/2908401547 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f195c0a3dc0 msgr2=0x7f195c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.428+0000 7f1979ceb640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f195c0a3dc0 0x7f195c0a6280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f196c005cc0 tx=0x7f196c0324a0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.428+0000 7f1979ceb640 1 -- 172.21.15.82:0/2908401547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1974103b20 msgr2=0x7f1974101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.428+0000 7f1979ceb640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1974103b20 0x7f1974101390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f196800e9f0 tx=0x7f196800edf0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.428+0000 7f1979ceb640 1 -- 172.21.15.82:0/2908401547 shutdown_connections 2024-04-21T08:48:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.428+0000 7f1979ceb640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f195c0a3dc0 0x7f195c0a6280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.428+0000 7f1979ceb640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f197410c5d0 0x7f19741018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.428+0000 7f1979ceb640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1974103b20 0x7f1974101390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.428+0000 7f1979ceb640 1 --2- 172.21.15.82:0/2908401547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1974103200 0x7f1974100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.428+0000 7f1979ceb640 1 -- 172.21.15.82:0/2908401547 >> 172.21.15.82:0/2908401547 conn(0x7f19740771d0 msgr2=0x7f19740fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.428+0000 7f1979ceb640 1 -- 172.21.15.82:0/2908401547 shutdown_connections 2024-04-21T08:48:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:48:57.428+0000 7f1979ceb640 1 -- 172.21.15.82:0/2908401547 wait complete. 2024-04-21T08:48:58.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:48:58 smithi134 ceph-mon[31772]: pgmap v61: 129 pgs: 129 active+clean; 186 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 622 op/s 2024-04-21T08:48:58.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:48:58 smithi082 ceph-mon[27967]: pgmap v61: 129 pgs: 129 active+clean; 186 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 622 op/s 2024-04-21T08:48:58.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:48:58 smithi106 ceph-mon[31072]: pgmap v61: 129 pgs: 129 active+clean; 186 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.2 MiB/s wr, 622 op/s 2024-04-21T08:49:00.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:00 smithi134 ceph-mon[31772]: pgmap v62: 129 pgs: 129 active+clean; 213 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.7 MiB/s wr, 808 op/s 2024-04-21T08:49:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:00 smithi082 ceph-mon[27967]: pgmap v62: 129 pgs: 129 active+clean; 213 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.7 MiB/s wr, 808 op/s 2024-04-21T08:49:00.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:00 smithi106 ceph-mon[31072]: pgmap v62: 129 pgs: 129 active+clean; 213 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.7 MiB/s wr, 808 op/s 2024-04-21T08:49:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:02 smithi134 ceph-mon[31772]: pgmap v63: 129 pgs: 129 active+clean; 213 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 633 op/s 2024-04-21T08:49:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:02 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:02.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:02 smithi082 ceph-mon[27967]: pgmap v63: 129 pgs: 129 active+clean; 213 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 633 op/s 2024-04-21T08:49:02.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:02 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:02 smithi106 ceph-mon[31072]: pgmap v63: 129 pgs: 129 active+clean; 213 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 633 op/s 2024-04-21T08:49:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:02 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:04.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:04 smithi134 ceph-mon[31772]: pgmap v64: 129 pgs: 129 active+clean; 216 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.6 MiB/s wr, 696 op/s 2024-04-21T08:49:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:04 smithi082 ceph-mon[27967]: pgmap v64: 129 pgs: 129 active+clean; 216 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.6 MiB/s wr, 696 op/s 2024-04-21T08:49:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:04 smithi106 ceph-mon[31072]: pgmap v64: 129 pgs: 129 active+clean; 216 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.6 MiB/s wr, 696 op/s 2024-04-21T08:49:06.222 INFO:tasks.workunit.client.0.smithi082.stdout: 57 1290263 605 853070 511 622315 1239 2024-04-21T08:49:06.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:06 smithi134 ceph-mon[31772]: pgmap v65: 129 pgs: 129 active+clean; 216 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 308 op/s 2024-04-21T08:49:06.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:06 smithi082 ceph-mon[27967]: pgmap v65: 129 pgs: 129 active+clean; 216 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 308 op/s 2024-04-21T08:49:06.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:06 smithi106 ceph-mon[31072]: pgmap v65: 129 pgs: 129 active+clean; 216 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 308 op/s 2024-04-21T08:49:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:07 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:49:07.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:07 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:49:07.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:07 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:49:08.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:08 smithi134 ceph-mon[31772]: pgmap v66: 129 pgs: 129 active+clean; 216 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 320 op/s 2024-04-21T08:49:08.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:08 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:49:08.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:08 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:49:08.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:08 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:49:08.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:08 smithi082 ceph-mon[27967]: pgmap v66: 129 pgs: 129 active+clean; 216 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 320 op/s 2024-04-21T08:49:08.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:08 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:49:08.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:08 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:49:08.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:08 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:49:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:08 smithi106 ceph-mon[31072]: pgmap v66: 129 pgs: 129 active+clean; 216 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 320 op/s 2024-04-21T08:49:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:08 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:49:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:08 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:49:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:08 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:49:09.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:09 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:49:09.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:09 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:49:09.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:09 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:49:09.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:09 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:49:09.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:09 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:49:09.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:09 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:49:09.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:09 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:49:09.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:09 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:49:09.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:09 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:49:09.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:09 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:49:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:09 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:49:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:09 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:49:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:09 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:49:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:09 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:49:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:09 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:49:10.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:10 smithi134 ceph-mon[31772]: pgmap v67: 129 pgs: 129 active+clean; 227 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 338 op/s 2024-04-21T08:49:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.529+0000 7efef7ad2640 1 -- 172.21.15.82:0/2346800640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efef01043e0 msgr2=0x7efef01047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.529+0000 7efef7ad2640 1 --2- 172.21.15.82:0/2346800640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efef01043e0 0x7efef01047c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efeec009900 tx=0x7efeec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.529+0000 7efef7ad2640 1 -- 172.21.15.82:0/2346800640 shutdown_connections 2024-04-21T08:49:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.529+0000 7efef7ad2640 1 --2- 172.21.15.82:0/2346800640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef01094b0 0x7efef0111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.529+0000 7efef7ad2640 1 --2- 172.21.15.82:0/2346800640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efef0104d90 0x7efef0108dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.529+0000 7efef7ad2640 1 --2- 172.21.15.82:0/2346800640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efef01043e0 0x7efef01047c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.529+0000 7efef7ad2640 1 -- 172.21.15.82:0/2346800640 >> 172.21.15.82:0/2346800640 conn(0x7efef00ffc60 msgr2=0x7efef0102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.529+0000 7efef7ad2640 1 -- 172.21.15.82:0/2346800640 shutdown_connections 2024-04-21T08:49:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.529+0000 7efef7ad2640 1 -- 172.21.15.82:0/2346800640 wait complete. 2024-04-21T08:49:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef7ad2640 1 Processor -- start 2024-04-21T08:49:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef7ad2640 1 -- start start 2024-04-21T08:49:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef7ad2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efef01043e0 0x7efef01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef7ad2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efef01094b0 0x7efef01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef7ad2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef019a770 0x7efef019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef7ad2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efef0114410 con 0x7efef019a770 2024-04-21T08:49:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef7ad2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efef0114290 con 0x7efef01094b0 2024-04-21T08:49:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef7ad2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efef0114590 con 0x7efef01043e0 2024-04-21T08:49:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef606f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef019a770 0x7efef019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef606f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef019a770 0x7efef019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47700/0 (socket says 172.21.15.82:47700) 2024-04-21T08:49:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef586e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efef01043e0 0x7efef01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef606f640 1 -- 172.21.15.82:0/2584495584 learned_addr learned my addr 172.21.15.82:0/2584495584 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef506d640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efef01094b0 0x7efef01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef506d640 1 -- 172.21.15.82:0/2584495584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efef01043e0 msgr2=0x7efef01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef506d640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efef01043e0 0x7efef01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef506d640 1 -- 172.21.15.82:0/2584495584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef019a770 msgr2=0x7efef019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef506d640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef019a770 0x7efef019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef506d640 1 -- 172.21.15.82:0/2584495584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef019b370 con 0x7efef01094b0 2024-04-21T08:49:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.530+0000 7efef586e640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efef01043e0 0x7efef01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.531+0000 7efef606f640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef019a770 0x7efef019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.531+0000 7efef506d640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efef01094b0 0x7efef01a0bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efee000e9c0 tx=0x7efee000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.531+0000 7efee6ffd640 1 -- 172.21.15.82:0/2584495584 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee000ccc0 con 0x7efef01094b0 2024-04-21T08:49:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.531+0000 7efef7ad2640 1 -- 172.21.15.82:0/2584495584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef01a7450 con 0x7efef01094b0 2024-04-21T08:49:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.531+0000 7efee6ffd640 1 -- 172.21.15.82:0/2584495584 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee0005020 con 0x7efef01094b0 2024-04-21T08:49:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.531+0000 7efee6ffd640 1 -- 172.21.15.82:0/2584495584 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0004a00 con 0x7efef01094b0 2024-04-21T08:49:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.531+0000 7efef7ad2640 1 -- 172.21.15.82:0/2584495584 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef01a7850 con 0x7efef01094b0 2024-04-21T08:49:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efefa7a0740 1 Processor -- start 2024-04-21T08:49:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efee6ffd640 1 -- 172.21.15.82:0/2584495584 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efee0004ba0 con 0x7efef01094b0 2024-04-21T08:49:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efefa7a0740 1 -- start start 2024-04-21T08:49:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efefa7a0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f91620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efefa7a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010f91fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efefa7a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 0x563010f9d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efefa7a0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563010ef68d0 con 0x563010f928d0 2024-04-21T08:49:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efefa7a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563010e93e80 con 0x563010f91b60 2024-04-21T08:49:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efefa7a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563010ef2890 con 0x563010f9b0e0 2024-04-21T08:49:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efef606f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f91620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efef586e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010f91fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efef586e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010f91fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:59828/0 (socket says 172.21.15.82:59828) 2024-04-21T08:49:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efef506d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 0x563010f9d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efef586e640 1 -- 172.21.15.82:0/1079936782 learned_addr learned my addr 172.21.15.82:0/1079936782 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.534+0000 7efef606f640 1 --2- 172.21.15.82:0/1079936782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f91620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47708/0 (socket says 172.21.15.82:47708) 2024-04-21T08:49:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efee6ffd640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efedc0a3dc0 0x7efedc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efef586e640 1 -- 172.21.15.82:0/1079936782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 msgr2=0x563010f9d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efef586e640 1 --2- 172.21.15.82:0/1079936782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 0x563010f9d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efef586e640 1 -- 172.21.15.82:0/1079936782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 msgr2=0x563010f91620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efef586e640 1 --2- 172.21.15.82:0/1079936782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f91620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efef586e640 1 -- 172.21.15.82:0/1079936782 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563010eb5320 con 0x563010f91b60 2024-04-21T08:49:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efef606f640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efedc0a3dc0 0x7efedc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efee6ffd640 1 -- 172.21.15.82:0/2584495584 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7efee00d0b80 con 0x7efef01094b0 2024-04-21T08:49:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efef586e640 1 --2- 172.21.15.82:0/1079936782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010f91fe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efeec009900 tx=0x7efeec0095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c78ac8efb317a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efef606f640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efedc0a3dc0 0x7efedc0a6280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efee80049d0 tx=0x7efee8004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efedbfff640 1 -- 172.21.15.82:0/1079936782 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec045070 con 0x563010f91b60 2024-04-21T08:49:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efedbfff640 1 -- 172.21.15.82:0/1079936782 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeec02c7e0 con 0x563010f91b60 2024-04-21T08:49:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.535+0000 7efedbfff640 1 -- 172.21.15.82:0/1079936782 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec003a50 con 0x563010f91b60 2024-04-21T08:49:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.536+0000 7efefa7a0740 1 -- 172.21.15.82:0/1079936782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 msgr2=0x563010f91fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.536+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1079936782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010f91fe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efeec009900 tx=0x7efeec0095f0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.536+0000 7efefa7a0740 1 -- 172.21.15.82:0/1079936782 shutdown_connections 2024-04-21T08:49:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.536+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1079936782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 0x563010f9d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.536+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1079936782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010f91fe0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.536+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1079936782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f91620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.536+0000 7efefa7a0740 1 -- 172.21.15.82:0/1079936782 >> 172.21.15.82:0/1079936782 conn(0x563010f58c90 msgr2=0x563010f8c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.536+0000 7efefa7a0740 1 -- 172.21.15.82:0/1079936782 shutdown_connections 2024-04-21T08:49:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.536+0000 7efefa7a0740 1 -- 172.21.15.82:0/1079936782 wait complete. 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.536+0000 7efefa7a0740 1 -- start start 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efefa7a0740 1 Processor -- start 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efefa7a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010f9aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efefa7a0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f97d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efefa7a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 0x563010f982d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efefa7a0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563010f9ddb0 con 0x563010f928d0 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efefa7a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563010e7eeb0 con 0x563010f91b60 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efefa7a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563010f9df30 con 0x563010f9b0e0 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efefa7a0740 10 client.0 _ll_register_callbacks cb 0x563010eabb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efef506d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f97d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efef586e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010f9aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efef606f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 0x563010f982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efef506d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f97d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47718/0 (socket says 172.21.15.82:47718) 2024-04-21T08:49:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.537+0000 7efef506d640 1 -- 172.21.15.82:0/1293143181 learned_addr learned my addr 172.21.15.82:0/1293143181 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efef506d640 1 -- 172.21.15.82:0/1293143181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 msgr2=0x563010f982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efef506d640 1 --2- 172.21.15.82:0/1293143181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 0x563010f982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efef506d640 1 -- 172.21.15.82:0/1293143181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 msgr2=0x563010f9aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efef506d640 1 --2- 172.21.15.82:0/1293143181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010f9aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efef506d640 1 -- 172.21.15.82:0/1293143181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563010c09210 con 0x563010f928d0 2024-04-21T08:49:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efef506d640 1 --2- 172.21.15.82:0/1293143181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f97d90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7efee009c400 tx=0x7efee0004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa82dbae407abe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efed97fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efed97fa640 1 -- 172.21.15.82:0/1293143181 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee001c070 con 0x563010f928d0 2024-04-21T08:49:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efed97fa640 1 -- 172.21.15.82:0/1293143181 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7efee00054f0 con 0x563010f928d0 2024-04-21T08:49:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efefa7a0740 1 -- 172.21.15.82:0/1293143181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeec02c7e0 con 0x563010f928d0 2024-04-21T08:49:10.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efed97fa640 1 -- 172.21.15.82:0/1293143181 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efee009c060 con 0x563010f928d0 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efefa7a0740 10 client.16638 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efefa7a0740 1 -- 172.21.15.82:0/1293143181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563010cd5620 con 0x563010f928d0 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.538+0000 7efed97fa640 1 -- 172.21.15.82:0/1293143181 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeec02c7e0 con 0x563010f928d0 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.539+0000 7efed97fa640 1 -- 172.21.15.82:0/1293143181 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7efee00b5020 con 0x563010f928d0 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.539+0000 7efefa7a0740 10 client.16638 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.539+0000 7efefa7a0740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.539+0000 7efefa7a0740 20 client.16638 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.539+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1293143181 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563010fab060 0x563010fad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.539+0000 7efefa7a0740 4 client.16638 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.539+0000 7efefa7a0740 1 -- 172.21.15.82:0/1293143181 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563010f0a880 con 0x563010fab060 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.539+0000 7efef586e640 1 --2- 172.21.15.82:0/1293143181 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563010fab060 0x563010fad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.539+0000 7efed97fa640 10 client.16638 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:10.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.539+0000 7efef586e640 1 --2- 172.21.15.82:0/1293143181 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563010fab060 0x563010fad470 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e11efe81ef415ad server_cookie=d4c96e8615cbef7d in_seq=0 out_seq=0 2024-04-21T08:49:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efed97fa640 1 -- 172.21.15.82:0/1293143181 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563010f0a880 con 0x563010fab060 2024-04-21T08:49:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efed97fa640 10 client.16638 handle_command_reply: tid=0 2024-04-21T08:49:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efefa7a0740 1 client.16638 shutdown 2024-04-21T08:49:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efefa7a0740 1 -- 172.21.15.82:0/1293143181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 msgr2=0x563010f97d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1293143181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f97d90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7efee009c400 tx=0x7efee0004f50 comp rx=0 tx=0).stop 2024-04-21T08:49:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efefa7a0740 1 -- 172.21.15.82:0/1293143181 shutdown_connections 2024-04-21T08:49:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1293143181 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563010fab060 0x563010fad470 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1293143181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 0x563010f982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1293143181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f97d90 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1293143181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010f9aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efed97fa640 0 client.16638 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efefa7a0740 1 -- 172.21.15.82:0/1293143181 >> 172.21.15.82:0/1293143181 conn(0x563010f58c90 msgr2=0x563010f8e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efefa7a0740 1 -- 172.21.15.82:0/1293143181 shutdown_connections 2024-04-21T08:49:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efefa7a0740 1 -- 172.21.15.82:0/1293143181 wait complete. 2024-04-21T08:49:10.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.540+0000 7efefa7a0740 20 client.16638 trim_cache size 0 max 16384 2024-04-21T08:49:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.556+0000 7efefa7a0740 1 Processor -- start 2024-04-21T08:49:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.556+0000 7efefa7a0740 1 -- start start 2024-04-21T08:49:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efefa7a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f91b60 0x563010fa6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efefa7a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f928d0 0x563010fb2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efefa7a0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f9b0e0 0x563010fb32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efefa7a0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efef01b1500 con 0x563010f9b0e0 2024-04-21T08:49:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efefa7a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efef01b3130 con 0x563010f928d0 2024-04-21T08:49:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efefa7a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efef01b1bd0 con 0x563010f91b60 2024-04-21T08:49:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efef606f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f9b0e0 0x563010fb32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efef606f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f9b0e0 0x563010fb32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47726/0 (socket says 172.21.15.82:47726) 2024-04-21T08:49:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efef606f640 1 -- 172.21.15.82:0/1362745705 learned_addr learned my addr 172.21.15.82:0/1362745705 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efef586e640 1 --2- 172.21.15.82:0/1362745705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f91b60 0x563010fa6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efef506d640 1 --2- 172.21.15.82:0/1362745705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f928d0 0x563010fb2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efef606f640 1 -- 172.21.15.82:0/1362745705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f91b60 msgr2=0x563010fa6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efef606f640 1 --2- 172.21.15.82:0/1362745705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f91b60 0x563010fa6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efef606f640 1 -- 172.21.15.82:0/1362745705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f928d0 msgr2=0x563010fb2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efef606f640 1 --2- 172.21.15.82:0/1362745705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f928d0 0x563010fb2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efef606f640 1 -- 172.21.15.82:0/1362745705 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563010f962e0 con 0x563010f9b0e0 2024-04-21T08:49:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efef606f640 1 --2- 172.21.15.82:0/1362745705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f9b0e0 0x563010fb32c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7efee800fd00 tx=0x7efee801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ae0798dc3445cd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efedb7fe640 1 -- 172.21.15.82:0/1362745705 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee800fd30 con 0x563010f9b0e0 2024-04-21T08:49:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efedb7fe640 1 -- 172.21.15.82:0/1362745705 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee802d070 con 0x563010f9b0e0 2024-04-21T08:49:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.557+0000 7efedb7fe640 1 -- 172.21.15.82:0/1362745705 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee800e610 con 0x563010f9b0e0 2024-04-21T08:49:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.558+0000 7efefa7a0740 1 -- 172.21.15.82:0/1362745705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f9b0e0 msgr2=0x563010fb32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.558+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1362745705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f9b0e0 0x563010fb32c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7efee800fd00 tx=0x7efee801a000 comp rx=0 tx=0).stop 2024-04-21T08:49:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.558+0000 7efefa7a0740 1 -- 172.21.15.82:0/1362745705 shutdown_connections 2024-04-21T08:49:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.558+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1362745705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f9b0e0 0x563010fb32c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.558+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1362745705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f928d0 0x563010fb2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.558+0000 7efefa7a0740 1 --2- 172.21.15.82:0/1362745705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f91b60 0x563010fa6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.558+0000 7efefa7a0740 1 -- 172.21.15.82:0/1362745705 >> 172.21.15.82:0/1362745705 conn(0x563010f58c90 msgr2=0x563010f8c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.558+0000 7efefa7a0740 1 -- 172.21.15.82:0/1362745705 shutdown_connections 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.558+0000 7efefa7a0740 1 -- 172.21.15.82:0/1362745705 wait complete. 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.558+0000 7efefa7a0740 1 -- start start 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.558+0000 7efefa7a0740 1 Processor -- start 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efefa7a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010fa3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efefa7a0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f946c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efefa7a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 0x563010f94c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efefa7a0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efef01b2a30 con 0x563010f928d0 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efefa7a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efef01b2360 con 0x563010f91b60 2024-04-21T08:49:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efefa7a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efef01af4e0 con 0x563010f9b0e0 2024-04-21T08:49:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efefa7a0740 10 client.0 _ll_register_callbacks cb 0x7efee00d4320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efef506d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efef506d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f946c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47738/0 (socket says 172.21.15.82:47738) 2024-04-21T08:49:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efef506d640 1 -- 172.21.15.82:0/396778011 learned_addr learned my addr 172.21.15.82:0/396778011 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efef586e640 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010fa3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efef606f640 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 0x563010f94c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efef506d640 1 -- 172.21.15.82:0/396778011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 msgr2=0x563010f94c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efef506d640 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 0x563010f94c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efef506d640 1 -- 172.21.15.82:0/396778011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 msgr2=0x563010fa3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efef506d640 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010fa3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efef506d640 1 -- 172.21.15.82:0/396778011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563010f952c0 con 0x563010f928d0 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efef506d640 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f946c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x563010f975f0 tx=0x7efee0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900e4471f1287e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efed97fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.559+0000 7efed97fa640 1 -- 172.21.15.82:0/396778011 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee001c040 con 0x563010f928d0 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efed97fa640 1 -- 172.21.15.82:0/396778011 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7efee009bd60 con 0x563010f928d0 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efefa7a0740 1 -- 172.21.15.82:0/396778011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee802d070 con 0x563010f928d0 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efed97fa640 1 -- 172.21.15.82:0/396778011 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efee00051d0 con 0x563010f928d0 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efefa7a0740 10 client.16644 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efefa7a0740 1 -- 172.21.15.82:0/396778011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563011011520 con 0x563010f928d0 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efed97fa640 1 -- 172.21.15.82:0/396778011 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee802d070 con 0x563010f928d0 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efed97fa640 1 -- 172.21.15.82:0/396778011 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7efee00b5040 con 0x563010f928d0 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efefa7a0740 10 client.16644 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:10.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efefa7a0740 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efefa7a0740 20 client.16644 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efefa7a0740 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563011011b40 0x563011013f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efefa7a0740 4 client.16644 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:49:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efefa7a0740 1 -- 172.21.15.82:0/396778011 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563010e963d0 con 0x563011011b40 2024-04-21T08:49:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.560+0000 7efef586e640 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563011011b40 0x563011013f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.561+0000 7efed97fa640 10 client.16644 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.561+0000 7efef586e640 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563011011b40 0x563011013f60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df8c22bf544d7fa6 server_cookie=fb298208578dc3b9 in_seq=0 out_seq=0 2024-04-21T08:49:10.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:10 smithi082 ceph-mon[27967]: pgmap v67: 129 pgs: 129 active+clean; 227 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 338 op/s 2024-04-21T08:49:10.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:10 smithi106 ceph-mon[31072]: pgmap v67: 129 pgs: 129 active+clean; 227 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 338 op/s 2024-04-21T08:49:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.959+0000 7efed97fa640 1 -- 172.21.15.82:0/396778011 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563010e963d0 con 0x563011011b40 2024-04-21T08:49:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.959+0000 7efed97fa640 10 client.16644 handle_command_reply: tid=0 2024-04-21T08:49:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efefa7a0740 1 client.16644 shutdown 2024-04-21T08:49:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efefa7a0740 1 -- 172.21.15.82:0/396778011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 msgr2=0x563010f946c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efefa7a0740 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f946c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x563010f975f0 tx=0x7efee0005720 comp rx=0 tx=0).stop 2024-04-21T08:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efefa7a0740 1 -- 172.21.15.82:0/396778011 shutdown_connections 2024-04-21T08:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efefa7a0740 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563011011b40 0x563011013f60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efefa7a0740 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563010f9b0e0 0x563010f94c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efed97fa640 0 client.16644 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efefa7a0740 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563010f928d0 0x563010f946c0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efefa7a0740 1 --2- 172.21.15.82:0/396778011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563010f91b60 0x563010fa3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efefa7a0740 1 -- 172.21.15.82:0/396778011 >> 172.21.15.82:0/396778011 conn(0x563010f58c90 msgr2=0x563010f965a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efefa7a0740 1 -- 172.21.15.82:0/396778011 shutdown_connections 2024-04-21T08:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efefa7a0740 1 -- 172.21.15.82:0/396778011 wait complete. 2024-04-21T08:49:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.960+0000 7efefa7a0740 20 client.16644 trim_cache size 0 max 16384 2024-04-21T08:49:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:49:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:49:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:49:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:49:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.961+0000 7efef7ad2640 1 -- 172.21.15.82:0/2584495584 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efedc0a3dc0 msgr2=0x7efedc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.961+0000 7efef7ad2640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efedc0a3dc0 0x7efedc0a6280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efee80049d0 tx=0x7efee8004a40 comp rx=0 tx=0).stop 2024-04-21T08:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.961+0000 7efef7ad2640 1 -- 172.21.15.82:0/2584495584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efef01094b0 msgr2=0x7efef01a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.961+0000 7efef7ad2640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efef01094b0 0x7efef01a0bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efee000e9c0 tx=0x7efee000ee80 comp rx=0 tx=0).stop 2024-04-21T08:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.962+0000 7efef7ad2640 1 -- 172.21.15.82:0/2584495584 shutdown_connections 2024-04-21T08:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.962+0000 7efef7ad2640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7efedc0a3dc0 0x7efedc0a6280 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.962+0000 7efef7ad2640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef019a770 0x7efef019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.962+0000 7efef7ad2640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efef01094b0 0x7efef01a0bf0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efee000e9c0 tx=0x7efee000ee80 comp rx=0 tx=0).stop 2024-04-21T08:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.962+0000 7efef7ad2640 1 --2- 172.21.15.82:0/2584495584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efef01043e0 0x7efef01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.962+0000 7efef7ad2640 1 -- 172.21.15.82:0/2584495584 >> 172.21.15.82:0/2584495584 conn(0x7efef00ffc60 msgr2=0x7efef0101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.962+0000 7efef7ad2640 1 -- 172.21.15.82:0/2584495584 shutdown_connections 2024-04-21T08:49:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:10.962+0000 7efef7ad2640 1 -- 172.21.15.82:0/2584495584 wait complete. 2024-04-21T08:49:12.429 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:12 smithi106 ceph-mon[31072]: pgmap v68: 129 pgs: 129 active+clean; 227 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-04-21T08:49:12.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:12 smithi134 ceph-mon[31772]: pgmap v68: 129 pgs: 129 active+clean; 227 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-04-21T08:49:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:12 smithi082 ceph-mon[27967]: pgmap v68: 129 pgs: 129 active+clean; 227 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-04-21T08:49:14.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:14 smithi134 ceph-mon[31772]: pgmap v69: 129 pgs: 129 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 265 op/s 2024-04-21T08:49:14.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:14 smithi082 ceph-mon[27967]: pgmap v69: 129 pgs: 129 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 265 op/s 2024-04-21T08:49:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:14 smithi106 ceph-mon[31072]: pgmap v69: 129 pgs: 129 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 265 op/s 2024-04-21T08:49:16.222 INFO:tasks.workunit.client.0.smithi082.stdout: 72 1368977 561 933742 860 744314 1646 2024-04-21T08:49:16.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:16 smithi134 ceph-mon[31772]: pgmap v70: 129 pgs: 129 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-04-21T08:49:16.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:16 smithi082 ceph-mon[27967]: pgmap v70: 129 pgs: 129 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-04-21T08:49:16.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:16 smithi106 ceph-mon[31072]: pgmap v70: 129 pgs: 129 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-04-21T08:49:17.819 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-21T08:49:17.820 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-21T08:49:17.820 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:49:17.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:49:17.887 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:49:18.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.340+0000 7f0b0146d640 1 -- 172.21.15.82:0/1137035548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0afc108f50 msgr2=0x7f0afc109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.340+0000 7f0b0146d640 1 --2- 172.21.15.82:0/1137035548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0afc108f50 0x7f0afc109330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009ca0 tx=0x7f0af402c120 comp rx=0 tx=0).stop 2024-04-21T08:49:18.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.340+0000 7f0b0146d640 1 -- 172.21.15.82:0/1137035548 shutdown_connections 2024-04-21T08:49:18.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.340+0000 7f0b0146d640 1 --2- 172.21.15.82:0/1137035548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0afc1150a0 0x7f0afc117490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.340+0000 7f0b0146d640 1 --2- 172.21.15.82:0/1137035548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0afc109870 0x7f0afc114b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.340+0000 7f0b0146d640 1 --2- 172.21.15.82:0/1137035548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0afc108f50 0x7f0afc109330 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.340+0000 7f0b0146d640 1 -- 172.21.15.82:0/1137035548 >> 172.21.15.82:0/1137035548 conn(0x7f0afc06ce90 msgr2=0x7f0afc06d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.342+0000 7f0b0146d640 1 -- 172.21.15.82:0/1137035548 shutdown_connections 2024-04-21T08:49:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.342+0000 7f0b0146d640 1 -- 172.21.15.82:0/1137035548 wait complete. 2024-04-21T08:49:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.343+0000 7f0b0146d640 1 Processor -- start 2024-04-21T08:49:18.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.343+0000 7f0b0146d640 1 -- start start 2024-04-21T08:49:18.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0b0146d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0afc108f50 0x7f0afc10c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0b0146d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0afc109870 0x7f0afc10a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0b0146d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0afc1150a0 0x7f0afc10b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0b0146d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc11a860 con 0x7f0afc1150a0 2024-04-21T08:49:18.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0b0146d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc11a6e0 con 0x7f0afc108f50 2024-04-21T08:49:18.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0b0146d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc11a9e0 con 0x7f0afc109870 2024-04-21T08:49:18.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0afaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0afc108f50 0x7f0afc10c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0afaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0afc108f50 0x7f0afc10c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:60314/0 (socket says 172.21.15.82:60314) 2024-04-21T08:49:18.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0afaffd640 1 -- 172.21.15.82:0/653927918 learned_addr learned my addr 172.21.15.82:0/653927918 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:18.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0afa7fc640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0afc109870 0x7f0afc10a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0afaffd640 1 -- 172.21.15.82:0/653927918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0afc109870 msgr2=0x7f0afc10a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0afaffd640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0afc109870 0x7f0afc10a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0afaffd640 1 -- 172.21.15.82:0/653927918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0afc1150a0 msgr2=0x7f0afc10b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0afb7fe640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0afc1150a0 0x7f0afc10b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0afaffd640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0afc1150a0 0x7f0afc10b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0afaffd640 1 -- 172.21.15.82:0/653927918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0afc1b3740 con 0x7f0afc108f50 2024-04-21T08:49:18.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.344+0000 7f0afb7fe640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0afc1150a0 0x7f0afc10b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:18.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.345+0000 7f0afaffd640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0afc108f50 0x7f0afc10c5c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0af402c730 tx=0x7f0af4035ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.349+0000 7f0ae3fff640 1 -- 172.21.15.82:0/653927918 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4039040 con 0x7f0afc108f50 2024-04-21T08:49:18.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.349+0000 7f0ae3fff640 1 -- 172.21.15.82:0/653927918 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af4044070 con 0x7f0afc108f50 2024-04-21T08:49:18.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.349+0000 7f0b0146d640 1 -- 172.21.15.82:0/653927918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0afc1b3910 con 0x7f0afc108f50 2024-04-21T08:49:18.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.349+0000 7f0ae3fff640 1 -- 172.21.15.82:0/653927918 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4004510 con 0x7f0afc108f50 2024-04-21T08:49:18.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.349+0000 7f0b0146d640 1 -- 172.21.15.82:0/653927918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0afc1b3dd0 con 0x7f0afc108f50 2024-04-21T08:49:18.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.351+0000 7f0b0146d640 1 -- 172.21.15.82:0/653927918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0afc10a640 con 0x7f0afc108f50 2024-04-21T08:49:18.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.352+0000 7f0ae3fff640 1 -- 172.21.15.82:0/653927918 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0af40046b0 con 0x7f0afc108f50 2024-04-21T08:49:18.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.353+0000 7f0ae3fff640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0ae40a3f20 0x7f0ae40a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.354+0000 7f0afa7fc640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0ae40a3f20 0x7f0ae40a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.354+0000 7f0afa7fc640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0ae40a3f20 0x7f0ae40a63e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0afc10c130 tx=0x7f0aec0090e0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.354+0000 7f0ae3fff640 1 -- 172.21.15.82:0/653927918 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f0af40f3120 con 0x7f0afc108f50 2024-04-21T08:49:18.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:18 smithi082 ceph-mon[27967]: pgmap v71: 129 pgs: 129 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-04-21T08:49:18.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.358+0000 7f0ae3fff640 1 -- 172.21.15.82:0/653927918 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0af40bf4b0 con 0x7f0afc108f50 2024-04-21T08:49:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.420+0000 7f076d46d640 1 -- 172.21.15.82:0/2123127898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0768101080 msgr2=0x7f0768101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.420+0000 7f076d46d640 1 --2- 172.21.15.82:0/2123127898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0768101080 0x7f0768101460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0758009900 tx=0x7f075802bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.420+0000 7f076d46d640 1 -- 172.21.15.82:0/2123127898 shutdown_connections 2024-04-21T08:49:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.420+0000 7f076d46d640 1 --2- 172.21.15.82:0/2123127898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0768106150 0x7f076810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.420+0000 7f076d46d640 1 --2- 172.21.15.82:0/2123127898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0768101a30 0x7f0768105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.420+0000 7f076d46d640 1 --2- 172.21.15.82:0/2123127898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0768101080 0x7f0768101460 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.420+0000 7f076d46d640 1 -- 172.21.15.82:0/2123127898 >> 172.21.15.82:0/2123127898 conn(0x7f0768078000 msgr2=0x7f0768078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.420+0000 7f076d46d640 1 -- 172.21.15.82:0/2123127898 shutdown_connections 2024-04-21T08:49:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.420+0000 7f076d46d640 1 -- 172.21.15.82:0/2123127898 wait complete. 2024-04-21T08:49:18.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.421+0000 7f076d46d640 1 Processor -- start 2024-04-21T08:49:18.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.421+0000 7f076d46d640 1 -- start start 2024-04-21T08:49:18.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.421+0000 7f076d46d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0768101080 0x7f0768197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.421+0000 7f076d46d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0768106150 0x7f0768197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.421+0000 7f076d46d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076819bf00 0x7f076819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f076d46d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0768111100 con 0x7f076819bf00 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f076d46d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0768110f80 con 0x7f0768106150 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f076d46d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0768111280 con 0x7f0768101080 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f07677fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076819bf00 0x7f076819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f0766ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0768101080 0x7f0768197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f07667fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0768106150 0x7f0768197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f0766ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0768101080 0x7f0768197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:49654/0 (socket says 172.21.15.82:49654) 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f0766ffd640 1 -- 172.21.15.82:0/349476502 learned_addr learned my addr 172.21.15.82:0/349476502 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f07667fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0768106150 0x7f0768197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:60336/0 (socket says 172.21.15.82:60336) 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f0766ffd640 1 -- 172.21.15.82:0/349476502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0768106150 msgr2=0x7f0768197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f0766ffd640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0768106150 0x7f0768197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f0766ffd640 1 -- 172.21.15.82:0/349476502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076819bf00 msgr2=0x7f076819c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f0766ffd640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076819bf00 0x7f076819c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f0766ffd640 1 -- 172.21.15.82:0/349476502 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076819c9f0 con 0x7f0768101080 2024-04-21T08:49:18.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f07677fe640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076819bf00 0x7f076819c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:18.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.422+0000 7f0766ffd640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0768101080 0x7f0768197640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0758000c00 tx=0x7f075802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.423+0000 7f074ffff640 1 -- 172.21.15.82:0/349476502 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758043020 con 0x7f0768101080 2024-04-21T08:49:18.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.423+0000 7f076d46d640 1 -- 172.21.15.82:0/349476502 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076819cc80 con 0x7f0768101080 2024-04-21T08:49:18.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.423+0000 7f074ffff640 1 -- 172.21.15.82:0/349476502 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f075802caf0 con 0x7f0768101080 2024-04-21T08:49:18.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.423+0000 7f074ffff640 1 -- 172.21.15.82:0/349476502 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758003ab0 con 0x7f0768101080 2024-04-21T08:49:18.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.423+0000 7f076d46d640 1 -- 172.21.15.82:0/349476502 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07680693d0 con 0x7f0768101080 2024-04-21T08:49:18.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.425+0000 7f074ffff640 1 -- 172.21.15.82:0/349476502 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0758046020 con 0x7f0768101080 2024-04-21T08:49:18.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.425+0000 7f0770068740 1 Processor -- start 2024-04-21T08:49:18.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.425+0000 7f0770068740 1 -- start start 2024-04-21T08:49:18.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.425+0000 7f0770068740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 0x557765c13960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.425+0000 7f0770068740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c13ea0 0x557765c1c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.425+0000 7f0770068740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c1ca80 0x557765c1ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.425+0000 7f0770068740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557765b781e0 con 0x557765c1ca80 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.425+0000 7f0770068740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557765b15390 con 0x557765c13ea0 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.425+0000 7f0770068740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557765b741a0 con 0x557765c13580 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.426+0000 7f07667fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c1ca80 0x557765c1ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.426+0000 7f07677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 0x557765c13960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.426+0000 7f07677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 0x557765c13960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:49668/0 (socket says 172.21.15.82:49668) 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.426+0000 7f07677fe640 1 -- 172.21.15.82:0/911858538 learned_addr learned my addr 172.21.15.82:0/911858538 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.426+0000 7f074ffff640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f07500a3da0 0x7f07500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.426+0000 7f07677fe640 1 -- 172.21.15.82:0/911858538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c13ea0 msgr2=0x557765c1c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.426+0000 7f074ffff640 1 -- 172.21.15.82:0/349476502 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f07580f2cb0 con 0x7f0768101080 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.427+0000 7f0766ffd640 1 --2- 172.21.15.82:0/911858538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c13ea0 0x557765c1c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.427+0000 7f07677fe640 1 --2- 172.21.15.82:0/911858538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c13ea0 0x557765c1c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.427+0000 7f07677fe640 1 -- 172.21.15.82:0/911858538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c1ca80 msgr2=0x557765c1ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.427+0000 7f07677fe640 1 --2- 172.21.15.82:0/911858538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c1ca80 0x557765c1ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.427+0000 7f07677fe640 1 -- 172.21.15.82:0/911858538 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577659547d0 con 0x557765c13580 2024-04-21T08:49:18.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.427+0000 7f07667fc640 1 --2- 172.21.15.82:0/911858538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c1ca80 0x557765c1ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:18.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.427+0000 7f07677fe640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f07500a3da0 0x7f07500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.427+0000 7f07677fe640 1 --2- 172.21.15.82:0/911858538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 0x557765c13960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0760010b40 tx=0x7f076000b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14a441eb8536e766 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.427+0000 7f074d7fa640 1 -- 172.21.15.82:0/911858538 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760021040 con 0x557765c13580 2024-04-21T08:49:18.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.427+0000 7f074d7fa640 1 -- 172.21.15.82:0/911858538 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0760028db0 con 0x557765c13580 2024-04-21T08:49:18.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.427+0000 7f074d7fa640 1 -- 172.21.15.82:0/911858538 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07600275b0 con 0x557765c13580 2024-04-21T08:49:18.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.428+0000 7f0770068740 1 -- 172.21.15.82:0/911858538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 msgr2=0x557765c13960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.428+0000 7f0770068740 1 --2- 172.21.15.82:0/911858538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 0x557765c13960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0760010b40 tx=0x7f076000b2c0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.428+0000 7f07677fe640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f07500a3da0 0x7f07500a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0760027000 tx=0x7f0760028000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.428+0000 7f0770068740 1 -- 172.21.15.82:0/911858538 shutdown_connections 2024-04-21T08:49:18.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.428+0000 7f0770068740 1 --2- 172.21.15.82:0/911858538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c1ca80 0x557765c1ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.428+0000 7f0770068740 1 --2- 172.21.15.82:0/911858538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c13ea0 0x557765c1c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.428+0000 7f0770068740 1 --2- 172.21.15.82:0/911858538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 0x557765c13960 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.428+0000 7f0770068740 1 -- 172.21.15.82:0/911858538 >> 172.21.15.82:0/911858538 conn(0x557765c0dcb0 msgr2=0x557765c0e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.428+0000 7f0770068740 1 -- 172.21.15.82:0/911858538 shutdown_connections 2024-04-21T08:49:18.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.428+0000 7f0770068740 1 -- 172.21.15.82:0/911858538 wait complete. 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.429+0000 7f0770068740 1 -- start start 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.429+0000 7f0770068740 1 Processor -- start 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.429+0000 7f0770068740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 0x557765c19b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.429+0000 7f0770068740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 0x557765c1a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f0770068740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c1a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f0770068740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557765c1f870 con 0x557765c13580 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f0770068740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557765b312f0 con 0x557765c1ca80 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f07667fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 0x557765c1a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f07667fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 0x557765c1a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:49676/0 (socket says 172.21.15.82:49676) 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f07677fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c1a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f07667fc640 1 -- 172.21.15.82:0/3535137399 learned_addr learned my addr 172.21.15.82:0/3535137399 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f0770068740 1 -- 172.21.15.82:0/3535137399 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557765c1f9f0 con 0x557765c13ea0 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f0766ffd640 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 0x557765c19b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f0770068740 10 client.0 _ll_register_callbacks cb 0x557765b3e580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:18.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f07677fe640 1 -- 172.21.15.82:0/3535137399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 msgr2=0x557765c1a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f07677fe640 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 0x557765c1a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f07677fe640 1 -- 172.21.15.82:0/3535137399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 msgr2=0x557765c19b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f07677fe640 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 0x557765c19b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f07677fe640 1 -- 172.21.15.82:0/3535137399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55776588a3f0 con 0x557765c1ca80 2024-04-21T08:49:18.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f07667fc640 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 0x557765c1a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:18.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.430+0000 7f0766ffd640 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 0x557765c19b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:18.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.431+0000 7f07677fe640 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c1a5d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0760010b40 tx=0x7f0760029ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f69ffe8b29a515b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.431+0000 7f074aff5640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:49:18.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.431+0000 7f074aff5640 1 -- 172.21.15.82:0/3535137399 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760021050 con 0x557765c1ca80 2024-04-21T08:49:18.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.431+0000 7f0770068740 1 -- 172.21.15.82:0/3535137399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0760028db0 con 0x557765c1ca80 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.431+0000 7f074aff5640 1 -- 172.21.15.82:0/3535137399 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f076001d600 con 0x557765c1ca80 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.431+0000 7f074aff5640 1 -- 172.21.15.82:0/3535137399 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f076003e050 con 0x557765c1ca80 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.431+0000 7f0770068740 10 client.26003 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.431+0000 7f0770068740 1 -- 172.21.15.82:0/3535137399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55776584a0f0 con 0x557765c1ca80 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.432+0000 7f074aff5640 1 -- 172.21.15.82:0/3535137399 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0760028db0 con 0x557765c1ca80 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.432+0000 7f074aff5640 1 -- 172.21.15.82:0/3535137399 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f076002f040 con 0x557765c1ca80 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.432+0000 7f0770068740 10 client.26003 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.432+0000 7f0770068740 10 client.26003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.432+0000 7f0770068740 20 client.26003 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.432+0000 7f0770068740 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557765c2cec0 0x557765c2f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.432+0000 7f0770068740 4 client.26003 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.432+0000 7f0770068740 1 -- 172.21.15.82:0/3535137399 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557765b85c00 con 0x557765c2cec0 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.432+0000 7f0766ffd640 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557765c2cec0 0x557765c2f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.433+0000 7f074aff5640 10 client.26003 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:18.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.433+0000 7f0766ffd640 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557765c2cec0 0x557765c2f360 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e7051760633bef server_cookie=7f5a2fe7f6975bf0 in_seq=0 out_seq=0 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f074aff5640 1 -- 172.21.15.82:0/3535137399 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557765b85c00 con 0x557765c2cec0 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f074aff5640 10 client.26003 handle_command_reply: tid=0 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f0770068740 1 client.26003 shutdown 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f0770068740 1 -- 172.21.15.82:0/3535137399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 msgr2=0x557765c1a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f0770068740 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c1a5d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0760010b40 tx=0x7f0760029ff0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f0770068740 1 -- 172.21.15.82:0/3535137399 shutdown_connections 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f0770068740 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557765c2cec0 0x557765c2f360 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f0770068740 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c1a5d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f0770068740 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 0x557765c1a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f0770068740 1 --2- 172.21.15.82:0/3535137399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 0x557765c19b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f0770068740 1 -- 172.21.15.82:0/3535137399 >> 172.21.15.82:0/3535137399 conn(0x557765c0dcb0 msgr2=0x557765c0fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f074aff5640 0 client.26003 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f0770068740 1 -- 172.21.15.82:0/3535137399 shutdown_connections 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.434+0000 7f0770068740 1 -- 172.21.15.82:0/3535137399 wait complete. 2024-04-21T08:49:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.435+0000 7f0770068740 20 client.26003 trim_cache size 0 max 16384 2024-04-21T08:49:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f0770068740 1 Processor -- start 2024-04-21T08:49:18.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f0770068740 1 -- start start 2024-04-21T08:49:18.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f0770068740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 0x557765c1c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f0770068740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13ea0 0x557765c28730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f0770068740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c34c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f0770068740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f07681ad260 con 0x557765c13ea0 2024-04-21T08:49:18.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f0770068740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f07681aee90 con 0x557765c1ca80 2024-04-21T08:49:18.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f0770068740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f07681ad930 con 0x557765c13580 2024-04-21T08:49:18.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f07667fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13ea0 0x557765c28730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f07677fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c34c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f07677fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c34c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:60360/0 (socket says 172.21.15.82:60360) 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f0766ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 0x557765c1c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.456+0000 7f07677fe640 1 -- 172.21.15.82:0/661087529 learned_addr learned my addr 172.21.15.82:0/661087529 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.457+0000 7f07667fc640 1 -- 172.21.15.82:0/661087529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 msgr2=0x557765c1c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.457+0000 7f07667fc640 1 --2- 172.21.15.82:0/661087529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 0x557765c1c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.457+0000 7f07667fc640 1 -- 172.21.15.82:0/661087529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 msgr2=0x557765c34c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.457+0000 7f07667fc640 1 --2- 172.21.15.82:0/661087529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c34c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.457+0000 7f07667fc640 1 -- 172.21.15.82:0/661087529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557765c0d4f0 con 0x557765c13ea0 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.457+0000 7f0766ffd640 1 --2- 172.21.15.82:0/661087529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 0x557765c1c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.457+0000 7f07677fe640 1 --2- 172.21.15.82:0/661087529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c34c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.457+0000 7f07667fc640 1 --2- 172.21.15.82:0/661087529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13ea0 0x557765c28730 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x557765c193b0 tx=0x7f075c00bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c536edac1d92f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.457+0000 7f074cff9640 1 -- 172.21.15.82:0/661087529 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c003d50 con 0x557765c13ea0 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.457+0000 7f074cff9640 1 -- 172.21.15.82:0/661087529 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f075c020070 con 0x557765c13ea0 2024-04-21T08:49:18.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.457+0000 7f074cff9640 1 -- 172.21.15.82:0/661087529 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c011e60 con 0x557765c13ea0 2024-04-21T08:49:18.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.459+0000 7f0770068740 1 -- 172.21.15.82:0/661087529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13ea0 msgr2=0x557765c28730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.459+0000 7f0770068740 1 --2- 172.21.15.82:0/661087529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13ea0 0x557765c28730 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x557765c193b0 tx=0x7f075c00bc90 comp rx=0 tx=0).stop 2024-04-21T08:49:18.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.459+0000 7f0770068740 1 -- 172.21.15.82:0/661087529 shutdown_connections 2024-04-21T08:49:18.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.459+0000 7f0770068740 1 --2- 172.21.15.82:0/661087529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c34c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.459+0000 7f0770068740 1 --2- 172.21.15.82:0/661087529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13ea0 0x557765c28730 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.459+0000 7f0770068740 1 --2- 172.21.15.82:0/661087529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13580 0x557765c1c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.459+0000 7f0770068740 1 -- 172.21.15.82:0/661087529 >> 172.21.15.82:0/661087529 conn(0x557765c0dcb0 msgr2=0x557765c14e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.459+0000 7f0770068740 1 -- 172.21.15.82:0/661087529 shutdown_connections 2024-04-21T08:49:18.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.459+0000 7f0770068740 1 -- 172.21.15.82:0/661087529 wait complete. 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.459+0000 7f0770068740 1 -- start start 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.459+0000 7f0770068740 1 Processor -- start 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f0770068740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 0x557765c24c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f0770068740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 0x557765c25190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f0770068740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c1fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f0770068740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f07681ae790 con 0x557765c13580 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f0770068740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f07681ae0c0 con 0x557765c1ca80 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f0770068740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f07681ab240 con 0x557765c13ea0 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f0770068740 10 client.0 _ll_register_callbacks cb 0x7f075c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f0766ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 0x557765c24c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f07677fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c1fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f0766ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 0x557765c24c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49828/0 (socket says 172.21.15.82:49828) 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f0766ffd640 1 -- 172.21.15.82:0/1751340435 learned_addr learned my addr 172.21.15.82:0/1751340435 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f07667fc640 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 0x557765c25190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f07677fe640 1 -- 172.21.15.82:0/1751340435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 msgr2=0x557765c25190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f07677fe640 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 0x557765c25190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f07677fe640 1 -- 172.21.15.82:0/1751340435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 msgr2=0x557765c24c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f07677fe640 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 0x557765c24c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f07677fe640 1 -- 172.21.15.82:0/1751340435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557765c20630 con 0x557765c1ca80 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f0766ffd640 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 0x557765c24c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.460+0000 7f07667fc640 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 0x557765c25190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.461+0000 7f07677fe640 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c1fe00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0760002230 tx=0x7f07600109f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9529fee29b93f985 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.461+0000 7f074aff5640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.461+0000 7f074aff5640 1 -- 172.21.15.82:0/1751340435 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760046070 con 0x557765c1ca80 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.461+0000 7f074aff5640 1 -- 172.21.15.82:0/1751340435 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f0760030810 con 0x557765c1ca80 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.461+0000 7f0770068740 1 -- 172.21.15.82:0/1751340435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f075c020070 con 0x557765c1ca80 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.461+0000 7f074aff5640 1 -- 172.21.15.82:0/1751340435 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0760052050 con 0x557765c1ca80 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.461+0000 7f0770068740 10 client.26009 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.461+0000 7f0770068740 1 -- 172.21.15.82:0/1751340435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557765c920c0 con 0x557765c1ca80 2024-04-21T08:49:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.462+0000 7f074aff5640 1 -- 172.21.15.82:0/1751340435 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0760035070 con 0x557765c1ca80 2024-04-21T08:49:18.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.462+0000 7f074aff5640 1 -- 172.21.15.82:0/1751340435 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f076002f050 con 0x557765c1ca80 2024-04-21T08:49:18.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.462+0000 7f0770068740 10 client.26009 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:18.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.462+0000 7f0770068740 10 client.26009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:18.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.462+0000 7f0770068740 20 client.26009 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:18.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.462+0000 7f0770068740 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557765c928a0 0x557765c94cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.462+0000 7f0770068740 4 client.26009 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:49:18.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.462+0000 7f0770068740 1 -- 172.21.15.82:0/1751340435 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f075c020070 con 0x557765c928a0 2024-04-21T08:49:18.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.462+0000 7f0766ffd640 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557765c928a0 0x557765c94cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.463+0000 7f0766ffd640 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557765c928a0 0x557765c94cc0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a0c18d85839fc0 server_cookie=5c800e9dc4807927 in_seq=0 out_seq=0 2024-04-21T08:49:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.463+0000 7f074aff5640 10 client.26009 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.463+0000 7f074aff5640 1 -- 172.21.15.82:0/1751340435 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f075c020070 con 0x557765c928a0 2024-04-21T08:49:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.463+0000 7f074aff5640 10 client.26009 handle_command_reply: tid=0 2024-04-21T08:49:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.463+0000 7f0770068740 1 client.26009 shutdown 2024-04-21T08:49:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.463+0000 7f0770068740 1 -- 172.21.15.82:0/1751340435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 msgr2=0x557765c1fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.463+0000 7f0770068740 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c1fe00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0760002230 tx=0x7f07600109f0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.464+0000 7f0770068740 1 -- 172.21.15.82:0/1751340435 shutdown_connections 2024-04-21T08:49:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.464+0000 7f0770068740 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557765c928a0 0x557765c94cc0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.464+0000 7f074aff5640 0 client.26009 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.464+0000 7f0770068740 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557765c1ca80 0x557765c1fe00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.464+0000 7f0770068740 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557765c13ea0 0x557765c25190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.464+0000 7f0770068740 1 --2- 172.21.15.82:0/1751340435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557765c13580 0x557765c24c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.464+0000 7f0770068740 1 -- 172.21.15.82:0/1751340435 >> 172.21.15.82:0/1751340435 conn(0x557765c0dcb0 msgr2=0x557765c18360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.464+0000 7f0770068740 1 -- 172.21.15.82:0/1751340435 shutdown_connections 2024-04-21T08:49:18.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.464+0000 7f0770068740 1 -- 172.21.15.82:0/1751340435 wait complete. 2024-04-21T08:49:18.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.464+0000 7f0770068740 20 client.26009 trim_cache size 0 max 16384 2024-04-21T08:49:18.469 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:49:18.469 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:49:18.469 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:49:18.469 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:49:18.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.465+0000 7f076d46d640 1 -- 172.21.15.82:0/349476502 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f07500a3da0 msgr2=0x7f07500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.465+0000 7f076d46d640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f07500a3da0 0x7f07500a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0760027000 tx=0x7f0760028000 comp rx=0 tx=0).stop 2024-04-21T08:49:18.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.465+0000 7f076d46d640 1 -- 172.21.15.82:0/349476502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0768101080 msgr2=0x7f0768197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.465+0000 7f076d46d640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0768101080 0x7f0768197640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0758000c00 tx=0x7f075802c6c0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.465+0000 7f076d46d640 1 -- 172.21.15.82:0/349476502 shutdown_connections 2024-04-21T08:49:18.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.465+0000 7f076d46d640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f07500a3da0 0x7f07500a6260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.465+0000 7f076d46d640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f076819bf00 0x7f076819c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.465+0000 7f076d46d640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0768106150 0x7f0768197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.465+0000 7f076d46d640 1 --2- 172.21.15.82:0/349476502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0768101080 0x7f0768197640 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.465+0000 7f076d46d640 1 -- 172.21.15.82:0/349476502 >> 172.21.15.82:0/349476502 conn(0x7f0768078000 msgr2=0x7f07680ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.466+0000 7f076d46d640 1 -- 172.21.15.82:0/349476502 shutdown_connections 2024-04-21T08:49:18.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.466+0000 7f076d46d640 1 -- 172.21.15.82:0/349476502 wait complete. 2024-04-21T08:49:18.475 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:18 smithi134 ceph-mon[31772]: pgmap v71: 129 pgs: 129 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-04-21T08:49:18.483 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:49:18.484 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:49:18.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:18 smithi106 ceph-mon[31072]: pgmap v71: 129 pgs: 129 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-04-21T08:49:18.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.646+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/16935162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc004830 msgr2=0x7fa6dc094a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.646+0000 7fa6ea9e1640 1 --2- 172.21.15.82:0/16935162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc004830 0x7fa6dc094a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009930 tx=0x7fa6d402bc60 comp rx=0 tx=0).stop 2024-04-21T08:49:18.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.650+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/16935162 shutdown_connections 2024-04-21T08:49:18.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.650+0000 7fa6ea9e1640 1 --2- 172.21.15.82:0/16935162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6dc094190 0x7fa6dc0a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.650+0000 7fa6ea9e1640 1 --2- 172.21.15.82:0/16935162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc004830 0x7fa6dc094a20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.650+0000 7fa6ea9e1640 1 --2- 172.21.15.82:0/16935162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc098f00 0x7fa6dc0992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.650+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/16935162 >> 172.21.15.82:0/16935162 conn(0x7fa6dc009a80 msgr2=0x7fa6dc009e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.661+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/16935162 shutdown_connections 2024-04-21T08:49:18.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.661+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/16935162 wait complete. 2024-04-21T08:49:18.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.662+0000 7fa6ea9e1640 1 Processor -- start 2024-04-21T08:49:18.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.663+0000 7fa6ea9e1640 1 -- start start 2024-04-21T08:49:18.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.663+0000 7fa6ea9e1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6dc004830 0x7fa6dc09ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.663+0000 7fa6ea9e1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc094190 0x7fa6dc0a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.663+0000 7fa6ea9e1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc098f00 0x7fa6dc0a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.663+0000 7fa6ea9e1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0b5f40 con 0x7fa6dc004830 2024-04-21T08:49:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.663+0000 7fa6ea9e1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0b5dc0 con 0x7fa6dc094190 2024-04-21T08:49:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.663+0000 7fa6ea9e1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0b60c0 con 0x7fa6dc098f00 2024-04-21T08:49:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.666+0000 7fa6ea1e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc098f00 0x7fa6dc0a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.666+0000 7fa6ea1e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc098f00 0x7fa6dc0a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:49706/0 (socket says 172.21.15.82:49706) 2024-04-21T08:49:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.666+0000 7fa6ea1e0640 1 -- 172.21.15.82:0/313951752 learned_addr learned my addr 172.21.15.82:0/313951752 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.666+0000 7fa6e91de640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc094190 0x7fa6dc0a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.666+0000 7fa6e99df640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6dc004830 0x7fa6dc09ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.666+0000 7fa6ea1e0640 1 -- 172.21.15.82:0/313951752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc094190 msgr2=0x7fa6dc0a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.666+0000 7fa6ea1e0640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc094190 0x7fa6dc0a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.666+0000 7fa6ea1e0640 1 -- 172.21.15.82:0/313951752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6dc004830 msgr2=0x7fa6dc09ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.666+0000 7fa6ea1e0640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6dc004830 0x7fa6dc09ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.666+0000 7fa6ea1e0640 1 -- 172.21.15.82:0/313951752 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc14ba50 con 0x7fa6dc098f00 2024-04-21T08:49:18.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.666+0000 7fa6e99df640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6dc004830 0x7fa6dc09ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:18.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.666+0000 7fa6ea1e0640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc098f00 0x7fa6dc0a0a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000c970 tx=0x7fa6e000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.670+0000 7fa6daffd640 1 -- 172.21.15.82:0/313951752 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0007bb0 con 0x7fa6dc098f00 2024-04-21T08:49:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.671+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/313951752 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc14bd40 con 0x7fa6dc098f00 2024-04-21T08:49:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.671+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/313951752 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc14c260 con 0x7fa6dc098f00 2024-04-21T08:49:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.671+0000 7fa6daffd640 1 -- 172.21.15.82:0/313951752 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6e0007d50 con 0x7fa6dc098f00 2024-04-21T08:49:18.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.671+0000 7fa6daffd640 1 -- 172.21.15.82:0/313951752 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0018770 con 0x7fa6dc098f00 2024-04-21T08:49:18.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.672+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/313951752 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6dc005510 con 0x7fa6dc098f00 2024-04-21T08:49:18.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.675+0000 7fa6daffd640 1 -- 172.21.15.82:0/313951752 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa6e00213f0 con 0x7fa6dc098f00 2024-04-21T08:49:18.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.676+0000 7fa6daffd640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa6d00a3f30 0x7fa6d00a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.676+0000 7fa6daffd640 1 -- 172.21.15.82:0/313951752 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa6e00d0cd0 con 0x7fa6dc098f00 2024-04-21T08:49:18.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.677+0000 7fa6e99df640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa6d00a3f30 0x7fa6d00a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.677+0000 7fa6e99df640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa6d00a3f30 0x7fa6d00a63f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4065470 tx=0x7fa6e406d040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.680+0000 7fa6daffd640 1 -- 172.21.15.82:0/313951752 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6e00d6020 con 0x7fa6dc098f00 2024-04-21T08:49:18.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.726+0000 7f0b0146d640 1 -- 172.21.15.82:0/653927918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0afc110220 con 0x7f0afc108f50 2024-04-21T08:49:18.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.727+0000 7f0ae3fff640 1 -- 172.21.15.82:0/653927918 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11952 (secure 0 0 0) 0x7f0af40bedb0 con 0x7f0afc108f50 2024-04-21T08:49:18.728 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:49:18.729 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:48:22.138089+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:49:18.729 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 32 2024-04-21T08:49:18.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.730+0000 7f0b0146d640 1 -- 172.21.15.82:0/653927918 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0ae40a3f20 msgr2=0x7f0ae40a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.730+0000 7f0b0146d640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0ae40a3f20 0x7f0ae40a63e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0afc10c130 tx=0x7f0aec0090e0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.730+0000 7f0b0146d640 1 -- 172.21.15.82:0/653927918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0afc108f50 msgr2=0x7f0afc10c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.730+0000 7f0b0146d640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0afc108f50 0x7f0afc10c5c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0af402c730 tx=0x7f0af4035ce0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.730+0000 7f0b0146d640 1 -- 172.21.15.82:0/653927918 shutdown_connections 2024-04-21T08:49:18.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.730+0000 7f0b0146d640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0ae40a3f20 0x7f0ae40a63e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.730+0000 7f0b0146d640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0afc1150a0 0x7f0afc10b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.730+0000 7f0b0146d640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0afc109870 0x7f0afc10a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.730+0000 7f0b0146d640 1 --2- 172.21.15.82:0/653927918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0afc108f50 0x7f0afc10c5c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.730+0000 7f0b0146d640 1 -- 172.21.15.82:0/653927918 >> 172.21.15.82:0/653927918 conn(0x7f0afc06ce90 msgr2=0x7f0afc0709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.730+0000 7f0b0146d640 1 -- 172.21.15.82:0/653927918 shutdown_connections 2024-04-21T08:49:18.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.730+0000 7f0b0146d640 1 -- 172.21.15.82:0/653927918 wait complete. 2024-04-21T08:49:18.743 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 30 --expiration 90.0 --await' 2024-04-21T08:49:18.743 DEBUG:teuthology.orchestra.run.smithi082:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db / --timeout 30 --expiration 90.0 --await 2024-04-21T08:49:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.867+0000 7f5553fca640 1 -- 172.21.15.82:0/1194298546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f554c106170 msgr2=0x7f554c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.867+0000 7f5553fca640 1 --2- 172.21.15.82:0/1194298546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f554c106170 0x7f554c10dd40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f553c009900 tx=0x7f553c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.867+0000 7f5553fca640 1 -- 172.21.15.82:0/1194298546 shutdown_connections 2024-04-21T08:49:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.867+0000 7f5553fca640 1 --2- 172.21.15.82:0/1194298546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f554c106170 0x7f554c10dd40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.867+0000 7f5553fca640 1 --2- 172.21.15.82:0/1194298546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f554c101a50 0x7f554c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.867+0000 7f5553fca640 1 --2- 172.21.15.82:0/1194298546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f554c1010a0 0x7f554c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.867+0000 7f5553fca640 1 -- 172.21.15.82:0/1194298546 >> 172.21.15.82:0/1194298546 conn(0x7f554c077f50 msgr2=0x7f554c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.867+0000 7f5553fca640 1 -- 172.21.15.82:0/1194298546 shutdown_connections 2024-04-21T08:49:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.867+0000 7f5553fca640 1 -- 172.21.15.82:0/1194298546 wait complete. 2024-04-21T08:49:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.867+0000 7f5553fca640 1 Processor -- start 2024-04-21T08:49:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5553fca640 1 -- start start 2024-04-21T08:49:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5553fca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f554c1010a0 0x7f554c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5553fca640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f554c106170 0x7f554c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5553fca640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f554c19ceb0 0x7f554c19d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5553fca640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f554c1110c0 con 0x7f554c106170 2024-04-21T08:49:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5553fca640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f554c110f40 con 0x7f554c19ceb0 2024-04-21T08:49:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5553fca640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f554c111240 con 0x7f554c1010a0 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5551565640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f554c106170 0x7f554c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5551565640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f554c106170 0x7f554c198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49844/0 (socket says 172.21.15.82:49844) 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5551565640 1 -- 172.21.15.82:0/2295477921 learned_addr learned my addr 172.21.15.82:0/2295477921 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5552567640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f554c19ceb0 0x7f554c19d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5551d66640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f554c1010a0 0x7f554c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5551565640 1 -- 172.21.15.82:0/2295477921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f554c1010a0 msgr2=0x7f554c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5551565640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f554c1010a0 0x7f554c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5551565640 1 -- 172.21.15.82:0/2295477921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f554c19ceb0 msgr2=0x7f554c19d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5551565640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f554c19ceb0 0x7f554c19d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5551565640 1 -- 172.21.15.82:0/2295477921 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c19d9a0 con 0x7f554c106170 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5551d66640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f554c1010a0 0x7f554c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.868+0000 7f5551565640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f554c106170 0x7f554c198b30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f554800e9c0 tx=0x7f554800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.869+0000 7f5542ffd640 1 -- 172.21.15.82:0/2295477921 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554800cd80 con 0x7f554c106170 2024-04-21T08:49:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.869+0000 7f5542ffd640 1 -- 172.21.15.82:0/2295477921 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55480054d0 con 0x7f554c106170 2024-04-21T08:49:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.869+0000 7f5542ffd640 1 -- 172.21.15.82:0/2295477921 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548010430 con 0x7f554c106170 2024-04-21T08:49:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.869+0000 7f5553fca640 1 -- 172.21.15.82:0/2295477921 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554c19dc90 con 0x7f554c106170 2024-04-21T08:49:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.869+0000 7f5553fca640 1 -- 172.21.15.82:0/2295477921 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554c1a5430 con 0x7f554c106170 2024-04-21T08:49:18.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.872+0000 7f5556bb7740 1 Processor -- start 2024-04-21T08:49:18.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.872+0000 7f5556bb7740 1 -- start start 2024-04-21T08:49:18.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.872+0000 7f5556bb7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bb9c0 0x5601702bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.872+0000 7f5556bb7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bc2e0 0x5601702c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.872+0000 7f5556bb7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702c4ec0 0x5601702c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.872+0000 7f5556bb7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56017021c0b0 con 0x5601702bc2e0 2024-04-21T08:49:18.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.872+0000 7f5556bb7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5601701cd680 con 0x5601702bb9c0 2024-04-21T08:49:18.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.872+0000 7f5556bb7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5601701d9c60 con 0x5601702c4ec0 2024-04-21T08:49:18.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.872+0000 7f5552567640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bb9c0 0x5601702bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.872+0000 7f5552567640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bb9c0 0x5601702bbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:60402/0 (socket says 172.21.15.82:60402) 2024-04-21T08:49:18.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.872+0000 7f5552567640 1 -- 172.21.15.82:0/349818402 learned_addr learned my addr 172.21.15.82:0/349818402 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:18.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.872+0000 7f5551d66640 1 --2- 172.21.15.82:0/349818402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bc2e0 0x5601702c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.873+0000 7f5542ffd640 1 -- 172.21.15.82:0/2295477921 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5548004fd0 con 0x7f554c106170 2024-04-21T08:49:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.873+0000 7f5552567640 1 -- 172.21.15.82:0/349818402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702c4ec0 msgr2=0x5601702c72b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:49:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.873+0000 7f5552567640 1 --2- 172.21.15.82:0/349818402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702c4ec0 0x5601702c72b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.873+0000 7f5552567640 1 -- 172.21.15.82:0/349818402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bc2e0 msgr2=0x5601702c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.873+0000 7f5552567640 1 --2- 172.21.15.82:0/349818402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bc2e0 0x5601702c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.873+0000 7f5552567640 1 -- 172.21.15.82:0/349818402 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56016fffec90 con 0x5601702bb9c0 2024-04-21T08:49:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.873+0000 7f5551d66640 1 --2- 172.21.15.82:0/349818402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bc2e0 0x5601702c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.873+0000 7f5552567640 1 --2- 172.21.15.82:0/349818402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bb9c0 0x5601702bbda0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f553c009900 tx=0x7f553c02c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91caa24a55a16a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.873+0000 7f5537fff640 1 -- 172.21.15.82:0/349818402 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c045020 con 0x5601702bb9c0 2024-04-21T08:49:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.873+0000 7f5537fff640 1 -- 172.21.15.82:0/349818402 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f553c02c7e0 con 0x5601702bb9c0 2024-04-21T08:49:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.873+0000 7f5537fff640 1 -- 172.21.15.82:0/349818402 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c0046c0 con 0x5601702bb9c0 2024-04-21T08:49:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.873+0000 7f5542ffd640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f55380a3da0 0x7f55380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.874+0000 7f5551d66640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f55380a3da0 0x7f55380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.874+0000 7f5542ffd640 1 -- 172.21.15.82:0/2295477921 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f55480d0f30 con 0x7f554c106170 2024-04-21T08:49:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.874+0000 7f5556bb7740 1 -- 172.21.15.82:0/349818402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bb9c0 msgr2=0x5601702bbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.874+0000 7f5551d66640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f55380a3da0 0x7f55380a6260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5601702bb490 tx=0x7f5544008720 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.874+0000 7f5556bb7740 1 --2- 172.21.15.82:0/349818402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bb9c0 0x5601702bbda0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f553c009900 tx=0x7f553c02c5d0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.874+0000 7f5556bb7740 1 -- 172.21.15.82:0/349818402 shutdown_connections 2024-04-21T08:49:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.874+0000 7f5556bb7740 1 --2- 172.21.15.82:0/349818402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702c4ec0 0x5601702c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.874+0000 7f5556bb7740 1 --2- 172.21.15.82:0/349818402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bc2e0 0x5601702c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.874+0000 7f5556bb7740 1 --2- 172.21.15.82:0/349818402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bb9c0 0x5601702bbda0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.874+0000 7f5556bb7740 1 -- 172.21.15.82:0/349818402 >> 172.21.15.82:0/349818402 conn(0x5601702b60f0 msgr2=0x5601702b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.874+0000 7f5556bb7740 1 -- 172.21.15.82:0/349818402 shutdown_connections 2024-04-21T08:49:18.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.874+0000 7f5556bb7740 1 -- 172.21.15.82:0/349818402 wait complete. 2024-04-21T08:49:18.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5556bb7740 1 -- start start 2024-04-21T08:49:18.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5556bb7740 1 Processor -- start 2024-04-21T08:49:18.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5556bb7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bb9c0 0x5601702c2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5556bb7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bc2e0 0x5601702c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5556bb7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702c4ec0 0x5601702c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5556bb7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56017021f400 con 0x5601702bb9c0 2024-04-21T08:49:18.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5556bb7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5601701db840 con 0x5601702bc2e0 2024-04-21T08:49:18.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5556bb7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560170224720 con 0x5601702c4ec0 2024-04-21T08:49:18.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5551d66640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bb9c0 0x5601702c2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5556bb7740 10 client.0 _ll_register_callbacks cb 0x56017022cea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:18.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5551d66640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bb9c0 0x5601702c2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49854/0 (socket says 172.21.15.82:49854) 2024-04-21T08:49:18.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5551d66640 1 -- 172.21.15.82:0/699480478 learned_addr learned my addr 172.21.15.82:0/699480478 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:18.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.875+0000 7f5551565640 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bc2e0 0x5601702c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f5552567640 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702c4ec0 0x5601702c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f5551d66640 1 -- 172.21.15.82:0/699480478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702c4ec0 msgr2=0x5601702c2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f5551d66640 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702c4ec0 0x5601702c2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f5551d66640 1 -- 172.21.15.82:0/699480478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bc2e0 msgr2=0x5601702c2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f5551d66640 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bc2e0 0x5601702c2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f5551d66640 1 -- 172.21.15.82:0/699480478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017005f420 con 0x5601702bb9c0 2024-04-21T08:49:18.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f5552567640 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702c4ec0 0x5601702c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:18.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f5551565640 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bc2e0 0x5601702c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:18.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f5551d66640 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bb9c0 0x5601702c2020 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5544010460 tx=0x7f5544004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1770d1a781e5aa9d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f55357fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f55357fa640 1 -- 172.21.15.82:0/699480478 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554401bcc0 con 0x5601702bb9c0 2024-04-21T08:49:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f55357fa640 1 -- 172.21.15.82:0/699480478 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f5544026430 con 0x5601702bb9c0 2024-04-21T08:49:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f5556bb7740 1 -- 172.21.15.82:0/699480478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f553c02c7e0 con 0x5601702bb9c0 2024-04-21T08:49:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f55357fa640 1 -- 172.21.15.82:0/699480478 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f554402f050 con 0x5601702bb9c0 2024-04-21T08:49:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f5556bb7740 10 client.16674 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f5556bb7740 1 -- 172.21.15.82:0/699480478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56016fef2580 con 0x5601702bb9c0 2024-04-21T08:49:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.876+0000 7f55357fa640 1 -- 172.21.15.82:0/699480478 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f553c02c7e0 con 0x5601702bb9c0 2024-04-21T08:49:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.877+0000 7f55357fa640 1 -- 172.21.15.82:0/699480478 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f554401c710 con 0x5601702bb9c0 2024-04-21T08:49:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.877+0000 7f5556bb7740 10 client.16674 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.877+0000 7f5556bb7740 10 client.16674 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-21T08:49:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.877+0000 7f5556bb7740 20 client.16674 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.877+0000 7f5556bb7740 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5601702d4d60 0x5601702d7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.877+0000 7f5556bb7740 4 client.16674 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:18.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.877+0000 7f5556bb7740 1 -- 172.21.15.82:0/699480478 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56017022db10 con 0x5601702d4d60 2024-04-21T08:49:18.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.877+0000 7f5551565640 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5601702d4d60 0x5601702d7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.877+0000 7f5551565640 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5601702d4d60 0x5601702d7150 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b1a2cd8ba985fd server_cookie=aca4ac912bead331 in_seq=0 out_seq=0 2024-04-21T08:49:18.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.877+0000 7f55357fa640 10 client.16674 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:18.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.878+0000 7f55357fa640 1 -- 172.21.15.82:0/699480478 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56017022db10 con 0x5601702d4d60 2024-04-21T08:49:18.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.878+0000 7f55357fa640 10 client.16674 handle_command_reply: tid=0 2024-04-21T08:49:18.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.878+0000 7f5556bb7740 1 client.16674 shutdown 2024-04-21T08:49:18.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.878+0000 7f5556bb7740 1 -- 172.21.15.82:0/699480478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bb9c0 msgr2=0x5601702c2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.878+0000 7f5556bb7740 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bb9c0 0x5601702c2020 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5544010460 tx=0x7f5544004a40 comp rx=0 tx=0).stop 2024-04-21T08:49:18.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.879+0000 7f5556bb7740 1 -- 172.21.15.82:0/699480478 shutdown_connections 2024-04-21T08:49:18.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.879+0000 7f5556bb7740 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5601702d4d60 0x5601702d7150 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.879+0000 7f5556bb7740 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702c4ec0 0x5601702c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.879+0000 7f5556bb7740 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bc2e0 0x5601702c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.879+0000 7f55357fa640 0 client.16674 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.879+0000 7f5556bb7740 1 --2- 172.21.15.82:0/699480478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702bb9c0 0x5601702c2020 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.879+0000 7f5556bb7740 1 -- 172.21.15.82:0/699480478 >> 172.21.15.82:0/699480478 conn(0x5601702b60f0 msgr2=0x5601702b7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.879+0000 7f5556bb7740 1 -- 172.21.15.82:0/699480478 shutdown_connections 2024-04-21T08:49:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.879+0000 7f5556bb7740 1 -- 172.21.15.82:0/699480478 wait complete. 2024-04-21T08:49:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.879+0000 7f5556bb7740 20 client.16674 trim_cache size 0 max 16384 2024-04-21T08:49:18.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5556bb7740 1 Processor -- start 2024-04-21T08:49:18.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5556bb7740 1 -- start start 2024-04-21T08:49:18.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5556bb7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702bb9c0 0x5601702c4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5556bb7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bc2e0 0x5601702c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5556bb7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 0x5601702dca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5556bb7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f554c1b0e80 con 0x5601702c4ec0 2024-04-21T08:49:18.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5556bb7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f554c1af920 con 0x5601702bc2e0 2024-04-21T08:49:18.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5556bb7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f554c1b00b0 con 0x5601702bb9c0 2024-04-21T08:49:18.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5552567640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 0x5601702dca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5552567640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 0x5601702dca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49858/0 (socket says 172.21.15.82:49858) 2024-04-21T08:49:18.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5552567640 1 -- 172.21.15.82:0/3038032069 learned_addr learned my addr 172.21.15.82:0/3038032069 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:18.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5551565640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bc2e0 0x5601702c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5551d66640 1 --2- 172.21.15.82:0/3038032069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702bb9c0 0x5601702c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5552567640 1 -- 172.21.15.82:0/3038032069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702bb9c0 msgr2=0x5601702c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5552567640 1 --2- 172.21.15.82:0/3038032069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702bb9c0 0x5601702c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5552567640 1 -- 172.21.15.82:0/3038032069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bc2e0 msgr2=0x5601702c28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5552567640 1 --2- 172.21.15.82:0/3038032069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bc2e0 0x5601702c28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.898+0000 7f5552567640 1 -- 172.21.15.82:0/3038032069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601702bc110 con 0x5601702c4ec0 2024-04-21T08:49:18.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.899+0000 7f5552567640 1 --2- 172.21.15.82:0/3038032069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 0x5601702dca20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f553c005eb0 tx=0x7f553c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd80cc524fccaaa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.899+0000 7f55377fe640 1 -- 172.21.15.82:0/3038032069 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c045030 con 0x5601702c4ec0 2024-04-21T08:49:18.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.899+0000 7f55377fe640 1 -- 172.21.15.82:0/3038032069 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f553c04f070 con 0x5601702c4ec0 2024-04-21T08:49:18.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.899+0000 7f55377fe640 1 -- 172.21.15.82:0/3038032069 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c0046c0 con 0x5601702c4ec0 2024-04-21T08:49:18.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 -- 172.21.15.82:0/3038032069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 msgr2=0x5601702dca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 --2- 172.21.15.82:0/3038032069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 0x5601702dca20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f553c005eb0 tx=0x7f553c000bc0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 -- 172.21.15.82:0/3038032069 shutdown_connections 2024-04-21T08:49:18.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 --2- 172.21.15.82:0/3038032069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 0x5601702dca20 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 --2- 172.21.15.82:0/3038032069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bc2e0 0x5601702c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 --2- 172.21.15.82:0/3038032069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702bb9c0 0x5601702c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 -- 172.21.15.82:0/3038032069 >> 172.21.15.82:0/3038032069 conn(0x5601702b60f0 msgr2=0x5601702b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 -- 172.21.15.82:0/3038032069 shutdown_connections 2024-04-21T08:49:18.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 -- 172.21.15.82:0/3038032069 wait complete. 2024-04-21T08:49:18.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 -- start start 2024-04-21T08:49:18.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 Processor -- start 2024-04-21T08:49:18.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bb9c0 0x5601702c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702bc2e0 0x5601702c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 0x56017033a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f554c1ad230 con 0x5601702c4ec0 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f554c1b0780 con 0x5601702bb9c0 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.900+0000 7f5556bb7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f554c1ad8a0 con 0x5601702bc2e0 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5556bb7740 10 client.0 _ll_register_callbacks cb 0x7f5544008410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5552567640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 0x56017033a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5551d66640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bb9c0 0x5601702c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5552567640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 0x56017033a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49872/0 (socket says 172.21.15.82:49872) 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5552567640 1 -- 172.21.15.82:0/2150723320 learned_addr learned my addr 172.21.15.82:0/2150723320 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5551565640 1 --2- 172.21.15.82:0/2150723320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702bc2e0 0x5601702c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5552567640 1 -- 172.21.15.82:0/2150723320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702bc2e0 msgr2=0x5601702c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5552567640 1 --2- 172.21.15.82:0/2150723320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702bc2e0 0x5601702c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5552567640 1 -- 172.21.15.82:0/2150723320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bb9c0 msgr2=0x5601702c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:18.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5552567640 1 --2- 172.21.15.82:0/2150723320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bb9c0 0x5601702c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5552567640 1 -- 172.21.15.82:0/2150723320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017033a880 con 0x5601702c4ec0 2024-04-21T08:49:18.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5552567640 1 --2- 172.21.15.82:0/2150723320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 0x56017033a290 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f553c02c170 tx=0x7f553c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c1940e0cbbb5e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:18.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f55357fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:18.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f55357fa640 1 -- 172.21.15.82:0/2150723320 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c059020 con 0x5601702c4ec0 2024-04-21T08:49:18.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f55357fa640 1 -- 172.21.15.82:0/2150723320 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f553c0035f0 con 0x5601702c4ec0 2024-04-21T08:49:18.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f5556bb7740 1 -- 172.21.15.82:0/2150723320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f553c04f070 con 0x5601702c4ec0 2024-04-21T08:49:18.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.901+0000 7f55357fa640 1 -- 172.21.15.82:0/2150723320 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f553c045040 con 0x5601702c4ec0 2024-04-21T08:49:18.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.902+0000 7f5556bb7740 10 client.16680 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:18.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.902+0000 7f5556bb7740 1 -- 172.21.15.82:0/2150723320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56017033bb40 con 0x5601702c4ec0 2024-04-21T08:49:18.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.902+0000 7f55357fa640 1 -- 172.21.15.82:0/2150723320 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f553c04f070 con 0x5601702c4ec0 2024-04-21T08:49:18.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.902+0000 7f55357fa640 1 -- 172.21.15.82:0/2150723320 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f553c041850 con 0x5601702c4ec0 2024-04-21T08:49:18.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.902+0000 7f5556bb7740 10 client.16680 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:18.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.902+0000 7f5556bb7740 10 client.16680 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-21T08:49:18.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.902+0000 7f5556bb7740 20 client.16680 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:18.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.902+0000 7f5556bb7740 1 --2- 172.21.15.82:0/2150723320 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56017033bee0 0x56017033e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:18.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.902+0000 7f5556bb7740 4 client.16680 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 90.0, "await": true}] 2024-04-21T08:49:18.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.902+0000 7f5556bb7740 1 -- 172.21.15.82:0/2150723320 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 90.0, "await": true}) -- 0x560170227980 con 0x56017033bee0 2024-04-21T08:49:18.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.902+0000 7f5551d66640 1 --2- 172.21.15.82:0/2150723320 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56017033bee0 0x56017033e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:18.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.903+0000 7f55357fa640 10 client.16680 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:18.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.903+0000 7f5551d66640 1 --2- 172.21.15.82:0/2150723320 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56017033bee0 0x56017033e2d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1beca7202c19e77 server_cookie=3c80a44cd143a69e in_seq=0 out_seq=0 2024-04-21T08:49:18.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.946+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/313951752 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6dc0a1500 con 0x7fa6dc098f00 2024-04-21T08:49:18.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.947+0000 7fa6daffd640 1 -- 172.21.15.82:0/313951752 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11952 (secure 0 0 0) 0x7fa6e0014030 con 0x7fa6dc098f00 2024-04-21T08:49:18.948 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:49:18.948 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:48:22.138089+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:49:18.948 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 32 2024-04-21T08:49:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.950+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/313951752 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa6d00a3f30 msgr2=0x7fa6d00a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.950+0000 7fa6ea9e1640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa6d00a3f30 0x7fa6d00a63f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4065470 tx=0x7fa6e406d040 comp rx=0 tx=0).stop 2024-04-21T08:49:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.950+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/313951752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc098f00 msgr2=0x7fa6dc0a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.950+0000 7fa6ea9e1640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc098f00 0x7fa6dc0a0a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000c970 tx=0x7fa6e000ce30 comp rx=0 tx=0).stop 2024-04-21T08:49:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.950+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/313951752 shutdown_connections 2024-04-21T08:49:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.950+0000 7fa6ea9e1640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6dc098f00 0x7fa6dc0a0a50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.950+0000 7fa6ea9e1640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa6dc094190 0x7fa6dc0a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.950+0000 7fa6ea9e1640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa6d00a3f30 0x7fa6d00a63f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.950+0000 7fa6ea9e1640 1 --2- 172.21.15.82:0/313951752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6dc004830 0x7fa6dc09ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.950+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/313951752 >> 172.21.15.82:0/313951752 conn(0x7fa6dc009a80 msgr2=0x7fa6dc091940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.950+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/313951752 shutdown_connections 2024-04-21T08:49:18.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:18.950+0000 7fa6ea9e1640 1 -- 172.21.15.82:0/313951752 wait complete. 2024-04-21T08:49:18.963 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:49:19.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.082+0000 7f51951d5640 1 -- 172.21.15.82:0/2714979490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51901021c0 msgr2=0x7f5190105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.082+0000 7f51951d5640 1 --2- 172.21.15.82:0/2714979490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51901021c0 0x7f5190105a70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5184009900 tx=0x7f518402bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:19.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.082+0000 7f51951d5640 1 -- 172.21.15.82:0/2714979490 shutdown_connections 2024-04-21T08:49:19.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.082+0000 7f51951d5640 1 --2- 172.21.15.82:0/2714979490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5190106150 0x7f519010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.082+0000 7f51951d5640 1 --2- 172.21.15.82:0/2714979490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51901021c0 0x7f5190105a70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.082+0000 7f51951d5640 1 --2- 172.21.15.82:0/2714979490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5190101810 0x7f5190101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.083+0000 7f51951d5640 1 -- 172.21.15.82:0/2714979490 >> 172.21.15.82:0/2714979490 conn(0x7f5190077410 msgr2=0x7f51900fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.083+0000 7f51951d5640 1 -- 172.21.15.82:0/2714979490 shutdown_connections 2024-04-21T08:49:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.083+0000 7f51951d5640 1 -- 172.21.15.82:0/2714979490 wait complete. 2024-04-21T08:49:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.083+0000 7f51951d5640 1 Processor -- start 2024-04-21T08:49:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f51951d5640 1 -- start start 2024-04-21T08:49:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f51951d5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5190101810 0x7f5190197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f51951d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5190106150 0x7f5190197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f51951d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f519019bf20 0x7f519019c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f51951d5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f51901110e0 con 0x7f5190101810 2024-04-21T08:49:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f518ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5190101810 0x7f5190197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f51951d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5190110f60 con 0x7f5190106150 2024-04-21T08:49:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f51951d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5190111260 con 0x7f519019bf20 2024-04-21T08:49:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f518ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5190101810 0x7f5190197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49888/0 (socket says 172.21.15.82:49888) 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f518ed9d640 1 -- 172.21.15.82:0/3829277625 learned_addr learned my addr 172.21.15.82:0/3829277625 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f518e59c640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5190106150 0x7f5190197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f518f59e640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f519019bf20 0x7f519019c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f518ed9d640 1 -- 172.21.15.82:0/3829277625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f519019bf20 msgr2=0x7f519019c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f518ed9d640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f519019bf20 0x7f519019c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f518ed9d640 1 -- 172.21.15.82:0/3829277625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5190106150 msgr2=0x7f5190197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f518ed9d640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5190106150 0x7f5190197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.084+0000 7f518ed9d640 1 -- 172.21.15.82:0/3829277625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519019ca10 con 0x7f5190101810 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.085+0000 7f518ed9d640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5190101810 0x7f5190197660 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f518000e9f0 tx=0x7f518000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.085+0000 7f5177fff640 1 -- 172.21.15.82:0/3829277625 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518000cd80 con 0x7f5190101810 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.085+0000 7f51951d5640 1 -- 172.21.15.82:0/3829277625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519019cd00 con 0x7f5190101810 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.085+0000 7f51951d5640 1 -- 172.21.15.82:0/3829277625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5190100780 con 0x7f5190101810 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.085+0000 7f5177fff640 1 -- 172.21.15.82:0/3829277625 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51800054d0 con 0x7f5190101810 2024-04-21T08:49:19.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.085+0000 7f5177fff640 1 -- 172.21.15.82:0/3829277625 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180010690 con 0x7f5190101810 2024-04-21T08:49:19.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.087+0000 7f5177fff640 1 -- 172.21.15.82:0/3829277625 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f51800046c0 con 0x7f5190101810 2024-04-21T08:49:19.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.087+0000 7f5197dca740 1 Processor -- start 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f5197dca740 1 -- start start 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f5177fff640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f51780a3da0 0x7f51780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f5197dca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 0x55d4da6038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f5197dca740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da60c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f5197dca740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da60ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f5197dca740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d4da5681f0 con 0x55d4da603e80 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f5197dca740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d4da5054b0 con 0x55d4da60ca00 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f5197dca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4da5641b0 con 0x55d4da6034d0 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f518e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da60c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f518e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da60c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49896/0 (socket says 172.21.15.82:49896) 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f518f59e640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f51780a3da0 0x7f51780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f518e59c640 1 -- 172.21.15.82:0/2245727687 learned_addr learned my addr 172.21.15.82:0/2245727687 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f518ed9d640 1 --2- 172.21.15.82:0/2245727687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 0x55d4da6038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f5177fff640 1 -- 172.21.15.82:0/3829277625 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f51800d0f10 con 0x7f5190101810 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f518f59e640 1 --2- 172.21.15.82:0/2245727687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da60ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f518e59c640 1 -- 172.21.15.82:0/2245727687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 msgr2=0x55d4da6038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f518e59c640 1 --2- 172.21.15.82:0/2245727687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 0x55d4da6038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f518e59c640 1 -- 172.21.15.82:0/2245727687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 msgr2=0x55d4da60ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f518e59c640 1 --2- 172.21.15.82:0/2245727687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da60ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f518e59c640 1 -- 172.21.15.82:0/2245727687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4da4f71b0 con 0x55d4da603e80 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.088+0000 7f518f59e640 1 --2- 172.21.15.82:0/2245727687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da60ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.089+0000 7f518e59c640 1 --2- 172.21.15.82:0/2245727687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da60c430 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5184005ab0 tx=0x7f5184009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392ab4ac60cd3d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.089+0000 7f518f59e640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f51780a3da0 0x7f51780a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5188005fd0 tx=0x7f5188009d10 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.089+0000 7f51757fa640 1 -- 172.21.15.82:0/2245727687 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184045020 con 0x55d4da603e80 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.089+0000 7f51757fa640 1 -- 172.21.15.82:0/2245727687 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f518402c7e0 con 0x55d4da603e80 2024-04-21T08:49:19.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.089+0000 7f51757fa640 1 -- 172.21.15.82:0/2245727687 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184005600 con 0x55d4da603e80 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 -- 172.21.15.82:0/2245727687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 msgr2=0x55d4da60c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 --2- 172.21.15.82:0/2245727687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da60c430 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5184005ab0 tx=0x7f5184009670 comp rx=0 tx=0).stop 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 -- 172.21.15.82:0/2245727687 shutdown_connections 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 --2- 172.21.15.82:0/2245727687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da60ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 --2- 172.21.15.82:0/2245727687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da60c430 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 --2- 172.21.15.82:0/2245727687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 0x55d4da6038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 -- 172.21.15.82:0/2245727687 >> 172.21.15.82:0/2245727687 conn(0x55d4da5fdba0 msgr2=0x55d4da5fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 -- 172.21.15.82:0/2245727687 shutdown_connections 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 -- 172.21.15.82:0/2245727687 wait complete. 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 -- start start 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 Processor -- start 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 0x55d4da60c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da6095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da609b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d4da60f910 con 0x55d4da603e80 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d4da521500 con 0x55d4da60ca00 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4da60fa90 con 0x55d4da6034d0 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f5197dca740 10 client.0 _ll_register_callbacks cb 0x55d4da4efb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.090+0000 7f518e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da6095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f518e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da6095c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49902/0 (socket says 172.21.15.82:49902) 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f518e59c640 1 -- 172.21.15.82:0/448189461 learned_addr learned my addr 172.21.15.82:0/448189461 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f518f59e640 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da609b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f518ed9d640 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 0x55d4da60c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f518e59c640 1 -- 172.21.15.82:0/448189461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 msgr2=0x55d4da60c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f518e59c640 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 0x55d4da60c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f518e59c640 1 -- 172.21.15.82:0/448189461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 msgr2=0x55d4da609b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f518e59c640 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da609b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f518e59c640 1 -- 172.21.15.82:0/448189461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4da27a3f0 con 0x55d4da603e80 2024-04-21T08:49:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f518e59c640 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da6095c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5184036040 tx=0x7f518402cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ce4dc5e7701f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f5172ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f5172ff5640 1 -- 172.21.15.82:0/448189461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51840035f0 con 0x55d4da603e80 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f5172ff5640 1 -- 172.21.15.82:0/448189461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f5184003790 con 0x55d4da603e80 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f5197dca740 1 -- 172.21.15.82:0/448189461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f518402c7e0 con 0x55d4da603e80 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f5172ff5640 1 -- 172.21.15.82:0/448189461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f518405a050 con 0x55d4da603e80 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f5197dca740 10 client.16692 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.091+0000 7f5197dca740 1 -- 172.21.15.82:0/448189461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d4da3447d0 con 0x55d4da603e80 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.092+0000 7f5172ff5640 1 -- 172.21.15.82:0/448189461 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f518402c7e0 con 0x55d4da603e80 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.092+0000 7f5172ff5640 1 -- 172.21.15.82:0/448189461 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f5184041a20 con 0x55d4da603e80 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.092+0000 7f5197dca740 10 client.16692 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.092+0000 7f5197dca740 10 client.16692 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.092+0000 7f5197dca740 20 client.16692 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.092+0000 7f5197dca740 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55d4da61c930 0x55d4da61ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.092+0000 7f5197dca740 4 client.16692 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:19.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.092+0000 7f5197dca740 1 -- 172.21.15.82:0/448189461 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4da507930 con 0x55d4da61c930 2024-04-21T08:49:19.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.092+0000 7f518ed9d640 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55d4da61c930 0x55d4da61ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.092+0000 7f5172ff5640 10 client.16692 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:49:19.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.092+0000 7f518ed9d640 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55d4da61c930 0x55d4da61ed20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1fca2582a1a14321 server_cookie=e42b2d240c983496 in_seq=0 out_seq=0 2024-04-21T08:49:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5172ff5640 1 -- 172.21.15.82:0/448189461 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4da507930 con 0x55d4da61c930 2024-04-21T08:49:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5172ff5640 10 client.16692 handle_command_reply: tid=0 2024-04-21T08:49:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5197dca740 1 client.16692 shutdown 2024-04-21T08:49:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5197dca740 1 -- 172.21.15.82:0/448189461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 msgr2=0x55d4da6095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5197dca740 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da6095c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5184036040 tx=0x7f518402cbc0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5197dca740 1 -- 172.21.15.82:0/448189461 shutdown_connections 2024-04-21T08:49:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5197dca740 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55d4da61c930 0x55d4da61ed20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5197dca740 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da609b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5172ff5640 0 client.16692 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:49:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5197dca740 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da6095c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5197dca740 1 --2- 172.21.15.82:0/448189461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 0x55d4da60c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5197dca740 1 -- 172.21.15.82:0/448189461 >> 172.21.15.82:0/448189461 conn(0x55d4da5fdba0 msgr2=0x55d4da5ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5197dca740 1 -- 172.21.15.82:0/448189461 shutdown_connections 2024-04-21T08:49:19.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5197dca740 1 -- 172.21.15.82:0/448189461 wait complete. 2024-04-21T08:49:19.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.098+0000 7f5197dca740 20 client.16692 trim_cache size 0 max 16384 2024-04-21T08:49:19.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.112+0000 7f5197dca740 1 Processor -- start 2024-04-21T08:49:19.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.112+0000 7f5197dca740 1 -- start start 2024-04-21T08:49:19.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.112+0000 7f5197dca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 0x55d4da618830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.112+0000 7f5197dca740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da5fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.112+0000 7f5197dca740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da60bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.112+0000 7f5197dca740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f517c00ff70 con 0x55d4da603e80 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.112+0000 7f5197dca740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f517c011ba0 con 0x55d4da60ca00 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.112+0000 7f5197dca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f517c010640 con 0x55d4da6034d0 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f518e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da5fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f518e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da5fcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49910/0 (socket says 172.21.15.82:49910) 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f518e59c640 1 -- 172.21.15.82:0/3279787487 learned_addr learned my addr 172.21.15.82:0/3279787487 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f518f59e640 1 --2- 172.21.15.82:0/3279787487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da60bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f518ed9d640 1 --2- 172.21.15.82:0/3279787487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 0x55d4da618830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f518e59c640 1 -- 172.21.15.82:0/3279787487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 msgr2=0x55d4da618830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f518e59c640 1 --2- 172.21.15.82:0/3279787487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 0x55d4da618830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f518e59c640 1 -- 172.21.15.82:0/3279787487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 msgr2=0x55d4da60bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f518e59c640 1 --2- 172.21.15.82:0/3279787487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da60bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f518e59c640 1 -- 172.21.15.82:0/3279787487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4da5fd9d0 con 0x55d4da603e80 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f518e59c640 1 --2- 172.21.15.82:0/3279787487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da5fcc00 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5184041e70 tx=0x7f5184005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a42291172263d55 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f5174ff9640 1 -- 172.21.15.82:0/3279787487 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184045040 con 0x55d4da603e80 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f5174ff9640 1 -- 172.21.15.82:0/3279787487 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5184004310 con 0x55d4da603e80 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.113+0000 7f5174ff9640 1 -- 172.21.15.82:0/3279787487 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184003a70 con 0x55d4da603e80 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.114+0000 7f5197dca740 1 -- 172.21.15.82:0/3279787487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 msgr2=0x55d4da5fcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.114+0000 7f5197dca740 1 --2- 172.21.15.82:0/3279787487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da5fcc00 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5184041e70 tx=0x7f5184005fd0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.114+0000 7f5197dca740 1 -- 172.21.15.82:0/3279787487 shutdown_connections 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.114+0000 7f5197dca740 1 --2- 172.21.15.82:0/3279787487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da60ca00 0x55d4da60bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.114+0000 7f5197dca740 1 --2- 172.21.15.82:0/3279787487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da603e80 0x55d4da5fcc00 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.114+0000 7f5197dca740 1 --2- 172.21.15.82:0/3279787487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da6034d0 0x55d4da618830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.114+0000 7f5197dca740 1 -- 172.21.15.82:0/3279787487 >> 172.21.15.82:0/3279787487 conn(0x55d4da5fdba0 msgr2=0x55d4da604ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.114+0000 7f5197dca740 1 -- 172.21.15.82:0/3279787487 shutdown_connections 2024-04-21T08:49:19.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.114+0000 7f5197dca740 1 -- 172.21.15.82:0/3279787487 wait complete. 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.114+0000 7f5197dca740 1 -- start start 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.114+0000 7f5197dca740 1 Processor -- start 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5197dca740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da6034d0 0x55d4da615110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5197dca740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da603e80 0x55d4da605f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5197dca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da60ca00 0x55d4da6064d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5197dca740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f517c0114a0 con 0x55d4da6034d0 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5197dca740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f517c010dd0 con 0x55d4da603e80 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5197dca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f517c00df50 con 0x55d4da60ca00 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5197dca740 10 client.0 _ll_register_callbacks cb 0x7f5184009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f518ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da6034d0 0x55d4da615110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f518ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da6034d0 0x55d4da615110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49914/0 (socket says 172.21.15.82:49914) 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f518ed9d640 1 -- 172.21.15.82:0/3246551863 learned_addr learned my addr 172.21.15.82:0/3246551863 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f518e59c640 1 --2- 172.21.15.82:0/3246551863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da603e80 0x55d4da605f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f518ed9d640 1 -- 172.21.15.82:0/3246551863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da60ca00 msgr2=0x55d4da6064d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f518ed9d640 1 --2- 172.21.15.82:0/3246551863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da60ca00 0x55d4da6064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f518ed9d640 1 -- 172.21.15.82:0/3246551863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da603e80 msgr2=0x55d4da605f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f518ed9d640 1 --2- 172.21.15.82:0/3246551863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da603e80 0x55d4da605f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f518ed9d640 1 -- 172.21.15.82:0/3246551863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4da606d00 con 0x55d4da6034d0 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f518ed9d640 1 --2- 172.21.15.82:0/3246551863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da6034d0 0x55d4da615110 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f518009c7f0 tx=0x7f5180098670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475fc77324c0d00d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5172ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5172ff5640 1 -- 172.21.15.82:0/3246551863 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51800a0ea0 con 0x55d4da6034d0 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5172ff5640 1 -- 172.21.15.82:0/3246551863 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f518009adf0 con 0x55d4da6034d0 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5197dca740 1 -- 172.21.15.82:0/3246551863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5184004310 con 0x55d4da6034d0 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5172ff5640 1 -- 172.21.15.82:0/3246551863 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f51800a5760 con 0x55d4da6034d0 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5197dca740 10 client.16698 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.115+0000 7f5197dca740 1 -- 172.21.15.82:0/3246551863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d4da682d40 con 0x55d4da6034d0 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.116+0000 7f5172ff5640 1 -- 172.21.15.82:0/3246551863 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5184004310 con 0x55d4da6034d0 2024-04-21T08:49:19.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.116+0000 7f5172ff5640 1 -- 172.21.15.82:0/3246551863 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f518009cdc0 con 0x55d4da6034d0 2024-04-21T08:49:19.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.116+0000 7f5197dca740 10 client.16698 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:19.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.116+0000 7f5197dca740 10 client.16698 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:49:19.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.116+0000 7f5197dca740 20 client.16698 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:19.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.116+0000 7f5197dca740 1 --2- 172.21.15.82:0/3246551863 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55d4da683460 0x55d4da6858b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.116+0000 7f5197dca740 4 client.16698 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:19.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.116+0000 7f5197dca740 1 -- 172.21.15.82:0/3246551863 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4da575c10 con 0x55d4da683460 2024-04-21T08:49:19.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.116+0000 7f518e59c640 1 --2- 172.21.15.82:0/3246551863 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55d4da683460 0x55d4da6858b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.117+0000 7f518e59c640 1 --2- 172.21.15.82:0/3246551863 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55d4da683460 0x55d4da6858b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bef0b622decb602a server_cookie=300375e426547474 in_seq=0 out_seq=0 2024-04-21T08:49:19.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.117+0000 7f5172ff5640 10 client.16698 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5172ff5640 1 -- 172.21.15.82:0/3246551863 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4da575c10 con 0x55d4da683460 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5172ff5640 10 client.16698 handle_command_reply: tid=0 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5197dca740 1 client.16698 shutdown 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5197dca740 1 -- 172.21.15.82:0/3246551863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da6034d0 msgr2=0x55d4da615110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5197dca740 1 --2- 172.21.15.82:0/3246551863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da6034d0 0x55d4da615110 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f518009c7f0 tx=0x7f5180098670 comp rx=0 tx=0).stop 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5197dca740 1 -- 172.21.15.82:0/3246551863 shutdown_connections 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5197dca740 1 --2- 172.21.15.82:0/3246551863 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55d4da683460 0x55d4da6858b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5197dca740 1 --2- 172.21.15.82:0/3246551863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4da60ca00 0x55d4da6064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5172ff5640 0 client.16698 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5197dca740 1 --2- 172.21.15.82:0/3246551863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4da603e80 0x55d4da605f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5197dca740 1 --2- 172.21.15.82:0/3246551863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4da6034d0 0x55d4da615110 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5197dca740 1 -- 172.21.15.82:0/3246551863 >> 172.21.15.82:0/3246551863 conn(0x55d4da5fdba0 msgr2=0x55d4da5fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5197dca740 1 -- 172.21.15.82:0/3246551863 shutdown_connections 2024-04-21T08:49:19.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5197dca740 1 -- 172.21.15.82:0/3246551863 wait complete. 2024-04-21T08:49:19.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.121+0000 7f5197dca740 20 client.16698 trim_cache size 0 max 16384 2024-04-21T08:49:19.125 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:49:19.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.122+0000 7f51951d5640 1 -- 172.21.15.82:0/3829277625 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f51780a3da0 msgr2=0x7f51780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.122+0000 7f51951d5640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f51780a3da0 0x7f51780a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5188005fd0 tx=0x7f5188009d10 comp rx=0 tx=0).stop 2024-04-21T08:49:19.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.122+0000 7f51951d5640 1 -- 172.21.15.82:0/3829277625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5190101810 msgr2=0x7f5190197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.122+0000 7f51951d5640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5190101810 0x7f5190197660 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f518000e9f0 tx=0x7f518000edf0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.122+0000 7f51951d5640 1 -- 172.21.15.82:0/3829277625 shutdown_connections 2024-04-21T08:49:19.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.122+0000 7f51951d5640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f519019bf20 0x7f519019c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.122+0000 7f51951d5640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5190106150 0x7f5190197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.122+0000 7f51951d5640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f51780a3da0 0x7f51780a6260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.122+0000 7f51951d5640 1 --2- 172.21.15.82:0/3829277625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5190101810 0x7f5190197660 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.122+0000 7f51951d5640 1 -- 172.21.15.82:0/3829277625 >> 172.21.15.82:0/3829277625 conn(0x7f5190077410 msgr2=0x7f5190104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.122+0000 7f51951d5640 1 -- 172.21.15.82:0/3829277625 shutdown_connections 2024-04-21T08:49:19.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.122+0000 7f51951d5640 1 -- 172.21.15.82:0/3829277625 wait complete. 2024-04-21T08:49:19.135 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:49:19.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.254+0000 7ff0f313d640 1 -- 172.21.15.82:0/1021213063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0ec1021a0 msgr2=0x7ff0ec102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.254+0000 7ff0f313d640 1 --2- 172.21.15.82:0/1021213063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0ec1021a0 0x7ff0ec102580 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009900 tx=0x7ff0e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:19.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.255+0000 7ff0f313d640 1 -- 172.21.15.82:0/1021213063 shutdown_connections 2024-04-21T08:49:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.255+0000 7ff0f313d640 1 --2- 172.21.15.82:0/1021213063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0ec107270 0x7ff0ec10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.255+0000 7ff0f313d640 1 --2- 172.21.15.82:0/1021213063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ec102b50 0x7ff0ec106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.255+0000 7ff0f313d640 1 --2- 172.21.15.82:0/1021213063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0ec1021a0 0x7ff0ec102580 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.255+0000 7ff0f313d640 1 -- 172.21.15.82:0/1021213063 >> 172.21.15.82:0/1021213063 conn(0x7ff0ec079350 msgr2=0x7ff0ec079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.255+0000 7ff0f313d640 1 -- 172.21.15.82:0/1021213063 shutdown_connections 2024-04-21T08:49:19.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.255+0000 7ff0f313d640 1 -- 172.21.15.82:0/1021213063 wait complete. 2024-04-21T08:49:19.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.255+0000 7ff0f313d640 1 Processor -- start 2024-04-21T08:49:19.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.255+0000 7ff0f313d640 1 -- start start 2024-04-21T08:49:19.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.255+0000 7ff0f313d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ec1021a0 0x7ff0ec19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.255+0000 7ff0f313d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0ec107270 0x7ff0ec19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f313d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0ec198590 0x7ff0ec198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f313d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1121b0 con 0x7ff0ec198590 2024-04-21T08:49:19.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f313d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec112030 con 0x7ff0ec1021a0 2024-04-21T08:49:19.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f313d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec112330 con 0x7ff0ec107270 2024-04-21T08:49:19.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f16da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0ec198590 0x7ff0ec198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f0ed9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ec1021a0 0x7ff0ec19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f0ed9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ec1021a0 0x7ff0ec19e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:60532/0 (socket says 172.21.15.82:60532) 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f0ed9640 1 -- 172.21.15.82:0/2118881587 learned_addr learned my addr 172.21.15.82:0/2118881587 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f16da640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0ec198590 0x7ff0ec198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49934/0 (socket says 172.21.15.82:49934) 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0ec107270 0x7ff0ec19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f16da640 1 -- 172.21.15.82:0/2118881587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0ec107270 msgr2=0x7ff0ec19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f16da640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0ec107270 0x7ff0ec19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f16da640 1 -- 172.21.15.82:0/2118881587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ec1021a0 msgr2=0x7ff0ec19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f16da640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ec1021a0 0x7ff0ec19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f16da640 1 -- 172.21.15.82:0/2118881587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec1990b0 con 0x7ff0ec198590 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f0ed9640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ec1021a0 0x7ff0ec19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0f16da640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0ec198590 0x7ff0ec198a40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009fd0 tx=0x7ff0e400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0e9ffb640 1 -- 172.21.15.82:0/2118881587 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e400ce20 con 0x7ff0ec198590 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0e9ffb640 1 -- 172.21.15.82:0/2118881587 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e4005020 con 0x7ff0ec198590 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.256+0000 7ff0e9ffb640 1 -- 172.21.15.82:0/2118881587 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4010690 con 0x7ff0ec198590 2024-04-21T08:49:19.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.257+0000 7ff0f313d640 1 -- 172.21.15.82:0/2118881587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec1993a0 con 0x7ff0ec198590 2024-04-21T08:49:19.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.257+0000 7ff0f313d640 1 -- 172.21.15.82:0/2118881587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec074480 con 0x7ff0ec198590 2024-04-21T08:49:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.259+0000 7ff0e9ffb640 1 -- 172.21.15.82:0/2118881587 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff0e40054d0 con 0x7ff0ec198590 2024-04-21T08:49:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0f5e09740 1 Processor -- start 2024-04-21T08:49:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0f5e09740 1 -- start start 2024-04-21T08:49:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0f5e09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaa908b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0f5e09740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa90e80 0x563deaa99430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0f5e09740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa99a00 0x563deaa9be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0f5e09740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563dea9f51f0 con 0x563deaa90e80 2024-04-21T08:49:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0f5e09740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563dea9924b0 con 0x563deaa99a00 2024-04-21T08:49:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0f5e09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563dea9f11b0 con 0x563deaa904d0 2024-04-21T08:49:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0e9ffb640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff0dc0a3d20 0x7ff0dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0f0ed9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa90e80 0x563deaa99430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0f0ed9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa90e80 0x563deaa99430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49944/0 (socket says 172.21.15.82:49944) 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0f16da640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff0dc0a3d20 0x7ff0dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0f0ed9640 1 -- 172.21.15.82:0/3004554123 learned_addr learned my addr 172.21.15.82:0/3004554123 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/3004554123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa99a00 0x563deaa9be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/3004554123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaa908b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.260+0000 7ff0e9ffb640 1 -- 172.21.15.82:0/2118881587 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff0e40d1240 con 0x7ff0ec198590 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.261+0000 7ff0f0ed9640 1 -- 172.21.15.82:0/3004554123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 msgr2=0x563deaa908b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.261+0000 7ff0f0ed9640 1 --2- 172.21.15.82:0/3004554123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaa908b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.261+0000 7ff0f0ed9640 1 -- 172.21.15.82:0/3004554123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa99a00 msgr2=0x563deaa9be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.261+0000 7ff0f0ed9640 1 --2- 172.21.15.82:0/3004554123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa99a00 0x563deaa9be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.261+0000 7ff0f0ed9640 1 -- 172.21.15.82:0/3004554123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dea9841b0 con 0x563deaa90e80 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.261+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/3004554123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaa908b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.261+0000 7ff0f0ed9640 1 --2- 172.21.15.82:0/3004554123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa90e80 0x563deaa99430 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0036040 tx=0x7ff0e0009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6191b175c2257ce server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.261+0000 7ff0f16da640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff0dc0a3d20 0x7ff0dc0a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff0e409cdb0 tx=0x7ff0e40b8040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.261+0000 7ff0d6ffd640 1 -- 172.21.15.82:0/3004554123 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0045070 con 0x563deaa90e80 2024-04-21T08:49:19.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.261+0000 7ff0d6ffd640 1 -- 172.21.15.82:0/3004554123 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e002c7e0 con 0x563deaa90e80 2024-04-21T08:49:19.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.261+0000 7ff0d6ffd640 1 -- 172.21.15.82:0/3004554123 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0003b10 con 0x563deaa90e80 2024-04-21T08:49:19.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.262+0000 7ff0f5e09740 1 -- 172.21.15.82:0/3004554123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa90e80 msgr2=0x563deaa99430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.262+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/3004554123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa90e80 0x563deaa99430 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0036040 tx=0x7ff0e0009670 comp rx=0 tx=0).stop 2024-04-21T08:49:19.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.262+0000 7ff0f5e09740 1 -- 172.21.15.82:0/3004554123 shutdown_connections 2024-04-21T08:49:19.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.262+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/3004554123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa99a00 0x563deaa9be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.262+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/3004554123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa90e80 0x563deaa99430 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.262+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/3004554123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaa908b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.262+0000 7ff0f5e09740 1 -- 172.21.15.82:0/3004554123 >> 172.21.15.82:0/3004554123 conn(0x563deaa8aba0 msgr2=0x563deaa8af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.262+0000 7ff0f5e09740 1 -- 172.21.15.82:0/3004554123 shutdown_connections 2024-04-21T08:49:19.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.262+0000 7ff0f5e09740 1 -- 172.21.15.82:0/3004554123 wait complete. 2024-04-21T08:49:19.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.262+0000 7ff0f5e09740 1 -- start start 2024-04-21T08:49:19.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.262+0000 7ff0f5e09740 1 Processor -- start 2024-04-21T08:49:19.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0f5e09740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa904d0 0x563deaa992e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0f5e09740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 0x563deaa96670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0f5e09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa99a00 0x563deaa96bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0f5e09740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563deaa9c910 con 0x563deaa904d0 2024-04-21T08:49:19.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0f5e09740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563dea9ae500 con 0x563deaa90e80 2024-04-21T08:49:19.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0f5e09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563deaa9ca90 con 0x563deaa99a00 2024-04-21T08:49:19.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0ebfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa904d0 0x563deaa992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0f5e09740 10 client.0 _ll_register_callbacks cb 0x563dea97cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0ebfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa904d0 0x563deaa992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49956/0 (socket says 172.21.15.82:49956) 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0ebfff640 1 -- 172.21.15.82:0/251229645 learned_addr learned my addr 172.21.15.82:0/251229645 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/251229645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa99a00 0x563deaa96bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0ebfff640 1 -- 172.21.15.82:0/251229645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa99a00 msgr2=0x563deaa96bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/251229645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa99a00 0x563deaa96bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0ebfff640 1 -- 172.21.15.82:0/251229645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 msgr2=0x563deaa96670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/251229645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 0x563deaa96670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0ebfff640 1 -- 172.21.15.82:0/251229645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dea7073f0 con 0x563deaa904d0 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/251229645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa904d0 0x563deaa992e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009e80 tx=0x7ff0d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11408b0703cd185f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.263+0000 7ff0cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0cbfff640 1 -- 172.21.15.82:0/251229645 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d80120d0 con 0x563deaa904d0 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0cbfff640 1 -- 172.21.15.82:0/251229645 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff0d8004050 con 0x563deaa904d0 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0f5e09740 1 -- 172.21.15.82:0/251229645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0e002c7e0 con 0x563deaa904d0 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0cbfff640 1 -- 172.21.15.82:0/251229645 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff0d8028050 con 0x563deaa904d0 2024-04-21T08:49:19.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0f5e09740 10 client.16710 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0f5e09740 1 -- 172.21.15.82:0/251229645 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563dea7d17d0 con 0x563deaa904d0 2024-04-21T08:49:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0cbfff640 1 -- 172.21.15.82:0/251229645 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e002c7e0 con 0x563deaa904d0 2024-04-21T08:49:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0cbfff640 1 -- 172.21.15.82:0/251229645 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7ff0d8003bc0 con 0x563deaa904d0 2024-04-21T08:49:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0f5e09740 10 client.16710 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0f5e09740 10 client.16710 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:49:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0f5e09740 20 client.16710 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/251229645 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x563deaaa9b20 0x563deaaabfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0f5e09740 4 client.16710 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.264+0000 7ff0f5e09740 1 -- 172.21.15.82:0/251229645 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dea994930 con 0x563deaaa9b20 2024-04-21T08:49:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.265+0000 7ff0f0ed9640 1 --2- 172.21.15.82:0/251229645 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x563deaaa9b20 0x563deaaabfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.265+0000 7ff0cbfff640 10 client.16710 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:49:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.265+0000 7ff0f0ed9640 1 --2- 172.21.15.82:0/251229645 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x563deaaa9b20 0x563deaaabfc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8bba416cb37ac45 server_cookie=ebf268fabfce4c11 in_seq=0 out_seq=0 2024-04-21T08:49:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0cbfff640 1 -- 172.21.15.82:0/251229645 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563dea994930 con 0x563deaaa9b20 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0cbfff640 10 client.16710 handle_command_reply: tid=0 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0f5e09740 1 client.16710 shutdown 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0f5e09740 1 -- 172.21.15.82:0/251229645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa904d0 msgr2=0x563deaa992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/251229645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa904d0 0x563deaa992e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009e80 tx=0x7ff0d80046c0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0f5e09740 1 -- 172.21.15.82:0/251229645 shutdown_connections 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/251229645 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x563deaaa9b20 0x563deaaabfc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0cbfff640 0 client.16710 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/251229645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa99a00 0x563deaa96bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/251229645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 0x563deaa96670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/251229645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa904d0 0x563deaa992e0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0f5e09740 1 -- 172.21.15.82:0/251229645 >> 172.21.15.82:0/251229645 conn(0x563deaa8aba0 msgr2=0x563deaa8c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0f5e09740 1 -- 172.21.15.82:0/251229645 shutdown_connections 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0f5e09740 1 -- 172.21.15.82:0/251229645 wait complete. 2024-04-21T08:49:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.267+0000 7ff0f5e09740 20 client.16710 trim_cache size 0 max 16384 2024-04-21T08:49:19.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0f5e09740 1 Processor -- start 2024-04-21T08:49:19.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0f5e09740 1 -- start start 2024-04-21T08:49:19.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0f5e09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaaa57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0f5e09740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 0x563deaa89a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0f5e09740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 0x563deaa98da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0f5e09740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1b15c0 con 0x563deaa99a00 2024-04-21T08:49:19.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0f5e09740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1b31f0 con 0x563deaa90e80 2024-04-21T08:49:19.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0f5e09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1b1c90 con 0x563deaa904d0 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0ebfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 0x563deaa98da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0ebfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 0x563deaa98da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49960/0 (socket says 172.21.15.82:49960) 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0ebfff640 1 -- 172.21.15.82:0/2789829502 learned_addr learned my addr 172.21.15.82:0/2789829502 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/2789829502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaaa57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.284+0000 7ff0f0ed9640 1 --2- 172.21.15.82:0/2789829502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 0x563deaa89a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.285+0000 7ff0ebfff640 1 -- 172.21.15.82:0/2789829502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 msgr2=0x563deaaa57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.285+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/2789829502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaaa57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.285+0000 7ff0ebfff640 1 -- 172.21.15.82:0/2789829502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 msgr2=0x563deaa89a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.285+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/2789829502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 0x563deaa89a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.285+0000 7ff0ebfff640 1 -- 172.21.15.82:0/2789829502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563deaa94bc0 con 0x563deaa99a00 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.285+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/2789829502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 0x563deaa98da0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009fd0 tx=0x7ff0d800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c50bc85dfd4c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.285+0000 7ff0d67fc640 1 -- 172.21.15.82:0/2789829502 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d80106d0 con 0x563deaa99a00 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.285+0000 7ff0d67fc640 1 -- 172.21.15.82:0/2789829502 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d80043e0 con 0x563deaa99a00 2024-04-21T08:49:19.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.285+0000 7ff0d67fc640 1 -- 172.21.15.82:0/2789829502 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d80035f0 con 0x563deaa99a00 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.286+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2789829502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 msgr2=0x563deaa98da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.286+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/2789829502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 0x563deaa98da0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009fd0 tx=0x7ff0d800efd0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.286+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2789829502 shutdown_connections 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.286+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/2789829502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 0x563deaa98da0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.286+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/2789829502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 0x563deaa89a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.286+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/2789829502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaaa57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.286+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2789829502 >> 172.21.15.82:0/2789829502 conn(0x563deaa8aba0 msgr2=0x563deaa89370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.286+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2789829502 shutdown_connections 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.286+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2789829502 wait complete. 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.286+0000 7ff0f5e09740 1 -- start start 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.286+0000 7ff0f5e09740 1 Processor -- start 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0f5e09740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaaa2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0f5e09740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 0x563deaa9d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0f5e09740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 0x563deaa9dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0f5e09740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1b2af0 con 0x563deaa99a00 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0f5e09740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1b2420 con 0x563deaa90e80 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0f5e09740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1af5a0 con 0x563deaa904d0 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0f5e09740 10 client.0 _ll_register_callbacks cb 0x7ff0d8012a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0ebfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 0x563deaa9dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0ebfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 0x563deaa9dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49976/0 (socket says 172.21.15.82:49976) 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0ebfff640 1 -- 172.21.15.82:0/2218793235 learned_addr learned my addr 172.21.15.82:0/2218793235 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaaa2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0f0ed9640 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 0x563deaa9d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0ebfff640 1 -- 172.21.15.82:0/2218793235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 msgr2=0x563deaaa2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaaa2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0ebfff640 1 -- 172.21.15.82:0/2218793235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 msgr2=0x563deaa9d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 0x563deaa9d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0ebfff640 1 -- 172.21.15.82:0/2218793235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563deab17910 con 0x563deaa99a00 2024-04-21T08:49:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0ebfff640 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 0x563deaa9dca0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8012420 tx=0x7ff0d80056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe5221918b167d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.287+0000 7ff0cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0cbfff640 1 -- 172.21.15.82:0/2218793235 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d80120d0 con 0x563deaa99a00 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0cbfff640 1 -- 172.21.15.82:0/2218793235 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff0d80106d0 con 0x563deaa99a00 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2218793235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0d80043e0 con 0x563deaa99a00 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0cbfff640 1 -- 172.21.15.82:0/2218793235 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff0d80109f0 con 0x563deaa99a00 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0f5e09740 10 client.16716 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2218793235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563deab18ab0 con 0x563deaa99a00 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0cbfff640 1 -- 172.21.15.82:0/2218793235 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d80043e0 con 0x563deaa99a00 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0cbfff640 1 -- 172.21.15.82:0/2218793235 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7ff0d8035020 con 0x563deaa99a00 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0f5e09740 10 client.16716 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0f5e09740 10 client.16716 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0f5e09740 20 client.16716 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x563deaaa9b20 0x563deab1b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0f5e09740 4 client.16716 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.288+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2218793235 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563deaa02c10 con 0x563deaaa9b20 2024-04-21T08:49:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.289+0000 7ff0f0ed9640 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x563deaaa9b20 0x563deab1b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.292+0000 7ff0f0ed9640 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x563deaaa9b20 0x563deab1b120 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb02f5319257086b server_cookie=e04f62ea76940ddc in_seq=0 out_seq=0 2024-04-21T08:49:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.292+0000 7ff0cbfff640 10 client.16716 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:49:19.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.297+0000 7ff0cbfff640 1 -- 172.21.15.82:0/2218793235 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563deaa02c10 con 0x563deaaa9b20 2024-04-21T08:49:19.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.297+0000 7ff0cbfff640 10 client.16716 handle_command_reply: tid=0 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.297+0000 7ff0f5e09740 1 client.16716 shutdown 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.297+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2218793235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 msgr2=0x563deaa9dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.297+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 0x563deaa9dca0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8012420 tx=0x7ff0d80056d0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.297+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2218793235 shutdown_connections 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.297+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x563deaaa9b20 0x563deab1b120 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.297+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563deaa99a00 0x563deaa9dca0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.297+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563deaa90e80 0x563deaa9d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.297+0000 7ff0cbfff640 0 client.16716 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.297+0000 7ff0f5e09740 1 --2- 172.21.15.82:0/2218793235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563deaa904d0 0x563deaaa2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.297+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2218793235 >> 172.21.15.82:0/2218793235 conn(0x563deaa8aba0 msgr2=0x563deaa91b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.298+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2218793235 shutdown_connections 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.298+0000 7ff0f5e09740 1 -- 172.21.15.82:0/2218793235 wait complete. 2024-04-21T08:49:19.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.298+0000 7ff0f5e09740 20 client.16716 trim_cache size 0 max 16384 2024-04-21T08:49:19.300 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:49:19.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.298+0000 7ff0f313d640 1 -- 172.21.15.82:0/2118881587 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff0dc0a3d20 msgr2=0x7ff0dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.298+0000 7ff0f313d640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff0dc0a3d20 0x7ff0dc0a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff0e409cdb0 tx=0x7ff0e40b8040 comp rx=0 tx=0).stop 2024-04-21T08:49:19.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.299+0000 7ff0f313d640 1 -- 172.21.15.82:0/2118881587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0ec198590 msgr2=0x7ff0ec198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.299+0000 7ff0f313d640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0ec198590 0x7ff0ec198a40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009fd0 tx=0x7ff0e400ef10 comp rx=0 tx=0).stop 2024-04-21T08:49:19.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.299+0000 7ff0f313d640 1 -- 172.21.15.82:0/2118881587 shutdown_connections 2024-04-21T08:49:19.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.299+0000 7ff0f313d640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff0dc0a3d20 0x7ff0dc0a61e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.299+0000 7ff0f313d640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0ec198590 0x7ff0ec198a40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.299+0000 7ff0f313d640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0ec107270 0x7ff0ec19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.299+0000 7ff0f313d640 1 --2- 172.21.15.82:0/2118881587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ec1021a0 0x7ff0ec19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.299+0000 7ff0f313d640 1 -- 172.21.15.82:0/2118881587 >> 172.21.15.82:0/2118881587 conn(0x7ff0ec079350 msgr2=0x7ff0ec1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.299+0000 7ff0f313d640 1 -- 172.21.15.82:0/2118881587 shutdown_connections 2024-04-21T08:49:19.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.299+0000 7ff0f313d640 1 -- 172.21.15.82:0/2118881587 wait complete. 2024-04-21T08:49:19.312 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:49:19.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.430+0000 7fa54e211640 1 -- 172.21.15.82:0/442708899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5481043a0 msgr2=0x7fa548104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.430+0000 7fa54e211640 1 --2- 172.21.15.82:0/442708899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5481043a0 0x7fa548104780 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa538009900 tx=0x7fa53802bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:19.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.430+0000 7fa54e211640 1 -- 172.21.15.82:0/442708899 shutdown_connections 2024-04-21T08:49:19.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.430+0000 7fa54e211640 1 --2- 172.21.15.82:0/442708899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa548109470 0x7fa548111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.430+0000 7fa54e211640 1 --2- 172.21.15.82:0/442708899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa548104d50 0x7fa548108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.430+0000 7fa54e211640 1 --2- 172.21.15.82:0/442708899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5481043a0 0x7fa548104780 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.430+0000 7fa54e211640 1 -- 172.21.15.82:0/442708899 >> 172.21.15.82:0/442708899 conn(0x7fa5480ffc40 msgr2=0x7fa548102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.430+0000 7fa54e211640 1 -- 172.21.15.82:0/442708899 shutdown_connections 2024-04-21T08:49:19.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.430+0000 7fa54e211640 1 -- 172.21.15.82:0/442708899 wait complete. 2024-04-21T08:49:19.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.430+0000 7fa54e211640 1 Processor -- start 2024-04-21T08:49:19.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa54e211640 1 -- start start 2024-04-21T08:49:19.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa54e211640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5481043a0 0x7fa5481a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa54e211640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa548109470 0x7fa5481a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa54e211640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa54819a820 0x7fa54819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa54e211640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa548114420 con 0x7fa54819a820 2024-04-21T08:49:19.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa54e211640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481142a0 con 0x7fa548109470 2024-04-21T08:49:19.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa54e211640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481145a0 con 0x7fa5481043a0 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa547fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa54819a820 0x7fa54819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa547fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa54819a820 0x7fa54819acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49988/0 (socket says 172.21.15.82:49988) 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa547fff640 1 -- 172.21.15.82:0/1002994101 learned_addr learned my addr 172.21.15.82:0/1002994101 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa546ffd640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa548109470 0x7fa5481a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa5477fe640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5481043a0 0x7fa5481a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa547fff640 1 -- 172.21.15.82:0/1002994101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5481043a0 msgr2=0x7fa5481a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa547fff640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5481043a0 0x7fa5481a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa547fff640 1 -- 172.21.15.82:0/1002994101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa548109470 msgr2=0x7fa5481a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa547fff640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa548109470 0x7fa5481a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.431+0000 7fa547fff640 1 -- 172.21.15.82:0/1002994101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54819b3b0 con 0x7fa54819a820 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.432+0000 7fa547fff640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa54819a820 0x7fa54819acd0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa540009fd0 tx=0x7fa54000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.432+0000 7fa544ff9640 1 -- 172.21.15.82:0/1002994101 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540017070 con 0x7fa54819a820 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.432+0000 7fa54e211640 1 -- 172.21.15.82:0/1002994101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54819b6a0 con 0x7fa54819a820 2024-04-21T08:49:19.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.432+0000 7fa544ff9640 1 -- 172.21.15.82:0/1002994101 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa540003e20 con 0x7fa54819a820 2024-04-21T08:49:19.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.432+0000 7fa544ff9640 1 -- 172.21.15.82:0/1002994101 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540010460 con 0x7fa54819a820 2024-04-21T08:49:19.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.432+0000 7fa54e211640 1 -- 172.21.15.82:0/1002994101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5481a7830 con 0x7fa54819a820 2024-04-21T08:49:19.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.434+0000 7fa544ff9640 1 -- 172.21.15.82:0/1002994101 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa540003970 con 0x7fa54819a820 2024-04-21T08:49:19.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa550dfe740 1 Processor -- start 2024-04-21T08:49:19.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa544ff9640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5300a3d20 0x7fa5300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa550dfe740 1 -- start start 2024-04-21T08:49:19.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa5477fe640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5300a3d20 0x7fa5300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa544ff9640 1 -- 172.21.15.82:0/1002994101 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa5400d0810 con 0x7fa54819a820 2024-04-21T08:49:19.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa550dfe740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34dd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa550dfe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34dde80 0x55a2b34e6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa550dfe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34e6a00 0x55a2b34e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa550dfe740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a2b34421f0 con 0x55a2b34dd4d0 2024-04-21T08:49:19.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa550dfe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a2b33df4b0 con 0x55a2b34e6a00 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa550dfe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a2b343e1b0 con 0x55a2b34dde80 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa5477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34dd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa5477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34dd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49994/0 (socket says 172.21.15.82:49994) 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa5477fe640 1 -- 172.21.15.82:0/3019087594 learned_addr learned my addr 172.21.15.82:0/3019087594 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa547fff640 1 --2- 172.21.15.82:0/3019087594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34e6a00 0x55a2b34e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa546ffd640 1 --2- 172.21.15.82:0/3019087594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34dde80 0x55a2b34e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa5477fe640 1 -- 172.21.15.82:0/3019087594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34dde80 msgr2=0x55a2b34e6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa5477fe640 1 --2- 172.21.15.82:0/3019087594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34dde80 0x55a2b34e6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa5477fe640 1 -- 172.21.15.82:0/3019087594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34e6a00 msgr2=0x55a2b34e8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.435+0000 7fa5477fe640 1 --2- 172.21.15.82:0/3019087594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34e6a00 0x55a2b34e8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.436+0000 7fa5477fe640 1 -- 172.21.15.82:0/3019087594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2b33d11b0 con 0x55a2b34dd4d0 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.436+0000 7fa5477fe640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5300a3d20 0x7fa5300a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa538000c00 tx=0x7fa53802c700 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.436+0000 7fa5477fe640 1 --2- 172.21.15.82:0/3019087594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34dd8b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa538009230 tx=0x7fa538047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676c0db5304147a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.436+0000 7fa52dffb640 1 -- 172.21.15.82:0/3019087594 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53802cbc0 con 0x55a2b34dd4d0 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.436+0000 7fa52dffb640 1 -- 172.21.15.82:0/3019087594 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa538005410 con 0x55a2b34dd4d0 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.436+0000 7fa52dffb640 1 -- 172.21.15.82:0/3019087594 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538003850 con 0x55a2b34dd4d0 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.437+0000 7fa550dfe740 1 -- 172.21.15.82:0/3019087594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 msgr2=0x55a2b34dd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.437+0000 7fa550dfe740 1 --2- 172.21.15.82:0/3019087594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34dd8b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa538009230 tx=0x7fa538047000 comp rx=0 tx=0).stop 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.437+0000 7fa550dfe740 1 -- 172.21.15.82:0/3019087594 shutdown_connections 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.437+0000 7fa550dfe740 1 --2- 172.21.15.82:0/3019087594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34e6a00 0x55a2b34e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.437+0000 7fa550dfe740 1 --2- 172.21.15.82:0/3019087594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34dde80 0x55a2b34e6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.437+0000 7fa550dfe740 1 --2- 172.21.15.82:0/3019087594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34dd8b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.437+0000 7fa550dfe740 1 -- 172.21.15.82:0/3019087594 >> 172.21.15.82:0/3019087594 conn(0x55a2b34d7ba0 msgr2=0x55a2b34d7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.437+0000 7fa550dfe740 1 -- 172.21.15.82:0/3019087594 shutdown_connections 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.437+0000 7fa550dfe740 1 -- 172.21.15.82:0/3019087594 wait complete. 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.437+0000 7fa550dfe740 1 -- start start 2024-04-21T08:49:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.437+0000 7fa550dfe740 1 Processor -- start 2024-04-21T08:49:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa550dfe740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34e63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa550dfe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34dde80 0x55a2b34e3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa550dfe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34e6a00 0x55a2b34e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa550dfe740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a2b34e9910 con 0x55a2b34dd4d0 2024-04-21T08:49:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa550dfe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a2b33fb500 con 0x55a2b34dde80 2024-04-21T08:49:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa550dfe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a2b34e9a90 con 0x55a2b34e6a00 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa550dfe740 10 client.0 _ll_register_callbacks cb 0x55a2b33c9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa5477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34e63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa5477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34e63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49996/0 (socket says 172.21.15.82:49996) 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa5477fe640 1 -- 172.21.15.82:0/4102038253 learned_addr learned my addr 172.21.15.82:0/4102038253 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa546ffd640 1 --2- 172.21.15.82:0/4102038253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34dde80 0x55a2b34e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa5477fe640 1 -- 172.21.15.82:0/4102038253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34e6a00 msgr2=0x55a2b34e3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa5477fe640 1 --2- 172.21.15.82:0/4102038253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34e6a00 0x55a2b34e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa5477fe640 1 -- 172.21.15.82:0/4102038253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34dde80 msgr2=0x55a2b34e3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa5477fe640 1 --2- 172.21.15.82:0/4102038253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34dde80 0x55a2b34e3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa5477fe640 1 -- 172.21.15.82:0/4102038253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2b31543f0 con 0x55a2b34dd4d0 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa5477fe640 1 --2- 172.21.15.82:0/4102038253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34e63b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa5380092e0 tx=0x7fa538009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0b272fe6483773 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.438+0000 7fa526ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.439+0000 7fa526ffd640 1 -- 172.21.15.82:0/4102038253 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53802cbc0 con 0x55a2b34dd4d0 2024-04-21T08:49:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.439+0000 7fa526ffd640 1 -- 172.21.15.82:0/4102038253 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa5380046c0 con 0x55a2b34dd4d0 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.439+0000 7fa550dfe740 1 -- 172.21.15.82:0/4102038253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa538005410 con 0x55a2b34dd4d0 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.439+0000 7fa526ffd640 1 -- 172.21.15.82:0/4102038253 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa5380048c0 con 0x55a2b34dd4d0 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.439+0000 7fa550dfe740 10 client.16728 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.439+0000 7fa550dfe740 1 -- 172.21.15.82:0/4102038253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a2b321e7d0 con 0x55a2b34dd4d0 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.440+0000 7fa526ffd640 1 -- 172.21.15.82:0/4102038253 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa538005410 con 0x55a2b34dd4d0 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.440+0000 7fa526ffd640 1 -- 172.21.15.82:0/4102038253 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fa538034440 con 0x55a2b34dd4d0 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.440+0000 7fa550dfe740 10 client.16728 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.440+0000 7fa550dfe740 10 client.16728 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.440+0000 7fa550dfe740 20 client.16728 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.440+0000 7fa550dfe740 1 --2- 172.21.15.82:0/4102038253 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a2b34f6b10 0x55a2b34f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.440+0000 7fa550dfe740 4 client.16728 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.440+0000 7fa550dfe740 1 -- 172.21.15.82:0/4102038253 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2b33e1930 con 0x55a2b34f6b10 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.440+0000 7fa546ffd640 1 --2- 172.21.15.82:0/4102038253 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a2b34f6b10 0x55a2b34f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.440+0000 7fa546ffd640 1 --2- 172.21.15.82:0/4102038253 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a2b34f6b10 0x55a2b34f8f00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=889ef738d677b783 server_cookie=d51d775f2e9dcb33 in_seq=0 out_seq=0 2024-04-21T08:49:19.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.440+0000 7fa526ffd640 10 client.16728 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa526ffd640 1 -- 172.21.15.82:0/4102038253 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2b33e1930 con 0x55a2b34f6b10 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa526ffd640 10 client.16728 handle_command_reply: tid=0 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa550dfe740 1 client.16728 shutdown 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa550dfe740 1 -- 172.21.15.82:0/4102038253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 msgr2=0x55a2b34e63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa550dfe740 1 --2- 172.21.15.82:0/4102038253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34e63b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa5380092e0 tx=0x7fa538009520 comp rx=0 tx=0).stop 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa550dfe740 1 -- 172.21.15.82:0/4102038253 shutdown_connections 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa550dfe740 1 --2- 172.21.15.82:0/4102038253 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a2b34f6b10 0x55a2b34f8f00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa550dfe740 1 --2- 172.21.15.82:0/4102038253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34e6a00 0x55a2b34e3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa550dfe740 1 --2- 172.21.15.82:0/4102038253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34dde80 0x55a2b34e3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa526ffd640 0 client.16728 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa550dfe740 1 --2- 172.21.15.82:0/4102038253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34e63b0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa550dfe740 1 -- 172.21.15.82:0/4102038253 >> 172.21.15.82:0/4102038253 conn(0x55a2b34d7ba0 msgr2=0x55a2b34d98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa550dfe740 1 -- 172.21.15.82:0/4102038253 shutdown_connections 2024-04-21T08:49:19.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.445+0000 7fa550dfe740 1 -- 172.21.15.82:0/4102038253 wait complete. 2024-04-21T08:49:19.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.446+0000 7fa550dfe740 20 client.16728 trim_cache size 0 max 16384 2024-04-21T08:49:19.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.460+0000 7fa550dfe740 1 Processor -- start 2024-04-21T08:49:19.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.460+0000 7fa550dfe740 1 -- start start 2024-04-21T08:49:19.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.460+0000 7fa550dfe740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34f2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.460+0000 7fa550dfe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34dde80 0x55a2b34e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.460+0000 7fa550dfe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34e6a00 0x55a2b34d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.460+0000 7fa550dfe740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481b1710 con 0x55a2b34dd4d0 2024-04-21T08:49:19.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.460+0000 7fa550dfe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481b3340 con 0x55a2b34e6a00 2024-04-21T08:49:19.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.460+0000 7fa550dfe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481b1de0 con 0x55a2b34dde80 2024-04-21T08:49:19.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.460+0000 7fa5477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.460+0000 7fa5477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50000/0 (socket says 172.21.15.82:50000) 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.460+0000 7fa5477fe640 1 -- 172.21.15.82:0/2038845164 learned_addr learned my addr 172.21.15.82:0/2038845164 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.460+0000 7fa547fff640 1 --2- 172.21.15.82:0/2038845164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34e6a00 0x55a2b34d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.461+0000 7fa546ffd640 1 --2- 172.21.15.82:0/2038845164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34dde80 0x55a2b34e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.461+0000 7fa5477fe640 1 -- 172.21.15.82:0/2038845164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34dde80 msgr2=0x55a2b34e28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.461+0000 7fa5477fe640 1 --2- 172.21.15.82:0/2038845164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34dde80 0x55a2b34e28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.461+0000 7fa5477fe640 1 -- 172.21.15.82:0/2038845164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34e6a00 msgr2=0x55a2b34d6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.461+0000 7fa5477fe640 1 --2- 172.21.15.82:0/2038845164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34e6a00 0x55a2b34d6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.461+0000 7fa5477fe640 1 -- 172.21.15.82:0/2038845164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2b34d2040 con 0x55a2b34dd4d0 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.461+0000 7fa546ffd640 1 --2- 172.21.15.82:0/2038845164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34dde80 0x55a2b34e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.461+0000 7fa5477fe640 1 --2- 172.21.15.82:0/2038845164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34f2820 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa538009900 tx=0x7fa53802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b1a86c9a8a8ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.461+0000 7fa52d7fa640 1 -- 172.21.15.82:0/2038845164 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538005410 con 0x55a2b34dd4d0 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.461+0000 7fa52d7fa640 1 -- 172.21.15.82:0/2038845164 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa53805c070 con 0x55a2b34dd4d0 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.461+0000 7fa52d7fa640 1 -- 172.21.15.82:0/2038845164 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538034440 con 0x55a2b34dd4d0 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.462+0000 7fa550dfe740 1 -- 172.21.15.82:0/2038845164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 msgr2=0x55a2b34f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.462+0000 7fa550dfe740 1 --2- 172.21.15.82:0/2038845164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34f2820 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa538009900 tx=0x7fa53802bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:19.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.462+0000 7fa550dfe740 1 -- 172.21.15.82:0/2038845164 shutdown_connections 2024-04-21T08:49:19.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.462+0000 7fa550dfe740 1 --2- 172.21.15.82:0/2038845164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34e6a00 0x55a2b34d6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.462+0000 7fa550dfe740 1 --2- 172.21.15.82:0/2038845164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34dde80 0x55a2b34e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.462+0000 7fa550dfe740 1 --2- 172.21.15.82:0/2038845164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dd4d0 0x55a2b34f2820 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.462+0000 7fa550dfe740 1 -- 172.21.15.82:0/2038845164 >> 172.21.15.82:0/2038845164 conn(0x55a2b34d7ba0 msgr2=0x55a2b34d6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.462+0000 7fa550dfe740 1 -- 172.21.15.82:0/2038845164 shutdown_connections 2024-04-21T08:49:19.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.462+0000 7fa550dfe740 1 -- 172.21.15.82:0/2038845164 wait complete. 2024-04-21T08:49:19.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.462+0000 7fa550dfe740 1 -- start start 2024-04-21T08:49:19.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa550dfe740 1 Processor -- start 2024-04-21T08:49:19.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa550dfe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34dd4d0 0x55a2b34eead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa550dfe740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dde80 0x55a2b34ef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa550dfe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34e6a00 0x55a2b34ea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa550dfe740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481b2c40 con 0x55a2b34dde80 2024-04-21T08:49:19.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa550dfe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481b2570 con 0x55a2b34dd4d0 2024-04-21T08:49:19.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa550dfe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481af6f0 con 0x55a2b34e6a00 2024-04-21T08:49:19.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa550dfe740 10 client.0 _ll_register_callbacks cb 0x7fa538004d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:19.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa546ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dde80 0x55a2b34ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa546ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dde80 0x55a2b34ef010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50010/0 (socket says 172.21.15.82:50010) 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa546ffd640 1 -- 172.21.15.82:0/1585587724 learned_addr learned my addr 172.21.15.82:0/1585587724 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa5477fe640 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34dd4d0 0x55a2b34eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.463+0000 7fa547fff640 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34e6a00 0x55a2b34ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa546ffd640 1 -- 172.21.15.82:0/1585587724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34e6a00 msgr2=0x55a2b34ea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa546ffd640 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34e6a00 0x55a2b34ea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa546ffd640 1 -- 172.21.15.82:0/1585587724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34dd4d0 msgr2=0x55a2b34eead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa546ffd640 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34dd4d0 0x55a2b34eead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa546ffd640 1 -- 172.21.15.82:0/1585587724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2b34ea8e0 con 0x55a2b34dde80 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa546ffd640 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dde80 0x55a2b34ef010 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa53c005fd0 tx=0x7fa53c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540c55ea9b893218 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa526ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa526ffd640 1 -- 172.21.15.82:0/1585587724 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c005020 con 0x55a2b34dde80 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa526ffd640 1 -- 172.21.15.82:0/1585587724 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa53c0051c0 con 0x55a2b34dde80 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa550dfe740 1 -- 172.21.15.82:0/1585587724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa53805c070 con 0x55a2b34dde80 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa526ffd640 1 -- 172.21.15.82:0/1585587724 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa53c02f050 con 0x55a2b34dde80 2024-04-21T08:49:19.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa550dfe740 10 client.16734 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:19.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa550dfe740 1 -- 172.21.15.82:0/1585587724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a2b355c9e0 con 0x55a2b34dde80 2024-04-21T08:49:19.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa526ffd640 1 -- 172.21.15.82:0/1585587724 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa53805c070 con 0x55a2b34dde80 2024-04-21T08:49:19.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.464+0000 7fa526ffd640 1 -- 172.21.15.82:0/1585587724 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fa53c020150 con 0x55a2b34dde80 2024-04-21T08:49:19.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.465+0000 7fa550dfe740 10 client.16734 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:19.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.465+0000 7fa550dfe740 10 client.16734 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:49:19.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.465+0000 7fa550dfe740 20 client.16734 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:19.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.465+0000 7fa550dfe740 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a2b34f6b10 0x55a2b355f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.465+0000 7fa550dfe740 4 client.16734 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:19.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.465+0000 7fa550dfe740 1 -- 172.21.15.82:0/1585587724 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2b344fc10 con 0x55a2b34f6b10 2024-04-21T08:49:19.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.465+0000 7fa5477fe640 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a2b34f6b10 0x55a2b355f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.465+0000 7fa526ffd640 10 client.16734 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:49:19.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.465+0000 7fa5477fe640 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a2b34f6b10 0x55a2b355f310 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e108baf28438fb9 server_cookie=c71666c10311d37e in_seq=0 out_seq=0 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa526ffd640 1 -- 172.21.15.82:0/1585587724 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2b344fc10 con 0x55a2b34f6b10 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa526ffd640 10 client.16734 handle_command_reply: tid=0 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa550dfe740 1 client.16734 shutdown 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa550dfe740 1 -- 172.21.15.82:0/1585587724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dde80 msgr2=0x55a2b34ef010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa550dfe740 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dde80 0x55a2b34ef010 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa53c005fd0 tx=0x7fa53c00bdd0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa550dfe740 1 -- 172.21.15.82:0/1585587724 shutdown_connections 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa550dfe740 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a2b34f6b10 0x55a2b355f310 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa550dfe740 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2b34e6a00 0x55a2b34ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa550dfe740 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2b34dde80 0x55a2b34ef010 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa550dfe740 1 --2- 172.21.15.82:0/1585587724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a2b34dd4d0 0x55a2b34eead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa550dfe740 1 -- 172.21.15.82:0/1585587724 >> 172.21.15.82:0/1585587724 conn(0x55a2b34d7ba0 msgr2=0x55a2b34d6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa526ffd640 0 client.16734 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:49:19.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa550dfe740 1 -- 172.21.15.82:0/1585587724 shutdown_connections 2024-04-21T08:49:19.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa550dfe740 1 -- 172.21.15.82:0/1585587724 wait complete. 2024-04-21T08:49:19.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.470+0000 7fa550dfe740 20 client.16734 trim_cache size 0 max 16384 2024-04-21T08:49:19.473 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:49:19.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.471+0000 7fa54e211640 1 -- 172.21.15.82:0/1002994101 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5300a3d20 msgr2=0x7fa5300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.471+0000 7fa54e211640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5300a3d20 0x7fa5300a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa538000c00 tx=0x7fa53802c700 comp rx=0 tx=0).stop 2024-04-21T08:49:19.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.471+0000 7fa54e211640 1 -- 172.21.15.82:0/1002994101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa54819a820 msgr2=0x7fa54819acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.471+0000 7fa54e211640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa54819a820 0x7fa54819acd0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa540009fd0 tx=0x7fa54000b260 comp rx=0 tx=0).stop 2024-04-21T08:49:19.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.471+0000 7fa54e211640 1 -- 172.21.15.82:0/1002994101 shutdown_connections 2024-04-21T08:49:19.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.471+0000 7fa54e211640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5300a3d20 0x7fa5300a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.471+0000 7fa54e211640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa54819a820 0x7fa54819acd0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.471+0000 7fa54e211640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa548109470 0x7fa5481a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.471+0000 7fa54e211640 1 --2- 172.21.15.82:0/1002994101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5481043a0 0x7fa5481a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.471+0000 7fa54e211640 1 -- 172.21.15.82:0/1002994101 >> 172.21.15.82:0/1002994101 conn(0x7fa5480ffc40 msgr2=0x7fa548101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.471+0000 7fa54e211640 1 -- 172.21.15.82:0/1002994101 shutdown_connections 2024-04-21T08:49:19.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.471+0000 7fa54e211640 1 -- 172.21.15.82:0/1002994101 wait complete. 2024-04-21T08:49:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:19 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/653927918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:19.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:19 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/313951752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:19.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:19 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/653927918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:19.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:19 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/313951752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:19.484 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:49:19.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.601+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/3256291658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d181018e0 msgr2=0x7f9d18105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.601+0000 7f9d1c9b6640 1 --2- 172.21.15.82:0/3256291658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d181018e0 0x7f9d18105da0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9d08009900 tx=0x7f9d0802bd60 comp rx=0 tx=0).stop 2024-04-21T08:49:19.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.601+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/3256291658 shutdown_connections 2024-04-21T08:49:19.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.601+0000 7f9d1c9b6640 1 --2- 172.21.15.82:0/3256291658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d181062e0 0x7f9d18106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.601+0000 7f9d1c9b6640 1 --2- 172.21.15.82:0/3256291658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d181018e0 0x7f9d18105da0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.601+0000 7f9d1c9b6640 1 --2- 172.21.15.82:0/3256291658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d18100fc0 0x7f9d181013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.601+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/3256291658 >> 172.21.15.82:0/3256291658 conn(0x7f9d18077f50 msgr2=0x7f9d18078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.601+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/3256291658 shutdown_connections 2024-04-21T08:49:19.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.601+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/3256291658 wait complete. 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d1c9b6640 1 Processor -- start 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d1c9b6640 1 -- start start 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d1c9b6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d18100fc0 0x7f9d1819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d1c9b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d181018e0 0x7f9d1819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d1c9b6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d181062e0 0x7f9d181a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d1c9b6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181187f0 con 0x7f9d181062e0 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d1c9b6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d18118670 con 0x7f9d18100fc0 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d1c9b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d18118970 con 0x7f9d181018e0 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d16d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d181062e0 0x7f9d181a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d16d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d181062e0 0x7f9d181a35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50042/0 (socket says 172.21.15.82:50042) 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d16d9d640 1 -- 172.21.15.82:0/2297827080 learned_addr learned my addr 172.21.15.82:0/2297827080 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d1659c640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d18100fc0 0x7f9d1819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.602+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d181018e0 0x7f9d1819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.603+0000 7f9d16d9d640 1 -- 172.21.15.82:0/2297827080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d181018e0 msgr2=0x7f9d1819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.603+0000 7f9d16d9d640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d181018e0 0x7f9d1819f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.603+0000 7f9d16d9d640 1 -- 172.21.15.82:0/2297827080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d18100fc0 msgr2=0x7f9d1819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.603+0000 7f9d16d9d640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d18100fc0 0x7f9d1819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.603+0000 7f9d16d9d640 1 -- 172.21.15.82:0/2297827080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d181a3d20 con 0x7f9d181062e0 2024-04-21T08:49:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.603+0000 7f9d16d9d640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d181062e0 0x7f9d181a35f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000eb10 tx=0x7f9d1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.603+0000 7f9d0f7fe640 1 -- 172.21.15.82:0/2297827080 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10004b20 con 0x7f9d181062e0 2024-04-21T08:49:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.603+0000 7f9d0f7fe640 1 -- 172.21.15.82:0/2297827080 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d10004cc0 con 0x7f9d181062e0 2024-04-21T08:49:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.603+0000 7f9d0f7fe640 1 -- 172.21.15.82:0/2297827080 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10010670 con 0x7f9d181062e0 2024-04-21T08:49:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.603+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/2297827080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d181a4010 con 0x7f9d181062e0 2024-04-21T08:49:19.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.603+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/2297827080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d181ab8a0 con 0x7f9d181062e0 2024-04-21T08:49:19.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.605+0000 7f9d0f7fe640 1 -- 172.21.15.82:0/2297827080 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9d10005020 con 0x7f9d181062e0 2024-04-21T08:49:19.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.606+0000 7f9d1f5aa740 1 Processor -- start 2024-04-21T08:49:19.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.606+0000 7f9d1f5aa740 1 -- start start 2024-04-21T08:49:19.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.606+0000 7f9d1f5aa740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 0x562c18a4f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.606+0000 7f9d1f5aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4fe80 0x562c18a58430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.606+0000 7f9d1f5aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a5ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.606+0000 7f9d1f5aa740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562c189b41f0 con 0x562c18a4f4d0 2024-04-21T08:49:19.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.606+0000 7f9d1f5aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562c189514b0 con 0x562c18a58a00 2024-04-21T08:49:19.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.606+0000 7f9d1f5aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562c189b01b0 con 0x562c18a4fe80 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.606+0000 7f9d0f7fe640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9d040a3da0 0x7f9d040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.606+0000 7f9d15d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 0x562c18a4f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d1659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4fe80 0x562c18a58430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d1659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4fe80 0x562c18a58430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:49912/0 (socket says 172.21.15.82:49912) 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d1659c640 1 -- 172.21.15.82:0/1521440904 learned_addr learned my addr 172.21.15.82:0/1521440904 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d16d9d640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9d040a3da0 0x7f9d040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d15d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 0x562c18a4f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50056/0 (socket says 172.21.15.82:50056) 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/1521440904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a5ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d0f7fe640 1 -- 172.21.15.82:0/2297827080 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f9d100d0fd0 con 0x7f9d181062e0 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d15d9b640 1 -- 172.21.15.82:0/1521440904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4fe80 msgr2=0x562c18a58430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/1521440904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4fe80 0x562c18a58430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d15d9b640 1 -- 172.21.15.82:0/1521440904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 msgr2=0x562c18a5ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/1521440904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a5ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d15d9b640 1 -- 172.21.15.82:0/1521440904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c189431b0 con 0x562c18a4f4d0 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/1521440904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 0x562c18a4f8b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9d0802c2a0 tx=0x7f9d08005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10aa5e38ff76ba39 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d16d9d640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9d040a3da0 0x7f9d040a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d1009c8b0 tx=0x7f9d10089040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d0cff9640 1 -- 172.21.15.82:0/1521440904 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08048070 con 0x562c18a4f4d0 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d0cff9640 1 -- 172.21.15.82:0/1521440904 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0802cdc0 con 0x562c18a4f4d0 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.607+0000 7f9d0cff9640 1 -- 172.21.15.82:0/1521440904 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080055f0 con 0x562c18a4f4d0 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.608+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1521440904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 msgr2=0x562c18a4f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.608+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1521440904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 0x562c18a4f8b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9d0802c2a0 tx=0x7f9d08005fd0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.608+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1521440904 shutdown_connections 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.608+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1521440904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a5ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.608+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1521440904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4fe80 0x562c18a58430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.608+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1521440904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 0x562c18a4f8b0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.608+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1521440904 >> 172.21.15.82:0/1521440904 conn(0x562c18a49ba0 msgr2=0x562c18a49f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.608+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1521440904 shutdown_connections 2024-04-21T08:49:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.608+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1521440904 wait complete. 2024-04-21T08:49:19.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1f5aa740 1 -- start start 2024-04-21T08:49:19.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1f5aa740 1 Processor -- start 2024-04-21T08:49:19.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1f5aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a4f4d0 0x562c18a58340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1f5aa740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 0x562c18a556d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1f5aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a58a00 0x562c18a55ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1f5aa740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562c18a5b910 con 0x562c18a4fe80 2024-04-21T08:49:19.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1f5aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562c1896d500 con 0x562c18a4f4d0 2024-04-21T08:49:19.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1f5aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562c18a5ba90 con 0x562c18a58a00 2024-04-21T08:49:19.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1f5aa740 10 client.0 _ll_register_callbacks cb 0x562c1893bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d15d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a4f4d0 0x562c18a58340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 0x562c18a556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d15d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a58a00 0x562c18a55ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 0x562c18a556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50064/0 (socket says 172.21.15.82:50064) 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d15d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a58a00 0x562c18a55ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:49924/0 (socket says 172.21.15.82:49924) 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1659c640 1 -- 172.21.15.82:0/1528260284 learned_addr learned my addr 172.21.15.82:0/1528260284 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1659c640 1 -- 172.21.15.82:0/1528260284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a58a00 msgr2=0x562c18a55ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.609+0000 7f9d1659c640 1 --2- 172.21.15.82:0/1528260284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a58a00 0x562c18a55ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9d1659c640 1 -- 172.21.15.82:0/1528260284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a4f4d0 msgr2=0x562c18a58340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9d1659c640 1 --2- 172.21.15.82:0/1528260284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a4f4d0 0x562c18a58340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9d1659c640 1 -- 172.21.15.82:0/1528260284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c186c63f0 con 0x562c18a4fe80 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9d1659c640 1 --2- 172.21.15.82:0/1528260284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 0x562c18a556d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9d00012720 tx=0x7f9d00012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c569e0123ee1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9cfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:19.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9cfdffb640 1 -- 172.21.15.82:0/1528260284 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00004050 con 0x562c18a4fe80 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9cfdffb640 1 -- 172.21.15.82:0/1528260284 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f9d000041f0 con 0x562c18a4fe80 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1528260284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d0802cdc0 con 0x562c18a4fe80 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9cfdffb640 1 -- 172.21.15.82:0/1528260284 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9d000043f0 con 0x562c18a4fe80 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9d1f5aa740 10 client.16746 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1528260284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c187907d0 con 0x562c18a4fe80 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9cfdffb640 1 -- 172.21.15.82:0/1528260284 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0802cdc0 con 0x562c18a4fe80 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9cfdffb640 1 -- 172.21.15.82:0/1528260284 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f9d00003bc0 con 0x562c18a4fe80 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9d1f5aa740 10 client.16746 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9d1f5aa740 10 client.16746 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9d1f5aa740 20 client.16746 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.610+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1528260284 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562c18a68c30 0x562c18a6b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.611+0000 7f9d1f5aa740 4 client.16746 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.611+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1528260284 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c18953930 con 0x562c18a68c30 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.611+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/1528260284 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562c18a68c30 0x562c18a6b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.611+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/1528260284 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562c18a68c30 0x562c18a6b020 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0c9d869e41a95c4 server_cookie=dbfb71fe4207a134 in_seq=0 out_seq=0 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.611+0000 7f9cfdffb640 10 client.16746 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:49:19.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.612+0000 7f9cfdffb640 1 -- 172.21.15.82:0/1528260284 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c18953930 con 0x562c18a68c30 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.612+0000 7f9cfdffb640 10 client.16746 handle_command_reply: tid=0 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.612+0000 7f9d1f5aa740 1 client.16746 shutdown 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.612+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1528260284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 msgr2=0x562c18a556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.612+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1528260284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 0x562c18a556d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9d00012720 tx=0x7f9d00012be0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.612+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1528260284 shutdown_connections 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.612+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1528260284 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562c18a68c30 0x562c18a6b020 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.612+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1528260284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a58a00 0x562c18a55ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.612+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1528260284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 0x562c18a556d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.612+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1528260284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a4f4d0 0x562c18a58340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.612+0000 7f9cfdffb640 0 client.16746 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.612+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1528260284 >> 172.21.15.82:0/1528260284 conn(0x562c18a49ba0 msgr2=0x562c18a4b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.613+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1528260284 shutdown_connections 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.613+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1528260284 wait complete. 2024-04-21T08:49:19.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.613+0000 7f9d1f5aa740 20 client.16746 trim_cache size 0 max 16384 2024-04-21T08:49:19.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:19 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/653927918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:19.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:19 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/313951752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:19.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.627+0000 7f9d1f5aa740 1 Processor -- start 2024-04-21T08:49:19.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.627+0000 7f9d1f5aa740 1 -- start start 2024-04-21T08:49:19.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.627+0000 7f9d1f5aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4f4d0 0x562c18a57e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.627+0000 7f9d1f5aa740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 0x562c18a58340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.627+0000 7f9d1f5aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a708f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.627+0000 7f9d1f5aa740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181b59f0 con 0x562c18a4fe80 2024-04-21T08:49:19.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.627+0000 7f9d1f5aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181b7620 con 0x562c18a58a00 2024-04-21T08:49:19.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.627+0000 7f9d1f5aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181b60c0 con 0x562c18a4f4d0 2024-04-21T08:49:19.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9d1659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 0x562c18a58340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9d1659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 0x562c18a58340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50068/0 (socket says 172.21.15.82:50068) 2024-04-21T08:49:19.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9d1659c640 1 -- 172.21.15.82:0/1050645826 learned_addr learned my addr 172.21.15.82:0/1050645826 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/1050645826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9d1659c640 1 -- 172.21.15.82:0/1050645826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4f4d0 msgr2=0x562c18a57e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9d1659c640 1 --2- 172.21.15.82:0/1050645826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4f4d0 0x562c18a57e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9d1659c640 1 -- 172.21.15.82:0/1050645826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 msgr2=0x562c18a708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9d1659c640 1 --2- 172.21.15.82:0/1050645826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a708f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9d1659c640 1 -- 172.21.15.82:0/1050645826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c18a4fc20 con 0x562c18a4fe80 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/1050645826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9d1659c640 1 --2- 172.21.15.82:0/1050645826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 0x562c18a58340 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9d000126f0 tx=0x7f9d000096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5475d9bcb8a582c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9cfffff640 1 -- 172.21.15.82:0/1050645826 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00003f90 con 0x562c18a4fe80 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9cfffff640 1 -- 172.21.15.82:0/1050645826 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0000fdd0 con 0x562c18a4fe80 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.628+0000 7f9cfffff640 1 -- 172.21.15.82:0/1050645826 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00005610 con 0x562c18a4fe80 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.629+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1050645826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 msgr2=0x562c18a58340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.629+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1050645826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 0x562c18a58340 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9d000126f0 tx=0x7f9d000096c0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.629+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1050645826 shutdown_connections 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.629+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1050645826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.629+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1050645826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4fe80 0x562c18a58340 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.629+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1050645826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4f4d0 0x562c18a57e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.629+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1050645826 >> 172.21.15.82:0/1050645826 conn(0x562c18a49ba0 msgr2=0x562c18a50260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.629+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1050645826 shutdown_connections 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.629+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1050645826 wait complete. 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.629+0000 7f9d1f5aa740 1 -- start start 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.629+0000 7f9d1f5aa740 1 Processor -- start 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d1f5aa740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 0x562c18a5c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d1f5aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4fe80 0x562c18a5cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d1f5aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a5d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d1f5aa740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181b6f20 con 0x562c18a4f4d0 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d1f5aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181b6850 con 0x562c18a58a00 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d1f5aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181b39d0 con 0x562c18a4fe80 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d1f5aa740 10 client.0 _ll_register_callbacks cb 0x7f9d00009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d15d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 0x562c18a5c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d15d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 0x562c18a5c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50070/0 (socket says 172.21.15.82:50070) 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d15d9b640 1 -- 172.21.15.82:0/1909057686 learned_addr learned my addr 172.21.15.82:0/1909057686 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d1659c640 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4fe80 0x562c18a5cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a5d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d15d9b640 1 -- 172.21.15.82:0/1909057686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4fe80 msgr2=0x562c18a5cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4fe80 0x562c18a5cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d15d9b640 1 -- 172.21.15.82:0/1909057686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 msgr2=0x562c18a5d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a5d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d15d9b640 1 -- 172.21.15.82:0/1909057686 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c18ace210 con 0x562c18a4f4d0 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d1659c640 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4fe80 0x562c18a5cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d15d9b640 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 0x562c18a5c7a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9d0802c340 tx=0x7f9d08000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9561675206369c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9cfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9cfdffb640 1 -- 172.21.15.82:0/1909057686 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0805c020 con 0x562c18a4f4d0 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9cfdffb640 1 -- 172.21.15.82:0/1909057686 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f9d08005180 con 0x562c18a4f4d0 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1909057686 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d0000fdd0 con 0x562c18a4f4d0 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9cfdffb640 1 -- 172.21.15.82:0/1909057686 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9d0802ce60 con 0x562c18a4f4d0 2024-04-21T08:49:19.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d1f5aa740 10 client.16752 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.630+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1909057686 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c18acf4f0 con 0x562c18a4f4d0 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.631+0000 7f9cfdffb640 1 -- 172.21.15.82:0/1909057686 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0000fdd0 con 0x562c18a4f4d0 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.631+0000 7f9cfdffb640 1 -- 172.21.15.82:0/1909057686 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f9d08040040 con 0x562c18a4f4d0 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.631+0000 7f9d1f5aa740 10 client.16752 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.631+0000 7f9d1f5aa740 10 client.16752 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.631+0000 7f9d1f5aa740 20 client.16752 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.631+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562c18acf780 0x562c18ad1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.631+0000 7f9d1f5aa740 4 client.16752 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.631+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1909057686 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c189c1c10 con 0x562c18acf780 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.631+0000 7f9d1659c640 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562c18acf780 0x562c18ad1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.631+0000 7f9d1659c640 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562c18acf780 0x562c18ad1b70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=464cb7a1942607e4 server_cookie=ea0eec754b53fb0a in_seq=0 out_seq=0 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.631+0000 7f9cfdffb640 10 client.16752 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.631+0000 7f9cfdffb640 1 -- 172.21.15.82:0/1909057686 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c189c1c10 con 0x562c18acf780 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9cfdffb640 10 client.16752 handle_command_reply: tid=0 2024-04-21T08:49:19.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1f5aa740 1 client.16752 shutdown 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1909057686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 msgr2=0x562c18a5c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 0x562c18a5c7a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9d0802c340 tx=0x7f9d08000bc0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1909057686 shutdown_connections 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562c18acf780 0x562c18ad1b70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c18a58a00 0x562c18a5d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c18a4fe80 0x562c18a5cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9cfdffb640 0 client.16752 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1f5aa740 1 --2- 172.21.15.82:0/1909057686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c18a4f4d0 0x562c18a5c7a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1909057686 >> 172.21.15.82:0/1909057686 conn(0x562c18a49ba0 msgr2=0x562c18a50af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1909057686 shutdown_connections 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1f5aa740 1 -- 172.21.15.82:0/1909057686 wait complete. 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1f5aa740 20 client.16752 trim_cache size 0 max 16384 2024-04-21T08:49:19.639 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:49:19.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/2297827080 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9d040a3da0 msgr2=0x7f9d040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1c9b6640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9d040a3da0 0x7f9d040a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d1009c8b0 tx=0x7f9d10089040 comp rx=0 tx=0).stop 2024-04-21T08:49:19.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/2297827080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d181062e0 msgr2=0x7f9d181a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1c9b6640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d181062e0 0x7f9d181a35f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000eb10 tx=0x7f9d1000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/2297827080 shutdown_connections 2024-04-21T08:49:19.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1c9b6640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9d040a3da0 0x7f9d040a6260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1c9b6640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d181062e0 0x7f9d181a35f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1c9b6640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d181018e0 0x7f9d1819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1c9b6640 1 --2- 172.21.15.82:0/2297827080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d18100fc0 0x7f9d1819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/2297827080 >> 172.21.15.82:0/2297827080 conn(0x7f9d18077f50 msgr2=0x7f9d18103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.632+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/2297827080 shutdown_connections 2024-04-21T08:49:19.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.633+0000 7f9d1c9b6640 1 -- 172.21.15.82:0/2297827080 wait complete. 2024-04-21T08:49:19.644 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:49:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.754+0000 7f9598fdb640 1 -- 172.21.15.82:0/1073188527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9594101a40 msgr2=0x7f9594105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.754+0000 7f9598fdb640 1 --2- 172.21.15.82:0/1073188527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9594101a40 0x7f9594105a80 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9588009900 tx=0x7f958802bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.754+0000 7f9598fdb640 1 -- 172.21.15.82:0/1073188527 shutdown_connections 2024-04-21T08:49:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.754+0000 7f9598fdb640 1 --2- 172.21.15.82:0/1073188527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9594106160 0x7f959410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.754+0000 7f9598fdb640 1 --2- 172.21.15.82:0/1073188527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9594101a40 0x7f9594105a80 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.754+0000 7f9598fdb640 1 --2- 172.21.15.82:0/1073188527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9594101090 0x7f9594101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.754+0000 7f9598fdb640 1 -- 172.21.15.82:0/1073188527 >> 172.21.15.82:0/1073188527 conn(0x7f95940771d0 msgr2=0x7f95940775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.754+0000 7f9598fdb640 1 -- 172.21.15.82:0/1073188527 shutdown_connections 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.754+0000 7f9598fdb640 1 -- 172.21.15.82:0/1073188527 wait complete. 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f9598fdb640 1 Processor -- start 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f9598fdb640 1 -- start start 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f9598fdb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9594101090 0x7f95941a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f9598fdb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9594106160 0x7f95941a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f9598fdb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959419a780 0x7f959419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f9598fdb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f95941110d0 con 0x7f959419a780 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f9598fdb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9594110f50 con 0x7f9594106160 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f9598fdb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9594111250 con 0x7f9594101090 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f9592d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959419a780 0x7f959419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f9592d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959419a780 0x7f959419ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50082/0 (socket says 172.21.15.82:50082) 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f9592d9d640 1 -- 172.21.15.82:0/2595529208 learned_addr learned my addr 172.21.15.82:0/2595529208 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f9591d9b640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9594106160 0x7f95941a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.755+0000 7f959259c640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9594101090 0x7f95941a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.756+0000 7f9592d9d640 1 -- 172.21.15.82:0/2595529208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9594101090 msgr2=0x7f95941a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.756+0000 7f9592d9d640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9594101090 0x7f95941a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.756+0000 7f9592d9d640 1 -- 172.21.15.82:0/2595529208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9594106160 msgr2=0x7f95941a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.756+0000 7f9592d9d640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9594106160 0x7f95941a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.756+0000 7f9592d9d640 1 -- 172.21.15.82:0/2595529208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959419b310 con 0x7f959419a780 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.756+0000 7f9592d9d640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959419a780 0x7f959419ac30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f958c00eb10 tx=0x7f958c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.756+0000 7f95837fe640 1 -- 172.21.15.82:0/2595529208 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c004b20 con 0x7f959419a780 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.756+0000 7f95837fe640 1 -- 172.21.15.82:0/2595529208 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f958c004cc0 con 0x7f959419a780 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.756+0000 7f9598fdb640 1 -- 172.21.15.82:0/2595529208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959419b600 con 0x7f959419a780 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.756+0000 7f95837fe640 1 -- 172.21.15.82:0/2595529208 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c010670 con 0x7f959419a780 2024-04-21T08:49:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.756+0000 7f9598fdb640 1 -- 172.21.15.82:0/2595529208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95941a7760 con 0x7f959419a780 2024-04-21T08:49:19.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.758+0000 7f95837fe640 1 -- 172.21.15.82:0/2595529208 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f958c005020 con 0x7f959419a780 2024-04-21T08:49:19.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.759+0000 7f95837fe640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f957c0a3d20 0x7f957c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.759+0000 7f959259c640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f957c0a3d20 0x7f957c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.759+0000 7f95837fe640 1 -- 172.21.15.82:0/2595529208 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f958c0d10c0 con 0x7f959419a780 2024-04-21T08:49:19.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.759+0000 7f959bbc5740 1 Processor -- start 2024-04-21T08:49:19.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.759+0000 7f959259c640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f957c0a3d20 0x7f957c0a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f958400bbe0 tx=0x7f958400b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.759+0000 7f959bbc5740 1 -- start start 2024-04-21T08:49:19.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959bbc5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d0567608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959bbc5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056760e80 0x55d056769430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959bbc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056769a00 0x55d05676be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959bbc5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d0566c51f0 con 0x55d0567604d0 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959bbc5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d0566624b0 con 0x55d056760e80 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959bbc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0566c11b0 con 0x55d056769a00 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d0567608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d0567608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50092/0 (socket says 172.21.15.82:50092) 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959259c640 1 -- 172.21.15.82:0/3989039728 learned_addr learned my addr 172.21.15.82:0/3989039728 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f9591d9b640 1 --2- 172.21.15.82:0/3989039728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056760e80 0x55d056769430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f9592d9d640 1 --2- 172.21.15.82:0/3989039728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056769a00 0x55d05676be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959259c640 1 -- 172.21.15.82:0/3989039728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056769a00 msgr2=0x55d05676be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959259c640 1 --2- 172.21.15.82:0/3989039728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056769a00 0x55d05676be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959259c640 1 -- 172.21.15.82:0/3989039728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056760e80 msgr2=0x55d056769430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959259c640 1 --2- 172.21.15.82:0/3989039728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056760e80 0x55d056769430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959259c640 1 -- 172.21.15.82:0/3989039728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0566541b0 con 0x55d0567604d0 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f959259c640 1 --2- 172.21.15.82:0/3989039728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d0567608b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9584005d00 tx=0x7f9584013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f2fa760467714f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f9580ff9640 1 -- 172.21.15.82:0/3989039728 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584005220 con 0x55d0567604d0 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f9580ff9640 1 -- 172.21.15.82:0/3989039728 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9584004c50 con 0x55d0567604d0 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.760+0000 7f9580ff9640 1 -- 172.21.15.82:0/3989039728 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958401e930 con 0x55d0567604d0 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.761+0000 7f959bbc5740 1 -- 172.21.15.82:0/3989039728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 msgr2=0x55d0567608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.761+0000 7f959bbc5740 1 --2- 172.21.15.82:0/3989039728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d0567608b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9584005d00 tx=0x7f9584013000 comp rx=0 tx=0).stop 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.761+0000 7f959bbc5740 1 -- 172.21.15.82:0/3989039728 shutdown_connections 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.761+0000 7f959bbc5740 1 --2- 172.21.15.82:0/3989039728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056769a00 0x55d05676be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.761+0000 7f959bbc5740 1 --2- 172.21.15.82:0/3989039728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056760e80 0x55d056769430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.761+0000 7f959bbc5740 1 --2- 172.21.15.82:0/3989039728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d0567608b0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.761+0000 7f959bbc5740 1 -- 172.21.15.82:0/3989039728 >> 172.21.15.82:0/3989039728 conn(0x55d05675aba0 msgr2=0x55d05675af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.761+0000 7f959bbc5740 1 -- 172.21.15.82:0/3989039728 shutdown_connections 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.761+0000 7f959bbc5740 1 -- 172.21.15.82:0/3989039728 wait complete. 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959bbc5740 1 -- start start 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959bbc5740 1 Processor -- start 2024-04-21T08:49:19.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959bbc5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056769330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959bbc5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056760e80 0x55d0567666c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959bbc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056769a00 0x55d056766cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959bbc5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d05676c970 con 0x55d0567604d0 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959bbc5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d05667e500 con 0x55d056760e80 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959bbc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d05676caf0 con 0x55d056769a00 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959bbc5740 10 client.0 _ll_register_callbacks cb 0x55d05664cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056769330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f9591d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056760e80 0x55d0567666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056769330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50098/0 (socket says 172.21.15.82:50098) 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959259c640 1 -- 172.21.15.82:0/1363631627 learned_addr learned my addr 172.21.15.82:0/1363631627 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f9592d9d640 1 --2- 172.21.15.82:0/1363631627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056769a00 0x55d056766cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959259c640 1 -- 172.21.15.82:0/1363631627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056769a00 msgr2=0x55d056766cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959259c640 1 --2- 172.21.15.82:0/1363631627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056769a00 0x55d056766cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959259c640 1 -- 172.21.15.82:0/1363631627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056760e80 msgr2=0x55d0567666c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959259c640 1 --2- 172.21.15.82:0/1363631627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056760e80 0x55d0567666c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.762+0000 7f959259c640 1 -- 172.21.15.82:0/1363631627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0563d73f0 con 0x55d0567604d0 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f959259c640 1 --2- 172.21.15.82:0/1363631627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056769330 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9584005c90 tx=0x7f95840048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fe244a000a1890 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f9579ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f9579ffb640 1 -- 172.21.15.82:0/1363631627 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584036060 con 0x55d0567604d0 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f9579ffb640 1 -- 172.21.15.82:0/1363631627 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f9584005410 con 0x55d0567604d0 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f959bbc5740 1 -- 172.21.15.82:0/1363631627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9584004c50 con 0x55d0567604d0 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f9579ffb640 1 -- 172.21.15.82:0/1363631627 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f958403d050 con 0x55d0567604d0 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f959bbc5740 10 client.16764 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f959bbc5740 1 -- 172.21.15.82:0/1363631627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0564a17d0 con 0x55d0567604d0 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f9579ffb640 1 -- 172.21.15.82:0/1363631627 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9584004c50 con 0x55d0567604d0 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f9579ffb640 1 -- 172.21.15.82:0/1363631627 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f9584026400 con 0x55d0567604d0 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f959bbc5740 10 client.16764 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f959bbc5740 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f959bbc5740 20 client.16764 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:19.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f959bbc5740 1 --2- 172.21.15.82:0/1363631627 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d056779bd0 0x55d05677bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f959bbc5740 4 client.16764 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.763+0000 7f959bbc5740 1 -- 172.21.15.82:0/1363631627 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d056664930 con 0x55d056779bd0 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.764+0000 7f9591d9b640 1 --2- 172.21.15.82:0/1363631627 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d056779bd0 0x55d05677bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.764+0000 7f9591d9b640 1 --2- 172.21.15.82:0/1363631627 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d056779bd0 0x55d05677bfc0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14a67f8fab67a4d0 server_cookie=93bf75232e4b2836 in_seq=0 out_seq=0 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.764+0000 7f9579ffb640 10 client.16764 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f9579ffb640 1 -- 172.21.15.82:0/1363631627 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d056664930 con 0x55d056779bd0 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f9579ffb640 10 client.16764 handle_command_reply: tid=0 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f959bbc5740 1 client.16764 shutdown 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f959bbc5740 1 -- 172.21.15.82:0/1363631627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 msgr2=0x55d056769330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f959bbc5740 1 --2- 172.21.15.82:0/1363631627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056769330 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9584005c90 tx=0x7f95840048f0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f959bbc5740 1 -- 172.21.15.82:0/1363631627 shutdown_connections 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f959bbc5740 1 --2- 172.21.15.82:0/1363631627 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d056779bd0 0x55d05677bfc0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f959bbc5740 1 --2- 172.21.15.82:0/1363631627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056769a00 0x55d056766cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f959bbc5740 1 --2- 172.21.15.82:0/1363631627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056760e80 0x55d0567666c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f9579ffb640 0 client.16764 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f959bbc5740 1 --2- 172.21.15.82:0/1363631627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056769330 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f959bbc5740 1 -- 172.21.15.82:0/1363631627 >> 172.21.15.82:0/1363631627 conn(0x55d05675aba0 msgr2=0x55d05675c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f959bbc5740 1 -- 172.21.15.82:0/1363631627 shutdown_connections 2024-04-21T08:49:19.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f959bbc5740 1 -- 172.21.15.82:0/1363631627 wait complete. 2024-04-21T08:49:19.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.765+0000 7f959bbc5740 20 client.16764 trim_cache size 0 max 16384 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.780+0000 7f959bbc5740 1 Processor -- start 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.780+0000 7f959bbc5740 1 -- start start 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.780+0000 7f959bbc5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056775830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.780+0000 7f959bbc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056760e80 0x55d056768df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.780+0000 7f959bbc5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056769a00 0x55d056769330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.780+0000 7f959bbc5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f95941b15b0 con 0x55d0567604d0 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.780+0000 7f959bbc5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f95941b31e0 con 0x55d056769a00 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.780+0000 7f959bbc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95941b1c80 con 0x55d056760e80 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.780+0000 7f959259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056775830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.780+0000 7f959259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056775830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50110/0 (socket says 172.21.15.82:50110) 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.780+0000 7f959259c640 1 -- 172.21.15.82:0/3308397294 learned_addr learned my addr 172.21.15.82:0/3308397294 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.780+0000 7f9592d9d640 1 --2- 172.21.15.82:0/3308397294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056769a00 0x55d056769330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.781+0000 7f9591d9b640 1 --2- 172.21.15.82:0/3308397294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056760e80 0x55d056768df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.781+0000 7f959259c640 1 -- 172.21.15.82:0/3308397294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056760e80 msgr2=0x55d056768df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.781+0000 7f959259c640 1 --2- 172.21.15.82:0/3308397294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056760e80 0x55d056768df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.781+0000 7f959259c640 1 -- 172.21.15.82:0/3308397294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056769a00 msgr2=0x55d056769330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.781+0000 7f959259c640 1 --2- 172.21.15.82:0/3308397294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056769a00 0x55d056769330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.781+0000 7f959259c640 1 -- 172.21.15.82:0/3308397294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0567600b0 con 0x55d0567604d0 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.781+0000 7f9592d9d640 1 --2- 172.21.15.82:0/3308397294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056769a00 0x55d056769330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.781+0000 7f959259c640 1 --2- 172.21.15.82:0/3308397294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056775830 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f95840267d0 tx=0x7f9584010fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a36337330000af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.781+0000 7f957bfff640 1 -- 172.21.15.82:0/3308397294 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584036070 con 0x55d0567604d0 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.781+0000 7f957bfff640 1 -- 172.21.15.82:0/3308397294 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9584026a40 con 0x55d0567604d0 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.781+0000 7f957bfff640 1 -- 172.21.15.82:0/3308397294 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584004c50 con 0x55d0567604d0 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 -- 172.21.15.82:0/3308397294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 msgr2=0x55d056775830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 --2- 172.21.15.82:0/3308397294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056775830 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f95840267d0 tx=0x7f9584010fb0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 -- 172.21.15.82:0/3308397294 shutdown_connections 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 --2- 172.21.15.82:0/3308397294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056769a00 0x55d056769330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 --2- 172.21.15.82:0/3308397294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056760e80 0x55d056768df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 --2- 172.21.15.82:0/3308397294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056775830 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 -- 172.21.15.82:0/3308397294 >> 172.21.15.82:0/3308397294 conn(0x55d05675aba0 msgr2=0x55d056759c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 -- 172.21.15.82:0/3308397294 shutdown_connections 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 -- 172.21.15.82:0/3308397294 wait complete. 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 -- start start 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 Processor -- start 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056772010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056760e80 0x55d056762f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056769a00 0x55d056763490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f95941b2ae0 con 0x55d0567604d0 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f95941b2410 con 0x55d056769a00 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95941af590 con 0x55d056760e80 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959bbc5740 10 client.0 _ll_register_callbacks cb 0x7f9584010cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056772010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.782+0000 7f959259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056772010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50118/0 (socket says 172.21.15.82:50118) 2024-04-21T08:49:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f959259c640 1 -- 172.21.15.82:0/2608724929 learned_addr learned my addr 172.21.15.82:0/2608724929 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f9592d9d640 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056769a00 0x55d056763490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f9591d9b640 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056760e80 0x55d056762f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f959259c640 1 -- 172.21.15.82:0/2608724929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056760e80 msgr2=0x55d056762f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f959259c640 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056760e80 0x55d056762f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f959259c640 1 -- 172.21.15.82:0/2608724929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056769a00 msgr2=0x55d056763490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f959259c640 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056769a00 0x55d056763490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f959259c640 1 -- 172.21.15.82:0/2608724929 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d056763b50 con 0x55d0567604d0 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f9592d9d640 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056769a00 0x55d056763490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f9591d9b640 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056760e80 0x55d056762f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f959259c640 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056772010 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f958400b4e0 tx=0x7f9584026710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3327aa0b157b7ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f9579ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f9579ffb640 1 -- 172.21.15.82:0/2608724929 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584036070 con 0x55d0567604d0 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f9579ffb640 1 -- 172.21.15.82:0/2608724929 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f95840049d0 con 0x55d0567604d0 2024-04-21T08:49:19.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f959bbc5740 1 -- 172.21.15.82:0/2608724929 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9584026a40 con 0x55d0567604d0 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f9579ffb640 1 -- 172.21.15.82:0/2608724929 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9584052050 con 0x55d0567604d0 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f959bbc5740 10 client.16770 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f959bbc5740 1 -- 172.21.15.82:0/2608724929 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0567dfea0 con 0x55d0567604d0 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.783+0000 7f9579ffb640 1 -- 172.21.15.82:0/2608724929 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9584026a40 con 0x55d0567604d0 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.784+0000 7f9579ffb640 1 -- 172.21.15.82:0/2608724929 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f958401ebc0 con 0x55d0567604d0 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.784+0000 7f959bbc5740 10 client.16770 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.784+0000 7f959bbc5740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.784+0000 7f959bbc5740 20 client.16770 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.784+0000 7f959bbc5740 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d056779bd0 0x55d0567e25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.784+0000 7f959bbc5740 4 client.16770 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.784+0000 7f959bbc5740 1 -- 172.21.15.82:0/2608724929 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0566d2c10 con 0x55d056779bd0 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.784+0000 7f9591d9b640 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d056779bd0 0x55d0567e25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.784+0000 7f9591d9b640 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d056779bd0 0x55d0567e25a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a183720cf35e0ff9 server_cookie=be2946b797216970 in_seq=0 out_seq=0 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.784+0000 7f9579ffb640 10 client.16770 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:19.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.784+0000 7f9579ffb640 1 -- 172.21.15.82:0/2608724929 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0566d2c10 con 0x55d056779bd0 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.784+0000 7f9579ffb640 10 client.16770 handle_command_reply: tid=0 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f959bbc5740 1 client.16770 shutdown 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f959bbc5740 1 -- 172.21.15.82:0/2608724929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 msgr2=0x55d056772010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f959bbc5740 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056772010 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f958400b4e0 tx=0x7f9584026710 comp rx=0 tx=0).stop 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f959bbc5740 1 -- 172.21.15.82:0/2608724929 shutdown_connections 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f959bbc5740 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d056779bd0 0x55d0567e25a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f959bbc5740 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d056769a00 0x55d056763490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f959bbc5740 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d056760e80 0x55d056762f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f9579ffb640 0 client.16770 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f959bbc5740 1 --2- 172.21.15.82:0/2608724929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0567604d0 0x55d056772010 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f959bbc5740 1 -- 172.21.15.82:0/2608724929 >> 172.21.15.82:0/2608724929 conn(0x55d05675aba0 msgr2=0x55d05675bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f959bbc5740 1 -- 172.21.15.82:0/2608724929 shutdown_connections 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f959bbc5740 1 -- 172.21.15.82:0/2608724929 wait complete. 2024-04-21T08:49:19.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f959bbc5740 20 client.16770 trim_cache size 0 max 16384 2024-04-21T08:49:19.793 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:49:19.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f9598fdb640 1 -- 172.21.15.82:0/2595529208 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f957c0a3d20 msgr2=0x7f957c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f9598fdb640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f957c0a3d20 0x7f957c0a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f958400bbe0 tx=0x7f958400b390 comp rx=0 tx=0).stop 2024-04-21T08:49:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f9598fdb640 1 -- 172.21.15.82:0/2595529208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959419a780 msgr2=0x7f959419ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f9598fdb640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959419a780 0x7f959419ac30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f958c00eb10 tx=0x7f958c00c2d0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f9598fdb640 1 -- 172.21.15.82:0/2595529208 shutdown_connections 2024-04-21T08:49:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f9598fdb640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f957c0a3d20 0x7f957c0a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f9598fdb640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959419a780 0x7f959419ac30 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f9598fdb640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9594106160 0x7f95941a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.785+0000 7f9598fdb640 1 --2- 172.21.15.82:0/2595529208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9594101090 0x7f95941a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.786+0000 7f9598fdb640 1 -- 172.21.15.82:0/2595529208 >> 172.21.15.82:0/2595529208 conn(0x7f95940771d0 msgr2=0x7f95940ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.786+0000 7f9598fdb640 1 -- 172.21.15.82:0/2595529208 shutdown_connections 2024-04-21T08:49:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:19.786+0000 7f9598fdb640 1 -- 172.21.15.82:0/2595529208 wait complete. 2024-04-21T08:49:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:20 smithi134 ceph-mon[31772]: pgmap v72: 129 pgs: 129 active+clean; 249 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-21T08:49:20.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:20 smithi082 ceph-mon[27967]: pgmap v72: 129 pgs: 129 active+clean; 249 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-21T08:49:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:20 smithi106 ceph-mon[31072]: pgmap v72: 129 pgs: 129 active+clean; 249 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-21T08:49:20.802 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:49:20.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:49:20.803 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:49:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.914+0000 7f19d9a06640 1 -- 172.21.15.82:0/2633982848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19d410e830 msgr2=0x7f19d4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.914+0000 7f19d9a06640 1 --2- 172.21.15.82:0/2633982848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19d410e830 0x7f19d4110ca0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009ca0 tx=0x7f19cc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.914+0000 7f19d9a06640 1 -- 172.21.15.82:0/2633982848 shutdown_connections 2024-04-21T08:49:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.914+0000 7f19d9a06640 1 --2- 172.21.15.82:0/2633982848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19d410e830 0x7f19d4110ca0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.914+0000 7f19d9a06640 1 --2- 172.21.15.82:0/2633982848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19d406aaa0 0x7f19d406aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.914+0000 7f19d9a06640 1 --2- 172.21.15.82:0/2633982848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19d4102f40 0x7f19d406a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.914+0000 7f19d9a06640 1 -- 172.21.15.82:0/2633982848 >> 172.21.15.82:0/2633982848 conn(0x7f19d4077e30 msgr2=0x7f19d4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.914+0000 7f19d9a06640 1 -- 172.21.15.82:0/2633982848 shutdown_connections 2024-04-21T08:49:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.914+0000 7f19d9a06640 1 -- 172.21.15.82:0/2633982848 wait complete. 2024-04-21T08:49:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.915+0000 7f19d9a06640 1 Processor -- start 2024-04-21T08:49:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.915+0000 7f19d9a06640 1 -- start start 2024-04-21T08:49:20.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.915+0000 7f19d9a06640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19d406aaa0 0x7f19d41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.915+0000 7f19d9a06640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19d4102f40 0x7f19d41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.915+0000 7f19d9a06640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19d410e830 0x7f19d419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.915+0000 7f19d9a06640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f19d4114470 con 0x7f19d4102f40 2024-04-21T08:49:20.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.915+0000 7f19d9a06640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41142f0 con 0x7f19d410e830 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.915+0000 7f19d9a06640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41145f0 con 0x7f19d406aaa0 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.915+0000 7f19ca7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19d4102f40 0x7f19d41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.915+0000 7f19ca7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19d4102f40 0x7f19d41a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50122/0 (socket says 172.21.15.82:50122) 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.915+0000 7f19ca7fc640 1 -- 172.21.15.82:0/542057655 learned_addr learned my addr 172.21.15.82:0/542057655 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.915+0000 7f19d37fe640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19d410e830 0x7f19d419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.916+0000 7f19d2ffd640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19d406aaa0 0x7f19d41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.916+0000 7f19ca7fc640 1 -- 172.21.15.82:0/542057655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19d406aaa0 msgr2=0x7f19d41a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.916+0000 7f19ca7fc640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19d406aaa0 0x7f19d41a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.916+0000 7f19ca7fc640 1 -- 172.21.15.82:0/542057655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19d410e830 msgr2=0x7f19d419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.916+0000 7f19ca7fc640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19d410e830 0x7f19d419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.916+0000 7f19ca7fc640 1 -- 172.21.15.82:0/542057655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d419b010 con 0x7f19d4102f40 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.916+0000 7f19ca7fc640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19d4102f40 0x7f19d41a0bd0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00c970 tx=0x7f19bc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.916+0000 7f19d8a04640 1 -- 172.21.15.82:0/542057655 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc007bb0 con 0x7f19d4102f40 2024-04-21T08:49:20.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.916+0000 7f19d9a06640 1 -- 172.21.15.82:0/542057655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d419b300 con 0x7f19d4102f40 2024-04-21T08:49:20.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.916+0000 7f19d8a04640 1 -- 172.21.15.82:0/542057655 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19bc007d50 con 0x7f19d4102f40 2024-04-21T08:49:20.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.916+0000 7f19d8a04640 1 -- 172.21.15.82:0/542057655 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00f450 con 0x7f19d4102f40 2024-04-21T08:49:20.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.916+0000 7f19d9a06640 1 -- 172.21.15.82:0/542057655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d41a7610 con 0x7f19d4102f40 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.918+0000 7f19d8a04640 1 -- 172.21.15.82:0/542057655 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f19bc004f60 con 0x7f19d4102f40 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.918+0000 7f19dc5f9740 1 Processor -- start 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19dc5f9740 1 -- start start 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19dc5f9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40de8e0 0x563bd40decc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19dc5f9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 0x563bd40e7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19dc5f9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40e7e10 0x563bd40ea260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19dc5f9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563bd4042f00 con 0x563bd40df290 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19dc5f9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563bd3fe0590 con 0x563bd40e7e10 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19dc5f9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563bd403eec0 con 0x563bd40de8e0 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19d2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 0x563bd40e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19d2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 0x563bd40e7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50124/0 (socket says 172.21.15.82:50124) 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19d2ffd640 1 -- 172.21.15.82:0/2816164971 learned_addr learned my addr 172.21.15.82:0/2816164971 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19ca7fc640 1 --2- 172.21.15.82:0/2816164971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40e7e10 0x563bd40ea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19d37fe640 1 --2- 172.21.15.82:0/2816164971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40de8e0 0x563bd40decc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19d2ffd640 1 -- 172.21.15.82:0/2816164971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40de8e0 msgr2=0x563bd40decc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19d2ffd640 1 --2- 172.21.15.82:0/2816164971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40de8e0 0x563bd40decc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19d2ffd640 1 -- 172.21.15.82:0/2816164971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40e7e10 msgr2=0x563bd40ea260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19d2ffd640 1 --2- 172.21.15.82:0/2816164971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40e7e10 0x563bd40ea260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19d2ffd640 1 -- 172.21.15.82:0/2816164971 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd3e1d7e0 con 0x563bd40df290 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19d8a04640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19c00a3da0 0x7f19c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19ca7fc640 1 --2- 172.21.15.82:0/2816164971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40e7e10 0x563bd40ea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19d37fe640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19c00a3da0 0x7f19c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.919+0000 7f19d2ffd640 1 --2- 172.21.15.82:0/2816164971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 0x563bd40e7840 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009cf0 tx=0x7f19c4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972bfd35b54c3a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19d8a04640 1 -- 172.21.15.82:0/542057655 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f19bc0d0800 con 0x7f19d4102f40 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19c9ffb640 1 -- 172.21.15.82:0/2816164971 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4004230 con 0x563bd40df290 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19c9ffb640 1 -- 172.21.15.82:0/2816164971 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19c40043d0 con 0x563bd40df290 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19c9ffb640 1 -- 172.21.15.82:0/2816164971 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4019880 con 0x563bd40df290 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2816164971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 msgr2=0x563bd40e7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:20.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19d37fe640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19c00a3da0 0x7f19c00a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19cc004af0 tx=0x7f19cc005c50 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2816164971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 0x563bd40e7840 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009cf0 tx=0x7f19c4011810 comp rx=0 tx=0).stop 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2816164971 shutdown_connections 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2816164971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40e7e10 0x563bd40ea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2816164971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 0x563bd40e7840 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2816164971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40de8e0 0x563bd40decc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2816164971 >> 172.21.15.82:0/2816164971 conn(0x563bd40d8fb0 msgr2=0x563bd40d9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2816164971 shutdown_connections 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.920+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2816164971 wait complete. 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19dc5f9740 1 -- start start 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19dc5f9740 1 Processor -- start 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19dc5f9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40de8e0 0x563bd40e4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19dc5f9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 0x563bd40e5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19dc5f9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40e7e10 0x563bd40e5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19dc5f9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563bd40ead80 con 0x563bd40df290 2024-04-21T08:49:20.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19dc5f9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563bd3ffc580 con 0x563bd40de8e0 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19dc5f9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563bd40eaf00 con 0x563bd40e7e10 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19dc5f9740 10 client.0 _ll_register_callbacks cb 0x563bd4002650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19d2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40de8e0 0x563bd40e4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19d2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40de8e0 0x563bd40e4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:60782/0 (socket says 172.21.15.82:60782) 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19d2ffd640 1 -- 172.21.15.82:0/2099345976 learned_addr learned my addr 172.21.15.82:0/2099345976 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.921+0000 7f19ca7fc640 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 0x563bd40e5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19d37fe640 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40e7e10 0x563bd40e5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19d2ffd640 1 -- 172.21.15.82:0/2099345976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40e7e10 msgr2=0x563bd40e5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19d2ffd640 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40e7e10 0x563bd40e5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19d2ffd640 1 -- 172.21.15.82:0/2099345976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 msgr2=0x563bd40e5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19d2ffd640 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 0x563bd40e5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19d2ffd640 1 -- 172.21.15.82:0/2099345976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd3e7df70 con 0x563bd40de8e0 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19d37fe640 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40e7e10 0x563bd40e5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19ca7fc640 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 0x563bd40e5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19d2ffd640 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40de8e0 0x563bd40e4dc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f19c401a000 tx=0x7f19c40038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a7383c261d14c91 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19b2ffd640 1 -- 172.21.15.82:0/2099345976 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4027070 con 0x563bd40de8e0 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19b2ffd640 1 -- 172.21.15.82:0/2099345976 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f19c4005240 con 0x563bd40de8e0 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2099345976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19c40043d0 con 0x563bd40de8e0 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19b2ffd640 1 -- 172.21.15.82:0/2099345976 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f19c4033050 con 0x563bd40de8e0 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19dc5f9740 10 client.26126 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2099345976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563bd3d15170 con 0x563bd40de8e0 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.922+0000 7f19b2ffd640 1 -- 172.21.15.82:0/2099345976 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19c40043d0 con 0x563bd40de8e0 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.923+0000 7f19b2ffd640 1 -- 172.21.15.82:0/2099345976 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f19c4004b70 con 0x563bd40de8e0 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.923+0000 7f19dc5f9740 10 client.26126 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.923+0000 7f19dc5f9740 10 client.26126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.923+0000 7f19dc5f9740 20 client.26126 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.923+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563bd40f8260 0x563bd40fa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.923+0000 7f19dc5f9740 4 client.26126 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.923+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2099345976 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bd4050920 con 0x563bd40f8260 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.923+0000 7f19ca7fc640 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563bd40f8260 0x563bd40fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.923+0000 7f19b2ffd640 10 client.26126 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.923+0000 7f19ca7fc640 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563bd40f8260 0x563bd40fa650 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=921528b28df37c7c server_cookie=4c971a347617c5d6 in_seq=0 out_seq=0 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19b2ffd640 1 -- 172.21.15.82:0/2099345976 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563bd4050920 con 0x563bd40f8260 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19b2ffd640 10 client.26126 handle_command_reply: tid=0 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19dc5f9740 1 client.26126 shutdown 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2099345976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40de8e0 msgr2=0x563bd40e4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40de8e0 0x563bd40e4dc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f19c401a000 tx=0x7f19c40038a0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2099345976 shutdown_connections 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563bd40f8260 0x563bd40fa650 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40e7e10 0x563bd40e5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40df290 0x563bd40e5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19b2ffd640 0 client.26126 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:20.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2099345976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40de8e0 0x563bd40e4dc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2099345976 >> 172.21.15.82:0/2099345976 conn(0x563bd40d8fb0 msgr2=0x563bd40dad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:20.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2099345976 shutdown_connections 2024-04-21T08:49:20.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2099345976 wait complete. 2024-04-21T08:49:20.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.924+0000 7f19dc5f9740 20 client.26126 trim_cache size 0 max 16384 2024-04-21T08:49:20.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19dc5f9740 1 Processor -- start 2024-04-21T08:49:20.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19dc5f9740 1 -- start start 2024-04-21T08:49:20.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19dc5f9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40de8e0 0x563bd40f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19dc5f9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40df290 0x563bd40e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19dc5f9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 0x563bd40dab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19dc5f9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41b1560 con 0x563bd40e7e10 2024-04-21T08:49:20.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19dc5f9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41b3190 con 0x563bd40df290 2024-04-21T08:49:20.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19dc5f9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41b1c30 con 0x563bd40de8e0 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19d37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 0x563bd40dab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19ca7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40df290 0x563bd40e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19d37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 0x563bd40dab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50138/0 (socket says 172.21.15.82:50138) 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19d37fe640 1 -- 172.21.15.82:0/2325973574 learned_addr learned my addr 172.21.15.82:0/2325973574 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19d2ffd640 1 --2- 172.21.15.82:0/2325973574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40de8e0 0x563bd40f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19d37fe640 1 -- 172.21.15.82:0/2325973574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40de8e0 msgr2=0x563bd40f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19d37fe640 1 --2- 172.21.15.82:0/2325973574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40de8e0 0x563bd40f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19d37fe640 1 -- 172.21.15.82:0/2325973574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40df290 msgr2=0x563bd40e46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19d37fe640 1 --2- 172.21.15.82:0/2325973574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40df290 0x563bd40e46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19d37fe640 1 -- 172.21.15.82:0/2325973574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd40e3310 con 0x563bd40e7e10 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19d37fe640 1 --2- 172.21.15.82:0/2325973574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 0x563bd40dab10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009ca0 tx=0x7f19cc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81c308124fb13fa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.944+0000 7f19c97fa640 1 -- 172.21.15.82:0/2325973574 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc051070 con 0x563bd40e7e10 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.945+0000 7f19c97fa640 1 -- 172.21.15.82:0/2325973574 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19cc032940 con 0x563bd40e7e10 2024-04-21T08:49:20.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.945+0000 7f19c97fa640 1 -- 172.21.15.82:0/2325973574 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc039040 con 0x563bd40e7e10 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.945+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2325973574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 msgr2=0x563bd40dab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.945+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2325973574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 0x563bd40dab10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009ca0 tx=0x7f19cc041000 comp rx=0 tx=0).stop 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.945+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2325973574 shutdown_connections 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.945+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2325973574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 0x563bd40dab10 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.945+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2325973574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40df290 0x563bd40e46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.945+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2325973574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40de8e0 0x563bd40f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.945+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2325973574 >> 172.21.15.82:0/2325973574 conn(0x563bd40d8fb0 msgr2=0x563bd40da400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.945+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2325973574 shutdown_connections 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.945+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2325973574 wait complete. 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.945+0000 7f19dc5f9740 1 -- start start 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19dc5f9740 1 Processor -- start 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19dc5f9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40de8e0 0x563bd40ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19dc5f9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40df290 0x563bd40ec470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19dc5f9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 0x563bd415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19dc5f9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41b2a90 con 0x563bd40e7e10 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19dc5f9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41b23c0 con 0x563bd40de8e0 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19dc5f9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41af540 con 0x563bd40df290 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19dc5f9740 10 client.0 _ll_register_callbacks cb 0x7f19bc0d6420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19d37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 0x563bd415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19d2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40de8e0 0x563bd40ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19d37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 0x563bd415d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50150/0 (socket says 172.21.15.82:50150) 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19d37fe640 1 -- 172.21.15.82:0/2766371940 learned_addr learned my addr 172.21.15.82:0/2766371940 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19ca7fc640 1 --2- 172.21.15.82:0/2766371940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40df290 0x563bd40ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19d37fe640 1 -- 172.21.15.82:0/2766371940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40df290 msgr2=0x563bd40ec470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19d37fe640 1 --2- 172.21.15.82:0/2766371940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40df290 0x563bd40ec470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19d37fe640 1 -- 172.21.15.82:0/2766371940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40de8e0 msgr2=0x563bd40ebf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19d37fe640 1 --2- 172.21.15.82:0/2766371940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40de8e0 0x563bd40ebf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19d37fe640 1 -- 172.21.15.82:0/2766371940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd40ecd40 con 0x563bd40e7e10 2024-04-21T08:49:20.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19d37fe640 1 --2- 172.21.15.82:0/2766371940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 0x563bd415d6d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f19cc02c630 tx=0x7f19cc003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6714b75a65d201a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.946+0000 7f19b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19b2ffd640 1 -- 172.21.15.82:0/2766371940 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc039070 con 0x563bd40e7e10 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19b2ffd640 1 -- 172.21.15.82:0/2766371940 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f19cc004450 con 0x563bd40e7e10 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2766371940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19cc032940 con 0x563bd40e7e10 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19b2ffd640 1 -- 172.21.15.82:0/2766371940 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f19cc067050 con 0x563bd40e7e10 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19dc5f9740 10 client.16788 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2766371940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563bd415ed50 con 0x563bd40e7e10 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19b2ffd640 1 -- 172.21.15.82:0/2766371940 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19cc032940 con 0x563bd40e7e10 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19b2ffd640 1 -- 172.21.15.82:0/2766371940 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f19cc04dcc0 con 0x563bd40e7e10 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19dc5f9740 10 client.16788 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19dc5f9740 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19dc5f9740 20 client.16788 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2766371940 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563bd415f440 0x563bd4161860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19dc5f9740 4 client.16788 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:49:20.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2766371940 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563bd404a790 con 0x563bd415f440 2024-04-21T08:49:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.947+0000 7f19d2ffd640 1 --2- 172.21.15.82:0/2766371940 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563bd415f440 0x563bd4161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.948+0000 7f19b2ffd640 10 client.16788 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:20.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.948+0000 7f19d2ffd640 1 --2- 172.21.15.82:0/2766371940 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563bd415f440 0x563bd4161860 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd52427511e62d3 server_cookie=d886914973420a56 in_seq=0 out_seq=0 2024-04-21T08:49:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19b2ffd640 1 -- 172.21.15.82:0/2766371940 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563bd404a790 con 0x563bd415f440 2024-04-21T08:49:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19b2ffd640 10 client.16788 handle_command_reply: tid=0 2024-04-21T08:49:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19dc5f9740 1 client.16788 shutdown 2024-04-21T08:49:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2766371940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 msgr2=0x563bd415d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2766371940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 0x563bd415d6d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f19cc02c630 tx=0x7f19cc003e50 comp rx=0 tx=0).stop 2024-04-21T08:49:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2766371940 shutdown_connections 2024-04-21T08:49:20.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2766371940 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563bd415f440 0x563bd4161860 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2766371940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bd40e7e10 0x563bd415d6d0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2766371940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bd40df290 0x563bd40ec470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19dc5f9740 1 --2- 172.21.15.82:0/2766371940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bd40de8e0 0x563bd40ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19b2ffd640 0 client.16788 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2766371940 >> 172.21.15.82:0/2766371940 conn(0x563bd40d8fb0 msgr2=0x563bd40dac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2766371940 shutdown_connections 2024-04-21T08:49:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.949+0000 7f19dc5f9740 1 -- 172.21.15.82:0/2766371940 wait complete. 2024-04-21T08:49:20.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19dc5f9740 20 client.16788 trim_cache size 0 max 16384 2024-04-21T08:49:20.956 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:49:20.956 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:49:20.956 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "480a3453-9908-4a12-b55d-cbecc05557f7", 2024-04-21T08:49:20.957 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:49:20.957 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:49:20.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19d9a06640 1 -- 172.21.15.82:0/542057655 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19c00a3da0 msgr2=0x7f19c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:20.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19d9a06640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19c00a3da0 0x7f19c00a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19cc004af0 tx=0x7f19cc005c50 comp rx=0 tx=0).stop 2024-04-21T08:49:20.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19d9a06640 1 -- 172.21.15.82:0/542057655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19d4102f40 msgr2=0x7f19d41a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:20.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19d9a06640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19d4102f40 0x7f19d41a0bd0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00c970 tx=0x7f19bc00ce30 comp rx=0 tx=0).stop 2024-04-21T08:49:20.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19d9a06640 1 -- 172.21.15.82:0/542057655 shutdown_connections 2024-04-21T08:49:20.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19d9a06640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19d410e830 0x7f19d419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19d9a06640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19c00a3da0 0x7f19c00a6260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19d9a06640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19d4102f40 0x7f19d41a0bd0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19d9a06640 1 --2- 172.21.15.82:0/542057655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19d406aaa0 0x7f19d41a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:20.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19d9a06640 1 -- 172.21.15.82:0/542057655 >> 172.21.15.82:0/542057655 conn(0x7f19d4077e30 msgr2=0x7f19d410f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:20.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19d9a06640 1 -- 172.21.15.82:0/542057655 shutdown_connections 2024-04-21T08:49:20.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:20.950+0000 7f19d9a06640 1 -- 172.21.15.82:0/542057655 wait complete. 2024-04-21T08:49:20.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:49:20.962 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:49:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.071+0000 7f301c2ba640 1 -- 172.21.15.82:0/1205596954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3014101a30 msgr2=0x7f3014105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.071+0000 7f301c2ba640 1 --2- 172.21.15.82:0/1205596954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3014101a30 0x7f3014105a70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3008009900 tx=0x7f300802bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.071+0000 7f301c2ba640 1 -- 172.21.15.82:0/1205596954 shutdown_connections 2024-04-21T08:49:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.071+0000 7f301c2ba640 1 --2- 172.21.15.82:0/1205596954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3014106150 0x7f301410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.071+0000 7f301c2ba640 1 --2- 172.21.15.82:0/1205596954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3014101a30 0x7f3014105a70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.071+0000 7f301c2ba640 1 --2- 172.21.15.82:0/1205596954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3014101080 0x7f3014101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301c2ba640 1 -- 172.21.15.82:0/1205596954 >> 172.21.15.82:0/1205596954 conn(0x7f3014078000 msgr2=0x7f3014078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301c2ba640 1 -- 172.21.15.82:0/1205596954 shutdown_connections 2024-04-21T08:49:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301c2ba640 1 -- 172.21.15.82:0/1205596954 wait complete. 2024-04-21T08:49:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301c2ba640 1 Processor -- start 2024-04-21T08:49:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301c2ba640 1 -- start start 2024-04-21T08:49:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301c2ba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3014101080 0x7f3014198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301c2ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3014106150 0x7f3014198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301c2ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f301419cff0 0x7f301419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301c2ba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f30141110c0 con 0x7f3014101080 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301c2ba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3014110f40 con 0x7f301419cff0 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301c2ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3014111240 con 0x7f3014106150 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301a056640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3014101080 0x7f3014198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.072+0000 7f301a056640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3014101080 0x7f3014198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50168/0 (socket says 172.21.15.82:50168) 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f301a056640 1 -- 172.21.15.82:0/614279417 learned_addr learned my addr 172.21.15.82:0/614279417 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f301a857640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f301419cff0 0x7f301419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f3019855640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3014106150 0x7f3014198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f301a056640 1 -- 172.21.15.82:0/614279417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3014106150 msgr2=0x7f3014198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f301a056640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3014106150 0x7f3014198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f301a056640 1 -- 172.21.15.82:0/614279417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f301419cff0 msgr2=0x7f301419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f301a056640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f301419cff0 0x7f301419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f301a056640 1 -- 172.21.15.82:0/614279417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301419dae0 con 0x7f3014101080 2024-04-21T08:49:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f3019855640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3014106150 0x7f3014198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f301a056640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3014101080 0x7f3014198730 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f300c00ebc0 tx=0x7f300c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f30077fe640 1 -- 172.21.15.82:0/614279417 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c010630 con 0x7f3014101080 2024-04-21T08:49:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f30077fe640 1 -- 172.21.15.82:0/614279417 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f300c010c80 con 0x7f3014101080 2024-04-21T08:49:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f301c2ba640 1 -- 172.21.15.82:0/614279417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301419ddd0 con 0x7f3014101080 2024-04-21T08:49:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f30077fe640 1 -- 172.21.15.82:0/614279417 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c018b60 con 0x7f3014101080 2024-04-21T08:49:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.073+0000 7f301c2ba640 1 -- 172.21.15.82:0/614279417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30141a5610 con 0x7f3014101080 2024-04-21T08:49:21.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.075+0000 7f30077fe640 1 -- 172.21.15.82:0/614279417 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f300c018d00 con 0x7f3014101080 2024-04-21T08:49:21.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301ef87740 1 Processor -- start 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301ef87740 1 -- start start 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301ef87740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8e1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301ef87740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8ea540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f30077fe640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f30000a3da0 0x7f30000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301ef87740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8ece70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301ef87740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff8461e0 con 0x55f3ff8e1580 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301ef87740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff7e3390 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301ef87740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff8421a0 con 0x55f3ff8eaa80 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301a857640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8e1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301a056640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f30000a3da0 0x7f30000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f3019855640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8ea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301a857640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8e1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50174/0 (socket says 172.21.15.82:50174) 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301a857640 1 -- 172.21.15.82:0/2659112626 learned_addr learned my addr 172.21.15.82:0/2659112626 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:21.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f30077fe640 1 -- 172.21.15.82:0/614279417 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f300c0d11c0 con 0x7f3014101080 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.076+0000 7f301a857640 1 --2- 172.21.15.82:0/2659112626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8ece70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.077+0000 7f3019855640 1 -- 172.21.15.82:0/2659112626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 msgr2=0x55f3ff8ece70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.077+0000 7f3019855640 1 --2- 172.21.15.82:0/2659112626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8ece70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.077+0000 7f3019855640 1 -- 172.21.15.82:0/2659112626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 msgr2=0x55f3ff8e1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.077+0000 7f3019855640 1 --2- 172.21.15.82:0/2659112626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8e1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.077+0000 7f3019855640 1 -- 172.21.15.82:0/2659112626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ff6227d0 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.077+0000 7f301a857640 1 --2- 172.21.15.82:0/2659112626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8e1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.077+0000 7f301a056640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f30000a3da0 0x7f30000a6260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f300c00cba0 tx=0x7f300c092230 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.077+0000 7f3019855640 1 --2- 172.21.15.82:0/2659112626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8ea540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3008036040 tx=0x7f3008005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed5caf050a9b37 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.077+0000 7f3004bfe640 1 -- 172.21.15.82:0/2659112626 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008043020 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.077+0000 7f3004bfe640 1 -- 172.21.15.82:0/2659112626 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f300802c7e0 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.077+0000 7f3004bfe640 1 -- 172.21.15.82:0/2659112626 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30080055d0 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.078+0000 7f301ef87740 1 -- 172.21.15.82:0/2659112626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 msgr2=0x55f3ff8ea540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.078+0000 7f301ef87740 1 --2- 172.21.15.82:0/2659112626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8ea540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3008036040 tx=0x7f3008005e80 comp rx=0 tx=0).stop 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.078+0000 7f301ef87740 1 -- 172.21.15.82:0/2659112626 shutdown_connections 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.078+0000 7f301ef87740 1 --2- 172.21.15.82:0/2659112626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8ece70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.078+0000 7f301ef87740 1 --2- 172.21.15.82:0/2659112626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8ea540 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.078+0000 7f301ef87740 1 --2- 172.21.15.82:0/2659112626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8e1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.078+0000 7f301ef87740 1 -- 172.21.15.82:0/2659112626 >> 172.21.15.82:0/2659112626 conn(0x55f3ff8dbcb0 msgr2=0x55f3ff8dc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.078+0000 7f301ef87740 1 -- 172.21.15.82:0/2659112626 shutdown_connections 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.078+0000 7f301ef87740 1 -- 172.21.15.82:0/2659112626 wait complete. 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.078+0000 7f301ef87740 1 -- start start 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f301ef87740 1 Processor -- start 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f301ef87740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1580 0x55f3ff8e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f301ef87740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f301ef87740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f301ef87740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff8ed810 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f301ef87740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff7ff2f0 con 0x55f3ff8e1580 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f301ef87740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff8ed990 con 0x55f3ff8eaa80 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f301ef87740 10 client.0 _ll_register_callbacks cb 0x55f3ff80c580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f3019855640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f3019855640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8e8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50180/0 (socket says 172.21.15.82:50180) 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f3019855640 1 -- 172.21.15.82:0/2025369952 learned_addr learned my addr 172.21.15.82:0/2025369952 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f301a857640 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1580 0x55f3ff8e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f301a857640 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f3019855640 1 -- 172.21.15.82:0/2025369952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 msgr2=0x55f3ff8e8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f3019855640 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8e8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f3019855640 1 -- 172.21.15.82:0/2025369952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1580 msgr2=0x55f3ff8e7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f3019855640 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1580 0x55f3ff8e7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f3019855640 1 -- 172.21.15.82:0/2025369952 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ff5583f0 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f301a857640 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f3019855640 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8e8040 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f300802cc40 tx=0x7f300802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e2cde8ccfa7a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f2ffdffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.079+0000 7f2ffdffb640 1 -- 172.21.15.82:0/2025369952 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300804c070 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f2ffdffb640 1 -- 172.21.15.82:0/2025369952 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f3008004460 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f301ef87740 1 -- 172.21.15.82:0/2025369952 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f300802c7e0 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f2ffdffb640 1 -- 172.21.15.82:0/2025369952 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3008043040 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f301ef87740 10 client.16800 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f301ef87740 1 -- 172.21.15.82:0/2025369952 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f3ff5180f0 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f2ffdffb640 1 -- 172.21.15.82:0/2025369952 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f300802c7e0 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f2ffdffb640 1 -- 172.21.15.82:0/2025369952 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f3008004960 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f301ef87740 10 client.16800 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f301ef87740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f301ef87740 20 client.16800 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f301ef87740 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f3ff8fa8f0 0x55f3ff8fcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f301ef87740 4 client.16800 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f301ef87740 1 -- 172.21.15.82:0/2025369952 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3ff853c00 con 0x55f3ff8fa8f0 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.080+0000 7f301a857640 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f3ff8fa8f0 0x55f3ff8fcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.081+0000 7f301a857640 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f3ff8fa8f0 0x55f3ff8fcce0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8c1cc8d93b55ab3 server_cookie=666b0b4b61ba55fa in_seq=0 out_seq=0 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.081+0000 7f2ffdffb640 10 client.16800 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.081+0000 7f2ffdffb640 1 -- 172.21.15.82:0/2025369952 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3ff853c00 con 0x55f3ff8fa8f0 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.081+0000 7f2ffdffb640 10 client.16800 handle_command_reply: tid=0 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.081+0000 7f301ef87740 1 client.16800 shutdown 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.081+0000 7f301ef87740 1 -- 172.21.15.82:0/2025369952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1ea0 msgr2=0x55f3ff8e8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.081+0000 7f301ef87740 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8e8040 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f300802cc40 tx=0x7f300802cc70 comp rx=0 tx=0).stop 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.082+0000 7f301ef87740 1 -- 172.21.15.82:0/2025369952 shutdown_connections 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.082+0000 7f301ef87740 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f3ff8fa8f0 0x55f3ff8fcce0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.082+0000 7f301ef87740 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.082+0000 7f301ef87740 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8e8040 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.082+0000 7f2ffdffb640 0 client.16800 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:21.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.082+0000 7f301ef87740 1 --2- 172.21.15.82:0/2025369952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1580 0x55f3ff8e7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.082+0000 7f301ef87740 1 -- 172.21.15.82:0/2025369952 >> 172.21.15.82:0/2025369952 conn(0x55f3ff8dbcb0 msgr2=0x55f3ff8dda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:21.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.082+0000 7f301ef87740 1 -- 172.21.15.82:0/2025369952 shutdown_connections 2024-04-21T08:49:21.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.082+0000 7f301ef87740 1 -- 172.21.15.82:0/2025369952 wait complete. 2024-04-21T08:49:21.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.082+0000 7f301ef87740 20 client.16800 trim_cache size 0 max 16384 2024-04-21T08:49:21.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.100+0000 7f301ef87740 1 Processor -- start 2024-04-21T08:49:21.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.100+0000 7f301ef87740 1 -- start start 2024-04-21T08:49:21.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.100+0000 7f301ef87740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8ea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.100+0000 7f301ef87740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301ef87740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301ef87740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f30141af3a0 con 0x55f3ff8e1580 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301ef87740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f30141b0fd0 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301ef87740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f30141afa70 con 0x55f3ff8eaa80 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301a857640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301a857640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8ea230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50194/0 (socket says 172.21.15.82:50194) 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301a857640 1 -- 172.21.15.82:0/3148975688 learned_addr learned my addr 172.21.15.82:0/3148975688 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f3019855640 1 --2- 172.21.15.82:0/3148975688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301a857640 1 -- 172.21.15.82:0/3148975688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 msgr2=0x55f3ff8f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301a857640 1 --2- 172.21.15.82:0/3148975688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301a857640 1 -- 172.21.15.82:0/3148975688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 msgr2=0x55f3ff8e6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301a857640 1 --2- 172.21.15.82:0/3148975688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8e6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301a857640 1 -- 172.21.15.82:0/3148975688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ff8e1cd0 con 0x55f3ff8e1580 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f301a857640 1 --2- 172.21.15.82:0/3148975688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8ea230 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x55f3ff8fd280 tx=0x7f301001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7586b7d96907322b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f2ffffff640 1 -- 172.21.15.82:0/3148975688 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010012aa0 con 0x55f3ff8e1580 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f2ffffff640 1 -- 172.21.15.82:0/3148975688 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3010027070 con 0x55f3ff8e1580 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.101+0000 7f2ffffff640 1 -- 172.21.15.82:0/3148975688 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010004a00 con 0x55f3ff8e1580 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 -- 172.21.15.82:0/3148975688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 msgr2=0x55f3ff8ea230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 --2- 172.21.15.82:0/3148975688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8ea230 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x55f3ff8fd280 tx=0x7f301001b040 comp rx=0 tx=0).stop 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 -- 172.21.15.82:0/3148975688 shutdown_connections 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 --2- 172.21.15.82:0/3148975688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 --2- 172.21.15.82:0/3148975688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8e6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 --2- 172.21.15.82:0/3148975688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8ea230 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 -- 172.21.15.82:0/3148975688 >> 172.21.15.82:0/3148975688 conn(0x55f3ff8dbcb0 msgr2=0x55f3ff8e8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 -- 172.21.15.82:0/3148975688 shutdown_connections 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 -- 172.21.15.82:0/3148975688 wait complete. 2024-04-21T08:49:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 -- start start 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 Processor -- start 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8f3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f30141b08d0 con 0x55f3ff8e1580 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f30141b0200 con 0x55f3ff8e1ea0 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f30141ad380 con 0x55f3ff8eaa80 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.102+0000 7f301ef87740 10 client.0 _ll_register_callbacks cb 0x7f3010004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301a857640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301a857640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8f2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50202/0 (socket says 172.21.15.82:50202) 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301a857640 1 -- 172.21.15.82:0/4255210072 learned_addr learned my addr 172.21.15.82:0/4255210072 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f3019855640 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301a857640 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301a857640 1 -- 172.21.15.82:0/4255210072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 msgr2=0x55f3ff8efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301a857640 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8efbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301a857640 1 -- 172.21.15.82:0/4255210072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 msgr2=0x55f3ff8f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301a857640 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301a857640 1 -- 172.21.15.82:0/4255210072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ff8f0240 con 0x55f3ff8e1580 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f3019855640 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301a857640 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8f2be0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f301000bba0 tx=0x7f301000bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b4bf45c6222c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f2ffdffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f2ffdffb640 1 -- 172.21.15.82:0/4255210072 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010005230 con 0x55f3ff8e1580 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f2ffdffb640 1 -- 172.21.15.82:0/4255210072 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f30100053d0 con 0x55f3ff8e1580 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301ef87740 1 -- 172.21.15.82:0/4255210072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3010027070 con 0x55f3ff8e1580 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f2ffdffb640 1 -- 172.21.15.82:0/4255210072 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f30100055d0 con 0x55f3ff8e1580 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301ef87740 10 client.16806 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f301ef87740 1 -- 172.21.15.82:0/4255210072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f3ff965900 con 0x55f3ff8e1580 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.103+0000 7f2ffdffb640 1 -- 172.21.15.82:0/4255210072 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3010027070 con 0x55f3ff8e1580 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f2ffdffb640 1 -- 172.21.15.82:0/4255210072 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f301002b7b0 con 0x55f3ff8e1580 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f301ef87740 10 client.16806 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f301ef87740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f301ef87740 20 client.16806 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f301ef87740 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f3ff965fd0 0x55f3ff9683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f301ef87740 4 client.16806 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f301ef87740 1 -- 172.21.15.82:0/4255210072 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3ff84da70 con 0x55f3ff965fd0 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f3019855640 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f3ff965fd0 0x55f3ff9683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f2ffdffb640 10 client.16806 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f3019855640 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f3ff965fd0 0x55f3ff9683f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa8b2ae503305152 server_cookie=2e1e6682633f1bd4 in_seq=0 out_seq=0 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f2ffdffb640 1 -- 172.21.15.82:0/4255210072 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55f3ff84da70 con 0x55f3ff965fd0 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f2ffdffb640 10 client.16806 handle_command_reply: tid=0 2024-04-21T08:49:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f301ef87740 1 client.16806 shutdown 2024-04-21T08:49:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f301ef87740 1 -- 172.21.15.82:0/4255210072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 msgr2=0x55f3ff8f2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.104+0000 7f301ef87740 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8f2be0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f301000bba0 tx=0x7f301000bb30 comp rx=0 tx=0).stop 2024-04-21T08:49:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301ef87740 1 -- 172.21.15.82:0/4255210072 shutdown_connections 2024-04-21T08:49:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301ef87740 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f3ff965fd0 0x55f3ff9683f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301ef87740 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3ff8eaa80 0x55f3ff8efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301ef87740 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ff8e1ea0 0x55f3ff8f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f2ffdffb640 0 client.16806 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301ef87740 1 --2- 172.21.15.82:0/4255210072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f3ff8e1580 0x55f3ff8f2be0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301ef87740 1 -- 172.21.15.82:0/4255210072 >> 172.21.15.82:0/4255210072 conn(0x55f3ff8dbcb0 msgr2=0x55f3ff8e7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301ef87740 1 -- 172.21.15.82:0/4255210072 shutdown_connections 2024-04-21T08:49:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301ef87740 1 -- 172.21.15.82:0/4255210072 wait complete. 2024-04-21T08:49:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301ef87740 20 client.16806 trim_cache size 0 max 16384 2024-04-21T08:49:21.113 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:49:21.113 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "scrub active (1 inodes in the stack)", 2024-04-21T08:49:21.113 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:49:21.113 INFO:teuthology.orchestra.run.smithi082.stdout: "480a3453-9908-4a12-b55d-cbecc05557f7": { 2024-04-21T08:49:21.113 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:49:21.113 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "480a3453-9908-4a12-b55d-cbecc05557f7", 2024-04-21T08:49:21.113 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:49:21.113 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:49:21.113 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:49:21.113 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:49:21.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301c2ba640 1 -- 172.21.15.82:0/614279417 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f30000a3da0 msgr2=0x7f30000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301c2ba640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f30000a3da0 0x7f30000a6260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f300c00cba0 tx=0x7f300c092230 comp rx=0 tx=0).stop 2024-04-21T08:49:21.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301c2ba640 1 -- 172.21.15.82:0/614279417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3014101080 msgr2=0x7f3014198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301c2ba640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3014101080 0x7f3014198730 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f300c00ebc0 tx=0x7f300c00c6a0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301c2ba640 1 -- 172.21.15.82:0/614279417 shutdown_connections 2024-04-21T08:49:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301c2ba640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f301419cff0 0x7f301419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301c2ba640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3014106150 0x7f3014198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301c2ba640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f30000a3da0 0x7f30000a6260 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301c2ba640 1 --2- 172.21.15.82:0/614279417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3014101080 0x7f3014198730 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301c2ba640 1 -- 172.21.15.82:0/614279417 >> 172.21.15.82:0/614279417 conn(0x7f3014078000 msgr2=0x7f30140694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301c2ba640 1 -- 172.21.15.82:0/614279417 shutdown_connections 2024-04-21T08:49:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:21.105+0000 7f301c2ba640 1 -- 172.21.15.82:0/614279417 wait complete. 2024-04-21T08:49:21.117 INFO:tasks.cephfs.filesystem:scrub status for tag:480a3453-9908-4a12-b55d-cbecc05557f7 - {'path': '/', 'tag': '480a3453-9908-4a12-b55d-cbecc05557f7', 'options': 'recursive,force'} 2024-04-21T08:49:22.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: pgmap v73: 129 pgs: 129 active+clean; 249 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-04-21T08:49:22.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:49:22.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:49:22.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:49:22.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:22 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: pgmap v73: 129 pgs: 129 active+clean; 249 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-04-21T08:49:22.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:49:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:49:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:49:22.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:22 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: pgmap v73: 129 pgs: 129 active+clean; 249 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-04-21T08:49:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:49:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:49:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:49:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:49:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:49:22.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:22 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:23.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:23 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:23.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:23 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:23.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:23 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.060+0000 7f1122784640 1 -- 172.21.15.82:0/413947260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c106170 msgr2=0x7f111c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.060+0000 7f1122784640 1 --2- 172.21.15.82:0/413947260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c106170 0x7f111c10dd40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f1114009ca0 tx=0x7f111402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.060+0000 7f1122784640 1 -- 172.21.15.82:0/413947260 shutdown_connections 2024-04-21T08:49:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.060+0000 7f1122784640 1 --2- 172.21.15.82:0/413947260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c106170 0x7f111c10dd40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.060+0000 7f1122784640 1 --2- 172.21.15.82:0/413947260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f111c101aa0 0x7f111c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.060+0000 7f1122784640 1 --2- 172.21.15.82:0/413947260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f111c1010f0 0x7f111c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.060+0000 7f1122784640 1 -- 172.21.15.82:0/413947260 >> 172.21.15.82:0/413947260 conn(0x7f111c077fd0 msgr2=0x7f111c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.060+0000 7f1122784640 1 -- 172.21.15.82:0/413947260 shutdown_connections 2024-04-21T08:49:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.060+0000 7f1122784640 1 -- 172.21.15.82:0/413947260 wait complete. 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.060+0000 7f1122784640 1 Processor -- start 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.060+0000 7f1122784640 1 -- start start 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f1122784640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1010f0 0x7f111c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f1122784640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f111c106170 0x7f111c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f1122784640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f111c19a6c0 0x7f111c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f1122784640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f111c10ffd0 con 0x7f111c1010f0 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f1122784640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f111c10fe50 con 0x7f111c19a6c0 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f1122784640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f111c110150 con 0x7f111c106170 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f111bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1010f0 0x7f111c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f111bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1010f0 0x7f111c1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54346/0 (socket says 172.21.15.82:54346) 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f111bfff640 1 -- 172.21.15.82:0/446672248 learned_addr learned my addr 172.21.15.82:0/446672248 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f1120d21640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f111c19a6c0 0x7f111c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f111b7fe640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f111c106170 0x7f111c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f111bfff640 1 -- 172.21.15.82:0/446672248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f111c106170 msgr2=0x7f111c1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f111bfff640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f111c106170 0x7f111c1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f111bfff640 1 -- 172.21.15.82:0/446672248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f111c19a6c0 msgr2=0x7f111c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f111bfff640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f111c19a6c0 0x7f111c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f111bfff640 1 -- 172.21.15.82:0/446672248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c19b2c0 con 0x7f111c1010f0 2024-04-21T08:49:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f111b7fe640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f111c106170 0x7f111c1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f111bfff640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1010f0 0x7f111c1a0600 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f110400e9f0 tx=0x7f110400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f11197fa640 1 -- 172.21.15.82:0/446672248 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110400cd80 con 0x7f111c1010f0 2024-04-21T08:49:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.061+0000 7f1122784640 1 -- 172.21.15.82:0/446672248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111c1a73a0 con 0x7f111c1010f0 2024-04-21T08:49:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.062+0000 7f11197fa640 1 -- 172.21.15.82:0/446672248 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11040054d0 con 0x7f111c1010f0 2024-04-21T08:49:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.062+0000 7f11197fa640 1 -- 172.21.15.82:0/446672248 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104010690 con 0x7f111c1010f0 2024-04-21T08:49:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.062+0000 7f1122784640 1 -- 172.21.15.82:0/446672248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111c1a7750 con 0x7f111c1010f0 2024-04-21T08:49:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.064+0000 7f11197fa640 1 -- 172.21.15.82:0/446672248 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1104004e90 con 0x7f111c1010f0 2024-04-21T08:49:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.064+0000 7f112537b740 1 Processor -- start 2024-04-21T08:49:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.064+0000 7f112537b740 1 -- start start 2024-04-21T08:49:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f112537b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c147a8d0 0x5654c1479620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f112537b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 0x5654c1479fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f112537b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c14830e0 0x5654c14854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f112537b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5654c13de8d0 con 0x5654c1479b60 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f112537b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5654c137be80 con 0x5654c14830e0 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f112537b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5654c13da890 con 0x5654c147a8d0 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f111bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 0x5654c1479fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f111bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 0x5654c1479fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54354/0 (socket says 172.21.15.82:54354) 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f111bfff640 1 -- 172.21.15.82:0/3920569845 learned_addr learned my addr 172.21.15.82:0/3920569845 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f111b7fe640 1 --2- 172.21.15.82:0/3920569845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c14830e0 0x5654c14854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f11197fa640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f11080a3dc0 0x7f11080a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f1120d21640 1 --2- 172.21.15.82:0/3920569845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c147a8d0 0x5654c1479620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f1120d21640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f11080a3dc0 0x7f11080a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f111b7fe640 1 -- 172.21.15.82:0/3920569845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c147a8d0 msgr2=0x5654c1479620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f111b7fe640 1 --2- 172.21.15.82:0/3920569845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c147a8d0 0x5654c1479620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f111b7fe640 1 -- 172.21.15.82:0/3920569845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 msgr2=0x5654c1479fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f111b7fe640 1 --2- 172.21.15.82:0/3920569845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 0x5654c1479fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f111b7fe640 1 -- 172.21.15.82:0/3920569845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c139d320 con 0x5654c14830e0 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f11197fa640 1 -- 172.21.15.82:0/446672248 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f11040d1060 con 0x7f111c1010f0 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f111bfff640 1 --2- 172.21.15.82:0/3920569845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 0x5654c1479fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f111b7fe640 1 --2- 172.21.15.82:0/3920569845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c14830e0 0x5654c14854d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f111c06b730 tx=0x7f110c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=785e8b022c7839f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f1120d21640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f11080a3dc0 0x7f11080a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1114004af0 tx=0x7f111402c0f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.065+0000 7f11127fc640 1 -- 172.21.15.82:0/3920569845 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c009e50 con 0x5654c14830e0 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.066+0000 7f11127fc640 1 -- 172.21.15.82:0/3920569845 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110c0035f0 con 0x5654c14830e0 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.066+0000 7f11127fc640 1 -- 172.21.15.82:0/3920569845 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0044e0 con 0x5654c14830e0 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.066+0000 7f112537b740 1 -- 172.21.15.82:0/3920569845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c14830e0 msgr2=0x5654c14854d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.066+0000 7f112537b740 1 --2- 172.21.15.82:0/3920569845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c14830e0 0x5654c14854d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f111c06b730 tx=0x7f110c010740 comp rx=0 tx=0).stop 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.066+0000 7f112537b740 1 -- 172.21.15.82:0/3920569845 shutdown_connections 2024-04-21T08:49:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.066+0000 7f112537b740 1 --2- 172.21.15.82:0/3920569845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c14830e0 0x5654c14854d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.066+0000 7f112537b740 1 --2- 172.21.15.82:0/3920569845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 0x5654c1479fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.066+0000 7f112537b740 1 --2- 172.21.15.82:0/3920569845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c147a8d0 0x5654c1479620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.066+0000 7f112537b740 1 -- 172.21.15.82:0/3920569845 >> 172.21.15.82:0/3920569845 conn(0x5654c1440c90 msgr2=0x5654c1474850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.066+0000 7f112537b740 1 -- 172.21.15.82:0/3920569845 shutdown_connections 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.066+0000 7f112537b740 1 -- 172.21.15.82:0/3920569845 wait complete. 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f112537b740 1 -- start start 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f112537b740 1 Processor -- start 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f112537b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c1479b60 0x5654c1482a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f112537b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c147a8d0 0x5654c147fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f112537b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c14830e0 0x5654c1480330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f112537b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5654c1485e10 con 0x5654c14830e0 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f112537b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5654c1366eb0 con 0x5654c147a8d0 2024-04-21T08:49:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f112537b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5654c1485f90 con 0x5654c1479b60 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f112537b740 10 client.0 _ll_register_callbacks cb 0x5654c1393b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f1120d21640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c14830e0 0x5654c1480330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f111b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c147a8d0 0x5654c147fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f1120d21640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c14830e0 0x5654c1480330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54358/0 (socket says 172.21.15.82:54358) 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f1120d21640 1 -- 172.21.15.82:0/3929189252 learned_addr learned my addr 172.21.15.82:0/3929189252 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f111bfff640 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c1479b60 0x5654c1482a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f1120d21640 1 -- 172.21.15.82:0/3929189252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c1479b60 msgr2=0x5654c1482a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f1120d21640 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c1479b60 0x5654c1482a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f1120d21640 1 -- 172.21.15.82:0/3929189252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c147a8d0 msgr2=0x5654c147fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f1120d21640 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c147a8d0 0x5654c147fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f1120d21640 1 -- 172.21.15.82:0/3929189252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c10f1210 con 0x5654c14830e0 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.067+0000 7f111b7fe640 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c147a8d0 0x5654c147fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f1120d21640 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c14830e0 0x5654c1480330 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1114036000 tx=0x7f11140095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bc4567bff2bd9c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f10fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f10fb7fe640 1 -- 172.21.15.82:0/3929189252 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114053060 con 0x5654c14830e0 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f10fb7fe640 1 -- 172.21.15.82:0/3929189252 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1114003d80 con 0x5654c14830e0 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f112537b740 1 -- 172.21.15.82:0/3929189252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f110c0035f0 con 0x5654c14830e0 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f10fb7fe640 1 -- 172.21.15.82:0/3929189252 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f111405a050 con 0x5654c14830e0 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f112537b740 10 client.16818 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f112537b740 1 -- 172.21.15.82:0/3929189252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5654c11bd620 con 0x5654c14830e0 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f10fb7fe640 1 -- 172.21.15.82:0/3929189252 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110c0035f0 con 0x5654c14830e0 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f10fb7fe640 1 -- 172.21.15.82:0/3929189252 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f111403f070 con 0x5654c14830e0 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f112537b740 10 client.16818 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f112537b740 10 client.16818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f112537b740 20 client.16818 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f112537b740 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5654c1493180 0x5654c14955d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f112537b740 4 client.16818 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.068+0000 7f112537b740 1 -- 172.21.15.82:0/3929189252 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654c13f2880 con 0x5654c1493180 2024-04-21T08:49:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.069+0000 7f111bfff640 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5654c1493180 0x5654c14955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.069+0000 7f111bfff640 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5654c1493180 0x5654c14955d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=862568b46b66f6d2 server_cookie=4c759f7c4a9de2de in_seq=0 out_seq=0 2024-04-21T08:49:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.069+0000 7f10fb7fe640 10 client.16818 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.069+0000 7f10fb7fe640 1 -- 172.21.15.82:0/3929189252 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654c13f2880 con 0x5654c1493180 2024-04-21T08:49:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.069+0000 7f10fb7fe640 10 client.16818 handle_command_reply: tid=0 2024-04-21T08:49:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.069+0000 7f112537b740 1 client.16818 shutdown 2024-04-21T08:49:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.070+0000 7f112537b740 1 -- 172.21.15.82:0/3929189252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c14830e0 msgr2=0x5654c1480330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.070+0000 7f112537b740 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c14830e0 0x5654c1480330 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1114036000 tx=0x7f11140095c0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.070+0000 7f112537b740 1 -- 172.21.15.82:0/3929189252 shutdown_connections 2024-04-21T08:49:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.070+0000 7f112537b740 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5654c1493180 0x5654c14955d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.070+0000 7f112537b740 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c14830e0 0x5654c1480330 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.070+0000 7f112537b740 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c147a8d0 0x5654c147fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.070+0000 7f112537b740 1 --2- 172.21.15.82:0/3929189252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c1479b60 0x5654c1482a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.070+0000 7f10fb7fe640 0 client.16818 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.070+0000 7f112537b740 1 -- 172.21.15.82:0/3929189252 >> 172.21.15.82:0/3929189252 conn(0x5654c1440c90 msgr2=0x5654c14765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.070+0000 7f112537b740 1 -- 172.21.15.82:0/3929189252 shutdown_connections 2024-04-21T08:49:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.070+0000 7f112537b740 1 -- 172.21.15.82:0/3929189252 wait complete. 2024-04-21T08:49:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.070+0000 7f112537b740 20 client.16818 trim_cache size 0 max 16384 2024-04-21T08:49:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.084+0000 7f112537b740 1 Processor -- start 2024-04-21T08:49:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f112537b740 1 -- start start 2024-04-21T08:49:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f112537b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c1479b60 0x5654c148ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f112537b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c147a8d0 0x5654c1482500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f112537b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c14830e0 0x5654c1482a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f112537b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1b1550 con 0x5654c147a8d0 2024-04-21T08:49:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f112537b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1b3180 con 0x5654c1479b60 2024-04-21T08:49:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f112537b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1b1c20 con 0x5654c14830e0 2024-04-21T08:49:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f111b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c147a8d0 0x5654c1482500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f111b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c147a8d0 0x5654c1482500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54368/0 (socket says 172.21.15.82:54368) 2024-04-21T08:49:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f111b7fe640 1 -- 172.21.15.82:0/1503883486 learned_addr learned my addr 172.21.15.82:0/1503883486 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f111bfff640 1 --2- 172.21.15.82:0/1503883486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c1479b60 0x5654c148ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f1120d21640 1 --2- 172.21.15.82:0/1503883486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c14830e0 0x5654c1482a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f111b7fe640 1 -- 172.21.15.82:0/1503883486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c14830e0 msgr2=0x5654c1482a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f111b7fe640 1 --2- 172.21.15.82:0/1503883486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c14830e0 0x5654c1482a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f111b7fe640 1 -- 172.21.15.82:0/1503883486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c1479b60 msgr2=0x5654c148ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f111b7fe640 1 --2- 172.21.15.82:0/1503883486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c1479b60 0x5654c148ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f111b7fe640 1 -- 172.21.15.82:0/1503883486 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c1478e30 con 0x5654c147a8d0 2024-04-21T08:49:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f111b7fe640 1 --2- 172.21.15.82:0/1503883486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c147a8d0 0x5654c1482500 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x5654c147a600 tx=0x7f110c0098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c7829fbb860354 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f1111ffb640 1 -- 172.21.15.82:0/1503883486 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0052c0 con 0x5654c147a8d0 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f1111ffb640 1 -- 172.21.15.82:0/1503883486 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110c005460 con 0x5654c147a8d0 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.085+0000 7f1111ffb640 1 -- 172.21.15.82:0/1503883486 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c012a80 con 0x5654c147a8d0 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.086+0000 7f112537b740 1 -- 172.21.15.82:0/1503883486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c147a8d0 msgr2=0x5654c1482500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.086+0000 7f112537b740 1 --2- 172.21.15.82:0/1503883486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c147a8d0 0x5654c1482500 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x5654c147a600 tx=0x7f110c0098d0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.086+0000 7f112537b740 1 -- 172.21.15.82:0/1503883486 shutdown_connections 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.086+0000 7f112537b740 1 --2- 172.21.15.82:0/1503883486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c14830e0 0x5654c1482a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.086+0000 7f112537b740 1 --2- 172.21.15.82:0/1503883486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c147a8d0 0x5654c1482500 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.086+0000 7f112537b740 1 --2- 172.21.15.82:0/1503883486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c1479b60 0x5654c148ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.086+0000 7f112537b740 1 -- 172.21.15.82:0/1503883486 >> 172.21.15.82:0/1503883486 conn(0x5654c1440c90 msgr2=0x5654c147b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.086+0000 7f112537b740 1 -- 172.21.15.82:0/1503883486 shutdown_connections 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.086+0000 7f112537b740 1 -- 172.21.15.82:0/1503883486 wait complete. 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.086+0000 7f112537b740 1 -- start start 2024-04-21T08:49:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f112537b740 1 Processor -- start 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f112537b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 0x5654c148b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f112537b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c147a8d0 0x5654c1486650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f112537b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c14830e0 0x5654c1486b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f112537b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1b2a80 con 0x5654c1479b60 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f112537b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1b23b0 con 0x5654c14830e0 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f112537b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1af530 con 0x5654c147a8d0 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f112537b740 10 client.0 _ll_register_callbacks cb 0x7f1104005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f111bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 0x5654c148b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f111bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 0x5654c148b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54372/0 (socket says 172.21.15.82:54372) 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f1120d21640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c14830e0 0x5654c1486b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f111bfff640 1 -- 172.21.15.82:0/1959971275 learned_addr learned my addr 172.21.15.82:0/1959971275 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f111b7fe640 1 --2- 172.21.15.82:0/1959971275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c147a8d0 0x5654c1486650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f111bfff640 1 -- 172.21.15.82:0/1959971275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c147a8d0 msgr2=0x5654c1486650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f111bfff640 1 --2- 172.21.15.82:0/1959971275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c147a8d0 0x5654c1486650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f111bfff640 1 -- 172.21.15.82:0/1959971275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c14830e0 msgr2=0x5654c1486b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f111bfff640 1 --2- 172.21.15.82:0/1959971275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c14830e0 0x5654c1486b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f111bfff640 1 -- 172.21.15.82:0/1959971275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c14870d0 con 0x5654c1479b60 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f111bfff640 1 --2- 172.21.15.82:0/1959971275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 0x5654c148b2f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1104004ad0 tx=0x7f1104099010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc2f10651b5a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.087+0000 7f10fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f10fb7fe640 1 -- 172.21.15.82:0/1959971275 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104005180 con 0x5654c1479b60 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f10fb7fe640 1 -- 172.21.15.82:0/1959971275 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f11040046c0 con 0x5654c1479b60 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f112537b740 1 -- 172.21.15.82:0/1959971275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f110c005460 con 0x5654c1479b60 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f10fb7fe640 1 -- 172.21.15.82:0/1959971275 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1104005680 con 0x5654c1479b60 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f112537b740 10 client.16824 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f112537b740 1 -- 172.21.15.82:0/1959971275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5654c14f93f0 con 0x5654c1479b60 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f10fb7fe640 1 -- 172.21.15.82:0/1959971275 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110c005460 con 0x5654c1479b60 2024-04-21T08:49:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f10fb7fe640 1 -- 172.21.15.82:0/1959971275 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f11040ce670 con 0x5654c1479b60 2024-04-21T08:49:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f112537b740 10 client.16824 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f112537b740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f112537b740 20 client.16824 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f112537b740 1 --2- 172.21.15.82:0/1959971275 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5654c14f9ac0 0x5654c14fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f112537b740 4 client.16824 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:49:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f112537b740 1 -- 172.21.15.82:0/1959971275 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654c137e3d0 con 0x5654c14f9ac0 2024-04-21T08:49:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.088+0000 7f111b7fe640 1 --2- 172.21.15.82:0/1959971275 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5654c14f9ac0 0x5654c14fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.089+0000 7f111b7fe640 1 --2- 172.21.15.82:0/1959971275 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5654c14f9ac0 0x5654c14fbee0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9addbdd0bc3d3bd0 server_cookie=93b4c8d1b507e5c2 in_seq=0 out_seq=0 2024-04-21T08:49:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:49:24.089+0000 7f10fb7fe640 10 client.16824 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:24.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:24 smithi134 ceph-mon[31772]: pgmap v74: 129 pgs: 129 active+clean; 261 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.1 MiB/s wr, 395 op/s 2024-04-21T08:49:24.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:24 smithi082 ceph-mon[27967]: pgmap v74: 129 pgs: 129 active+clean; 261 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.1 MiB/s wr, 395 op/s 2024-04-21T08:49:24.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:24 smithi106 ceph-mon[31072]: pgmap v74: 129 pgs: 129 active+clean; 261 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.1 MiB/s wr, 395 op/s 2024-04-21T08:49:26.222 INFO:tasks.workunit.client.0.smithi082.stdout: 75 457062 139 320277 170 251651 195 2024-04-21T08:49:26.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:26 smithi134 ceph-mon[31772]: pgmap v75: 129 pgs: 129 active+clean; 261 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 249 op/s 2024-04-21T08:49:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:26 smithi082 ceph-mon[27967]: pgmap v75: 129 pgs: 129 active+clean; 261 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 249 op/s 2024-04-21T08:49:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:26 smithi106 ceph-mon[31072]: pgmap v75: 129 pgs: 129 active+clean; 261 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 249 op/s 2024-04-21T08:49:28.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:28 smithi134 ceph-mon[31772]: pgmap v76: 129 pgs: 129 active+clean; 266 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-04-21T08:49:28.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:28 smithi082 ceph-mon[27967]: pgmap v76: 129 pgs: 129 active+clean; 266 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-04-21T08:49:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:28 smithi106 ceph-mon[31072]: pgmap v76: 129 pgs: 129 active+clean; 266 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-04-21T08:49:30.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:30 smithi134 ceph-mon[31772]: pgmap v77: 129 pgs: 129 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 555 op/s 2024-04-21T08:49:30.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:30 smithi082 ceph-mon[27967]: pgmap v77: 129 pgs: 129 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 555 op/s 2024-04-21T08:49:30.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:30 smithi106 ceph-mon[31072]: pgmap v77: 129 pgs: 129 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 555 op/s 2024-04-21T08:49:32.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:32 smithi134 ceph-mon[31772]: pgmap v78: 129 pgs: 129 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 528 op/s 2024-04-21T08:49:32.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:32 smithi082 ceph-mon[27967]: pgmap v78: 129 pgs: 129 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 528 op/s 2024-04-21T08:49:32.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:32 smithi106 ceph-mon[31072]: pgmap v78: 129 pgs: 129 active+clean; 287 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 528 op/s 2024-04-21T08:49:34.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:34 smithi134 ceph-mon[31772]: pgmap v79: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 851 op/s 2024-04-21T08:49:34.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:34 smithi082 ceph-mon[27967]: pgmap v79: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 851 op/s 2024-04-21T08:49:34.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:34 smithi106 ceph-mon[31072]: pgmap v79: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 851 op/s 2024-04-21T08:49:36.222 INFO:tasks.workunit.client.0.smithi082.stdout: 75 184 0 85 0 22 0 2024-04-21T08:49:36.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f55357fa640 1 -- 172.21.15.82:0/2150723320 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x560170227980 con 0x56017033bee0 2024-04-21T08:49:36.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f55357fa640 10 client.16680 handle_command_reply: tid=0 2024-04-21T08:49:36.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f5556bb7740 1 client.16680 shutdown 2024-04-21T08:49:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f5556bb7740 1 -- 172.21.15.82:0/2150723320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 msgr2=0x56017033a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f5556bb7740 1 --2- 172.21.15.82:0/2150723320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 0x56017033a290 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f553c02c170 tx=0x7f553c02c6c0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f5556bb7740 1 -- 172.21.15.82:0/2150723320 shutdown_connections 2024-04-21T08:49:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f5556bb7740 1 --2- 172.21.15.82:0/2150723320 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56017033bee0 0x56017033e2d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f5556bb7740 1 --2- 172.21.15.82:0/2150723320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601702c4ec0 0x56017033a290 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f5556bb7740 1 --2- 172.21.15.82:0/2150723320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5601702bc2e0 0x5601702c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f5556bb7740 1 --2- 172.21.15.82:0/2150723320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601702bb9c0 0x5601702c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f55357fa640 0 client.16680 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f5556bb7740 1 -- 172.21.15.82:0/2150723320 >> 172.21.15.82:0/2150723320 conn(0x5601702b60f0 msgr2=0x5601702b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f5556bb7740 1 -- 172.21.15.82:0/2150723320 shutdown_connections 2024-04-21T08:49:36.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f5556bb7740 1 -- 172.21.15.82:0/2150723320 wait complete. 2024-04-21T08:49:36.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.310+0000 7f5556bb7740 20 client.16680 trim_cache size 0 max 16384 2024-04-21T08:49:36.312 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:49:36.312 INFO:teuthology.orchestra.run.smithi082.stdout: "epoch": 32, 2024-04-21T08:49:36.312 INFO:teuthology.orchestra.run.smithi082.stdout: "leader": 24509, 2024-04-21T08:49:36.312 INFO:teuthology.orchestra.run.smithi082.stdout: "set_version": 2, 2024-04-21T08:49:36.312 INFO:teuthology.orchestra.run.smithi082.stdout: "sets": { 2024-04-21T08:49:36.312 INFO:teuthology.orchestra.run.smithi082.stdout: "9b977ccd": { 2024-04-21T08:49:36.313 INFO:teuthology.orchestra.run.smithi082.stdout: "version": 2, 2024-04-21T08:49:36.313 INFO:teuthology.orchestra.run.smithi082.stdout: "age_ref": 0.0, 2024-04-21T08:49:36.313 INFO:teuthology.orchestra.run.smithi082.stdout: "state": { 2024-04-21T08:49:36.313 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "QUIESCED", 2024-04-21T08:49:36.313 INFO:teuthology.orchestra.run.smithi082.stdout: "age": 0.0 2024-04-21T08:49:36.313 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:49:36.313 INFO:teuthology.orchestra.run.smithi082.stdout: "timeout": 30.0, 2024-04-21T08:49:36.313 INFO:teuthology.orchestra.run.smithi082.stdout: "expiration": 90.0, 2024-04-21T08:49:36.313 INFO:teuthology.orchestra.run.smithi082.stdout: "members": { 2024-04-21T08:49:36.313 INFO:teuthology.orchestra.run.smithi082.stdout: "file:/": { 2024-04-21T08:49:36.314 INFO:teuthology.orchestra.run.smithi082.stdout: "excluded": false, 2024-04-21T08:49:36.314 INFO:teuthology.orchestra.run.smithi082.stdout: "state": { 2024-04-21T08:49:36.314 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "QUIESCED", 2024-04-21T08:49:36.314 INFO:teuthology.orchestra.run.smithi082.stdout: "age": 0.0 2024-04-21T08:49:36.314 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:49:36.314 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:49:36.314 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:49:36.314 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:49:36.314 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:49:36.314 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:49:36.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.311+0000 7f5553fca640 1 -- 172.21.15.82:0/2295477921 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f55380a3da0 msgr2=0x7f55380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.311+0000 7f5553fca640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f55380a3da0 0x7f55380a6260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5601702bb490 tx=0x7f5544008720 comp rx=0 tx=0).stop 2024-04-21T08:49:36.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.311+0000 7f5553fca640 1 -- 172.21.15.82:0/2295477921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f554c106170 msgr2=0x7f554c198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:36.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.311+0000 7f5553fca640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f554c106170 0x7f554c198b30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f554800e9c0 tx=0x7f554800ee80 comp rx=0 tx=0).stop 2024-04-21T08:49:36.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.311+0000 7f5553fca640 1 -- 172.21.15.82:0/2295477921 shutdown_connections 2024-04-21T08:49:36.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.311+0000 7f5553fca640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f554c19ceb0 0x7f554c19d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.311+0000 7f5553fca640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f55380a3da0 0x7f55380a6260 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.311+0000 7f5553fca640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f554c106170 0x7f554c198b30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.311+0000 7f5553fca640 1 --2- 172.21.15.82:0/2295477921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f554c1010a0 0x7f554c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:36.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.311+0000 7f5553fca640 1 -- 172.21.15.82:0/2295477921 >> 172.21.15.82:0/2295477921 conn(0x7f554c077f50 msgr2=0x7f554c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:36.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.311+0000 7f5553fca640 1 -- 172.21.15.82:0/2295477921 shutdown_connections 2024-04-21T08:49:36.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:36.311+0000 7f5553fca640 1 -- 172.21.15.82:0/2295477921 wait complete. 2024-04-21T08:49:36.324 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: 9b977ccd, quiesce duration: 30.0 2024-04-21T08:49:36.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:36 smithi134 ceph-mon[31772]: pgmap v80: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 637 op/s 2024-04-21T08:49:36.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:36 smithi082 ceph-mon[27967]: pgmap v80: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 637 op/s 2024-04-21T08:49:36.642 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:36 smithi106 ceph-mon[31072]: pgmap v80: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.8 MiB/s wr, 637 op/s 2024-04-21T08:49:38.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:38 smithi134 ceph-mon[31772]: pgmap v81: 129 pgs: 129 active+clean; 317 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 662 op/s 2024-04-21T08:49:38.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:38 smithi082 ceph-mon[27967]: pgmap v81: 129 pgs: 129 active+clean; 317 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 662 op/s 2024-04-21T08:49:38.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:38 smithi106 ceph-mon[31072]: pgmap v81: 129 pgs: 129 active+clean; 317 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 662 op/s 2024-04-21T08:49:40.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:40 smithi134 ceph-mon[31772]: pgmap v82: 129 pgs: 129 active+clean; 341 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 669 op/s 2024-04-21T08:49:40.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:40 smithi082 ceph-mon[27967]: pgmap v82: 129 pgs: 129 active+clean; 341 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 669 op/s 2024-04-21T08:49:40.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:40 smithi106 ceph-mon[31072]: pgmap v82: 129 pgs: 129 active+clean; 341 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.8 MiB/s wr, 669 op/s 2024-04-21T08:49:42.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:42 smithi082 ceph-mon[27967]: pgmap v83: 129 pgs: 129 active+clean; 341 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 411 op/s 2024-04-21T08:49:42.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:42 smithi106 ceph-mon[31072]: pgmap v83: 129 pgs: 129 active+clean; 341 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 411 op/s 2024-04-21T08:49:42.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:42 smithi134 ceph-mon[31772]: pgmap v83: 129 pgs: 129 active+clean; 341 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 411 op/s 2024-04-21T08:49:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:44 smithi082 ceph-mon[27967]: pgmap v84: 129 pgs: 129 active+clean; 363 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.0 MiB/s wr, 673 op/s 2024-04-21T08:49:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:44 smithi106 ceph-mon[31072]: pgmap v84: 129 pgs: 129 active+clean; 363 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.0 MiB/s wr, 673 op/s 2024-04-21T08:49:44.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:44 smithi134 ceph-mon[31772]: pgmap v84: 129 pgs: 129 active+clean; 363 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.0 MiB/s wr, 673 op/s 2024-04-21T08:49:46.222 INFO:tasks.workunit.client.0.smithi082.stdout: 75 14029 0 10493 0 8932 0 2024-04-21T08:49:46.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:46 smithi082 ceph-mon[27967]: pgmap v85: 129 pgs: 129 active+clean; 363 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 350 op/s 2024-04-21T08:49:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:46 smithi106 ceph-mon[31072]: pgmap v85: 129 pgs: 129 active+clean; 363 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 350 op/s 2024-04-21T08:49:46.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:46 smithi134 ceph-mon[31772]: pgmap v85: 129 pgs: 129 active+clean; 363 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 350 op/s 2024-04-21T08:49:48.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:48 smithi082 ceph-mon[27967]: pgmap v86: 129 pgs: 129 active+clean; 370 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.2 MiB/s wr, 420 op/s 2024-04-21T08:49:48.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:48 smithi106 ceph-mon[31072]: pgmap v86: 129 pgs: 129 active+clean; 370 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.2 MiB/s wr, 420 op/s 2024-04-21T08:49:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:48 smithi134 ceph-mon[31772]: pgmap v86: 129 pgs: 129 active+clean; 370 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.2 MiB/s wr, 420 op/s 2024-04-21T08:49:50.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:50 smithi082 ceph-mon[27967]: pgmap v87: 129 pgs: 129 active+clean; 382 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.9 MiB/s wr, 564 op/s 2024-04-21T08:49:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:50 smithi106 ceph-mon[31072]: pgmap v87: 129 pgs: 129 active+clean; 382 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.9 MiB/s wr, 564 op/s 2024-04-21T08:49:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:50 smithi134 ceph-mon[31772]: pgmap v87: 129 pgs: 129 active+clean; 382 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.9 MiB/s wr, 564 op/s 2024-04-21T08:49:51.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:49:51.117 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:49:51.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.229+0000 7fc97b88e640 1 -- 172.21.15.82:0/3359485692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9741010a0 msgr2=0x7fc974101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.229+0000 7fc97b88e640 1 --2- 172.21.15.82:0/3359485692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9741010a0 0x7fc974101500 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc964009900 tx=0x7fc96402bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 -- 172.21.15.82:0/3359485692 shutdown_connections 2024-04-21T08:49:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 --2- 172.21.15.82:0/3359485692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc974101a40 0x7fc97410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 --2- 172.21.15.82:0/3359485692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9741010a0 0x7fc974101500 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 --2- 172.21.15.82:0/3359485692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc974105dd0 0x7fc9741061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 -- 172.21.15.82:0/3359485692 >> 172.21.15.82:0/3359485692 conn(0x7fc974077c40 msgr2=0x7fc974078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 -- 172.21.15.82:0/3359485692 shutdown_connections 2024-04-21T08:49:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 -- 172.21.15.82:0/3359485692 wait complete. 2024-04-21T08:49:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 Processor -- start 2024-04-21T08:49:51.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 -- start start 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9741010a0 0x7fc9740feba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc974101a40 0x7fc9740ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc974105dd0 0x7fc9740ff620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc974111130 con 0x7fc974101a40 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc974110fb0 con 0x7fc974105dd0 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.230+0000 7fc97b88e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741112b0 con 0x7fc9741010a0 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc978e29640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc974101a40 0x7fc9740ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc978e29640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc974101a40 0x7fc9740ff0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34422/0 (socket says 172.21.15.82:34422) 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc978e29640 1 -- 172.21.15.82:0/560581383 learned_addr learned my addr 172.21.15.82:0/560581383 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc97962a640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9741010a0 0x7fc9740feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc979e2b640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc974105dd0 0x7fc9740ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc978e29640 1 -- 172.21.15.82:0/560581383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9741010a0 msgr2=0x7fc9740feba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc978e29640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9741010a0 0x7fc9740feba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc978e29640 1 -- 172.21.15.82:0/560581383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc974105dd0 msgr2=0x7fc9740ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:51.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc978e29640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc974105dd0 0x7fc9740ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc978e29640 1 -- 172.21.15.82:0/560581383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9741a6c20 con 0x7fc974101a40 2024-04-21T08:49:51.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc979e2b640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc974105dd0 0x7fc9740ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:51.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc978e29640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc974101a40 0x7fc9740ff0e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc964005940 tx=0x7fc96402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:51.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc96a7fc640 1 -- 172.21.15.82:0/560581383 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964043020 con 0x7fc974101a40 2024-04-21T08:49:51.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc96a7fc640 1 -- 172.21.15.82:0/560581383 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc96402cbe0 con 0x7fc974101a40 2024-04-21T08:49:51.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc97b88e640 1 -- 172.21.15.82:0/560581383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9741a6eb0 con 0x7fc974101a40 2024-04-21T08:49:51.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc96a7fc640 1 -- 172.21.15.82:0/560581383 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964003bf0 con 0x7fc974101a40 2024-04-21T08:49:51.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.231+0000 7fc97b88e640 1 -- 172.21.15.82:0/560581383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9741a7370 con 0x7fc974101a40 2024-04-21T08:49:51.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc96a7fc640 1 -- 172.21.15.82:0/560581383 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc964046020 con 0x7fc974101a40 2024-04-21T08:49:51.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc97e47d740 1 Processor -- start 2024-04-21T08:49:51.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc97e47d740 1 -- start start 2024-04-21T08:49:51.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc97e47d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e60153580 0x562e60153960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:51.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc97e47d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 0x562e6015c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:51.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc97e47d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e6015ca80 0x562e6015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:51.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc97e47d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562e600b81e0 con 0x562e60153ea0 2024-04-21T08:49:51.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc97e47d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562e60055390 con 0x562e60153580 2024-04-21T08:49:51.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc97e47d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562e600b41a0 con 0x562e6015ca80 2024-04-21T08:49:51.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc97962a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 0x562e6015c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:51.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc96a7fc640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc9600a3d20 0x7fc9600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:51.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc97962a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 0x562e6015c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34432/0 (socket says 172.21.15.82:34432) 2024-04-21T08:49:51.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.234+0000 7fc97962a640 1 -- 172.21.15.82:0/501802530 learned_addr learned my addr 172.21.15.82:0/501802530 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:51.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc978e29640 1 --2- 172.21.15.82:0/501802530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e6015ca80 0x562e6015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:51.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc979e2b640 1 --2- 172.21.15.82:0/501802530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e60153580 0x562e60153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:51.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc979e2b640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc9600a3d20 0x7fc9600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:51.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc96a7fc640 1 -- 172.21.15.82:0/560581383 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc9640f2cf0 con 0x7fc974101a40 2024-04-21T08:49:51.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc97962a640 1 -- 172.21.15.82:0/501802530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e6015ca80 msgr2=0x562e6015ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:51.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc97962a640 1 --2- 172.21.15.82:0/501802530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e6015ca80 0x562e6015ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc97962a640 1 -- 172.21.15.82:0/501802530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e60153580 msgr2=0x562e60153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc97962a640 1 --2- 172.21.15.82:0/501802530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e60153580 0x562e60153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc97962a640 1 -- 172.21.15.82:0/501802530 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e5fe947d0 con 0x562e60153ea0 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc979e2b640 1 --2- 172.21.15.82:0/501802530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e60153580 0x562e60153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc97962a640 1 --2- 172.21.15.82:0/501802530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 0x562e6015c540 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc97000bc90 tx=0x7fc9700107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b16bd50578ea83 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc979e2b640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc9600a3d20 0x7fc9600a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc96c005c40 tx=0x7fc96c0094a0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc95f7fe640 1 -- 172.21.15.82:0/501802530 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97001c480 con 0x562e60153ea0 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc95f7fe640 1 -- 172.21.15.82:0/501802530 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc97001cad0 con 0x562e60153ea0 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.235+0000 7fc95f7fe640 1 -- 172.21.15.82:0/501802530 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970011af0 con 0x562e60153ea0 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.236+0000 7fc97e47d740 1 -- 172.21.15.82:0/501802530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 msgr2=0x562e6015c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.236+0000 7fc97e47d740 1 --2- 172.21.15.82:0/501802530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 0x562e6015c540 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc97000bc90 tx=0x7fc9700107a0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.236+0000 7fc97e47d740 1 -- 172.21.15.82:0/501802530 shutdown_connections 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.236+0000 7fc97e47d740 1 --2- 172.21.15.82:0/501802530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e6015ca80 0x562e6015ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.236+0000 7fc97e47d740 1 --2- 172.21.15.82:0/501802530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 0x562e6015c540 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.236+0000 7fc97e47d740 1 --2- 172.21.15.82:0/501802530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e60153580 0x562e60153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.236+0000 7fc97e47d740 1 -- 172.21.15.82:0/501802530 >> 172.21.15.82:0/501802530 conn(0x562e6014dcb0 msgr2=0x562e6014e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.236+0000 7fc97e47d740 1 -- 172.21.15.82:0/501802530 shutdown_connections 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.236+0000 7fc97e47d740 1 -- 172.21.15.82:0/501802530 wait complete. 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.236+0000 7fc97e47d740 1 -- start start 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.236+0000 7fc97e47d740 1 Processor -- start 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97e47d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 0x562e60159a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97e47d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e60153ea0 0x562e60159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97e47d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e6015ca80 0x562e6015a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97e47d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562e6015f870 con 0x562e60153580 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97e47d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562e600712f0 con 0x562e60153ea0 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97e47d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562e6015f9f0 con 0x562e6015ca80 2024-04-21T08:49:51.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97e47d740 10 client.0 _ll_register_callbacks cb 0x562e6007e580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97962a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 0x562e60159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97962a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 0x562e60159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34438/0 (socket says 172.21.15.82:34438) 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97962a640 1 -- 172.21.15.82:0/2297153758 learned_addr learned my addr 172.21.15.82:0/2297153758 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc978e29640 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e60153ea0 0x562e60159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc979e2b640 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e6015ca80 0x562e6015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97962a640 1 -- 172.21.15.82:0/2297153758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e6015ca80 msgr2=0x562e6015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97962a640 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e6015ca80 0x562e6015a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97962a640 1 -- 172.21.15.82:0/2297153758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e60153ea0 msgr2=0x562e60159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97962a640 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e60153ea0 0x562e60159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97962a640 1 -- 172.21.15.82:0/2297153758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e5fdca3f0 con 0x562e60153580 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc978e29640 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e60153ea0 0x562e60159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc979e2b640 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e6015ca80 0x562e6015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc97962a640 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 0x562e60159a10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc97000ba00 tx=0x7fc970011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cead474be995e475 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc95cff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:49:51.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.237+0000 7fc95cff9640 1 -- 172.21.15.82:0/2297153758 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97001c890 con 0x562e60153580 2024-04-21T08:49:51.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.238+0000 7fc95cff9640 1 -- 172.21.15.82:0/2297153758 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc970004020 con 0x562e60153580 2024-04-21T08:49:51.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.238+0000 7fc97e47d740 1 -- 172.21.15.82:0/2297153758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc97001cad0 con 0x562e60153580 2024-04-21T08:49:51.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.240+0000 7fc95cff9640 1 -- 172.21.15.82:0/2297153758 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc97002e050 con 0x562e60153580 2024-04-21T08:49:51.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.240+0000 7fc97e47d740 10 client.16836 fetch_fsmap learned FSMap version 32 2024-04-21T08:49:51.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.240+0000 7fc97e47d740 1 -- 172.21.15.82:0/2297153758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562e5fd8a0f0 con 0x562e60153580 2024-04-21T08:49:51.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.240+0000 7fc95cff9640 1 -- 172.21.15.82:0/2297153758 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc97001cad0 con 0x562e60153580 2024-04-21T08:49:51.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.240+0000 7fc95cff9640 1 -- 172.21.15.82:0/2297153758 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fc970025070 con 0x562e60153580 2024-04-21T08:49:51.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.241+0000 7fc97e47d740 10 client.16836 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:49:51.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.241+0000 7fc97e47d740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:51.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.241+0000 7fc97e47d740 20 client.16836 populate_metadata read hostname 'smithi082' 2024-04-21T08:49:51.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.241+0000 7fc97e47d740 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e6016c910 0x562e6016ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:51.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.241+0000 7fc97e47d740 4 client.16836 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:51.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.241+0000 7fc97e47d740 1 -- 172.21.15.82:0/2297153758 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e600c5c00 con 0x562e6016c910 2024-04-21T08:49:51.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.241+0000 7fc978e29640 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e6016c910 0x562e6016ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:51.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.241+0000 7fc978e29640 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e6016c910 0x562e6016ed00 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41302a10fd742561 server_cookie=98e8faf26bc509fd in_seq=0 out_seq=0 2024-04-21T08:49:51.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:49:51.241+0000 7fc95cff9640 10 client.16836 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:49:52.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:52 smithi082 ceph-mon[27967]: pgmap v88: 129 pgs: 129 active+clean; 382 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 501 op/s 2024-04-21T08:49:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:52 smithi106 ceph-mon[31072]: pgmap v88: 129 pgs: 129 active+clean; 382 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 501 op/s 2024-04-21T08:49:52.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:52 smithi134 ceph-mon[31772]: pgmap v88: 129 pgs: 129 active+clean; 382 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 501 op/s 2024-04-21T08:49:54.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:54 smithi082 ceph-mon[27967]: pgmap v89: 129 pgs: 129 active+clean; 407 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.0 MiB/s wr, 672 op/s 2024-04-21T08:49:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:54 smithi106 ceph-mon[31072]: pgmap v89: 129 pgs: 129 active+clean; 407 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.0 MiB/s wr, 672 op/s 2024-04-21T08:49:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:54 smithi134 ceph-mon[31772]: pgmap v89: 129 pgs: 129 active+clean; 407 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.0 MiB/s wr, 672 op/s 2024-04-21T08:49:56.222 INFO:tasks.workunit.client.0.smithi082.stdout: 75 31278 0 22838 0 16744 0 2024-04-21T08:49:56.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:56 smithi082 ceph-mon[27967]: pgmap v90: 129 pgs: 129 active+clean; 407 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 410 op/s 2024-04-21T08:49:56.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:56 smithi106 ceph-mon[31072]: pgmap v90: 129 pgs: 129 active+clean; 407 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 410 op/s 2024-04-21T08:49:56.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:56 smithi134 ceph-mon[31772]: pgmap v90: 129 pgs: 129 active+clean; 407 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 410 op/s 2024-04-21T08:49:58.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:49:58 smithi082 ceph-mon[27967]: pgmap v91: 129 pgs: 129 active+clean; 407 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 414 op/s 2024-04-21T08:49:58.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:49:58 smithi106 ceph-mon[31072]: pgmap v91: 129 pgs: 129 active+clean; 407 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 414 op/s 2024-04-21T08:49:58.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:49:58 smithi134 ceph-mon[31772]: pgmap v91: 129 pgs: 129 active+clean; 407 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 414 op/s 2024-04-21T08:50:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:00 smithi082 ceph-mon[27967]: pgmap v92: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 354 op/s 2024-04-21T08:50:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:00 smithi082 ceph-mon[27967]: overall HEALTH_OK 2024-04-21T08:50:00.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:00 smithi106 ceph-mon[31072]: pgmap v92: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 354 op/s 2024-04-21T08:50:00.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:00 smithi106 ceph-mon[31072]: overall HEALTH_OK 2024-04-21T08:50:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:00 smithi134 ceph-mon[31772]: pgmap v92: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 354 op/s 2024-04-21T08:50:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:00 smithi134 ceph-mon[31772]: overall HEALTH_OK 2024-04-21T08:50:02.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:02 smithi082 ceph-mon[27967]: pgmap v93: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-04-21T08:50:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:02 smithi106 ceph-mon[31072]: pgmap v93: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-04-21T08:50:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:02 smithi134 ceph-mon[31772]: pgmap v93: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-04-21T08:50:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:04 smithi082 ceph-mon[27967]: pgmap v94: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-04-21T08:50:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:04 smithi106 ceph-mon[31072]: pgmap v94: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-04-21T08:50:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:04 smithi134 ceph-mon[31772]: pgmap v94: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.3 MiB/s wr, 205 op/s 2024-04-21T08:50:06.223 INFO:tasks.workunit.client.0.smithi082.stdout: 75 0 0 0 0 0 0 2024-04-21T08:50:06.325 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:50:06.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.438+0000 7effd9575640 1 -- 172.21.15.82:0/3885911342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effd4101ad0 msgr2=0x7effd410dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:06.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.438+0000 7effd9575640 1 --2- 172.21.15.82:0/3885911342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effd4101ad0 0x7effd410dcb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7effcc009ca0 tx=0x7effcc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.438+0000 7effd9575640 1 -- 172.21.15.82:0/3885911342 shutdown_connections 2024-04-21T08:50:06.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.438+0000 7effd9575640 1 --2- 172.21.15.82:0/3885911342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effd4101ad0 0x7effd410dcb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.438+0000 7effd9575640 1 --2- 172.21.15.82:0/3885911342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effd4101130 0x7effd4101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.438+0000 7effd9575640 1 --2- 172.21.15.82:0/3885911342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd4106f10 0x7effd41072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effd9575640 1 -- 172.21.15.82:0/3885911342 >> 172.21.15.82:0/3885911342 conn(0x7effd4077e30 msgr2=0x7effd4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:06.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effd9575640 1 -- 172.21.15.82:0/3885911342 shutdown_connections 2024-04-21T08:50:06.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effd9575640 1 -- 172.21.15.82:0/3885911342 wait complete. 2024-04-21T08:50:06.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effd9575640 1 Processor -- start 2024-04-21T08:50:06.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effd9575640 1 -- start start 2024-04-21T08:50:06.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effd9575640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effd4101130 0x7effd419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effd9575640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effd4101ad0 0x7effd419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effd9575640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd4106f10 0x7effd4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effd9575640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7effd41110f0 con 0x7effd4101ad0 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effd9575640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7effd4110f70 con 0x7effd4106f10 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effd9575640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7effd4111270 con 0x7effd4101130 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effcbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effd4101ad0 0x7effd419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effcbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effd4101ad0 0x7effd419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36228/0 (socket says 172.21.15.82:36228) 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.439+0000 7effcbfff640 1 -- 172.21.15.82:0/2902435416 learned_addr learned my addr 172.21.15.82:0/2902435416 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effd2ffd640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effd4101130 0x7effd419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effd37fe640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd4106f10 0x7effd4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effcbfff640 1 -- 172.21.15.82:0/2902435416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effd4101130 msgr2=0x7effd419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effcbfff640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effd4101130 0x7effd419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effcbfff640 1 -- 172.21.15.82:0/2902435416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd4106f10 msgr2=0x7effd4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effcbfff640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd4106f10 0x7effd4198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effcbfff640 1 -- 172.21.15.82:0/2902435416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd4198d80 con 0x7effd4101ad0 2024-04-21T08:50:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effd37fe640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd4106f10 0x7effd4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effcbfff640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effd4101ad0 0x7effd419e940 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7effc400d770 tx=0x7effc400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effd0ff9640 1 -- 172.21.15.82:0/2902435416 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4017070 con 0x7effd4101ad0 2024-04-21T08:50:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effd0ff9640 1 -- 172.21.15.82:0/2902435416 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc40054d0 con 0x7effd4101ad0 2024-04-21T08:50:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effd9575640 1 -- 172.21.15.82:0/2902435416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd4199070 con 0x7effd4101ad0 2024-04-21T08:50:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effd0ff9640 1 -- 172.21.15.82:0/2902435416 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4004c30 con 0x7effd4101ad0 2024-04-21T08:50:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.440+0000 7effd9575640 1 -- 172.21.15.82:0/2902435416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd41a52c0 con 0x7effd4101ad0 2024-04-21T08:50:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.441+0000 7effd9575640 1 -- 172.21.15.82:0/2902435416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd4069230 con 0x7effd4101ad0 2024-04-21T08:50:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.442+0000 7effd0ff9640 1 -- 172.21.15.82:0/2902435416 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7effc4004e10 con 0x7effd4101ad0 2024-04-21T08:50:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.443+0000 7effd0ff9640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7effc00a3da0 0x7effc00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.443+0000 7effd2ffd640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7effc00a3da0 0x7effc00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.443+0000 7effd0ff9640 1 -- 172.21.15.82:0/2902435416 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7effc40d06f0 con 0x7effd4101ad0 2024-04-21T08:50:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.443+0000 7effd2ffd640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7effc00a3da0 0x7effc00a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7effbc00bb60 tx=0x7effbc00b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.448+0000 7effd0ff9640 1 -- 172.21.15.82:0/2902435416 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effc409c970 con 0x7effd4101ad0 2024-04-21T08:50:06.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:06 smithi082 ceph-mon[27967]: pgmap v95: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 149 KiB/s wr, 34 op/s 2024-04-21T08:50:06.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:06 smithi106 ceph-mon[31072]: pgmap v95: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 149 KiB/s wr, 34 op/s 2024-04-21T08:50:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.670+0000 7effd9575640 1 -- 172.21.15.82:0/2902435416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effd41a5490 con 0x7effd4101ad0 2024-04-21T08:50:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.671+0000 7effd0ff9640 1 -- 172.21.15.82:0/2902435416 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11952 (secure 0 0 0) 0x7effc409c270 con 0x7effd4101ad0 2024-04-21T08:50:06.671 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:50:06.671 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:48:22.138089+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:50:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 32 2024-04-21T08:50:06.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.673+0000 7effd9575640 1 -- 172.21.15.82:0/2902435416 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7effc00a3da0 msgr2=0x7effc00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:06.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.673+0000 7effd9575640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7effc00a3da0 0x7effc00a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7effbc00bb60 tx=0x7effbc00b390 comp rx=0 tx=0).stop 2024-04-21T08:50:06.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.673+0000 7effd9575640 1 -- 172.21.15.82:0/2902435416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effd4101ad0 msgr2=0x7effd419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:06.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.673+0000 7effd9575640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effd4101ad0 0x7effd419e940 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7effc400d770 tx=0x7effc400dc30 comp rx=0 tx=0).stop 2024-04-21T08:50:06.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.673+0000 7effd9575640 1 -- 172.21.15.82:0/2902435416 shutdown_connections 2024-04-21T08:50:06.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.673+0000 7effd9575640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effd4106f10 0x7effd4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.673+0000 7effd9575640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7effc00a3da0 0x7effc00a6260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.673+0000 7effd9575640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effd4101ad0 0x7effd419e940 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.673+0000 7effd9575640 1 --2- 172.21.15.82:0/2902435416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effd4101130 0x7effd419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.673+0000 7effd9575640 1 -- 172.21.15.82:0/2902435416 >> 172.21.15.82:0/2902435416 conn(0x7effd4077e30 msgr2=0x7effd4105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:06.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.673+0000 7effd9575640 1 -- 172.21.15.82:0/2902435416 shutdown_connections 2024-04-21T08:50:06.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.673+0000 7effd9575640 1 -- 172.21.15.82:0/2902435416 wait complete. 2024-04-21T08:50:06.684 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id 9b977ccd --release --await' 2024-04-21T08:50:06.684 DEBUG:teuthology.orchestra.run.smithi082:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --set-id 9b977ccd --release --await 2024-04-21T08:50:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:06 smithi134 ceph-mon[31772]: pgmap v95: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 149 KiB/s wr, 34 op/s 2024-04-21T08:50:06.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.790+0000 7fe407cf2640 1 -- 172.21.15.82:0/3807443647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400100e10 msgr2=0x7fe40010ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:06.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.790+0000 7fe407cf2640 1 --2- 172.21.15.82:0/3807443647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400100e10 0x7fe40010ede0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009cd0 tx=0x7fe3f802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.790+0000 7fe407cf2640 1 -- 172.21.15.82:0/3807443647 shutdown_connections 2024-04-21T08:50:06.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.790+0000 7fe407cf2640 1 --2- 172.21.15.82:0/3807443647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400100e10 0x7fe40010ede0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.790+0000 7fe407cf2640 1 --2- 172.21.15.82:0/3807443647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe400100450 0x7fe4001008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.790+0000 7fe407cf2640 1 --2- 172.21.15.82:0/3807443647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe400106f30 0x7fe400107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.790+0000 7fe407cf2640 1 -- 172.21.15.82:0/3807443647 >> 172.21.15.82:0/3807443647 conn(0x7fe400077410 msgr2=0x7fe4000fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:06.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.790+0000 7fe407cf2640 1 -- 172.21.15.82:0/3807443647 shutdown_connections 2024-04-21T08:50:06.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.790+0000 7fe407cf2640 1 -- 172.21.15.82:0/3807443647 wait complete. 2024-04-21T08:50:06.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe407cf2640 1 Processor -- start 2024-04-21T08:50:06.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe407cf2640 1 -- start start 2024-04-21T08:50:06.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe407cf2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe400100450 0x7fe40019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe407cf2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe400100e10 0x7fe40019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe407cf2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400106f30 0x7fe400198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe407cf2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001122f0 con 0x7fe400106f30 2024-04-21T08:50:06.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe407cf2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe400112170 con 0x7fe400100e10 2024-04-21T08:50:06.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe407cf2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe400112470 con 0x7fe400100450 2024-04-21T08:50:06.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe40628f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400106f30 0x7fe400198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe40628f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400106f30 0x7fe400198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36242/0 (socket says 172.21.15.82:36242) 2024-04-21T08:50:06.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe40628f640 1 -- 172.21.15.82:0/327634121 learned_addr learned my addr 172.21.15.82:0/327634121 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:50:06.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe40628f640 1 -- 172.21.15.82:0/327634121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe400100450 msgr2=0x7fe40019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:06.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe40628f640 1 --2- 172.21.15.82:0/327634121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe400100450 0x7fe40019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe40628f640 1 -- 172.21.15.82:0/327634121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe400100e10 msgr2=0x7fe40019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:06.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe40628f640 1 --2- 172.21.15.82:0/327634121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe400100e10 0x7fe40019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe40628f640 1 -- 172.21.15.82:0/327634121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe400198ef0 con 0x7fe400106f30 2024-04-21T08:50:06.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.791+0000 7fe40628f640 1 --2- 172.21.15.82:0/327634121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400106f30 0x7fe400198630 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8037040 tx=0x7fe3f8035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:06.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.792+0000 7fe3f6ffd640 1 -- 172.21.15.82:0/327634121 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8039040 con 0x7fe400106f30 2024-04-21T08:50:06.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.792+0000 7fe3f6ffd640 1 -- 172.21.15.82:0/327634121 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3f8044070 con 0x7fe400106f30 2024-04-21T08:50:06.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.792+0000 7fe407cf2640 1 -- 172.21.15.82:0/327634121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe400199180 con 0x7fe400106f30 2024-04-21T08:50:06.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.792+0000 7fe3f6ffd640 1 -- 172.21.15.82:0/327634121 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80042a0 con 0x7fe400106f30 2024-04-21T08:50:06.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.792+0000 7fe407cf2640 1 -- 172.21.15.82:0/327634121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4001a54f0 con 0x7fe400106f30 2024-04-21T08:50:06.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.794+0000 7fe3f6ffd640 1 -- 172.21.15.82:0/327634121 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe3f8004440 con 0x7fe400106f30 2024-04-21T08:50:06.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.794+0000 7fe40a9c1740 1 Processor -- start 2024-04-21T08:50:06.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe40a9c1740 1 -- start start 2024-04-21T08:50:06.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe3f6ffd640 1 --2- 172.21.15.82:0/327634121 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe3ec0a3da0 0x7fe3ec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe40a9c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4310 0x55f549eb46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe40a9c1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549ebd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe40a9c1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549ebd810 0x55f549ebfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe40a9c1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f549e19500 con 0x55f549eb4c30 2024-04-21T08:50:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe40a9c1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f549dbf6a0 con 0x55f549ebd810 2024-04-21T08:50:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe40a9c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f549e154c0 con 0x55f549eb4310 2024-04-21T08:50:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe3f6ffd640 1 -- 172.21.15.82:0/327634121 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe3f80f3040 con 0x7fe400106f30 2024-04-21T08:50:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe40528d640 1 --2- 172.21.15.82:0/327634121 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe3ec0a3da0 0x7fe3ec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe405a8e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4310 0x55f549eb46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe405a8e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4310 0x55f549eb46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:33444/0 (socket says 172.21.15.82:33444) 2024-04-21T08:50:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe40628f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549ebd810 0x55f549ebfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe405a8e640 1 -- 172.21.15.82:0/445324389 learned_addr learned my addr 172.21.15.82:0/445324389 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:50:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe40528d640 1 --2- 172.21.15.82:0/445324389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549ebd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe405a8e640 1 -- 172.21.15.82:0/445324389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549ebd810 msgr2=0x55f549ebfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe405a8e640 1 --2- 172.21.15.82:0/445324389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549ebd810 0x55f549ebfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe405a8e640 1 -- 172.21.15.82:0/445324389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 msgr2=0x55f549ebd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe405a8e640 1 --2- 172.21.15.82:0/445324389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549ebd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe405a8e640 1 -- 172.21.15.82:0/445324389 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f549dde5e0 con 0x55f549eb4310 2024-04-21T08:50:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe40628f640 1 --2- 172.21.15.82:0/445324389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549ebd810 0x55f549ebfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:50:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.795+0000 7fe40528d640 1 --2- 172.21.15.82:0/445324389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549ebd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:50:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.796+0000 7fe40528d640 1 --2- 172.21.15.82:0/327634121 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe3ec0a3da0 0x7fe3ec0a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000cfd0 tx=0x7fe3f000c890 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.796+0000 7fe405a8e640 1 --2- 172.21.15.82:0/445324389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4310 0x55f549eb46f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc009820 tx=0x7fe3fc009ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ef2801489a633a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.796+0000 7fe3ebfff640 1 -- 172.21.15.82:0/445324389 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc015020 con 0x55f549eb4310 2024-04-21T08:50:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.796+0000 7fe3ebfff640 1 -- 172.21.15.82:0/445324389 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3fc011420 con 0x55f549eb4310 2024-04-21T08:50:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.796+0000 7fe3ebfff640 1 -- 172.21.15.82:0/445324389 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0105e0 con 0x55f549eb4310 2024-04-21T08:50:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.797+0000 7fe40a9c1740 1 -- 172.21.15.82:0/445324389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4310 msgr2=0x55f549eb46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.797+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/445324389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4310 0x55f549eb46f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc009820 tx=0x7fe3fc009ce0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.797+0000 7fe40a9c1740 1 -- 172.21.15.82:0/445324389 shutdown_connections 2024-04-21T08:50:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.797+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/445324389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549ebd810 0x55f549ebfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.797+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/445324389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549ebd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.797+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/445324389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4310 0x55f549eb46f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.797+0000 7fe40a9c1740 1 -- 172.21.15.82:0/445324389 >> 172.21.15.82:0/445324389 conn(0x55f549eaea40 msgr2=0x55f549eaee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:06.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.797+0000 7fe40a9c1740 1 -- 172.21.15.82:0/445324389 shutdown_connections 2024-04-21T08:50:06.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.797+0000 7fe40a9c1740 1 -- 172.21.15.82:0/445324389 wait complete. 2024-04-21T08:50:06.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.797+0000 7fe40a9c1740 1 -- start start 2024-04-21T08:50:06.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.797+0000 7fe40a9c1740 1 Processor -- start 2024-04-21T08:50:06.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40a9c1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549ebd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40a9c1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549eba610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40a9c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549ebd810 0x55f549ebad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40a9c1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f549ec0600 con 0x55f549eb4c30 2024-04-21T08:50:06.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40a9c1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f549dd2a60 con 0x55f549eb4310 2024-04-21T08:50:06.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40a9c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f549ec0780 con 0x55f549ebd810 2024-04-21T08:50:06.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40a9c1740 10 client.0 _ll_register_callbacks cb 0x55f549b2b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:06.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40528d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549eba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40528d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549eba610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36262/0 (socket says 172.21.15.82:36262) 2024-04-21T08:50:06.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40528d640 1 -- 172.21.15.82:0/3551132012 learned_addr learned my addr 172.21.15.82:0/3551132012 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:50:06.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe405a8e640 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549ebd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40628f640 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549ebd810 0x55f549ebad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40528d640 1 -- 172.21.15.82:0/3551132012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549ebd810 msgr2=0x55f549ebad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:06.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40528d640 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549ebd810 0x55f549ebad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40528d640 1 -- 172.21.15.82:0/3551132012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 msgr2=0x55f549ebd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40528d640 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549ebd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40528d640 1 -- 172.21.15.82:0/3551132012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f549b2b920 con 0x55f549eb4c30 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40528d640 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549eba610 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00116a0 tx=0x7fe3f0005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf7750f4c71e404 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe3e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe3e97fa640 1 -- 172.21.15.82:0/3551132012 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f001e610 con 0x55f549eb4c30 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe3e97fa640 1 -- 172.21.15.82:0/3551132012 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe3f001e7b0 con 0x55f549eb4c30 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3551132012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3fc011420 con 0x55f549eb4c30 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe3e97fa640 1 -- 172.21.15.82:0/3551132012 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe3f0030050 con 0x55f549eb4c30 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40a9c1740 10 client.16854 fetch_fsmap learned FSMap version 32 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.798+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3551132012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f549bf7d30 con 0x55f549eb4c30 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.799+0000 7fe3e97fa640 1 -- 172.21.15.82:0/3551132012 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3fc011420 con 0x55f549eb4c30 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.799+0000 7fe3e97fa640 1 -- 172.21.15.82:0/3551132012 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7fe3f0027070 con 0x55f549eb4c30 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.799+0000 7fe40a9c1740 10 client.16854 fetch_fsmap finished waiting for FSMap version 32 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.799+0000 7fe40a9c1740 10 client.16854 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-21T08:50:06.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.799+0000 7fe40a9c1740 20 client.16854 populate_metadata read hostname 'smithi082' 2024-04-21T08:50:06.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.799+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f549ecd920 0x55f549ecfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:06.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.799+0000 7fe40a9c1740 4 client.16854 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:06.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.799+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3551132012 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f549deca10 con 0x55f549ecd920 2024-04-21T08:50:06.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.799+0000 7fe405a8e640 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f549ecd920 0x55f549ecfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:06.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.800+0000 7fe405a8e640 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f549ecd920 0x55f549ecfda0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe97f2d7596e44a server_cookie=6c4c493d57895b9c in_seq=0 out_seq=0 2024-04-21T08:50:06.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:50:06.800+0000 7fe3e97fa640 10 client.16854 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:50:07.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:07 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2902435416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:07.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:07 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2902435416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:07.747 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:07 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2902435416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:08.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:08 smithi082 ceph-mon[27967]: pgmap v96: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 149 KiB/s wr, 35 op/s 2024-04-21T08:50:08.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:08 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:50:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:08 smithi106 ceph-mon[31072]: pgmap v96: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 149 KiB/s wr, 35 op/s 2024-04-21T08:50:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:08 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:50:08.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:08 smithi134 ceph-mon[31772]: pgmap v96: 129 pgs: 129 active+clean; 408 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 149 KiB/s wr, 35 op/s 2024-04-21T08:50:08.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:08 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:50:09.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:09 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:50:09.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:09 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:50:09.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:09 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:50:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:09 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:50:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:09 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:50:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:09 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:50:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:09 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:50:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:09 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:50:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:09 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:50:10.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:10 smithi082 ceph-mon[27967]: pgmap v97: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 167 KiB/s wr, 36 op/s 2024-04-21T08:50:10.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:10 smithi106 ceph-mon[31072]: pgmap v97: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 167 KiB/s wr, 36 op/s 2024-04-21T08:50:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:10 smithi134 ceph-mon[31772]: pgmap v97: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 167 KiB/s wr, 36 op/s 2024-04-21T08:50:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:12 smithi082 ceph-mon[27967]: pgmap v98: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 73 KiB/s wr, 26 op/s 2024-04-21T08:50:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:12 smithi106 ceph-mon[31072]: pgmap v98: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 73 KiB/s wr, 26 op/s 2024-04-21T08:50:12.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:12 smithi134 ceph-mon[31772]: pgmap v98: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 73 KiB/s wr, 26 op/s 2024-04-21T08:50:14.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:14 smithi082 ceph-mon[27967]: pgmap v99: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 76 KiB/s wr, 31 op/s 2024-04-21T08:50:14.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:14 smithi106 ceph-mon[31072]: pgmap v99: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 76 KiB/s wr, 31 op/s 2024-04-21T08:50:14.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:14 smithi134 ceph-mon[31772]: pgmap v99: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 76 KiB/s wr, 31 op/s 2024-04-21T08:50:16.223 INFO:tasks.workunit.client.0.smithi082.stdout: 75 0 0 0 0 0 0 2024-04-21T08:50:16.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:16 smithi082 ceph-mon[27967]: pgmap v100: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2024-04-21T08:50:16.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:16 smithi106 ceph-mon[31072]: pgmap v100: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2024-04-21T08:50:16.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:16 smithi134 ceph-mon[31772]: pgmap v100: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 11 op/s 2024-04-21T08:50:18.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:18 smithi082 ceph-mon[27967]: pgmap v101: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-04-21T08:50:18.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:18 smithi106 ceph-mon[31072]: pgmap v101: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-04-21T08:50:18.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:18 smithi134 ceph-mon[31772]: pgmap v101: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-04-21T08:50:20.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:20 smithi082 ceph-mon[27967]: pgmap v102: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-04-21T08:50:20.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:20 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:50:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:20 smithi106 ceph-mon[31072]: pgmap v102: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-04-21T08:50:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:20 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:50:20.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:20 smithi134 ceph-mon[31772]: pgmap v102: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-04-21T08:50:20.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:20 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:50:22.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:22 smithi082 ceph-mon[27967]: pgmap v103: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-21T08:50:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:22 smithi106 ceph-mon[31072]: pgmap v103: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-21T08:50:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:22 smithi134 ceph-mon[31772]: pgmap v103: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-21T08:50:24.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:24 smithi106 ceph-mon[31072]: pgmap v104: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-21T08:50:24.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:24 smithi082 ceph-mon[27967]: pgmap v104: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-21T08:50:24.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:24 smithi134 ceph-mon[31772]: pgmap v104: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-21T08:50:26.223 INFO:tasks.workunit.client.0.smithi082.stdout: 75 0 0 0 0 0 0 2024-04-21T08:50:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:26 smithi082 ceph-mon[27967]: pgmap v105: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-21T08:50:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:26 smithi106 ceph-mon[31072]: pgmap v105: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-21T08:50:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:26 smithi134 ceph-mon[31772]: pgmap v105: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-21T08:50:28.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:28 smithi082 ceph-mon[27967]: pgmap v106: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-21T08:50:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:28 smithi106 ceph-mon[31072]: pgmap v106: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-21T08:50:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:28 smithi134 ceph-mon[31772]: pgmap v106: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-21T08:50:30.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:30 smithi082 ceph-mon[27967]: pgmap v107: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-04-21T08:50:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:30 smithi106 ceph-mon[31072]: pgmap v107: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-04-21T08:50:30.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:30 smithi134 ceph-mon[31772]: pgmap v107: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-04-21T08:50:32.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:32 smithi082 ceph-mon[27967]: pgmap v108: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-21T08:50:32.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:32 smithi106 ceph-mon[31072]: pgmap v108: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-21T08:50:32.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:32 smithi134 ceph-mon[31772]: pgmap v108: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-21T08:50:34.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:34 smithi082 ceph-mon[27967]: pgmap v109: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-21T08:50:34.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:34 smithi106 ceph-mon[31072]: pgmap v109: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-21T08:50:34.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:34 smithi134 ceph-mon[31772]: pgmap v109: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-21T08:50:36.223 INFO:tasks.workunit.client.0.smithi082.stdout: 75 0 0 0 0 0 0 2024-04-21T08:50:36.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:36 smithi082 ceph-mon[27967]: pgmap v110: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-21T08:50:36.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:36 smithi106 ceph-mon[31072]: pgmap v110: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-21T08:50:36.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:36 smithi134 ceph-mon[31772]: pgmap v110: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-21T08:50:38.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:38 smithi082 ceph-mon[27967]: pgmap v111: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-04-21T08:50:38.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:38 smithi106 ceph-mon[31072]: pgmap v111: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-04-21T08:50:38.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:38 smithi134 ceph-mon[31772]: pgmap v111: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-04-21T08:50:40.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:40 smithi082 ceph-mon[27967]: pgmap v112: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-21T08:50:40.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:40 smithi106 ceph-mon[31072]: pgmap v112: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-21T08:50:40.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:40 smithi134 ceph-mon[31772]: pgmap v112: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-21T08:50:42.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:42 smithi082 ceph-mon[27967]: pgmap v113: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-04-21T08:50:42.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:42 smithi106 ceph-mon[31072]: pgmap v113: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-04-21T08:50:42.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:42 smithi134 ceph-mon[31772]: pgmap v113: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-04-21T08:50:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:44 smithi082 ceph-mon[27967]: pgmap v114: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-04-21T08:50:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:44 smithi106 ceph-mon[31072]: pgmap v114: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-04-21T08:50:44.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:44 smithi134 ceph-mon[31772]: pgmap v114: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-04-21T08:50:46.223 INFO:tasks.workunit.client.0.smithi082.stdout: 75 0 0 0 0 0 0 2024-04-21T08:50:46.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:46 smithi082 ceph-mon[27967]: pgmap v115: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-04-21T08:50:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:46 smithi106 ceph-mon[31072]: pgmap v115: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-04-21T08:50:46.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:46 smithi134 ceph-mon[31772]: pgmap v115: 129 pgs: 129 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-04-21T08:50:48.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:48 smithi082 ceph-mon[27967]: pgmap v116: 129 pgs: 129 active+clean; 412 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 265 KiB/s wr, 12 op/s 2024-04-21T08:50:48.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:48 smithi106 ceph-mon[31072]: pgmap v116: 129 pgs: 129 active+clean; 412 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 265 KiB/s wr, 12 op/s 2024-04-21T08:50:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:48 smithi134 ceph-mon[31772]: pgmap v116: 129 pgs: 129 active+clean; 412 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 265 KiB/s wr, 12 op/s 2024-04-21T08:50:50.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:50 smithi082 ceph-mon[27967]: pgmap v117: 129 pgs: 129 active+clean; 412 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 14 op/s 2024-04-21T08:50:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:50 smithi106 ceph-mon[31072]: pgmap v117: 129 pgs: 129 active+clean; 412 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 14 op/s 2024-04-21T08:50:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:50 smithi134 ceph-mon[31772]: pgmap v117: 129 pgs: 129 active+clean; 412 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 270 KiB/s wr, 14 op/s 2024-04-21T08:50:52.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:52 smithi082 ceph-mon[27967]: pgmap v118: 129 pgs: 129 active+clean; 412 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 10 op/s 2024-04-21T08:50:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:52 smithi106 ceph-mon[31072]: pgmap v118: 129 pgs: 129 active+clean; 412 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 10 op/s 2024-04-21T08:50:52.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:52 smithi134 ceph-mon[31772]: pgmap v118: 129 pgs: 129 active+clean; 412 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 266 KiB/s wr, 10 op/s 2024-04-21T08:50:54.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:54 smithi082 ceph-mon[27967]: pgmap v119: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 405 KiB/s wr, 17 op/s 2024-04-21T08:50:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:54 smithi106 ceph-mon[31072]: pgmap v119: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 405 KiB/s wr, 17 op/s 2024-04-21T08:50:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:54 smithi134 ceph-mon[31772]: pgmap v119: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 405 KiB/s wr, 17 op/s 2024-04-21T08:50:56.223 INFO:tasks.workunit.client.0.smithi082.stdout: 75 0 0 0 0 0 0 2024-04-21T08:50:56.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:56 smithi106 ceph-mon[31072]: pgmap v120: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 404 KiB/s wr, 12 op/s 2024-04-21T08:50:56.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:56 smithi134 ceph-mon[31772]: pgmap v120: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 404 KiB/s wr, 12 op/s 2024-04-21T08:50:56.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:56 smithi082 ceph-mon[27967]: pgmap v120: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 404 KiB/s wr, 12 op/s 2024-04-21T08:50:58.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:50:58 smithi106 ceph-mon[31072]: pgmap v121: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 404 KiB/s wr, 14 op/s 2024-04-21T08:50:58.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:50:58 smithi134 ceph-mon[31772]: pgmap v121: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 404 KiB/s wr, 14 op/s 2024-04-21T08:50:58.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:50:58 smithi082 ceph-mon[27967]: pgmap v121: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 404 KiB/s wr, 14 op/s 2024-04-21T08:51:00.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:00 smithi106 ceph-mon[31072]: pgmap v122: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2024-04-21T08:51:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:00 smithi134 ceph-mon[31772]: pgmap v122: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2024-04-21T08:51:00.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:00 smithi082 ceph-mon[27967]: pgmap v122: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 155 KiB/s wr, 15 op/s 2024-04-21T08:51:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:02 smithi106 ceph-mon[31072]: pgmap v123: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 12 op/s 2024-04-21T08:51:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:02 smithi134 ceph-mon[31772]: pgmap v123: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 12 op/s 2024-04-21T08:51:02.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:02 smithi082 ceph-mon[27967]: pgmap v123: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 150 KiB/s wr, 12 op/s 2024-04-21T08:51:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:04 smithi106 ceph-mon[31072]: pgmap v124: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 175 KiB/s wr, 31 op/s 2024-04-21T08:51:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:04 smithi134 ceph-mon[31772]: pgmap v124: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 175 KiB/s wr, 31 op/s 2024-04-21T08:51:04.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:04 smithi082 ceph-mon[27967]: pgmap v124: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 175 KiB/s wr, 31 op/s 2024-04-21T08:51:06.223 INFO:tasks.workunit.client.0.smithi082.stdout: 75 0 0 0 0 0 0 2024-04-21T08:51:06.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:06 smithi106 ceph-mon[31072]: pgmap v125: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 KiB/s wr, 25 op/s 2024-04-21T08:51:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:06 smithi134 ceph-mon[31772]: pgmap v125: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 KiB/s wr, 25 op/s 2024-04-21T08:51:06.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:06 smithi082 ceph-mon[27967]: pgmap v125: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 KiB/s wr, 25 op/s 2024-04-21T08:51:06.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.994+0000 7fc95cff9640 1 -- 172.21.15.82:0/2297153758 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e600c5c00 con 0x562e6016c910 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.994+0000 7fc95cff9640 10 client.16836 handle_command_reply: tid=0 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.994+0000 7fc97e47d740 1 client.16836 shutdown 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.994+0000 7fc97e47d740 1 -- 172.21.15.82:0/2297153758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 msgr2=0x562e60159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.994+0000 7fc97e47d740 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 0x562e60159a10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc97000ba00 tx=0x7fc970011440 comp rx=0 tx=0).stop 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.994+0000 7fc97e47d740 1 -- 172.21.15.82:0/2297153758 shutdown_connections 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.994+0000 7fc97e47d740 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e6016c910 0x562e6016ed00 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.994+0000 7fc97e47d740 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e6015ca80 0x562e6015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.994+0000 7fc95cff9640 0 client.16836 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.994+0000 7fc97e47d740 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e60153ea0 0x562e60159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.994+0000 7fc97e47d740 1 --2- 172.21.15.82:0/2297153758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 0x562e60159a10 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.994+0000 7fc97e47d740 1 -- 172.21.15.82:0/2297153758 >> 172.21.15.82:0/2297153758 conn(0x562e6014dcb0 msgr2=0x562e6014fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fc97e47d740 1 -- 172.21.15.82:0/2297153758 shutdown_connections 2024-04-21T08:51:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fc97e47d740 1 -- 172.21.15.82:0/2297153758 wait complete. 2024-04-21T08:51:06.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe3e97fa640 1 -- 172.21.15.82:0/3551132012 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f549deca10 con 0x55f549ecd920 2024-04-21T08:51:06.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe3e97fa640 10 client.16854 handle_command_reply: tid=0 2024-04-21T08:51:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f10fb7fe640 1 -- 172.21.15.82:0/1959971275 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654c137e3d0 con 0x5654c14f9ac0 2024-04-21T08:51:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f10fb7fe640 10 client.16824 handle_command_reply: tid=0 2024-04-21T08:51:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f112537b740 1 client.16824 shutdown 2024-04-21T08:51:06.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe40a9c1740 1 client.16854 shutdown 2024-04-21T08:51:06.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fc97e47d740 20 client.16836 trim_cache size 0 max 16384 2024-04-21T08:51:06.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3551132012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 msgr2=0x55f549eba610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:06.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549eba610 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00116a0 tx=0x7fe3f0005ac0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f112537b740 1 -- 172.21.15.82:0/1959971275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 msgr2=0x5654c148b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f112537b740 1 --2- 172.21.15.82:0/1959971275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 0x5654c148b2f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1104004ad0 tx=0x7f1104099010 comp rx=0 tx=0).stop 2024-04-21T08:51:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f112537b740 1 -- 172.21.15.82:0/1959971275 shutdown_connections 2024-04-21T08:51:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f112537b740 1 --2- 172.21.15.82:0/1959971275 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5654c14f9ac0 0x5654c14fbee0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f112537b740 1 --2- 172.21.15.82:0/1959971275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5654c14830e0 0x5654c1486b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f112537b740 1 --2- 172.21.15.82:0/1959971275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5654c147a8d0 0x5654c1486650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f112537b740 1 --2- 172.21.15.82:0/1959971275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5654c1479b60 0x5654c148b2f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f112537b740 1 -- 172.21.15.82:0/1959971275 >> 172.21.15.82:0/1959971275 conn(0x5654c1440c90 msgr2=0x5654c147e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3551132012 shutdown_connections 2024-04-21T08:51:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f549ecd920 0x55f549ecfda0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549ebd810 0x55f549ebad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549eba610 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/3551132012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549ebd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe3e97fa640 0 client.16854 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:07.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3551132012 >> 172.21.15.82:0/3551132012 conn(0x55f549eaea40 msgr2=0x55f549eb0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f10fb7fe640 0 client.16824 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f112537b740 1 -- 172.21.15.82:0/1959971275 shutdown_connections 2024-04-21T08:51:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f112537b740 1 -- 172.21.15.82:0/1959971275 wait complete. 2024-04-21T08:51:07.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3551132012 shutdown_connections 2024-04-21T08:51:07.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3551132012 wait complete. 2024-04-21T08:51:07.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:06.995+0000 7fe40a9c1740 20 client.16854 trim_cache size 0 max 16384 2024-04-21T08:51:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.995+0000 7f112537b740 20 client.16824 trim_cache size 0 max 16384 2024-04-21T08:51:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:51:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:51:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:51:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.996+0000 7f1122784640 1 -- 172.21.15.82:0/446672248 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f11080a3dc0 msgr2=0x7f11080a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.996+0000 7f1122784640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f11080a3dc0 0x7f11080a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1114004af0 tx=0x7f111402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.996+0000 7f1122784640 1 -- 172.21.15.82:0/446672248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1010f0 msgr2=0x7f111c1a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.996+0000 7f1122784640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1010f0 0x7f111c1a0600 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f110400e9f0 tx=0x7f110400edf0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.996+0000 7f1122784640 1 -- 172.21.15.82:0/446672248 shutdown_connections 2024-04-21T08:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.996+0000 7f1122784640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f111c19a6c0 0x7f111c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.996+0000 7f1122784640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f111c106170 0x7f111c1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.996+0000 7f1122784640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f11080a3dc0 0x7f11080a6280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.996+0000 7f1122784640 1 --2- 172.21.15.82:0/446672248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111c1010f0 0x7f111c1a0600 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.996+0000 7f1122784640 1 -- 172.21.15.82:0/446672248 >> 172.21.15.82:0/446672248 conn(0x7f111c077fd0 msgr2=0x7f111c10c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.997+0000 7f1122784640 1 -- 172.21.15.82:0/446672248 shutdown_connections 2024-04-21T08:51:07.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:06.997+0000 7f1122784640 1 -- 172.21.15.82:0/446672248 wait complete. 2024-04-21T08:51:07.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.036+0000 7fc97e47d740 1 Processor -- start 2024-04-21T08:51:07.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.036+0000 7fc97e47d740 1 -- start start 2024-04-21T08:51:07.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.036+0000 7fc97e47d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e60153580 0x562e6015c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.036+0000 7fc97e47d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 0x562e60168730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.036+0000 7fc97e47d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e6015ca80 0x562e601589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.036+0000 7fc97e47d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741b14e0 con 0x562e60153ea0 2024-04-21T08:51:07.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.036+0000 7fc97e47d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741b3110 con 0x562e6015ca80 2024-04-21T08:51:07.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.036+0000 7fc97e47d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741b1bb0 con 0x562e60153580 2024-04-21T08:51:07.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc979e2b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e6015ca80 0x562e601589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc97962a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e60153580 0x562e6015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc979e2b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e6015ca80 0x562e601589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:46426/0 (socket says 172.21.15.82:46426) 2024-04-21T08:51:07.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc979e2b640 1 -- 172.21.15.82:0/286609285 learned_addr learned my addr 172.21.15.82:0/286609285 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:07.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc978e29640 1 --2- 172.21.15.82:0/286609285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 0x562e60168730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc979e2b640 1 -- 172.21.15.82:0/286609285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e60153580 msgr2=0x562e6015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc979e2b640 1 --2- 172.21.15.82:0/286609285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e60153580 0x562e6015c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc979e2b640 1 -- 172.21.15.82:0/286609285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 msgr2=0x562e60168730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc979e2b640 1 --2- 172.21.15.82:0/286609285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 0x562e60168730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc979e2b640 1 -- 172.21.15.82:0/286609285 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e60158f30 con 0x562e6015ca80 2024-04-21T08:51:07.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc97962a640 1 --2- 172.21.15.82:0/286609285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e60153580 0x562e6015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:07.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc978e29640 1 --2- 172.21.15.82:0/286609285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 0x562e60168730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:07.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.037+0000 7fc979e2b640 1 --2- 172.21.15.82:0/286609285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e6015ca80 0x562e601589f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc96c010620 tx=0x7fc96c009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bc732b1d68711c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:07.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.038+0000 7fc95effd640 1 -- 172.21.15.82:0/286609285 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c004b20 con 0x562e6015ca80 2024-04-21T08:51:07.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.038+0000 7fc95effd640 1 -- 172.21.15.82:0/286609285 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc96c004cc0 con 0x562e6015ca80 2024-04-21T08:51:07.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.038+0000 7fc95effd640 1 -- 172.21.15.82:0/286609285 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c005390 con 0x562e6015ca80 2024-04-21T08:51:07.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe40a9c1740 1 Processor -- start 2024-04-21T08:51:07.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe40a9c1740 1 -- start start 2024-04-21T08:51:07.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe40a9c1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549ec5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe40a9c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4c30 0x55f549ebcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe40a9c1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549ebd810 0x55f549ebd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fc97e47d740 1 -- 172.21.15.82:0/286609285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e6015ca80 msgr2=0x562e601589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fc97e47d740 1 --2- 172.21.15.82:0/286609285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e6015ca80 0x562e601589f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc96c010620 tx=0x7fc96c009770 comp rx=0 tx=0).stop 2024-04-21T08:51:07.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe40a9c1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001af370 con 0x55f549ebd810 2024-04-21T08:51:07.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe40a9c1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001b0fa0 con 0x55f549eb4310 2024-04-21T08:51:07.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe40a9c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001afa40 con 0x55f549eb4c30 2024-04-21T08:51:07.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe40528d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4c30 0x55f549ebcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fc97e47d740 1 -- 172.21.15.82:0/286609285 shutdown_connections 2024-04-21T08:51:07.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fc97e47d740 1 --2- 172.21.15.82:0/286609285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e6015ca80 0x562e601589f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fc97e47d740 1 --2- 172.21.15.82:0/286609285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153ea0 0x562e60168730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fc97e47d740 1 --2- 172.21.15.82:0/286609285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e60153580 0x562e6015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fc97e47d740 1 -- 172.21.15.82:0/286609285 >> 172.21.15.82:0/286609285 conn(0x562e6014dcb0 msgr2=0x562e601582e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe405a8e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549ec5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe40528d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4c30 0x55f549ebcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:36206/0 (socket says 172.21.15.82:36206) 2024-04-21T08:51:07.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe40528d640 1 -- 172.21.15.82:0/3806840089 learned_addr learned my addr 172.21.15.82:0/3806840089 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:07.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.039+0000 7fe40628f640 1 --2- 172.21.15.82:0/3806840089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549ebd810 0x55f549ebd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fc97e47d740 1 -- 172.21.15.82:0/286609285 shutdown_connections 2024-04-21T08:51:07.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fc97e47d740 1 -- 172.21.15.82:0/286609285 wait complete. 2024-04-21T08:51:07.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fe405a8e640 1 -- 172.21.15.82:0/3806840089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4c30 msgr2=0x55f549ebcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fe405a8e640 1 --2- 172.21.15.82:0/3806840089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4c30 0x55f549ebcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fe405a8e640 1 -- 172.21.15.82:0/3806840089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549ebd810 msgr2=0x55f549ebd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fe405a8e640 1 --2- 172.21.15.82:0/3806840089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549ebd810 0x55f549ebd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fe405a8e640 1 -- 172.21.15.82:0/3806840089 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f549eaf610 con 0x55f549eb4310 2024-04-21T08:51:07.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fe40528d640 1 --2- 172.21.15.82:0/3806840089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4c30 0x55f549ebcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:07.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fe40628f640 1 --2- 172.21.15.82:0/3806840089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549ebd810 0x55f549ebd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:07.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fe405a8e640 1 --2- 172.21.15.82:0/3806840089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549ec5db0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc011d60 tx=0x7fe3fc01c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f77eccc1ed724615 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:07.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fe3eb7fe640 1 -- 172.21.15.82:0/3806840089 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc015040 con 0x55f549eb4310 2024-04-21T08:51:07.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fe3eb7fe640 1 -- 172.21.15.82:0/3806840089 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3fc00b4b0 con 0x55f549eb4310 2024-04-21T08:51:07.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fe3eb7fe640 1 -- 172.21.15.82:0/3806840089 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc011950 con 0x55f549eb4310 2024-04-21T08:51:07.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fc97e47d740 1 -- start start 2024-04-21T08:51:07.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.040+0000 7fc97e47d740 1 Processor -- start 2024-04-21T08:51:07.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc97e47d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 0x562e60164ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc97e47d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e60153ea0 0x562e60160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc97e47d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e6015ca80 0x562e60160740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc97e47d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741b2a10 con 0x562e60153580 2024-04-21T08:51:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc97e47d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741b2340 con 0x562e6015ca80 2024-04-21T08:51:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc97e47d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9741af4c0 con 0x562e60153ea0 2024-04-21T08:51:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc97e47d740 10 client.0 _ll_register_callbacks cb 0x7fc97000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc97962a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 0x562e60164ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc978e29640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e60153ea0 0x562e60160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc979e2b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e6015ca80 0x562e60160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc97962a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 0x562e60164ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58762/0 (socket says 172.21.15.82:58762) 2024-04-21T08:51:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc97962a640 1 -- 172.21.15.82:0/2821028842 learned_addr learned my addr 172.21.15.82:0/2821028842 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:07.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3806840089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 msgr2=0x55f549ec5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/3806840089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549ec5db0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc011d60 tx=0x7fe3fc01c040 comp rx=0 tx=0).stop 2024-04-21T08:51:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3806840089 shutdown_connections 2024-04-21T08:51:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/3806840089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549ebd810 0x55f549ebd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/3806840089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549eb4c30 0x55f549ebcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/3806840089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549ec5db0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3806840089 >> 172.21.15.82:0/3806840089 conn(0x55f549eaea40 msgr2=0x55f549eadb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc978e29640 1 -- 172.21.15.82:0/2821028842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e6015ca80 msgr2=0x562e60160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc978e29640 1 --2- 172.21.15.82:0/2821028842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e6015ca80 0x562e60160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc978e29640 1 -- 172.21.15.82:0/2821028842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 msgr2=0x562e60164ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc978e29640 1 --2- 172.21.15.82:0/2821028842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 0x562e60164ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.041+0000 7fc978e29640 1 -- 172.21.15.82:0/2821028842 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e60160d80 con 0x562e60153ea0 2024-04-21T08:51:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3806840089 shutdown_connections 2024-04-21T08:51:07.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fe40a9c1740 1 -- 172.21.15.82:0/3806840089 wait complete. 2024-04-21T08:51:07.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fc978e29640 1 --2- 172.21.15.82:0/2821028842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e60153ea0 0x562e60160200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc9640f0160 tx=0x7fc964088fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a5b5bcade7ae7f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:07.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fc95cff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:51:07.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fc95cff9640 1 -- 172.21.15.82:0/2821028842 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9640894f0 con 0x562e60153ea0 2024-04-21T08:51:07.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fe40a9c1740 1 -- start start 2024-04-21T08:51:07.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fc95cff9640 1 -- 172.21.15.82:0/2821028842 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc9640054d0 con 0x562e60153ea0 2024-04-21T08:51:07.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fc97e47d740 1 -- 172.21.15.82:0/2821028842 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc96c004cc0 con 0x562e60153ea0 2024-04-21T08:51:07.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fe40a9c1740 1 Processor -- start 2024-04-21T08:51:07.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fc95cff9640 1 -- 172.21.15.82:0/2821028842 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc9640bf0e0 con 0x562e60153ea0 2024-04-21T08:51:07.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fc97e47d740 10 client.25951 fetch_fsmap learned FSMap version 33 2024-04-21T08:51:07.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fc97e47d740 1 -- 172.21.15.82:0/2821028842 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562e601d2af0 con 0x562e60153ea0 2024-04-21T08:51:07.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fc95cff9640 1 -- 172.21.15.82:0/2821028842 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc96c004cc0 con 0x562e60153ea0 2024-04-21T08:51:07.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fe40a9c1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549eb7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fe40a9c1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549eb7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fe40a9c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549ebd810 0x55f549eb8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fe40a9c1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001b08a0 con 0x55f549eb4c30 2024-04-21T08:51:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fe40a9c1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001b01d0 con 0x55f549eb4310 2024-04-21T08:51:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fe40a9c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4001ad350 con 0x55f549ebd810 2024-04-21T08:51:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.042+0000 7fe40a9c1740 10 client.0 _ll_register_callbacks cb 0x7fe3f0011580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fc95cff9640 1 -- 172.21.15.82:0/2821028842 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7fc9640f0300 con 0x562e60153ea0 2024-04-21T08:51:07.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe405a8e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549eb7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe40528d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549eb7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe405a8e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549eb7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:46444/0 (socket says 172.21.15.82:46444) 2024-04-21T08:51:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe405a8e640 1 -- 172.21.15.82:0/1036382751 learned_addr learned my addr 172.21.15.82:0/1036382751 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe40628f640 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549ebd810 0x55f549eb8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fc97e47d740 10 client.25951 fetch_fsmap finished waiting for FSMap version 33 2024-04-21T08:51:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fc97e47d740 10 client.25951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fc97e47d740 20 client.25951 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fc97e47d740 1 --2- 172.21.15.82:0/2821028842 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e6016c910 0x562e601d5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fc97e47d740 4 client.25951 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:51:07.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fc97e47d740 1 -- 172.21.15.82:0/2821028842 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e600bfa70 con 0x562e6016c910 2024-04-21T08:51:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe40528d640 1 -- 172.21.15.82:0/1036382751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549ebd810 msgr2=0x55f549eb8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe40528d640 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549ebd810 0x55f549eb8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe40528d640 1 -- 172.21.15.82:0/1036382751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 msgr2=0x55f549eb7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe40528d640 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549eb7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe40528d640 1 -- 172.21.15.82:0/1036382751 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f549ebddd0 con 0x55f549eb4c30 2024-04-21T08:51:07.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe405a8e640 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549eb7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fc97962a640 1 --2- 172.21.15.82:0/2821028842 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e6016c910 0x562e601d5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe40528d640 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549eb7dd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0004700 tx=0x7fe3f0015070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711e2f9ca5189503 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:07.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe3e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:51:07.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe3e97fa640 1 -- 172.21.15.82:0/1036382751 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0013400 con 0x55f549eb4c30 2024-04-21T08:51:07.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe3e97fa640 1 -- 172.21.15.82:0/1036382751 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe3f0004020 con 0x55f549eb4c30 2024-04-21T08:51:07.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fe40a9c1740 1 -- 172.21.15.82:0/1036382751 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3fc00b4b0 con 0x55f549eb4c30 2024-04-21T08:51:07.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fc97962a640 1 --2- 172.21.15.82:0/2821028842 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e6016c910 0x562e601d5260 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f17931d4d2fb28 server_cookie=31c7c3109c908aa1 in_seq=0 out_seq=0 2024-04-21T08:51:07.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.043+0000 7fc95cff9640 10 client.25951 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:07.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fe3e97fa640 1 -- 172.21.15.82:0/1036382751 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe3f0004220 con 0x55f549eb4c30 2024-04-21T08:51:07.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fe40a9c1740 10 client.16866 fetch_fsmap learned FSMap version 33 2024-04-21T08:51:07.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fe40a9c1740 1 -- 172.21.15.82:0/1036382751 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f549f39390 con 0x55f549eb4c30 2024-04-21T08:51:07.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc95cff9640 1 -- 172.21.15.82:0/2821028842 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562e600bfa70 con 0x562e6016c910 2024-04-21T08:51:07.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc95cff9640 10 client.25951 handle_command_reply: tid=0 2024-04-21T08:51:07.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc97e47d740 1 client.25951 shutdown 2024-04-21T08:51:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fe3e97fa640 1 -- 172.21.15.82:0/1036382751 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3fc00b4b0 con 0x55f549eb4c30 2024-04-21T08:51:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc97e47d740 1 -- 172.21.15.82:0/2821028842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e60153ea0 msgr2=0x562e60160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc97e47d740 1 --2- 172.21.15.82:0/2821028842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e60153ea0 0x562e60160200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc9640f0160 tx=0x7fc964088fe0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc97e47d740 1 -- 172.21.15.82:0/2821028842 shutdown_connections 2024-04-21T08:51:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc97e47d740 1 --2- 172.21.15.82:0/2821028842 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562e6016c910 0x562e601d5260 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc97e47d740 1 --2- 172.21.15.82:0/2821028842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562e6015ca80 0x562e60160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc97e47d740 1 --2- 172.21.15.82:0/2821028842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562e60153ea0 0x562e60160200 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc97e47d740 1 --2- 172.21.15.82:0/2821028842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e60153580 0x562e60164ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc97e47d740 1 -- 172.21.15.82:0/2821028842 >> 172.21.15.82:0/2821028842 conn(0x562e6014dcb0 msgr2=0x562e601599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc95cff9640 0 client.25951 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:07.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fe3e97fa640 1 -- 172.21.15.82:0/1036382751 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) ==== 8378+0+0 (secure 0 0 0) 0x7fe3f0027040 con 0x55f549eb4c30 2024-04-21T08:51:07.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fe40a9c1740 10 client.16866 fetch_fsmap finished waiting for FSMap version 33 2024-04-21T08:51:07.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fe40a9c1740 10 client.16866 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-21T08:51:07.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fe40a9c1740 20 client.16866 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:07.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc97e47d740 1 -- 172.21.15.82:0/2821028842 shutdown_connections 2024-04-21T08:51:07.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f549f39a80 0x55f549f3bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fe40a9c1740 4 client.16866 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9b977ccd", "release": true, "await": true}] 2024-04-21T08:51:07.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fe40a9c1740 1 -- 172.21.15.82:0/1036382751 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9b977ccd", "release": true, "await": true}) -- 0x55f549e26f20 con 0x55f549f39a80 2024-04-21T08:51:07.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc97e47d740 1 -- 172.21.15.82:0/2821028842 wait complete. 2024-04-21T08:51:07.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fc97e47d740 20 client.25951 trim_cache size 0 max 16384 2024-04-21T08:51:07.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.044+0000 7fe405a8e640 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f549f39a80 0x55f549f3bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.064 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:51:07.064 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:51:07.064 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:51:07.065 INFO:teuthology.orchestra.run.smithi082.stdout: "480a3453-9908-4a12-b55d-cbecc05557f7": { 2024-04-21T08:51:07.065 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:51:07.065 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "480a3453-9908-4a12-b55d-cbecc05557f7", 2024-04-21T08:51:07.065 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:51:07.065 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:51:07.065 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:51:07.065 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:51:07.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe3e97fa640 10 client.16866 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:07.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe405a8e640 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f549f39a80 0x55f549f3bea0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a4fdc9f18fbd1a9 server_cookie=5083dcf1c02f0d24 in_seq=0 out_seq=0 2024-04-21T08:51:07.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fc97b88e640 1 -- 172.21.15.82:0/560581383 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc9600a3d20 msgr2=0x7fc9600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fc97b88e640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc9600a3d20 0x7fc9600a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc96c005c40 tx=0x7fc96c0094a0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fc97b88e640 1 -- 172.21.15.82:0/560581383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc974101a40 msgr2=0x7fc9740ff0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fc97b88e640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc974101a40 0x7fc9740ff0e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc964005940 tx=0x7fc96402c7e0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fc97b88e640 1 -- 172.21.15.82:0/560581383 shutdown_connections 2024-04-21T08:51:07.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fc97b88e640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc974105dd0 0x7fc9740ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fc97b88e640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc9600a3d20 0x7fc9600a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fc97b88e640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc974101a40 0x7fc9740ff0e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fc97b88e640 1 --2- 172.21.15.82:0/560581383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9741010a0 0x7fc9740feba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fc97b88e640 1 -- 172.21.15.82:0/560581383 >> 172.21.15.82:0/560581383 conn(0x7fc974077c40 msgr2=0x7fc97406a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fc97b88e640 1 -- 172.21.15.82:0/560581383 shutdown_connections 2024-04-21T08:51:07.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fc97b88e640 1 -- 172.21.15.82:0/560581383 wait complete. 2024-04-21T08:51:07.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe3e97fa640 1 -- 172.21.15.82:0/1036382751 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: -1 ) ==== 8+0+588 (crc 0 0 0) 0x55f549e26f20 con 0x55f549f39a80 2024-04-21T08:51:07.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe3e97fa640 10 client.16866 handle_command_reply: tid=0 2024-04-21T08:51:07.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe40a9c1740 1 client.16866 shutdown 2024-04-21T08:51:07.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe40a9c1740 1 -- 172.21.15.82:0/1036382751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 msgr2=0x55f549eb7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549eb7dd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0004700 tx=0x7fe3f0015070 comp rx=0 tx=0).stop 2024-04-21T08:51:07.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe40a9c1740 1 -- 172.21.15.82:0/1036382751 shutdown_connections 2024-04-21T08:51:07.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55f549f39a80 0x55f549f3bea0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f549ebd810 0x55f549eb8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f549eb4c30 0x55f549eb7dd0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe40a9c1740 1 --2- 172.21.15.82:0/1036382751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f549eb4310 0x55f549eb7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe40a9c1740 1 -- 172.21.15.82:0/1036382751 >> 172.21.15.82:0/1036382751 conn(0x55f549eaea40 msgr2=0x55f549eafbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe3e97fa640 0 client.16866 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:07.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe40a9c1740 1 -- 172.21.15.82:0/1036382751 shutdown_connections 2024-04-21T08:51:07.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.045+0000 7fe40a9c1740 1 -- 172.21.15.82:0/1036382751 wait complete. 2024-04-21T08:51:07.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:07.046+0000 7fe40a9c1740 20 client.16866 trim_cache size 0 max 16384 2024-04-21T08:51:07.069 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: "epoch": 33, 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: "leader": 24509, 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: "set_version": 3, 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: "sets": { 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: "9b977ccd": { 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: "version": 3, 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: "age_ref": 0.7, 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: "state": { 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "EXPIRED", 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: "age": 0.0 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: "timeout": 30.0, 2024-04-21T08:51:07.070 INFO:teuthology.orchestra.run.smithi082.stdout: "expiration": 90.0, 2024-04-21T08:51:07.071 INFO:teuthology.orchestra.run.smithi082.stdout: "members": { 2024-04-21T08:51:07.071 INFO:teuthology.orchestra.run.smithi082.stdout: "file:/": { 2024-04-21T08:51:07.071 INFO:teuthology.orchestra.run.smithi082.stdout: "excluded": false, 2024-04-21T08:51:07.071 INFO:teuthology.orchestra.run.smithi082.stdout: "state": { 2024-04-21T08:51:07.071 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "QUIESCED", 2024-04-21T08:51:07.071 INFO:teuthology.orchestra.run.smithi082.stdout: "age": 90.0 2024-04-21T08:51:07.071 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:51:07.071 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:51:07.071 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:51:07.071 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:51:07.071 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:51:07.071 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:51:07.072 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-21T08:51:07.073 INFO:teuthology.orchestra.run.smithi082.stderr:Error EPERM: 2024-04-21T08:51:07.073 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't release root with rc: -124 (Unknown), stdout: { "epoch": 33, "leader": 24509, "set_version": 3, "sets": { "9b977ccd": { "version": 3, "age_ref": 0.7, "state": { "name": "EXPIRED", "age": 0.0 }, "timeout": 30.0, "expiration": 90.0, "members": { "file:/": { "excluded": false, "state": { "name": "QUIESCED", "age": 90.0 } } } } } } 2024-04-21T08:51:07.073 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/quiescer.py", line 240, in _run self.do_quiesce(d) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/quiescer.py", line 216, in do_quiesce raise RuntimeError(f"Error releasing root: {rcinfo}") RuntimeError: Error releasing root: -124 (Unknown) 2024-04-21T08:51:07.075 INFO:tasks.cephfs.filesystem:scrub status for tag:480a3453-9908-4a12-b55d-cbecc05557f7 - {'path': '/', 'tag': '480a3453-9908-4a12-b55d-cbecc05557f7', 'options': 'recursive,force'} 2024-04-21T08:51:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:08 smithi106 ceph-mon[31072]: pgmap v126: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 KiB/s wr, 28 op/s 2024-04-21T08:51:08.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:08 smithi134 ceph-mon[31772]: pgmap v126: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 KiB/s wr, 28 op/s 2024-04-21T08:51:08.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:08 smithi082 ceph-mon[27967]: pgmap v126: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 KiB/s wr, 28 op/s 2024-04-21T08:51:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:09 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:09 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:51:09.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:09 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:09.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:09 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:51:09.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:09 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:09.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:09 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:51:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:10 smithi134 ceph-mon[31772]: pgmap v127: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 KiB/s wr, 29 op/s 2024-04-21T08:51:11.245 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:10 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:51:11.245 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:10 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:51:11.245 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:10 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:51:11.247 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:10 smithi106 ceph-mon[31072]: pgmap v127: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 KiB/s wr, 29 op/s 2024-04-21T08:51:11.247 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:10 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:51:11.247 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:10 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:51:11.247 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:10 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:51:11.248 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:10 smithi082 ceph-mon[27967]: pgmap v127: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 48 KiB/s wr, 29 op/s 2024-04-21T08:51:11.248 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:10 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:51:11.248 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:10 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:51:11.248 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:10 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:51:12.680 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:12 smithi106 ceph-mon[31072]: pgmap v128: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 KiB/s wr, 25 op/s 2024-04-21T08:51:12.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:12 smithi134 ceph-mon[31772]: pgmap v128: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 KiB/s wr, 25 op/s 2024-04-21T08:51:12.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:12 smithi082 ceph-mon[27967]: pgmap v128: 129 pgs: 129 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 KiB/s wr, 25 op/s 2024-04-21T08:51:14.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:14 smithi134 ceph-mon[31772]: pgmap v129: 129 pgs: 129 active+clean; 421 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.0 MiB/s wr, 192 op/s 2024-04-21T08:51:14.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:14 smithi082 ceph-mon[27967]: pgmap v129: 129 pgs: 129 active+clean; 421 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.0 MiB/s wr, 192 op/s 2024-04-21T08:51:14.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:14 smithi106 ceph-mon[31072]: pgmap v129: 129 pgs: 129 active+clean; 421 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.0 MiB/s wr, 192 op/s 2024-04-21T08:51:16.228 INFO:tasks.workunit.client.0.smithi082.stdout: 79 920864 446 639151 291 481100 1101 2024-04-21T08:51:16.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:16 smithi134 ceph-mon[31772]: pgmap v130: 129 pgs: 129 active+clean; 421 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.0 MiB/s wr, 173 op/s 2024-04-21T08:51:16.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:16 smithi082 ceph-mon[27967]: pgmap v130: 129 pgs: 129 active+clean; 421 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.0 MiB/s wr, 173 op/s 2024-04-21T08:51:16.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:16 smithi106 ceph-mon[31072]: pgmap v130: 129 pgs: 129 active+clean; 421 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.0 MiB/s wr, 173 op/s 2024-04-21T08:51:18.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:18 smithi134 ceph-mon[31772]: pgmap v131: 129 pgs: 129 active+clean; 426 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-04-21T08:51:18.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:18 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:51:18.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:18 smithi082 ceph-mon[27967]: pgmap v131: 129 pgs: 129 active+clean; 426 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-04-21T08:51:18.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:18 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:51:18.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:18 smithi106 ceph-mon[31072]: pgmap v131: 129 pgs: 129 active+clean; 426 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-04-21T08:51:18.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:18 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:51:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:19 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:51:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:19 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:51:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:19 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:51:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:19 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:51:19.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:19 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:51:19.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:19 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:51:19.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:19 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:51:19.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:19 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:51:19.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:19 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:51:19.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:19 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:51:19.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:19 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:51:19.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:19 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:51:19.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:19 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:51:19.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:19 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:51:19.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:19 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:51:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.155+0000 7f7b59f9f640 1 -- 172.21.15.82:0/1150916657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b54106f30 msgr2=0x7f7b54107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.155+0000 7f7b59f9f640 1 --2- 172.21.15.82:0/1150916657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b54106f30 0x7f7b54107310 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009930 tx=0x7f7b4402bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.155+0000 7f7b59f9f640 1 -- 172.21.15.82:0/1150916657 shutdown_connections 2024-04-21T08:51:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.155+0000 7f7b59f9f640 1 --2- 172.21.15.82:0/1150916657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b54100e10 0x7f7b5410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.155+0000 7f7b59f9f640 1 --2- 172.21.15.82:0/1150916657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b54100450 0x7f7b541008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.155+0000 7f7b59f9f640 1 --2- 172.21.15.82:0/1150916657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b54106f30 0x7f7b54107310 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.155+0000 7f7b59f9f640 1 -- 172.21.15.82:0/1150916657 >> 172.21.15.82:0/1150916657 conn(0x7f7b540773e0 msgr2=0x7f7b540fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.155+0000 7f7b59f9f640 1 -- 172.21.15.82:0/1150916657 shutdown_connections 2024-04-21T08:51:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.155+0000 7f7b59f9f640 1 -- 172.21.15.82:0/1150916657 wait complete. 2024-04-21T08:51:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.155+0000 7f7b59f9f640 1 Processor -- start 2024-04-21T08:51:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.156+0000 7f7b59f9f640 1 -- start start 2024-04-21T08:51:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.156+0000 7f7b59f9f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b54100450 0x7f7b54103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.156+0000 7f7b59f9f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b54100e10 0x7f7b54101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.156+0000 7f7b59f9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b54106f30 0x7f7b54101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.156+0000 7f7b59f9f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54112290 con 0x7f7b54100e10 2024-04-21T08:51:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.156+0000 7f7b52ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b54100e10 0x7f7b54101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.156+0000 7f7b59f9f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54112110 con 0x7f7b54100450 2024-04-21T08:51:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.156+0000 7f7b59f9f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54112410 con 0x7f7b54106f30 2024-04-21T08:51:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.156+0000 7f7b537fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b54100450 0x7f7b54103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.156+0000 7f7b52ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b54100e10 0x7f7b54101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34598/0 (socket says 172.21.15.82:34598) 2024-04-21T08:51:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.156+0000 7f7b537fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b54100450 0x7f7b54103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34338/0 (socket says 172.21.15.82:34338) 2024-04-21T08:51:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.157+0000 7f7b52ffd640 1 -- 172.21.15.82:0/210530100 learned_addr learned my addr 172.21.15.82:0/210530100 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.157+0000 7f7b53fff640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b54106f30 0x7f7b54101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.157+0000 7f7b537fe640 1 -- 172.21.15.82:0/210530100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b54106f30 msgr2=0x7f7b54101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.157+0000 7f7b537fe640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b54106f30 0x7f7b54101c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.157+0000 7f7b537fe640 1 -- 172.21.15.82:0/210530100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b54100e10 msgr2=0x7f7b54101630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.157+0000 7f7b537fe640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b54100e10 0x7f7b54101630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.157+0000 7f7b537fe640 1 -- 172.21.15.82:0/210530100 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b541ab130 con 0x7f7b54100450 2024-04-21T08:51:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.157+0000 7f7b537fe640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b54100450 0x7f7b54103230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009230 tx=0x7f7b440045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.158+0000 7f7b50ff9640 1 -- 172.21.15.82:0/210530100 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44043020 con 0x7f7b54100450 2024-04-21T08:51:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.158+0000 7f7b50ff9640 1 -- 172.21.15.82:0/210530100 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b4402ca70 con 0x7f7b54100450 2024-04-21T08:51:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.158+0000 7f7b50ff9640 1 -- 172.21.15.82:0/210530100 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44003a80 con 0x7f7b54100450 2024-04-21T08:51:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.158+0000 7f7b59f9f640 1 -- 172.21.15.82:0/210530100 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b541ab300 con 0x7f7b54100450 2024-04-21T08:51:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.159+0000 7f7b59f9f640 1 -- 172.21.15.82:0/210530100 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b541ab710 con 0x7f7b54100450 2024-04-21T08:51:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.162+0000 7f7b50ff9640 1 -- 172.21.15.82:0/210530100 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7b44046020 con 0x7f7b54100450 2024-04-21T08:51:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.162+0000 7f7b5cb8a740 1 Processor -- start 2024-04-21T08:51:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.163+0000 7f7b5cb8a740 1 -- start start 2024-04-21T08:51:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.163+0000 7f7b5cb8a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219de8d0 0x5574219dd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.163+0000 7f7b5cb8a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219ddb60 0x5574219ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.163+0000 7f7b5cb8a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.163+0000 7f7b5cb8a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5574219428d0 con 0x5574219ddb60 2024-04-21T08:51:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.163+0000 7f7b5cb8a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5574218dfe80 con 0x5574219e70e0 2024-04-21T08:51:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.163+0000 7f7b5cb8a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55742193e890 con 0x5574219de8d0 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.163+0000 7f7b52ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.163+0000 7f7b52ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219e94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34350/0 (socket says 172.21.15.82:34350) 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.163+0000 7f7b52ffd640 1 -- 172.21.15.82:0/1792885094 learned_addr learned my addr 172.21.15.82:0/1792885094 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.163+0000 7f7b53fff640 1 --2- 172.21.15.82:0/1792885094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219de8d0 0x5574219dd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.163+0000 7f7b537fe640 1 --2- 172.21.15.82:0/1792885094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219ddb60 0x5574219ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.164+0000 7f7b537fe640 1 -- 172.21.15.82:0/1792885094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219de8d0 msgr2=0x5574219dd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.164+0000 7f7b537fe640 1 --2- 172.21.15.82:0/1792885094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219de8d0 0x5574219dd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.164+0000 7f7b537fe640 1 -- 172.21.15.82:0/1792885094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 msgr2=0x5574219e94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.164+0000 7f7b537fe640 1 --2- 172.21.15.82:0/1792885094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219e94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.164+0000 7f7b537fe640 1 -- 172.21.15.82:0/1792885094 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557421901320 con 0x5574219ddb60 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.164+0000 7f7b52ffd640 1 --2- 172.21.15.82:0/1792885094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.164+0000 7f7b537fe640 1 --2- 172.21.15.82:0/1792885094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219ddb60 0x5574219ddfe0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7b440eff40 tx=0x7f7b440f6a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b167486ba1ecb4bc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.165+0000 7f7b50ff9640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7b3c0a3e40 0x7f7b3c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.165+0000 7f7b39ffb640 1 -- 172.21.15.82:0/1792885094 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44105070 con 0x5574219ddb60 2024-04-21T08:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.165+0000 7f7b39ffb640 1 -- 172.21.15.82:0/1792885094 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b440fbac0 con 0x5574219ddb60 2024-04-21T08:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.165+0000 7f7b39ffb640 1 -- 172.21.15.82:0/1792885094 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b440fc040 con 0x5574219ddb60 2024-04-21T08:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.165+0000 7f7b53fff640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7b3c0a3e40 0x7f7b3c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.165+0000 7f7b50ff9640 1 -- 172.21.15.82:0/210530100 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f7b440f2b00 con 0x7f7b54100450 2024-04-21T08:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.166+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1792885094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219ddb60 msgr2=0x5574219ddfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.166+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/1792885094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219ddb60 0x5574219ddfe0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7b440eff40 tx=0x7f7b440f6a90 comp rx=0 tx=0).stop 2024-04-21T08:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.166+0000 7f7b53fff640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7b3c0a3e40 0x7f7b3c0a6300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c005fd0 tx=0x7f7b4c009500 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.166+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1792885094 shutdown_connections 2024-04-21T08:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.166+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/1792885094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.166+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/1792885094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219ddb60 0x5574219ddfe0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.166+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/1792885094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219de8d0 0x5574219dd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.166+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1792885094 >> 172.21.15.82:0/1792885094 conn(0x5574219a4c90 msgr2=0x5574219d8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.166+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1792885094 shutdown_connections 2024-04-21T08:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.166+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1792885094 wait complete. 2024-04-21T08:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.167+0000 7f7b5cb8a740 1 -- start start 2024-04-21T08:51:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.167+0000 7f7b5cb8a740 1 Processor -- start 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.167+0000 7f7b5cb8a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219ddb60 0x5574219e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.167+0000 7f7b5cb8a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219de8d0 0x5574219e3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.167+0000 7f7b5cb8a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219e4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.167+0000 7f7b5cb8a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5574219fabe0 con 0x5574219de8d0 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.167+0000 7f7b5cb8a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5574218caeb0 con 0x5574219e70e0 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.167+0000 7f7b5cb8a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5574219fad60 con 0x5574219ddb60 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.167+0000 7f7b5cb8a740 10 client.0 _ll_register_callbacks cb 0x5574218f7b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.168+0000 7f7b53fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.168+0000 7f7b53fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219e4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34366/0 (socket says 172.21.15.82:34366) 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.168+0000 7f7b537fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219ddb60 0x5574219e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.168+0000 7f7b53fff640 1 -- 172.21.15.82:0/429850971 learned_addr learned my addr 172.21.15.82:0/429850971 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.168+0000 7f7b53fff640 1 -- 172.21.15.82:0/429850971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219ddb60 msgr2=0x5574219e6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.168+0000 7f7b53fff640 1 --2- 172.21.15.82:0/429850971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219ddb60 0x5574219e6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.168+0000 7f7b53fff640 1 -- 172.21.15.82:0/429850971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219de8d0 msgr2=0x5574219e3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.168+0000 7f7b53fff640 1 --2- 172.21.15.82:0/429850971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219de8d0 0x5574219e3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.168+0000 7f7b53fff640 1 -- 172.21.15.82:0/429850971 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557421655210 con 0x5574219e70e0 2024-04-21T08:51:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.169+0000 7f7b53fff640 1 --2- 172.21.15.82:0/429850971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219e4370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0049d0 tx=0x7f7b4c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5059de3736b1b9ea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.169+0000 7f7b32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:51:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.169+0000 7f7b32ffd640 1 -- 172.21.15.82:0/429850971 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c02c060 con 0x5574219e70e0 2024-04-21T08:51:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.169+0000 7f7b32ffd640 1 -- 172.21.15.82:0/429850971 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f7b4c017030 con 0x5574219e70e0 2024-04-21T08:51:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.169+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/429850971 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b440fbac0 con 0x5574219e70e0 2024-04-21T08:51:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.170+0000 7f7b32ffd640 1 -- 172.21.15.82:0/429850971 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7b4c033050 con 0x5574219e70e0 2024-04-21T08:51:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.170+0000 7f7b5cb8a740 10 client.26213 fetch_fsmap learned FSMap version 34 2024-04-21T08:51:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.170+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/429850971 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557421721620 con 0x5574219e70e0 2024-04-21T08:51:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.171+0000 7f7b32ffd640 1 -- 172.21.15.82:0/429850971 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b440fbac0 con 0x5574219e70e0 2024-04-21T08:51:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.171+0000 7f7b32ffd640 1 -- 172.21.15.82:0/429850971 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f7b4c027a30 con 0x5574219e70e0 2024-04-21T08:51:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.172+0000 7f7b5cb8a740 10 client.26213 fetch_fsmap finished waiting for FSMap version 34 2024-04-21T08:51:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.172+0000 7f7b5cb8a740 10 client.26213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.172+0000 7f7b5cb8a740 20 client.26213 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.172+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/429850971 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557421a07f00 0x557421a0a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.172+0000 7f7b5cb8a740 4 client.26213 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.172+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/429850971 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557421956880 con 0x557421a07f00 2024-04-21T08:51:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.173+0000 7f7b537fe640 1 --2- 172.21.15.82:0/429850971 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557421a07f00 0x557421a0a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.173+0000 7f7b32ffd640 10 client.26213 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.173+0000 7f7b537fe640 1 --2- 172.21.15.82:0/429850971 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557421a07f00 0x557421a0a370 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d4a204325f7ce6 server_cookie=c77ca9c474ebfd6a in_seq=0 out_seq=0 2024-04-21T08:51:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.174+0000 7f7b32ffd640 1 -- 172.21.15.82:0/429850971 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557421956880 con 0x557421a07f00 2024-04-21T08:51:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.174+0000 7f7b32ffd640 10 client.26213 handle_command_reply: tid=0 2024-04-21T08:51:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.175+0000 7f7b5cb8a740 1 client.26213 shutdown 2024-04-21T08:51:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.175+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/429850971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 msgr2=0x5574219e4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.175+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/429850971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219e4370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0049d0 tx=0x7f7b4c004ab0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.175+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/429850971 shutdown_connections 2024-04-21T08:51:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.175+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/429850971 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557421a07f00 0x557421a0a370 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.175+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/429850971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219e4370 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.175+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/429850971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219de8d0 0x5574219e3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.175+0000 7f7b32ffd640 0 client.26213 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.175+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/429850971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219ddb60 0x5574219e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.175+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/429850971 >> 172.21.15.82:0/429850971 conn(0x5574219a4c90 msgr2=0x5574219da5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.175+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/429850971 shutdown_connections 2024-04-21T08:51:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.175+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/429850971 wait complete. 2024-04-21T08:51:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.176+0000 7f7b5cb8a740 20 client.26213 trim_cache size 0 max 16384 2024-04-21T08:51:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.204+0000 7f7b5cb8a740 1 Processor -- start 2024-04-21T08:51:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.204+0000 7f7b5cb8a740 1 -- start start 2024-04-21T08:51:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.204+0000 7f7b5cb8a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219ddb60 0x557421a03ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.204+0000 7f7b5cb8a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219de8d0 0x5574219e6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.204+0000 7f7b5cb8a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219e70e0 0x5574219e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.204+0000 7f7b537fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219ddb60 0x557421a03ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.204+0000 7f7b537fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219ddb60 0x557421a03ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34380/0 (socket says 172.21.15.82:34380) 2024-04-21T08:51:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.204+0000 7f7b537fe640 1 -- 172.21.15.82:0/1990915435 learned_addr learned my addr 172.21.15.82:0/1990915435 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.204+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1990915435 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541ad140 con 0x5574219de8d0 2024-04-21T08:51:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.204+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1990915435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541aed10 con 0x5574219ddb60 2024-04-21T08:51:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.204+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1990915435 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541ad810 con 0x5574219e70e0 2024-04-21T08:51:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.205+0000 7f7b52ffd640 1 --2- 172.21.15.82:0/1990915435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219de8d0 0x5574219e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.205+0000 7f7b537fe640 1 -- 172.21.15.82:0/1990915435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219e70e0 msgr2=0x5574219e6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.205+0000 7f7b537fe640 1 --2- 172.21.15.82:0/1990915435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219e70e0 0x5574219e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.205+0000 7f7b537fe640 1 -- 172.21.15.82:0/1990915435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219de8d0 msgr2=0x5574219e6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.205+0000 7f7b537fe640 1 --2- 172.21.15.82:0/1990915435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219de8d0 0x5574219e6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.205+0000 7f7b537fe640 1 -- 172.21.15.82:0/1990915435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574219e9840 con 0x5574219ddb60 2024-04-21T08:51:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.205+0000 7f7b537fe640 1 --2- 172.21.15.82:0/1990915435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219ddb60 0x557421a03ae0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7b440f6920 tx=0x7f7b4402c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b516f14f1632eed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.206+0000 7f7b397fa640 1 -- 172.21.15.82:0/1990915435 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b440baa30 con 0x5574219ddb60 2024-04-21T08:51:20.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.206+0000 7f7b397fa640 1 -- 172.21.15.82:0/1990915435 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b440bee90 con 0x5574219ddb60 2024-04-21T08:51:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.207+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1990915435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219ddb60 msgr2=0x557421a03ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.207+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/1990915435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219ddb60 0x557421a03ae0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7b440f6920 tx=0x7f7b4402c740 comp rx=0 tx=0).stop 2024-04-21T08:51:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.207+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1990915435 shutdown_connections 2024-04-21T08:51:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.207+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/1990915435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219e70e0 0x5574219e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.207+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/1990915435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219de8d0 0x5574219e6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.207+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/1990915435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219ddb60 0x557421a03ae0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.207+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1990915435 >> 172.21.15.82:0/1990915435 conn(0x5574219a4c90 msgr2=0x5574219df060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.207+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1990915435 shutdown_connections 2024-04-21T08:51:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.207+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/1990915435 wait complete. 2024-04-21T08:51:20.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.208+0000 7f7b5cb8a740 1 -- start start 2024-04-21T08:51:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.208+0000 7f7b5cb8a740 1 Processor -- start 2024-04-21T08:51:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.208+0000 7f7b5cb8a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219ddb60 0x5574219ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.208+0000 7f7b5cb8a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219de8d0 0x557421a00530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.208+0000 7f7b5cb8a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219fcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.208+0000 7f7b5cb8a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541ae640 con 0x5574219ddb60 2024-04-21T08:51:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.208+0000 7f7b5cb8a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7b541adf70 con 0x5574219e70e0 2024-04-21T08:51:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.208+0000 7f7b5cb8a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54073960 con 0x5574219de8d0 2024-04-21T08:51:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.208+0000 7f7b5cb8a740 10 client.0 _ll_register_callbacks cb 0x7f7b4c009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.209+0000 7f7b52ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219de8d0 0x557421a00530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.209+0000 7f7b52ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219de8d0 0x557421a00530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:33714/0 (socket says 172.21.15.82:33714) 2024-04-21T08:51:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.209+0000 7f7b52ffd640 1 -- 172.21.15.82:0/890306336 learned_addr learned my addr 172.21.15.82:0/890306336 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.209+0000 7f7b53fff640 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219fcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.209+0000 7f7b537fe640 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219ddb60 0x5574219ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.209+0000 7f7b52ffd640 1 -- 172.21.15.82:0/890306336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 msgr2=0x5574219fcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.209+0000 7f7b52ffd640 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219fcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.209+0000 7f7b52ffd640 1 -- 172.21.15.82:0/890306336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219ddb60 msgr2=0x5574219ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.209+0000 7f7b52ffd640 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219ddb60 0x5574219ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.209+0000 7f7b52ffd640 1 -- 172.21.15.82:0/890306336 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574219fd500 con 0x5574219de8d0 2024-04-21T08:51:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.209+0000 7f7b53fff640 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219fcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.209+0000 7f7b537fe640 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219ddb60 0x5574219ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.210+0000 7f7b52ffd640 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219de8d0 0x557421a00530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5574219de600 tx=0x7f7b48010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=390fda859b6838eb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.210+0000 7f7b32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:51:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.211+0000 7f7b32ffd640 1 -- 172.21.15.82:0/890306336 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48021060 con 0x5574219de8d0 2024-04-21T08:51:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.211+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/890306336 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b440bee90 con 0x5574219de8d0 2024-04-21T08:51:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.211+0000 7f7b32ffd640 1 -- 172.21.15.82:0/890306336 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f7b4800fc80 con 0x5574219de8d0 2024-04-21T08:51:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.211+0000 7f7b32ffd640 1 -- 172.21.15.82:0/890306336 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7b48028050 con 0x5574219de8d0 2024-04-21T08:51:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.212+0000 7f7b5cb8a740 10 client.25969 fetch_fsmap learned FSMap version 34 2024-04-21T08:51:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.212+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/890306336 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557421a6ed70 con 0x5574219de8d0 2024-04-21T08:51:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.212+0000 7f7b32ffd640 1 -- 172.21.15.82:0/890306336 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b440bee90 con 0x5574219de8d0 2024-04-21T08:51:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.212+0000 7f7b32ffd640 1 -- 172.21.15.82:0/890306336 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f7b4801b070 con 0x5574219de8d0 2024-04-21T08:51:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.212+0000 7f7b5cb8a740 10 client.25969 fetch_fsmap finished waiting for FSMap version 34 2024-04-21T08:51:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.212+0000 7f7b5cb8a740 10 client.25969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.212+0000 7f7b5cb8a740 20 client.25969 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.212+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557421a6f490 0x557421a718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.212+0000 7f7b5cb8a740 4 client.25969 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:51:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.212+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/890306336 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574218e23d0 con 0x557421a6f490 2024-04-21T08:51:20.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.213+0000 7f7b537fe640 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557421a6f490 0x557421a718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.214+0000 7f7b537fe640 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557421a6f490 0x557421a718d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d104c5ebdcac611c server_cookie=e0858ebb3ea8ab7e in_seq=0 out_seq=0 2024-04-21T08:51:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.215+0000 7f7b32ffd640 10 client.25969 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:20.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:20 smithi134 ceph-mon[31772]: pgmap v132: 129 pgs: 129 active+clean; 434 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-04-21T08:51:20.759 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:20 smithi082 ceph-mon[27967]: pgmap v132: 129 pgs: 129 active+clean; 434 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-04-21T08:51:20.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:20 smithi106 ceph-mon[31072]: pgmap v132: 129 pgs: 129 active+clean; 434 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 264 op/s 2024-04-21T08:51:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b32ffd640 1 -- 172.21.15.82:0/890306336 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574218e23d0 con 0x557421a6f490 2024-04-21T08:51:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b32ffd640 10 client.25969 handle_command_reply: tid=0 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b5cb8a740 1 client.25969 shutdown 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/890306336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219de8d0 msgr2=0x557421a00530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219de8d0 0x557421a00530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5574219de600 tx=0x7f7b48010c80 comp rx=0 tx=0).stop 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/890306336 shutdown_connections 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557421a6f490 0x557421a718d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5574219e70e0 0x5574219fcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b32ffd640 0 client.25969 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574219de8d0 0x557421a00530 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b5cb8a740 1 --2- 172.21.15.82:0/890306336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574219ddb60 0x5574219ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/890306336 >> 172.21.15.82:0/890306336 conn(0x5574219a4c90 msgr2=0x5574219d7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/890306336 shutdown_connections 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b5cb8a740 1 -- 172.21.15.82:0/890306336 wait complete. 2024-04-21T08:51:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.981+0000 7f7b5cb8a740 20 client.25969 trim_cache size 0 max 16384 2024-04-21T08:51:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:51:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:51:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:51:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:51:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.982+0000 7f7b59f9f640 1 -- 172.21.15.82:0/210530100 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7b3c0a3e40 msgr2=0x7f7b3c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.982+0000 7f7b59f9f640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7b3c0a3e40 0x7f7b3c0a6300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c005fd0 tx=0x7f7b4c009500 comp rx=0 tx=0).stop 2024-04-21T08:51:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.982+0000 7f7b59f9f640 1 -- 172.21.15.82:0/210530100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b54100450 msgr2=0x7f7b54103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.982+0000 7f7b59f9f640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b54100450 0x7f7b54103230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009230 tx=0x7f7b440045e0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.982+0000 7f7b59f9f640 1 -- 172.21.15.82:0/210530100 shutdown_connections 2024-04-21T08:51:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.982+0000 7f7b59f9f640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b54106f30 0x7f7b54101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.982+0000 7f7b59f9f640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7b3c0a3e40 0x7f7b3c0a6300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.982+0000 7f7b59f9f640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b54100e10 0x7f7b54101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.982+0000 7f7b59f9f640 1 --2- 172.21.15.82:0/210530100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b54100450 0x7f7b54103230 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.982+0000 7f7b59f9f640 1 -- 172.21.15.82:0/210530100 >> 172.21.15.82:0/210530100 conn(0x7f7b540773e0 msgr2=0x7f7b540fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.982+0000 7f7b59f9f640 1 -- 172.21.15.82:0/210530100 shutdown_connections 2024-04-21T08:51:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:20.982+0000 7f7b59f9f640 1 -- 172.21.15.82:0/210530100 wait complete. 2024-04-21T08:51:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:22 smithi134 ceph-mon[31772]: pgmap v133: 129 pgs: 129 active+clean; 434 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 261 op/s 2024-04-21T08:51:22.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:22 smithi082 ceph-mon[27967]: pgmap v133: 129 pgs: 129 active+clean; 434 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 261 op/s 2024-04-21T08:51:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:22 smithi106 ceph-mon[31072]: pgmap v133: 129 pgs: 129 active+clean; 434 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 261 op/s 2024-04-21T08:51:23.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:23 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:51:23.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:23 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:51:23.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:23 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:51:24.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:24 smithi134 ceph-mon[31772]: pgmap v134: 129 pgs: 129 active+clean; 473 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.7 MiB/s wr, 520 op/s 2024-04-21T08:51:24.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:24 smithi082 ceph-mon[27967]: pgmap v134: 129 pgs: 129 active+clean; 473 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.7 MiB/s wr, 520 op/s 2024-04-21T08:51:24.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:24 smithi106 ceph-mon[31072]: pgmap v134: 129 pgs: 129 active+clean; 473 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.7 MiB/s wr, 520 op/s 2024-04-21T08:51:26.231 INFO:tasks.workunit.client.0.smithi082.stdout: 85 1512510 518 1042554 326 746489 1117 2024-04-21T08:51:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:26 smithi134 ceph-mon[31772]: pgmap v135: 129 pgs: 129 active+clean; 473 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 353 op/s 2024-04-21T08:51:26.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:26 smithi082 ceph-mon[27967]: pgmap v135: 129 pgs: 129 active+clean; 473 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 353 op/s 2024-04-21T08:51:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:26 smithi106 ceph-mon[31072]: pgmap v135: 129 pgs: 129 active+clean; 473 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 353 op/s 2024-04-21T08:51:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:28 smithi134 ceph-mon[31772]: pgmap v136: 129 pgs: 129 active+clean; 475 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 387 op/s 2024-04-21T08:51:28.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:28 smithi082 ceph-mon[27967]: pgmap v136: 129 pgs: 129 active+clean; 475 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 387 op/s 2024-04-21T08:51:28.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:28 smithi106 ceph-mon[31072]: pgmap v136: 129 pgs: 129 active+clean; 475 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.9 MiB/s wr, 387 op/s 2024-04-21T08:51:30.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:30 smithi134 ceph-mon[31772]: pgmap v137: 129 pgs: 129 active+clean; 482 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 350 op/s 2024-04-21T08:51:30.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:30 smithi082 ceph-mon[27967]: pgmap v137: 129 pgs: 129 active+clean; 482 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 350 op/s 2024-04-21T08:51:30.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:30 smithi106 ceph-mon[31072]: pgmap v137: 129 pgs: 129 active+clean; 482 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 350 op/s 2024-04-21T08:51:32.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:32 smithi134 ceph-mon[31772]: pgmap v138: 129 pgs: 129 active+clean; 482 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 314 op/s 2024-04-21T08:51:32.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:32 smithi082 ceph-mon[27967]: pgmap v138: 129 pgs: 129 active+clean; 482 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 314 op/s 2024-04-21T08:51:32.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:32 smithi106 ceph-mon[31072]: pgmap v138: 129 pgs: 129 active+clean; 482 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 314 op/s 2024-04-21T08:51:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.078+0000 7f7e61115640 1 -- 172.21.15.82:0/4168239032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e5c101aa0 msgr2=0x7f7e5c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.078+0000 7f7e61115640 1 --2- 172.21.15.82:0/4168239032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e5c101aa0 0x7f7e5c105a90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009900 tx=0x7f7e5002bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.078+0000 7f7e61115640 1 -- 172.21.15.82:0/4168239032 shutdown_connections 2024-04-21T08:51:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.078+0000 7f7e61115640 1 --2- 172.21.15.82:0/4168239032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c106170 0x7f7e5c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.078+0000 7f7e61115640 1 --2- 172.21.15.82:0/4168239032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e5c101aa0 0x7f7e5c105a90 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.078+0000 7f7e61115640 1 --2- 172.21.15.82:0/4168239032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5c1010f0 0x7f7e5c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.078+0000 7f7e61115640 1 -- 172.21.15.82:0/4168239032 >> 172.21.15.82:0/4168239032 conn(0x7f7e5c077fd0 msgr2=0x7f7e5c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.078+0000 7f7e61115640 1 -- 172.21.15.82:0/4168239032 shutdown_connections 2024-04-21T08:51:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.078+0000 7f7e61115640 1 -- 172.21.15.82:0/4168239032 wait complete. 2024-04-21T08:51:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e61115640 1 Processor -- start 2024-04-21T08:51:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e61115640 1 -- start start 2024-04-21T08:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e61115640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e5c1010f0 0x7f7e5c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e61115640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c106170 0x7f7e5c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e61115640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5c1985a0 0x7f7e5c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e61115640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c111110 con 0x7f7e5c1010f0 2024-04-21T08:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e61115640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c110f90 con 0x7f7e5c106170 2024-04-21T08:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e61115640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c111290 con 0x7f7e5c1985a0 2024-04-21T08:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e5ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e5c1010f0 0x7f7e5c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e5ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e5c1010f0 0x7f7e5c19e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50048/0 (socket says 172.21.15.82:50048) 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e5ad9d640 1 -- 172.21.15.82:0/3683859685 learned_addr learned my addr 172.21.15.82:0/3683859685 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e5a59c640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c106170 0x7f7e5c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.079+0000 7f7e5b59e640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5c1985a0 0x7f7e5c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.080+0000 7f7e5ad9d640 1 -- 172.21.15.82:0/3683859685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5c1985a0 msgr2=0x7f7e5c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.080+0000 7f7e5ad9d640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5c1985a0 0x7f7e5c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.080+0000 7f7e5ad9d640 1 -- 172.21.15.82:0/3683859685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c106170 msgr2=0x7f7e5c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.080+0000 7f7e5ad9d640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c106170 0x7f7e5c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.080+0000 7f7e5ad9d640 1 -- 172.21.15.82:0/3683859685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c1990c0 con 0x7f7e5c1010f0 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.080+0000 7f7e5b59e640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5c1985a0 0x7f7e5c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.080+0000 7f7e5ad9d640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e5c1010f0 0x7f7e5c19e3f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00b4a0 tx=0x7f7e4c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.080+0000 7f7e43fff640 1 -- 172.21.15.82:0/3683859685 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c014070 con 0x7f7e5c1010f0 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.080+0000 7f7e61115640 1 -- 172.21.15.82:0/3683859685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5c1993b0 con 0x7f7e5c1010f0 2024-04-21T08:51:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.080+0000 7f7e43fff640 1 -- 172.21.15.82:0/3683859685 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e4c010ac0 con 0x7f7e5c1010f0 2024-04-21T08:51:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.080+0000 7f7e43fff640 1 -- 172.21.15.82:0/3683859685 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c017c30 con 0x7f7e5c1010f0 2024-04-21T08:51:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.080+0000 7f7e61115640 1 -- 172.21.15.82:0/3683859685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5c1a55b0 con 0x7f7e5c1010f0 2024-04-21T08:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.082+0000 7f7e43fff640 1 -- 172.21.15.82:0/3683859685 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7e4c017070 con 0x7f7e5c1010f0 2024-04-21T08:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.082+0000 7f7e63de3740 1 Processor -- start 2024-04-21T08:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.082+0000 7f7e63de3740 1 -- start start 2024-04-21T08:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e63de3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388578d0 0x55ce38856620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e63de3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 0x55ce38856fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e63de3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce388600e0 0x55ce388624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e63de3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ce387bb8d0 con 0x55ce38856b60 2024-04-21T08:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e63de3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ce38758e80 con 0x55ce388600e0 2024-04-21T08:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e63de3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ce387b7890 con 0x55ce388578d0 2024-04-21T08:51:34.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 0x55ce38856fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 0x55ce38856fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50060/0 (socket says 172.21.15.82:50060) 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5ad9d640 1 -- 172.21.15.82:0/3136545698 learned_addr learned my addr 172.21.15.82:0/3136545698 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5a59c640 1 --2- 172.21.15.82:0/3136545698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce388600e0 0x55ce388624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e43fff640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7e440a3dc0 0x7f7e440a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5b59e640 1 --2- 172.21.15.82:0/3136545698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388578d0 0x55ce38856620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5a59c640 1 -- 172.21.15.82:0/3136545698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388578d0 msgr2=0x55ce38856620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5a59c640 1 --2- 172.21.15.82:0/3136545698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388578d0 0x55ce38856620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5a59c640 1 -- 172.21.15.82:0/3136545698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 msgr2=0x55ce38856fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5a59c640 1 --2- 172.21.15.82:0/3136545698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 0x55ce38856fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5a59c640 1 -- 172.21.15.82:0/3136545698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce3877a320 con 0x55ce388600e0 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5b59e640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7e440a3dc0 0x7f7e440a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e43fff640 1 -- 172.21.15.82:0/3683859685 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f7e4c0cfe60 con 0x7f7e5c1010f0 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5a59c640 1 --2- 172.21.15.82:0/3136545698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce388600e0 0x55ce388624d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e50036040 tx=0x7f7e5002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=811c341cc0d3f5b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e417fa640 1 -- 172.21.15.82:0/3136545698 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50045020 con 0x55ce388600e0 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e5b59e640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7e440a3dc0 0x7f7e440a6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7e54005fd0 tx=0x7f7e54004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e417fa640 1 -- 172.21.15.82:0/3136545698 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e5002c7e0 con 0x55ce388600e0 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.083+0000 7f7e417fa640 1 -- 172.21.15.82:0/3136545698 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e500046c0 con 0x55ce388600e0 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.084+0000 7f7e63de3740 1 -- 172.21.15.82:0/3136545698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce388600e0 msgr2=0x55ce388624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.084+0000 7f7e63de3740 1 --2- 172.21.15.82:0/3136545698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce388600e0 0x55ce388624d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e50036040 tx=0x7f7e5002c6c0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.084+0000 7f7e63de3740 1 -- 172.21.15.82:0/3136545698 shutdown_connections 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.084+0000 7f7e63de3740 1 --2- 172.21.15.82:0/3136545698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce388600e0 0x55ce388624d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.084+0000 7f7e63de3740 1 --2- 172.21.15.82:0/3136545698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 0x55ce38856fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.084+0000 7f7e63de3740 1 --2- 172.21.15.82:0/3136545698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388578d0 0x55ce38856620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.084+0000 7f7e63de3740 1 -- 172.21.15.82:0/3136545698 >> 172.21.15.82:0/3136545698 conn(0x55ce3881dc90 msgr2=0x55ce38851850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.084+0000 7f7e63de3740 1 -- 172.21.15.82:0/3136545698 shutdown_connections 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.084+0000 7f7e63de3740 1 -- 172.21.15.82:0/3136545698 wait complete. 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.084+0000 7f7e63de3740 1 -- start start 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e63de3740 1 Processor -- start 2024-04-21T08:51:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e63de3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 0x55ce3885fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e63de3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce388578d0 0x55ce3885cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e63de3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388600e0 0x55ce3885d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e63de3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ce38862db0 con 0x55ce38856b60 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e63de3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ce38743eb0 con 0x55ce388578d0 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e63de3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ce38862f30 con 0x55ce388600e0 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e63de3740 10 client.0 _ll_register_callbacks cb 0x55ce38770b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e5ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 0x55ce3885fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e5a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce388578d0 0x55ce3885cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e5ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 0x55ce3885fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50074/0 (socket says 172.21.15.82:50074) 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e5ad9d640 1 -- 172.21.15.82:0/1019946399 learned_addr learned my addr 172.21.15.82:0/1019946399 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e5b59e640 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388600e0 0x55ce3885d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e5a59c640 1 -- 172.21.15.82:0/1019946399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388600e0 msgr2=0x55ce3885d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e5a59c640 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388600e0 0x55ce3885d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e5a59c640 1 -- 172.21.15.82:0/1019946399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 msgr2=0x55ce3885fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e5a59c640 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 0x55ce3885fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e5a59c640 1 -- 172.21.15.82:0/1019946399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce384ce210 con 0x55ce388578d0 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e5ad9d640 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 0x55ce3885fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e5a59c640 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce388578d0 0x55ce3885cda0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55ce388529e0 tx=0x7f7e5002c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9016d14ad3b44006 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.085+0000 7f7e3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e3eff5640 1 -- 172.21.15.82:0/1019946399 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e500035f0 con 0x55ce388578d0 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e63de3740 1 -- 172.21.15.82:0/1019946399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e5002c7e0 con 0x55ce388578d0 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e3eff5640 1 -- 172.21.15.82:0/1019946399 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f7e50003790 con 0x55ce388578d0 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e3eff5640 1 -- 172.21.15.82:0/1019946399 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7e5005a050 con 0x55ce388578d0 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e63de3740 10 client.26231 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e63de3740 1 -- 172.21.15.82:0/1019946399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ce3859a620 con 0x55ce388578d0 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e3eff5640 1 -- 172.21.15.82:0/1019946399 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e5002c7e0 con 0x55ce388578d0 2024-04-21T08:51:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e3eff5640 1 -- 172.21.15.82:0/1019946399 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f7e50041b80 con 0x55ce388578d0 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e63de3740 10 client.26231 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e63de3740 10 client.26231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e63de3740 20 client.26231 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e63de3740 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ce3886fec0 0x55ce388722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e63de3740 4 client.26231 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e63de3740 1 -- 172.21.15.82:0/1019946399 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce387cf880 con 0x55ce3886fec0 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.086+0000 7f7e5ad9d640 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ce3886fec0 0x55ce388722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.087+0000 7f7e5ad9d640 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ce3886fec0 0x55ce388722b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c132fd0af03cdb32 server_cookie=b261305acbd94f78 in_seq=0 out_seq=0 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.087+0000 7f7e3eff5640 10 client.26231 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.087+0000 7f7e3eff5640 1 -- 172.21.15.82:0/1019946399 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce387cf880 con 0x55ce3886fec0 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.087+0000 7f7e3eff5640 10 client.26231 handle_command_reply: tid=0 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.087+0000 7f7e63de3740 1 client.26231 shutdown 2024-04-21T08:51:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.088+0000 7f7e63de3740 1 -- 172.21.15.82:0/1019946399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce388578d0 msgr2=0x55ce3885cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.088+0000 7f7e63de3740 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce388578d0 0x55ce3885cda0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55ce388529e0 tx=0x7f7e5002c3e0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.088+0000 7f7e63de3740 1 -- 172.21.15.82:0/1019946399 shutdown_connections 2024-04-21T08:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.088+0000 7f7e63de3740 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ce3886fec0 0x55ce388722b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.088+0000 7f7e63de3740 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388600e0 0x55ce3885d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.088+0000 7f7e3eff5640 0 client.26231 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.088+0000 7f7e63de3740 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce388578d0 0x55ce3885cda0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.088+0000 7f7e63de3740 1 --2- 172.21.15.82:0/1019946399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce38856b60 0x55ce3885fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.088+0000 7f7e63de3740 1 -- 172.21.15.82:0/1019946399 >> 172.21.15.82:0/1019946399 conn(0x55ce3881dc90 msgr2=0x55ce388535a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.088+0000 7f7e63de3740 1 -- 172.21.15.82:0/1019946399 shutdown_connections 2024-04-21T08:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.088+0000 7f7e63de3740 1 -- 172.21.15.82:0/1019946399 wait complete. 2024-04-21T08:51:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.088+0000 7f7e63de3740 20 client.26231 trim_cache size 0 max 16384 2024-04-21T08:51:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.104+0000 7f7e63de3740 1 Processor -- start 2024-04-21T08:51:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.104+0000 7f7e63de3740 1 -- start start 2024-04-21T08:51:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.104+0000 7f7e63de3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce3881dc90 0x55ce3886bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.104+0000 7f7e63de3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce38856b60 0x55ce3884c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.104+0000 7f7e63de3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce388578d0 0x55ce3885f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.104+0000 7f7e63de3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c1af330 con 0x55ce388578d0 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.104+0000 7f7e63de3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c1b0ed0 con 0x55ce3881dc90 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.104+0000 7f7e63de3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c1afa00 con 0x55ce38856b60 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.104+0000 7f7e5b59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce388578d0 0x55ce3885f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.104+0000 7f7e5ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce3881dc90 0x55ce3886bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.104+0000 7f7e5b59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce388578d0 0x55ce3885f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50082/0 (socket says 172.21.15.82:50082) 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.104+0000 7f7e5b59e640 1 -- 172.21.15.82:0/3993427108 learned_addr learned my addr 172.21.15.82:0/3993427108 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.105+0000 7f7e5a59c640 1 --2- 172.21.15.82:0/3993427108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce38856b60 0x55ce3884c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.105+0000 7f7e5b59e640 1 -- 172.21.15.82:0/3993427108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce38856b60 msgr2=0x55ce3884c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.105+0000 7f7e5b59e640 1 --2- 172.21.15.82:0/3993427108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce38856b60 0x55ce3884c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.105+0000 7f7e5b59e640 1 -- 172.21.15.82:0/3993427108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce3881dc90 msgr2=0x55ce3886bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.105+0000 7f7e5b59e640 1 --2- 172.21.15.82:0/3993427108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce3881dc90 0x55ce3886bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.105+0000 7f7e5b59e640 1 -- 172.21.15.82:0/3993427108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce3884cf30 con 0x55ce388578d0 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.105+0000 7f7e5ad9d640 1 --2- 172.21.15.82:0/3993427108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce3881dc90 0x55ce3886bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.105+0000 7f7e5b59e640 1 --2- 172.21.15.82:0/3993427108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce388578d0 0x55ce3885f4d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7e54004410 tx=0x7f7e54014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbd60ed7558e58b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.105+0000 7f7e40ff9640 1 -- 172.21.15.82:0/3993427108 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e54012df0 con 0x55ce388578d0 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.105+0000 7f7e40ff9640 1 -- 172.21.15.82:0/3993427108 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e54016070 con 0x55ce388578d0 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.105+0000 7f7e40ff9640 1 -- 172.21.15.82:0/3993427108 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5400f6d0 con 0x55ce388578d0 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 -- 172.21.15.82:0/3993427108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce388578d0 msgr2=0x55ce3885f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 --2- 172.21.15.82:0/3993427108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce388578d0 0x55ce3885f4d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7e54004410 tx=0x7f7e54014070 comp rx=0 tx=0).stop 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 -- 172.21.15.82:0/3993427108 shutdown_connections 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 --2- 172.21.15.82:0/3993427108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce388578d0 0x55ce3885f4d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 --2- 172.21.15.82:0/3993427108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce38856b60 0x55ce3884c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 --2- 172.21.15.82:0/3993427108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce3881dc90 0x55ce3886bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 -- 172.21.15.82:0/3993427108 >> 172.21.15.82:0/3993427108 conn(0x55ce3885d440 msgr2=0x55ce3885b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 -- 172.21.15.82:0/3993427108 shutdown_connections 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 -- 172.21.15.82:0/3993427108 wait complete. 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 -- start start 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 Processor -- start 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce3881dc90 0x55ce388683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce38856b60 0x55ce38859610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388578d0 0x55ce38859b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c1b0800 con 0x55ce3881dc90 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c1b0130 con 0x55ce38856b60 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c1ad310 con 0x55ce388578d0 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e63de3740 10 client.0 _ll_register_callbacks cb 0x7f7e4c097e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e5ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce3881dc90 0x55ce388683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e5ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce3881dc90 0x55ce388683b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50084/0 (socket says 172.21.15.82:50084) 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e5ad9d640 1 -- 172.21.15.82:0/1198032123 learned_addr learned my addr 172.21.15.82:0/1198032123 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.106+0000 7f7e5a59c640 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce38856b60 0x55ce38859610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e5b59e640 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388578d0 0x55ce38859b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e5ad9d640 1 -- 172.21.15.82:0/1198032123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388578d0 msgr2=0x55ce38859b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e5ad9d640 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388578d0 0x55ce38859b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e5ad9d640 1 -- 172.21.15.82:0/1198032123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce38856b60 msgr2=0x55ce38859610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e5ad9d640 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce38856b60 0x55ce38859610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e5ad9d640 1 -- 172.21.15.82:0/1198032123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce3885a0c0 con 0x55ce3881dc90 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e5ad9d640 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce3881dc90 0x55ce388683b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0cd3b0 tx=0x7f7e4c0d2370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac1e3b5739316a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e3eff5640 1 -- 172.21.15.82:0/1198032123 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c01e040 con 0x55ce3881dc90 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e3eff5640 1 -- 172.21.15.82:0/1198032123 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f7e4c0bc0b0 con 0x55ce3881dc90 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e63de3740 1 -- 172.21.15.82:0/1198032123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e54016070 con 0x55ce3881dc90 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e3eff5640 1 -- 172.21.15.82:0/1198032123 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7e4c09ff00 con 0x55ce3881dc90 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e63de3740 10 client.16899 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e63de3740 1 -- 172.21.15.82:0/1198032123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ce388da370 con 0x55ce3881dc90 2024-04-21T08:51:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e3eff5640 1 -- 172.21.15.82:0/1198032123 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e54016070 con 0x55ce3881dc90 2024-04-21T08:51:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e3eff5640 1 -- 172.21.15.82:0/1198032123 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f7e4c0cd470 con 0x55ce3881dc90 2024-04-21T08:51:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e63de3740 10 client.16899 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e63de3740 10 client.16899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e63de3740 20 client.16899 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e63de3740 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ce388600e0 0x55ce388dcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e63de3740 4 client.16899 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:51:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.107+0000 7f7e63de3740 1 -- 172.21.15.82:0/1198032123 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce3875b3d0 con 0x55ce388600e0 2024-04-21T08:51:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.108+0000 7f7e5a59c640 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ce388600e0 0x55ce388dcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.108+0000 7f7e5a59c640 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ce388600e0 0x55ce388dcad0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd7413706983a3aa server_cookie=fb93b09bcc14b549 in_seq=0 out_seq=0 2024-04-21T08:51:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.108+0000 7f7e3eff5640 10 client.16899 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:34.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:34 smithi134 ceph-mon[31772]: pgmap v139: 129 pgs: 129 active+clean; 494 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 401 op/s 2024-04-21T08:51:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.817+0000 7f7e3eff5640 1 -- 172.21.15.82:0/1198032123 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ce3875b3d0 con 0x55ce388600e0 2024-04-21T08:51:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.817+0000 7f7e3eff5640 10 client.16899 handle_command_reply: tid=0 2024-04-21T08:51:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.817+0000 7f7e63de3740 1 client.16899 shutdown 2024-04-21T08:51:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.820+0000 7f7e63de3740 1 -- 172.21.15.82:0/1198032123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce3881dc90 msgr2=0x55ce388683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.820+0000 7f7e63de3740 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce3881dc90 0x55ce388683b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0cd3b0 tx=0x7f7e4c0d2370 comp rx=0 tx=0).stop 2024-04-21T08:51:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.820+0000 7f7e63de3740 1 -- 172.21.15.82:0/1198032123 shutdown_connections 2024-04-21T08:51:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.820+0000 7f7e63de3740 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ce388600e0 0x55ce388dcad0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.820+0000 7f7e63de3740 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce388578d0 0x55ce38859b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.820+0000 7f7e63de3740 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ce38856b60 0x55ce38859610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.820+0000 7f7e63de3740 1 --2- 172.21.15.82:0/1198032123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ce3881dc90 0x55ce388683b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.820+0000 7f7e63de3740 1 -- 172.21.15.82:0/1198032123 >> 172.21.15.82:0/1198032123 conn(0x55ce3885d440 msgr2=0x55ce38852c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.820+0000 7f7e3eff5640 0 client.16899 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.821+0000 7f7e63de3740 1 -- 172.21.15.82:0/1198032123 shutdown_connections 2024-04-21T08:51:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.821+0000 7f7e63de3740 1 -- 172.21.15.82:0/1198032123 wait complete. 2024-04-21T08:51:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.821+0000 7f7e63de3740 20 client.16899 trim_cache size 0 max 16384 2024-04-21T08:51:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:51:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:51:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:51:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:51:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.837+0000 7f7e61115640 1 -- 172.21.15.82:0/3683859685 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7e440a3dc0 msgr2=0x7f7e440a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.837+0000 7f7e61115640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7e440a3dc0 0x7f7e440a6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7e54005fd0 tx=0x7f7e54004a40 comp rx=0 tx=0).stop 2024-04-21T08:51:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.837+0000 7f7e61115640 1 -- 172.21.15.82:0/3683859685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e5c1010f0 msgr2=0x7f7e5c19e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.837+0000 7f7e61115640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e5c1010f0 0x7f7e5c19e3f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00b4a0 tx=0x7f7e4c00b960 comp rx=0 tx=0).stop 2024-04-21T08:51:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.841+0000 7f7e61115640 1 -- 172.21.15.82:0/3683859685 shutdown_connections 2024-04-21T08:51:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.841+0000 7f7e61115640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e5c1985a0 0x7f7e5c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.841+0000 7f7e61115640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e5c106170 0x7f7e5c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.841+0000 7f7e61115640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7e440a3dc0 0x7f7e440a6280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.841+0000 7f7e61115640 1 --2- 172.21.15.82:0/3683859685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e5c1010f0 0x7f7e5c19e3f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.841+0000 7f7e61115640 1 -- 172.21.15.82:0/3683859685 >> 172.21.15.82:0/3683859685 conn(0x7f7e5c077fd0 msgr2=0x7f7e5c0699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.845+0000 7f7e61115640 1 -- 172.21.15.82:0/3683859685 shutdown_connections 2024-04-21T08:51:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:34.845+0000 7f7e61115640 1 -- 172.21.15.82:0/3683859685 wait complete. 2024-04-21T08:51:34.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:34 smithi082 ceph-mon[27967]: pgmap v139: 129 pgs: 129 active+clean; 494 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 401 op/s 2024-04-21T08:51:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:34 smithi106 ceph-mon[31072]: pgmap v139: 129 pgs: 129 active+clean; 494 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.6 MiB/s wr, 401 op/s 2024-04-21T08:51:36.231 INFO:tasks.workunit.client.0.smithi082.stdout: 93 1998155 512 1391694 504 1022692 1539 2024-04-21T08:51:36.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:36 smithi134 ceph-mon[31772]: pgmap v140: 129 pgs: 129 active+clean; 494 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-04-21T08:51:36.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:36 smithi082 ceph-mon[27967]: pgmap v140: 129 pgs: 129 active+clean; 494 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-04-21T08:51:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:36 smithi106 ceph-mon[31072]: pgmap v140: 129 pgs: 129 active+clean; 494 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-04-21T08:51:37.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:51:37.076 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:51:37.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.386+0000 7f6821a1e640 1 -- 172.21.15.82:0/3634892366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c1031c0 msgr2=0x7f681c1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:37.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.386+0000 7f6821a1e640 1 --2- 172.21.15.82:0/3634892366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c1031c0 0x7f681c1035a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f680c009900 tx=0x7f680c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:51:37.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.386+0000 7f6821a1e640 1 -- 172.21.15.82:0/3634892366 shutdown_connections 2024-04-21T08:51:37.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.386+0000 7f6821a1e640 1 --2- 172.21.15.82:0/3634892366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f681c10c500 0x7f681c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.386+0000 7f6821a1e640 1 --2- 172.21.15.82:0/3634892366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f681c103ae0 0x7f681c10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.386+0000 7f6821a1e640 1 --2- 172.21.15.82:0/3634892366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c1031c0 0x7f681c1035a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.386+0000 7f6821a1e640 1 -- 172.21.15.82:0/3634892366 >> 172.21.15.82:0/3634892366 conn(0x7f681c078000 msgr2=0x7f681c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:37.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.386+0000 7f6821a1e640 1 -- 172.21.15.82:0/3634892366 shutdown_connections 2024-04-21T08:51:37.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.386+0000 7f6821a1e640 1 -- 172.21.15.82:0/3634892366 wait complete. 2024-04-21T08:51:37.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.387+0000 7f6821a1e640 1 Processor -- start 2024-04-21T08:51:37.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.387+0000 7f6821a1e640 1 -- start start 2024-04-21T08:51:37.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.387+0000 7f6821a1e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c1031c0 0x7f681c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.387+0000 7f6821a1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f681c103ae0 0x7f681c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.387+0000 7f6821a1e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f681c10c500 0x7f681c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.387+0000 7f6821a1e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1121e0 con 0x7f681c1031c0 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.387+0000 7f6821a1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f681c112060 con 0x7f681c103ae0 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.387+0000 7f6821a1e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f681c112360 con 0x7f681c10c500 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f681a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f681c103ae0 0x7f681c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f681a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f681c103ae0 0x7f681c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34142/0 (socket says 172.21.15.82:34142) 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f681affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c1031c0 0x7f681c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f681a7fc640 1 -- 172.21.15.82:0/190869796 learned_addr learned my addr 172.21.15.82:0/190869796 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f681b7fe640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f681c10c500 0x7f681c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f681a7fc640 1 -- 172.21.15.82:0/190869796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f681c10c500 msgr2=0x7f681c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f681a7fc640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f681c10c500 0x7f681c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f681a7fc640 1 -- 172.21.15.82:0/190869796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c1031c0 msgr2=0x7f681c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f681a7fc640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c1031c0 0x7f681c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f681a7fc640 1 -- 172.21.15.82:0/190869796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681c19d730 con 0x7f681c103ae0 2024-04-21T08:51:37.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f681b7fe640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f681c10c500 0x7f681c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:37.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f681a7fc640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f681c103ae0 0x7f681c198c80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f681000e9f0 tx=0x7f681000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:37.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f6820a1c640 1 -- 172.21.15.82:0/190869796 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681000cd80 con 0x7f681c103ae0 2024-04-21T08:51:37.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f6820a1c640 1 -- 172.21.15.82:0/190869796 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68100054d0 con 0x7f681c103ae0 2024-04-21T08:51:37.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f6821a1e640 1 -- 172.21.15.82:0/190869796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681c19da20 con 0x7f681c103ae0 2024-04-21T08:51:37.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f6820a1c640 1 -- 172.21.15.82:0/190869796 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810010690 con 0x7f681c103ae0 2024-04-21T08:51:37.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.388+0000 7f6821a1e640 1 -- 172.21.15.82:0/190869796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681c1a52b0 con 0x7f681c103ae0 2024-04-21T08:51:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f6824605740 1 Processor -- start 2024-04-21T08:51:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f6824605740 1 -- start start 2024-04-21T08:51:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f6824605740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f6824605740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f6824605740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f6824605740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e7fb3151e0 con 0x55e7fb3b9a80 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f6824605740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e7fb2b2390 con 0x55e7fb3b0580 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f6820a1c640 1 -- 172.21.15.82:0/190869796 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6810004e90 con 0x7f681c103ae0 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f6824605740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e7fb3111a0 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f681b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f681b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34152/0 (socket says 172.21.15.82:34152) 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f681b7fe640 1 -- 172.21.15.82:0/1887544727 learned_addr learned my addr 172.21.15.82:0/1887544727 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.391+0000 7f681a7fc640 1 --2- 172.21.15.82:0/1887544727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f681affd640 1 --2- 172.21.15.82:0/1887544727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f681b7fe640 1 -- 172.21.15.82:0/1887544727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b0ea0 msgr2=0x55e7fb3b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f681b7fe640 1 --2- 172.21.15.82:0/1887544727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3b9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f681b7fe640 1 -- 172.21.15.82:0/1887544727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b9a80 msgr2=0x55e7fb3bbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f681b7fe640 1 --2- 172.21.15.82:0/1887544727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3bbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f681b7fe640 1 -- 172.21.15.82:0/1887544727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7fb0f17d0 con 0x55e7fb3b0580 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f6820a1c640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68040a3d20 0x7f68040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f681b7fe640 1 --2- 172.21.15.82:0/1887544727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b0960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6814009fd0 tx=0x7f6814010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6140718b96bce0d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f681affd640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68040a3d20 0x7f68040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f6801ffb640 1 -- 172.21.15.82:0/1887544727 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814011700 con 0x55e7fb3b0580 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f6820a1c640 1 -- 172.21.15.82:0/190869796 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f68100d10c0 con 0x7f681c103ae0 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f6801ffb640 1 -- 172.21.15.82:0/1887544727 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6814011d50 con 0x55e7fb3b0580 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.392+0000 7f6801ffb640 1 -- 172.21.15.82:0/1887544727 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681400e3c0 con 0x55e7fb3b0580 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.393+0000 7f6824605740 1 -- 172.21.15.82:0/1887544727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 msgr2=0x55e7fb3b0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.393+0000 7f6824605740 1 --2- 172.21.15.82:0/1887544727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b0960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6814009fd0 tx=0x7f6814010860 comp rx=0 tx=0).stop 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.393+0000 7f681affd640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68040a3d20 0x7f68040a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f680c005ab0 tx=0x7f680c005a20 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.393+0000 7f6824605740 1 -- 172.21.15.82:0/1887544727 shutdown_connections 2024-04-21T08:51:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.393+0000 7f6824605740 1 --2- 172.21.15.82:0/1887544727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3bbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.393+0000 7f6824605740 1 --2- 172.21.15.82:0/1887544727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.393+0000 7f6824605740 1 --2- 172.21.15.82:0/1887544727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b0960 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.393+0000 7f6824605740 1 -- 172.21.15.82:0/1887544727 >> 172.21.15.82:0/1887544727 conn(0x55e7fb3aacb0 msgr2=0x55e7fb3ab090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.393+0000 7f6824605740 1 -- 172.21.15.82:0/1887544727 shutdown_connections 2024-04-21T08:51:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.393+0000 7f6824605740 1 -- 172.21.15.82:0/1887544727 wait complete. 2024-04-21T08:51:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f6824605740 1 -- start start 2024-04-21T08:51:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f6824605740 1 Processor -- start 2024-04-21T08:51:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f6824605740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f6824605740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f6824605740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f6824605740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e7fb3bc870 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f6824605740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e7fb2ce2f0 con 0x55e7fb3b0580 2024-04-21T08:51:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f6824605740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e7fb3bc9f0 con 0x55e7fb3b9a80 2024-04-21T08:51:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f6824605740 10 client.0 _ll_register_callbacks cb 0x55e7fb2db580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f681affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f681a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f681affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34162/0 (socket says 172.21.15.82:34162) 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f681affd640 1 -- 172.21.15.82:0/3863206759 learned_addr learned my addr 172.21.15.82:0/3863206759 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f681a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3b7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50116/0 (socket says 172.21.15.82:50116) 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.394+0000 7f681b7fe640 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f681a7fc640 1 -- 172.21.15.82:0/3863206759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b9a80 msgr2=0x55e7fb3b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f681a7fc640 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f681a7fc640 1 -- 172.21.15.82:0/3863206759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 msgr2=0x55e7fb3b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f681a7fc640 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f681a7fc640 1 -- 172.21.15.82:0/3863206759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7fb0273f0 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f681b7fe640 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f681affd640 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f681a7fc640 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3b7080 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f68100ce3f0 tx=0x7f6810098ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac95c05084b586fd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:37.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f67faffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f67faffd640 1 -- 172.21.15.82:0/3863206759 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681009c5a0 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f67faffd640 1 -- 172.21.15.82:0/3863206759 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f68100d4ce0 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f6824605740 1 -- 172.21.15.82:0/3863206759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6814011d50 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f67faffd640 1 -- 172.21.15.82:0/3863206759 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6810005180 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f6824605740 10 client.16911 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f6824605740 1 -- 172.21.15.82:0/3863206759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e7fafe70f0 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f67faffd640 1 -- 172.21.15.82:0/3863206759 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6814011d50 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f67faffd640 1 -- 172.21.15.82:0/3863206759 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f68100ce670 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f6824605740 10 client.16911 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f6824605740 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.395+0000 7f6824605740 20 client.16911 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.396+0000 7f6824605740 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e7fb3c9b60 0x55e7fb3cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.396+0000 7f6824605740 4 client.16911 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.396+0000 7f6824605740 1 -- 172.21.15.82:0/3863206759 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7fb322c00 con 0x55e7fb3c9b60 2024-04-21T08:51:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.396+0000 7f681affd640 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e7fb3c9b60 0x55e7fb3cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.396+0000 7f67faffd640 10 client.16911 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.396+0000 7f681affd640 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e7fb3c9b60 0x55e7fb3cbf50 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=894c7c4153b3862a server_cookie=cc6780390c8b9da2 in_seq=0 out_seq=0 2024-04-21T08:51:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f67faffd640 1 -- 172.21.15.82:0/3863206759 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7fb322c00 con 0x55e7fb3c9b60 2024-04-21T08:51:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f67faffd640 10 client.16911 handle_command_reply: tid=0 2024-04-21T08:51:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f6824605740 1 client.16911 shutdown 2024-04-21T08:51:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f6824605740 1 -- 172.21.15.82:0/3863206759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 msgr2=0x55e7fb3b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f6824605740 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3b7080 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f68100ce3f0 tx=0x7f6810098ee0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f6824605740 1 -- 172.21.15.82:0/3863206759 shutdown_connections 2024-04-21T08:51:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f6824605740 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e7fb3c9b60 0x55e7fb3cbf50 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f6824605740 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f6824605740 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3b7080 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f6824605740 1 --2- 172.21.15.82:0/3863206759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f6824605740 1 -- 172.21.15.82:0/3863206759 >> 172.21.15.82:0/3863206759 conn(0x55e7fb3aacb0 msgr2=0x55e7fb3aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:37.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f67faffd640 0 client.16911 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:37.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f6824605740 1 -- 172.21.15.82:0/3863206759 shutdown_connections 2024-04-21T08:51:37.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f6824605740 1 -- 172.21.15.82:0/3863206759 wait complete. 2024-04-21T08:51:37.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.397+0000 7f6824605740 20 client.16911 trim_cache size 0 max 16384 2024-04-21T08:51:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.416+0000 7f6824605740 1 Processor -- start 2024-04-21T08:51:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.416+0000 7f6824605740 1 -- start start 2024-04-21T08:51:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.416+0000 7f6824605740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.416+0000 7f6824605740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.416+0000 7f6824605740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.416+0000 7f6824605740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1af430 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.416+0000 7f6824605740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1b0fd0 con 0x55e7fb3b0580 2024-04-21T08:51:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.416+0000 7f6824605740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1afb00 con 0x55e7fb3b9a80 2024-04-21T08:51:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.416+0000 7f681a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.416+0000 7f681a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3c5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50130/0 (socket says 172.21.15.82:50130) 2024-04-21T08:51:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.416+0000 7f681a7fc640 1 -- 172.21.15.82:0/1611772228 learned_addr learned my addr 172.21.15.82:0/1611772228 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.416+0000 7f681affd640 1 --2- 172.21.15.82:0/1611772228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.417+0000 7f681b7fe640 1 --2- 172.21.15.82:0/1611772228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.417+0000 7f681a7fc640 1 -- 172.21.15.82:0/1611772228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b9a80 msgr2=0x55e7fb3a9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.417+0000 7f681a7fc640 1 --2- 172.21.15.82:0/1611772228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3a9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.417+0000 7f681a7fc640 1 -- 172.21.15.82:0/1611772228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 msgr2=0x55e7fb3b9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.417+0000 7f681a7fc640 1 --2- 172.21.15.82:0/1611772228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.417+0000 7f681a7fc640 1 -- 172.21.15.82:0/1611772228 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7fb3b0cd0 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.417+0000 7f681a7fc640 1 --2- 172.21.15.82:0/1611772228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3c5730 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f681009cbd0 tx=0x7f6810099010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408b3716656f7729 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.417+0000 7f68017fa640 1 -- 172.21.15.82:0/1611772228 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681009c5a0 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.417+0000 7f68017fa640 1 -- 172.21.15.82:0/1611772228 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6810087040 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.417+0000 7f68017fa640 1 -- 172.21.15.82:0/1611772228 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68100a0ff0 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.418+0000 7f6824605740 1 -- 172.21.15.82:0/1611772228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 msgr2=0x55e7fb3c5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.418+0000 7f6824605740 1 --2- 172.21.15.82:0/1611772228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3c5730 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f681009cbd0 tx=0x7f6810099010 comp rx=0 tx=0).stop 2024-04-21T08:51:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.418+0000 7f6824605740 1 -- 172.21.15.82:0/1611772228 shutdown_connections 2024-04-21T08:51:37.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.418+0000 7f6824605740 1 --2- 172.21.15.82:0/1611772228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.418+0000 7f6824605740 1 --2- 172.21.15.82:0/1611772228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3c5730 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.418+0000 7f6824605740 1 --2- 172.21.15.82:0/1611772228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b0580 0x55e7fb3b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.418+0000 7f6824605740 1 -- 172.21.15.82:0/1611772228 >> 172.21.15.82:0/1611772228 conn(0x55e7fb3aacb0 msgr2=0x55e7fb3a94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:37.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.418+0000 7f6824605740 1 -- 172.21.15.82:0/1611772228 shutdown_connections 2024-04-21T08:51:37.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.418+0000 7f6824605740 1 -- 172.21.15.82:0/1611772228 wait complete. 2024-04-21T08:51:37.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.418+0000 7f6824605740 1 -- start start 2024-04-21T08:51:37.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.419+0000 7f6824605740 1 Processor -- start 2024-04-21T08:51:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.419+0000 7f6824605740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0580 0x55e7fb3c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.419+0000 7f6824605740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3bd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.419+0000 7f6824605740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.419+0000 7f6824605740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1b0900 con 0x55e7fb3b0580 2024-04-21T08:51:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.419+0000 7f6824605740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1b0230 con 0x55e7fb3b9a80 2024-04-21T08:51:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.419+0000 7f6824605740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1ad410 con 0x55e7fb3b0ea0 2024-04-21T08:51:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.419+0000 7f681affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0580 0x55e7fb3c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f681affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0580 0x55e7fb3c1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50138/0 (socket says 172.21.15.82:50138) 2024-04-21T08:51:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f681b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f681affd640 1 -- 172.21.15.82:0/2285936310 learned_addr learned my addr 172.21.15.82:0/2285936310 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f6824605740 10 client.0 _ll_register_callbacks cb 0x7f68100d4860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:37.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f681a7fc640 1 --2- 172.21.15.82:0/2285936310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3bd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f681affd640 1 -- 172.21.15.82:0/2285936310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b0ea0 msgr2=0x55e7fb3bd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f681affd640 1 --2- 172.21.15.82:0/2285936310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3bd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f681affd640 1 -- 172.21.15.82:0/2285936310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b9a80 msgr2=0x55e7fb3bd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f681affd640 1 --2- 172.21.15.82:0/2285936310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3bd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f681affd640 1 -- 172.21.15.82:0/2285936310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7fb3be020 con 0x55e7fb3b0580 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f681affd640 1 --2- 172.21.15.82:0/2285936310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0580 0x55e7fb3c1f70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f680c02c370 tx=0x7f680c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3cf90a9114230b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f67faffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f67faffd640 1 -- 172.21.15.82:0/2285936310 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c055070 con 0x55e7fb3b0580 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f67faffd640 1 -- 172.21.15.82:0/2285936310 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f680c005250 con 0x55e7fb3b0580 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f6824605740 1 -- 172.21.15.82:0/2285936310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6810087040 con 0x55e7fb3b0580 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f67faffd640 1 -- 172.21.15.82:0/2285936310 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f680c061050 con 0x55e7fb3b0580 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f6824605740 10 client.16917 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.420+0000 7f6824605740 1 -- 172.21.15.82:0/2285936310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e7fb430010 con 0x55e7fb3b0580 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.421+0000 7f67faffd640 1 -- 172.21.15.82:0/2285936310 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6810087040 con 0x55e7fb3b0580 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.421+0000 7f67faffd640 1 -- 172.21.15.82:0/2285936310 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f680c051270 con 0x55e7fb3b0580 2024-04-21T08:51:37.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.421+0000 7f6824605740 10 client.16917 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.421+0000 7f6824605740 10 client.16917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.421+0000 7f6824605740 20 client.16917 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.421+0000 7f6824605740 1 --2- 172.21.15.82:0/2285936310 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e7fb430760 0x55e7fb432b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.421+0000 7f6824605740 4 client.16917 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.421+0000 7f6824605740 1 -- 172.21.15.82:0/2285936310 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e7fb31ca70 con 0x55e7fb430760 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.421+0000 7f681a7fc640 1 --2- 172.21.15.82:0/2285936310 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e7fb430760 0x55e7fb432b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.421+0000 7f67faffd640 10 client.16917 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.421+0000 7f681a7fc640 1 --2- 172.21.15.82:0/2285936310 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e7fb430760 0x55e7fb432b80 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f2dad6f631a866 server_cookie=54aa4cba1e7eb935 in_seq=0 out_seq=0 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f67faffd640 1 -- 172.21.15.82:0/2285936310 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e7fb31ca70 con 0x55e7fb430760 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f67faffd640 10 client.16917 handle_command_reply: tid=0 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6824605740 1 client.16917 shutdown 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6824605740 1 -- 172.21.15.82:0/2285936310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0580 msgr2=0x55e7fb3c1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:37.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6824605740 1 --2- 172.21.15.82:0/2285936310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0580 0x55e7fb3c1f70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f680c02c370 tx=0x7f680c009260 comp rx=0 tx=0).stop 2024-04-21T08:51:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6824605740 1 -- 172.21.15.82:0/2285936310 shutdown_connections 2024-04-21T08:51:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6824605740 1 --2- 172.21.15.82:0/2285936310 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e7fb430760 0x55e7fb432b80 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6824605740 1 --2- 172.21.15.82:0/2285936310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7fb3b9a80 0x55e7fb3bd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6824605740 1 --2- 172.21.15.82:0/2285936310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7fb3b0ea0 0x55e7fb3bd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f67faffd640 0 client.16917 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6824605740 1 --2- 172.21.15.82:0/2285936310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7fb3b0580 0x55e7fb3c1f70 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6824605740 1 -- 172.21.15.82:0/2285936310 >> 172.21.15.82:0/2285936310 conn(0x55e7fb3aacb0 msgr2=0x55e7fb3a9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6824605740 1 -- 172.21.15.82:0/2285936310 shutdown_connections 2024-04-21T08:51:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6824605740 1 -- 172.21.15.82:0/2285936310 wait complete. 2024-04-21T08:51:37.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6824605740 20 client.16917 trim_cache size 0 max 16384 2024-04-21T08:51:37.427 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:51:37.428 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:51:37.428 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:51:37.428 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:51:37.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6821a1e640 1 -- 172.21.15.82:0/190869796 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68040a3d20 msgr2=0x7f68040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:37.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.422+0000 7f6821a1e640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68040a3d20 0x7f68040a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f680c005ab0 tx=0x7f680c005a20 comp rx=0 tx=0).stop 2024-04-21T08:51:37.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.423+0000 7f6821a1e640 1 -- 172.21.15.82:0/190869796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f681c103ae0 msgr2=0x7f681c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:37.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.423+0000 7f6821a1e640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f681c103ae0 0x7f681c198c80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f681000e9f0 tx=0x7f681000edf0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.423+0000 7f6821a1e640 1 -- 172.21.15.82:0/190869796 shutdown_connections 2024-04-21T08:51:37.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.423+0000 7f6821a1e640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f681c10c500 0x7f681c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.423+0000 7f6821a1e640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f681c103ae0 0x7f681c198c80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.423+0000 7f6821a1e640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68040a3d20 0x7f68040a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.423+0000 7f6821a1e640 1 --2- 172.21.15.82:0/190869796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f681c1031c0 0x7f681c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.423+0000 7f6821a1e640 1 -- 172.21.15.82:0/190869796 >> 172.21.15.82:0/190869796 conn(0x7f681c078000 msgr2=0x7f681c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:37.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.423+0000 7f6821a1e640 1 -- 172.21.15.82:0/190869796 shutdown_connections 2024-04-21T08:51:37.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.423+0000 7f6821a1e640 1 -- 172.21.15.82:0/190869796 wait complete. 2024-04-21T08:51:37.434 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:51:37.435 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:51:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.542+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/4269447064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc14101b60 msgr2=0x7fdc1410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.542+0000 7fdc1b7f4640 1 --2- 172.21.15.82:0/4269447064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc14101b60 0x7fdc1410dc90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009cd0 tx=0x7fdc0c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.542+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/4269447064 shutdown_connections 2024-04-21T08:51:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.542+0000 7fdc1b7f4640 1 --2- 172.21.15.82:0/4269447064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc14101b60 0x7fdc1410dc90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.542+0000 7fdc1b7f4640 1 --2- 172.21.15.82:0/4269447064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc141011a0 0x7fdc14101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.542+0000 7fdc1b7f4640 1 --2- 172.21.15.82:0/4269447064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc14105e30 0x7fdc14106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.542+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/4269447064 >> 172.21.15.82:0/4269447064 conn(0x7fdc14077e30 msgr2=0x7fdc14078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.542+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/4269447064 shutdown_connections 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.542+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/4269447064 wait complete. 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.542+0000 7fdc1b7f4640 1 Processor -- start 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.542+0000 7fdc1b7f4640 1 -- start start 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc1b7f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc141011a0 0x7fdc1410e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc1b7f4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc14101b60 0x7fdc1410ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc1b7f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc14105e30 0x7fdc1410f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc1b7f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdc14112240 con 0x7fdc14105e30 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc1b7f4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141120c0 con 0x7fdc14101b60 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc1b7f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141123c0 con 0x7fdc141011a0 2024-04-21T08:51:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc19d91640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc14105e30 0x7fdc1410f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc18d8f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc14101b60 0x7fdc1410ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc19d91640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc14105e30 0x7fdc1410f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50152/0 (socket says 172.21.15.82:50152) 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc19d91640 1 -- 172.21.15.82:0/116562875 learned_addr learned my addr 172.21.15.82:0/116562875 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc19590640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc141011a0 0x7fdc1410e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc19d91640 1 -- 172.21.15.82:0/116562875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc141011a0 msgr2=0x7fdc1410e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc19d91640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc141011a0 0x7fdc1410e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc19d91640 1 -- 172.21.15.82:0/116562875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc14101b60 msgr2=0x7fdc1410ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc19d91640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc14101b60 0x7fdc1410ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc19d91640 1 -- 172.21.15.82:0/116562875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc141a6bc0 con 0x7fdc14105e30 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc18d8f640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc14101b60 0x7fdc1410ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc19d91640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc14105e30 0x7fdc1410f1a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009ca0 tx=0x7fdc0c034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc0a7fc640 1 -- 172.21.15.82:0/116562875 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c041020 con 0x7fdc14105e30 2024-04-21T08:51:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc0a7fc640 1 -- 172.21.15.82:0/116562875 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c034ca0 con 0x7fdc14105e30 2024-04-21T08:51:37.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/116562875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc141a6e50 con 0x7fdc14105e30 2024-04-21T08:51:37.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc0a7fc640 1 -- 172.21.15.82:0/116562875 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c039040 con 0x7fdc14105e30 2024-04-21T08:51:37.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.543+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/116562875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc141a7310 con 0x7fdc14105e30 2024-04-21T08:51:37.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.544+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/116562875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc14068de0 con 0x7fdc14105e30 2024-04-21T08:51:37.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.546+0000 7fdc0a7fc640 1 -- 172.21.15.82:0/116562875 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc0c044020 con 0x7fdc14105e30 2024-04-21T08:51:37.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.547+0000 7fdc0a7fc640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdc000a3d20 0x7fdc000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:37.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.547+0000 7fdc19590640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdc000a3d20 0x7fdc000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:37.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.547+0000 7fdc0a7fc640 1 -- 172.21.15.82:0/116562875 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fdc0c0f2cf0 con 0x7fdc14105e30 2024-04-21T08:51:37.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.547+0000 7fdc19590640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdc000a3d20 0x7fdc000a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc100095f0 tx=0x7fdc10005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:37.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:37.551+0000 7fdc0a7fc640 1 -- 172.21.15.82:0/116562875 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc0c04b020 con 0x7fdc14105e30 2024-04-21T08:51:38.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.028+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/116562875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc14109350 con 0x7fdc14105e30 2024-04-21T08:51:38.032 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:51:38.032 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:51:22.169271+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24481,24509,24476,24409,24407]},"id":1}]} 2024-04-21T08:51:38.032 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 35 2024-04-21T08:51:38.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.030+0000 7fdc0a7fc640 1 -- 172.21.15.82:0/116562875 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11947 (secure 0 0 0) 0x7fdc0c003e20 con 0x7fdc14105e30 2024-04-21T08:51:38.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.038+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/116562875 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdc000a3d20 msgr2=0x7fdc000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:38.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.038+0000 7fdc1b7f4640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdc000a3d20 0x7fdc000a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc100095f0 tx=0x7fdc10005f90 comp rx=0 tx=0).stop 2024-04-21T08:51:38.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.038+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/116562875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc14105e30 msgr2=0x7fdc1410f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:38.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.038+0000 7fdc1b7f4640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc14105e30 0x7fdc1410f1a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009ca0 tx=0x7fdc0c034670 comp rx=0 tx=0).stop 2024-04-21T08:51:38.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.038+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/116562875 shutdown_connections 2024-04-21T08:51:38.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.038+0000 7fdc1b7f4640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdc000a3d20 0x7fdc000a61e0 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc100095f0 tx=0x7fdc10005f90 comp rx=0 tx=0).stop 2024-04-21T08:51:38.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.038+0000 7fdc1b7f4640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc14105e30 0x7fdc1410f1a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.038+0000 7fdc1b7f4640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc14101b60 0x7fdc1410ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.038+0000 7fdc1b7f4640 1 --2- 172.21.15.82:0/116562875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc141011a0 0x7fdc1410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.038+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/116562875 >> 172.21.15.82:0/116562875 conn(0x7fdc14077e30 msgr2=0x7fdc140fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:38.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.043+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/116562875 shutdown_connections 2024-04-21T08:51:38.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.044+0000 7fdc1b7f4640 1 -- 172.21.15.82:0/116562875 wait complete. 2024-04-21T08:51:38.076 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:51:38.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.371+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/3297929628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f0810e390 msgr2=0x7f6f0810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:38.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.371+0000 7f6f0c8f3640 1 --2- 172.21.15.82:0/3297929628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f0810e390 0x7f6f0810e770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80074e0 tx=0x7f6ef802c960 comp rx=0 tx=0).stop 2024-04-21T08:51:38.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.374+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/3297929628 shutdown_connections 2024-04-21T08:51:38.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.374+0000 7f6f0c8f3640 1 --2- 172.21.15.82:0/3297929628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f0810c690 0x7f6f0807a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.374+0000 7f6f0c8f3640 1 --2- 172.21.15.82:0/3297929628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f0810ecb0 0x7f6f0810bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.374+0000 7f6f0c8f3640 1 --2- 172.21.15.82:0/3297929628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f0810e390 0x7f6f0810e770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.374+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/3297929628 >> 172.21.15.82:0/3297929628 conn(0x7f6f0806ced0 msgr2=0x7f6f0806d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:38.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.377+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/3297929628 shutdown_connections 2024-04-21T08:51:38.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.378+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/3297929628 wait complete. 2024-04-21T08:51:38.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.379+0000 7f6f0c8f3640 1 Processor -- start 2024-04-21T08:51:38.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.379+0000 7f6f0c8f3640 1 -- start start 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.379+0000 7f6f0c8f3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f0810c690 0x7f6f08084bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.379+0000 7f6f0c8f3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f0810e390 0x7f6f0807ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.379+0000 7f6f0c8f3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f0810ecb0 0x7f6f0807f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.379+0000 7f6f0c8f3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0807d600 con 0x7f6f0810c690 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.379+0000 7f6f0c8f3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0807d480 con 0x7f6f0810e390 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.379+0000 7f6f0c8f3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0807d780 con 0x7f6f0810ecb0 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.379+0000 7f6f0659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f0810e390 0x7f6f0807ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.380+0000 7f6f0659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f0810e390 0x7f6f0807ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34230/0 (socket says 172.21.15.82:34230) 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.380+0000 7f6f0659c640 1 -- 172.21.15.82:0/1209058127 learned_addr learned my addr 172.21.15.82:0/1209058127 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.380+0000 7f6f0759e640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f0810ecb0 0x7f6f0807f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.380+0000 7f6f0759e640 1 -- 172.21.15.82:0/1209058127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f0810e390 msgr2=0x7f6f0807ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.380+0000 7f6f0759e640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f0810e390 0x7f6f0807ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.380+0000 7f6f0759e640 1 -- 172.21.15.82:0/1209058127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f0810c690 msgr2=0x7f6f08084bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:38.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.380+0000 7f6f0759e640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f0810c690 0x7f6f08084bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.380+0000 7f6f0759e640 1 -- 172.21.15.82:0/1209058127 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f081b5fb0 con 0x7f6f0810ecb0 2024-04-21T08:51:38.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.382+0000 7f6f0759e640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f0810ecb0 0x7f6f0807f3d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00b3e0 tx=0x7f6efc00b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:38.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.384+0000 7f6eeffff640 1 -- 172.21.15.82:0/1209058127 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc015020 con 0x7f6f0810ecb0 2024-04-21T08:51:38.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.384+0000 7f6eeffff640 1 -- 172.21.15.82:0/1209058127 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6efc011ce0 con 0x7f6f0810ecb0 2024-04-21T08:51:38.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.384+0000 7f6eeffff640 1 -- 172.21.15.82:0/1209058127 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc010e00 con 0x7f6f0810ecb0 2024-04-21T08:51:38.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.384+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/1209058127 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f081b62a0 con 0x7f6f0810ecb0 2024-04-21T08:51:38.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.384+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/1209058127 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f081b67a0 con 0x7f6f0810ecb0 2024-04-21T08:51:38.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.388+0000 7f6f0fdbd740 1 Processor -- start 2024-04-21T08:51:38.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.389+0000 7f6f0fdbd740 1 -- start start 2024-04-21T08:51:38.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.389+0000 7f6eeffff640 1 -- 172.21.15.82:0/1209058127 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6efc018020 con 0x7f6f0810ecb0 2024-04-21T08:51:38.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.389+0000 7f6f0fdbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e866d0 0x556e34e86ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.389+0000 7f6f0fdbd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34e8f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.389+0000 7f6f0fdbd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e8fc00 0x556e34e92050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.389+0000 7f6f0fdbd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556e34deb1f0 con 0x556e34e87080 2024-04-21T08:51:38.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.389+0000 7f6f0fdbd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556e34d884b0 con 0x556e34e8fc00 2024-04-21T08:51:38.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.389+0000 7f6f0fdbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556e34de71b0 con 0x556e34e866d0 2024-04-21T08:51:38.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.389+0000 7f6f06d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34e8f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.389+0000 7f6f06d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34e8f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50170/0 (socket says 172.21.15.82:50170) 2024-04-21T08:51:38.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.389+0000 7f6f06d9d640 1 -- 172.21.15.82:0/3046363587 learned_addr learned my addr 172.21.15.82:0/3046363587 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:38.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.390+0000 7f6eeffff640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6ef00a3f20 0x7f6ef00a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.390+0000 7f6eeffff640 1 -- 172.21.15.82:0/1209058127 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6efc0d0050 con 0x7f6f0810ecb0 2024-04-21T08:51:38.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.390+0000 7f6f06d9d640 1 -- 172.21.15.82:0/3046363587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e866d0 msgr2=0x556e34e86ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:51:38.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.390+0000 7f6f06d9d640 1 --2- 172.21.15.82:0/3046363587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e866d0 0x556e34e86ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.390+0000 7f6f06d9d640 1 -- 172.21.15.82:0/3046363587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e8fc00 msgr2=0x556e34e92050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:51:38.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.390+0000 7f6f06d9d640 1 --2- 172.21.15.82:0/3046363587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e8fc00 0x556e34e92050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.390+0000 7f6f06d9d640 1 -- 172.21.15.82:0/3046363587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e34d7a1b0 con 0x556e34e87080 2024-04-21T08:51:38.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.390+0000 7f6f0759e640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6ef00a3f20 0x7f6ef00a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.391+0000 7f6f06d9d640 1 --2- 172.21.15.82:0/3046363587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34e8f630 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6ef802cef0 tx=0x7f6ef8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafcad00b9940039 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:38.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.392+0000 7f6eed7fa640 1 -- 172.21.15.82:0/3046363587 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8008e10 con 0x556e34e87080 2024-04-21T08:51:38.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.392+0000 7f6eed7fa640 1 -- 172.21.15.82:0/3046363587 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef8040070 con 0x556e34e87080 2024-04-21T08:51:38.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.392+0000 7f6eed7fa640 1 -- 172.21.15.82:0/3046363587 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800a0d0 con 0x556e34e87080 2024-04-21T08:51:38.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.392+0000 7f6f0759e640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6ef00a3f20 0x7f6ef00a63e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6efc006fd0 tx=0x7f6efc010680 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:38.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.393+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/3046363587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 msgr2=0x556e34e8f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:38.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.393+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/3046363587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34e8f630 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6ef802cef0 tx=0x7f6ef8005780 comp rx=0 tx=0).stop 2024-04-21T08:51:38.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.395+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/3046363587 shutdown_connections 2024-04-21T08:51:38.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.395+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/3046363587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e8fc00 0x556e34e92050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.395+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/3046363587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34e8f630 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.395+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/3046363587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e866d0 0x556e34e86ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.395+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/3046363587 >> 172.21.15.82:0/3046363587 conn(0x556e34e80da0 msgr2=0x556e34e81180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:38.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.396+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/3046363587 shutdown_connections 2024-04-21T08:51:38.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.396+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/3046363587 wait complete. 2024-04-21T08:51:38.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.397+0000 7f6f0fdbd740 1 -- start start 2024-04-21T08:51:38.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.397+0000 7f6f0fdbd740 1 Processor -- start 2024-04-21T08:51:38.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.397+0000 7f6f0fdbd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e866d0 0x556e34e8f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.397+0000 7f6f0fdbd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34e8c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.397+0000 7f6f0fdbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e8fc00 0x556e34e8cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.397+0000 7f6f0fdbd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556e34e92b10 con 0x556e34e87080 2024-04-21T08:51:38.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.397+0000 7f6f0fdbd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556e34da4500 con 0x556e34e866d0 2024-04-21T08:51:38.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.397+0000 7f6f0fdbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556e34e92c90 con 0x556e34e8fc00 2024-04-21T08:51:38.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.397+0000 7f6f0fdbd740 10 client.0 _ll_register_callbacks cb 0x556e34d72b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.398+0000 7f6f06d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34e8c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.398+0000 7f6f06d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34e8c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50172/0 (socket says 172.21.15.82:50172) 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.398+0000 7f6f06d9d640 1 -- 172.21.15.82:0/2458267930 learned_addr learned my addr 172.21.15.82:0/2458267930 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.398+0000 7f6f0659c640 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e866d0 0x556e34e8f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.398+0000 7f6f0659c640 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e8fc00 0x556e34e8cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.398+0000 7f6f06d9d640 1 -- 172.21.15.82:0/2458267930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e8fc00 msgr2=0x556e34e8cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.398+0000 7f6f06d9d640 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e8fc00 0x556e34e8cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.398+0000 7f6f06d9d640 1 -- 172.21.15.82:0/2458267930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e866d0 msgr2=0x556e34e8f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.398+0000 7f6f06d9d640 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e866d0 0x556e34e8f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.398+0000 7f6f06d9d640 1 -- 172.21.15.82:0/2458267930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e34afd3f0 con 0x556e34e87080 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.399+0000 7f6f06d9d640 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34e8c910 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8003ca0 tx=0x7f6ef8003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e30c9719164318 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.399+0000 7f6eeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:51:38.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.399+0000 7f6eeaff5640 1 -- 172.21.15.82:0/2458267930 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef804a070 con 0x556e34e87080 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.399+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2458267930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ef8040070 con 0x556e34e87080 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.399+0000 7f6eeaff5640 1 -- 172.21.15.82:0/2458267930 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6ef8004150 con 0x556e34e87080 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.399+0000 7f6eeaff5640 1 -- 172.21.15.82:0/2458267930 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6ef8035040 con 0x556e34e87080 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.400+0000 7f6f0fdbd740 10 client.16929 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.400+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2458267930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556e34bc77d0 con 0x556e34e87080 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.400+0000 7f6eeaff5640 1 -- 172.21.15.82:0/2458267930 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef8040070 con 0x556e34e87080 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.401+0000 7f6eeaff5640 1 -- 172.21.15.82:0/2458267930 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f6ef8030050 con 0x556e34e87080 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.401+0000 7f6f0fdbd740 10 client.16929 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.401+0000 7f6f0fdbd740 10 client.16929 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.401+0000 7f6f0fdbd740 20 client.16929 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.401+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556e34e9fd20 0x556e34ea2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.401+0000 7f6f0fdbd740 4 client.16929 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.401+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2458267930 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e34d8a930 con 0x556e34e9fd20 2024-04-21T08:51:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.401+0000 7f6f0659c640 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556e34e9fd20 0x556e34ea2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.404+0000 7f6f0659c640 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556e34e9fd20 0x556e34ea2110 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b2fb7112678eb0a server_cookie=911f494cf58ff7a9 in_seq=0 out_seq=0 2024-04-21T08:51:38.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.404+0000 7f6eeaff5640 10 client.16929 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:51:38.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.405+0000 7f6eeaff5640 1 -- 172.21.15.82:0/2458267930 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e34d8a930 con 0x556e34e9fd20 2024-04-21T08:51:38.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.405+0000 7f6eeaff5640 10 client.16929 handle_command_reply: tid=0 2024-04-21T08:51:38.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.405+0000 7f6f0fdbd740 1 client.16929 shutdown 2024-04-21T08:51:38.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.405+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2458267930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 msgr2=0x556e34e8c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:38.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.405+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34e8c910 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8003ca0 tx=0x7f6ef8003960 comp rx=0 tx=0).stop 2024-04-21T08:51:38.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.408+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2458267930 shutdown_connections 2024-04-21T08:51:38.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.408+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556e34e9fd20 0x556e34ea2110 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.408+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e8fc00 0x556e34e8cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.408+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34e8c910 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.408+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2458267930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e866d0 0x556e34e8f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.408+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2458267930 >> 172.21.15.82:0/2458267930 conn(0x556e34e80da0 msgr2=0x556e34e82af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:38.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.408+0000 7f6eeaff5640 0 client.16929 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:51:38.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.410+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2458267930 shutdown_connections 2024-04-21T08:51:38.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.412+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2458267930 wait complete. 2024-04-21T08:51:38.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.413+0000 7f6f0fdbd740 20 client.16929 trim_cache size 0 max 16384 2024-04-21T08:51:38.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.445+0000 7f6f0fdbd740 1 Processor -- start 2024-04-21T08:51:38.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.445+0000 7f6f0fdbd740 1 -- start start 2024-04-21T08:51:38.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.445+0000 7f6f0fdbd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e866d0 0x556e34e8b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.445+0000 7f6f0fdbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e87080 0x556e34e8f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.446+0000 7f6f0fdbd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e8fc00 0x556e34e8f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.446+0000 7f6f0fdbd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081d79f0 con 0x556e34e866d0 2024-04-21T08:51:38.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.446+0000 7f6f0fdbd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081d9590 con 0x556e34e8fc00 2024-04-21T08:51:38.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.446+0000 7f6f0659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e866d0 0x556e34e8b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.446+0000 7f6f0659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e8fc00 0x556e34e8f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.446+0000 7f6f0659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e8fc00 0x556e34e8f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34246/0 (socket says 172.21.15.82:34246) 2024-04-21T08:51:38.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.446+0000 7f6f0659c640 1 -- 172.21.15.82:0/2051464412 learned_addr learned my addr 172.21.15.82:0/2051464412 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:38.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.446+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2051464412 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081d80c0 con 0x556e34e87080 2024-04-21T08:51:38.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.446+0000 7f6f06d9d640 1 --2- 172.21.15.82:0/2051464412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e87080 0x556e34e8f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.447+0000 7f6f0659c640 1 -- 172.21.15.82:0/2051464412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e87080 msgr2=0x556e34e8f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.447+0000 7f6f0659c640 1 --2- 172.21.15.82:0/2051464412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e87080 0x556e34e8f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.447+0000 7f6f0659c640 1 -- 172.21.15.82:0/2051464412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e866d0 msgr2=0x556e34e8b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.447+0000 7f6f0659c640 1 --2- 172.21.15.82:0/2051464412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e866d0 0x556e34e8b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.447+0000 7f6f0659c640 1 -- 172.21.15.82:0/2051464412 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e34e8ae60 con 0x556e34e8fc00 2024-04-21T08:51:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.448+0000 7f6f0659c640 1 --2- 172.21.15.82:0/2051464412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e8fc00 0x556e34e8f580 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f00004a80 tx=0x7f6f00004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=829e7e725c87347a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.448+0000 7f6eecff9640 1 -- 172.21.15.82:0/2051464412 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00004b20 con 0x556e34e8fc00 2024-04-21T08:51:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.448+0000 7f6eecff9640 1 -- 172.21.15.82:0/2051464412 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f00003e20 con 0x556e34e8fc00 2024-04-21T08:51:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.448+0000 7f6eecff9640 1 -- 172.21.15.82:0/2051464412 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0000fd60 con 0x556e34e8fc00 2024-04-21T08:51:38.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.449+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2051464412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e8fc00 msgr2=0x556e34e8f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:38.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.449+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2051464412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e8fc00 0x556e34e8f580 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f00004a80 tx=0x7f6f00004ab0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.449+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2051464412 shutdown_connections 2024-04-21T08:51:38.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.449+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2051464412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e8fc00 0x556e34e8f580 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.449+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2051464412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e87080 0x556e34e8f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.449+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2051464412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e866d0 0x556e34e8b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.449+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2051464412 >> 172.21.15.82:0/2051464412 conn(0x556e34e80da0 msgr2=0x556e34e9b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:38.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.449+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2051464412 shutdown_connections 2024-04-21T08:51:38.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.450+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2051464412 wait complete. 2024-04-21T08:51:38.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.450+0000 7f6f0fdbd740 1 -- start start 2024-04-21T08:51:38.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.450+0000 7f6f0fdbd740 1 Processor -- start 2024-04-21T08:51:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.451+0000 7f6f0fdbd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e866d0 0x556e34ea5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.451+0000 7f6f0fdbd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34ea56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.451+0000 7f6f0fdbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e8fc00 0x556e34ea2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.451+0000 7f6f0fdbd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081d8ec0 con 0x556e34e87080 2024-04-21T08:51:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.451+0000 7f6f0fdbd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081d87f0 con 0x556e34e866d0 2024-04-21T08:51:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.451+0000 7f6f0fdbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081d59d0 con 0x556e34e8fc00 2024-04-21T08:51:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.451+0000 7f6f0fdbd740 10 client.0 _ll_register_callbacks cb 0x7f6ef80035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:38.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.452+0000 7f6f06d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34ea56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.453+0000 7f6f06d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34ea56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50192/0 (socket says 172.21.15.82:50192) 2024-04-21T08:51:38.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.453+0000 7f6f06d9d640 1 -- 172.21.15.82:0/2368562287 learned_addr learned my addr 172.21.15.82:0/2368562287 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:38.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.453+0000 7f6f0659c640 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e866d0 0x556e34ea5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.453+0000 7f6f0659c640 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e8fc00 0x556e34ea2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.453+0000 7f6f0659c640 1 -- 172.21.15.82:0/2368562287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e8fc00 msgr2=0x556e34ea2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:38.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.453+0000 7f6f0659c640 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e8fc00 0x556e34ea2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.453+0000 7f6f0659c640 1 -- 172.21.15.82:0/2368562287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 msgr2=0x556e34ea56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:38.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.453+0000 7f6f0659c640 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34ea56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.453+0000 7f6f0659c640 1 -- 172.21.15.82:0/2368562287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e34ea2f20 con 0x556e34e866d0 2024-04-21T08:51:38.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.454+0000 7f6f0659c640 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e866d0 0x556e34ea5190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6f00019b90 tx=0x7f6f00007940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6e1951ccc85ba2a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:38.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.454+0000 7f6eeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:51:38.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.455+0000 7f6eeaff5640 1 -- 172.21.15.82:0/2368562287 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0000f790 con 0x556e34e866d0 2024-04-21T08:51:38.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.455+0000 7f6eeaff5640 1 -- 172.21.15.82:0/2368562287 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6f0000f930 con 0x556e34e866d0 2024-04-21T08:51:38.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.455+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2368562287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f00003e20 con 0x556e34e866d0 2024-04-21T08:51:38.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.457+0000 7f6eeaff5640 1 -- 172.21.15.82:0/2368562287 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6f0000fb30 con 0x556e34e866d0 2024-04-21T08:51:38.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.457+0000 7f6f0fdbd740 10 client.26273 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:38.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.457+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2368562287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556e34f05b90 con 0x556e34e866d0 2024-04-21T08:51:38.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.458+0000 7f6eeaff5640 1 -- 172.21.15.82:0/2368562287 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f00003e20 con 0x556e34e866d0 2024-04-21T08:51:38.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.458+0000 7f6eeaff5640 1 -- 172.21.15.82:0/2368562287 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f6f00010430 con 0x556e34e866d0 2024-04-21T08:51:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.459+0000 7f6f0fdbd740 10 client.26273 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.459+0000 7f6f0fdbd740 10 client.26273 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:51:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.459+0000 7f6f0fdbd740 20 client.26273 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.459+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556e34f061e0 0x556e34f08600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.459+0000 7f6f0fdbd740 4 client.26273 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.459+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2368562287 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e34df8c10 con 0x556e34f061e0 2024-04-21T08:51:38.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.459+0000 7f6f06d9d640 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556e34f061e0 0x556e34f08600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:38.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.460+0000 7f6f06d9d640 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556e34f061e0 0x556e34f08600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1c8f8d0936ab0032 server_cookie=4efa70e9fc2db940 in_seq=0 out_seq=0 2024-04-21T08:51:38.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.460+0000 7f6eeaff5640 10 client.26273 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:51:38.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.461+0000 7f6eeaff5640 1 -- 172.21.15.82:0/2368562287 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e34df8c10 con 0x556e34f061e0 2024-04-21T08:51:38.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.461+0000 7f6eeaff5640 10 client.26273 handle_command_reply: tid=0 2024-04-21T08:51:38.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.462+0000 7f6f0fdbd740 1 client.26273 shutdown 2024-04-21T08:51:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.463+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2368562287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e866d0 msgr2=0x556e34ea5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.463+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e866d0 0x556e34ea5190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6f00019b90 tx=0x7f6f00007940 comp rx=0 tx=0).stop 2024-04-21T08:51:38.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.464+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2368562287 shutdown_connections 2024-04-21T08:51:38.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.464+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556e34f061e0 0x556e34f08600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.464+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556e34e8fc00 0x556e34ea2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.464+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e34e87080 0x556e34ea56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.464+0000 7f6f0fdbd740 1 --2- 172.21.15.82:0/2368562287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e34e866d0 0x556e34ea5190 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.464+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2368562287 >> 172.21.15.82:0/2368562287 conn(0x556e34e80da0 msgr2=0x556e34e8b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:38.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.464+0000 7f6eeaff5640 0 client.26273 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:51:38.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.465+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2368562287 shutdown_connections 2024-04-21T08:51:38.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.467+0000 7f6f0fdbd740 1 -- 172.21.15.82:0/2368562287 wait complete. 2024-04-21T08:51:38.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.467+0000 7f6f0fdbd740 20 client.26273 trim_cache size 0 max 16384 2024-04-21T08:51:38.467 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:51:38.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.470+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/1209058127 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6ef00a3f20 msgr2=0x7f6ef00a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:38.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.470+0000 7f6f0c8f3640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6ef00a3f20 0x7f6ef00a63e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6efc006fd0 tx=0x7f6efc010680 comp rx=0 tx=0).stop 2024-04-21T08:51:38.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.470+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/1209058127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f0810ecb0 msgr2=0x7f6f0807f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:38.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.470+0000 7f6f0c8f3640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f0810ecb0 0x7f6f0807f3d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00b3e0 tx=0x7f6efc00b8a0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.470+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/1209058127 shutdown_connections 2024-04-21T08:51:38.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.470+0000 7f6f0c8f3640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f0810ecb0 0x7f6f0807f3d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.470+0000 7f6f0c8f3640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f0810e390 0x7f6f0807ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.471+0000 7f6f0c8f3640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6ef00a3f20 0x7f6ef00a63e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.471+0000 7f6f0c8f3640 1 --2- 172.21.15.82:0/1209058127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f0810c690 0x7f6f08084bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:38.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.471+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/1209058127 >> 172.21.15.82:0/1209058127 conn(0x7f6f0806ced0 msgr2=0x7f6f08071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:38.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.472+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/1209058127 shutdown_connections 2024-04-21T08:51:38.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:38.472+0000 7f6f0c8f3640 1 -- 172.21.15.82:0/1209058127 wait complete. 2024-04-21T08:51:38.493 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:51:38.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:38 smithi134 ceph-mon[31772]: pgmap v141: 129 pgs: 129 active+clean; 498 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-04-21T08:51:38.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:38 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/116562875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:38.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:38 smithi082 ceph-mon[27967]: pgmap v141: 129 pgs: 129 active+clean; 498 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-04-21T08:51:38.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:38 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/116562875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:38 smithi106 ceph-mon[31072]: pgmap v141: 129 pgs: 129 active+clean; 498 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-04-21T08:51:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:38 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/116562875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:39.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.041+0000 7face9559640 1 -- 172.21.15.82:0/619990258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face41105c0 msgr2=0x7face41109a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.041+0000 7face9559640 1 --2- 172.21.15.82:0/619990258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face41105c0 0x7face41109a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facdc009ca0 tx=0x7facdc02c140 comp rx=0 tx=0).stop 2024-04-21T08:51:39.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.044+0000 7face9559640 1 -- 172.21.15.82:0/619990258 shutdown_connections 2024-04-21T08:51:39.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.044+0000 7face9559640 1 --2- 172.21.15.82:0/619990258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7face411b730 0x7face411db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.044+0000 7face9559640 1 --2- 172.21.15.82:0/619990258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7face4108fe0 0x7face4109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.044+0000 7face9559640 1 --2- 172.21.15.82:0/619990258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face41105c0 0x7face41109a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.044+0000 7face9559640 1 -- 172.21.15.82:0/619990258 >> 172.21.15.82:0/619990258 conn(0x7face406cf30 msgr2=0x7face406d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.047+0000 7face9559640 1 -- 172.21.15.82:0/619990258 shutdown_connections 2024-04-21T08:51:39.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.047+0000 7face9559640 1 -- 172.21.15.82:0/619990258 wait complete. 2024-04-21T08:51:39.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.048+0000 7face9559640 1 Processor -- start 2024-04-21T08:51:39.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.048+0000 7face9559640 1 -- start start 2024-04-21T08:51:39.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.049+0000 7face9559640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face4108fe0 0x7face41a7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.049+0000 7face9559640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7face41105c0 0x7face41a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.049+0000 7face9559640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7face411b730 0x7face41abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.049+0000 7face9559640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7face4120f90 con 0x7face411b730 2024-04-21T08:51:39.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.049+0000 7face9559640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7face4120e10 con 0x7face41105c0 2024-04-21T08:51:39.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.049+0000 7face9559640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7face4121110 con 0x7face4108fe0 2024-04-21T08:51:39.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.049+0000 7face2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face4108fe0 0x7face41a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.049+0000 7face27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7face41105c0 0x7face41a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.049+0000 7face27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7face41105c0 0x7face41a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34266/0 (socket says 172.21.15.82:34266) 2024-04-21T08:51:39.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.049+0000 7face27fc640 1 -- 172.21.15.82:0/2834231628 learned_addr learned my addr 172.21.15.82:0/2834231628 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.050+0000 7face27fc640 1 -- 172.21.15.82:0/2834231628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face4108fe0 msgr2=0x7face41a7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.050+0000 7face27fc640 1 --2- 172.21.15.82:0/2834231628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face4108fe0 0x7face41a7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.050+0000 7face27fc640 1 -- 172.21.15.82:0/2834231628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7face411b730 msgr2=0x7face41abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.050+0000 7face27fc640 1 --2- 172.21.15.82:0/2834231628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7face411b730 0x7face41abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.050+0000 7face27fc640 1 -- 172.21.15.82:0/2834231628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face41ac4f0 con 0x7face41105c0 2024-04-21T08:51:39.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.051+0000 7face27fc640 1 --2- 172.21.15.82:0/2834231628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7face41105c0 0x7face41a7a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facd400b4d0 tx=0x7facd400b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.064+0000 7faccbfff640 1 -- 172.21.15.82:0/2834231628 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4014070 con 0x7face41105c0 2024-04-21T08:51:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.064+0000 7faccbfff640 1 -- 172.21.15.82:0/2834231628 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd4010d20 con 0x7face41105c0 2024-04-21T08:51:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.064+0000 7faccbfff640 1 -- 172.21.15.82:0/2834231628 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4017360 con 0x7face41105c0 2024-04-21T08:51:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.064+0000 7face9559640 1 -- 172.21.15.82:0/2834231628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face41ac7e0 con 0x7face41105c0 2024-04-21T08:51:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.065+0000 7face9559640 1 -- 172.21.15.82:0/2834231628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face4110f10 con 0x7face41105c0 2024-04-21T08:51:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.069+0000 7facec22e740 1 Processor -- start 2024-04-21T08:51:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.071+0000 7facec22e740 1 -- start start 2024-04-21T08:51:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.071+0000 7facec22e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe17e6d0 0x55a8fe17eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.071+0000 7facec22e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17f080 0x55a8fe187630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.071+0000 7facec22e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe187c00 0x55a8fe18a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.071+0000 7facec22e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a8fe0e31f0 con 0x55a8fe187c00 2024-04-21T08:51:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.071+0000 7facec22e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a8fe0804b0 con 0x55a8fe17e6d0 2024-04-21T08:51:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.071+0000 7facec22e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a8fe0df1b0 con 0x55a8fe17f080 2024-04-21T08:51:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.071+0000 7faccbfff640 1 -- 172.21.15.82:0/2834231628 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7facd4017070 con 0x7face41105c0 2024-04-21T08:51:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.072+0000 7faccbfff640 1 --2- 172.21.15.82:0/2834231628 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faccc0a3ea0 0x7faccc0a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.073+0000 7faccbfff640 1 -- 172.21.15.82:0/2834231628 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7facd40d00e0 con 0x7face41105c0 2024-04-21T08:51:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.073+0000 7face2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17f080 0x55a8fe187630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.073+0000 7face2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17f080 0x55a8fe187630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:35330/0 (socket says 172.21.15.82:35330) 2024-04-21T08:51:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.073+0000 7face2ffd640 1 -- 172.21.15.82:0/3815796654 learned_addr learned my addr 172.21.15.82:0/3815796654 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.073+0000 7face37fe640 1 --2- 172.21.15.82:0/3815796654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe17e6d0 0x55a8fe17eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.074+0000 7face2ffd640 1 -- 172.21.15.82:0/3815796654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe17e6d0 msgr2=0x55a8fe17eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.074+0000 7face2ffd640 1 --2- 172.21.15.82:0/3815796654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe17e6d0 0x55a8fe17eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.074+0000 7face2ffd640 1 -- 172.21.15.82:0/3815796654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe187c00 msgr2=0x55a8fe18a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.074+0000 7face2ffd640 1 --2- 172.21.15.82:0/3815796654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe187c00 0x55a8fe18a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.074+0000 7face2ffd640 1 -- 172.21.15.82:0/3815796654 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8fe0721b0 con 0x55a8fe17f080 2024-04-21T08:51:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.074+0000 7face2ffd640 1 --2- 172.21.15.82:0/3815796654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17f080 0x55a8fe187630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7facdc009ca0 tx=0x7facdc0099f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eba209e8080e743 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.075+0000 7face37fe640 1 --2- 172.21.15.82:0/2834231628 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faccc0a3ea0 0x7faccc0a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.080+0000 7facc97fa640 1 -- 172.21.15.82:0/3815796654 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc039040 con 0x55a8fe17f080 2024-04-21T08:51:39.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.080+0000 7facc97fa640 1 -- 172.21.15.82:0/3815796654 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facdc032e70 con 0x55a8fe17f080 2024-04-21T08:51:39.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.081+0000 7facec22e740 1 -- 172.21.15.82:0/3815796654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17f080 msgr2=0x55a8fe187630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.081+0000 7facec22e740 1 --2- 172.21.15.82:0/3815796654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17f080 0x55a8fe187630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7facdc009ca0 tx=0x7facdc0099f0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.083+0000 7facec22e740 1 -- 172.21.15.82:0/3815796654 shutdown_connections 2024-04-21T08:51:39.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.083+0000 7facec22e740 1 --2- 172.21.15.82:0/3815796654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe187c00 0x55a8fe18a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.083+0000 7facec22e740 1 --2- 172.21.15.82:0/3815796654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17f080 0x55a8fe187630 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.083+0000 7facec22e740 1 --2- 172.21.15.82:0/3815796654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe17e6d0 0x55a8fe17eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.083+0000 7facec22e740 1 -- 172.21.15.82:0/3815796654 >> 172.21.15.82:0/3815796654 conn(0x55a8fe178da0 msgr2=0x55a8fe179180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.083+0000 7facec22e740 1 -- 172.21.15.82:0/3815796654 shutdown_connections 2024-04-21T08:51:39.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.084+0000 7facec22e740 1 -- 172.21.15.82:0/3815796654 wait complete. 2024-04-21T08:51:39.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.084+0000 7facec22e740 1 -- start start 2024-04-21T08:51:39.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.084+0000 7facec22e740 1 Processor -- start 2024-04-21T08:51:39.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.084+0000 7facec22e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe17e6d0 0x55a8fe1875e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.085+0000 7facec22e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17f080 0x55a8fe184970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.085+0000 7facec22e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 0x55a8fe1850b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.085+0000 7facec22e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a8fe18ab60 con 0x55a8fe17e6d0 2024-04-21T08:51:39.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.085+0000 7facec22e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a8fe09c500 con 0x55a8fe187c00 2024-04-21T08:51:39.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.085+0000 7facec22e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a8fe18ace0 con 0x55a8fe17f080 2024-04-21T08:51:39.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.085+0000 7facec22e740 10 client.0 _ll_register_callbacks cb 0x55a8fe06ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.085+0000 7face37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 0x55a8fe1850b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.086+0000 7face37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 0x55a8fe1850b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34288/0 (socket says 172.21.15.82:34288) 2024-04-21T08:51:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.086+0000 7face37fe640 1 -- 172.21.15.82:0/1857017548 learned_addr learned my addr 172.21.15.82:0/1857017548 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.086+0000 7face37fe640 1 -- 172.21.15.82:0/1857017548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17f080 msgr2=0x55a8fe184970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:51:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.086+0000 7face37fe640 1 --2- 172.21.15.82:0/1857017548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17f080 0x55a8fe184970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.086+0000 7face37fe640 1 -- 172.21.15.82:0/1857017548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe17e6d0 msgr2=0x55a8fe1875e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.086+0000 7face37fe640 1 --2- 172.21.15.82:0/1857017548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe17e6d0 0x55a8fe1875e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.086+0000 7face37fe640 1 -- 172.21.15.82:0/1857017548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8fddf53f0 con 0x55a8fe187c00 2024-04-21T08:51:39.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.086+0000 7face37fe640 1 --2- 172.21.15.82:0/1857017548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 0x55a8fe1850b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7facd8018e90 tx=0x7facd8016680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dbe2df2aa7d63e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.089+0000 7facc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:51:39.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.089+0000 7facc6ff5640 1 -- 172.21.15.82:0/1857017548 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd801e070 con 0x55a8fe187c00 2024-04-21T08:51:39.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.089+0000 7facec22e740 1 -- 172.21.15.82:0/1857017548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8fe082930 con 0x55a8fe187c00 2024-04-21T08:51:39.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.089+0000 7facc6ff5640 1 -- 172.21.15.82:0/1857017548 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7facd801a820 con 0x55a8fe187c00 2024-04-21T08:51:39.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.089+0000 7facc6ff5640 1 -- 172.21.15.82:0/1857017548 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7facd802b050 con 0x55a8fe187c00 2024-04-21T08:51:39.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.090+0000 7facec22e740 10 client.26282 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:39.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.090+0000 7facec22e740 1 -- 172.21.15.82:0/1857017548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a8fdebf7d0 con 0x55a8fe187c00 2024-04-21T08:51:39.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.097+0000 7facc6ff5640 1 -- 172.21.15.82:0/1857017548 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a8fe082930 con 0x55a8fe187c00 2024-04-21T08:51:39.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.097+0000 7facc6ff5640 1 -- 172.21.15.82:0/1857017548 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7facd8023990 con 0x55a8fe187c00 2024-04-21T08:51:39.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.097+0000 7facec22e740 10 client.26282 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:39.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.097+0000 7facec22e740 10 client.26282 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:51:39.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.097+0000 7facec22e740 20 client.26282 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:39.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.097+0000 7facec22e740 1 --2- 172.21.15.82:0/1857017548 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55a8fe198460 0x55a8fe19a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.097+0000 7facec22e740 4 client.26282 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:39.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.097+0000 7facec22e740 1 -- 172.21.15.82:0/1857017548 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8fe0f0c10 con 0x55a8fe198460 2024-04-21T08:51:39.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.097+0000 7face2ffd640 1 --2- 172.21.15.82:0/1857017548 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55a8fe198460 0x55a8fe19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.100+0000 7facc6ff5640 10 client.26282 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:51:39.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.100+0000 7face2ffd640 1 --2- 172.21.15.82:0/1857017548 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55a8fe198460 0x55a8fe19a920 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=958e8a2b2beb1b6 server_cookie=ec75b19793e9a0fa in_seq=0 out_seq=0 2024-04-21T08:51:39.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.100+0000 7facc6ff5640 1 -- 172.21.15.82:0/1857017548 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8fe0f0c10 con 0x55a8fe198460 2024-04-21T08:51:39.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.100+0000 7facc6ff5640 10 client.26282 handle_command_reply: tid=0 2024-04-21T08:51:39.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.101+0000 7facec22e740 1 client.26282 shutdown 2024-04-21T08:51:39.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.101+0000 7facec22e740 1 -- 172.21.15.82:0/1857017548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 msgr2=0x55a8fe1850b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.101+0000 7facec22e740 1 --2- 172.21.15.82:0/1857017548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 0x55a8fe1850b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7facd8018e90 tx=0x7facd8016680 comp rx=0 tx=0).stop 2024-04-21T08:51:39.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.104+0000 7facec22e740 1 -- 172.21.15.82:0/1857017548 shutdown_connections 2024-04-21T08:51:39.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.104+0000 7facec22e740 1 --2- 172.21.15.82:0/1857017548 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55a8fe198460 0x55a8fe19a920 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.104+0000 7facec22e740 1 --2- 172.21.15.82:0/1857017548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 0x55a8fe1850b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.104+0000 7facec22e740 1 --2- 172.21.15.82:0/1857017548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17f080 0x55a8fe184970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.104+0000 7facec22e740 1 --2- 172.21.15.82:0/1857017548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe17e6d0 0x55a8fe1875e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.104+0000 7facec22e740 1 -- 172.21.15.82:0/1857017548 >> 172.21.15.82:0/1857017548 conn(0x55a8fe178da0 msgr2=0x55a8fe17aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.104+0000 7face37fe640 1 --2- 172.21.15.82:0/2834231628 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faccc0a3ea0 0x7faccc0a6360 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7facd80207f0 tx=0x7facd8020820 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.104+0000 7facc6ff5640 0 client.26282 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:51:39.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.104+0000 7facec22e740 1 -- 172.21.15.82:0/1857017548 shutdown_connections 2024-04-21T08:51:39.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.104+0000 7facec22e740 1 -- 172.21.15.82:0/1857017548 wait complete. 2024-04-21T08:51:39.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.104+0000 7facec22e740 20 client.26282 trim_cache size 0 max 16384 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.154+0000 7facec22e740 1 Processor -- start 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.155+0000 7facec22e740 1 -- start start 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.155+0000 7facec22e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17e6d0 0x55a8fe193a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.155+0000 7facec22e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe17f080 0x55a8fe1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.155+0000 7face2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17e6d0 0x55a8fe193a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.155+0000 7face2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17e6d0 0x55a8fe193a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:35358/0 (socket says 172.21.15.82:35358) 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.156+0000 7face27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe17f080 0x55a8fe1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.156+0000 7face27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe17f080 0x55a8fe1a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50228/0 (socket says 172.21.15.82:50228) 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.156+0000 7face2ffd640 1 -- 172.21.15.82:0/310823886 learned_addr learned my addr 172.21.15.82:0/310823886 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.156+0000 7facec22e740 1 --2- 172.21.15.82:0/310823886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 0x55a8fe1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.157+0000 7facec22e740 1 -- 172.21.15.82:0/310823886 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7face41c8d50 con 0x55a8fe17f080 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.157+0000 7facec22e740 1 -- 172.21.15.82:0/310823886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7face41c6ae0 con 0x55a8fe187c00 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.157+0000 7face37fe640 1 --2- 172.21.15.82:0/310823886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 0x55a8fe1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.157+0000 7facec22e740 1 -- 172.21.15.82:0/310823886 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7face41c8680 con 0x55a8fe17e6d0 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.157+0000 7face37fe640 1 -- 172.21.15.82:0/310823886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17e6d0 msgr2=0x55a8fe193a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.157+0000 7face37fe640 1 --2- 172.21.15.82:0/310823886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17e6d0 0x55a8fe193a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.157+0000 7face37fe640 1 -- 172.21.15.82:0/310823886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe17f080 msgr2=0x55a8fe1a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.157+0000 7face37fe640 1 --2- 172.21.15.82:0/310823886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe17f080 0x55a8fe1a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.157+0000 7face37fe640 1 -- 172.21.15.82:0/310823886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8fe1785f0 con 0x55a8fe187c00 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.160+0000 7face37fe640 1 --2- 172.21.15.82:0/310823886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 0x55a8fe1a0770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7facd801b000 tx=0x7facd802f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2971fe975103a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.160+0000 7facc8ff9640 1 -- 172.21.15.82:0/310823886 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8034070 con 0x55a8fe187c00 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.161+0000 7facc8ff9640 1 -- 172.21.15.82:0/310823886 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd8010040 con 0x55a8fe187c00 2024-04-21T08:51:39.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.162+0000 7facec22e740 1 -- 172.21.15.82:0/310823886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 msgr2=0x55a8fe1a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.162+0000 7facec22e740 1 --2- 172.21.15.82:0/310823886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 0x55a8fe1a0770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7facd801b000 tx=0x7facd802f000 comp rx=0 tx=0).stop 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.163+0000 7facec22e740 1 -- 172.21.15.82:0/310823886 shutdown_connections 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.163+0000 7facec22e740 1 --2- 172.21.15.82:0/310823886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe187c00 0x55a8fe1a0770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.163+0000 7facec22e740 1 --2- 172.21.15.82:0/310823886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe17f080 0x55a8fe1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.163+0000 7facec22e740 1 --2- 172.21.15.82:0/310823886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17e6d0 0x55a8fe193a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.163+0000 7facec22e740 1 -- 172.21.15.82:0/310823886 >> 172.21.15.82:0/310823886 conn(0x55a8fe178da0 msgr2=0x55a8fe177e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.170+0000 7facec22e740 1 -- 172.21.15.82:0/310823886 shutdown_connections 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.170+0000 7facec22e740 1 -- 172.21.15.82:0/310823886 wait complete. 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.170+0000 7facec22e740 1 -- start start 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.171+0000 7facec22e740 1 Processor -- start 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.171+0000 7facec22e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17e6d0 0x55a8fe18af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.171+0000 7facec22e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe17f080 0x55a8fe18b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7facec22e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe187c00 0x55a8fe1fca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7face2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17e6d0 0x55a8fe18af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7face2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17e6d0 0x55a8fe18af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:35366/0 (socket says 172.21.15.82:35366) 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7face27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe17f080 0x55a8fe18b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7face27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe17f080 0x55a8fe18b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34304/0 (socket says 172.21.15.82:34304) 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7face2ffd640 1 -- 172.21.15.82:0/4280601380 learned_addr learned my addr 172.21.15.82:0/4280601380 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7facec22e740 1 -- 172.21.15.82:0/4280601380 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7face41c78e0 con 0x55a8fe187c00 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7facec22e740 1 -- 172.21.15.82:0/4280601380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7face41c71b0 con 0x55a8fe17f080 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7facec22e740 1 -- 172.21.15.82:0/4280601380 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7face41c7fb0 con 0x55a8fe17e6d0 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7facec22e740 10 client.0 _ll_register_callbacks cb 0x7facd8018a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7face37fe640 1 --2- 172.21.15.82:0/4280601380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe187c00 0x55a8fe1fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7face27fc640 1 -- 172.21.15.82:0/4280601380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17e6d0 msgr2=0x55a8fe18af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7face27fc640 1 --2- 172.21.15.82:0/4280601380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17e6d0 0x55a8fe18af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7face27fc640 1 -- 172.21.15.82:0/4280601380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe187c00 msgr2=0x55a8fe1fca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7face27fc640 1 --2- 172.21.15.82:0/4280601380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe187c00 0x55a8fe1fca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.172+0000 7face27fc640 1 -- 172.21.15.82:0/4280601380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8fe1fcfd0 con 0x55a8fe17f080 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.173+0000 7face27fc640 1 --2- 172.21.15.82:0/4280601380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe17f080 0x55a8fe18b480 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7facd40cd580 tx=0x7facd4006e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0da872792579e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.177+0000 7facc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.177+0000 7facc6ff5640 1 -- 172.21.15.82:0/4280601380 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd40b5060 con 0x55a8fe17f080 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.177+0000 7facec22e740 1 -- 172.21.15.82:0/4280601380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd8010040 con 0x55a8fe17f080 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.177+0000 7facc6ff5640 1 -- 172.21.15.82:0/4280601380 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7facd40cd6f0 con 0x55a8fe17f080 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.177+0000 7facc6ff5640 1 -- 172.21.15.82:0/4280601380 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7facd401e040 con 0x55a8fe17f080 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.177+0000 7facec22e740 10 client.26288 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.177+0000 7facec22e740 1 -- 172.21.15.82:0/4280601380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a8fe1fe240 con 0x55a8fe17f080 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.180+0000 7facc6ff5640 1 -- 172.21.15.82:0/4280601380 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd8010040 con 0x55a8fe17f080 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.180+0000 7facc6ff5640 1 -- 172.21.15.82:0/4280601380 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7facd40d2110 con 0x55a8fe17f080 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.182+0000 7facec22e740 10 client.26288 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.182+0000 7facec22e740 10 client.26288 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.182+0000 7facec22e740 20 client.26288 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.182+0000 7facec22e740 1 --2- 172.21.15.82:0/4280601380 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55a8fe1fea90 0x55a8fe200eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.182+0000 7facec22e740 4 client.26288 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.182+0000 7facec22e740 1 -- 172.21.15.82:0/4280601380 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8fe0eaa80 con 0x55a8fe1fea90 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.182+0000 7face2ffd640 1 --2- 172.21.15.82:0/4280601380 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55a8fe1fea90 0x55a8fe200eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.183+0000 7face2ffd640 1 --2- 172.21.15.82:0/4280601380 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55a8fe1fea90 0x55a8fe200eb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=200d4e3292dd1646 server_cookie=ccff9ed8903016d9 in_seq=0 out_seq=0 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.183+0000 7facc6ff5640 10 client.26288 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.183+0000 7facc6ff5640 1 -- 172.21.15.82:0/4280601380 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8fe0eaa80 con 0x55a8fe1fea90 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.183+0000 7facc6ff5640 10 client.26288 handle_command_reply: tid=0 2024-04-21T08:51:39.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.186+0000 7facec22e740 1 client.26288 shutdown 2024-04-21T08:51:39.191 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:51:39.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.191+0000 7facec22e740 1 -- 172.21.15.82:0/4280601380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe17f080 msgr2=0x55a8fe18b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.191+0000 7facec22e740 1 --2- 172.21.15.82:0/4280601380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe17f080 0x55a8fe18b480 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7facd40cd580 tx=0x7facd4006e40 comp rx=0 tx=0).stop 2024-04-21T08:51:39.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.191+0000 7facec22e740 1 -- 172.21.15.82:0/4280601380 shutdown_connections 2024-04-21T08:51:39.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.191+0000 7facec22e740 1 --2- 172.21.15.82:0/4280601380 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55a8fe1fea90 0x55a8fe200eb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.191+0000 7facec22e740 1 --2- 172.21.15.82:0/4280601380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a8fe187c00 0x55a8fe1fca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.191+0000 7facec22e740 1 --2- 172.21.15.82:0/4280601380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a8fe17f080 0x55a8fe18b480 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.191+0000 7facec22e740 1 --2- 172.21.15.82:0/4280601380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a8fe17e6d0 0x55a8fe18af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.191+0000 7facec22e740 1 -- 172.21.15.82:0/4280601380 >> 172.21.15.82:0/4280601380 conn(0x55a8fe178da0 msgr2=0x55a8fe177590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.191+0000 7facc6ff5640 0 client.26288 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:51:39.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.191+0000 7facec22e740 1 -- 172.21.15.82:0/4280601380 shutdown_connections 2024-04-21T08:51:39.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.191+0000 7facec22e740 1 -- 172.21.15.82:0/4280601380 wait complete. 2024-04-21T08:51:39.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.191+0000 7facec22e740 20 client.26288 trim_cache size 0 max 16384 2024-04-21T08:51:39.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.202+0000 7face9559640 1 -- 172.21.15.82:0/2834231628 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faccc0a3ea0 msgr2=0x7faccc0a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.202+0000 7face9559640 1 --2- 172.21.15.82:0/2834231628 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faccc0a3ea0 0x7faccc0a6360 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7facd80207f0 tx=0x7facd8020820 comp rx=0 tx=0).stop 2024-04-21T08:51:39.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.202+0000 7face9559640 1 -- 172.21.15.82:0/2834231628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7face41105c0 msgr2=0x7face41a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.202+0000 7face9559640 1 --2- 172.21.15.82:0/2834231628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7face41105c0 0x7face41a7a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facd400b4d0 tx=0x7facd400b990 comp rx=0 tx=0).stop 2024-04-21T08:51:39.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.203+0000 7face9559640 1 -- 172.21.15.82:0/2834231628 shutdown_connections 2024-04-21T08:51:39.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.203+0000 7face9559640 1 --2- 172.21.15.82:0/2834231628 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faccc0a3ea0 0x7faccc0a6360 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.203+0000 7face9559640 1 --2- 172.21.15.82:0/2834231628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7face411b730 0x7face41abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.203+0000 7face9559640 1 --2- 172.21.15.82:0/2834231628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7face41105c0 0x7face41a7a70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.203+0000 7face9559640 1 --2- 172.21.15.82:0/2834231628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face4108fe0 0x7face41a7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.203+0000 7face9559640 1 -- 172.21.15.82:0/2834231628 >> 172.21.15.82:0/2834231628 conn(0x7face406cf30 msgr2=0x7face4119720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.207+0000 7face9559640 1 -- 172.21.15.82:0/2834231628 shutdown_connections 2024-04-21T08:51:39.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.207+0000 7face9559640 1 -- 172.21.15.82:0/2834231628 wait complete. 2024-04-21T08:51:39.237 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:51:39.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.350+0000 7fbad6d8f640 1 -- 172.21.15.82:0/3322609340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbad01015a0 msgr2=0x7fbad0101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.350+0000 7fbad6d8f640 1 --2- 172.21.15.82:0/3322609340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbad01015a0 0x7fbad0101980 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009900 tx=0x7fbac402bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.350+0000 7fbad6d8f640 1 -- 172.21.15.82:0/3322609340 shutdown_connections 2024-04-21T08:51:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.350+0000 7fbad6d8f640 1 --2- 172.21.15.82:0/3322609340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad0106150 0x7fbad010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.350+0000 7fbad6d8f640 1 --2- 172.21.15.82:0/3322609340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad0101f50 0x7fbad0105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.350+0000 7fbad6d8f640 1 --2- 172.21.15.82:0/3322609340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbad01015a0 0x7fbad0101980 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.350+0000 7fbad6d8f640 1 -- 172.21.15.82:0/3322609340 >> 172.21.15.82:0/3322609340 conn(0x7fbad00769d0 msgr2=0x7fbad0076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.350+0000 7fbad6d8f640 1 -- 172.21.15.82:0/3322609340 shutdown_connections 2024-04-21T08:51:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.350+0000 7fbad6d8f640 1 -- 172.21.15.82:0/3322609340 wait complete. 2024-04-21T08:51:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad6d8f640 1 Processor -- start 2024-04-21T08:51:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad6d8f640 1 -- start start 2024-04-21T08:51:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad6d8f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbad01015a0 0x7fbad0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad6d8f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad0106150 0x7fbad0198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad6d8f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad019d010 0x7fbad019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad6d8f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0112210 con 0x7fbad01015a0 2024-04-21T08:51:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad6d8f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0112090 con 0x7fbad0106150 2024-04-21T08:51:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad6d8f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0112390 con 0x7fbad019d010 2024-04-21T08:51:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad4b2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbad01015a0 0x7fbad0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad4b2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbad01015a0 0x7fbad0198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50266/0 (socket says 172.21.15.82:50266) 2024-04-21T08:51:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad4b2b640 1 -- 172.21.15.82:0/172147287 learned_addr learned my addr 172.21.15.82:0/172147287 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbacffff640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad0106150 0x7fbad0198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad532c640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad019d010 0x7fbad019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbacffff640 1 -- 172.21.15.82:0/172147287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad019d010 msgr2=0x7fbad019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbacffff640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad019d010 0x7fbad019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbacffff640 1 -- 172.21.15.82:0/172147287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbad01015a0 msgr2=0x7fbad0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbacffff640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbad01015a0 0x7fbad0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbacffff640 1 -- 172.21.15.82:0/172147287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad019db00 con 0x7fbad0106150 2024-04-21T08:51:39.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.351+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbad01015a0 0x7fbad0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:39.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.352+0000 7fbacffff640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad0106150 0x7fbad0198c90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e9c0 tx=0x7fbabc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.352+0000 7fbacdffb640 1 -- 172.21.15.82:0/172147287 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc00ccc0 con 0x7fbad0106150 2024-04-21T08:51:39.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.352+0000 7fbad6d8f640 1 -- 172.21.15.82:0/172147287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad019ddf0 con 0x7fbad0106150 2024-04-21T08:51:39.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.352+0000 7fbacdffb640 1 -- 172.21.15.82:0/172147287 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbabc0108e0 con 0x7fbad0106150 2024-04-21T08:51:39.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.352+0000 7fbacdffb640 1 -- 172.21.15.82:0/172147287 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc0187a0 con 0x7fbad0106150 2024-04-21T08:51:39.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.352+0000 7fbad6d8f640 1 -- 172.21.15.82:0/172147287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad010f850 con 0x7fbad0106150 2024-04-21T08:51:39.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.355+0000 7fbacdffb640 1 -- 172.21.15.82:0/172147287 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbabc010430 con 0x7fbad0106150 2024-04-21T08:51:39.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbad9980740 1 Processor -- start 2024-04-21T08:51:39.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbad9980740 1 -- start start 2024-04-21T08:51:39.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbad9980740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063a4d0 0x5653a063a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbad9980740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a0643430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbad9980740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a0643a00 0x5653a0645e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbad9980740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5653a059f1f0 con 0x5653a0643a00 2024-04-21T08:51:39.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbad9980740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5653a053c4b0 con 0x5653a063ae80 2024-04-21T08:51:39.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbad9980740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5653a059b1b0 con 0x5653a063a4d0 2024-04-21T08:51:39.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbacdffb640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbac00a3d20 0x7fbac00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbacffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a0643a00 0x5653a0645e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbad4b2b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a0643430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbacffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a0643a00 0x5653a0645e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50280/0 (socket says 172.21.15.82:50280) 2024-04-21T08:51:39.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbacffff640 1 -- 172.21.15.82:0/4031156352 learned_addr learned my addr 172.21.15.82:0/4031156352 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbad532c640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbac00a3d20 0x7fbac00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.356+0000 7fbad532c640 1 --2- 172.21.15.82:0/4031156352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063a4d0 0x5653a063a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.357+0000 7fbad4b2b640 1 -- 172.21.15.82:0/4031156352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063a4d0 msgr2=0x5653a063a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.357+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/4031156352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063a4d0 0x5653a063a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.357+0000 7fbad4b2b640 1 -- 172.21.15.82:0/4031156352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a0643a00 msgr2=0x5653a0645e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.357+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/4031156352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a0643a00 0x5653a0645e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.357+0000 7fbad4b2b640 1 -- 172.21.15.82:0/4031156352 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653a052e1b0 con 0x5653a063ae80 2024-04-21T08:51:39.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.357+0000 7fbacdffb640 1 -- 172.21.15.82:0/172147287 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fbabc0d0ea0 con 0x7fbad0106150 2024-04-21T08:51:39.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.357+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/4031156352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a0643430 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbac4006010 tx=0x7fbac402c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b974f21ff3b728ac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.357+0000 7fbad532c640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbac00a3d20 0x7fbac00a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbac80098b0 tx=0x7fbac8009d90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.357+0000 7fbabaffd640 1 -- 172.21.15.82:0/4031156352 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4045070 con 0x5653a063ae80 2024-04-21T08:51:39.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.357+0000 7fbabaffd640 1 -- 172.21.15.82:0/4031156352 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbac402c7e0 con 0x5653a063ae80 2024-04-21T08:51:39.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.357+0000 7fbabaffd640 1 -- 172.21.15.82:0/4031156352 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4003c10 con 0x5653a063ae80 2024-04-21T08:51:39.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.358+0000 7fbad9980740 1 -- 172.21.15.82:0/4031156352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 msgr2=0x5653a0643430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.358+0000 7fbad9980740 1 --2- 172.21.15.82:0/4031156352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a0643430 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbac4006010 tx=0x7fbac402c690 comp rx=0 tx=0).stop 2024-04-21T08:51:39.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.358+0000 7fbad9980740 1 -- 172.21.15.82:0/4031156352 shutdown_connections 2024-04-21T08:51:39.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.358+0000 7fbad9980740 1 --2- 172.21.15.82:0/4031156352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a0643a00 0x5653a0645e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.358+0000 7fbad9980740 1 --2- 172.21.15.82:0/4031156352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a0643430 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.358+0000 7fbad9980740 1 --2- 172.21.15.82:0/4031156352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063a4d0 0x5653a063a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.358+0000 7fbad9980740 1 -- 172.21.15.82:0/4031156352 >> 172.21.15.82:0/4031156352 conn(0x5653a0634ba0 msgr2=0x5653a0634f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.358+0000 7fbad9980740 1 -- 172.21.15.82:0/4031156352 shutdown_connections 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.358+0000 7fbad9980740 1 -- 172.21.15.82:0/4031156352 wait complete. 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbad9980740 1 -- start start 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbad9980740 1 Processor -- start 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbad9980740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063a4d0 0x5653a0643310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbad9980740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a06406a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbad9980740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a0643a00 0x5653a0640c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbad9980740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5653a0646910 con 0x5653a0643a00 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbad9980740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5653a0558500 con 0x5653a063ae80 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbad9980740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5653a0646a90 con 0x5653a063a4d0 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbad9980740 10 client.0 _ll_register_callbacks cb 0x5653a0526b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:39.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbacffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a06406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbacffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a06406a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34334/0 (socket says 172.21.15.82:34334) 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbacffff640 1 -- 172.21.15.82:0/4042160207 learned_addr learned my addr 172.21.15.82:0/4042160207 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbad532c640 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a0643a00 0x5653a0640c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.359+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063a4d0 0x5653a0643310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbacffff640 1 -- 172.21.15.82:0/4042160207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063a4d0 msgr2=0x5653a0643310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbacffff640 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063a4d0 0x5653a0643310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbacffff640 1 -- 172.21.15.82:0/4042160207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a0643a00 msgr2=0x5653a0640c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbacffff640 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a0643a00 0x5653a0640c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbacffff640 1 -- 172.21.15.82:0/4042160207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653a02b13f0 con 0x5653a063ae80 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbad532c640 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a0643a00 0x5653a0640c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbacffff640 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a06406a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbabc06f5f0 tx=0x7fbabc06c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=432b6c3422ed2e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbaaffff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:51:39.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbaaffff640 1 -- 172.21.15.82:0/4042160207 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc0a56f0 con 0x5653a063ae80 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbaaffff640 1 -- 172.21.15.82:0/4042160207 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fbabc0840e0 con 0x5653a063ae80 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbad9980740 1 -- 172.21.15.82:0/4042160207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac402c7e0 con 0x5653a063ae80 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbaaffff640 1 -- 172.21.15.82:0/4042160207 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbabc09cb30 con 0x5653a063ae80 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbad9980740 10 client.26300 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.360+0000 7fbad9980740 1 -- 172.21.15.82:0/4042160207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5653a037b7d0 con 0x5653a063ae80 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.361+0000 7fbaaffff640 1 -- 172.21.15.82:0/4042160207 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbac402c7e0 con 0x5653a063ae80 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.361+0000 7fbaaffff640 1 -- 172.21.15.82:0/4042160207 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fbabc06b1a0 con 0x5653a063ae80 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.361+0000 7fbad9980740 10 client.26300 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.361+0000 7fbad9980740 10 client.26300 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.361+0000 7fbad9980740 20 client.26300 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.361+0000 7fbad9980740 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5653a0653bd0 0x5653a0656050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.361+0000 7fbad9980740 4 client.26300 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.361+0000 7fbad9980740 1 -- 172.21.15.82:0/4042160207 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653a053e930 con 0x5653a0653bd0 2024-04-21T08:51:39.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.361+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5653a0653bd0 0x5653a0656050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.361+0000 7fbaaffff640 10 client.26300 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:51:39.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.362+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5653a0653bd0 0x5653a0656050 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f03578cecc64049 server_cookie=9d02a1c53b8d4030 in_seq=0 out_seq=0 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbaaffff640 1 -- 172.21.15.82:0/4042160207 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653a053e930 con 0x5653a0653bd0 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbaaffff640 10 client.26300 handle_command_reply: tid=0 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbad9980740 1 client.26300 shutdown 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbad9980740 1 -- 172.21.15.82:0/4042160207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 msgr2=0x5653a06406a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbad9980740 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a06406a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbabc06f5f0 tx=0x7fbabc06c410 comp rx=0 tx=0).stop 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbad9980740 1 -- 172.21.15.82:0/4042160207 shutdown_connections 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbad9980740 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5653a0653bd0 0x5653a0656050 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbad9980740 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a0643a00 0x5653a0640c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbaaffff640 0 client.26300 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbad9980740 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a06406a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbad9980740 1 --2- 172.21.15.82:0/4042160207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063a4d0 0x5653a0643310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbad9980740 1 -- 172.21.15.82:0/4042160207 >> 172.21.15.82:0/4042160207 conn(0x5653a0634ba0 msgr2=0x5653a06368f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.363+0000 7fbad9980740 1 -- 172.21.15.82:0/4042160207 shutdown_connections 2024-04-21T08:51:39.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.364+0000 7fbad9980740 1 -- 172.21.15.82:0/4042160207 wait complete. 2024-04-21T08:51:39.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.364+0000 7fbad9980740 20 client.26300 trim_cache size 0 max 16384 2024-04-21T08:51:39.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad9980740 1 Processor -- start 2024-04-21T08:51:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad9980740 1 -- start start 2024-04-21T08:51:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad9980740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 0x5653a064f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad9980740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063ae80 0x5653a063f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad9980740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a0643a00 0x5653a063bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad9980740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01b1500 con 0x5653a063a4d0 2024-04-21T08:51:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad9980740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01b30a0 con 0x5653a0643a00 2024-04-21T08:51:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad9980740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01b1bd0 con 0x5653a063ae80 2024-04-21T08:51:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad4b2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 0x5653a064f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad4b2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 0x5653a064f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50296/0 (socket says 172.21.15.82:50296) 2024-04-21T08:51:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad4b2b640 1 -- 172.21.15.82:0/2711238192 learned_addr learned my addr 172.21.15.82:0/2711238192 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad532c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a0643a00 0x5653a063bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbacffff640 1 --2- 172.21.15.82:0/2711238192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063ae80 0x5653a063f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad4b2b640 1 -- 172.21.15.82:0/2711238192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063ae80 msgr2=0x5653a063f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/2711238192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063ae80 0x5653a063f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad4b2b640 1 -- 172.21.15.82:0/2711238192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a0643a00 msgr2=0x5653a063bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/2711238192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a0643a00 0x5653a063bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad4b2b640 1 -- 172.21.15.82:0/2711238192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653a0630470 con 0x5653a063a4d0 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.391+0000 7fbad532c640 1 --2- 172.21.15.82:0/2711238192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a0643a00 0x5653a063bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.392+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/2711238192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 0x5653a064f800 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbac402c320 tx=0x7fbac4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e59238282f0be10 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.392+0000 7fbaba7fc640 1 -- 172.21.15.82:0/2711238192 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4045040 con 0x5653a063a4d0 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.392+0000 7fbaba7fc640 1 -- 172.21.15.82:0/2711238192 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbac402c7e0 con 0x5653a063a4d0 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.392+0000 7fbaba7fc640 1 -- 172.21.15.82:0/2711238192 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac404a450 con 0x5653a063a4d0 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.392+0000 7fbad9980740 1 -- 172.21.15.82:0/2711238192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 msgr2=0x5653a064f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.392+0000 7fbad9980740 1 --2- 172.21.15.82:0/2711238192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 0x5653a064f800 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbac402c320 tx=0x7fbac4005a20 comp rx=0 tx=0).stop 2024-04-21T08:51:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.393+0000 7fbad9980740 1 -- 172.21.15.82:0/2711238192 shutdown_connections 2024-04-21T08:51:39.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.393+0000 7fbad9980740 1 --2- 172.21.15.82:0/2711238192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a0643a00 0x5653a063bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.393+0000 7fbad9980740 1 --2- 172.21.15.82:0/2711238192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a063ae80 0x5653a063f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.393+0000 7fbad9980740 1 --2- 172.21.15.82:0/2711238192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 0x5653a064f800 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.393+0000 7fbad9980740 1 -- 172.21.15.82:0/2711238192 >> 172.21.15.82:0/2711238192 conn(0x5653a0634ba0 msgr2=0x5653a063b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.393+0000 7fbad9980740 1 -- 172.21.15.82:0/2711238192 shutdown_connections 2024-04-21T08:51:39.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.393+0000 7fbad9980740 1 -- 172.21.15.82:0/2711238192 wait complete. 2024-04-21T08:51:39.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.393+0000 7fbad9980740 1 -- start start 2024-04-21T08:51:39.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.393+0000 7fbad9980740 1 Processor -- start 2024-04-21T08:51:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.393+0000 7fbad9980740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 0x5653a0646ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad9980740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a0647400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad9980740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a0643a00 0x5653a06b8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad9980740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01b29d0 con 0x5653a063a4d0 2024-04-21T08:51:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad9980740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01b2300 con 0x5653a063ae80 2024-04-21T08:51:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad9980740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01af4e0 con 0x5653a0643a00 2024-04-21T08:51:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad4b2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 0x5653a0646ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad4b2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 0x5653a0646ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50306/0 (socket says 172.21.15.82:50306) 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbacffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a0647400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad9980740 10 client.0 _ll_register_callbacks cb 0x7fbabc0ce1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad4b2b640 1 -- 172.21.15.82:0/3272666741 learned_addr learned my addr 172.21.15.82:0/3272666741 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad532c640 1 --2- 172.21.15.82:0/3272666741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a0643a00 0x5653a06b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad4b2b640 1 -- 172.21.15.82:0/3272666741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a0643a00 msgr2=0x5653a06b8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/3272666741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a0643a00 0x5653a06b8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad4b2b640 1 -- 172.21.15.82:0/3272666741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 msgr2=0x5653a0647400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/3272666741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a0647400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad4b2b640 1 -- 172.21.15.82:0/3272666741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653a06b86c0 con 0x5653a063a4d0 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad4b2b640 1 --2- 172.21.15.82:0/3272666741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 0x5653a0646ec0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbac40096c0 tx=0x7fbac4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1cb4e3aede5e11 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbaaffff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbaaffff640 1 -- 172.21.15.82:0/3272666741 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4045030 con 0x5653a063a4d0 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbaaffff640 1 -- 172.21.15.82:0/3272666741 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fbac402cb70 con 0x5653a063a4d0 2024-04-21T08:51:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.394+0000 7fbad9980740 1 -- 172.21.15.82:0/3272666741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac402c7e0 con 0x5653a063a4d0 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.395+0000 7fbaaffff640 1 -- 172.21.15.82:0/3272666741 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbac406c050 con 0x5653a063a4d0 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.395+0000 7fbad9980740 10 client.16959 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.395+0000 7fbad9980740 1 -- 172.21.15.82:0/3272666741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5653a06b9880 con 0x5653a063a4d0 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.395+0000 7fbaaffff640 1 -- 172.21.15.82:0/3272666741 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbac402c7e0 con 0x5653a063a4d0 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.395+0000 7fbaaffff640 1 -- 172.21.15.82:0/3272666741 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fbac404a450 con 0x5653a063a4d0 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.395+0000 7fbad9980740 10 client.16959 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.395+0000 7fbad9980740 10 client.16959 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.395+0000 7fbad9980740 20 client.16959 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.395+0000 7fbad9980740 1 --2- 172.21.15.82:0/3272666741 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5653a06b9bc0 0x5653a06bbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.395+0000 7fbad9980740 4 client.16959 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.395+0000 7fbad9980740 1 -- 172.21.15.82:0/3272666741 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653a05acc10 con 0x5653a06b9bc0 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.395+0000 7fbacffff640 1 --2- 172.21.15.82:0/3272666741 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5653a06b9bc0 0x5653a06bbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.396+0000 7fbaaffff640 10 client.16959 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.396+0000 7fbacffff640 1 --2- 172.21.15.82:0/3272666741 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5653a06b9bc0 0x5653a06bbfb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=161e061241c99130 server_cookie=19407680c9469fd8 in_seq=0 out_seq=0 2024-04-21T08:51:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.397+0000 7fbaaffff640 1 -- 172.21.15.82:0/3272666741 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653a05acc10 con 0x5653a06b9bc0 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.397+0000 7fbaaffff640 10 client.16959 handle_command_reply: tid=0 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.397+0000 7fbad9980740 1 client.16959 shutdown 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.398+0000 7fbad9980740 1 -- 172.21.15.82:0/3272666741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 msgr2=0x5653a0646ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.398+0000 7fbad9980740 1 --2- 172.21.15.82:0/3272666741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 0x5653a0646ec0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbac40096c0 tx=0x7fbac4000bc0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.398+0000 7fbad9980740 1 -- 172.21.15.82:0/3272666741 shutdown_connections 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.398+0000 7fbad9980740 1 --2- 172.21.15.82:0/3272666741 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5653a06b9bc0 0x5653a06bbfb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.398+0000 7fbad9980740 1 --2- 172.21.15.82:0/3272666741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5653a0643a00 0x5653a06b8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.398+0000 7fbaaffff640 0 client.16959 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.398+0000 7fbad9980740 1 --2- 172.21.15.82:0/3272666741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653a063ae80 0x5653a0647400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.398+0000 7fbad9980740 1 --2- 172.21.15.82:0/3272666741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653a063a4d0 0x5653a0646ec0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.398+0000 7fbad9980740 1 -- 172.21.15.82:0/3272666741 >> 172.21.15.82:0/3272666741 conn(0x5653a0634ba0 msgr2=0x5653a063bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.398+0000 7fbad9980740 1 -- 172.21.15.82:0/3272666741 shutdown_connections 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.398+0000 7fbad9980740 1 -- 172.21.15.82:0/3272666741 wait complete. 2024-04-21T08:51:39.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.398+0000 7fbad9980740 20 client.16959 trim_cache size 0 max 16384 2024-04-21T08:51:39.402 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:51:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.399+0000 7fbad6d8f640 1 -- 172.21.15.82:0/172147287 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbac00a3d20 msgr2=0x7fbac00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.399+0000 7fbad6d8f640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbac00a3d20 0x7fbac00a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbac80098b0 tx=0x7fbac8009d90 comp rx=0 tx=0).stop 2024-04-21T08:51:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.399+0000 7fbad6d8f640 1 -- 172.21.15.82:0/172147287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad0106150 msgr2=0x7fbad0198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.399+0000 7fbad6d8f640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad0106150 0x7fbad0198c90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e9c0 tx=0x7fbabc00ee80 comp rx=0 tx=0).stop 2024-04-21T08:51:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.399+0000 7fbad6d8f640 1 -- 172.21.15.82:0/172147287 shutdown_connections 2024-04-21T08:51:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.399+0000 7fbad6d8f640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad019d010 0x7fbad019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.399+0000 7fbad6d8f640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbad0106150 0x7fbad0198c90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.399+0000 7fbad6d8f640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbac00a3d20 0x7fbac00a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.399+0000 7fbad6d8f640 1 --2- 172.21.15.82:0/172147287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbad01015a0 0x7fbad0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.399+0000 7fbad6d8f640 1 -- 172.21.15.82:0/172147287 >> 172.21.15.82:0/172147287 conn(0x7fbad00769d0 msgr2=0x7fbad00ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.399+0000 7fbad6d8f640 1 -- 172.21.15.82:0/172147287 shutdown_connections 2024-04-21T08:51:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.399+0000 7fbad6d8f640 1 -- 172.21.15.82:0/172147287 wait complete. 2024-04-21T08:51:39.411 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:51:39.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.515+0000 7f6843959640 1 -- 172.21.15.82:0/1542498039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c1011a0 msgr2=0x7f683c101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.515+0000 7f6843959640 1 --2- 172.21.15.82:0/1542498039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c1011a0 0x7f683c101600 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f682c009930 tx=0x7f682c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:39.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.515+0000 7f6843959640 1 -- 172.21.15.82:0/1542498039 shutdown_connections 2024-04-21T08:51:39.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.515+0000 7f6843959640 1 --2- 172.21.15.82:0/1542498039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f683c101b40 0x7f683c10dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.515+0000 7f6843959640 1 --2- 172.21.15.82:0/1542498039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c1011a0 0x7f683c101600 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.515+0000 7f6843959640 1 --2- 172.21.15.82:0/1542498039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f683c107030 0x7f683c107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.515+0000 7f6843959640 1 -- 172.21.15.82:0/1542498039 >> 172.21.15.82:0/1542498039 conn(0x7f683c076c50 msgr2=0x7f683c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.515+0000 7f6843959640 1 -- 172.21.15.82:0/1542498039 shutdown_connections 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.515+0000 7f6843959640 1 -- 172.21.15.82:0/1542498039 wait complete. 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f6843959640 1 Processor -- start 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f6843959640 1 -- start start 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f6843959640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f683c1011a0 0x7f683c19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f6843959640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f683c101b40 0x7f683c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f6843959640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c107030 0x7f683c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f6843959640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1110e0 con 0x7f683c107030 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f6843959640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f683c110f60 con 0x7f683c1011a0 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f6843959640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f683c111260 con 0x7f683c101b40 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f6841ef6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c107030 0x7f683c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f68416f5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f683c1011a0 0x7f683c19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f6841ef6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c107030 0x7f683c198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50334/0 (socket says 172.21.15.82:50334) 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f6841ef6640 1 -- 172.21.15.82:0/355037940 learned_addr learned my addr 172.21.15.82:0/355037940 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.516+0000 7f6840ef4640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f683c101b40 0x7f683c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.517+0000 7f6841ef6640 1 -- 172.21.15.82:0/355037940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f683c101b40 msgr2=0x7f683c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.517+0000 7f6841ef6640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f683c101b40 0x7f683c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.517+0000 7f6841ef6640 1 -- 172.21.15.82:0/355037940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f683c1011a0 msgr2=0x7f683c19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.517+0000 7f6841ef6640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f683c1011a0 0x7f683c19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.517+0000 7f6841ef6640 1 -- 172.21.15.82:0/355037940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c198d90 con 0x7f683c107030 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.517+0000 7f6841ef6640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c107030 0x7f683c198610 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f683400eb10 tx=0x7f683400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.517+0000 7f68327fc640 1 -- 172.21.15.82:0/355037940 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6834004b20 con 0x7f683c107030 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.517+0000 7f68327fc640 1 -- 172.21.15.82:0/355037940 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6834004cc0 con 0x7f683c107030 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.517+0000 7f6843959640 1 -- 172.21.15.82:0/355037940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683c199080 con 0x7f683c107030 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.517+0000 7f68327fc640 1 -- 172.21.15.82:0/355037940 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6834010670 con 0x7f683c107030 2024-04-21T08:51:39.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.517+0000 7f6843959640 1 -- 172.21.15.82:0/355037940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683c1a52d0 con 0x7f683c107030 2024-04-21T08:51:39.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.519+0000 7f68327fc640 1 -- 172.21.15.82:0/355037940 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6834005020 con 0x7f683c107030 2024-04-21T08:51:39.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f684662e740 1 Processor -- start 2024-04-21T08:51:39.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f684662e740 1 -- start start 2024-04-21T08:51:39.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f684662e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 0x55a789a8e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f684662e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 0x55a789a97430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f684662e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a97a00 0x55a789a99e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f684662e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a7899f31f0 con 0x55a789a8ee80 2024-04-21T08:51:39.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f684662e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a7899904b0 con 0x55a789a97a00 2024-04-21T08:51:39.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f684662e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a7899ef1b0 con 0x55a789a8e4d0 2024-04-21T08:51:39.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f68327fc640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68280a3da0 0x7f68280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f68416f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 0x55a789a97430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f68416f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 0x55a789a97430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50350/0 (socket says 172.21.15.82:50350) 2024-04-21T08:51:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f68416f5640 1 -- 172.21.15.82:0/4093676653 learned_addr learned my addr 172.21.15.82:0/4093676653 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f6840ef4640 1 --2- 172.21.15.82:0/4093676653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a97a00 0x55a789a99e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f6840ef4640 1 --2- 172.21.15.82:0/4093676653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 0x55a789a8e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f6841ef6640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68280a3da0 0x7f68280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f68327fc640 1 -- 172.21.15.82:0/355037940 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f68340d1100 con 0x7f683c107030 2024-04-21T08:51:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f68416f5640 1 -- 172.21.15.82:0/4093676653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 msgr2=0x55a789a8e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f68416f5640 1 --2- 172.21.15.82:0/4093676653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 0x55a789a8e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f68416f5640 1 -- 172.21.15.82:0/4093676653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a97a00 msgr2=0x55a789a99e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f68416f5640 1 --2- 172.21.15.82:0/4093676653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a97a00 0x55a789a99e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.520+0000 7f68416f5640 1 -- 172.21.15.82:0/4093676653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7899821b0 con 0x55a789a8ee80 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.521+0000 7f68416f5640 1 --2- 172.21.15.82:0/4093676653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 0x55a789a97430 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6838009d20 tx=0x7f6838011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e989c0d902ab90 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.521+0000 7f6841ef6640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68280a3da0 0x7f68280a6260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f68340cea90 tx=0x7f683406f7d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.521+0000 7f68277fe640 1 -- 172.21.15.82:0/4093676653 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683801e070 con 0x55a789a8ee80 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.521+0000 7f68277fe640 1 -- 172.21.15.82:0/4093676653 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6838009e10 con 0x55a789a8ee80 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.521+0000 7f68277fe640 1 -- 172.21.15.82:0/4093676653 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68380187d0 con 0x55a789a8ee80 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.521+0000 7f684662e740 1 -- 172.21.15.82:0/4093676653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 msgr2=0x55a789a97430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.521+0000 7f684662e740 1 --2- 172.21.15.82:0/4093676653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 0x55a789a97430 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6838009d20 tx=0x7f6838011810 comp rx=0 tx=0).stop 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.521+0000 7f684662e740 1 -- 172.21.15.82:0/4093676653 shutdown_connections 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.521+0000 7f684662e740 1 --2- 172.21.15.82:0/4093676653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a97a00 0x55a789a99e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.521+0000 7f684662e740 1 --2- 172.21.15.82:0/4093676653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 0x55a789a97430 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.521+0000 7f684662e740 1 --2- 172.21.15.82:0/4093676653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 0x55a789a8e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f684662e740 1 -- 172.21.15.82:0/4093676653 >> 172.21.15.82:0/4093676653 conn(0x55a789a88ba0 msgr2=0x55a789a88f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f684662e740 1 -- 172.21.15.82:0/4093676653 shutdown_connections 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f684662e740 1 -- 172.21.15.82:0/4093676653 wait complete. 2024-04-21T08:51:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f684662e740 1 -- start start 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f684662e740 1 Processor -- start 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f684662e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a8e4d0 0x55a789a97380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f684662e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8ee80 0x55a789a94710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f684662e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a97a00 0x55a789a94d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f684662e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a789a9a970 con 0x55a789a97a00 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f684662e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a7899ac500 con 0x55a789a8e4d0 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f684662e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a789a9aaf0 con 0x55a789a8ee80 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f684662e740 10 client.0 _ll_register_callbacks cb 0x55a78997ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.522+0000 7f6840ef4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a97a00 0x55a789a94d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6840ef4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a8e4d0 0x55a789a97380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6840ef4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a8e4d0 0x55a789a97380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34382/0 (socket says 172.21.15.82:34382) 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6840ef4640 1 -- 172.21.15.82:0/2283017741 learned_addr learned my addr 172.21.15.82:0/2283017741 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f68416f5640 1 --2- 172.21.15.82:0/2283017741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8ee80 0x55a789a94710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6840ef4640 1 -- 172.21.15.82:0/2283017741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8ee80 msgr2=0x55a789a94710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6840ef4640 1 --2- 172.21.15.82:0/2283017741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8ee80 0x55a789a94710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6840ef4640 1 -- 172.21.15.82:0/2283017741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a97a00 msgr2=0x55a789a94d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6840ef4640 1 --2- 172.21.15.82:0/2283017741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a97a00 0x55a789a94d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6840ef4640 1 -- 172.21.15.82:0/2283017741 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7897053f0 con 0x55a789a8e4d0 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6840ef4640 1 --2- 172.21.15.82:0/2283017741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a8e4d0 0x55a789a97380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f682c009900 tx=0x7f682c005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed67d5a6c25bbb48 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6824ff9640 1 -- 172.21.15.82:0/2283017741 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c005480 con 0x55a789a8e4d0 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f684662e740 1 -- 172.21.15.82:0/2283017741 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6838009e10 con 0x55a789a8e4d0 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6824ff9640 1 -- 172.21.15.82:0/2283017741 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f682c0046c0 con 0x55a789a8e4d0 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6824ff9640 1 -- 172.21.15.82:0/2283017741 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f682c0048c0 con 0x55a789a8e4d0 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f684662e740 10 client.26318 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f684662e740 1 -- 172.21.15.82:0/2283017741 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a7897cf7d0 con 0x55a789a8e4d0 2024-04-21T08:51:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.523+0000 7f6824ff9640 1 -- 172.21.15.82:0/2283017741 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6838009e10 con 0x55a789a8e4d0 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.524+0000 7f6824ff9640 1 -- 172.21.15.82:0/2283017741 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f682c0039d0 con 0x55a789a8e4d0 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.524+0000 7f684662e740 10 client.26318 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.524+0000 7f684662e740 10 client.26318 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.524+0000 7f684662e740 20 client.26318 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.524+0000 7f684662e740 1 --2- 172.21.15.82:0/2283017741 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a789aa7c60 0x55a789aaa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.524+0000 7f684662e740 4 client.26318 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.524+0000 7f684662e740 1 -- 172.21.15.82:0/2283017741 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a789992930 con 0x55a789aa7c60 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.524+0000 7f68416f5640 1 --2- 172.21.15.82:0/2283017741 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a789aa7c60 0x55a789aaa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.524+0000 7f6824ff9640 10 client.26318 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.524+0000 7f68416f5640 1 --2- 172.21.15.82:0/2283017741 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a789aa7c60 0x55a789aaa050 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e514578120ac0b67 server_cookie=d9f55094092876d7 in_seq=0 out_seq=0 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f6824ff9640 1 -- 172.21.15.82:0/2283017741 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a789992930 con 0x55a789aa7c60 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f6824ff9640 10 client.26318 handle_command_reply: tid=0 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f684662e740 1 client.26318 shutdown 2024-04-21T08:51:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f684662e740 1 -- 172.21.15.82:0/2283017741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a8e4d0 msgr2=0x55a789a97380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f684662e740 1 --2- 172.21.15.82:0/2283017741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a8e4d0 0x55a789a97380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f682c009900 tx=0x7f682c005d90 comp rx=0 tx=0).stop 2024-04-21T08:51:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f684662e740 1 -- 172.21.15.82:0/2283017741 shutdown_connections 2024-04-21T08:51:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f684662e740 1 --2- 172.21.15.82:0/2283017741 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a789aa7c60 0x55a789aaa050 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f684662e740 1 --2- 172.21.15.82:0/2283017741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a97a00 0x55a789a94d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f684662e740 1 --2- 172.21.15.82:0/2283017741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8ee80 0x55a789a94710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f684662e740 1 --2- 172.21.15.82:0/2283017741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a8e4d0 0x55a789a97380 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f6824ff9640 0 client.26318 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:51:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f684662e740 1 -- 172.21.15.82:0/2283017741 >> 172.21.15.82:0/2283017741 conn(0x55a789a88ba0 msgr2=0x55a789a8a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f684662e740 1 -- 172.21.15.82:0/2283017741 shutdown_connections 2024-04-21T08:51:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f684662e740 1 -- 172.21.15.82:0/2283017741 wait complete. 2024-04-21T08:51:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.525+0000 7f684662e740 20 client.26318 trim_cache size 0 max 16384 2024-04-21T08:51:39.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.538+0000 7f684662e740 1 Processor -- start 2024-04-21T08:51:39.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f684662e740 1 -- start start 2024-04-21T08:51:39.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f684662e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 0x55a789aa3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f684662e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 0x55a789a96e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f684662e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a97a00 0x55a789a97380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f684662e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1af3c0 con 0x55a789a8ee80 2024-04-21T08:51:39.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f684662e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1b0f60 con 0x55a789a97a00 2024-04-21T08:51:39.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f684662e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1afa90 con 0x55a789a8e4d0 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f68416f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 0x55a789a96e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f68416f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 0x55a789a96e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50364/0 (socket says 172.21.15.82:50364) 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f6840ef4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a97a00 0x55a789a97380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f68416f5640 1 -- 172.21.15.82:0/1070447493 learned_addr learned my addr 172.21.15.82:0/1070447493 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f6840ef4640 1 --2- 172.21.15.82:0/1070447493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 0x55a789aa3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f68416f5640 1 -- 172.21.15.82:0/1070447493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 msgr2=0x55a789aa3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f68416f5640 1 --2- 172.21.15.82:0/1070447493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 0x55a789aa3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f68416f5640 1 -- 172.21.15.82:0/1070447493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a97a00 msgr2=0x55a789a97380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f68416f5640 1 --2- 172.21.15.82:0/1070447493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a97a00 0x55a789a97380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f68416f5640 1 -- 172.21.15.82:0/1070447493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a789a95090 con 0x55a789a8ee80 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f6840ef4640 1 --2- 172.21.15.82:0/1070447493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a97a00 0x55a789a97380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f68416f5640 1 --2- 172.21.15.82:0/1070447493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 0x55a789a96e40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f683801c040 tx=0x7f68380183b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6644e094ced69992 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f6826ffd640 1 -- 172.21.15.82:0/1070447493 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683801e040 con 0x55a789a8ee80 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f6826ffd640 1 -- 172.21.15.82:0/1070447493 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6838009780 con 0x55a789a8ee80 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.539+0000 7f6826ffd640 1 -- 172.21.15.82:0/1070447493 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68380035f0 con 0x55a789a8ee80 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.540+0000 7f684662e740 1 -- 172.21.15.82:0/1070447493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 msgr2=0x55a789a96e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.540+0000 7f684662e740 1 --2- 172.21.15.82:0/1070447493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 0x55a789a96e40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f683801c040 tx=0x7f68380183b0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.540+0000 7f684662e740 1 -- 172.21.15.82:0/1070447493 shutdown_connections 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.540+0000 7f684662e740 1 --2- 172.21.15.82:0/1070447493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a97a00 0x55a789a97380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.540+0000 7f684662e740 1 --2- 172.21.15.82:0/1070447493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a8ee80 0x55a789a96e40 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.540+0000 7f684662e740 1 --2- 172.21.15.82:0/1070447493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 0x55a789aa3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.540+0000 7f684662e740 1 -- 172.21.15.82:0/1070447493 >> 172.21.15.82:0/1070447493 conn(0x55a789a88ba0 msgr2=0x55a789a87c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.540+0000 7f684662e740 1 -- 172.21.15.82:0/1070447493 shutdown_connections 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.540+0000 7f684662e740 1 -- 172.21.15.82:0/1070447493 wait complete. 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.540+0000 7f684662e740 1 -- start start 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f684662e740 1 Processor -- start 2024-04-21T08:51:39.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f684662e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 0x55a789aa0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f684662e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a8ee80 0x55a789a9b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f684662e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a97a00 0x55a789a9b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f684662e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1b0890 con 0x55a789a97a00 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f684662e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1b01c0 con 0x55a789a8ee80 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f684662e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1ad3a0 con 0x55a789a8e4d0 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f684662e740 10 client.0 _ll_register_callbacks cb 0x7f682c02cc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f6840ef4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a97a00 0x55a789a9b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f6840ef4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a97a00 0x55a789a9b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50366/0 (socket says 172.21.15.82:50366) 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f6840ef4640 1 -- 172.21.15.82:0/612198955 learned_addr learned my addr 172.21.15.82:0/612198955 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f68416f5640 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a8ee80 0x55a789a9b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f6840ef4640 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 0x55a789aa0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f6840ef4640 1 -- 172.21.15.82:0/612198955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 msgr2=0x55a789aa0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f6840ef4640 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 0x55a789aa0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f6840ef4640 1 -- 172.21.15.82:0/612198955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a8ee80 msgr2=0x55a789a9b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f6840ef4640 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a8ee80 0x55a789a9b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.541+0000 7f6840ef4640 1 -- 172.21.15.82:0/612198955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a789a9c0f0 con 0x55a789a97a00 2024-04-21T08:51:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f6840ef4640 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a97a00 0x55a789a9b9b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f682c02c170 tx=0x7f682c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b20430739a4dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f6824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:51:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f6824ff9640 1 -- 172.21.15.82:0/612198955 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c004ea0 con 0x55a789a97a00 2024-04-21T08:51:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f684662e740 1 -- 172.21.15.82:0/612198955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6838009780 con 0x55a789a97a00 2024-04-21T08:51:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f6824ff9640 1 -- 172.21.15.82:0/612198955 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f682c0035f0 con 0x55a789a97a00 2024-04-21T08:51:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f6824ff9640 1 -- 172.21.15.82:0/612198955 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f682c0037f0 con 0x55a789a97a00 2024-04-21T08:51:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f684662e740 10 client.16977 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f684662e740 1 -- 172.21.15.82:0/612198955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a789b0e080 con 0x55a789a97a00 2024-04-21T08:51:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f6824ff9640 1 -- 172.21.15.82:0/612198955 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6838009780 con 0x55a789a97a00 2024-04-21T08:51:39.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f6824ff9640 1 -- 172.21.15.82:0/612198955 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f682c043ac0 con 0x55a789a97a00 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f684662e740 10 client.16977 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f684662e740 10 client.16977 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f684662e740 20 client.16977 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f684662e740 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a789b0e740 0x55a789b10b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f684662e740 4 client.16977 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.542+0000 7f684662e740 1 -- 172.21.15.82:0/612198955 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a789a00c10 con 0x55a789b0e740 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f68416f5640 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a789b0e740 0x55a789b10b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f6824ff9640 10 client.16977 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f68416f5640 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a789b0e740 0x55a789b10b60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cabc6ae7052e7c38 server_cookie=7527dd2dc3b639b7 in_seq=0 out_seq=0 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f6824ff9640 1 -- 172.21.15.82:0/612198955 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a789a00c10 con 0x55a789b0e740 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f6824ff9640 10 client.16977 handle_command_reply: tid=0 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f684662e740 1 client.16977 shutdown 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f684662e740 1 -- 172.21.15.82:0/612198955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a97a00 msgr2=0x55a789a9b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f684662e740 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a97a00 0x55a789a9b9b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f682c02c170 tx=0x7f682c005d90 comp rx=0 tx=0).stop 2024-04-21T08:51:39.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f684662e740 1 -- 172.21.15.82:0/612198955 shutdown_connections 2024-04-21T08:51:39.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f684662e740 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a789b0e740 0x55a789b10b60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f684662e740 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a789a97a00 0x55a789a9b9b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f6824ff9640 0 client.16977 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:51:39.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f684662e740 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a789a8ee80 0x55a789a9b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f684662e740 1 --2- 172.21.15.82:0/612198955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a789a8e4d0 0x55a789aa0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.543+0000 7f684662e740 1 -- 172.21.15.82:0/612198955 >> 172.21.15.82:0/612198955 conn(0x55a789a88ba0 msgr2=0x55a789a8f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f684662e740 1 -- 172.21.15.82:0/612198955 shutdown_connections 2024-04-21T08:51:39.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f684662e740 1 -- 172.21.15.82:0/612198955 wait complete. 2024-04-21T08:51:39.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f684662e740 20 client.16977 trim_cache size 0 max 16384 2024-04-21T08:51:39.552 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:51:39.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f6843959640 1 -- 172.21.15.82:0/355037940 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68280a3da0 msgr2=0x7f68280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f6843959640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68280a3da0 0x7f68280a6260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f68340cea90 tx=0x7f683406f7d0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f6843959640 1 -- 172.21.15.82:0/355037940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c107030 msgr2=0x7f683c198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f6843959640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c107030 0x7f683c198610 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f683400eb10 tx=0x7f683400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f6843959640 1 -- 172.21.15.82:0/355037940 shutdown_connections 2024-04-21T08:51:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f6843959640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f68280a3da0 0x7f68280a6260 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f6843959640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c107030 0x7f683c198610 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f6843959640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f683c101b40 0x7f683c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f6843959640 1 --2- 172.21.15.82:0/355037940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f683c1011a0 0x7f683c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f6843959640 1 -- 172.21.15.82:0/355037940 >> 172.21.15.82:0/355037940 conn(0x7f683c076c50 msgr2=0x7f683c06a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f6843959640 1 -- 172.21.15.82:0/355037940 shutdown_connections 2024-04-21T08:51:39.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.544+0000 7f6843959640 1 -- 172.21.15.82:0/355037940 wait complete. 2024-04-21T08:51:39.555 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:51:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.662+0000 7f29b6a25640 1 -- 172.21.15.82:0/4135079817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b01072e0 msgr2=0x7f29b010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.662+0000 7f29b6a25640 1 --2- 172.21.15.82:0/4135079817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b01072e0 0x7f29b010ee60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009ca0 tx=0x7f29a802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.662+0000 7f29b6a25640 1 -- 172.21.15.82:0/4135079817 shutdown_connections 2024-04-21T08:51:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.662+0000 7f29b6a25640 1 --2- 172.21.15.82:0/4135079817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b01072e0 0x7f29b010ee60 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.662+0000 7f29b6a25640 1 --2- 172.21.15.82:0/4135079817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b0101080 0x7f29b0106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.662+0000 7f29b6a25640 1 --2- 172.21.15.82:0/4135079817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b01006d0 0x7f29b0100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.662+0000 7f29b6a25640 1 -- 172.21.15.82:0/4135079817 >> 172.21.15.82:0/4135079817 conn(0x7f29b00773f0 msgr2=0x7f29b00fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.662+0000 7f29b6a25640 1 -- 172.21.15.82:0/4135079817 shutdown_connections 2024-04-21T08:51:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.662+0000 7f29b6a25640 1 -- 172.21.15.82:0/4135079817 wait complete. 2024-04-21T08:51:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.663+0000 7f29b6a25640 1 Processor -- start 2024-04-21T08:51:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.663+0000 7f29b6a25640 1 -- start start 2024-04-21T08:51:39.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.663+0000 7f29b6a25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b01006d0 0x7f29b0101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.663+0000 7f29b6a25640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b01072e0 0x7f29b0101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.663+0000 7f29b6a25640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b01020c0 0x7f29b01a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.663+0000 7f29b6a25640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01121e0 con 0x7f29b01020c0 2024-04-21T08:51:39.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.663+0000 7f29b6a25640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29b0112060 con 0x7f29b01006d0 2024-04-21T08:51:39.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.663+0000 7f29b6a25640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29b0112360 con 0x7f29b01072e0 2024-04-21T08:51:39.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.663+0000 7f29b4fc2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b01020c0 0x7f29b01a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.663+0000 7f29b4fc2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b01020c0 0x7f29b01a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50378/0 (socket says 172.21.15.82:50378) 2024-04-21T08:51:39.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.663+0000 7f29affff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b01006d0 0x7f29b0101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.663+0000 7f29b4fc2640 1 -- 172.21.15.82:0/2345877215 learned_addr learned my addr 172.21.15.82:0/2345877215 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29af7fe640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b01072e0 0x7f29b0101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29b4fc2640 1 -- 172.21.15.82:0/2345877215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b01072e0 msgr2=0x7f29b0101b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29b4fc2640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b01072e0 0x7f29b0101b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29b4fc2640 1 -- 172.21.15.82:0/2345877215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b01006d0 msgr2=0x7f29b0101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29b4fc2640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b01006d0 0x7f29b0101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29b4fc2640 1 -- 172.21.15.82:0/2345877215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b01a7010 con 0x7f29b01020c0 2024-04-21T08:51:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29affff640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b01006d0 0x7f29b0101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29b4fc2640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b01020c0 0x7f29b01a6910 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f29a8005e30 tx=0x7f29a80346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29ad7fa640 1 -- 172.21.15.82:0/2345877215 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8041020 con 0x7f29b01020c0 2024-04-21T08:51:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29ad7fa640 1 -- 172.21.15.82:0/2345877215 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a8032970 con 0x7f29b01020c0 2024-04-21T08:51:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29b6a25640 1 -- 172.21.15.82:0/2345877215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b01a72a0 con 0x7f29b01020c0 2024-04-21T08:51:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29ad7fa640 1 -- 172.21.15.82:0/2345877215 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8039040 con 0x7f29b01020c0 2024-04-21T08:51:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.664+0000 7f29b6a25640 1 -- 172.21.15.82:0/2345877215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b01a7790 con 0x7f29b01020c0 2024-04-21T08:51:39.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.666+0000 7f29ad7fa640 1 -- 172.21.15.82:0/2345877215 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f29a8044020 con 0x7f29b01020c0 2024-04-21T08:51:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29b9613740 1 Processor -- start 2024-04-21T08:51:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29b9613740 1 -- start start 2024-04-21T08:51:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29b9613740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975d4d0 0x55b05975d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29b9613740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975de80 0x55b059766430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29b9613740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b059766a00 0x55b059768e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29b9613740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b0596c21f0 con 0x55b05975de80 2024-04-21T08:51:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29b9613740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b05965f4b0 con 0x55b059766a00 2024-04-21T08:51:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29b9613740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b0596be1b0 con 0x55b05975d4d0 2024-04-21T08:51:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975de80 0x55b059766430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975de80 0x55b059766430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50380/0 (socket says 172.21.15.82:50380) 2024-04-21T08:51:39.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29affff640 1 -- 172.21.15.82:0/2944248795 learned_addr learned my addr 172.21.15.82:0/2944248795 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29ad7fa640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29980a3d50 0x7f29980a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29af7fe640 1 --2- 172.21.15.82:0/2944248795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b059766a00 0x55b059768e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29af7fe640 1 --2- 172.21.15.82:0/2944248795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975d4d0 0x55b05975d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29b4fc2640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29980a3d50 0x7f29980a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29affff640 1 -- 172.21.15.82:0/2944248795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975d4d0 msgr2=0x55b05975d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29affff640 1 --2- 172.21.15.82:0/2944248795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975d4d0 0x55b05975d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29affff640 1 -- 172.21.15.82:0/2944248795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b059766a00 msgr2=0x55b059768e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29affff640 1 --2- 172.21.15.82:0/2944248795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b059766a00 0x55b059768e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29affff640 1 -- 172.21.15.82:0/2944248795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0596511b0 con 0x55b05975de80 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29ad7fa640 1 -- 172.21.15.82:0/2345877215 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f29a80f2640 con 0x7f29b01020c0 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.667+0000 7f29affff640 1 --2- 172.21.15.82:0/2944248795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975de80 0x55b059766430 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f29a00115f0 tx=0x7f29a0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14cea111c051ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29967fc640 1 -- 172.21.15.82:0/2944248795 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a001e070 con 0x55b05975de80 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29b4fc2640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29980a3d50 0x7f29980a6210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f29a80be030 tx=0x7f29a802e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29967fc640 1 -- 172.21.15.82:0/2944248795 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a000fa70 con 0x55b05975de80 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29967fc640 1 -- 172.21.15.82:0/2944248795 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0019a40 con 0x55b05975de80 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29b9613740 1 -- 172.21.15.82:0/2944248795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975de80 msgr2=0x55b059766430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29b9613740 1 --2- 172.21.15.82:0/2944248795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975de80 0x55b059766430 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f29a00115f0 tx=0x7f29a0011ab0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29b9613740 1 -- 172.21.15.82:0/2944248795 shutdown_connections 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29b9613740 1 --2- 172.21.15.82:0/2944248795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b059766a00 0x55b059768e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29b9613740 1 --2- 172.21.15.82:0/2944248795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975de80 0x55b059766430 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29b9613740 1 --2- 172.21.15.82:0/2944248795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975d4d0 0x55b05975d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29b9613740 1 -- 172.21.15.82:0/2944248795 >> 172.21.15.82:0/2944248795 conn(0x55b059757ba0 msgr2=0x55b059757f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29b9613740 1 -- 172.21.15.82:0/2944248795 shutdown_connections 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.668+0000 7f29b9613740 1 -- 172.21.15.82:0/2944248795 wait complete. 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29b9613740 1 -- start start 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29b9613740 1 Processor -- start 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29b9613740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 0x55b059766350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29b9613740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b05975de80 0x55b0597636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29b9613740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b059766a00 0x55b059763cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29b9613740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b059769970 con 0x55b05975d4d0 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29b9613740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b05967b500 con 0x55b05975de80 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29b9613740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b059769af0 con 0x55b059766a00 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29b9613740 10 client.0 _ll_register_callbacks cb 0x55b059649b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 0x55b059766350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29affff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b05975de80 0x55b0597636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 0x55b059766350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50384/0 (socket says 172.21.15.82:50384) 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.669+0000 7f29af7fe640 1 -- 172.21.15.82:0/3939948310 learned_addr learned my addr 172.21.15.82:0/3939948310 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29af7fe640 1 -- 172.21.15.82:0/3939948310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b059766a00 msgr2=0x55b059763cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29af7fe640 1 --2- 172.21.15.82:0/3939948310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b059766a00 0x55b059763cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29af7fe640 1 -- 172.21.15.82:0/3939948310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b05975de80 msgr2=0x55b0597636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29af7fe640 1 --2- 172.21.15.82:0/3939948310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b05975de80 0x55b0597636e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29af7fe640 1 -- 172.21.15.82:0/3939948310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0593d43f0 con 0x55b05975d4d0 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29af7fe640 1 --2- 172.21.15.82:0/3939948310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 0x55b059766350 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f29a40058a0 tx=0x7f29a400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9209eabd4d2f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f298f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f298f7fe640 1 -- 172.21.15.82:0/3939948310 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4022060 con 0x55b05975d4d0 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f298f7fe640 1 -- 172.21.15.82:0/3939948310 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f29a400ec60 con 0x55b05975d4d0 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29b9613740 1 -- 172.21.15.82:0/3939948310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29a000fa70 con 0x55b05975d4d0 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f298f7fe640 1 -- 172.21.15.82:0/3939948310 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f29a4029050 con 0x55b05975d4d0 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29b9613740 10 client.16989 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29b9613740 1 -- 172.21.15.82:0/3939948310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b05949e7d0 con 0x55b05975d4d0 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f298f7fe640 1 -- 172.21.15.82:0/3939948310 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a000fa70 con 0x55b05975d4d0 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f298f7fe640 1 -- 172.21.15.82:0/3939948310 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f29a401d410 con 0x55b05975d4d0 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29b9613740 10 client.16989 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29b9613740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29b9613740 20 client.16989 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29b9613740 1 --2- 172.21.15.82:0/3939948310 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55b059776ce0 0x55b0597790d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29b9613740 4 client.16989 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.670+0000 7f29b9613740 1 -- 172.21.15.82:0/3939948310 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b059661930 con 0x55b059776ce0 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.671+0000 7f29affff640 1 --2- 172.21.15.82:0/3939948310 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55b059776ce0 0x55b0597790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.671+0000 7f298f7fe640 10 client.16989 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.671+0000 7f29affff640 1 --2- 172.21.15.82:0/3939948310 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55b059776ce0 0x55b0597790d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32bd14426c069a11 server_cookie=886cb6ca90f8cd84 in_seq=0 out_seq=0 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f298f7fe640 1 -- 172.21.15.82:0/3939948310 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b059661930 con 0x55b059776ce0 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f298f7fe640 10 client.16989 handle_command_reply: tid=0 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f29b9613740 1 client.16989 shutdown 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f29b9613740 1 -- 172.21.15.82:0/3939948310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 msgr2=0x55b059766350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f29b9613740 1 --2- 172.21.15.82:0/3939948310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 0x55b059766350 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f29a40058a0 tx=0x7f29a400f790 comp rx=0 tx=0).stop 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f29b9613740 1 -- 172.21.15.82:0/3939948310 shutdown_connections 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f29b9613740 1 --2- 172.21.15.82:0/3939948310 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55b059776ce0 0x55b0597790d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f29b9613740 1 --2- 172.21.15.82:0/3939948310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b059766a00 0x55b059763cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f29b9613740 1 --2- 172.21.15.82:0/3939948310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b05975de80 0x55b0597636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f298f7fe640 0 client.16989 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f29b9613740 1 --2- 172.21.15.82:0/3939948310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 0x55b059766350 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f29b9613740 1 -- 172.21.15.82:0/3939948310 >> 172.21.15.82:0/3939948310 conn(0x55b059757ba0 msgr2=0x55b0597598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f29b9613740 1 -- 172.21.15.82:0/3939948310 shutdown_connections 2024-04-21T08:51:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f29b9613740 1 -- 172.21.15.82:0/3939948310 wait complete. 2024-04-21T08:51:39.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.672+0000 7f29b9613740 20 client.16989 trim_cache size 0 max 16384 2024-04-21T08:51:39.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29b9613740 1 Processor -- start 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29b9613740 1 -- start start 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29b9613740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 0x55b059772860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29b9613740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975de80 0x55b059765e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29b9613740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b059766a00 0x55b059766350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29b9613740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01b1670 con 0x55b05975d4d0 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29b9613740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01b3210 con 0x55b059766a00 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29b9613740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01b1d40 con 0x55b05975de80 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 0x55b059772860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 0x55b059772860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50388/0 (socket says 172.21.15.82:50388) 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29af7fe640 1 -- 172.21.15.82:0/401376225 learned_addr learned my addr 172.21.15.82:0/401376225 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29af7fe640 1 --2- 172.21.15.82:0/401376225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b059766a00 0x55b059766350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29affff640 1 --2- 172.21.15.82:0/401376225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975de80 0x55b059765e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29af7fe640 1 -- 172.21.15.82:0/401376225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975de80 msgr2=0x55b059765e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29af7fe640 1 --2- 172.21.15.82:0/401376225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975de80 0x55b059765e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29af7fe640 1 -- 172.21.15.82:0/401376225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b059766a00 msgr2=0x55b059766350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29af7fe640 1 --2- 172.21.15.82:0/401376225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b059766a00 0x55b059766350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.686+0000 7f29af7fe640 1 -- 172.21.15.82:0/401376225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b059779700 con 0x55b05975d4d0 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.687+0000 7f29af7fe640 1 --2- 172.21.15.82:0/401376225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 0x55b059772860 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009b20 tx=0x7f29a4009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c843d39698e7e06 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.687+0000 7f2995ffb640 1 -- 172.21.15.82:0/401376225 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4034070 con 0x55b05975d4d0 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.687+0000 7f2995ffb640 1 -- 172.21.15.82:0/401376225 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a4014070 con 0x55b05975d4d0 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.687+0000 7f2995ffb640 1 -- 172.21.15.82:0/401376225 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a400ee50 con 0x55b05975d4d0 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.687+0000 7f29b9613740 1 -- 172.21.15.82:0/401376225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 msgr2=0x55b059772860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.687+0000 7f29b9613740 1 --2- 172.21.15.82:0/401376225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 0x55b059772860 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009b20 tx=0x7f29a4009a90 comp rx=0 tx=0).stop 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 -- 172.21.15.82:0/401376225 shutdown_connections 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 --2- 172.21.15.82:0/401376225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b059766a00 0x55b059766350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 --2- 172.21.15.82:0/401376225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975de80 0x55b059765e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 --2- 172.21.15.82:0/401376225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b05975d4d0 0x55b059772860 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 -- 172.21.15.82:0/401376225 >> 172.21.15.82:0/401376225 conn(0x55b059757ba0 msgr2=0x55b059756c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 -- 172.21.15.82:0/401376225 shutdown_connections 2024-04-21T08:51:39.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 -- 172.21.15.82:0/401376225 wait complete. 2024-04-21T08:51:39.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 -- start start 2024-04-21T08:51:39.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 Processor -- start 2024-04-21T08:51:39.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975d4d0 0x55b05976ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b05975de80 0x55b05975ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b059766a00 0x55b059760490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01b2b40 con 0x55b059766a00 2024-04-21T08:51:39.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01b2470 con 0x55b05975de80 2024-04-21T08:51:39.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01af650 con 0x55b05975d4d0 2024-04-21T08:51:39.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29b9613740 10 client.0 _ll_register_callbacks cb 0x7f29a400e8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:39.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b059766a00 0x55b059760490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29affff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b05975de80 0x55b05975ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b059766a00 0x55b059760490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50392/0 (socket says 172.21.15.82:50392) 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.688+0000 7f29af7fe640 1 -- 172.21.15.82:0/550825109 learned_addr learned my addr 172.21.15.82:0/550825109 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f29af7fe640 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975d4d0 0x55b05976ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f29affff640 1 -- 172.21.15.82:0/550825109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975d4d0 msgr2=0x55b05976ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f29affff640 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975d4d0 0x55b05976ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f29affff640 1 -- 172.21.15.82:0/550825109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b059766a00 msgr2=0x55b059760490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f29affff640 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b059766a00 0x55b059760490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f29affff640 1 -- 172.21.15.82:0/550825109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b059760a00 con 0x55b05975de80 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f29af7fe640 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b059766a00 0x55b059760490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f29affff640 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b05975de80 0x55b05975ff50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55b059758c10 tx=0x7f29a00092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4daa63bea44ca6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f298f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f298f7fe640 1 -- 172.21.15.82:0/550825109 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a001e040 con 0x55b05975de80 2024-04-21T08:51:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f29b9613740 1 -- 172.21.15.82:0/550825109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29a4014070 con 0x55b05975de80 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f298f7fe640 1 -- 172.21.15.82:0/550825109 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f29a0009b50 con 0x55b05975de80 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f298f7fe640 1 -- 172.21.15.82:0/550825109 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f29a0038050 con 0x55b05975de80 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f29b9613740 10 client.26342 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.689+0000 7f29b9613740 1 -- 172.21.15.82:0/550825109 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b0597dcf40 con 0x55b05975de80 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f298f7fe640 1 -- 172.21.15.82:0/550825109 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a4014070 con 0x55b05975de80 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f298f7fe640 1 -- 172.21.15.82:0/550825109 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f29a002a250 con 0x55b05975de80 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f29b9613740 10 client.26342 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f29b9613740 10 client.26342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f29b9613740 20 client.26342 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f29b9613740 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55b0597dd520 0x55b0597df940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f29b9613740 4 client.26342 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f29b9613740 1 -- 172.21.15.82:0/550825109 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0596cfc10 con 0x55b0597dd520 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f29af7fe640 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55b0597dd520 0x55b0597df940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f298f7fe640 10 client.26342 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f29af7fe640 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55b0597dd520 0x55b0597df940 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b46b2e3affacff3 server_cookie=4350a95dbbfd192c in_seq=0 out_seq=0 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f298f7fe640 1 -- 172.21.15.82:0/550825109 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0596cfc10 con 0x55b0597dd520 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f298f7fe640 10 client.26342 handle_command_reply: tid=0 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.690+0000 7f29b9613740 1 client.26342 shutdown 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b9613740 1 -- 172.21.15.82:0/550825109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b05975de80 msgr2=0x55b05975ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b9613740 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b05975de80 0x55b05975ff50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55b059758c10 tx=0x7f29a00092e0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b9613740 1 -- 172.21.15.82:0/550825109 shutdown_connections 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b9613740 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55b0597dd520 0x55b0597df940 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b9613740 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b059766a00 0x55b059760490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b9613740 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b05975de80 0x55b05975ff50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b9613740 1 --2- 172.21.15.82:0/550825109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b05975d4d0 0x55b05976ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f298f7fe640 0 client.26342 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b9613740 1 -- 172.21.15.82:0/550825109 >> 172.21.15.82:0/550825109 conn(0x55b059757ba0 msgr2=0x55b059756370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b9613740 1 -- 172.21.15.82:0/550825109 shutdown_connections 2024-04-21T08:51:39.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b9613740 1 -- 172.21.15.82:0/550825109 wait complete. 2024-04-21T08:51:39.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b9613740 20 client.26342 trim_cache size 0 max 16384 2024-04-21T08:51:39.697 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:51:39.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b6a25640 1 -- 172.21.15.82:0/2345877215 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29980a3d50 msgr2=0x7f29980a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b6a25640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29980a3d50 0x7f29980a6210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f29a80be030 tx=0x7f29a802e040 comp rx=0 tx=0).stop 2024-04-21T08:51:39.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b6a25640 1 -- 172.21.15.82:0/2345877215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b01020c0 msgr2=0x7f29b01a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b6a25640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b01020c0 0x7f29b01a6910 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f29a8005e30 tx=0x7f29a80346d0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b6a25640 1 -- 172.21.15.82:0/2345877215 shutdown_connections 2024-04-21T08:51:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b6a25640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29980a3d50 0x7f29980a6210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b6a25640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b01020c0 0x7f29b01a6910 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b6a25640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29b01072e0 0x7f29b0101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b6a25640 1 --2- 172.21.15.82:0/2345877215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29b01006d0 0x7f29b0101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.691+0000 7f29b6a25640 1 -- 172.21.15.82:0/2345877215 >> 172.21.15.82:0/2345877215 conn(0x7f29b00773f0 msgr2=0x7f29b01054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.692+0000 7f29b6a25640 1 -- 172.21.15.82:0/2345877215 shutdown_connections 2024-04-21T08:51:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:39.692+0000 7f29b6a25640 1 -- 172.21.15.82:0/2345877215 wait complete. 2024-04-21T08:51:40.702 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:51:40.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:51:40.703 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:51:40.725 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:40 smithi082 ceph-mon[27967]: pgmap v142: 129 pgs: 129 active+clean; 506 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2024-04-21T08:51:40.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:40 smithi134 ceph-mon[31772]: pgmap v142: 129 pgs: 129 active+clean; 506 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2024-04-21T08:51:40.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.819+0000 7f63b5bdb640 1 -- 172.21.15.82:0/2324770677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63b0106160 msgr2=0x7f63b010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:40.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.819+0000 7f63b5bdb640 1 --2- 172.21.15.82:0/2324770677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63b0106160 0x7f63b010dd30 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009ca0 tx=0x7f63a802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.819+0000 7f63b5bdb640 1 -- 172.21.15.82:0/2324770677 shutdown_connections 2024-04-21T08:51:40.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.819+0000 7f63b5bdb640 1 --2- 172.21.15.82:0/2324770677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63b0106160 0x7f63b010dd30 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.819+0000 7f63b5bdb640 1 --2- 172.21.15.82:0/2324770677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63b0101a40 0x7f63b0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.819+0000 7f63b5bdb640 1 --2- 172.21.15.82:0/2324770677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b0101090 0x7f63b0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.819+0000 7f63b5bdb640 1 -- 172.21.15.82:0/2324770677 >> 172.21.15.82:0/2324770677 conn(0x7f63b0077f50 msgr2=0x7f63b0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:40.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.820+0000 7f63b5bdb640 1 -- 172.21.15.82:0/2324770677 shutdown_connections 2024-04-21T08:51:40.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.820+0000 7f63b5bdb640 1 -- 172.21.15.82:0/2324770677 wait complete. 2024-04-21T08:51:40.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.820+0000 7f63b5bdb640 1 Processor -- start 2024-04-21T08:51:40.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.820+0000 7f63b5bdb640 1 -- start start 2024-04-21T08:51:40.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.820+0000 7f63b5bdb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63b0101090 0x7f63b0197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.820+0000 7f63b5bdb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63b0106160 0x7f63b0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.820+0000 7f63b5bdb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b019bef0 0x7f63b019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.820+0000 7f63b5bdb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01110d0 con 0x7f63b0101090 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.820+0000 7f63b5bdb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0110f50 con 0x7f63b019bef0 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.820+0000 7f63b5bdb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0111250 con 0x7f63b0106160 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63b0101090 0x7f63b0197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63b0101090 0x7f63b0197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50408/0 (socket says 172.21.15.82:50408) 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63b0106160 0x7f63b0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63af7fe640 1 -- 172.21.15.82:0/337873386 learned_addr learned my addr 172.21.15.82:0/337873386 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63affff640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b019bef0 0x7f63b019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63affff640 1 -- 172.21.15.82:0/337873386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63b0106160 msgr2=0x7f63b0197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63affff640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63b0106160 0x7f63b0197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63affff640 1 -- 172.21.15.82:0/337873386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63b0101090 msgr2=0x7f63b0197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63affff640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63b0101090 0x7f63b0197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63affff640 1 -- 172.21.15.82:0/337873386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b019c9e0 con 0x7f63b019bef0 2024-04-21T08:51:40.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63af7fe640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63b0101090 0x7f63b0197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:40.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63affff640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b019bef0 0x7f63b019c3a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63a802c700 tx=0x7f63a80346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:40.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63acff9640 1 -- 172.21.15.82:0/337873386 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8041020 con 0x7f63b019bef0 2024-04-21T08:51:40.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63acff9640 1 -- 172.21.15.82:0/337873386 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a8032970 con 0x7f63b019bef0 2024-04-21T08:51:40.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63b5bdb640 1 -- 172.21.15.82:0/337873386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b019cc70 con 0x7f63b019bef0 2024-04-21T08:51:40.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63acff9640 1 -- 172.21.15.82:0/337873386 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8039040 con 0x7f63b019bef0 2024-04-21T08:51:40.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.821+0000 7f63b5bdb640 1 -- 172.21.15.82:0/337873386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01a4430 con 0x7f63b019bef0 2024-04-21T08:51:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.823+0000 7f63acff9640 1 -- 172.21.15.82:0/337873386 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f63a8044020 con 0x7f63b019bef0 2024-04-21T08:51:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.824+0000 7f63b87ce740 1 Processor -- start 2024-04-21T08:51:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.824+0000 7f63acff9640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f63980a3da0 0x7f63980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.824+0000 7f63b87ce740 1 -- start start 2024-04-21T08:51:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.824+0000 7f63aeffd640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f63980a3da0 0x7f63980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.824+0000 7f63acff9640 1 -- 172.21.15.82:0/337873386 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f63a80f26b0 con 0x7f63b019bef0 2024-04-21T08:51:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.824+0000 7f63b87ce740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aad68e0 0x56109aad6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.824+0000 7f63b87ce740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aadf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.824+0000 7f63b87ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aadfe10 0x56109aae2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.824+0000 7f63b87ce740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56109aa3af00 con 0x56109aad7290 2024-04-21T08:51:40.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.824+0000 7f63b87ce740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56109a9d8590 con 0x56109aad68e0 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.824+0000 7f63b87ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56109aa36ec0 con 0x56109aadfe10 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aadf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63af7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aad68e0 0x56109aad6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aadf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50416/0 (socket says 172.21.15.82:50416) 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63affff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aadfe10 0x56109aae2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63aeffd640 1 -- 172.21.15.82:0/4256264014 learned_addr learned my addr 172.21.15.82:0/4256264014 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63aeffd640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f63980a3da0 0x7f63980a6260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f63b01009b0 tx=0x7f63a4009f70 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63af7fe640 1 -- 172.21.15.82:0/4256264014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aadfe10 msgr2=0x56109aae2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63af7fe640 1 --2- 172.21.15.82:0/4256264014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aadfe10 0x56109aae2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63af7fe640 1 -- 172.21.15.82:0/4256264014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 msgr2=0x56109aadf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63af7fe640 1 --2- 172.21.15.82:0/4256264014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aadf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63af7fe640 1 -- 172.21.15.82:0/4256264014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56109a8157e0 con 0x56109aad68e0 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63aeffd640 1 --2- 172.21.15.82:0/4256264014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aadf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63affff640 1 --2- 172.21.15.82:0/4256264014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aadfe10 0x56109aae2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f63af7fe640 1 --2- 172.21.15.82:0/4256264014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aad68e0 0x56109aad6cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f63a0011600 tx=0x7f63a0011ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70c806f0f5611098 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f6395ffb640 1 -- 172.21.15.82:0/4256264014 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0004030 con 0x56109aad68e0 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f6395ffb640 1 -- 172.21.15.82:0/4256264014 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a000e6e0 con 0x56109aad68e0 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.825+0000 7f6395ffb640 1 -- 172.21.15.82:0/4256264014 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a00039f0 con 0x56109aad68e0 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.826+0000 7f63b87ce740 1 -- 172.21.15.82:0/4256264014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aad68e0 msgr2=0x56109aad6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.826+0000 7f63b87ce740 1 --2- 172.21.15.82:0/4256264014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aad68e0 0x56109aad6cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f63a0011600 tx=0x7f63a0011ac0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.826+0000 7f63b87ce740 1 -- 172.21.15.82:0/4256264014 shutdown_connections 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.826+0000 7f63b87ce740 1 --2- 172.21.15.82:0/4256264014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aadfe10 0x56109aae2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.826+0000 7f63b87ce740 1 --2- 172.21.15.82:0/4256264014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aadf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.826+0000 7f63b87ce740 1 --2- 172.21.15.82:0/4256264014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aad68e0 0x56109aad6cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.826+0000 7f63b87ce740 1 -- 172.21.15.82:0/4256264014 >> 172.21.15.82:0/4256264014 conn(0x56109aad0fb0 msgr2=0x56109aad1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.826+0000 7f63b87ce740 1 -- 172.21.15.82:0/4256264014 shutdown_connections 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.826+0000 7f63b87ce740 1 -- 172.21.15.82:0/4256264014 wait complete. 2024-04-21T08:51:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.827+0000 7f63b87ce740 1 -- start start 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.827+0000 7f63b87ce740 1 Processor -- start 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.827+0000 7f63b87ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aadcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.827+0000 7f63b87ce740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aad7290 0x56109aadd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.827+0000 7f63b87ce740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aadfe10 0x56109aadd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.827+0000 7f63b87ce740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56109aae2d80 con 0x56109aadfe10 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.827+0000 7f63b87ce740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56109a9f4580 con 0x56109aad7290 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.827+0000 7f63b87ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56109aae2f00 con 0x56109aad68e0 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.827+0000 7f63b87ce740 10 client.0 _ll_register_callbacks cb 0x56109a9fa650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63af7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aadcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63af7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aadcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:35562/0 (socket says 172.21.15.82:35562) 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aadfe10 0x56109aadd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63af7fe640 1 -- 172.21.15.82:0/765273328 learned_addr learned my addr 172.21.15.82:0/765273328 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63aeffd640 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aad7290 0x56109aadd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63af7fe640 1 -- 172.21.15.82:0/765273328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aad7290 msgr2=0x56109aadd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63af7fe640 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aad7290 0x56109aadd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63af7fe640 1 -- 172.21.15.82:0/765273328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aadfe10 msgr2=0x56109aadd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63af7fe640 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aadfe10 0x56109aadd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63af7fe640 1 -- 172.21.15.82:0/765273328 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56109a875f70 con 0x56109aad68e0 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63affff640 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aadfe10 0x56109aadd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63af7fe640 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aadcdd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63a00097c0 tx=0x7f63a0009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=169794e61903154f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f638effd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f638effd640 1 -- 172.21.15.82:0/765273328 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0028020 con 0x56109aad68e0 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f63b87ce740 1 -- 172.21.15.82:0/765273328 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a000e6e0 con 0x56109aad68e0 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f638effd640 1 -- 172.21.15.82:0/765273328 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f63a001a710 con 0x56109aad68e0 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.828+0000 7f638effd640 1 -- 172.21.15.82:0/765273328 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f63a0033050 con 0x56109aad68e0 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.829+0000 7f63b87ce740 10 client.26083 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.829+0000 7f63b87ce740 1 -- 172.21.15.82:0/765273328 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56109a70d170 con 0x56109aad68e0 2024-04-21T08:51:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.829+0000 7f638effd640 1 -- 172.21.15.82:0/765273328 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a000e6e0 con 0x56109aad68e0 2024-04-21T08:51:40.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.829+0000 7f638effd640 1 -- 172.21.15.82:0/765273328 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f63a0024ba0 con 0x56109aad68e0 2024-04-21T08:51:40.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.829+0000 7f63b87ce740 10 client.26083 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:40.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.829+0000 7f63b87ce740 10 client.26083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:40.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.829+0000 7f63b87ce740 20 client.26083 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:40.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.829+0000 7f63b87ce740 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56109aaf0220 0x56109aaf2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.829+0000 7f63b87ce740 4 client.26083 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:40.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.829+0000 7f63b87ce740 1 -- 172.21.15.82:0/765273328 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56109aa48920 con 0x56109aaf0220 2024-04-21T08:51:40.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.829+0000 7f63aeffd640 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56109aaf0220 0x56109aaf2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f638effd640 10 client.26083 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:40.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f63aeffd640 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56109aaf0220 0x56109aaf2610 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c005521aba90a09d server_cookie=452dc659ebbcb193 in_seq=0 out_seq=0 2024-04-21T08:51:40.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f638effd640 1 -- 172.21.15.82:0/765273328 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56109aa48920 con 0x56109aaf0220 2024-04-21T08:51:40.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f638effd640 10 client.26083 handle_command_reply: tid=0 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f63b87ce740 1 client.26083 shutdown 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f63b87ce740 1 -- 172.21.15.82:0/765273328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 msgr2=0x56109aadcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f63b87ce740 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aadcdd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63a00097c0 tx=0x7f63a0009cd0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f63b87ce740 1 -- 172.21.15.82:0/765273328 shutdown_connections 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f63b87ce740 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56109aaf0220 0x56109aaf2610 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f63b87ce740 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aadfe10 0x56109aadd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f63b87ce740 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aad7290 0x56109aadd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f638effd640 0 client.26083 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f63b87ce740 1 --2- 172.21.15.82:0/765273328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aadcdd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.830+0000 7f63b87ce740 1 -- 172.21.15.82:0/765273328 >> 172.21.15.82:0/765273328 conn(0x56109aad0fb0 msgr2=0x56109aad2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.831+0000 7f63b87ce740 1 -- 172.21.15.82:0/765273328 shutdown_connections 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.831+0000 7f63b87ce740 1 -- 172.21.15.82:0/765273328 wait complete. 2024-04-21T08:51:40.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.831+0000 7f63b87ce740 20 client.26083 trim_cache size 0 max 16384 2024-04-21T08:51:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.850+0000 7f63b87ce740 1 Processor -- start 2024-04-21T08:51:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.850+0000 7f63b87ce740 1 -- start start 2024-04-21T08:51:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.850+0000 7f63b87ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aadf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.850+0000 7f63b87ce740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aaebc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.850+0000 7f63b87ce740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aadfe10 0x56109aad2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.850+0000 7f63b87ce740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ad1a0 con 0x56109aad7290 2024-04-21T08:51:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.850+0000 7f63b87ce740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aed40 con 0x56109aadfe10 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.850+0000 7f63b87ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ad870 con 0x56109aad68e0 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aaebc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aaebc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50432/0 (socket says 172.21.15.82:50432) 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63aeffd640 1 -- 172.21.15.82:0/2209485188 learned_addr learned my addr 172.21.15.82:0/2209485188 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63af7fe640 1 --2- 172.21.15.82:0/2209485188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aadf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63affff640 1 --2- 172.21.15.82:0/2209485188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aadfe10 0x56109aad2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63aeffd640 1 -- 172.21.15.82:0/2209485188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 msgr2=0x56109aadf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63aeffd640 1 --2- 172.21.15.82:0/2209485188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aadf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63aeffd640 1 -- 172.21.15.82:0/2209485188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aadfe10 msgr2=0x56109aad2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63aeffd640 1 --2- 172.21.15.82:0/2209485188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aadfe10 0x56109aad2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63aeffd640 1 -- 172.21.15.82:0/2209485188 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56109aad5ba0 con 0x56109aad7290 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63aeffd640 1 --2- 172.21.15.82:0/2209485188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aaebc40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f63a4005d00 tx=0x7f63a401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14df6e075de3fbbb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63957fa640 1 -- 172.21.15.82:0/2209485188 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4017040 con 0x56109aad7290 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63957fa640 1 -- 172.21.15.82:0/2209485188 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a402e070 con 0x56109aad7290 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.851+0000 7f63957fa640 1 -- 172.21.15.82:0/2209485188 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a400f820 con 0x56109aad7290 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.852+0000 7f63b87ce740 1 -- 172.21.15.82:0/2209485188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 msgr2=0x56109aaebc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.852+0000 7f63b87ce740 1 --2- 172.21.15.82:0/2209485188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aaebc40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f63a4005d00 tx=0x7f63a401c000 comp rx=0 tx=0).stop 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.852+0000 7f63b87ce740 1 -- 172.21.15.82:0/2209485188 shutdown_connections 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.852+0000 7f63b87ce740 1 --2- 172.21.15.82:0/2209485188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aadfe10 0x56109aad2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.852+0000 7f63b87ce740 1 --2- 172.21.15.82:0/2209485188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aaebc40 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.852+0000 7f63b87ce740 1 --2- 172.21.15.82:0/2209485188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aadf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.852+0000 7f63b87ce740 1 -- 172.21.15.82:0/2209485188 >> 172.21.15.82:0/2209485188 conn(0x56109aad0fb0 msgr2=0x56109aad2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.852+0000 7f63b87ce740 1 -- 172.21.15.82:0/2209485188 shutdown_connections 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.852+0000 7f63b87ce740 1 -- 172.21.15.82:0/2209485188 wait complete. 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.852+0000 7f63b87ce740 1 -- start start 2024-04-21T08:51:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63b87ce740 1 Processor -- start 2024-04-21T08:51:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63b87ce740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aae8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63b87ce740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aad95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63b87ce740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aadfe10 0x56109aad9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63b87ce740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ae670 con 0x56109aad7290 2024-04-21T08:51:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63b87ce740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01adfa0 con 0x56109aadfe10 2024-04-21T08:51:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63b87ce740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ab180 con 0x56109aad68e0 2024-04-21T08:51:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63b87ce740 10 client.0 _ll_register_callbacks cb 0x7f63a40098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aad95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aad95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50448/0 (socket says 172.21.15.82:50448) 2024-04-21T08:51:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63aeffd640 1 -- 172.21.15.82:0/2211508147 learned_addr learned my addr 172.21.15.82:0/2211508147 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63af7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aae8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63affff640 1 --2- 172.21.15.82:0/2211508147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aadfe10 0x56109aad9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63aeffd640 1 -- 172.21.15.82:0/2211508147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 msgr2=0x56109aae8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63aeffd640 1 --2- 172.21.15.82:0/2211508147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aae8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63aeffd640 1 -- 172.21.15.82:0/2211508147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aadfe10 msgr2=0x56109aad9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63aeffd640 1 --2- 172.21.15.82:0/2211508147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aadfe10 0x56109aad9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63aeffd640 1 -- 172.21.15.82:0/2211508147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56109aada040 con 0x56109aad7290 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f63aeffd640 1 --2- 172.21.15.82:0/2211508147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aad95c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f63a402b040 tx=0x7f63a40043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02c47799563dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.853+0000 7f638effd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f638effd640 1 -- 172.21.15.82:0/2211508147 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4036070 con 0x56109aad7290 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f638effd640 1 -- 172.21.15.82:0/2211508147 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f63a4029730 con 0x56109aad7290 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f63b87ce740 1 -- 172.21.15.82:0/2211508147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a402e070 con 0x56109aad7290 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f638effd640 1 -- 172.21.15.82:0/2211508147 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f63a4042050 con 0x56109aad7290 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f63b87ce740 10 client.17013 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f63b87ce740 1 -- 172.21.15.82:0/2211508147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56109ab56300 con 0x56109aad7290 2024-04-21T08:51:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f638effd640 1 -- 172.21.15.82:0/2211508147 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a402e070 con 0x56109aad7290 2024-04-21T08:51:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f638effd640 1 -- 172.21.15.82:0/2211508147 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f63a40046c0 con 0x56109aad7290 2024-04-21T08:51:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f63b87ce740 10 client.17013 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f63b87ce740 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f63b87ce740 20 client.17013 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f63b87ce740 1 --2- 172.21.15.82:0/2211508147 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56109ab569f0 0x56109ab58e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f63b87ce740 4 client.17013 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:51:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f63b87ce740 1 -- 172.21.15.82:0/2211508147 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56109aa42790 con 0x56109ab569f0 2024-04-21T08:51:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.854+0000 7f63af7fe640 1 --2- 172.21.15.82:0/2211508147 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56109ab569f0 0x56109ab58e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.855+0000 7f638effd640 10 client.17013 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.855+0000 7f63af7fe640 1 --2- 172.21.15.82:0/2211508147 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56109ab569f0 0x56109ab58e10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3268df3c3cc4d9c server_cookie=651c8d9ad6b8ae1e in_seq=0 out_seq=0 2024-04-21T08:51:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f638effd640 1 -- 172.21.15.82:0/2211508147 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56109aa42790 con 0x56109ab569f0 2024-04-21T08:51:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f638effd640 10 client.17013 handle_command_reply: tid=0 2024-04-21T08:51:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f63b87ce740 1 client.17013 shutdown 2024-04-21T08:51:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f63b87ce740 1 -- 172.21.15.82:0/2211508147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 msgr2=0x56109aad95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f63b87ce740 1 --2- 172.21.15.82:0/2211508147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aad95c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f63a402b040 tx=0x7f63a40043a0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f63b87ce740 1 -- 172.21.15.82:0/2211508147 shutdown_connections 2024-04-21T08:51:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f63b87ce740 1 --2- 172.21.15.82:0/2211508147 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56109ab569f0 0x56109ab58e10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f63b87ce740 1 --2- 172.21.15.82:0/2211508147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56109aadfe10 0x56109aad9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f638effd640 0 client.17013 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f63b87ce740 1 --2- 172.21.15.82:0/2211508147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56109aad7290 0x56109aad95c0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f63b87ce740 1 --2- 172.21.15.82:0/2211508147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56109aad68e0 0x56109aae8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f63b87ce740 1 -- 172.21.15.82:0/2211508147 >> 172.21.15.82:0/2211508147 conn(0x56109aad0fb0 msgr2=0x56109aad2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:40.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f63b87ce740 1 -- 172.21.15.82:0/2211508147 shutdown_connections 2024-04-21T08:51:40.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.856+0000 7f63b87ce740 1 -- 172.21.15.82:0/2211508147 wait complete. 2024-04-21T08:51:40.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b87ce740 20 client.17013 trim_cache size 0 max 16384 2024-04-21T08:51:40.863 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:51:40.863 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:51:40.863 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "67068658-7e18-4c0e-a274-59f6fb85db36", 2024-04-21T08:51:40.863 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:51:40.863 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:51:40.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b5bdb640 1 -- 172.21.15.82:0/337873386 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f63980a3da0 msgr2=0x7f63980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:40.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b5bdb640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f63980a3da0 0x7f63980a6260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f63b01009b0 tx=0x7f63a4009f70 comp rx=0 tx=0).stop 2024-04-21T08:51:40.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b5bdb640 1 -- 172.21.15.82:0/337873386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b019bef0 msgr2=0x7f63b019c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:40.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b5bdb640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b019bef0 0x7f63b019c3a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63a802c700 tx=0x7f63a80346f0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b5bdb640 1 -- 172.21.15.82:0/337873386 shutdown_connections 2024-04-21T08:51:40.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b5bdb640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b019bef0 0x7f63b019c3a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b5bdb640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63b0106160 0x7f63b0197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b5bdb640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f63980a3da0 0x7f63980a6260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b5bdb640 1 --2- 172.21.15.82:0/337873386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63b0101090 0x7f63b0197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:40.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b5bdb640 1 -- 172.21.15.82:0/337873386 >> 172.21.15.82:0/337873386 conn(0x7f63b0077f50 msgr2=0x7f63b00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:40.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b5bdb640 1 -- 172.21.15.82:0/337873386 shutdown_connections 2024-04-21T08:51:40.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:40.857+0000 7f63b5bdb640 1 -- 172.21.15.82:0/337873386 wait complete. 2024-04-21T08:51:40.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:51:40.868 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:51:40.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:40 smithi106 ceph-mon[31072]: pgmap v142: 129 pgs: 129 active+clean; 506 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2024-04-21T08:51:41.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.120+0000 7f5305b7e640 1 -- 172.21.15.82:0/3918641007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530010eb40 msgr2=0x7f530011d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:41.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.120+0000 7f5305b7e640 1 --2- 172.21.15.82:0/3918641007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530010eb40 0x7f530011d740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009be0 tx=0x7f52f802bfa0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.135+0000 7f5305b7e640 1 -- 172.21.15.82:0/3918641007 shutdown_connections 2024-04-21T08:51:41.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.135+0000 7f5305b7e640 1 --2- 172.21.15.82:0/3918641007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530010eb40 0x7f530011d740 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.135+0000 7f5305b7e640 1 --2- 172.21.15.82:0/3918641007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f530010e0c0 0x7f530010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.135+0000 7f5305b7e640 1 --2- 172.21.15.82:0/3918641007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5300110550 0x7f5300110930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.135+0000 7f5305b7e640 1 -- 172.21.15.82:0/3918641007 >> 172.21.15.82:0/3918641007 conn(0x7f530006ce60 msgr2=0x7f530006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:41.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.135+0000 7f5305b7e640 1 -- 172.21.15.82:0/3918641007 shutdown_connections 2024-04-21T08:51:41.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.135+0000 7f5305b7e640 1 -- 172.21.15.82:0/3918641007 wait complete. 2024-04-21T08:51:41.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.136+0000 7f5305b7e640 1 Processor -- start 2024-04-21T08:51:41.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.136+0000 7f5305b7e640 1 -- start start 2024-04-21T08:51:41.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.137+0000 7f5305b7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530010e0c0 0x7f53001a7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.137+0000 7f5305b7e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530010eb40 0x7f53001a7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.137+0000 7f5305b7e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5300110550 0x7f53001abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.137+0000 7f5305b7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5300120e30 con 0x7f530010e0c0 2024-04-21T08:51:41.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.137+0000 7f5305b7e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5300120cb0 con 0x7f5300110550 2024-04-21T08:51:41.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.137+0000 7f5305b7e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5300120fb0 con 0x7f530010eb40 2024-04-21T08:51:41.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.137+0000 7f52fffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5300110550 0x7f53001abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.137+0000 7f52fffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5300110550 0x7f53001abbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34536/0 (socket says 172.21.15.82:34536) 2024-04-21T08:51:41.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.137+0000 7f52fffff640 1 -- 172.21.15.82:0/2993782241 learned_addr learned my addr 172.21.15.82:0/2993782241 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:41.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.137+0000 7f52ff7fe640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530010e0c0 0x7f53001a7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.138+0000 7f52fffff640 1 -- 172.21.15.82:0/2993782241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530010eb40 msgr2=0x7f53001a7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:41.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.138+0000 7f52feffd640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530010eb40 0x7f53001a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.138+0000 7f52fffff640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530010eb40 0x7f53001a7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.138+0000 7f52fffff640 1 -- 172.21.15.82:0/2993782241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530010e0c0 msgr2=0x7f53001a7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:41.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.138+0000 7f52fffff640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530010e0c0 0x7f53001a7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.138+0000 7f52fffff640 1 -- 172.21.15.82:0/2993782241 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53001ac300 con 0x7f5300110550 2024-04-21T08:51:41.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.138+0000 7f52feffd640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530010eb40 0x7f53001a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:41.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.138+0000 7f52ff7fe640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530010e0c0 0x7f53001a7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:41.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.138+0000 7f52fffff640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5300110550 0x7f53001abbd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f52f8006010 tx=0x7f52f802cbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:41.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.152+0000 7f52fcff9640 1 -- 172.21.15.82:0/2993782241 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8043020 con 0x7f5300110550 2024-04-21T08:51:41.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.152+0000 7f52fcff9640 1 -- 172.21.15.82:0/2993782241 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f803eb10 con 0x7f5300110550 2024-04-21T08:51:41.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.152+0000 7f52fcff9640 1 -- 172.21.15.82:0/2993782241 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f803fb10 con 0x7f5300110550 2024-04-21T08:51:41.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.152+0000 7f5305b7e640 1 -- 172.21.15.82:0/2993782241 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53001ac590 con 0x7f5300110550 2024-04-21T08:51:41.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.152+0000 7f5305b7e640 1 -- 172.21.15.82:0/2993782241 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53001b3e80 con 0x7f5300110550 2024-04-21T08:51:41.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.155+0000 7f52fcff9640 1 -- 172.21.15.82:0/2993782241 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f52f8046020 con 0x7f5300110550 2024-04-21T08:51:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.157+0000 7f52fcff9640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f52e80a3f20 0x7f52e80a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.157+0000 7f52fcff9640 1 -- 172.21.15.82:0/2993782241 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f52f80f3370 con 0x7f5300110550 2024-04-21T08:51:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.158+0000 7f52ff7fe640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f52e80a3f20 0x7f52e80a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.158+0000 7f530876a740 1 Processor -- start 2024-04-21T08:51:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.159+0000 7f52ff7fe640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f52e80a3f20 0x7f52e80a63e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f52f4005fd0 tx=0x7f52f400f730 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:41.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.159+0000 7f530876a740 1 -- start start 2024-04-21T08:51:41.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.159+0000 7f530876a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10b700 0x55570b10bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.159+0000 7f530876a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b10c020 0x55570b1146c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.159+0000 7f530876a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b114c00 0x55570b116ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.159+0000 7f530876a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55570b0701e0 con 0x55570b10c020 2024-04-21T08:51:41.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.159+0000 7f530876a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55570b00d390 con 0x55570b10b700 2024-04-21T08:51:41.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.159+0000 7f530876a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55570b06c1a0 con 0x55570b114c00 2024-04-21T08:51:41.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.160+0000 7f52feffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b10c020 0x55570b1146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.160+0000 7f52feffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b10c020 0x55570b1146c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50464/0 (socket says 172.21.15.82:50464) 2024-04-21T08:51:41.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.160+0000 7f52feffd640 1 -- 172.21.15.82:0/229391813 learned_addr learned my addr 172.21.15.82:0/229391813 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:41.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.160+0000 7f52feffd640 1 -- 172.21.15.82:0/229391813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b114c00 msgr2=0x55570b116ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:51:41.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.160+0000 7f52feffd640 1 --2- 172.21.15.82:0/229391813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b114c00 0x55570b116ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.160+0000 7f52feffd640 1 -- 172.21.15.82:0/229391813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10b700 msgr2=0x55570b10bae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:51:41.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.160+0000 7f52feffd640 1 --2- 172.21.15.82:0/229391813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10b700 0x55570b10bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.160+0000 7f52feffd640 1 -- 172.21.15.82:0/229391813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570ae4c7d0 con 0x55570b10c020 2024-04-21T08:51:41.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.161+0000 7f52feffd640 1 --2- 172.21.15.82:0/229391813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b10c020 0x55570b1146c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009c50 tx=0x7f52f0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc198b7571ed953 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:41.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.161+0000 7f52edffb640 1 -- 172.21.15.82:0/229391813 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0004030 con 0x55570b10c020 2024-04-21T08:51:41.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.161+0000 7f52edffb640 1 -- 172.21.15.82:0/229391813 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f00041d0 con 0x55570b10c020 2024-04-21T08:51:41.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.161+0000 7f52edffb640 1 -- 172.21.15.82:0/229391813 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f00126b0 con 0x55570b10c020 2024-04-21T08:51:41.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.162+0000 7f530876a740 1 -- 172.21.15.82:0/229391813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b10c020 msgr2=0x55570b1146c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:41.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.162+0000 7f530876a740 1 --2- 172.21.15.82:0/229391813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b10c020 0x55570b1146c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009c50 tx=0x7f52f0010740 comp rx=0 tx=0).stop 2024-04-21T08:51:41.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.163+0000 7f530876a740 1 -- 172.21.15.82:0/229391813 shutdown_connections 2024-04-21T08:51:41.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.163+0000 7f530876a740 1 --2- 172.21.15.82:0/229391813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b114c00 0x55570b116ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.163+0000 7f530876a740 1 --2- 172.21.15.82:0/229391813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b10c020 0x55570b1146c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.163+0000 7f530876a740 1 --2- 172.21.15.82:0/229391813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10b700 0x55570b10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.163+0000 7f530876a740 1 -- 172.21.15.82:0/229391813 >> 172.21.15.82:0/229391813 conn(0x55570b105e30 msgr2=0x55570b106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:41.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.163+0000 7f530876a740 1 -- 172.21.15.82:0/229391813 shutdown_connections 2024-04-21T08:51:41.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.163+0000 7f530876a740 1 -- 172.21.15.82:0/229391813 wait complete. 2024-04-21T08:51:41.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.163+0000 7f530876a740 1 -- start start 2024-04-21T08:51:41.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.164+0000 7f530876a740 1 Processor -- start 2024-04-21T08:51:41.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.164+0000 7f530876a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10b700 0x55570b111da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.164+0000 7f530876a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 0x55570b1122e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.164+0000 7f530876a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b112820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.164+0000 7f530876a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55570b128a00 con 0x55570b114c00 2024-04-21T08:51:41.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.164+0000 7f530876a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55570b0292f0 con 0x55570b10c020 2024-04-21T08:51:41.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.164+0000 7f530876a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55570b128b80 con 0x55570b10b700 2024-04-21T08:51:41.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.164+0000 7f530876a740 10 client.0 _ll_register_callbacks cb 0x55570b036580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:41.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.165+0000 7f52feffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 0x55570b1122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.165+0000 7f52feffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 0x55570b1122e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34552/0 (socket says 172.21.15.82:34552) 2024-04-21T08:51:41.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.165+0000 7f52feffd640 1 -- 172.21.15.82:0/2035596798 learned_addr learned my addr 172.21.15.82:0/2035596798 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:41.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.165+0000 7f52ff7fe640 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10b700 0x55570b111da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.166+0000 7f52ff7fe640 1 -- 172.21.15.82:0/2035596798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 msgr2=0x55570b1122e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:41.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.166+0000 7f52ff7fe640 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 0x55570b1122e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.166+0000 7f52ff7fe640 1 -- 172.21.15.82:0/2035596798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 msgr2=0x55570b112820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:41.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.166+0000 7f52fffff640 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b112820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.166+0000 7f52ff7fe640 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b112820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.166+0000 7f52ff7fe640 1 -- 172.21.15.82:0/2035596798 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570ad823f0 con 0x55570b10b700 2024-04-21T08:51:41.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.166+0000 7f52fffff640 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b112820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:41.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.166+0000 7f52ff7fe640 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10b700 0x55570b111da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52f4004260 tx=0x7f52f40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da7c6d19d3cf481e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:41.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.184+0000 7f52e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:51:41.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.184+0000 7f52e2ffd640 1 -- 172.21.15.82:0/2035596798 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f400b3b0 con 0x55570b10b700 2024-04-21T08:51:41.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.184+0000 7f530876a740 1 -- 172.21.15.82:0/2035596798 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f00041d0 con 0x55570b10b700 2024-04-21T08:51:41.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.184+0000 7f52e2ffd640 1 -- 172.21.15.82:0/2035596798 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f52f401f460 con 0x55570b10b700 2024-04-21T08:51:41.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.184+0000 7f52e2ffd640 1 -- 172.21.15.82:0/2035596798 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f52f402d050 con 0x55570b10b700 2024-04-21T08:51:41.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.185+0000 7f530876a740 10 client.26095 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:41.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.185+0000 7f530876a740 1 -- 172.21.15.82:0/2035596798 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55570ad420f0 con 0x55570b10b700 2024-04-21T08:51:41.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.185+0000 7f52e2ffd640 1 -- 172.21.15.82:0/2035596798 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f00041d0 con 0x55570b10b700 2024-04-21T08:51:41.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.188+0000 7f52e2ffd640 1 -- 172.21.15.82:0/2035596798 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f52f4011770 con 0x55570b10b700 2024-04-21T08:51:41.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.189+0000 7f530876a740 10 client.26095 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:41.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.189+0000 7f530876a740 10 client.26095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:41.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.189+0000 7f530876a740 20 client.26095 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:41.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.189+0000 7f530876a740 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55570b136170 0x55570b1385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.189+0000 7f530876a740 4 client.26095 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:41.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.189+0000 7f530876a740 1 -- 172.21.15.82:0/2035596798 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55570b07dc00 con 0x55570b136170 2024-04-21T08:51:41.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.190+0000 7f52feffd640 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55570b136170 0x55570b1385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.190+0000 7f52feffd640 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55570b136170 0x55570b1385f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad064dd43dd9f68 server_cookie=cbfb07a0eb7af943 in_seq=0 out_seq=0 2024-04-21T08:51:41.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.191+0000 7f52e2ffd640 10 client.26095 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:41.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.196+0000 7f52e2ffd640 1 -- 172.21.15.82:0/2035596798 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55570b07dc00 con 0x55570b136170 2024-04-21T08:51:41.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.197+0000 7f52e2ffd640 10 client.26095 handle_command_reply: tid=0 2024-04-21T08:51:41.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.197+0000 7f530876a740 1 client.26095 shutdown 2024-04-21T08:51:41.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.276+0000 7f530876a740 1 -- 172.21.15.82:0/2035596798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10b700 msgr2=0x55570b111da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:41.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.276+0000 7f530876a740 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10b700 0x55570b111da0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52f4004260 tx=0x7f52f40046c0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.279+0000 7f530876a740 1 -- 172.21.15.82:0/2035596798 shutdown_connections 2024-04-21T08:51:41.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.279+0000 7f530876a740 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55570b136170 0x55570b1385f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.279+0000 7f530876a740 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b112820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.279+0000 7f530876a740 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 0x55570b1122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.279+0000 7f530876a740 1 --2- 172.21.15.82:0/2035596798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10b700 0x55570b111da0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.279+0000 7f530876a740 1 -- 172.21.15.82:0/2035596798 >> 172.21.15.82:0/2035596798 conn(0x55570b105e30 msgr2=0x55570b107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:41.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.279+0000 7f52e2ffd640 0 client.26095 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:41.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.279+0000 7f530876a740 1 -- 172.21.15.82:0/2035596798 shutdown_connections 2024-04-21T08:51:41.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.283+0000 7f530876a740 1 -- 172.21.15.82:0/2035596798 wait complete. 2024-04-21T08:51:41.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.283+0000 7f530876a740 20 client.26095 trim_cache size 0 max 16384 2024-04-21T08:51:41.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.544+0000 7f530876a740 1 Processor -- start 2024-04-21T08:51:41.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.544+0000 7f530876a740 1 -- start start 2024-04-21T08:51:41.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.545+0000 7f530876a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10b700 0x55570b1318c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.545+0000 7f530876a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10c020 0x55570b1106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.545+0000 7f530876a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b107990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.545+0000 7f52ff7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10b700 0x55570b1318c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.545+0000 7f52ff7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10b700 0x55570b1318c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34554/0 (socket says 172.21.15.82:34554) 2024-04-21T08:51:41.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.545+0000 7f52ff7fe640 1 -- 172.21.15.82:0/2918245704 learned_addr learned my addr 172.21.15.82:0/2918245704 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:41.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.545+0000 7f52feffd640 1 --2- 172.21.15.82:0/2918245704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10c020 0x55570b1106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.545+0000 7f530876a740 1 -- 172.21.15.82:0/2918245704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f53001b56d0 con 0x55570b114c00 2024-04-21T08:51:41.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.545+0000 7f530876a740 1 -- 172.21.15.82:0/2918245704 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f53001b7270 con 0x55570b10b700 2024-04-21T08:51:41.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.545+0000 7f530876a740 1 -- 172.21.15.82:0/2918245704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53001b5da0 con 0x55570b10c020 2024-04-21T08:51:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.546+0000 7f52feffd640 1 -- 172.21.15.82:0/2918245704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10b700 msgr2=0x55570b1318c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.546+0000 7f52feffd640 1 --2- 172.21.15.82:0/2918245704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10b700 0x55570b1318c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.546+0000 7f52feffd640 1 -- 172.21.15.82:0/2918245704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 msgr2=0x55570b107990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.546+0000 7f52feffd640 1 --2- 172.21.15.82:0/2918245704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.546+0000 7f52feffd640 1 -- 172.21.15.82:0/2918245704 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570b106770 con 0x55570b10c020 2024-04-21T08:51:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.546+0000 7f52feffd640 1 --2- 172.21.15.82:0/2918245704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10c020 0x55570b1106d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f52f0003b90 tx=0x7f52f00095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=228f6ae16129c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:41.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.564+0000 7f52ed7fa640 1 -- 172.21.15.82:0/2918245704 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0009710 con 0x55570b10c020 2024-04-21T08:51:41.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.564+0000 7f52ed7fa640 1 -- 172.21.15.82:0/2918245704 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f0031070 con 0x55570b10c020 2024-04-21T08:51:41.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.564+0000 7f52ed7fa640 1 -- 172.21.15.82:0/2918245704 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f002b220 con 0x55570b10c020 2024-04-21T08:51:41.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.565+0000 7f530876a740 1 -- 172.21.15.82:0/2918245704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10c020 msgr2=0x55570b1106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:41.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.566+0000 7f530876a740 1 --2- 172.21.15.82:0/2918245704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10c020 0x55570b1106d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f52f0003b90 tx=0x7f52f00095f0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.568+0000 7f530876a740 1 -- 172.21.15.82:0/2918245704 shutdown_connections 2024-04-21T08:51:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.568+0000 7f530876a740 1 --2- 172.21.15.82:0/2918245704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.568+0000 7f530876a740 1 --2- 172.21.15.82:0/2918245704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10c020 0x55570b1106d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.568+0000 7f530876a740 1 --2- 172.21.15.82:0/2918245704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10b700 0x55570b1318c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.568+0000 7f530876a740 1 -- 172.21.15.82:0/2918245704 >> 172.21.15.82:0/2918245704 conn(0x55570b105e30 msgr2=0x55570b107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.568+0000 7f530876a740 1 -- 172.21.15.82:0/2918245704 shutdown_connections 2024-04-21T08:51:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.568+0000 7f530876a740 1 -- 172.21.15.82:0/2918245704 wait complete. 2024-04-21T08:51:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.569+0000 7f530876a740 1 -- start start 2024-04-21T08:51:41.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.570+0000 7f530876a740 1 Processor -- start 2024-04-21T08:51:41.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.570+0000 7f530876a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10b700 0x55570b12dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.570+0000 7f530876a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 0x55570b10e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.570+0000 7f530876a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b10e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52ff7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10b700 0x55570b12dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52ff7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10b700 0x55570b12dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:35658/0 (socket says 172.21.15.82:35658) 2024-04-21T08:51:41.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f530876a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f53001b6ba0 con 0x55570b114c00 2024-04-21T08:51:41.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f530876a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f53001b64d0 con 0x55570b10c020 2024-04-21T08:51:41.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f530876a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5300068cd0 con 0x55570b10b700 2024-04-21T08:51:41.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52feffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 0x55570b10e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52feffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 0x55570b10e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:34568/0 (socket says 172.21.15.82:34568) 2024-04-21T08:51:41.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52feffd640 1 -- 172.21.15.82:0/1844985546 learned_addr learned my addr 172.21.15.82:0/1844985546 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:41.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f530876a740 10 client.0 _ll_register_callbacks cb 0x7f52f4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:41.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52fffff640 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b10e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52fffff640 1 -- 172.21.15.82:0/1844985546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10b700 msgr2=0x55570b12dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:41.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52fffff640 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10b700 0x55570b12dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52fffff640 1 -- 172.21.15.82:0/1844985546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 msgr2=0x55570b10e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:41.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52fffff640 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 0x55570b10e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52fffff640 1 -- 172.21.15.82:0/1844985546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570b10eec0 con 0x55570b114c00 2024-04-21T08:51:41.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52feffd640 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 0x55570b10e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:41.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.571+0000 7f52fffff640 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b10e980 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f52f80f07b0 tx=0x7f52f80be960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5397296a864c08 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:41.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.577+0000 7f52e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:51:41.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.577+0000 7f52e2ffd640 1 -- 172.21.15.82:0/1844985546 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f80d8060 con 0x55570b114c00 2024-04-21T08:51:41.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.577+0000 7f530876a740 1 -- 172.21.15.82:0/1844985546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f0031070 con 0x55570b114c00 2024-04-21T08:51:41.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.577+0000 7f52e2ffd640 1 -- 172.21.15.82:0/1844985546 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f52f80f0920 con 0x55570b114c00 2024-04-21T08:51:41.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.578+0000 7f52e2ffd640 1 -- 172.21.15.82:0/1844985546 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f52f80f0b20 con 0x55570b114c00 2024-04-21T08:51:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.579+0000 7f530876a740 10 client.17025 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.579+0000 7f530876a740 1 -- 172.21.15.82:0/1844985546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55570b19c440 con 0x55570b114c00 2024-04-21T08:51:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.580+0000 7f52e2ffd640 1 -- 172.21.15.82:0/1844985546 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f80bf000 con 0x55570b114c00 2024-04-21T08:51:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.580+0000 7f52e2ffd640 1 -- 172.21.15.82:0/1844985546 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f52f8004b40 con 0x55570b114c00 2024-04-21T08:51:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.580+0000 7f530876a740 10 client.17025 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.580+0000 7f530876a740 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.580+0000 7f530876a740 20 client.17025 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.580+0000 7f530876a740 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55570b19cb90 0x55570b19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.580+0000 7f530876a740 4 client.17025 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:51:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.580+0000 7f530876a740 1 -- 172.21.15.82:0/1844985546 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f52f0031070 con 0x55570b19cb90 2024-04-21T08:51:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.580+0000 7f52ff7fe640 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55570b19cb90 0x55570b19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.581+0000 7f52ff7fe640 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55570b19cb90 0x55570b19efb0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e1f66070649d60b server_cookie=22b7fdcab00ea2f2 in_seq=0 out_seq=0 2024-04-21T08:51:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.582+0000 7f52e2ffd640 10 client.17025 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.582+0000 7f52e2ffd640 1 -- 172.21.15.82:0/1844985546 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f52f0031070 con 0x55570b19cb90 2024-04-21T08:51:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.582+0000 7f52e2ffd640 10 client.17025 handle_command_reply: tid=0 2024-04-21T08:51:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.582+0000 7f530876a740 1 client.17025 shutdown 2024-04-21T08:51:41.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.607+0000 7f530876a740 1 -- 172.21.15.82:0/1844985546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 msgr2=0x55570b10e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:41.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.607+0000 7f530876a740 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b10e980 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f52f80f07b0 tx=0x7f52f80be960 comp rx=0 tx=0).stop 2024-04-21T08:51:41.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.607+0000 7f530876a740 1 -- 172.21.15.82:0/1844985546 shutdown_connections 2024-04-21T08:51:41.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.607+0000 7f530876a740 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55570b19cb90 0x55570b19efb0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.607+0000 7f530876a740 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55570b114c00 0x55570b10e980 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.607+0000 7f530876a740 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55570b10c020 0x55570b10e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.607+0000 7f530876a740 1 --2- 172.21.15.82:0/1844985546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55570b10b700 0x55570b12dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.607+0000 7f530876a740 1 -- 172.21.15.82:0/1844985546 >> 172.21.15.82:0/1844985546 conn(0x55570b105e30 msgr2=0x55570b104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:41.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.607+0000 7f52e2ffd640 0 client.17025 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:41.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.607+0000 7f530876a740 1 -- 172.21.15.82:0/1844985546 shutdown_connections 2024-04-21T08:51:41.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.608+0000 7f530876a740 1 -- 172.21.15.82:0/1844985546 wait complete. 2024-04-21T08:51:41.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.608+0000 7f530876a740 20 client.17025 trim_cache size 0 max 16384 2024-04-21T08:51:41.609 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:51:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:51:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:51:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "67068658-7e18-4c0e-a274-59f6fb85db36": { 2024-04-21T08:51:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:51:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "67068658-7e18-4c0e-a274-59f6fb85db36", 2024-04-21T08:51:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:51:41.609 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:51:41.610 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:51:41.610 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:51:41.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.614+0000 7f5305b7e640 1 -- 172.21.15.82:0/2993782241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f52e80a3f20 msgr2=0x7f52e80a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:41.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.614+0000 7f5305b7e640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f52e80a3f20 0x7f52e80a63e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f52f4005fd0 tx=0x7f52f400f730 comp rx=0 tx=0).stop 2024-04-21T08:51:41.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.614+0000 7f5305b7e640 1 -- 172.21.15.82:0/2993782241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5300110550 msgr2=0x7f53001abbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:41.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.614+0000 7f5305b7e640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5300110550 0x7f53001abbd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f52f8006010 tx=0x7f52f802cbb0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.615+0000 7f5305b7e640 1 -- 172.21.15.82:0/2993782241 shutdown_connections 2024-04-21T08:51:41.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.615+0000 7f5305b7e640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5300110550 0x7f53001abbd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.615+0000 7f5305b7e640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f530010eb40 0x7f53001a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.615+0000 7f5305b7e640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f52e80a3f20 0x7f52e80a63e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.615+0000 7f5305b7e640 1 --2- 172.21.15.82:0/2993782241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530010e0c0 0x7f53001a7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:41.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.615+0000 7f5305b7e640 1 -- 172.21.15.82:0/2993782241 >> 172.21.15.82:0/2993782241 conn(0x7f530006ce60 msgr2=0x7f53000712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:41.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.615+0000 7f5305b7e640 1 -- 172.21.15.82:0/2993782241 shutdown_connections 2024-04-21T08:51:41.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:51:41.615+0000 7f5305b7e640 1 -- 172.21.15.82:0/2993782241 wait complete. 2024-04-21T08:51:41.667 INFO:tasks.cephfs.filesystem:scrub status for tag:67068658-7e18-4c0e-a274-59f6fb85db36 - {'path': '/', 'tag': '67068658-7e18-4c0e-a274-59f6fb85db36', 'options': 'recursive,force'} 2024-04-21T08:51:41.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:41 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:51:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:41 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:41 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:51:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:41 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:51:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:41 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:41 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:51:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:41 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:51:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:41 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:41 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:51:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:41 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:41 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:51:41.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:41 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:41 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:51:41.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:41 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:41 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:51:41.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:41 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:51:41.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:41 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:41 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:51:41.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:41 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:51:41.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:41 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:41 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:51:41.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:41 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:41 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:51:41.879 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:41 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:41 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:51:41.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:41 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:41 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:51:41.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:41 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:51:41.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:41 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:41 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:51:41.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:41 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:51:41.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:41 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:41 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:51:41.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:41 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:41.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:41 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:51:41.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:41 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: pgmap v143: 129 pgs: 129 active+clean; 506 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 236 op/s 2024-04-21T08:51:42.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:51:42.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:51:42.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:51:42.780 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.780 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:51:42.780 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:51:42.780 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:51:42.780 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.780 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.780 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:42 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: pgmap v143: 129 pgs: 129 active+clean; 506 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 236 op/s 2024-04-21T08:51:42.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:51:42.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:51:42.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:51:42.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:51:42.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:51:42.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:51:42.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:42 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: pgmap v143: 129 pgs: 129 active+clean; 506 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 236 op/s 2024-04-21T08:51:42.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:51:42.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:51:42.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:51:42.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:51:42.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:51:42.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:51:42.878 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.878 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:42.878 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:42 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:44.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:44 smithi082 ceph-mon[27967]: pgmap v144: 129 pgs: 129 active+clean; 551 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.3 MiB/s wr, 579 op/s 2024-04-21T08:51:44.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:44 smithi106 ceph-mon[31072]: pgmap v144: 129 pgs: 129 active+clean; 551 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.3 MiB/s wr, 579 op/s 2024-04-21T08:51:44.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:44 smithi134 ceph-mon[31772]: pgmap v144: 129 pgs: 129 active+clean; 551 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.3 MiB/s wr, 579 op/s 2024-04-21T08:51:46.231 INFO:tasks.workunit.client.0.smithi082.stdout: 96 1526411 298 1070364 248 804351 673 2024-04-21T08:51:46.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:46 smithi082 ceph-mon[27967]: pgmap v145: 129 pgs: 129 active+clean; 551 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 492 op/s 2024-04-21T08:51:46.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:46 smithi106 ceph-mon[31072]: pgmap v145: 129 pgs: 129 active+clean; 551 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 492 op/s 2024-04-21T08:51:46.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:46 smithi134 ceph-mon[31772]: pgmap v145: 129 pgs: 129 active+clean; 551 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 492 op/s 2024-04-21T08:51:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.660+0000 7fbfbe7fc640 1 -- 172.21.15.82:0/3387546989 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8035710 con 0x7fbfc010ece0 2024-04-21T08:51:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.660+0000 7fbfc5dff640 1 -- 172.21.15.82:0/3387546989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc010ece0 msgr2=0x7fbfc010d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.660+0000 7fbfc5dff640 1 --2- 172.21.15.82:0/3387546989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc010ece0 0x7fbfc010d000 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800afd0 tx=0x7fbfb802dd60 comp rx=0 tx=0).stop 2024-04-21T08:51:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.660+0000 7fbfc5dff640 1 -- 172.21.15.82:0/3387546989 shutdown_connections 2024-04-21T08:51:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.660+0000 7fbfc5dff640 1 --2- 172.21.15.82:0/3387546989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfc010d730 0x7fbfc007a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.660+0000 7fbfc5dff640 1 --2- 172.21.15.82:0/3387546989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc010ece0 0x7fbfc010d000 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.660+0000 7fbfc5dff640 1 --2- 172.21.15.82:0/3387546989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc010e330 0x7fbfc010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.660+0000 7fbfc5dff640 1 -- 172.21.15.82:0/3387546989 >> 172.21.15.82:0/3387546989 conn(0x7fbfc006cf00 msgr2=0x7fbfc006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.662+0000 7fbfc5dff640 1 -- 172.21.15.82:0/3387546989 shutdown_connections 2024-04-21T08:51:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.662+0000 7fbfc5dff640 1 -- 172.21.15.82:0/3387546989 wait complete. 2024-04-21T08:51:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.663+0000 7fbfc5dff640 1 Processor -- start 2024-04-21T08:51:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.663+0000 7fbfc5dff640 1 -- start start 2024-04-21T08:51:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.663+0000 7fbfc5dff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfc010d730 0x7fbfc0083090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.663+0000 7fbfc5dff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc010e330 0x7fbfc00835d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.663+0000 7fbfc5dff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc0083b10 0x7fbfc01bffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.663+0000 7fbfc5dff640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc007d5d0 con 0x7fbfc0083b10 2024-04-21T08:51:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.663+0000 7fbfc5dff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc007d450 con 0x7fbfc010d730 2024-04-21T08:51:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.663+0000 7fbfc5dff640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc007d750 con 0x7fbfc010e330 2024-04-21T08:51:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.664+0000 7fbfbf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfc010d730 0x7fbfc0083090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.664+0000 7fbfbf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfc010d730 0x7fbfc0083090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:59622/0 (socket says 172.21.15.82:59622) 2024-04-21T08:51:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.664+0000 7fbfbeffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc010e330 0x7fbfc00835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.664+0000 7fbfbf7fe640 1 -- 172.21.15.82:0/1733757424 learned_addr learned my addr 172.21.15.82:0/1733757424 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.664+0000 7fbfbffff640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc0083b10 0x7fbfc01bffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.664+0000 7fbfbeffd640 1 -- 172.21.15.82:0/1733757424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfc010d730 msgr2=0x7fbfc0083090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.664+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfc010d730 0x7fbfc0083090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.664+0000 7fbfbeffd640 1 -- 172.21.15.82:0/1733757424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc0083b10 msgr2=0x7fbfc01bffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.664+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc0083b10 0x7fbfc01bffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.664+0000 7fbfbeffd640 1 -- 172.21.15.82:0/1733757424 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc01c06f0 con 0x7fbfc010e330 2024-04-21T08:51:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.664+0000 7fbfbffff640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc0083b10 0x7fbfc01bffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.664+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc010e330 0x7fbfc00835d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8007bd0 tx=0x7fbfb8035c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.665+0000 7fbfbcff9640 1 -- 172.21.15.82:0/1733757424 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8045070 con 0x7fbfc010e330 2024-04-21T08:51:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.665+0000 7fbfc5dff640 1 -- 172.21.15.82:0/1733757424 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc01c0980 con 0x7fbfc010e330 2024-04-21T08:51:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.665+0000 7fbfc5dff640 1 -- 172.21.15.82:0/1733757424 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc01c0ef0 con 0x7fbfc010e330 2024-04-21T08:51:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.665+0000 7fbfbcff9640 1 -- 172.21.15.82:0/1733757424 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb8035e30 con 0x7fbfc010e330 2024-04-21T08:51:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.666+0000 7fbfbcff9640 1 -- 172.21.15.82:0/1733757424 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8045070 con 0x7fbfc010e330 2024-04-21T08:51:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.669+0000 7fbfbcff9640 1 -- 172.21.15.82:0/1733757424 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbfb8004040 con 0x7fbfc010e330 2024-04-21T08:51:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.669+0000 7fbfc89f2740 1 Processor -- start 2024-04-21T08:51:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.671+0000 7fbfbcff9640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbfa80a4150 0x7fbfa80a6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.671+0000 7fbfbcff9640 1 -- 172.21.15.82:0/1733757424 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fbfb803d070 con 0x7fbfc010e330 2024-04-21T08:51:48.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.672+0000 7fbfbffff640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbfa80a4150 0x7fbfa80a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.672+0000 7fbfc89f2740 1 -- start start 2024-04-21T08:51:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.672+0000 7fbfc89f2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c137ac0 0x56183c136810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.673+0000 7fbfbffff640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbfa80a4150 0x7fbfa80a6610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0006fd0 tx=0x7fbfb0008040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.673+0000 7fbfc89f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 0x56183c1371d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.673+0000 7fbfc89f2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 0x56183c1426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.673+0000 7fbfbf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c137ac0 0x56183c136810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.673+0000 7fbfbf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c137ac0 0x56183c136810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:59628/0 (socket says 172.21.15.82:59628) 2024-04-21T08:51:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.673+0000 7fbfbf7fe640 1 -- 172.21.15.82:0/3294545085 learned_addr learned my addr 172.21.15.82:0/3294545085 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.673+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/3294545085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 0x56183c1371d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.674+0000 7fbfbffff640 1 --2- 172.21.15.82:0/3294545085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 0x56183c1426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.674+0000 7fbfc89f2740 1 -- 172.21.15.82:0/3294545085 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56183c09b8d0 con 0x56183c1402d0 2024-04-21T08:51:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.674+0000 7fbfc89f2740 1 -- 172.21.15.82:0/3294545085 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56183c038e80 con 0x56183c137ac0 2024-04-21T08:51:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.674+0000 7fbfc89f2740 1 -- 172.21.15.82:0/3294545085 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56183c097890 con 0x56183c136d50 2024-04-21T08:51:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.674+0000 7fbfbeffd640 1 -- 172.21.15.82:0/3294545085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c137ac0 msgr2=0x56183c136810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.674+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/3294545085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c137ac0 0x56183c136810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.674+0000 7fbfbeffd640 1 -- 172.21.15.82:0/3294545085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 msgr2=0x56183c1426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.674+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/3294545085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 0x56183c1426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.674+0000 7fbfbeffd640 1 -- 172.21.15.82:0/3294545085 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56183c05a320 con 0x56183c136d50 2024-04-21T08:51:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.675+0000 7fbfbffff640 1 --2- 172.21.15.82:0/3294545085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 0x56183c1426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.675+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/3294545085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 0x56183c1371d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80ee570 tx=0x7fbfb80bc990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c578ff93ffc5f7d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.675+0000 7fbfa5ffb640 1 -- 172.21.15.82:0/3294545085 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8045070 con 0x56183c136d50 2024-04-21T08:51:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.675+0000 7fbfa5ffb640 1 -- 172.21.15.82:0/3294545085 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb80bf030 con 0x56183c136d50 2024-04-21T08:51:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.676+0000 7fbfc89f2740 1 -- 172.21.15.82:0/3294545085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 msgr2=0x56183c1371d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.676+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/3294545085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 0x56183c1371d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80ee570 tx=0x7fbfb80bc990 comp rx=0 tx=0).stop 2024-04-21T08:51:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.676+0000 7fbfc89f2740 1 -- 172.21.15.82:0/3294545085 shutdown_connections 2024-04-21T08:51:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.676+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/3294545085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 0x56183c1426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.676+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/3294545085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 0x56183c1371d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.676+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/3294545085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c137ac0 0x56183c136810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.676+0000 7fbfc89f2740 1 -- 172.21.15.82:0/3294545085 >> 172.21.15.82:0/3294545085 conn(0x56183c0fdc90 msgr2=0x56183c131a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.677+0000 7fbfc89f2740 1 -- 172.21.15.82:0/3294545085 shutdown_connections 2024-04-21T08:51:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.677+0000 7fbfc89f2740 1 -- 172.21.15.82:0/3294545085 wait complete. 2024-04-21T08:51:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.677+0000 7fbfc89f2740 1 -- start start 2024-04-21T08:51:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.677+0000 7fbfc89f2740 1 Processor -- start 2024-04-21T08:51:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfc89f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 0x56183c13fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfc89f2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c137ac0 0x56183c13cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfc89f2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c1402d0 0x56183c13d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfc89f2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56183c142dc0 con 0x56183c137ac0 2024-04-21T08:51:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfc89f2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56183c023eb0 con 0x56183c1402d0 2024-04-21T08:51:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfc89f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56183c142f40 con 0x56183c136d50 2024-04-21T08:51:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfc89f2740 10 client.0 _ll_register_callbacks cb 0x56183c050b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfbf7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 0x56183c13fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfbf7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 0x56183c13fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:37938/0 (socket says 172.21.15.82:37938) 2024-04-21T08:51:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfbf7fe640 1 -- 172.21.15.82:0/2865218372 learned_addr learned my addr 172.21.15.82:0/2865218372 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c137ac0 0x56183c13cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfbffff640 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c1402d0 0x56183c13d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfbf7fe640 1 -- 172.21.15.82:0/2865218372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c1402d0 msgr2=0x56183c13d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfbf7fe640 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c1402d0 0x56183c13d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.678+0000 7fbfbf7fe640 1 -- 172.21.15.82:0/2865218372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c137ac0 msgr2=0x56183c13cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.679+0000 7fbfbf7fe640 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c137ac0 0x56183c13cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.679+0000 7fbfbf7fe640 1 -- 172.21.15.82:0/2865218372 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56183bdae210 con 0x56183c136d50 2024-04-21T08:51:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.679+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c137ac0 0x56183c13cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.679+0000 7fbfbf7fe640 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 0x56183c13fc00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4012980 tx=0x7fbfb4012d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d1e2e4f5c6d7798 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.679+0000 7fbfa37f6640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:51:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.679+0000 7fbfa37f6640 1 -- 172.21.15.82:0/2865218372 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400f920 con 0x56183c136d50 2024-04-21T08:51:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.679+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2865218372 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb80bf030 con 0x56183c136d50 2024-04-21T08:51:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.679+0000 7fbfa37f6640 1 -- 172.21.15.82:0/2865218372 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fbfb400fac0 con 0x56183c136d50 2024-04-21T08:51:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.680+0000 7fbfa37f6640 1 -- 172.21.15.82:0/2865218372 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbfb400fcc0 con 0x56183c136d50 2024-04-21T08:51:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.680+0000 7fbfc89f2740 10 client.26113 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.680+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2865218372 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56183be7a620 con 0x56183c136d50 2024-04-21T08:51:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.680+0000 7fbfa37f6640 1 -- 172.21.15.82:0/2865218372 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb80bf030 con 0x56183c136d50 2024-04-21T08:51:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.681+0000 7fbfa37f6640 1 -- 172.21.15.82:0/2865218372 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fbfb4010430 con 0x56183c136d50 2024-04-21T08:51:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.681+0000 7fbfc89f2740 10 client.26113 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.681+0000 7fbfc89f2740 10 client.26113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.681+0000 7fbfc89f2740 20 client.26113 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.681+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56183c147580 0x56183c1499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.681+0000 7fbfc89f2740 4 client.26113 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.681+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2865218372 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56183c0af880 con 0x56183c147580 2024-04-21T08:51:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.681+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56183c147580 0x56183c1499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.682+0000 7fbfa37f6640 10 client.26113 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.682+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56183c147580 0x56183c1499f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef5a069e418671d server_cookie=232f2bc64f09a09f in_seq=0 out_seq=0 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfa37f6640 1 -- 172.21.15.82:0/2865218372 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56183c0af880 con 0x56183c147580 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfa37f6640 10 client.26113 handle_command_reply: tid=0 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfc89f2740 1 client.26113 shutdown 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2865218372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 msgr2=0x56183c13fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 0x56183c13fc00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4012980 tx=0x7fbfb4012d80 comp rx=0 tx=0).stop 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2865218372 shutdown_connections 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56183c147580 0x56183c1499f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c1402d0 0x56183c13d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfa37f6640 0 client.26113 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c137ac0 0x56183c13cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/2865218372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c136d50 0x56183c13fc00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2865218372 >> 172.21.15.82:0/2865218372 conn(0x56183c0fdc90 msgr2=0x56183c133790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2865218372 shutdown_connections 2024-04-21T08:51:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2865218372 wait complete. 2024-04-21T08:51:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.683+0000 7fbfc89f2740 20 client.26113 trim_cache size 0 max 16384 2024-04-21T08:51:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.710+0000 7fbfc89f2740 1 Processor -- start 2024-04-21T08:51:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.710+0000 7fbfc89f2740 1 -- start start 2024-04-21T08:51:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfc89f2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c136d50 0x56183c13f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfc89f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c137ac0 0x56183c13fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfc89f2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 0x56183c16ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfc89f2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac00ffb0 con 0x56183c1402d0 2024-04-21T08:51:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfc89f2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac011b50 con 0x56183c136d50 2024-04-21T08:51:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfc89f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac010680 con 0x56183c137ac0 2024-04-21T08:51:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 0x56183c16ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c136d50 0x56183c13f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 0x56183c16ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44622/0 (socket says 172.21.15.82:44622) 2024-04-21T08:51:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c136d50 0x56183c13f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:59642/0 (socket says 172.21.15.82:59642) 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbffff640 1 -- 172.21.15.82:0/1337834908 learned_addr learned my addr 172.21.15.82:0/1337834908 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/1337834908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c137ac0 0x56183c13fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbeffd640 1 -- 172.21.15.82:0/1337834908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c136d50 msgr2=0x56183c13f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/1337834908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c136d50 0x56183c13f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbeffd640 1 -- 172.21.15.82:0/1337834908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 msgr2=0x56183c16ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/1337834908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 0x56183c16ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbeffd640 1 -- 172.21.15.82:0/1337834908 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56183c13b4e0 con 0x56183c137ac0 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbffff640 1 --2- 172.21.15.82:0/1337834908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 0x56183c16ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbf7fe640 1 --2- 172.21.15.82:0/1337834908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c136d50 0x56183c13f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/1337834908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c137ac0 0x56183c13fc00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80bcb50 tx=0x7fbfb80dd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56d9415d47411d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.711+0000 7fbfa57fa640 1 -- 172.21.15.82:0/1337834908 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8045040 con 0x56183c137ac0 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.712+0000 7fbfa57fa640 1 -- 172.21.15.82:0/1337834908 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb80bf040 con 0x56183c137ac0 2024-04-21T08:51:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.712+0000 7fbfa57fa640 1 -- 172.21.15.82:0/1337834908 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80c5a30 con 0x56183c137ac0 2024-04-21T08:51:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.712+0000 7fbfc89f2740 1 -- 172.21.15.82:0/1337834908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c137ac0 msgr2=0x56183c13fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.712+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/1337834908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c137ac0 0x56183c13fc00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80bcb50 tx=0x7fbfb80dd040 comp rx=0 tx=0).stop 2024-04-21T08:51:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 -- 172.21.15.82:0/1337834908 shutdown_connections 2024-04-21T08:51:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/1337834908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c1402d0 0x56183c16ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/1337834908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c137ac0 0x56183c13fc00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/1337834908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c136d50 0x56183c13f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 -- 172.21.15.82:0/1337834908 >> 172.21.15.82:0/1337834908 conn(0x56183c0fdc90 msgr2=0x56183c13cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 -- 172.21.15.82:0/1337834908 shutdown_connections 2024-04-21T08:51:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 -- 172.21.15.82:0/1337834908 wait complete. 2024-04-21T08:51:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 -- start start 2024-04-21T08:51:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 Processor -- start 2024-04-21T08:51:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c136d50 0x56183c14b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c137ac0 0x56183c14b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c1402d0 0x56183c1c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac011480 con 0x56183c136d50 2024-04-21T08:51:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac010db0 con 0x56183c1402d0 2024-04-21T08:51:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.713+0000 7fbfc89f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac00df90 con 0x56183c137ac0 2024-04-21T08:51:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfc89f2740 10 client.0 _ll_register_callbacks cb 0x7fbfb40099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfbffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c1402d0 0x56183c1c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfbffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c1402d0 0x56183c1c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:59656/0 (socket says 172.21.15.82:59656) 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfbffff640 1 -- 172.21.15.82:0/2034734992 learned_addr learned my addr 172.21.15.82:0/2034734992 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfbf7fe640 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c136d50 0x56183c14b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfbeffd640 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c137ac0 0x56183c14b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfbffff640 1 -- 172.21.15.82:0/2034734992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c137ac0 msgr2=0x56183c14b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfbffff640 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c137ac0 0x56183c14b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfbffff640 1 -- 172.21.15.82:0/2034734992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c136d50 msgr2=0x56183c14b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfbffff640 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c136d50 0x56183c14b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfbffff640 1 -- 172.21.15.82:0/2034734992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56183c1c68f0 con 0x56183c1402d0 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfbffff640 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c1402d0 0x56183c1c6380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00146e0 tx=0x7fbfb0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4016051dcafcdb5c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfa37f6640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfa37f6640 1 -- 172.21.15.82:0/2034734992 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0026070 con 0x56183c1402d0 2024-04-21T08:51:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfa37f6640 1 -- 172.21.15.82:0/2034734992 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fbfb000f040 con 0x56183c1402d0 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.714+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2034734992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb80bf040 con 0x56183c1402d0 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.715+0000 7fbfa37f6640 1 -- 172.21.15.82:0/2034734992 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbfb0032050 con 0x56183c1402d0 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.715+0000 7fbfc89f2740 10 client.26390 fetch_fsmap learned FSMap version 35 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.715+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2034734992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56183c1c7c60 con 0x56183c1402d0 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.715+0000 7fbfa37f6640 1 -- 172.21.15.82:0/2034734992 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb80bf040 con 0x56183c1402d0 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.715+0000 7fbfa37f6640 1 -- 172.21.15.82:0/2034734992 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fbfb0013070 con 0x56183c1402d0 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.715+0000 7fbfc89f2740 10 client.26390 fetch_fsmap finished waiting for FSMap version 35 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.715+0000 7fbfc89f2740 10 client.26390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.715+0000 7fbfc89f2740 20 client.26390 populate_metadata read hostname 'smithi082' 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.715+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56183c1c8430 0x56183c1ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.715+0000 7fbfc89f2740 4 client.26390 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.715+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2034734992 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56183c136ae0 con 0x56183c1c8430 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.715+0000 7fbfbf7fe640 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56183c1c8430 0x56183c1ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.716+0000 7fbfa37f6640 10 client.26390 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:48.716+0000 7fbfbf7fe640 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56183c1c8430 0x56183c1ca850 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=104c6a13d06103c server_cookie=8a3cad73824e973a in_seq=0 out_seq=0 2024-04-21T08:51:48.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:48 smithi082 ceph-mon[27967]: pgmap v146: 129 pgs: 129 active+clean; 569 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.8 MiB/s wr, 620 op/s 2024-04-21T08:51:48.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:48 smithi106 ceph-mon[31072]: pgmap v146: 129 pgs: 129 active+clean; 569 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.8 MiB/s wr, 620 op/s 2024-04-21T08:51:48.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:48 smithi134 ceph-mon[31772]: pgmap v146: 129 pgs: 129 active+clean; 569 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.8 MiB/s wr, 620 op/s 2024-04-21T08:51:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.674+0000 7fbfa37f6640 1 -- 172.21.15.82:0/2034734992 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56183c136ae0 con 0x56183c1c8430 2024-04-21T08:51:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.674+0000 7fbfa37f6640 10 client.26390 handle_command_reply: tid=0 2024-04-21T08:51:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.674+0000 7fbfc89f2740 1 client.26390 shutdown 2024-04-21T08:51:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.674+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2034734992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c1402d0 msgr2=0x56183c1c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.674+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c1402d0 0x56183c1c6380 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00146e0 tx=0x7fbfb0004750 comp rx=0 tx=0).stop 2024-04-21T08:51:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.675+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2034734992 shutdown_connections 2024-04-21T08:51:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.675+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56183c1c8430 0x56183c1ca850 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.675+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56183c1402d0 0x56183c1c6380 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.675+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56183c137ac0 0x56183c14b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.675+0000 7fbfa37f6640 0 client.26390 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:51:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.675+0000 7fbfc89f2740 1 --2- 172.21.15.82:0/2034734992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56183c136d50 0x56183c14b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:49.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.675+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2034734992 >> 172.21.15.82:0/2034734992 conn(0x56183c0fdc90 msgr2=0x56183c153dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.675+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2034734992 shutdown_connections 2024-04-21T08:51:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.675+0000 7fbfc89f2740 1 -- 172.21.15.82:0/2034734992 wait complete. 2024-04-21T08:51:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.675+0000 7fbfc89f2740 20 client.26390 trim_cache size 0 max 16384 2024-04-21T08:51:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:51:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:51:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:51:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:51:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.683+0000 7fbfc5dff640 1 -- 172.21.15.82:0/1733757424 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbfa80a4150 msgr2=0x7fbfa80a6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.683+0000 7fbfc5dff640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbfa80a4150 0x7fbfa80a6610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0006fd0 tx=0x7fbfb0008040 comp rx=0 tx=0).stop 2024-04-21T08:51:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.683+0000 7fbfc5dff640 1 -- 172.21.15.82:0/1733757424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc010e330 msgr2=0x7fbfc00835d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.683+0000 7fbfc5dff640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc010e330 0x7fbfc00835d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8007bd0 tx=0x7fbfb8035c40 comp rx=0 tx=0).stop 2024-04-21T08:51:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.684+0000 7fbfc5dff640 1 -- 172.21.15.82:0/1733757424 shutdown_connections 2024-04-21T08:51:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.684+0000 7fbfc5dff640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbfa80a4150 0x7fbfa80a6610 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.684+0000 7fbfc5dff640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc0083b10 0x7fbfc01bffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.684+0000 7fbfc5dff640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc010e330 0x7fbfc00835d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.684+0000 7fbfc5dff640 1 --2- 172.21.15.82:0/1733757424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbfc010d730 0x7fbfc0083090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.684+0000 7fbfc5dff640 1 -- 172.21.15.82:0/1733757424 >> 172.21.15.82:0/1733757424 conn(0x7fbfc006cf00 msgr2=0x7fbfc010b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.685+0000 7fbfc5dff640 1 -- 172.21.15.82:0/1733757424 shutdown_connections 2024-04-21T08:51:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:51:49.685+0000 7fbfc5dff640 1 -- 172.21.15.82:0/1733757424 wait complete. 2024-04-21T08:51:50.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:50 smithi082 ceph-mon[27967]: pgmap v147: 129 pgs: 129 active+clean; 590 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 641 op/s 2024-04-21T08:51:50.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:50 smithi106 ceph-mon[31072]: pgmap v147: 129 pgs: 129 active+clean; 590 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 641 op/s 2024-04-21T08:51:50.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:50 smithi134 ceph-mon[31772]: pgmap v147: 129 pgs: 129 active+clean; 590 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.3 MiB/s wr, 641 op/s 2024-04-21T08:51:52.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:52 smithi082 ceph-mon[27967]: pgmap v148: 129 pgs: 129 active+clean; 590 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.6 MiB/s wr, 572 op/s 2024-04-21T08:51:52.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:52 smithi106 ceph-mon[31072]: pgmap v148: 129 pgs: 129 active+clean; 590 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.6 MiB/s wr, 572 op/s 2024-04-21T08:51:52.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:52 smithi134 ceph-mon[31772]: pgmap v148: 129 pgs: 129 active+clean; 590 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.6 MiB/s wr, 572 op/s 2024-04-21T08:51:53.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:53 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:51:53.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:53 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:51:53.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:53 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:51:54.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:54 smithi082 ceph-mon[27967]: pgmap v149: 129 pgs: 129 active+clean; 602 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 703 op/s 2024-04-21T08:51:54.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:54 smithi106 ceph-mon[31072]: pgmap v149: 129 pgs: 129 active+clean; 602 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 703 op/s 2024-04-21T08:51:54.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:54 smithi134 ceph-mon[31772]: pgmap v149: 129 pgs: 129 active+clean; 602 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.7 MiB/s wr, 703 op/s 2024-04-21T08:51:56.231 INFO:tasks.workunit.client.0.smithi082.stdout: 102 1373193 448 963590 349 735714 1300 2024-04-21T08:51:56.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:56 smithi082 ceph-mon[27967]: pgmap v150: 129 pgs: 129 active+clean; 602 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 359 op/s 2024-04-21T08:51:56.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:56 smithi106 ceph-mon[31072]: pgmap v150: 129 pgs: 129 active+clean; 602 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 359 op/s 2024-04-21T08:51:56.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:56 smithi134 ceph-mon[31772]: pgmap v150: 129 pgs: 129 active+clean; 602 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 359 op/s 2024-04-21T08:51:57.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:57 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:57.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:57 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:57.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:57 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:51:58.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:51:58 smithi082 ceph-mon[27967]: pgmap v151: 129 pgs: 129 active+clean; 602 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 502 op/s 2024-04-21T08:51:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:51:58 smithi106 ceph-mon[31072]: pgmap v151: 129 pgs: 129 active+clean; 602 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 502 op/s 2024-04-21T08:51:58.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:51:58 smithi134 ceph-mon[31772]: pgmap v151: 129 pgs: 129 active+clean; 602 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 502 op/s 2024-04-21T08:52:00.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:00 smithi082 ceph-mon[27967]: pgmap v152: 129 pgs: 129 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 506 op/s 2024-04-21T08:52:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:00 smithi106 ceph-mon[31072]: pgmap v152: 129 pgs: 129 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 506 op/s 2024-04-21T08:52:00.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:00 smithi134 ceph-mon[31772]: pgmap v152: 129 pgs: 129 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 506 op/s 2024-04-21T08:52:02.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:02 smithi082 ceph-mon[27967]: pgmap v153: 129 pgs: 129 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 405 op/s 2024-04-21T08:52:02.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:02 smithi106 ceph-mon[31072]: pgmap v153: 129 pgs: 129 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 405 op/s 2024-04-21T08:52:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.941+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/2783601887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c0106170 msgr2=0x7fa9c010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.941+0000 7fa9c5fb6640 1 --2- 172.21.15.82:0/2783601887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c0106170 0x7fa9c010dd40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009ca0 tx=0x7fa9b802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.941+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/2783601887 shutdown_connections 2024-04-21T08:52:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.941+0000 7fa9c5fb6640 1 --2- 172.21.15.82:0/2783601887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c0106170 0x7fa9c010dd40 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.941+0000 7fa9c5fb6640 1 --2- 172.21.15.82:0/2783601887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0101aa0 0x7fa9c0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.941+0000 7fa9c5fb6640 1 --2- 172.21.15.82:0/2783601887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9c01010f0 0x7fa9c01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.941+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/2783601887 >> 172.21.15.82:0/2783601887 conn(0x7fa9c0077fd0 msgr2=0x7fa9c00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.941+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/2783601887 shutdown_connections 2024-04-21T08:52:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.941+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/2783601887 wait complete. 2024-04-21T08:52:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.942+0000 7fa9c5fb6640 1 Processor -- start 2024-04-21T08:52:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.942+0000 7fa9c5fb6640 1 -- start start 2024-04-21T08:52:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.942+0000 7fa9c5fb6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9c01010f0 0x7fa9c01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.942+0000 7fa9c5fb6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0106170 0x7fa9c01a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.942+0000 7fa9c5fb6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c019a6f0 0x7fa9c019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.942+0000 7fa9c5fb6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c010ffd0 con 0x7fa9c019a6f0 2024-04-21T08:52:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.942+0000 7fa9c5fb6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c010fe50 con 0x7fa9c01010f0 2024-04-21T08:52:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.942+0000 7fa9c5fb6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0110150 con 0x7fa9c0106170 2024-04-21T08:52:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9c01010f0 0x7fa9c01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9c01010f0 0x7fa9c01a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:54096/0 (socket says 172.21.15.82:54096) 2024-04-21T08:52:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bf7fe640 1 -- 172.21.15.82:0/1142142686 learned_addr learned my addr 172.21.15.82:0/1142142686 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9beffd640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0106170 0x7fa9c01a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bffff640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c019a6f0 0x7fa9c019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bf7fe640 1 -- 172.21.15.82:0/1142142686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0106170 msgr2=0x7fa9c01a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bf7fe640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0106170 0x7fa9c01a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bf7fe640 1 -- 172.21.15.82:0/1142142686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c019a6f0 msgr2=0x7fa9c019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bf7fe640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c019a6f0 0x7fa9c019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bf7fe640 1 -- 172.21.15.82:0/1142142686 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c019b2f0 con 0x7fa9c01010f0 2024-04-21T08:52:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9beffd640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0106170 0x7fa9c01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bffff640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c019a6f0 0x7fa9c019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bf7fe640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9c01010f0 0x7fa9c01a0630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400bfd0 tx=0x7fa9b400ae10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bcff9640 1 -- 172.21.15.82:0/1142142686 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4019070 con 0x7fa9c01010f0 2024-04-21T08:52:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bcff9640 1 -- 172.21.15.82:0/1142142686 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b4014410 con 0x7fa9c01010f0 2024-04-21T08:52:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/1142142686 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c01a73d0 con 0x7fa9c01010f0 2024-04-21T08:52:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.943+0000 7fa9bcff9640 1 -- 172.21.15.82:0/1142142686 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4013610 con 0x7fa9c01010f0 2024-04-21T08:52:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.944+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/1142142686 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c01a77d0 con 0x7fa9c01010f0 2024-04-21T08:52:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.947+0000 7fa9bcff9640 1 -- 172.21.15.82:0/1142142686 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa9b401c070 con 0x7fa9c01010f0 2024-04-21T08:52:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.948+0000 7fa9bcff9640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa9a80a3dc0 0x7fa9a80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.948+0000 7fa9c8ba9740 1 Processor -- start 2024-04-21T08:52:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.948+0000 7fa9bcff9640 1 -- 172.21.15.82:0/1142142686 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa9b40cfdd0 con 0x7fa9c01010f0 2024-04-21T08:52:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.948+0000 7fa9beffd640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa9a80a3dc0 0x7fa9a80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.949+0000 7fa9c8ba9740 1 -- start start 2024-04-21T08:52:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.949+0000 7fa9c8ba9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b69379e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.949+0000 7fa9c8ba9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937a370 0x564b6937a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.949+0000 7fa9beffd640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa9a80a3dc0 0x7fa9a80a6280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9c019bf40 tx=0x7fa9b000b410 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.949+0000 7fa9c8ba9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b693838f0 0x564b69385ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.949+0000 7fa9bf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b69379e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.949+0000 7fa9bf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b69379e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:54098/0 (socket says 172.21.15.82:54098) 2024-04-21T08:52:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.949+0000 7fa9bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b693838f0 0x564b69385ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b693838f0 0x564b69385ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41056/0 (socket says 172.21.15.82:41056) 2024-04-21T08:52:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9beffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937a370 0x564b6937a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9beffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937a370 0x564b6937a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:33294/0 (socket says 172.21.15.82:33294) 2024-04-21T08:52:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9bf7fe640 1 -- 172.21.15.82:0/2189155343 learned_addr learned my addr 172.21.15.82:0/2189155343 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2189155343 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564b692d68d0 con 0x564b693838f0 2024-04-21T08:52:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2189155343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564b69273e80 con 0x564b6937b0e0 2024-04-21T08:52:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2189155343 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564b692d2890 con 0x564b6937a370 2024-04-21T08:52:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9beffd640 1 -- 172.21.15.82:0/2189155343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 msgr2=0x564b69379e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9beffd640 1 --2- 172.21.15.82:0/2189155343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b69379e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9beffd640 1 -- 172.21.15.82:0/2189155343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b693838f0 msgr2=0x564b69385ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9beffd640 1 --2- 172.21.15.82:0/2189155343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b693838f0 0x564b69385ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9beffd640 1 -- 172.21.15.82:0/2189155343 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b69295320 con 0x564b6937a370 2024-04-21T08:52:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.950+0000 7fa9bffff640 1 --2- 172.21.15.82:0/2189155343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b693838f0 0x564b69385ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.951+0000 7fa9beffd640 1 --2- 172.21.15.82:0/2189155343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937a370 0x564b6937a7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005d00 tx=0x7fa9b0013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d75f854622d3074a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.951+0000 7fa9a5ffb640 1 -- 172.21.15.82:0/2189155343 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0004220 con 0x564b6937a370 2024-04-21T08:52:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.951+0000 7fa9a5ffb640 1 -- 172.21.15.82:0/2189155343 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b00043c0 con 0x564b6937a370 2024-04-21T08:52:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.952+0000 7fa9a5ffb640 1 -- 172.21.15.82:0/2189155343 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b001ca80 con 0x564b6937a370 2024-04-21T08:52:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.952+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2189155343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937a370 msgr2=0x564b6937a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.952+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/2189155343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937a370 0x564b6937a7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005d00 tx=0x7fa9b0013000 comp rx=0 tx=0).stop 2024-04-21T08:52:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.953+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2189155343 shutdown_connections 2024-04-21T08:52:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.953+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/2189155343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b693838f0 0x564b69385ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.953+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/2189155343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937a370 0x564b6937a7f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.953+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/2189155343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b69379e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.953+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2189155343 >> 172.21.15.82:0/2189155343 conn(0x564b69338c90 msgr2=0x564b69375060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.953+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2189155343 shutdown_connections 2024-04-21T08:52:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.953+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2189155343 wait complete. 2024-04-21T08:52:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.953+0000 7fa9c8ba9740 1 -- start start 2024-04-21T08:52:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.954+0000 7fa9c8ba9740 1 Processor -- start 2024-04-21T08:52:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.954+0000 7fa9c8ba9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937a370 0x564b69383280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.954+0000 7fa9c8ba9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b69380610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.954+0000 7fa9c8ba9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b693838f0 0x564b69380b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.954+0000 7fa9c8ba9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564b69386670 con 0x564b693838f0 2024-04-21T08:52:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.954+0000 7fa9c8ba9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564b6925eeb0 con 0x564b6937b0e0 2024-04-21T08:52:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.954+0000 7fa9c8ba9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564b693867f0 con 0x564b6937a370 2024-04-21T08:52:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.954+0000 7fa9c8ba9740 10 client.0 _ll_register_callbacks cb 0x564b6928bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.954+0000 7fa9beffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b69380610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.954+0000 7fa9beffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b69380610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:54114/0 (socket says 172.21.15.82:54114) 2024-04-21T08:52:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.954+0000 7fa9beffd640 1 -- 172.21.15.82:0/1656317282 learned_addr learned my addr 172.21.15.82:0/1656317282 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.954+0000 7fa9bffff640 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b693838f0 0x564b69380b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.955+0000 7fa9bf7fe640 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937a370 0x564b69383280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.955+0000 7fa9beffd640 1 -- 172.21.15.82:0/1656317282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937a370 msgr2=0x564b69383280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.955+0000 7fa9beffd640 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937a370 0x564b69383280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.955+0000 7fa9beffd640 1 -- 172.21.15.82:0/1656317282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b693838f0 msgr2=0x564b69380b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.955+0000 7fa9beffd640 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b693838f0 0x564b69380b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.955+0000 7fa9beffd640 1 -- 172.21.15.82:0/1656317282 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b68fe9210 con 0x564b6937b0e0 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.955+0000 7fa9beffd640 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b69380610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000bd60 tx=0x7fa9b00042b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb0fdbb71007eb26 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.955+0000 7fa99effd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.955+0000 7fa99effd640 1 -- 172.21.15.82:0/1656317282 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0036060 con 0x564b6937b0e0 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.955+0000 7fa99effd640 1 -- 172.21.15.82:0/1656317282 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa9b001cb90 con 0x564b6937b0e0 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.955+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/1656317282 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b00043c0 con 0x564b6937b0e0 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.956+0000 7fa99effd640 1 -- 172.21.15.82:0/1656317282 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa9b003d050 con 0x564b6937b0e0 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.956+0000 7fa9c8ba9740 10 client.26402 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.956+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/1656317282 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564b690b5620 con 0x564b6937b0e0 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.956+0000 7fa99effd640 1 -- 172.21.15.82:0/1656317282 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b00043c0 con 0x564b6937b0e0 2024-04-21T08:52:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.956+0000 7fa99effd640 1 -- 172.21.15.82:0/1656317282 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fa9b0031470 con 0x564b6937b0e0 2024-04-21T08:52:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.956+0000 7fa9c8ba9740 10 client.26402 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.956+0000 7fa9c8ba9740 10 client.26402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.956+0000 7fa9c8ba9740 20 client.26402 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.956+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564b693938d0 0x564b69395cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.956+0000 7fa9c8ba9740 4 client.26402 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.956+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/1656317282 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b692ea880 con 0x564b693938d0 2024-04-21T08:52:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.956+0000 7fa9bf7fe640 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564b693938d0 0x564b69395cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.957+0000 7fa99effd640 10 client.26402 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.957+0000 7fa9bf7fe640 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564b693938d0 0x564b69395cc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f4ef539b46455a server_cookie=ba7fdd5f5740a899 in_seq=0 out_seq=0 2024-04-21T08:52:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa99effd640 1 -- 172.21.15.82:0/1656317282 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b692ea880 con 0x564b693938d0 2024-04-21T08:52:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa99effd640 10 client.26402 handle_command_reply: tid=0 2024-04-21T08:52:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa9c8ba9740 1 client.26402 shutdown 2024-04-21T08:52:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/1656317282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 msgr2=0x564b69380610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b69380610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000bd60 tx=0x7fa9b00042b0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/1656317282 shutdown_connections 2024-04-21T08:52:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564b693938d0 0x564b69395cc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b693838f0 0x564b69380b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b69380610 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa99effd640 0 client.26402 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/1656317282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937a370 0x564b69383280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/1656317282 >> 172.21.15.82:0/1656317282 conn(0x564b69338c90 msgr2=0x564b69376db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/1656317282 shutdown_connections 2024-04-21T08:52:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/1656317282 wait complete. 2024-04-21T08:52:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.958+0000 7fa9c8ba9740 20 client.26402 trim_cache size 0 max 16384 2024-04-21T08:52:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.975+0000 7fa9c8ba9740 1 Processor -- start 2024-04-21T08:52:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9c8ba9740 1 -- start start 2024-04-21T08:52:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9c8ba9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 0x564b6938f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9c8ba9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b6937f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9c8ba9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b693838f0 0x564b69382d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9c8ba9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01ad1c0 con 0x564b6937a370 2024-04-21T08:52:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9c8ba9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01aed90 con 0x564b6937b0e0 2024-04-21T08:52:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9c8ba9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01ad890 con 0x564b693838f0 2024-04-21T08:52:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9bf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 0x564b6938f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9bf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 0x564b6938f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41066/0 (socket says 172.21.15.82:41066) 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9beffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b6937f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9bf7fe640 1 -- 172.21.15.82:0/453447969 learned_addr learned my addr 172.21.15.82:0/453447969 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9bffff640 1 --2- 172.21.15.82:0/453447969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b693838f0 0x564b69382d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9bf7fe640 1 -- 172.21.15.82:0/453447969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b693838f0 msgr2=0x564b69382d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9bf7fe640 1 --2- 172.21.15.82:0/453447969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b693838f0 0x564b69382d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9bf7fe640 1 -- 172.21.15.82:0/453447969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 msgr2=0x564b6937f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9bf7fe640 1 --2- 172.21.15.82:0/453447969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b6937f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.976+0000 7fa9bf7fe640 1 -- 172.21.15.82:0/453447969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b69375c30 con 0x564b6937a370 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.977+0000 7fa9bf7fe640 1 --2- 172.21.15.82:0/453447969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 0x564b6938f5c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4097ec0 tx=0x7fa9b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7149a2d16f6999 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.977+0000 7fa9a57fa640 1 -- 172.21.15.82:0/453447969 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b40a4620 con 0x564b6937a370 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.977+0000 7fa9a57fa640 1 -- 172.21.15.82:0/453447969 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b4099b00 con 0x564b6937a370 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.977+0000 7fa9a57fa640 1 -- 172.21.15.82:0/453447969 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b40148e0 con 0x564b6937a370 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.978+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/453447969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 msgr2=0x564b6938f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.978+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/453447969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 0x564b6938f5c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4097ec0 tx=0x7fa9b4000bc0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.978+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/453447969 shutdown_connections 2024-04-21T08:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.978+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/453447969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b693838f0 0x564b69382d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.978+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/453447969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b6937b0e0 0x564b6937f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.978+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/453447969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 0x564b6938f5c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.978+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/453447969 >> 172.21.15.82:0/453447969 conn(0x564b69338c90 msgr2=0x564b6937eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.978+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/453447969 shutdown_connections 2024-04-21T08:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.978+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/453447969 wait complete. 2024-04-21T08:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.978+0000 7fa9c8ba9740 1 -- start start 2024-04-21T08:52:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.978+0000 7fa9c8ba9740 1 Processor -- start 2024-04-21T08:52:02.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:02 smithi134 ceph-mon[31772]: pgmap v153: 129 pgs: 129 active+clean; 607 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 405 op/s 2024-04-21T08:52:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9c8ba9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 0x564b6938bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9c8ba9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937b0e0 0x564b6938c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9c8ba9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b693838f0 0x564b69387250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9c8ba9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01ae6c0 con 0x564b6937a370 2024-04-21T08:52:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9c8ba9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01adff0 con 0x564b693838f0 2024-04-21T08:52:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9c8ba9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c01ab1a0 con 0x564b6937b0e0 2024-04-21T08:52:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9bffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b693838f0 0x564b69387250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9bf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 0x564b6938bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9bffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b693838f0 0x564b69387250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:54132/0 (socket says 172.21.15.82:54132) 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9bf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 0x564b6938bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41072/0 (socket says 172.21.15.82:41072) 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9bffff640 1 -- 172.21.15.82:0/2060130719 learned_addr learned my addr 172.21.15.82:0/2060130719 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9c8ba9740 10 client.0 _ll_register_callbacks cb 0x7fa9b8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9beffd640 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937b0e0 0x564b6938c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9bffff640 1 -- 172.21.15.82:0/2060130719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937b0e0 msgr2=0x564b6938c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9bffff640 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937b0e0 0x564b6938c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9bffff640 1 -- 172.21.15.82:0/2060130719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 msgr2=0x564b6938bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9bffff640 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 0x564b6938bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9bffff640 1 -- 172.21.15.82:0/2060130719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b69387a50 con 0x564b693838f0 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9beffd640 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937b0e0 0x564b6938c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa9bffff640 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b693838f0 0x564b69387250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8037040 tx=0x7fa9b802cf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b516756656942a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.979+0000 7fa99effd640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:52:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa99effd640 1 -- 172.21.15.82:0/2060130719 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8039040 con 0x564b693838f0 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa99effd640 1 -- 172.21.15.82:0/2060130719 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa9b8003970 con 0x564b693838f0 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2060130719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b4099b00 con 0x564b693838f0 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa99effd640 1 -- 172.21.15.82:0/2060130719 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa9b8054050 con 0x564b693838f0 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa9c8ba9740 10 client.26408 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2060130719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564b693f98f0 con 0x564b693838f0 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa99effd640 1 -- 172.21.15.82:0/2060130719 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b8003b70 con 0x564b693838f0 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa99effd640 1 -- 172.21.15.82:0/2060130719 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fa9b804b480 con 0x564b693838f0 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa9c8ba9740 10 client.26408 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa9c8ba9740 10 client.26408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa9c8ba9740 20 client.26408 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564b693fa0a0 0x564b693fc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa9c8ba9740 4 client.26408 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:52:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.980+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2060130719 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa9b4099b00 con 0x564b693fa0a0 2024-04-21T08:52:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.981+0000 7fa9bf7fe640 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564b693fa0a0 0x564b693fc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.981+0000 7fa99effd640 10 client.26408 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:02.981+0000 7fa9bf7fe640 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564b693fa0a0 0x564b693fc4c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a78c4f133e6988 server_cookie=de2d880305c64e41 in_seq=0 out_seq=0 2024-04-21T08:52:03.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:03 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:52:03.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:03 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:52:03.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:03 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:52:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa99effd640 1 -- 172.21.15.82:0/2060130719 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa9b4099b00 con 0x564b693fa0a0 2024-04-21T08:52:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa99effd640 10 client.26408 handle_command_reply: tid=0 2024-04-21T08:52:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa9c8ba9740 1 client.26408 shutdown 2024-04-21T08:52:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2060130719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b693838f0 msgr2=0x564b69387250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b693838f0 0x564b69387250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8037040 tx=0x7fa9b802cf80 comp rx=0 tx=0).stop 2024-04-21T08:52:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2060130719 shutdown_connections 2024-04-21T08:52:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564b693fa0a0 0x564b693fc4c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564b693838f0 0x564b69387250 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b6937b0e0 0x564b6938c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa99effd640 0 client.26408 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa9c8ba9740 1 --2- 172.21.15.82:0/2060130719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b6937a370 0x564b6938bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2060130719 >> 172.21.15.82:0/2060130719 conn(0x564b69338c90 msgr2=0x564b6936b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.345+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2060130719 shutdown_connections 2024-04-21T08:52:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.346+0000 7fa9c8ba9740 1 -- 172.21.15.82:0/2060130719 wait complete. 2024-04-21T08:52:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.346+0000 7fa9c8ba9740 20 client.26408 trim_cache size 0 max 16384 2024-04-21T08:52:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:52:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:52:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:52:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:52:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.347+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/1142142686 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa9a80a3dc0 msgr2=0x7fa9a80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.347+0000 7fa9c5fb6640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa9a80a3dc0 0x7fa9a80a6280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9c019bf40 tx=0x7fa9b000b410 comp rx=0 tx=0).stop 2024-04-21T08:52:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.347+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/1142142686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9c01010f0 msgr2=0x7fa9c01a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.347+0000 7fa9c5fb6640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9c01010f0 0x7fa9c01a0630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400bfd0 tx=0x7fa9b400ae10 comp rx=0 tx=0).stop 2024-04-21T08:52:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.347+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/1142142686 shutdown_connections 2024-04-21T08:52:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.347+0000 7fa9c5fb6640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa9a80a3dc0 0x7fa9a80a6280 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.347+0000 7fa9c5fb6640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9c019a6f0 0x7fa9c019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.347+0000 7fa9c5fb6640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9c0106170 0x7fa9c01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.347+0000 7fa9c5fb6640 1 --2- 172.21.15.82:0/1142142686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9c01010f0 0x7fa9c01a0630 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.347+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/1142142686 >> 172.21.15.82:0/1142142686 conn(0x7fa9c0077fd0 msgr2=0x7fa9c00fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.347+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/1142142686 shutdown_connections 2024-04-21T08:52:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:04.347+0000 7fa9c5fb6640 1 -- 172.21.15.82:0/1142142686 wait complete. 2024-04-21T08:52:04.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:04 smithi082 ceph-mon[27967]: pgmap v154: 129 pgs: 129 active+clean; 616 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 591 op/s 2024-04-21T08:52:04.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:04 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:52:04.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:04 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:52:04.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:04 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:52:04.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:04 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:52:04.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:04 smithi106 ceph-mon[31072]: pgmap v154: 129 pgs: 129 active+clean; 616 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 591 op/s 2024-04-21T08:52:04.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:04 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:52:04.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:04 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:52:04.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:04 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:52:04.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:04 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:52:04.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:04 smithi134 ceph-mon[31772]: pgmap v154: 129 pgs: 129 active+clean; 616 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 591 op/s 2024-04-21T08:52:04.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:04 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:52:04.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:04 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:52:04.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:04 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:52:04.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:04 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:52:06.231 INFO:tasks.workunit.client.0.smithi082.stdout: 110 1232261 304 850661 302 668255 987 2024-04-21T08:52:06.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:06 smithi082 ceph-mon[27967]: pgmap v155: 129 pgs: 129 active+clean; 616 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 460 op/s 2024-04-21T08:52:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:06 smithi106 ceph-mon[31072]: pgmap v155: 129 pgs: 129 active+clean; 616 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 460 op/s 2024-04-21T08:52:06.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:06 smithi134 ceph-mon[31772]: pgmap v155: 129 pgs: 129 active+clean; 616 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 460 op/s 2024-04-21T08:52:08.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:08 smithi082 ceph-mon[27967]: pgmap v156: 129 pgs: 129 active+clean; 647 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 744 op/s 2024-04-21T08:52:08.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:08 smithi106 ceph-mon[31072]: pgmap v156: 129 pgs: 129 active+clean; 647 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 744 op/s 2024-04-21T08:52:08.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:08 smithi134 ceph-mon[31772]: pgmap v156: 129 pgs: 129 active+clean; 647 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.4 MiB/s wr, 744 op/s 2024-04-21T08:52:10.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:10 smithi082 ceph-mon[27967]: pgmap v157: 129 pgs: 129 active+clean; 669 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.1 MiB/s wr, 861 op/s 2024-04-21T08:52:10.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:10 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:52:10.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:10 smithi106 ceph-mon[31072]: pgmap v157: 129 pgs: 129 active+clean; 669 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.1 MiB/s wr, 861 op/s 2024-04-21T08:52:10.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:10 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:52:10.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:10 smithi134 ceph-mon[31772]: pgmap v157: 129 pgs: 129 active+clean; 669 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.1 MiB/s wr, 861 op/s 2024-04-21T08:52:10.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:10 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:52:11.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:52:11.669 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:52:11.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.774+0000 7fe505552640 1 -- 172.21.15.82:0/1786684496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe50010c500 msgr2=0x7fe50010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:11.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.774+0000 7fe505552640 1 --2- 172.21.15.82:0/1786684496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe50010c500 0x7fe50010e9a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009900 tx=0x7fe4f002bd60 comp rx=0 tx=0).stop 2024-04-21T08:52:11.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.774+0000 7fe505552640 1 -- 172.21.15.82:0/1786684496 shutdown_connections 2024-04-21T08:52:11.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.774+0000 7fe505552640 1 --2- 172.21.15.82:0/1786684496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe50010c500 0x7fe50010e9a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.774+0000 7fe505552640 1 --2- 172.21.15.82:0/1786684496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe500103ae0 0x7fe50010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.774+0000 7fe505552640 1 --2- 172.21.15.82:0/1786684496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5001031c0 0x7fe5001035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.774+0000 7fe505552640 1 -- 172.21.15.82:0/1786684496 >> 172.21.15.82:0/1786684496 conn(0x7fe500078000 msgr2=0x7fe500078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.774+0000 7fe505552640 1 -- 172.21.15.82:0/1786684496 shutdown_connections 2024-04-21T08:52:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.774+0000 7fe505552640 1 -- 172.21.15.82:0/1786684496 wait complete. 2024-04-21T08:52:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.774+0000 7fe505552640 1 Processor -- start 2024-04-21T08:52:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.774+0000 7fe505552640 1 -- start start 2024-04-21T08:52:11.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe505552640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5001031c0 0x7fe5001986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe505552640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe500103ae0 0x7fe500198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe505552640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe50010c500 0x7fe50019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe505552640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe500112190 con 0x7fe5001031c0 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe505552640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe500112010 con 0x7fe50010c500 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe505552640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe500112310 con 0x7fe500103ae0 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4feffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5001031c0 0x7fe5001986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4feffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5001031c0 0x7fe5001986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41082/0 (socket says 172.21.15.82:41082) 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4feffd640 1 -- 172.21.15.82:0/3757537852 learned_addr learned my addr 172.21.15.82:0/3757537852 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4ff7fe640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe50010c500 0x7fe50019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4fe7fc640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe500103ae0 0x7fe500198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4feffd640 1 -- 172.21.15.82:0/3757537852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe500103ae0 msgr2=0x7fe500198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4feffd640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe500103ae0 0x7fe500198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4feffd640 1 -- 172.21.15.82:0/3757537852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe50010c500 msgr2=0x7fe50019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4feffd640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe50010c500 0x7fe50019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4feffd640 1 -- 172.21.15.82:0/3757537852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50019d6a0 con 0x7fe5001031c0 2024-04-21T08:52:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4feffd640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5001031c0 0x7fe5001986b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800ebc0 tx=0x7fe4f800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4e7fff640 1 -- 172.21.15.82:0/3757537852 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8010630 con 0x7fe5001031c0 2024-04-21T08:52:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe505552640 1 -- 172.21.15.82:0/3757537852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50019d990 con 0x7fe5001031c0 2024-04-21T08:52:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4e7fff640 1 -- 172.21.15.82:0/3757537852 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f8010c80 con 0x7fe5001031c0 2024-04-21T08:52:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe4e7fff640 1 -- 172.21.15.82:0/3757537852 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8018b60 con 0x7fe5001031c0 2024-04-21T08:52:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.775+0000 7fe505552640 1 -- 172.21.15.82:0/3757537852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5001a5220 con 0x7fe5001031c0 2024-04-21T08:52:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.777+0000 7fe4e7fff640 1 -- 172.21.15.82:0/3757537852 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4f8018d00 con 0x7fe5001031c0 2024-04-21T08:52:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe508221740 1 Processor -- start 2024-04-21T08:52:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe508221740 1 -- start start 2024-04-21T08:52:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe508221740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaac580 0x560e1aaac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe508221740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaacea0 0x560e1aab5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe508221740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 0x560e1aab7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe508221740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560e1aa111e0 con 0x560e1aab5a80 2024-04-21T08:52:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe508221740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560e1a9ae390 con 0x560e1aaacea0 2024-04-21T08:52:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe508221740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560e1aa0d1a0 con 0x560e1aaac580 2024-04-21T08:52:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe4e7fff640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe4e80a3d20 0x7fe4e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe4fe7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 0x560e1aab7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe4fe7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 0x560e1aab7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41092/0 (socket says 172.21.15.82:41092) 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe4fe7fc640 1 -- 172.21.15.82:0/943719442 learned_addr learned my addr 172.21.15.82:0/943719442 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe4feffd640 1 --2- 172.21.15.82:0/943719442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaacea0 0x560e1aab5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.778+0000 7fe4ff7fe640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe4e80a3d20 0x7fe4e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4ff7fe640 1 --2- 172.21.15.82:0/943719442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaac580 0x560e1aaac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4e7fff640 1 -- 172.21.15.82:0/3757537852 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe4f80d1270 con 0x7fe5001031c0 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4fe7fc640 1 -- 172.21.15.82:0/943719442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaac580 msgr2=0x560e1aaac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4fe7fc640 1 --2- 172.21.15.82:0/943719442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaac580 0x560e1aaac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4fe7fc640 1 -- 172.21.15.82:0/943719442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaacea0 msgr2=0x560e1aab5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4fe7fc640 1 --2- 172.21.15.82:0/943719442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaacea0 0x560e1aab5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4fe7fc640 1 -- 172.21.15.82:0/943719442 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1a7ed7d0 con 0x560e1aab5a80 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4ff7fe640 1 --2- 172.21.15.82:0/943719442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaac580 0x560e1aaac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4fe7fc640 1 --2- 172.21.15.82:0/943719442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 0x560e1aab7e70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009ae0 tx=0x7fe4f400e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6784e188ebd3ff08 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:11.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4ff7fe640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe4e80a3d20 0x7fe4e80a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0005ab0 tx=0x7fe4f0005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4e57fa640 1 -- 172.21.15.82:0/943719442 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4016070 con 0x560e1aab5a80 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4e57fa640 1 -- 172.21.15.82:0/943719442 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f400ecc0 con 0x560e1aab5a80 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.779+0000 7fe4e57fa640 1 -- 172.21.15.82:0/943719442 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40106a0 con 0x560e1aab5a80 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 -- 172.21.15.82:0/943719442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 msgr2=0x560e1aab7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 --2- 172.21.15.82:0/943719442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 0x560e1aab7e70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009ae0 tx=0x7fe4f400e2f0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 -- 172.21.15.82:0/943719442 shutdown_connections 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 --2- 172.21.15.82:0/943719442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 0x560e1aab7e70 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 --2- 172.21.15.82:0/943719442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaacea0 0x560e1aab5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 --2- 172.21.15.82:0/943719442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaac580 0x560e1aaac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 -- 172.21.15.82:0/943719442 >> 172.21.15.82:0/943719442 conn(0x560e1aaa6cb0 msgr2=0x560e1aaa7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 -- 172.21.15.82:0/943719442 shutdown_connections 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 -- 172.21.15.82:0/943719442 wait complete. 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 -- start start 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 Processor -- start 2024-04-21T08:52:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaac580 0x560e1aab2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 0x560e1aab2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aab5a80 0x560e1aab3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560e1aab8870 con 0x560e1aaacea0 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560e1a9ca2f0 con 0x560e1aab5a80 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560e1aab89f0 con 0x560e1aaac580 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.780+0000 7fe508221740 10 client.0 _ll_register_callbacks cb 0x560e1a9d7580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4fe7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 0x560e1aab2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4feffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaac580 0x560e1aab2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4fe7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 0x560e1aab2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41108/0 (socket says 172.21.15.82:41108) 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4ff7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aab5a80 0x560e1aab3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4fe7fc640 1 -- 172.21.15.82:0/2697454993 learned_addr learned my addr 172.21.15.82:0/2697454993 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4fe7fc640 1 -- 172.21.15.82:0/2697454993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaac580 msgr2=0x560e1aab2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4fe7fc640 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaac580 0x560e1aab2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4fe7fc640 1 -- 172.21.15.82:0/2697454993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aab5a80 msgr2=0x560e1aab3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4fe7fc640 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aab5a80 0x560e1aab3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4fe7fc640 1 -- 172.21.15.82:0/2697454993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1a7233f0 con 0x560e1aaacea0 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4ff7fe640 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aab5a80 0x560e1aab3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4feffd640 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaac580 0x560e1aab2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4fe7fc640 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 0x560e1aab2fe0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4011ee0 tx=0x7fe4f4005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b052931e3929a2b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4e2ff5640 1 -- 172.21.15.82:0/2697454993 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4028060 con 0x560e1aaacea0 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4e2ff5640 1 -- 172.21.15.82:0/2697454993 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe4f4011470 con 0x560e1aaacea0 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe508221740 1 -- 172.21.15.82:0/2697454993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f400ecc0 con 0x560e1aaacea0 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4e2ff5640 1 -- 172.21.15.82:0/2697454993 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe4f402f050 con 0x560e1aaacea0 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe508221740 10 client.17073 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe508221740 1 -- 172.21.15.82:0/2697454993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560e1a6e30f0 con 0x560e1aaacea0 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.781+0000 7fe4e2ff5640 1 -- 172.21.15.82:0/2697454993 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f400ecc0 con 0x560e1aaacea0 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.782+0000 7fe4e2ff5640 1 -- 172.21.15.82:0/2697454993 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fe4f400f040 con 0x560e1aaacea0 2024-04-21T08:52:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.782+0000 7fe508221740 10 client.17073 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.782+0000 7fe508221740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.782+0000 7fe508221740 20 client.17073 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.782+0000 7fe508221740 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560e1aac5960 0x560e1aac7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.782+0000 7fe508221740 4 client.17073 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.782+0000 7fe508221740 1 -- 172.21.15.82:0/2697454993 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e1aa1ec00 con 0x560e1aac5960 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.782+0000 7fe4feffd640 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560e1aac5960 0x560e1aac7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.782+0000 7fe4e2ff5640 10 client.17073 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.782+0000 7fe4feffd640 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560e1aac5960 0x560e1aac7d50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166a074e3c63a0d6 server_cookie=5196a5eb2c749137 in_seq=0 out_seq=0 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe4e2ff5640 1 -- 172.21.15.82:0/2697454993 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e1aa1ec00 con 0x560e1aac5960 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe4e2ff5640 10 client.17073 handle_command_reply: tid=0 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe508221740 1 client.17073 shutdown 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe508221740 1 -- 172.21.15.82:0/2697454993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 msgr2=0x560e1aab2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe508221740 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 0x560e1aab2fe0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4011ee0 tx=0x7fe4f4005750 comp rx=0 tx=0).stop 2024-04-21T08:52:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe508221740 1 -- 172.21.15.82:0/2697454993 shutdown_connections 2024-04-21T08:52:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe508221740 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560e1aac5960 0x560e1aac7d50 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe508221740 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aab5a80 0x560e1aab3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe508221740 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 0x560e1aab2fe0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe508221740 1 --2- 172.21.15.82:0/2697454993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaac580 0x560e1aab2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe508221740 1 -- 172.21.15.82:0/2697454993 >> 172.21.15.82:0/2697454993 conn(0x560e1aaa6cb0 msgr2=0x560e1aaa8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe4e2ff5640 0 client.17073 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe508221740 1 -- 172.21.15.82:0/2697454993 shutdown_connections 2024-04-21T08:52:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe508221740 1 -- 172.21.15.82:0/2697454993 wait complete. 2024-04-21T08:52:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.783+0000 7fe508221740 20 client.17073 trim_cache size 0 max 16384 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.801+0000 7fe508221740 1 Processor -- start 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.801+0000 7fe508221740 1 -- start start 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.801+0000 7fe508221740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaac580 0x560e1aac1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.801+0000 7fe508221740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaacea0 0x560e1aacd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.801+0000 7fe508221740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 0x560e1aacd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.801+0000 7fe508221740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001af3d0 con 0x560e1aab5a80 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.801+0000 7fe508221740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001b0fa0 con 0x560e1aaac580 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.801+0000 7fe508221740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001afaa0 con 0x560e1aaacea0 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4ff7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 0x560e1aacd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4ff7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 0x560e1aacd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41120/0 (socket says 172.21.15.82:41120) 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4ff7fe640 1 -- 172.21.15.82:0/102055998 learned_addr learned my addr 172.21.15.82:0/102055998 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4feffd640 1 --2- 172.21.15.82:0/102055998 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaac580 0x560e1aac1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4fe7fc640 1 --2- 172.21.15.82:0/102055998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaacea0 0x560e1aacd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4ff7fe640 1 -- 172.21.15.82:0/102055998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaacea0 msgr2=0x560e1aacd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4ff7fe640 1 --2- 172.21.15.82:0/102055998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaacea0 0x560e1aacd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4ff7fe640 1 -- 172.21.15.82:0/102055998 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaac580 msgr2=0x560e1aac1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4ff7fe640 1 --2- 172.21.15.82:0/102055998 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaac580 0x560e1aac1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4ff7fe640 1 -- 172.21.15.82:0/102055998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1aaad850 con 0x560e1aab5a80 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4fe7fc640 1 --2- 172.21.15.82:0/102055998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaacea0 0x560e1aacd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4ff7fe640 1 --2- 172.21.15.82:0/102055998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 0x560e1aacd8c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009900 tx=0x7fe4f00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ff1fcfff08f617 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4e4ff9640 1 -- 172.21.15.82:0/102055998 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0051070 con 0x560e1aab5a80 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4e4ff9640 1 -- 172.21.15.82:0/102055998 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f0045070 con 0x560e1aab5a80 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.802+0000 7fe4e4ff9640 1 -- 172.21.15.82:0/102055998 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0003850 con 0x560e1aab5a80 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 -- 172.21.15.82:0/102055998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 msgr2=0x560e1aacd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 --2- 172.21.15.82:0/102055998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 0x560e1aacd8c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009900 tx=0x7fe4f00058d0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 -- 172.21.15.82:0/102055998 shutdown_connections 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 --2- 172.21.15.82:0/102055998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aab5a80 0x560e1aacd8c0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 --2- 172.21.15.82:0/102055998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aaacea0 0x560e1aacd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 --2- 172.21.15.82:0/102055998 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaac580 0x560e1aac1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 -- 172.21.15.82:0/102055998 >> 172.21.15.82:0/102055998 conn(0x560e1aaa6cb0 msgr2=0x560e1aab1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 -- 172.21.15.82:0/102055998 shutdown_connections 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 -- 172.21.15.82:0/102055998 wait complete. 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 -- start start 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 Processor -- start 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaac580 0x560e1aabd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 0x560e1aabde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aab5a80 0x560e1aab8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001b08d0 con 0x560e1aaacea0 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001b0200 con 0x560e1aaac580 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001ad3b0 con 0x560e1aab5a80 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.803+0000 7fe508221740 10 client.0 _ll_register_callbacks cb 0x7fe4f809cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4fe7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 0x560e1aabde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4fe7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 0x560e1aabde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41136/0 (socket says 172.21.15.82:41136) 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4fe7fc640 1 -- 172.21.15.82:0/1443648369 learned_addr learned my addr 172.21.15.82:0/1443648369 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4feffd640 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaac580 0x560e1aabd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4ff7fe640 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aab5a80 0x560e1aab8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4fe7fc640 1 -- 172.21.15.82:0/1443648369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aab5a80 msgr2=0x560e1aab8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4fe7fc640 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aab5a80 0x560e1aab8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4fe7fc640 1 -- 172.21.15.82:0/1443648369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaac580 msgr2=0x560e1aabd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4fe7fc640 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaac580 0x560e1aabd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4fe7fc640 1 -- 172.21.15.82:0/1443648369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1aab9550 con 0x560e1aaacea0 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4fe7fc640 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 0x560e1aabde90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4004f70 tx=0x7fe4f401b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33567f981a76f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4e2ff5640 1 -- 172.21.15.82:0/1443648369 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4028070 con 0x560e1aaacea0 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4e2ff5640 1 -- 172.21.15.82:0/1443648369 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fe4f401b780 con 0x560e1aaacea0 2024-04-21T08:52:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe508221740 1 -- 172.21.15.82:0/1443648369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f0045070 con 0x560e1aaacea0 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4e2ff5640 1 -- 172.21.15.82:0/1443648369 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe4f4016070 con 0x560e1aaacea0 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe508221740 10 client.17079 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe508221740 1 -- 172.21.15.82:0/1443648369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560e1ab2b670 con 0x560e1aaacea0 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.804+0000 7fe4e2ff5640 1 -- 172.21.15.82:0/1443648369 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f0045070 con 0x560e1aaacea0 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe4e2ff5640 1 -- 172.21.15.82:0/1443648369 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fe4f4003860 con 0x560e1aaacea0 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 10 client.17079 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 20 client.17079 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560e1ab2be70 0x560e1ab2e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 4 client.17079 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 1 -- 172.21.15.82:0/1443648369 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e1aa18a70 con 0x560e1ab2be70 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe4feffd640 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560e1ab2be70 0x560e1ab2e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe4e2ff5640 10 client.17079 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe4feffd640 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560e1ab2be70 0x560e1ab2e290 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4e3d21cbe87fc1 server_cookie=80f1661fcd0bf369 in_seq=0 out_seq=0 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe4e2ff5640 1 -- 172.21.15.82:0/1443648369 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560e1aa18a70 con 0x560e1ab2be70 2024-04-21T08:52:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe4e2ff5640 10 client.17079 handle_command_reply: tid=0 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 1 client.17079 shutdown 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 1 -- 172.21.15.82:0/1443648369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 msgr2=0x560e1aabde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 0x560e1aabde90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4004f70 tx=0x7fe4f401b700 comp rx=0 tx=0).stop 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 1 -- 172.21.15.82:0/1443648369 shutdown_connections 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560e1ab2be70 0x560e1ab2e290 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e1aab5a80 0x560e1aab8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1aaacea0 0x560e1aabde90 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 1 --2- 172.21.15.82:0/1443648369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560e1aaac580 0x560e1aabd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe4e2ff5640 0 client.17079 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.805+0000 7fe508221740 1 -- 172.21.15.82:0/1443648369 >> 172.21.15.82:0/1443648369 conn(0x560e1aaa6cb0 msgr2=0x560e1aab2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe508221740 1 -- 172.21.15.82:0/1443648369 shutdown_connections 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe508221740 1 -- 172.21.15.82:0/1443648369 wait complete. 2024-04-21T08:52:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe508221740 20 client.17079 trim_cache size 0 max 16384 2024-04-21T08:52:11.813 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:52:11.813 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:52:11.813 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:52:11.813 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:52:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe505552640 1 -- 172.21.15.82:0/3757537852 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe4e80a3d20 msgr2=0x7fe4e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe505552640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe4e80a3d20 0x7fe4e80a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0005ab0 tx=0x7fe4f0005fd0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe505552640 1 -- 172.21.15.82:0/3757537852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5001031c0 msgr2=0x7fe5001986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe505552640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5001031c0 0x7fe5001986b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800ebc0 tx=0x7fe4f800c6a0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe505552640 1 -- 172.21.15.82:0/3757537852 shutdown_connections 2024-04-21T08:52:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe505552640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe50010c500 0x7fe50019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe505552640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe500103ae0 0x7fe500198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe505552640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe4e80a3d20 0x7fe4e80a61e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe505552640 1 --2- 172.21.15.82:0/3757537852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5001031c0 0x7fe5001986b0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe505552640 1 -- 172.21.15.82:0/3757537852 >> 172.21.15.82:0/3757537852 conn(0x7fe500078000 msgr2=0x7fe5000fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe505552640 1 -- 172.21.15.82:0/3757537852 shutdown_connections 2024-04-21T08:52:11.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.806+0000 7fe505552640 1 -- 172.21.15.82:0/3757537852 wait complete. 2024-04-21T08:52:11.817 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:52:11.817 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:52:11.838 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:11 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:52:11.838 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:11 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:52:11.838 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:11 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:52:11.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:11 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:52:11.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:11 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:52:11.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:11 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:52:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.921+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/3937289979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781011a0 msgr2=0x7ffa78101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.921+0000 7ffa7e5b6640 1 --2- 172.21.15.82:0/3937289979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781011a0 0x7ffa78101620 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009930 tx=0x7ffa6c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:52:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.921+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/3937289979 shutdown_connections 2024-04-21T08:52:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.921+0000 7ffa7e5b6640 1 --2- 172.21.15.82:0/3937289979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa78101b60 0x7ffa7810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.921+0000 7ffa7e5b6640 1 --2- 172.21.15.82:0/3937289979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781011a0 0x7ffa78101620 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.921+0000 7ffa7e5b6640 1 --2- 172.21.15.82:0/3937289979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa78105e30 0x7ffa78106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.921+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/3937289979 >> 172.21.15.82:0/3937289979 conn(0x7ffa78077e30 msgr2=0x7ffa78078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.921+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/3937289979 shutdown_connections 2024-04-21T08:52:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.921+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/3937289979 wait complete. 2024-04-21T08:52:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.921+0000 7ffa7e5b6640 1 Processor -- start 2024-04-21T08:52:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.921+0000 7ffa7e5b6640 1 -- start start 2024-04-21T08:52:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7e5b6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa781011a0 0x7ffa78102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7e5b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa78101b60 0x7ffa78103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7e5b6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa78105e30 0x7ffa78103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7e5b6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa781110e0 con 0x7ffa78105e30 2024-04-21T08:52:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7e5b6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78110f60 con 0x7ffa781011a0 2024-04-21T08:52:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7e5b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78111260 con 0x7ffa78101b60 2024-04-21T08:52:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7cb53640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa78105e30 0x7ffa78103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7cb53640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa78105e30 0x7ffa78103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41152/0 (socket says 172.21.15.82:41152) 2024-04-21T08:52:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7cb53640 1 -- 172.21.15.82:0/1864731679 learned_addr learned my addr 172.21.15.82:0/1864731679 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa77fff640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa781011a0 0x7ffa78102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa777fe640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa78101b60 0x7ffa78103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7cb53640 1 -- 172.21.15.82:0/1864731679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa78101b60 msgr2=0x7ffa78103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7cb53640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa78101b60 0x7ffa78103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7cb53640 1 -- 172.21.15.82:0/1864731679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa781011a0 msgr2=0x7ffa78102ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7cb53640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa781011a0 0x7ffa78102ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7cb53640 1 -- 172.21.15.82:0/1864731679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa781a6c20 con 0x7ffa78105e30 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa77fff640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa781011a0 0x7ffa78102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa7cb53640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa78105e30 0x7ffa78103550 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000eb10 tx=0x7ffa7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa757fa640 1 -- 172.21.15.82:0/1864731679 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70004b20 con 0x7ffa78105e30 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.922+0000 7ffa757fa640 1 -- 172.21.15.82:0/1864731679 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa70004cc0 con 0x7ffa78105e30 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.923+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/1864731679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa781a6f10 con 0x7ffa78105e30 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.923+0000 7ffa757fa640 1 -- 172.21.15.82:0/1864731679 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70010670 con 0x7ffa78105e30 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.923+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/1864731679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa781a7430 con 0x7ffa78105e30 2024-04-21T08:52:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.923+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/1864731679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa78068de0 con 0x7ffa78105e30 2024-04-21T08:52:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.925+0000 7ffa757fa640 1 -- 172.21.15.82:0/1864731679 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa70005000 con 0x7ffa78105e30 2024-04-21T08:52:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.925+0000 7ffa757fa640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa600a3da0 0x7ffa600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.926+0000 7ffa77fff640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa600a3da0 0x7ffa600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.926+0000 7ffa757fa640 1 -- 172.21.15.82:0/1864731679 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ffa700d12b0 con 0x7ffa78105e30 2024-04-21T08:52:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.926+0000 7ffa77fff640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa600a3da0 0x7ffa600a6260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009f40 tx=0x7ffa68005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:11.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:11.930+0000 7ffa757fa640 1 -- 172.21.15.82:0/1864731679 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa70053ff0 con 0x7ffa78105e30 2024-04-21T08:52:11.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:11 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:52:11.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:11 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:52:11.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:11 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:52:12.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.163+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/1864731679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa78104000 con 0x7ffa78105e30 2024-04-21T08:52:12.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.164+0000 7ffa757fa640 1 -- 172.21.15.82:0/1864731679 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11949 (secure 0 0 0) 0x7ffa7009d530 con 0x7ffa78105e30 2024-04-21T08:52:12.164 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:52:12.164 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:51:52.176351+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:52:12.164 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 36 2024-04-21T08:52:12.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.166+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/1864731679 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa600a3da0 msgr2=0x7ffa600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.166+0000 7ffa7e5b6640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa600a3da0 0x7ffa600a6260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009f40 tx=0x7ffa68005f90 comp rx=0 tx=0).stop 2024-04-21T08:52:12.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.166+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/1864731679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa78105e30 msgr2=0x7ffa78103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.166+0000 7ffa7e5b6640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa78105e30 0x7ffa78103550 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000eb10 tx=0x7ffa7000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.166+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/1864731679 shutdown_connections 2024-04-21T08:52:12.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.166+0000 7ffa7e5b6640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa600a3da0 0x7ffa600a6260 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.166+0000 7ffa7e5b6640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa78105e30 0x7ffa78103550 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.166+0000 7ffa7e5b6640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa78101b60 0x7ffa78103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.166+0000 7ffa7e5b6640 1 --2- 172.21.15.82:0/1864731679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa781011a0 0x7ffa78102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.166+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/1864731679 >> 172.21.15.82:0/1864731679 conn(0x7ffa78077e30 msgr2=0x7ffa78104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.166+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/1864731679 shutdown_connections 2024-04-21T08:52:12.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.166+0000 7ffa7e5b6640 1 -- 172.21.15.82:0/1864731679 wait complete. 2024-04-21T08:52:12.177 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:52:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.387+0000 7f4db687e640 1 -- 172.21.15.82:0/1343349002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db01099d0 msgr2=0x7f4db011d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.387+0000 7f4db687e640 1 --2- 172.21.15.82:0/1343349002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db01099d0 0x7f4db011d740 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009860 tx=0x7f4da002bc30 comp rx=0 tx=0).stop 2024-04-21T08:52:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.387+0000 7f4db687e640 1 -- 172.21.15.82:0/1343349002 shutdown_connections 2024-04-21T08:52:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.387+0000 7f4db687e640 1 --2- 172.21.15.82:0/1343349002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db01099d0 0x7f4db011d740 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.387+0000 7f4db687e640 1 --2- 172.21.15.82:0/1343349002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4db0108f50 0x7f4db01093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.387+0000 7f4db687e640 1 --2- 172.21.15.82:0/1343349002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db01105f0 0x7f4db01109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.387+0000 7f4db687e640 1 -- 172.21.15.82:0/1343349002 >> 172.21.15.82:0/1343349002 conn(0x7f4db006ced0 msgr2=0x7f4db006d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.432+0000 7f4db687e640 1 -- 172.21.15.82:0/1343349002 shutdown_connections 2024-04-21T08:52:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.432+0000 7f4db687e640 1 -- 172.21.15.82:0/1343349002 wait complete. 2024-04-21T08:52:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.432+0000 7f4db687e640 1 Processor -- start 2024-04-21T08:52:12.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.433+0000 7f4db687e640 1 -- start start 2024-04-21T08:52:12.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.433+0000 7f4db687e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4db0108f50 0x7f4db01a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.433+0000 7f4db687e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db01a7780 0x7f4db01abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.433+0000 7f4db687e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db01ac040 0x7f4db01ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.433+0000 7f4db687e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0120f20 con 0x7f4db0108f50 2024-04-21T08:52:12.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.433+0000 7f4db687e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0120da0 con 0x7f4db01ac040 2024-04-21T08:52:12.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.433+0000 7f4db687e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01210a0 con 0x7f4db01a7780 2024-04-21T08:52:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.433+0000 7f4db507b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db01a7780 0x7f4db01abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.433+0000 7f4db507b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db01a7780 0x7f4db01abb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:33410/0 (socket says 172.21.15.82:33410) 2024-04-21T08:52:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.433+0000 7f4db507b640 1 -- 172.21.15.82:0/4124313457 learned_addr learned my addr 172.21.15.82:0/4124313457 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.434+0000 7f4db587c640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4db0108f50 0x7f4db01a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.434+0000 7f4db607d640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db01ac040 0x7f4db01ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.434+0000 7f4db587c640 1 -- 172.21.15.82:0/4124313457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db01a7780 msgr2=0x7f4db01abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.434+0000 7f4db587c640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db01a7780 0x7f4db01abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.434+0000 7f4db587c640 1 -- 172.21.15.82:0/4124313457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db01ac040 msgr2=0x7f4db01ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.434+0000 7f4db587c640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db01ac040 0x7f4db01ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.435+0000 7f4db587c640 1 -- 172.21.15.82:0/4124313457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db0110f10 con 0x7f4db0108f50 2024-04-21T08:52:12.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.435+0000 7f4db607d640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db01ac040 0x7f4db01ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.441+0000 7f4db587c640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4db0108f50 0x7f4db01a7240 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4db006dab0 tx=0x7f4dac00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.443+0000 7f4da6ffd640 1 -- 172.21.15.82:0/4124313457 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac015020 con 0x7f4db0108f50 2024-04-21T08:52:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.443+0000 7f4db687e640 1 -- 172.21.15.82:0/4124313457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db0111200 con 0x7f4db0108f50 2024-04-21T08:52:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.443+0000 7f4db687e640 1 -- 172.21.15.82:0/4124313457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db0111750 con 0x7f4db0108f50 2024-04-21T08:52:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.447+0000 7f4dba7ae740 1 Processor -- start 2024-04-21T08:52:12.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.449+0000 7f4dba7ae740 1 -- start start 2024-04-21T08:52:12.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.449+0000 7f4dba7ae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f29ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4dba7ae740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f32630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4dba7ae740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f35050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4db607d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f29ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4dba7ae740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5572b4e8e1f0 con 0x5572b4f2a080 2024-04-21T08:52:12.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4dba7ae740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5572b4e2b4b0 con 0x5572b4f32c00 2024-04-21T08:52:12.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4dba7ae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5572b4e8a1b0 con 0x5572b4f296d0 2024-04-21T08:52:12.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4db607d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f29ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:33426/0 (socket says 172.21.15.82:33426) 2024-04-21T08:52:12.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4db607d640 1 -- 172.21.15.82:0/3879739852 learned_addr learned my addr 172.21.15.82:0/3879739852 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:12.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4db507b640 1 --2- 172.21.15.82:0/3879739852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f35050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4da6ffd640 1 -- 172.21.15.82:0/4124313457 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dac011ce0 con 0x7f4db0108f50 2024-04-21T08:52:12.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4da6ffd640 1 -- 172.21.15.82:0/4124313457 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac010d20 con 0x7f4db0108f50 2024-04-21T08:52:12.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4db607d640 1 -- 172.21.15.82:0/3879739852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 msgr2=0x5572b4f35050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4db607d640 1 --2- 172.21.15.82:0/3879739852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f35050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4db607d640 1 -- 172.21.15.82:0/3879739852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 msgr2=0x5572b4f32630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.450+0000 7f4db607d640 1 --2- 172.21.15.82:0/3879739852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f32630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.451+0000 7f4db607d640 1 -- 172.21.15.82:0/3879739852 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b4e1d1b0 con 0x5572b4f296d0 2024-04-21T08:52:12.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.452+0000 7f4db607d640 1 --2- 172.21.15.82:0/3879739852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f29ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4da0037040 tx=0x7f4da0009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=795ed2831b23e941 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:12.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.456+0000 7f4da6ffd640 1 -- 172.21.15.82:0/4124313457 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4dac018020 con 0x7f4db0108f50 2024-04-21T08:52:12.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.461+0000 7f4da6ffd640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4d9c0a3f30 0x7f4d9c0a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.461+0000 7f4da6ffd640 1 -- 172.21.15.82:0/4124313457 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f4dac0cff60 con 0x7f4db0108f50 2024-04-21T08:52:12.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.462+0000 7f4db587c640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4d9c0a3f30 0x7f4d9c0a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.466+0000 7f4db587c640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4d9c0a3f30 0x7f4d9c0a63f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009fd0 tx=0x7f4dac000bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:12.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.469+0000 7f4d9bfff640 1 -- 172.21.15.82:0/3879739852 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0042020 con 0x5572b4f296d0 2024-04-21T08:52:12.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.469+0000 7f4d9bfff640 1 -- 172.21.15.82:0/3879739852 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da0003aa0 con 0x5572b4f296d0 2024-04-21T08:52:12.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.469+0000 7f4d9bfff640 1 -- 172.21.15.82:0/3879739852 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0039040 con 0x5572b4f296d0 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.470+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3879739852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 msgr2=0x5572b4f29ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.470+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/3879739852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f29ab0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4da0037040 tx=0x7f4da0009400 comp rx=0 tx=0).stop 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.470+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3879739852 shutdown_connections 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.470+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/3879739852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f35050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.470+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/3879739852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f32630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.470+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/3879739852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f29ab0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.470+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3879739852 >> 172.21.15.82:0/3879739852 conn(0x5572b4f23da0 msgr2=0x5572b4f24180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.470+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3879739852 shutdown_connections 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.471+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3879739852 wait complete. 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.471+0000 7f4dba7ae740 1 -- start start 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.471+0000 7f4dba7ae740 1 Processor -- start 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.472+0000 7f4dba7ae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f32510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.472+0000 7f4dba7ae740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f2f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.472+0000 7f4dba7ae740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f2fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.472+0000 7f4dba7ae740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5572b4f35b10 con 0x5572b4f2a080 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.472+0000 7f4dba7ae740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5572b4e47500 con 0x5572b4f32c00 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.472+0000 7f4dba7ae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5572b4f35c90 con 0x5572b4f296d0 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.472+0000 7f4dba7ae740 10 client.0 _ll_register_callbacks cb 0x5572b4e15b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.472+0000 7f4db607d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f2fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.472+0000 7f4db607d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f2fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:54246/0 (socket says 172.21.15.82:54246) 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.472+0000 7f4db607d640 1 -- 172.21.15.82:0/1270691811 learned_addr learned my addr 172.21.15.82:0/1270691811 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.472+0000 7f4db607d640 1 --2- 172.21.15.82:0/1270691811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f32510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.473+0000 7f4db607d640 1 -- 172.21.15.82:0/1270691811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 msgr2=0x5572b4f32510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.473+0000 7f4db607d640 1 --2- 172.21.15.82:0/1270691811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f32510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.473+0000 7f4db607d640 1 -- 172.21.15.82:0/1270691811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 msgr2=0x5572b4f2f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.473+0000 7f4db607d640 1 --2- 172.21.15.82:0/1270691811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f2f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.473+0000 7f4db607d640 1 -- 172.21.15.82:0/1270691811 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b4ba03f0 con 0x5572b4f32c00 2024-04-21T08:52:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.473+0000 7f4db607d640 1 --2- 172.21.15.82:0/1270691811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f2fe40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4da0005fd0 tx=0x7f4da002c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d81bff51cbc231de server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:12.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.498+0000 7f4d997fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:52:12.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.498+0000 7f4d997fa640 1 -- 172.21.15.82:0/1270691811 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0039070 con 0x5572b4f32c00 2024-04-21T08:52:12.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.498+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1270691811 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da0003aa0 con 0x5572b4f32c00 2024-04-21T08:52:12.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.498+0000 7f4d997fa640 1 -- 172.21.15.82:0/1270691811 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f4da0005020 con 0x5572b4f32c00 2024-04-21T08:52:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.504+0000 7f4d997fa640 1 -- 172.21.15.82:0/1270691811 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4da0057050 con 0x5572b4f32c00 2024-04-21T08:52:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.512+0000 7f4dba7ae740 10 client.26444 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.512+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1270691811 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5572b4c6a7d0 con 0x5572b4f32c00 2024-04-21T08:52:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.513+0000 7f4d997fa640 1 -- 172.21.15.82:0/1270691811 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da0003aa0 con 0x5572b4f32c00 2024-04-21T08:52:12.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.514+0000 7f4d997fa640 1 -- 172.21.15.82:0/1270691811 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f4da00325a0 con 0x5572b4f32c00 2024-04-21T08:52:12.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.514+0000 7f4dba7ae740 10 client.26444 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:12.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.514+0000 7f4dba7ae740 10 client.26444 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:52:12.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.514+0000 7f4dba7ae740 20 client.26444 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:12.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.514+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/1270691811 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5572b4f42df0 0x5572b4f45250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.514+0000 7f4dba7ae740 4 client.26444 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:12.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.514+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1270691811 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572b4e2d930 con 0x5572b4f42df0 2024-04-21T08:52:12.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.515+0000 7f4db507b640 1 --2- 172.21.15.82:0/1270691811 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5572b4f42df0 0x5572b4f45250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.517+0000 7f4db507b640 1 --2- 172.21.15.82:0/1270691811 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5572b4f42df0 0x5572b4f45250 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2adc8efbc0ecb676 server_cookie=2a89717020c7c94e in_seq=0 out_seq=0 2024-04-21T08:52:12.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.518+0000 7f4d997fa640 10 client.26444 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:52:12.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.518+0000 7f4d997fa640 1 -- 172.21.15.82:0/1270691811 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572b4e2d930 con 0x5572b4f42df0 2024-04-21T08:52:12.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.518+0000 7f4d997fa640 10 client.26444 handle_command_reply: tid=0 2024-04-21T08:52:12.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.518+0000 7f4dba7ae740 1 client.26444 shutdown 2024-04-21T08:52:12.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.521+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1270691811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 msgr2=0x5572b4f2fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.522+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/1270691811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f2fe40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4da0005fd0 tx=0x7f4da002c750 comp rx=0 tx=0).stop 2024-04-21T08:52:12.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.522+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1270691811 shutdown_connections 2024-04-21T08:52:12.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.522+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/1270691811 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5572b4f42df0 0x5572b4f45250 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.522+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/1270691811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f2fe40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.522+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/1270691811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f2f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.522+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/1270691811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f32510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.522+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1270691811 >> 172.21.15.82:0/1270691811 conn(0x5572b4f23da0 msgr2=0x5572b4f25af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.522+0000 7f4d997fa640 0 client.26444 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:52:12.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.528+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1270691811 shutdown_connections 2024-04-21T08:52:12.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.528+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1270691811 wait complete. 2024-04-21T08:52:12.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.528+0000 7f4dba7ae740 20 client.26444 trim_cache size 0 max 16384 2024-04-21T08:52:12.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.599+0000 7f4dba7ae740 1 Processor -- start 2024-04-21T08:52:12.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.605+0000 7f4dba7ae740 1 -- start start 2024-04-21T08:52:12.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.605+0000 7f4dba7ae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f3e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.610+0000 7f4db607d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f3e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.610+0000 7f4db607d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f3e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:33450/0 (socket says 172.21.15.82:33450) 2024-04-21T08:52:12.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.610+0000 7f4db607d640 1 -- 172.21.15.82:0/3181059171 learned_addr learned my addr 172.21.15.82:0/3181059171 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:12.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.610+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/3181059171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f22ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.611+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/3181059171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f31fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.611+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3181059171 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9400fea0 con 0x5572b4f2a080 2024-04-21T08:52:12.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.611+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3181059171 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4d94011a70 con 0x5572b4f32c00 2024-04-21T08:52:12.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.611+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3181059171 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d94010570 con 0x5572b4f296d0 2024-04-21T08:52:12.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.612+0000 7f4db607d640 1 --2- 172.21.15.82:0/3181059171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f31fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.618+0000 7f4db507b640 1 --2- 172.21.15.82:0/3181059171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f22ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.621+0000 7f4db607d640 1 -- 172.21.15.82:0/3181059171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 msgr2=0x5572b4f31fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.621+0000 7f4db607d640 1 --2- 172.21.15.82:0/3181059171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f31fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.621+0000 7f4db607d640 1 -- 172.21.15.82:0/3181059171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 msgr2=0x5572b4f22ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.621+0000 7f4db607d640 1 --2- 172.21.15.82:0/3181059171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f22ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.621+0000 7f4db607d640 1 -- 172.21.15.82:0/3181059171 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b4f1f480 con 0x5572b4f296d0 2024-04-21T08:52:12.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.622+0000 7f4db607d640 1 --2- 172.21.15.82:0/3181059171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f3e9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4da0055040 tx=0x7f4da0001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcab447ded91b943 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:12.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.622+0000 7f4d9b7fe640 1 -- 172.21.15.82:0/3181059171 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0039040 con 0x5572b4f296d0 2024-04-21T08:52:12.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.622+0000 7f4d9b7fe640 1 -- 172.21.15.82:0/3181059171 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da0005730 con 0x5572b4f296d0 2024-04-21T08:52:12.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.622+0000 7f4d9b7fe640 1 -- 172.21.15.82:0/3181059171 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0042040 con 0x5572b4f296d0 2024-04-21T08:52:12.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.625+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3181059171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 msgr2=0x5572b4f3e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.625+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/3181059171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f3e9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4da0055040 tx=0x7f4da0001e70 comp rx=0 tx=0).stop 2024-04-21T08:52:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.626+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3181059171 shutdown_connections 2024-04-21T08:52:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.626+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/3181059171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f31fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.626+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/3181059171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f22ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.626+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/3181059171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f3e9d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.626+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3181059171 >> 172.21.15.82:0/3181059171 conn(0x5572b4f23da0 msgr2=0x5572b4f22590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.632+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3181059171 shutdown_connections 2024-04-21T08:52:12.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.636+0000 7f4dba7ae740 1 -- 172.21.15.82:0/3181059171 wait complete. 2024-04-21T08:52:12.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.636+0000 7f4dba7ae740 1 -- start start 2024-04-21T08:52:12.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.641+0000 7f4dba7ae740 1 Processor -- start 2024-04-21T08:52:12.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.642+0000 7f4dba7ae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f3b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.642+0000 7f4dba7ae740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f36440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.642+0000 7f4dba7ae740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f36980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.643+0000 7f4dba7ae740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4d940113a0 con 0x5572b4f2a080 2024-04-21T08:52:12.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.643+0000 7f4dba7ae740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4d94010cd0 con 0x5572b4f32c00 2024-04-21T08:52:12.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.643+0000 7f4dba7ae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9400df30 con 0x5572b4f296d0 2024-04-21T08:52:12.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.643+0000 7f4dba7ae740 10 client.0 _ll_register_callbacks cb 0x7f4da0004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:12.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.644+0000 7f4db507b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f36440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.644+0000 7f4db507b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f36440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37062/0 (socket says 172.21.15.82:37062) 2024-04-21T08:52:12.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.644+0000 7f4db507b640 1 -- 172.21.15.82:0/1723353501 learned_addr learned my addr 172.21.15.82:0/1723353501 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:12.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.649+0000 7f4db607d640 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f3b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.649+0000 7f4db607d640 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f36980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.649+0000 7f4db607d640 1 -- 172.21.15.82:0/1723353501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 msgr2=0x5572b4f36980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.649+0000 7f4db607d640 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f36980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.649+0000 7f4db607d640 1 -- 172.21.15.82:0/1723353501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 msgr2=0x5572b4f36440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.649+0000 7f4db607d640 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f36440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.649+0000 7f4db607d640 1 -- 172.21.15.82:0/1723353501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b4f36fc0 con 0x5572b4f296d0 2024-04-21T08:52:12.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.658+0000 7f4db607d640 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f3b0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4da0032a90 tx=0x7f4da0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d885c83d0c4b74a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.658+0000 7f4d997fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:52:12.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.660+0000 7f4d997fa640 1 -- 172.21.15.82:0/1723353501 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0039040 con 0x5572b4f296d0 2024-04-21T08:52:12.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.661+0000 7f4d997fa640 1 -- 172.21.15.82:0/1723353501 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f4da0032b90 con 0x5572b4f296d0 2024-04-21T08:52:12.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.661+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1723353501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da0005730 con 0x5572b4f296d0 2024-04-21T08:52:12.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.667+0000 7f4d997fa640 1 -- 172.21.15.82:0/1723353501 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4da0042040 con 0x5572b4f296d0 2024-04-21T08:52:12.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.668+0000 7f4dba7ae740 10 client.26170 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:12.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.668+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1723353501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5572b4fa9200 con 0x5572b4f296d0 2024-04-21T08:52:12.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.669+0000 7f4d997fa640 1 -- 172.21.15.82:0/1723353501 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da0005730 con 0x5572b4f296d0 2024-04-21T08:52:12.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.670+0000 7f4d997fa640 1 -- 172.21.15.82:0/1723353501 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f4da005cad0 con 0x5572b4f296d0 2024-04-21T08:52:12.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.671+0000 7f4dba7ae740 10 client.26170 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:12.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.671+0000 7f4dba7ae740 10 client.26170 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:52:12.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.671+0000 7f4dba7ae740 20 client.26170 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:12.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.671+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5572b4fa98e0 0x5572b4fabd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.671+0000 7f4dba7ae740 4 client.26170 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:52:12.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.671+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1723353501 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572b4e9bc10 con 0x5572b4fa98e0 2024-04-21T08:52:12.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.674+0000 7f4db507b640 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5572b4fa98e0 0x5572b4fabd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.675+0000 7f4db507b640 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5572b4fa98e0 0x5572b4fabd00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cbe94674d07f9e79 server_cookie=7ea433cb21f9e579 in_seq=0 out_seq=0 2024-04-21T08:52:12.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.676+0000 7f4d997fa640 10 client.26170 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:52:12.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.676+0000 7f4d997fa640 1 -- 172.21.15.82:0/1723353501 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572b4e9bc10 con 0x5572b4fa98e0 2024-04-21T08:52:12.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.676+0000 7f4d997fa640 10 client.26170 handle_command_reply: tid=0 2024-04-21T08:52:12.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.683+0000 7f4dba7ae740 1 client.26170 shutdown 2024-04-21T08:52:12.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.683+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1723353501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 msgr2=0x5572b4f3b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.683+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f3b0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4da0032a90 tx=0x7f4da0000bc0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.683+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1723353501 shutdown_connections 2024-04-21T08:52:12.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.683+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x5572b4fa98e0 0x5572b4fabd00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.683+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572b4f32c00 0x5572b4f36980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.683+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5572b4f2a080 0x5572b4f36440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.683+0000 7f4dba7ae740 1 --2- 172.21.15.82:0/1723353501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5572b4f296d0 0x5572b4f3b0d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.683+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1723353501 >> 172.21.15.82:0/1723353501 conn(0x5572b4f23da0 msgr2=0x5572b4f22e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.684+0000 7f4d997fa640 0 client.26170 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:52:12.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.687+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1723353501 shutdown_connections 2024-04-21T08:52:12.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.687+0000 7f4dba7ae740 1 -- 172.21.15.82:0/1723353501 wait complete. 2024-04-21T08:52:12.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.687+0000 7f4dba7ae740 20 client.26170 trim_cache size 0 max 16384 2024-04-21T08:52:12.689 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:52:12.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.698+0000 7f4db687e640 1 -- 172.21.15.82:0/4124313457 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4d9c0a3f30 msgr2=0x7f4d9c0a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.698+0000 7f4db687e640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4d9c0a3f30 0x7f4d9c0a63f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009fd0 tx=0x7f4dac000bc0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.699+0000 7f4db687e640 1 -- 172.21.15.82:0/4124313457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4db0108f50 msgr2=0x7f4db01a7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.699+0000 7f4db687e640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4db0108f50 0x7f4db01a7240 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4db006dab0 tx=0x7f4dac00b870 comp rx=0 tx=0).stop 2024-04-21T08:52:12.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.700+0000 7f4db687e640 1 -- 172.21.15.82:0/4124313457 shutdown_connections 2024-04-21T08:52:12.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.700+0000 7f4db687e640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4db01ac040 0x7f4db01ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.700+0000 7f4db687e640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db01a7780 0x7f4db01abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.700+0000 7f4db687e640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4d9c0a3f30 0x7f4d9c0a63f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.700+0000 7f4db687e640 1 --2- 172.21.15.82:0/4124313457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4db0108f50 0x7f4db01a7240 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.700+0000 7f4db687e640 1 -- 172.21.15.82:0/4124313457 >> 172.21.15.82:0/4124313457 conn(0x7f4db006ced0 msgr2=0x7f4db0071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.700+0000 7f4db687e640 1 -- 172.21.15.82:0/4124313457 shutdown_connections 2024-04-21T08:52:12.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.700+0000 7f4db687e640 1 -- 172.21.15.82:0/4124313457 wait complete. 2024-04-21T08:52:12.715 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:52:12.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.820+0000 7ff6f3550640 1 -- 172.21.15.82:0/2668453511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ec101a40 msgr2=0x7ff6ec105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.820+0000 7ff6f3550640 1 --2- 172.21.15.82:0/2668453511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ec101a40 0x7ff6ec105a80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009900 tx=0x7ff6e802bc30 comp rx=0 tx=0).stop 2024-04-21T08:52:12.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.820+0000 7ff6f3550640 1 -- 172.21.15.82:0/2668453511 shutdown_connections 2024-04-21T08:52:12.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.820+0000 7ff6f3550640 1 --2- 172.21.15.82:0/2668453511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6ec106160 0x7ff6ec10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.820+0000 7ff6f3550640 1 --2- 172.21.15.82:0/2668453511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ec101a40 0x7ff6ec105a80 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.820+0000 7ff6f3550640 1 --2- 172.21.15.82:0/2668453511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6ec101090 0x7ff6ec101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.820+0000 7ff6f3550640 1 -- 172.21.15.82:0/2668453511 >> 172.21.15.82:0/2668453511 conn(0x7ff6ec077f50 msgr2=0x7ff6ec078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.821+0000 7ff6f3550640 1 -- 172.21.15.82:0/2668453511 shutdown_connections 2024-04-21T08:52:12.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.821+0000 7ff6f3550640 1 -- 172.21.15.82:0/2668453511 wait complete. 2024-04-21T08:52:12.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.821+0000 7ff6f3550640 1 Processor -- start 2024-04-21T08:52:12.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.821+0000 7ff6f3550640 1 -- start start 2024-04-21T08:52:12.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.821+0000 7ff6f3550640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6ec101090 0x7ff6ec198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.821+0000 7ff6f3550640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ec106160 0x7ff6ec198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.821+0000 7ff6f3550640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6ec19cfe0 0x7ff6ec19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.821+0000 7ff6f3550640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1121e0 con 0x7ff6ec106160 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.821+0000 7ff6f3550640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec112060 con 0x7ff6ec101090 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.821+0000 7ff6f3550640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec112360 con 0x7ff6ec19cfe0 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f0aeb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ec106160 0x7ff6ec198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f12ec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6ec101090 0x7ff6ec198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f0aeb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ec106160 0x7ff6ec198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37070/0 (socket says 172.21.15.82:37070) 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f0aeb640 1 -- 172.21.15.82:0/2634457670 learned_addr learned my addr 172.21.15.82:0/2634457670 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f1aed640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6ec19cfe0 0x7ff6ec19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f0aeb640 1 -- 172.21.15.82:0/2634457670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6ec19cfe0 msgr2=0x7ff6ec19d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f0aeb640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6ec19cfe0 0x7ff6ec19d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f0aeb640 1 -- 172.21.15.82:0/2634457670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6ec101090 msgr2=0x7ff6ec198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f0aeb640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6ec101090 0x7ff6ec198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f0aeb640 1 -- 172.21.15.82:0/2634457670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ec19dad0 con 0x7ff6ec106160 2024-04-21T08:52:12.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f0aeb640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ec106160 0x7ff6ec198c60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8036040 tx=0x7ff6e80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6e27fc640 1 -- 172.21.15.82:0/2634457670 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8043020 con 0x7ff6ec106160 2024-04-21T08:52:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6e27fc640 1 -- 172.21.15.82:0/2634457670 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e802cb70 con 0x7ff6ec106160 2024-04-21T08:52:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f3550640 1 -- 172.21.15.82:0/2634457670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ec19dd60 con 0x7ff6ec106160 2024-04-21T08:52:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6e27fc640 1 -- 172.21.15.82:0/2634457670 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8003bf0 con 0x7ff6ec106160 2024-04-21T08:52:12.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.822+0000 7ff6f3550640 1 -- 172.21.15.82:0/2634457670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ec1a5580 con 0x7ff6ec106160 2024-04-21T08:52:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.824+0000 7ff6e27fc640 1 -- 172.21.15.82:0/2634457670 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff6e8046020 con 0x7ff6ec106160 2024-04-21T08:52:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f621d740 1 Processor -- start 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f621d740 1 -- start start 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6e27fc640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6d80a3d20 0x7ff6d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f621d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e454d0 0x561927e458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f621d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e4e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f621d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e4ea00 0x561927e50e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f621d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561927daa1f0 con 0x561927e45e80 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f621d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561927d474b0 con 0x561927e454d0 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f621d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561927da61b0 con 0x561927e4ea00 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f1aed640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6d80a3d20 0x7ff6d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f0aeb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e4e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f12ec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e454d0 0x561927e458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f12ec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e454d0 0x561927e458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:36392/0 (socket says 172.21.15.82:36392) 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f12ec640 1 -- 172.21.15.82:0/2590886269 learned_addr learned my addr 172.21.15.82:0/2590886269 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f0aeb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e4e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37072/0 (socket says 172.21.15.82:37072) 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6e27fc640 1 -- 172.21.15.82:0/2634457670 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff6e80f2bc0 con 0x7ff6ec106160 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.825+0000 7ff6f1aed640 1 --2- 172.21.15.82:0/2590886269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e4ea00 0x561927e50e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.826+0000 7ff6f12ec640 1 -- 172.21.15.82:0/2590886269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e4ea00 msgr2=0x561927e50e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.826+0000 7ff6f12ec640 1 --2- 172.21.15.82:0/2590886269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e4ea00 0x561927e50e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.826+0000 7ff6f12ec640 1 -- 172.21.15.82:0/2590886269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 msgr2=0x561927e4e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.826+0000 7ff6f12ec640 1 --2- 172.21.15.82:0/2590886269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e4e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.826+0000 7ff6f12ec640 1 -- 172.21.15.82:0/2590886269 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561927d391b0 con 0x561927e454d0 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.826+0000 7ff6f1aed640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6d80a3d20 0x7ff6d80a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009fd0 tx=0x7ff6e4013b00 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.826+0000 7ff6f12ec640 1 --2- 172.21.15.82:0/2590886269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e454d0 0x561927e458b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00b760 tx=0x7ff6dc00bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89de5668a07eccab server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.826+0000 7ff6d77fe640 1 -- 172.21.15.82:0/2590886269 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc003dd0 con 0x561927e454d0 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.826+0000 7ff6d77fe640 1 -- 172.21.15.82:0/2590886269 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6dc015070 con 0x561927e454d0 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.826+0000 7ff6d77fe640 1 -- 172.21.15.82:0/2590886269 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc004c90 con 0x561927e454d0 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 -- 172.21.15.82:0/2590886269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e454d0 msgr2=0x561927e458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 --2- 172.21.15.82:0/2590886269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e454d0 0x561927e458b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00b760 tx=0x7ff6dc00bc20 comp rx=0 tx=0).stop 2024-04-21T08:52:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 -- 172.21.15.82:0/2590886269 shutdown_connections 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 --2- 172.21.15.82:0/2590886269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e4ea00 0x561927e50e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 --2- 172.21.15.82:0/2590886269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e4e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 --2- 172.21.15.82:0/2590886269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e454d0 0x561927e458b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 -- 172.21.15.82:0/2590886269 >> 172.21.15.82:0/2590886269 conn(0x561927e3fba0 msgr2=0x561927e3ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 -- 172.21.15.82:0/2590886269 shutdown_connections 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 -- 172.21.15.82:0/2590886269 wait complete. 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 -- start start 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 Processor -- start 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e454d0 0x561927e4e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e45e80 0x561927e4b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e4ea00 0x561927e4bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561927e51970 con 0x561927e4ea00 2024-04-21T08:52:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561927d63500 con 0x561927e454d0 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.827+0000 7ff6f621d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561927e51af0 con 0x561927e45e80 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f621d740 10 client.0 _ll_register_callbacks cb 0x561927d31b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f1aed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e4ea00 0x561927e4bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f1aed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e4ea00 0x561927e4bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37088/0 (socket says 172.21.15.82:37088) 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f12ec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e454d0 0x561927e4e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f0aeb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e45e80 0x561927e4b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f1aed640 1 -- 172.21.15.82:0/3751168582 learned_addr learned my addr 172.21.15.82:0/3751168582 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f1aed640 1 -- 172.21.15.82:0/3751168582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e45e80 msgr2=0x561927e4b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f1aed640 1 --2- 172.21.15.82:0/3751168582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e45e80 0x561927e4b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f1aed640 1 -- 172.21.15.82:0/3751168582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e454d0 msgr2=0x561927e4e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f1aed640 1 --2- 172.21.15.82:0/3751168582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e454d0 0x561927e4e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f1aed640 1 -- 172.21.15.82:0/3751168582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561927abc3f0 con 0x561927e4ea00 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f1aed640 1 --2- 172.21.15.82:0/3751168582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e4ea00 0x561927e4bc50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4003da0 tx=0x7ff6e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e1b1f92f538d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:52:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6d4ff9640 1 -- 172.21.15.82:0/3751168582 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4024020 con 0x561927e4ea00 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6d4ff9640 1 -- 172.21.15.82:0/3751168582 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff6e4004300 con 0x561927e4ea00 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f621d740 1 -- 172.21.15.82:0/3751168582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6dc015070 con 0x561927e4ea00 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6d4ff9640 1 -- 172.21.15.82:0/3751168582 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff6e402f050 con 0x561927e4ea00 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f621d740 10 client.17109 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.828+0000 7ff6f621d740 1 -- 172.21.15.82:0/3751168582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561927b867d0 con 0x561927e4ea00 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.829+0000 7ff6d4ff9640 1 -- 172.21.15.82:0/3751168582 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6dc015070 con 0x561927e4ea00 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.829+0000 7ff6d4ff9640 1 -- 172.21.15.82:0/3751168582 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7ff6e401f070 con 0x561927e4ea00 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.829+0000 7ff6f621d740 10 client.17109 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.829+0000 7ff6f621d740 10 client.17109 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.829+0000 7ff6f621d740 20 client.17109 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.829+0000 7ff6f621d740 1 --2- 172.21.15.82:0/3751168582 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561927e5ecb0 0x561927e61100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.829+0000 7ff6f621d740 4 client.17109 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.829+0000 7ff6f621d740 1 -- 172.21.15.82:0/3751168582 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561927d49930 con 0x561927e5ecb0 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.829+0000 7ff6f12ec640 1 --2- 172.21.15.82:0/3751168582 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561927e5ecb0 0x561927e61100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.829+0000 7ff6f12ec640 1 --2- 172.21.15.82:0/3751168582 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561927e5ecb0 0x561927e61100 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8fd79fbe6b1507a8 server_cookie=17aec16a457dc23b in_seq=0 out_seq=0 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.829+0000 7ff6d4ff9640 10 client.17109 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6d4ff9640 1 -- 172.21.15.82:0/3751168582 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561927d49930 con 0x561927e5ecb0 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6d4ff9640 10 client.17109 handle_command_reply: tid=0 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6f621d740 1 client.17109 shutdown 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6f621d740 1 -- 172.21.15.82:0/3751168582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e4ea00 msgr2=0x561927e4bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6f621d740 1 --2- 172.21.15.82:0/3751168582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e4ea00 0x561927e4bc50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4003da0 tx=0x7ff6e4004ab0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6f621d740 1 -- 172.21.15.82:0/3751168582 shutdown_connections 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6f621d740 1 --2- 172.21.15.82:0/3751168582 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561927e5ecb0 0x561927e61100 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6f621d740 1 --2- 172.21.15.82:0/3751168582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e4ea00 0x561927e4bc50 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6f621d740 1 --2- 172.21.15.82:0/3751168582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e45e80 0x561927e4b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6f621d740 1 --2- 172.21.15.82:0/3751168582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e454d0 0x561927e4e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6d4ff9640 0 client.17109 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6f621d740 1 -- 172.21.15.82:0/3751168582 >> 172.21.15.82:0/3751168582 conn(0x561927e3fba0 msgr2=0x561927e418f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6f621d740 1 -- 172.21.15.82:0/3751168582 shutdown_connections 2024-04-21T08:52:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6f621d740 1 -- 172.21.15.82:0/3751168582 wait complete. 2024-04-21T08:52:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.830+0000 7ff6f621d740 20 client.17109 trim_cache size 0 max 16384 2024-04-21T08:52:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f621d740 1 Processor -- start 2024-04-21T08:52:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f621d740 1 -- start start 2024-04-21T08:52:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f621d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e454d0 0x561927e5a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f621d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e4dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f621d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e4ea00 0x561927e4e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f621d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1ad0a0 con 0x561927e45e80 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f621d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1aec70 con 0x561927e4ea00 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f621d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1ad770 con 0x561927e454d0 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f0aeb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e4dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f0aeb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e4dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37102/0 (socket says 172.21.15.82:37102) 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f0aeb640 1 -- 172.21.15.82:0/4123835972 learned_addr learned my addr 172.21.15.82:0/4123835972 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f1aed640 1 --2- 172.21.15.82:0/4123835972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e4ea00 0x561927e4e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f0aeb640 1 -- 172.21.15.82:0/4123835972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e454d0 msgr2=0x561927e5a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f0aeb640 1 --2- 172.21.15.82:0/4123835972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e454d0 0x561927e5a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f0aeb640 1 -- 172.21.15.82:0/4123835972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e4ea00 msgr2=0x561927e4e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f0aeb640 1 --2- 172.21.15.82:0/4123835972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e4ea00 0x561927e4e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f0aeb640 1 -- 172.21.15.82:0/4123835972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561927e49c00 con 0x561927e45e80 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6f0aeb640 1 --2- 172.21.15.82:0/4123835972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e4dde0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80ba590 tx=0x7ff6e80a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39552c63a856bf04 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.844+0000 7ff6d6ffd640 1 -- 172.21.15.82:0/4123835972 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80be3f0 con 0x561927e45e80 2024-04-21T08:52:12.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.845+0000 7ff6d6ffd640 1 -- 172.21.15.82:0/4123835972 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e80ec150 con 0x561927e45e80 2024-04-21T08:52:12.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.845+0000 7ff6d6ffd640 1 -- 172.21.15.82:0/4123835972 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80e9c20 con 0x561927e45e80 2024-04-21T08:52:12.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.845+0000 7ff6f621d740 1 -- 172.21.15.82:0/4123835972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 msgr2=0x561927e4dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.845+0000 7ff6f621d740 1 --2- 172.21.15.82:0/4123835972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e4dde0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80ba590 tx=0x7ff6e80a9040 comp rx=0 tx=0).stop 2024-04-21T08:52:12.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.845+0000 7ff6f621d740 1 -- 172.21.15.82:0/4123835972 shutdown_connections 2024-04-21T08:52:12.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.845+0000 7ff6f621d740 1 --2- 172.21.15.82:0/4123835972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e4ea00 0x561927e4e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.845+0000 7ff6f621d740 1 --2- 172.21.15.82:0/4123835972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e4dde0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.845+0000 7ff6f621d740 1 --2- 172.21.15.82:0/4123835972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e454d0 0x561927e5a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.845+0000 7ff6f621d740 1 -- 172.21.15.82:0/4123835972 >> 172.21.15.82:0/4123835972 conn(0x561927e3fba0 msgr2=0x561927e3ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.845+0000 7ff6f621d740 1 -- 172.21.15.82:0/4123835972 shutdown_connections 2024-04-21T08:52:12.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.845+0000 7ff6f621d740 1 -- 172.21.15.82:0/4123835972 wait complete. 2024-04-21T08:52:12.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.845+0000 7ff6f621d740 1 -- start start 2024-04-21T08:52:12.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f621d740 1 Processor -- start 2024-04-21T08:52:12.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f621d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e454d0 0x561927e57200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f621d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e480d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f621d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e4ea00 0x561927e48670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f621d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1ae5a0 con 0x561927e45e80 2024-04-21T08:52:12.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f621d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1aded0 con 0x561927e4ea00 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f621d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1ab080 con 0x561927e454d0 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f621d740 10 client.0 _ll_register_callbacks cb 0x7ff6e4003a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f0aeb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f0aeb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e480d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37116/0 (socket says 172.21.15.82:37116) 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f0aeb640 1 -- 172.21.15.82:0/1217776947 learned_addr learned my addr 172.21.15.82:0/1217776947 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f1aed640 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e4ea00 0x561927e48670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f12ec640 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e454d0 0x561927e57200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f0aeb640 1 -- 172.21.15.82:0/1217776947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e454d0 msgr2=0x561927e57200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f0aeb640 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e454d0 0x561927e57200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f0aeb640 1 -- 172.21.15.82:0/1217776947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e4ea00 msgr2=0x561927e48670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f0aeb640 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e4ea00 0x561927e48670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f0aeb640 1 -- 172.21.15.82:0/1217776947 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561927e48ea0 con 0x561927e45e80 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6f0aeb640 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e480d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8000c00 tx=0x7ff6e802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5669e0092af8b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.846+0000 7ff6d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6d4ff9640 1 -- 172.21.15.82:0/1217776947 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80be210 con 0x561927e45e80 2024-04-21T08:52:12.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6d4ff9640 1 -- 172.21.15.82:0/1217776947 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff6e8003d90 con 0x561927e45e80 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6f621d740 1 -- 172.21.15.82:0/1217776947 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e80ec150 con 0x561927e45e80 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6d4ff9640 1 -- 172.21.15.82:0/1217776947 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff6e80c7520 con 0x561927e45e80 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6f621d740 10 client.17115 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6f621d740 1 -- 172.21.15.82:0/1217776947 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561927ec52b0 con 0x561927e45e80 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6d4ff9640 1 -- 172.21.15.82:0/1217776947 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e80ec150 con 0x561927e45e80 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6d4ff9640 1 -- 172.21.15.82:0/1217776947 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7ff6e80a90b0 con 0x561927e45e80 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6f621d740 10 client.17115 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6f621d740 10 client.17115 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6f621d740 20 client.17115 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6f621d740 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561927ec59e0 0x561927ec7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6f621d740 4 client.17115 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6f621d740 1 -- 172.21.15.82:0/1217776947 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561927db7c10 con 0x561927ec59e0 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.847+0000 7ff6f12ec640 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561927ec59e0 0x561927ec7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6d4ff9640 10 client.17115 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:52:12.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f12ec640 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561927ec59e0 0x561927ec7e00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=548ae2778209fa5c server_cookie=3277e49570a14ac8 in_seq=0 out_seq=0 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6d4ff9640 1 -- 172.21.15.82:0/1217776947 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561927db7c10 con 0x561927ec59e0 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6d4ff9640 10 client.17115 handle_command_reply: tid=0 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f621d740 1 client.17115 shutdown 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f621d740 1 -- 172.21.15.82:0/1217776947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 msgr2=0x561927e480d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f621d740 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e480d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8000c00 tx=0x7ff6e802bc30 comp rx=0 tx=0).stop 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f621d740 1 -- 172.21.15.82:0/1217776947 shutdown_connections 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f621d740 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561927ec59e0 0x561927ec7e00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f621d740 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561927e4ea00 0x561927e48670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6d4ff9640 0 client.17115 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f621d740 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561927e45e80 0x561927e480d0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f621d740 1 --2- 172.21.15.82:0/1217776947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561927e454d0 0x561927e57200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f621d740 1 -- 172.21.15.82:0/1217776947 >> 172.21.15.82:0/1217776947 conn(0x561927e3fba0 msgr2=0x561927e46260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f621d740 1 -- 172.21.15.82:0/1217776947 shutdown_connections 2024-04-21T08:52:12.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f621d740 1 -- 172.21.15.82:0/1217776947 wait complete. 2024-04-21T08:52:12.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.848+0000 7ff6f621d740 20 client.17115 trim_cache size 0 max 16384 2024-04-21T08:52:12.859 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:52:12.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.849+0000 7ff6f3550640 1 -- 172.21.15.82:0/2634457670 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6d80a3d20 msgr2=0x7ff6d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.849+0000 7ff6f3550640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6d80a3d20 0x7ff6d80a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009fd0 tx=0x7ff6e4013b00 comp rx=0 tx=0).stop 2024-04-21T08:52:12.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.849+0000 7ff6f3550640 1 -- 172.21.15.82:0/2634457670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ec106160 msgr2=0x7ff6ec198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:12.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.849+0000 7ff6f3550640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ec106160 0x7ff6ec198c60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8036040 tx=0x7ff6e80045e0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.849+0000 7ff6f3550640 1 -- 172.21.15.82:0/2634457670 shutdown_connections 2024-04-21T08:52:12.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.849+0000 7ff6f3550640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6ec19cfe0 0x7ff6ec19d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.849+0000 7ff6f3550640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6d80a3d20 0x7ff6d80a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.849+0000 7ff6f3550640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ec106160 0x7ff6ec198c60 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.849+0000 7ff6f3550640 1 --2- 172.21.15.82:0/2634457670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6ec101090 0x7ff6ec198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:12.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.849+0000 7ff6f3550640 1 -- 172.21.15.82:0/2634457670 >> 172.21.15.82:0/2634457670 conn(0x7ff6ec077f50 msgr2=0x7ff6ec0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:12.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.849+0000 7ff6f3550640 1 -- 172.21.15.82:0/2634457670 shutdown_connections 2024-04-21T08:52:12.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:12.849+0000 7ff6f3550640 1 -- 172.21.15.82:0/2634457670 wait complete. 2024-04-21T08:52:12.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:12 smithi082 ceph-mon[27967]: pgmap v158: 129 pgs: 129 active+clean; 669 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 729 op/s 2024-04-21T08:52:12.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:12 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1864731679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:12.873 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:52:12.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:12 smithi134 ceph-mon[31772]: pgmap v158: 129 pgs: 129 active+clean; 669 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 729 op/s 2024-04-21T08:52:12.877 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:12 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1864731679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:12.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:12 smithi106 ceph-mon[31072]: pgmap v158: 129 pgs: 129 active+clean; 669 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 729 op/s 2024-04-21T08:52:12.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:12 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1864731679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:13.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.880+0000 7fcc32984640 1 -- 172.21.15.82:0/2427087233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc2c10ecb0 msgr2=0x7fcc2c10bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:13.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.880+0000 7fcc32984640 1 --2- 172.21.15.82:0/2427087233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc2c10ecb0 0x7fcc2c10bf80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009ca0 tx=0x7fcc2402d310 comp rx=0 tx=0).stop 2024-04-21T08:52:13.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.896+0000 7fcc32984640 1 -- 172.21.15.82:0/2427087233 shutdown_connections 2024-04-21T08:52:13.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.896+0000 7fcc32984640 1 --2- 172.21.15.82:0/2427087233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc2c10c690 0x7fcc2c07a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.896+0000 7fcc32984640 1 --2- 172.21.15.82:0/2427087233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc2c10ecb0 0x7fcc2c10bf80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.896+0000 7fcc32984640 1 --2- 172.21.15.82:0/2427087233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc2c10e390 0x7fcc2c10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.896+0000 7fcc32984640 1 -- 172.21.15.82:0/2427087233 >> 172.21.15.82:0/2427087233 conn(0x7fcc2c06ced0 msgr2=0x7fcc2c06d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:13.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.896+0000 7fcc32984640 1 -- 172.21.15.82:0/2427087233 shutdown_connections 2024-04-21T08:52:13.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.896+0000 7fcc32984640 1 -- 172.21.15.82:0/2427087233 wait complete. 2024-04-21T08:52:13.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.897+0000 7fcc32984640 1 Processor -- start 2024-04-21T08:52:13.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.898+0000 7fcc32984640 1 -- start start 2024-04-21T08:52:13.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.898+0000 7fcc32984640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc2c10c690 0x7fcc2c0806e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:13.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.898+0000 7fcc32984640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc2c10e390 0x7fcc2c07eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:13.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.898+0000 7fcc32984640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc2c10ecb0 0x7fcc2c07f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:13.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.898+0000 7fcc32984640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c07d5e0 con 0x7fcc2c10e390 2024-04-21T08:52:13.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.898+0000 7fcc32984640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c07d460 con 0x7fcc2c10ecb0 2024-04-21T08:52:13.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.898+0000 7fcc32984640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c07d760 con 0x7fcc2c10c690 2024-04-21T08:52:13.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.898+0000 7fcc31181640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc2c10e390 0x7fcc2c07eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:13.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.898+0000 7fcc31181640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc2c10e390 0x7fcc2c07eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37136/0 (socket says 172.21.15.82:37136) 2024-04-21T08:52:13.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.898+0000 7fcc31181640 1 -- 172.21.15.82:0/852936333 learned_addr learned my addr 172.21.15.82:0/852936333 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:13.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.898+0000 7fcc31982640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc2c10c690 0x7fcc2c0806e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:13.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.898+0000 7fcc32183640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc2c10ecb0 0x7fcc2c07f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:13.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.899+0000 7fcc32183640 1 -- 172.21.15.82:0/852936333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc2c10c690 msgr2=0x7fcc2c0806e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:13.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.899+0000 7fcc32183640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc2c10c690 0x7fcc2c0806e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.899+0000 7fcc32183640 1 -- 172.21.15.82:0/852936333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc2c10e390 msgr2=0x7fcc2c07eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:13.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.899+0000 7fcc32183640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc2c10e390 0x7fcc2c07eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.899+0000 7fcc32183640 1 -- 172.21.15.82:0/852936333 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c1c47f0 con 0x7fcc2c10ecb0 2024-04-21T08:52:13.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.899+0000 7fcc31181640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc2c10e390 0x7fcc2c07eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:13.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.899+0000 7fcc32183640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc2c10ecb0 0x7fcc2c07f140 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00b410 tx=0x7fcc1c00b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.905+0000 7fcc22ffd640 1 -- 172.21.15.82:0/852936333 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c015020 con 0x7fcc2c10ecb0 2024-04-21T08:52:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.905+0000 7fcc32984640 1 -- 172.21.15.82:0/852936333 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2c1c4a20 con 0x7fcc2c10ecb0 2024-04-21T08:52:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.905+0000 7fcc32984640 1 -- 172.21.15.82:0/852936333 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2c1c4f40 con 0x7fcc2c10ecb0 2024-04-21T08:52:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.905+0000 7fcc22ffd640 1 -- 172.21.15.82:0/852936333 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c011ce0 con 0x7fcc2c10ecb0 2024-04-21T08:52:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.905+0000 7fcc22ffd640 1 -- 172.21.15.82:0/852936333 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c010da0 con 0x7fcc2c10ecb0 2024-04-21T08:52:13.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.909+0000 7fcc367d1740 1 Processor -- start 2024-04-21T08:52:13.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.912+0000 7fcc22ffd640 1 -- 172.21.15.82:0/852936333 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcc1c018020 con 0x7fcc2c10ecb0 2024-04-21T08:52:13.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.913+0000 7fcc22ffd640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcc180a3f20 0x7fcc180a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:13.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.914+0000 7fcc31181640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcc180a3f20 0x7fcc180a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:13.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.914+0000 7fcc367d1740 1 -- start start 2024-04-21T08:52:13.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.914+0000 7fcc367d1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x560027181ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:13.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.915+0000 7fcc367d1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560027182080 0x56002718a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:13.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.915+0000 7fcc367d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56002718ac00 0x56002718d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:13.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.915+0000 7fcc367d1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5600270e61f0 con 0x5600271816d0 2024-04-21T08:52:13.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.915+0000 7fcc31982640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x560027181ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:13.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.915+0000 7fcc31982640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x560027181ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37150/0 (socket says 172.21.15.82:37150) 2024-04-21T08:52:13.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.915+0000 7fcc31982640 1 -- 172.21.15.82:0/2679800134 learned_addr learned my addr 172.21.15.82:0/2679800134 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:13.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.915+0000 7fcc367d1740 1 -- 172.21.15.82:0/2679800134 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5600270834b0 con 0x560027182080 2024-04-21T08:52:13.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.915+0000 7fcc367d1740 1 -- 172.21.15.82:0/2679800134 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5600270e21b0 con 0x56002718ac00 2024-04-21T08:52:13.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.915+0000 7fcc32183640 1 --2- 172.21.15.82:0/2679800134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56002718ac00 0x56002718d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:13.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.915+0000 7fcc22ffd640 1 -- 172.21.15.82:0/852936333 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fcc1c0d1100 con 0x7fcc2c10ecb0 2024-04-21T08:52:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.915+0000 7fcc31181640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcc180a3f20 0x7fcc180a63e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc24000f80 tx=0x7fcc2402d830 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.916+0000 7fcc31982640 1 -- 172.21.15.82:0/2679800134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56002718ac00 msgr2=0x56002718d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.916+0000 7fcc31982640 1 --2- 172.21.15.82:0/2679800134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56002718ac00 0x56002718d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.916+0000 7fcc31982640 1 -- 172.21.15.82:0/2679800134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560027182080 msgr2=0x56002718a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.916+0000 7fcc31181640 1 --2- 172.21.15.82:0/2679800134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560027182080 0x56002718a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.916+0000 7fcc31982640 1 --2- 172.21.15.82:0/2679800134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560027182080 0x56002718a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.916+0000 7fcc31982640 1 -- 172.21.15.82:0/2679800134 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600270751b0 con 0x5600271816d0 2024-04-21T08:52:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.916+0000 7fcc31982640 1 --2- 172.21.15.82:0/2679800134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x560027181ab0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800ad00 tx=0x7fcc2800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9a44fe09e8fdc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.916+0000 7fcc32183640 1 --2- 172.21.15.82:0/2679800134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56002718ac00 0x56002718d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.916+0000 7fcc31181640 1 --2- 172.21.15.82:0/2679800134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560027182080 0x56002718a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:13.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.929+0000 7fcc17fff640 1 -- 172.21.15.82:0/2679800134 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2801d070 con 0x5600271816d0 2024-04-21T08:52:13.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.929+0000 7fcc17fff640 1 -- 172.21.15.82:0/2679800134 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc28018af0 con 0x5600271816d0 2024-04-21T08:52:13.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.929+0000 7fcc17fff640 1 -- 172.21.15.82:0/2679800134 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28010b50 con 0x5600271816d0 2024-04-21T08:52:13.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.930+0000 7fcc367d1740 1 -- 172.21.15.82:0/2679800134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 msgr2=0x560027181ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:13.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.930+0000 7fcc367d1740 1 --2- 172.21.15.82:0/2679800134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x560027181ab0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800ad00 tx=0x7fcc2800f790 comp rx=0 tx=0).stop 2024-04-21T08:52:13.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.934+0000 7fcc367d1740 1 -- 172.21.15.82:0/2679800134 shutdown_connections 2024-04-21T08:52:13.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.934+0000 7fcc367d1740 1 --2- 172.21.15.82:0/2679800134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56002718ac00 0x56002718d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.934+0000 7fcc367d1740 1 --2- 172.21.15.82:0/2679800134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560027182080 0x56002718a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.934+0000 7fcc367d1740 1 --2- 172.21.15.82:0/2679800134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x560027181ab0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.934+0000 7fcc367d1740 1 -- 172.21.15.82:0/2679800134 >> 172.21.15.82:0/2679800134 conn(0x56002717bda0 msgr2=0x56002717c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:13.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.935+0000 7fcc367d1740 1 -- 172.21.15.82:0/2679800134 shutdown_connections 2024-04-21T08:52:13.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.935+0000 7fcc367d1740 1 -- 172.21.15.82:0/2679800134 wait complete. 2024-04-21T08:52:13.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.935+0000 7fcc367d1740 1 -- start start 2024-04-21T08:52:13.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.936+0000 7fcc367d1740 1 Processor -- start 2024-04-21T08:52:13.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc367d1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x56002718a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:13.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc367d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560027182080 0x5600271878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:13.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc367d1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56002718ac00 0x560027187e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:13.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc367d1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56002718db70 con 0x5600271816d0 2024-04-21T08:52:13.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc367d1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56002709f500 con 0x56002718ac00 2024-04-21T08:52:13.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc367d1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56002718dcf0 con 0x560027182080 2024-04-21T08:52:13.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc367d1740 10 client.0 _ll_register_callbacks cb 0x56002706db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:13.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc31181640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560027182080 0x5600271878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:13.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc31181640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560027182080 0x5600271878a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:51934/0 (socket says 172.21.15.82:51934) 2024-04-21T08:52:13.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc31181640 1 -- 172.21.15.82:0/1389297216 learned_addr learned my addr 172.21.15.82:0/1389297216 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:13.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc31181640 1 -- 172.21.15.82:0/1389297216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56002718ac00 msgr2=0x560027187e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:13.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc32183640 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56002718ac00 0x560027187e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:13.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc31181640 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56002718ac00 0x560027187e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc31982640 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x56002718a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:13.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc31181640 1 -- 172.21.15.82:0/1389297216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 msgr2=0x56002718a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:13.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc31181640 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x56002718a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.937+0000 7fcc31181640 1 -- 172.21.15.82:0/1389297216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560026df83f0 con 0x560027182080 2024-04-21T08:52:13.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.938+0000 7fcc31982640 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x56002718a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:13.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.938+0000 7fcc31181640 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560027182080 0x5600271878a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009dc0 tx=0x7fcc240095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0300662c156033 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:13.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.955+0000 7fcc157fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:52:13.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.955+0000 7fcc157fa640 1 -- 172.21.15.82:0/1389297216 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2404f070 con 0x560027182080 2024-04-21T08:52:13.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.955+0000 7fcc367d1740 1 -- 172.21.15.82:0/1389297216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc28018af0 con 0x560027182080 2024-04-21T08:52:13.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.955+0000 7fcc157fa640 1 -- 172.21.15.82:0/1389297216 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fcc24004140 con 0x560027182080 2024-04-21T08:52:13.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.960+0000 7fcc157fa640 1 -- 172.21.15.82:0/1389297216 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcc2405b050 con 0x560027182080 2024-04-21T08:52:13.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.965+0000 7fcc367d1740 10 client.26194 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:13.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.965+0000 7fcc367d1740 1 -- 172.21.15.82:0/1389297216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560026ec27d0 con 0x560027182080 2024-04-21T08:52:13.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.966+0000 7fcc157fa640 1 -- 172.21.15.82:0/1389297216 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc28018af0 con 0x560027182080 2024-04-21T08:52:13.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.966+0000 7fcc157fa640 1 -- 172.21.15.82:0/1389297216 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fcc2404a070 con 0x560027182080 2024-04-21T08:52:13.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.966+0000 7fcc367d1740 10 client.26194 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:13.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.966+0000 7fcc367d1740 10 client.26194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:52:13.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.966+0000 7fcc367d1740 20 client.26194 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:13.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.966+0000 7fcc367d1740 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56002719b2b0 0x56002719d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:13.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.966+0000 7fcc367d1740 4 client.26194 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:13.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.966+0000 7fcc367d1740 1 -- 172.21.15.82:0/1389297216 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560027085930 con 0x56002719b2b0 2024-04-21T08:52:13.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.966+0000 7fcc31982640 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56002719b2b0 0x56002719d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:13.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.967+0000 7fcc31982640 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56002719b2b0 0x56002719d770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4bf0e2d037b3549e server_cookie=10ea8ae8e8e1b55b in_seq=0 out_seq=0 2024-04-21T08:52:13.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.969+0000 7fcc157fa640 10 client.26194 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:52:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.972+0000 7fcc157fa640 1 -- 172.21.15.82:0/1389297216 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560027085930 con 0x56002719b2b0 2024-04-21T08:52:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.972+0000 7fcc157fa640 10 client.26194 handle_command_reply: tid=0 2024-04-21T08:52:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.972+0000 7fcc367d1740 1 client.26194 shutdown 2024-04-21T08:52:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.973+0000 7fcc367d1740 1 -- 172.21.15.82:0/1389297216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560027182080 msgr2=0x5600271878a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.973+0000 7fcc367d1740 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560027182080 0x5600271878a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009dc0 tx=0x7fcc240095c0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.973+0000 7fcc367d1740 1 -- 172.21.15.82:0/1389297216 shutdown_connections 2024-04-21T08:52:13.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.973+0000 7fcc367d1740 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56002719b2b0 0x56002719d770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.973+0000 7fcc367d1740 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56002718ac00 0x560027187e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.973+0000 7fcc367d1740 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560027182080 0x5600271878a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.973+0000 7fcc367d1740 1 --2- 172.21.15.82:0/1389297216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x56002718a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:13.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.973+0000 7fcc367d1740 1 -- 172.21.15.82:0/1389297216 >> 172.21.15.82:0/1389297216 conn(0x56002717bda0 msgr2=0x56002717daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:13.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.973+0000 7fcc157fa640 0 client.26194 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:52:13.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.973+0000 7fcc367d1740 1 -- 172.21.15.82:0/1389297216 shutdown_connections 2024-04-21T08:52:13.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.973+0000 7fcc367d1740 1 -- 172.21.15.82:0/1389297216 wait complete. 2024-04-21T08:52:13.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:13.974+0000 7fcc367d1740 20 client.26194 trim_cache size 0 max 16384 2024-04-21T08:52:14.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.020+0000 7fcc367d1740 1 Processor -- start 2024-04-21T08:52:14.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.023+0000 7fcc367d1740 1 -- start start 2024-04-21T08:52:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.023+0000 7fcc367d1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x560027196a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.023+0000 7fcc367d1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560027182080 0x560027189fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.023+0000 7fcc367d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56002718ac00 0x56002718a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.024+0000 7fcc367d1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcc10010090 con 0x5600271816d0 2024-04-21T08:52:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.024+0000 7fcc31982640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x560027196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.024+0000 7fcc31982640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x560027196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37168/0 (socket says 172.21.15.82:37168) 2024-04-21T08:52:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.024+0000 7fcc31181640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560027182080 0x560027189fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.024+0000 7fcc31982640 1 -- 172.21.15.82:0/4266428830 learned_addr learned my addr 172.21.15.82:0/4266428830 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.024+0000 7fcc32183640 1 --2- 172.21.15.82:0/4266428830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56002718ac00 0x56002718a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.024+0000 7fcc367d1740 1 -- 172.21.15.82:0/4266428830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcc10011c60 con 0x560027182080 2024-04-21T08:52:14.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.024+0000 7fcc367d1740 1 -- 172.21.15.82:0/4266428830 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc10010760 con 0x56002718ac00 2024-04-21T08:52:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.024+0000 7fcc32183640 1 -- 172.21.15.82:0/4266428830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560027182080 msgr2=0x560027189fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.024+0000 7fcc32183640 1 --2- 172.21.15.82:0/4266428830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560027182080 0x560027189fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.024+0000 7fcc32183640 1 -- 172.21.15.82:0/4266428830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 msgr2=0x560027196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.025+0000 7fcc32183640 1 --2- 172.21.15.82:0/4266428830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x560027196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.025+0000 7fcc32183640 1 -- 172.21.15.82:0/4266428830 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560027181e20 con 0x56002718ac00 2024-04-21T08:52:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.025+0000 7fcc31982640 1 --2- 172.21.15.82:0/4266428830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x560027196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:14.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.025+0000 7fcc32183640 1 --2- 172.21.15.82:0/4266428830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56002718ac00 0x56002718a510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0ce540 tx=0x7fcc1c0bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa2af2abca9bfc77 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:14.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.029+0000 7fcc177fe640 1 -- 172.21.15.82:0/4266428830 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c09d490 con 0x56002718ac00 2024-04-21T08:52:14.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.029+0000 7fcc177fe640 1 -- 172.21.15.82:0/4266428830 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c09cd90 con 0x56002718ac00 2024-04-21T08:52:14.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.029+0000 7fcc177fe640 1 -- 172.21.15.82:0/4266428830 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c099400 con 0x56002718ac00 2024-04-21T08:52:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.030+0000 7fcc367d1740 1 -- 172.21.15.82:0/4266428830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56002718ac00 msgr2=0x56002718a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.031+0000 7fcc367d1740 1 --2- 172.21.15.82:0/4266428830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56002718ac00 0x56002718a510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0ce540 tx=0x7fcc1c0bd040 comp rx=0 tx=0).stop 2024-04-21T08:52:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.031+0000 7fcc367d1740 1 -- 172.21.15.82:0/4266428830 shutdown_connections 2024-04-21T08:52:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.031+0000 7fcc367d1740 1 --2- 172.21.15.82:0/4266428830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56002718ac00 0x56002718a510 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.031+0000 7fcc367d1740 1 --2- 172.21.15.82:0/4266428830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560027182080 0x560027189fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.031+0000 7fcc367d1740 1 --2- 172.21.15.82:0/4266428830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600271816d0 0x560027196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.031+0000 7fcc367d1740 1 -- 172.21.15.82:0/4266428830 >> 172.21.15.82:0/4266428830 conn(0x56002717bda0 msgr2=0x560027182460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.033+0000 7fcc367d1740 1 -- 172.21.15.82:0/4266428830 shutdown_connections 2024-04-21T08:52:14.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.033+0000 7fcc367d1740 1 -- 172.21.15.82:0/4266428830 wait complete. 2024-04-21T08:52:14.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.033+0000 7fcc367d1740 1 -- start start 2024-04-21T08:52:14.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.034+0000 7fcc367d1740 1 Processor -- start 2024-04-21T08:52:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.034+0000 7fcc367d1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600271816d0 0x5600271931a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc367d1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560027182080 0x56002718e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc31982640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600271816d0 0x5600271931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc31982640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600271816d0 0x5600271931a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:51940/0 (socket says 172.21.15.82:51940) 2024-04-21T08:52:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc31181640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560027182080 0x56002718e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc31982640 1 -- 172.21.15.82:0/3585194029 learned_addr learned my addr 172.21.15.82:0/3585194029 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc31181640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560027182080 0x56002718e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37182/0 (socket says 172.21.15.82:37182) 2024-04-21T08:52:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc367d1740 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56002718ac00 0x56002718ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc367d1740 1 -- 172.21.15.82:0/3585194029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcc10011590 con 0x560027182080 2024-04-21T08:52:14.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc367d1740 1 -- 172.21.15.82:0/3585194029 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcc10010ec0 con 0x56002718ac00 2024-04-21T08:52:14.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc367d1740 1 -- 172.21.15.82:0/3585194029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1000e070 con 0x5600271816d0 2024-04-21T08:52:14.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc32183640 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56002718ac00 0x56002718ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc31982640 1 -- 172.21.15.82:0/3585194029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56002718ac00 msgr2=0x56002718ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:14.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc31982640 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56002718ac00 0x56002718ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.035+0000 7fcc31982640 1 -- 172.21.15.82:0/3585194029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560027182080 msgr2=0x56002718e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.036+0000 7fcc31982640 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560027182080 0x56002718e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.036+0000 7fcc31982640 1 -- 172.21.15.82:0/3585194029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56002718f090 con 0x5600271816d0 2024-04-21T08:52:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.036+0000 7fcc367d1740 10 client.0 _ll_register_callbacks cb 0x7fcc24003a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.036+0000 7fcc32183640 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56002718ac00 0x56002718ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.036+0000 7fcc367d1740 1 -- 172.21.15.82:0/3585194029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc1c09cd90 con 0x5600271816d0 2024-04-21T08:52:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.036+0000 7fcc31982640 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600271816d0 0x5600271931a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc28010770 tx=0x7fcc2800a960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6adefb707f20f5e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.036+0000 7fcc31181640 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560027182080 0x56002718e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:14.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.070+0000 7fcc157fa640 10 client.26200 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:52:14.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.070+0000 7fcc157fa640 1 -- 172.21.15.82:0/3585194029 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2802f060 con 0x5600271816d0 2024-04-21T08:52:14.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.070+0000 7fcc157fa640 1 -- 172.21.15.82:0/3585194029 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fcc28005250 con 0x5600271816d0 2024-04-21T08:52:14.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.070+0000 7fcc157fa640 1 -- 172.21.15.82:0/3585194029 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcc28036050 con 0x5600271816d0 2024-04-21T08:52:14.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.070+0000 7fcc367d1740 10 client.26200 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:14.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.070+0000 7fcc367d1740 1 -- 172.21.15.82:0/3585194029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({fsmap=36}) -- 0x560027201640 con 0x5600271816d0 2024-04-21T08:52:14.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.073+0000 7fcc157fa640 1 -- 172.21.15.82:0/3585194029 <== mon.1 v2:172.21.15.134:3300/0 4 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fcc240333c0 con 0x5600271816d0 2024-04-21T08:52:14.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.073+0000 7fcc367d1740 10 client.26200 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:14.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.073+0000 7fcc367d1740 10 client.26200 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:52:14.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.073+0000 7fcc367d1740 20 client.26200 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:14.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.073+0000 7fcc367d1740 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x560027201a20 0x560027203e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.073+0000 7fcc367d1740 4 client.26200 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:52:14.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.073+0000 7fcc367d1740 1 -- 172.21.15.82:0/3585194029 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600270f3c10 con 0x560027201a20 2024-04-21T08:52:14.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.073+0000 7fcc31181640 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x560027201a20 0x560027203e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.074+0000 7fcc31181640 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x560027201a20 0x560027203e40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b62aff37a90a363d server_cookie=15d750467e519fef in_seq=0 out_seq=0 2024-04-21T08:52:14.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.074+0000 7fcc157fa640 10 client.26200 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:52:14.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.076+0000 7fcc157fa640 1 -- 172.21.15.82:0/3585194029 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600270f3c10 con 0x560027201a20 2024-04-21T08:52:14.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.076+0000 7fcc157fa640 10 client.26200 handle_command_reply: tid=0 2024-04-21T08:52:14.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.076+0000 7fcc367d1740 1 client.26200 shutdown 2024-04-21T08:52:14.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.076+0000 7fcc367d1740 1 -- 172.21.15.82:0/3585194029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600271816d0 msgr2=0x5600271931a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.076+0000 7fcc367d1740 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600271816d0 0x5600271931a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc28010770 tx=0x7fcc2800a960 comp rx=0 tx=0).stop 2024-04-21T08:52:14.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.077+0000 7fcc367d1740 1 -- 172.21.15.82:0/3585194029 shutdown_connections 2024-04-21T08:52:14.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.077+0000 7fcc367d1740 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x560027201a20 0x560027203e40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.077+0000 7fcc367d1740 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56002718ac00 0x56002718ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.077+0000 7fcc367d1740 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560027182080 0x56002718e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.077+0000 7fcc367d1740 1 --2- 172.21.15.82:0/3585194029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600271816d0 0x5600271931a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.077+0000 7fcc367d1740 1 -- 172.21.15.82:0/3585194029 >> 172.21.15.82:0/3585194029 conn(0x56002717bda0 msgr2=0x560027182cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:14.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.077+0000 7fcc157fa640 0 client.26200 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:52:14.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.077+0000 7fcc367d1740 1 -- 172.21.15.82:0/3585194029 shutdown_connections 2024-04-21T08:52:14.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.077+0000 7fcc367d1740 1 -- 172.21.15.82:0/3585194029 wait complete. 2024-04-21T08:52:14.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.077+0000 7fcc367d1740 20 client.26200 trim_cache size 0 max 16384 2024-04-21T08:52:14.078 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:52:14.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.091+0000 7fcc32984640 1 -- 172.21.15.82:0/852936333 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcc180a3f20 msgr2=0x7fcc180a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.091+0000 7fcc32984640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcc180a3f20 0x7fcc180a63e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc24000f80 tx=0x7fcc2402d830 comp rx=0 tx=0).stop 2024-04-21T08:52:14.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.091+0000 7fcc32984640 1 -- 172.21.15.82:0/852936333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc2c10ecb0 msgr2=0x7fcc2c07f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.091+0000 7fcc32984640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc2c10ecb0 0x7fcc2c07f140 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00b410 tx=0x7fcc1c00b810 comp rx=0 tx=0).stop 2024-04-21T08:52:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.091+0000 7fcc32984640 1 -- 172.21.15.82:0/852936333 shutdown_connections 2024-04-21T08:52:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.091+0000 7fcc32984640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc2c10ecb0 0x7fcc2c07f140 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.091+0000 7fcc32984640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcc180a3f20 0x7fcc180a63e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.092+0000 7fcc32984640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc2c10e390 0x7fcc2c07eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.092+0000 7fcc32984640 1 --2- 172.21.15.82:0/852936333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc2c10c690 0x7fcc2c0806e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.092+0000 7fcc32984640 1 -- 172.21.15.82:0/852936333 >> 172.21.15.82:0/852936333 conn(0x7fcc2c06ced0 msgr2=0x7fcc2c072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.093+0000 7fcc32984640 1 -- 172.21.15.82:0/852936333 shutdown_connections 2024-04-21T08:52:14.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.093+0000 7fcc32984640 1 -- 172.21.15.82:0/852936333 wait complete. 2024-04-21T08:52:14.162 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:52:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:14 smithi082 ceph-mon[27967]: pgmap v159: 129 pgs: 129 active+clean; 682 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.5 MiB/s wr, 1.10k op/s 2024-04-21T08:52:14.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:14 smithi106 ceph-mon[31072]: pgmap v159: 129 pgs: 129 active+clean; 682 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.5 MiB/s wr, 1.10k op/s 2024-04-21T08:52:14.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.932+0000 7f18899c3640 1 -- 172.21.15.82:0/2160309885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c094e10 msgr2=0x7f187c095290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.932+0000 7f18899c3640 1 --2- 172.21.15.82:0/2160309885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c094e10 0x7f187c095290 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1870009930 tx=0x7f187002bc30 comp rx=0 tx=0).stop 2024-04-21T08:52:14.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.932+0000 7f18899c3640 1 -- 172.21.15.82:0/2160309885 shutdown_connections 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.932+0000 7f18899c3640 1 --2- 172.21.15.82:0/2160309885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187c0957d0 0x7f187c0a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.932+0000 7f18899c3640 1 --2- 172.21.15.82:0/2160309885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c094e10 0x7f187c095290 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.932+0000 7f18899c3640 1 --2- 172.21.15.82:0/2160309885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f187c099bf0 0x7f187c099fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.932+0000 7f18899c3640 1 -- 172.21.15.82:0/2160309885 >> 172.21.15.82:0/2160309885 conn(0x7f187c00a7c0 msgr2=0x7f187c00abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.932+0000 7f18899c3640 1 -- 172.21.15.82:0/2160309885 shutdown_connections 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.932+0000 7f18899c3640 1 -- 172.21.15.82:0/2160309885 wait complete. 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f18899c3640 1 Processor -- start 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f18899c3640 1 -- start start 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f18899c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f187c094e10 0x7f187c12b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f18899c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187c0957d0 0x7f187c12b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f18899c3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c099bf0 0x7f187c12fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f18899c3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f187c0a4ec0 con 0x7f187c0957d0 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f18899c3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f187c0a4d40 con 0x7f187c099bf0 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f18899c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f187c0a5040 con 0x7f187c094e10 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f1883fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187c0957d0 0x7f187c12b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f1883fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187c0957d0 0x7f187c12b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37202/0 (socket says 172.21.15.82:37202) 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f18891c2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c099bf0 0x7f187c12fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f1883fff640 1 -- 172.21.15.82:0/2974010677 learned_addr learned my addr 172.21.15.82:0/2974010677 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.933+0000 7f18889c1640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f187c094e10 0x7f187c12b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.934+0000 7f18891c2640 1 -- 172.21.15.82:0/2974010677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f187c094e10 msgr2=0x7f187c12b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.934+0000 7f18891c2640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f187c094e10 0x7f187c12b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.934+0000 7f18891c2640 1 -- 172.21.15.82:0/2974010677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187c0957d0 msgr2=0x7f187c12b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.934+0000 7f18891c2640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187c0957d0 0x7f187c12b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.934+0000 7f18891c2640 1 -- 172.21.15.82:0/2974010677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c1303d0 con 0x7f187c099bf0 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.934+0000 7f1883fff640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187c0957d0 0x7f187c12b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.934+0000 7f18891c2640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c099bf0 0x7f187c12fca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1884067c50 tx=0x7f188407aae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.934+0000 7f1881ffb640 1 -- 172.21.15.82:0/2974010677 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840698a0 con 0x7f187c099bf0 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.934+0000 7f1881ffb640 1 -- 172.21.15.82:0/2974010677 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1884084420 con 0x7f187c099bf0 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.934+0000 7f1881ffb640 1 -- 172.21.15.82:0/2974010677 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884079670 con 0x7f187c099bf0 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.934+0000 7f18899c3640 1 -- 172.21.15.82:0/2974010677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187c1306c0 con 0x7f187c099bf0 2024-04-21T08:52:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.935+0000 7f18899c3640 1 -- 172.21.15.82:0/2974010677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187c137f50 con 0x7f187c099bf0 2024-04-21T08:52:14.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.938+0000 7f1881ffb640 1 -- 172.21.15.82:0/2974010677 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1884069a40 con 0x7f187c099bf0 2024-04-21T08:52:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f188d812740 1 Processor -- start 2024-04-21T08:52:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f188d812740 1 -- start start 2024-04-21T08:52:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f1881ffb640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18740a3da0 0x7f18740a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f188d812740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3570 0x55c252ce3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f188d812740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3f20 0x55c252cee840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f188d812740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 0x55c252cf1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f188d812740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c252c481f0 con 0x55c252ceee10 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f188d812740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c252be54b0 con 0x55c252ce3f20 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f188d812740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c252c441b0 con 0x55c252ce3570 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f18891c2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 0x55c252cf1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f18889c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3570 0x55c252ce3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f18891c2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 0x55c252cf1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37210/0 (socket says 172.21.15.82:37210) 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f18891c2640 1 -- 172.21.15.82:0/1965486873 learned_addr learned my addr 172.21.15.82:0/1965486873 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f1883fff640 1 --2- 172.21.15.82:0/1965486873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3f20 0x55c252cee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f1881ffb640 1 -- 172.21.15.82:0/2974010677 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f188413d0a0 con 0x7f187c099bf0 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f1883fff640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18740a3da0 0x7f18740a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f18889c1640 1 -- 172.21.15.82:0/1965486873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3f20 msgr2=0x55c252cee840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f18889c1640 1 --2- 172.21.15.82:0/1965486873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3f20 0x55c252cee840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f18889c1640 1 -- 172.21.15.82:0/1965486873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 msgr2=0x55c252cf1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f18889c1640 1 --2- 172.21.15.82:0/1965486873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 0x55c252cf1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.939+0000 7f18889c1640 1 -- 172.21.15.82:0/1965486873 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c252bd71b0 con 0x55c252ce3570 2024-04-21T08:52:14.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.940+0000 7f18889c1640 1 --2- 172.21.15.82:0/1965486873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3570 0x55c252ce3950 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1878009b40 tx=0x7f18780113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dd543a6c1446387 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:14.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.940+0000 7f186effd640 1 -- 172.21.15.82:0/1965486873 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1878011e10 con 0x55c252ce3570 2024-04-21T08:52:14.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.940+0000 7f186effd640 1 -- 172.21.15.82:0/1965486873 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1878012900 con 0x55c252ce3570 2024-04-21T08:52:14.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.940+0000 7f186effd640 1 -- 172.21.15.82:0/1965486873 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187800fa00 con 0x55c252ce3570 2024-04-21T08:52:14.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.941+0000 7f188d812740 1 -- 172.21.15.82:0/1965486873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3570 msgr2=0x55c252ce3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.941+0000 7f188d812740 1 --2- 172.21.15.82:0/1965486873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3570 0x55c252ce3950 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1878009b40 tx=0x7f18780113e0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.941+0000 7f1883fff640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18740a3da0 0x7f18740a6260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1870006010 tx=0x7f18700058d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.941+0000 7f188d812740 1 -- 172.21.15.82:0/1965486873 shutdown_connections 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.941+0000 7f188d812740 1 --2- 172.21.15.82:0/1965486873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 0x55c252cf1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.941+0000 7f188d812740 1 --2- 172.21.15.82:0/1965486873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3f20 0x55c252cee840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.941+0000 7f188d812740 1 --2- 172.21.15.82:0/1965486873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3570 0x55c252ce3950 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.941+0000 7f188d812740 1 -- 172.21.15.82:0/1965486873 >> 172.21.15.82:0/1965486873 conn(0x55c252cddda0 msgr2=0x55c252cde180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.941+0000 7f188d812740 1 -- 172.21.15.82:0/1965486873 shutdown_connections 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.941+0000 7f188d812740 1 -- 172.21.15.82:0/1965486873 wait complete. 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.941+0000 7f188d812740 1 -- start start 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f188d812740 1 Processor -- start 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f188d812740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3570 0x55c252cee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f188d812740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ce3f20 0x55c252ceba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f188d812740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ceee10 0x55c252cebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f188d812740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c252cf1d40 con 0x55c252ce3f20 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f188d812740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c252c01500 con 0x55c252ce3570 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f188d812740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c252cf1ec0 con 0x55c252ceee10 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f188d812740 10 client.0 _ll_register_callbacks cb 0x55c252bcfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f18889c1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3570 0x55c252cee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f1883fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ce3f20 0x55c252ceba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f18889c1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3570 0x55c252cee6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:36510/0 (socket says 172.21.15.82:36510) 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f18889c1640 1 -- 172.21.15.82:0/3552015480 learned_addr learned my addr 172.21.15.82:0/3552015480 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f1883fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ce3f20 0x55c252ceba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37218/0 (socket says 172.21.15.82:37218) 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.942+0000 7f18891c2640 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ceee10 0x55c252cebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.943+0000 7f18891c2640 1 -- 172.21.15.82:0/3552015480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3570 msgr2=0x55c252cee6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.943+0000 7f18891c2640 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3570 0x55c252cee6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.943+0000 7f18891c2640 1 -- 172.21.15.82:0/3552015480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ce3f20 msgr2=0x55c252ceba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.943+0000 7f18891c2640 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ce3f20 0x55c252ceba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.943+0000 7f18891c2640 1 -- 172.21.15.82:0/3552015480 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c25295a3f0 con 0x55c252ceee10 2024-04-21T08:52:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.943+0000 7f18889c1640 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3570 0x55c252cee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.943+0000 7f1883fff640 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ce3f20 0x55c252ceba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.943+0000 7f18891c2640 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ceee10 0x55c252cebf70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f188413a4e0 tx=0x7f18841087c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30554fb02070cdc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.943+0000 7f1863fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:52:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.943+0000 7f1863fff640 1 -- 172.21.15.82:0/3552015480 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18841166e0 con 0x55c252ceee10 2024-04-21T08:52:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.943+0000 7f1863fff640 1 -- 172.21.15.82:0/3552015480 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f18840ea260 con 0x55c252ceee10 2024-04-21T08:52:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.943+0000 7f188d812740 1 -- 172.21.15.82:0/3552015480 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1878012900 con 0x55c252ceee10 2024-04-21T08:52:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.944+0000 7f1863fff640 1 -- 172.21.15.82:0/3552015480 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f18841127b0 con 0x55c252ceee10 2024-04-21T08:52:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.944+0000 7f188d812740 10 client.26212 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.944+0000 7f188d812740 1 -- 172.21.15.82:0/3552015480 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c252a247d0 con 0x55c252ceee10 2024-04-21T08:52:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.944+0000 7f1863fff640 1 -- 172.21.15.82:0/3552015480 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18840ff080 con 0x55c252ceee10 2024-04-21T08:52:14.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.944+0000 7f1863fff640 1 -- 172.21.15.82:0/3552015480 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f188408c030 con 0x55c252ceee10 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.944+0000 7f188d812740 10 client.26212 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.944+0000 7f188d812740 10 client.26212 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.944+0000 7f188d812740 20 client.26212 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.944+0000 7f188d812740 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c252cfef40 0x55c252d01350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.944+0000 7f188d812740 4 client.26212 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.944+0000 7f188d812740 1 -- 172.21.15.82:0/3552015480 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1878012900 con 0x55c252cfef40 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.945+0000 7f18889c1640 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c252cfef40 0x55c252d01350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.945+0000 7f1863fff640 10 client.26212 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.945+0000 7f18889c1640 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c252cfef40 0x55c252d01350 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98aebda10d8e9bb2 server_cookie=c1a0271cec523ad6 in_seq=0 out_seq=0 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f1863fff640 1 -- 172.21.15.82:0/3552015480 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f1878012900 con 0x55c252cfef40 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f1863fff640 10 client.26212 handle_command_reply: tid=0 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f188d812740 1 client.26212 shutdown 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f188d812740 1 -- 172.21.15.82:0/3552015480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ceee10 msgr2=0x55c252cebf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f188d812740 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ceee10 0x55c252cebf70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f188413a4e0 tx=0x7f18841087c0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f188d812740 1 -- 172.21.15.82:0/3552015480 shutdown_connections 2024-04-21T08:52:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f188d812740 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c252cfef40 0x55c252d01350 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f1863fff640 0 client.26212 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:52:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f188d812740 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ceee10 0x55c252cebf70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f188d812740 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ce3f20 0x55c252ceba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f188d812740 1 --2- 172.21.15.82:0/3552015480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3570 0x55c252cee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f188d812740 1 -- 172.21.15.82:0/3552015480 >> 172.21.15.82:0/3552015480 conn(0x55c252cddda0 msgr2=0x55c252cdfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f188d812740 1 -- 172.21.15.82:0/3552015480 shutdown_connections 2024-04-21T08:52:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.946+0000 7f188d812740 1 -- 172.21.15.82:0/3552015480 wait complete. 2024-04-21T08:52:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.947+0000 7f188d812740 20 client.26212 trim_cache size 0 max 16384 2024-04-21T08:52:14.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:14 smithi134 ceph-mon[31772]: pgmap v159: 129 pgs: 129 active+clean; 682 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.5 MiB/s wr, 1.10k op/s 2024-04-21T08:52:14.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.976+0000 7f188d812740 1 Processor -- start 2024-04-21T08:52:14.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.976+0000 7f188d812740 1 -- start start 2024-04-21T08:52:14.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f188d812740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3570 0x55c252cfac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f188d812740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ce3f20 0x55c252ce6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f188d812740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ceee10 0x55c252cee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f188d812740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f187c140e40 con 0x55c252ce3f20 2024-04-21T08:52:14.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f188d812740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f187c142a10 con 0x55c252ceee10 2024-04-21T08:52:14.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f188d812740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f187c141510 con 0x55c252ce3570 2024-04-21T08:52:14.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f18891c2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ceee10 0x55c252cee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f1883fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ce3f20 0x55c252ce6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f18891c2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ceee10 0x55c252cee160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:36526/0 (socket says 172.21.15.82:36526) 2024-04-21T08:52:14.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f18891c2640 1 -- 172.21.15.82:0/1708300807 learned_addr learned my addr 172.21.15.82:0/1708300807 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:14.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f18889c1640 1 --2- 172.21.15.82:0/1708300807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3570 0x55c252cfac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f18891c2640 1 -- 172.21.15.82:0/1708300807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3570 msgr2=0x55c252cfac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f18891c2640 1 --2- 172.21.15.82:0/1708300807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3570 0x55c252cfac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f18891c2640 1 -- 172.21.15.82:0/1708300807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ce3f20 msgr2=0x55c252ce6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f18891c2640 1 --2- 172.21.15.82:0/1708300807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ce3f20 0x55c252ce6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f18891c2640 1 -- 172.21.15.82:0/1708300807 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c252d01930 con 0x55c252ceee10 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.977+0000 7f18889c1640 1 --2- 172.21.15.82:0/1708300807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3570 0x55c252cfac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.978+0000 7f18891c2640 1 --2- 172.21.15.82:0/1708300807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ceee10 0x55c252cee160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1884108c10 tx=0x7f1884069f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad3e3c4decdc1771 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.978+0000 7f186e7fc640 1 -- 172.21.15.82:0/1708300807 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188410d030 con 0x55c252ceee10 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.978+0000 7f186e7fc640 1 -- 172.21.15.82:0/1708300807 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18841118f0 con 0x55c252ceee10 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.978+0000 7f186e7fc640 1 -- 172.21.15.82:0/1708300807 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884084830 con 0x55c252ceee10 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.979+0000 7f188d812740 1 -- 172.21.15.82:0/1708300807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ceee10 msgr2=0x55c252cee160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.979+0000 7f188d812740 1 --2- 172.21.15.82:0/1708300807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ceee10 0x55c252cee160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1884108c10 tx=0x7f1884069f40 comp rx=0 tx=0).stop 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.979+0000 7f188d812740 1 -- 172.21.15.82:0/1708300807 shutdown_connections 2024-04-21T08:52:14.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.979+0000 7f188d812740 1 --2- 172.21.15.82:0/1708300807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ceee10 0x55c252cee160 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.979+0000 7f188d812740 1 --2- 172.21.15.82:0/1708300807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ce3f20 0x55c252ce6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.979+0000 7f188d812740 1 --2- 172.21.15.82:0/1708300807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3570 0x55c252cfac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.979+0000 7f188d812740 1 -- 172.21.15.82:0/1708300807 >> 172.21.15.82:0/1708300807 conn(0x55c252cddda0 msgr2=0x55c252ce67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:14.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.979+0000 7f188d812740 1 -- 172.21.15.82:0/1708300807 shutdown_connections 2024-04-21T08:52:14.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.979+0000 7f188d812740 1 -- 172.21.15.82:0/1708300807 wait complete. 2024-04-21T08:52:14.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.979+0000 7f188d812740 1 -- start start 2024-04-21T08:52:14.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.979+0000 7f188d812740 1 Processor -- start 2024-04-21T08:52:14.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f188d812740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3570 0x55c252cf7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f188d812740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3f20 0x55c252cf26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f188d812740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 0x55c252cf2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f188d812740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f187c142340 con 0x55c252ceee10 2024-04-21T08:52:14.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f188d812740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f187c141c70 con 0x55c252ce3570 2024-04-21T08:52:14.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f188d812740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f187c13ee20 con 0x55c252ce3f20 2024-04-21T08:52:14.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f188d812740 10 client.0 _ll_register_callbacks cb 0x7f1884104ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:14.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f18889c1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3570 0x55c252cf7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f18891c2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 0x55c252cf2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f18891c2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 0x55c252cf2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37244/0 (socket says 172.21.15.82:37244) 2024-04-21T08:52:14.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f18891c2640 1 -- 172.21.15.82:0/2985073179 learned_addr learned my addr 172.21.15.82:0/2985073179 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:14.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f1883fff640 1 --2- 172.21.15.82:0/2985073179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3f20 0x55c252cf26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f18891c2640 1 -- 172.21.15.82:0/2985073179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3f20 msgr2=0x55c252cf26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:14.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f18891c2640 1 --2- 172.21.15.82:0/2985073179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3f20 0x55c252cf26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f18891c2640 1 -- 172.21.15.82:0/2985073179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3570 msgr2=0x55c252cf7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f18891c2640 1 --2- 172.21.15.82:0/2985073179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3570 0x55c252cf7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f18891c2640 1 -- 172.21.15.82:0/2985073179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c252cf3170 con 0x55c252ceee10 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f18891c2640 1 --2- 172.21.15.82:0/2985073179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 0x55c252cf2c30 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1884065700 tx=0x7f1884107190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd7d842a8124277 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.980+0000 7f1863fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f1863fff640 1 -- 172.21.15.82:0/2985073179 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884111460 con 0x55c252ceee10 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f188d812740 1 -- 172.21.15.82:0/2985073179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18841118f0 con 0x55c252ceee10 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f1863fff640 1 -- 172.21.15.82:0/2985073179 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f18840847e0 con 0x55c252ceee10 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f1863fff640 1 -- 172.21.15.82:0/2985073179 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1884108c80 con 0x55c252ceee10 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f188d812740 10 client.17145 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f188d812740 1 -- 172.21.15.82:0/2985073179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c252ce4950 con 0x55c252ceee10 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f1863fff640 1 -- 172.21.15.82:0/2985073179 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18841118f0 con 0x55c252ceee10 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f1863fff640 1 -- 172.21.15.82:0/2985073179 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f1884119070 con 0x55c252ceee10 2024-04-21T08:52:14.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f188d812740 10 client.17145 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f188d812740 10 client.17145 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f188d812740 20 client.17145 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f188d812740 1 --2- 172.21.15.82:0/2985073179 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c252cfef40 0x55c252ce4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f188d812740 4 client.17145 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.981+0000 7f188d812740 1 -- 172.21.15.82:0/2985073179 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c252be7930 con 0x55c252cfef40 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.982+0000 7f18889c1640 1 --2- 172.21.15.82:0/2985073179 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c252cfef40 0x55c252ce4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.982+0000 7f1863fff640 10 client.17145 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.982+0000 7f18889c1640 1 --2- 172.21.15.82:0/2985073179 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c252cfef40 0x55c252ce4ea0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70d80da9ad736b7d server_cookie=99de9b2f63c4939b in_seq=0 out_seq=0 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.982+0000 7f1863fff640 1 -- 172.21.15.82:0/2985073179 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c252be7930 con 0x55c252cfef40 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.982+0000 7f1863fff640 10 client.17145 handle_command_reply: tid=0 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.982+0000 7f188d812740 1 client.17145 shutdown 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.983+0000 7f188d812740 1 -- 172.21.15.82:0/2985073179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 msgr2=0x55c252cf2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.983+0000 7f188d812740 1 --2- 172.21.15.82:0/2985073179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 0x55c252cf2c30 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1884065700 tx=0x7f1884107190 comp rx=0 tx=0).stop 2024-04-21T08:52:14.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.983+0000 7f188d812740 1 -- 172.21.15.82:0/2985073179 shutdown_connections 2024-04-21T08:52:14.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.983+0000 7f188d812740 1 --2- 172.21.15.82:0/2985073179 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55c252cfef40 0x55c252ce4ea0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.983+0000 7f1863fff640 0 client.17145 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:52:14.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.983+0000 7f188d812740 1 --2- 172.21.15.82:0/2985073179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c252ceee10 0x55c252cf2c30 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.983+0000 7f188d812740 1 --2- 172.21.15.82:0/2985073179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c252ce3f20 0x55c252cf26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.983+0000 7f188d812740 1 --2- 172.21.15.82:0/2985073179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c252ce3570 0x55c252cf7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.983+0000 7f188d812740 1 -- 172.21.15.82:0/2985073179 >> 172.21.15.82:0/2985073179 conn(0x55c252cddda0 msgr2=0x55c252ce7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:14.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.983+0000 7f188d812740 1 -- 172.21.15.82:0/2985073179 shutdown_connections 2024-04-21T08:52:14.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.983+0000 7f188d812740 1 -- 172.21.15.82:0/2985073179 wait complete. 2024-04-21T08:52:14.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.983+0000 7f188d812740 20 client.17145 trim_cache size 0 max 16384 2024-04-21T08:52:14.988 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:52:14.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.984+0000 7f18899c3640 1 -- 172.21.15.82:0/2974010677 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18740a3da0 msgr2=0x7f18740a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.984+0000 7f18899c3640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18740a3da0 0x7f18740a6260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1870006010 tx=0x7f18700058d0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.984+0000 7f18899c3640 1 -- 172.21.15.82:0/2974010677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c099bf0 msgr2=0x7f187c12fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:14.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.984+0000 7f18899c3640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c099bf0 0x7f187c12fca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1884067c50 tx=0x7f188407aae0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.984+0000 7f18899c3640 1 -- 172.21.15.82:0/2974010677 shutdown_connections 2024-04-21T08:52:14.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.984+0000 7f18899c3640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c099bf0 0x7f187c12fca0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.984+0000 7f18899c3640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18740a3da0 0x7f18740a6260 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.984+0000 7f18899c3640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f187c0957d0 0x7f187c12b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.984+0000 7f18899c3640 1 --2- 172.21.15.82:0/2974010677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f187c094e10 0x7f187c12b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:14.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.984+0000 7f18899c3640 1 -- 172.21.15.82:0/2974010677 >> 172.21.15.82:0/2974010677 conn(0x7f187c00a7c0 msgr2=0x7f187c092ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:14.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.984+0000 7f18899c3640 1 -- 172.21.15.82:0/2974010677 shutdown_connections 2024-04-21T08:52:14.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:14.984+0000 7f18899c3640 1 -- 172.21.15.82:0/2974010677 wait complete. 2024-04-21T08:52:14.998 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:52:15.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.118+0000 7f8d7530e640 1 -- 172.21.15.82:0/2932355526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d70102190 msgr2=0x7f8d70102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:15.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.118+0000 7f8d7530e640 1 --2- 172.21.15.82:0/2932355526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d70102190 0x7f8d70102570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009900 tx=0x7f8d6002bc30 comp rx=0 tx=0).stop 2024-04-21T08:52:15.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.118+0000 7f8d7530e640 1 -- 172.21.15.82:0/2932355526 shutdown_connections 2024-04-21T08:52:15.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.118+0000 7f8d7530e640 1 --2- 172.21.15.82:0/2932355526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d70107260 0x7f8d7010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.118+0000 7f8d7530e640 1 --2- 172.21.15.82:0/2932355526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d70102b40 0x7f8d70106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.118+0000 7f8d7530e640 1 --2- 172.21.15.82:0/2932355526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d70102190 0x7f8d70102570 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.118+0000 7f8d7530e640 1 -- 172.21.15.82:0/2932355526 >> 172.21.15.82:0/2932355526 conn(0x7f8d700793a0 msgr2=0x7f8d700797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.118+0000 7f8d7530e640 1 -- 172.21.15.82:0/2932355526 shutdown_connections 2024-04-21T08:52:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.118+0000 7f8d7530e640 1 -- 172.21.15.82:0/2932355526 wait complete. 2024-04-21T08:52:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d7530e640 1 Processor -- start 2024-04-21T08:52:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d7530e640 1 -- start start 2024-04-21T08:52:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d7530e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d70102190 0x7f8d7019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d7530e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d70107260 0x7f8d7019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d7530e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d701985c0 0x7f8d70198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d7530e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701121c0 con 0x7f8d701985c0 2024-04-21T08:52:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d7530e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70112040 con 0x7f8d70102190 2024-04-21T08:52:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d7530e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70112340 con 0x7f8d70107260 2024-04-21T08:52:15.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d6f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d701985c0 0x7f8d70198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d6effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d70102190 0x7f8d7019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d6f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d701985c0 0x7f8d70198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37268/0 (socket says 172.21.15.82:37268) 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d6f7fe640 1 -- 172.21.15.82:0/3663220121 learned_addr learned my addr 172.21.15.82:0/3663220121 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.119+0000 7f8d667fc640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d70107260 0x7f8d7019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d6effd640 1 -- 172.21.15.82:0/3663220121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d70107260 msgr2=0x7f8d7019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d6effd640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d70107260 0x7f8d7019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d6effd640 1 -- 172.21.15.82:0/3663220121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d701985c0 msgr2=0x7f8d70198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d6effd640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d701985c0 0x7f8d70198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d6effd640 1 -- 172.21.15.82:0/3663220121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d701990e0 con 0x7f8d70102190 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d667fc640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d70107260 0x7f8d7019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d6f7fe640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d701985c0 0x7f8d70198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d6effd640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d70102190 0x7f8d7019e410 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8d60036040 tx=0x7f8d600045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d6cff9640 1 -- 172.21.15.82:0/3663220121 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60054020 con 0x7f8d70102190 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d7530e640 1 -- 172.21.15.82:0/3663220121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d70199370 con 0x7f8d70102190 2024-04-21T08:52:15.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d6cff9640 1 -- 172.21.15.82:0/3663220121 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d6002ca70 con 0x7f8d70102190 2024-04-21T08:52:15.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d6cff9640 1 -- 172.21.15.82:0/3663220121 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60003a50 con 0x7f8d70102190 2024-04-21T08:52:15.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.120+0000 7f8d7530e640 1 -- 172.21.15.82:0/3663220121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7006bcb0 con 0x7f8d70102190 2024-04-21T08:52:15.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.122+0000 7f8d6cff9640 1 -- 172.21.15.82:0/3663220121 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8d60057020 con 0x7f8d70102190 2024-04-21T08:52:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.123+0000 7f8d6cff9640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f8d5c0a3d20 0x7f8d5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.123+0000 7f8d6cff9640 1 -- 172.21.15.82:0/3663220121 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f8d60103af0 con 0x7f8d70102190 2024-04-21T08:52:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.123+0000 7f8d667fc640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f8d5c0a3d20 0x7f8d5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.124+0000 7f8d667fc640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f8d5c0a3d20 0x7f8d5c0a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8d70101920 tx=0x7f8d58005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.124+0000 7f8d77fdc740 1 Processor -- start 2024-04-21T08:52:15.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.124+0000 7f8d77fdc740 1 -- start start 2024-04-21T08:52:15.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.124+0000 7f8d77fdc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b291a4d0 0x5565b291a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d77fdc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 0x5565b2923430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d77fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b2923a00 0x5565b2925e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d77fdc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5565b287f1f0 con 0x5565b291ae80 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d77fdc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5565b281c4b0 con 0x5565b291a4d0 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d77fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565b287b1b0 con 0x5565b2923a00 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d6effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b291a4d0 0x5565b291a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d667fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 0x5565b2923430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d6effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b291a4d0 0x5565b291a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:36562/0 (socket says 172.21.15.82:36562) 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d6effd640 1 -- 172.21.15.82:0/3707437643 learned_addr learned my addr 172.21.15.82:0/3707437643 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d6f7fe640 1 --2- 172.21.15.82:0/3707437643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b2923a00 0x5565b2925e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d6effd640 1 -- 172.21.15.82:0/3707437643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b2923a00 msgr2=0x5565b2925e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d6effd640 1 --2- 172.21.15.82:0/3707437643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b2923a00 0x5565b2925e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d6effd640 1 -- 172.21.15.82:0/3707437643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 msgr2=0x5565b2923430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d6effd640 1 --2- 172.21.15.82:0/3707437643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 0x5565b2923430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d6effd640 1 -- 172.21.15.82:0/3707437643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b280e1b0 con 0x5565b291a4d0 2024-04-21T08:52:15.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d6effd640 1 --2- 172.21.15.82:0/3707437643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b291a4d0 0x5565b291a8b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8d600cbc40 tx=0x7f8d600cf3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8308f7a477531501 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d657fa640 1 -- 172.21.15.82:0/3707437643 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d600cf7e0 con 0x5565b291a4d0 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d657fa640 1 -- 172.21.15.82:0/3707437643 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d600d3b90 con 0x5565b291a4d0 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.125+0000 7f8d657fa640 1 -- 172.21.15.82:0/3707437643 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d600cfa40 con 0x5565b291a4d0 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.126+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3707437643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b291a4d0 msgr2=0x5565b291a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.126+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/3707437643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b291a4d0 0x5565b291a8b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8d600cbc40 tx=0x7f8d600cf3e0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.126+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3707437643 shutdown_connections 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.126+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/3707437643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b2923a00 0x5565b2925e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.126+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/3707437643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 0x5565b2923430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.126+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/3707437643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b291a4d0 0x5565b291a8b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.126+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3707437643 >> 172.21.15.82:0/3707437643 conn(0x5565b2914ba0 msgr2=0x5565b2914f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.126+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3707437643 shutdown_connections 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.126+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3707437643 wait complete. 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d77fdc740 1 -- start start 2024-04-21T08:52:15.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d77fdc740 1 Processor -- start 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d77fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b2923360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d77fdc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 0x5565b29206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d77fdc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b2923a00 0x5565b2920d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d77fdc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5565b2926730 con 0x5565b291ae80 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d77fdc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5565b2838500 con 0x5565b2923a00 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d77fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565b29268b0 con 0x5565b291a4d0 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d77fdc740 10 client.0 _ll_register_callbacks cb 0x5565b2806b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d6f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b2923a00 0x5565b2920d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d667fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 0x5565b29206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d6effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b2923360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d6f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b2923a00 0x5565b2920d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:36574/0 (socket says 172.21.15.82:36574) 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d6f7fe640 1 -- 172.21.15.82:0/3103867716 learned_addr learned my addr 172.21.15.82:0/3103867716 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d6f7fe640 1 -- 172.21.15.82:0/3103867716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 msgr2=0x5565b2923360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:15.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d6f7fe640 1 --2- 172.21.15.82:0/3103867716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b2923360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d6f7fe640 1 -- 172.21.15.82:0/3103867716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 msgr2=0x5565b29206f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d6f7fe640 1 --2- 172.21.15.82:0/3103867716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 0x5565b29206f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.127+0000 7f8d6f7fe640 1 -- 172.21.15.82:0/3103867716 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b25913f0 con 0x5565b2923a00 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.128+0000 7f8d6f7fe640 1 --2- 172.21.15.82:0/3103867716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b2923a00 0x5565b2920d00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800ef20 tx=0x7f8d68011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23594fec6bb785c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.128+0000 7f8d567fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.128+0000 7f8d567fc640 1 -- 172.21.15.82:0/3103867716 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6801e020 con 0x5565b2923a00 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.128+0000 7f8d567fc640 1 -- 172.21.15.82:0/3103867716 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f8d68003970 con 0x5565b2923a00 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.128+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3103867716 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d600d3b90 con 0x5565b2923a00 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.128+0000 7f8d567fc640 1 -- 172.21.15.82:0/3103867716 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8d68026050 con 0x5565b2923a00 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.128+0000 7f8d77fdc740 10 client.26507 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.128+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3103867716 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5565b265b7d0 con 0x5565b2923a00 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.128+0000 7f8d567fc640 1 -- 172.21.15.82:0/3103867716 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d600d3b90 con 0x5565b2923a00 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.128+0000 7f8d567fc640 1 -- 172.21.15.82:0/3103867716 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f8d68019070 con 0x5565b2923a00 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.129+0000 7f8d77fdc740 10 client.26507 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.129+0000 7f8d77fdc740 10 client.26507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:15.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.129+0000 7f8d77fdc740 20 client.26507 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:15.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.129+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/3103867716 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5565b2933a30 0x5565b2935e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.129+0000 7f8d77fdc740 4 client.26507 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:15.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.129+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3103867716 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565b281e930 con 0x5565b2933a30 2024-04-21T08:52:15.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.129+0000 7f8d6effd640 1 --2- 172.21.15.82:0/3103867716 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5565b2933a30 0x5565b2935e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.129+0000 7f8d567fc640 10 client.26507 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:15.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.130+0000 7f8d6effd640 1 --2- 172.21.15.82:0/3103867716 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5565b2933a30 0x5565b2935e60 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d71f58053ac825 server_cookie=67fe364b2545c598 in_seq=0 out_seq=0 2024-04-21T08:52:15.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d567fc640 1 -- 172.21.15.82:0/3103867716 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565b281e930 con 0x5565b2933a30 2024-04-21T08:52:15.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d567fc640 10 client.26507 handle_command_reply: tid=0 2024-04-21T08:52:15.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d77fdc740 1 client.26507 shutdown 2024-04-21T08:52:15.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3103867716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b2923a00 msgr2=0x5565b2920d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:15.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/3103867716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b2923a00 0x5565b2920d00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800ef20 tx=0x7f8d68011810 comp rx=0 tx=0).stop 2024-04-21T08:52:15.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3103867716 shutdown_connections 2024-04-21T08:52:15.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/3103867716 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5565b2933a30 0x5565b2935e60 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/3103867716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b2923a00 0x5565b2920d00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d567fc640 0 client.26507 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:15.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/3103867716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 0x5565b29206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/3103867716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b2923360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3103867716 >> 172.21.15.82:0/3103867716 conn(0x5565b2914ba0 msgr2=0x5565b29168f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:15.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3103867716 shutdown_connections 2024-04-21T08:52:15.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d77fdc740 1 -- 172.21.15.82:0/3103867716 wait complete. 2024-04-21T08:52:15.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.131+0000 7f8d77fdc740 20 client.26507 trim_cache size 0 max 16384 2024-04-21T08:52:15.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.146+0000 7f8d77fdc740 1 Processor -- start 2024-04-21T08:52:15.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.146+0000 7f8d77fdc740 1 -- start start 2024-04-21T08:52:15.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.146+0000 7f8d77fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b292f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.146+0000 7f8d77fdc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 0x5565b2922e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d77fdc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b2923a00 0x5565b2923360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d77fdc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70075ac0 con 0x5565b291ae80 2024-04-21T08:52:15.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d77fdc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701a65d0 con 0x5565b2923a00 2024-04-21T08:52:15.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d77fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70076190 con 0x5565b291a4d0 2024-04-21T08:52:15.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d667fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 0x5565b2922e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d6effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b292f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d6effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b292f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52062/0 (socket says 172.21.15.82:52062) 2024-04-21T08:52:15.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d6effd640 1 -- 172.21.15.82:0/2716270589 learned_addr learned my addr 172.21.15.82:0/2716270589 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:15.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d6f7fe640 1 --2- 172.21.15.82:0/2716270589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b2923a00 0x5565b2923360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d6effd640 1 -- 172.21.15.82:0/2716270589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b2923a00 msgr2=0x5565b2923360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:15.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d6effd640 1 --2- 172.21.15.82:0/2716270589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b2923a00 0x5565b2923360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d6effd640 1 -- 172.21.15.82:0/2716270589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 msgr2=0x5565b2922e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:15.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d6effd640 1 --2- 172.21.15.82:0/2716270589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 0x5565b2922e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d6effd640 1 -- 172.21.15.82:0/2716270589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b291ec60 con 0x5565b291a4d0 2024-04-21T08:52:15.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d6effd640 1 --2- 172.21.15.82:0/2716270589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b292f470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8d6002c170 tx=0x7f8d60009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13a28e2cdbbe75cd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:15.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.147+0000 7f8d64ff9640 1 -- 172.21.15.82:0/2716270589 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d600cf7e0 con 0x5565b291a4d0 2024-04-21T08:52:15.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.148+0000 7f8d64ff9640 1 -- 172.21.15.82:0/2716270589 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d600cdae0 con 0x5565b291a4d0 2024-04-21T08:52:15.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.148+0000 7f8d64ff9640 1 -- 172.21.15.82:0/2716270589 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60005340 con 0x5565b291a4d0 2024-04-21T08:52:15.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.148+0000 7f8d77fdc740 1 -- 172.21.15.82:0/2716270589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 msgr2=0x5565b292f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:15.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.148+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/2716270589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b292f470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8d6002c170 tx=0x7f8d60009560 comp rx=0 tx=0).stop 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.148+0000 7f8d77fdc740 1 -- 172.21.15.82:0/2716270589 shutdown_connections 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.148+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/2716270589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b2923a00 0x5565b2923360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.148+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/2716270589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b291ae80 0x5565b2922e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.148+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/2716270589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b292f470 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.148+0000 7f8d77fdc740 1 -- 172.21.15.82:0/2716270589 >> 172.21.15.82:0/2716270589 conn(0x5565b2914ba0 msgr2=0x5565b2916730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d77fdc740 1 -- 172.21.15.82:0/2716270589 shutdown_connections 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d77fdc740 1 -- 172.21.15.82:0/2716270589 wait complete. 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d77fdc740 1 -- start start 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d77fdc740 1 Processor -- start 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d77fdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b292b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d77fdc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b291ae80 0x5565b292bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d77fdc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b2923a00 0x5565b2928950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d77fdc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701a5f00 con 0x5565b2923a00 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d77fdc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701a5830 con 0x5565b291ae80 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d77fdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70073aa0 con 0x5565b291a4d0 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d77fdc740 10 client.0 _ll_register_callbacks cb 0x7f8d6800eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d6effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b292b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d6f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b2923a00 0x5565b2928950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d6f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b2923a00 0x5565b2928950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37304/0 (socket says 172.21.15.82:37304) 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d6effd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b292b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52078/0 (socket says 172.21.15.82:52078) 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.149+0000 7f8d6f7fe640 1 -- 172.21.15.82:0/603443812 learned_addr learned my addr 172.21.15.82:0/603443812 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d667fc640 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b291ae80 0x5565b292bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d6effd640 1 -- 172.21.15.82:0/603443812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b291ae80 msgr2=0x5565b292bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d6effd640 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b291ae80 0x5565b292bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d6effd640 1 -- 172.21.15.82:0/603443812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b2923a00 msgr2=0x5565b2928950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:15.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d6effd640 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b2923a00 0x5565b2928950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d6effd640 1 -- 172.21.15.82:0/603443812 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b2929150 con 0x5565b291a4d0 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d6f7fe640 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b2923a00 0x5565b2928950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d6effd640 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b292b980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d600cbc40 tx=0x7f8d600cdc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=176f6ed34bec9d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d567fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d567fc640 1 -- 172.21.15.82:0/603443812 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d600de060 con 0x5565b291a4d0 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d567fc640 1 -- 172.21.15.82:0/603443812 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f8d60101100 con 0x5565b291a4d0 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d77fdc740 1 -- 172.21.15.82:0/603443812 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d600cdae0 con 0x5565b291a4d0 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d567fc640 1 -- 172.21.15.82:0/603443812 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8d600d3b90 con 0x5565b291a4d0 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d77fdc740 10 client.26236 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.150+0000 7f8d77fdc740 1 -- 172.21.15.82:0/603443812 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5565b299aa50 con 0x5565b291a4d0 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.151+0000 7f8d567fc640 1 -- 172.21.15.82:0/603443812 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d600cdae0 con 0x5565b291a4d0 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.151+0000 7f8d567fc640 1 -- 172.21.15.82:0/603443812 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f8d600cfa40 con 0x5565b291a4d0 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.151+0000 7f8d77fdc740 10 client.26236 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.151+0000 7f8d77fdc740 10 client.26236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.151+0000 7f8d77fdc740 20 client.26236 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.151+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5565b299ade0 0x5565b299d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.151+0000 7f8d77fdc740 4 client.26236 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.151+0000 7f8d77fdc740 1 -- 172.21.15.82:0/603443812 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565b288cc10 con 0x5565b299ade0 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.151+0000 7f8d667fc640 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5565b299ade0 0x5565b299d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d567fc640 10 client.26236 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d667fc640 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5565b299ade0 0x5565b299d1d0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8438d33d4c8799cd server_cookie=22f27c4aafb2b42e in_seq=0 out_seq=0 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d567fc640 1 -- 172.21.15.82:0/603443812 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565b288cc10 con 0x5565b299ade0 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d567fc640 10 client.26236 handle_command_reply: tid=0 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d77fdc740 1 client.26236 shutdown 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d77fdc740 1 -- 172.21.15.82:0/603443812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 msgr2=0x5565b292b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b292b980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d600cbc40 tx=0x7f8d600cdc90 comp rx=0 tx=0).stop 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d77fdc740 1 -- 172.21.15.82:0/603443812 shutdown_connections 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5565b299ade0 0x5565b299d1d0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d567fc640 0 client.26236 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565b2923a00 0x5565b2928950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5565b291ae80 0x5565b292bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d77fdc740 1 --2- 172.21.15.82:0/603443812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565b291a4d0 0x5565b292b980 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d77fdc740 1 -- 172.21.15.82:0/603443812 >> 172.21.15.82:0/603443812 conn(0x5565b2914ba0 msgr2=0x5565b291b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:15.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d77fdc740 1 -- 172.21.15.82:0/603443812 shutdown_connections 2024-04-21T08:52:15.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d77fdc740 1 -- 172.21.15.82:0/603443812 wait complete. 2024-04-21T08:52:15.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.152+0000 7f8d77fdc740 20 client.26236 trim_cache size 0 max 16384 2024-04-21T08:52:15.158 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:52:15.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.153+0000 7f8d7530e640 1 -- 172.21.15.82:0/3663220121 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f8d5c0a3d20 msgr2=0x7f8d5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:15.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.153+0000 7f8d7530e640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f8d5c0a3d20 0x7f8d5c0a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8d70101920 tx=0x7f8d58005f90 comp rx=0 tx=0).stop 2024-04-21T08:52:15.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.153+0000 7f8d7530e640 1 -- 172.21.15.82:0/3663220121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d70102190 msgr2=0x7f8d7019e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:15.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.153+0000 7f8d7530e640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d70102190 0x7f8d7019e410 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8d60036040 tx=0x7f8d600045e0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.153+0000 7f8d7530e640 1 -- 172.21.15.82:0/3663220121 shutdown_connections 2024-04-21T08:52:15.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.153+0000 7f8d7530e640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f8d5c0a3d20 0x7f8d5c0a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.153+0000 7f8d7530e640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d701985c0 0x7f8d70198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.153+0000 7f8d7530e640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d70107260 0x7f8d7019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.153+0000 7f8d7530e640 1 --2- 172.21.15.82:0/3663220121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d70102190 0x7f8d7019e410 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:15.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.153+0000 7f8d7530e640 1 -- 172.21.15.82:0/3663220121 >> 172.21.15.82:0/3663220121 conn(0x7f8d700793a0 msgr2=0x7f8d70100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:15.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.153+0000 7f8d7530e640 1 -- 172.21.15.82:0/3663220121 shutdown_connections 2024-04-21T08:52:15.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:15.153+0000 7f8d7530e640 1 -- 172.21.15.82:0/3663220121 wait complete. 2024-04-21T08:52:16.173 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:52:16.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:52:16.174 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:52:16.231 INFO:tasks.workunit.client.0.smithi082.stdout: 113 1190101 290 827234 226 646689 747 2024-04-21T08:52:16.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:16 smithi082 ceph-mon[27967]: pgmap v160: 129 pgs: 129 active+clean; 682 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.5 MiB/s wr, 915 op/s 2024-04-21T08:52:16.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:16 smithi106 ceph-mon[31072]: pgmap v160: 129 pgs: 129 active+clean; 682 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.5 MiB/s wr, 915 op/s 2024-04-21T08:52:16.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:16 smithi134 ceph-mon[31772]: pgmap v160: 129 pgs: 129 active+clean; 682 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.5 MiB/s wr, 915 op/s 2024-04-21T08:52:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.723+0000 7f88b8f59640 1 -- 172.21.15.82:0/762827189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4110550 msgr2=0x7f88b4110930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:17.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.723+0000 7f88b8f59640 1 --2- 172.21.15.82:0/762827189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4110550 0x7f88b4110930 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009900 tx=0x7f88a402bcf0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.732+0000 7f88b8f59640 1 -- 172.21.15.82:0/762827189 shutdown_connections 2024-04-21T08:52:17.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.732+0000 7f88b8f59640 1 --2- 172.21.15.82:0/762827189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88b410eb40 0x7f88b411d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.732+0000 7f88b8f59640 1 --2- 172.21.15.82:0/762827189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88b410e0c0 0x7f88b410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.732+0000 7f88b8f59640 1 --2- 172.21.15.82:0/762827189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4110550 0x7f88b4110930 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.732+0000 7f88b8f59640 1 -- 172.21.15.82:0/762827189 >> 172.21.15.82:0/762827189 conn(0x7f88b406ce60 msgr2=0x7f88b406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:17.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.735+0000 7f88b8f59640 1 -- 172.21.15.82:0/762827189 shutdown_connections 2024-04-21T08:52:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.735+0000 7f88b8f59640 1 -- 172.21.15.82:0/762827189 wait complete. 2024-04-21T08:52:17.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.736+0000 7f88b8f59640 1 Processor -- start 2024-04-21T08:52:17.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.736+0000 7f88b8f59640 1 -- start start 2024-04-21T08:52:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.736+0000 7f88b8f59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88b410e0c0 0x7f88b411cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.736+0000 7f88b8f59640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88b410eb40 0x7f88b4115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b8f59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4110550 0x7f88b4116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b8f59640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4120e30 con 0x7f88b4110550 2024-04-21T08:52:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b8f59640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4120cb0 con 0x7f88b410eb40 2024-04-21T08:52:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b8f59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4120fb0 con 0x7f88b410e0c0 2024-04-21T08:52:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88b410e0c0 0x7f88b411cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4110550 0x7f88b4116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4110550 0x7f88b4116340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37328/0 (socket says 172.21.15.82:37328) 2024-04-21T08:52:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b2d9d640 1 -- 172.21.15.82:0/3819832909 learned_addr learned my addr 172.21.15.82:0/3819832909 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88b410eb40 0x7f88b4115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b1d9b640 1 -- 172.21.15.82:0/3819832909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88b410e0c0 msgr2=0x7f88b411cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88b410e0c0 0x7f88b411cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b1d9b640 1 -- 172.21.15.82:0/3819832909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4110550 msgr2=0x7f88b4116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4110550 0x7f88b4116340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.737+0000 7f88b1d9b640 1 -- 172.21.15.82:0/3819832909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b41168d0 con 0x7f88b410eb40 2024-04-21T08:52:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.738+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88b410eb40 0x7f88b4115e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f88a800d6d0 tx=0x7f88a800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.757+0000 7f88a37fe640 1 -- 172.21.15.82:0/3819832909 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8017070 con 0x7f88b410eb40 2024-04-21T08:52:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.757+0000 7f88b8f59640 1 -- 172.21.15.82:0/3819832909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b4116bc0 con 0x7f88b410eb40 2024-04-21T08:52:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.757+0000 7f88b8f59640 1 -- 172.21.15.82:0/3819832909 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b411abc0 con 0x7f88b410eb40 2024-04-21T08:52:17.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.759+0000 7f88a37fe640 1 -- 172.21.15.82:0/3819832909 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a8009820 con 0x7f88b410eb40 2024-04-21T08:52:17.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.759+0000 7f88a37fe640 1 -- 172.21.15.82:0/3819832909 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8004c40 con 0x7f88b410eb40 2024-04-21T08:52:17.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.762+0000 7f88bbc2e740 1 Processor -- start 2024-04-21T08:52:17.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.763+0000 7f88bbc2e740 1 -- start start 2024-04-21T08:52:17.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.763+0000 7f88a37fe640 1 -- 172.21.15.82:0/3819832909 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f88a8010460 con 0x7f88b410eb40 2024-04-21T08:52:17.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.764+0000 7f88bbc2e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620703a80 0x55e620703e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.764+0000 7f88bbc2e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620704430 0x55e62070c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.764+0000 7f88bbc2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 0x55e62070f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.764+0000 7f88bbc2e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e620667f00 con 0x55e620704430 2024-04-21T08:52:17.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.764+0000 7f88bbc2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e620605590 con 0x55e62070cfb0 2024-04-21T08:52:17.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.764+0000 7f88bbc2e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e620663ec0 con 0x55e620703a80 2024-04-21T08:52:17.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.764+0000 7f88b1d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 0x55e62070f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.764+0000 7f88b1d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 0x55e62070f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:36622/0 (socket says 172.21.15.82:36622) 2024-04-21T08:52:17.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.764+0000 7f88b1d9b640 1 -- 172.21.15.82:0/660566483 learned_addr learned my addr 172.21.15.82:0/660566483 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:17.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.764+0000 7f88b2d9d640 1 --2- 172.21.15.82:0/660566483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620703a80 0x55e620703e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.764+0000 7f88b259c640 1 --2- 172.21.15.82:0/660566483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620704430 0x55e62070c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.765+0000 7f88b1d9b640 1 -- 172.21.15.82:0/660566483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620703a80 msgr2=0x55e620703e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:17.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.765+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/660566483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620703a80 0x55e620703e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.765+0000 7f88b1d9b640 1 -- 172.21.15.82:0/660566483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620704430 msgr2=0x55e62070c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:17.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.765+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/660566483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620704430 0x55e62070c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.765+0000 7f88b1d9b640 1 -- 172.21.15.82:0/660566483 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6204427e0 con 0x55e62070cfb0 2024-04-21T08:52:17.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.765+0000 7f88b259c640 1 --2- 172.21.15.82:0/660566483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620704430 0x55e62070c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:17.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.765+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/660566483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 0x55e62070f400 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f88a80cd580 tx=0x7f88a80d6a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b3bfcfb11cbef24 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:17.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.766+0000 7f88a37fe640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f889c0a3f30 0x7f889c0a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.766+0000 7f88b2d9d640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f889c0a3f30 0x7f889c0a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.767+0000 7f88b2d9d640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f889c0a3f30 0x7f889c0a63f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f88ac007f20 tx=0x7f88ac007560 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:17.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.767+0000 7f88a37fe640 1 -- 172.21.15.82:0/3819832909 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f88a80d0310 con 0x7f88b410eb40 2024-04-21T08:52:17.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.778+0000 7f88a0ff9640 1 -- 172.21.15.82:0/660566483 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a80e3070 con 0x55e62070cfb0 2024-04-21T08:52:17.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.779+0000 7f88a0ff9640 1 -- 172.21.15.82:0/660566483 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a80d0ce0 con 0x55e62070cfb0 2024-04-21T08:52:17.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.780+0000 7f88bbc2e740 1 -- 172.21.15.82:0/660566483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 msgr2=0x55e62070f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:17.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.780+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/660566483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 0x55e62070f400 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f88a80cd580 tx=0x7f88a80d6a70 comp rx=0 tx=0).stop 2024-04-21T08:52:17.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.791+0000 7f88bbc2e740 1 -- 172.21.15.82:0/660566483 shutdown_connections 2024-04-21T08:52:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.791+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/660566483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 0x55e62070f400 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.791+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/660566483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620704430 0x55e62070c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.791+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/660566483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620703a80 0x55e620703e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.791+0000 7f88bbc2e740 1 -- 172.21.15.82:0/660566483 >> 172.21.15.82:0/660566483 conn(0x55e6206fe150 msgr2=0x55e6206fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.791+0000 7f88bbc2e740 1 -- 172.21.15.82:0/660566483 shutdown_connections 2024-04-21T08:52:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.791+0000 7f88bbc2e740 1 -- 172.21.15.82:0/660566483 wait complete. 2024-04-21T08:52:17.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.792+0000 7f88bbc2e740 1 -- start start 2024-04-21T08:52:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.792+0000 7f88bbc2e740 1 Processor -- start 2024-04-21T08:52:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.792+0000 7f88bbc2e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620703a80 0x55e620709ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.792+0000 7f88bbc2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620704430 0x55e62070a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.792+0000 7f88bbc2e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e62070cfb0 0x55e62070a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.792+0000 7f88bbc2e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e62070ff70 con 0x55e62070cfb0 2024-04-21T08:52:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.792+0000 7f88bbc2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e620621580 con 0x55e620704430 2024-04-21T08:52:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.792+0000 7f88bbc2e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e6207100f0 con 0x55e620703a80 2024-04-21T08:52:17.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.792+0000 7f88bbc2e740 10 client.0 _ll_register_callbacks cb 0x55e620627650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:17.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.793+0000 7f88b1d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620704430 0x55e62070a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.793+0000 7f88b1d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620704430 0x55e62070a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:36634/0 (socket says 172.21.15.82:36634) 2024-04-21T08:52:17.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.793+0000 7f88b1d9b640 1 -- 172.21.15.82:0/993961989 learned_addr learned my addr 172.21.15.82:0/993961989 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:17.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.793+0000 7f88b2d9d640 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e62070cfb0 0x55e62070a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.793+0000 7f88b259c640 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620703a80 0x55e620709ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.793+0000 7f88b1d9b640 1 -- 172.21.15.82:0/993961989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620703a80 msgr2=0x55e620709ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:17.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.793+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620703a80 0x55e620709ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.793+0000 7f88b1d9b640 1 -- 172.21.15.82:0/993961989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e62070cfb0 msgr2=0x55e62070a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:17.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.793+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e62070cfb0 0x55e62070a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.793+0000 7f88b1d9b640 1 -- 172.21.15.82:0/993961989 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6204a2f70 con 0x55e620704430 2024-04-21T08:52:17.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.793+0000 7f88b2d9d640 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e62070cfb0 0x55e62070a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:17.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.793+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620704430 0x55e62070a3e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009610 tx=0x7f88a8005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7de06271353e52ac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.809+0000 7f8899ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:52:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.809+0000 7f8899ffb640 1 -- 172.21.15.82:0/993961989 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8020050 con 0x55e620704430 2024-04-21T08:52:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.811+0000 7f88bbc2e740 1 -- 172.21.15.82:0/993961989 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a80d0ce0 con 0x55e620704430 2024-04-21T08:52:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.811+0000 7f8899ffb640 1 -- 172.21.15.82:0/993961989 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f88a80cd5b0 con 0x55e620704430 2024-04-21T08:52:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.812+0000 7f8899ffb640 1 -- 172.21.15.82:0/993961989 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f88a8098130 con 0x55e620704430 2024-04-21T08:52:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.812+0000 7f88bbc2e740 10 client.26522 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.812+0000 7f88bbc2e740 1 -- 172.21.15.82:0/993961989 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e62033a170 con 0x55e620704430 2024-04-21T08:52:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.815+0000 7f8899ffb640 1 -- 172.21.15.82:0/993961989 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a80d0ce0 con 0x55e620704430 2024-04-21T08:52:17.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.816+0000 7f8899ffb640 1 -- 172.21.15.82:0/993961989 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f88a8098500 con 0x55e620704430 2024-04-21T08:52:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.816+0000 7f88bbc2e740 10 client.26522 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.816+0000 7f88bbc2e740 10 client.26522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.816+0000 7f88bbc2e740 20 client.26522 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.816+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e62071d2f0 0x55e62071f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.816+0000 7f88bbc2e740 4 client.26522 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.816+0000 7f88bbc2e740 1 -- 172.21.15.82:0/993961989 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e620675920 con 0x55e62071d2f0 2024-04-21T08:52:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.817+0000 7f88b259c640 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e62071d2f0 0x55e62071f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.817+0000 7f88b259c640 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e62071d2f0 0x55e62071f6e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a83cce7f3f416778 server_cookie=810efbc8f92eb161 in_seq=0 out_seq=0 2024-04-21T08:52:17.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.820+0000 7f8899ffb640 10 client.26522 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:17.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.820+0000 7f8899ffb640 1 -- 172.21.15.82:0/993961989 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e620675920 con 0x55e62071d2f0 2024-04-21T08:52:17.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.820+0000 7f8899ffb640 10 client.26522 handle_command_reply: tid=0 2024-04-21T08:52:17.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.820+0000 7f88bbc2e740 1 client.26522 shutdown 2024-04-21T08:52:17.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.823+0000 7f88bbc2e740 1 -- 172.21.15.82:0/993961989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620704430 msgr2=0x55e62070a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:17.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.823+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620704430 0x55e62070a3e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009610 tx=0x7f88a8005d30 comp rx=0 tx=0).stop 2024-04-21T08:52:17.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.823+0000 7f88bbc2e740 1 -- 172.21.15.82:0/993961989 shutdown_connections 2024-04-21T08:52:17.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.823+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e62071d2f0 0x55e62071f6e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.823+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e62070cfb0 0x55e62070a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.823+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620704430 0x55e62070a3e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.823+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/993961989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620703a80 0x55e620709ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.823+0000 7f88bbc2e740 1 -- 172.21.15.82:0/993961989 >> 172.21.15.82:0/993961989 conn(0x55e6206fe150 msgr2=0x55e6206ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:17.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.824+0000 7f8899ffb640 0 client.26522 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:17.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.824+0000 7f88bbc2e740 1 -- 172.21.15.82:0/993961989 shutdown_connections 2024-04-21T08:52:17.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.824+0000 7f88bbc2e740 1 -- 172.21.15.82:0/993961989 wait complete. 2024-04-21T08:52:17.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.825+0000 7f88bbc2e740 20 client.26522 trim_cache size 0 max 16384 2024-04-21T08:52:17.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:17 smithi106 ceph-mon[31072]: pgmap v161: 129 pgs: 129 active+clean; 692 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.4 MiB/s wr, 1.04k op/s 2024-04-21T08:52:17.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.947+0000 7f88bbc2e740 1 Processor -- start 2024-04-21T08:52:17.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.950+0000 7f88bbc2e740 1 -- start start 2024-04-21T08:52:17.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.950+0000 7f88bbc2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620703a80 0x55e62070c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.950+0000 7f88bbc2e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620704430 0x55e620718b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.950+0000 7f88bbc2e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e62070cfb0 0x55e620722320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.950+0000 7f88b259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620703a80 0x55e62070c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.950+0000 7f88b259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620703a80 0x55e62070c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:36644/0 (socket says 172.21.15.82:36644) 2024-04-21T08:52:17.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.950+0000 7f88b259c640 1 -- 172.21.15.82:0/889765064 learned_addr learned my addr 172.21.15.82:0/889765064 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:17.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.950+0000 7f88bbc2e740 1 -- 172.21.15.82:0/889765064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f889400fef0 con 0x55e620704430 2024-04-21T08:52:17.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.950+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/889765064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620704430 0x55e620718b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.951+0000 7f88b2d9d640 1 --2- 172.21.15.82:0/889765064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e62070cfb0 0x55e620722320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.952+0000 7f88bbc2e740 1 -- 172.21.15.82:0/889765064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8894011ac0 con 0x55e620703a80 2024-04-21T08:52:17.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.952+0000 7f88bbc2e740 1 -- 172.21.15.82:0/889765064 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88940105c0 con 0x55e62070cfb0 2024-04-21T08:52:17.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.953+0000 7f88b1d9b640 1 -- 172.21.15.82:0/889765064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e62070cfb0 msgr2=0x55e620722320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:17.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.953+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/889765064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e62070cfb0 0x55e620722320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.953+0000 7f88b1d9b640 1 -- 172.21.15.82:0/889765064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620703a80 msgr2=0x55e62070c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:17.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.953+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/889765064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620703a80 0x55e62070c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.953+0000 7f88b1d9b640 1 -- 172.21.15.82:0/889765064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62070ae70 con 0x55e620704430 2024-04-21T08:52:17.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.954+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/889765064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620704430 0x55e620718b00 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f88a800d6a0 tx=0x7f88a8009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674fc27f2d96e7ed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:17.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.955+0000 7f88b2d9d640 1 --2- 172.21.15.82:0/889765064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e62070cfb0 0x55e620722320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:17.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:17 smithi134 ceph-mon[31772]: pgmap v161: 129 pgs: 129 active+clean; 692 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.4 MiB/s wr, 1.04k op/s 2024-04-21T08:52:17.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.990+0000 7f889bfff640 1 -- 172.21.15.82:0/889765064 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8020070 con 0x55e620704430 2024-04-21T08:52:17.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.991+0000 7f889bfff640 1 -- 172.21.15.82:0/889765064 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a80b2e50 con 0x55e620704430 2024-04-21T08:52:17.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.991+0000 7f889bfff640 1 -- 172.21.15.82:0/889765064 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8015030 con 0x55e620704430 2024-04-21T08:52:17.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.992+0000 7f88bbc2e740 1 -- 172.21.15.82:0/889765064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620704430 msgr2=0x55e620718b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.992+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/889765064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620704430 0x55e620718b00 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f88a800d6a0 tx=0x7f88a8009760 comp rx=0 tx=0).stop 2024-04-21T08:52:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.992+0000 7f88bbc2e740 1 -- 172.21.15.82:0/889765064 shutdown_connections 2024-04-21T08:52:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.992+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/889765064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e62070cfb0 0x55e620722320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.992+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/889765064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620704430 0x55e620718b00 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.992+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/889765064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e620703a80 0x55e62070c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.992+0000 7f88bbc2e740 1 -- 172.21.15.82:0/889765064 >> 172.21.15.82:0/889765064 conn(0x55e6206fe150 msgr2=0x55e6207183f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:17.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.993+0000 7f88bbc2e740 1 -- 172.21.15.82:0/889765064 shutdown_connections 2024-04-21T08:52:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.993+0000 7f88bbc2e740 1 -- 172.21.15.82:0/889765064 wait complete. 2024-04-21T08:52:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.993+0000 7f88bbc2e740 1 -- start start 2024-04-21T08:52:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.993+0000 7f88bbc2e740 1 Processor -- start 2024-04-21T08:52:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.994+0000 7f88bbc2e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620703a80 0x55e6207200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.994+0000 7f88bbc2e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620704430 0x55e620720600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.994+0000 7f88bbc2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 0x55e6207824d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.994+0000 7f88bbc2e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f88940113f0 con 0x55e620703a80 2024-04-21T08:52:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.994+0000 7f88bbc2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8894010d20 con 0x55e62070cfb0 2024-04-21T08:52:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.994+0000 7f88bbc2e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f889400df80 con 0x55e620704430 2024-04-21T08:52:17.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.994+0000 7f88bbc2e740 10 client.0 _ll_register_callbacks cb 0x7f88a80e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:17.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.994+0000 7f88b259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620703a80 0x55e6207200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.994+0000 7f88b259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620703a80 0x55e6207200c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37364/0 (socket says 172.21.15.82:37364) 2024-04-21T08:52:17.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.994+0000 7f88b259c640 1 -- 172.21.15.82:0/83183157 learned_addr learned my addr 172.21.15.82:0/83183157 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:17.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.994+0000 7f88b2d9d640 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 0x55e6207824d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.995+0000 7f88b1d9b640 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620704430 0x55e620720600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.995+0000 7f88b2d9d640 1 -- 172.21.15.82:0/83183157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620704430 msgr2=0x55e620720600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.995+0000 7f88b2d9d640 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620704430 0x55e620720600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.995+0000 7f88b2d9d640 1 -- 172.21.15.82:0/83183157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620703a80 msgr2=0x55e6207200c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.995+0000 7f88b2d9d640 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620703a80 0x55e6207200c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.995+0000 7f88b2d9d640 1 -- 172.21.15.82:0/83183157 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e620720e90 con 0x55e62070cfb0 2024-04-21T08:52:17.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:17.995+0000 7f88b2d9d640 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 0x55e6207824d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f88ac007350 tx=0x7f88ac004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b427ca75f01dae18 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.007+0000 7f8899ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:52:18.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.007+0000 7f8899ffb640 1 -- 172.21.15.82:0/83183157 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac00fc40 con 0x55e62070cfb0 2024-04-21T08:52:18.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.007+0000 7f88bbc2e740 1 -- 172.21.15.82:0/83183157 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a80b2e50 con 0x55e62070cfb0 2024-04-21T08:52:18.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.007+0000 7f8899ffb640 1 -- 172.21.15.82:0/83183157 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f88ac00fde0 con 0x55e62070cfb0 2024-04-21T08:52:18.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.008+0000 7f8899ffb640 1 -- 172.21.15.82:0/83183157 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f88ac031050 con 0x55e62070cfb0 2024-04-21T08:52:18.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.008+0000 7f88bbc2e740 10 client.26528 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:18.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.008+0000 7f88bbc2e740 1 -- 172.21.15.82:0/83183157 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e620783af0 con 0x55e62070cfb0 2024-04-21T08:52:18.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.010+0000 7f8899ffb640 1 -- 172.21.15.82:0/83183157 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a80b2e50 con 0x55e62070cfb0 2024-04-21T08:52:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.011+0000 7f8899ffb640 1 -- 172.21.15.82:0/83183157 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f88ac025500 con 0x55e62070cfb0 2024-04-21T08:52:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.011+0000 7f88bbc2e740 10 client.26528 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.011+0000 7f88bbc2e740 10 client.26528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.011+0000 7f88bbc2e740 20 client.26528 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.011+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e6207842a0 0x55e6207866c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.012+0000 7f88bbc2e740 4 client.26528 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:52:18.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.012+0000 7f88bbc2e740 1 -- 172.21.15.82:0/83183157 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e62066f790 con 0x55e6207842a0 2024-04-21T08:52:18.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.012+0000 7f88b259c640 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e6207842a0 0x55e6207866c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.012+0000 7f88b259c640 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e6207842a0 0x55e6207866c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474533ea839350fd server_cookie=d738ee71d87cb035 in_seq=0 out_seq=0 2024-04-21T08:52:18.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.013+0000 7f8899ffb640 10 client.26528 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:18.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.015+0000 7f8899ffb640 1 -- 172.21.15.82:0/83183157 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e62066f790 con 0x55e6207842a0 2024-04-21T08:52:18.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.015+0000 7f8899ffb640 10 client.26528 handle_command_reply: tid=0 2024-04-21T08:52:18.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.015+0000 7f88bbc2e740 1 client.26528 shutdown 2024-04-21T08:52:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.018+0000 7f88bbc2e740 1 -- 172.21.15.82:0/83183157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 msgr2=0x55e6207824d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.018+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 0x55e6207824d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f88ac007350 tx=0x7f88ac004ab0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.018+0000 7f88bbc2e740 1 -- 172.21.15.82:0/83183157 shutdown_connections 2024-04-21T08:52:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.018+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e6207842a0 0x55e6207866c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.018+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e62070cfb0 0x55e6207824d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.018+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e620704430 0x55e620720600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.018+0000 7f88bbc2e740 1 --2- 172.21.15.82:0/83183157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e620703a80 0x55e6207200c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.018+0000 7f88bbc2e740 1 -- 172.21.15.82:0/83183157 >> 172.21.15.82:0/83183157 conn(0x55e6206fe150 msgr2=0x55e62070a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.018+0000 7f8899ffb640 0 client.26528 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:18.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.019+0000 7f88bbc2e740 1 -- 172.21.15.82:0/83183157 shutdown_connections 2024-04-21T08:52:18.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.019+0000 7f88bbc2e740 1 -- 172.21.15.82:0/83183157 wait complete. 2024-04-21T08:52:18.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.019+0000 7f88bbc2e740 20 client.26528 trim_cache size 0 max 16384 2024-04-21T08:52:18.020 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:52:18.020 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:52:18.020 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "64552535-4733-45da-b784-7a1f2b81d8dd", 2024-04-21T08:52:18.020 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:52:18.020 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:52:18.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.034+0000 7f88b8f59640 1 -- 172.21.15.82:0/3819832909 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f889c0a3f30 msgr2=0x7f889c0a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.034+0000 7f88b8f59640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f889c0a3f30 0x7f889c0a63f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f88ac007f20 tx=0x7f88ac007560 comp rx=0 tx=0).stop 2024-04-21T08:52:18.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.035+0000 7f88b8f59640 1 -- 172.21.15.82:0/3819832909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88b410eb40 msgr2=0x7f88b4115e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.035+0000 7f88b8f59640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88b410eb40 0x7f88b4115e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f88a800d6d0 tx=0x7f88a800db90 comp rx=0 tx=0).stop 2024-04-21T08:52:18.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.035+0000 7f88b8f59640 1 -- 172.21.15.82:0/3819832909 shutdown_connections 2024-04-21T08:52:18.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.035+0000 7f88b8f59640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f889c0a3f30 0x7f889c0a63f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.035+0000 7f88b8f59640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4110550 0x7f88b4116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.035+0000 7f88b8f59640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88b410eb40 0x7f88b4115e00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.035+0000 7f88b8f59640 1 --2- 172.21.15.82:0/3819832909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88b410e0c0 0x7f88b411cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.035+0000 7f88b8f59640 1 -- 172.21.15.82:0/3819832909 >> 172.21.15.82:0/3819832909 conn(0x7f88b406ce60 msgr2=0x7f88b4071330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.036+0000 7f88b8f59640 1 -- 172.21.15.82:0/3819832909 shutdown_connections 2024-04-21T08:52:18.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.036+0000 7f88b8f59640 1 -- 172.21.15.82:0/3819832909 wait complete. 2024-04-21T08:52:18.099 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:17 smithi082 ceph-mon[27967]: pgmap v161: 129 pgs: 129 active+clean; 692 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.4 MiB/s wr, 1.04k op/s 2024-04-21T08:52:18.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:52:18.100 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:52:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.419+0000 7fc104d50640 1 -- 172.21.15.82:0/1636786620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1001109f0 msgr2=0x7fc10010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.419+0000 7fc104d50640 1 --2- 172.21.15.82:0/1636786620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1001109f0 0x7fc10010e600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc0f001a910 tx=0x7fc0f003cd40 comp rx=0 tx=0).stop 2024-04-21T08:52:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.420+0000 7fc104d50640 1 -- 172.21.15.82:0/1636786620 shutdown_connections 2024-04-21T08:52:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.420+0000 7fc104d50640 1 --2- 172.21.15.82:0/1636786620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10010ec20 0x7fc10011d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.420+0000 7fc104d50640 1 --2- 172.21.15.82:0/1636786620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1001109f0 0x7fc10010e600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.420+0000 7fc104d50640 1 --2- 172.21.15.82:0/1636786620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc100110610 0x7fc10010e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.420+0000 7fc104d50640 1 -- 172.21.15.82:0/1636786620 >> 172.21.15.82:0/1636786620 conn(0x7fc10006ce90 msgr2=0x7fc10006d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.430+0000 7fc104d50640 1 -- 172.21.15.82:0/1636786620 shutdown_connections 2024-04-21T08:52:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.430+0000 7fc104d50640 1 -- 172.21.15.82:0/1636786620 wait complete. 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.432+0000 7fc104d50640 1 Processor -- start 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.432+0000 7fc104d50640 1 -- start start 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.432+0000 7fc104d50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10010ec20 0x7fc10011d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.432+0000 7fc104d50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100110610 0x7fc1001193b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.432+0000 7fc104d50640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1001109f0 0x7fc1001198f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc104d50640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc100113790 con 0x7fc10010ec20 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc104d50640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc100113610 con 0x7fc1001109f0 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc104d50640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc100113910 con 0x7fc100110610 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc0feffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100110610 0x7fc1001193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc0ff7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10010ec20 0x7fc10011d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc0feffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100110610 0x7fc1001193b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52146/0 (socket says 172.21.15.82:52146) 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc0feffd640 1 -- 172.21.15.82:0/1795269457 learned_addr learned my addr 172.21.15.82:0/1795269457 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc0fffff640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1001109f0 0x7fc1001198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc0feffd640 1 -- 172.21.15.82:0/1795269457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1001109f0 msgr2=0x7fc1001198f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc0feffd640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1001109f0 0x7fc1001198f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc0feffd640 1 -- 172.21.15.82:0/1795269457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10010ec20 msgr2=0x7fc10011d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc0feffd640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10010ec20 0x7fc10011d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.433+0000 7fc0feffd640 1 -- 172.21.15.82:0/1795269457 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10011a020 con 0x7fc100110610 2024-04-21T08:52:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.434+0000 7fc0ff7fe640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10010ec20 0x7fc10011d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.434+0000 7fc0feffd640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100110610 0x7fc1001193b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0006010 tx=0x7fc0f0004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.439+0000 7fc0fcff9640 1 -- 172.21.15.82:0/1795269457 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0053020 con 0x7fc100110610 2024-04-21T08:52:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.440+0000 7fc104d50640 1 -- 172.21.15.82:0/1795269457 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1001ab3d0 con 0x7fc100110610 2024-04-21T08:52:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.440+0000 7fc104d50640 1 -- 172.21.15.82:0/1795269457 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1001ab780 con 0x7fc100110610 2024-04-21T08:52:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.442+0000 7fc0fcff9640 1 -- 172.21.15.82:0/1795269457 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f0005520 con 0x7fc100110610 2024-04-21T08:52:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.442+0000 7fc0fcff9640 1 -- 172.21.15.82:0/1795269457 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f003d8f0 con 0x7fc100110610 2024-04-21T08:52:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.448+0000 7fc0fcff9640 1 -- 172.21.15.82:0/1795269457 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc0f0056020 con 0x7fc100110610 2024-04-21T08:52:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.450+0000 7fc0fcff9640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc0e80a3ee0 0x7fc0e80a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.450+0000 7fc0fcff9640 1 -- 172.21.15.82:0/1795269457 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc0f01049c0 con 0x7fc100110610 2024-04-21T08:52:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.451+0000 7fc108ba5740 1 Processor -- start 2024-04-21T08:52:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.451+0000 7fc108ba5740 1 -- start start 2024-04-21T08:52:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.451+0000 7fc108ba5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f2812ac0 0x55a7f2811810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.451+0000 7fc108ba5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2811d50 0x55a7f28121d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.451+0000 7fc108ba5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f281b2d0 0x55a7f281d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.451+0000 7fc108ba5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a7f27768d0 con 0x55a7f2811d50 2024-04-21T08:52:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.451+0000 7fc108ba5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a7f2713e80 con 0x55a7f2812ac0 2024-04-21T08:52:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.451+0000 7fc108ba5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a7f2772890 con 0x55a7f281b2d0 2024-04-21T08:52:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.453+0000 7fc0feffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2811d50 0x55a7f28121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.453+0000 7fc0feffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2811d50 0x55a7f28121d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37394/0 (socket says 172.21.15.82:37394) 2024-04-21T08:52:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.453+0000 7fc0feffd640 1 -- 172.21.15.82:0/1863734351 learned_addr learned my addr 172.21.15.82:0/1863734351 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.453+0000 7fc0ff7fe640 1 --2- 172.21.15.82:0/1863734351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f2812ac0 0x55a7f2811810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.454+0000 7fc0fffff640 1 --2- 172.21.15.82:0/1863734351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f281b2d0 0x55a7f281d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.454+0000 7fc0fffff640 1 -- 172.21.15.82:0/1863734351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f2812ac0 msgr2=0x55a7f2811810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.454+0000 7fc0fffff640 1 --2- 172.21.15.82:0/1863734351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f2812ac0 0x55a7f2811810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.454+0000 7fc0fffff640 1 -- 172.21.15.82:0/1863734351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2811d50 msgr2=0x55a7f28121d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.454+0000 7fc0fffff640 1 --2- 172.21.15.82:0/1863734351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2811d50 0x55a7f28121d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.454+0000 7fc0fffff640 1 -- 172.21.15.82:0/1863734351 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f2735320 con 0x55a7f281b2d0 2024-04-21T08:52:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.455+0000 7fc0fffff640 1 --2- 172.21.15.82:0/1863734351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f281b2d0 0x55a7f281d6c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009f30 tx=0x7fc0f8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e0bcff9fad8d5cd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.457+0000 7fc0ff7fe640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc0e80a3ee0 0x7fc0e80a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.463+0000 7fc0ff7fe640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc0e80a3ee0 0x7fc0e80a63a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4006fd0 tx=0x7fc0f40046e0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.469+0000 7fc0e5ffb640 1 -- 172.21.15.82:0/1863734351 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8016070 con 0x55a7f281b2d0 2024-04-21T08:52:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.469+0000 7fc0e5ffb640 1 -- 172.21.15.82:0/1863734351 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f8004d20 con 0x55a7f281b2d0 2024-04-21T08:52:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.471+0000 7fc0e5ffb640 1 -- 172.21.15.82:0/1863734351 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f80127d0 con 0x55a7f281b2d0 2024-04-21T08:52:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.474+0000 7fc108ba5740 1 -- 172.21.15.82:0/1863734351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f281b2d0 msgr2=0x55a7f281d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.474+0000 7fc108ba5740 1 --2- 172.21.15.82:0/1863734351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f281b2d0 0x55a7f281d6c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009f30 tx=0x7fc0f8010800 comp rx=0 tx=0).stop 2024-04-21T08:52:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.505+0000 7fc108ba5740 1 -- 172.21.15.82:0/1863734351 shutdown_connections 2024-04-21T08:52:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.505+0000 7fc108ba5740 1 --2- 172.21.15.82:0/1863734351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f281b2d0 0x55a7f281d6c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.505+0000 7fc108ba5740 1 --2- 172.21.15.82:0/1863734351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2811d50 0x55a7f28121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.505+0000 7fc108ba5740 1 --2- 172.21.15.82:0/1863734351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f2812ac0 0x55a7f2811810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.505+0000 7fc108ba5740 1 -- 172.21.15.82:0/1863734351 >> 172.21.15.82:0/1863734351 conn(0x55a7f27d8c90 msgr2=0x55a7f280ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.513+0000 7fc108ba5740 1 -- 172.21.15.82:0/1863734351 shutdown_connections 2024-04-21T08:52:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.520+0000 7fc108ba5740 1 -- 172.21.15.82:0/1863734351 wait complete. 2024-04-21T08:52:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.520+0000 7fc108ba5740 1 -- start start 2024-04-21T08:52:18.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.520+0000 7fc108ba5740 1 Processor -- start 2024-04-21T08:52:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.521+0000 7fc108ba5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 0x55a7f281aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.521+0000 7fc108ba5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2812ac0 0x55a7f2818030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.521+0000 7fc108ba5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f281b2d0 0x55a7f2818640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.521+0000 7fc108ba5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a7f281e000 con 0x55a7f2812ac0 2024-04-21T08:52:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.521+0000 7fc108ba5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a7f26feeb0 con 0x55a7f281b2d0 2024-04-21T08:52:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.521+0000 7fc108ba5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a7f281e180 con 0x55a7f2811d50 2024-04-21T08:52:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.521+0000 7fc108ba5740 10 client.0 _ll_register_callbacks cb 0x55a7f272bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.521+0000 7fc0ff7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 0x55a7f281aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.521+0000 7fc0ff7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 0x55a7f281aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52154/0 (socket says 172.21.15.82:52154) 2024-04-21T08:52:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.521+0000 7fc0ff7fe640 1 -- 172.21.15.82:0/418397767 learned_addr learned my addr 172.21.15.82:0/418397767 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.521+0000 7fc0fffff640 1 --2- 172.21.15.82:0/418397767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f281b2d0 0x55a7f2818640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.522+0000 7fc0ff7fe640 1 -- 172.21.15.82:0/418397767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f281b2d0 msgr2=0x55a7f2818640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.522+0000 7fc0ff7fe640 1 --2- 172.21.15.82:0/418397767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f281b2d0 0x55a7f2818640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.522+0000 7fc0ff7fe640 1 -- 172.21.15.82:0/418397767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2812ac0 msgr2=0x55a7f2818030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.522+0000 7fc0ff7fe640 1 --2- 172.21.15.82:0/418397767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2812ac0 0x55a7f2818030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.522+0000 7fc0ff7fe640 1 -- 172.21.15.82:0/418397767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f2489210 con 0x55a7f2811d50 2024-04-21T08:52:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.522+0000 7fc0ff7fe640 1 --2- 172.21.15.82:0/418397767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 0x55a7f281aca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4014500 tx=0x7fc0f4006d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=584392e517b50654 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.525+0000 7fc0deffd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:52:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.525+0000 7fc0deffd640 1 -- 172.21.15.82:0/418397767 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f401b050 con 0x55a7f2811d50 2024-04-21T08:52:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.525+0000 7fc108ba5740 1 -- 172.21.15.82:0/418397767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f8004d20 con 0x55a7f2811d50 2024-04-21T08:52:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.525+0000 7fc0deffd640 1 -- 172.21.15.82:0/418397767 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc0f4015820 con 0x55a7f2811d50 2024-04-21T08:52:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.526+0000 7fc0deffd640 1 -- 172.21.15.82:0/418397767 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc0f4015a20 con 0x55a7f2811d50 2024-04-21T08:52:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.526+0000 7fc108ba5740 10 client.26263 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.526+0000 7fc108ba5740 1 -- 172.21.15.82:0/418397767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a7f2555620 con 0x55a7f2811d50 2024-04-21T08:52:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.527+0000 7fc0deffd640 1 -- 172.21.15.82:0/418397767 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f8004d20 con 0x55a7f2811d50 2024-04-21T08:52:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.527+0000 7fc0deffd640 1 -- 172.21.15.82:0/418397767 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fc0f4006330 con 0x55a7f2811d50 2024-04-21T08:52:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.527+0000 7fc108ba5740 10 client.26263 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.527+0000 7fc108ba5740 10 client.26263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.527+0000 7fc108ba5740 20 client.26263 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.527+0000 7fc108ba5740 1 --2- 172.21.15.82:0/418397767 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a7f282b760 0x55a7f282dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.527+0000 7fc108ba5740 4 client.26263 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.527+0000 7fc108ba5740 1 -- 172.21.15.82:0/418397767 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7f278a880 con 0x55a7f282b760 2024-04-21T08:52:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.527+0000 7fc0feffd640 1 --2- 172.21.15.82:0/418397767 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a7f282b760 0x55a7f282dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.530+0000 7fc0feffd640 1 --2- 172.21.15.82:0/418397767 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a7f282b760 0x55a7f282dc20 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef7f36d7d3da2a3d server_cookie=e92a9fb8b6115623 in_seq=0 out_seq=0 2024-04-21T08:52:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.530+0000 7fc0deffd640 10 client.26263 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.534+0000 7fc0deffd640 1 -- 172.21.15.82:0/418397767 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7f278a880 con 0x55a7f282b760 2024-04-21T08:52:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.534+0000 7fc0deffd640 10 client.26263 handle_command_reply: tid=0 2024-04-21T08:52:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.534+0000 7fc108ba5740 1 client.26263 shutdown 2024-04-21T08:52:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.539+0000 7fc108ba5740 1 -- 172.21.15.82:0/418397767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 msgr2=0x55a7f281aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.539+0000 7fc108ba5740 1 --2- 172.21.15.82:0/418397767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 0x55a7f281aca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4014500 tx=0x7fc0f4006d50 comp rx=0 tx=0).stop 2024-04-21T08:52:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.542+0000 7fc108ba5740 1 -- 172.21.15.82:0/418397767 shutdown_connections 2024-04-21T08:52:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.542+0000 7fc108ba5740 1 --2- 172.21.15.82:0/418397767 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a7f282b760 0x55a7f282dc20 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.542+0000 7fc108ba5740 1 --2- 172.21.15.82:0/418397767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f281b2d0 0x55a7f2818640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.542+0000 7fc108ba5740 1 --2- 172.21.15.82:0/418397767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2812ac0 0x55a7f2818030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.542+0000 7fc108ba5740 1 --2- 172.21.15.82:0/418397767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 0x55a7f281aca0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.542+0000 7fc108ba5740 1 -- 172.21.15.82:0/418397767 >> 172.21.15.82:0/418397767 conn(0x55a7f27d8c90 msgr2=0x55a7f280e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.542+0000 7fc0deffd640 0 client.26263 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.549+0000 7fc108ba5740 1 -- 172.21.15.82:0/418397767 shutdown_connections 2024-04-21T08:52:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.549+0000 7fc108ba5740 1 -- 172.21.15.82:0/418397767 wait complete. 2024-04-21T08:52:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.549+0000 7fc108ba5740 20 client.26263 trim_cache size 0 max 16384 2024-04-21T08:52:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.624+0000 7fc108ba5740 1 Processor -- start 2024-04-21T08:52:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.624+0000 7fc108ba5740 1 -- start start 2024-04-21T08:52:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.624+0000 7fc108ba5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 0x55a7f2826f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.625+0000 7fc108ba5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2812ac0 0x55a7f281a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.625+0000 7fc0ff7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 0x55a7f2826f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.625+0000 7fc0ff7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 0x55a7f2826f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52156/0 (socket says 172.21.15.82:52156) 2024-04-21T08:52:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.625+0000 7fc0ff7fe640 1 -- 172.21.15.82:0/4211737128 learned_addr learned my addr 172.21.15.82:0/4211737128 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.625+0000 7fc108ba5740 1 --2- 172.21.15.82:0/4211737128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f281b2d0 0x55a7f281aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.625+0000 7fc108ba5740 1 -- 172.21.15.82:0/4211737128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001b5950 con 0x55a7f2812ac0 2024-04-21T08:52:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.625+0000 7fc108ba5740 1 -- 172.21.15.82:0/4211737128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001b7520 con 0x55a7f281b2d0 2024-04-21T08:52:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.625+0000 7fc108ba5740 1 -- 172.21.15.82:0/4211737128 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001b6020 con 0x55a7f2811d50 2024-04-21T08:52:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.625+0000 7fc0fffff640 1 --2- 172.21.15.82:0/4211737128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f281b2d0 0x55a7f281aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.626+0000 7fc0feffd640 1 --2- 172.21.15.82:0/4211737128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2812ac0 0x55a7f281a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.627+0000 7fc0feffd640 1 -- 172.21.15.82:0/4211737128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 msgr2=0x55a7f2826f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.627+0000 7fc0feffd640 1 --2- 172.21.15.82:0/4211737128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 0x55a7f2826f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.627+0000 7fc0feffd640 1 -- 172.21.15.82:0/4211737128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f281b2d0 msgr2=0x55a7f281aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.627+0000 7fc0feffd640 1 --2- 172.21.15.82:0/4211737128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f281b2d0 0x55a7f281aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.627+0000 7fc0feffd640 1 -- 172.21.15.82:0/4211737128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f2811020 con 0x55a7f2812ac0 2024-04-21T08:52:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.631+0000 7fc0feffd640 1 --2- 172.21.15.82:0/4211737128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2812ac0 0x55a7f281a760 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00d8d90 tx=0x7fc0f004df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bd235945c57bf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.635+0000 7fc0e57fa640 1 -- 172.21.15.82:0/4211737128 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f00d0650 con 0x55a7f2812ac0 2024-04-21T08:52:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.635+0000 7fc0e57fa640 1 -- 172.21.15.82:0/4211737128 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f00d4a00 con 0x55a7f2812ac0 2024-04-21T08:52:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.635+0000 7fc0e57fa640 1 -- 172.21.15.82:0/4211737128 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0003740 con 0x55a7f2812ac0 2024-04-21T08:52:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.639+0000 7fc108ba5740 1 -- 172.21.15.82:0/4211737128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2812ac0 msgr2=0x55a7f281a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.639+0000 7fc108ba5740 1 --2- 172.21.15.82:0/4211737128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2812ac0 0x55a7f281a760 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00d8d90 tx=0x7fc0f004df90 comp rx=0 tx=0).stop 2024-04-21T08:52:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.639+0000 7fc108ba5740 1 -- 172.21.15.82:0/4211737128 shutdown_connections 2024-04-21T08:52:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.639+0000 7fc108ba5740 1 --2- 172.21.15.82:0/4211737128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f281b2d0 0x55a7f281aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.639+0000 7fc108ba5740 1 --2- 172.21.15.82:0/4211737128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f2812ac0 0x55a7f281a760 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.639+0000 7fc108ba5740 1 --2- 172.21.15.82:0/4211737128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2811d50 0x55a7f2826f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.639+0000 7fc108ba5740 1 -- 172.21.15.82:0/4211737128 >> 172.21.15.82:0/4211737128 conn(0x55a7f27d8c90 msgr2=0x55a7f280c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.639+0000 7fc108ba5740 1 -- 172.21.15.82:0/4211737128 shutdown_connections 2024-04-21T08:52:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.639+0000 7fc108ba5740 1 -- 172.21.15.82:0/4211737128 wait complete. 2024-04-21T08:52:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.640+0000 7fc108ba5740 1 -- start start 2024-04-21T08:52:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.641+0000 7fc108ba5740 1 Processor -- start 2024-04-21T08:52:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.641+0000 7fc108ba5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f2811d50 0x55a7f2823630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.641+0000 7fc108ba5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2812ac0 0x55a7f28152b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.641+0000 7fc108ba5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f281b2d0 0x55a7f28157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.642+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f281b2d0 0x55a7f28157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.642+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f281b2d0 0x55a7f28157f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37402/0 (socket says 172.21.15.82:37402) 2024-04-21T08:52:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.642+0000 7fc0fffff640 1 -- 172.21.15.82:0/1199114773 learned_addr learned my addr 172.21.15.82:0/1199114773 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.643+0000 7fc108ba5740 1 -- 172.21.15.82:0/1199114773 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001b6e50 con 0x55a7f281b2d0 2024-04-21T08:52:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.643+0000 7fc0ff7fe640 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f2811d50 0x55a7f2823630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.643+0000 7fc108ba5740 1 -- 172.21.15.82:0/1199114773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001b6780 con 0x55a7f2811d50 2024-04-21T08:52:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.643+0000 7fc0feffd640 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2812ac0 0x55a7f28152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.643+0000 7fc108ba5740 1 -- 172.21.15.82:0/1199114773 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001b3930 con 0x55a7f2812ac0 2024-04-21T08:52:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.643+0000 7fc0fffff640 1 -- 172.21.15.82:0/1199114773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2812ac0 msgr2=0x55a7f28152b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.643+0000 7fc0fffff640 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2812ac0 0x55a7f28152b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.643+0000 7fc0fffff640 1 -- 172.21.15.82:0/1199114773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f2811d50 msgr2=0x55a7f2823630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.643+0000 7fc0fffff640 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f2811d50 0x55a7f2823630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.643+0000 7fc0fffff640 1 -- 172.21.15.82:0/1199114773 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f2815d30 con 0x55a7f281b2d0 2024-04-21T08:52:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.643+0000 7fc0ff7fe640 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f2811d50 0x55a7f2823630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.644+0000 7fc0feffd640 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2812ac0 0x55a7f28152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.644+0000 7fc108ba5740 10 client.0 _ll_register_callbacks cb 0x7fc0f400a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.644+0000 7fc108ba5740 1 -- 172.21.15.82:0/1199114773 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f00d4a00 con 0x55a7f281b2d0 2024-04-21T08:52:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.657+0000 7fc0fffff640 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f281b2d0 0x55a7f28157f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009f30 tx=0x7fc0f8004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd93b34361713241 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.657+0000 7fc0deffd640 10 client.17190 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:52:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.670+0000 7fc0deffd640 1 -- 172.21.15.82:0/1199114773 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8016040 con 0x55a7f281b2d0 2024-04-21T08:52:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.670+0000 7fc0deffd640 1 -- 172.21.15.82:0/1199114773 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fc0f8012650 con 0x55a7f281b2d0 2024-04-21T08:52:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.670+0000 7fc0deffd640 1 -- 172.21.15.82:0/1199114773 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc0f80128b0 con 0x55a7f281b2d0 2024-04-21T08:52:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.671+0000 7fc108ba5740 10 client.17190 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:18.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.671+0000 7fc108ba5740 1 -- 172.21.15.82:0/1199114773 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=36}) -- 0x55a7f281c660 con 0x55a7f281b2d0 2024-04-21T08:52:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.678+0000 7fc0deffd640 1 -- 172.21.15.82:0/1199114773 <== mon.0 v2:172.21.15.82:3300/0 4 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fc0f802d030 con 0x55a7f281b2d0 2024-04-21T08:52:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.678+0000 7fc108ba5740 10 client.17190 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.678+0000 7fc108ba5740 10 client.17190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.678+0000 7fc108ba5740 20 client.17190 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.678+0000 7fc108ba5740 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a7f2891da0 0x55a7f28941b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.678+0000 7fc108ba5740 4 client.17190 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:52:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.678+0000 7fc108ba5740 1 -- 172.21.15.82:0/1199114773 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7f27163d0 con 0x55a7f2891da0 2024-04-21T08:52:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.678+0000 7fc0ff7fe640 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a7f2891da0 0x55a7f28941b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.679+0000 7fc0deffd640 10 client.17190 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.679+0000 7fc0ff7fe640 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a7f2891da0 0x55a7f28941b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e1f30230ada31f server_cookie=e9759da3abadfd5e in_seq=0 out_seq=0 2024-04-21T08:52:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.783+0000 7fc0deffd640 1 -- 172.21.15.82:0/1199114773 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7f27163d0 con 0x55a7f2891da0 2024-04-21T08:52:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.783+0000 7fc0deffd640 10 client.17190 handle_command_reply: tid=0 2024-04-21T08:52:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.783+0000 7fc108ba5740 1 client.17190 shutdown 2024-04-21T08:52:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.783+0000 7fc108ba5740 1 -- 172.21.15.82:0/1199114773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f281b2d0 msgr2=0x55a7f28157f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.783+0000 7fc108ba5740 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f281b2d0 0x55a7f28157f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009f30 tx=0x7fc0f8004c30 comp rx=0 tx=0).stop 2024-04-21T08:52:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.783+0000 7fc108ba5740 1 -- 172.21.15.82:0/1199114773 shutdown_connections 2024-04-21T08:52:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.783+0000 7fc108ba5740 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a7f2891da0 0x55a7f28941b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.783+0000 7fc108ba5740 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f281b2d0 0x55a7f28157f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.783+0000 7fc108ba5740 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7f2812ac0 0x55a7f28152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.783+0000 7fc108ba5740 1 --2- 172.21.15.82:0/1199114773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7f2811d50 0x55a7f2823630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.783+0000 7fc108ba5740 1 -- 172.21.15.82:0/1199114773 >> 172.21.15.82:0/1199114773 conn(0x55a7f27d8c90 msgr2=0x55a7f280b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.783+0000 7fc0deffd640 0 client.17190 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.784+0000 7fc108ba5740 1 -- 172.21.15.82:0/1199114773 shutdown_connections 2024-04-21T08:52:18.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.784+0000 7fc108ba5740 1 -- 172.21.15.82:0/1199114773 wait complete. 2024-04-21T08:52:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.785+0000 7fc108ba5740 20 client.17190 trim_cache size 0 max 16384 2024-04-21T08:52:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:52:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:52:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:52:18.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.800+0000 7fc104d50640 1 -- 172.21.15.82:0/1795269457 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc0e80a3ee0 msgr2=0x7fc0e80a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.800+0000 7fc104d50640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc0e80a3ee0 0x7fc0e80a63a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4006fd0 tx=0x7fc0f40046e0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.800+0000 7fc104d50640 1 -- 172.21.15.82:0/1795269457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100110610 msgr2=0x7fc1001193b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.800+0000 7fc104d50640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100110610 0x7fc1001193b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0006010 tx=0x7fc0f0004740 comp rx=0 tx=0).stop 2024-04-21T08:52:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.804+0000 7fc104d50640 1 -- 172.21.15.82:0/1795269457 shutdown_connections 2024-04-21T08:52:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.804+0000 7fc104d50640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1001109f0 0x7fc1001198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.804+0000 7fc104d50640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100110610 0x7fc1001193b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.804+0000 7fc104d50640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc0e80a3ee0 0x7fc0e80a63a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.804+0000 7fc104d50640 1 --2- 172.21.15.82:0/1795269457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10010ec20 0x7fc10011d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.804+0000 7fc104d50640 1 -- 172.21.15.82:0/1795269457 >> 172.21.15.82:0/1795269457 conn(0x7fc10006ce90 msgr2=0x7fc10010d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.804+0000 7fc104d50640 1 -- 172.21.15.82:0/1795269457 shutdown_connections 2024-04-21T08:52:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:18.804+0000 7fc104d50640 1 -- 172.21.15.82:0/1795269457 wait complete. 2024-04-21T08:52:18.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.815+0000 7f1cafd66640 1 -- 172.21.15.82:0/577326079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca811b730 msgr2=0x7f1ca811db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.815+0000 7f1cafd66640 1 --2- 172.21.15.82:0/577326079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca811b730 0x7f1ca811db20 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009cd0 tx=0x7f1ca002c100 comp rx=0 tx=0).stop 2024-04-21T08:52:18.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.815+0000 7f1cafd66640 1 -- 172.21.15.82:0/577326079 shutdown_connections 2024-04-21T08:52:18.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.815+0000 7f1cafd66640 1 --2- 172.21.15.82:0/577326079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca811b730 0x7f1ca811db20 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.815+0000 7f1cafd66640 1 --2- 172.21.15.82:0/577326079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ca810e7b0 0x7f1ca810ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.815+0000 7f1cafd66640 1 --2- 172.21.15.82:0/577326079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca81106d0 0x7f1ca810e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.815+0000 7f1cafd66640 1 -- 172.21.15.82:0/577326079 >> 172.21.15.82:0/577326079 conn(0x7f1ca806d070 msgr2=0x7f1ca806d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.815+0000 7f1cafd66640 1 -- 172.21.15.82:0/577326079 shutdown_connections 2024-04-21T08:52:18.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.815+0000 7f1cafd66640 1 -- 172.21.15.82:0/577326079 wait complete. 2024-04-21T08:52:18.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.816+0000 7f1cafd66640 1 Processor -- start 2024-04-21T08:52:18.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.816+0000 7f1cafd66640 1 -- start start 2024-04-21T08:52:18.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.816+0000 7f1cafd66640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca810e7b0 0x7f1ca81a7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.817+0000 7f1cafd66640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ca81106d0 0x7f1ca81a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.817+0000 7f1cafd66640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca811b730 0x7f1ca81abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.817+0000 7f1cafd66640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8121040 con 0x7f1ca810e7b0 2024-04-21T08:52:18.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.817+0000 7f1cafd66640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8120ec0 con 0x7f1ca81106d0 2024-04-21T08:52:18.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.817+0000 7f1cafd66640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca81211c0 con 0x7f1ca811b730 2024-04-21T08:52:18.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.817+0000 7f1cadb02640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca810e7b0 0x7f1ca81a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.817+0000 7f1cadb02640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca810e7b0 0x7f1ca81a7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37422/0 (socket says 172.21.15.82:37422) 2024-04-21T08:52:18.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.817+0000 7f1cadb02640 1 -- 172.21.15.82:0/3008105488 learned_addr learned my addr 172.21.15.82:0/3008105488 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:18.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.817+0000 7f1cad301640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ca81106d0 0x7f1ca81a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.817+0000 7f1cae303640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca811b730 0x7f1ca81abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.818+0000 7f1cae303640 1 -- 172.21.15.82:0/3008105488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ca81106d0 msgr2=0x7f1ca81a7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.818+0000 7f1cae303640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ca81106d0 0x7f1ca81a7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.818+0000 7f1cae303640 1 -- 172.21.15.82:0/3008105488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca810e7b0 msgr2=0x7f1ca81a7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.818+0000 7f1cae303640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca810e7b0 0x7f1ca81a7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.818+0000 7f1cae303640 1 -- 172.21.15.82:0/3008105488 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca81ac4f0 con 0x7f1ca811b730 2024-04-21T08:52:18.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.818+0000 7f1cadb02640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca810e7b0 0x7f1ca81a7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:18.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.818+0000 7f1cad301640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ca81106d0 0x7f1ca81a7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:18.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.818+0000 7f1cae303640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca811b730 0x7f1ca81abdc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009bf0 tx=0x7f1ca0035ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.827+0000 7f1c9effd640 1 -- 172.21.15.82:0/3008105488 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0039040 con 0x7f1ca811b730 2024-04-21T08:52:18.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.827+0000 7f1c9effd640 1 -- 172.21.15.82:0/3008105488 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca0044070 con 0x7f1ca811b730 2024-04-21T08:52:18.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.827+0000 7f1c9effd640 1 -- 172.21.15.82:0/3008105488 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0004760 con 0x7f1ca811b730 2024-04-21T08:52:18.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.827+0000 7f1cafd66640 1 -- 172.21.15.82:0/3008105488 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca81ac780 con 0x7f1ca811b730 2024-04-21T08:52:18.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.827+0000 7f1cafd66640 1 -- 172.21.15.82:0/3008105488 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca81acbf0 con 0x7f1ca811b730 2024-04-21T08:52:18.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.833+0000 7f1cb2a31740 1 Processor -- start 2024-04-21T08:52:18.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.834+0000 7f1c9effd640 1 -- 172.21.15.82:0/3008105488 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1ca0032660 con 0x7f1ca811b730 2024-04-21T08:52:18.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.834+0000 7f1cb2a31740 1 -- start start 2024-04-21T08:52:18.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.834+0000 7f1cb2a31740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaab580 0x560b6eaab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.834+0000 7f1cb2a31740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eaabea0 0x560b6eab4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.835+0000 7f1cad301640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaab580 0x560b6eaab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.835+0000 7f1cad301640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaab580 0x560b6eaab960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:36722/0 (socket says 172.21.15.82:36722) 2024-04-21T08:52:18.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.835+0000 7f1cadb02640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eaabea0 0x560b6eab4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.835+0000 7f1cadb02640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eaabea0 0x560b6eab4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52210/0 (socket says 172.21.15.82:52210) 2024-04-21T08:52:18.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.835+0000 7f1cb2a31740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.835+0000 7f1cad301640 1 -- 172.21.15.82:0/3175404093 learned_addr learned my addr 172.21.15.82:0/3175404093 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:18.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.835+0000 7f1cb2a31740 1 -- 172.21.15.82:0/3175404093 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560b6ea101e0 con 0x560b6eab4a80 2024-04-21T08:52:18.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.835+0000 7f1cb2a31740 1 -- 172.21.15.82:0/3175404093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560b6e9ad390 con 0x560b6eaab580 2024-04-21T08:52:18.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.835+0000 7f1cb2a31740 1 -- 172.21.15.82:0/3175404093 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560b6ea0c1a0 con 0x560b6eaabea0 2024-04-21T08:52:18.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.836+0000 7f1cad301640 1 -- 172.21.15.82:0/3175404093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eaabea0 msgr2=0x560b6eab4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.836+0000 7f1cad301640 1 --2- 172.21.15.82:0/3175404093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eaabea0 0x560b6eab4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.836+0000 7f1cad301640 1 -- 172.21.15.82:0/3175404093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 msgr2=0x560b6eab6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.836+0000 7f1cad301640 1 --2- 172.21.15.82:0/3175404093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.836+0000 7f1cad301640 1 -- 172.21.15.82:0/3175404093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b6e7ec7d0 con 0x560b6eaab580 2024-04-21T08:52:18.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.836+0000 7f1cad301640 1 --2- 172.21.15.82:0/3175404093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaab580 0x560b6eaab960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800bdd0 tx=0x7f1c98011750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80f6eeb9238bfc4e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.837+0000 7f1c9effd640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1c940a3eb0 0x7f1c940a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.838+0000 7f1c9effd640 1 -- 172.21.15.82:0/3008105488 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1ca00f2e20 con 0x7f1ca811b730 2024-04-21T08:52:18.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.840+0000 7f1cadb02640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1c940a3eb0 0x7f1c940a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.846+0000 7f1c93fff640 1 -- 172.21.15.82:0/3175404093 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98011e60 con 0x560b6eaab580 2024-04-21T08:52:18.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.846+0000 7f1c93fff640 1 -- 172.21.15.82:0/3175404093 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c98004230 con 0x560b6eaab580 2024-04-21T08:52:18.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.847+0000 7f1cb2a31740 1 -- 172.21.15.82:0/3175404093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaab580 msgr2=0x560b6eaab960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.847+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/3175404093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaab580 0x560b6eaab960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800bdd0 tx=0x7f1c98011750 comp rx=0 tx=0).stop 2024-04-21T08:52:18.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.847+0000 7f1cb2a31740 1 -- 172.21.15.82:0/3175404093 shutdown_connections 2024-04-21T08:52:18.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.847+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/3175404093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.847+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/3175404093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eaabea0 0x560b6eab4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.847+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/3175404093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaab580 0x560b6eaab960 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.847+0000 7f1cb2a31740 1 -- 172.21.15.82:0/3175404093 >> 172.21.15.82:0/3175404093 conn(0x560b6eaa5cb0 msgr2=0x560b6eaa6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.847+0000 7f1cadb02640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1c940a3eb0 0x7f1c940a6370 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x560b6eaab050 tx=0x7f1ca40091c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.848+0000 7f1cb2a31740 1 -- 172.21.15.82:0/3175404093 shutdown_connections 2024-04-21T08:52:18.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.848+0000 7f1cb2a31740 1 -- 172.21.15.82:0/3175404093 wait complete. 2024-04-21T08:52:18.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.848+0000 7f1cb2a31740 1 -- start start 2024-04-21T08:52:18.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.848+0000 7f1cb2a31740 1 Processor -- start 2024-04-21T08:52:18.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.849+0000 7f1cb2a31740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eaabea0 0x560b6eab4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.849+0000 7f1cb2a31740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab4a80 0x560b6eab1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.849+0000 7f1cb2a31740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eab1ba0 0x560b6eab2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cb2a31740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560b6eab78c0 con 0x560b6eaabea0 2024-04-21T08:52:18.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cb2a31740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560b6e9c92f0 con 0x560b6eab4a80 2024-04-21T08:52:18.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cb2a31740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560b6eab7a40 con 0x560b6eab1ba0 2024-04-21T08:52:18.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cad301640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab4a80 0x560b6eab1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cb2a31740 10 client.0 _ll_register_callbacks cb 0x560b6e9d6580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:18.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cad301640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab4a80 0x560b6eab1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:36728/0 (socket says 172.21.15.82:36728) 2024-04-21T08:52:18.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cad301640 1 -- 172.21.15.82:0/1028717068 learned_addr learned my addr 172.21.15.82:0/1028717068 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:18.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cae303640 1 --2- 172.21.15.82:0/1028717068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eab1ba0 0x560b6eab2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.849+0000 7f1cadb02640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eaabea0 0x560b6eab4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cae303640 1 -- 172.21.15.82:0/1028717068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab4a80 msgr2=0x560b6eab1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cae303640 1 --2- 172.21.15.82:0/1028717068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab4a80 0x560b6eab1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cae303640 1 -- 172.21.15.82:0/1028717068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eaabea0 msgr2=0x560b6eab4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cae303640 1 --2- 172.21.15.82:0/1028717068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eaabea0 0x560b6eab4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cae303640 1 -- 172.21.15.82:0/1028717068 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b6e7223f0 con 0x560b6eab1ba0 2024-04-21T08:52:18.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.850+0000 7f1cae303640 1 --2- 172.21.15.82:0/1028717068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eab1ba0 0x560b6eab2020 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00f0290 tx=0x7f1ca00bbeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b1eda02e07b4dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.854+0000 7f1c917fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:52:18.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.854+0000 7f1c917fa640 1 -- 172.21.15.82:0/1028717068 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00098d0 con 0x560b6eab1ba0 2024-04-21T08:52:18.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.854+0000 7f1cb2a31740 1 -- 172.21.15.82:0/1028717068 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560b6ea1dc00 con 0x560b6eab1ba0 2024-04-21T08:52:18.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.854+0000 7f1c917fa640 1 -- 172.21.15.82:0/1028717068 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1ca00df0f0 con 0x560b6eab1ba0 2024-04-21T08:52:18.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.857+0000 7f1c917fa640 1 -- 172.21.15.82:0/1028717068 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1ca00df2f0 con 0x560b6eab1ba0 2024-04-21T08:52:18.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.857+0000 7f1cb2a31740 10 client.26278 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:18.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.857+0000 7f1cb2a31740 1 -- 172.21.15.82:0/1028717068 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560b6e6e20f0 con 0x560b6eab1ba0 2024-04-21T08:52:18.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.857+0000 7f1c917fa640 1 -- 172.21.15.82:0/1028717068 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560b6ea1dc00 con 0x560b6eab1ba0 2024-04-21T08:52:18.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.858+0000 7f1c917fa640 1 -- 172.21.15.82:0/1028717068 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f1ca003c030 con 0x560b6eab1ba0 2024-04-21T08:52:18.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.858+0000 7f1cb2a31740 10 client.26278 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:18.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.858+0000 7f1cb2a31740 10 client.26278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:18.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.858+0000 7f1cb2a31740 20 client.26278 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:18.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.858+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/1028717068 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560b6eac4ea0 0x560b6eac72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.858+0000 7f1cb2a31740 4 client.26278 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:18.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.858+0000 7f1cb2a31740 1 -- 172.21.15.82:0/1028717068 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b6ea17a70 con 0x560b6eac4ea0 2024-04-21T08:52:18.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.858+0000 7f1cadb02640 1 --2- 172.21.15.82:0/1028717068 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560b6eac4ea0 0x560b6eac72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:18 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:52:18.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:18 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:18.860 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:18 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:18.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:18 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:18.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:18 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:18.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:18 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:18.861 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:18 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:18.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.859+0000 7f1c917fa640 10 client.26278 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:18.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.860+0000 7f1cadb02640 1 --2- 172.21.15.82:0/1028717068 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560b6eac4ea0 0x560b6eac72e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0eea65d242166be server_cookie=eed667e34126f7bd in_seq=0 out_seq=0 2024-04-21T08:52:18.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.861+0000 7f1c917fa640 1 -- 172.21.15.82:0/1028717068 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b6ea17a70 con 0x560b6eac4ea0 2024-04-21T08:52:18.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.861+0000 7f1c917fa640 10 client.26278 handle_command_reply: tid=0 2024-04-21T08:52:18.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.861+0000 7f1cb2a31740 1 client.26278 shutdown 2024-04-21T08:52:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.872+0000 7f1cb2a31740 1 -- 172.21.15.82:0/1028717068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eab1ba0 msgr2=0x560b6eab2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.872+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/1028717068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eab1ba0 0x560b6eab2020 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00f0290 tx=0x7f1ca00bbeb0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.872+0000 7f1cb2a31740 1 -- 172.21.15.82:0/1028717068 shutdown_connections 2024-04-21T08:52:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.872+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/1028717068 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560b6eac4ea0 0x560b6eac72e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.872+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/1028717068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eab1ba0 0x560b6eab2020 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.872+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/1028717068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab4a80 0x560b6eab1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.872+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/1028717068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eaabea0 0x560b6eab4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.872+0000 7f1c917fa640 0 client.26278 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.872+0000 7f1cb2a31740 1 -- 172.21.15.82:0/1028717068 >> 172.21.15.82:0/1028717068 conn(0x560b6eaa5cb0 msgr2=0x560b6eaa7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.872+0000 7f1cb2a31740 1 -- 172.21.15.82:0/1028717068 shutdown_connections 2024-04-21T08:52:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.872+0000 7f1cb2a31740 1 -- 172.21.15.82:0/1028717068 wait complete. 2024-04-21T08:52:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.872+0000 7f1cb2a31740 20 client.26278 trim_cache size 0 max 16384 2024-04-21T08:52:18.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.952+0000 7f1cb2a31740 1 Processor -- start 2024-04-21T08:52:18.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.952+0000 7f1cb2a31740 1 -- start start 2024-04-21T08:52:18.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.953+0000 7f1cb2a31740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaabea0 0x560b6eab05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.953+0000 7f1cadb02640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaabea0 0x560b6eab05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.961+0000 7f1cadb02640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaabea0 0x560b6eab05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:36734/0 (socket says 172.21.15.82:36734) 2024-04-21T08:52:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.961+0000 7f1cadb02640 1 -- 172.21.15.82:0/123730888 learned_addr learned my addr 172.21.15.82:0/123730888 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.961+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/123730888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eab1ba0 0x560b6eaa76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.961+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/123730888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.961+0000 7f1cad301640 1 --2- 172.21.15.82:0/123730888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eab1ba0 0x560b6eaa76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.961+0000 7f1cb2a31740 1 -- 172.21.15.82:0/123730888 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c00ffe0 con 0x560b6eab4a80 2024-04-21T08:52:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.961+0000 7f1cb2a31740 1 -- 172.21.15.82:0/123730888 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c011bb0 con 0x560b6eaabea0 2024-04-21T08:52:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.961+0000 7f1cb2a31740 1 -- 172.21.15.82:0/123730888 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c0106b0 con 0x560b6eab1ba0 2024-04-21T08:52:18.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.961+0000 7f1cae303640 1 --2- 172.21.15.82:0/123730888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.962+0000 7f1cad301640 1 -- 172.21.15.82:0/123730888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaabea0 msgr2=0x560b6eab05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.962+0000 7f1cad301640 1 --2- 172.21.15.82:0/123730888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaabea0 0x560b6eab05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.962+0000 7f1cad301640 1 -- 172.21.15.82:0/123730888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 msgr2=0x560b6eab3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.962+0000 7f1cad301640 1 --2- 172.21.15.82:0/123730888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.962+0000 7f1cad301640 1 -- 172.21.15.82:0/123730888 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b6eab71e0 con 0x560b6eab1ba0 2024-04-21T08:52:18.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.962+0000 7f1cae303640 1 --2- 172.21.15.82:0/123730888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:18.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.963+0000 7f1cad301640 1 --2- 172.21.15.82:0/123730888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eab1ba0 0x560b6eaa76c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800bda0 tx=0x7f1c980123f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dfd839f65a51835 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.966+0000 7f1c937fe640 1 -- 172.21.15.82:0/123730888 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98029070 con 0x560b6eab1ba0 2024-04-21T08:52:18.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.966+0000 7f1c937fe640 1 -- 172.21.15.82:0/123730888 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c98012880 con 0x560b6eab1ba0 2024-04-21T08:52:18.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.966+0000 7f1c937fe640 1 -- 172.21.15.82:0/123730888 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98003920 con 0x560b6eab1ba0 2024-04-21T08:52:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.968+0000 7f1cb2a31740 1 -- 172.21.15.82:0/123730888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eab1ba0 msgr2=0x560b6eaa76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.968+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/123730888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eab1ba0 0x560b6eaa76c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800bda0 tx=0x7f1c980123f0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.968+0000 7f1cb2a31740 1 -- 172.21.15.82:0/123730888 shutdown_connections 2024-04-21T08:52:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.968+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/123730888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.968+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/123730888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eab1ba0 0x560b6eaa76c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.968+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/123730888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eaabea0 0x560b6eab05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.968+0000 7f1cb2a31740 1 -- 172.21.15.82:0/123730888 >> 172.21.15.82:0/123730888 conn(0x560b6eaa5cb0 msgr2=0x560b6eaa6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.968+0000 7f1cb2a31740 1 -- 172.21.15.82:0/123730888 shutdown_connections 2024-04-21T08:52:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.968+0000 7f1cb2a31740 1 -- 172.21.15.82:0/123730888 wait complete. 2024-04-21T08:52:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.968+0000 7f1cb2a31740 1 -- start start 2024-04-21T08:52:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.969+0000 7f1cb2a31740 1 Processor -- start 2024-04-21T08:52:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.969+0000 7f1cb2a31740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eaabea0 0x560b6eabcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.969+0000 7f1cb2a31740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab1ba0 0x560b6eabd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.969+0000 7f1cb2a31740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.969+0000 7f1cb2a31740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c0114e0 con 0x560b6eab4a80 2024-04-21T08:52:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.969+0000 7f1cb2a31740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c010e10 con 0x560b6eab1ba0 2024-04-21T08:52:18.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.969+0000 7f1cb2a31740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c00dfc0 con 0x560b6eaabea0 2024-04-21T08:52:18.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.969+0000 7f1cb2a31740 10 client.0 _ll_register_callbacks cb 0x7f1ca0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:18.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.970+0000 7f1cad301640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab1ba0 0x560b6eabd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.970+0000 7f1cad301640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab1ba0 0x560b6eabd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:36744/0 (socket says 172.21.15.82:36744) 2024-04-21T08:52:18.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.970+0000 7f1cad301640 1 -- 172.21.15.82:0/2827763623 learned_addr learned my addr 172.21.15.82:0/2827763623 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.970+0000 7f1cadb02640 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eaabea0 0x560b6eabcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.970+0000 7f1cae303640 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.970+0000 7f1cad301640 1 -- 172.21.15.82:0/2827763623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eaabea0 msgr2=0x560b6eabcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.970+0000 7f1cad301640 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eaabea0 0x560b6eabcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.970+0000 7f1cad301640 1 -- 172.21.15.82:0/2827763623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 msgr2=0x560b6eab7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.970+0000 7f1cad301640 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.970+0000 7f1cad301640 1 -- 172.21.15.82:0/2827763623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b6eab8730 con 0x560b6eab1ba0 2024-04-21T08:52:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.970+0000 7f1cae303640 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.971+0000 7f1cad301640 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab1ba0 0x560b6eabd2c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800bd70 tx=0x7f1c98014c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59395f0c263a413c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.971+0000 7f1c917fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:52:18.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.971+0000 7f1c917fa640 1 -- 172.21.15.82:0/2827763623 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98029040 con 0x560b6eab1ba0 2024-04-21T08:52:18.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.971+0000 7f1c917fa640 1 -- 172.21.15.82:0/2827763623 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1c98011d80 con 0x560b6eab1ba0 2024-04-21T08:52:18.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.971+0000 7f1cb2a31740 1 -- 172.21.15.82:0/2827763623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c98012880 con 0x560b6eab1ba0 2024-04-21T08:52:18.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.971+0000 7f1c917fa640 1 -- 172.21.15.82:0/2827763623 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1c98012a30 con 0x560b6eab1ba0 2024-04-21T08:52:18.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.971+0000 7f1cb2a31740 10 client.26561 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:18.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.971+0000 7f1cb2a31740 1 -- 172.21.15.82:0/2827763623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560b6eb2c060 con 0x560b6eab1ba0 2024-04-21T08:52:18.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.972+0000 7f1c917fa640 1 -- 172.21.15.82:0/2827763623 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c98012880 con 0x560b6eab1ba0 2024-04-21T08:52:18.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.972+0000 7f1c917fa640 1 -- 172.21.15.82:0/2827763623 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f1c98036030 con 0x560b6eab1ba0 2024-04-21T08:52:18.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.972+0000 7f1cb2a31740 10 client.26561 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:18.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.972+0000 7f1cb2a31740 10 client.26561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:18.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.972+0000 7f1cb2a31740 20 client.26561 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:18.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.972+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560b6eb2c4a0 0x560b6eb2e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:18.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.972+0000 7f1cb2a31740 4 client.26561 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:52:18.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.972+0000 7f1cb2a31740 1 -- 172.21.15.82:0/2827763623 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b6e993590 con 0x560b6eb2c4a0 2024-04-21T08:52:18.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.972+0000 7f1cadb02640 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560b6eb2c4a0 0x560b6eb2e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:18.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.973+0000 7f1cadb02640 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560b6eb2c4a0 0x560b6eb2e8c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b11dfcbf8c7c87 server_cookie=ed39510a670d9201 in_seq=0 out_seq=0 2024-04-21T08:52:18.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.973+0000 7f1c917fa640 10 client.26561 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.973+0000 7f1c917fa640 1 -- 172.21.15.82:0/2827763623 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560b6e993590 con 0x560b6eb2c4a0 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.973+0000 7f1c917fa640 10 client.26561 handle_command_reply: tid=0 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.973+0000 7f1cb2a31740 1 client.26561 shutdown 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.974+0000 7f1cb2a31740 1 -- 172.21.15.82:0/2827763623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab1ba0 msgr2=0x560b6eabd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.974+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab1ba0 0x560b6eabd2c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800bd70 tx=0x7f1c98014c80 comp rx=0 tx=0).stop 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.974+0000 7f1cb2a31740 1 -- 172.21.15.82:0/2827763623 shutdown_connections 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.974+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560b6eb2c4a0 0x560b6eb2e8c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.974+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b6eab4a80 0x560b6eab7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.974+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6eab1ba0 0x560b6eabd2c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.974+0000 7f1cb2a31740 1 --2- 172.21.15.82:0/2827763623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b6eaabea0 0x560b6eabcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.974+0000 7f1cb2a31740 1 -- 172.21.15.82:0/2827763623 >> 172.21.15.82:0/2827763623 conn(0x560b6eaa5cb0 msgr2=0x560b6eaa44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.974+0000 7f1c917fa640 0 client.26561 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.974+0000 7f1cb2a31740 1 -- 172.21.15.82:0/2827763623 shutdown_connections 2024-04-21T08:52:18.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.974+0000 7f1cb2a31740 1 -- 172.21.15.82:0/2827763623 wait complete. 2024-04-21T08:52:18.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.974+0000 7f1cb2a31740 20 client.26561 trim_cache size 0 max 16384 2024-04-21T08:52:18.976 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:52:18.976 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:52:18.976 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:52:18.976 INFO:teuthology.orchestra.run.smithi082.stdout: "64552535-4733-45da-b784-7a1f2b81d8dd": { 2024-04-21T08:52:18.976 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:52:18.976 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "64552535-4733-45da-b784-7a1f2b81d8dd", 2024-04-21T08:52:18.976 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:52:18.976 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:52:18.976 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:52:18.977 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:52:18.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.975+0000 7f1cafd66640 1 -- 172.21.15.82:0/3008105488 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1c940a3eb0 msgr2=0x7f1c940a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.975+0000 7f1cafd66640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1c940a3eb0 0x7f1c940a6370 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x560b6eaab050 tx=0x7f1ca40091c0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.976+0000 7f1cafd66640 1 -- 172.21.15.82:0/3008105488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca811b730 msgr2=0x7f1ca81abdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:18.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.976+0000 7f1cafd66640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca811b730 0x7f1ca81abdc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009bf0 tx=0x7f1ca0035ce0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.976+0000 7f1cafd66640 1 -- 172.21.15.82:0/3008105488 shutdown_connections 2024-04-21T08:52:18.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.976+0000 7f1cafd66640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ca811b730 0x7f1ca81abdc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.976+0000 7f1cafd66640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ca81106d0 0x7f1ca81a7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.976+0000 7f1cafd66640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1c940a3eb0 0x7f1c940a6370 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.976+0000 7f1cafd66640 1 --2- 172.21.15.82:0/3008105488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca810e7b0 0x7f1ca81a7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:18.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.976+0000 7f1cafd66640 1 -- 172.21.15.82:0/3008105488 >> 172.21.15.82:0/3008105488 conn(0x7f1ca806d070 msgr2=0x7f1ca811a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:18.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.976+0000 7f1cafd66640 1 -- 172.21.15.82:0/3008105488 shutdown_connections 2024-04-21T08:52:18.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:18.976+0000 7f1cafd66640 1 -- 172.21.15.82:0/3008105488 wait complete. 2024-04-21T08:52:18.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:18 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:52:18.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:18 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:18.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:18 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:18.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:18 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:18.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:18 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:18.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:18 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:18.979 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:18 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:19.005 INFO:tasks.cephfs.filesystem:scrub status for tag:64552535-4733-45da-b784-7a1f2b81d8dd - {'path': '/', 'tag': '64552535-4733-45da-b784-7a1f2b81d8dd', 'options': 'recursive,force'} 2024-04-21T08:52:19.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:18 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:52:19.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:18 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:19.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:18 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:19.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:18 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:19.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:18 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:19.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:18 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:19.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:18 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:19.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:19.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:19.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:19.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:19.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:19.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:19.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:19.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:19.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: pgmap v162: 129 pgs: 129 active+clean; 692 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 856 op/s 2024-04-21T08:52:19.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:19.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:19.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:19.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:19.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:19.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:19 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: pgmap v162: 129 pgs: 129 active+clean; 692 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 856 op/s 2024-04-21T08:52:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:20.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:20.110 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:19 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:20.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:20.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:20.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:20.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:20.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: pgmap v162: 129 pgs: 129 active+clean; 692 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 856 op/s 2024-04-21T08:52:20.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:20.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:20.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:19 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:20.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:20.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:20.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:20.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:20.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:20.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:20.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:52:20.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:20 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:21.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:52:21.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:20 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:21.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:21.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:21.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:21.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:21.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:21.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:52:21.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:20 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:52:21.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:21 smithi134 ceph-mon[31772]: pgmap v163: 129 pgs: 129 active+clean; 692 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 596 op/s 2024-04-21T08:52:22.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:21 smithi082 ceph-mon[27967]: pgmap v163: 129 pgs: 129 active+clean; 692 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 596 op/s 2024-04-21T08:52:22.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:21 smithi106 ceph-mon[31072]: pgmap v163: 129 pgs: 129 active+clean; 692 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 596 op/s 2024-04-21T08:52:24.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:23 smithi082 ceph-mon[27967]: pgmap v164: 129 pgs: 129 active+clean; 674 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 822 op/s 2024-04-21T08:52:24.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:23 smithi106 ceph-mon[31072]: pgmap v164: 129 pgs: 129 active+clean; 674 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 822 op/s 2024-04-21T08:52:24.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:23 smithi134 ceph-mon[31772]: pgmap v164: 129 pgs: 129 active+clean; 674 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.7 MiB/s wr, 822 op/s 2024-04-21T08:52:26.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:25 smithi082 ceph-mon[27967]: pgmap v165: 129 pgs: 129 active+clean; 674 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 450 op/s 2024-04-21T08:52:26.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:25 smithi106 ceph-mon[31072]: pgmap v165: 129 pgs: 129 active+clean; 674 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 450 op/s 2024-04-21T08:52:26.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:25 smithi134 ceph-mon[31772]: pgmap v165: 129 pgs: 129 active+clean; 674 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 450 op/s 2024-04-21T08:52:26.231 INFO:tasks.workunit.client.0.smithi082.stdout: 121 1866837 551 1287661 379 1035893 1414 2024-04-21T08:52:28.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:27 smithi082 ceph-mon[27967]: pgmap v166: 129 pgs: 129 active+clean; 682 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 508 op/s 2024-04-21T08:52:28.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:27 smithi106 ceph-mon[31072]: pgmap v166: 129 pgs: 129 active+clean; 682 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 508 op/s 2024-04-21T08:52:28.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:27 smithi134 ceph-mon[31772]: pgmap v166: 129 pgs: 129 active+clean; 682 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 508 op/s 2024-04-21T08:52:30.111 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:29 smithi082 ceph-mon[27967]: pgmap v167: 129 pgs: 129 active+clean; 686 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 440 op/s 2024-04-21T08:52:30.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:29 smithi106 ceph-mon[31072]: pgmap v167: 129 pgs: 129 active+clean; 686 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 440 op/s 2024-04-21T08:52:30.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:29 smithi134 ceph-mon[31772]: pgmap v167: 129 pgs: 129 active+clean; 686 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 440 op/s 2024-04-21T08:52:32.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:32 smithi082 ceph-mon[27967]: pgmap v168: 129 pgs: 129 active+clean; 686 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 340 op/s 2024-04-21T08:52:32.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:32 smithi106 ceph-mon[31072]: pgmap v168: 129 pgs: 129 active+clean; 686 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 340 op/s 2024-04-21T08:52:32.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:32 smithi134 ceph-mon[31772]: pgmap v168: 129 pgs: 129 active+clean; 686 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 340 op/s 2024-04-21T08:52:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.312+0000 7f1c93bce640 1 -- 172.21.15.82:0/1220403101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c8c10a8f0 msgr2=0x7f1c8c10acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:33.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.312+0000 7f1c93bce640 1 --2- 172.21.15.82:0/1220403101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c8c10a8f0 0x7f1c8c10acd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009900 tx=0x7f1c8802bd10 comp rx=0 tx=0).stop 2024-04-21T08:52:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.315+0000 7f1c93bce640 1 -- 172.21.15.82:0/1220403101 shutdown_connections 2024-04-21T08:52:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.315+0000 7f1c93bce640 1 --2- 172.21.15.82:0/1220403101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8c113ea0 0x7f1c8c116290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.315+0000 7f1c93bce640 1 --2- 172.21.15.82:0/1220403101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c8c10b210 0x7f1c8c113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.315+0000 7f1c93bce640 1 --2- 172.21.15.82:0/1220403101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c8c10a8f0 0x7f1c8c10acd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.315+0000 7f1c93bce640 1 -- 172.21.15.82:0/1220403101 >> 172.21.15.82:0/1220403101 conn(0x7f1c8c06cf00 msgr2=0x7f1c8c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.316+0000 7f1c93bce640 1 -- 172.21.15.82:0/1220403101 shutdown_connections 2024-04-21T08:52:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.316+0000 7f1c93bce640 1 -- 172.21.15.82:0/1220403101 wait complete. 2024-04-21T08:52:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.317+0000 7f1c93bce640 1 Processor -- start 2024-04-21T08:52:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.317+0000 7f1c93bce640 1 -- start start 2024-04-21T08:52:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c93bce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c8c10a8f0 0x7f1c8c1a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c93bce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8c10b210 0x7f1c8c1a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c93bce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c8c113ea0 0x7f1c8c1abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c93bce640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c119860 con 0x7f1c8c10b210 2024-04-21T08:52:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c93bce640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1196e0 con 0x7f1c8c10a8f0 2024-04-21T08:52:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c93bce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1199e0 con 0x7f1c8c113ea0 2024-04-21T08:52:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c9196a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c8c10a8f0 0x7f1c8c1a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c9216b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c8c113ea0 0x7f1c8c1abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c9196a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c8c10a8f0 0x7f1c8c1a7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:41378/0 (socket says 172.21.15.82:41378) 2024-04-21T08:52:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c9216b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c8c113ea0 0x7f1c8c1abd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:40496/0 (socket says 172.21.15.82:40496) 2024-04-21T08:52:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c9196a640 1 -- 172.21.15.82:0/1729348718 learned_addr learned my addr 172.21.15.82:0/1729348718 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c9216b640 1 -- 172.21.15.82:0/1729348718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c8c10a8f0 msgr2=0x7f1c8c1a7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c9216b640 1 --2- 172.21.15.82:0/1729348718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c8c10a8f0 0x7f1c8c1a7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c9216b640 1 -- 172.21.15.82:0/1729348718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8c10b210 msgr2=0x7f1c8c1a7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c9216b640 1 --2- 172.21.15.82:0/1729348718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8c10b210 0x7f1c8c1a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.318+0000 7f1c9216b640 1 -- 172.21.15.82:0/1729348718 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c1ac430 con 0x7f1c8c113ea0 2024-04-21T08:52:33.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.319+0000 7f1c9216b640 1 --2- 172.21.15.82:0/1729348718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c8c113ea0 0x7f1c8c1abd00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400bdf0 tx=0x7f1c8400bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.335+0000 7f1c82ffd640 1 -- 172.21.15.82:0/1729348718 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84019070 con 0x7f1c8c113ea0 2024-04-21T08:52:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.335+0000 7f1c82ffd640 1 -- 172.21.15.82:0/1729348718 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c840148f0 con 0x7f1c8c113ea0 2024-04-21T08:52:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.335+0000 7f1c82ffd640 1 -- 172.21.15.82:0/1729348718 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84012820 con 0x7f1c8c113ea0 2024-04-21T08:52:33.338 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.339+0000 7f1c93bce640 1 -- 172.21.15.82:0/1729348718 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8c1ac720 con 0x7f1c8c113ea0 2024-04-21T08:52:33.338 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.339+0000 7f1c93bce640 1 -- 172.21.15.82:0/1729348718 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c8c1b3f70 con 0x7f1c8c113ea0 2024-04-21T08:52:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.343+0000 7f1c967bf740 1 Processor -- start 2024-04-21T08:52:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.345+0000 7f1c82ffd640 1 -- 172.21.15.82:0/1729348718 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1c84014aa0 con 0x7f1c8c113ea0 2024-04-21T08:52:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.345+0000 7f1c967bf740 1 -- start start 2024-04-21T08:52:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c967bf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd6ac0 0x557ccbbd5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c967bf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd5d50 0x557ccbbd61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c967bf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbdf2d0 0x557ccbbe16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c967bf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557ccbb3a8d0 con 0x557ccbbdf2d0 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c967bf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557ccbad7e80 con 0x557ccbbd5d50 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c967bf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557ccbb36890 con 0x557ccbbd6ac0 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c9196a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd5d50 0x557ccbbd61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c82ffd640 1 --2- 172.21.15.82:0/1729348718 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1c780a3ea0 0x7f1c780a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c9196a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd5d50 0x557ccbbd61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:41386/0 (socket says 172.21.15.82:41386) 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c9196a640 1 -- 172.21.15.82:0/3655511 learned_addr learned my addr 172.21.15.82:0/3655511 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c82ffd640 1 -- 172.21.15.82:0/1729348718 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1c840d0470 con 0x7f1c8c113ea0 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c9196a640 1 -- 172.21.15.82:0/3655511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd6ac0 msgr2=0x557ccbbd5810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c9196a640 1 --2- 172.21.15.82:0/3655511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd6ac0 0x557ccbbd5810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c9196a640 1 -- 172.21.15.82:0/3655511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbdf2d0 msgr2=0x557ccbbe16c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c9196a640 1 --2- 172.21.15.82:0/3655511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbdf2d0 0x557ccbbe16c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c9196a640 1 -- 172.21.15.82:0/3655511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ccbaf9320 con 0x557ccbbd5d50 2024-04-21T08:52:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.346+0000 7f1c9216b640 1 --2- 172.21.15.82:0/1729348718 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1c780a3ea0 0x7f1c780a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.347+0000 7f1c9196a640 1 --2- 172.21.15.82:0/3655511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd5d50 0x557ccbbd61d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802c320 tx=0x7f1c8802c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3209519f3ba2c72 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.354+0000 7f1c77fff640 1 -- 172.21.15.82:0/3655511 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88004040 con 0x557ccbbd5d50 2024-04-21T08:52:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.354+0000 7f1c77fff640 1 -- 172.21.15.82:0/3655511 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c88065030 con 0x557ccbbd5d50 2024-04-21T08:52:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.355+0000 7f1c967bf740 1 -- 172.21.15.82:0/3655511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd5d50 msgr2=0x557ccbbd61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.355+0000 7f1c967bf740 1 --2- 172.21.15.82:0/3655511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd5d50 0x557ccbbd61d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802c320 tx=0x7f1c8802c940 comp rx=0 tx=0).stop 2024-04-21T08:52:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.355+0000 7f1c967bf740 1 -- 172.21.15.82:0/3655511 shutdown_connections 2024-04-21T08:52:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.355+0000 7f1c967bf740 1 --2- 172.21.15.82:0/3655511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbdf2d0 0x557ccbbe16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.355+0000 7f1c967bf740 1 --2- 172.21.15.82:0/3655511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd5d50 0x557ccbbd61d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.355+0000 7f1c967bf740 1 --2- 172.21.15.82:0/3655511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd6ac0 0x557ccbbd5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.355+0000 7f1c967bf740 1 -- 172.21.15.82:0/3655511 >> 172.21.15.82:0/3655511 conn(0x557ccbb9cc90 msgr2=0x557ccbbd0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:33.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.357+0000 7f1c9216b640 1 --2- 172.21.15.82:0/1729348718 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1c780a3ea0 0x7f1c780a6360 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1c840cd9c0 tx=0x7f1c8400e040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:33.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.361+0000 7f1c967bf740 1 -- 172.21.15.82:0/3655511 shutdown_connections 2024-04-21T08:52:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.361+0000 7f1c967bf740 1 -- 172.21.15.82:0/3655511 wait complete. 2024-04-21T08:52:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.361+0000 7f1c967bf740 1 -- start start 2024-04-21T08:52:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.361+0000 7f1c967bf740 1 Processor -- start 2024-04-21T08:52:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.362+0000 7f1c967bf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd5d50 0x557ccbbdec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.362+0000 7f1c967bf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd6ac0 0x557ccbbdbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.362+0000 7f1c967bf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbdf2d0 0x557ccbbdc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.362+0000 7f1c967bf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557ccbbe1e30 con 0x557ccbbd6ac0 2024-04-21T08:52:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.362+0000 7f1c967bf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557ccbac2eb0 con 0x557ccbbdf2d0 2024-04-21T08:52:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.362+0000 7f1c967bf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557ccbbe1fb0 con 0x557ccbbd5d50 2024-04-21T08:52:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.362+0000 7f1c967bf740 10 client.0 _ll_register_callbacks cb 0x557ccbaefb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.363+0000 7f1c91169640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbdf2d0 0x557ccbbdc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.363+0000 7f1c91169640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbdf2d0 0x557ccbbdc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:41394/0 (socket says 172.21.15.82:41394) 2024-04-21T08:52:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.363+0000 7f1c91169640 1 -- 172.21.15.82:0/3793518255 learned_addr learned my addr 172.21.15.82:0/3793518255 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.363+0000 7f1c91169640 1 --2- 172.21.15.82:0/3793518255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd5d50 0x557ccbbdec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.363+0000 7f1c91169640 1 -- 172.21.15.82:0/3793518255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbdf2d0 msgr2=0x557ccbbdc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.363+0000 7f1c91169640 1 --2- 172.21.15.82:0/3793518255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbdf2d0 0x557ccbbdc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.363+0000 7f1c91169640 1 -- 172.21.15.82:0/3793518255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd6ac0 msgr2=0x557ccbbdbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.363+0000 7f1c91169640 1 --2- 172.21.15.82:0/3793518255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd6ac0 0x557ccbbdbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.363+0000 7f1c91169640 1 -- 172.21.15.82:0/3793518255 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ccb84d210 con 0x557ccbbd5d50 2024-04-21T08:52:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.363+0000 7f1c91169640 1 --2- 172.21.15.82:0/3793518255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd5d50 0x557ccbbdec20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c1a8570 tx=0x7f1c7c001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=befef1ba2b672084 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.366+0000 7f1c757fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:52:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.366+0000 7f1c757fa640 1 -- 172.21.15.82:0/3793518255 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c012040 con 0x557ccbbd5d50 2024-04-21T08:52:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.367+0000 7f1c967bf740 1 -- 172.21.15.82:0/3793518255 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c88065030 con 0x557ccbbd5d50 2024-04-21T08:52:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.367+0000 7f1c757fa640 1 -- 172.21.15.82:0/3793518255 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1c7c0072f0 con 0x557ccbbd5d50 2024-04-21T08:52:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.367+0000 7f1c757fa640 1 -- 172.21.15.82:0/3793518255 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1c7c021050 con 0x557ccbbd5d50 2024-04-21T08:52:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.367+0000 7f1c967bf740 10 client.26290 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.367+0000 7f1c967bf740 1 -- 172.21.15.82:0/3793518255 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557ccb919620 con 0x557ccbbd5d50 2024-04-21T08:52:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.368+0000 7f1c757fa640 1 -- 172.21.15.82:0/3793518255 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c88065030 con 0x557ccbbd5d50 2024-04-21T08:52:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.369+0000 7f1c757fa640 1 -- 172.21.15.82:0/3793518255 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f1c7c008b30 con 0x557ccbbd5d50 2024-04-21T08:52:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.369+0000 7f1c967bf740 10 client.26290 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.369+0000 7f1c967bf740 10 client.26290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.369+0000 7f1c967bf740 20 client.26290 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.370+0000 7f1c967bf740 1 --2- 172.21.15.82:0/3793518255 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ccbbef5f0 0x557ccbbf1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.370+0000 7f1c967bf740 4 client.26290 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.370+0000 7f1c967bf740 1 -- 172.21.15.82:0/3793518255 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ccbb4e880 con 0x557ccbbef5f0 2024-04-21T08:52:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.370+0000 7f1c9196a640 1 --2- 172.21.15.82:0/3793518255 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ccbbef5f0 0x557ccbbf1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.371+0000 7f1c9196a640 1 --2- 172.21.15.82:0/3793518255 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ccbbef5f0 0x557ccbbf1ab0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a9a08b982ffdc3f server_cookie=fdbbdb9ef9b3af5d in_seq=0 out_seq=0 2024-04-21T08:52:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.371+0000 7f1c757fa640 10 client.26290 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.373+0000 7f1c757fa640 1 -- 172.21.15.82:0/3793518255 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ccbb4e880 con 0x557ccbbef5f0 2024-04-21T08:52:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.373+0000 7f1c757fa640 10 client.26290 handle_command_reply: tid=0 2024-04-21T08:52:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.373+0000 7f1c967bf740 1 client.26290 shutdown 2024-04-21T08:52:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.375+0000 7f1c967bf740 1 -- 172.21.15.82:0/3793518255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd5d50 msgr2=0x557ccbbdec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.375+0000 7f1c967bf740 1 --2- 172.21.15.82:0/3793518255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd5d50 0x557ccbbdec20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c1a8570 tx=0x7f1c7c001120 comp rx=0 tx=0).stop 2024-04-21T08:52:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.375+0000 7f1c967bf740 1 -- 172.21.15.82:0/3793518255 shutdown_connections 2024-04-21T08:52:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.375+0000 7f1c967bf740 1 --2- 172.21.15.82:0/3793518255 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ccbbef5f0 0x557ccbbf1ab0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.375+0000 7f1c967bf740 1 --2- 172.21.15.82:0/3793518255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbdf2d0 0x557ccbbdc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.375+0000 7f1c967bf740 1 --2- 172.21.15.82:0/3793518255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd6ac0 0x557ccbbdbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.375+0000 7f1c967bf740 1 --2- 172.21.15.82:0/3793518255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd5d50 0x557ccbbdec20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.375+0000 7f1c967bf740 1 -- 172.21.15.82:0/3793518255 >> 172.21.15.82:0/3793518255 conn(0x557ccbb9cc90 msgr2=0x557ccbbd2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.375+0000 7f1c757fa640 0 client.26290 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.376+0000 7f1c967bf740 1 -- 172.21.15.82:0/3793518255 shutdown_connections 2024-04-21T08:52:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.376+0000 7f1c967bf740 1 -- 172.21.15.82:0/3793518255 wait complete. 2024-04-21T08:52:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.376+0000 7f1c967bf740 20 client.26290 trim_cache size 0 max 16384 2024-04-21T08:52:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.407+0000 7f1c967bf740 1 Processor -- start 2024-04-21T08:52:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.407+0000 7f1c967bf740 1 -- start start 2024-04-21T08:52:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.408+0000 7f1c967bf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd5d50 0x557ccbbcb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.408+0000 7f1c967bf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd6ac0 0x557ccbbde6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.408+0000 7f1c967bf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbdf2d0 0x557ccbbdec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.408+0000 7f1c967bf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1b57d0 con 0x557ccbbd5d50 2024-04-21T08:52:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c967bf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1b73a0 con 0x557ccbbdf2d0 2024-04-21T08:52:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c9196a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd6ac0 0x557ccbbde6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c9196a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd6ac0 0x557ccbbde6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:40512/0 (socket says 172.21.15.82:40512) 2024-04-21T08:52:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c9196a640 1 -- 172.21.15.82:0/1596741512 learned_addr learned my addr 172.21.15.82:0/1596741512 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c91169640 1 --2- 172.21.15.82:0/1596741512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbdf2d0 0x557ccbbdec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c967bf740 1 -- 172.21.15.82:0/1596741512 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1b5ea0 con 0x557ccbbd6ac0 2024-04-21T08:52:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c91169640 1 --2- 172.21.15.82:0/1596741512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd5d50 0x557ccbbcb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c91169640 1 -- 172.21.15.82:0/1596741512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd6ac0 msgr2=0x557ccbbde6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c91169640 1 --2- 172.21.15.82:0/1596741512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd6ac0 0x557ccbbde6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c91169640 1 -- 172.21.15.82:0/1596741512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd5d50 msgr2=0x557ccbbcb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c91169640 1 --2- 172.21.15.82:0/1596741512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd5d50 0x557ccbbcb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c91169640 1 -- 172.21.15.82:0/1596741512 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ccbbeba30 con 0x557ccbbdf2d0 2024-04-21T08:52:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.409+0000 7f1c91169640 1 --2- 172.21.15.82:0/1596741512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbdf2d0 0x557ccbbdec20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0017e0 tx=0x7f1c7c006040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b52151c26886fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.411+0000 7f1c777fe640 1 -- 172.21.15.82:0/1596741512 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c008b00 con 0x557ccbbdf2d0 2024-04-21T08:52:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.411+0000 7f1c777fe640 1 -- 172.21.15.82:0/1596741512 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7c009be0 con 0x557ccbbdf2d0 2024-04-21T08:52:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.411+0000 7f1c777fe640 1 -- 172.21.15.82:0/1596741512 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c009300 con 0x557ccbbdf2d0 2024-04-21T08:52:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.412+0000 7f1c967bf740 1 -- 172.21.15.82:0/1596741512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbdf2d0 msgr2=0x557ccbbdec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.412+0000 7f1c967bf740 1 --2- 172.21.15.82:0/1596741512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbdf2d0 0x557ccbbdec20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0017e0 tx=0x7f1c7c006040 comp rx=0 tx=0).stop 2024-04-21T08:52:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.413+0000 7f1c967bf740 1 -- 172.21.15.82:0/1596741512 shutdown_connections 2024-04-21T08:52:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.413+0000 7f1c967bf740 1 --2- 172.21.15.82:0/1596741512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbdf2d0 0x557ccbbdec20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.413+0000 7f1c967bf740 1 --2- 172.21.15.82:0/1596741512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbd6ac0 0x557ccbbde6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.413+0000 7f1c967bf740 1 --2- 172.21.15.82:0/1596741512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd5d50 0x557ccbbcb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.413+0000 7f1c967bf740 1 -- 172.21.15.82:0/1596741512 >> 172.21.15.82:0/1596741512 conn(0x557ccbb9cc90 msgr2=0x557ccbbd7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.413+0000 7f1c967bf740 1 -- 172.21.15.82:0/1596741512 shutdown_connections 2024-04-21T08:52:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.413+0000 7f1c967bf740 1 -- 172.21.15.82:0/1596741512 wait complete. 2024-04-21T08:52:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.414+0000 7f1c967bf740 1 -- start start 2024-04-21T08:52:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.414+0000 7f1c967bf740 1 Processor -- start 2024-04-21T08:52:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c967bf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd5d50 0x557ccbbf7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c967bf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd6ac0 0x557ccbbf8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c967bf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbdf2d0 0x557ccbbf2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c967bf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1b6cd0 con 0x557ccbbd5d50 2024-04-21T08:52:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c967bf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1b6600 con 0x557ccbbd6ac0 2024-04-21T08:52:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c967bf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c050530 con 0x557ccbbdf2d0 2024-04-21T08:52:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c967bf740 10 client.0 _ll_register_callbacks cb 0x7f1c7c001880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c9196a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd6ac0 0x557ccbbf8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c9196a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd6ac0 0x557ccbbf8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:41412/0 (socket says 172.21.15.82:41412) 2024-04-21T08:52:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c9196a640 1 -- 172.21.15.82:0/1312688759 learned_addr learned my addr 172.21.15.82:0/1312688759 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c91169640 1 --2- 172.21.15.82:0/1312688759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbdf2d0 0x557ccbbf2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c9196a640 1 -- 172.21.15.82:0/1312688759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbdf2d0 msgr2=0x557ccbbf2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c9196a640 1 --2- 172.21.15.82:0/1312688759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbdf2d0 0x557ccbbf2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c9196a640 1 -- 172.21.15.82:0/1312688759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd5d50 msgr2=0x557ccbbf7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c9196a640 1 --2- 172.21.15.82:0/1312688759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd5d50 0x557ccbbf7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.415+0000 7f1c9196a640 1 -- 172.21.15.82:0/1312688759 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ccbbf33b0 con 0x557ccbbd6ac0 2024-04-21T08:52:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.416+0000 7f1c9196a640 1 --2- 172.21.15.82:0/1312688759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd6ac0 0x557ccbbf8300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802ce00 tx=0x7f1c88009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=305733d798bcf7ad server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.416+0000 7f1c757fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:52:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.416+0000 7f1c757fa640 1 -- 172.21.15.82:0/1312688759 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88076030 con 0x557ccbbd6ac0 2024-04-21T08:52:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.416+0000 7f1c967bf740 1 -- 172.21.15.82:0/1312688759 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c7c009be0 con 0x557ccbbd6ac0 2024-04-21T08:52:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.416+0000 7f1c757fa640 1 -- 172.21.15.82:0/1312688759 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1c8802ca60 con 0x557ccbbd6ac0 2024-04-21T08:52:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.417+0000 7f1c757fa640 1 -- 172.21.15.82:0/1312688759 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1c88083050 con 0x557ccbbd6ac0 2024-04-21T08:52:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.417+0000 7f1c967bf740 10 client.26576 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.417+0000 7f1c967bf740 1 -- 172.21.15.82:0/1312688759 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557ccbc54f90 con 0x557ccbbd6ac0 2024-04-21T08:52:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.417+0000 7f1c757fa640 1 -- 172.21.15.82:0/1312688759 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7c009be0 con 0x557ccbbd6ac0 2024-04-21T08:52:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.417+0000 7f1c757fa640 1 -- 172.21.15.82:0/1312688759 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f1c8807d4e0 con 0x557ccbbd6ac0 2024-04-21T08:52:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.417+0000 7f1c967bf740 10 client.26576 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.417+0000 7f1c967bf740 10 client.26576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.417+0000 7f1c967bf740 20 client.26576 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.417+0000 7f1c967bf740 1 --2- 172.21.15.82:0/1312688759 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ccbc55640 0x557ccbc57a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.417+0000 7f1c967bf740 4 client.26576 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:52:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.417+0000 7f1c967bf740 1 -- 172.21.15.82:0/1312688759 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ccbada3d0 con 0x557ccbc55640 2024-04-21T08:52:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.418+0000 7f1c91169640 1 --2- 172.21.15.82:0/1312688759 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ccbc55640 0x557ccbc57a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.418+0000 7f1c757fa640 10 client.26576 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:33.419+0000 7f1c91169640 1 --2- 172.21.15.82:0/1312688759 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ccbc55640 0x557ccbc57a60 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d7c8691df17495 server_cookie=fe20325c6f17bb0b in_seq=0 out_seq=0 2024-04-21T08:52:34.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:34 smithi082 ceph-mon[27967]: pgmap v169: 129 pgs: 129 active+clean; 747 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.4 MiB/s wr, 736 op/s 2024-04-21T08:52:34.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:34 smithi106 ceph-mon[31072]: pgmap v169: 129 pgs: 129 active+clean; 747 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.4 MiB/s wr, 736 op/s 2024-04-21T08:52:34.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:34 smithi134 ceph-mon[31772]: pgmap v169: 129 pgs: 129 active+clean; 747 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.4 MiB/s wr, 736 op/s 2024-04-21T08:52:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.558+0000 7f1c757fa640 1 -- 172.21.15.82:0/1312688759 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ccbada3d0 con 0x557ccbc55640 2024-04-21T08:52:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.558+0000 7f1c757fa640 10 client.26576 handle_command_reply: tid=0 2024-04-21T08:52:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.559+0000 7f1c967bf740 1 client.26576 shutdown 2024-04-21T08:52:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.559+0000 7f1c967bf740 1 -- 172.21.15.82:0/1312688759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd6ac0 msgr2=0x557ccbbf8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.559+0000 7f1c967bf740 1 --2- 172.21.15.82:0/1312688759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd6ac0 0x557ccbbf8300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802ce00 tx=0x7f1c88009260 comp rx=0 tx=0).stop 2024-04-21T08:52:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.559+0000 7f1c967bf740 1 -- 172.21.15.82:0/1312688759 shutdown_connections 2024-04-21T08:52:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.559+0000 7f1c967bf740 1 --2- 172.21.15.82:0/1312688759 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ccbc55640 0x557ccbc57a60 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.559+0000 7f1c967bf740 1 --2- 172.21.15.82:0/1312688759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ccbbdf2d0 0x557ccbbf2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.559+0000 7f1c967bf740 1 --2- 172.21.15.82:0/1312688759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ccbbd6ac0 0x557ccbbf8300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.559+0000 7f1c967bf740 1 --2- 172.21.15.82:0/1312688759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ccbbd5d50 0x557ccbbf7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.559+0000 7f1c967bf740 1 -- 172.21.15.82:0/1312688759 >> 172.21.15.82:0/1312688759 conn(0x557ccbb9cc90 msgr2=0x557ccbbda800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.559+0000 7f1c757fa640 0 client.26576 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.560+0000 7f1c967bf740 1 -- 172.21.15.82:0/1312688759 shutdown_connections 2024-04-21T08:52:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.560+0000 7f1c967bf740 1 -- 172.21.15.82:0/1312688759 wait complete. 2024-04-21T08:52:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.560+0000 7f1c967bf740 20 client.26576 trim_cache size 0 max 16384 2024-04-21T08:52:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:52:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:52:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:52:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:52:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.608+0000 7f1c93bce640 1 -- 172.21.15.82:0/1729348718 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1c780a3ea0 msgr2=0x7f1c780a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.608+0000 7f1c93bce640 1 --2- 172.21.15.82:0/1729348718 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1c780a3ea0 0x7f1c780a6360 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1c840cd9c0 tx=0x7f1c8400e040 comp rx=0 tx=0).stop 2024-04-21T08:52:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.608+0000 7f1c93bce640 1 -- 172.21.15.82:0/1729348718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c8c113ea0 msgr2=0x7f1c8c1abd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.608+0000 7f1c93bce640 1 --2- 172.21.15.82:0/1729348718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c8c113ea0 0x7f1c8c1abd00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400bdf0 tx=0x7f1c8400bee0 comp rx=0 tx=0).stop 2024-04-21T08:52:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.612+0000 7f1c93bce640 1 -- 172.21.15.82:0/1729348718 shutdown_connections 2024-04-21T08:52:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.612+0000 7f1c93bce640 1 --2- 172.21.15.82:0/1729348718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c8c113ea0 0x7f1c8c1abd00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.612+0000 7f1c93bce640 1 --2- 172.21.15.82:0/1729348718 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1c780a3ea0 0x7f1c780a6360 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.612+0000 7f1c93bce640 1 --2- 172.21.15.82:0/1729348718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8c10b210 0x7f1c8c1a7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.612+0000 7f1c93bce640 1 --2- 172.21.15.82:0/1729348718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c8c10a8f0 0x7f1c8c1a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.612+0000 7f1c93bce640 1 -- 172.21.15.82:0/1729348718 >> 172.21.15.82:0/1729348718 conn(0x7f1c8c06cf00 msgr2=0x7f1c8c1098a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.613+0000 7f1c93bce640 1 -- 172.21.15.82:0/1729348718 shutdown_connections 2024-04-21T08:52:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:34.613+0000 7f1c93bce640 1 -- 172.21.15.82:0/1729348718 wait complete. 2024-04-21T08:52:36.231 INFO:tasks.workunit.client.0.smithi082.stdout: 124 1797023 226 1232875 302 1018855 805 2024-04-21T08:52:36.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:36 smithi106 ceph-mon[31072]: pgmap v170: 129 pgs: 129 active+clean; 747 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 511 op/s 2024-04-21T08:52:36.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:36 smithi134 ceph-mon[31772]: pgmap v170: 129 pgs: 129 active+clean; 747 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 511 op/s 2024-04-21T08:52:36.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:36 smithi082 ceph-mon[27967]: pgmap v170: 129 pgs: 129 active+clean; 747 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 511 op/s 2024-04-21T08:52:38.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:38 smithi106 ceph-mon[31072]: pgmap v171: 129 pgs: 129 active+clean; 762 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.9 MiB/s wr, 686 op/s 2024-04-21T08:52:38.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:38 smithi134 ceph-mon[31772]: pgmap v171: 129 pgs: 129 active+clean; 762 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.9 MiB/s wr, 686 op/s 2024-04-21T08:52:38.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:38 smithi082 ceph-mon[27967]: pgmap v171: 129 pgs: 129 active+clean; 762 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.9 MiB/s wr, 686 op/s 2024-04-21T08:52:40.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:40 smithi134 ceph-mon[31772]: pgmap v172: 129 pgs: 129 active+clean; 767 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 800 op/s 2024-04-21T08:52:40.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:40 smithi082 ceph-mon[27967]: pgmap v172: 129 pgs: 129 active+clean; 767 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 800 op/s 2024-04-21T08:52:40.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:40 smithi106 ceph-mon[31072]: pgmap v172: 129 pgs: 129 active+clean; 767 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 800 op/s 2024-04-21T08:52:42.430 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:42 smithi106 ceph-mon[31072]: pgmap v173: 129 pgs: 129 active+clean; 767 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.4 MiB/s wr, 743 op/s 2024-04-21T08:52:42.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:42 smithi134 ceph-mon[31772]: pgmap v173: 129 pgs: 129 active+clean; 767 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.4 MiB/s wr, 743 op/s 2024-04-21T08:52:42.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:42 smithi082 ceph-mon[27967]: pgmap v173: 129 pgs: 129 active+clean; 767 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.4 MiB/s wr, 743 op/s 2024-04-21T08:52:44.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:44 smithi134 ceph-mon[31772]: pgmap v174: 129 pgs: 129 active+clean; 747 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.5 MiB/s wr, 913 op/s 2024-04-21T08:52:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:44 smithi082 ceph-mon[27967]: pgmap v174: 129 pgs: 129 active+clean; 747 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.5 MiB/s wr, 913 op/s 2024-04-21T08:52:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:44 smithi106 ceph-mon[31072]: pgmap v174: 129 pgs: 129 active+clean; 747 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.5 MiB/s wr, 913 op/s 2024-04-21T08:52:46.231 INFO:tasks.workunit.client.0.smithi082.stdout: 133 2165044 470 1510362 444 1258595 1440 2024-04-21T08:52:46.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:46 smithi134 ceph-mon[31772]: pgmap v175: 129 pgs: 129 active+clean; 747 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 517 op/s 2024-04-21T08:52:46.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:46 smithi082 ceph-mon[27967]: pgmap v175: 129 pgs: 129 active+clean; 747 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 517 op/s 2024-04-21T08:52:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:46 smithi106 ceph-mon[31072]: pgmap v175: 129 pgs: 129 active+clean; 747 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 517 op/s 2024-04-21T08:52:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.869+0000 7f98e2915640 1 -- 172.21.15.82:0/101935917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98dc101aa0 msgr2=0x7f98dc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.869+0000 7f98e2915640 1 --2- 172.21.15.82:0/101935917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98dc101aa0 0x7f98dc105a90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009900 tx=0x7f98c402bc30 comp rx=0 tx=0).stop 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.869+0000 7f98e2915640 1 -- 172.21.15.82:0/101935917 shutdown_connections 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.869+0000 7f98e2915640 1 --2- 172.21.15.82:0/101935917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98dc106170 0x7f98dc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.869+0000 7f98e2915640 1 --2- 172.21.15.82:0/101935917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98dc101aa0 0x7f98dc105a90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.869+0000 7f98e2915640 1 --2- 172.21.15.82:0/101935917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98dc1010f0 0x7f98dc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.869+0000 7f98e2915640 1 -- 172.21.15.82:0/101935917 >> 172.21.15.82:0/101935917 conn(0x7f98dc077fd0 msgr2=0x7f98dc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.869+0000 7f98e2915640 1 -- 172.21.15.82:0/101935917 shutdown_connections 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.869+0000 7f98e2915640 1 -- 172.21.15.82:0/101935917 wait complete. 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.869+0000 7f98e2915640 1 Processor -- start 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.869+0000 7f98e2915640 1 -- start start 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e2915640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98dc1010f0 0x7f98dc199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e2915640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98dc106170 0x7f98dc199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e2915640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98dc19a020 0x7f98dc1a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e2915640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1110b0 con 0x7f98dc1010f0 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e2915640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc110f30 con 0x7f98dc19a020 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e2915640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc111230 con 0x7f98dc106170 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98dc1010f0 0x7f98dc199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e0eb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98dc19a020 0x7f98dc1a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98dc106170 0x7f98dc199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e0eb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98dc19a020 0x7f98dc1a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:49724/0 (socket says 172.21.15.82:49724) 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e0eb2640 1 -- 172.21.15.82:0/3548557478 learned_addr learned my addr 172.21.15.82:0/3548557478 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98dc1010f0 0x7f98dc199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46530/0 (socket says 172.21.15.82:46530) 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98dc106170 0x7f98dc199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:56174/0 (socket says 172.21.15.82:56174) 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e0eb2640 1 -- 172.21.15.82:0/3548557478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98dc106170 msgr2=0x7f98dc199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e0eb2640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98dc106170 0x7f98dc199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e0eb2640 1 -- 172.21.15.82:0/3548557478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98dc1010f0 msgr2=0x7f98dc199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:47.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e0eb2640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98dc1010f0 0x7f98dc199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e0eb2640 1 -- 172.21.15.82:0/3548557478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc1a4da0 con 0x7f98dc19a020 2024-04-21T08:52:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98d37fe640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98dc106170 0x7f98dc199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e0eb2640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98dc19a020 0x7f98dc1a45d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f98d400e970 tx=0x7f98d400ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98d9ffb640 1 -- 172.21.15.82:0/3548557478 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d400ccc0 con 0x7f98dc19a020 2024-04-21T08:52:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98d9ffb640 1 -- 172.21.15.82:0/3548557478 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d400ce60 con 0x7f98dc19a020 2024-04-21T08:52:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98e2915640 1 -- 172.21.15.82:0/3548557478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98dc1a5090 con 0x7f98dc19a020 2024-04-21T08:52:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.870+0000 7f98d9ffb640 1 -- 172.21.15.82:0/3548557478 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4010430 con 0x7f98dc19a020 2024-04-21T08:52:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.871+0000 7f98e2915640 1 -- 172.21.15.82:0/3548557478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98dc1a5600 con 0x7f98dc19a020 2024-04-21T08:52:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.873+0000 7f98d9ffb640 1 -- 172.21.15.82:0/3548557478 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f98d4004f90 con 0x7f98dc19a020 2024-04-21T08:52:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.873+0000 7f98e55e3740 1 Processor -- start 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.873+0000 7f98e55e3740 1 -- start start 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.873+0000 7f98e55e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fcc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.873+0000 7f98e55e3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fccb60 0x55d4d1fccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.873+0000 7f98e55e3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.873+0000 7f98e55e3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d4d1f318d0 con 0x55d4d1fccb60 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.873+0000 7f98e55e3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d4d1ecee80 con 0x55d4d1fd60e0 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.873+0000 7f98e55e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4d1f2d890 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d9ffb640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f98c80a3e40 0x7f98c80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fcc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fcc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:56184/0 (socket says 172.21.15.82:56184) 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fccb60 0x55d4d1fccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d37fe640 1 -- 172.21.15.82:0/2641684367 learned_addr learned my addr 172.21.15.82:0/2641684367 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d37fe640 1 --2- 172.21.15.82:0/2641684367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98e0eb2640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f98c80a3e40 0x7f98c80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d9ffb640 1 -- 172.21.15.82:0/3548557478 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f98d40d0e70 con 0x7f98dc19a020 2024-04-21T08:52:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d37fe640 1 -- 172.21.15.82:0/2641684367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fcd8d0 msgr2=0x55d4d1fcc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d37fe640 1 --2- 172.21.15.82:0/2641684367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fcc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d37fe640 1 -- 172.21.15.82:0/2641684367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fccb60 msgr2=0x55d4d1fccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d37fe640 1 --2- 172.21.15.82:0/2641684367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fccb60 0x55d4d1fccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d37fe640 1 -- 172.21.15.82:0/2641684367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4d1ef0320 con 0x55d4d1fd60e0 2024-04-21T08:52:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d37fe640 1 --2- 172.21.15.82:0/2641684367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd84d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98c402c780 tx=0x7f98c4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f9f85d3259a06db server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98e0eb2640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f98c80a3e40 0x7f98c80a6300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f98d40ce800 tx=0x7f98d40bd040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d27fc640 1 -- 172.21.15.82:0/2641684367 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c402c7e0 con 0x55d4d1fd60e0 2024-04-21T08:52:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d27fc640 1 -- 172.21.15.82:0/2641684367 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c40035f0 con 0x55d4d1fd60e0 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.874+0000 7f98d27fc640 1 -- 172.21.15.82:0/2641684367 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4043690 con 0x55d4d1fd60e0 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.875+0000 7f98e55e3740 1 -- 172.21.15.82:0/2641684367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fd60e0 msgr2=0x55d4d1fd84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.875+0000 7f98e55e3740 1 --2- 172.21.15.82:0/2641684367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd84d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98c402c780 tx=0x7f98c4005fd0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.875+0000 7f98e55e3740 1 -- 172.21.15.82:0/2641684367 shutdown_connections 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.875+0000 7f98e55e3740 1 --2- 172.21.15.82:0/2641684367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd84d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.875+0000 7f98e55e3740 1 --2- 172.21.15.82:0/2641684367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fccb60 0x55d4d1fccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.875+0000 7f98e55e3740 1 --2- 172.21.15.82:0/2641684367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fcc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.875+0000 7f98e55e3740 1 -- 172.21.15.82:0/2641684367 >> 172.21.15.82:0/2641684367 conn(0x55d4d1f93c90 msgr2=0x55d4d1fc7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.875+0000 7f98e55e3740 1 -- 172.21.15.82:0/2641684367 shutdown_connections 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.875+0000 7f98e55e3740 1 -- 172.21.15.82:0/2641684367 wait complete. 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.875+0000 7f98e55e3740 1 -- start start 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98e55e3740 1 Processor -- start 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98e55e3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fccb60 0x55d4d1fd5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98e55e3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fd2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98e55e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98e55e3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d4d1fd8db0 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98e55e3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d4d1eb9eb0 con 0x55d4d1fccb60 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98e55e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d4d1fd8f30 con 0x55d4d1fd60e0 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98e55e3740 10 client.0 _ll_register_callbacks cb 0x55d4d1ee6b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fd2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fccb60 0x55d4d1fd5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fd2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46550/0 (socket says 172.21.15.82:46550) 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98dbfff640 1 -- 172.21.15.82:0/4145797862 learned_addr learned my addr 172.21.15.82:0/4145797862 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98d37fe640 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98dbfff640 1 -- 172.21.15.82:0/4145797862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 msgr2=0x55d4d1fd3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98dbfff640 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98dbfff640 1 -- 172.21.15.82:0/4145797862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fccb60 msgr2=0x55d4d1fd5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:47.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98dbfff640 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fccb60 0x55d4d1fd5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98dbfff640 1 -- 172.21.15.82:0/4145797862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4d1c44210 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.876+0000 7f98d37fe640 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98dbfff640 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fd2ec0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f98cc011d10 tx=0x7f98cc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48ba3ea85411742 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98bb7fe640 1 -- 172.21.15.82:0/4145797862 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc01f060 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98bb7fe640 1 -- 172.21.15.82:0/4145797862 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f98cc017050 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98e55e3740 1 -- 172.21.15.82:0/4145797862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c40035f0 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98bb7fe640 1 -- 172.21.15.82:0/4145797862 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f98cc026050 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98e55e3740 10 client.17217 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98e55e3740 1 -- 172.21.15.82:0/4145797862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d4d1d10620 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98bb7fe640 1 -- 172.21.15.82:0/4145797862 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c40035f0 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98bb7fe640 1 -- 172.21.15.82:0/4145797862 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f98cc019030 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98e55e3740 10 client.17217 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98e55e3740 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98e55e3740 20 client.17217 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98e55e3740 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d4d1fe6170 0x55d4d1fe8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98e55e3740 4 client.17217 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.877+0000 7f98e55e3740 1 -- 172.21.15.82:0/4145797862 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4d1f45880 con 0x55d4d1fe6170 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.878+0000 7f98d37fe640 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d4d1fe6170 0x55d4d1fe8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.878+0000 7f98d37fe640 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d4d1fe6170 0x55d4d1fe8580 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=455c01d9218be178 server_cookie=cf375c08ba1c58ff in_seq=0 out_seq=0 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.878+0000 7f98bb7fe640 10 client.17217 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98bb7fe640 1 -- 172.21.15.82:0/4145797862 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4d1f45880 con 0x55d4d1fe6170 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98bb7fe640 10 client.17217 handle_command_reply: tid=0 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98e55e3740 1 client.17217 shutdown 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98e55e3740 1 -- 172.21.15.82:0/4145797862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 msgr2=0x55d4d1fd2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98e55e3740 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fd2ec0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f98cc011d10 tx=0x7f98cc00f790 comp rx=0 tx=0).stop 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98e55e3740 1 -- 172.21.15.82:0/4145797862 shutdown_connections 2024-04-21T08:52:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98e55e3740 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d4d1fe6170 0x55d4d1fe8580 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98e55e3740 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98e55e3740 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fd2ec0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98bb7fe640 0 client.17217 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98e55e3740 1 --2- 172.21.15.82:0/4145797862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fccb60 0x55d4d1fd5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98e55e3740 1 -- 172.21.15.82:0/4145797862 >> 172.21.15.82:0/4145797862 conn(0x55d4d1f93c90 msgr2=0x55d4d1fc95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98e55e3740 1 -- 172.21.15.82:0/4145797862 shutdown_connections 2024-04-21T08:52:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98e55e3740 1 -- 172.21.15.82:0/4145797862 wait complete. 2024-04-21T08:52:47.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.879+0000 7f98e55e3740 20 client.17217 trim_cache size 0 max 16384 2024-04-21T08:52:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98e55e3740 1 Processor -- start 2024-04-21T08:52:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98e55e3740 1 -- start start 2024-04-21T08:52:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98e55e3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fccb60 0x55d4d1fe1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98e55e3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98e55e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98e55e3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1af370 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98e55e3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1b0f40 con 0x55d4d1fccb60 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98e55e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1afa40 con 0x55d4d1fd60e0 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fccb60 0x55d4d1fe1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fede90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46564/0 (socket says 172.21.15.82:46564) 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98dbfff640 1 -- 172.21.15.82:0/535944514 learned_addr learned my addr 172.21.15.82:0/535944514 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98d37fe640 1 --2- 172.21.15.82:0/535944514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98dbfff640 1 -- 172.21.15.82:0/535944514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 msgr2=0x55d4d1fee3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98dbfff640 1 --2- 172.21.15.82:0/535944514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fee3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98dbfff640 1 -- 172.21.15.82:0/535944514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fccb60 msgr2=0x55d4d1fe1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98dbfff640 1 --2- 172.21.15.82:0/535944514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fccb60 0x55d4d1fe1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98dbfff640 1 -- 172.21.15.82:0/535944514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4d1fcbe30 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.896+0000 7f98d37fe640 1 --2- 172.21.15.82:0/535944514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fccb60 0x55d4d1fe1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:52:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.897+0000 7f98dbfff640 1 --2- 172.21.15.82:0/535944514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fede90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009a00 tx=0x7f98cc011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5cc9eec22a1a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.897+0000 7f98d1ffb640 1 -- 172.21.15.82:0/535944514 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc02f070 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.897+0000 7f98d1ffb640 1 -- 172.21.15.82:0/535944514 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98cc003e20 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.897+0000 7f98d1ffb640 1 -- 172.21.15.82:0/535944514 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc005110 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.897+0000 7f98e55e3740 1 -- 172.21.15.82:0/535944514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 msgr2=0x55d4d1fede90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.897+0000 7f98e55e3740 1 --2- 172.21.15.82:0/535944514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fede90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009a00 tx=0x7f98cc011900 comp rx=0 tx=0).stop 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 -- 172.21.15.82:0/535944514 shutdown_connections 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 --2- 172.21.15.82:0/535944514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fee3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 --2- 172.21.15.82:0/535944514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fede90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 --2- 172.21.15.82:0/535944514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fccb60 0x55d4d1fe1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 -- 172.21.15.82:0/535944514 >> 172.21.15.82:0/535944514 conn(0x55d4d1f93c90 msgr2=0x55d4d1fd17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 -- 172.21.15.82:0/535944514 shutdown_connections 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 -- 172.21.15.82:0/535944514 wait complete. 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 -- start start 2024-04-21T08:52:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 Processor -- start 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fccb60 0x55d4d1fde3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fd0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1b0870 con 0x55d4d1fccb60 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1b01a0 con 0x55d4d1fcd8d0 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1ad350 con 0x55d4d1fd60e0 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98e55e3740 10 client.0 _ll_register_callbacks cb 0x7f98cc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fccb60 0x55d4d1fde3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98d37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fccb60 0x55d4d1fde3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46580/0 (socket says 172.21.15.82:46580) 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.898+0000 7f98d37fe640 1 -- 172.21.15.82:0/1435420707 learned_addr learned my addr 172.21.15.82:0/1435420707 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98dbfff640 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fd0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98d37fe640 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98d37fe640 1 -- 172.21.15.82:0/1435420707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 msgr2=0x55d4d1fd06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98d37fe640 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98d37fe640 1 -- 172.21.15.82:0/1435420707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fcd8d0 msgr2=0x55d4d1fd0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98d37fe640 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fd0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98d37fe640 1 -- 172.21.15.82:0/1435420707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4d1fd0ce0 con 0x55d4d1fccb60 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98d37fe640 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fccb60 0x55d4d1fde3b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f98c402bc70 tx=0x7f98c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e26993c3793040a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98bb7fe640 1 -- 172.21.15.82:0/1435420707 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c402cba0 con 0x55d4d1fccb60 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98e55e3740 1 -- 172.21.15.82:0/1435420707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98cc003e20 con 0x55d4d1fccb60 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98bb7fe640 1 -- 172.21.15.82:0/1435420707 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f98c405a360 con 0x55d4d1fccb60 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98bb7fe640 1 -- 172.21.15.82:0/1435420707 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f98c405a560 con 0x55d4d1fccb60 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98e55e3740 10 client.17223 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98e55e3740 1 -- 172.21.15.82:0/1435420707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d4d2050b10 con 0x55d4d1fccb60 2024-04-21T08:52:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.899+0000 7f98bb7fe640 1 -- 172.21.15.82:0/1435420707 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98cc003e20 con 0x55d4d1fccb60 2024-04-21T08:52:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.900+0000 7f98bb7fe640 1 -- 172.21.15.82:0/1435420707 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f98c405ace0 con 0x55d4d1fccb60 2024-04-21T08:52:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.900+0000 7f98e55e3740 10 client.17223 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.900+0000 7f98e55e3740 10 client.17223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.900+0000 7f98e55e3740 20 client.17223 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.900+0000 7f98e55e3740 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d4d2050ce0 0x55d4d20530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.900+0000 7f98e55e3740 4 client.17223 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:52:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.900+0000 7f98e55e3740 1 -- 172.21.15.82:0/1435420707 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d4d1ed13d0 con 0x55d4d2050ce0 2024-04-21T08:52:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.900+0000 7f98dbfff640 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d4d2050ce0 0x55d4d20530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.900+0000 7f98dbfff640 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d4d2050ce0 0x55d4d20530d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d20da379b50c44 server_cookie=11ff77736df8f600 in_seq=0 out_seq=0 2024-04-21T08:52:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:47.900+0000 7f98bb7fe640 10 client.17223 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.133+0000 7f98bb7fe640 1 -- 172.21.15.82:0/1435420707 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d4d1ed13d0 con 0x55d4d2050ce0 2024-04-21T08:52:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.133+0000 7f98bb7fe640 10 client.17223 handle_command_reply: tid=0 2024-04-21T08:52:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98e55e3740 1 client.17223 shutdown 2024-04-21T08:52:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98e55e3740 1 -- 172.21.15.82:0/1435420707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fccb60 msgr2=0x55d4d1fde3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98e55e3740 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fccb60 0x55d4d1fde3b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f98c402bc70 tx=0x7f98c4005fd0 comp rx=0 tx=0).stop 2024-04-21T08:52:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98e55e3740 1 -- 172.21.15.82:0/1435420707 shutdown_connections 2024-04-21T08:52:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98e55e3740 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d4d2050ce0 0x55d4d20530d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98bb7fe640 0 client.17223 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98e55e3740 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d4d1fd60e0 0x55d4d1fd06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98e55e3740 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d4d1fcd8d0 0x55d4d1fd0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98e55e3740 1 --2- 172.21.15.82:0/1435420707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4d1fccb60 0x55d4d1fde3b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98e55e3740 1 -- 172.21.15.82:0/1435420707 >> 172.21.15.82:0/1435420707 conn(0x55d4d1f93c90 msgr2=0x55d4d1fcdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98e55e3740 1 -- 172.21.15.82:0/1435420707 shutdown_connections 2024-04-21T08:52:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98e55e3740 1 -- 172.21.15.82:0/1435420707 wait complete. 2024-04-21T08:52:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.134+0000 7f98e55e3740 20 client.17223 trim_cache size 0 max 16384 2024-04-21T08:52:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:52:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:52:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:52:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:52:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.135+0000 7f98e2915640 1 -- 172.21.15.82:0/3548557478 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f98c80a3e40 msgr2=0x7f98c80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.135+0000 7f98e2915640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f98c80a3e40 0x7f98c80a6300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f98d40ce800 tx=0x7f98d40bd040 comp rx=0 tx=0).stop 2024-04-21T08:52:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.135+0000 7f98e2915640 1 -- 172.21.15.82:0/3548557478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98dc19a020 msgr2=0x7f98dc1a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.135+0000 7f98e2915640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98dc19a020 0x7f98dc1a45d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f98d400e970 tx=0x7f98d400ed70 comp rx=0 tx=0).stop 2024-04-21T08:52:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.135+0000 7f98e2915640 1 -- 172.21.15.82:0/3548557478 shutdown_connections 2024-04-21T08:52:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.135+0000 7f98e2915640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98dc19a020 0x7f98dc1a45d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.135+0000 7f98e2915640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98dc106170 0x7f98dc199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.135+0000 7f98e2915640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f98c80a3e40 0x7f98c80a6300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.135+0000 7f98e2915640 1 --2- 172.21.15.82:0/3548557478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98dc1010f0 0x7f98dc199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.135+0000 7f98e2915640 1 -- 172.21.15.82:0/3548557478 >> 172.21.15.82:0/3548557478 conn(0x7f98dc077fd0 msgr2=0x7f98dc0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.135+0000 7f98e2915640 1 -- 172.21.15.82:0/3548557478 shutdown_connections 2024-04-21T08:52:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:52:48.135+0000 7f98e2915640 1 -- 172.21.15.82:0/3548557478 wait complete. 2024-04-21T08:52:48.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:48 smithi134 ceph-mon[31772]: pgmap v176: 129 pgs: 129 active+clean; 755 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 654 op/s 2024-04-21T08:52:48.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:48 smithi082 ceph-mon[27967]: pgmap v176: 129 pgs: 129 active+clean; 755 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 654 op/s 2024-04-21T08:52:48.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:48 smithi106 ceph-mon[31072]: pgmap v176: 129 pgs: 129 active+clean; 755 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 654 op/s 2024-04-21T08:52:49.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:52:49.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:52:49.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.709+0000 7f5d97387640 1 -- 172.21.15.82:0/1832554709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d9010e930 msgr2=0x7f5d9010ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:49.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.709+0000 7f5d97387640 1 --2- 172.21.15.82:0/1832554709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d9010e930 0x7f5d9010ed10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0085a0 tx=0x7f5d8c02dc70 comp rx=0 tx=0).stop 2024-04-21T08:52:49.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.715+0000 7f5d97387640 1 -- 172.21.15.82:0/1832554709 shutdown_connections 2024-04-21T08:52:49.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.715+0000 7f5d97387640 1 --2- 172.21.15.82:0/1832554709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d90109910 0x7f5d901157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.715+0000 7f5d97387640 1 --2- 172.21.15.82:0/1832554709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d90108f50 0x7f5d901093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.715+0000 7f5d97387640 1 --2- 172.21.15.82:0/1832554709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d9010e930 0x7f5d9010ed10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.715+0000 7f5d97387640 1 -- 172.21.15.82:0/1832554709 >> 172.21.15.82:0/1832554709 conn(0x7f5d9006d4d0 msgr2=0x7f5d9006d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:49.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.720+0000 7f5d97387640 1 -- 172.21.15.82:0/1832554709 shutdown_connections 2024-04-21T08:52:49.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.722+0000 7f5d97387640 1 -- 172.21.15.82:0/1832554709 wait complete. 2024-04-21T08:52:49.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.723+0000 7f5d97387640 1 Processor -- start 2024-04-21T08:52:49.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.724+0000 7f5d97387640 1 -- start start 2024-04-21T08:52:49.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.724+0000 7f5d97387640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d90108f50 0x7f5d9019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.724+0000 7f5d97387640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d90109910 0x7f5d9019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.724+0000 7f5d97387640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9010e930 0x7f5d9019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.724+0000 7f5d97387640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901187b0 con 0x7f5d9010e930 2024-04-21T08:52:49.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.724+0000 7f5d97387640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d90118630 con 0x7f5d90109910 2024-04-21T08:52:49.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.724+0000 7f5d97387640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d90118930 con 0x7f5d90108f50 2024-04-21T08:52:49.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.725+0000 7f5d94922640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d90109910 0x7f5d9019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:49.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.725+0000 7f5d94922640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d90109910 0x7f5d9019eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:49766/0 (socket says 172.21.15.82:49766) 2024-04-21T08:52:49.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.725+0000 7f5d94922640 1 -- 172.21.15.82:0/1242029455 learned_addr learned my addr 172.21.15.82:0/1242029455 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:49.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.727+0000 7f5d94922640 1 -- 172.21.15.82:0/1242029455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d90108f50 msgr2=0x7f5d9019eab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:52:49.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.727+0000 7f5d94922640 1 --2- 172.21.15.82:0/1242029455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d90108f50 0x7f5d9019eab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.727+0000 7f5d94922640 1 -- 172.21.15.82:0/1242029455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9010e930 msgr2=0x7f5d9019f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:49.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.727+0000 7f5d94922640 1 --2- 172.21.15.82:0/1242029455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9010e930 0x7f5d9019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.727+0000 7f5d94922640 1 -- 172.21.15.82:0/1242029455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d901a2ef0 con 0x7f5d90109910 2024-04-21T08:52:49.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.730+0000 7f5d94922640 1 --2- 172.21.15.82:0/1242029455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d90109910 0x7f5d9019eff0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800cce0 tx=0x7f5d8800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:49.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.732+0000 7f5d867fc640 1 -- 172.21.15.82:0/1242029455 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88004b20 con 0x7f5d90109910 2024-04-21T08:52:49.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.732+0000 7f5d867fc640 1 -- 172.21.15.82:0/1242029455 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d88004cc0 con 0x7f5d90109910 2024-04-21T08:52:49.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.732+0000 7f5d867fc640 1 -- 172.21.15.82:0/1242029455 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8800f740 con 0x7f5d90109910 2024-04-21T08:52:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.733+0000 7f5d97387640 1 -- 172.21.15.82:0/1242029455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d901a31e0 con 0x7f5d90109910 2024-04-21T08:52:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.733+0000 7f5d97387640 1 -- 172.21.15.82:0/1242029455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d901a3730 con 0x7f5d90109910 2024-04-21T08:52:49.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.740+0000 7f5d99f70740 1 Processor -- start 2024-04-21T08:52:49.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.740+0000 7f5d867fc640 1 -- 172.21.15.82:0/1242029455 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d8800f8e0 con 0x7f5d90109910 2024-04-21T08:52:49.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.741+0000 7f5d99f70740 1 -- start start 2024-04-21T08:52:49.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.741+0000 7f5d99f70740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90f580 0x55d2ab90f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.741+0000 7f5d99f70740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 0x55d2ab918540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.741+0000 7f5d95123640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 0x55d2ab918540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:49.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.741+0000 7f5d99f70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab918a80 0x55d2ab91ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.741+0000 7f5d99f70740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d2ab8741e0 con 0x55d2ab90f580 2024-04-21T08:52:49.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.741+0000 7f5d99f70740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d2ab811390 con 0x55d2ab90fea0 2024-04-21T08:52:49.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.741+0000 7f5d99f70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d2ab8701a0 con 0x55d2ab918a80 2024-04-21T08:52:49.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.741+0000 7f5d95123640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 0x55d2ab918540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:49780/0 (socket says 172.21.15.82:49780) 2024-04-21T08:52:49.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.741+0000 7f5d95123640 1 -- 172.21.15.82:0/3914582158 learned_addr learned my addr 172.21.15.82:0/3914582158 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:49.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.742+0000 7f5d95924640 1 --2- 172.21.15.82:0/3914582158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90f580 0x55d2ab90f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:49.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.743+0000 7f5d95123640 1 -- 172.21.15.82:0/3914582158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab918a80 msgr2=0x55d2ab91ae70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:52:49.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.743+0000 7f5d95123640 1 --2- 172.21.15.82:0/3914582158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab918a80 0x55d2ab91ae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.743+0000 7f5d95123640 1 -- 172.21.15.82:0/3914582158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90f580 msgr2=0x55d2ab90f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:49.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.743+0000 7f5d95123640 1 --2- 172.21.15.82:0/3914582158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90f580 0x55d2ab90f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.743+0000 7f5d867fc640 1 --2- 172.21.15.82:0/1242029455 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5d7c0a3f20 0x7f5d7c0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.743+0000 7f5d95924640 1 --2- 172.21.15.82:0/1242029455 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5d7c0a3f20 0x7f5d7c0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:49.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.744+0000 7f5d95924640 1 --2- 172.21.15.82:0/1242029455 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5d7c0a3f20 0x7f5d7c0a63e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d80005fd0 tx=0x7f5d80005de0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:49.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.744+0000 7f5d867fc640 1 -- 172.21.15.82:0/1242029455 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f5d880d1c00 con 0x7f5d90109910 2024-04-21T08:52:49.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.744+0000 7f5d95123640 1 -- 172.21.15.82:0/3914582158 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ab6507d0 con 0x55d2ab90fea0 2024-04-21T08:52:49.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.754+0000 7f5d95123640 1 --2- 172.21.15.82:0/3914582158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 0x55d2ab918540 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c02e280 tx=0x7f5d8c00ae70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d0da886e35cca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:49.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.756+0000 7f5d7b7fe640 1 -- 172.21.15.82:0/3914582158 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c02ee60 con 0x55d2ab90fea0 2024-04-21T08:52:49.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.756+0000 7f5d7b7fe640 1 -- 172.21.15.82:0/3914582158 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c00a0d0 con 0x55d2ab90fea0 2024-04-21T08:52:49.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.756+0000 7f5d7b7fe640 1 -- 172.21.15.82:0/3914582158 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c036660 con 0x55d2ab90fea0 2024-04-21T08:52:49.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.757+0000 7f5d99f70740 1 -- 172.21.15.82:0/3914582158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 msgr2=0x55d2ab918540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:49.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.757+0000 7f5d99f70740 1 --2- 172.21.15.82:0/3914582158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 0x55d2ab918540 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c02e280 tx=0x7f5d8c00ae70 comp rx=0 tx=0).stop 2024-04-21T08:52:49.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.760+0000 7f5d99f70740 1 -- 172.21.15.82:0/3914582158 shutdown_connections 2024-04-21T08:52:49.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.760+0000 7f5d99f70740 1 --2- 172.21.15.82:0/3914582158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab918a80 0x55d2ab91ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.760+0000 7f5d99f70740 1 --2- 172.21.15.82:0/3914582158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 0x55d2ab918540 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.760+0000 7f5d99f70740 1 --2- 172.21.15.82:0/3914582158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90f580 0x55d2ab90f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.760+0000 7f5d99f70740 1 -- 172.21.15.82:0/3914582158 >> 172.21.15.82:0/3914582158 conn(0x55d2ab909cb0 msgr2=0x55d2ab90a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:49.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.762+0000 7f5d99f70740 1 -- 172.21.15.82:0/3914582158 shutdown_connections 2024-04-21T08:52:49.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.766+0000 7f5d99f70740 1 -- 172.21.15.82:0/3914582158 wait complete. 2024-04-21T08:52:49.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.766+0000 7f5d99f70740 1 -- start start 2024-04-21T08:52:49.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.766+0000 7f5d99f70740 1 Processor -- start 2024-04-21T08:52:49.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.769+0000 7f5d99f70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 0x55d2ab915ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.769+0000 7f5d99f70740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 0x55d2ab916010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.769+0000 7f5d99f70740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab918a80 0x55d2ab916550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.769+0000 7f5d99f70740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d2ab91b7b0 con 0x55d2ab918a80 2024-04-21T08:52:49.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.769+0000 7f5d99f70740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d2ab82d2f0 con 0x55d2ab90fea0 2024-04-21T08:52:49.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.769+0000 7f5d99f70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d2ab91b930 con 0x55d2ab90f580 2024-04-21T08:52:49.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.770+0000 7f5d99f70740 10 client.0 _ll_register_callbacks cb 0x55d2ab83a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:49.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.771+0000 7f5d95123640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 0x55d2ab915ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:49.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.771+0000 7f5d95123640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 0x55d2ab915ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:56244/0 (socket says 172.21.15.82:56244) 2024-04-21T08:52:49.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.771+0000 7f5d95123640 1 -- 172.21.15.82:0/3719650381 learned_addr learned my addr 172.21.15.82:0/3719650381 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.771+0000 7f5d95123640 1 -- 172.21.15.82:0/3719650381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 msgr2=0x55d2ab916010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.771+0000 7f5d94922640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 0x55d2ab916010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.771+0000 7f5d95123640 1 --2- 172.21.15.82:0/3719650381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 0x55d2ab916010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.771+0000 7f5d95123640 1 -- 172.21.15.82:0/3719650381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab918a80 msgr2=0x55d2ab916550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.771+0000 7f5d95123640 1 --2- 172.21.15.82:0/3719650381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab918a80 0x55d2ab916550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.771+0000 7f5d95123640 1 -- 172.21.15.82:0/3719650381 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2ab5863f0 con 0x55d2ab90f580 2024-04-21T08:52:49.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.772+0000 7f5d94922640 1 --2- 172.21.15.82:0/3719650381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 0x55d2ab916010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:49.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.772+0000 7f5d95123640 1 --2- 172.21.15.82:0/3719650381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 0x55d2ab915ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c004270 tx=0x7f5d8c0042a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca60cf1a89f9cd3d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:49.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.772+0000 7f5d78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:52:49.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.784+0000 7f5d78ff9640 1 -- 172.21.15.82:0/3719650381 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00a480 con 0x55d2ab90f580 2024-04-21T08:52:49.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.784+0000 7f5d78ff9640 1 -- 172.21.15.82:0/3719650381 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f5d8c00a620 con 0x55d2ab90f580 2024-04-21T08:52:49.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.787+0000 7f5d99f70740 1 -- 172.21.15.82:0/3719650381 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d8c00a0d0 con 0x55d2ab90f580 2024-04-21T08:52:49.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.797+0000 7f5d78ff9640 1 -- 172.21.15.82:0/3719650381 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5d8c00a820 con 0x55d2ab90f580 2024-04-21T08:52:49.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.800+0000 7f5d99f70740 10 client.26320 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:49.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.800+0000 7f5d99f70740 1 -- 172.21.15.82:0/3719650381 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d2ab5460f0 con 0x55d2ab90f580 2024-04-21T08:52:49.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.805+0000 7f5d78ff9640 1 -- 172.21.15.82:0/3719650381 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c00a0d0 con 0x55d2ab90f580 2024-04-21T08:52:49.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.805+0000 7f5d78ff9640 1 -- 172.21.15.82:0/3719650381 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f5d8c0476f0 con 0x55d2ab90f580 2024-04-21T08:52:49.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.805+0000 7f5d99f70740 10 client.26320 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:49.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.805+0000 7f5d99f70740 10 client.26320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:49.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.805+0000 7f5d99f70740 20 client.26320 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:49.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.805+0000 7f5d99f70740 1 --2- 172.21.15.82:0/3719650381 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2ab928b50 0x55d2ab92afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.805+0000 7f5d99f70740 4 client.26320 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:52:49.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.805+0000 7f5d99f70740 1 -- 172.21.15.82:0/3719650381 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2ab881c00 con 0x55d2ab928b50 2024-04-21T08:52:49.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.806+0000 7f5d94922640 1 --2- 172.21.15.82:0/3719650381 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2ab928b50 0x55d2ab92afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:49.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.809+0000 7f5d94922640 1 --2- 172.21.15.82:0/3719650381 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2ab928b50 0x55d2ab92afd0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16cefa2081d347f1 server_cookie=d7539e6f08420672 in_seq=0 out_seq=0 2024-04-21T08:52:49.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.810+0000 7f5d78ff9640 10 client.26320 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:49.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.816+0000 7f5d78ff9640 1 -- 172.21.15.82:0/3719650381 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2ab881c00 con 0x55d2ab928b50 2024-04-21T08:52:49.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.816+0000 7f5d78ff9640 10 client.26320 handle_command_reply: tid=0 2024-04-21T08:52:49.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.816+0000 7f5d99f70740 1 client.26320 shutdown 2024-04-21T08:52:49.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.824+0000 7f5d99f70740 1 -- 172.21.15.82:0/3719650381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 msgr2=0x55d2ab915ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:49.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.824+0000 7f5d99f70740 1 --2- 172.21.15.82:0/3719650381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 0x55d2ab915ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c004270 tx=0x7f5d8c0042a0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.830+0000 7f5d99f70740 1 -- 172.21.15.82:0/3719650381 shutdown_connections 2024-04-21T08:52:49.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.830+0000 7f5d99f70740 1 --2- 172.21.15.82:0/3719650381 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2ab928b50 0x55d2ab92afd0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.830+0000 7f5d99f70740 1 --2- 172.21.15.82:0/3719650381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab918a80 0x55d2ab916550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.830+0000 7f5d99f70740 1 --2- 172.21.15.82:0/3719650381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90fea0 0x55d2ab916010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.830+0000 7f5d99f70740 1 --2- 172.21.15.82:0/3719650381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 0x55d2ab915ad0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.830+0000 7f5d99f70740 1 -- 172.21.15.82:0/3719650381 >> 172.21.15.82:0/3719650381 conn(0x55d2ab909cb0 msgr2=0x55d2ab90ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:49.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.830+0000 7f5d78ff9640 0 client.26320 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:49.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.837+0000 7f5d99f70740 1 -- 172.21.15.82:0/3719650381 shutdown_connections 2024-04-21T08:52:49.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.838+0000 7f5d99f70740 1 -- 172.21.15.82:0/3719650381 wait complete. 2024-04-21T08:52:49.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.840+0000 7f5d99f70740 20 client.26320 trim_cache size 0 max 16384 2024-04-21T08:52:49.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.878+0000 7f5d99f70740 1 Processor -- start 2024-04-21T08:52:49.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.878+0000 7f5d99f70740 1 -- start start 2024-04-21T08:52:49.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.879+0000 7f5d99f70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 0x55d2ab918200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.879+0000 7f5d99f70740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90fea0 0x55d2ab9246d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.879+0000 7f5d99f70740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab918a80 0x55d2ab916370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.879+0000 7f5d99f70740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901c6af0 con 0x55d2ab90fea0 2024-04-21T08:52:49.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.879+0000 7f5d99f70740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901c86c0 con 0x55d2ab918a80 2024-04-21T08:52:49.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.879+0000 7f5d99f70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901c71c0 con 0x55d2ab90f580 2024-04-21T08:52:49.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.879+0000 7f5d95924640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab918a80 0x55d2ab916370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:49.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.879+0000 7f5d95123640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 0x55d2ab918200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:49.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.879+0000 7f5d95123640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 0x55d2ab918200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:56246/0 (socket says 172.21.15.82:56246) 2024-04-21T08:52:49.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.879+0000 7f5d95123640 1 -- 172.21.15.82:0/546112120 learned_addr learned my addr 172.21.15.82:0/546112120 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.882+0000 7f5d95123640 1 -- 172.21.15.82:0/546112120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab918a80 msgr2=0x55d2ab916370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.882+0000 7f5d95123640 1 --2- 172.21.15.82:0/546112120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab918a80 0x55d2ab916370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.882+0000 7f5d95123640 1 -- 172.21.15.82:0/546112120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90fea0 msgr2=0x55d2ab9246d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:52:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.882+0000 7f5d95123640 1 --2- 172.21.15.82:0/546112120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90fea0 0x55d2ab9246d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.882+0000 7f5d95123640 1 -- 172.21.15.82:0/546112120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ab90fcd0 con 0x55d2ab90f580 2024-04-21T08:52:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.884+0000 7f5d95123640 1 --2- 172.21.15.82:0/546112120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 0x55d2ab918200 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c000c00 tx=0x7f5d8c005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c50fa757792b806b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.885+0000 7f5d7affd640 1 -- 172.21.15.82:0/546112120 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c051070 con 0x55d2ab90f580 2024-04-21T08:52:49.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.885+0000 7f5d7affd640 1 -- 172.21.15.82:0/546112120 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c0478c0 con 0x55d2ab90f580 2024-04-21T08:52:49.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.885+0000 7f5d7affd640 1 -- 172.21.15.82:0/546112120 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0035f0 con 0x55d2ab90f580 2024-04-21T08:52:49.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.886+0000 7f5d99f70740 1 -- 172.21.15.82:0/546112120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 msgr2=0x55d2ab918200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:49.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.886+0000 7f5d99f70740 1 --2- 172.21.15.82:0/546112120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 0x55d2ab918200 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c000c00 tx=0x7f5d8c005b50 comp rx=0 tx=0).stop 2024-04-21T08:52:49.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.889+0000 7f5d99f70740 1 -- 172.21.15.82:0/546112120 shutdown_connections 2024-04-21T08:52:49.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.889+0000 7f5d99f70740 1 --2- 172.21.15.82:0/546112120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab918a80 0x55d2ab916370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.889+0000 7f5d99f70740 1 --2- 172.21.15.82:0/546112120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90fea0 0x55d2ab9246d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.889+0000 7f5d99f70740 1 --2- 172.21.15.82:0/546112120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab90f580 0x55d2ab918200 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.889+0000 7f5d99f70740 1 -- 172.21.15.82:0/546112120 >> 172.21.15.82:0/546112120 conn(0x55d2ab909cb0 msgr2=0x55d2ab9143a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:49.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.893+0000 7f5d99f70740 1 -- 172.21.15.82:0/546112120 shutdown_connections 2024-04-21T08:52:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.894+0000 7f5d99f70740 1 -- 172.21.15.82:0/546112120 wait complete. 2024-04-21T08:52:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.894+0000 7f5d99f70740 1 -- start start 2024-04-21T08:52:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.895+0000 7f5d99f70740 1 Processor -- start 2024-04-21T08:52:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.895+0000 7f5d99f70740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90f580 0x55d2ab9210b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.895+0000 7f5d99f70740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90fea0 0x55d2ab913100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.895+0000 7f5d99f70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab918a80 0x55d2ab913710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.895+0000 7f5d99f70740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901c7ff0 con 0x55d2ab90fea0 2024-04-21T08:52:49.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.895+0000 7f5d99f70740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901c7920 con 0x55d2ab90f580 2024-04-21T08:52:49.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.895+0000 7f5d99f70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901c4ad0 con 0x55d2ab918a80 2024-04-21T08:52:49.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.895+0000 7f5d99f70740 10 client.0 _ll_register_callbacks cb 0x7f5d8c00ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:52:49.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.897+0000 7f5d94922640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90fea0 0x55d2ab913100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:49.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.897+0000 7f5d94922640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90fea0 0x55d2ab913100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46626/0 (socket says 172.21.15.82:46626) 2024-04-21T08:52:49.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.897+0000 7f5d94922640 1 -- 172.21.15.82:0/1210598737 learned_addr learned my addr 172.21.15.82:0/1210598737 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:52:49.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.899+0000 7f5d94922640 1 -- 172.21.15.82:0/1210598737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab918a80 msgr2=0x55d2ab913710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:52:49.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.899+0000 7f5d94922640 1 --2- 172.21.15.82:0/1210598737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab918a80 0x55d2ab913710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.899+0000 7f5d94922640 1 -- 172.21.15.82:0/1210598737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90f580 msgr2=0x55d2ab9210b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:52:49.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.899+0000 7f5d94922640 1 --2- 172.21.15.82:0/1210598737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90f580 0x55d2ab9210b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.899+0000 7f5d94922640 1 -- 172.21.15.82:0/1210598737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2ab911f10 con 0x55d2ab90fea0 2024-04-21T08:52:49.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.900+0000 7f5d94922640 1 --2- 172.21.15.82:0/1210598737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90fea0 0x55d2ab913100 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5d88099be0 tx=0x7f5d8800ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f666c097d2289 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:49.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.900+0000 7f5d78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:52:49.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.900+0000 7f5d78ff9640 1 -- 172.21.15.82:0/1210598737 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8801d040 con 0x55d2ab90fea0 2024-04-21T08:52:49.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.900+0000 7f5d99f70740 1 -- 172.21.15.82:0/1210598737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d8c0478c0 con 0x55d2ab90fea0 2024-04-21T08:52:49.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.900+0000 7f5d78ff9640 1 -- 172.21.15.82:0/1210598737 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f5d88099df0 con 0x55d2ab90fea0 2024-04-21T08:52:49.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.901+0000 7f5d78ff9640 1 -- 172.21.15.82:0/1210598737 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5d880a1b30 con 0x55d2ab90fea0 2024-04-21T08:52:49.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.903+0000 7f5d99f70740 10 client.17229 fetch_fsmap learned FSMap version 36 2024-04-21T08:52:49.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.903+0000 7f5d99f70740 1 -- 172.21.15.82:0/1210598737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d2ab98e680 con 0x55d2ab90fea0 2024-04-21T08:52:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.905+0000 7f5d78ff9640 1 -- 172.21.15.82:0/1210598737 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c0478c0 con 0x55d2ab90fea0 2024-04-21T08:52:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.905+0000 7f5d78ff9640 1 -- 172.21.15.82:0/1210598737 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f5d88099040 con 0x55d2ab90fea0 2024-04-21T08:52:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.906+0000 7f5d99f70740 10 client.17229 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:52:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.906+0000 7f5d99f70740 10 client.17229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:52:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.906+0000 7f5d99f70740 20 client.17229 populate_metadata read hostname 'smithi082' 2024-04-21T08:52:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.906+0000 7f5d99f70740 1 --2- 172.21.15.82:0/1210598737 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2ab98ede0 0x55d2ab991200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.906+0000 7f5d99f70740 4 client.17229 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:52:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.906+0000 7f5d99f70740 1 -- 172.21.15.82:0/1210598737 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d2ab87ba70 con 0x55d2ab98ede0 2024-04-21T08:52:49.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.912+0000 7f5d95123640 1 --2- 172.21.15.82:0/1210598737 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2ab98ede0 0x55d2ab991200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:49.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.913+0000 7f5d95123640 1 --2- 172.21.15.82:0/1210598737 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2ab98ede0 0x55d2ab991200 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76bd07680d701e1a server_cookie=f5a771c5b1a3035a in_seq=0 out_seq=0 2024-04-21T08:52:49.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.913+0000 7f5d78ff9640 10 client.17229 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:49.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.913+0000 7f5d78ff9640 1 -- 172.21.15.82:0/1210598737 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d2ab87ba70 con 0x55d2ab98ede0 2024-04-21T08:52:49.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.913+0000 7f5d78ff9640 10 client.17229 handle_command_reply: tid=0 2024-04-21T08:52:49.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.913+0000 7f5d99f70740 1 client.17229 shutdown 2024-04-21T08:52:49.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.922+0000 7f5d99f70740 1 -- 172.21.15.82:0/1210598737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90fea0 msgr2=0x55d2ab913100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:49.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.922+0000 7f5d99f70740 1 --2- 172.21.15.82:0/1210598737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90fea0 0x55d2ab913100 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5d88099be0 tx=0x7f5d8800ac50 comp rx=0 tx=0).stop 2024-04-21T08:52:49.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.924+0000 7f5d99f70740 1 -- 172.21.15.82:0/1210598737 shutdown_connections 2024-04-21T08:52:49.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.924+0000 7f5d99f70740 1 --2- 172.21.15.82:0/1210598737 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2ab98ede0 0x55d2ab991200 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.924+0000 7f5d99f70740 1 --2- 172.21.15.82:0/1210598737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2ab918a80 0x55d2ab913710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.924+0000 7f5d99f70740 1 --2- 172.21.15.82:0/1210598737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2ab90fea0 0x55d2ab913100 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.924+0000 7f5d99f70740 1 --2- 172.21.15.82:0/1210598737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ab90f580 0x55d2ab9210b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.924+0000 7f5d99f70740 1 -- 172.21.15.82:0/1210598737 >> 172.21.15.82:0/1210598737 conn(0x55d2ab909cb0 msgr2=0x55d2ab9084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:49.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.925+0000 7f5d78ff9640 0 client.17229 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:52:49.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.931+0000 7f5d99f70740 1 -- 172.21.15.82:0/1210598737 shutdown_connections 2024-04-21T08:52:49.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.933+0000 7f5d99f70740 1 -- 172.21.15.82:0/1210598737 wait complete. 2024-04-21T08:52:49.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.934+0000 7f5d99f70740 20 client.17229 trim_cache size 0 max 16384 2024-04-21T08:52:49.938 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:52:49.938 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:52:49.938 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:52:49.938 INFO:teuthology.orchestra.run.smithi082.stdout: "64552535-4733-45da-b784-7a1f2b81d8dd": { 2024-04-21T08:52:49.938 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:52:49.939 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "64552535-4733-45da-b784-7a1f2b81d8dd", 2024-04-21T08:52:49.939 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:52:49.939 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:52:49.939 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:52:49.939 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:52:49.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.939+0000 7f5d97387640 1 -- 172.21.15.82:0/1242029455 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5d7c0a3f20 msgr2=0x7f5d7c0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:49.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.939+0000 7f5d97387640 1 --2- 172.21.15.82:0/1242029455 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5d7c0a3f20 0x7f5d7c0a63e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d80005fd0 tx=0x7f5d80005de0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.939+0000 7f5d97387640 1 -- 172.21.15.82:0/1242029455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d90109910 msgr2=0x7f5d9019eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:49.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.939+0000 7f5d97387640 1 --2- 172.21.15.82:0/1242029455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d90109910 0x7f5d9019eff0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800cce0 tx=0x7f5d8800a620 comp rx=0 tx=0).stop 2024-04-21T08:52:49.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.943+0000 7f5d97387640 1 -- 172.21.15.82:0/1242029455 shutdown_connections 2024-04-21T08:52:49.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.943+0000 7f5d97387640 1 --2- 172.21.15.82:0/1242029455 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5d7c0a3f20 0x7f5d7c0a63e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.943+0000 7f5d97387640 1 --2- 172.21.15.82:0/1242029455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9010e930 0x7f5d9019f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.943+0000 7f5d97387640 1 --2- 172.21.15.82:0/1242029455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d90109910 0x7f5d9019eff0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.943+0000 7f5d97387640 1 --2- 172.21.15.82:0/1242029455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d90108f50 0x7f5d9019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:49.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.943+0000 7f5d97387640 1 -- 172.21.15.82:0/1242029455 >> 172.21.15.82:0/1242029455 conn(0x7f5d9006d4d0 msgr2=0x7f5d90072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:49.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.946+0000 7f5d97387640 1 -- 172.21.15.82:0/1242029455 shutdown_connections 2024-04-21T08:52:49.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:52:49.947+0000 7f5d97387640 1 -- 172.21.15.82:0/1242029455 wait complete. 2024-04-21T08:52:49.966 INFO:tasks.cephfs.filesystem:scrub status for tag:64552535-4733-45da-b784-7a1f2b81d8dd - {'path': '/', 'tag': '64552535-4733-45da-b784-7a1f2b81d8dd', 'options': 'recursive,force'} 2024-04-21T08:52:50.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:50 smithi082 ceph-mon[27967]: pgmap v177: 129 pgs: 129 active+clean; 757 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 672 op/s 2024-04-21T08:52:50.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:50 smithi134 ceph-mon[31772]: pgmap v177: 129 pgs: 129 active+clean; 757 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 672 op/s 2024-04-21T08:52:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:50 smithi106 ceph-mon[31072]: pgmap v177: 129 pgs: 129 active+clean; 757 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 672 op/s 2024-04-21T08:52:52.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:52 smithi134 ceph-mon[31772]: pgmap v178: 129 pgs: 129 active+clean; 757 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 500 op/s 2024-04-21T08:52:52.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:52 smithi082 ceph-mon[27967]: pgmap v178: 129 pgs: 129 active+clean; 757 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 500 op/s 2024-04-21T08:52:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:52 smithi106 ceph-mon[31072]: pgmap v178: 129 pgs: 129 active+clean; 757 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 500 op/s 2024-04-21T08:52:54.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:54 smithi134 ceph-mon[31772]: pgmap v179: 129 pgs: 129 active+clean; 815 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.6 MiB/s wr, 957 op/s 2024-04-21T08:52:54.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:54 smithi082 ceph-mon[27967]: pgmap v179: 129 pgs: 129 active+clean; 815 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.6 MiB/s wr, 957 op/s 2024-04-21T08:52:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:54 smithi106 ceph-mon[31072]: pgmap v179: 129 pgs: 129 active+clean; 815 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.6 MiB/s wr, 957 op/s 2024-04-21T08:52:56.231 INFO:tasks.workunit.client.0.smithi082.stdout: 138 1044786 296 731885 198 594011 755 2024-04-21T08:52:56.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:56 smithi134 ceph-mon[31772]: pgmap v180: 129 pgs: 129 active+clean; 815 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.5 MiB/s wr, 788 op/s 2024-04-21T08:52:56.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:56 smithi082 ceph-mon[27967]: pgmap v180: 129 pgs: 129 active+clean; 815 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.5 MiB/s wr, 788 op/s 2024-04-21T08:52:56.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:56 smithi106 ceph-mon[31072]: pgmap v180: 129 pgs: 129 active+clean; 815 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.5 MiB/s wr, 788 op/s 2024-04-21T08:52:58.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:52:58 smithi134 ceph-mon[31772]: pgmap v181: 129 pgs: 129 active+clean; 823 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 961 op/s 2024-04-21T08:52:58.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:52:58 smithi082 ceph-mon[27967]: pgmap v181: 129 pgs: 129 active+clean; 823 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 961 op/s 2024-04-21T08:52:58.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:52:58 smithi106 ceph-mon[31072]: pgmap v181: 129 pgs: 129 active+clean; 823 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 961 op/s 2024-04-21T08:53:00.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:00 smithi134 ceph-mon[31772]: pgmap v182: 129 pgs: 129 active+clean; 828 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.0 MiB/s wr, 1.03k op/s 2024-04-21T08:53:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:00 smithi082 ceph-mon[27967]: pgmap v182: 129 pgs: 129 active+clean; 828 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.0 MiB/s wr, 1.03k op/s 2024-04-21T08:53:00.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:00 smithi106 ceph-mon[31072]: pgmap v182: 129 pgs: 129 active+clean; 828 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.0 MiB/s wr, 1.03k op/s 2024-04-21T08:53:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:02 smithi134 ceph-mon[31772]: pgmap v183: 129 pgs: 129 active+clean; 828 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 836 op/s 2024-04-21T08:53:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.529+0000 7f6657058640 1 -- 172.21.15.82:0/2702217459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f665006e240 msgr2=0x7f6650108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.529+0000 7f6657058640 1 --2- 172.21.15.82:0/2702217459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f665006e240 0x7f6650108f50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6640009ca0 tx=0x7f664002c120 comp rx=0 tx=0).stop 2024-04-21T08:53:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.537+0000 7f6657058640 1 -- 172.21.15.82:0/2702217459 shutdown_connections 2024-04-21T08:53:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.537+0000 7f6657058640 1 --2- 172.21.15.82:0/2702217459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6650114c90 0x7f6650117080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.537+0000 7f6657058640 1 --2- 172.21.15.82:0/2702217459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6650109520 0x7f66501099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.537+0000 7f6657058640 1 --2- 172.21.15.82:0/2702217459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f665006e240 0x7f6650108f50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.537+0000 7f6657058640 1 -- 172.21.15.82:0/2702217459 >> 172.21.15.82:0/2702217459 conn(0x7f665006ce90 msgr2=0x7f665006d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.543+0000 7f6657058640 1 -- 172.21.15.82:0/2702217459 shutdown_connections 2024-04-21T08:53:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.543+0000 7f6657058640 1 -- 172.21.15.82:0/2702217459 wait complete. 2024-04-21T08:53:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.544+0000 7f6657058640 1 Processor -- start 2024-04-21T08:53:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.544+0000 7f6657058640 1 -- start start 2024-04-21T08:53:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.544+0000 7f6657058640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f665006e240 0x7f66501180a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.544+0000 7f6657058640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6650109520 0x7f66501185e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.544+0000 7f6657058640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650114c90 0x7f6650118b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.544+0000 7f6657058640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f665011cae0 con 0x7f6650109520 2024-04-21T08:53:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.544+0000 7f6657058640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f665011c960 con 0x7f6650114c90 2024-04-21T08:53:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.544+0000 7f6657058640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f665011cc60 con 0x7f665006e240 2024-04-21T08:53:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.544+0000 7f6656056640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f665006e240 0x7f66501180a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.544+0000 7f6656857640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650114c90 0x7f6650118b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.544+0000 7f6656056640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f665006e240 0x7f66501180a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:57290/0 (socket says 172.21.15.82:57290) 2024-04-21T08:53:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.544+0000 7f6656056640 1 -- 172.21.15.82:0/461938896 learned_addr learned my addr 172.21.15.82:0/461938896 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.545+0000 7f6656056640 1 -- 172.21.15.82:0/461938896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650114c90 msgr2=0x7f6650118b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.545+0000 7f6656056640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650114c90 0x7f6650118b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.545+0000 7f6656056640 1 -- 172.21.15.82:0/461938896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6650109520 msgr2=0x7f66501185e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:53:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.545+0000 7f6655855640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6650109520 0x7f66501185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.545+0000 7f6656056640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6650109520 0x7f66501185e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.545+0000 7f6656056640 1 -- 172.21.15.82:0/461938896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66501a2f10 con 0x7f665006e240 2024-04-21T08:53:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.545+0000 7f6655855640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6650109520 0x7f66501185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:53:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.546+0000 7f6656056640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f665006e240 0x7f66501180a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f664002c6b0 tx=0x7f6640035ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.556+0000 7f664f7fe640 1 -- 172.21.15.82:0/461938896 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640039040 con 0x7f665006e240 2024-04-21T08:53:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.556+0000 7f664f7fe640 1 -- 172.21.15.82:0/461938896 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6640044070 con 0x7f665006e240 2024-04-21T08:53:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.556+0000 7f664f7fe640 1 -- 172.21.15.82:0/461938896 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66400043c0 con 0x7f665006e240 2024-04-21T08:53:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.556+0000 7f6657058640 1 -- 172.21.15.82:0/461938896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66501a31a0 con 0x7f665006e240 2024-04-21T08:53:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.556+0000 7f6657058640 1 -- 172.21.15.82:0/461938896 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66501a3690 con 0x7f665006e240 2024-04-21T08:53:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.561+0000 7f664f7fe640 1 -- 172.21.15.82:0/461938896 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6640004560 con 0x7f665006e240 2024-04-21T08:53:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.563+0000 7f665af83740 1 Processor -- start 2024-04-21T08:53:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.563+0000 7f665af83740 1 -- start start 2024-04-21T08:53:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.564+0000 7f665af83740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 0x56374164f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.564+0000 7f6656857640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 0x56374164f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.564+0000 7f6656857640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 0x56374164f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35158/0 (socket says 172.21.15.82:35158) 2024-04-21T08:53:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.565+0000 7f6656857640 1 -- 172.21.15.82:0/3909934014 learned_addr learned my addr 172.21.15.82:0/3909934014 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.565+0000 7f665af83740 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56374164fd50 0x5637416501d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.565+0000 7f665af83740 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637416592d0 0x56374165b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.565+0000 7f6656056640 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56374164fd50 0x5637416501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.565+0000 7f665af83740 1 -- 172.21.15.82:0/3909934014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5637415b48d0 con 0x563741650ac0 2024-04-21T08:53:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.565+0000 7f665af83740 1 -- 172.21.15.82:0/3909934014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563741551e80 con 0x56374164fd50 2024-04-21T08:53:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.565+0000 7f665af83740 1 -- 172.21.15.82:0/3909934014 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5637415b0890 con 0x5637416592d0 2024-04-21T08:53:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.565+0000 7f6655855640 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637416592d0 0x56374165b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.566+0000 7f6656857640 1 -- 172.21.15.82:0/3909934014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637416592d0 msgr2=0x56374165b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.566+0000 7f6656857640 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637416592d0 0x56374165b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.566+0000 7f6656857640 1 -- 172.21.15.82:0/3909934014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56374164fd50 msgr2=0x5637416501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.566+0000 7f6656857640 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56374164fd50 0x5637416501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.566+0000 7f6656857640 1 -- 172.21.15.82:0/3909934014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563741573320 con 0x563741650ac0 2024-04-21T08:53:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.566+0000 7f6656857640 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 0x56374164f810 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f664800b6b0 tx=0x7f664800bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305cf2f40b31cb70 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.567+0000 7f6656056640 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56374164fd50 0x5637416501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:53:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.568+0000 7f6655855640 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637416592d0 0x56374165b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:53:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.569+0000 7f664f7fe640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66380a3f20 0x7f66380a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.569+0000 7f6656056640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66380a3f20 0x7f66380a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.570+0000 7f6656056640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66380a3f20 0x7f66380a63e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x563741651220 tx=0x7f66400f8040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.570+0000 7f664f7fe640 1 -- 172.21.15.82:0/461938896 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f66400f38d0 con 0x7f665006e240 2024-04-21T08:53:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.587+0000 7f664cbfe640 1 -- 172.21.15.82:0/3909934014 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648003bd0 con 0x563741650ac0 2024-04-21T08:53:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.587+0000 7f664cbfe640 1 -- 172.21.15.82:0/3909934014 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6648015070 con 0x563741650ac0 2024-04-21T08:53:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.588+0000 7f665af83740 1 -- 172.21.15.82:0/3909934014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 msgr2=0x56374164f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.588+0000 7f665af83740 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 0x56374164f810 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f664800b6b0 tx=0x7f664800bb70 comp rx=0 tx=0).stop 2024-04-21T08:53:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.590+0000 7f665af83740 1 -- 172.21.15.82:0/3909934014 shutdown_connections 2024-04-21T08:53:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.590+0000 7f665af83740 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637416592d0 0x56374165b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.590+0000 7f665af83740 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56374164fd50 0x5637416501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.590+0000 7f665af83740 1 --2- 172.21.15.82:0/3909934014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 0x56374164f810 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.590+0000 7f665af83740 1 -- 172.21.15.82:0/3909934014 >> 172.21.15.82:0/3909934014 conn(0x563741616c90 msgr2=0x56374164aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.591+0000 7f665af83740 1 -- 172.21.15.82:0/3909934014 shutdown_connections 2024-04-21T08:53:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.591+0000 7f665af83740 1 -- 172.21.15.82:0/3909934014 wait complete. 2024-04-21T08:53:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.591+0000 7f665af83740 1 -- start start 2024-04-21T08:53:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.591+0000 7f665af83740 1 Processor -- start 2024-04-21T08:53:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.592+0000 7f665af83740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56374164fd50 0x563741658b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.592+0000 7f665af83740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 0x563741655f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.592+0000 7f665af83740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637416592d0 0x563741656460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.592+0000 7f665af83740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56374165c050 con 0x563741650ac0 2024-04-21T08:53:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.592+0000 7f665af83740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56374153ceb0 con 0x56374164fd50 2024-04-21T08:53:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.592+0000 7f665af83740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56374165c1d0 con 0x5637416592d0 2024-04-21T08:53:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.592+0000 7f665af83740 10 client.0 _ll_register_callbacks cb 0x563741569b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.593+0000 7f6655855640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 0x563741655f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.593+0000 7f6656857640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637416592d0 0x563741656460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.593+0000 7f6656857640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56374164fd50 0x563741658b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.593+0000 7f6655855640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 0x563741655f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35160/0 (socket says 172.21.15.82:35160) 2024-04-21T08:53:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.593+0000 7f6655855640 1 -- 172.21.15.82:0/2386166187 learned_addr learned my addr 172.21.15.82:0/2386166187 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.593+0000 7f6656857640 1 -- 172.21.15.82:0/2386166187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56374164fd50 msgr2=0x563741658b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:02.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.593+0000 7f6656857640 1 --2- 172.21.15.82:0/2386166187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56374164fd50 0x563741658b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.593+0000 7f6656857640 1 -- 172.21.15.82:0/2386166187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 msgr2=0x563741655f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.594+0000 7f6656857640 1 --2- 172.21.15.82:0/2386166187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 0x563741655f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.594+0000 7f6656857640 1 -- 172.21.15.82:0/2386166187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637412c7210 con 0x5637416592d0 2024-04-21T08:53:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.594+0000 7f6655855640 1 --2- 172.21.15.82:0/2386166187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 0x563741655f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:02.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.594+0000 7f6656857640 1 --2- 172.21.15.82:0/2386166187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637416592d0 0x563741656460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f664801ad30 tx=0x7f664801ad90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=657e6ac5072f9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:02.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:02 smithi082 ceph-mon[27967]: pgmap v183: 129 pgs: 129 active+clean; 828 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 836 op/s 2024-04-21T08:53:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.621+0000 7f663dffb640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:53:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.621+0000 7f663dffb640 1 -- 172.21.15.82:0/2386166187 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648023070 con 0x5637416592d0 2024-04-21T08:53:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.621+0000 7f665af83740 1 -- 172.21.15.82:0/2386166187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6648015070 con 0x5637416592d0 2024-04-21T08:53:02.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.621+0000 7f663dffb640 1 -- 172.21.15.82:0/2386166187 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6648011cf0 con 0x5637416592d0 2024-04-21T08:53:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.621+0000 7f663dffb640 1 -- 172.21.15.82:0/2386166187 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6648026040 con 0x5637416592d0 2024-04-21T08:53:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.621+0000 7f665af83740 10 client.26329 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.621+0000 7f665af83740 1 -- 172.21.15.82:0/2386166187 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563741393620 con 0x5637416592d0 2024-04-21T08:53:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.622+0000 7f663dffb640 1 -- 172.21.15.82:0/2386166187 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6648015070 con 0x5637416592d0 2024-04-21T08:53:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.622+0000 7f663dffb640 1 -- 172.21.15.82:0/2386166187 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f6648005390 con 0x5637416592d0 2024-04-21T08:53:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.622+0000 7f665af83740 10 client.26329 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.622+0000 7f665af83740 10 client.26329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.622+0000 7f665af83740 20 client.26329 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.622+0000 7f665af83740 1 --2- 172.21.15.82:0/2386166187 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563741669690 0x56374166bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.622+0000 7f665af83740 4 client.26329 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.622+0000 7f665af83740 1 -- 172.21.15.82:0/2386166187 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637415c8880 con 0x563741669690 2024-04-21T08:53:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.622+0000 7f6655855640 1 --2- 172.21.15.82:0/2386166187 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563741669690 0x56374166bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.623+0000 7f6655855640 1 --2- 172.21.15.82:0/2386166187 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563741669690 0x56374166bab0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=121676b2393bfb4e server_cookie=dd7ac5f0d8cafe56 in_seq=0 out_seq=0 2024-04-21T08:53:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.624+0000 7f663dffb640 10 client.26329 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.626+0000 7f663dffb640 1 -- 172.21.15.82:0/2386166187 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637415c8880 con 0x563741669690 2024-04-21T08:53:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.626+0000 7f663dffb640 10 client.26329 handle_command_reply: tid=0 2024-04-21T08:53:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.626+0000 7f665af83740 1 client.26329 shutdown 2024-04-21T08:53:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.626+0000 7f665af83740 1 -- 172.21.15.82:0/2386166187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637416592d0 msgr2=0x563741656460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.626+0000 7f665af83740 1 --2- 172.21.15.82:0/2386166187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637416592d0 0x563741656460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f664801ad30 tx=0x7f664801ad90 comp rx=0 tx=0).stop 2024-04-21T08:53:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.626+0000 7f665af83740 1 -- 172.21.15.82:0/2386166187 shutdown_connections 2024-04-21T08:53:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.626+0000 7f665af83740 1 --2- 172.21.15.82:0/2386166187 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563741669690 0x56374166bab0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.626+0000 7f665af83740 1 --2- 172.21.15.82:0/2386166187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637416592d0 0x563741656460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.626+0000 7f665af83740 1 --2- 172.21.15.82:0/2386166187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563741650ac0 0x563741655f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.626+0000 7f665af83740 1 --2- 172.21.15.82:0/2386166187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56374164fd50 0x563741658b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.626+0000 7f665af83740 1 -- 172.21.15.82:0/2386166187 >> 172.21.15.82:0/2386166187 conn(0x563741616c90 msgr2=0x56374164c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.627+0000 7f663dffb640 0 client.26329 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.627+0000 7f665af83740 1 -- 172.21.15.82:0/2386166187 shutdown_connections 2024-04-21T08:53:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.627+0000 7f665af83740 1 -- 172.21.15.82:0/2386166187 wait complete. 2024-04-21T08:53:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.627+0000 7f665af83740 20 client.26329 trim_cache size 0 max 16384 2024-04-21T08:53:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:02 smithi106 ceph-mon[31072]: pgmap v183: 129 pgs: 129 active+clean; 828 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.8 MiB/s wr, 836 op/s 2024-04-21T08:53:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.674+0000 7f665af83740 1 Processor -- start 2024-04-21T08:53:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.675+0000 7f665af83740 1 -- start start 2024-04-21T08:53:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.675+0000 7f665af83740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56374164fd50 0x563741665030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.675+0000 7f665af83740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563741650ac0 0x563741645a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.675+0000 7f665af83740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637416592d0 0x56374164c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.675+0000 7f665af83740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6634010080 con 0x5637416592d0 2024-04-21T08:53:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.675+0000 7f665af83740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6634011c50 con 0x563741650ac0 2024-04-21T08:53:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.676+0000 7f665af83740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6634010750 con 0x56374164fd50 2024-04-21T08:53:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.676+0000 7f6655855640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563741650ac0 0x563741645a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.676+0000 7f6655855640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563741650ac0 0x563741645a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:47232/0 (socket says 172.21.15.82:47232) 2024-04-21T08:53:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.676+0000 7f6655855640 1 -- 172.21.15.82:0/3141362750 learned_addr learned my addr 172.21.15.82:0/3141362750 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.676+0000 7f6656857640 1 --2- 172.21.15.82:0/3141362750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637416592d0 0x56374164c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.676+0000 7f6656857640 1 --2- 172.21.15.82:0/3141362750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56374164fd50 0x563741665030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.676+0000 7f6655855640 1 -- 172.21.15.82:0/3141362750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56374164fd50 msgr2=0x563741665030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.676+0000 7f6655855640 1 --2- 172.21.15.82:0/3141362750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56374164fd50 0x563741665030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.676+0000 7f6655855640 1 -- 172.21.15.82:0/3141362750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637416592d0 msgr2=0x56374164c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.676+0000 7f6655855640 1 --2- 172.21.15.82:0/3141362750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637416592d0 0x56374164c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.677+0000 7f6655855640 1 -- 172.21.15.82:0/3141362750 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563741654ed0 con 0x563741650ac0 2024-04-21T08:53:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.677+0000 7f6655855640 1 --2- 172.21.15.82:0/3141362750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563741650ac0 0x563741645a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f664400b240 tx=0x7f6644008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c90bc66f246036 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.677+0000 7f663ffff640 1 -- 172.21.15.82:0/3141362750 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644015c30 con 0x563741650ac0 2024-04-21T08:53:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.677+0000 7f663ffff640 1 -- 172.21.15.82:0/3141362750 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6644015dd0 con 0x563741650ac0 2024-04-21T08:53:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.678+0000 7f665af83740 1 -- 172.21.15.82:0/3141362750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563741650ac0 msgr2=0x563741645a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.678+0000 7f665af83740 1 --2- 172.21.15.82:0/3141362750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563741650ac0 0x563741645a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f664400b240 tx=0x7f6644008040 comp rx=0 tx=0).stop 2024-04-21T08:53:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.679+0000 7f665af83740 1 -- 172.21.15.82:0/3141362750 shutdown_connections 2024-04-21T08:53:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.679+0000 7f665af83740 1 --2- 172.21.15.82:0/3141362750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637416592d0 0x56374164c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.679+0000 7f665af83740 1 --2- 172.21.15.82:0/3141362750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563741650ac0 0x563741645a20 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f664400b240 tx=0x7f6644008040 comp rx=0 tx=0).stop 2024-04-21T08:53:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.679+0000 7f665af83740 1 --2- 172.21.15.82:0/3141362750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56374164fd50 0x563741665030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.679+0000 7f665af83740 1 -- 172.21.15.82:0/3141362750 >> 172.21.15.82:0/3141362750 conn(0x563741616c90 msgr2=0x56374164be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.710+0000 7f665af83740 1 -- 172.21.15.82:0/3141362750 shutdown_connections 2024-04-21T08:53:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.710+0000 7f665af83740 1 -- 172.21.15.82:0/3141362750 wait complete. 2024-04-21T08:53:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.710+0000 7f665af83740 1 -- start start 2024-04-21T08:53:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.710+0000 7f665af83740 1 Processor -- start 2024-04-21T08:53:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.711+0000 7f665af83740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56374164fd50 0x56374165c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.711+0000 7f665af83740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637416592d0 0x56374165ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.711+0000 7f665af83740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56374165cf80 0x5637416cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.711+0000 7f665af83740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6634011580 con 0x56374164fd50 2024-04-21T08:53:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.711+0000 7f665af83740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6634010eb0 con 0x5637416592d0 2024-04-21T08:53:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.711+0000 7f665af83740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f663400e060 con 0x56374165cf80 2024-04-21T08:53:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.711+0000 7f665af83740 10 client.0 _ll_register_callbacks cb 0x7f66480044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.711+0000 7f6655855640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637416592d0 0x56374165ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.711+0000 7f6655855640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637416592d0 0x56374165ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:47244/0 (socket says 172.21.15.82:47244) 2024-04-21T08:53:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.711+0000 7f6655855640 1 -- 172.21.15.82:0/1622510788 learned_addr learned my addr 172.21.15.82:0/1622510788 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.711+0000 7f6656857640 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56374164fd50 0x56374165c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.712+0000 7f6655855640 1 -- 172.21.15.82:0/1622510788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56374165cf80 msgr2=0x5637416cdc20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:53:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.712+0000 7f6655855640 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56374165cf80 0x5637416cdc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.712+0000 7f6655855640 1 -- 172.21.15.82:0/1622510788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56374164fd50 msgr2=0x56374165c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.712+0000 7f6655855640 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56374164fd50 0x56374165c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.712+0000 7f6655855640 1 -- 172.21.15.82:0/1622510788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637416ce190 con 0x5637416592d0 2024-04-21T08:53:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.712+0000 7f6656857640 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56374164fd50 0x56374165c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.712+0000 7f6655855640 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637416592d0 0x56374165ca40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f664400b310 tx=0x7f6644015ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=736080e93bde7e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.717+0000 7f663dffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:53:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.717+0000 7f663dffb640 1 -- 172.21.15.82:0/1622510788 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644004250 con 0x5637416592d0 2024-04-21T08:53:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.717+0000 7f665af83740 1 -- 172.21.15.82:0/1622510788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563741646100 con 0x5637416592d0 2024-04-21T08:53:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.717+0000 7f663dffb640 1 -- 172.21.15.82:0/1622510788 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f66440043f0 con 0x5637416592d0 2024-04-21T08:53:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.717+0000 7f663dffb640 1 -- 172.21.15.82:0/1622510788 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6644036050 con 0x5637416592d0 2024-04-21T08:53:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.717+0000 7f665af83740 10 client.26618 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.717+0000 7f665af83740 1 -- 172.21.15.82:0/1622510788 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5637416cf4f0 con 0x5637416592d0 2024-04-21T08:53:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.718+0000 7f663dffb640 1 -- 172.21.15.82:0/1622510788 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563741646100 con 0x5637416592d0 2024-04-21T08:53:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.719+0000 7f663dffb640 1 -- 172.21.15.82:0/1622510788 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f6644003a00 con 0x5637416592d0 2024-04-21T08:53:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.720+0000 7f665af83740 10 client.26618 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.720+0000 7f665af83740 10 client.26618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.720+0000 7f665af83740 20 client.26618 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.720+0000 7f665af83740 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5637416cfb10 0x5637416d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.720+0000 7f665af83740 4 client.26618 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:53:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.720+0000 7f665af83740 1 -- 172.21.15.82:0/1622510788 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637415543d0 con 0x5637416cfb10 2024-04-21T08:53:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.720+0000 7f6656857640 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5637416cfb10 0x5637416d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.720+0000 7f6656857640 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5637416cfb10 0x5637416d1f50 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d198059160e5ce1 server_cookie=2446fa9a4ec1e706 in_seq=0 out_seq=0 2024-04-21T08:53:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:02.721+0000 7f663dffb640 10 client.26618 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.433+0000 7f663dffb640 1 -- 172.21.15.82:0/1622510788 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637415543d0 con 0x5637416cfb10 2024-04-21T08:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.433+0000 7f663dffb640 10 client.26618 handle_command_reply: tid=0 2024-04-21T08:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.433+0000 7f665af83740 1 client.26618 shutdown 2024-04-21T08:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.433+0000 7f665af83740 1 -- 172.21.15.82:0/1622510788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637416592d0 msgr2=0x56374165ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.433+0000 7f665af83740 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637416592d0 0x56374165ca40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f664400b310 tx=0x7f6644015ba0 comp rx=0 tx=0).stop 2024-04-21T08:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.433+0000 7f665af83740 1 -- 172.21.15.82:0/1622510788 shutdown_connections 2024-04-21T08:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.433+0000 7f665af83740 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5637416cfb10 0x5637416d1f50 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.433+0000 7f665af83740 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56374165cf80 0x5637416cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.433+0000 7f665af83740 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637416592d0 0x56374165ca40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.433+0000 7f663dffb640 0 client.26618 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.433+0000 7f665af83740 1 --2- 172.21.15.82:0/1622510788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56374164fd50 0x56374165c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.433+0000 7f665af83740 1 -- 172.21.15.82:0/1622510788 >> 172.21.15.82:0/1622510788 conn(0x563741616c90 msgr2=0x56374164c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.434+0000 7f665af83740 1 -- 172.21.15.82:0/1622510788 shutdown_connections 2024-04-21T08:53:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.434+0000 7f665af83740 1 -- 172.21.15.82:0/1622510788 wait complete. 2024-04-21T08:53:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.434+0000 7f665af83740 20 client.26618 trim_cache size 0 max 16384 2024-04-21T08:53:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:53:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:53:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:53:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:53:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.450+0000 7f6657058640 1 -- 172.21.15.82:0/461938896 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66380a3f20 msgr2=0x7f66380a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.450+0000 7f6657058640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66380a3f20 0x7f66380a63e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x563741651220 tx=0x7f66400f8040 comp rx=0 tx=0).stop 2024-04-21T08:53:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.450+0000 7f6657058640 1 -- 172.21.15.82:0/461938896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f665006e240 msgr2=0x7f66501180a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.450+0000 7f6657058640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f665006e240 0x7f66501180a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f664002c6b0 tx=0x7f6640035ce0 comp rx=0 tx=0).stop 2024-04-21T08:53:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.454+0000 7f6657058640 1 -- 172.21.15.82:0/461938896 shutdown_connections 2024-04-21T08:53:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.454+0000 7f6657058640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6650114c90 0x7f6650118b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.454+0000 7f6657058640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66380a3f20 0x7f66380a63e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.454+0000 7f6657058640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6650109520 0x7f66501185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.454+0000 7f6657058640 1 --2- 172.21.15.82:0/461938896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f665006e240 0x7f66501180a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.454+0000 7f6657058640 1 -- 172.21.15.82:0/461938896 >> 172.21.15.82:0/461938896 conn(0x7f665006ce90 msgr2=0x7f6650070840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.455+0000 7f6657058640 1 -- 172.21.15.82:0/461938896 shutdown_connections 2024-04-21T08:53:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:03.455+0000 7f6657058640 1 -- 172.21.15.82:0/461938896 wait complete. 2024-04-21T08:53:04.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:04 smithi134 ceph-mon[31772]: pgmap v184: 129 pgs: 129 active+clean; 836 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.8 MiB/s wr, 1.01k op/s 2024-04-21T08:53:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:04 smithi082 ceph-mon[27967]: pgmap v184: 129 pgs: 129 active+clean; 836 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.8 MiB/s wr, 1.01k op/s 2024-04-21T08:53:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:04 smithi106 ceph-mon[31072]: pgmap v184: 129 pgs: 129 active+clean; 836 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.8 MiB/s wr, 1.01k op/s 2024-04-21T08:53:06.232 INFO:tasks.workunit.client.0.smithi082.stdout: 145 2158476 360 1507709 456 1216419 1373 2024-04-21T08:53:06.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:06 smithi134 ceph-mon[31772]: pgmap v185: 129 pgs: 129 active+clean; 836 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 552 op/s 2024-04-21T08:53:06.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:06 smithi082 ceph-mon[27967]: pgmap v185: 129 pgs: 129 active+clean; 836 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 552 op/s 2024-04-21T08:53:06.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:06 smithi106 ceph-mon[31072]: pgmap v185: 129 pgs: 129 active+clean; 836 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 552 op/s 2024-04-21T08:53:08.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:08 smithi134 ceph-mon[31772]: pgmap v186: 129 pgs: 129 active+clean; 824 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 673 op/s 2024-04-21T08:53:08.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:08 smithi082 ceph-mon[27967]: pgmap v186: 129 pgs: 129 active+clean; 824 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 673 op/s 2024-04-21T08:53:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:08 smithi106 ceph-mon[31072]: pgmap v186: 129 pgs: 129 active+clean; 824 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 673 op/s 2024-04-21T08:53:09.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:09 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:09.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:09 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:09 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:10.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:10 smithi134 ceph-mon[31772]: pgmap v187: 129 pgs: 129 active+clean; 812 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 594 op/s 2024-04-21T08:53:10.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:10 smithi082 ceph-mon[27967]: pgmap v187: 129 pgs: 129 active+clean; 812 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 594 op/s 2024-04-21T08:53:10.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:10 smithi106 ceph-mon[31072]: pgmap v187: 129 pgs: 129 active+clean; 812 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 594 op/s 2024-04-21T08:53:11.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:11 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:53:11.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:11 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:53:11.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:11 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:53:12.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:12 smithi134 ceph-mon[31772]: pgmap v188: 129 pgs: 129 active+clean; 812 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 388 op/s 2024-04-21T08:53:12.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:12 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:53:12.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:12 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:53:12.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:12 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:53:12.529 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:12 smithi106 ceph-mon[31072]: pgmap v188: 129 pgs: 129 active+clean; 812 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 388 op/s 2024-04-21T08:53:12.529 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:12 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:53:12.529 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:12 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:53:12.529 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:12 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:53:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:12 smithi082 ceph-mon[27967]: pgmap v188: 129 pgs: 129 active+clean; 812 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 388 op/s 2024-04-21T08:53:12.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:12 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:53:12.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:12 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:53:12.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:12 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:53:14.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:14 smithi134 ceph-mon[31772]: pgmap v189: 129 pgs: 129 active+clean; 864 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.7 MiB/s wr, 952 op/s 2024-04-21T08:53:14.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:14 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:53:14.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:14 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:53:14.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:14 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:53:14.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:14 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:53:14.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:14 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:53:14.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:14 smithi082 ceph-mon[27967]: pgmap v189: 129 pgs: 129 active+clean; 864 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.7 MiB/s wr, 952 op/s 2024-04-21T08:53:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:14 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:53:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:14 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:53:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:14 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:53:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:14 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:53:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:14 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:53:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:14 smithi106 ceph-mon[31072]: pgmap v189: 129 pgs: 129 active+clean; 864 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.7 MiB/s wr, 952 op/s 2024-04-21T08:53:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:14 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:53:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:14 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:53:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:14 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:53:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:14 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:53:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:14 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:53:15.183 INFO:tasks.workunit.client.0.smithi082.stdout: 2024-04-21T08:53:15.183 INFO:tasks.workunit.client.0.smithi082.stdout:Final score for writes: 145 2024-04-21T08:53:15.183 INFO:tasks.workunit.client.0.smithi082.stdout:Final score for reads : 516039 2024-04-21T08:53:15.183 INFO:tasks.workunit.client.0.smithi082.stdout: 2024-04-21T08:53:15.184 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-21T08:53:15.184 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-21T08:53:16.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:16 smithi134 ceph-mon[31772]: pgmap v190: 129 pgs: 129 active+clean; 864 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.7 MiB/s wr, 779 op/s 2024-04-21T08:53:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.581+0000 7ff28a8a8640 1 -- 172.21.15.82:0/584244940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff284107270 msgr2=0x7ff28410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.581+0000 7ff28a8a8640 1 --2- 172.21.15.82:0/584244940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff284107270 0x7ff28410ee40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009ca0 tx=0x7ff27c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.581+0000 7ff28a8a8640 1 -- 172.21.15.82:0/584244940 shutdown_connections 2024-04-21T08:53:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.581+0000 7ff28a8a8640 1 --2- 172.21.15.82:0/584244940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff284107270 0x7ff28410ee40 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.581+0000 7ff28a8a8640 1 --2- 172.21.15.82:0/584244940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff284102b50 0x7ff284106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.581+0000 7ff28a8a8640 1 --2- 172.21.15.82:0/584244940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2841021a0 0x7ff284102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.581+0000 7ff28a8a8640 1 -- 172.21.15.82:0/584244940 >> 172.21.15.82:0/584244940 conn(0x7ff2840fda00 msgr2=0x7ff2840ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.581+0000 7ff28a8a8640 1 -- 172.21.15.82:0/584244940 shutdown_connections 2024-04-21T08:53:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.581+0000 7ff28a8a8640 1 -- 172.21.15.82:0/584244940 wait complete. 2024-04-21T08:53:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.581+0000 7ff28a8a8640 1 Processor -- start 2024-04-21T08:53:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.581+0000 7ff28a8a8640 1 -- start start 2024-04-21T08:53:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff28a8a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2841021a0 0x7ff2841985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff28a8a8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff284107270 0x7ff284198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff28a8a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28419cea0 0x7ff28419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff28a8a8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841121c0 con 0x7ff28419cea0 2024-04-21T08:53:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff28a8a8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff284112040 con 0x7ff284107270 2024-04-21T08:53:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff28a8a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff284112340 con 0x7ff2841021a0 2024-04-21T08:53:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff288e45640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28419cea0 0x7ff28419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff284107270 0x7ff284198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff288e45640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28419cea0 0x7ff28419d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36982/0 (socket says 172.21.15.82:36982) 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff288e45640 1 -- 172.21.15.82:0/3246209983 learned_addr learned my addr 172.21.15.82:0/3246209983 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff283fff640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2841021a0 0x7ff2841985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff288e45640 1 -- 172.21.15.82:0/3246209983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2841021a0 msgr2=0x7ff2841985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff288e45640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2841021a0 0x7ff2841985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff288e45640 1 -- 172.21.15.82:0/3246209983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff284107270 msgr2=0x7ff284198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff288e45640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff284107270 0x7ff284198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff288e45640 1 -- 172.21.15.82:0/3246209983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28419d990 con 0x7ff28419cea0 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff283fff640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2841021a0 0x7ff2841985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff288e45640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28419cea0 0x7ff28419d350 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff27c005e30 tx=0x7ff27c032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff2817fa640 1 -- 172.21.15.82:0/3246209983 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c041020 con 0x7ff28419cea0 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff28a8a8640 1 -- 172.21.15.82:0/3246209983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28419dc20 con 0x7ff28419cea0 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff2817fa640 1 -- 172.21.15.82:0/3246209983 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff27c032970 con 0x7ff28419cea0 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.582+0000 7ff2817fa640 1 -- 172.21.15.82:0/3246209983 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c039040 con 0x7ff28419cea0 2024-04-21T08:53:16.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.583+0000 7ff28a8a8640 1 -- 172.21.15.82:0/3246209983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2841a53f0 con 0x7ff28419cea0 2024-04-21T08:53:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.585+0000 7ff2817fa640 1 -- 172.21.15.82:0/3246209983 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff27c044020 con 0x7ff28419cea0 2024-04-21T08:53:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.585+0000 7ff2817fa640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff26c0a3dc0 0x7ff26c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.585+0000 7ff28d57a740 1 Processor -- start 2024-04-21T08:53:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.585+0000 7ff28d57a740 1 -- start start 2024-04-21T08:53:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.585+0000 7ff283fff640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff26c0a3dc0 0x7ff26c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.585+0000 7ff2817fa640 1 -- 172.21.15.82:0/3246209983 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff27c0f2840 con 0x7ff28419cea0 2024-04-21T08:53:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.585+0000 7ff28d57a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d4e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.585+0000 7ff28d57a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d4efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff28d57a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d580e0 0x55a6f5d5a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff28d57a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a6f5cb38d0 con 0x55a6f5d580e0 2024-04-21T08:53:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff28d57a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a6f5c50e80 con 0x55a6f5d4eb60 2024-04-21T08:53:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff28d57a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a6f5caf890 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d4efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff288e45640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d580e0 0x55a6f5d5a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d4efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:58722/0 (socket says 172.21.15.82:58722) 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff2837fe640 1 -- 172.21.15.82:0/1372803534 learned_addr learned my addr 172.21.15.82:0/1372803534 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff283fff640 1 --2- 172.21.15.82:0/1372803534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d4e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff283fff640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff26c0a3dc0 0x7ff26c0a6280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff27400bab0 tx=0x7ff27400beb0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff2837fe640 1 -- 172.21.15.82:0/1372803534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d4f8d0 msgr2=0x55a6f5d4e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff2837fe640 1 --2- 172.21.15.82:0/1372803534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d4e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff2837fe640 1 -- 172.21.15.82:0/1372803534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d580e0 msgr2=0x55a6f5d5a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff2837fe640 1 --2- 172.21.15.82:0/1372803534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d580e0 0x55a6f5d5a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff2837fe640 1 -- 172.21.15.82:0/1372803534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f5c72320 con 0x55a6f5d4eb60 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff288e45640 1 --2- 172.21.15.82:0/1372803534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d580e0 0x55a6f5d5a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff2837fe640 1 --2- 172.21.15.82:0/1372803534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d4efe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff278009c20 tx=0x7ff278011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a77990049dd8a7c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff26a7fc640 1 -- 172.21.15.82:0/1372803534 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278004050 con 0x55a6f5d4eb60 2024-04-21T08:53:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff26a7fc640 1 -- 172.21.15.82:0/1372803534 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2780054d0 con 0x55a6f5d4eb60 2024-04-21T08:53:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.586+0000 7ff26a7fc640 1 -- 172.21.15.82:0/1372803534 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278004c40 con 0x55a6f5d4eb60 2024-04-21T08:53:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.587+0000 7ff28d57a740 1 -- 172.21.15.82:0/1372803534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 msgr2=0x55a6f5d4efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.587+0000 7ff28d57a740 1 --2- 172.21.15.82:0/1372803534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d4efe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff278009c20 tx=0x7ff278011810 comp rx=0 tx=0).stop 2024-04-21T08:53:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.587+0000 7ff28d57a740 1 -- 172.21.15.82:0/1372803534 shutdown_connections 2024-04-21T08:53:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.587+0000 7ff28d57a740 1 --2- 172.21.15.82:0/1372803534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d580e0 0x55a6f5d5a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.587+0000 7ff28d57a740 1 --2- 172.21.15.82:0/1372803534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d4efe0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.587+0000 7ff28d57a740 1 --2- 172.21.15.82:0/1372803534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d4e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.587+0000 7ff28d57a740 1 -- 172.21.15.82:0/1372803534 >> 172.21.15.82:0/1372803534 conn(0x55a6f5d15c90 msgr2=0x55a6f5d49850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.587+0000 7ff28d57a740 1 -- 172.21.15.82:0/1372803534 shutdown_connections 2024-04-21T08:53:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.587+0000 7ff28d57a740 1 -- 172.21.15.82:0/1372803534 wait complete. 2024-04-21T08:53:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.587+0000 7ff28d57a740 1 -- start start 2024-04-21T08:53:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff28d57a740 1 Processor -- start 2024-04-21T08:53:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff28d57a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d57a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff28d57a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d54dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff28d57a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 0x55a6f5d55330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff28d57a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a6f5d5f220 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff28d57a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a6f5c3beb0 con 0x55a6f5d4eb60 2024-04-21T08:53:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff28d57a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a6f5d5f3a0 con 0x55a6f5d580e0 2024-04-21T08:53:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff28d57a740 10 client.0 _ll_register_callbacks cb 0x55a6f5c68b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d54dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d54dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37008/0 (socket says 172.21.15.82:37008) 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff2837fe640 1 -- 172.21.15.82:0/2128522543 learned_addr learned my addr 172.21.15.82:0/2128522543 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff283fff640 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d57a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff288e45640 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 0x55a6f5d55330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff2837fe640 1 -- 172.21.15.82:0/2128522543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 msgr2=0x55a6f5d55330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff2837fe640 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 0x55a6f5d55330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff2837fe640 1 -- 172.21.15.82:0/2128522543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 msgr2=0x55a6f5d57a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff2837fe640 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d57a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff2837fe640 1 -- 172.21.15.82:0/2128522543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f59c6210 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff288e45640 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 0x55a6f5d55330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff2837fe640 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d54dd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff278009bf0 tx=0x7ff278004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4c11d1d11f0e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.588+0000 7ff2637fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff2637fe640 1 -- 172.21.15.82:0/2128522543 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278029070 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff28d57a740 1 -- 172.21.15.82:0/2128522543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2780054d0 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff2637fe640 1 -- 172.21.15.82:0/2128522543 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff278003a00 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff2637fe640 1 -- 172.21.15.82:0/2128522543 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff278035050 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff28d57a740 10 client.17256 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff28d57a740 1 -- 172.21.15.82:0/2128522543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a6f5a92620 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff2637fe640 1 -- 172.21.15.82:0/2128522543 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2780054d0 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff2637fe640 1 -- 172.21.15.82:0/2128522543 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7ff278005110 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff28d57a740 10 client.17256 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff28d57a740 10 client.17256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff28d57a740 20 client.17256 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff28d57a740 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a6f5d6c3b0 0x55a6f5d6e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff28d57a740 4 client.17256 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff28d57a740 1 -- 172.21.15.82:0/2128522543 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6f5cc7880 con 0x55a6f5d6c3b0 2024-04-21T08:53:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.589+0000 7ff283fff640 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a6f5d6c3b0 0x55a6f5d6e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.590+0000 7ff2637fe640 10 client.17256 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.590+0000 7ff283fff640 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a6f5d6c3b0 0x55a6f5d6e7a0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa1bc82283a16fb server_cookie=d0c0dbc8083a466e in_seq=0 out_seq=0 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.590+0000 7ff2637fe640 1 -- 172.21.15.82:0/2128522543 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6f5cc7880 con 0x55a6f5d6c3b0 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.590+0000 7ff2637fe640 10 client.17256 handle_command_reply: tid=0 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.590+0000 7ff28d57a740 1 client.17256 shutdown 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.591+0000 7ff28d57a740 1 -- 172.21.15.82:0/2128522543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 msgr2=0x55a6f5d54dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.591+0000 7ff28d57a740 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d54dd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff278009bf0 tx=0x7ff278004740 comp rx=0 tx=0).stop 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.591+0000 7ff28d57a740 1 -- 172.21.15.82:0/2128522543 shutdown_connections 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.591+0000 7ff28d57a740 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a6f5d6c3b0 0x55a6f5d6e7a0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.591+0000 7ff28d57a740 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 0x55a6f5d55330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.591+0000 7ff2637fe640 0 client.17256 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.591+0000 7ff28d57a740 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d54dd0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.591+0000 7ff28d57a740 1 --2- 172.21.15.82:0/2128522543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d57a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.591+0000 7ff28d57a740 1 -- 172.21.15.82:0/2128522543 >> 172.21.15.82:0/2128522543 conn(0x55a6f5d15c90 msgr2=0x55a6f5d4b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.591+0000 7ff28d57a740 1 -- 172.21.15.82:0/2128522543 shutdown_connections 2024-04-21T08:53:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.591+0000 7ff28d57a740 1 -- 172.21.15.82:0/2128522543 wait complete. 2024-04-21T08:53:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.591+0000 7ff28d57a740 20 client.17256 trim_cache size 0 max 16384 2024-04-21T08:53:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff28d57a740 1 Processor -- start 2024-04-21T08:53:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff28d57a740 1 -- start start 2024-04-21T08:53:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff28d57a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d68120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff28d57a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d57500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff28d57a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 0x55a6f5d57a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff28d57a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841ad050 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff28d57a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841aec20 con 0x55a6f5d4eb60 2024-04-21T08:53:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff28d57a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841ad720 con 0x55a6f5d580e0 2024-04-21T08:53:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d57500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d57500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37014/0 (socket says 172.21.15.82:37014) 2024-04-21T08:53:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff2837fe640 1 -- 172.21.15.82:0/870446975 learned_addr learned my addr 172.21.15.82:0/870446975 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff283fff640 1 --2- 172.21.15.82:0/870446975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d68120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff2837fe640 1 -- 172.21.15.82:0/870446975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 msgr2=0x55a6f5d57a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff2837fe640 1 --2- 172.21.15.82:0/870446975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 0x55a6f5d57a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff2837fe640 1 -- 172.21.15.82:0/870446975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 msgr2=0x55a6f5d68120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff2837fe640 1 --2- 172.21.15.82:0/870446975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d68120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff2837fe640 1 -- 172.21.15.82:0/870446975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f5d4de30 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff283fff640 1 --2- 172.21.15.82:0/870446975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d68120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.606+0000 7ff2837fe640 1 --2- 172.21.15.82:0/870446975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d57500 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff278009730 tx=0x7ff278005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64cc57f9b44b140 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.607+0000 7ff269ffb640 1 -- 172.21.15.82:0/870446975 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278029030 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.607+0000 7ff269ffb640 1 -- 172.21.15.82:0/870446975 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff278003800 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.607+0000 7ff269ffb640 1 -- 172.21.15.82:0/870446975 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27802e6a0 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.607+0000 7ff28d57a740 1 -- 172.21.15.82:0/870446975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 msgr2=0x55a6f5d57500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.607+0000 7ff28d57a740 1 --2- 172.21.15.82:0/870446975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d57500 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff278009730 tx=0x7ff278005b90 comp rx=0 tx=0).stop 2024-04-21T08:53:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.607+0000 7ff28d57a740 1 -- 172.21.15.82:0/870446975 shutdown_connections 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.607+0000 7ff28d57a740 1 --2- 172.21.15.82:0/870446975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 0x55a6f5d57a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.607+0000 7ff28d57a740 1 --2- 172.21.15.82:0/870446975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d57500 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.607+0000 7ff28d57a740 1 --2- 172.21.15.82:0/870446975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d68120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.607+0000 7ff28d57a740 1 -- 172.21.15.82:0/870446975 >> 172.21.15.82:0/870446975 conn(0x55a6f5d15c90 msgr2=0x55a6f5d49380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.607+0000 7ff28d57a740 1 -- 172.21.15.82:0/870446975 shutdown_connections 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.607+0000 7ff28d57a740 1 -- 172.21.15.82:0/870446975 wait complete. 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff28d57a740 1 -- start start 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff28d57a740 1 Processor -- start 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff28d57a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d64630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff28d57a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d64b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff28d57a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 0x55a6f5d52050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff28d57a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841ae550 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff28d57a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841ade80 con 0x55a6f5d4eb60 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff28d57a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff2841ab030 con 0x55a6f5d580e0 2024-04-21T08:53:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff28d57a740 10 client.0 _ll_register_callbacks cb 0x7ff27c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d64b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d64b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37018/0 (socket says 172.21.15.82:37018) 2024-04-21T08:53:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff2837fe640 1 -- 172.21.15.82:0/998440525 learned_addr learned my addr 172.21.15.82:0/998440525 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff283fff640 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d64630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.613 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:16 smithi082 ceph-mon[27967]: pgmap v190: 129 pgs: 129 active+clean; 864 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.7 MiB/s wr, 779 op/s 2024-04-21T08:53:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff2837fe640 1 -- 172.21.15.82:0/998440525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 msgr2=0x55a6f5d52050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff288e45640 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 0x55a6f5d52050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff2837fe640 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 0x55a6f5d52050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff2837fe640 1 -- 172.21.15.82:0/998440525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 msgr2=0x55a6f5d64630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff2837fe640 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d64630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.608+0000 7ff2837fe640 1 -- 172.21.15.82:0/998440525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f5d52740 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff283fff640 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d64630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff2837fe640 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d64b70 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff278011d10 tx=0x7ff278004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac428b7bd89f83bb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff2637fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff2637fe640 1 -- 172.21.15.82:0/998440525 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278029040 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff2637fe640 1 -- 172.21.15.82:0/998440525 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff2780039b0 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff28d57a740 1 -- 172.21.15.82:0/998440525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff278003800 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff2637fe640 1 -- 172.21.15.82:0/998440525 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff27804b050 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff28d57a740 10 client.17262 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff28d57a740 1 -- 172.21.15.82:0/998440525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a6f5d58e90 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff2637fe640 1 -- 172.21.15.82:0/998440525 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff278003800 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff2637fe640 1 -- 172.21.15.82:0/998440525 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7ff278004770 con 0x55a6f5d4f8d0 2024-04-21T08:53:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff28d57a740 10 client.17262 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff28d57a740 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff28d57a740 20 client.17262 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff28d57a740 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a6f5dd2840 0x55a6f5dd4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff28d57a740 4 client.17262 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:53:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.609+0000 7ff28d57a740 1 -- 172.21.15.82:0/998440525 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6f5c533d0 con 0x55a6f5dd2840 2024-04-21T08:53:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.610+0000 7ff283fff640 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a6f5dd2840 0x55a6f5dd4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.610+0000 7ff283fff640 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a6f5dd2840 0x55a6f5dd4c30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc02766b9b524d43 server_cookie=a46f3f39f585aaa0 in_seq=0 out_seq=0 2024-04-21T08:53:16.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.610+0000 7ff2637fe640 10 client.17262 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:16.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:16 smithi106 ceph-mon[31072]: pgmap v190: 129 pgs: 129 active+clean; 864 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.7 MiB/s wr, 779 op/s 2024-04-21T08:53:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff2637fe640 1 -- 172.21.15.82:0/998440525 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6f5c533d0 con 0x55a6f5dd2840 2024-04-21T08:53:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff2637fe640 10 client.17262 handle_command_reply: tid=0 2024-04-21T08:53:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff28d57a740 1 client.17262 shutdown 2024-04-21T08:53:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff28d57a740 1 -- 172.21.15.82:0/998440525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 msgr2=0x55a6f5d64b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff28d57a740 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d64b70 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff278011d10 tx=0x7ff278004600 comp rx=0 tx=0).stop 2024-04-21T08:53:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff28d57a740 1 -- 172.21.15.82:0/998440525 shutdown_connections 2024-04-21T08:53:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff28d57a740 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55a6f5dd2840 0x55a6f5dd4c30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff28d57a740 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a6f5d580e0 0x55a6f5d52050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff28d57a740 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f5d4f8d0 0x55a6f5d64b70 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff2637fe640 0 client.17262 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff28d57a740 1 --2- 172.21.15.82:0/998440525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6f5d4eb60 0x55a6f5d64630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff28d57a740 1 -- 172.21.15.82:0/998440525 >> 172.21.15.82:0/998440525 conn(0x55a6f5d15c90 msgr2=0x55a6f5d532a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff28d57a740 1 -- 172.21.15.82:0/998440525 shutdown_connections 2024-04-21T08:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff28d57a740 1 -- 172.21.15.82:0/998440525 wait complete. 2024-04-21T08:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.793+0000 7ff28d57a740 20 client.17262 trim_cache size 0 max 16384 2024-04-21T08:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.794+0000 7ff28a8a8640 1 -- 172.21.15.82:0/3246209983 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff26c0a3dc0 msgr2=0x7ff26c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.794+0000 7ff28a8a8640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff26c0a3dc0 0x7ff26c0a6280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff27400bab0 tx=0x7ff27400beb0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.794+0000 7ff28a8a8640 1 -- 172.21.15.82:0/3246209983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28419cea0 msgr2=0x7ff28419d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.794+0000 7ff28a8a8640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28419cea0 0x7ff28419d350 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff27c005e30 tx=0x7ff27c032420 comp rx=0 tx=0).stop 2024-04-21T08:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.794+0000 7ff28a8a8640 1 -- 172.21.15.82:0/3246209983 shutdown_connections 2024-04-21T08:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.794+0000 7ff28a8a8640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff26c0a3dc0 0x7ff26c0a6280 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.794+0000 7ff28a8a8640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28419cea0 0x7ff28419d350 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.794+0000 7ff28a8a8640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff284107270 0x7ff284198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.794+0000 7ff28a8a8640 1 --2- 172.21.15.82:0/3246209983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2841021a0 0x7ff2841985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.794+0000 7ff28a8a8640 1 -- 172.21.15.82:0/3246209983 >> 172.21.15.82:0/3246209983 conn(0x7ff2840fda00 msgr2=0x7ff2840fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.794+0000 7ff28a8a8640 1 -- 172.21.15.82:0/3246209983 shutdown_connections 2024-04-21T08:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:16.794+0000 7ff28a8a8640 1 -- 172.21.15.82:0/3246209983 wait complete. 2024-04-21T08:53:18.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:18 smithi134 ceph-mon[31772]: pgmap v191: 129 pgs: 129 active+clean; 876 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 839 op/s 2024-04-21T08:53:18.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:18 smithi082 ceph-mon[27967]: pgmap v191: 129 pgs: 129 active+clean; 876 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 839 op/s 2024-04-21T08:53:18.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:18 smithi106 ceph-mon[31072]: pgmap v191: 129 pgs: 129 active+clean; 876 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 839 op/s 2024-04-21T08:53:19.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:53:19.967 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:53:20.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.109+0000 7fa9496c7640 1 -- 172.21.15.82:0/3090908007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa94410c500 msgr2=0x7fa94410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.109+0000 7fa9496c7640 1 --2- 172.21.15.82:0/3090908007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa94410c500 0x7fa94410e9a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009ca0 tx=0x7fa93c02c230 comp rx=0 tx=0).stop 2024-04-21T08:53:20.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.109+0000 7fa9496c7640 1 -- 172.21.15.82:0/3090908007 shutdown_connections 2024-04-21T08:53:20.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.109+0000 7fa9496c7640 1 --2- 172.21.15.82:0/3090908007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa94410c500 0x7fa94410e9a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.109+0000 7fa9496c7640 1 --2- 172.21.15.82:0/3090908007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa944103ae0 0x7fa94410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.109+0000 7fa9496c7640 1 --2- 172.21.15.82:0/3090908007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9441031c0 0x7fa9441035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.109+0000 7fa9496c7640 1 -- 172.21.15.82:0/3090908007 >> 172.21.15.82:0/3090908007 conn(0x7fa944078000 msgr2=0x7fa944078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.109+0000 7fa9496c7640 1 -- 172.21.15.82:0/3090908007 shutdown_connections 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.109+0000 7fa9496c7640 1 -- 172.21.15.82:0/3090908007 wait complete. 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.110+0000 7fa9496c7640 1 Processor -- start 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.110+0000 7fa9496c7640 1 -- start start 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.110+0000 7fa9496c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9441031c0 0x7fa9441986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.110+0000 7fa9496c7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa944103ae0 0x7fa944198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.110+0000 7fa9496c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa94410c500 0x7fa94419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.110+0000 7fa9496c7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa944112190 con 0x7fa9441031c0 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.110+0000 7fa9496c7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa944112010 con 0x7fa944103ae0 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.110+0000 7fa9496c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa944112310 con 0x7fa94410c500 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.110+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9441031c0 0x7fa9441986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.110+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9441031c0 0x7fa9441986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37046/0 (socket says 172.21.15.82:37046) 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.110+0000 7fa942ffd640 1 -- 172.21.15.82:0/4232284582 learned_addr learned my addr 172.21.15.82:0/4232284582 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.110+0000 7fa9437fe640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa94410c500 0x7fa94419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.111+0000 7fa9427fc640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa944103ae0 0x7fa944198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.111+0000 7fa942ffd640 1 -- 172.21.15.82:0/4232284582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa94410c500 msgr2=0x7fa94419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.111+0000 7fa942ffd640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa94410c500 0x7fa94419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.111+0000 7fa942ffd640 1 -- 172.21.15.82:0/4232284582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa944103ae0 msgr2=0x7fa944198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.111+0000 7fa942ffd640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa944103ae0 0x7fa944198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.111+0000 7fa942ffd640 1 -- 172.21.15.82:0/4232284582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94419d6a0 con 0x7fa9441031c0 2024-04-21T08:53:20.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.111+0000 7fa942ffd640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9441031c0 0x7fa9441986b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa93400d950 tx=0x7fa93400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.111+0000 7fa92bfff640 1 -- 172.21.15.82:0/4232284582 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934017070 con 0x7fa9441031c0 2024-04-21T08:53:20.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.111+0000 7fa9496c7640 1 -- 172.21.15.82:0/4232284582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94419d990 con 0x7fa9441031c0 2024-04-21T08:53:20.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.111+0000 7fa9496c7640 1 -- 172.21.15.82:0/4232284582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9441a5220 con 0x7fa9441031c0 2024-04-21T08:53:20.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.111+0000 7fa92bfff640 1 -- 172.21.15.82:0/4232284582 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa934009c30 con 0x7fa9441031c0 2024-04-21T08:53:20.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.111+0000 7fa92bfff640 1 -- 172.21.15.82:0/4232284582 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93400f620 con 0x7fa9441031c0 2024-04-21T08:53:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.113+0000 7fa92bfff640 1 -- 172.21.15.82:0/4232284582 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa93400f7c0 con 0x7fa9441031c0 2024-04-21T08:53:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa94c38a740 1 Processor -- start 2024-04-21T08:53:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa94c38a740 1 -- start start 2024-04-21T08:53:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa92bfff640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa92c0a3e20 0x7fa92c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa94c38a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641580 0x55c82c641960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa9437fe640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa92c0a3e20 0x7fa92c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa92bfff640 1 -- 172.21.15.82:0/4232284582 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa9340d0700 con 0x7fa9441031c0 2024-04-21T08:53:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641580 0x55c82c641960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641580 0x55c82c641960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37052/0 (socket says 172.21.15.82:37052) 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa942ffd640 1 -- 172.21.15.82:0/3174110331 learned_addr learned my addr 172.21.15.82:0/3174110331 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa94c38a740 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641ea0 0x55c82c64a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa94c38a740 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c64aa80 0x55c82c64ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa94c38a740 1 -- 172.21.15.82:0/3174110331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c82c5a61e0 con 0x55c82c641580 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa94c38a740 1 -- 172.21.15.82:0/3174110331 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c82c543390 con 0x55c82c641ea0 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa94c38a740 1 -- 172.21.15.82:0/3174110331 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c82c5a21a0 con 0x55c82c64aa80 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa9427fc640 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641ea0 0x55c82c64a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa942ffd640 1 -- 172.21.15.82:0/3174110331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c64aa80 msgr2=0x55c82c64ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa9437fe640 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c64aa80 0x55c82c64ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa942ffd640 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c64aa80 0x55c82c64ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa942ffd640 1 -- 172.21.15.82:0/3174110331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641ea0 msgr2=0x55c82c64a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa942ffd640 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641ea0 0x55c82c64a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa942ffd640 1 -- 172.21.15.82:0/3174110331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82c3827d0 con 0x55c82c641580 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa9427fc640 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641ea0 0x55c82c64a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa9437fe640 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c64aa80 0x55c82c64ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa9437fe640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa92c0a3e20 0x7fa92c0a62e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa93c004af0 tx=0x7fa93c032360 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.114+0000 7fa942ffd640 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641580 0x55c82c641960 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa93400bfd0 tx=0x7fa93409c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6d8f1c7be8ad41 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.115+0000 7fa9297fa640 1 -- 172.21.15.82:0/3174110331 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934020040 con 0x55c82c641580 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.115+0000 7fa9297fa640 1 -- 172.21.15.82:0/3174110331 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9340cf070 con 0x55c82c641580 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.115+0000 7fa9297fa640 1 -- 172.21.15.82:0/3174110331 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9340a4ef0 con 0x55c82c641580 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.115+0000 7fa94c38a740 1 -- 172.21.15.82:0/3174110331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641580 msgr2=0x55c82c641960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.115+0000 7fa94c38a740 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641580 0x55c82c641960 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa93400bfd0 tx=0x7fa93409c980 comp rx=0 tx=0).stop 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.115+0000 7fa94c38a740 1 -- 172.21.15.82:0/3174110331 shutdown_connections 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.115+0000 7fa94c38a740 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c64aa80 0x55c82c64ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.115+0000 7fa94c38a740 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641ea0 0x55c82c64a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.115+0000 7fa94c38a740 1 --2- 172.21.15.82:0/3174110331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641580 0x55c82c641960 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.115+0000 7fa94c38a740 1 -- 172.21.15.82:0/3174110331 >> 172.21.15.82:0/3174110331 conn(0x55c82c63bcb0 msgr2=0x55c82c63c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.115+0000 7fa94c38a740 1 -- 172.21.15.82:0/3174110331 shutdown_connections 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.115+0000 7fa94c38a740 1 -- 172.21.15.82:0/3174110331 wait complete. 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa94c38a740 1 -- start start 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa94c38a740 1 Processor -- start 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa94c38a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c641580 0x55c82c6479d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa94c38a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641ea0 0x55c82c647f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa94c38a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c64aa80 0x55c82c648450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa94c38a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c82c64d870 con 0x55c82c64aa80 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa94c38a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c82c55f2f0 con 0x55c82c641ea0 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa94c38a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c82c64d9f0 con 0x55c82c641580 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa9437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c64aa80 0x55c82c648450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c641580 0x55c82c6479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa9427fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641ea0 0x55c82c647f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c641580 0x55c82c6479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:47918/0 (socket says 172.21.15.82:47918) 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa942ffd640 1 -- 172.21.15.82:0/3778205535 learned_addr learned my addr 172.21.15.82:0/3778205535 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa94c38a740 10 client.0 _ll_register_callbacks cb 0x55c82c56c580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa942ffd640 1 -- 172.21.15.82:0/3778205535 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641ea0 msgr2=0x55c82c647f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa942ffd640 1 --2- 172.21.15.82:0/3778205535 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641ea0 0x55c82c647f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa942ffd640 1 -- 172.21.15.82:0/3778205535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c64aa80 msgr2=0x55c82c648450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa942ffd640 1 --2- 172.21.15.82:0/3778205535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c64aa80 0x55c82c648450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa942ffd640 1 -- 172.21.15.82:0/3778205535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82c2b83f0 con 0x55c82c641580 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa942ffd640 1 --2- 172.21.15.82:0/3778205535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c641580 0x55c82c6479d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa934009610 tx=0x7fa93409c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28601767d7c16721 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.116+0000 7fa926ff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa926ff5640 1 -- 172.21.15.82:0/3778205535 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa934020070 con 0x55c82c641580 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa926ff5640 1 -- 172.21.15.82:0/3778205535 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa9340cdcb0 con 0x55c82c641580 2024-04-21T08:53:20.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa94c38a740 1 -- 172.21.15.82:0/3778205535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9340cf070 con 0x55c82c641580 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa926ff5640 1 -- 172.21.15.82:0/3778205535 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa9340cdeb0 con 0x55c82c641580 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa94c38a740 10 client.26344 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa94c38a740 1 -- 172.21.15.82:0/3778205535 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c82c2780f0 con 0x55c82c641580 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa926ff5640 1 -- 172.21.15.82:0/3778205535 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9340cf070 con 0x55c82c641580 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa926ff5640 1 -- 172.21.15.82:0/3778205535 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fa934067020 con 0x55c82c641580 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa94c38a740 10 client.26344 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa94c38a740 10 client.26344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa94c38a740 20 client.26344 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa94c38a740 1 --2- 172.21.15.82:0/3778205535 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c82c65a7a0 0x55c82c65cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa94c38a740 4 client.26344 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa94c38a740 1 -- 172.21.15.82:0/3778205535 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c82c5b3c00 con 0x55c82c65a7a0 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.117+0000 7fa9427fc640 1 --2- 172.21.15.82:0/3778205535 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c82c65a7a0 0x55c82c65cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.118+0000 7fa9427fc640 1 --2- 172.21.15.82:0/3778205535 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c82c65a7a0 0x55c82c65cb90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a7d98988d902b1 server_cookie=525ba813b581eb36 in_seq=0 out_seq=0 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.118+0000 7fa926ff5640 10 client.26344 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.118+0000 7fa926ff5640 1 -- 172.21.15.82:0/3778205535 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c82c5b3c00 con 0x55c82c65a7a0 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.118+0000 7fa926ff5640 10 client.26344 handle_command_reply: tid=0 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.118+0000 7fa94c38a740 1 client.26344 shutdown 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.118+0000 7fa94c38a740 1 -- 172.21.15.82:0/3778205535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c641580 msgr2=0x55c82c6479d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.118+0000 7fa94c38a740 1 --2- 172.21.15.82:0/3778205535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c641580 0x55c82c6479d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa934009610 tx=0x7fa93409c370 comp rx=0 tx=0).stop 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.118+0000 7fa94c38a740 1 -- 172.21.15.82:0/3778205535 shutdown_connections 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.118+0000 7fa94c38a740 1 --2- 172.21.15.82:0/3778205535 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c82c65a7a0 0x55c82c65cb90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.119+0000 7fa94c38a740 1 --2- 172.21.15.82:0/3778205535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c64aa80 0x55c82c648450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.119+0000 7fa94c38a740 1 --2- 172.21.15.82:0/3778205535 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641ea0 0x55c82c647f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.119+0000 7fa94c38a740 1 --2- 172.21.15.82:0/3778205535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c641580 0x55c82c6479d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.119+0000 7fa94c38a740 1 -- 172.21.15.82:0/3778205535 >> 172.21.15.82:0/3778205535 conn(0x55c82c63bcb0 msgr2=0x55c82c63da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.119+0000 7fa926ff5640 0 client.26344 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.119+0000 7fa94c38a740 1 -- 172.21.15.82:0/3778205535 shutdown_connections 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.119+0000 7fa94c38a740 1 -- 172.21.15.82:0/3778205535 wait complete. 2024-04-21T08:53:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.119+0000 7fa94c38a740 20 client.26344 trim_cache size 0 max 16384 2024-04-21T08:53:20.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa94c38a740 1 Processor -- start 2024-04-21T08:53:20.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa94c38a740 1 -- start start 2024-04-21T08:53:20.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa94c38a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641580 0x55c82c64a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa94c38a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641ea0 0x55c82c656760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa94c38a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c64aa80 0x55c82c662130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa94c38a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa93000ff70 con 0x55c82c641ea0 2024-04-21T08:53:20.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa94c38a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa930011b40 con 0x55c82c641580 2024-04-21T08:53:20.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa94c38a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa930010640 con 0x55c82c64aa80 2024-04-21T08:53:20.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa9427fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641ea0 0x55c82c656760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa9427fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641ea0 0x55c82c656760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37074/0 (socket says 172.21.15.82:37074) 2024-04-21T08:53:20.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa9427fc640 1 -- 172.21.15.82:0/2847384914 learned_addr learned my addr 172.21.15.82:0/2847384914 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641580 0x55c82c64a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa9437fe640 1 --2- 172.21.15.82:0/2847384914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c64aa80 0x55c82c662130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa9427fc640 1 -- 172.21.15.82:0/2847384914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c64aa80 msgr2=0x55c82c662130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa9427fc640 1 --2- 172.21.15.82:0/2847384914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c64aa80 0x55c82c662130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa9427fc640 1 -- 172.21.15.82:0/2847384914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641580 msgr2=0x55c82c64a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa9427fc640 1 --2- 172.21.15.82:0/2847384914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641580 0x55c82c64a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.137+0000 7fa9427fc640 1 -- 172.21.15.82:0/2847384914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82c63bae0 con 0x55c82c641ea0 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.138+0000 7fa9427fc640 1 --2- 172.21.15.82:0/2847384914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641ea0 0x55c82c656760 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fa938009e30 tx=0x7fa938009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd3be5aef9f5f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.138+0000 7fa928ff9640 1 -- 172.21.15.82:0/2847384914 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938022070 con 0x55c82c641ea0 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.138+0000 7fa928ff9640 1 -- 172.21.15.82:0/2847384914 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa93801d8e0 con 0x55c82c641ea0 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.138+0000 7fa928ff9640 1 -- 172.21.15.82:0/2847384914 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938011c60 con 0x55c82c641ea0 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.138+0000 7fa94c38a740 1 -- 172.21.15.82:0/2847384914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641ea0 msgr2=0x55c82c656760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.138+0000 7fa94c38a740 1 --2- 172.21.15.82:0/2847384914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641ea0 0x55c82c656760 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fa938009e30 tx=0x7fa938009be0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.138+0000 7fa94c38a740 1 -- 172.21.15.82:0/2847384914 shutdown_connections 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.138+0000 7fa94c38a740 1 --2- 172.21.15.82:0/2847384914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c64aa80 0x55c82c662130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.138+0000 7fa94c38a740 1 --2- 172.21.15.82:0/2847384914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c641ea0 0x55c82c656760 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.138+0000 7fa94c38a740 1 --2- 172.21.15.82:0/2847384914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641580 0x55c82c64a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.138+0000 7fa94c38a740 1 -- 172.21.15.82:0/2847384914 >> 172.21.15.82:0/2847384914 conn(0x55c82c63bcb0 msgr2=0x55c82c63cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa94c38a740 1 -- 172.21.15.82:0/2847384914 shutdown_connections 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa94c38a740 1 -- 172.21.15.82:0/2847384914 wait complete. 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa94c38a740 1 -- start start 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa94c38a740 1 Processor -- start 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa94c38a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641580 0x55c82c652c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa94c38a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c641ea0 0x55c82c6531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa94c38a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c64aa80 0x55c82c64fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa94c38a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa930011470 con 0x55c82c64aa80 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa94c38a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa930010da0 con 0x55c82c641580 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa94c38a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa93000df50 con 0x55c82c641ea0 2024-04-21T08:53:20.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa94c38a740 10 client.0 _ll_register_callbacks cb 0x7fa93409bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa9437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c64aa80 0x55c82c64fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa942ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641580 0x55c82c652c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa9437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c64aa80 0x55c82c64fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37078/0 (socket says 172.21.15.82:37078) 2024-04-21T08:53:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa9437fe640 1 -- 172.21.15.82:0/2291385944 learned_addr learned my addr 172.21.15.82:0/2291385944 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa9427fc640 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c641ea0 0x55c82c6531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa9437fe640 1 -- 172.21.15.82:0/2291385944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c641ea0 msgr2=0x55c82c6531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa9437fe640 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c641ea0 0x55c82c6531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa9437fe640 1 -- 172.21.15.82:0/2291385944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641580 msgr2=0x55c82c652c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa9437fe640 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641580 0x55c82c652c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa9437fe640 1 -- 172.21.15.82:0/2291385944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82c63b4f0 con 0x55c82c64aa80 2024-04-21T08:53:20.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.139+0000 7fa942ffd640 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641580 0x55c82c652c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa9437fe640 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c64aa80 0x55c82c64fcc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fa93c037040 tx=0x7fa93c02c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406d8912acd12b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa926ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa926ff5640 1 -- 172.21.15.82:0/2291385944 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c052020 con 0x55c82c64aa80 2024-04-21T08:53:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa926ff5640 1 -- 172.21.15.82:0/2291385944 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa93c004510 con 0x55c82c64aa80 2024-04-21T08:53:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa94c38a740 1 -- 172.21.15.82:0/2291385944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa93801d8e0 con 0x55c82c64aa80 2024-04-21T08:53:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa926ff5640 1 -- 172.21.15.82:0/2291385944 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa93c05d050 con 0x55c82c64aa80 2024-04-21T08:53:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa94c38a740 10 client.17280 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa94c38a740 1 -- 172.21.15.82:0/2291385944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c82c64ce70 con 0x55c82c64aa80 2024-04-21T08:53:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa926ff5640 1 -- 172.21.15.82:0/2291385944 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa93801d8e0 con 0x55c82c64aa80 2024-04-21T08:53:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa926ff5640 1 -- 172.21.15.82:0/2291385944 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fa93c03f040 con 0x55c82c64aa80 2024-04-21T08:53:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa94c38a740 10 client.17280 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa94c38a740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa94c38a740 20 client.17280 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa94c38a740 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c82c6c1970 0x55c82c6c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa94c38a740 4 client.17280 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:53:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.140+0000 7fa94c38a740 1 -- 172.21.15.82:0/2291385944 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c82c5ada70 con 0x55c82c6c1970 2024-04-21T08:53:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa942ffd640 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c82c6c1970 0x55c82c6c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa942ffd640 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c82c6c1970 0x55c82c6c3d90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641114e5aaa329ba server_cookie=da3958846ce2d422 in_seq=0 out_seq=0 2024-04-21T08:53:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa926ff5640 10 client.17280 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa926ff5640 1 -- 172.21.15.82:0/2291385944 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c82c5ada70 con 0x55c82c6c1970 2024-04-21T08:53:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa926ff5640 10 client.17280 handle_command_reply: tid=0 2024-04-21T08:53:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa94c38a740 1 client.17280 shutdown 2024-04-21T08:53:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa94c38a740 1 -- 172.21.15.82:0/2291385944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c64aa80 msgr2=0x55c82c64fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa94c38a740 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c64aa80 0x55c82c64fcc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fa93c037040 tx=0x7fa93c02c230 comp rx=0 tx=0).stop 2024-04-21T08:53:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa94c38a740 1 -- 172.21.15.82:0/2291385944 shutdown_connections 2024-04-21T08:53:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa94c38a740 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55c82c6c1970 0x55c82c6c3d90 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa94c38a740 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c82c64aa80 0x55c82c64fcc0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa94c38a740 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c82c641ea0 0x55c82c6531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa926ff5640 0 client.17280 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa94c38a740 1 --2- 172.21.15.82:0/2291385944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c82c641580 0x55c82c652c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa94c38a740 1 -- 172.21.15.82:0/2291385944 >> 172.21.15.82:0/2291385944 conn(0x55c82c63bcb0 msgr2=0x55c82c63ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa94c38a740 1 -- 172.21.15.82:0/2291385944 shutdown_connections 2024-04-21T08:53:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa94c38a740 1 -- 172.21.15.82:0/2291385944 wait complete. 2024-04-21T08:53:20.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.141+0000 7fa94c38a740 20 client.17280 trim_cache size 0 max 16384 2024-04-21T08:53:20.149 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:53:20.150 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:53:20.150 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:53:20.150 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:53:20.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.142+0000 7fa9496c7640 1 -- 172.21.15.82:0/4232284582 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa92c0a3e20 msgr2=0x7fa92c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.142+0000 7fa9496c7640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa92c0a3e20 0x7fa92c0a62e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa93c004af0 tx=0x7fa93c032360 comp rx=0 tx=0).stop 2024-04-21T08:53:20.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.142+0000 7fa9496c7640 1 -- 172.21.15.82:0/4232284582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9441031c0 msgr2=0x7fa9441986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.142+0000 7fa9496c7640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9441031c0 0x7fa9441986b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa93400d950 tx=0x7fa93400de10 comp rx=0 tx=0).stop 2024-04-21T08:53:20.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.142+0000 7fa9496c7640 1 -- 172.21.15.82:0/4232284582 shutdown_connections 2024-04-21T08:53:20.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.142+0000 7fa9496c7640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa94410c500 0x7fa94419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.142+0000 7fa9496c7640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa944103ae0 0x7fa944198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.142+0000 7fa9496c7640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa92c0a3e20 0x7fa92c0a62e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.142+0000 7fa9496c7640 1 --2- 172.21.15.82:0/4232284582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9441031c0 0x7fa9441986b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.142+0000 7fa9496c7640 1 -- 172.21.15.82:0/4232284582 >> 172.21.15.82:0/4232284582 conn(0x7fa944078000 msgr2=0x7fa9440fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.142+0000 7fa9496c7640 1 -- 172.21.15.82:0/4232284582 shutdown_connections 2024-04-21T08:53:20.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.142+0000 7fa9496c7640 1 -- 172.21.15.82:0/4232284582 wait complete. 2024-04-21T08:53:20.153 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:53:20.153 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:53:20.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.254+0000 7f74ab3e2640 1 -- 172.21.15.82:0/3764414191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74a4069620 msgr2=0x7f74a4069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.254+0000 7f74ab3e2640 1 --2- 172.21.15.82:0/3764414191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74a4069620 0x7f74a4069aa0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7494009900 tx=0x7f749402bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.254+0000 7f74ab3e2640 1 -- 172.21.15.82:0/3764414191 shutdown_connections 2024-04-21T08:53:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.254+0000 7f74ab3e2640 1 --2- 172.21.15.82:0/3764414191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74a410e810 0x7f74a4110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.254+0000 7f74ab3e2640 1 --2- 172.21.15.82:0/3764414191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74a4069620 0x7f74a4069aa0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.254+0000 7f74ab3e2640 1 --2- 172.21.15.82:0/3764414191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74a4102e30 0x7f74a40690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.254+0000 7f74ab3e2640 1 -- 172.21.15.82:0/3764414191 >> 172.21.15.82:0/3764414191 conn(0x7f74a4077e30 msgr2=0x7f74a4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74ab3e2640 1 -- 172.21.15.82:0/3764414191 shutdown_connections 2024-04-21T08:53:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74ab3e2640 1 -- 172.21.15.82:0/3764414191 wait complete. 2024-04-21T08:53:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74ab3e2640 1 Processor -- start 2024-04-21T08:53:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74ab3e2640 1 -- start start 2024-04-21T08:53:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74ab3e2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74a4069620 0x7f74a41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74ab3e2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74a4102e30 0x7f74a41a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74ab3e2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74a410e810 0x7f74a419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74ab3e2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f74a41143a0 con 0x7f74a410e810 2024-04-21T08:53:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74ab3e2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f74a4114220 con 0x7f74a4069620 2024-04-21T08:53:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74ab3e2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f74a4114520 con 0x7f74a4102e30 2024-04-21T08:53:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74a997f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74a410e810 0x7f74a419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74a997f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74a410e810 0x7f74a419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37092/0 (socket says 172.21.15.82:37092) 2024-04-21T08:53:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74a997f640 1 -- 172.21.15.82:0/2908663114 learned_addr learned my addr 172.21.15.82:0/2908663114 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74a897d640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74a4102e30 0x7f74a41a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.255+0000 7f74a917e640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74a4069620 0x7f74a41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.256+0000 7f74a997f640 1 -- 172.21.15.82:0/2908663114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74a4102e30 msgr2=0x7f74a41a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.256+0000 7f74a997f640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74a4102e30 0x7f74a41a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.256+0000 7f74a997f640 1 -- 172.21.15.82:0/2908663114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74a4069620 msgr2=0x7f74a41a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.256+0000 7f74a997f640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74a4069620 0x7f74a41a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.256+0000 7f74a997f640 1 -- 172.21.15.82:0/2908663114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a419af70 con 0x7f74a410e810 2024-04-21T08:53:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.256+0000 7f74a997f640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74a410e810 0x7f74a419a7a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f749c00eb10 tx=0x7f749c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.256+0000 7f749a7fc640 1 -- 172.21.15.82:0/2908663114 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c004b20 con 0x7f74a410e810 2024-04-21T08:53:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.256+0000 7f749a7fc640 1 -- 172.21.15.82:0/2908663114 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f749c004cc0 con 0x7f74a410e810 2024-04-21T08:53:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.256+0000 7f74ab3e2640 1 -- 172.21.15.82:0/2908663114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a419b260 con 0x7f74a410e810 2024-04-21T08:53:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.256+0000 7f749a7fc640 1 -- 172.21.15.82:0/2908663114 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c0107b0 con 0x7f74a410e810 2024-04-21T08:53:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.256+0000 7f74ab3e2640 1 -- 172.21.15.82:0/2908663114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a41a7480 con 0x7f74a410e810 2024-04-21T08:53:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.257+0000 7f74ab3e2640 1 -- 172.21.15.82:0/2908663114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a4101430 con 0x7f74a410e810 2024-04-21T08:53:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.258+0000 7f749a7fc640 1 -- 172.21.15.82:0/2908663114 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f749c005020 con 0x7f74a410e810 2024-04-21T08:53:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.259+0000 7f749a7fc640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f74900a3d20 0x7f74900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.259+0000 7f74a917e640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f74900a3d20 0x7f74900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.259+0000 7f749a7fc640 1 -- 172.21.15.82:0/2908663114 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f749c0d1280 con 0x7f74a410e810 2024-04-21T08:53:20.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.259+0000 7f74a917e640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f74900a3d20 0x7f74900a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009bd0 tx=0x7f74a0005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.264+0000 7f749a7fc640 1 -- 172.21.15.82:0/2908663114 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f749c09d560 con 0x7f74a410e810 2024-04-21T08:53:20.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:20 smithi082 ceph-mon[27967]: pgmap v192: 129 pgs: 129 active+clean; 883 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.8 MiB/s wr, 756 op/s 2024-04-21T08:53:20.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.494+0000 7f74ab3e2640 1 -- 172.21.15.82:0/2908663114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74a41a76f0 con 0x7f74a410e810 2024-04-21T08:53:20.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.495+0000 7f749a7fc640 1 -- 172.21.15.82:0/2908663114 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11949 (secure 0 0 0) 0x7f749c09ce60 con 0x7f74a410e810 2024-04-21T08:53:20.495 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:53:20.495 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:51:52.176351+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:53:20.495 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 36 2024-04-21T08:53:20.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.497+0000 7f74ab3e2640 1 -- 172.21.15.82:0/2908663114 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f74900a3d20 msgr2=0x7f74900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.497+0000 7f74ab3e2640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f74900a3d20 0x7f74900a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009bd0 tx=0x7f74a0005f90 comp rx=0 tx=0).stop 2024-04-21T08:53:20.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.497+0000 7f74ab3e2640 1 -- 172.21.15.82:0/2908663114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74a410e810 msgr2=0x7f74a419a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.497+0000 7f74ab3e2640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74a410e810 0x7f74a419a7a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f749c00eb10 tx=0x7f749c00c2d0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.497+0000 7f74ab3e2640 1 -- 172.21.15.82:0/2908663114 shutdown_connections 2024-04-21T08:53:20.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.497+0000 7f74ab3e2640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f74900a3d20 0x7f74900a61e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.497+0000 7f74ab3e2640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74a410e810 0x7f74a419a7a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.497+0000 7f74ab3e2640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74a4102e30 0x7f74a41a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.497+0000 7f74ab3e2640 1 --2- 172.21.15.82:0/2908663114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74a4069620 0x7f74a41a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.497+0000 7f74ab3e2640 1 -- 172.21.15.82:0/2908663114 >> 172.21.15.82:0/2908663114 conn(0x7f74a4077e30 msgr2=0x7f74a410f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.497+0000 7f74ab3e2640 1 -- 172.21.15.82:0/2908663114 shutdown_connections 2024-04-21T08:53:20.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.497+0000 7f74ab3e2640 1 -- 172.21.15.82:0/2908663114 wait complete. 2024-04-21T08:53:20.507 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:53:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.610+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/1464253556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28107030 msgr2=0x7f3d28107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.610+0000 7f3d2d4c1640 1 --2- 172.21.15.82:0/1464253556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28107030 0x7f3d28107410 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009930 tx=0x7f3d1802bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.610+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/1464253556 shutdown_connections 2024-04-21T08:53:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.610+0000 7f3d2d4c1640 1 --2- 172.21.15.82:0/1464253556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d28101b40 0x7f3d2810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.610+0000 7f3d2d4c1640 1 --2- 172.21.15.82:0/1464253556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d281011a0 0x7f3d28101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.610+0000 7f3d2d4c1640 1 --2- 172.21.15.82:0/1464253556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28107030 0x7f3d28107410 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.610+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/1464253556 >> 172.21.15.82:0/1464253556 conn(0x7f3d28076c50 msgr2=0x7f3d28077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.610+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/1464253556 shutdown_connections 2024-04-21T08:53:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.610+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/1464253556 wait complete. 2024-04-21T08:53:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d2d4c1640 1 Processor -- start 2024-04-21T08:53:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d2d4c1640 1 -- start start 2024-04-21T08:53:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d2d4c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d281011a0 0x7f3d28197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d2d4c1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28101b40 0x7f3d28197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d2d4c1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d28107030 0x7f3d2819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d2d4c1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281110c0 con 0x7f3d28101b40 2024-04-21T08:53:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d2d4c1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28110f40 con 0x7f3d28107030 2024-04-21T08:53:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d2d4c1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28111240 con 0x7f3d281011a0 2024-04-21T08:53:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d1ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28101b40 0x7f3d28197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d277fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d28107030 0x7f3d2819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d1ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28101b40 0x7f3d28197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37102/0 (socket says 172.21.15.82:37102) 2024-04-21T08:53:20.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d1ffff640 1 -- 172.21.15.82:0/3378834987 learned_addr learned my addr 172.21.15.82:0/3378834987 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.611+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d281011a0 0x7f3d28197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.612+0000 7f3d1ffff640 1 -- 172.21.15.82:0/3378834987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d281011a0 msgr2=0x7f3d28197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.612+0000 7f3d1ffff640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d281011a0 0x7f3d28197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.612+0000 7f3d1ffff640 1 -- 172.21.15.82:0/3378834987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d28107030 msgr2=0x7f3d2819bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.612+0000 7f3d1ffff640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d28107030 0x7f3d2819bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.612+0000 7f3d1ffff640 1 -- 172.21.15.82:0/3378834987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2819c670 con 0x7f3d28101b40 2024-04-21T08:53:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.612+0000 7f3d277fe640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d28107030 0x7f3d2819bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.612+0000 7f3d1ffff640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28101b40 0x7f3d28197bc0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000c890 tx=0x7f3d1000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.612+0000 7f3d24ff9640 1 -- 172.21.15.82:0/3378834987 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10004220 con 0x7f3d28101b40 2024-04-21T08:53:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.612+0000 7f3d24ff9640 1 -- 172.21.15.82:0/3378834987 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d100043c0 con 0x7f3d28101b40 2024-04-21T08:53:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.612+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/3378834987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d2819c960 con 0x7f3d28101b40 2024-04-21T08:53:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.612+0000 7f3d24ff9640 1 -- 172.21.15.82:0/3378834987 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10004a00 con 0x7f3d28101b40 2024-04-21T08:53:20.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.612+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/3378834987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281a41f0 con 0x7f3d28101b40 2024-04-21T08:53:20.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.614+0000 7f3d24ff9640 1 -- 172.21.15.82:0/3378834987 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3d100079b0 con 0x7f3d28101b40 2024-04-21T08:53:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.614+0000 7f3d3018f740 1 Processor -- start 2024-04-21T08:53:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d3018f740 1 -- start start 2024-04-21T08:53:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d3018f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0b470 0x55c351d0b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d3018f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 0x55c351d16640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d3018f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 0x55c351d19020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d3018f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c351c701f0 con 0x55c351d0be20 2024-04-21T08:53:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d3018f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c351c0d4b0 con 0x55c351d16c10 2024-04-21T08:53:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d3018f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c351c6c1b0 con 0x55c351d0b470 2024-04-21T08:53:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 0x55c351d16640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d24ff9640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3d140a3d20 0x7f3d140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 0x55c351d16640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37104/0 (socket says 172.21.15.82:37104) 2024-04-21T08:53:20.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d26ffd640 1 -- 172.21.15.82:0/3276561860 learned_addr learned my addr 172.21.15.82:0/3276561860 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d1ffff640 1 --2- 172.21.15.82:0/3276561860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 0x55c351d19020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d277fe640 1 --2- 172.21.15.82:0/3276561860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0b470 0x55c351d0b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d277fe640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3d140a3d20 0x7f3d140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d26ffd640 1 -- 172.21.15.82:0/3276561860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0b470 msgr2=0x55c351d0b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/3276561860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0b470 0x55c351d0b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d26ffd640 1 -- 172.21.15.82:0/3276561860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 msgr2=0x55c351d19020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/3276561860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 0x55c351d19020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d26ffd640 1 -- 172.21.15.82:0/3276561860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c351bff1b0 con 0x55c351d0be20 2024-04-21T08:53:20.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d277fe640 1 --2- 172.21.15.82:0/3276561860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0b470 0x55c351d0b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:20.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.615+0000 7f3d24ff9640 1 -- 172.21.15.82:0/3378834987 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f3d100d0860 con 0x7f3d28101b40 2024-04-21T08:53:20.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.616+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/3276561860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 0x55c351d16640 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3d18036040 tx=0x7f3d1802c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d71faacd63b6a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.616+0000 7f3d277fe640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3d140a3d20 0x7f3d140a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009990 tx=0x7f3d200096a0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.616+0000 7f3d1d7fa640 1 -- 172.21.15.82:0/3276561860 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18045020 con 0x55c351d0be20 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.616+0000 7f3d1d7fa640 1 -- 172.21.15.82:0/3276561860 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d1802c7e0 con 0x55c351d0be20 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.616+0000 7f3d1d7fa640 1 -- 172.21.15.82:0/3276561860 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18005570 con 0x55c351d0be20 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.616+0000 7f3d3018f740 1 -- 172.21.15.82:0/3276561860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 msgr2=0x55c351d16640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.616+0000 7f3d3018f740 1 --2- 172.21.15.82:0/3276561860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 0x55c351d16640 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f3d18036040 tx=0x7f3d1802c680 comp rx=0 tx=0).stop 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 -- 172.21.15.82:0/3276561860 shutdown_connections 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 --2- 172.21.15.82:0/3276561860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 0x55c351d19020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 --2- 172.21.15.82:0/3276561860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 0x55c351d16640 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 --2- 172.21.15.82:0/3276561860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0b470 0x55c351d0b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 -- 172.21.15.82:0/3276561860 >> 172.21.15.82:0/3276561860 conn(0x55c351d05ba0 msgr2=0x55c351d05f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 -- 172.21.15.82:0/3276561860 shutdown_connections 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 -- 172.21.15.82:0/3276561860 wait complete. 2024-04-21T08:53:20.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 -- start start 2024-04-21T08:53:20.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 Processor -- start 2024-04-21T08:53:20.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 0x55c351d164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0be20 0x55c351d13830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 0x55c351d13d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55c351d19ae0 con 0x55c351d0b470 2024-04-21T08:53:20.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c351c29500 con 0x55c351d16c10 2024-04-21T08:53:20.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c351d19c60 con 0x55c351d0be20 2024-04-21T08:53:20.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d3018f740 10 client.0 _ll_register_callbacks cb 0x55c351bf7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:20.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 0x55c351d164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 0x55c351d164a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37118/0 (socket says 172.21.15.82:37118) 2024-04-21T08:53:20.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d26ffd640 1 -- 172.21.15.82:0/871978678 learned_addr learned my addr 172.21.15.82:0/871978678 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d1ffff640 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0be20 0x55c351d13830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.617+0000 7f3d277fe640 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 0x55c351d13d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d26ffd640 1 -- 172.21.15.82:0/871978678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0be20 msgr2=0x55c351d13830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0be20 0x55c351d13830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d26ffd640 1 -- 172.21.15.82:0/871978678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 msgr2=0x55c351d13d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 0x55c351d13d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d26ffd640 1 -- 172.21.15.82:0/871978678 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3519823f0 con 0x55c351d0b470 2024-04-21T08:53:20.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 0x55c351d164a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009900 tx=0x7f3d180055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeabbe3de0017848 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d0e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:20.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d0e7fc640 1 -- 172.21.15.82:0/871978678 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1804e070 con 0x55c351d0b470 2024-04-21T08:53:20.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d0e7fc640 1 -- 172.21.15.82:0/871978678 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f3d180035f0 con 0x55c351d0b470 2024-04-21T08:53:20.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d3018f740 1 -- 172.21.15.82:0/871978678 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d1802c7e0 con 0x55c351d0b470 2024-04-21T08:53:20.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d0e7fc640 1 -- 172.21.15.82:0/871978678 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3d18045030 con 0x55c351d0b470 2024-04-21T08:53:20.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d3018f740 10 client.17298 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:20.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d3018f740 1 -- 172.21.15.82:0/871978678 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c351a4c7d0 con 0x55c351d0b470 2024-04-21T08:53:20.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d0e7fc640 1 -- 172.21.15.82:0/871978678 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d1802c7e0 con 0x55c351d0b470 2024-04-21T08:53:20.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d0e7fc640 1 -- 172.21.15.82:0/871978678 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f3d18004b00 con 0x55c351d0b470 2024-04-21T08:53:20.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d3018f740 10 client.17298 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:20.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d3018f740 10 client.17298 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:53:20.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d3018f740 20 client.17298 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:20.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d3018f740 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55c351d26bf0 0x55c351d28fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d3018f740 4 client.17298 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:20.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.618+0000 7f3d3018f740 1 -- 172.21.15.82:0/871978678 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c351c0f930 con 0x55c351d26bf0 2024-04-21T08:53:20.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.619+0000 7f3d1ffff640 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55c351d26bf0 0x55c351d28fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.619+0000 7f3d0e7fc640 10 client.17298 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:53:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.619+0000 7f3d1ffff640 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55c351d26bf0 0x55c351d28fe0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8b63cd2fdcdb3a66 server_cookie=d3f13237b06e9614 in_seq=0 out_seq=0 2024-04-21T08:53:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d0e7fc640 1 -- 172.21.15.82:0/871978678 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c351c0f930 con 0x55c351d26bf0 2024-04-21T08:53:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d0e7fc640 10 client.17298 handle_command_reply: tid=0 2024-04-21T08:53:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d3018f740 1 client.17298 shutdown 2024-04-21T08:53:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d3018f740 1 -- 172.21.15.82:0/871978678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 msgr2=0x55c351d164a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d3018f740 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 0x55c351d164a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009900 tx=0x7f3d180055a0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d3018f740 1 -- 172.21.15.82:0/871978678 shutdown_connections 2024-04-21T08:53:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d3018f740 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55c351d26bf0 0x55c351d28fe0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d3018f740 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 0x55c351d13d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d3018f740 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0be20 0x55c351d13830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d3018f740 1 --2- 172.21.15.82:0/871978678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 0x55c351d164a0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d0e7fc640 0 client.17298 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:53:20.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d3018f740 1 -- 172.21.15.82:0/871978678 >> 172.21.15.82:0/871978678 conn(0x55c351d05ba0 msgr2=0x55c351d078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d3018f740 1 -- 172.21.15.82:0/871978678 shutdown_connections 2024-04-21T08:53:20.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d3018f740 1 -- 172.21.15.82:0/871978678 wait complete. 2024-04-21T08:53:20.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.620+0000 7f3d3018f740 20 client.17298 trim_cache size 0 max 16384 2024-04-21T08:53:20.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:20 smithi106 ceph-mon[31072]: pgmap v192: 129 pgs: 129 active+clean; 883 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.8 MiB/s wr, 756 op/s 2024-04-21T08:53:20.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.633+0000 7f3d3018f740 1 Processor -- start 2024-04-21T08:53:20.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.633+0000 7f3d3018f740 1 -- start start 2024-04-21T08:53:20.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d3018f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 0x55c351d2c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d3018f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0be20 0x55c351d04a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d3018f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 0x55c351d15f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d3018f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281ad120 con 0x55c351d0b470 2024-04-21T08:53:20.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d3018f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281aecf0 con 0x55c351d16c10 2024-04-21T08:53:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d3018f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281ad7f0 con 0x55c351d0be20 2024-04-21T08:53:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 0x55c351d2c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 0x55c351d2c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37132/0 (socket says 172.21.15.82:37132) 2024-04-21T08:53:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d26ffd640 1 -- 172.21.15.82:0/2966396280 learned_addr learned my addr 172.21.15.82:0/2966396280 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d277fe640 1 --2- 172.21.15.82:0/2966396280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 0x55c351d15f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d26ffd640 1 -- 172.21.15.82:0/2966396280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0be20 msgr2=0x55c351d04a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/2966396280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0be20 0x55c351d04a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d26ffd640 1 -- 172.21.15.82:0/2966396280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 msgr2=0x55c351d15f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/2966396280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 0x55c351d15f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d26ffd640 1 -- 172.21.15.82:0/2966396280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c351d11d80 con 0x55c351d0b470 2024-04-21T08:53:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/2966396280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 0x55c351d2c3f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3d180051c0 tx=0x7f3d18005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbc3c8b7f67d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d1cff9640 1 -- 172.21.15.82:0/2966396280 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1804e040 con 0x55c351d0b470 2024-04-21T08:53:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d1cff9640 1 -- 172.21.15.82:0/2966396280 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d1802c7e0 con 0x55c351d0b470 2024-04-21T08:53:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.634+0000 7f3d1cff9640 1 -- 172.21.15.82:0/2966396280 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18003850 con 0x55c351d0b470 2024-04-21T08:53:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.635+0000 7f3d3018f740 1 -- 172.21.15.82:0/2966396280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 msgr2=0x55c351d2c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.635+0000 7f3d3018f740 1 --2- 172.21.15.82:0/2966396280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 0x55c351d2c3f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3d180051c0 tx=0x7f3d18005fd0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.635+0000 7f3d3018f740 1 -- 172.21.15.82:0/2966396280 shutdown_connections 2024-04-21T08:53:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.635+0000 7f3d3018f740 1 --2- 172.21.15.82:0/2966396280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d16c10 0x55c351d15f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.635+0000 7f3d3018f740 1 --2- 172.21.15.82:0/2966396280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d0be20 0x55c351d04a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.635+0000 7f3d3018f740 1 --2- 172.21.15.82:0/2966396280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0b470 0x55c351d2c3f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.635+0000 7f3d3018f740 1 -- 172.21.15.82:0/2966396280 >> 172.21.15.82:0/2966396280 conn(0x55c351d05ba0 msgr2=0x55c351d04370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.635+0000 7f3d3018f740 1 -- 172.21.15.82:0/2966396280 shutdown_connections 2024-04-21T08:53:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.635+0000 7f3d3018f740 1 -- 172.21.15.82:0/2966396280 wait complete. 2024-04-21T08:53:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d3018f740 1 -- start start 2024-04-21T08:53:20.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d3018f740 1 Processor -- start 2024-04-21T08:53:20.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d3018f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d0b470 0x55c351d87550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d3018f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 0x55c351d87a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d3018f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d16c10 0x55c351d0e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d3018f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281ae620 con 0x55c351d0be20 2024-04-21T08:53:20.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d3018f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281adf50 con 0x55c351d0b470 2024-04-21T08:53:20.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d3018f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281ab100 con 0x55c351d16c10 2024-04-21T08:53:20.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d3018f740 10 client.0 _ll_register_callbacks cb 0x7f3d1802cd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:20.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d1ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 0x55c351d87a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d1ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 0x55c351d87a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37142/0 (socket says 172.21.15.82:37142) 2024-04-21T08:53:20.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d1ffff640 1 -- 172.21.15.82:0/1375961569 learned_addr learned my addr 172.21.15.82:0/1375961569 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d26ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d0b470 0x55c351d87550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.636+0000 7f3d277fe640 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d16c10 0x55c351d0e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d1ffff640 1 -- 172.21.15.82:0/1375961569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d16c10 msgr2=0x55c351d0e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d1ffff640 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d16c10 0x55c351d0e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d1ffff640 1 -- 172.21.15.82:0/1375961569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d0b470 msgr2=0x55c351d87550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d1ffff640 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d0b470 0x55c351d87550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d1ffff640 1 -- 172.21.15.82:0/1375961569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c351d0e940 con 0x55c351d0be20 2024-04-21T08:53:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d0b470 0x55c351d87550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d1ffff640 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 0x55c351d87a90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3d100ce1f0 tx=0x7f3d1009bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2707707a16e02e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d0e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d0e7fc640 1 -- 172.21.15.82:0/1375961569 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1001d040 con 0x55c351d0be20 2024-04-21T08:53:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d0e7fc640 1 -- 172.21.15.82:0/1375961569 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f3d100cde70 con 0x55c351d0be20 2024-04-21T08:53:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d3018f740 1 -- 172.21.15.82:0/1375961569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d1802c7e0 con 0x55c351d0be20 2024-04-21T08:53:20.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d0e7fc640 1 -- 172.21.15.82:0/1375961569 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3d100ba050 con 0x55c351d0be20 2024-04-21T08:53:20.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d3018f740 10 client.17304 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:20.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.637+0000 7f3d3018f740 1 -- 172.21.15.82:0/1375961569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c351d8ed70 con 0x55c351d0be20 2024-04-21T08:53:20.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.638+0000 7f3d0e7fc640 1 -- 172.21.15.82:0/1375961569 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d1802c7e0 con 0x55c351d0be20 2024-04-21T08:53:20.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.638+0000 7f3d0e7fc640 1 -- 172.21.15.82:0/1375961569 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f3d100b5bc0 con 0x55c351d0be20 2024-04-21T08:53:20.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.638+0000 7f3d3018f740 10 client.17304 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:20.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.638+0000 7f3d3018f740 10 client.17304 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:53:20.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.638+0000 7f3d3018f740 20 client.17304 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:20.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.638+0000 7f3d3018f740 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55c351d8f520 0x55c351d91910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.638+0000 7f3d3018f740 4 client.17304 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:53:20.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.638+0000 7f3d3018f740 1 -- 172.21.15.82:0/1375961569 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c351d14100 con 0x55c351d8f520 2024-04-21T08:53:20.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.638+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55c351d8f520 0x55c351d91910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.638+0000 7f3d26ffd640 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55c351d8f520 0x55c351d91910 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c64fbab31f98eca6 server_cookie=5ef4986a1cb3f649 in_seq=0 out_seq=0 2024-04-21T08:53:20.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.638+0000 7f3d0e7fc640 10 client.17304 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:53:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.641+0000 7f3d0e7fc640 1 -- 172.21.15.82:0/1375961569 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c351d14100 con 0x55c351d8f520 2024-04-21T08:53:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.641+0000 7f3d0e7fc640 10 client.17304 handle_command_reply: tid=0 2024-04-21T08:53:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.641+0000 7f3d3018f740 1 client.17304 shutdown 2024-04-21T08:53:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d3018f740 1 -- 172.21.15.82:0/1375961569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 msgr2=0x55c351d87a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d3018f740 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 0x55c351d87a90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3d100ce1f0 tx=0x7f3d1009bf70 comp rx=0 tx=0).stop 2024-04-21T08:53:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d3018f740 1 -- 172.21.15.82:0/1375961569 shutdown_connections 2024-04-21T08:53:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d3018f740 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55c351d8f520 0x55c351d91910 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d3018f740 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c351d16c10 0x55c351d0e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d3018f740 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c351d0be20 0x55c351d87a90 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d0e7fc640 0 client.17304 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:53:20.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d3018f740 1 --2- 172.21.15.82:0/1375961569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c351d0b470 0x55c351d87550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d3018f740 1 -- 172.21.15.82:0/1375961569 >> 172.21.15.82:0/1375961569 conn(0x55c351d05ba0 msgr2=0x55c351d04c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d3018f740 1 -- 172.21.15.82:0/1375961569 shutdown_connections 2024-04-21T08:53:20.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d3018f740 1 -- 172.21.15.82:0/1375961569 wait complete. 2024-04-21T08:53:20.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d3018f740 20 client.17304 trim_cache size 0 max 16384 2024-04-21T08:53:20.646 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:53:20.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/3378834987 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3d140a3d20 msgr2=0x7f3d140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d2d4c1640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3d140a3d20 0x7f3d140a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009990 tx=0x7f3d200096a0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/3378834987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28101b40 msgr2=0x7f3d28197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.642+0000 7f3d2d4c1640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28101b40 0x7f3d28197bc0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000c890 tx=0x7f3d1000cd50 comp rx=0 tx=0).stop 2024-04-21T08:53:20.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.643+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/3378834987 shutdown_connections 2024-04-21T08:53:20.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.643+0000 7f3d2d4c1640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d28107030 0x7f3d2819bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.643+0000 7f3d2d4c1640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3d140a3d20 0x7f3d140a61e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.643+0000 7f3d2d4c1640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d28101b40 0x7f3d28197bc0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.643+0000 7f3d2d4c1640 1 --2- 172.21.15.82:0/3378834987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d281011a0 0x7f3d28197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.643+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/3378834987 >> 172.21.15.82:0/3378834987 conn(0x7f3d28076c50 msgr2=0x7f3d280ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.643+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/3378834987 shutdown_connections 2024-04-21T08:53:20.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.643+0000 7f3d2d4c1640 1 -- 172.21.15.82:0/3378834987 wait complete. 2024-04-21T08:53:20.662 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:53:20.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:20 smithi134 ceph-mon[31772]: pgmap v192: 129 pgs: 129 active+clean; 883 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.8 MiB/s wr, 756 op/s 2024-04-21T08:53:20.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.812+0000 7f29afddb640 1 -- 172.21.15.82:0/779463525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29a8101080 msgr2=0x7f29a8106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.812+0000 7f29afddb640 1 --2- 172.21.15.82:0/779463525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29a8101080 0x7f29a8106bb0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009900 tx=0x7f29a402bc60 comp rx=0 tx=0).stop 2024-04-21T08:53:20.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.814+0000 7f29afddb640 1 -- 172.21.15.82:0/779463525 shutdown_connections 2024-04-21T08:53:20.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.814+0000 7f29afddb640 1 --2- 172.21.15.82:0/779463525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a81072e0 0x7f29a810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.814+0000 7f29afddb640 1 --2- 172.21.15.82:0/779463525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29a8101080 0x7f29a8106bb0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.814+0000 7f29afddb640 1 --2- 172.21.15.82:0/779463525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29a81006d0 0x7f29a8100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.814+0000 7f29afddb640 1 -- 172.21.15.82:0/779463525 >> 172.21.15.82:0/779463525 conn(0x7f29a80773f0 msgr2=0x7f29a80fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.816+0000 7f29afddb640 1 -- 172.21.15.82:0/779463525 shutdown_connections 2024-04-21T08:53:20.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.819+0000 7f29afddb640 1 -- 172.21.15.82:0/779463525 wait complete. 2024-04-21T08:53:20.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.820+0000 7f29afddb640 1 Processor -- start 2024-04-21T08:53:20.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.820+0000 7f29afddb640 1 -- start start 2024-04-21T08:53:20.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.820+0000 7f29afddb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29a81006d0 0x7f29a819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.820+0000 7f29afddb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29a81072e0 0x7f29a819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.821+0000 7f29afddb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8198750 0x7f29a8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.821+0000 7f29afddb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8112300 con 0x7f29a81006d0 2024-04-21T08:53:20.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.821+0000 7f29afddb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8112180 con 0x7f29a81072e0 2024-04-21T08:53:20.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.821+0000 7f29afddb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8112480 con 0x7f29a8198750 2024-04-21T08:53:20.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.821+0000 7f29ae378640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8198750 0x7f29a8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.821+0000 7f29ae378640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8198750 0x7f29a8198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:48024/0 (socket says 172.21.15.82:48024) 2024-04-21T08:53:20.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.821+0000 7f29ae378640 1 -- 172.21.15.82:0/2510139369 learned_addr learned my addr 172.21.15.82:0/2510139369 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.821+0000 7f29ad376640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29a81072e0 0x7f29a819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.822+0000 7f29ae378640 1 -- 172.21.15.82:0/2510139369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29a81072e0 msgr2=0x7f29a819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.822+0000 7f29ae378640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29a81072e0 0x7f29a819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.822+0000 7f29ae378640 1 -- 172.21.15.82:0/2510139369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29a81006d0 msgr2=0x7f29a819e690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:53:20.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.822+0000 7f29ae378640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29a81006d0 0x7f29a819e690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.822+0000 7f29ae378640 1 -- 172.21.15.82:0/2510139369 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a8199350 con 0x7f29a8198750 2024-04-21T08:53:20.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.822+0000 7f29ad376640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29a81072e0 0x7f29a819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:20.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.822+0000 7f29ae378640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8198750 0x7f29a8198bd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f29a000b970 tx=0x7f29a000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.823+0000 7f299effd640 1 -- 172.21.15.82:0/2510139369 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0019070 con 0x7f29a8198750 2024-04-21T08:53:20.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.823+0000 7f299effd640 1 -- 172.21.15.82:0/2510139369 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a0014440 con 0x7f29a8198750 2024-04-21T08:53:20.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.824+0000 7f29afddb640 1 -- 172.21.15.82:0/2510139369 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a81a5430 con 0x7f29a8198750 2024-04-21T08:53:20.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.824+0000 7f299effd640 1 -- 172.21.15.82:0/2510139369 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00075d0 con 0x7f29a8198750 2024-04-21T08:53:20.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.824+0000 7f29afddb640 1 -- 172.21.15.82:0/2510139369 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a8101570 con 0x7f29a8198750 2024-04-21T08:53:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.827+0000 7f29b29cf740 1 Processor -- start 2024-04-21T08:53:20.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.828+0000 7f29b29cf740 1 -- start start 2024-04-21T08:53:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.828+0000 7f29b29cf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5a89850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.828+0000 7f29b29cf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 0x5586d5a9ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.828+0000 7f29b29cf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a9d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.828+0000 7f29b29cf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5586d59ee1f0 con 0x5586d5a9b210 2024-04-21T08:53:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.828+0000 7f29b29cf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5586d598b4b0 con 0x5586d5a89470 2024-04-21T08:53:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.828+0000 7f29b29cf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5586d59ea1b0 con 0x5586d5a89e20 2024-04-21T08:53:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.828+0000 7f29ad376640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5a89850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.829+0000 7f29ad376640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5a89850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:58918/0 (socket says 172.21.15.82:58918) 2024-04-21T08:53:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.829+0000 7f29ad376640 1 -- 172.21.15.82:0/3617668384 learned_addr learned my addr 172.21.15.82:0/3617668384 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.829+0000 7f29ad376640 1 --2- 172.21.15.82:0/3617668384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a9d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.829+0000 7f29adb77640 1 --2- 172.21.15.82:0/3617668384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 0x5586d5a9ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.829+0000 7f29ad376640 1 -- 172.21.15.82:0/3617668384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 msgr2=0x5586d5a9ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.829+0000 7f29ad376640 1 --2- 172.21.15.82:0/3617668384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 0x5586d5a9ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.829+0000 7f29ad376640 1 -- 172.21.15.82:0/3617668384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 msgr2=0x5586d5a9d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.829+0000 7f29ad376640 1 --2- 172.21.15.82:0/3617668384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a9d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.829+0000 7f29ad376640 1 -- 172.21.15.82:0/3617668384 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d597d1b0 con 0x5586d5a89470 2024-04-21T08:53:20.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.830+0000 7f299effd640 1 -- 172.21.15.82:0/2510139369 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f29a000cb40 con 0x7f29a8198750 2024-04-21T08:53:20.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.831+0000 7f299effd640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29940a3de0 0x7f29940a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.831+0000 7f299effd640 1 -- 172.21.15.82:0/2510139369 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f29a00d00a0 con 0x7f29a8198750 2024-04-21T08:53:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.831+0000 7f29ad376640 1 --2- 172.21.15.82:0/3617668384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5a89850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f29a80689f0 tx=0x7f29a4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47534190389ef362 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.832+0000 7f29adb77640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29940a3de0 0x7f29940a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.832+0000 7f2993fff640 1 -- 172.21.15.82:0/3617668384 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4049070 con 0x5586d5a89470 2024-04-21T08:53:20.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.832+0000 7f2993fff640 1 -- 172.21.15.82:0/3617668384 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a4005500 con 0x5586d5a89470 2024-04-21T08:53:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.833+0000 7f29b29cf740 1 -- 172.21.15.82:0/3617668384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 msgr2=0x5586d5a89850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.833+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3617668384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5a89850 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f29a80689f0 tx=0x7f29a4005fd0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.834+0000 7f29adb77640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29940a3de0 0x7f29940a62a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2998006fd0 tx=0x7f2998008040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.837+0000 7f29b29cf740 1 -- 172.21.15.82:0/3617668384 shutdown_connections 2024-04-21T08:53:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.837+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3617668384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a9d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.837+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3617668384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 0x5586d5a9ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.837+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3617668384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5a89850 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.837+0000 7f29b29cf740 1 -- 172.21.15.82:0/3617668384 >> 172.21.15.82:0/3617668384 conn(0x5586d5a83ba0 msgr2=0x5586d5a83f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.837+0000 7f29b29cf740 1 -- 172.21.15.82:0/3617668384 shutdown_connections 2024-04-21T08:53:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.837+0000 7f29b29cf740 1 -- 172.21.15.82:0/3617668384 wait complete. 2024-04-21T08:53:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.837+0000 7f29b29cf740 1 -- start start 2024-04-21T08:53:20.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.837+0000 7f29b29cf740 1 Processor -- start 2024-04-21T08:53:20.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.838+0000 7f29b29cf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5a9ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.838+0000 7f29b29cf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 0x5586d5a97ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.838+0000 7f29b29cf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a98420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.838+0000 7f29b29cf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5586d5a9e130 con 0x5586d5a9b210 2024-04-21T08:53:20.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.838+0000 7f29b29cf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5586d59a7500 con 0x5586d5a89470 2024-04-21T08:53:20.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.838+0000 7f29b29cf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5586d5a9e2b0 con 0x5586d5a89e20 2024-04-21T08:53:20.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.838+0000 7f29b29cf740 10 client.0 _ll_register_callbacks cb 0x5586d5975b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:20.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.839+0000 7f29adb77640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5a9ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.839+0000 7f29adb77640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5a9ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:58922/0 (socket says 172.21.15.82:58922) 2024-04-21T08:53:20.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.839+0000 7f29adb77640 1 -- 172.21.15.82:0/1473082767 learned_addr learned my addr 172.21.15.82:0/1473082767 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.839+0000 7f29ae378640 1 --2- 172.21.15.82:0/1473082767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a98420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.839+0000 7f29adb77640 1 -- 172.21.15.82:0/1473082767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 msgr2=0x5586d5a97ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:53:20.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.839+0000 7f29adb77640 1 --2- 172.21.15.82:0/1473082767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 0x5586d5a97ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.839+0000 7f29adb77640 1 -- 172.21.15.82:0/1473082767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 msgr2=0x5586d5a98420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.839+0000 7f29adb77640 1 --2- 172.21.15.82:0/1473082767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a98420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.839+0000 7f29adb77640 1 -- 172.21.15.82:0/1473082767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d57003f0 con 0x5586d5a89470 2024-04-21T08:53:20.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.840+0000 7f29adb77640 1 --2- 172.21.15.82:0/1473082767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5a9ab10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f299800bfa0 tx=0x7f2998004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa5f03efdea08f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.840+0000 7f29917fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:53:20.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.841+0000 7f29917fa640 1 -- 172.21.15.82:0/1473082767 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998044020 con 0x5586d5a89470 2024-04-21T08:53:20.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.841+0000 7f29917fa640 1 -- 172.21.15.82:0/1473082767 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f2998040400 con 0x5586d5a89470 2024-04-21T08:53:20.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.841+0000 7f29b29cf740 1 -- 172.21.15.82:0/1473082767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29a4005500 con 0x5586d5a89470 2024-04-21T08:53:20.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.842+0000 7f29917fa640 1 -- 172.21.15.82:0/1473082767 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f299804f050 con 0x5586d5a89470 2024-04-21T08:53:20.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.842+0000 7f29b29cf740 10 client.26684 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:20.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.843+0000 7f29b29cf740 1 -- 172.21.15.82:0/1473082767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5586d57ca7d0 con 0x5586d5a89470 2024-04-21T08:53:20.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.843+0000 7f29917fa640 1 -- 172.21.15.82:0/1473082767 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a4005500 con 0x5586d5a89470 2024-04-21T08:53:20.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.844+0000 7f29917fa640 1 -- 172.21.15.82:0/1473082767 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f2998013050 con 0x5586d5a89470 2024-04-21T08:53:20.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.844+0000 7f29b29cf740 10 client.26684 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:20.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.844+0000 7f29b29cf740 10 client.26684 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:53:20.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.844+0000 7f29b29cf740 20 client.26684 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:20.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.844+0000 7f29b29cf740 1 --2- 172.21.15.82:0/1473082767 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5586d5a8e680 0x5586d5a90b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.844+0000 7f29b29cf740 4 client.26684 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:20.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.844+0000 7f29b29cf740 1 -- 172.21.15.82:0/1473082767 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586d598d930 con 0x5586d5a8e680 2024-04-21T08:53:20.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.844+0000 7f29ad376640 1 --2- 172.21.15.82:0/1473082767 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5586d5a8e680 0x5586d5a90b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.847+0000 7f29ad376640 1 --2- 172.21.15.82:0/1473082767 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5586d5a8e680 0x5586d5a90b20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f71899130374c816 server_cookie=d2edb88c9e2aa6f6 in_seq=0 out_seq=0 2024-04-21T08:53:20.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.847+0000 7f29917fa640 10 client.26684 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:53:20.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.849+0000 7f29917fa640 1 -- 172.21.15.82:0/1473082767 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586d598d930 con 0x5586d5a8e680 2024-04-21T08:53:20.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.849+0000 7f29917fa640 10 client.26684 handle_command_reply: tid=0 2024-04-21T08:53:20.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.850+0000 7f29b29cf740 1 client.26684 shutdown 2024-04-21T08:53:20.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.850+0000 7f29b29cf740 1 -- 172.21.15.82:0/1473082767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 msgr2=0x5586d5a9ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.850+0000 7f29b29cf740 1 --2- 172.21.15.82:0/1473082767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5a9ab10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f299800bfa0 tx=0x7f2998004770 comp rx=0 tx=0).stop 2024-04-21T08:53:20.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.852+0000 7f29b29cf740 1 -- 172.21.15.82:0/1473082767 shutdown_connections 2024-04-21T08:53:20.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.852+0000 7f29b29cf740 1 --2- 172.21.15.82:0/1473082767 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5586d5a8e680 0x5586d5a90b20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.852+0000 7f29b29cf740 1 --2- 172.21.15.82:0/1473082767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a98420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.852+0000 7f29b29cf740 1 --2- 172.21.15.82:0/1473082767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 0x5586d5a97ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.852+0000 7f29b29cf740 1 --2- 172.21.15.82:0/1473082767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5a9ab10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.852+0000 7f29b29cf740 1 -- 172.21.15.82:0/1473082767 >> 172.21.15.82:0/1473082767 conn(0x5586d5a83ba0 msgr2=0x5586d5a858f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.852+0000 7f29917fa640 0 client.26684 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:53:20.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.853+0000 7f29b29cf740 1 -- 172.21.15.82:0/1473082767 shutdown_connections 2024-04-21T08:53:20.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.854+0000 7f29b29cf740 1 -- 172.21.15.82:0/1473082767 wait complete. 2024-04-21T08:53:20.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.854+0000 7f29b29cf740 20 client.26684 trim_cache size 0 max 16384 2024-04-21T08:53:20.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.886+0000 7f29b29cf740 1 Processor -- start 2024-04-21T08:53:20.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.888+0000 7f29b29cf740 1 -- start start 2024-04-21T08:53:20.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.888+0000 7f29b29cf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89470 0x5586d5aa3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.888+0000 7f29b29cf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89e20 0x5586d5aa3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.888+0000 7f29b29cf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a96770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.888+0000 7f29b29cf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81ad3f0 con 0x5586d5a9b210 2024-04-21T08:53:20.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.889+0000 7f29b29cf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81aefc0 con 0x5586d5a89e20 2024-04-21T08:53:20.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.889+0000 7f29ad376640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89e20 0x5586d5aa3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.889+0000 7f29ad376640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89e20 0x5586d5aa3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:58938/0 (socket says 172.21.15.82:58938) 2024-04-21T08:53:20.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.889+0000 7f29ad376640 1 -- 172.21.15.82:0/3247360561 learned_addr learned my addr 172.21.15.82:0/3247360561 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.889+0000 7f29b29cf740 1 -- 172.21.15.82:0/3247360561 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81adac0 con 0x5586d5a89470 2024-04-21T08:53:20.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.889+0000 7f29ae378640 1 --2- 172.21.15.82:0/3247360561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a96770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.889+0000 7f29adb77640 1 --2- 172.21.15.82:0/3247360561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89470 0x5586d5aa3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.889+0000 7f29ae378640 1 -- 172.21.15.82:0/3247360561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89470 msgr2=0x5586d5aa3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.889+0000 7f29ae378640 1 --2- 172.21.15.82:0/3247360561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89470 0x5586d5aa3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.889+0000 7f29ae378640 1 -- 172.21.15.82:0/3247360561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89e20 msgr2=0x5586d5aa3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.889+0000 7f29ae378640 1 --2- 172.21.15.82:0/3247360561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89e20 0x5586d5aa3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.889+0000 7f29ae378640 1 -- 172.21.15.82:0/3247360561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d5a89bc0 con 0x5586d5a9b210 2024-04-21T08:53:20.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.890+0000 7f29ae378640 1 --2- 172.21.15.82:0/3247360561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a96770 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f29a00cd4e0 tx=0x7f29a0014d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72279ac943d3a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.891+0000 7f29937fe640 1 -- 172.21.15.82:0/3247360561 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00a49a0 con 0x5586d5a9b210 2024-04-21T08:53:20.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.891+0000 7f29937fe640 1 -- 172.21.15.82:0/3247360561 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a00b5070 con 0x5586d5a9b210 2024-04-21T08:53:20.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.891+0000 7f29937fe640 1 -- 172.21.15.82:0/3247360561 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00cd650 con 0x5586d5a9b210 2024-04-21T08:53:20.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.892+0000 7f29b29cf740 1 -- 172.21.15.82:0/3247360561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 msgr2=0x5586d5a96770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.892+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3247360561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a96770 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f29a00cd4e0 tx=0x7f29a0014d00 comp rx=0 tx=0).stop 2024-04-21T08:53:20.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.893+0000 7f29b29cf740 1 -- 172.21.15.82:0/3247360561 shutdown_connections 2024-04-21T08:53:20.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.893+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3247360561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5a96770 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.893+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3247360561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89e20 0x5586d5aa3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.893+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3247360561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89470 0x5586d5aa3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.893+0000 7f29b29cf740 1 -- 172.21.15.82:0/3247360561 >> 172.21.15.82:0/3247360561 conn(0x5586d5a83ba0 msgr2=0x5586d5a90690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.893+0000 7f29b29cf740 1 -- 172.21.15.82:0/3247360561 shutdown_connections 2024-04-21T08:53:20.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.894+0000 7f29b29cf740 1 -- 172.21.15.82:0/3247360561 wait complete. 2024-04-21T08:53:20.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.894+0000 7f29b29cf740 1 -- start start 2024-04-21T08:53:20.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.894+0000 7f29b29cf740 1 Processor -- start 2024-04-21T08:53:20.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29b29cf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5aa37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29b29cf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 0x5586d5aa0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29b29cf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5aa10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29b29cf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81ae8f0 con 0x5586d5a9b210 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29b29cf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81ae220 con 0x5586d5a89470 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29b29cf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81ab3f0 con 0x5586d5a89e20 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29b29cf740 10 client.0 _ll_register_callbacks cb 0x7f299800da90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29ae378640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5aa10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29adb77640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5aa37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29ae378640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5aa10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37192/0 (socket says 172.21.15.82:37192) 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29ae378640 1 -- 172.21.15.82:0/3005133518 learned_addr learned my addr 172.21.15.82:0/3005133518 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29ad376640 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 0x5586d5aa0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29ae378640 1 -- 172.21.15.82:0/3005133518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 msgr2=0x5586d5aa0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29ae378640 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 0x5586d5aa0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29ae378640 1 -- 172.21.15.82:0/3005133518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 msgr2=0x5586d5aa37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29ae378640 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5aa37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29ae378640 1 -- 172.21.15.82:0/3005133518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d5aa17a0 con 0x5586d5a9b210 2024-04-21T08:53:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.895+0000 7f29adb77640 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5aa37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.896+0000 7f29ae378640 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5aa10b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f29a00cd4b0 tx=0x7f29a009beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecbb81de61e6a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.896+0000 7f29917fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.896+0000 7f29917fa640 1 -- 172.21.15.82:0/3005133518 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00a4710 con 0x5586d5a9b210 2024-04-21T08:53:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.896+0000 7f29b29cf740 1 -- 172.21.15.82:0/3005133518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29a00b5070 con 0x5586d5a9b210 2024-04-21T08:53:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.896+0000 7f29917fa640 1 -- 172.21.15.82:0/3005133518 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f29a00bc0b0 con 0x5586d5a9b210 2024-04-21T08:53:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.896+0000 7f29917fa640 1 -- 172.21.15.82:0/3005133518 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f29a00bc2d0 con 0x5586d5a9b210 2024-04-21T08:53:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.896+0000 7f29b29cf740 10 client.17319 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.896+0000 7f29b29cf740 1 -- 172.21.15.82:0/3005133518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5586d5b08dd0 con 0x5586d5a9b210 2024-04-21T08:53:20.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.897+0000 7f29917fa640 1 -- 172.21.15.82:0/3005133518 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a00b5070 con 0x5586d5a9b210 2024-04-21T08:53:20.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.897+0000 7f29917fa640 1 -- 172.21.15.82:0/3005133518 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f29a00cdb20 con 0x5586d5a9b210 2024-04-21T08:53:20.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.897+0000 7f29b29cf740 10 client.17319 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:20.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.897+0000 7f29b29cf740 10 client.17319 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:53:20.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.897+0000 7f29b29cf740 20 client.17319 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:20.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.897+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5586d5b09480 0x5586d5b0b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:20.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.897+0000 7f29b29cf740 4 client.17319 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:53:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.897+0000 7f29b29cf740 1 -- 172.21.15.82:0/3005133518 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586d59fbc10 con 0x5586d5b09480 2024-04-21T08:53:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.898+0000 7f29adb77640 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5586d5b09480 0x5586d5b0b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.898+0000 7f29adb77640 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5586d5b09480 0x5586d5b0b8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46b2cd986ee5c6b3 server_cookie=e2e6ac6fa0eea19c in_seq=0 out_seq=0 2024-04-21T08:53:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.898+0000 7f29917fa640 10 client.17319 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:53:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.899+0000 7f29917fa640 1 -- 172.21.15.82:0/3005133518 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586d59fbc10 con 0x5586d5b09480 2024-04-21T08:53:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.899+0000 7f29917fa640 10 client.17319 handle_command_reply: tid=0 2024-04-21T08:53:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.899+0000 7f29b29cf740 1 client.17319 shutdown 2024-04-21T08:53:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.899+0000 7f29b29cf740 1 -- 172.21.15.82:0/3005133518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 msgr2=0x5586d5aa10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.899+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5aa10b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f29a00cd4b0 tx=0x7f29a009beb0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.900+0000 7f29b29cf740 1 -- 172.21.15.82:0/3005133518 shutdown_connections 2024-04-21T08:53:20.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.900+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5586d5b09480 0x5586d5b0b8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.900+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586d5a9b210 0x5586d5aa10b0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.900+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586d5a89e20 0x5586d5aa0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.900+0000 7f29b29cf740 1 --2- 172.21.15.82:0/3005133518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5586d5a89470 0x5586d5aa37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.900+0000 7f29b29cf740 1 -- 172.21.15.82:0/3005133518 >> 172.21.15.82:0/3005133518 conn(0x5586d5a83ba0 msgr2=0x5586d5a97730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.901+0000 7f29917fa640 0 client.17319 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:53:20.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.901+0000 7f29b29cf740 1 -- 172.21.15.82:0/3005133518 shutdown_connections 2024-04-21T08:53:20.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.902+0000 7f29b29cf740 1 -- 172.21.15.82:0/3005133518 wait complete. 2024-04-21T08:53:20.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.902+0000 7f29b29cf740 20 client.17319 trim_cache size 0 max 16384 2024-04-21T08:53:20.902 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:53:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.904+0000 7f29afddb640 1 -- 172.21.15.82:0/2510139369 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29940a3de0 msgr2=0x7f29940a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.904+0000 7f29afddb640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29940a3de0 0x7f29940a62a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2998006fd0 tx=0x7f2998008040 comp rx=0 tx=0).stop 2024-04-21T08:53:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.904+0000 7f29afddb640 1 -- 172.21.15.82:0/2510139369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8198750 msgr2=0x7f29a8198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.904+0000 7f29afddb640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8198750 0x7f29a8198bd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f29a000b970 tx=0x7f29a000be30 comp rx=0 tx=0).stop 2024-04-21T08:53:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.904+0000 7f29afddb640 1 -- 172.21.15.82:0/2510139369 shutdown_connections 2024-04-21T08:53:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.904+0000 7f29afddb640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8198750 0x7f29a8198bd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.904+0000 7f29afddb640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29a81072e0 0x7f29a819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.904+0000 7f29afddb640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29940a3de0 0x7f29940a62a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.904+0000 7f29afddb640 1 --2- 172.21.15.82:0/2510139369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29a81006d0 0x7f29a819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:20.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.904+0000 7f29afddb640 1 -- 172.21.15.82:0/2510139369 >> 172.21.15.82:0/2510139369 conn(0x7f29a80773f0 msgr2=0x7f29a81056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:20.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.904+0000 7f29afddb640 1 -- 172.21.15.82:0/2510139369 shutdown_connections 2024-04-21T08:53:20.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:20.904+0000 7f29afddb640 1 -- 172.21.15.82:0/2510139369 wait complete. 2024-04-21T08:53:20.923 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:53:21.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.092+0000 7f87c7cce640 1 -- 172.21.15.82:0/2144028102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87c006a360 msgr2=0x7f87c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.092+0000 7f87c7cce640 1 --2- 172.21.15.82:0/2144028102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87c006a360 0x7f87c006a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009900 tx=0x7f87bc02bc40 comp rx=0 tx=0).stop 2024-04-21T08:53:21.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.093+0000 7f87c7cce640 1 -- 172.21.15.82:0/2144028102 shutdown_connections 2024-04-21T08:53:21.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.093+0000 7f87c7cce640 1 --2- 172.21.15.82:0/2144028102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c006ace0 0x7f87c010ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.093+0000 7f87c7cce640 1 --2- 172.21.15.82:0/2144028102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87c006a360 0x7f87c006a7a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.093+0000 7f87c7cce640 1 --2- 172.21.15.82:0/2144028102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c0107fd0 0x7f87c01083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.093+0000 7f87c7cce640 1 -- 172.21.15.82:0/2144028102 >> 172.21.15.82:0/2144028102 conn(0x7f87c0077ef0 msgr2=0x7f87c0078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.094+0000 7f87c7cce640 1 -- 172.21.15.82:0/2144028102 shutdown_connections 2024-04-21T08:53:21.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.094+0000 7f87c7cce640 1 -- 172.21.15.82:0/2144028102 wait complete. 2024-04-21T08:53:21.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c7cce640 1 Processor -- start 2024-04-21T08:53:21.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c7cce640 1 -- start start 2024-04-21T08:53:21.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c7cce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c006a360 0x7f87c01a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c7cce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87c006ace0 0x7f87c01a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c7cce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c0107fd0 0x7f87c010cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c7cce640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f87c01121d0 con 0x7f87c0107fd0 2024-04-21T08:53:21.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c7cce640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f87c0112050 con 0x7f87c006ace0 2024-04-21T08:53:21.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c7cce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87c0112350 con 0x7f87c006a360 2024-04-21T08:53:21.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c5a6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c006a360 0x7f87c01a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c5a6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c006a360 0x7f87c01a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:48078/0 (socket says 172.21.15.82:48078) 2024-04-21T08:53:21.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c5a6a640 1 -- 172.21.15.82:0/2659217963 learned_addr learned my addr 172.21.15.82:0/2659217963 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c5269640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87c006ace0 0x7f87c01a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.095+0000 7f87c626b640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c0107fd0 0x7f87c010cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.096+0000 7f87c626b640 1 -- 172.21.15.82:0/2659217963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c006a360 msgr2=0x7f87c01a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.096+0000 7f87c626b640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c006a360 0x7f87c01a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.096+0000 7f87c626b640 1 -- 172.21.15.82:0/2659217963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87c006ace0 msgr2=0x7f87c01a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.096+0000 7f87c626b640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87c006ace0 0x7f87c01a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.096+0000 7f87c626b640 1 -- 172.21.15.82:0/2659217963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c010d530 con 0x7f87c0107fd0 2024-04-21T08:53:21.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.096+0000 7f87c5a6a640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c006a360 0x7f87c01a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:21.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.096+0000 7f87c5269640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87c006ace0 0x7f87c01a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:21.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.096+0000 7f87c626b640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c0107fd0 0x7f87c010cde0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f87b800e940 tx=0x7f87b800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.097+0000 7f87b6ffd640 1 -- 172.21.15.82:0/2659217963 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8004d20 con 0x7f87c0107fd0 2024-04-21T08:53:21.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.097+0000 7f87b6ffd640 1 -- 172.21.15.82:0/2659217963 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b8005020 con 0x7f87c0107fd0 2024-04-21T08:53:21.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.097+0000 7f87b6ffd640 1 -- 172.21.15.82:0/2659217963 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b80116d0 con 0x7f87c0107fd0 2024-04-21T08:53:21.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.097+0000 7f87c7cce640 1 -- 172.21.15.82:0/2659217963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c010d820 con 0x7f87c0107fd0 2024-04-21T08:53:21.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.097+0000 7f87c7cce640 1 -- 172.21.15.82:0/2659217963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c006b970 con 0x7f87c0107fd0 2024-04-21T08:53:21.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.101+0000 7f87ca995740 1 Processor -- start 2024-04-21T08:53:21.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.101+0000 7f87b6ffd640 1 -- 172.21.15.82:0/2659217963 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f87b800cb20 con 0x7f87c0107fd0 2024-04-21T08:53:21.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87ca995740 1 -- start start 2024-04-21T08:53:21.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87ca995740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f9654d0 0x55653f9658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87ca995740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f965e80 0x55653f96e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87ca995740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f96ea00 0x55653f970e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87ca995740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55653f8ca1f0 con 0x55653f96ea00 2024-04-21T08:53:21.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87ca995740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55653f8674b0 con 0x55653f9654d0 2024-04-21T08:53:21.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87ca995740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55653f8c61b0 con 0x55653f965e80 2024-04-21T08:53:21.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87c5a6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f965e80 0x55653f96e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87c5269640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f96ea00 0x55653f970e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87c5a6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f965e80 0x55653f96e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:48086/0 (socket says 172.21.15.82:48086) 2024-04-21T08:53:21.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87c5269640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f96ea00 0x55653f970e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37222/0 (socket says 172.21.15.82:37222) 2024-04-21T08:53:21.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87c5a6a640 1 -- 172.21.15.82:0/3077868351 learned_addr learned my addr 172.21.15.82:0/3077868351 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87c5269640 1 --2- 172.21.15.82:0/3077868351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f9654d0 0x55653f9658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.102+0000 7f87b6ffd640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f87ac0a3f20 0x7f87ac0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.103+0000 7f87c626b640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f87ac0a3f20 0x7f87ac0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.103+0000 7f87c5269640 1 -- 172.21.15.82:0/3077868351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f965e80 msgr2=0x55653f96e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.103+0000 7f87c5269640 1 --2- 172.21.15.82:0/3077868351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f965e80 0x55653f96e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.103+0000 7f87c5269640 1 -- 172.21.15.82:0/3077868351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f96ea00 msgr2=0x55653f970e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.103+0000 7f87c5269640 1 --2- 172.21.15.82:0/3077868351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f96ea00 0x55653f970e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.103+0000 7f87c5269640 1 -- 172.21.15.82:0/3077868351 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653f8591b0 con 0x55653f9654d0 2024-04-21T08:53:21.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.103+0000 7f87b6ffd640 1 -- 172.21.15.82:0/2659217963 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f87b80d0b80 con 0x7f87c0107fd0 2024-04-21T08:53:21.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.103+0000 7f87c626b640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f87ac0a3f20 0x7f87ac0a63e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f87b809c4b0 tx=0x7f87b8098b10 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.104+0000 7f87c5269640 1 --2- 172.21.15.82:0/3077868351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f9654d0 0x55653f9658b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f87bc02c250 tx=0x7f87bc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4aaef6591f0711 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.104+0000 7f87abfff640 1 -- 172.21.15.82:0/3077868351 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0041e0 con 0x55653f9654d0 2024-04-21T08:53:21.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.104+0000 7f87abfff640 1 -- 172.21.15.82:0/3077868351 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87bc004380 con 0x55653f9654d0 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.104+0000 7f87abfff640 1 -- 172.21.15.82:0/3077868351 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0436b0 con 0x55653f9654d0 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.105+0000 7f87ca995740 1 -- 172.21.15.82:0/3077868351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f9654d0 msgr2=0x55653f9658b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.105+0000 7f87ca995740 1 --2- 172.21.15.82:0/3077868351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f9654d0 0x55653f9658b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f87bc02c250 tx=0x7f87bc000bc0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.105+0000 7f87ca995740 1 -- 172.21.15.82:0/3077868351 shutdown_connections 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.105+0000 7f87ca995740 1 --2- 172.21.15.82:0/3077868351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f96ea00 0x55653f970e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.105+0000 7f87ca995740 1 --2- 172.21.15.82:0/3077868351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f965e80 0x55653f96e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.105+0000 7f87ca995740 1 --2- 172.21.15.82:0/3077868351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f9654d0 0x55653f9658b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.105+0000 7f87ca995740 1 -- 172.21.15.82:0/3077868351 >> 172.21.15.82:0/3077868351 conn(0x55653f95fba0 msgr2=0x55653f95ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.105+0000 7f87ca995740 1 -- 172.21.15.82:0/3077868351 shutdown_connections 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.105+0000 7f87ca995740 1 -- 172.21.15.82:0/3077868351 wait complete. 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87ca995740 1 -- start start 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87ca995740 1 Processor -- start 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87ca995740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 0x55653f96e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87ca995740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f965e80 0x55653f96b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87ca995740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f96ea00 0x55653f96bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87ca995740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55653f971910 con 0x55653f9654d0 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87ca995740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55653f883500 con 0x55653f965e80 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87ca995740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55653f971a90 con 0x55653f96ea00 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87ca995740 10 client.0 _ll_register_callbacks cb 0x55653f851b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87c5269640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 0x55653f96e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87c5a6a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f965e80 0x55653f96b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87c5269640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 0x55653f96e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37226/0 (socket says 172.21.15.82:37226) 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.106+0000 7f87c5269640 1 -- 172.21.15.82:0/2194465154 learned_addr learned my addr 172.21.15.82:0/2194465154 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87c5269640 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f96ea00 0x55653f96bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87c5a6a640 1 -- 172.21.15.82:0/2194465154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f96ea00 msgr2=0x55653f96bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87c5a6a640 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f96ea00 0x55653f96bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87c5a6a640 1 -- 172.21.15.82:0/2194465154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 msgr2=0x55653f96e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87c5a6a640 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 0x55653f96e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87c5a6a640 1 -- 172.21.15.82:0/2194465154 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653f5dc3f0 con 0x55653f965e80 2024-04-21T08:53:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87c5269640 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 0x55653f96e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87c5a6a640 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f965e80 0x55653f96b6d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55653f960c10 tx=0x7f87b00039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5125170c6f8568f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:53:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87a97fa640 1 -- 172.21.15.82:0/2194465154 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0011710 con 0x55653f965e80 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87a97fa640 1 -- 172.21.15.82:0/2194465154 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f87b00118b0 con 0x55653f965e80 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87ca995740 1 -- 172.21.15.82:0/2194465154 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87bc004380 con 0x55653f965e80 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87a97fa640 1 -- 172.21.15.82:0/2194465154 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f87b0024050 con 0x55653f965e80 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87ca995740 10 client.26702 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.107+0000 7f87ca995740 1 -- 172.21.15.82:0/2194465154 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55653f6a67d0 con 0x55653f965e80 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.108+0000 7f87a97fa640 1 -- 172.21.15.82:0/2194465154 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87bc004380 con 0x55653f965e80 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.108+0000 7f87a97fa640 1 -- 172.21.15.82:0/2194465154 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f87b001bc50 con 0x55653f965e80 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.108+0000 7f87ca995740 10 client.26702 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.108+0000 7f87ca995740 10 client.26702 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.108+0000 7f87ca995740 20 client.26702 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.108+0000 7f87ca995740 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55653f97ec80 0x55653f981070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.108+0000 7f87ca995740 4 client.26702 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.108+0000 7f87ca995740 1 -- 172.21.15.82:0/2194465154 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55653f869930 con 0x55653f97ec80 2024-04-21T08:53:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.108+0000 7f87c5269640 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55653f97ec80 0x55653f981070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.108+0000 7f87a97fa640 10 client.26702 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.108+0000 7f87c5269640 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55653f97ec80 0x55653f981070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4bd5783fa83e953d server_cookie=80b493b31bc99a4e in_seq=0 out_seq=0 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87a97fa640 1 -- 172.21.15.82:0/2194465154 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55653f869930 con 0x55653f97ec80 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87a97fa640 10 client.26702 handle_command_reply: tid=0 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87ca995740 1 client.26702 shutdown 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87ca995740 1 -- 172.21.15.82:0/2194465154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f965e80 msgr2=0x55653f96b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87ca995740 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f965e80 0x55653f96b6d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55653f960c10 tx=0x7f87b00039c0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87ca995740 1 -- 172.21.15.82:0/2194465154 shutdown_connections 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87ca995740 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55653f97ec80 0x55653f981070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87ca995740 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f96ea00 0x55653f96bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87ca995740 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f965e80 0x55653f96b6d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87a97fa640 0 client.26702 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87ca995740 1 --2- 172.21.15.82:0/2194465154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 0x55653f96e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87ca995740 1 -- 172.21.15.82:0/2194465154 >> 172.21.15.82:0/2194465154 conn(0x55653f95fba0 msgr2=0x55653f9618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87ca995740 1 -- 172.21.15.82:0/2194465154 shutdown_connections 2024-04-21T08:53:21.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.110+0000 7f87ca995740 1 -- 172.21.15.82:0/2194465154 wait complete. 2024-04-21T08:53:21.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.111+0000 7f87ca995740 20 client.26702 trim_cache size 0 max 16384 2024-04-21T08:53:21.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87ca995740 1 Processor -- start 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87ca995740 1 -- start start 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87ca995740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f9654d0 0x55653f961800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87ca995740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f965e80 0x55653f96de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87ca995740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f96ea00 0x55653f96e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87ca995740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f87c01be190 con 0x55653f965e80 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87ca995740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f87c01bfd60 con 0x55653f96ea00 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87ca995740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87c01be860 con 0x55653f9654d0 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87c5a6a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f965e80 0x55653f96de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87c5a6a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f965e80 0x55653f96de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37238/0 (socket says 172.21.15.82:37238) 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87c5a6a640 1 -- 172.21.15.82:0/2481350145 learned_addr learned my addr 172.21.15.82:0/2481350145 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87c5269640 1 --2- 172.21.15.82:0/2481350145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f96ea00 0x55653f96e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.124+0000 7f87c5269640 1 --2- 172.21.15.82:0/2481350145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f9654d0 0x55653f961800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.125+0000 7f87c5a6a640 1 -- 172.21.15.82:0/2481350145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f9654d0 msgr2=0x55653f961800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.125+0000 7f87c5a6a640 1 --2- 172.21.15.82:0/2481350145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f9654d0 0x55653f961800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.125+0000 7f87c5a6a640 1 -- 172.21.15.82:0/2481350145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f96ea00 msgr2=0x55653f96e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.125+0000 7f87c5a6a640 1 --2- 172.21.15.82:0/2481350145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f96ea00 0x55653f96e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.125+0000 7f87c5a6a640 1 -- 172.21.15.82:0/2481350145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653f9816f0 con 0x55653f965e80 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.125+0000 7f87c5a6a640 1 --2- 172.21.15.82:0/2481350145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f965e80 0x55653f96de00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f87b0000c00 tx=0x7f87b000b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb7d1c6fd7f3d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.125+0000 7f87ab7fe640 1 -- 172.21.15.82:0/2481350145 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0003ec0 con 0x55653f965e80 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.125+0000 7f87ab7fe640 1 -- 172.21.15.82:0/2481350145 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b0010730 con 0x55653f965e80 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.125+0000 7f87ab7fe640 1 -- 172.21.15.82:0/2481350145 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0011710 con 0x55653f965e80 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.126+0000 7f87ca995740 1 -- 172.21.15.82:0/2481350145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f965e80 msgr2=0x55653f96de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.126+0000 7f87ca995740 1 --2- 172.21.15.82:0/2481350145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f965e80 0x55653f96de00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f87b0000c00 tx=0x7f87b000b5d0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.126+0000 7f87ca995740 1 -- 172.21.15.82:0/2481350145 shutdown_connections 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.126+0000 7f87ca995740 1 --2- 172.21.15.82:0/2481350145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f96ea00 0x55653f96e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.126+0000 7f87ca995740 1 --2- 172.21.15.82:0/2481350145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f965e80 0x55653f96de00 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.126+0000 7f87ca995740 1 --2- 172.21.15.82:0/2481350145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f9654d0 0x55653f961800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.126+0000 7f87ca995740 1 -- 172.21.15.82:0/2481350145 >> 172.21.15.82:0/2481350145 conn(0x55653f95fba0 msgr2=0x55653f966260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.126+0000 7f87ca995740 1 -- 172.21.15.82:0/2481350145 shutdown_connections 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.126+0000 7f87ca995740 1 -- 172.21.15.82:0/2481350145 wait complete. 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.126+0000 7f87ca995740 1 -- start start 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.126+0000 7f87ca995740 1 Processor -- start 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87ca995740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 0x55653f972b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87ca995740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f965e80 0x55653f973060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87ca995740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f96ea00 0x55653f9e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87ca995740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f87c01bf690 con 0x55653f9654d0 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87ca995740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f87c01befc0 con 0x55653f96ea00 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87ca995740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87c0103670 con 0x55653f965e80 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87c5269640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 0x55653f972b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87c5269640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 0x55653f972b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37254/0 (socket says 172.21.15.82:37254) 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87c5269640 1 -- 172.21.15.82:0/3177757995 learned_addr learned my addr 172.21.15.82:0/3177757995 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87ca995740 10 client.0 _ll_register_callbacks cb 0x7f87b0004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87c5269640 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f96ea00 0x55653f9e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87c5a6a640 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f965e80 0x55653f973060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87c5269640 1 -- 172.21.15.82:0/3177757995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f965e80 msgr2=0x55653f973060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87c5269640 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f965e80 0x55653f973060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87c5269640 1 -- 172.21.15.82:0/3177757995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f96ea00 msgr2=0x55653f9e41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87c5269640 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f96ea00 0x55653f9e41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87c5269640 1 -- 172.21.15.82:0/3177757995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653f9e4730 con 0x55653f9654d0 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87c5269640 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 0x55653f972b20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0099f0 tx=0x7f87bc02c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df44c114bc3ce server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87a97fa640 1 -- 172.21.15.82:0/3177757995 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc05c3d0 con 0x55653f9654d0 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87a97fa640 1 -- 172.21.15.82:0/3177757995 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f87bc05c570 con 0x55653f9654d0 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.127+0000 7f87ca995740 1 -- 172.21.15.82:0/3177757995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87b0010730 con 0x55653f9654d0 2024-04-21T08:53:21.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.128+0000 7f87a97fa640 1 -- 172.21.15.82:0/3177757995 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f87bc05c770 con 0x55653f9654d0 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.128+0000 7f87ca995740 10 client.17337 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.128+0000 7f87ca995740 1 -- 172.21.15.82:0/3177757995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55653f9e58f0 con 0x55653f9654d0 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.128+0000 7f87a97fa640 1 -- 172.21.15.82:0/3177757995 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b0010730 con 0x55653f9654d0 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.128+0000 7f87a97fa640 1 -- 172.21.15.82:0/3177757995 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f87bc0434e0 con 0x55653f9654d0 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.128+0000 7f87ca995740 10 client.17337 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.128+0000 7f87ca995740 10 client.17337 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.128+0000 7f87ca995740 20 client.17337 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.128+0000 7f87ca995740 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55653f9e5c20 0x55653f9e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.128+0000 7f87ca995740 4 client.17337 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.128+0000 7f87ca995740 1 -- 172.21.15.82:0/3177757995 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55653f8d7c10 con 0x55653f9e5c20 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.128+0000 7f87c5a6a640 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55653f9e5c20 0x55653f9e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.129+0000 7f87c5a6a640 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55653f9e5c20 0x55653f9e8040 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d3482903001d74 server_cookie=150572735d24871a in_seq=0 out_seq=0 2024-04-21T08:53:21.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.129+0000 7f87a97fa640 10 client.17337 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:53:21.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.130+0000 7f87a97fa640 1 -- 172.21.15.82:0/3177757995 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55653f8d7c10 con 0x55653f9e5c20 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.130+0000 7f87a97fa640 10 client.17337 handle_command_reply: tid=0 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.130+0000 7f87ca995740 1 client.17337 shutdown 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87ca995740 1 -- 172.21.15.82:0/3177757995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 msgr2=0x55653f972b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87ca995740 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 0x55653f972b20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0099f0 tx=0x7f87bc02c810 comp rx=0 tx=0).stop 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87ca995740 1 -- 172.21.15.82:0/3177757995 shutdown_connections 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87ca995740 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55653f9e5c20 0x55653f9e8040 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87ca995740 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55653f96ea00 0x55653f9e41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87ca995740 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55653f965e80 0x55653f973060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87ca995740 1 --2- 172.21.15.82:0/3177757995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55653f9654d0 0x55653f972b20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87a97fa640 0 client.17337 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87ca995740 1 -- 172.21.15.82:0/3177757995 >> 172.21.15.82:0/3177757995 conn(0x55653f95fba0 msgr2=0x55653f966af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87ca995740 1 -- 172.21.15.82:0/3177757995 shutdown_connections 2024-04-21T08:53:21.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87ca995740 1 -- 172.21.15.82:0/3177757995 wait complete. 2024-04-21T08:53:21.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87ca995740 20 client.17337 trim_cache size 0 max 16384 2024-04-21T08:53:21.137 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:53:21.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87c7cce640 1 -- 172.21.15.82:0/2659217963 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f87ac0a3f20 msgr2=0x7f87ac0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87c7cce640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f87ac0a3f20 0x7f87ac0a63e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f87b809c4b0 tx=0x7f87b8098b10 comp rx=0 tx=0).stop 2024-04-21T08:53:21.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87c7cce640 1 -- 172.21.15.82:0/2659217963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c0107fd0 msgr2=0x7f87c010cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87c7cce640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c0107fd0 0x7f87c010cde0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f87b800e940 tx=0x7f87b800ee00 comp rx=0 tx=0).stop 2024-04-21T08:53:21.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87c7cce640 1 -- 172.21.15.82:0/2659217963 shutdown_connections 2024-04-21T08:53:21.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87c7cce640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f87ac0a3f20 0x7f87ac0a63e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87c7cce640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87c0107fd0 0x7f87c010cde0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87c7cce640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87c006ace0 0x7f87c01a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87c7cce640 1 --2- 172.21.15.82:0/2659217963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c006a360 0x7f87c01a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87c7cce640 1 -- 172.21.15.82:0/2659217963 >> 172.21.15.82:0/2659217963 conn(0x7f87c0077ef0 msgr2=0x7f87c01017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.131+0000 7f87c7cce640 1 -- 172.21.15.82:0/2659217963 shutdown_connections 2024-04-21T08:53:21.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.132+0000 7f87c7cce640 1 -- 172.21.15.82:0/2659217963 wait complete. 2024-04-21T08:53:21.143 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:53:21.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.248+0000 7faaf1552640 1 -- 172.21.15.82:0/1554020782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faaec106160 msgr2=0x7faaec10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.248+0000 7faaf1552640 1 --2- 172.21.15.82:0/1554020782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faaec106160 0x7faaec10dd30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7faad4009900 tx=0x7faad402bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:21.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.248+0000 7faaf1552640 1 -- 172.21.15.82:0/1554020782 shutdown_connections 2024-04-21T08:53:21.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.248+0000 7faaf1552640 1 --2- 172.21.15.82:0/1554020782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faaec106160 0x7faaec10dd30 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.248+0000 7faaf1552640 1 --2- 172.21.15.82:0/1554020782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaec101a40 0x7faaec105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.248+0000 7faaf1552640 1 --2- 172.21.15.82:0/1554020782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaec101090 0x7faaec101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.248+0000 7faaf1552640 1 -- 172.21.15.82:0/1554020782 >> 172.21.15.82:0/1554020782 conn(0x7faaec077f50 msgr2=0x7faaec078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.248+0000 7faaf1552640 1 -- 172.21.15.82:0/1554020782 shutdown_connections 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.248+0000 7faaf1552640 1 -- 172.21.15.82:0/1554020782 wait complete. 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaf1552640 1 Processor -- start 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaf1552640 1 -- start start 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaf1552640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaec101090 0x7faaec102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaf1552640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faaec106160 0x7faaec103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaf1552640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaec1035a0 0x7faaec1a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaf1552640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1110d0 con 0x7faaec106160 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaf1552640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faaec110f50 con 0x7faaec101090 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaf1552640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faaec111250 con 0x7faaec1035a0 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faaec106160 0x7faaec103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faaec106160 0x7faaec103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37264/0 (socket says 172.21.15.82:37264) 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaea7fc640 1 -- 172.21.15.82:0/1890729629 learned_addr learned my addr 172.21.15.82:0/1890729629 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaeaffd640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaec101090 0x7faaec102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaeb7fe640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaec1035a0 0x7faaec1a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaea7fc640 1 -- 172.21.15.82:0/1890729629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaec1035a0 msgr2=0x7faaec1a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaea7fc640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaec1035a0 0x7faaec1a68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaea7fc640 1 -- 172.21.15.82:0/1890729629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaec101090 msgr2=0x7faaec102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaea7fc640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaec101090 0x7faaec102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.249+0000 7faaea7fc640 1 -- 172.21.15.82:0/1890729629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec1a6f70 con 0x7faaec106160 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.250+0000 7faaea7fc640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faaec106160 0x7faaec103060 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7faadc00e9c0 tx=0x7faadc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.250+0000 7faae3fff640 1 -- 172.21.15.82:0/1890729629 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc00cd80 con 0x7faaec106160 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.250+0000 7faae3fff640 1 -- 172.21.15.82:0/1890729629 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faadc0054d0 con 0x7faaec106160 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.250+0000 7faaf1552640 1 -- 172.21.15.82:0/1890729629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec1a7260 con 0x7faaec106160 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.250+0000 7faae3fff640 1 -- 172.21.15.82:0/1890729629 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc010430 con 0x7faaec106160 2024-04-21T08:53:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.250+0000 7faaf1552640 1 -- 172.21.15.82:0/1890729629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec1a77b0 con 0x7faaec106160 2024-04-21T08:53:21.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.252+0000 7faae3fff640 1 -- 172.21.15.82:0/1890729629 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faadc004fd0 con 0x7faaec106160 2024-04-21T08:53:21.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.252+0000 7faaf4224740 1 Processor -- start 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaf4224740 1 -- start start 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faae3fff640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faad80a3d20 0x7faad80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaeb7fe640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faad80a3d20 0x7faad80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faae3fff640 1 -- 172.21.15.82:0/1890729629 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7faadc0d0fe0 con 0x7faaec106160 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaf4224740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaf4224740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52c4e80 0x558be52cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaf4224740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 0x558be52cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaf4224740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558be52291f0 con 0x558be52cda00 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaf4224740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558be51c64b0 con 0x558be52c4e80 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaf4224740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558be52251b0 con 0x558be52c44d0 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaeb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 0x558be52cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaea7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52c4e80 0x558be52cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaeb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 0x558be52cfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37272/0 (socket says 172.21.15.82:37272) 2024-04-21T08:53:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaeb7fe640 1 -- 172.21.15.82:0/2955309495 learned_addr learned my addr 172.21.15.82:0/2955309495 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaeaffd640 1 --2- 172.21.15.82:0/2955309495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaeb7fe640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faad80a3d20 0x7faad80a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faad4036040 tx=0x7faad402c5d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaeb7fe640 1 -- 172.21.15.82:0/2955309495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 msgr2=0x558be52c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaeb7fe640 1 --2- 172.21.15.82:0/2955309495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaeb7fe640 1 -- 172.21.15.82:0/2955309495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52c4e80 msgr2=0x558be52cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaeb7fe640 1 --2- 172.21.15.82:0/2955309495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52c4e80 0x558be52cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaeb7fe640 1 -- 172.21.15.82:0/2955309495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be51b81b0 con 0x558be52cda00 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.253+0000 7faaea7fc640 1 --2- 172.21.15.82:0/2955309495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52c4e80 0x558be52cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.254+0000 7faaeb7fe640 1 --2- 172.21.15.82:0/2955309495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 0x558be52cfe50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7faad4004200 tx=0x7faad4044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58fd6f6969814c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.254+0000 7faae17fa640 1 -- 172.21.15.82:0/2955309495 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad404d070 con 0x558be52cda00 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.254+0000 7faae17fa640 1 -- 172.21.15.82:0/2955309495 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faad40054f0 con 0x558be52cda00 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.254+0000 7faae17fa640 1 -- 172.21.15.82:0/2955309495 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4045070 con 0x558be52cda00 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.254+0000 7faaf4224740 1 -- 172.21.15.82:0/2955309495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 msgr2=0x558be52cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.254+0000 7faaf4224740 1 --2- 172.21.15.82:0/2955309495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 0x558be52cfe50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7faad4004200 tx=0x7faad4044000 comp rx=0 tx=0).stop 2024-04-21T08:53:21.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 -- 172.21.15.82:0/2955309495 shutdown_connections 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 --2- 172.21.15.82:0/2955309495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 0x558be52cfe50 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 --2- 172.21.15.82:0/2955309495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52c4e80 0x558be52cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 --2- 172.21.15.82:0/2955309495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 -- 172.21.15.82:0/2955309495 >> 172.21.15.82:0/2955309495 conn(0x558be52beba0 msgr2=0x558be52bef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 -- 172.21.15.82:0/2955309495 shutdown_connections 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 -- 172.21.15.82:0/2955309495 wait complete. 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 -- start start 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 Processor -- start 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52c4e80 0x558be52ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 0x558be52cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558be52d0910 con 0x558be52cda00 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558be51e2500 con 0x558be52c4e80 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558be52d0a90 con 0x558be52c44d0 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaf4224740 10 client.0 _ll_register_callbacks cb 0x558be51b0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaeb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 0x558be52cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaeb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 0x558be52cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37288/0 (socket says 172.21.15.82:37288) 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaeb7fe640 1 -- 172.21.15.82:0/3911504261 learned_addr learned my addr 172.21.15.82:0/3911504261 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaea7fc640 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52c4e80 0x558be52ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.255+0000 7faaeaffd640 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faaeb7fe640 1 -- 172.21.15.82:0/3911504261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 msgr2=0x558be52cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faaeb7fe640 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faaeb7fe640 1 -- 172.21.15.82:0/3911504261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52c4e80 msgr2=0x558be52ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faaeb7fe640 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52c4e80 0x558be52ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faaeb7fe640 1 -- 172.21.15.82:0/3911504261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be4f3b3f0 con 0x558be52cda00 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faaeaffd640 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faaeb7fe640 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 0x558be52cacd0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7faad40041d0 tx=0x7faad402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51f2a592db07f4a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faad27fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:21.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faad27fc640 1 -- 172.21.15.82:0/3911504261 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4058070 con 0x558be52cda00 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faad27fc640 1 -- 172.21.15.82:0/3911504261 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7faad40038c0 con 0x558be52cda00 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faaf4224740 1 -- 172.21.15.82:0/3911504261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faad40054f0 con 0x558be52cda00 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faad27fc640 1 -- 172.21.15.82:0/3911504261 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7faad4064050 con 0x558be52cda00 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faaf4224740 10 client.17349 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faaf4224740 1 -- 172.21.15.82:0/3911504261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558be50057d0 con 0x558be52cda00 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faad27fc640 1 -- 172.21.15.82:0/3911504261 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faad40054f0 con 0x558be52cda00 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faad27fc640 1 -- 172.21.15.82:0/3911504261 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7faad4048070 con 0x558be52cda00 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faaf4224740 10 client.17349 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.256+0000 7faaf4224740 10 client.17349 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.257+0000 7faaf4224740 20 client.17349 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.257+0000 7faaf4224740 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x558be52dda60 0x558be52dfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.257+0000 7faaf4224740 4 client.17349 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.257+0000 7faaf4224740 1 -- 172.21.15.82:0/3911504261 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558be51c8930 con 0x558be52dda60 2024-04-21T08:53:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.257+0000 7faaeaffd640 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x558be52dda60 0x558be52dfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.259+0000 7faad27fc640 10 client.17349 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:53:21.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.259+0000 7faaeaffd640 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x558be52dda60 0x558be52dfe50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ed04e840066ac9a server_cookie=3b092abec186d82 in_seq=0 out_seq=0 2024-04-21T08:53:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.260+0000 7faad27fc640 1 -- 172.21.15.82:0/3911504261 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558be51c8930 con 0x558be52dda60 2024-04-21T08:53:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.260+0000 7faad27fc640 10 client.17349 handle_command_reply: tid=0 2024-04-21T08:53:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.260+0000 7faaf4224740 1 client.17349 shutdown 2024-04-21T08:53:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.260+0000 7faaf4224740 1 -- 172.21.15.82:0/3911504261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 msgr2=0x558be52cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.260+0000 7faaf4224740 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 0x558be52cacd0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7faad40041d0 tx=0x7faad402bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.260+0000 7faaf4224740 1 -- 172.21.15.82:0/3911504261 shutdown_connections 2024-04-21T08:53:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.260+0000 7faaf4224740 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x558be52dda60 0x558be52dfe50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.261+0000 7faaf4224740 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52cda00 0x558be52cacd0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.261+0000 7faaf4224740 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52c4e80 0x558be52ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.261+0000 7faad27fc640 0 client.17349 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:53:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.261+0000 7faaf4224740 1 --2- 172.21.15.82:0/3911504261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.261+0000 7faaf4224740 1 -- 172.21.15.82:0/3911504261 >> 172.21.15.82:0/3911504261 conn(0x558be52beba0 msgr2=0x558be52c08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.261+0000 7faaf4224740 1 -- 172.21.15.82:0/3911504261 shutdown_connections 2024-04-21T08:53:21.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.261+0000 7faaf4224740 1 -- 172.21.15.82:0/3911504261 wait complete. 2024-04-21T08:53:21.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.261+0000 7faaf4224740 20 client.17349 trim_cache size 0 max 16384 2024-04-21T08:53:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.274+0000 7faaf4224740 1 Processor -- start 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.274+0000 7faaf4224740 1 -- start start 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.274+0000 7faaf4224740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.274+0000 7faaf4224740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 0x558be52cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.274+0000 7faaf4224740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52cda00 0x558be52c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.274+0000 7faaf4224740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1b1570 con 0x558be52c4e80 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.274+0000 7faaf4224740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1b3140 con 0x558be52cda00 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.274+0000 7faaf4224740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1b1c40 con 0x558be52c44d0 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faaea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 0x558be52cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faaea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 0x558be52cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37302/0 (socket says 172.21.15.82:37302) 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faaea7fc640 1 -- 172.21.15.82:0/2337518341 learned_addr learned my addr 172.21.15.82:0/2337518341 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faaeb7fe640 1 --2- 172.21.15.82:0/2337518341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52cda00 0x558be52c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faaeaffd640 1 --2- 172.21.15.82:0/2337518341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faaea7fc640 1 -- 172.21.15.82:0/2337518341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 msgr2=0x558be52ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faaea7fc640 1 --2- 172.21.15.82:0/2337518341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faaea7fc640 1 -- 172.21.15.82:0/2337518341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52cda00 msgr2=0x558be52c05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faaea7fc640 1 --2- 172.21.15.82:0/2337518341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52cda00 0x558be52c05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faaea7fc640 1 -- 172.21.15.82:0/2337518341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be52be6d0 con 0x558be52c4e80 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faaeaffd640 1 --2- 172.21.15.82:0/2337518341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faaea7fc640 1 --2- 172.21.15.82:0/2337518341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 0x558be52cd330 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7faadc099020 tx=0x7faadc005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13655f76fb27d2f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faae0ff9640 1 -- 172.21.15.82:0/2337518341 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc09d2c0 con 0x558be52c4e80 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faae0ff9640 1 -- 172.21.15.82:0/2337518341 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faadc09cbc0 con 0x558be52c4e80 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.275+0000 7faae0ff9640 1 -- 172.21.15.82:0/2337518341 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc0ce780 con 0x558be52c4e80 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 -- 172.21.15.82:0/2337518341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 msgr2=0x558be52cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 --2- 172.21.15.82:0/2337518341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 0x558be52cd330 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7faadc099020 tx=0x7faadc005680 comp rx=0 tx=0).stop 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 -- 172.21.15.82:0/2337518341 shutdown_connections 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 --2- 172.21.15.82:0/2337518341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52cda00 0x558be52c05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 --2- 172.21.15.82:0/2337518341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 0x558be52cd330 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 --2- 172.21.15.82:0/2337518341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 -- 172.21.15.82:0/2337518341 >> 172.21.15.82:0/2337518341 conn(0x558be52beba0 msgr2=0x558be52bfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 -- 172.21.15.82:0/2337518341 shutdown_connections 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 -- 172.21.15.82:0/2337518341 wait complete. 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 -- start start 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 Processor -- start 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 0x558be52e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52cda00 0x558be52e0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1b2a70 con 0x558be52c4e80 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1b23a0 con 0x558be52cda00 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1af550 con 0x558be52c44d0 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.276+0000 7faaf4224740 10 client.0 _ll_register_callbacks cb 0x7faad4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 0x558be52e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 0x558be52e0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37318/0 (socket says 172.21.15.82:37318) 2024-04-21T08:53:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaea7fc640 1 -- 172.21.15.82:0/1930302814 learned_addr learned my addr 172.21.15.82:0/1930302814 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaeb7fe640 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52cda00 0x558be52e0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaeaffd640 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaea7fc640 1 -- 172.21.15.82:0/1930302814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 msgr2=0x558be52e2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaea7fc640 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52e2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaea7fc640 1 -- 172.21.15.82:0/1930302814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52cda00 msgr2=0x558be52e0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaea7fc640 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52cda00 0x558be52e0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaea7fc640 1 -- 172.21.15.82:0/1930302814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be52e0eb0 con 0x558be52c4e80 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaea7fc640 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 0x558be52e0430 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7faadc004960 tx=0x7faadc09c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8a8f20d14d9f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faad27fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faad27fc640 1 -- 172.21.15.82:0/1930302814 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc065020 con 0x558be52c4e80 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faad27fc640 1 -- 172.21.15.82:0/1930302814 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7faadc099230 con 0x558be52c4e80 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaf4224740 1 -- 172.21.15.82:0/1930302814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faadc09cbc0 con 0x558be52c4e80 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faad27fc640 1 -- 172.21.15.82:0/1930302814 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7faadc0a5830 con 0x558be52c4e80 2024-04-21T08:53:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaf4224740 10 client.17355 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faaf4224740 1 -- 172.21.15.82:0/1930302814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558be53439b0 con 0x558be52c4e80 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faad27fc640 1 -- 172.21.15.82:0/1930302814 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faadc09cbc0 con 0x558be52c4e80 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.277+0000 7faad27fc640 1 -- 172.21.15.82:0/1930302814 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7faadc0989a0 con 0x558be52c4e80 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 10 client.17355 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 10 client.17355 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 20 client.17355 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x558be52dda60 0x558be5346030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 4 client.17355 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 -- 172.21.15.82:0/1930302814 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558be5236c10 con 0x558be52dda60 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaeaffd640 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x558be52dda60 0x558be5346030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faad27fc640 10 client.17355 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaeaffd640 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x558be52dda60 0x558be5346030 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be323a01ae369add server_cookie=5ef59d372274408c in_seq=0 out_seq=0 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faad27fc640 1 -- 172.21.15.82:0/1930302814 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558be5236c10 con 0x558be52dda60 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faad27fc640 10 client.17355 handle_command_reply: tid=0 2024-04-21T08:53:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 client.17355 shutdown 2024-04-21T08:53:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 -- 172.21.15.82:0/1930302814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 msgr2=0x558be52e0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 0x558be52e0430 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7faadc004960 tx=0x7faadc09c990 comp rx=0 tx=0).stop 2024-04-21T08:53:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 -- 172.21.15.82:0/1930302814 shutdown_connections 2024-04-21T08:53:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x558be52dda60 0x558be5346030 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558be52cda00 0x558be52e0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558be52c4e80 0x558be52e0430 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faad27fc640 0 client.17355 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:53:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 --2- 172.21.15.82:0/1930302814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558be52c44d0 0x558be52e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 -- 172.21.15.82:0/1930302814 >> 172.21.15.82:0/1930302814 conn(0x558be52beba0 msgr2=0x558be52c5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 -- 172.21.15.82:0/1930302814 shutdown_connections 2024-04-21T08:53:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.278+0000 7faaf4224740 1 -- 172.21.15.82:0/1930302814 wait complete. 2024-04-21T08:53:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf4224740 20 client.17355 trim_cache size 0 max 16384 2024-04-21T08:53:21.286 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:53:21.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf1552640 1 -- 172.21.15.82:0/1890729629 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faad80a3d20 msgr2=0x7faad80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf1552640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faad80a3d20 0x7faad80a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faad4036040 tx=0x7faad402c5d0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf1552640 1 -- 172.21.15.82:0/1890729629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faaec106160 msgr2=0x7faaec103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf1552640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faaec106160 0x7faaec103060 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7faadc00e9c0 tx=0x7faadc00ee80 comp rx=0 tx=0).stop 2024-04-21T08:53:21.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf1552640 1 -- 172.21.15.82:0/1890729629 shutdown_connections 2024-04-21T08:53:21.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf1552640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaec1035a0 0x7faaec1a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf1552640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faad80a3d20 0x7faad80a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf1552640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faaec106160 0x7faaec103060 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf1552640 1 --2- 172.21.15.82:0/1890729629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faaec101090 0x7faaec102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf1552640 1 -- 172.21.15.82:0/1890729629 >> 172.21.15.82:0/1890729629 conn(0x7faaec077f50 msgr2=0x7faaec104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf1552640 1 -- 172.21.15.82:0/1890729629 shutdown_connections 2024-04-21T08:53:21.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.279+0000 7faaf1552640 1 -- 172.21.15.82:0/1890729629 wait complete. 2024-04-21T08:53:21.289 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:53:21.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.393+0000 7f2663575640 1 -- 172.21.15.82:0/899185514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f265c101a40 msgr2=0x7f265c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.393+0000 7f2663575640 1 --2- 172.21.15.82:0/899185514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f265c101a40 0x7f265c105a80 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f264c009900 tx=0x7f264c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:21.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.393+0000 7f2663575640 1 -- 172.21.15.82:0/899185514 shutdown_connections 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.393+0000 7f2663575640 1 --2- 172.21.15.82:0/899185514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f265c106160 0x7f265c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.393+0000 7f2663575640 1 --2- 172.21.15.82:0/899185514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f265c101a40 0x7f265c105a80 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.393+0000 7f2663575640 1 --2- 172.21.15.82:0/899185514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265c101090 0x7f265c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.393+0000 7f2663575640 1 -- 172.21.15.82:0/899185514 >> 172.21.15.82:0/899185514 conn(0x7f265c077f50 msgr2=0x7f265c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.393+0000 7f2663575640 1 -- 172.21.15.82:0/899185514 shutdown_connections 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.393+0000 7f2663575640 1 -- 172.21.15.82:0/899185514 wait complete. 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.394+0000 7f2663575640 1 Processor -- start 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.394+0000 7f2663575640 1 -- start start 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.394+0000 7f2663575640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f265c101090 0x7f265c1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.394+0000 7f2663575640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f265c106160 0x7f265c197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.394+0000 7f2663575640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265c19bf80 0x7f265c19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.394+0000 7f2663575640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f265c111100 con 0x7f265c101090 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.394+0000 7f2663575640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f265c110f80 con 0x7f265c19bf80 2024-04-21T08:53:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.394+0000 7f2663575640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f265c111280 con 0x7f265c106160 2024-04-21T08:53:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.394+0000 7f2661311640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f265c101090 0x7f265c1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.394+0000 7f2661311640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f265c101090 0x7f265c1976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37334/0 (socket says 172.21.15.82:37334) 2024-04-21T08:53:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.394+0000 7f2661311640 1 -- 172.21.15.82:0/565800181 learned_addr learned my addr 172.21.15.82:0/565800181 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.394+0000 7f2661b12640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265c19bf80 0x7f265c19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.395+0000 7f2661311640 1 -- 172.21.15.82:0/565800181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f265c106160 msgr2=0x7f265c197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.395+0000 7f2661311640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f265c106160 0x7f265c197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.395+0000 7f2661311640 1 -- 172.21.15.82:0/565800181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265c19bf80 msgr2=0x7f265c19c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.395+0000 7f2661311640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265c19bf80 0x7f265c19c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.395+0000 7f2661311640 1 -- 172.21.15.82:0/565800181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c19ca70 con 0x7f265c101090 2024-04-21T08:53:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.395+0000 7f2661311640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f265c101090 0x7f265c1976c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f265800eb00 tx=0x7f2658009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.395+0000 7f26527fc640 1 -- 172.21.15.82:0/565800181 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265801c070 con 0x7f265c101090 2024-04-21T08:53:21.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.395+0000 7f26527fc640 1 -- 172.21.15.82:0/565800181 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f265800bc30 con 0x7f265c101090 2024-04-21T08:53:21.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.395+0000 7f26527fc640 1 -- 172.21.15.82:0/565800181 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658016610 con 0x7f265c101090 2024-04-21T08:53:21.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.395+0000 7f2663575640 1 -- 172.21.15.82:0/565800181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c19cd60 con 0x7f265c101090 2024-04-21T08:53:21.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.395+0000 7f2663575640 1 -- 172.21.15.82:0/565800181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c0693b0 con 0x7f265c101090 2024-04-21T08:53:21.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.398+0000 7f26527fc640 1 -- 172.21.15.82:0/565800181 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f265801f070 con 0x7f265c101090 2024-04-21T08:53:21.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.398+0000 7f2666164740 1 Processor -- start 2024-04-21T08:53:21.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.398+0000 7f2666164740 1 -- start start 2024-04-21T08:53:21.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.398+0000 7f2666164740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 0x560d9f12f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.398+0000 7f26527fc640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f26480a3d20 0x7f26480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.398+0000 7f2666164740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f12fe80 0x560d9f138430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.398+0000 7f2666164740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f138a00 0x560d9f13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.398+0000 7f2666164740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560d9f0941f0 con 0x560d9f138a00 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.398+0000 7f2666164740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560d9f0314b0 con 0x560d9f12f4d0 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.398+0000 7f2666164740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d9f0901b0 con 0x560d9f12fe80 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.398+0000 7f2661b12640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f138a00 0x560d9f13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f2661b12640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f138a00 0x560d9f13ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37340/0 (socket says 172.21.15.82:37340) 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f2661311640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f26480a3d20 0x7f26480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f2661b12640 1 -- 172.21.15.82:0/1659596940 learned_addr learned my addr 172.21.15.82:0/1659596940 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f26527fc640 1 -- 172.21.15.82:0/565800181 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f26580d21c0 con 0x7f265c101090 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f2660b10640 1 --2- 172.21.15.82:0/1659596940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f12fe80 0x560d9f138430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f2661b12640 1 --2- 172.21.15.82:0/1659596940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 0x560d9f12f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f2661b12640 1 -- 172.21.15.82:0/1659596940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f12fe80 msgr2=0x560d9f138430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f2661b12640 1 --2- 172.21.15.82:0/1659596940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f12fe80 0x560d9f138430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f2661b12640 1 -- 172.21.15.82:0/1659596940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 msgr2=0x560d9f12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f2661b12640 1 --2- 172.21.15.82:0/1659596940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 0x560d9f12f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f2661b12640 1 -- 172.21.15.82:0/1659596940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d9f0231b0 con 0x560d9f138a00 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f2661311640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f26480a3d20 0x7f26480a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26580cfb50 tx=0x7f26580932e0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f2661b12640 1 --2- 172.21.15.82:0/1659596940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f138a00 0x560d9f13ae50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f265400bd60 tx=0x7f2654009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845d58d88774cb66 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f26477fe640 1 -- 172.21.15.82:0/1659596940 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654014040 con 0x560d9f138a00 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f26477fe640 1 -- 172.21.15.82:0/1659596940 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f265400be50 con 0x560d9f138a00 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.399+0000 7f26477fe640 1 -- 172.21.15.82:0/1659596940 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265401a3e0 con 0x560d9f138a00 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.400+0000 7f2666164740 1 -- 172.21.15.82:0/1659596940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f138a00 msgr2=0x560d9f13ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.400+0000 7f2666164740 1 --2- 172.21.15.82:0/1659596940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f138a00 0x560d9f13ae50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f265400bd60 tx=0x7f2654009b80 comp rx=0 tx=0).stop 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.400+0000 7f2666164740 1 -- 172.21.15.82:0/1659596940 shutdown_connections 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.400+0000 7f2666164740 1 --2- 172.21.15.82:0/1659596940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f138a00 0x560d9f13ae50 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.400+0000 7f2666164740 1 --2- 172.21.15.82:0/1659596940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f12fe80 0x560d9f138430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.400+0000 7f2666164740 1 --2- 172.21.15.82:0/1659596940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 0x560d9f12f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.400+0000 7f2666164740 1 -- 172.21.15.82:0/1659596940 >> 172.21.15.82:0/1659596940 conn(0x560d9f129ba0 msgr2=0x560d9f129f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.400+0000 7f2666164740 1 -- 172.21.15.82:0/1659596940 shutdown_connections 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.400+0000 7f2666164740 1 -- 172.21.15.82:0/1659596940 wait complete. 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.400+0000 7f2666164740 1 -- start start 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.400+0000 7f2666164740 1 Processor -- start 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2666164740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 0x560d9f138300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2666164740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f135690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2666164740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f138a00 0x560d9f135c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2666164740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560d9f13b910 con 0x560d9f12fe80 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2666164740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560d9f04d500 con 0x560d9f12f4d0 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2666164740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d9f13ba90 con 0x560d9f138a00 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2666164740 10 client.0 _ll_register_callbacks cb 0x560d9f01bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2660b10640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f135690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2660b10640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f135690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37348/0 (socket says 172.21.15.82:37348) 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2660b10640 1 -- 172.21.15.82:0/2321537095 learned_addr learned my addr 172.21.15.82:0/2321537095 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2661b12640 1 --2- 172.21.15.82:0/2321537095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 0x560d9f138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2660b10640 1 -- 172.21.15.82:0/2321537095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f138a00 msgr2=0x560d9f135c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2660b10640 1 --2- 172.21.15.82:0/2321537095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f138a00 0x560d9f135c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2660b10640 1 -- 172.21.15.82:0/2321537095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 msgr2=0x560d9f138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2660b10640 1 --2- 172.21.15.82:0/2321537095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 0x560d9f138300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2660b10640 1 -- 172.21.15.82:0/2321537095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d9eda63f0 con 0x560d9f12fe80 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2660b10640 1 --2- 172.21.15.82:0/2321537095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f135690 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f265c06b6e0 tx=0x7f264c02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f0272bc97729bf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2644ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2644ff9640 1 -- 172.21.15.82:0/2321537095 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c044070 con 0x560d9f12fe80 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2644ff9640 1 -- 172.21.15.82:0/2321537095 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f264c0037f0 con 0x560d9f12fe80 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.401+0000 7f2666164740 1 -- 172.21.15.82:0/2321537095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f265400be50 con 0x560d9f12fe80 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.402+0000 7f2644ff9640 1 -- 172.21.15.82:0/2321537095 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f264c005d50 con 0x560d9f12fe80 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.402+0000 7f2666164740 10 client.17367 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.402+0000 7f2666164740 1 -- 172.21.15.82:0/2321537095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560d9ee707d0 con 0x560d9f12fe80 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.402+0000 7f2644ff9640 1 -- 172.21.15.82:0/2321537095 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f265400be50 con 0x560d9f12fe80 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.402+0000 7f2644ff9640 1 -- 172.21.15.82:0/2321537095 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f264c0053c0 con 0x560d9f12fe80 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.402+0000 7f2666164740 10 client.17367 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:21.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.402+0000 7f2666164740 10 client.17367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.402+0000 7f2666164740 20 client.17367 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.402+0000 7f2666164740 1 --2- 172.21.15.82:0/2321537095 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560d9f148b30 0x560d9f14af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.402+0000 7f2666164740 4 client.17367 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.402+0000 7f2666164740 1 -- 172.21.15.82:0/2321537095 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d9f033930 con 0x560d9f148b30 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.402+0000 7f2661b12640 1 --2- 172.21.15.82:0/2321537095 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560d9f148b30 0x560d9f14af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2644ff9640 10 client.17367 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2661b12640 1 --2- 172.21.15.82:0/2321537095 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560d9f148b30 0x560d9f14af90 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8bc45e71e352aa3 server_cookie=8147159d3e9de688 in_seq=0 out_seq=0 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2644ff9640 1 -- 172.21.15.82:0/2321537095 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d9f033930 con 0x560d9f148b30 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2644ff9640 10 client.17367 handle_command_reply: tid=0 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2666164740 1 client.17367 shutdown 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2666164740 1 -- 172.21.15.82:0/2321537095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 msgr2=0x560d9f135690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2666164740 1 --2- 172.21.15.82:0/2321537095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f135690 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f265c06b6e0 tx=0x7f264c02ca10 comp rx=0 tx=0).stop 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2666164740 1 -- 172.21.15.82:0/2321537095 shutdown_connections 2024-04-21T08:53:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2666164740 1 --2- 172.21.15.82:0/2321537095 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560d9f148b30 0x560d9f14af90 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2666164740 1 --2- 172.21.15.82:0/2321537095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f138a00 0x560d9f135c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2666164740 1 --2- 172.21.15.82:0/2321537095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f135690 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2666164740 1 --2- 172.21.15.82:0/2321537095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 0x560d9f138300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2644ff9640 0 client.17367 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.403+0000 7f2666164740 1 -- 172.21.15.82:0/2321537095 >> 172.21.15.82:0/2321537095 conn(0x560d9f129ba0 msgr2=0x560d9f12b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.404+0000 7f2666164740 1 -- 172.21.15.82:0/2321537095 shutdown_connections 2024-04-21T08:53:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.404+0000 7f2666164740 1 -- 172.21.15.82:0/2321537095 wait complete. 2024-04-21T08:53:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.404+0000 7f2666164740 20 client.17367 trim_cache size 0 max 16384 2024-04-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2666164740 1 Processor -- start 2024-04-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2666164740 1 -- start start 2024-04-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2666164740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 0x560d9f1447d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2666164740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2666164740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f138a00 0x560d9f138300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2666164740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1ad2c0 con 0x560d9f12fe80 2024-04-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2666164740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1aee90 con 0x560d9f12f4d0 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2666164740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1ad990 con 0x560d9f138a00 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2660b10640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2660b10640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f137dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37358/0 (socket says 172.21.15.82:37358) 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2660b10640 1 -- 172.21.15.82:0/2837076367 learned_addr learned my addr 172.21.15.82:0/2837076367 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2661b12640 1 --2- 172.21.15.82:0/2837076367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f138a00 0x560d9f138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2660b10640 1 -- 172.21.15.82:0/2837076367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f138a00 msgr2=0x560d9f138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2660b10640 1 --2- 172.21.15.82:0/2837076367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f138a00 0x560d9f138300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2660b10640 1 -- 172.21.15.82:0/2837076367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 msgr2=0x560d9f1447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2660b10640 1 --2- 172.21.15.82:0/2837076367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 0x560d9f1447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.417+0000 7f2660b10640 1 -- 172.21.15.82:0/2837076367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d9f133be0 con 0x560d9f12fe80 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2660b10640 1 --2- 172.21.15.82:0/2837076367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f137dc0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f264c004710 tx=0x7f264c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89baf0867920b333 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2646ffd640 1 -- 172.21.15.82:0/2837076367 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c044070 con 0x560d9f12fe80 2024-04-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2646ffd640 1 -- 172.21.15.82:0/2837076367 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f264c004b80 con 0x560d9f12fe80 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2646ffd640 1 -- 172.21.15.82:0/2837076367 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0035f0 con 0x560d9f12fe80 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2666164740 1 -- 172.21.15.82:0/2837076367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 msgr2=0x560d9f137dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2666164740 1 --2- 172.21.15.82:0/2837076367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f137dc0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f264c004710 tx=0x7f264c001e70 comp rx=0 tx=0).stop 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2666164740 1 -- 172.21.15.82:0/2837076367 shutdown_connections 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2666164740 1 --2- 172.21.15.82:0/2837076367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f138a00 0x560d9f138300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2666164740 1 --2- 172.21.15.82:0/2837076367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f137dc0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2666164740 1 --2- 172.21.15.82:0/2837076367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f12f4d0 0x560d9f1447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2666164740 1 -- 172.21.15.82:0/2837076367 >> 172.21.15.82:0/2837076367 conn(0x560d9f129ba0 msgr2=0x560d9f130260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2666164740 1 -- 172.21.15.82:0/2837076367 shutdown_connections 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.418+0000 7f2666164740 1 -- 172.21.15.82:0/2837076367 wait complete. 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2666164740 1 -- start start 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2666164740 1 Processor -- start 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2666164740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f12f4d0 0x560d9f140d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2666164740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f13c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2666164740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f138a00 0x560d9f13c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2666164740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1ae7c0 con 0x560d9f12fe80 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2666164740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1ae0f0 con 0x560d9f138a00 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2666164740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f265c1ab2a0 con 0x560d9f12f4d0 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2660b10640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f13c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2666164740 10 client.0 _ll_register_callbacks cb 0x7f264c004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2660b10640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f13c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37374/0 (socket says 172.21.15.82:37374) 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2660b10640 1 -- 172.21.15.82:0/144505872 learned_addr learned my addr 172.21.15.82:0/144505872 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2661b12640 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f138a00 0x560d9f13c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.419+0000 7f2661b12640 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f12f4d0 0x560d9f140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2660b10640 1 -- 172.21.15.82:0/144505872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f12f4d0 msgr2=0x560d9f140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2660b10640 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f12f4d0 0x560d9f140d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2660b10640 1 -- 172.21.15.82:0/144505872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f138a00 msgr2=0x560d9f13c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2660b10640 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f138a00 0x560d9f13c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2660b10640 1 -- 172.21.15.82:0/144505872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d9f13ccc0 con 0x560d9f12fe80 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2661b12640 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f12f4d0 0x560d9f140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2660b10640 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f13c240 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f264c009850 tx=0x7f264c005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6445b4982791ab09 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2644ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2644ff9640 1 -- 172.21.15.82:0/144505872 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c044040 con 0x560d9f12fe80 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2644ff9640 1 -- 172.21.15.82:0/144505872 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f264c003800 con 0x560d9f12fe80 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2666164740 1 -- 172.21.15.82:0/144505872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f264c004b80 con 0x560d9f12fe80 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2644ff9640 1 -- 172.21.15.82:0/144505872 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f264c02c2b0 con 0x560d9f12fe80 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2666164740 10 client.17373 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2666164740 1 -- 172.21.15.82:0/144505872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560d9f1aedc0 con 0x560d9f12fe80 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2644ff9640 1 -- 172.21.15.82:0/144505872 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f264c004b80 con 0x560d9f12fe80 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2644ff9640 1 -- 172.21.15.82:0/144505872 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f264c0498d0 con 0x560d9f12fe80 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2666164740 10 client.17373 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2666164740 10 client.17373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2666164740 20 client.17373 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:21.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2666164740 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560d9f1af290 0x560d9f1b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2666164740 4 client.17373 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.420+0000 7f2666164740 1 -- 172.21.15.82:0/144505872 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d9f0a1c10 con 0x560d9f1af290 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2661b12640 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560d9f1af290 0x560d9f1b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2644ff9640 10 client.17373 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2661b12640 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560d9f1af290 0x560d9f1b16b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d023223a2538f0 server_cookie=c1273105e36d52b4 in_seq=0 out_seq=0 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2644ff9640 1 -- 172.21.15.82:0/144505872 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d9f0a1c10 con 0x560d9f1af290 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2644ff9640 10 client.17373 handle_command_reply: tid=0 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2666164740 1 client.17373 shutdown 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2666164740 1 -- 172.21.15.82:0/144505872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 msgr2=0x560d9f13c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2666164740 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f13c240 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f264c009850 tx=0x7f264c005390 comp rx=0 tx=0).stop 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2666164740 1 -- 172.21.15.82:0/144505872 shutdown_connections 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2666164740 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560d9f1af290 0x560d9f1b16b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2666164740 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d9f138a00 0x560d9f13c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2666164740 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d9f12fe80 0x560d9f13c240 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2644ff9640 0 client.17373 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:21.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2666164740 1 --2- 172.21.15.82:0/144505872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d9f12f4d0 0x560d9f140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2666164740 1 -- 172.21.15.82:0/144505872 >> 172.21.15.82:0/144505872 conn(0x560d9f129ba0 msgr2=0x560d9f130af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2666164740 1 -- 172.21.15.82:0/144505872 shutdown_connections 2024-04-21T08:53:21.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2666164740 1 -- 172.21.15.82:0/144505872 wait complete. 2024-04-21T08:53:21.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.421+0000 7f2666164740 20 client.17373 trim_cache size 0 max 16384 2024-04-21T08:53:21.428 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:53:21.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.422+0000 7f2663575640 1 -- 172.21.15.82:0/565800181 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f26480a3d20 msgr2=0x7f26480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.422+0000 7f2663575640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f26480a3d20 0x7f26480a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26580cfb50 tx=0x7f26580932e0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.422+0000 7f2663575640 1 -- 172.21.15.82:0/565800181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f265c101090 msgr2=0x7f265c1976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:21.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.422+0000 7f2663575640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f265c101090 0x7f265c1976c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f265800eb00 tx=0x7f2658009f20 comp rx=0 tx=0).stop 2024-04-21T08:53:21.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.422+0000 7f2663575640 1 -- 172.21.15.82:0/565800181 shutdown_connections 2024-04-21T08:53:21.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.422+0000 7f2663575640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f265c19bf80 0x7f265c19c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.422+0000 7f2663575640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f265c106160 0x7f265c197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.422+0000 7f2663575640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f26480a3d20 0x7f26480a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.422+0000 7f2663575640 1 --2- 172.21.15.82:0/565800181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f265c101090 0x7f265c1976c0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:21.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.422+0000 7f2663575640 1 -- 172.21.15.82:0/565800181 >> 172.21.15.82:0/565800181 conn(0x7f265c077f50 msgr2=0x7f265c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:21.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.422+0000 7f2663575640 1 -- 172.21.15.82:0/565800181 shutdown_connections 2024-04-21T08:53:21.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:21.422+0000 7f2663575640 1 -- 172.21.15.82:0/565800181 wait complete. 2024-04-21T08:53:21.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:21 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2908663114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:53:21.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:21 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2908663114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:53:21.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:21 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2908663114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:53:22.434 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:53:22.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:53:22.434 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:53:22.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.543+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/1120429763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8106160 msgr2=0x7f6fc810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.543+0000 7f6fcdaf2640 1 --2- 172.21.15.82:0/1120429763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8106160 0x7f6fc810dd30 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009ca0 tx=0x7f6fc002c120 comp rx=0 tx=0).stop 2024-04-21T08:53:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.543+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/1120429763 shutdown_connections 2024-04-21T08:53:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.543+0000 7f6fcdaf2640 1 --2- 172.21.15.82:0/1120429763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8106160 0x7f6fc810dd30 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.543+0000 7f6fcdaf2640 1 --2- 172.21.15.82:0/1120429763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fc8101a40 0x7f6fc8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.543+0000 7f6fcdaf2640 1 --2- 172.21.15.82:0/1120429763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fc8101090 0x7f6fc8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.543+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/1120429763 >> 172.21.15.82:0/1120429763 conn(0x7f6fc8077f50 msgr2=0x7f6fc8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.543+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/1120429763 shutdown_connections 2024-04-21T08:53:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.543+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/1120429763 wait complete. 2024-04-21T08:53:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.544+0000 7f6fcdaf2640 1 Processor -- start 2024-04-21T08:53:22.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.544+0000 7f6fcdaf2640 1 -- start start 2024-04-21T08:53:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.544+0000 7f6fcdaf2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fc8101090 0x7f6fc819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.544+0000 7f6fcdaf2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fc8106160 0x7f6fc819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.544+0000 7f6fcdaf2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc81985c0 0x7f6fc8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.544+0000 7f6fcdaf2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc8112240 con 0x7f6fc81985c0 2024-04-21T08:53:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.544+0000 7f6fcdaf2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81120c0 con 0x7f6fc8106160 2024-04-21T08:53:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.544+0000 7f6fcdaf2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81123c0 con 0x7f6fc8101090 2024-04-21T08:53:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.544+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc81985c0 0x7f6fc8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.544+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc81985c0 0x7f6fc8198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37394/0 (socket says 172.21.15.82:37394) 2024-04-21T08:53:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.544+0000 7f6fc6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fc8106160 0x7f6fc819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.544+0000 7f6fc7fff640 1 -- 172.21.15.82:0/3199847729 learned_addr learned my addr 172.21.15.82:0/3199847729 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:22.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.545+0000 7f6fc7fff640 1 -- 172.21.15.82:0/3199847729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fc8101090 msgr2=0x7f6fc819e410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:53:22.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.545+0000 7f6fc7fff640 1 --2- 172.21.15.82:0/3199847729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fc8101090 0x7f6fc819e410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.545+0000 7f6fc7fff640 1 -- 172.21.15.82:0/3199847729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fc8106160 msgr2=0x7f6fc819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.545+0000 7f6fc7fff640 1 --2- 172.21.15.82:0/3199847729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fc8106160 0x7f6fc819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.545+0000 7f6fc7fff640 1 -- 172.21.15.82:0/3199847729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc81990e0 con 0x7f6fc81985c0 2024-04-21T08:53:22.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.545+0000 7f6fc7fff640 1 --2- 172.21.15.82:0/3199847729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc81985c0 0x7f6fc8198a70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6fc002c730 tx=0x7f6fc0035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:22.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.545+0000 7f6fc4ff9640 1 -- 172.21.15.82:0/3199847729 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0039040 con 0x7f6fc81985c0 2024-04-21T08:53:22.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.545+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/3199847729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc8199370 con 0x7f6fc81985c0 2024-04-21T08:53:22.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.545+0000 7f6fc4ff9640 1 -- 172.21.15.82:0/3199847729 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fc0044070 con 0x7f6fc81985c0 2024-04-21T08:53:22.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.545+0000 7f6fc4ff9640 1 -- 172.21.15.82:0/3199847729 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0004330 con 0x7f6fc81985c0 2024-04-21T08:53:22.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.545+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/3199847729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc81a55c0 con 0x7f6fc81985c0 2024-04-21T08:53:22.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.547+0000 7f6fc4ff9640 1 -- 172.21.15.82:0/3199847729 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6fc00044d0 con 0x7f6fc81985c0 2024-04-21T08:53:22.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fd07c2740 1 Processor -- start 2024-04-21T08:53:22.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fd07c2740 1 -- start start 2024-04-21T08:53:22.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fd07c2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 0x5612c9edfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fd07c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee0290 0x5612c9ee8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fd07c2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9ee8e10 0x5612c9eeb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fd07c2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5612c9e43f00 con 0x5612c9edf8e0 2024-04-21T08:53:22.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fd07c2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5612c9de1590 con 0x5612c9ee8e10 2024-04-21T08:53:22.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fd07c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5612c9e3fec0 con 0x5612c9ee0290 2024-04-21T08:53:22.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 0x5612c9edfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 0x5612c9edfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37402/0 (socket says 172.21.15.82:37402) 2024-04-21T08:53:22.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc6ffd640 1 -- 172.21.15.82:0/2110291948 learned_addr learned my addr 172.21.15.82:0/2110291948 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:22.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc4ff9640 1 --2- 172.21.15.82:0/3199847729 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6fb40a3d20 0x7f6fb40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/2110291948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9ee8e10 0x5612c9eeb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc77fe640 1 --2- 172.21.15.82:0/2110291948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee0290 0x5612c9ee8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc7fff640 1 --2- 172.21.15.82:0/3199847729 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6fb40a3d20 0x7f6fb40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc6ffd640 1 -- 172.21.15.82:0/2110291948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee0290 msgr2=0x5612c9ee8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/2110291948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee0290 0x5612c9ee8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc6ffd640 1 -- 172.21.15.82:0/2110291948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9ee8e10 msgr2=0x5612c9eeb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/2110291948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9ee8e10 0x5612c9eeb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc6ffd640 1 -- 172.21.15.82:0/2110291948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612c9c1e7e0 con 0x5612c9edf8e0 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc4ff9640 1 -- 172.21.15.82:0/3199847729 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6fc00f2ff0 con 0x7f6fc81985c0 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.548+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/2110291948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 0x5612c9edfcc0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00137c0 tx=0x7f6fb0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1d6e985c1bc716 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fbdffb640 1 -- 172.21.15.82:0/2110291948 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0013120 con 0x5612c9edf8e0 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fc7fff640 1 --2- 172.21.15.82:0/3199847729 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6fb40a3d20 0x7f6fb40a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0005e30 tx=0x7f6fc0090700 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fbdffb640 1 -- 172.21.15.82:0/2110291948 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb000f7c0 con 0x5612c9edf8e0 2024-04-21T08:53:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fbdffb640 1 -- 172.21.15.82:0/2110291948 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00132c0 con 0x5612c9edf8e0 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fd07c2740 1 -- 172.21.15.82:0/2110291948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 msgr2=0x5612c9edfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/2110291948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 0x5612c9edfcc0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00137c0 tx=0x7f6fb0005850 comp rx=0 tx=0).stop 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fd07c2740 1 -- 172.21.15.82:0/2110291948 shutdown_connections 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/2110291948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9ee8e10 0x5612c9eeb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/2110291948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee0290 0x5612c9ee8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/2110291948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 0x5612c9edfcc0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fd07c2740 1 -- 172.21.15.82:0/2110291948 >> 172.21.15.82:0/2110291948 conn(0x5612c9ed9fb0 msgr2=0x5612c9eda390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fd07c2740 1 -- 172.21.15.82:0/2110291948 shutdown_connections 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.549+0000 7f6fd07c2740 1 -- 172.21.15.82:0/2110291948 wait complete. 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.550+0000 7f6fd07c2740 1 -- start start 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.550+0000 7f6fd07c2740 1 Processor -- start 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.550+0000 7f6fd07c2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9edf8e0 0x5612c9ee5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.550+0000 7f6fd07c2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ee0290 0x5612c9ee6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.550+0000 7f6fd07c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee8e10 0x5612c9ee6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.550+0000 7f6fd07c2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5612c9eebd20 con 0x5612c9ee0290 2024-04-21T08:53:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.550+0000 7f6fd07c2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5612c9dfd580 con 0x5612c9edf8e0 2024-04-21T08:53:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.550+0000 7f6fd07c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5612c9eebea0 con 0x5612c9ee8e10 2024-04-21T08:53:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.550+0000 7f6fd07c2740 10 client.0 _ll_register_callbacks cb 0x5612c9e03650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.550+0000 7f6fc77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ee0290 0x5612c9ee6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fc77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ee0290 0x5612c9ee6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37412/0 (socket says 172.21.15.82:37412) 2024-04-21T08:53:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fc77fe640 1 -- 172.21.15.82:0/1633144947 learned_addr learned my addr 172.21.15.82:0/1633144947 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9edf8e0 0x5612c9ee5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee8e10 0x5612c9ee6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fc77fe640 1 -- 172.21.15.82:0/1633144947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee8e10 msgr2=0x5612c9ee6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fc77fe640 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee8e10 0x5612c9ee6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fc77fe640 1 -- 172.21.15.82:0/1633144947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9edf8e0 msgr2=0x5612c9ee5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fc77fe640 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9edf8e0 0x5612c9ee5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fc77fe640 1 -- 172.21.15.82:0/1633144947 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612c9c7ef70 con 0x5612c9ee0290 2024-04-21T08:53:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fc77fe640 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ee0290 0x5612c9ee6350 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800bf60 tx=0x7f6fb800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849a18492efc7f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fa6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fa6ffd640 1 -- 172.21.15.82:0/1633144947 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8014430 con 0x5612c9ee0290 2024-04-21T08:53:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fa6ffd640 1 -- 172.21.15.82:0/1633144947 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6fb80145d0 con 0x5612c9ee0290 2024-04-21T08:53:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1633144947 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb000f7c0 con 0x5612c9ee0290 2024-04-21T08:53:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fa6ffd640 1 -- 172.21.15.82:0/1633144947 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6fb8004020 con 0x5612c9ee0290 2024-04-21T08:53:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fd07c2740 10 client.17385 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.551+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1633144947 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5612c9b16170 con 0x5612c9ee0290 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.552+0000 7f6fa6ffd640 1 -- 172.21.15.82:0/1633144947 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb000f7c0 con 0x5612c9ee0290 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.552+0000 7f6fa6ffd640 1 -- 172.21.15.82:0/1633144947 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f6fb8013070 con 0x5612c9ee0290 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.552+0000 7f6fd07c2740 10 client.17385 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.552+0000 7f6fd07c2740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.552+0000 7f6fd07c2740 20 client.17385 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.552+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5612c9ef93a0 0x5612c9efb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.552+0000 7f6fd07c2740 4 client.17385 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.552+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1633144947 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612c9e51920 con 0x5612c9ef93a0 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.552+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5612c9ef93a0 0x5612c9efb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5612c9ef93a0 0x5612c9efb820 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f172ed699ac532a server_cookie=eddfc0c7425754a8 in_seq=0 out_seq=0 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fa6ffd640 10 client.17385 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fa6ffd640 1 -- 172.21.15.82:0/1633144947 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612c9e51920 con 0x5612c9ef93a0 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fa6ffd640 10 client.17385 handle_command_reply: tid=0 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fd07c2740 1 client.17385 shutdown 2024-04-21T08:53:22.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1633144947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ee0290 msgr2=0x5612c9ee6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ee0290 0x5612c9ee6350 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800bf60 tx=0x7f6fb800bf90 comp rx=0 tx=0).stop 2024-04-21T08:53:22.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1633144947 shutdown_connections 2024-04-21T08:53:22.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5612c9ef93a0 0x5612c9efb820 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee8e10 0x5612c9ee6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ee0290 0x5612c9ee6350 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/1633144947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9edf8e0 0x5612c9ee5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fa6ffd640 0 client.17385 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:22.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.553+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1633144947 >> 172.21.15.82:0/1633144947 conn(0x5612c9ed9fb0 msgr2=0x5612c9edbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:22.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.554+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1633144947 shutdown_connections 2024-04-21T08:53:22.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.554+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1633144947 wait complete. 2024-04-21T08:53:22.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.554+0000 7f6fd07c2740 20 client.17385 trim_cache size 0 max 16384 2024-04-21T08:53:22.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.571+0000 7f6fd07c2740 1 Processor -- start 2024-04-21T08:53:22.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.571+0000 7f6fd07c2740 1 -- start start 2024-04-21T08:53:22.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.571+0000 7f6fd07c2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ed9fb0 0x5612c9ee8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.571+0000 7f6fd07c2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9edf8e0 0x5612c9ee4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.571+0000 7f6fd07c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee0290 0x5612c9ef4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.571+0000 7f6fd07c2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81af350 con 0x5612c9ed9fb0 2024-04-21T08:53:22.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.571+0000 7f6fd07c2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81b0f20 con 0x5612c9edf8e0 2024-04-21T08:53:22.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.571+0000 7f6fd07c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81afa20 con 0x5612c9ee0290 2024-04-21T08:53:22.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fc6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ed9fb0 0x5612c9ee8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fc6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ed9fb0 0x5612c9ee8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37424/0 (socket says 172.21.15.82:37424) 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fc6ffd640 1 -- 172.21.15.82:0/3683632300 learned_addr learned my addr 172.21.15.82:0/3683632300 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fc77fe640 1 --2- 172.21.15.82:0/3683632300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9edf8e0 0x5612c9ee4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fc6ffd640 1 -- 172.21.15.82:0/3683632300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee0290 msgr2=0x5612c9ef4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/3683632300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee0290 0x5612c9ef4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fc6ffd640 1 -- 172.21.15.82:0/3683632300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9edf8e0 msgr2=0x5612c9ee4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/3683632300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9edf8e0 0x5612c9ee4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fc6ffd640 1 -- 172.21.15.82:0/3683632300 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612c9ed43f0 con 0x5612c9ed9fb0 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/3683632300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ed9fb0 0x5612c9ee8540 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009ca0 tx=0x7f6fb0009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee0fbe8edf263dc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fbd7fa640 1 -- 172.21.15.82:0/3683632300 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000f790 con 0x5612c9ed9fb0 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fbd7fa640 1 -- 172.21.15.82:0/3683632300 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb00050c0 con 0x5612c9ed9fb0 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.572+0000 7f6fbd7fa640 1 -- 172.21.15.82:0/3683632300 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00036e0 con 0x5612c9ed9fb0 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.573+0000 7f6fd07c2740 1 -- 172.21.15.82:0/3683632300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ed9fb0 msgr2=0x5612c9ee8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.573+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/3683632300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ed9fb0 0x5612c9ee8540 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009ca0 tx=0x7f6fb0009eb0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.573+0000 7f6fd07c2740 1 -- 172.21.15.82:0/3683632300 shutdown_connections 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.573+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/3683632300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ee0290 0x5612c9ef4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.573+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/3683632300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9edf8e0 0x5612c9ee4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.573+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/3683632300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9ed9fb0 0x5612c9ee8540 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.573+0000 7f6fd07c2740 1 -- 172.21.15.82:0/3683632300 >> 172.21.15.82:0/3683632300 conn(0x5612c9ee09e0 msgr2=0x5612c9ee6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.573+0000 7f6fd07c2740 1 -- 172.21.15.82:0/3683632300 shutdown_connections 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.573+0000 7f6fd07c2740 1 -- 172.21.15.82:0/3683632300 wait complete. 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.573+0000 7f6fd07c2740 1 -- start start 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.573+0000 7f6fd07c2740 1 Processor -- start 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.573+0000 7f6fd07c2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ed9fb0 0x5612c9ef1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fd07c2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 0x5612c9eee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fd07c2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9ee0290 0x5612c9eee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fd07c2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81b0850 con 0x5612c9edf8e0 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fd07c2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81b0180 con 0x5612c9ee0290 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fd07c2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81ad330 con 0x5612c9ed9fb0 2024-04-21T08:53:22.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fd07c2740 10 client.0 _ll_register_callbacks cb 0x7f6fb800b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fc77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 0x5612c9eee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fc77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 0x5612c9eee0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37436/0 (socket says 172.21.15.82:37436) 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fc77fe640 1 -- 172.21.15.82:0/1862104570 learned_addr learned my addr 172.21.15.82:0/1862104570 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9ee0290 0x5612c9eee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ed9fb0 0x5612c9ef1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fc77fe640 1 -- 172.21.15.82:0/1862104570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ed9fb0 msgr2=0x5612c9ef1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fc77fe640 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ed9fb0 0x5612c9ef1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fc77fe640 1 -- 172.21.15.82:0/1862104570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9ee0290 msgr2=0x5612c9eee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fc77fe640 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9ee0290 0x5612c9eee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fc77fe640 1 -- 172.21.15.82:0/1862104570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612c9f5f9b0 con 0x5612c9edf8e0 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fc77fe640 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 0x5612c9eee0c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8000c00 tx=0x7f6fb800b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fffed6a769c072 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fa6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fa6ffd640 1 -- 172.21.15.82:0/1862104570 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8004490 con 0x5612c9edf8e0 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fa6ffd640 1 -- 172.21.15.82:0/1862104570 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f6fb8003920 con 0x5612c9edf8e0 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1862104570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb00050c0 con 0x5612c9edf8e0 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fa6ffd640 1 -- 172.21.15.82:0/1862104570 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6fb8003b20 con 0x5612c9edf8e0 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fd07c2740 10 client.17391 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.574+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1862104570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5612c9f60aa0 con 0x5612c9edf8e0 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.575+0000 7f6fa6ffd640 1 -- 172.21.15.82:0/1862104570 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb00050c0 con 0x5612c9edf8e0 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.575+0000 7f6fa6ffd640 1 -- 172.21.15.82:0/1862104570 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f6fb8014c80 con 0x5612c9edf8e0 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.575+0000 7f6fd07c2740 10 client.17391 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.575+0000 7f6fd07c2740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.575+0000 7f6fd07c2740 20 client.17391 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.575+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5612c9ee8e10 0x5612c9f63230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.575+0000 7f6fd07c2740 4 client.17391 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.575+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1862104570 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5612c9e4b790 con 0x5612c9ee8e10 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.575+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5612c9ee8e10 0x5612c9f63230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.575+0000 7f6fa6ffd640 10 client.17391 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:22.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.575+0000 7f6fc6ffd640 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5612c9ee8e10 0x5612c9f63230 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac818f9e2a2f9a4 server_cookie=7651da7296737e35 in_seq=0 out_seq=0 2024-04-21T08:53:22.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:22 smithi082 ceph-mon[27967]: pgmap v193: 129 pgs: 129 active+clean; 883 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 662 op/s 2024-04-21T08:53:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:22 smithi106 ceph-mon[31072]: pgmap v193: 129 pgs: 129 active+clean; 883 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 662 op/s 2024-04-21T08:53:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:22 smithi134 ceph-mon[31772]: pgmap v193: 129 pgs: 129 active+clean; 883 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 662 op/s 2024-04-21T08:53:22.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fa6ffd640 1 -- 172.21.15.82:0/1862104570 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5612c9e4b790 con 0x5612c9ee8e10 2024-04-21T08:53:22.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fa6ffd640 10 client.17391 handle_command_reply: tid=0 2024-04-21T08:53:22.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fd07c2740 1 client.17391 shutdown 2024-04-21T08:53:22.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1862104570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 msgr2=0x5612c9eee0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 0x5612c9eee0c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8000c00 tx=0x7f6fb800b3a0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1862104570 shutdown_connections 2024-04-21T08:53:22.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5612c9ee8e10 0x5612c9f63230 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612c9ee0290 0x5612c9eee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fa6ffd640 0 client.17391 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:22.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612c9edf8e0 0x5612c9eee0c0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fd07c2740 1 --2- 172.21.15.82:0/1862104570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5612c9ed9fb0 0x5612c9ef1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1862104570 >> 172.21.15.82:0/1862104570 conn(0x5612c9ee09e0 msgr2=0x5612c9ee5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:22.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1862104570 shutdown_connections 2024-04-21T08:53:22.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fd07c2740 1 -- 172.21.15.82:0/1862104570 wait complete. 2024-04-21T08:53:22.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.737+0000 7f6fd07c2740 20 client.17391 trim_cache size 0 max 16384 2024-04-21T08:53:22.739 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:53:22.739 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:53:22.739 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "1df5c305-d29f-46bd-88d2-1f62f7d52625", 2024-04-21T08:53:22.739 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:53:22.739 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:53:22.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.738+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/3199847729 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6fb40a3d20 msgr2=0x7f6fb40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.738+0000 7f6fcdaf2640 1 --2- 172.21.15.82:0/3199847729 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6fb40a3d20 0x7f6fb40a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0005e30 tx=0x7f6fc0090700 comp rx=0 tx=0).stop 2024-04-21T08:53:22.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.738+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/3199847729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc81985c0 msgr2=0x7f6fc8198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.738+0000 7f6fcdaf2640 1 --2- 172.21.15.82:0/3199847729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc81985c0 0x7f6fc8198a70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6fc002c730 tx=0x7f6fc0035d60 comp rx=0 tx=0).stop 2024-04-21T08:53:22.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.738+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/3199847729 shutdown_connections 2024-04-21T08:53:22.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.738+0000 7f6fcdaf2640 1 --2- 172.21.15.82:0/3199847729 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6fb40a3d20 0x7f6fb40a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.738+0000 7f6fcdaf2640 1 --2- 172.21.15.82:0/3199847729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc81985c0 0x7f6fc8198a70 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.738+0000 7f6fcdaf2640 1 --2- 172.21.15.82:0/3199847729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fc8106160 0x7f6fc819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.738+0000 7f6fcdaf2640 1 --2- 172.21.15.82:0/3199847729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fc8101090 0x7f6fc819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.738+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/3199847729 >> 172.21.15.82:0/3199847729 conn(0x7f6fc8077f50 msgr2=0x7f6fc80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:22.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.738+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/3199847729 shutdown_connections 2024-04-21T08:53:22.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.738+0000 7f6fcdaf2640 1 -- 172.21.15.82:0/3199847729 wait complete. 2024-04-21T08:53:22.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:53:22.756 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:53:22.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.914+0000 7f2fd798f640 1 -- 172.21.15.82:0/1478174332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fd0101a30 msgr2=0x7f2fd0105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.914+0000 7f2fd798f640 1 --2- 172.21.15.82:0/1478174332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fd0101a30 0x7f2fd0105a70 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009900 tx=0x7f2fc402bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:22.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.914+0000 7f2fd798f640 1 -- 172.21.15.82:0/1478174332 shutdown_connections 2024-04-21T08:53:22.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.914+0000 7f2fd798f640 1 --2- 172.21.15.82:0/1478174332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fd0106150 0x7f2fd010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.914+0000 7f2fd798f640 1 --2- 172.21.15.82:0/1478174332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fd0101a30 0x7f2fd0105a70 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.914+0000 7f2fd798f640 1 --2- 172.21.15.82:0/1478174332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fd0101080 0x7f2fd0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.914+0000 7f2fd798f640 1 -- 172.21.15.82:0/1478174332 >> 172.21.15.82:0/1478174332 conn(0x7f2fd0078000 msgr2=0x7f2fd0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:22.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.914+0000 7f2fd798f640 1 -- 172.21.15.82:0/1478174332 shutdown_connections 2024-04-21T08:53:22.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.914+0000 7f2fd798f640 1 -- 172.21.15.82:0/1478174332 wait complete. 2024-04-21T08:53:22.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd798f640 1 Processor -- start 2024-04-21T08:53:22.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd798f640 1 -- start start 2024-04-21T08:53:22.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd798f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fd0101080 0x7f2fd019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd798f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fd0106150 0x7f2fd019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd798f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fd0198590 0x7f2fd0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd798f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd0112210 con 0x7f2fd0198590 2024-04-21T08:53:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd798f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd0112090 con 0x7f2fd0106150 2024-04-21T08:53:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd798f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd0112390 con 0x7f2fd0101080 2024-04-21T08:53:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd5f2c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fd0198590 0x7f2fd0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd5f2c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fd0198590 0x7f2fd0198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60588/0 (socket says 172.21.15.82:60588) 2024-04-21T08:53:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd5f2c640 1 -- 172.21.15.82:0/2270337288 learned_addr learned my addr 172.21.15.82:0/2270337288 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:22.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fd0106150 0x7f2fd019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.915+0000 7f2fd572b640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fd0101080 0x7f2fd019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.916+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/2270337288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fd0101080 msgr2=0x7f2fd019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.916+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fd0101080 0x7f2fd019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.916+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/2270337288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fd0198590 msgr2=0x7f2fd0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.916+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fd0198590 0x7f2fd0198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.916+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/2270337288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd01990b0 con 0x7f2fd0106150 2024-04-21T08:53:22.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.918+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fd0106150 0x7f2fd019e920 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4036040 tx=0x7f2fc40045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:22.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.918+0000 7f2fc27fc640 1 -- 172.21.15.82:0/2270337288 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc402cb70 con 0x7f2fd0106150 2024-04-21T08:53:22.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.918+0000 7f2fc27fc640 1 -- 172.21.15.82:0/2270337288 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc40052b0 con 0x7f2fd0106150 2024-04-21T08:53:22.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.919+0000 7f2fd798f640 1 -- 172.21.15.82:0/2270337288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd0199340 con 0x7f2fd0106150 2024-04-21T08:53:22.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.919+0000 7f2fc27fc640 1 -- 172.21.15.82:0/2270337288 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc403eda0 con 0x7f2fd0106150 2024-04-21T08:53:22.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.919+0000 7f2fd798f640 1 -- 172.21.15.82:0/2270337288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd010e8d0 con 0x7f2fd0106150 2024-04-21T08:53:22.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.922+0000 7f2fda57f740 1 Processor -- start 2024-04-21T08:53:22.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.922+0000 7f2fda57f740 1 -- start start 2024-04-21T08:53:22.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.923+0000 7f2fda57f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557193fd580 0x5557193fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.923+0000 7f2fda57f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557193fdea0 0x555719406540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.923+0000 7f2fc27fc640 1 -- 172.21.15.82:0/2270337288 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2fc4004c50 con 0x7f2fd0106150 2024-04-21T08:53:22.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.923+0000 7f2fda57f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555719406a80 0x555719408e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.923+0000 7f2fda57f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5557193621e0 con 0x5557193fdea0 2024-04-21T08:53:22.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.923+0000 7f2fda57f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5557192ff390 con 0x5557193fd580 2024-04-21T08:53:22.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.923+0000 7f2fda57f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55571935e1a0 con 0x555719406a80 2024-04-21T08:53:22.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.923+0000 7f2fd572b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557193fdea0 0x555719406540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.923+0000 7f2fd5f2c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557193fd580 0x5557193fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.923+0000 7f2fd572b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557193fdea0 0x555719406540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60598/0 (socket says 172.21.15.82:60598) 2024-04-21T08:53:22.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.923+0000 7f2fd572b640 1 -- 172.21.15.82:0/1160648180 learned_addr learned my addr 172.21.15.82:0/1160648180 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:22.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.923+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/1160648180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555719406a80 0x555719408e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.924+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/1160648180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557193fd580 msgr2=0x5557193fd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.924+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/1160648180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557193fd580 0x5557193fd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.924+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/1160648180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557193fdea0 msgr2=0x555719406540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.924+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/1160648180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557193fdea0 0x555719406540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.924+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/1160648180 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55571913e7d0 con 0x555719406a80 2024-04-21T08:53:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.924+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/1160648180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555719406a80 0x555719408e70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40f3b30 tx=0x7f2fc40fcb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1139c4a58a861ed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.924+0000 7f2fc27fc640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2fbc0a3e70 0x7f2fbc0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.925+0000 7f2fc27fc640 1 -- 172.21.15.82:0/2270337288 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f2fc40f66f0 con 0x7f2fd0106150 2024-04-21T08:53:22.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.926+0000 7f2fd5f2c640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2fbc0a3e70 0x7f2fbc0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.927+0000 7f2fbb7fe640 1 -- 172.21.15.82:0/1160648180 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40c1ce0 con 0x555719406a80 2024-04-21T08:53:22.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.927+0000 7f2fd5f2c640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2fbc0a3e70 0x7f2fbc0a6330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80049d0 tx=0x7f2fc8009500 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:22.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.927+0000 7f2fbb7fe640 1 -- 172.21.15.82:0/1160648180 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc4106020 con 0x555719406a80 2024-04-21T08:53:22.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.927+0000 7f2fbb7fe640 1 -- 172.21.15.82:0/1160648180 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40c1ec0 con 0x555719406a80 2024-04-21T08:53:22.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.928+0000 7f2fda57f740 1 -- 172.21.15.82:0/1160648180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555719406a80 msgr2=0x555719408e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.928+0000 7f2fda57f740 1 --2- 172.21.15.82:0/1160648180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555719406a80 0x555719408e70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40f3b30 tx=0x7f2fc40fcb20 comp rx=0 tx=0).stop 2024-04-21T08:53:22.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.928+0000 7f2fda57f740 1 -- 172.21.15.82:0/1160648180 shutdown_connections 2024-04-21T08:53:22.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.928+0000 7f2fda57f740 1 --2- 172.21.15.82:0/1160648180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555719406a80 0x555719408e70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.928+0000 7f2fda57f740 1 --2- 172.21.15.82:0/1160648180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557193fdea0 0x555719406540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.928+0000 7f2fda57f740 1 --2- 172.21.15.82:0/1160648180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557193fd580 0x5557193fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.928+0000 7f2fda57f740 1 -- 172.21.15.82:0/1160648180 >> 172.21.15.82:0/1160648180 conn(0x5557193f7cb0 msgr2=0x5557193f8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:22.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.928+0000 7f2fda57f740 1 -- 172.21.15.82:0/1160648180 shutdown_connections 2024-04-21T08:53:22.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.928+0000 7f2fda57f740 1 -- 172.21.15.82:0/1160648180 wait complete. 2024-04-21T08:53:22.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.929+0000 7f2fda57f740 1 -- start start 2024-04-21T08:53:22.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.929+0000 7f2fda57f740 1 Processor -- start 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.930+0000 7f2fda57f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 0x555719403af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.930+0000 7f2fda57f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557193fdea0 0x555719404030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.930+0000 7f2fda57f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555719406a80 0x555719404570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.930+0000 7f2fda57f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555719409630 con 0x5557193fdea0 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.930+0000 7f2fd572b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 0x555719403af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.930+0000 7f2fd572b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 0x555719403af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:39756/0 (socket says 172.21.15.82:39756) 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.930+0000 7f2fd572b640 1 -- 172.21.15.82:0/1438597338 learned_addr learned my addr 172.21.15.82:0/1438597338 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.930+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557193fdea0 0x555719404030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.930+0000 7f2fda57f740 1 -- 172.21.15.82:0/1438597338 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55571931b2f0 con 0x555719406a80 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.930+0000 7f2fda57f740 1 -- 172.21.15.82:0/1438597338 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5557194097b0 con 0x5557193fd580 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.930+0000 7f2fda57f740 10 client.0 _ll_register_callbacks cb 0x555719328580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.930+0000 7f2fd5f2c640 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555719406a80 0x555719404570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/1438597338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 msgr2=0x555719403af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 0x555719403af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/1438597338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555719406a80 msgr2=0x555719404570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555719406a80 0x555719404570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/1438597338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557190743f0 con 0x5557193fdea0 2024-04-21T08:53:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557193fdea0 0x555719404030 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40c2310 tx=0x7f2fc403e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828e04d3e987f118 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fb8ff9640 1 -- 172.21.15.82:0/1438597338 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40f5070 con 0x5557193fdea0 2024-04-21T08:53:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fb8ff9640 1 -- 172.21.15.82:0/1438597338 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f2fc4005460 con 0x5557193fdea0 2024-04-21T08:53:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fda57f740 1 -- 172.21.15.82:0/1438597338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc4106020 con 0x5557193fdea0 2024-04-21T08:53:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fb8ff9640 1 -- 172.21.15.82:0/1438597338 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2fc4105070 con 0x5557193fdea0 2024-04-21T08:53:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fda57f740 10 client.17403 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.931+0000 7f2fda57f740 1 -- 172.21.15.82:0/1438597338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5557190340f0 con 0x5557193fdea0 2024-04-21T08:53:22.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.932+0000 7f2fb8ff9640 1 -- 172.21.15.82:0/1438597338 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc40c2380 con 0x5557193fdea0 2024-04-21T08:53:22.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.932+0000 7f2fb8ff9640 1 -- 172.21.15.82:0/1438597338 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f2fc40be9f0 con 0x5557193fdea0 2024-04-21T08:53:22.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.932+0000 7f2fda57f740 10 client.17403 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:22.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.932+0000 7f2fda57f740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:22.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.932+0000 7f2fda57f740 20 client.17403 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:22.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.932+0000 7f2fda57f740 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555719416640 0x555719418a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.932+0000 7f2fda57f740 4 client.17403 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:22.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.932+0000 7f2fda57f740 1 -- 172.21.15.82:0/1438597338 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2fc4106020 con 0x555719416640 2024-04-21T08:53:22.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.932+0000 7f2fd572b640 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555719416640 0x555719418a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.932+0000 7f2fb8ff9640 10 client.17403 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:22.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.932+0000 7f2fd572b640 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555719416640 0x555719418a30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c8cff2c0755f7b7 server_cookie=ed32f5cb2485b31f in_seq=0 out_seq=0 2024-04-21T08:53:22.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.934+0000 7f2fb8ff9640 1 -- 172.21.15.82:0/1438597338 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f2fc4106020 con 0x555719416640 2024-04-21T08:53:22.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.934+0000 7f2fb8ff9640 10 client.17403 handle_command_reply: tid=0 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.934+0000 7f2fda57f740 1 client.17403 shutdown 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.935+0000 7f2fda57f740 1 -- 172.21.15.82:0/1438597338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557193fdea0 msgr2=0x555719404030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.935+0000 7f2fda57f740 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557193fdea0 0x555719404030 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40c2310 tx=0x7f2fc403e8a0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.935+0000 7f2fda57f740 1 -- 172.21.15.82:0/1438597338 shutdown_connections 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.935+0000 7f2fda57f740 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555719416640 0x555719418a30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.935+0000 7f2fda57f740 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555719406a80 0x555719404570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.935+0000 7f2fda57f740 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557193fdea0 0x555719404030 secure :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40c2310 tx=0x7f2fc403e8a0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.935+0000 7f2fda57f740 1 --2- 172.21.15.82:0/1438597338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 0x555719403af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.935+0000 7f2fda57f740 1 -- 172.21.15.82:0/1438597338 >> 172.21.15.82:0/1438597338 conn(0x5557193f7cb0 msgr2=0x5557193f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.935+0000 7f2fb8ff9640 0 client.17403 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.935+0000 7f2fda57f740 1 -- 172.21.15.82:0/1438597338 shutdown_connections 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.935+0000 7f2fda57f740 1 -- 172.21.15.82:0/1438597338 wait complete. 2024-04-21T08:53:22.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.936+0000 7f2fda57f740 20 client.17403 trim_cache size 0 max 16384 2024-04-21T08:53:22.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.970+0000 7f2fda57f740 1 Processor -- start 2024-04-21T08:53:22.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.970+0000 7f2fda57f740 1 -- start start 2024-04-21T08:53:22.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.970+0000 7f2fda57f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557193fd580 0x555719406220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.970+0000 7f2fda57f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555719406a80 0x555719412460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.970+0000 7f2fda57f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557194041c0 0x55571941e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.970+0000 7f2fda57f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd01ad170 con 0x555719406a80 2024-04-21T08:53:22.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.970+0000 7f2fda57f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd01aed40 con 0x5557193fd580 2024-04-21T08:53:22.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.970+0000 7f2fda57f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd01ad840 con 0x5557194041c0 2024-04-21T08:53:22.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fd4f2a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555719406a80 0x555719412460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fd5f2c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557194041c0 0x55571941e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fd4f2a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555719406a80 0x555719412460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60620/0 (socket says 172.21.15.82:60620) 2024-04-21T08:53:22.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/3911837216 learned_addr learned my addr 172.21.15.82:0/3911837216 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:22.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/3911837216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557194041c0 msgr2=0x55571941e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/3911837216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557194041c0 0x55571941e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/3911837216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557193fd580 msgr2=0x555719406220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/3911837216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557193fd580 0x555719406220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fd4f2a640 1 -- 172.21.15.82:0/3911837216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555719408e70 con 0x555719406a80 2024-04-21T08:53:22.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/3911837216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555719406a80 0x555719412460 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4006010 tx=0x7f2fc40056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb2eccffae9a276 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:22.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fbaffd640 1 -- 172.21.15.82:0/3911837216 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40f5070 con 0x555719406a80 2024-04-21T08:53:22.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fbaffd640 1 -- 172.21.15.82:0/3911837216 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc40c2a80 con 0x555719406a80 2024-04-21T08:53:22.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.971+0000 7f2fbaffd640 1 -- 172.21.15.82:0/3911837216 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40c2150 con 0x555719406a80 2024-04-21T08:53:22.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.972+0000 7f2fda57f740 1 -- 172.21.15.82:0/3911837216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555719406a80 msgr2=0x555719412460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.972+0000 7f2fda57f740 1 --2- 172.21.15.82:0/3911837216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555719406a80 0x555719412460 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4006010 tx=0x7f2fc40056f0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.973+0000 7f2fda57f740 1 -- 172.21.15.82:0/3911837216 shutdown_connections 2024-04-21T08:53:22.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.973+0000 7f2fda57f740 1 --2- 172.21.15.82:0/3911837216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557194041c0 0x55571941e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.973+0000 7f2fda57f740 1 --2- 172.21.15.82:0/3911837216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555719406a80 0x555719412460 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.973+0000 7f2fda57f740 1 --2- 172.21.15.82:0/3911837216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557193fd580 0x555719406220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.973+0000 7f2fda57f740 1 -- 172.21.15.82:0/3911837216 >> 172.21.15.82:0/3911837216 conn(0x5557193f7cb0 msgr2=0x555719403ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.973+0000 7f2fda57f740 1 -- 172.21.15.82:0/3911837216 shutdown_connections 2024-04-21T08:53:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.973+0000 7f2fda57f740 1 -- 172.21.15.82:0/3911837216 wait complete. 2024-04-21T08:53:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.973+0000 7f2fda57f740 1 -- start start 2024-04-21T08:53:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.973+0000 7f2fda57f740 1 Processor -- start 2024-04-21T08:53:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.973+0000 7f2fda57f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 0x55571940aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.973+0000 7f2fda57f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557194041c0 0x55571940af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fda57f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555719406a80 0x555719484080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fda57f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd01ae670 con 0x5557194041c0 2024-04-21T08:53:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fda57f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd01adfa0 con 0x555719406a80 2024-04-21T08:53:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fda57f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd01ab150 con 0x5557193fd580 2024-04-21T08:53:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fda57f740 10 client.0 _ll_register_callbacks cb 0x7f2fcc00b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:22.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fd572b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 0x55571940aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fd572b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 0x55571940aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:39774/0 (socket says 172.21.15.82:39774) 2024-04-21T08:53:22.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fd572b640 1 -- 172.21.15.82:0/4134677435 learned_addr learned my addr 172.21.15.82:0/4134677435 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:22.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557194041c0 0x55571940af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fd5f2c640 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555719406a80 0x555719484080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fd572b640 1 -- 172.21.15.82:0/4134677435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555719406a80 msgr2=0x555719484080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fd572b640 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555719406a80 0x555719484080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fd572b640 1 -- 172.21.15.82:0/4134677435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557194041c0 msgr2=0x55571940af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fd572b640 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557194041c0 0x55571940af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fd572b640 1 -- 172.21.15.82:0/4134677435 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557194845c0 con 0x5557193fd580 2024-04-21T08:53:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557194041c0 0x55571940af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fd572b640 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 0x55571940aa10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x555719418f70 tx=0x7f2fcc006f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5042e7b8efebff67 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.974+0000 7f2fb8ff9640 1 -- 172.21.15.82:0/4134677435 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc016c50 con 0x5557193fd580 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.975+0000 7f2fda57f740 1 -- 172.21.15.82:0/4134677435 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc40c2a80 con 0x5557193fd580 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.975+0000 7f2fb8ff9640 1 -- 172.21.15.82:0/4134677435 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f2fcc00f040 con 0x5557193fd580 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.975+0000 7f2fb8ff9640 1 -- 172.21.15.82:0/4134677435 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2fcc02d050 con 0x5557193fd580 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.975+0000 7f2fda57f740 10 client.26443 fetch_fsmap learned FSMap version 36 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.975+0000 7f2fda57f740 1 -- 172.21.15.82:0/4134677435 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5557194858e0 con 0x5557193fd580 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.975+0000 7f2fb8ff9640 1 -- 172.21.15.82:0/4134677435 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc40c2a80 con 0x5557193fd580 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.975+0000 7f2fb8ff9640 1 -- 172.21.15.82:0/4134677435 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f2fcc025720 con 0x5557193fd580 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.975+0000 7f2fda57f740 10 client.26443 fetch_fsmap finished waiting for FSMap version 36 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.975+0000 7f2fda57f740 10 client.26443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.975+0000 7f2fda57f740 20 client.26443 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fda57f740 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555719486060 0x555719488480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fda57f740 4 client.26443 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fda57f740 1 -- 172.21.15.82:0/4134677435 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55571936fc00 con 0x555719486060 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555719486060 0x555719488480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fb8ff9640 10 client.26443 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fd4f2a640 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555719486060 0x555719488480 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3df0fafc3ad76c server_cookie=ee0938a386a94cd in_seq=0 out_seq=0 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fb8ff9640 1 -- 172.21.15.82:0/4134677435 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55571936fc00 con 0x555719486060 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fb8ff9640 10 client.26443 handle_command_reply: tid=0 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fda57f740 1 client.26443 shutdown 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fda57f740 1 -- 172.21.15.82:0/4134677435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 msgr2=0x55571940aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fda57f740 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 0x55571940aa10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x555719418f70 tx=0x7f2fcc006f50 comp rx=0 tx=0).stop 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fda57f740 1 -- 172.21.15.82:0/4134677435 shutdown_connections 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fda57f740 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555719486060 0x555719488480 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fda57f740 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555719406a80 0x555719484080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fda57f740 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557194041c0 0x55571940af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fb8ff9640 0 client.26443 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fda57f740 1 --2- 172.21.15.82:0/4134677435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557193fd580 0x55571940aa10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.976+0000 7f2fda57f740 1 -- 172.21.15.82:0/4134677435 >> 172.21.15.82:0/4134677435 conn(0x5557193f7cb0 msgr2=0x5557193f64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:22.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fda57f740 1 -- 172.21.15.82:0/4134677435 shutdown_connections 2024-04-21T08:53:22.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fda57f740 1 -- 172.21.15.82:0/4134677435 wait complete. 2024-04-21T08:53:22.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fda57f740 20 client.26443 trim_cache size 0 max 16384 2024-04-21T08:53:22.985 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:53:22.985 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:53:22.985 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:53:22.985 INFO:teuthology.orchestra.run.smithi082.stdout: "1df5c305-d29f-46bd-88d2-1f62f7d52625": { 2024-04-21T08:53:22.985 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:53:22.986 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "1df5c305-d29f-46bd-88d2-1f62f7d52625", 2024-04-21T08:53:22.986 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:53:22.986 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:53:22.986 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:53:22.986 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:53:22.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fd798f640 1 -- 172.21.15.82:0/2270337288 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2fbc0a3e70 msgr2=0x7f2fbc0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fd798f640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2fbc0a3e70 0x7f2fbc0a6330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80049d0 tx=0x7f2fc8009500 comp rx=0 tx=0).stop 2024-04-21T08:53:22.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fd798f640 1 -- 172.21.15.82:0/2270337288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fd0106150 msgr2=0x7f2fd019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:22.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fd798f640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fd0106150 0x7f2fd019e920 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4036040 tx=0x7f2fc40045e0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fd798f640 1 -- 172.21.15.82:0/2270337288 shutdown_connections 2024-04-21T08:53:22.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fd798f640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2fbc0a3e70 0x7f2fbc0a6330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fd798f640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fd0198590 0x7f2fd0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fd798f640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fd0106150 0x7f2fd019e920 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fd798f640 1 --2- 172.21.15.82:0/2270337288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fd0101080 0x7f2fd019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:22.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.977+0000 7f2fd798f640 1 -- 172.21.15.82:0/2270337288 >> 172.21.15.82:0/2270337288 conn(0x7f2fd0078000 msgr2=0x7f2fd00fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:22.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.978+0000 7f2fd798f640 1 -- 172.21.15.82:0/2270337288 shutdown_connections 2024-04-21T08:53:22.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:22.978+0000 7f2fd798f640 1 -- 172.21.15.82:0/2270337288 wait complete. 2024-04-21T08:53:22.989 INFO:tasks.cephfs.filesystem:scrub status for tag:1df5c305-d29f-46bd-88d2-1f62f7d52625 - {'path': '/', 'tag': '1df5c305-d29f-46bd-88d2-1f62f7d52625', 'options': 'recursive,force'} 2024-04-21T08:53:23.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:23 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:23.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:23 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:23.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:23 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:23.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:23 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:23.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:23 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:23.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:23 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:23.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:23 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:23.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:23 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:23.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:23 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:23.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:23 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:23.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:23 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:23.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:23 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:23.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:23 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:23.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:23 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:23.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:23 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:23.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:23 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:23.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:23 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:23.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:23 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:23.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:23 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:23.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:23 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:23.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:23 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:53:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: pgmap v194: 129 pgs: 129 active+clean; 859 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.4 MiB/s wr, 848 op/s 2024-04-21T08:53:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:24.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:24 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:53:24.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: pgmap v194: 129 pgs: 129 active+clean; 859 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.4 MiB/s wr, 848 op/s 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:24.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:24.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:24.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:24.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:24.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:24 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:53:24.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:24.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: pgmap v194: 129 pgs: 129 active+clean; 859 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.4 MiB/s wr, 848 op/s 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:24.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:24.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:24.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:24.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:24.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:24 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:25.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:25 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:53:25.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:25 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:53:25.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:25 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:53:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:26 smithi082 ceph-mon[27967]: pgmap v195: 129 pgs: 129 active+clean; 859 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 284 op/s 2024-04-21T08:53:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:26 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:53:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:26 smithi106 ceph-mon[31072]: pgmap v195: 129 pgs: 129 active+clean; 859 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 284 op/s 2024-04-21T08:53:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:26 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:53:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:26 smithi134 ceph-mon[31772]: pgmap v195: 129 pgs: 129 active+clean; 859 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 284 op/s 2024-04-21T08:53:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:26 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:53:28.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:28 smithi082 ceph-mon[27967]: pgmap v196: 129 pgs: 129 active+clean; 867 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 453 op/s 2024-04-21T08:53:28.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:28 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:53:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:28 smithi106 ceph-mon[31072]: pgmap v196: 129 pgs: 129 active+clean; 867 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 453 op/s 2024-04-21T08:53:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:28 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:53:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:28 smithi134 ceph-mon[31772]: pgmap v196: 129 pgs: 129 active+clean; 867 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 453 op/s 2024-04-21T08:53:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:28 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:53:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.929+0000 7ff70a6c0640 1 -- 172.21.15.82:0/393743104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff704100f80 msgr2=0x7ff70410d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.929+0000 7ff70a6c0640 1 --2- 172.21.15.82:0/393743104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff704100f80 0x7ff70410d440 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009be0 tx=0x7ff6fc02bef0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.929+0000 7ff70a6c0640 1 -- 172.21.15.82:0/393743104 shutdown_connections 2024-04-21T08:53:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.929+0000 7ff70a6c0640 1 --2- 172.21.15.82:0/393743104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff704100f80 0x7ff70410d440 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.929+0000 7ff70a6c0640 1 --2- 172.21.15.82:0/393743104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7041005c0 0x7ff704100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.929+0000 7ff70a6c0640 1 --2- 172.21.15.82:0/393743104 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff704106680 0x7ff704106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.929+0000 7ff70a6c0640 1 -- 172.21.15.82:0/393743104 >> 172.21.15.82:0/393743104 conn(0x7ff7040773e0 msgr2=0x7ff7040fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.929+0000 7ff70a6c0640 1 -- 172.21.15.82:0/393743104 shutdown_connections 2024-04-21T08:53:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.929+0000 7ff70a6c0640 1 -- 172.21.15.82:0/393743104 wait complete. 2024-04-21T08:53:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.930+0000 7ff70a6c0640 1 Processor -- start 2024-04-21T08:53:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.931+0000 7ff70a6c0640 1 -- start start 2024-04-21T08:53:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.931+0000 7ff70a6c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7041005c0 0x7ff70410d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.931+0000 7ff70a6c0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff704100f80 0x7ff70410a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.931+0000 7ff70a6c0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff704106680 0x7ff70410a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.931+0000 7ff70a6c0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff70410fca0 con 0x7ff7041005c0 2024-04-21T08:53:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.931+0000 7ff70a6c0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff70410fb20 con 0x7ff704106680 2024-04-21T08:53:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.931+0000 7ff70a6c0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff70410fe20 con 0x7ff704100f80 2024-04-21T08:53:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.932+0000 7ff708c5d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff704106680 0x7ff70410a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.932+0000 7ff708c5d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff704106680 0x7ff70410a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:50532/0 (socket says 172.21.15.82:50532) 2024-04-21T08:53:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.932+0000 7ff708c5d640 1 -- 172.21.15.82:0/2538252374 learned_addr learned my addr 172.21.15.82:0/2538252374 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.932+0000 7ff703fff640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7041005c0 0x7ff70410d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.933+0000 7ff7037fe640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff704100f80 0x7ff70410a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.933+0000 7ff703fff640 1 -- 172.21.15.82:0/2538252374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff704100f80 msgr2=0x7ff70410a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.933+0000 7ff703fff640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff704100f80 0x7ff70410a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.933+0000 7ff703fff640 1 -- 172.21.15.82:0/2538252374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff704106680 msgr2=0x7ff70410a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.933+0000 7ff703fff640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff704106680 0x7ff70410a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.933+0000 7ff703fff640 1 -- 172.21.15.82:0/2538252374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70410b160 con 0x7ff7041005c0 2024-04-21T08:53:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.934+0000 7ff703fff640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7041005c0 0x7ff70410d250 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800d950 tx=0x7ff6f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.935+0000 7ff7017fa640 1 -- 172.21.15.82:0/2538252374 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8009c30 con 0x7ff7041005c0 2024-04-21T08:53:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.935+0000 7ff7017fa640 1 -- 172.21.15.82:0/2538252374 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6f8009dd0 con 0x7ff7041005c0 2024-04-21T08:53:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.935+0000 7ff7017fa640 1 -- 172.21.15.82:0/2538252374 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8010460 con 0x7ff7041005c0 2024-04-21T08:53:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.936+0000 7ff70a6c0640 1 -- 172.21.15.82:0/2538252374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7041ab0d0 con 0x7ff7041005c0 2024-04-21T08:53:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.936+0000 7ff70a6c0640 1 -- 172.21.15.82:0/2538252374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7041ab5a0 con 0x7ff7041005c0 2024-04-21T08:53:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.939+0000 7ff70d382740 1 Processor -- start 2024-04-21T08:53:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.940+0000 7ff70d382740 1 -- start start 2024-04-21T08:53:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.940+0000 7ff70d382740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a6658d0 0x562c3a664620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.940+0000 7ff70d382740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a664b60 0x562c3a664fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.940+0000 7ff70d382740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a66e0e0 0x562c3a6704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.940+0000 7ff70d382740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562c3a5c98d0 con 0x562c3a66e0e0 2024-04-21T08:53:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.940+0000 7ff70d382740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562c3a566e80 con 0x562c3a6658d0 2024-04-21T08:53:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.940+0000 7ff70d382740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562c3a5c5890 con 0x562c3a664b60 2024-04-21T08:53:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.941+0000 7ff708c5d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a6658d0 0x562c3a664620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.941+0000 7ff708c5d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a6658d0 0x562c3a664620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:50540/0 (socket says 172.21.15.82:50540) 2024-04-21T08:53:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.941+0000 7ff708c5d640 1 -- 172.21.15.82:0/1494133324 learned_addr learned my addr 172.21.15.82:0/1494133324 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.941+0000 7ff7017fa640 1 -- 172.21.15.82:0/2538252374 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff6f80035f0 con 0x7ff7041005c0 2024-04-21T08:53:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.942+0000 7ff7017fa640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6ec0a3f20 0x7ff6ec0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.942+0000 7ff7037fe640 1 --2- 172.21.15.82:0/1494133324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a66e0e0 0x562c3a6704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.942+0000 7ff703fff640 1 --2- 172.21.15.82:0/1494133324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a664b60 0x562c3a664fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.942+0000 7ff7037fe640 1 -- 172.21.15.82:0/1494133324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a664b60 msgr2=0x562c3a664fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.942+0000 7ff7037fe640 1 --2- 172.21.15.82:0/1494133324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a664b60 0x562c3a664fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.942+0000 7ff7037fe640 1 -- 172.21.15.82:0/1494133324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a6658d0 msgr2=0x562c3a664620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.942+0000 7ff7037fe640 1 --2- 172.21.15.82:0/1494133324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a6658d0 0x562c3a664620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.942+0000 7ff7037fe640 1 -- 172.21.15.82:0/1494133324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c3a588320 con 0x562c3a66e0e0 2024-04-21T08:53:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.943+0000 7ff7017fa640 1 -- 172.21.15.82:0/2538252374 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff6f80d04b0 con 0x7ff7041005c0 2024-04-21T08:53:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.943+0000 7ff7037fe640 1 --2- 172.21.15.82:0/1494133324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a66e0e0 0x562c3a6704d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff70410bb40 tx=0x7ff6f400e3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919dd2abda6f4aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.943+0000 7ff6ea7fc640 1 -- 172.21.15.82:0/1494133324 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f401c070 con 0x562c3a66e0e0 2024-04-21T08:53:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.943+0000 7ff6ea7fc640 1 -- 172.21.15.82:0/1494133324 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6f40178c0 con 0x562c3a66e0e0 2024-04-21T08:53:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.943+0000 7ff6ea7fc640 1 -- 172.21.15.82:0/1494133324 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400c970 con 0x562c3a66e0e0 2024-04-21T08:53:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.945+0000 7ff708c5d640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6ec0a3f20 0x7ff6ec0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.945+0000 7ff70d382740 1 -- 172.21.15.82:0/1494133324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a66e0e0 msgr2=0x562c3a6704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.945+0000 7ff70d382740 1 --2- 172.21.15.82:0/1494133324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a66e0e0 0x562c3a6704d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff70410bb40 tx=0x7ff6f400e3f0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.945+0000 7ff708c5d640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6ec0a3f20 0x7ff6ec0a63e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc033680 tx=0x7ff6fc009500 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.945+0000 7ff70d382740 1 -- 172.21.15.82:0/1494133324 shutdown_connections 2024-04-21T08:53:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.945+0000 7ff70d382740 1 --2- 172.21.15.82:0/1494133324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a66e0e0 0x562c3a6704d0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.945+0000 7ff70d382740 1 --2- 172.21.15.82:0/1494133324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a664b60 0x562c3a664fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.945+0000 7ff70d382740 1 --2- 172.21.15.82:0/1494133324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a6658d0 0x562c3a664620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.945+0000 7ff70d382740 1 -- 172.21.15.82:0/1494133324 >> 172.21.15.82:0/1494133324 conn(0x562c3a62bc90 msgr2=0x562c3a65f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.945+0000 7ff70d382740 1 -- 172.21.15.82:0/1494133324 shutdown_connections 2024-04-21T08:53:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.945+0000 7ff70d382740 1 -- 172.21.15.82:0/1494133324 wait complete. 2024-04-21T08:53:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.945+0000 7ff70d382740 1 -- start start 2024-04-21T08:53:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.946+0000 7ff70d382740 1 Processor -- start 2024-04-21T08:53:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.946+0000 7ff70d382740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a664b60 0x562c3a66da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.946+0000 7ff70d382740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a6658d0 0x562c3a66ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.946+0000 7ff70d382740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a66e0e0 0x562c3a66b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.946+0000 7ff70d382740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562c3a670e10 con 0x562c3a664b60 2024-04-21T08:53:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.946+0000 7ff70d382740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562c3a551eb0 con 0x562c3a6658d0 2024-04-21T08:53:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.946+0000 7ff70d382740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562c3a670f90 con 0x562c3a66e0e0 2024-04-21T08:53:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.946+0000 7ff70d382740 10 client.0 _ll_register_callbacks cb 0x562c3a57eb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.946+0000 7ff708c5d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a66e0e0 0x562c3a66b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.947+0000 7ff708c5d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a66e0e0 0x562c3a66b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:39818/0 (socket says 172.21.15.82:39818) 2024-04-21T08:53:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.947+0000 7ff708c5d640 1 -- 172.21.15.82:0/273380581 learned_addr learned my addr 172.21.15.82:0/273380581 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.947+0000 7ff703fff640 1 --2- 172.21.15.82:0/273380581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a664b60 0x562c3a66da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.947+0000 7ff708c5d640 1 -- 172.21.15.82:0/273380581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a6658d0 msgr2=0x562c3a66ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.947+0000 7ff708c5d640 1 --2- 172.21.15.82:0/273380581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a6658d0 0x562c3a66ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.947+0000 7ff708c5d640 1 -- 172.21.15.82:0/273380581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a664b60 msgr2=0x562c3a66da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.947+0000 7ff708c5d640 1 --2- 172.21.15.82:0/273380581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a664b60 0x562c3a66da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.947+0000 7ff708c5d640 1 -- 172.21.15.82:0/273380581 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c3a2dc210 con 0x562c3a66e0e0 2024-04-21T08:53:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.948+0000 7ff708c5d640 1 --2- 172.21.15.82:0/273380581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a66e0e0 0x562c3a66b3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc033fc0 tx=0x7ff6fc02cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f52ed72ed7ab0e38 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.950+0000 7ff6e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:53:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.950+0000 7ff6e37fe640 1 -- 172.21.15.82:0/273380581 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc04f070 con 0x562c3a66e0e0 2024-04-21T08:53:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.950+0000 7ff70d382740 1 -- 172.21.15.82:0/273380581 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6f40178c0 con 0x562c3a66e0e0 2024-04-21T08:53:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.950+0000 7ff6e37fe640 1 -- 172.21.15.82:0/273380581 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff6fc034980 con 0x562c3a66e0e0 2024-04-21T08:53:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.953+0000 7ff6e37fe640 1 -- 172.21.15.82:0/273380581 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff6fc059050 con 0x562c3a66e0e0 2024-04-21T08:53:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.953+0000 7ff70d382740 10 client.26452 fetch_fsmap learned FSMap version 39 2024-04-21T08:53:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.953+0000 7ff70d382740 1 -- 172.21.15.82:0/273380581 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562c3a3a8620 con 0x562c3a66e0e0 2024-04-21T08:53:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.954+0000 7ff6e37fe640 1 -- 172.21.15.82:0/273380581 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6f40178c0 con 0x562c3a66e0e0 2024-04-21T08:53:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.955+0000 7ff6e37fe640 1 -- 172.21.15.82:0/273380581 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 39) ==== 8386+0+0 (secure 0 0 0) 0x7ff6fc004330 con 0x562c3a66e0e0 2024-04-21T08:53:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.956+0000 7ff70d382740 10 client.26452 fetch_fsmap finished waiting for FSMap version 39 2024-04-21T08:53:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.956+0000 7ff70d382740 10 client.26452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.956+0000 7ff70d382740 20 client.26452 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.956+0000 7ff70d382740 1 --2- 172.21.15.82:0/273380581 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562c3a67e190 0x562c3a6805e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.956+0000 7ff70d382740 4 client.26452 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.956+0000 7ff70d382740 1 -- 172.21.15.82:0/273380581 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c3a5dd880 con 0x562c3a67e190 2024-04-21T08:53:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.958+0000 7ff703fff640 1 --2- 172.21.15.82:0/273380581 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562c3a67e190 0x562c3a6805e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.959+0000 7ff703fff640 1 --2- 172.21.15.82:0/273380581 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562c3a67e190 0x562c3a6805e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2b198d2689e089 server_cookie=b2bd6633160e6846 in_seq=0 out_seq=0 2024-04-21T08:53:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.959+0000 7ff6e37fe640 10 client.26452 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.960+0000 7ff6e37fe640 1 -- 172.21.15.82:0/273380581 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c3a5dd880 con 0x562c3a67e190 2024-04-21T08:53:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.960+0000 7ff6e37fe640 10 client.26452 handle_command_reply: tid=0 2024-04-21T08:53:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.961+0000 7ff70d382740 1 client.26452 shutdown 2024-04-21T08:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.961+0000 7ff70d382740 1 -- 172.21.15.82:0/273380581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a66e0e0 msgr2=0x562c3a66b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.961+0000 7ff70d382740 1 --2- 172.21.15.82:0/273380581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a66e0e0 0x562c3a66b3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc033fc0 tx=0x7ff6fc02cc60 comp rx=0 tx=0).stop 2024-04-21T08:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.961+0000 7ff70d382740 1 -- 172.21.15.82:0/273380581 shutdown_connections 2024-04-21T08:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.961+0000 7ff70d382740 1 --2- 172.21.15.82:0/273380581 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562c3a67e190 0x562c3a6805e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.961+0000 7ff70d382740 1 --2- 172.21.15.82:0/273380581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a66e0e0 0x562c3a66b3c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.961+0000 7ff70d382740 1 --2- 172.21.15.82:0/273380581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a6658d0 0x562c3a66ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.961+0000 7ff70d382740 1 --2- 172.21.15.82:0/273380581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a664b60 0x562c3a66da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.961+0000 7ff70d382740 1 -- 172.21.15.82:0/273380581 >> 172.21.15.82:0/273380581 conn(0x562c3a62bc90 msgr2=0x562c3a6615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.961+0000 7ff6e37fe640 0 client.26452 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.962+0000 7ff70d382740 1 -- 172.21.15.82:0/273380581 shutdown_connections 2024-04-21T08:53:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.962+0000 7ff70d382740 1 -- 172.21.15.82:0/273380581 wait complete. 2024-04-21T08:53:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.962+0000 7ff70d382740 20 client.26452 trim_cache size 0 max 16384 2024-04-21T08:53:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.988+0000 7ff70d382740 1 Processor -- start 2024-04-21T08:53:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.990+0000 7ff70d382740 1 -- start start 2024-04-21T08:53:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.990+0000 7ff70d382740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a664b60 0x562c3a679d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.990+0000 7ff70d382740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a6658d0 0x562c3a66d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.990+0000 7ff703fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a664b60 0x562c3a679d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.990+0000 7ff703fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a664b60 0x562c3a679d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60656/0 (socket says 172.21.15.82:60656) 2024-04-21T08:53:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.991+0000 7ff7037fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a6658d0 0x562c3a66d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.991+0000 7ff7037fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a6658d0 0x562c3a66d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:39830/0 (socket says 172.21.15.82:39830) 2024-04-21T08:53:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.990+0000 7ff70d382740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a66e0e0 0x562c3a66da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.991+0000 7ff70d382740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041b5a20 con 0x562c3a664b60 2024-04-21T08:53:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.991+0000 7ff70d382740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041b7650 con 0x562c3a66e0e0 2024-04-21T08:53:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.991+0000 7ff70d382740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041b60f0 con 0x562c3a6658d0 2024-04-21T08:53:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.991+0000 7ff703fff640 1 -- 172.21.15.82:0/3648860612 learned_addr learned my addr 172.21.15.82:0/3648860612 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.992+0000 7ff7037fe640 1 -- 172.21.15.82:0/3648860612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a66e0e0 msgr2=0x562c3a66da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.992+0000 7ff708c5d640 1 --2- 172.21.15.82:0/3648860612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a66e0e0 0x562c3a66da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.992+0000 7ff7037fe640 1 --2- 172.21.15.82:0/3648860612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a66e0e0 0x562c3a66da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.992+0000 7ff7037fe640 1 -- 172.21.15.82:0/3648860612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a664b60 msgr2=0x562c3a679d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.992+0000 7ff7037fe640 1 --2- 172.21.15.82:0/3648860612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a664b60 0x562c3a679d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.992+0000 7ff7037fe640 1 -- 172.21.15.82:0/3648860612 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c3a65af30 con 0x562c3a6658d0 2024-04-21T08:53:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.992+0000 7ff703fff640 1 --2- 172.21.15.82:0/3648860612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a664b60 0x562c3a679d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.996+0000 7ff7037fe640 1 --2- 172.21.15.82:0/3648860612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a6658d0 0x562c3a66d550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x562c3a6609e0 tx=0x7ff6f400a850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ab55e7ba7c7314e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.997+0000 7ff6e9ffb640 1 -- 172.21.15.82:0/3648860612 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f401c070 con 0x562c3a6658d0 2024-04-21T08:53:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.997+0000 7ff6e9ffb640 1 -- 172.21.15.82:0/3648860612 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6f4005570 con 0x562c3a6658d0 2024-04-21T08:53:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.997+0000 7ff6e9ffb640 1 -- 172.21.15.82:0/3648860612 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4022410 con 0x562c3a6658d0 2024-04-21T08:53:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.998+0000 7ff70d382740 1 -- 172.21.15.82:0/3648860612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a6658d0 msgr2=0x562c3a66d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.998+0000 7ff70d382740 1 --2- 172.21.15.82:0/3648860612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a6658d0 0x562c3a66d550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x562c3a6609e0 tx=0x7ff6f400a850 comp rx=0 tx=0).stop 2024-04-21T08:53:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.998+0000 7ff70d382740 1 -- 172.21.15.82:0/3648860612 shutdown_connections 2024-04-21T08:53:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.998+0000 7ff70d382740 1 --2- 172.21.15.82:0/3648860612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a66e0e0 0x562c3a66da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.998+0000 7ff70d382740 1 --2- 172.21.15.82:0/3648860612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a6658d0 0x562c3a66d550 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.998+0000 7ff70d382740 1 --2- 172.21.15.82:0/3648860612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a664b60 0x562c3a679d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.998+0000 7ff70d382740 1 -- 172.21.15.82:0/3648860612 >> 172.21.15.82:0/3648860612 conn(0x562c3a62bc90 msgr2=0x562c3a664080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.999+0000 7ff70d382740 1 -- 172.21.15.82:0/3648860612 shutdown_connections 2024-04-21T08:53:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:29.999+0000 7ff70d382740 1 -- 172.21.15.82:0/3648860612 wait complete. 2024-04-21T08:53:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.000+0000 7ff70d382740 1 -- start start 2024-04-21T08:53:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.000+0000 7ff70d382740 1 Processor -- start 2024-04-21T08:53:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.001+0000 7ff70d382740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a664b60 0x562c3a676790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.001+0000 7ff70d382740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a6658d0 0x562c3a671b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.001+0000 7ff70d382740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a66e0e0 0x562c3a6721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.001+0000 7ff70d382740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041b6f50 con 0x562c3a6658d0 2024-04-21T08:53:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.001+0000 7ff70d382740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041b6850 con 0x562c3a66e0e0 2024-04-21T08:53:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.001+0000 7ff703fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a664b60 0x562c3a676790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.001+0000 7ff703fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a664b60 0x562c3a676790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:39838/0 (socket says 172.21.15.82:39838) 2024-04-21T08:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.001+0000 7ff703fff640 1 -- 172.21.15.82:0/87931493 learned_addr learned my addr 172.21.15.82:0/87931493 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.002+0000 7ff708c5d640 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a66e0e0 0x562c3a6721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.002+0000 7ff70d382740 1 -- 172.21.15.82:0/87931493 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7041b39d0 con 0x562c3a664b60 2024-04-21T08:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.002+0000 7ff70d382740 10 client.0 _ll_register_callbacks cb 0x7ff6fc009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.002+0000 7ff7037fe640 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a6658d0 0x562c3a671b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.002+0000 7ff703fff640 1 -- 172.21.15.82:0/87931493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a66e0e0 msgr2=0x562c3a6721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.002+0000 7ff703fff640 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a66e0e0 0x562c3a6721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.002+0000 7ff703fff640 1 -- 172.21.15.82:0/87931493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a6658d0 msgr2=0x562c3a671b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.002+0000 7ff703fff640 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a6658d0 0x562c3a671b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.002+0000 7ff703fff640 1 -- 172.21.15.82:0/87931493 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c3a6e3760 con 0x562c3a664b60 2024-04-21T08:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.003+0000 7ff703fff640 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a664b60 0x562c3a676790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8000c00 tx=0x7ff6f800bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e75aefa4692dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.003+0000 7ff6e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:53:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.003+0000 7ff6e37fe640 1 -- 172.21.15.82:0/87931493 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80b4070 con 0x562c3a664b60 2024-04-21T08:53:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.003+0000 7ff6e37fe640 1 -- 172.21.15.82:0/87931493 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7ff6f8098720 con 0x562c3a664b60 2024-04-21T08:53:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.003+0000 7ff70d382740 1 -- 172.21.15.82:0/87931493 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6f4005570 con 0x562c3a664b60 2024-04-21T08:53:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.003+0000 7ff6e37fe640 1 -- 172.21.15.82:0/87931493 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff6f80a0460 con 0x562c3a664b60 2024-04-21T08:53:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.003+0000 7ff70d382740 10 client.26458 fetch_fsmap learned FSMap version 39 2024-04-21T08:53:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.003+0000 7ff70d382740 1 -- 172.21.15.82:0/87931493 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562c3a6e4910 con 0x562c3a664b60 2024-04-21T08:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.004+0000 7ff6e37fe640 1 -- 172.21.15.82:0/87931493 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6f4005570 con 0x562c3a664b60 2024-04-21T08:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.004+0000 7ff6e37fe640 1 -- 172.21.15.82:0/87931493 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 39) ==== 8386+0+0 (secure 0 0 0) 0x7ff6f80cda60 con 0x562c3a664b60 2024-04-21T08:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.005+0000 7ff70d382740 10 client.26458 fetch_fsmap finished waiting for FSMap version 39 2024-04-21T08:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.005+0000 7ff70d382740 10 client.26458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.005+0000 7ff70d382740 20 client.26458 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.005+0000 7ff70d382740 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562c3a6e5010 0x562c3a6e7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.005+0000 7ff70d382740 4 client.26458 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:53:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.005+0000 7ff70d382740 1 -- 172.21.15.82:0/87931493 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c3a5693d0 con 0x562c3a6e5010 2024-04-21T08:53:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.005+0000 7ff7037fe640 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562c3a6e5010 0x562c3a6e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.011+0000 7ff7037fe640 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562c3a6e5010 0x562c3a6e7430 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a7d8d5c07a2a612 server_cookie=fb982167131572a1 in_seq=0 out_seq=0 2024-04-21T08:53:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.011+0000 7ff6e37fe640 10 client.26458 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.036+0000 7ff6e37fe640 1 -- 172.21.15.82:0/87931493 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c3a5693d0 con 0x562c3a6e5010 2024-04-21T08:53:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.036+0000 7ff6e37fe640 10 client.26458 handle_command_reply: tid=0 2024-04-21T08:53:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.036+0000 7ff70d382740 1 client.26458 shutdown 2024-04-21T08:53:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.036+0000 7ff70d382740 1 -- 172.21.15.82:0/87931493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a664b60 msgr2=0x562c3a676790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.036+0000 7ff70d382740 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a664b60 0x562c3a676790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8000c00 tx=0x7ff6f800bf60 comp rx=0 tx=0).stop 2024-04-21T08:53:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.037+0000 7ff70d382740 1 -- 172.21.15.82:0/87931493 shutdown_connections 2024-04-21T08:53:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.037+0000 7ff70d382740 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562c3a6e5010 0x562c3a6e7430 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.037+0000 7ff70d382740 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c3a66e0e0 0x562c3a6721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.037+0000 7ff70d382740 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c3a6658d0 0x562c3a671b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.037+0000 7ff70d382740 1 --2- 172.21.15.82:0/87931493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c3a664b60 0x562c3a676790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.037+0000 7ff70d382740 1 -- 172.21.15.82:0/87931493 >> 172.21.15.82:0/87931493 conn(0x562c3a62bc90 msgr2=0x562c3a665e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.037+0000 7ff6e37fe640 0 client.26458 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.037+0000 7ff70d382740 1 -- 172.21.15.82:0/87931493 shutdown_connections 2024-04-21T08:53:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.037+0000 7ff70d382740 1 -- 172.21.15.82:0/87931493 wait complete. 2024-04-21T08:53:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.037+0000 7ff70d382740 20 client.26458 trim_cache size 0 max 16384 2024-04-21T08:53:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:53:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:53:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:53:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:53:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.038+0000 7ff70a6c0640 1 -- 172.21.15.82:0/2538252374 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6ec0a3f20 msgr2=0x7ff6ec0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.038+0000 7ff70a6c0640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6ec0a3f20 0x7ff6ec0a63e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc033680 tx=0x7ff6fc009500 comp rx=0 tx=0).stop 2024-04-21T08:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.038+0000 7ff70a6c0640 1 -- 172.21.15.82:0/2538252374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7041005c0 msgr2=0x7ff70410d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.038+0000 7ff70a6c0640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7041005c0 0x7ff70410d250 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800d950 tx=0x7ff6f800de10 comp rx=0 tx=0).stop 2024-04-21T08:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.038+0000 7ff70a6c0640 1 -- 172.21.15.82:0/2538252374 shutdown_connections 2024-04-21T08:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.038+0000 7ff70a6c0640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff704106680 0x7ff70410a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.039+0000 7ff70a6c0640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff704100f80 0x7ff70410a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.039+0000 7ff70a6c0640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff6ec0a3f20 0x7ff6ec0a63e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.039+0000 7ff70a6c0640 1 --2- 172.21.15.82:0/2538252374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7041005c0 0x7ff70410d250 secure :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800d950 tx=0x7ff6f800de10 comp rx=0 tx=0).stop 2024-04-21T08:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.039+0000 7ff70a6c0640 1 -- 172.21.15.82:0/2538252374 >> 172.21.15.82:0/2538252374 conn(0x7ff7040773e0 msgr2=0x7ff7040fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.039+0000 7ff70a6c0640 1 -- 172.21.15.82:0/2538252374 shutdown_connections 2024-04-21T08:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:30.039+0000 7ff70a6c0640 1 -- 172.21.15.82:0/2538252374 wait complete. 2024-04-21T08:53:30.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:30 smithi082 ceph-mon[27967]: pgmap v197: 129 pgs: 129 active+clean; 868 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 556 op/s 2024-04-21T08:53:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:30 smithi106 ceph-mon[31072]: pgmap v197: 129 pgs: 129 active+clean; 868 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 556 op/s 2024-04-21T08:53:30.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:30 smithi134 ceph-mon[31772]: pgmap v197: 129 pgs: 129 active+clean; 868 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 556 op/s 2024-04-21T08:53:31.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:31 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:53:31.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:31 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:53:31.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:31 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:53:32.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:32 smithi082 ceph-mon[27967]: pgmap v198: 129 pgs: 129 active+clean; 868 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 517 op/s 2024-04-21T08:53:32.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:32 smithi106 ceph-mon[31072]: pgmap v198: 129 pgs: 129 active+clean; 868 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 517 op/s 2024-04-21T08:53:32.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:32 smithi134 ceph-mon[31772]: pgmap v198: 129 pgs: 129 active+clean; 868 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 517 op/s 2024-04-21T08:53:34.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:34 smithi082 ceph-mon[27967]: pgmap v199: 129 pgs: 129 active+clean; 879 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 922 op/s 2024-04-21T08:53:34.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:34 smithi106 ceph-mon[31072]: pgmap v199: 129 pgs: 129 active+clean; 879 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 922 op/s 2024-04-21T08:53:34.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:34 smithi134 ceph-mon[31772]: pgmap v199: 129 pgs: 129 active+clean; 879 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 922 op/s 2024-04-21T08:53:36.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:36 smithi082 ceph-mon[27967]: pgmap v200: 129 pgs: 129 active+clean; 879 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 736 op/s 2024-04-21T08:53:36.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:36 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:36.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:36 smithi106 ceph-mon[31072]: pgmap v200: 129 pgs: 129 active+clean; 879 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 736 op/s 2024-04-21T08:53:36.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:36 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:36.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:36 smithi134 ceph-mon[31772]: pgmap v200: 129 pgs: 129 active+clean; 879 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 736 op/s 2024-04-21T08:53:36.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:36 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:38.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:38 smithi082 ceph-mon[27967]: pgmap v201: 129 pgs: 129 active+clean; 889 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 977 op/s 2024-04-21T08:53:38.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:38 smithi106 ceph-mon[31072]: pgmap v201: 129 pgs: 129 active+clean; 889 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 977 op/s 2024-04-21T08:53:38.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:38 smithi134 ceph-mon[31772]: pgmap v201: 129 pgs: 129 active+clean; 889 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 977 op/s 2024-04-21T08:53:40.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:40 smithi082 ceph-mon[27967]: pgmap v202: 129 pgs: 129 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 1.04k op/s 2024-04-21T08:53:40.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:40 smithi106 ceph-mon[31072]: pgmap v202: 129 pgs: 129 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 1.04k op/s 2024-04-21T08:53:40.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:40 smithi134 ceph-mon[31772]: pgmap v202: 129 pgs: 129 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 1.04k op/s 2024-04-21T08:53:42.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:42 smithi082 ceph-mon[27967]: pgmap v203: 129 pgs: 129 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 875 op/s 2024-04-21T08:53:42.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:42 smithi106 ceph-mon[31072]: pgmap v203: 129 pgs: 129 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 875 op/s 2024-04-21T08:53:42.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:42 smithi134 ceph-mon[31772]: pgmap v203: 129 pgs: 129 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 875 op/s 2024-04-21T08:53:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.177+0000 7f62494eb640 1 -- 172.21.15.82:0/1204396445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6244108f50 msgr2=0x7f6244109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.177+0000 7f62494eb640 1 --2- 172.21.15.82:0/1204396445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6244108f50 0x7f6244109330 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6238009900 tx=0x7f623802bcc0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.177+0000 7f62494eb640 1 -- 172.21.15.82:0/1204396445 shutdown_connections 2024-04-21T08:53:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.177+0000 7f62494eb640 1 --2- 172.21.15.82:0/1204396445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62441150a0 0x7f6244117490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.177+0000 7f62494eb640 1 --2- 172.21.15.82:0/1204396445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6244109870 0x7f6244114b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.177+0000 7f62494eb640 1 --2- 172.21.15.82:0/1204396445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6244108f50 0x7f6244109330 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.177+0000 7f62494eb640 1 -- 172.21.15.82:0/1204396445 >> 172.21.15.82:0/1204396445 conn(0x7f624406d020 msgr2=0x7f624406d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.179+0000 7f62494eb640 1 -- 172.21.15.82:0/1204396445 shutdown_connections 2024-04-21T08:53:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.179+0000 7f62494eb640 1 -- 172.21.15.82:0/1204396445 wait complete. 2024-04-21T08:53:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.180+0000 7f62494eb640 1 Processor -- start 2024-04-21T08:53:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.180+0000 7f62494eb640 1 -- start start 2024-04-21T08:53:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f62494eb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6244108f50 0x7f62441a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f62494eb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6244109870 0x7f624419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f62494eb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62441150a0 0x7f624419f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f62494eb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f62441188d0 con 0x7f6244109870 2024-04-21T08:53:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f62494eb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6244118750 con 0x7f6244108f50 2024-04-21T08:53:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f62494eb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6244118a50 con 0x7f62441150a0 2024-04-21T08:53:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f62427fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6244109870 0x7f624419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f6242ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6244108f50 0x7f62441a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f6242ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6244108f50 0x7f62441a5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:39072/0 (socket says 172.21.15.82:39072) 2024-04-21T08:53:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f62427fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6244109870 0x7f624419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53522/0 (socket says 172.21.15.82:53522) 2024-04-21T08:53:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f6242ffd640 1 -- 172.21.15.82:0/3356772651 learned_addr learned my addr 172.21.15.82:0/3356772651 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f62437fe640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62441150a0 0x7f624419f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f6242ffd640 1 -- 172.21.15.82:0/3356772651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62441150a0 msgr2=0x7f624419f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f6242ffd640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62441150a0 0x7f624419f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f6242ffd640 1 -- 172.21.15.82:0/3356772651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6244109870 msgr2=0x7f624419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.181+0000 7f6242ffd640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6244109870 0x7f624419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.182+0000 7f6242ffd640 1 -- 172.21.15.82:0/3356772651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624419f7a0 con 0x7f6244108f50 2024-04-21T08:53:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.182+0000 7f6242ffd640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6244108f50 0x7f62441a5ba0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f623802c2d0 tx=0x7f623802da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.182+0000 7f622bfff640 1 -- 172.21.15.82:0/3356772651 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238043020 con 0x7f6244108f50 2024-04-21T08:53:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.182+0000 7f62494eb640 1 -- 172.21.15.82:0/3356772651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624419fa30 con 0x7f6244108f50 2024-04-21T08:53:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.182+0000 7f62494eb640 1 -- 172.21.15.82:0/3356772651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62441a3a30 con 0x7f6244108f50 2024-04-21T08:53:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.182+0000 7f622bfff640 1 -- 172.21.15.82:0/3356772651 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f623802ddd0 con 0x7f6244108f50 2024-04-21T08:53:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.182+0000 7f622bfff640 1 -- 172.21.15.82:0/3356772651 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238003bb0 con 0x7f6244108f50 2024-04-21T08:53:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.186+0000 7f624c1ba740 1 Processor -- start 2024-04-21T08:53:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.186+0000 7f624c1ba740 1 -- start start 2024-04-21T08:53:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.186+0000 7f624c1ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36898d0 0x5557e3688620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.186+0000 7f624c1ba740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e3688b60 0x5557e3688fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.186+0000 7f624c1ba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e36944d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.187+0000 7f624c1ba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5557e35ed8d0 con 0x5557e36920e0 2024-04-21T08:53:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.187+0000 7f624c1ba740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5557e358ae80 con 0x5557e3688b60 2024-04-21T08:53:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.187+0000 7f624c1ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5557e35e9890 con 0x5557e36898d0 2024-04-21T08:53:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.187+0000 7f62437fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36898d0 0x5557e3688620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.187+0000 7f62437fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36898d0 0x5557e3688620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:43656/0 (socket says 172.21.15.82:43656) 2024-04-21T08:53:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.187+0000 7f62437fe640 1 -- 172.21.15.82:0/841232370 learned_addr learned my addr 172.21.15.82:0/841232370 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.187+0000 7f62427fc640 1 --2- 172.21.15.82:0/841232370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e36944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.188+0000 7f62427fc640 1 -- 172.21.15.82:0/841232370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36898d0 msgr2=0x5557e3688620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.188+0000 7f62427fc640 1 --2- 172.21.15.82:0/841232370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36898d0 0x5557e3688620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.188+0000 7f62427fc640 1 -- 172.21.15.82:0/841232370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e3688b60 msgr2=0x5557e3688fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:53:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.188+0000 7f62427fc640 1 --2- 172.21.15.82:0/841232370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e3688b60 0x5557e3688fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.188+0000 7f62427fc640 1 -- 172.21.15.82:0/841232370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557e35ac320 con 0x5557e36920e0 2024-04-21T08:53:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.188+0000 7f622bfff640 1 -- 172.21.15.82:0/3356772651 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6238046020 con 0x7f6244108f50 2024-04-21T08:53:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.189+0000 7f62437fe640 1 --2- 172.21.15.82:0/841232370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36898d0 0x5557e3688620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.189+0000 7f622bfff640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f622c0a3f30 0x7f622c0a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.190+0000 7f62427fc640 1 --2- 172.21.15.82:0/841232370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e36944d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f62441a0470 tx=0x7f623400bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f9ac605d62f6a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.190+0000 7f622bfff640 1 -- 172.21.15.82:0/3356772651 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f62380f2dd0 con 0x7f6244108f50 2024-04-21T08:53:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.190+0000 7f62437fe640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f622c0a3f30 0x7f622c0a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.190+0000 7f62297fa640 1 -- 172.21.15.82:0/841232370 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623400be10 con 0x5557e36920e0 2024-04-21T08:53:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.191+0000 7f62297fa640 1 -- 172.21.15.82:0/841232370 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6234019070 con 0x5557e36920e0 2024-04-21T08:53:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.191+0000 7f62297fa640 1 -- 172.21.15.82:0/841232370 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234014da0 con 0x5557e36920e0 2024-04-21T08:53:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.192+0000 7f62437fe640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f622c0a3f30 0x7f622c0a63f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f623c00f470 tx=0x7f623c007680 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.193+0000 7f624c1ba740 1 -- 172.21.15.82:0/841232370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 msgr2=0x5557e36944d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.193+0000 7f624c1ba740 1 --2- 172.21.15.82:0/841232370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e36944d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f62441a0470 tx=0x7f623400bbc0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.194+0000 7f624c1ba740 1 -- 172.21.15.82:0/841232370 shutdown_connections 2024-04-21T08:53:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.194+0000 7f624c1ba740 1 --2- 172.21.15.82:0/841232370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e36944d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.194+0000 7f624c1ba740 1 --2- 172.21.15.82:0/841232370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e3688b60 0x5557e3688fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.194+0000 7f624c1ba740 1 --2- 172.21.15.82:0/841232370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36898d0 0x5557e3688620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.194+0000 7f624c1ba740 1 -- 172.21.15.82:0/841232370 >> 172.21.15.82:0/841232370 conn(0x5557e364fc90 msgr2=0x5557e3683850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.195+0000 7f624c1ba740 1 -- 172.21.15.82:0/841232370 shutdown_connections 2024-04-21T08:53:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.195+0000 7f624c1ba740 1 -- 172.21.15.82:0/841232370 wait complete. 2024-04-21T08:53:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.196+0000 7f624c1ba740 1 -- start start 2024-04-21T08:53:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.196+0000 7f624c1ba740 1 Processor -- start 2024-04-21T08:53:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.196+0000 7f624c1ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e3688b60 0x5557e3691b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.196+0000 7f624c1ba740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e36898d0 0x5557e368eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.196+0000 7f624c1ba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e368f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.196+0000 7f624c1ba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5557e3694e10 con 0x5557e36920e0 2024-04-21T08:53:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.196+0000 7f624c1ba740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5557e3575eb0 con 0x5557e36898d0 2024-04-21T08:53:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.196+0000 7f624c1ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5557e3694f90 con 0x5557e3688b60 2024-04-21T08:53:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.196+0000 7f624c1ba740 10 client.0 _ll_register_callbacks cb 0x5557e35a2b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.197+0000 7f62437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e368f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.197+0000 7f62437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e368f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53546/0 (socket says 172.21.15.82:53546) 2024-04-21T08:53:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.197+0000 7f62437fe640 1 -- 172.21.15.82:0/2521776100 learned_addr learned my addr 172.21.15.82:0/2521776100 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.198+0000 7f62437fe640 1 -- 172.21.15.82:0/2521776100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e3688b60 msgr2=0x5557e3691b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:53:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.198+0000 7f62437fe640 1 --2- 172.21.15.82:0/2521776100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e3688b60 0x5557e3691b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.198+0000 7f62437fe640 1 -- 172.21.15.82:0/2521776100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e36898d0 msgr2=0x5557e368eec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:53:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.198+0000 7f62437fe640 1 --2- 172.21.15.82:0/2521776100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e36898d0 0x5557e368eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.198+0000 7f62437fe640 1 -- 172.21.15.82:0/2521776100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557e3300210 con 0x5557e36920e0 2024-04-21T08:53:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.199+0000 7f62437fe640 1 --2- 172.21.15.82:0/2521776100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e368f600 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f623c007fd0 tx=0x7f623c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649b3f6a06554215 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.199+0000 7f6226ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.199+0000 7f6226ff5640 1 -- 172.21.15.82:0/2521776100 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c023020 con 0x5557e36920e0 2024-04-21T08:53:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.199+0000 7f6226ff5640 1 -- 172.21.15.82:0/2521776100 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f623c01f440 con 0x5557e36920e0 2024-04-21T08:53:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.199+0000 7f624c1ba740 1 -- 172.21.15.82:0/2521776100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6234019070 con 0x5557e36920e0 2024-04-21T08:53:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.199+0000 7f6226ff5640 1 -- 172.21.15.82:0/2521776100 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f623c02e050 con 0x5557e36920e0 2024-04-21T08:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.199+0000 7f624c1ba740 10 client.17439 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.199+0000 7f624c1ba740 1 -- 172.21.15.82:0/2521776100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5557e33cc620 con 0x5557e36920e0 2024-04-21T08:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.200+0000 7f6226ff5640 1 -- 172.21.15.82:0/2521776100 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6234019070 con 0x5557e36920e0 2024-04-21T08:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.200+0000 7f6226ff5640 1 -- 172.21.15.82:0/2521776100 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f623c011040 con 0x5557e36920e0 2024-04-21T08:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.200+0000 7f624c1ba740 10 client.17439 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.200+0000 7f624c1ba740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.200+0000 7f624c1ba740 20 client.17439 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.200+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2521776100 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5557e36a2370 0x5557e36a4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.200+0000 7f624c1ba740 4 client.17439 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.200+0000 7f624c1ba740 1 -- 172.21.15.82:0/2521776100 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557e3601880 con 0x5557e36a2370 2024-04-21T08:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.201+0000 7f6242ffd640 1 --2- 172.21.15.82:0/2521776100 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5557e36a2370 0x5557e36a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.202+0000 7f6242ffd640 1 --2- 172.21.15.82:0/2521776100 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5557e36a2370 0x5557e36a4800 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74df0faced81cf22 server_cookie=e30d6f0581b5a320 in_seq=0 out_seq=0 2024-04-21T08:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.202+0000 7f6226ff5640 10 client.17439 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.203+0000 7f6226ff5640 1 -- 172.21.15.82:0/2521776100 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557e3601880 con 0x5557e36a2370 2024-04-21T08:53:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.203+0000 7f6226ff5640 10 client.17439 handle_command_reply: tid=0 2024-04-21T08:53:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f624c1ba740 1 client.17439 shutdown 2024-04-21T08:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f624c1ba740 1 -- 172.21.15.82:0/2521776100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 msgr2=0x5557e368f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2521776100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e368f600 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f623c007fd0 tx=0x7f623c004a40 comp rx=0 tx=0).stop 2024-04-21T08:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f624c1ba740 1 -- 172.21.15.82:0/2521776100 shutdown_connections 2024-04-21T08:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2521776100 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5557e36a2370 0x5557e36a4800 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2521776100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e368f600 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2521776100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e36898d0 0x5557e368eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2521776100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e3688b60 0x5557e3691b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f624c1ba740 1 -- 172.21.15.82:0/2521776100 >> 172.21.15.82:0/2521776100 conn(0x5557e364fc90 msgr2=0x5557e36855a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f6226ff5640 0 client.17439 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f624c1ba740 1 -- 172.21.15.82:0/2521776100 shutdown_connections 2024-04-21T08:53:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f624c1ba740 1 -- 172.21.15.82:0/2521776100 wait complete. 2024-04-21T08:53:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.204+0000 7f624c1ba740 20 client.17439 trim_cache size 0 max 16384 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.230+0000 7f624c1ba740 1 Processor -- start 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.230+0000 7f624c1ba740 1 -- start start 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.231+0000 7f624c1ba740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e3688b60 0x5557e369dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.231+0000 7f624c1ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36898d0 0x5557e36aa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.231+0000 7f624c1ba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e36aa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.231+0000 7f624c1ba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f62300100b0 con 0x5557e36920e0 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.231+0000 7f624c1ba740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6230011ce0 con 0x5557e3688b60 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.231+0000 7f624c1ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6230010780 con 0x5557e36898d0 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.231+0000 7f62437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e36aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.231+0000 7f62437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e36aa650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53552/0 (socket says 172.21.15.82:53552) 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.231+0000 7f62437fe640 1 -- 172.21.15.82:0/2743235062 learned_addr learned my addr 172.21.15.82:0/2743235062 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.231+0000 7f62427fc640 1 --2- 172.21.15.82:0/2743235062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36898d0 0x5557e36aa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.232+0000 7f62437fe640 1 -- 172.21.15.82:0/2743235062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36898d0 msgr2=0x5557e36aa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.232+0000 7f62437fe640 1 --2- 172.21.15.82:0/2743235062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36898d0 0x5557e36aa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.232+0000 7f62437fe640 1 -- 172.21.15.82:0/2743235062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e3688b60 msgr2=0x5557e369dda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.232+0000 7f62437fe640 1 --2- 172.21.15.82:0/2743235062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e3688b60 0x5557e369dda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.232+0000 7f62437fe640 1 -- 172.21.15.82:0/2743235062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557e367ef30 con 0x5557e36920e0 2024-04-21T08:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.232+0000 7f62437fe640 1 --2- 172.21.15.82:0/2743235062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e36aa650 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f623c014040 tx=0x7f623c0071b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9e925a96bca554 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.233+0000 7f6228ff9640 1 -- 172.21.15.82:0/2743235062 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c023050 con 0x5557e36920e0 2024-04-21T08:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.233+0000 7f6228ff9640 1 -- 172.21.15.82:0/2743235062 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f623c018030 con 0x5557e36920e0 2024-04-21T08:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.233+0000 7f6228ff9640 1 -- 172.21.15.82:0/2743235062 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c01f6a0 con 0x5557e36920e0 2024-04-21T08:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.234+0000 7f624c1ba740 1 -- 172.21.15.82:0/2743235062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 msgr2=0x5557e36aa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.234+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2743235062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e36aa650 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f623c014040 tx=0x7f623c0071b0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.234+0000 7f624c1ba740 1 -- 172.21.15.82:0/2743235062 shutdown_connections 2024-04-21T08:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.234+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2743235062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e36920e0 0x5557e36aa650 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.234+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2743235062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36898d0 0x5557e36aa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.234+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2743235062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e3688b60 0x5557e369dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.234+0000 7f624c1ba740 1 -- 172.21.15.82:0/2743235062 >> 172.21.15.82:0/2743235062 conn(0x5557e364fc90 msgr2=0x5557e36a4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.234+0000 7f624c1ba740 1 -- 172.21.15.82:0/2743235062 shutdown_connections 2024-04-21T08:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.235+0000 7f624c1ba740 1 -- 172.21.15.82:0/2743235062 wait complete. 2024-04-21T08:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.235+0000 7f624c1ba740 1 -- start start 2024-04-21T08:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.235+0000 7f624c1ba740 1 Processor -- start 2024-04-21T08:53:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.236+0000 7f624c1ba740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e3688b60 0x5557e369a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.236+0000 7f624c1ba740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e36898d0 0x5557e369a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.236+0000 7f624c1ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36920e0 0x5557e36956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.236+0000 7f624c1ba740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f62300115e0 con 0x5557e3688b60 2024-04-21T08:53:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.236+0000 7f624c1ba740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6230010ee0 con 0x5557e36898d0 2024-04-21T08:53:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.236+0000 7f624c1ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f623000e060 con 0x5557e36920e0 2024-04-21T08:53:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.236+0000 7f624c1ba740 10 client.0 _ll_register_callbacks cb 0x7f623c011aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.237+0000 7f62427fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e36898d0 0x5557e369a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.237+0000 7f62427fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e36898d0 0x5557e369a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:39112/0 (socket says 172.21.15.82:39112) 2024-04-21T08:53:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.237+0000 7f62427fc640 1 -- 172.21.15.82:0/2351922865 learned_addr learned my addr 172.21.15.82:0/2351922865 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.237+0000 7f62427fc640 1 -- 172.21.15.82:0/2351922865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36920e0 msgr2=0x5557e36956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.237+0000 7f62427fc640 1 --2- 172.21.15.82:0/2351922865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36920e0 0x5557e36956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.237+0000 7f62427fc640 1 -- 172.21.15.82:0/2351922865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e3688b60 msgr2=0x5557e369a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.237+0000 7f62427fc640 1 --2- 172.21.15.82:0/2351922865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e3688b60 0x5557e369a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.237+0000 7f62427fc640 1 -- 172.21.15.82:0/2351922865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557e3695ef0 con 0x5557e36898d0 2024-04-21T08:53:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.238+0000 7f62427fc640 1 --2- 172.21.15.82:0/2351922865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e36898d0 0x5557e369a720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6234004810 tx=0x7f623400d950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fc926b69ef53828 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.238+0000 7f6226ff5640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:53:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.239+0000 7f6226ff5640 1 -- 172.21.15.82:0/2351922865 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234014430 con 0x5557e36898d0 2024-04-21T08:53:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.239+0000 7f624c1ba740 1 -- 172.21.15.82:0/2351922865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f623c018030 con 0x5557e36898d0 2024-04-21T08:53:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.239+0000 7f6226ff5640 1 -- 172.21.15.82:0/2351922865 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f62340145d0 con 0x5557e36898d0 2024-04-21T08:53:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.239+0000 7f6226ff5640 1 -- 172.21.15.82:0/2351922865 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f62340147f0 con 0x5557e36898d0 2024-04-21T08:53:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.239+0000 7f624c1ba740 10 client.26789 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.239+0000 7f624c1ba740 1 -- 172.21.15.82:0/2351922865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5557e3708590 con 0x5557e36898d0 2024-04-21T08:53:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.240+0000 7f6226ff5640 1 -- 172.21.15.82:0/2351922865 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f623c018030 con 0x5557e36898d0 2024-04-21T08:53:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.240+0000 7f6226ff5640 1 -- 172.21.15.82:0/2351922865 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f6234013070 con 0x5557e36898d0 2024-04-21T08:53:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.240+0000 7f624c1ba740 10 client.26789 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.240+0000 7f624c1ba740 10 client.26789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.240+0000 7f624c1ba740 20 client.26789 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.240+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2351922865 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5557e3708b00 0x5557e370af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.240+0000 7f624c1ba740 4 client.26789 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:53:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.240+0000 7f624c1ba740 1 -- 172.21.15.82:0/2351922865 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557e358d3d0 con 0x5557e3708b00 2024-04-21T08:53:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.241+0000 7f6242ffd640 1 --2- 172.21.15.82:0/2351922865 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5557e3708b00 0x5557e370af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.245+0000 7f6226ff5640 10 client.26789 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.245+0000 7f6242ffd640 1 --2- 172.21.15.82:0/2351922865 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5557e3708b00 0x5557e370af40 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea57d0df2d483ff9 server_cookie=e96b6764eef853c3 in_seq=0 out_seq=0 2024-04-21T08:53:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.542+0000 7f6226ff5640 1 -- 172.21.15.82:0/2351922865 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557e358d3d0 con 0x5557e3708b00 2024-04-21T08:53:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.542+0000 7f6226ff5640 10 client.26789 handle_command_reply: tid=0 2024-04-21T08:53:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.542+0000 7f624c1ba740 1 client.26789 shutdown 2024-04-21T08:53:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f624c1ba740 1 -- 172.21.15.82:0/2351922865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e36898d0 msgr2=0x5557e369a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2351922865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e36898d0 0x5557e369a720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6234004810 tx=0x7f623400d950 comp rx=0 tx=0).stop 2024-04-21T08:53:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f624c1ba740 1 -- 172.21.15.82:0/2351922865 shutdown_connections 2024-04-21T08:53:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2351922865 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5557e3708b00 0x5557e370af40 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2351922865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557e36920e0 0x5557e36956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2351922865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557e36898d0 0x5557e369a720 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f624c1ba740 1 --2- 172.21.15.82:0/2351922865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5557e3688b60 0x5557e369a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f6226ff5640 0 client.26789 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f624c1ba740 1 -- 172.21.15.82:0/2351922865 >> 172.21.15.82:0/2351922865 conn(0x5557e364fc90 msgr2=0x5557e3689ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f624c1ba740 1 -- 172.21.15.82:0/2351922865 shutdown_connections 2024-04-21T08:53:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f624c1ba740 1 -- 172.21.15.82:0/2351922865 wait complete. 2024-04-21T08:53:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f624c1ba740 20 client.26789 trim_cache size 0 max 16384 2024-04-21T08:53:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:53:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:53:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:53:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:53:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f62494eb640 1 -- 172.21.15.82:0/3356772651 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f622c0a3f30 msgr2=0x7f622c0a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f62494eb640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f622c0a3f30 0x7f622c0a63f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f623c00f470 tx=0x7f623c007680 comp rx=0 tx=0).stop 2024-04-21T08:53:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f62494eb640 1 -- 172.21.15.82:0/3356772651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6244108f50 msgr2=0x7f62441a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f62494eb640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6244108f50 0x7f62441a5ba0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f623802c2d0 tx=0x7f623802da50 comp rx=0 tx=0).stop 2024-04-21T08:53:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f62494eb640 1 -- 172.21.15.82:0/3356772651 shutdown_connections 2024-04-21T08:53:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f62494eb640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62441150a0 0x7f624419f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f62494eb640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f622c0a3f30 0x7f622c0a63f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f62494eb640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6244109870 0x7f624419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f62494eb640 1 --2- 172.21.15.82:0/3356772651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6244108f50 0x7f62441a5ba0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.543+0000 7f62494eb640 1 -- 172.21.15.82:0/3356772651 >> 172.21.15.82:0/3356772651 conn(0x7f624406d020 msgr2=0x7f6244070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.544+0000 7f62494eb640 1 -- 172.21.15.82:0/3356772651 shutdown_connections 2024-04-21T08:53:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:43.544+0000 7f62494eb640 1 -- 172.21.15.82:0/3356772651 wait complete. 2024-04-21T08:53:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:44 smithi082 ceph-mon[27967]: pgmap v204: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 1.13k op/s 2024-04-21T08:53:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:44 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:53:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:44 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:53:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:44 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:53:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:44 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:53:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:44 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:53:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:44 smithi106 ceph-mon[31072]: pgmap v204: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 1.13k op/s 2024-04-21T08:53:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:44 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:53:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:44 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:53:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:44 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:53:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:44 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:53:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:44 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:53:44.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:44 smithi134 ceph-mon[31772]: pgmap v204: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 1.13k op/s 2024-04-21T08:53:44.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:44 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:53:44.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:44 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:53:44.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:44 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:53:44.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:44 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:53:44.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:44 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:53:46.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:46 smithi082 ceph-mon[27967]: pgmap v205: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 725 op/s 2024-04-21T08:53:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:46 smithi106 ceph-mon[31072]: pgmap v205: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 725 op/s 2024-04-21T08:53:46.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:46 smithi134 ceph-mon[31772]: pgmap v205: 129 pgs: 129 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 725 op/s 2024-04-21T08:53:48.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:48 smithi082 ceph-mon[27967]: pgmap v206: 129 pgs: 129 active+clean; 880 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 873 op/s 2024-04-21T08:53:48.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:48 smithi106 ceph-mon[31072]: pgmap v206: 129 pgs: 129 active+clean; 880 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 873 op/s 2024-04-21T08:53:48.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:48 smithi134 ceph-mon[31772]: pgmap v206: 129 pgs: 129 active+clean; 880 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 873 op/s 2024-04-21T08:53:50.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:50 smithi082 ceph-mon[27967]: pgmap v207: 129 pgs: 129 active+clean; 880 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 779 op/s 2024-04-21T08:53:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:50 smithi106 ceph-mon[31072]: pgmap v207: 129 pgs: 129 active+clean; 880 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 779 op/s 2024-04-21T08:53:50.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:50 smithi134 ceph-mon[31772]: pgmap v207: 129 pgs: 129 active+clean; 880 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 779 op/s 2024-04-21T08:53:52.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:52 smithi082 ceph-mon[27967]: pgmap v208: 129 pgs: 129 active+clean; 880 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 549 op/s 2024-04-21T08:53:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:52 smithi106 ceph-mon[31072]: pgmap v208: 129 pgs: 129 active+clean; 880 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 549 op/s 2024-04-21T08:53:52.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:52 smithi134 ceph-mon[31772]: pgmap v208: 129 pgs: 129 active+clean; 880 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 549 op/s 2024-04-21T08:53:52.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:53:52.990 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:53:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.093+0000 7f31d803a640 1 -- 172.21.15.82:0/2096457930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d0101a40 msgr2=0x7f31d0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.093+0000 7f31d803a640 1 --2- 172.21.15.82:0/2096457930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d0101a40 0x7f31d0105a80 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009900 tx=0x7f31cc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.093+0000 7f31d803a640 1 -- 172.21.15.82:0/2096457930 shutdown_connections 2024-04-21T08:53:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.093+0000 7f31d803a640 1 --2- 172.21.15.82:0/2096457930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31d0106160 0x7f31d010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.093+0000 7f31d803a640 1 --2- 172.21.15.82:0/2096457930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d0101a40 0x7f31d0105a80 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.093+0000 7f31d803a640 1 --2- 172.21.15.82:0/2096457930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0101090 0x7f31d0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.093+0000 7f31d803a640 1 -- 172.21.15.82:0/2096457930 >> 172.21.15.82:0/2096457930 conn(0x7f31d00771d0 msgr2=0x7f31d00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.093+0000 7f31d803a640 1 -- 172.21.15.82:0/2096457930 shutdown_connections 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.093+0000 7f31d803a640 1 -- 172.21.15.82:0/2096457930 wait complete. 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.093+0000 7f31d803a640 1 Processor -- start 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.093+0000 7f31d803a640 1 -- start start 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d803a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0101090 0x7f31d019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d803a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d0106160 0x7f31d019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d803a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31d01985a0 0x7f31d0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d803a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f31d0112200 con 0x7f31d0106160 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d803a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f31d0112080 con 0x7f31d01985a0 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d803a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f31d0112380 con 0x7f31d0101090 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d55d5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d0106160 0x7f31d019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d55d5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d0106160 0x7f31d019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44340/0 (socket says 172.21.15.82:44340) 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d55d5640 1 -- 172.21.15.82:0/2383554795 learned_addr learned my addr 172.21.15.82:0/2383554795 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d65d7640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31d01985a0 0x7f31d0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0101090 0x7f31d019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d55d5640 1 -- 172.21.15.82:0/2383554795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0101090 msgr2=0x7f31d019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d55d5640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0101090 0x7f31d019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d55d5640 1 -- 172.21.15.82:0/2383554795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31d01985a0 msgr2=0x7f31d0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d55d5640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31d01985a0 0x7f31d0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d55d5640 1 -- 172.21.15.82:0/2383554795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d01990c0 con 0x7f31d0106160 2024-04-21T08:53:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d65d7640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31d01985a0 0x7f31d0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d55d5640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d0106160 0x7f31d019e930 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f31cc036040 tx=0x7f31cc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31c6ffd640 1 -- 172.21.15.82:0/2383554795 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc043020 con 0x7f31d0106160 2024-04-21T08:53:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31c6ffd640 1 -- 172.21.15.82:0/2383554795 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31cc02cb70 con 0x7f31d0106160 2024-04-21T08:53:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31c6ffd640 1 -- 172.21.15.82:0/2383554795 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc003b80 con 0x7f31d0106160 2024-04-21T08:53:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d803a640 1 -- 172.21.15.82:0/2383554795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d0199350 con 0x7f31d0106160 2024-04-21T08:53:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.094+0000 7f31d803a640 1 -- 172.21.15.82:0/2383554795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d01a55a0 con 0x7f31d0106160 2024-04-21T08:53:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.096+0000 7f31c6ffd640 1 -- 172.21.15.82:0/2383554795 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f31cc046020 con 0x7f31d0106160 2024-04-21T08:53:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31dac1f740 1 Processor -- start 2024-04-21T08:53:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31dac1f740 1 -- start start 2024-04-21T08:53:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31c6ffd640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f31bc0a3d20 0x7f31bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31dac1f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x55939014b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31dac1f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014bea0 0x559390154540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31dac1f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 0x559390156e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31dac1f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5593900b01e0 con 0x55939014b580 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31dac1f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55939004d390 con 0x559390154a80 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31dac1f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5593900ac1a0 con 0x55939014bea0 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31d65d7640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f31bc0a3d20 0x7f31bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31c6ffd640 1 -- 172.21.15.82:0/2383554795 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f31cc0f2bb0 con 0x7f31d0106160 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31d5dd6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x55939014b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31d5dd6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x55939014b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44346/0 (socket says 172.21.15.82:44346) 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31d5dd6640 1 -- 172.21.15.82:0/2683807487 learned_addr learned my addr 172.21.15.82:0/2683807487 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.097+0000 7f31d65d7640 1 --2- 172.21.15.82:0/2683807487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 0x559390156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.098+0000 7f31d55d5640 1 --2- 172.21.15.82:0/2683807487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014bea0 0x559390154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.098+0000 7f31d5dd6640 1 -- 172.21.15.82:0/2683807487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014bea0 msgr2=0x559390154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.098+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/2683807487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014bea0 0x559390154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.098+0000 7f31d5dd6640 1 -- 172.21.15.82:0/2683807487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 msgr2=0x559390156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.098+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/2683807487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 0x559390156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.098+0000 7f31d5dd6640 1 -- 172.21.15.82:0/2683807487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55938fe8c7d0 con 0x55939014b580 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.098+0000 7f31d65d7640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f31bc0a3d20 0x7f31bc0a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31c80099e0 tx=0x7f31c8013660 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.098+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/2683807487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x55939014b960 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009a70 tx=0x7f31c0009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6627ab7b54393395 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.098+0000 7f31bbfff640 1 -- 172.21.15.82:0/2683807487 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c0017070 con 0x55939014b580 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.098+0000 7f31bbfff640 1 -- 172.21.15.82:0/2683807487 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c0013470 con 0x55939014b580 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.098+0000 7f31bbfff640 1 -- 172.21.15.82:0/2683807487 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c00125d0 con 0x55939014b580 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.099+0000 7f31dac1f740 1 -- 172.21.15.82:0/2683807487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 msgr2=0x55939014b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.099+0000 7f31dac1f740 1 --2- 172.21.15.82:0/2683807487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x55939014b960 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009a70 tx=0x7f31c0009f30 comp rx=0 tx=0).stop 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.099+0000 7f31dac1f740 1 -- 172.21.15.82:0/2683807487 shutdown_connections 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.099+0000 7f31dac1f740 1 --2- 172.21.15.82:0/2683807487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 0x559390156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.099+0000 7f31dac1f740 1 --2- 172.21.15.82:0/2683807487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014bea0 0x559390154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.099+0000 7f31dac1f740 1 --2- 172.21.15.82:0/2683807487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x55939014b960 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.099+0000 7f31dac1f740 1 -- 172.21.15.82:0/2683807487 >> 172.21.15.82:0/2683807487 conn(0x559390145cb0 msgr2=0x559390146090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.099+0000 7f31dac1f740 1 -- 172.21.15.82:0/2683807487 shutdown_connections 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.099+0000 7f31dac1f740 1 -- 172.21.15.82:0/2683807487 wait complete. 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.099+0000 7f31dac1f740 1 -- start start 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.099+0000 7f31dac1f740 1 Processor -- start 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31dac1f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x559390151a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31dac1f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014bea0 0x559390151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31dac1f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 0x5593901524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31dac1f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559390157870 con 0x55939014b580 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31dac1f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5593900692f0 con 0x559390154a80 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31dac1f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5593901579f0 con 0x55939014bea0 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31dac1f740 10 client.0 _ll_register_callbacks cb 0x559390076580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31d5dd6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x559390151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31d5dd6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x559390151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44354/0 (socket says 172.21.15.82:44354) 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31d5dd6640 1 -- 172.21.15.82:0/152064311 learned_addr learned my addr 172.21.15.82:0/152064311 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31d65d7640 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 0x5593901524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31d55d5640 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014bea0 0x559390151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31d5dd6640 1 -- 172.21.15.82:0/152064311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014bea0 msgr2=0x559390151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014bea0 0x559390151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31d5dd6640 1 -- 172.21.15.82:0/152064311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 msgr2=0x5593901524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 0x5593901524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31d5dd6640 1 -- 172.21.15.82:0/152064311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55938fdc23f0 con 0x55939014b580 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x559390151a70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f31c0004700 tx=0x7f31c000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac55ff01525c38e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31b97fa640 1 -- 172.21.15.82:0/152064311 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c0017050 con 0x55939014b580 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31b97fa640 1 -- 172.21.15.82:0/152064311 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f31c001c450 con 0x55939014b580 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.100+0000 7f31dac1f740 1 -- 172.21.15.82:0/152064311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c0013470 con 0x55939014b580 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.101+0000 7f31b97fa640 1 -- 172.21.15.82:0/152064311 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f31c002f050 con 0x55939014b580 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.101+0000 7f31dac1f740 10 client.17454 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:53.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.101+0000 7f31dac1f740 1 -- 172.21.15.82:0/152064311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55938fd820f0 con 0x55939014b580 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.101+0000 7f31b97fa640 1 -- 172.21.15.82:0/152064311 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c0013470 con 0x55939014b580 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.101+0000 7f31b97fa640 1 -- 172.21.15.82:0/152064311 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f31c0026680 con 0x55939014b580 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.101+0000 7f31dac1f740 10 client.17454 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.101+0000 7f31dac1f740 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.101+0000 7f31dac1f740 20 client.17454 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.101+0000 7f31dac1f740 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5593901649c0 0x559390166db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.101+0000 7f31dac1f740 4 client.17454 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.101+0000 7f31dac1f740 1 -- 172.21.15.82:0/152064311 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593900bdc00 con 0x5593901649c0 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.101+0000 7f31d55d5640 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5593901649c0 0x559390166db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.102+0000 7f31b97fa640 10 client.17454 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.102+0000 7f31d55d5640 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5593901649c0 0x559390166db0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19092bc76e0e1980 server_cookie=530737973189b269 in_seq=0 out_seq=0 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.102+0000 7f31b97fa640 1 -- 172.21.15.82:0/152064311 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593900bdc00 con 0x5593901649c0 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.102+0000 7f31b97fa640 10 client.17454 handle_command_reply: tid=0 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.102+0000 7f31dac1f740 1 client.17454 shutdown 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.103+0000 7f31dac1f740 1 -- 172.21.15.82:0/152064311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 msgr2=0x559390151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.103+0000 7f31dac1f740 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x559390151a70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f31c0004700 tx=0x7f31c000ed70 comp rx=0 tx=0).stop 2024-04-21T08:53:53.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.103+0000 7f31dac1f740 1 -- 172.21.15.82:0/152064311 shutdown_connections 2024-04-21T08:53:53.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.103+0000 7f31dac1f740 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5593901649c0 0x559390166db0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.103+0000 7f31dac1f740 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 0x5593901524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.103+0000 7f31dac1f740 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014bea0 0x559390151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.103+0000 7f31b97fa640 0 client.17454 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:53.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.103+0000 7f31dac1f740 1 --2- 172.21.15.82:0/152064311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x559390151a70 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.103+0000 7f31dac1f740 1 -- 172.21.15.82:0/152064311 >> 172.21.15.82:0/152064311 conn(0x559390145cb0 msgr2=0x559390147a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.103+0000 7f31dac1f740 1 -- 172.21.15.82:0/152064311 shutdown_connections 2024-04-21T08:53:53.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.103+0000 7f31dac1f740 1 -- 172.21.15.82:0/152064311 wait complete. 2024-04-21T08:53:53.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.103+0000 7f31dac1f740 20 client.17454 trim_cache size 0 max 16384 2024-04-21T08:53:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.120+0000 7f31dac1f740 1 Processor -- start 2024-04-21T08:53:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.120+0000 7f31dac1f740 1 -- start start 2024-04-21T08:53:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.120+0000 7f31dac1f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x5593901541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.120+0000 7f31dac1f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55939014bea0 0x559390160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.120+0000 7f31dac1f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559390154a80 0x5593901509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.120+0000 7f31dac1f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01af470 con 0x55939014b580 2024-04-21T08:53:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.120+0000 7f31dac1f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01b10a0 con 0x55939014bea0 2024-04-21T08:53:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.120+0000 7f31dac1f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01afb40 con 0x559390154a80 2024-04-21T08:53:53.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31d5dd6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x5593901541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31d5dd6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x5593901541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44370/0 (socket says 172.21.15.82:44370) 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31d5dd6640 1 -- 172.21.15.82:0/465185229 learned_addr learned my addr 172.21.15.82:0/465185229 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31d55d5640 1 --2- 172.21.15.82:0/465185229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55939014bea0 0x559390160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31d65d7640 1 --2- 172.21.15.82:0/465185229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559390154a80 0x5593901509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31d5dd6640 1 -- 172.21.15.82:0/465185229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559390154a80 msgr2=0x5593901509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/465185229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559390154a80 0x5593901509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31d5dd6640 1 -- 172.21.15.82:0/465185229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55939014bea0 msgr2=0x559390160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/465185229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55939014bea0 0x559390160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31d5dd6640 1 -- 172.21.15.82:0/465185229 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55939014bc90 con 0x55939014b580 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31d65d7640 1 --2- 172.21.15.82:0/465185229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559390154a80 0x5593901509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/465185229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x5593901541a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f31c0004810 tx=0x7f31c000e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9836f5a81f9aa3e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31bb7fe640 1 -- 172.21.15.82:0/465185229 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c0017070 con 0x55939014b580 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31bb7fe640 1 -- 172.21.15.82:0/465185229 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c000ede0 con 0x55939014b580 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.121+0000 7f31bb7fe640 1 -- 172.21.15.82:0/465185229 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c001c720 con 0x55939014b580 2024-04-21T08:53:53.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.122+0000 7f31dac1f740 1 -- 172.21.15.82:0/465185229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 msgr2=0x5593901541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.122+0000 7f31dac1f740 1 --2- 172.21.15.82:0/465185229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x5593901541a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f31c0004810 tx=0x7f31c000e790 comp rx=0 tx=0).stop 2024-04-21T08:53:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.122+0000 7f31dac1f740 1 -- 172.21.15.82:0/465185229 shutdown_connections 2024-04-21T08:53:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.122+0000 7f31dac1f740 1 --2- 172.21.15.82:0/465185229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559390154a80 0x5593901509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.122+0000 7f31dac1f740 1 --2- 172.21.15.82:0/465185229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55939014bea0 0x559390160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.122+0000 7f31dac1f740 1 --2- 172.21.15.82:0/465185229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014b580 0x5593901541a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.122+0000 7f31dac1f740 1 -- 172.21.15.82:0/465185229 >> 172.21.15.82:0/465185229 conn(0x559390145cb0 msgr2=0x559390144d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.122+0000 7f31dac1f740 1 -- 172.21.15.82:0/465185229 shutdown_connections 2024-04-21T08:53:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.122+0000 7f31dac1f740 1 -- 172.21.15.82:0/465185229 wait complete. 2024-04-21T08:53:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.122+0000 7f31dac1f740 1 -- start start 2024-04-21T08:53:53.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.122+0000 7f31dac1f740 1 Processor -- start 2024-04-21T08:53:53.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.122+0000 7f31dac1f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014b580 0x559390158e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31dac1f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014bea0 0x559390159380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31dac1f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 0x5593901ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31dac1f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01b09a0 con 0x55939014bea0 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31dac1f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01b02a0 con 0x559390154a80 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31dac1f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f31d01ad420 con 0x55939014b580 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31dac1f740 10 client.0 _ll_register_callbacks cb 0x7f31c8013bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31d55d5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014bea0 0x559390159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31d55d5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014bea0 0x559390159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44372/0 (socket says 172.21.15.82:44372) 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31d55d5640 1 -- 172.21.15.82:0/1169030347 learned_addr learned my addr 172.21.15.82:0/1169030347 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31d65d7640 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 0x5593901ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014b580 0x559390158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31d55d5640 1 -- 172.21.15.82:0/1169030347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014b580 msgr2=0x559390158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31d55d5640 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014b580 0x559390158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31d55d5640 1 -- 172.21.15.82:0/1169030347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 msgr2=0x5593901ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31d55d5640 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 0x5593901ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31d55d5640 1 -- 172.21.15.82:0/1169030347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593901ce6b0 con 0x55939014bea0 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31d55d5640 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014bea0 0x559390159380 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0bad00 tx=0x7f31cc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c49e7675ec82f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31b97fa640 1 -- 172.21.15.82:0/1169030347 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc0c7510 con 0x55939014bea0 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31b97fa640 1 -- 172.21.15.82:0/1169030347 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f31cc003d20 con 0x55939014bea0 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31dac1f740 1 -- 172.21.15.82:0/1169030347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c000ede0 con 0x55939014bea0 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31b97fa640 1 -- 172.21.15.82:0/1169030347 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f31cc0b90d0 con 0x55939014bea0 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31dac1f740 10 client.17460 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31dac1f740 1 -- 172.21.15.82:0/1169030347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5593901cf970 con 0x55939014bea0 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.123+0000 7f31b97fa640 1 -- 172.21.15.82:0/1169030347 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c000ede0 con 0x55939014bea0 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31b97fa640 1 -- 172.21.15.82:0/1169030347 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f31cc0baf10 con 0x55939014bea0 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31dac1f740 10 client.17460 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31dac1f740 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:53.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31dac1f740 20 client.17460 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31dac1f740 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5593901d0080 0x5593901d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31dac1f740 4 client.17460 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31dac1f740 1 -- 172.21.15.82:0/1169030347 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5593900b7a70 con 0x5593901d0080 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5593901d0080 0x5593901d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31b97fa640 10 client.17460 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31d5dd6640 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5593901d0080 0x5593901d24a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c49d7326e77356cb server_cookie=97947acf4bc61582 in_seq=0 out_seq=0 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31b97fa640 1 -- 172.21.15.82:0/1169030347 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5593900b7a70 con 0x5593901d0080 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31b97fa640 10 client.17460 handle_command_reply: tid=0 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.124+0000 7f31dac1f740 1 client.17460 shutdown 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31dac1f740 1 -- 172.21.15.82:0/1169030347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014bea0 msgr2=0x559390159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31dac1f740 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014bea0 0x559390159380 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f31cc0bad00 tx=0x7f31cc009260 comp rx=0 tx=0).stop 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31dac1f740 1 -- 172.21.15.82:0/1169030347 shutdown_connections 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31dac1f740 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5593901d0080 0x5593901d24a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31dac1f740 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559390154a80 0x5593901ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31dac1f740 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55939014bea0 0x559390159380 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31dac1f740 1 --2- 172.21.15.82:0/1169030347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55939014b580 0x559390158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31b97fa640 0 client.17460 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:53.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31dac1f740 1 -- 172.21.15.82:0/1169030347 >> 172.21.15.82:0/1169030347 conn(0x559390145cb0 msgr2=0x559390151a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31dac1f740 1 -- 172.21.15.82:0/1169030347 shutdown_connections 2024-04-21T08:53:53.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31dac1f740 1 -- 172.21.15.82:0/1169030347 wait complete. 2024-04-21T08:53:53.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31dac1f740 20 client.17460 trim_cache size 0 max 16384 2024-04-21T08:53:53.131 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:53:53.131 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:53:53.131 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:53:53.131 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:53:53.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31d803a640 1 -- 172.21.15.82:0/2383554795 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f31bc0a3d20 msgr2=0x7f31bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31d803a640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f31bc0a3d20 0x7f31bc0a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31c80099e0 tx=0x7f31c8013660 comp rx=0 tx=0).stop 2024-04-21T08:53:53.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31d803a640 1 -- 172.21.15.82:0/2383554795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d0106160 msgr2=0x7f31d019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31d803a640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d0106160 0x7f31d019e930 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f31cc036040 tx=0x7f31cc0045e0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31d803a640 1 -- 172.21.15.82:0/2383554795 shutdown_connections 2024-04-21T08:53:53.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31d803a640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31d01985a0 0x7f31d0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31d803a640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f31bc0a3d20 0x7f31bc0a61e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31d803a640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31d0106160 0x7f31d019e930 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31d803a640 1 --2- 172.21.15.82:0/2383554795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31d0101090 0x7f31d019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31d803a640 1 -- 172.21.15.82:0/2383554795 >> 172.21.15.82:0/2383554795 conn(0x7f31d00771d0 msgr2=0x7f31d00fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31d803a640 1 -- 172.21.15.82:0/2383554795 shutdown_connections 2024-04-21T08:53:53.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.125+0000 7f31d803a640 1 -- 172.21.15.82:0/2383554795 wait complete. 2024-04-21T08:53:53.136 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:53:53.136 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:53:53.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.284+0000 7f71689f4640 1 -- 172.21.15.82:0/2225520041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7164078130 msgr2=0x7f716407a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.284+0000 7f71689f4640 1 --2- 172.21.15.82:0/2225520041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7164078130 0x7f716407a5c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f715c005fd0 tx=0x7f715c02cc30 comp rx=0 tx=0).stop 2024-04-21T08:53:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.284+0000 7f71689f4640 1 -- 172.21.15.82:0/2225520041 shutdown_connections 2024-04-21T08:53:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.284+0000 7f71689f4640 1 --2- 172.21.15.82:0/2225520041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7164078130 0x7f716407a5c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.284+0000 7f71689f4640 1 --2- 172.21.15.82:0/2225520041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7164109490 0x7f7164109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.284+0000 7f71689f4640 1 --2- 172.21.15.82:0/2225520041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f716410ea50 0x7f7164108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.284+0000 7f71689f4640 1 -- 172.21.15.82:0/2225520041 >> 172.21.15.82:0/2225520041 conn(0x7f716406e410 msgr2=0x7f71640719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.284+0000 7f71689f4640 1 -- 172.21.15.82:0/2225520041 shutdown_connections 2024-04-21T08:53:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.284+0000 7f71689f4640 1 -- 172.21.15.82:0/2225520041 wait complete. 2024-04-21T08:53:53.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.285+0000 7f71689f4640 1 Processor -- start 2024-04-21T08:53:53.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.285+0000 7f71689f4640 1 -- start start 2024-04-21T08:53:53.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.286+0000 7f71689f4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7164078130 0x7f7164084930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.286+0000 7f71689f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7164109490 0x7f716407e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.286+0000 7f71689f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f716410ea50 0x7f716407eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.286+0000 7f71689f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f716407d5c0 con 0x7f7164109490 2024-04-21T08:53:53.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.286+0000 7f71689f4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f716407d440 con 0x7f7164078130 2024-04-21T08:53:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.286+0000 7f71689f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f716407d740 con 0x7f716410ea50 2024-04-21T08:53:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.286+0000 7f716259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7164078130 0x7f7164084930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.286+0000 7f716259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7164078130 0x7f7164084930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:57842/0 (socket says 172.21.15.82:57842) 2024-04-21T08:53:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.286+0000 7f716259c640 1 -- 172.21.15.82:0/782741893 learned_addr learned my addr 172.21.15.82:0/782741893 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.287+0000 7f716259c640 1 -- 172.21.15.82:0/782741893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f716410ea50 msgr2=0x7f716407eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.287+0000 7f716259c640 1 --2- 172.21.15.82:0/782741893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f716410ea50 0x7f716407eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.287+0000 7f716259c640 1 -- 172.21.15.82:0/782741893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7164109490 msgr2=0x7f716407e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.287+0000 7f716259c640 1 --2- 172.21.15.82:0/782741893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7164109490 0x7f716407e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.287+0000 7f716259c640 1 -- 172.21.15.82:0/782741893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716407f660 con 0x7f7164078130 2024-04-21T08:53:53.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.287+0000 7f716259c640 1 --2- 172.21.15.82:0/782741893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7164078130 0x7f7164084930 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f715800b4a0 tx=0x7f715800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.288+0000 7f71537fe640 1 -- 172.21.15.82:0/782741893 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158014070 con 0x7f7164078130 2024-04-21T08:53:53.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.288+0000 7f71537fe640 1 -- 172.21.15.82:0/782741893 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7158010ac0 con 0x7f7164078130 2024-04-21T08:53:53.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.288+0000 7f71537fe640 1 -- 172.21.15.82:0/782741893 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158017b60 con 0x7f7164078130 2024-04-21T08:53:53.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.289+0000 7f71689f4640 1 -- 172.21.15.82:0/782741893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71641c4850 con 0x7f7164078130 2024-04-21T08:53:53.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.289+0000 7f71689f4640 1 -- 172.21.15.82:0/782741893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71641c4cc0 con 0x7f7164078130 2024-04-21T08:53:53.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.291+0000 7f71689f4640 1 -- 172.21.15.82:0/782741893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7164073e70 con 0x7f7164078130 2024-04-21T08:53:53.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.293+0000 7f71537fe640 1 -- 172.21.15.82:0/782741893 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7158046020 con 0x7f7164078130 2024-04-21T08:53:53.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.293+0000 7f71537fe640 1 --2- 172.21.15.82:0/782741893 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f714c0a3f20 0x7f714c0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.294+0000 7f71537fe640 1 -- 172.21.15.82:0/782741893 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f71580d0830 con 0x7f7164078130 2024-04-21T08:53:53.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.294+0000 7f7161d9b640 1 --2- 172.21.15.82:0/782741893 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f714c0a3f20 0x7f714c0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.295+0000 7f7161d9b640 1 --2- 172.21.15.82:0/782741893 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f714c0a3f20 0x7f714c0a63e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f71540058c0 tx=0x7f7154005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.301+0000 7f71537fe640 1 -- 172.21.15.82:0/782741893 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f715801e030 con 0x7f7164078130 2024-04-21T08:53:53.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.718+0000 7f71689f4640 1 -- 172.21.15.82:0/782741893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7164068de0 con 0x7f7164078130 2024-04-21T08:53:53.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.719+0000 7f71537fe640 1 -- 172.21.15.82:0/782741893 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11953 (secure 0 0 0) 0x7f715809cc40 con 0x7f7164078130 2024-04-21T08:53:53.719 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:53:53.719 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:53:29.845577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:53:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 40 2024-04-21T08:53:53.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.722+0000 7f71689f4640 1 -- 172.21.15.82:0/782741893 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f714c0a3f20 msgr2=0x7f714c0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.722+0000 7f71689f4640 1 --2- 172.21.15.82:0/782741893 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f714c0a3f20 0x7f714c0a63e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f71540058c0 tx=0x7f7154005850 comp rx=0 tx=0).stop 2024-04-21T08:53:53.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.722+0000 7f71689f4640 1 -- 172.21.15.82:0/782741893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7164078130 msgr2=0x7f7164084930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.722+0000 7f71689f4640 1 --2- 172.21.15.82:0/782741893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7164078130 0x7f7164084930 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f715800b4a0 tx=0x7f715800b960 comp rx=0 tx=0).stop 2024-04-21T08:53:53.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.722+0000 7f71689f4640 1 -- 172.21.15.82:0/782741893 shutdown_connections 2024-04-21T08:53:53.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.722+0000 7f71689f4640 1 --2- 172.21.15.82:0/782741893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f716410ea50 0x7f716407eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.722+0000 7f71689f4640 1 --2- 172.21.15.82:0/782741893 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f714c0a3f20 0x7f714c0a63e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.722+0000 7f71689f4640 1 --2- 172.21.15.82:0/782741893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7164109490 0x7f716407e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.722+0000 7f71689f4640 1 --2- 172.21.15.82:0/782741893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7164078130 0x7f7164084930 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.722+0000 7f71689f4640 1 -- 172.21.15.82:0/782741893 >> 172.21.15.82:0/782741893 conn(0x7f716406e410 msgr2=0x7f716406f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.723+0000 7f71689f4640 1 -- 172.21.15.82:0/782741893 shutdown_connections 2024-04-21T08:53:53.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.723+0000 7f71689f4640 1 -- 172.21.15.82:0/782741893 wait complete. 2024-04-21T08:53:53.739 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:53:53.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.899+0000 7f2956ba4640 1 -- 172.21.15.82:0/3498161825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2950101110 msgr2=0x7f29501014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.899+0000 7f2956ba4640 1 --2- 172.21.15.82:0/3498161825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2950101110 0x7f29501014f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2944009900 tx=0x7f294402bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.899+0000 7f2956ba4640 1 -- 172.21.15.82:0/3498161825 shutdown_connections 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.899+0000 7f2956ba4640 1 --2- 172.21.15.82:0/3498161825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29501061e0 0x7f295010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.899+0000 7f2956ba4640 1 --2- 172.21.15.82:0/3498161825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2950101ac0 0x7f2950105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.899+0000 7f2956ba4640 1 --2- 172.21.15.82:0/3498161825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2950101110 0x7f29501014f0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2956ba4640 1 -- 172.21.15.82:0/3498161825 >> 172.21.15.82:0/3498161825 conn(0x7f2950077f80 msgr2=0x7f2950078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2956ba4640 1 -- 172.21.15.82:0/3498161825 shutdown_connections 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2956ba4640 1 -- 172.21.15.82:0/3498161825 wait complete. 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2956ba4640 1 Processor -- start 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2956ba4640 1 -- start start 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2956ba4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2950101110 0x7f295019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2956ba4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29501061e0 0x7f295019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2956ba4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2950198680 0x7f2950198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2956ba4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2950111190 con 0x7f2950198680 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2956ba4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2950111010 con 0x7f2950101110 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2956ba4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2950111310 con 0x7f29501061e0 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2955141640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2950198680 0x7f2950198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2955141640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2950198680 0x7f2950198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44412/0 (socket says 172.21.15.82:44412) 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2954940640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2950101110 0x7f295019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f2955141640 1 -- 172.21.15.82:0/3822102759 learned_addr learned my addr 172.21.15.82:0/3822102759 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.900+0000 7f294ffff640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29501061e0 0x7f295019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.901+0000 7f2955141640 1 -- 172.21.15.82:0/3822102759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29501061e0 msgr2=0x7f295019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.901+0000 7f2955141640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29501061e0 0x7f295019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.901+0000 7f2955141640 1 -- 172.21.15.82:0/3822102759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2950101110 msgr2=0x7f295019e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.901+0000 7f2955141640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2950101110 0x7f295019e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.901+0000 7f2955141640 1 -- 172.21.15.82:0/3822102759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2950199280 con 0x7f2950198680 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.901+0000 7f2954940640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2950101110 0x7f295019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:53.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.901+0000 7f2955141640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2950198680 0x7f2950198b00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f294800eb10 tx=0x7f294800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.901+0000 7f294dffb640 1 -- 172.21.15.82:0/3822102759 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948004b20 con 0x7f2950198680 2024-04-21T08:53:53.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.901+0000 7f294dffb640 1 -- 172.21.15.82:0/3822102759 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2948004cc0 con 0x7f2950198680 2024-04-21T08:53:53.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.901+0000 7f294dffb640 1 -- 172.21.15.82:0/3822102759 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948010670 con 0x7f2950198680 2024-04-21T08:53:53.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.901+0000 7f2956ba4640 1 -- 172.21.15.82:0/3822102759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29501a52c0 con 0x7f2950198680 2024-04-21T08:53:53.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.901+0000 7f2956ba4640 1 -- 172.21.15.82:0/3822102759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29501a5670 con 0x7f2950198680 2024-04-21T08:53:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f294dffb640 1 -- 172.21.15.82:0/3822102759 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2948005020 con 0x7f2950198680 2024-04-21T08:53:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f2959799740 1 Processor -- start 2024-04-21T08:53:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f2959799740 1 -- start start 2024-04-21T08:53:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f2959799740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba433b0 0x55a9eba43790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f2959799740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba4c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f2959799740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba4ce60 0x55a9eba4d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f2959799740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a9eb9a81f0 con 0x55a9eba43d60 2024-04-21T08:53:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f2959799740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a9eb9454b0 con 0x55a9eba433b0 2024-04-21T08:53:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f2959799740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a9eb9a41b0 con 0x55a9eba4ce60 2024-04-21T08:53:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f294dffb640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29400a3d20 0x7f29400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f2954940640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba4c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f2954940640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba4c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44426/0 (socket says 172.21.15.82:44426) 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f2954940640 1 -- 172.21.15.82:0/163694102 learned_addr learned my addr 172.21.15.82:0/163694102 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.904+0000 7f294ffff640 1 --2- 172.21.15.82:0/163694102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba4ce60 0x55a9eba4d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f294ffff640 1 --2- 172.21.15.82:0/163694102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba433b0 0x55a9eba43790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f2955141640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29400a3d20 0x7f29400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f294dffb640 1 -- 172.21.15.82:0/3822102759 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f29480d1100 con 0x7f2950198680 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f294ffff640 1 -- 172.21.15.82:0/163694102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba433b0 msgr2=0x55a9eba43790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f294ffff640 1 --2- 172.21.15.82:0/163694102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba433b0 0x55a9eba43790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f294ffff640 1 -- 172.21.15.82:0/163694102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 msgr2=0x55a9eba4c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f294ffff640 1 --2- 172.21.15.82:0/163694102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba4c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f294ffff640 1 -- 172.21.15.82:0/163694102 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9eb9371b0 con 0x55a9eba4ce60 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f294ffff640 1 --2- 172.21.15.82:0/163694102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba4ce60 0x55a9eba4d2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2950199ed0 tx=0x7f293c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18da05273fa90f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f2955141640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29400a3d20 0x7f29400a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f294809cc10 tx=0x7f294800efd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f293affd640 1 -- 172.21.15.82:0/163694102 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c004050 con 0x55a9eba4ce60 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f293affd640 1 -- 172.21.15.82:0/163694102 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293c00fc40 con 0x55a9eba4ce60 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.905+0000 7f293affd640 1 -- 172.21.15.82:0/163694102 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c003850 con 0x55a9eba4ce60 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.906+0000 7f2959799740 1 -- 172.21.15.82:0/163694102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba4ce60 msgr2=0x55a9eba4d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.906+0000 7f2959799740 1 --2- 172.21.15.82:0/163694102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba4ce60 0x55a9eba4d2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2950199ed0 tx=0x7f293c004730 comp rx=0 tx=0).stop 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.906+0000 7f2959799740 1 -- 172.21.15.82:0/163694102 shutdown_connections 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.906+0000 7f2959799740 1 --2- 172.21.15.82:0/163694102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba4ce60 0x55a9eba4d2c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.906+0000 7f2959799740 1 --2- 172.21.15.82:0/163694102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba4c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.906+0000 7f2959799740 1 --2- 172.21.15.82:0/163694102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba433b0 0x55a9eba43790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.906+0000 7f2959799740 1 -- 172.21.15.82:0/163694102 >> 172.21.15.82:0/163694102 conn(0x55a9eba3dba0 msgr2=0x55a9eba3df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.906+0000 7f2959799740 1 -- 172.21.15.82:0/163694102 shutdown_connections 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.906+0000 7f2959799740 1 -- 172.21.15.82:0/163694102 wait complete. 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.906+0000 7f2959799740 1 -- start start 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.906+0000 7f2959799740 1 Processor -- start 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2959799740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba53230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2959799740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba505c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2959799740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2959799740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a9eba53da0 con 0x55a9eba43d60 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2959799740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a9eb961500 con 0x55a9eba4ce60 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2959799740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a9eba53f20 con 0x55a9eba433b0 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2959799740 10 client.0 _ll_register_callbacks cb 0x55a9eb92fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2954940640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2954940640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44438/0 (socket says 172.21.15.82:44438) 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2954940640 1 -- 172.21.15.82:0/2805907344 learned_addr learned my addr 172.21.15.82:0/2805907344 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f294ffff640 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba50b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f294ffff640 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba53230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2954940640 1 -- 172.21.15.82:0/2805907344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 msgr2=0x55a9eba53230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2954940640 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba53230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2954940640 1 -- 172.21.15.82:0/2805907344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 msgr2=0x55a9eba50b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2954940640 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba50b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2954940640 1 -- 172.21.15.82:0/2805907344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9eb6ba3f0 con 0x55a9eba43d60 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f294ffff640 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f2954940640 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba505c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f294402c170 tx=0x7f294402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240e00c296706801 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.907+0000 7f292ffff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f292ffff640 1 -- 172.21.15.82:0/2805907344 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294402c5f0 con 0x55a9eba43d60 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f2959799740 1 -- 172.21.15.82:0/2805907344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f293c00fc40 con 0x55a9eba43d60 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f292ffff640 1 -- 172.21.15.82:0/2805907344 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f29440035f0 con 0x55a9eba43d60 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f292ffff640 1 -- 172.21.15.82:0/2805907344 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f29440037f0 con 0x55a9eba43d60 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f2959799740 10 client.17475 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f2959799740 1 -- 172.21.15.82:0/2805907344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a9eb7847d0 con 0x55a9eba43d60 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f292ffff640 1 -- 172.21.15.82:0/2805907344 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293c00fc40 con 0x55a9eba43d60 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f292ffff640 1 -- 172.21.15.82:0/2805907344 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f2944034ae0 con 0x55a9eba43d60 2024-04-21T08:53:53.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f2959799740 10 client.17475 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f2959799740 10 client.17475 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f2959799740 20 client.17475 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f2959799740 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a9eba60fb0 0x55a9eba633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f2959799740 4 client.17475 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.908+0000 7f2959799740 1 -- 172.21.15.82:0/2805907344 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9eb947930 con 0x55a9eba60fb0 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.909+0000 7f294ffff640 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a9eba60fb0 0x55a9eba633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.909+0000 7f292ffff640 10 client.17475 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.909+0000 7f294ffff640 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a9eba60fb0 0x55a9eba633d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eddb63fa8597dc07 server_cookie=410d4e339cde3eb6 in_seq=0 out_seq=0 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.909+0000 7f292ffff640 1 -- 172.21.15.82:0/2805907344 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9eb947930 con 0x55a9eba60fb0 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.909+0000 7f292ffff640 10 client.17475 handle_command_reply: tid=0 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.909+0000 7f2959799740 1 client.17475 shutdown 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.910+0000 7f2959799740 1 -- 172.21.15.82:0/2805907344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 msgr2=0x55a9eba505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.910+0000 7f2959799740 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba505c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f294402c170 tx=0x7f294402ca10 comp rx=0 tx=0).stop 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.910+0000 7f2959799740 1 -- 172.21.15.82:0/2805907344 shutdown_connections 2024-04-21T08:53:53.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.910+0000 7f2959799740 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a9eba60fb0 0x55a9eba633d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.910+0000 7f292ffff640 0 client.17475 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:53:53.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.910+0000 7f2959799740 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.910+0000 7f2959799740 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba505c0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.910+0000 7f2959799740 1 --2- 172.21.15.82:0/2805907344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba53230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.910+0000 7f2959799740 1 -- 172.21.15.82:0/2805907344 >> 172.21.15.82:0/2805907344 conn(0x55a9eba3dba0 msgr2=0x55a9eba3f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.910+0000 7f2959799740 1 -- 172.21.15.82:0/2805907344 shutdown_connections 2024-04-21T08:53:53.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.910+0000 7f2959799740 1 -- 172.21.15.82:0/2805907344 wait complete. 2024-04-21T08:53:53.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.910+0000 7f2959799740 20 client.17475 trim_cache size 0 max 16384 2024-04-21T08:53:53.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2959799740 1 Processor -- start 2024-04-21T08:53:53.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2959799740 1 -- start start 2024-04-21T08:53:53.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2959799740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba4e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2959799740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba5ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2959799740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba52cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2959799740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29501ad2d0 con 0x55a9eba43d60 2024-04-21T08:53:53.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2959799740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29501aef00 con 0x55a9eba4ce60 2024-04-21T08:53:53.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2959799740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29501ad9a0 con 0x55a9eba433b0 2024-04-21T08:53:53.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2954940640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba5ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2954940640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba5ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44448/0 (socket says 172.21.15.82:44448) 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f294ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba52cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2954940640 1 -- 172.21.15.82:0/2435365475 learned_addr learned my addr 172.21.15.82:0/2435365475 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f294ffff640 1 --2- 172.21.15.82:0/2435365475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba4e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2954940640 1 -- 172.21.15.82:0/2435365475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 msgr2=0x55a9eba4e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2954940640 1 --2- 172.21.15.82:0/2435365475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba4e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2954940640 1 -- 172.21.15.82:0/2435365475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 msgr2=0x55a9eba52cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2954940640 1 --2- 172.21.15.82:0/2435365475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba52cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2954940640 1 -- 172.21.15.82:0/2435365475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9eba63740 con 0x55a9eba43d60 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.924+0000 7f2954940640 1 --2- 172.21.15.82:0/2435365475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba5ccd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2944034e80 tx=0x7f294402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa80a9c72080dd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f293a7fc640 1 -- 172.21.15.82:0/2435365475 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294402c5f0 con 0x55a9eba43d60 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f293a7fc640 1 -- 172.21.15.82:0/2435365475 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2944003e40 con 0x55a9eba43d60 2024-04-21T08:53:53.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f293a7fc640 1 -- 172.21.15.82:0/2435365475 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944040cc0 con 0x55a9eba43d60 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f2959799740 1 -- 172.21.15.82:0/2435365475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 msgr2=0x55a9eba5ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f2959799740 1 --2- 172.21.15.82:0/2435365475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba5ccd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2944034e80 tx=0x7f294402bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f2959799740 1 -- 172.21.15.82:0/2435365475 shutdown_connections 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f2959799740 1 --2- 172.21.15.82:0/2435365475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba52cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f2959799740 1 --2- 172.21.15.82:0/2435365475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba5ccd0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f2959799740 1 --2- 172.21.15.82:0/2435365475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba4e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f2959799740 1 -- 172.21.15.82:0/2435365475 >> 172.21.15.82:0/2435365475 conn(0x55a9eba3dba0 msgr2=0x55a9eba3f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f2959799740 1 -- 172.21.15.82:0/2435365475 shutdown_connections 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f2959799740 1 -- 172.21.15.82:0/2435365475 wait complete. 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.925+0000 7f2959799740 1 -- start start 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2959799740 1 Processor -- start 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2959799740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2959799740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba59720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2959799740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba51b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2959799740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f29501ae800 con 0x55a9eba43d60 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2959799740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29501ae100 con 0x55a9eba4ce60 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2959799740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29501ab220 con 0x55a9eba433b0 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2959799740 10 client.0 _ll_register_callbacks cb 0x7f2944004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2954940640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba59720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2954940640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba59720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44458/0 (socket says 172.21.15.82:44458) 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2954940640 1 -- 172.21.15.82:0/1908736076 learned_addr learned my addr 172.21.15.82:0/1908736076 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f294ffff640 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba51b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f294ffff640 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2954940640 1 -- 172.21.15.82:0/1908736076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 msgr2=0x55a9eba591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2954940640 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2954940640 1 -- 172.21.15.82:0/1908736076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 msgr2=0x55a9eba51b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2954940640 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba51b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2954940640 1 -- 172.21.15.82:0/1908736076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9eba52260 con 0x55a9eba43d60 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f294ffff640 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f2954940640 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba59720 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f294402c170 tx=0x7f2944003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c32d91d2bcd399b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.926+0000 7f292ffff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.927+0000 7f292ffff640 1 -- 172.21.15.82:0/1908736076 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294402c5f0 con 0x55a9eba43d60 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.927+0000 7f292ffff640 1 -- 172.21.15.82:0/1908736076 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f2944040970 con 0x55a9eba43d60 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.927+0000 7f2959799740 1 -- 172.21.15.82:0/1908736076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2944003e40 con 0x55a9eba43d60 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.927+0000 7f292ffff640 1 -- 172.21.15.82:0/1908736076 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f294406e050 con 0x55a9eba43d60 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.927+0000 7f2959799740 10 client.17481 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.927+0000 7f2959799740 1 -- 172.21.15.82:0/1908736076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a9ebac6e90 con 0x55a9eba43d60 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.927+0000 7f292ffff640 1 -- 172.21.15.82:0/1908736076 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2944003e40 con 0x55a9eba43d60 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.927+0000 7f292ffff640 1 -- 172.21.15.82:0/1908736076 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f29440348f0 con 0x55a9eba43d60 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 10 client.17481 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 10 client.17481 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:53:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 20 client.17481 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a9eba60fb0 0x55a9ebac9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 4 client.17481 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 1 -- 172.21.15.82:0/1908736076 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9eb9b5c10 con 0x55a9eba60fb0 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f294ffff640 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a9eba60fb0 0x55a9ebac9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f292ffff640 10 client.17481 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f294ffff640 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a9eba60fb0 0x55a9ebac9510 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d0d4302ff201e492 server_cookie=3e3910de30217ef1 in_seq=0 out_seq=0 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f292ffff640 1 -- 172.21.15.82:0/1908736076 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9eb9b5c10 con 0x55a9eba60fb0 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f292ffff640 10 client.17481 handle_command_reply: tid=0 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 1 client.17481 shutdown 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 1 -- 172.21.15.82:0/1908736076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 msgr2=0x55a9eba59720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba59720 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f294402c170 tx=0x7f2944003c40 comp rx=0 tx=0).stop 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 1 -- 172.21.15.82:0/1908736076 shutdown_connections 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55a9eba60fb0 0x55a9ebac9510 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a9eba4ce60 0x55a9eba51b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9eba43d60 0x55a9eba59720 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f292ffff640 0 client.17481 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:53:53.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 1 --2- 172.21.15.82:0/1908736076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a9eba433b0 0x55a9eba591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.928+0000 7f2959799740 1 -- 172.21.15.82:0/1908736076 >> 172.21.15.82:0/1908736076 conn(0x55a9eba3dba0 msgr2=0x55a9eba3eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2959799740 1 -- 172.21.15.82:0/1908736076 shutdown_connections 2024-04-21T08:53:53.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2959799740 1 -- 172.21.15.82:0/1908736076 wait complete. 2024-04-21T08:53:53.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2959799740 20 client.17481 trim_cache size 0 max 16384 2024-04-21T08:53:53.935 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:53:53.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2956ba4640 1 -- 172.21.15.82:0/3822102759 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29400a3d20 msgr2=0x7f29400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2956ba4640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29400a3d20 0x7f29400a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f294809cc10 tx=0x7f294800efd0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2956ba4640 1 -- 172.21.15.82:0/3822102759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2950198680 msgr2=0x7f2950198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:53.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2956ba4640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2950198680 0x7f2950198b00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f294800eb10 tx=0x7f294800c2d0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2956ba4640 1 -- 172.21.15.82:0/3822102759 shutdown_connections 2024-04-21T08:53:53.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2956ba4640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f29400a3d20 0x7f29400a61e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2956ba4640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2950198680 0x7f2950198b00 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2956ba4640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29501061e0 0x7f295019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2956ba4640 1 --2- 172.21.15.82:0/3822102759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2950101110 0x7f295019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:53.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2956ba4640 1 -- 172.21.15.82:0/3822102759 >> 172.21.15.82:0/3822102759 conn(0x7f2950077f80 msgr2=0x7f2950104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:53.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2956ba4640 1 -- 172.21.15.82:0/3822102759 shutdown_connections 2024-04-21T08:53:53.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:53.929+0000 7f2956ba4640 1 -- 172.21.15.82:0/3822102759 wait complete. 2024-04-21T08:53:53.940 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:53:54.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.044+0000 7fa0b574a640 1 -- 172.21.15.82:0/2872202097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0b01010a0 msgr2=0x7fa0b0101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.044+0000 7fa0b574a640 1 --2- 172.21.15.82:0/2872202097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0b01010a0 0x7fa0b0101480 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009900 tx=0x7fa0a002bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:54.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.044+0000 7fa0b574a640 1 -- 172.21.15.82:0/2872202097 shutdown_connections 2024-04-21T08:53:54.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.044+0000 7fa0b574a640 1 --2- 172.21.15.82:0/2872202097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b0106170 0x7fa0b010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.044+0000 7fa0b574a640 1 --2- 172.21.15.82:0/2872202097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0b0101a50 0x7fa0b0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.044+0000 7fa0b574a640 1 --2- 172.21.15.82:0/2872202097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0b01010a0 0x7fa0b0101480 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.044+0000 7fa0b574a640 1 -- 172.21.15.82:0/2872202097 >> 172.21.15.82:0/2872202097 conn(0x7fa0b0077eb0 msgr2=0x7fa0b00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.044+0000 7fa0b574a640 1 -- 172.21.15.82:0/2872202097 shutdown_connections 2024-04-21T08:53:54.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.044+0000 7fa0b574a640 1 -- 172.21.15.82:0/2872202097 wait complete. 2024-04-21T08:53:54.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0b574a640 1 Processor -- start 2024-04-21T08:53:54.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0b574a640 1 -- start start 2024-04-21T08:53:54.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0b574a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b01010a0 0x7fa0b0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0b574a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0b0106170 0x7fa0b0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0b574a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0b019cff0 0x7fa0b019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0b574a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01121f0 con 0x7fa0b019cff0 2024-04-21T08:53:54.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0b574a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0112070 con 0x7fa0b0106170 2024-04-21T08:53:54.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0b574a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0112370 con 0x7fa0b01010a0 2024-04-21T08:53:54.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0b019cff0 0x7fa0b019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0b019cff0 0x7fa0b019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44474/0 (socket says 172.21.15.82:44474) 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0af7fe640 1 -- 172.21.15.82:0/766397952 learned_addr learned my addr 172.21.15.82:0/766397952 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0a67fc640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0b0106170 0x7fa0b0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.045+0000 7fa0aeffd640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b01010a0 0x7fa0b0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.046+0000 7fa0af7fe640 1 -- 172.21.15.82:0/766397952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b01010a0 msgr2=0x7fa0b0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.046+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b01010a0 0x7fa0b0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.046+0000 7fa0af7fe640 1 -- 172.21.15.82:0/766397952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0b0106170 msgr2=0x7fa0b0198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.046+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0b0106170 0x7fa0b0198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.046+0000 7fa0af7fe640 1 -- 172.21.15.82:0/766397952 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b019dae0 con 0x7fa0b019cff0 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.046+0000 7fa0a67fc640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0b0106170 0x7fa0b0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.046+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0b019cff0 0x7fa0b019d4a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009fd0 tx=0x7fa0a800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.046+0000 7fa0acff9640 1 -- 172.21.15.82:0/766397952 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8017070 con 0x7fa0b019cff0 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.046+0000 7fa0b574a640 1 -- 172.21.15.82:0/766397952 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b019ddd0 con 0x7fa0b019cff0 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.046+0000 7fa0acff9640 1 -- 172.21.15.82:0/766397952 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a8003e20 con 0x7fa0b019cff0 2024-04-21T08:53:54.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.046+0000 7fa0acff9640 1 -- 172.21.15.82:0/766397952 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8010460 con 0x7fa0b019cff0 2024-04-21T08:53:54.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.046+0000 7fa0b574a640 1 -- 172.21.15.82:0/766397952 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b01a5570 con 0x7fa0b019cff0 2024-04-21T08:53:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.048+0000 7fa0acff9640 1 -- 172.21.15.82:0/766397952 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa0a8003970 con 0x7fa0b019cff0 2024-04-21T08:53:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0b840f740 1 Processor -- start 2024-04-21T08:53:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0b840f740 1 -- start start 2024-04-21T08:53:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0acff9640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa09c0a3d20 0x7fa09c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0b840f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 0x565528daa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0b840f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 0x565528db3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0b840f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528db3a00 0x565528db5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0b840f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x565528d0f1f0 con 0x565528daa4d0 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0b840f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x565528cac4b0 con 0x565528db3a00 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0b840f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565528d0b1b0 con 0x565528daae80 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 0x565528daa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0a67fc640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa09c0a3d20 0x7fa09c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 0x565528daa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44484/0 (socket says 172.21.15.82:44484) 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0aeffd640 1 -- 172.21.15.82:0/3139398391 learned_addr learned my addr 172.21.15.82:0/3139398391 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0acff9640 1 -- 172.21.15.82:0/766397952 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa0a80d0900 con 0x7fa0b019cff0 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.049+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/3139398391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528db3a00 0x565528db5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.050+0000 7fa0a67fc640 1 --2- 172.21.15.82:0/3139398391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 0x565528db3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.050+0000 7fa0aeffd640 1 -- 172.21.15.82:0/3139398391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 msgr2=0x565528db3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.050+0000 7fa0aeffd640 1 --2- 172.21.15.82:0/3139398391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 0x565528db3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.050+0000 7fa0aeffd640 1 -- 172.21.15.82:0/3139398391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528db3a00 msgr2=0x565528db5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.050+0000 7fa0aeffd640 1 --2- 172.21.15.82:0/3139398391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528db3a00 0x565528db5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.050+0000 7fa0aeffd640 1 -- 172.21.15.82:0/3139398391 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565528c9e1b0 con 0x565528daa4d0 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.050+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/3139398391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528db3a00 0x565528db5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.050+0000 7fa0a67fc640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa09c0a3d20 0x7fa09c0a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0105c80 tx=0x7fa098013940 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.050+0000 7fa0aeffd640 1 --2- 172.21.15.82:0/3139398391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 0x565528daa8b0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009900 tx=0x7fa0a0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c7e0cd7bd3f640 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.050+0000 7fa0a57fa640 1 -- 172.21.15.82:0/3139398391 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0044020 con 0x565528daa4d0 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.050+0000 7fa0a57fa640 1 -- 172.21.15.82:0/3139398391 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a002cc90 con 0x565528daa4d0 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.050+0000 7fa0a57fa640 1 -- 172.21.15.82:0/3139398391 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00046c0 con 0x565528daa4d0 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.051+0000 7fa0b840f740 1 -- 172.21.15.82:0/3139398391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 msgr2=0x565528daa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.051+0000 7fa0b840f740 1 --2- 172.21.15.82:0/3139398391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 0x565528daa8b0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009900 tx=0x7fa0a0005dc0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.051+0000 7fa0b840f740 1 -- 172.21.15.82:0/3139398391 shutdown_connections 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.051+0000 7fa0b840f740 1 --2- 172.21.15.82:0/3139398391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528db3a00 0x565528db5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.051+0000 7fa0b840f740 1 --2- 172.21.15.82:0/3139398391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 0x565528db3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.051+0000 7fa0b840f740 1 --2- 172.21.15.82:0/3139398391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 0x565528daa8b0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.051+0000 7fa0b840f740 1 -- 172.21.15.82:0/3139398391 >> 172.21.15.82:0/3139398391 conn(0x565528da4ba0 msgr2=0x565528da4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.051+0000 7fa0b840f740 1 -- 172.21.15.82:0/3139398391 shutdown_connections 2024-04-21T08:53:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.051+0000 7fa0b840f740 1 -- 172.21.15.82:0/3139398391 wait complete. 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.051+0000 7fa0b840f740 1 -- start start 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.051+0000 7fa0b840f740 1 Processor -- start 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0b840f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 0x565528db32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0b840f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 0x565528db0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0b840f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528db3a00 0x565528db0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0b840f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x565528db6910 con 0x565528daa4d0 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0b840f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x565528cc8500 con 0x565528db3a00 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0b840f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565528db6a90 con 0x565528daae80 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 0x565528db32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 0x565528db32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44494/0 (socket says 172.21.15.82:44494) 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0aeffd640 1 -- 172.21.15.82:0/3142429869 learned_addr learned my addr 172.21.15.82:0/3142429869 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0b840f740 10 client.0 _ll_register_callbacks cb 0x565528c96b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:54.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528db3a00 0x565528db0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0a67fc640 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 0x565528db0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0aeffd640 1 -- 172.21.15.82:0/3142429869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 msgr2=0x565528db0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0aeffd640 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 0x565528db0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0aeffd640 1 -- 172.21.15.82:0/3142429869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528db3a00 msgr2=0x565528db0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0aeffd640 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528db3a00 0x565528db0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0aeffd640 1 -- 172.21.15.82:0/3142429869 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565528a213f0 con 0x565528daa4d0 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0aeffd640 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 0x565528db32f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0036040 tx=0x7fa0a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693e6c505d7a34f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0967fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0967fc640 1 -- 172.21.15.82:0/3142429869 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0044030 con 0x565528daa4d0 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0b840f740 1 -- 172.21.15.82:0/3142429869 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a002cc90 con 0x565528daa4d0 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0967fc640 1 -- 172.21.15.82:0/3142429869 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa0a0004450 con 0x565528daa4d0 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0967fc640 1 -- 172.21.15.82:0/3142429869 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa0a0059050 con 0x565528daa4d0 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0b840f740 10 client.17493 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.052+0000 7fa0b840f740 1 -- 172.21.15.82:0/3142429869 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x565528aeb7d0 con 0x565528daa4d0 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.053+0000 7fa0967fc640 1 -- 172.21.15.82:0/3142429869 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a002cc90 con 0x565528daa4d0 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.053+0000 7fa0967fc640 1 -- 172.21.15.82:0/3142429869 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7fa0a00407c0 con 0x565528daa4d0 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.053+0000 7fa0b840f740 10 client.17493 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.053+0000 7fa0b840f740 10 client.17493 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.053+0000 7fa0b840f740 20 client.17493 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.053+0000 7fa0b840f740 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x565528dc3a00 0x565528dc5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.053+0000 7fa0b840f740 4 client.17493 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.053+0000 7fa0b840f740 1 -- 172.21.15.82:0/3142429869 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565528cae930 con 0x565528dc3a00 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.053+0000 7fa0a67fc640 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x565528dc3a00 0x565528dc5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0a67fc640 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x565528dc3a00 0x565528dc5df0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=911ae0e432dc0e4a server_cookie=adb2e39e40326508 in_seq=0 out_seq=0 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0967fc640 10 client.17493 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0967fc640 1 -- 172.21.15.82:0/3142429869 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565528cae930 con 0x565528dc3a00 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0967fc640 10 client.17493 handle_command_reply: tid=0 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0b840f740 1 client.17493 shutdown 2024-04-21T08:53:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0b840f740 1 -- 172.21.15.82:0/3142429869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 msgr2=0x565528db32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0b840f740 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 0x565528db32f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0036040 tx=0x7fa0a0005fd0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0b840f740 1 -- 172.21.15.82:0/3142429869 shutdown_connections 2024-04-21T08:53:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0b840f740 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x565528dc3a00 0x565528dc5df0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0b840f740 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528db3a00 0x565528db0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0967fc640 0 client.17493 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:53:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0b840f740 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 0x565528db0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0b840f740 1 --2- 172.21.15.82:0/3142429869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daa4d0 0x565528db32f0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.054+0000 7fa0b840f740 1 -- 172.21.15.82:0/3142429869 >> 172.21.15.82:0/3142429869 conn(0x565528da4ba0 msgr2=0x565528da68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.055+0000 7fa0b840f740 1 -- 172.21.15.82:0/3142429869 shutdown_connections 2024-04-21T08:53:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.055+0000 7fa0b840f740 1 -- 172.21.15.82:0/3142429869 wait complete. 2024-04-21T08:53:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.055+0000 7fa0b840f740 20 client.17493 trim_cache size 0 max 16384 2024-04-21T08:53:54.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.069+0000 7fa0b840f740 1 Processor -- start 2024-04-21T08:53:54.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.069+0000 7fa0b840f740 1 -- start start 2024-04-21T08:53:54.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.070+0000 7fa0b840f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528daa4d0 0x565528dbf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.070+0000 7fa0b840f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 0x565528db2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.070+0000 7fa0b840f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528db3a00 0x565528db32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.070+0000 7fa0af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528db3a00 0x565528db32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.070+0000 7fa0af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528db3a00 0x565528db32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44502/0 (socket says 172.21.15.82:44502) 2024-04-21T08:53:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.070+0000 7fa0b840f740 1 -- 172.21.15.82:0/1731896007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01af340 con 0x565528db3a00 2024-04-21T08:53:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.070+0000 7fa0b840f740 1 -- 172.21.15.82:0/1731896007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01b0f70 con 0x565528daa4d0 2024-04-21T08:53:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.070+0000 7fa0b840f740 1 -- 172.21.15.82:0/1731896007 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01afa10 con 0x565528daae80 2024-04-21T08:53:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.070+0000 7fa0af7fe640 1 -- 172.21.15.82:0/1731896007 learned_addr learned my addr 172.21.15.82:0/1731896007 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.071+0000 7fa0af7fe640 1 -- 172.21.15.82:0/1731896007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 msgr2=0x565528db2db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:53:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.071+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/1731896007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 0x565528db2db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.071+0000 7fa0af7fe640 1 -- 172.21.15.82:0/1731896007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528daa4d0 msgr2=0x565528dbf7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:53:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.071+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/1731896007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528daa4d0 0x565528dbf7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.071+0000 7fa0af7fe640 1 -- 172.21.15.82:0/1731896007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565528daac20 con 0x565528db3a00 2024-04-21T08:53:54.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.072+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/1731896007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528db3a00 0x565528db32f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8098eb0 tx=0x7fa0a8003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c00f6d09694a9cb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.072+0000 7fa0a4ff9640 1 -- 172.21.15.82:0/1731896007 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8020040 con 0x565528db3a00 2024-04-21T08:53:54.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.072+0000 7fa0a4ff9640 1 -- 172.21.15.82:0/1731896007 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a809c4e0 con 0x565528db3a00 2024-04-21T08:53:54.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.072+0000 7fa0a4ff9640 1 -- 172.21.15.82:0/1731896007 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a80cdf10 con 0x565528db3a00 2024-04-21T08:53:54.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.073+0000 7fa0b840f740 1 -- 172.21.15.82:0/1731896007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528db3a00 msgr2=0x565528db32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.073+0000 7fa0b840f740 1 --2- 172.21.15.82:0/1731896007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528db3a00 0x565528db32f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8098eb0 tx=0x7fa0a8003d20 comp rx=0 tx=0).stop 2024-04-21T08:53:54.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.074+0000 7fa0b840f740 1 -- 172.21.15.82:0/1731896007 shutdown_connections 2024-04-21T08:53:54.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.074+0000 7fa0b840f740 1 --2- 172.21.15.82:0/1731896007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528db3a00 0x565528db32f0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.074+0000 7fa0b840f740 1 --2- 172.21.15.82:0/1731896007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528daae80 0x565528db2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.074+0000 7fa0b840f740 1 --2- 172.21.15.82:0/1731896007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528daa4d0 0x565528dbf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.074+0000 7fa0b840f740 1 -- 172.21.15.82:0/1731896007 >> 172.21.15.82:0/1731896007 conn(0x565528da4ba0 msgr2=0x565528da3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.076+0000 7fa0b840f740 1 -- 172.21.15.82:0/1731896007 shutdown_connections 2024-04-21T08:53:54.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.077+0000 7fa0b840f740 1 -- 172.21.15.82:0/1731896007 wait complete. 2024-04-21T08:53:54.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.077+0000 7fa0b840f740 1 -- start start 2024-04-21T08:53:54.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.077+0000 7fa0b840f740 1 Processor -- start 2024-04-21T08:53:54.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.078+0000 7fa0b840f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528daa4d0 0x565528dad310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.078+0000 7fa0b840f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daae80 0x565528dad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.078+0000 7fa0b840f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528db3a00 0x565528dbacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.078+0000 7fa0b840f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01b0870 con 0x565528daae80 2024-04-21T08:53:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.078+0000 7fa0b840f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01b0170 con 0x565528daa4d0 2024-04-21T08:53:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.078+0000 7fa0b840f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01ad2f0 con 0x565528db3a00 2024-04-21T08:53:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.078+0000 7fa0b840f740 10 client.0 _ll_register_callbacks cb 0x7fa0a002c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.079+0000 7fa0af7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528db3a00 0x565528dbacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.079+0000 7fa0af7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528db3a00 0x565528dbacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:44388/0 (socket says 172.21.15.82:44388) 2024-04-21T08:53:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.079+0000 7fa0af7fe640 1 -- 172.21.15.82:0/4007882830 learned_addr learned my addr 172.21.15.82:0/4007882830 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.079+0000 7fa0a67fc640 1 --2- 172.21.15.82:0/4007882830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daae80 0x565528dad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.079+0000 7fa0af7fe640 1 -- 172.21.15.82:0/4007882830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528daa4d0 msgr2=0x565528dad310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:53:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.079+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/4007882830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528daa4d0 0x565528dad310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.079+0000 7fa0af7fe640 1 -- 172.21.15.82:0/4007882830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daae80 msgr2=0x565528dad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.079+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/4007882830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daae80 0x565528dad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.079+0000 7fa0af7fe640 1 -- 172.21.15.82:0/4007882830 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565528dbb240 con 0x565528db3a00 2024-04-21T08:53:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.079+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/4007882830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528db3a00 0x565528dbacd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800b760 tx=0x7fa0a809cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abf9f08d9c5aaeff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.079+0000 7fa0967fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:53:54.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.080+0000 7fa0967fc640 1 -- 172.21.15.82:0/4007882830 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8020070 con 0x565528db3a00 2024-04-21T08:53:54.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.080+0000 7fa0967fc640 1 -- 172.21.15.82:0/4007882830 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fa0a8098b50 con 0x565528db3a00 2024-04-21T08:53:54.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.080+0000 7fa0b840f740 1 -- 172.21.15.82:0/4007882830 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a809c4e0 con 0x565528db3a00 2024-04-21T08:53:54.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.080+0000 7fa0967fc640 1 -- 172.21.15.82:0/4007882830 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa0a809a600 con 0x565528db3a00 2024-04-21T08:53:54.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.081+0000 7fa0b840f740 10 client.26506 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:54.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.081+0000 7fa0b840f740 1 -- 172.21.15.82:0/4007882830 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x565528dbc520 con 0x565528db3a00 2024-04-21T08:53:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.082+0000 7fa0967fc640 1 -- 172.21.15.82:0/4007882830 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a8087520 con 0x565528db3a00 2024-04-21T08:53:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.083+0000 7fa0967fc640 1 -- 172.21.15.82:0/4007882830 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7fa0a80bb280 con 0x565528db3a00 2024-04-21T08:53:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.083+0000 7fa0b840f740 10 client.26506 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.083+0000 7fa0b840f740 10 client.26506 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:53:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.083+0000 7fa0b840f740 20 client.26506 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.083+0000 7fa0b840f740 1 --2- 172.21.15.82:0/4007882830 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x565528dc3a00 0x565528e34c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.083+0000 7fa0b840f740 4 client.26506 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:53:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.083+0000 7fa0b840f740 1 -- 172.21.15.82:0/4007882830 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa0a809c4e0 con 0x565528dc3a00 2024-04-21T08:53:54.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.083+0000 7fa0aeffd640 1 --2- 172.21.15.82:0/4007882830 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x565528dc3a00 0x565528e34c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.084+0000 7fa0aeffd640 1 --2- 172.21.15.82:0/4007882830 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x565528dc3a00 0x565528e34c80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=58c6cf67659ea19 server_cookie=69a1375987ee80f7 in_seq=0 out_seq=0 2024-04-21T08:53:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.084+0000 7fa0967fc640 10 client.26506 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:53:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.084+0000 7fa0967fc640 1 -- 172.21.15.82:0/4007882830 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa0a809c4e0 con 0x565528dc3a00 2024-04-21T08:53:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.084+0000 7fa0967fc640 10 client.26506 handle_command_reply: tid=0 2024-04-21T08:53:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.084+0000 7fa0b840f740 1 client.26506 shutdown 2024-04-21T08:53:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.085+0000 7fa0b840f740 1 -- 172.21.15.82:0/4007882830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528db3a00 msgr2=0x565528dbacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.085+0000 7fa0b840f740 1 --2- 172.21.15.82:0/4007882830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528db3a00 0x565528dbacd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800b760 tx=0x7fa0a809cd20 comp rx=0 tx=0).stop 2024-04-21T08:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.085+0000 7fa0b840f740 1 -- 172.21.15.82:0/4007882830 shutdown_connections 2024-04-21T08:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.085+0000 7fa0b840f740 1 --2- 172.21.15.82:0/4007882830 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x565528dc3a00 0x565528e34c80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.085+0000 7fa0b840f740 1 --2- 172.21.15.82:0/4007882830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565528db3a00 0x565528dbacd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.085+0000 7fa0b840f740 1 --2- 172.21.15.82:0/4007882830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565528daae80 0x565528dad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.085+0000 7fa0b840f740 1 --2- 172.21.15.82:0/4007882830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565528daa4d0 0x565528dad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.085+0000 7fa0b840f740 1 -- 172.21.15.82:0/4007882830 >> 172.21.15.82:0/4007882830 conn(0x565528da4ba0 msgr2=0x565528da3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.085+0000 7fa0967fc640 0 client.26506 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:53:54.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.086+0000 7fa0b840f740 1 -- 172.21.15.82:0/4007882830 shutdown_connections 2024-04-21T08:53:54.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.086+0000 7fa0b840f740 1 -- 172.21.15.82:0/4007882830 wait complete. 2024-04-21T08:53:54.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.086+0000 7fa0b840f740 20 client.26506 trim_cache size 0 max 16384 2024-04-21T08:53:54.087 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:53:54.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.089+0000 7fa0b574a640 1 -- 172.21.15.82:0/766397952 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa09c0a3d20 msgr2=0x7fa09c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.089+0000 7fa0b574a640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa09c0a3d20 0x7fa09c0a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0105c80 tx=0x7fa098013940 comp rx=0 tx=0).stop 2024-04-21T08:53:54.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.089+0000 7fa0b574a640 1 -- 172.21.15.82:0/766397952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0b019cff0 msgr2=0x7fa0b019d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.089+0000 7fa0b574a640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0b019cff0 0x7fa0b019d4a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009fd0 tx=0x7fa0a800b260 comp rx=0 tx=0).stop 2024-04-21T08:53:54.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.090+0000 7fa0b574a640 1 -- 172.21.15.82:0/766397952 shutdown_connections 2024-04-21T08:53:54.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.090+0000 7fa0b574a640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa09c0a3d20 0x7fa09c0a61e0 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0105c80 tx=0x7fa098013940 comp rx=0 tx=0).stop 2024-04-21T08:53:54.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.090+0000 7fa0b574a640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0b019cff0 0x7fa0b019d4a0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.090+0000 7fa0b574a640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa0b0106170 0x7fa0b0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.090+0000 7fa0b574a640 1 --2- 172.21.15.82:0/766397952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b01010a0 0x7fa0b0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.090+0000 7fa0b574a640 1 -- 172.21.15.82:0/766397952 >> 172.21.15.82:0/766397952 conn(0x7fa0b0077eb0 msgr2=0x7fa0b0107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.091+0000 7fa0b574a640 1 -- 172.21.15.82:0/766397952 shutdown_connections 2024-04-21T08:53:54.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.092+0000 7fa0b574a640 1 -- 172.21.15.82:0/766397952 wait complete. 2024-04-21T08:53:54.111 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:53:54.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.272+0000 7f50ed8d2640 1 -- 172.21.15.82:0/3590942018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50e81015a0 msgr2=0x7f50e8101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.272+0000 7f50ed8d2640 1 --2- 172.21.15.82:0/3590942018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50e81015a0 0x7f50e8101980 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009860 tx=0x7f50d002bbb0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.273+0000 7f50ed8d2640 1 -- 172.21.15.82:0/3590942018 shutdown_connections 2024-04-21T08:53:54.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.273+0000 7f50ed8d2640 1 --2- 172.21.15.82:0/3590942018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e8106150 0x7f50e806a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.273+0000 7f50ed8d2640 1 --2- 172.21.15.82:0/3590942018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50e8101f50 0x7f50e8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.273+0000 7f50ed8d2640 1 --2- 172.21.15.82:0/3590942018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50e81015a0 0x7f50e8101980 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.273+0000 7f50ed8d2640 1 -- 172.21.15.82:0/3590942018 >> 172.21.15.82:0/3590942018 conn(0x7f50e80769d0 msgr2=0x7f50e8076de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.273+0000 7f50ed8d2640 1 -- 172.21.15.82:0/3590942018 shutdown_connections 2024-04-21T08:53:54.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.273+0000 7f50ed8d2640 1 -- 172.21.15.82:0/3590942018 wait complete. 2024-04-21T08:53:54.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.274+0000 7f50ed8d2640 1 Processor -- start 2024-04-21T08:53:54.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.274+0000 7f50ed8d2640 1 -- start start 2024-04-21T08:53:54.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50ed8d2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50e81015a0 0x7f50e810d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50ed8d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50e8106150 0x7f50e810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50ed8d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e81a4900 0x7f50e81a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50ed8d2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f50e81100b0 con 0x7f50e81015a0 2024-04-21T08:53:54.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50ed8d2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f50e810ff30 con 0x7f50e8106150 2024-04-21T08:53:54.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50ed8d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f50e8110230 con 0x7f50e81a4900 2024-04-21T08:53:54.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50e77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e81a4900 0x7f50e81a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50e77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e81a4900 0x7f50e81a4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:44394/0 (socket says 172.21.15.82:44394) 2024-04-21T08:53:54.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50e77fe640 1 -- 172.21.15.82:0/1967306124 learned_addr learned my addr 172.21.15.82:0/1967306124 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50deffd640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50e81015a0 0x7f50e810d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50e8106150 0x7f50e810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50e77fe640 1 -- 172.21.15.82:0/1967306124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50e8106150 msgr2=0x7f50e810d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.275+0000 7f50e77fe640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50e8106150 0x7f50e810d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.276+0000 7f50e77fe640 1 -- 172.21.15.82:0/1967306124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50e81015a0 msgr2=0x7f50e810d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.276+0000 7f50e77fe640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50e81015a0 0x7f50e810d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.276+0000 7f50e77fe640 1 -- 172.21.15.82:0/1967306124 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e81a53a0 con 0x7f50e81a4900 2024-04-21T08:53:54.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.278+0000 7f50e77fe640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e81a4900 0x7f50e81a4db0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f50e000db20 tx=0x7f50e000b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.278+0000 7f50ec8d0640 1 -- 172.21.15.82:0/1967306124 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0017070 con 0x7f50e81a4900 2024-04-21T08:53:54.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.279+0000 7f50ed8d2640 1 -- 172.21.15.82:0/1967306124 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e81a5690 con 0x7f50e81a4900 2024-04-21T08:53:54.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.279+0000 7f50ec8d0640 1 -- 172.21.15.82:0/1967306124 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e0003e20 con 0x7f50e81a4900 2024-04-21T08:53:54.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.279+0000 7f50ec8d0640 1 -- 172.21.15.82:0/1967306124 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e00106c0 con 0x7f50e81a4900 2024-04-21T08:53:54.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.279+0000 7f50ed8d2640 1 -- 172.21.15.82:0/1967306124 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e81abca0 con 0x7f50e81a4900 2024-04-21T08:53:54.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.283+0000 7f50ec8d0640 1 -- 172.21.15.82:0/1967306124 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f50e0010860 con 0x7f50e81a4900 2024-04-21T08:53:54.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.285+0000 7f50f05a2740 1 Processor -- start 2024-04-21T08:53:54.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.285+0000 7f50f05a2740 1 -- start start 2024-04-21T08:53:54.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.285+0000 7f50f05a2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 0x5595c5fa88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.285+0000 7f50f05a2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa8e80 0x5595c5fb1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.286+0000 7f50f05a2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c5fb3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.286+0000 7f50f05a2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5595c5f0d1f0 con 0x5595c5fb1a00 2024-04-21T08:53:54.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.286+0000 7f50f05a2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5595c5eaa4b0 con 0x5595c5fa84d0 2024-04-21T08:53:54.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.286+0000 7f50f05a2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5595c5f091b0 con 0x5595c5fa8e80 2024-04-21T08:53:54.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.286+0000 7f50ec8d0640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f50d40a3e20 0x7f50d40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.286+0000 7f50ec8d0640 1 -- 172.21.15.82:0/1967306124 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f50e00d1800 con 0x7f50e81a4900 2024-04-21T08:53:54.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.287+0000 7f50e6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 0x5595c5fa88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.287+0000 7f50e77fe640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f50d40a3e20 0x7f50d40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.287+0000 7f50e6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 0x5595c5fa88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:57964/0 (socket says 172.21.15.82:57964) 2024-04-21T08:53:54.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.287+0000 7f50e6ffd640 1 -- 172.21.15.82:0/1283130111 learned_addr learned my addr 172.21.15.82:0/1283130111 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.287+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/1283130111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c5fb3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.287+0000 7f50e6ffd640 1 -- 172.21.15.82:0/1283130111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa8e80 msgr2=0x5595c5fb1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.287+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/1283130111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa8e80 0x5595c5fb1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.287+0000 7f50e6ffd640 1 -- 172.21.15.82:0/1283130111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 msgr2=0x5595c5fb3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.287+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/1283130111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c5fb3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.287+0000 7f50e6ffd640 1 -- 172.21.15.82:0/1283130111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c5e9c1b0 con 0x5595c5fa84d0 2024-04-21T08:53:54.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.287+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/1283130111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 0x5595c5fa88b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009f90 tx=0x7f50d8009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e6fe5118e91d4ac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.288+0000 7f50e77fe640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f50d40a3e20 0x7f50d40a62e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50e00997e0 tx=0x7f50e0010000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.289+0000 7f50ddffb640 1 -- 172.21.15.82:0/1283130111 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8004020 con 0x5595c5fa84d0 2024-04-21T08:53:54.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.289+0000 7f50ddffb640 1 -- 172.21.15.82:0/1283130111 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d80041c0 con 0x5595c5fa84d0 2024-04-21T08:53:54.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.290+0000 7f50f05a2740 1 -- 172.21.15.82:0/1283130111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 msgr2=0x5595c5fa88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.290+0000 7f50f05a2740 1 --2- 172.21.15.82:0/1283130111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 0x5595c5fa88b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009f90 tx=0x7f50d8009600 comp rx=0 tx=0).stop 2024-04-21T08:53:54.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.292+0000 7f50f05a2740 1 -- 172.21.15.82:0/1283130111 shutdown_connections 2024-04-21T08:53:54.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.292+0000 7f50f05a2740 1 --2- 172.21.15.82:0/1283130111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c5fb3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.292+0000 7f50f05a2740 1 --2- 172.21.15.82:0/1283130111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa8e80 0x5595c5fb1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.292+0000 7f50f05a2740 1 --2- 172.21.15.82:0/1283130111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 0x5595c5fa88b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.292+0000 7f50f05a2740 1 -- 172.21.15.82:0/1283130111 >> 172.21.15.82:0/1283130111 conn(0x5595c5fa2ba0 msgr2=0x5595c5fa2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.293+0000 7f50f05a2740 1 -- 172.21.15.82:0/1283130111 shutdown_connections 2024-04-21T08:53:54.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.294+0000 7f50f05a2740 1 -- 172.21.15.82:0/1283130111 wait complete. 2024-04-21T08:53:54.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.294+0000 7f50f05a2740 1 -- start start 2024-04-21T08:53:54.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.295+0000 7f50f05a2740 1 Processor -- start 2024-04-21T08:53:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.295+0000 7f50f05a2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa84d0 0x5595c5fb11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.295+0000 7f50f05a2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 0x5595c5fae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.295+0000 7f50f05a2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c5faea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.295+0000 7f50f05a2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5595c5fc59d0 con 0x5595c5fb1a00 2024-04-21T08:53:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.295+0000 7f50f05a2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5595c5ec6500 con 0x5595c5fa8e80 2024-04-21T08:53:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.295+0000 7f50f05a2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5595c5fc5b50 con 0x5595c5fa84d0 2024-04-21T08:53:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.295+0000 7f50f05a2740 10 client.0 _ll_register_callbacks cb 0x5595c5e94b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.295+0000 7f50deffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 0x5595c5fae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.295+0000 7f50e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c5faea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.295+0000 7f50deffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 0x5595c5fae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:57980/0 (socket says 172.21.15.82:57980) 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.295+0000 7f50deffd640 1 -- 172.21.15.82:0/300338158 learned_addr learned my addr 172.21.15.82:0/300338158 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.296+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa84d0 0x5595c5fb11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.296+0000 7f50e6ffd640 1 -- 172.21.15.82:0/300338158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa84d0 msgr2=0x5595c5fb11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.296+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa84d0 0x5595c5fb11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.296+0000 7f50e6ffd640 1 -- 172.21.15.82:0/300338158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 msgr2=0x5595c5fae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.296+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 0x5595c5fae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.296+0000 7f50e6ffd640 1 -- 172.21.15.82:0/300338158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c5c1f3f0 con 0x5595c5fb1a00 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.296+0000 7f50deffd640 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 0x5595c5fae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.297+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c5faea80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f50d8005940 tx=0x7f50d8009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7a396fe6b99b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.297+0000 7f50c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.297+0000 7f50c6ffd640 1 -- 172.21.15.82:0/300338158 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8003890 con 0x5595c5fb1a00 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.297+0000 7f50f05a2740 1 -- 172.21.15.82:0/300338158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d80041c0 con 0x5595c5fb1a00 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.297+0000 7f50c6ffd640 1 -- 172.21.15.82:0/300338158 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f50d8003a30 con 0x5595c5fb1a00 2024-04-21T08:53:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.298+0000 7f50c6ffd640 1 -- 172.21.15.82:0/300338158 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f50d8005030 con 0x5595c5fb1a00 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.298+0000 7f50f05a2740 10 client.17511 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.298+0000 7f50f05a2740 1 -- 172.21.15.82:0/300338158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5595c5ce97d0 con 0x5595c5fb1a00 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.299+0000 7f50c6ffd640 1 -- 172.21.15.82:0/300338158 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d80041c0 con 0x5595c5fb1a00 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.299+0000 7f50c6ffd640 1 -- 172.21.15.82:0/300338158 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f50d8004b50 con 0x5595c5fb1a00 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.299+0000 7f50f05a2740 10 client.17511 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.299+0000 7f50f05a2740 10 client.17511 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.299+0000 7f50f05a2740 20 client.17511 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.299+0000 7f50f05a2740 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5595c5fd2d00 0x5595c5fd50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.299+0000 7f50f05a2740 4 client.17511 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.299+0000 7f50f05a2740 1 -- 172.21.15.82:0/300338158 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595c5eac930 con 0x5595c5fd2d00 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.299+0000 7f50deffd640 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5595c5fd2d00 0x5595c5fd50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.300+0000 7f50deffd640 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5595c5fd2d00 0x5595c5fd50f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d8b9d4adbda2ed2 server_cookie=d4fc19e81224a005 in_seq=0 out_seq=0 2024-04-21T08:53:54.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.300+0000 7f50c6ffd640 10 client.17511 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.302+0000 7f50c6ffd640 1 -- 172.21.15.82:0/300338158 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595c5eac930 con 0x5595c5fd2d00 2024-04-21T08:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.302+0000 7f50c6ffd640 10 client.17511 handle_command_reply: tid=0 2024-04-21T08:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.302+0000 7f50f05a2740 1 client.17511 shutdown 2024-04-21T08:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.302+0000 7f50f05a2740 1 -- 172.21.15.82:0/300338158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 msgr2=0x5595c5faea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.302+0000 7f50f05a2740 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c5faea80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f50d8005940 tx=0x7f50d8009460 comp rx=0 tx=0).stop 2024-04-21T08:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.302+0000 7f50f05a2740 1 -- 172.21.15.82:0/300338158 shutdown_connections 2024-04-21T08:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.302+0000 7f50f05a2740 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5595c5fd2d00 0x5595c5fd50f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.302+0000 7f50f05a2740 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c5faea80 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.302+0000 7f50f05a2740 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 0x5595c5fae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.303+0000 7f50f05a2740 1 --2- 172.21.15.82:0/300338158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa84d0 0x5595c5fb11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.303+0000 7f50f05a2740 1 -- 172.21.15.82:0/300338158 >> 172.21.15.82:0/300338158 conn(0x5595c5fa2ba0 msgr2=0x5595c5fa48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.303+0000 7f50c6ffd640 0 client.17511 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:53:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.303+0000 7f50f05a2740 1 -- 172.21.15.82:0/300338158 shutdown_connections 2024-04-21T08:53:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.303+0000 7f50f05a2740 1 -- 172.21.15.82:0/300338158 wait complete. 2024-04-21T08:53:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.303+0000 7f50f05a2740 20 client.17511 trim_cache size 0 max 16384 2024-04-21T08:53:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.322+0000 7f50f05a2740 1 Processor -- start 2024-04-21T08:53:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.322+0000 7f50f05a2740 1 -- start start 2024-04-21T08:53:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50f05a2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fa84d0 0x5595c5fce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50f05a2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 0x5595c5fb0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50f05a2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fb1a00 0x5595c5fb11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50f05a2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f50c80100f0 con 0x5595c5fa84d0 2024-04-21T08:53:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50f05a2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f50c8011d20 con 0x5595c5fa8e80 2024-04-21T08:53:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50f05a2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f50c80107c0 con 0x5595c5fb1a00 2024-04-21T08:53:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fa84d0 0x5595c5fce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fa84d0 0x5595c5fce500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44552/0 (socket says 172.21.15.82:44552) 2024-04-21T08:53:54.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50e6ffd640 1 -- 172.21.15.82:0/857806379 learned_addr learned my addr 172.21.15.82:0/857806379 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50deffd640 1 --2- 172.21.15.82:0/857806379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 0x5595c5fb0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/857806379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fb1a00 0x5595c5fb11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50deffd640 1 -- 172.21.15.82:0/857806379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fb1a00 msgr2=0x5595c5fb11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50deffd640 1 --2- 172.21.15.82:0/857806379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fb1a00 0x5595c5fb11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50deffd640 1 -- 172.21.15.82:0/857806379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fa84d0 msgr2=0x5595c5fce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50deffd640 1 --2- 172.21.15.82:0/857806379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fa84d0 0x5595c5fce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50deffd640 1 -- 172.21.15.82:0/857806379 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c5fa8c20 con 0x5595c5fa8e80 2024-04-21T08:53:54.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/857806379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fa84d0 0x5595c5fce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:54.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.323+0000 7f50deffd640 1 --2- 172.21.15.82:0/857806379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 0x5595c5fb0c70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5595c5fafb10 tx=0x7f50d0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=219356ea1a118361 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.324+0000 7f50dd7fa640 1 -- 172.21.15.82:0/857806379 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0039040 con 0x5595c5fa8e80 2024-04-21T08:53:54.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.324+0000 7f50dd7fa640 1 -- 172.21.15.82:0/857806379 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d0049030 con 0x5595c5fa8e80 2024-04-21T08:53:54.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.324+0000 7f50dd7fa640 1 -- 172.21.15.82:0/857806379 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0003970 con 0x5595c5fa8e80 2024-04-21T08:53:54.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.325+0000 7f50f05a2740 1 -- 172.21.15.82:0/857806379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 msgr2=0x5595c5fb0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.325+0000 7f50f05a2740 1 --2- 172.21.15.82:0/857806379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 0x5595c5fb0c70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5595c5fafb10 tx=0x7f50d0005f90 comp rx=0 tx=0).stop 2024-04-21T08:53:54.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.325+0000 7f50f05a2740 1 -- 172.21.15.82:0/857806379 shutdown_connections 2024-04-21T08:53:54.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.325+0000 7f50f05a2740 1 --2- 172.21.15.82:0/857806379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fb1a00 0x5595c5fb11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.325+0000 7f50f05a2740 1 --2- 172.21.15.82:0/857806379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa8e80 0x5595c5fb0c70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.325+0000 7f50f05a2740 1 --2- 172.21.15.82:0/857806379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fa84d0 0x5595c5fce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.325+0000 7f50f05a2740 1 -- 172.21.15.82:0/857806379 >> 172.21.15.82:0/857806379 conn(0x5595c5fa2ba0 msgr2=0x5595c5fcddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.325+0000 7f50f05a2740 1 -- 172.21.15.82:0/857806379 shutdown_connections 2024-04-21T08:53:54.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.325+0000 7f50f05a2740 1 -- 172.21.15.82:0/857806379 wait complete. 2024-04-21T08:53:54.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.325+0000 7f50f05a2740 1 -- start start 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.325+0000 7f50f05a2740 1 Processor -- start 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50f05a2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 0x5595c5fd5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50f05a2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa8e80 0x5595c5fd6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50f05a2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c6037ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50f05a2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f50c8011620 con 0x5595c5fb1a00 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50f05a2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f50c8010f20 con 0x5595c5fa84d0 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50f05a2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f50c800e040 con 0x5595c5fa8e80 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50f05a2740 10 client.0 _ll_register_callbacks cb 0x7f50d8004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50e6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 0x5595c5fd5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50e6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 0x5595c5fd5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:58004/0 (socket says 172.21.15.82:58004) 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50e6ffd640 1 -- 172.21.15.82:0/3880020357 learned_addr learned my addr 172.21.15.82:0/3880020357 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50deffd640 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa8e80 0x5595c5fd6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c6037ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50e6ffd640 1 -- 172.21.15.82:0/3880020357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa8e80 msgr2=0x5595c5fd6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa8e80 0x5595c5fd6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50e6ffd640 1 -- 172.21.15.82:0/3880020357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 msgr2=0x5595c5fd5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 0x5595c5fd5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50e6ffd640 1 -- 172.21.15.82:0/3880020357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c6038420 con 0x5595c5fb1a00 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50deffd640 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa8e80 0x5595c5fd6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50e6ffd640 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c6037ee0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f50d8001eb0 tx=0x7f50d8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e1bf7c9840d73e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50c6ffd640 1 -- 172.21.15.82:0/3880020357 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8005030 con 0x5595c5fb1a00 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.326+0000 7f50c6ffd640 1 -- 172.21.15.82:0/3880020357 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f50d8003a30 con 0x5595c5fb1a00 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50f05a2740 1 -- 172.21.15.82:0/3880020357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d0049030 con 0x5595c5fb1a00 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50c6ffd640 1 -- 172.21.15.82:0/3880020357 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f50d8009600 con 0x5595c5fb1a00 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50f05a2740 10 client.17517 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50f05a2740 1 -- 172.21.15.82:0/3880020357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5595c60395c0 con 0x5595c5fb1a00 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50c6ffd640 1 -- 172.21.15.82:0/3880020357 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d0049030 con 0x5595c5fb1a00 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50c6ffd640 1 -- 172.21.15.82:0/3880020357 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f50d803d060 con 0x5595c5fb1a00 2024-04-21T08:53:54.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50f05a2740 10 client.17517 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50f05a2740 10 client.17517 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50f05a2740 20 client.17517 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50f05a2740 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5595c60398f0 0x5595c603bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50f05a2740 4 client.17517 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50f05a2740 1 -- 172.21.15.82:0/3880020357 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595c5f1ac10 con 0x5595c60398f0 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.327+0000 7f50deffd640 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5595c60398f0 0x5595c603bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.328+0000 7f50c6ffd640 10 client.17517 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.328+0000 7f50deffd640 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5595c60398f0 0x5595c603bd10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37fc0ce52975e791 server_cookie=fa633cf3a8e44d81 in_seq=0 out_seq=0 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50c6ffd640 1 -- 172.21.15.82:0/3880020357 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595c5f1ac10 con 0x5595c60398f0 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50c6ffd640 10 client.17517 handle_command_reply: tid=0 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50f05a2740 1 client.17517 shutdown 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50f05a2740 1 -- 172.21.15.82:0/3880020357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 msgr2=0x5595c6037ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50f05a2740 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c6037ee0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f50d8001eb0 tx=0x7f50d8005fd0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50f05a2740 1 -- 172.21.15.82:0/3880020357 shutdown_connections 2024-04-21T08:53:54.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50f05a2740 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5595c60398f0 0x5595c603bd10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50f05a2740 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595c5fb1a00 0x5595c6037ee0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50f05a2740 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595c5fa8e80 0x5595c5fd6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50f05a2740 1 --2- 172.21.15.82:0/3880020357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595c5fa84d0 0x5595c5fd5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50c6ffd640 0 client.17517 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:53:54.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50f05a2740 1 -- 172.21.15.82:0/3880020357 >> 172.21.15.82:0/3880020357 conn(0x5595c5fa2ba0 msgr2=0x5595c5fcddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50f05a2740 1 -- 172.21.15.82:0/3880020357 shutdown_connections 2024-04-21T08:53:54.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50f05a2740 1 -- 172.21.15.82:0/3880020357 wait complete. 2024-04-21T08:53:54.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50f05a2740 20 client.17517 trim_cache size 0 max 16384 2024-04-21T08:53:54.334 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:53:54.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50ed8d2640 1 -- 172.21.15.82:0/1967306124 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f50d40a3e20 msgr2=0x7f50d40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50ed8d2640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f50d40a3e20 0x7f50d40a62e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50e00997e0 tx=0x7f50e0010000 comp rx=0 tx=0).stop 2024-04-21T08:53:54.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50ed8d2640 1 -- 172.21.15.82:0/1967306124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e81a4900 msgr2=0x7f50e81a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.329+0000 7f50ed8d2640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e81a4900 0x7f50e81a4db0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f50e000db20 tx=0x7f50e000b260 comp rx=0 tx=0).stop 2024-04-21T08:53:54.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.330+0000 7f50ed8d2640 1 -- 172.21.15.82:0/1967306124 shutdown_connections 2024-04-21T08:53:54.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.330+0000 7f50ed8d2640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e81a4900 0x7f50e81a4db0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.330+0000 7f50ed8d2640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50e8106150 0x7f50e810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.330+0000 7f50ed8d2640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f50d40a3e20 0x7f50d40a62e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.330+0000 7f50ed8d2640 1 --2- 172.21.15.82:0/1967306124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50e81015a0 0x7f50e810d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.330+0000 7f50ed8d2640 1 -- 172.21.15.82:0/1967306124 >> 172.21.15.82:0/1967306124 conn(0x7f50e80769d0 msgr2=0x7f50e8108280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.330+0000 7f50ed8d2640 1 -- 172.21.15.82:0/1967306124 shutdown_connections 2024-04-21T08:53:54.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.330+0000 7f50ed8d2640 1 -- 172.21.15.82:0/1967306124 wait complete. 2024-04-21T08:53:54.341 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:53:54.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:54 smithi082 ceph-mon[27967]: pgmap v209: 129 pgs: 129 active+clean; 889 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 845 op/s 2024-04-21T08:53:54.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:54 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/782741893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:53:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.445+0000 7f181777e640 1 -- 172.21.15.82:0/3584264105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f181006a820 msgr2=0x7f181006ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.445+0000 7f181777e640 1 --2- 172.21.15.82:0/3584264105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f181006a820 0x7f181006ac00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f180c009900 tx=0x7f180c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.445+0000 7f181777e640 1 -- 172.21.15.82:0/3584264105 shutdown_connections 2024-04-21T08:53:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.445+0000 7f181777e640 1 --2- 172.21.15.82:0/3584264105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1810107270 0x7f181010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.445+0000 7f181777e640 1 --2- 172.21.15.82:0/3584264105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f181006b1d0 0x7f1810106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.445+0000 7f181777e640 1 --2- 172.21.15.82:0/3584264105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f181006a820 0x7f181006ac00 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.445+0000 7f181777e640 1 -- 172.21.15.82:0/3584264105 >> 172.21.15.82:0/3584264105 conn(0x7f1810077eb0 msgr2=0x7f18100782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.445+0000 7f181777e640 1 -- 172.21.15.82:0/3584264105 shutdown_connections 2024-04-21T08:53:54.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.445+0000 7f181777e640 1 -- 172.21.15.82:0/3584264105 wait complete. 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.445+0000 7f181777e640 1 Processor -- start 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.445+0000 7f181777e640 1 -- start start 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181777e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f181006a820 0x7f18101986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181777e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1810107270 0x7f1810198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181777e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f181019cf90 0x7f181019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181777e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f18101121b0 con 0x7f181006a820 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181777e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1810112030 con 0x7f181019cf90 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181777e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1810112330 con 0x7f1810107270 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181551a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f181006a820 0x7f18101986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181551a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f181006a820 0x7f18101986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44584/0 (socket says 172.21.15.82:44584) 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181551a640 1 -- 172.21.15.82:0/1389594943 learned_addr learned my addr 172.21.15.82:0/1389594943 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f1815d1b640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f181019cf90 0x7f181019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f1814d19640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1810107270 0x7f1810198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181551a640 1 -- 172.21.15.82:0/1389594943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1810107270 msgr2=0x7f1810198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181551a640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1810107270 0x7f1810198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181551a640 1 -- 172.21.15.82:0/1389594943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f181019cf90 msgr2=0x7f181019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181551a640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f181019cf90 0x7f181019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181551a640 1 -- 172.21.15.82:0/1389594943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181019da80 con 0x7f181006a820 2024-04-21T08:53:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f1814d19640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1810107270 0x7f1810198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:53:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181551a640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f181006a820 0x7f18101986d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f180c005940 tx=0x7f180c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f18067fc640 1 -- 172.21.15.82:0/1389594943 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c043020 con 0x7f181006a820 2024-04-21T08:53:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181777e640 1 -- 172.21.15.82:0/1389594943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181019dd10 con 0x7f181006a820 2024-04-21T08:53:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f18067fc640 1 -- 172.21.15.82:0/1389594943 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f180c02cbe0 con 0x7f181006a820 2024-04-21T08:53:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f18067fc640 1 -- 172.21.15.82:0/1389594943 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c003bf0 con 0x7f181006a820 2024-04-21T08:53:54.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.446+0000 7f181777e640 1 -- 172.21.15.82:0/1389594943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18101a5530 con 0x7f181006a820 2024-04-21T08:53:54.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.448+0000 7f18067fc640 1 -- 172.21.15.82:0/1389594943 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f180c046020 con 0x7f181006a820 2024-04-21T08:53:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.449+0000 7f18067fc640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f17fc0a3d20 0x7f17fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.449+0000 7f181a36d740 1 Processor -- start 2024-04-21T08:53:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.449+0000 7f1814d19640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f17fc0a3d20 0x7f17fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.449+0000 7f18067fc640 1 -- 172.21.15.82:0/1389594943 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f180c0f2d20 con 0x7f181006a820 2024-04-21T08:53:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.449+0000 7f181a36d740 1 -- start start 2024-04-21T08:53:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.449+0000 7f181a36d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b36412f4d0 0x55b36412f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.449+0000 7f181a36d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412fe80 0x55b364138430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.449+0000 7f181a36d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b364138a00 0x55b36413ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.449+0000 7f181a36d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b3640941f0 con 0x55b36412fe80 2024-04-21T08:53:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.449+0000 7f181a36d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b3640314b0 con 0x55b36412f4d0 2024-04-21T08:53:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.449+0000 7f181a36d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b3640901b0 con 0x55b364138a00 2024-04-21T08:53:54.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f181551a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b36412f4d0 0x55b36412f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f1815d1b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b364138a00 0x55b36413ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f181551a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b36412f4d0 0x55b36412f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:58042/0 (socket says 172.21.15.82:58042) 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f181551a640 1 -- 172.21.15.82:0/43331430 learned_addr learned my addr 172.21.15.82:0/43331430 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f1814d19640 1 --2- 172.21.15.82:0/43331430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412fe80 0x55b364138430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f1814d19640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f17fc0a3d20 0x7f17fc0a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1800005d00 tx=0x7f1800009550 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f181551a640 1 -- 172.21.15.82:0/43331430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b364138a00 msgr2=0x55b36413ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f181551a640 1 --2- 172.21.15.82:0/43331430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b364138a00 0x55b36413ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f181551a640 1 -- 172.21.15.82:0/43331430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412fe80 msgr2=0x55b364138430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f181551a640 1 --2- 172.21.15.82:0/43331430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412fe80 0x55b364138430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f181551a640 1 -- 172.21.15.82:0/43331430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3640231b0 con 0x55b36412f4d0 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f181551a640 1 --2- 172.21.15.82:0/43331430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b36412f4d0 0x55b36412f8b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f180c0bae70 tx=0x7f180c0be910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5664b5e4826f2ac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f17fb7fe640 1 -- 172.21.15.82:0/43331430 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c0bea10 con 0x55b36412f4d0 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f17fb7fe640 1 -- 172.21.15.82:0/43331430 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f180c0c2dc0 con 0x55b36412f4d0 2024-04-21T08:53:54.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.450+0000 7f17fb7fe640 1 -- 172.21.15.82:0/43331430 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c0bb080 con 0x55b36412f4d0 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.451+0000 7f181a36d740 1 -- 172.21.15.82:0/43331430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b36412f4d0 msgr2=0x55b36412f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.451+0000 7f181a36d740 1 --2- 172.21.15.82:0/43331430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b36412f4d0 0x55b36412f8b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f180c0bae70 tx=0x7f180c0be910 comp rx=0 tx=0).stop 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.451+0000 7f181a36d740 1 -- 172.21.15.82:0/43331430 shutdown_connections 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.451+0000 7f181a36d740 1 --2- 172.21.15.82:0/43331430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b364138a00 0x55b36413ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.451+0000 7f181a36d740 1 --2- 172.21.15.82:0/43331430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412fe80 0x55b364138430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.451+0000 7f181a36d740 1 --2- 172.21.15.82:0/43331430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b36412f4d0 0x55b36412f8b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.451+0000 7f181a36d740 1 -- 172.21.15.82:0/43331430 >> 172.21.15.82:0/43331430 conn(0x55b364129ba0 msgr2=0x55b364129f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.451+0000 7f181a36d740 1 -- 172.21.15.82:0/43331430 shutdown_connections 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.451+0000 7f181a36d740 1 -- 172.21.15.82:0/43331430 wait complete. 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.451+0000 7f181a36d740 1 -- start start 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.451+0000 7f181a36d740 1 Processor -- start 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181a36d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b3641383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181a36d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b364135760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181a36d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364135e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181a36d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b36413b730 con 0x55b36412f4d0 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181a36d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b36404d500 con 0x55b364138a00 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181a36d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b36413b8b0 con 0x55b36412fe80 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181a36d740 10 client.0 _ll_register_callbacks cb 0x55b36401bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181551a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b3641383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181551a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b3641383d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44602/0 (socket says 172.21.15.82:44602) 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181551a640 1 -- 172.21.15.82:0/1267837002 learned_addr learned my addr 172.21.15.82:0/1267837002 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f1815d1b640 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364135e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f1814d19640 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b364135760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181551a640 1 -- 172.21.15.82:0/1267837002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 msgr2=0x55b364135760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181551a640 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b364135760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181551a640 1 -- 172.21.15.82:0/1267837002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 msgr2=0x55b364135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181551a640 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364135e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f181551a640 1 -- 172.21.15.82:0/1267837002 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b363da63f0 con 0x55b36412f4d0 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.452+0000 7f1815d1b640 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364135e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:54.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f181551a640 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b3641383d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f180c0c7150 tx=0x7f180c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c331280513b37927 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f17f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f17f8ff9640 1 -- 172.21.15.82:0/1267837002 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c0c7680 con 0x55b36412f4d0 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f17f8ff9640 1 -- 172.21.15.82:0/1267837002 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f180c0054d0 con 0x55b36412f4d0 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f181a36d740 1 -- 172.21.15.82:0/1267837002 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f180c0c2dc0 con 0x55b36412f4d0 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f17f8ff9640 1 -- 172.21.15.82:0/1267837002 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f180c0ddbd0 con 0x55b36412f4d0 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f181a36d740 10 client.17529 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f181a36d740 1 -- 172.21.15.82:0/1267837002 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b363e707d0 con 0x55b36412f4d0 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f17f8ff9640 1 -- 172.21.15.82:0/1267837002 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f180c0c2dc0 con 0x55b36412f4d0 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f17f8ff9640 1 -- 172.21.15.82:0/1267837002 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f180c089330 con 0x55b36412f4d0 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f181a36d740 10 client.17529 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f181a36d740 10 client.17529 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f181a36d740 20 client.17529 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f181a36d740 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55b364148810 0x55b36414ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f181a36d740 4 client.17529 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.453+0000 7f181a36d740 1 -- 172.21.15.82:0/1267837002 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b364033930 con 0x55b364148810 2024-04-21T08:53:54.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.454+0000 7f1814d19640 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55b364148810 0x55b36414ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.454+0000 7f17f8ff9640 10 client.17529 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.454+0000 7f1814d19640 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55b364148810 0x55b36414ac00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d516fffa5dcccea server_cookie=bfafe10568ffade7 in_seq=0 out_seq=0 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f17f8ff9640 1 -- 172.21.15.82:0/1267837002 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b364033930 con 0x55b364148810 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f17f8ff9640 10 client.17529 handle_command_reply: tid=0 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f181a36d740 1 client.17529 shutdown 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f181a36d740 1 -- 172.21.15.82:0/1267837002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 msgr2=0x55b3641383d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f181a36d740 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b3641383d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f180c0c7150 tx=0x7f180c0035f0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f181a36d740 1 -- 172.21.15.82:0/1267837002 shutdown_connections 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f181a36d740 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55b364148810 0x55b36414ac00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f181a36d740 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364135e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f181a36d740 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b364135760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f17f8ff9640 0 client.17529 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f181a36d740 1 --2- 172.21.15.82:0/1267837002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b3641383d0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f181a36d740 1 -- 172.21.15.82:0/1267837002 >> 172.21.15.82:0/1267837002 conn(0x55b364129ba0 msgr2=0x55b36412b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f181a36d740 1 -- 172.21.15.82:0/1267837002 shutdown_connections 2024-04-21T08:53:54.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f181a36d740 1 -- 172.21.15.82:0/1267837002 wait complete. 2024-04-21T08:53:54.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.455+0000 7f181a36d740 20 client.17529 trim_cache size 0 max 16384 2024-04-21T08:53:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.468+0000 7f181a36d740 1 Processor -- start 2024-04-21T08:53:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.468+0000 7f181a36d740 1 -- start start 2024-04-21T08:53:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.468+0000 7f181a36d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b3641501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.468+0000 7f181a36d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b3641506e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.468+0000 7f181a36d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364137e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.468+0000 7f181a36d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f18101af410 con 0x55b36412f4d0 2024-04-21T08:53:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.468+0000 7f181a36d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18101b1040 con 0x55b364138a00 2024-04-21T08:53:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.468+0000 7f181a36d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18101afae0 con 0x55b36412fe80 2024-04-21T08:53:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.468+0000 7f181551a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b3641501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f181551a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b3641501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44616/0 (socket says 172.21.15.82:44616) 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f181551a640 1 -- 172.21.15.82:0/3358934583 learned_addr learned my addr 172.21.15.82:0/3358934583 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f1814d19640 1 --2- 172.21.15.82:0/3358934583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b3641506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f1815d1b640 1 --2- 172.21.15.82:0/3358934583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f181551a640 1 -- 172.21.15.82:0/3358934583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 msgr2=0x55b3641506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f181551a640 1 --2- 172.21.15.82:0/3358934583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b3641506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f181551a640 1 -- 172.21.15.82:0/3358934583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 msgr2=0x55b364137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f181551a640 1 --2- 172.21.15.82:0/3358934583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364137e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f181551a640 1 -- 172.21.15.82:0/3358934583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b364129220 con 0x55b36412f4d0 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f1815d1b640 1 --2- 172.21.15.82:0/3358934583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364137e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f181551a640 1 --2- 172.21.15.82:0/3358934583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b3641501a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f180c0bb390 tx=0x7f180c003660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a26001a210ddcd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f17faffd640 1 -- 172.21.15.82:0/3358934583 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c0c7680 con 0x55b36412f4d0 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f17faffd640 1 -- 172.21.15.82:0/3358934583 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f180c0f0270 con 0x55b36412f4d0 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.469+0000 7f17faffd640 1 -- 172.21.15.82:0/3358934583 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c0054d0 con 0x55b36412f4d0 2024-04-21T08:53:54.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 -- 172.21.15.82:0/3358934583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 msgr2=0x55b3641501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 --2- 172.21.15.82:0/3358934583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b3641501a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f180c0bb390 tx=0x7f180c003660 comp rx=0 tx=0).stop 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 -- 172.21.15.82:0/3358934583 shutdown_connections 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 --2- 172.21.15.82:0/3358934583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364137e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 --2- 172.21.15.82:0/3358934583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b3641506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 --2- 172.21.15.82:0/3358934583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b3641501a0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 -- 172.21.15.82:0/3358934583 >> 172.21.15.82:0/3358934583 conn(0x55b364129ba0 msgr2=0x55b36412b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 -- 172.21.15.82:0/3358934583 shutdown_connections 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 -- 172.21.15.82:0/3358934583 wait complete. 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 -- start start 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 Processor -- start 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b364140880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b364140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364131a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f18101b0940 con 0x55b36412f4d0 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18101b0240 con 0x55b364138a00 2024-04-21T08:53:54.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18101ad3c0 con 0x55b36412fe80 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181a36d740 10 client.0 _ll_register_callbacks cb 0x7f1800009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.470+0000 7f181551a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b364140880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f181551a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b364140880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44632/0 (socket says 172.21.15.82:44632) 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f181551a640 1 -- 172.21.15.82:0/3238611578 learned_addr learned my addr 172.21.15.82:0/3238611578 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f1814d19640 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b364140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f1815d1b640 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364131a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f1814d19640 1 -- 172.21.15.82:0/3238611578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 msgr2=0x55b364131a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f1814d19640 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364131a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f1814d19640 1 -- 172.21.15.82:0/3238611578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 msgr2=0x55b364140880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f1814d19640 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b364140880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f1814d19640 1 -- 172.21.15.82:0/3238611578 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3641361f0 con 0x55b36412fe80 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f181551a640 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b364140880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f1814d19640 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b364140dc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1800009490 tx=0x7f1800004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2968d5a39da11989 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f17f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f17f8ff9640 1 -- 172.21.15.82:0/3238611578 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800011940 con 0x55b36412fe80 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f181a36d740 1 -- 172.21.15.82:0/3238611578 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b3640a1c10 con 0x55b36412fe80 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f17f8ff9640 1 -- 172.21.15.82:0/3238611578 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1800011ae0 con 0x55b36412fe80 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f17f8ff9640 1 -- 172.21.15.82:0/3238611578 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1800039050 con 0x55b36412fe80 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f181a36d740 10 client.26536 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.471+0000 7f181a36d740 1 -- 172.21.15.82:0/3238611578 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b3641ae150 con 0x55b36412fe80 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f17f8ff9640 1 -- 172.21.15.82:0/3238611578 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b3640a1c10 con 0x55b36412fe80 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f17f8ff9640 1 -- 172.21.15.82:0/3238611578 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f180000f040 con 0x55b36412fe80 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f181a36d740 10 client.26536 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f181a36d740 10 client.26536 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f181a36d740 20 client.26536 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f181a36d740 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55b3641ae900 0x55b3641b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f181a36d740 4 client.26536 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:53:54.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f181a36d740 1 -- 172.21.15.82:0/3238611578 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b36409ba80 con 0x55b3641ae900 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f181551a640 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55b3641ae900 0x55b3641b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f17f8ff9640 10 client.26536 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f181551a640 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55b3641ae900 0x55b3641b0d20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47a2ecbf143f7c4a server_cookie=a5e99c1d806cf067 in_seq=0 out_seq=0 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f17f8ff9640 1 -- 172.21.15.82:0/3238611578 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b36409ba80 con 0x55b3641ae900 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f17f8ff9640 10 client.26536 handle_command_reply: tid=0 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.472+0000 7f181a36d740 1 client.26536 shutdown 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181a36d740 1 -- 172.21.15.82:0/3238611578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 msgr2=0x55b364140dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181a36d740 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b364140dc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1800009490 tx=0x7f1800004730 comp rx=0 tx=0).stop 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181a36d740 1 -- 172.21.15.82:0/3238611578 shutdown_connections 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181a36d740 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55b3641ae900 0x55b3641b0d20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181a36d740 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b364138a00 0x55b364131a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181a36d740 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b36412fe80 0x55b364140dc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181a36d740 1 --2- 172.21.15.82:0/3238611578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b36412f4d0 0x55b364140880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f17f8ff9640 0 client.26536 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:53:54.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181a36d740 1 -- 172.21.15.82:0/3238611578 >> 172.21.15.82:0/3238611578 conn(0x55b364129ba0 msgr2=0x55b3641302b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181a36d740 1 -- 172.21.15.82:0/3238611578 shutdown_connections 2024-04-21T08:53:54.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181a36d740 1 -- 172.21.15.82:0/3238611578 wait complete. 2024-04-21T08:53:54.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181a36d740 20 client.26536 trim_cache size 0 max 16384 2024-04-21T08:53:54.480 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:53:54.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181777e640 1 -- 172.21.15.82:0/1389594943 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f17fc0a3d20 msgr2=0x7f17fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181777e640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f17fc0a3d20 0x7f17fc0a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1800005d00 tx=0x7f1800009550 comp rx=0 tx=0).stop 2024-04-21T08:53:54.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181777e640 1 -- 172.21.15.82:0/1389594943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f181006a820 msgr2=0x7f18101986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181777e640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f181006a820 0x7f18101986d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f180c005940 tx=0x7f180c02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181777e640 1 -- 172.21.15.82:0/1389594943 shutdown_connections 2024-04-21T08:53:54.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181777e640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f181019cf90 0x7f181019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181777e640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1810107270 0x7f1810198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181777e640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f17fc0a3d20 0x7f17fc0a61e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181777e640 1 --2- 172.21.15.82:0/1389594943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f181006a820 0x7f18101986d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181777e640 1 -- 172.21.15.82:0/1389594943 >> 172.21.15.82:0/1389594943 conn(0x7f1810077eb0 msgr2=0x7f1810108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181777e640 1 -- 172.21.15.82:0/1389594943 shutdown_connections 2024-04-21T08:53:54.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.473+0000 7f181777e640 1 -- 172.21.15.82:0/1389594943 wait complete. 2024-04-21T08:53:54.484 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:53:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:54 smithi106 ceph-mon[31072]: pgmap v209: 129 pgs: 129 active+clean; 889 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 845 op/s 2024-04-21T08:53:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:54 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/782741893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:53:54.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.630+0000 7fcf08b98640 1 -- 172.21.15.82:0/4004061854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf04106160 msgr2=0x7fcf0410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.630+0000 7fcf08b98640 1 --2- 172.21.15.82:0/4004061854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf04106160 0x7fcf0410dd30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009ca0 tx=0x7fcefc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.630+0000 7fcf08b98640 1 -- 172.21.15.82:0/4004061854 shutdown_connections 2024-04-21T08:53:54.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.630+0000 7fcf08b98640 1 --2- 172.21.15.82:0/4004061854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf04106160 0x7fcf0410dd30 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.630+0000 7fcf08b98640 1 --2- 172.21.15.82:0/4004061854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf04101a40 0x7fcf04105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.630+0000 7fcf08b98640 1 --2- 172.21.15.82:0/4004061854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf04101090 0x7fcf04101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.630+0000 7fcf08b98640 1 -- 172.21.15.82:0/4004061854 >> 172.21.15.82:0/4004061854 conn(0x7fcf040771d0 msgr2=0x7fcf040775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.630+0000 7fcf08b98640 1 -- 172.21.15.82:0/4004061854 shutdown_connections 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.630+0000 7fcf08b98640 1 -- 172.21.15.82:0/4004061854 wait complete. 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcf08b98640 1 Processor -- start 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcf08b98640 1 -- start start 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcf08b98640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf04101090 0x7fcf041986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcf08b98640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf04106160 0x7fcf04198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcf08b98640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf0419cf80 0x7fcf0419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcf08b98640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041121e0 con 0x7fcf0419cf80 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcf08b98640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcf04112060 con 0x7fcf04101090 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcf08b98640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf04112360 con 0x7fcf04106160 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcf02d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf0419cf80 0x7fcf0419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf04101090 0x7fcf041986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf04101090 0x7fcf041986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:58084/0 (socket says 172.21.15.82:58084) 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcf0259c640 1 -- 172.21.15.82:0/4276230166 learned_addr learned my addr 172.21.15.82:0/4276230166 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.631+0000 7fcefbfff640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf04106160 0x7fcf04198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.632+0000 7fcf02d9d640 1 -- 172.21.15.82:0/4276230166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf04106160 msgr2=0x7fcf04198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.632+0000 7fcf02d9d640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf04106160 0x7fcf04198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.632+0000 7fcf02d9d640 1 -- 172.21.15.82:0/4276230166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf04101090 msgr2=0x7fcf041986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.632+0000 7fcf02d9d640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf04101090 0x7fcf041986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.632+0000 7fcf02d9d640 1 -- 172.21.15.82:0/4276230166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0419da70 con 0x7fcf0419cf80 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.632+0000 7fcf02d9d640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf0419cf80 0x7fcf0419d430 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcefc02c700 tx=0x7fcefc0346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.632+0000 7fcefb7fe640 1 -- 172.21.15.82:0/4276230166 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc039040 con 0x7fcf0419cf80 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.632+0000 7fcf08b98640 1 -- 172.21.15.82:0/4276230166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0419dd00 con 0x7fcf0419cf80 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.632+0000 7fcefb7fe640 1 -- 172.21.15.82:0/4276230166 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcefc044070 con 0x7fcf0419cf80 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.632+0000 7fcefb7fe640 1 -- 172.21.15.82:0/4276230166 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0042a0 con 0x7fcf0419cf80 2024-04-21T08:53:54.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.632+0000 7fcf08b98640 1 -- 172.21.15.82:0/4276230166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf041a54c0 con 0x7fcf0419cf80 2024-04-21T08:53:54.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.634+0000 7fcefb7fe640 1 -- 172.21.15.82:0/4276230166 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcefc004440 con 0x7fcf0419cf80 2024-04-21T08:53:54.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.634+0000 7fcf0b782740 1 Processor -- start 2024-04-21T08:53:54.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0b782740 1 -- start start 2024-04-21T08:53:54.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0b782740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b14d0 0x559dfe1b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0b782740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1b1e80 0x559dfe1ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0b782740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1baa00 0x559dfe1bce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0b782740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559dfe1161f0 con 0x559dfe1baa00 2024-04-21T08:53:54.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0b782740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559dfe0b34b0 con 0x559dfe1b1e80 2024-04-21T08:53:54.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0b782740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559dfe1121b0 con 0x559dfe1b14d0 2024-04-21T08:53:54.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1b1e80 0x559dfe1ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcefb7fe640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcef00a3d20 0x7fcef00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcefbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1baa00 0x559dfe1bce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1b1e80 0x559dfe1ba430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:58100/0 (socket says 172.21.15.82:58100) 2024-04-21T08:53:54.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0259c640 1 -- 172.21.15.82:0/3312389116 learned_addr learned my addr 172.21.15.82:0/3312389116 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf02d9d640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcef00a3d20 0x7fcef00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcefb7fe640 1 -- 172.21.15.82:0/4276230166 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fcefc0f2c30 con 0x7fcf0419cf80 2024-04-21T08:53:54.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0259c640 1 -- 172.21.15.82:0/3312389116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b14d0 msgr2=0x559dfe1b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0259c640 1 --2- 172.21.15.82:0/3312389116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b14d0 0x559dfe1b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0259c640 1 -- 172.21.15.82:0/3312389116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1baa00 msgr2=0x559dfe1bce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0259c640 1 --2- 172.21.15.82:0/3312389116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1baa00 0x559dfe1bce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0259c640 1 -- 172.21.15.82:0/3312389116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dfe0a51b0 con 0x559dfe1b1e80 2024-04-21T08:53:54.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcefbfff640 1 --2- 172.21.15.82:0/3312389116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1baa00 0x559dfe1bce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:54.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf02d9d640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcef00a3d20 0x7fcef00a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0f05c0 tx=0x7fcefc0df040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.635+0000 7fcf0259c640 1 --2- 172.21.15.82:0/3312389116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1b1e80 0x559dfe1ba430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009dc0 tx=0x7fcef4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeccd7e2d9b357b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.636+0000 7fcef8ff9640 1 -- 172.21.15.82:0/3312389116 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4005220 con 0x559dfe1b1e80 2024-04-21T08:53:54.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.636+0000 7fcef8ff9640 1 -- 172.21.15.82:0/3312389116 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcef4004c50 con 0x559dfe1b1e80 2024-04-21T08:53:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.636+0000 7fcef8ff9640 1 -- 172.21.15.82:0/3312389116 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef40197a0 con 0x559dfe1b1e80 2024-04-21T08:53:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.636+0000 7fcf0b782740 1 -- 172.21.15.82:0/3312389116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1b1e80 msgr2=0x559dfe1ba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.636+0000 7fcf0b782740 1 --2- 172.21.15.82:0/3312389116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1b1e80 0x559dfe1ba430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009dc0 tx=0x7fcef4011810 comp rx=0 tx=0).stop 2024-04-21T08:53:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.636+0000 7fcf0b782740 1 -- 172.21.15.82:0/3312389116 shutdown_connections 2024-04-21T08:53:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.636+0000 7fcf0b782740 1 --2- 172.21.15.82:0/3312389116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1baa00 0x559dfe1bce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.636+0000 7fcf0b782740 1 --2- 172.21.15.82:0/3312389116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1b1e80 0x559dfe1ba430 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.636+0000 7fcf0b782740 1 --2- 172.21.15.82:0/3312389116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b14d0 0x559dfe1b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.636+0000 7fcf0b782740 1 -- 172.21.15.82:0/3312389116 >> 172.21.15.82:0/3312389116 conn(0x559dfe1abba0 msgr2=0x559dfe1abf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0b782740 1 -- 172.21.15.82:0/3312389116 shutdown_connections 2024-04-21T08:53:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0b782740 1 -- 172.21.15.82:0/3312389116 wait complete. 2024-04-21T08:53:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0b782740 1 -- start start 2024-04-21T08:53:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0b782740 1 Processor -- start 2024-04-21T08:53:54.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0b782740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b14d0 0x559dfe1ba360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0b782740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 0x559dfe1b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0b782740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1baa00 0x559dfe1b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0b782740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559dfe1bd970 con 0x559dfe1b1e80 2024-04-21T08:53:54.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0b782740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559dfe0cf500 con 0x559dfe1baa00 2024-04-21T08:53:54.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0b782740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559dfe1bdaf0 con 0x559dfe1b14d0 2024-04-21T08:53:54.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0b782740 10 client.0 _ll_register_callbacks cb 0x559dfe09db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:54.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 0x559dfe1b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 0x559dfe1b76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44654/0 (socket says 172.21.15.82:44654) 2024-04-21T08:53:54.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcf0259c640 1 -- 172.21.15.82:0/1232545602 learned_addr learned my addr 172.21.15.82:0/1232545602 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.637+0000 7fcefbfff640 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1baa00 0x559dfe1b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcefbfff640 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b14d0 0x559dfe1ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcf0259c640 1 -- 172.21.15.82:0/1232545602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b14d0 msgr2=0x559dfe1ba360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcf0259c640 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b14d0 0x559dfe1ba360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcf0259c640 1 -- 172.21.15.82:0/1232545602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1baa00 msgr2=0x559dfe1b7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcf0259c640 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1baa00 0x559dfe1b7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcf0259c640 1 -- 172.21.15.82:0/1232545602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dfde283f0 con 0x559dfe1b1e80 2024-04-21T08:53:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcf0259c640 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 0x559dfe1b76f0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fcef4000c00 tx=0x7fcef40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a143a1de3a2909 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcee9ffb640 1 -- 172.21.15.82:0/1232545602 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4027070 con 0x559dfe1b1e80 2024-04-21T08:53:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcee9ffb640 1 -- 172.21.15.82:0/1232545602 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fcef4005000 con 0x559dfe1b1e80 2024-04-21T08:53:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcf0b782740 1 -- 172.21.15.82:0/1232545602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef4004c50 con 0x559dfe1b1e80 2024-04-21T08:53:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcee9ffb640 1 -- 172.21.15.82:0/1232545602 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcef4033050 con 0x559dfe1b1e80 2024-04-21T08:53:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcf0b782740 10 client.17547 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcf0b782740 1 -- 172.21.15.82:0/1232545602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559dfdef27d0 con 0x559dfe1b1e80 2024-04-21T08:53:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcee9ffb640 1 -- 172.21.15.82:0/1232545602 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcef4004c50 con 0x559dfe1b1e80 2024-04-21T08:53:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.638+0000 7fcee9ffb640 1 -- 172.21.15.82:0/1232545602 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7fcef40194a0 con 0x559dfe1b1e80 2024-04-21T08:53:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.639+0000 7fcf0b782740 10 client.17547 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.639+0000 7fcf0b782740 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.639+0000 7fcf0b782740 20 client.17547 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.639+0000 7fcf0b782740 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559dfe1cab40 0x559dfe1ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.639+0000 7fcf0b782740 4 client.17547 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.639+0000 7fcf0b782740 1 -- 172.21.15.82:0/1232545602 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dfe0b5930 con 0x559dfe1cab40 2024-04-21T08:53:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.639+0000 7fcefbfff640 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559dfe1cab40 0x559dfe1ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.639+0000 7fcefbfff640 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559dfe1cab40 0x559dfe1ccf30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e211140116300550 server_cookie=1ebfbff182a67180 in_seq=0 out_seq=0 2024-04-21T08:53:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.639+0000 7fcee9ffb640 10 client.17547 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcee9ffb640 1 -- 172.21.15.82:0/1232545602 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559dfe0b5930 con 0x559dfe1cab40 2024-04-21T08:53:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcee9ffb640 10 client.17547 handle_command_reply: tid=0 2024-04-21T08:53:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcf0b782740 1 client.17547 shutdown 2024-04-21T08:53:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcf0b782740 1 -- 172.21.15.82:0/1232545602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 msgr2=0x559dfe1b76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcf0b782740 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 0x559dfe1b76f0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fcef4000c00 tx=0x7fcef40042f0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcf0b782740 1 -- 172.21.15.82:0/1232545602 shutdown_connections 2024-04-21T08:53:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcf0b782740 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559dfe1cab40 0x559dfe1ccf30 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcf0b782740 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1baa00 0x559dfe1b7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcee9ffb640 0 client.17547 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcf0b782740 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 0x559dfe1b76f0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcf0b782740 1 --2- 172.21.15.82:0/1232545602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b14d0 0x559dfe1ba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcf0b782740 1 -- 172.21.15.82:0/1232545602 >> 172.21.15.82:0/1232545602 conn(0x559dfe1abba0 msgr2=0x559dfe1ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcf0b782740 1 -- 172.21.15.82:0/1232545602 shutdown_connections 2024-04-21T08:53:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcf0b782740 1 -- 172.21.15.82:0/1232545602 wait complete. 2024-04-21T08:53:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.640+0000 7fcf0b782740 20 client.17547 trim_cache size 0 max 16384 2024-04-21T08:53:54.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.653+0000 7fcf0b782740 1 Processor -- start 2024-04-21T08:53:54.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.653+0000 7fcf0b782740 1 -- start start 2024-04-21T08:53:54.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcf0b782740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b14d0 0x559dfe1c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcf0b782740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b1e80 0x559dfe1c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcf0b782740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1baa00 0x559dfe1b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcf0b782740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041af3b0 con 0x559dfe1b14d0 2024-04-21T08:53:54.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcf0b782740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041b0fe0 con 0x559dfe1baa00 2024-04-21T08:53:54.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcf0b782740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041afa80 con 0x559dfe1b1e80 2024-04-21T08:53:54.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcefbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b14d0 0x559dfe1c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcefbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b14d0 0x559dfe1c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44666/0 (socket says 172.21.15.82:44666) 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcefbfff640 1 -- 172.21.15.82:0/2594311754 learned_addr learned my addr 172.21.15.82:0/2594311754 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcefbfff640 1 --2- 172.21.15.82:0/2594311754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1baa00 0x559dfe1b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcefbfff640 1 -- 172.21.15.82:0/2594311754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b1e80 msgr2=0x559dfe1c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcefbfff640 1 --2- 172.21.15.82:0/2594311754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b1e80 0x559dfe1c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcefbfff640 1 -- 172.21.15.82:0/2594311754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1baa00 msgr2=0x559dfe1b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcefbfff640 1 --2- 172.21.15.82:0/2594311754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1baa00 0x559dfe1b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcefbfff640 1 -- 172.21.15.82:0/2594311754 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dfe1bd1c0 con 0x559dfe1b14d0 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fcefbfff640 1 --2- 172.21.15.82:0/2594311754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b14d0 0x559dfe1c6830 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fceec004180 tx=0x7fceec00ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ee96994c0976b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fceebfff640 1 -- 172.21.15.82:0/2594311754 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec026020 con 0x559dfe1b14d0 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fceebfff640 1 -- 172.21.15.82:0/2594311754 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceec020800 con 0x559dfe1b14d0 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.654+0000 7fceebfff640 1 -- 172.21.15.82:0/2594311754 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec012b50 con 0x559dfe1b14d0 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 -- 172.21.15.82:0/2594311754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b14d0 msgr2=0x559dfe1c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 --2- 172.21.15.82:0/2594311754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b14d0 0x559dfe1c6830 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fceec004180 tx=0x7fceec00ba30 comp rx=0 tx=0).stop 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 -- 172.21.15.82:0/2594311754 shutdown_connections 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 --2- 172.21.15.82:0/2594311754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1baa00 0x559dfe1b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 --2- 172.21.15.82:0/2594311754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1b1e80 0x559dfe1c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 --2- 172.21.15.82:0/2594311754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b14d0 0x559dfe1c6830 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 -- 172.21.15.82:0/2594311754 >> 172.21.15.82:0/2594311754 conn(0x559dfe1abba0 msgr2=0x559dfe1b5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 -- 172.21.15.82:0/2594311754 shutdown_connections 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 -- 172.21.15.82:0/2594311754 wait complete. 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 -- start start 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 Processor -- start 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1b14d0 0x559dfe1b9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 0x559dfe1b9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.655+0000 7fcf0b782740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1baa00 0x559dfe1ba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0b782740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041b08e0 con 0x559dfe1b1e80 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0b782740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041b01e0 con 0x559dfe1b14d0 2024-04-21T08:53:54.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0b782740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041ad360 con 0x559dfe1baa00 2024-04-21T08:53:54.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0b782740 10 client.0 _ll_register_callbacks cb 0x7fcef4004a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:54.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 0x559dfe1b9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 0x559dfe1b9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44676/0 (socket says 172.21.15.82:44676) 2024-04-21T08:53:54.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcefbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1b14d0 0x559dfe1b9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0259c640 1 -- 172.21.15.82:0/3625527748 learned_addr learned my addr 172.21.15.82:0/3625527748 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:54.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcefbfff640 1 --2- 172.21.15.82:0/3625527748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1baa00 0x559dfe1ba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0259c640 1 -- 172.21.15.82:0/3625527748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1baa00 msgr2=0x559dfe1ba2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0259c640 1 --2- 172.21.15.82:0/3625527748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1baa00 0x559dfe1ba2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0259c640 1 -- 172.21.15.82:0/3625527748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1b14d0 msgr2=0x559dfe1b9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:54.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0259c640 1 --2- 172.21.15.82:0/3625527748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1b14d0 0x559dfe1b9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0259c640 1 -- 172.21.15.82:0/3625527748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dfe233f70 con 0x559dfe1b1e80 2024-04-21T08:53:54.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0259c640 1 --2- 172.21.15.82:0/3625527748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 0x559dfe1b9d60 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcef40059f0 tx=0x7fcef401af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e459484bb1e12af server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcee9ffb640 1 -- 172.21.15.82:0/3625527748 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef403c060 con 0x559dfe1b1e80 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcee9ffb640 1 -- 172.21.15.82:0/3625527748 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7fcef4005410 con 0x559dfe1b1e80 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0b782740 1 -- 172.21.15.82:0/3625527748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceec020800 con 0x559dfe1b1e80 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcee9ffb640 1 -- 172.21.15.82:0/3625527748 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcef4027050 con 0x559dfe1b1e80 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0b782740 10 client.17553 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcf0b782740 1 -- 172.21.15.82:0/3625527748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559dfe235230 con 0x559dfe1b1e80 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.656+0000 7fcee9ffb640 1 -- 172.21.15.82:0/3625527748 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceec020800 con 0x559dfe1b1e80 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcee9ffb640 1 -- 172.21.15.82:0/3625527748 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7fcef4019430 con 0x559dfe1b1e80 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcf0b782740 10 client.17553 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcf0b782740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcf0b782740 20 client.17553 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcf0b782740 1 --2- 172.21.15.82:0/3625527748 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559dfe235890 0x559dfe237cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:54.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcf0b782740 4 client.17553 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcf0b782740 1 -- 172.21.15.82:0/3625527748 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559dfe123c10 con 0x559dfe235890 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcefbfff640 1 --2- 172.21.15.82:0/3625527748 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559dfe235890 0x559dfe237cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcee9ffb640 10 client.17553 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcefbfff640 1 --2- 172.21.15.82:0/3625527748 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559dfe235890 0x559dfe237cb0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=431bba2916481649 server_cookie=85494655062e27d5 in_seq=0 out_seq=0 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcee9ffb640 1 -- 172.21.15.82:0/3625527748 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559dfe123c10 con 0x559dfe235890 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcee9ffb640 10 client.17553 handle_command_reply: tid=0 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcf0b782740 1 client.17553 shutdown 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcf0b782740 1 -- 172.21.15.82:0/3625527748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 msgr2=0x559dfe1b9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.657+0000 7fcf0b782740 1 --2- 172.21.15.82:0/3625527748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 0x559dfe1b9d60 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcef40059f0 tx=0x7fcef401af90 comp rx=0 tx=0).stop 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf0b782740 1 -- 172.21.15.82:0/3625527748 shutdown_connections 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf0b782740 1 --2- 172.21.15.82:0/3625527748 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559dfe235890 0x559dfe237cb0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf0b782740 1 --2- 172.21.15.82:0/3625527748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dfe1baa00 0x559dfe1ba2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf0b782740 1 --2- 172.21.15.82:0/3625527748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dfe1b1e80 0x559dfe1b9d60 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcee9ffb640 0 client.17553 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:54.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf0b782740 1 --2- 172.21.15.82:0/3625527748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559dfe1b14d0 0x559dfe1b9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf0b782740 1 -- 172.21.15.82:0/3625527748 >> 172.21.15.82:0/3625527748 conn(0x559dfe1abba0 msgr2=0x559dfe1b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf0b782740 1 -- 172.21.15.82:0/3625527748 shutdown_connections 2024-04-21T08:53:54.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf0b782740 1 -- 172.21.15.82:0/3625527748 wait complete. 2024-04-21T08:53:54.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf0b782740 20 client.17553 trim_cache size 0 max 16384 2024-04-21T08:53:54.665 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:53:54.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf08b98640 1 -- 172.21.15.82:0/4276230166 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcef00a3d20 msgr2=0x7fcef00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf08b98640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcef00a3d20 0x7fcef00a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0f05c0 tx=0x7fcefc0df040 comp rx=0 tx=0).stop 2024-04-21T08:53:54.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf08b98640 1 -- 172.21.15.82:0/4276230166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf0419cf80 msgr2=0x7fcf0419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:54.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf08b98640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf0419cf80 0x7fcf0419d430 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcefc02c700 tx=0x7fcefc0346f0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf08b98640 1 -- 172.21.15.82:0/4276230166 shutdown_connections 2024-04-21T08:53:54.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf08b98640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcef00a3d20 0x7fcef00a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf08b98640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf0419cf80 0x7fcf0419d430 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf08b98640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf04106160 0x7fcf04198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf08b98640 1 --2- 172.21.15.82:0/4276230166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf04101090 0x7fcf041986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:54.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf08b98640 1 -- 172.21.15.82:0/4276230166 >> 172.21.15.82:0/4276230166 conn(0x7fcf040771d0 msgr2=0x7fcf040fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:54.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf08b98640 1 -- 172.21.15.82:0/4276230166 shutdown_connections 2024-04-21T08:53:54.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:54.658+0000 7fcf08b98640 1 -- 172.21.15.82:0/4276230166 wait complete. 2024-04-21T08:53:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:54 smithi134 ceph-mon[31772]: pgmap v209: 129 pgs: 129 active+clean; 889 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 845 op/s 2024-04-21T08:53:54.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:54 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/782741893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:53:55.670 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:53:55.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:53:55.670 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:53:55.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.777+0000 7f663f45c640 1 -- 172.21.15.82:0/1905219854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6638102b40 msgr2=0x7f6638106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 --2- 172.21.15.82:0/1905219854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6638102b40 0x7f6638106b80 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f662c009900 tx=0x7f662c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 -- 172.21.15.82:0/1905219854 shutdown_connections 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 --2- 172.21.15.82:0/1905219854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6638107260 0x7f663810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 --2- 172.21.15.82:0/1905219854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6638102b40 0x7f6638106b80 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 --2- 172.21.15.82:0/1905219854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6638102190 0x7f6638102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 -- 172.21.15.82:0/1905219854 >> 172.21.15.82:0/1905219854 conn(0x7f66380793a0 msgr2=0x7f66380797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 -- 172.21.15.82:0/1905219854 shutdown_connections 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 -- 172.21.15.82:0/1905219854 wait complete. 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 Processor -- start 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 -- start start 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6638102190 0x7f6638198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6638107260 0x7f6638198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f663819cfc0 0x7f663819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6638112180 con 0x7f6638102190 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6638112000 con 0x7f663819cfc0 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663f45c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6638112300 con 0x7f6638107260 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.778+0000 7f663d1f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6638102190 0x7f6638198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f663d1f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6638102190 0x7f6638198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44690/0 (socket says 172.21.15.82:44690) 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f663d1f8640 1 -- 172.21.15.82:0/692317862 learned_addr learned my addr 172.21.15.82:0/692317862 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f663c9f7640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6638107260 0x7f6638198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f663d9f9640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f663819cfc0 0x7f663819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f663d1f8640 1 -- 172.21.15.82:0/692317862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6638107260 msgr2=0x7f6638198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f663d1f8640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6638107260 0x7f6638198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f663d1f8640 1 -- 172.21.15.82:0/692317862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f663819cfc0 msgr2=0x7f663819d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f663d1f8640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f663819cfc0 0x7f663819d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f663d1f8640 1 -- 172.21.15.82:0/692317862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663819dab0 con 0x7f6638102190 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f663d1f8640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6638102190 0x7f6638198700 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f663400d950 tx=0x7f663400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:55.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f662a7fc640 1 -- 172.21.15.82:0/692317862 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634017030 con 0x7f6638102190 2024-04-21T08:53:55.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f662a7fc640 1 -- 172.21.15.82:0/692317862 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6634004020 con 0x7f6638102190 2024-04-21T08:53:55.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f663f45c640 1 -- 172.21.15.82:0/692317862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663819dda0 con 0x7f6638102190 2024-04-21T08:53:55.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f662a7fc640 1 -- 172.21.15.82:0/692317862 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66340047a0 con 0x7f6638102190 2024-04-21T08:53:55.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.779+0000 7f663f45c640 1 -- 172.21.15.82:0/692317862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66381a5560 con 0x7f6638102190 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.782+0000 7f662a7fc640 1 -- 172.21.15.82:0/692317862 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6634009a30 con 0x7f6638102190 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.782+0000 7f6642053740 1 Processor -- start 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.782+0000 7f6642053740 1 -- start start 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.782+0000 7f6642053740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 0x56537bc4ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.782+0000 7f6642053740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4d290 0x56537bc55840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.782+0000 7f6642053740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc58260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.782+0000 7f6642053740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56537bbb0f00 con 0x56537bc4c8e0 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.782+0000 7f6642053740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56537bb4e590 con 0x56537bc55e10 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.782+0000 7f6642053740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56537bbacec0 con 0x56537bc4d290 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f663d9f9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 0x56537bc4ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f663d9f9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 0x56537bc4ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44702/0 (socket says 172.21.15.82:44702) 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f663d9f9640 1 -- 172.21.15.82:0/4193891230 learned_addr learned my addr 172.21.15.82:0/4193891230 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:55.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f663d1f8640 1 --2- 172.21.15.82:0/4193891230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4d290 0x56537bc55840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f663c9f7640 1 --2- 172.21.15.82:0/4193891230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc58260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f662a7fc640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66240a3d20 0x7f66240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f663d9f9640 1 -- 172.21.15.82:0/4193891230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4d290 msgr2=0x56537bc55840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f663d9f9640 1 --2- 172.21.15.82:0/4193891230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4d290 0x56537bc55840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f663d9f9640 1 -- 172.21.15.82:0/4193891230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 msgr2=0x56537bc58260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f663d9f9640 1 --2- 172.21.15.82:0/4193891230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc58260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f663d9f9640 1 -- 172.21.15.82:0/4193891230 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56537b98b7e0 con 0x56537bc4c8e0 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f662a7fc640 1 -- 172.21.15.82:0/692317862 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f66340d0590 con 0x7f6638102190 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.783+0000 7f663d9f9640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66240a3d20 0x7f66240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.784+0000 7f663d9f9640 1 --2- 172.21.15.82:0/4193891230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 0x56537bc4ccc0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f6630007ec0 tx=0x7f663000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965c7145c4b1f603 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.784+0000 7f66237fe640 1 -- 172.21.15.82:0/4193891230 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6630019b30 con 0x56537bc4c8e0 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.784+0000 7f66237fe640 1 -- 172.21.15.82:0/4193891230 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6630018830 con 0x56537bc4c8e0 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.784+0000 7f66237fe640 1 -- 172.21.15.82:0/4193891230 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66300236e0 con 0x56537bc4c8e0 2024-04-21T08:53:55.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.784+0000 7f663d9f9640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66240a3d20 0x7f66240a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6630020000 tx=0x7f6630021000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:55.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.784+0000 7f6642053740 1 -- 172.21.15.82:0/4193891230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 msgr2=0x56537bc4ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.784+0000 7f6642053740 1 --2- 172.21.15.82:0/4193891230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 0x56537bc4ccc0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f6630007ec0 tx=0x7f663000f790 comp rx=0 tx=0).stop 2024-04-21T08:53:55.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.784+0000 7f6642053740 1 -- 172.21.15.82:0/4193891230 shutdown_connections 2024-04-21T08:53:55.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.784+0000 7f6642053740 1 --2- 172.21.15.82:0/4193891230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc58260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.784+0000 7f6642053740 1 --2- 172.21.15.82:0/4193891230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4d290 0x56537bc55840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.784+0000 7f6642053740 1 --2- 172.21.15.82:0/4193891230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 0x56537bc4ccc0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f6642053740 1 -- 172.21.15.82:0/4193891230 >> 172.21.15.82:0/4193891230 conn(0x56537bc46fb0 msgr2=0x56537bc47390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:55.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f6642053740 1 -- 172.21.15.82:0/4193891230 shutdown_connections 2024-04-21T08:53:55.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f6642053740 1 -- 172.21.15.82:0/4193891230 wait complete. 2024-04-21T08:53:55.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f6642053740 1 -- start start 2024-04-21T08:53:55.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f6642053740 1 Processor -- start 2024-04-21T08:53:55.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f6642053740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 0x56537bc52de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f6642053740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4d290 0x56537bc53320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f6642053740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc53860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f6642053740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56537bc58d80 con 0x56537bc4c8e0 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f6642053740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56537bb6a580 con 0x56537bc55e10 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f6642053740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56537bc58f00 con 0x56537bc4d290 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f6642053740 10 client.0 _ll_register_callbacks cb 0x56537bb70650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f663d1f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 0x56537bc52de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f663d1f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 0x56537bc52de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44714/0 (socket says 172.21.15.82:44714) 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.785+0000 7f663d1f8640 1 -- 172.21.15.82:0/471405549 learned_addr learned my addr 172.21.15.82:0/471405549 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f663c9f7640 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4d290 0x56537bc53320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f663d9f9640 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc53860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f663c9f7640 1 -- 172.21.15.82:0/471405549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 msgr2=0x56537bc53860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f663c9f7640 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc53860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f663c9f7640 1 -- 172.21.15.82:0/471405549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 msgr2=0x56537bc52de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f663c9f7640 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 0x56537bc52de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f663c9f7640 1 -- 172.21.15.82:0/471405549 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537b9ebf70 con 0x56537bc4d290 2024-04-21T08:53:55.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f663d1f8640 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 0x56537bc52de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f663c9f7640 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4d290 0x56537bc53320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f662c02c170 tx=0x7f662c02ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58167b1bf7804b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f6620ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f6620ff9640 1 -- 172.21.15.82:0/471405549 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c02c610 con 0x56537bc4d290 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f6642053740 1 -- 172.21.15.82:0/471405549 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6630018830 con 0x56537bc4d290 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f6620ff9640 1 -- 172.21.15.82:0/471405549 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f662c005030 con 0x56537bc4d290 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f6620ff9640 1 -- 172.21.15.82:0/471405549 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f662c051050 con 0x56537bc4d290 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f6642053740 10 client.26557 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.786+0000 7f6642053740 1 -- 172.21.15.82:0/471405549 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56537b883170 con 0x56537bc4d290 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.787+0000 7f6620ff9640 1 -- 172.21.15.82:0/471405549 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6630018830 con 0x56537bc4d290 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.787+0000 7f6620ff9640 1 -- 172.21.15.82:0/471405549 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f662c0035f0 con 0x56537bc4d290 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.787+0000 7f6642053740 10 client.26557 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.787+0000 7f6642053740 10 client.26557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.787+0000 7f6642053740 20 client.26557 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.787+0000 7f6642053740 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56537bc662e0 0x56537bc68750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.787+0000 7f6642053740 4 client.26557 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.787+0000 7f6642053740 1 -- 172.21.15.82:0/471405549 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56537bbbe920 con 0x56537bc662e0 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.787+0000 7f663d1f8640 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56537bc662e0 0x56537bc68750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.787+0000 7f6620ff9640 10 client.26557 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.787+0000 7f663d1f8640 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56537bc662e0 0x56537bc68750 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f405927d63b9c0c server_cookie=50722ef12f083dc7 in_seq=0 out_seq=0 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6620ff9640 1 -- 172.21.15.82:0/471405549 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56537bbbe920 con 0x56537bc662e0 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6620ff9640 10 client.26557 handle_command_reply: tid=0 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6642053740 1 client.26557 shutdown 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6642053740 1 -- 172.21.15.82:0/471405549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4d290 msgr2=0x56537bc53320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6642053740 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4d290 0x56537bc53320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f662c02c170 tx=0x7f662c02ca10 comp rx=0 tx=0).stop 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6642053740 1 -- 172.21.15.82:0/471405549 shutdown_connections 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6642053740 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56537bc662e0 0x56537bc68750 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6642053740 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc53860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6642053740 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4d290 0x56537bc53320 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6642053740 1 --2- 172.21.15.82:0/471405549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4c8e0 0x56537bc52de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6620ff9640 0 client.26557 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:55.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6642053740 1 -- 172.21.15.82:0/471405549 >> 172.21.15.82:0/471405549 conn(0x56537bc46fb0 msgr2=0x56537bc48d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:55.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6642053740 1 -- 172.21.15.82:0/471405549 shutdown_connections 2024-04-21T08:53:55.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6642053740 1 -- 172.21.15.82:0/471405549 wait complete. 2024-04-21T08:53:55.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.788+0000 7f6642053740 20 client.26557 trim_cache size 0 max 16384 2024-04-21T08:53:55.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f6642053740 1 Processor -- start 2024-04-21T08:53:55.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f6642053740 1 -- start start 2024-04-21T08:53:55.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f6642053740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4c8e0 0x56537bc55510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f6642053740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 0x56537bc61c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f6642053740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc6e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f6642053740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f66381af340 con 0x56537bc4d290 2024-04-21T08:53:55.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f6642053740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66381b0f70 con 0x56537bc55e10 2024-04-21T08:53:55.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f6642053740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66381afa10 con 0x56537bc4c8e0 2024-04-21T08:53:55.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f663c9f7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 0x56537bc61c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f663c9f7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 0x56537bc61c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44716/0 (socket says 172.21.15.82:44716) 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f663c9f7640 1 -- 172.21.15.82:0/484173564 learned_addr learned my addr 172.21.15.82:0/484173564 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f663d9f9640 1 --2- 172.21.15.82:0/484173564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc6e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f663d1f8640 1 --2- 172.21.15.82:0/484173564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4c8e0 0x56537bc55510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f663c9f7640 1 -- 172.21.15.82:0/484173564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4c8e0 msgr2=0x56537bc55510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f663c9f7640 1 --2- 172.21.15.82:0/484173564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4c8e0 0x56537bc55510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f663c9f7640 1 -- 172.21.15.82:0/484173564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 msgr2=0x56537bc6e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f663c9f7640 1 --2- 172.21.15.82:0/484173564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc6e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f663c9f7640 1 -- 172.21.15.82:0/484173564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56537bc51db0 con 0x56537bc4d290 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f663d9f9640 1 --2- 172.21.15.82:0/484173564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc6e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.806+0000 7f663d1f8640 1 --2- 172.21.15.82:0/484173564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4c8e0 0x56537bc55510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f663c9f7640 1 --2- 172.21.15.82:0/484173564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 0x56537bc61c40 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f662c003a40 tx=0x7f662c0043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34655dfb04655499 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f6622ffd640 1 -- 172.21.15.82:0/484173564 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c05a070 con 0x56537bc4d290 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f6622ffd640 1 -- 172.21.15.82:0/484173564 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f662c003c30 con 0x56537bc4d290 2024-04-21T08:53:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f6622ffd640 1 -- 172.21.15.82:0/484173564 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c034dd0 con 0x56537bc4d290 2024-04-21T08:53:55.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f6642053740 1 -- 172.21.15.82:0/484173564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 msgr2=0x56537bc61c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f6642053740 1 --2- 172.21.15.82:0/484173564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 0x56537bc61c40 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f662c003a40 tx=0x7f662c0043e0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f6642053740 1 -- 172.21.15.82:0/484173564 shutdown_connections 2024-04-21T08:53:55.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f6642053740 1 --2- 172.21.15.82:0/484173564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc6e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f6642053740 1 --2- 172.21.15.82:0/484173564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 0x56537bc61c40 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f6642053740 1 --2- 172.21.15.82:0/484173564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4c8e0 0x56537bc55510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f6642053740 1 -- 172.21.15.82:0/484173564 >> 172.21.15.82:0/484173564 conn(0x56537bc46fb0 msgr2=0x56537bc53630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:55.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f6642053740 1 -- 172.21.15.82:0/484173564 shutdown_connections 2024-04-21T08:53:55.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.807+0000 7f6642053740 1 -- 172.21.15.82:0/484173564 wait complete. 2024-04-21T08:53:55.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f6642053740 1 -- start start 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f6642053740 1 Processor -- start 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f6642053740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4c8e0 0x56537bc5b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f6642053740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 0x56537bc5b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f6642053740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc5bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f6642053740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f66381b0870 con 0x56537bc4d290 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f6642053740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66381b0170 con 0x56537bc55e10 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f6642053740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66381ad2f0 con 0x56537bc4c8e0 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f6642053740 10 client.0 _ll_register_callbacks cb 0x7f662c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f663c9f7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 0x56537bc5b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f663c9f7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 0x56537bc5b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44718/0 (socket says 172.21.15.82:44718) 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f663c9f7640 1 -- 172.21.15.82:0/2564761256 learned_addr learned my addr 172.21.15.82:0/2564761256 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f663d9f9640 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc5bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.808+0000 7f663d1f8640 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4c8e0 0x56537bc5b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f663c9f7640 1 -- 172.21.15.82:0/2564761256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4c8e0 msgr2=0x56537bc5b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f663c9f7640 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4c8e0 0x56537bc5b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f663c9f7640 1 -- 172.21.15.82:0/2564761256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 msgr2=0x56537bc5bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f663c9f7640 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc5bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f663c9f7640 1 -- 172.21.15.82:0/2564761256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537bcd0d30 con 0x56537bc4d290 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f663c9f7640 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 0x56537bc5b660 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f662c006010 tx=0x7f662c004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909affb99dac37fe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6620ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6620ff9640 1 -- 172.21.15.82:0/2564761256 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c063060 con 0x56537bc4d290 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6642053740 1 -- 172.21.15.82:0/2564761256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662c003c30 con 0x56537bc4d290 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6620ff9640 1 -- 172.21.15.82:0/2564761256 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f662c034cd0 con 0x56537bc4d290 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6620ff9640 1 -- 172.21.15.82:0/2564761256 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f662c06a050 con 0x56537bc4d290 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6642053740 10 client.17571 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6642053740 1 -- 172.21.15.82:0/2564761256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56537bcd1fe0 con 0x56537bc4d290 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6620ff9640 1 -- 172.21.15.82:0/2564761256 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f662c003c30 con 0x56537bc4d290 2024-04-21T08:53:55.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6620ff9640 1 -- 172.21.15.82:0/2564761256 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f662c004bd0 con 0x56537bc4d290 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6642053740 10 client.17571 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6642053740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6642053740 20 client.17571 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6642053740 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56537bcd2640 0x56537bcd4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6642053740 4 client.17571 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.809+0000 7f6642053740 1 -- 172.21.15.82:0/2564761256 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56537bbb8790 con 0x56537bcd2640 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.810+0000 7f663d1f8640 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56537bcd2640 0x56537bcd4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.810+0000 7f663d1f8640 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56537bcd2640 0x56537bcd4a60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee977453f8526327 server_cookie=54d323100ffab03b in_seq=0 out_seq=0 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.810+0000 7f6620ff9640 10 client.17571 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6620ff9640 1 -- 172.21.15.82:0/2564761256 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56537bbb8790 con 0x56537bcd2640 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6620ff9640 10 client.17571 handle_command_reply: tid=0 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6642053740 1 client.17571 shutdown 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6642053740 1 -- 172.21.15.82:0/2564761256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 msgr2=0x56537bc5b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6642053740 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 0x56537bc5b660 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f662c006010 tx=0x7f662c004520 comp rx=0 tx=0).stop 2024-04-21T08:53:55.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6642053740 1 -- 172.21.15.82:0/2564761256 shutdown_connections 2024-04-21T08:53:55.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6642053740 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56537bcd2640 0x56537bcd4a60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6642053740 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56537bc55e10 0x56537bc5bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6642053740 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56537bc4d290 0x56537bc5b660 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6620ff9640 0 client.17571 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:55.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6642053740 1 --2- 172.21.15.82:0/2564761256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56537bc4c8e0 0x56537bc5b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6642053740 1 -- 172.21.15.82:0/2564761256 >> 172.21.15.82:0/2564761256 conn(0x56537bc46fb0 msgr2=0x56537bc52d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:55.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6642053740 1 -- 172.21.15.82:0/2564761256 shutdown_connections 2024-04-21T08:53:55.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6642053740 1 -- 172.21.15.82:0/2564761256 wait complete. 2024-04-21T08:53:55.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f6642053740 20 client.17571 trim_cache size 0 max 16384 2024-04-21T08:53:55.819 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:53:55.819 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:53:55.819 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "277adcaf-6bde-442b-a789-7fe7f9670fe1", 2024-04-21T08:53:55.819 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:53:55.819 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:53:55.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f663f45c640 1 -- 172.21.15.82:0/692317862 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66240a3d20 msgr2=0x7f66240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f663f45c640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66240a3d20 0x7f66240a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6630020000 tx=0x7f6630021000 comp rx=0 tx=0).stop 2024-04-21T08:53:55.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f663f45c640 1 -- 172.21.15.82:0/692317862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6638102190 msgr2=0x7f6638198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.811+0000 7f663f45c640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6638102190 0x7f6638198700 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f663400d950 tx=0x7f663400de10 comp rx=0 tx=0).stop 2024-04-21T08:53:55.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.812+0000 7f663f45c640 1 -- 172.21.15.82:0/692317862 shutdown_connections 2024-04-21T08:53:55.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.812+0000 7f663f45c640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f663819cfc0 0x7f663819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.812+0000 7f663f45c640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6638107260 0x7f6638198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.812+0000 7f663f45c640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f66240a3d20 0x7f66240a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.812+0000 7f663f45c640 1 --2- 172.21.15.82:0/692317862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6638102190 0x7f6638198700 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.812+0000 7f663f45c640 1 -- 172.21.15.82:0/692317862 >> 172.21.15.82:0/692317862 conn(0x7f66380793a0 msgr2=0x7f66380fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:55.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.812+0000 7f663f45c640 1 -- 172.21.15.82:0/692317862 shutdown_connections 2024-04-21T08:53:55.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.812+0000 7f663f45c640 1 -- 172.21.15.82:0/692317862 wait complete. 2024-04-21T08:53:55.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:53:55.823 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:53:55.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.927+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1835801806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bf4101a30 msgr2=0x7f1bf4105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.927+0000 7f1bfb0e0640 1 --2- 172.21.15.82:0/1835801806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bf4101a30 0x7f1bf4105a70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009900 tx=0x7f1be002bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:55.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1835801806 shutdown_connections 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 --2- 172.21.15.82:0/1835801806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4106150 0x7f1bf410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 --2- 172.21.15.82:0/1835801806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bf4101a30 0x7f1bf4105a70 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 --2- 172.21.15.82:0/1835801806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bf4101080 0x7f1bf4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1835801806 >> 172.21.15.82:0/1835801806 conn(0x7f1bf4078000 msgr2=0x7f1bf4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1835801806 shutdown_connections 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1835801806 wait complete. 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 Processor -- start 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 -- start start 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bf4101080 0x7f1bf41995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4106150 0x7f1bf4199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bf419a080 0x7f1bf41a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4111090 con 0x7f1bf419a080 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4110f10 con 0x7f1bf4101080 2024-04-21T08:53:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bfb0e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4111210 con 0x7f1bf4106150 2024-04-21T08:53:55.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.928+0000 7f1bf967d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bf419a080 0x7f1bf41a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf8e7c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bf4101080 0x7f1bf41995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf967d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bf419a080 0x7f1bf41a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44732/0 (socket says 172.21.15.82:44732) 2024-04-21T08:53:55.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf8e7c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bf4101080 0x7f1bf41995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:58176/0 (socket says 172.21.15.82:58176) 2024-04-21T08:53:55.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf967d640 1 -- 172.21.15.82:0/1187130385 learned_addr learned my addr 172.21.15.82:0/1187130385 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:55.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf3fff640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4106150 0x7f1bf4199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf967d640 1 -- 172.21.15.82:0/1187130385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4106150 msgr2=0x7f1bf4199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf967d640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4106150 0x7f1bf4199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf967d640 1 -- 172.21.15.82:0/1187130385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bf4101080 msgr2=0x7f1bf41995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf967d640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bf4101080 0x7f1bf41995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf967d640 1 -- 172.21.15.82:0/1187130385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf41a4e30 con 0x7f1bf419a080 2024-04-21T08:53:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf967d640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bf419a080 0x7f1bf41a4630 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00eaa0 tx=0x7f1bec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf1ffb640 1 -- 172.21.15.82:0/1187130385 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec00cdb0 con 0x7f1bf419a080 2024-04-21T08:53:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf1ffb640 1 -- 172.21.15.82:0/1187130385 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bec005020 con 0x7f1bf419a080 2024-04-21T08:53:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1187130385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf41a5120 con 0x7f1bf419a080 2024-04-21T08:53:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bf1ffb640 1 -- 172.21.15.82:0/1187130385 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec0116d0 con 0x7f1bf419a080 2024-04-21T08:53:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.929+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1187130385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf41a5690 con 0x7f1bf419a080 2024-04-21T08:53:55.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.931+0000 7f1bf1ffb640 1 -- 172.21.15.82:0/1187130385 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1bec0051d0 con 0x7f1bf419a080 2024-04-21T08:53:55.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bfddab740 1 Processor -- start 2024-04-21T08:53:55.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bfddab740 1 -- start start 2024-04-21T08:53:55.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bf1ffb640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1be40a3da0 0x7f1be40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bfddab740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286e72b0 0x5573286e7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bfddab740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286f4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bfddab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286f4e90 0x5573286f7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bfddab740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55732864c1e0 con 0x5573286e7bd0 2024-04-21T08:53:55.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bfddab740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5573285e9390 con 0x5573286e72b0 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bfddab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5573286481a0 con 0x5573286f4e90 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bf8e7c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286e72b0 0x5573286e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bf8e7c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286e72b0 0x5573286e7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:58178/0 (socket says 172.21.15.82:58178) 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bf967d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286f4e90 0x5573286f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bf3fff640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1be40a3da0 0x7f1be40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bf8e7c640 1 -- 172.21.15.82:0/3826593393 learned_addr learned my addr 172.21.15.82:0/3826593393 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bf1ffb640 1 -- 172.21.15.82:0/1187130385 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1bec0d0bd0 con 0x7f1bf419a080 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.932+0000 7f1bf3fff640 1 --2- 172.21.15.82:0/3826593393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.933+0000 7f1bf8e7c640 1 -- 172.21.15.82:0/3826593393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286f4e90 msgr2=0x5573286f7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.933+0000 7f1bf8e7c640 1 --2- 172.21.15.82:0/3826593393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286f4e90 0x5573286f7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.933+0000 7f1bf8e7c640 1 -- 172.21.15.82:0/3826593393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 msgr2=0x5573286f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.933+0000 7f1bf8e7c640 1 --2- 172.21.15.82:0/3826593393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.933+0000 7f1bf8e7c640 1 -- 172.21.15.82:0/3826593393 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573284287d0 con 0x5573286e72b0 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.933+0000 7f1bf3fff640 1 --2- 172.21.15.82:0/3826593393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.933+0000 7f1bf3fff640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1be40a3da0 0x7f1be40a6260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009230 tx=0x7f1be0005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.933+0000 7f1bf8e7c640 1 --2- 172.21.15.82:0/3826593393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286e72b0 0x5573286e7690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009cf0 tx=0x7f1be8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a70f61cb2c0bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.933+0000 7f1bdeffd640 1 -- 172.21.15.82:0/3826593393 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8004230 con 0x5573286e72b0 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.933+0000 7f1bdeffd640 1 -- 172.21.15.82:0/3826593393 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1be80043d0 con 0x5573286e72b0 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.933+0000 7f1bdeffd640 1 -- 172.21.15.82:0/3826593393 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8019880 con 0x5573286e72b0 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 -- 172.21.15.82:0/3826593393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286e72b0 msgr2=0x5573286e7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 --2- 172.21.15.82:0/3826593393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286e72b0 0x5573286e7690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009cf0 tx=0x7f1be8011810 comp rx=0 tx=0).stop 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 -- 172.21.15.82:0/3826593393 shutdown_connections 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 --2- 172.21.15.82:0/3826593393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286f4e90 0x5573286f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 --2- 172.21.15.82:0/3826593393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 --2- 172.21.15.82:0/3826593393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286e72b0 0x5573286e7690 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 -- 172.21.15.82:0/3826593393 >> 172.21.15.82:0/3826593393 conn(0x5573286e1cb0 msgr2=0x5573286e2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 -- 172.21.15.82:0/3826593393 shutdown_connections 2024-04-21T08:53:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 -- 172.21.15.82:0/3826593393 wait complete. 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 -- start start 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 Processor -- start 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286e72b0 0x5573286f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.934+0000 7f1bfddab740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286f4e90 0x5573286f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bfddab740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5573286f7c50 con 0x5573286e7bd0 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bfddab740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5573286052f0 con 0x5573286f4e90 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bfddab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5573286f7dd0 con 0x5573286e72b0 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bfddab740 10 client.0 _ll_register_callbacks cb 0x557328612580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44750/0 (socket says 172.21.15.82:44750) 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bf3fff640 1 -- 172.21.15.82:0/629364271 learned_addr learned my addr 172.21.15.82:0/629364271 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bf967d640 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286f4e90 0x5573286f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bf8e7c640 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286e72b0 0x5573286f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bf3fff640 1 -- 172.21.15.82:0/629364271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286e72b0 msgr2=0x5573286f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bf3fff640 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286e72b0 0x5573286f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bf3fff640 1 -- 172.21.15.82:0/629364271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286f4e90 msgr2=0x5573286f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bf3fff640 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286f4e90 0x5573286f28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bf3fff640 1 -- 172.21.15.82:0/629364271 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55732835e3f0 con 0x5573286e7bd0 2024-04-21T08:53:55.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bf967d640 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286f4e90 0x5573286f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:55.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bf3fff640 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286f23b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1be00058d0 tx=0x7f1be002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7044a735156a2fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:55.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:55.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bd3fff640 1 -- 172.21.15.82:0/629364271 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0052020 con 0x5573286e7bd0 2024-04-21T08:53:55.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bfddab740 1 -- 172.21.15.82:0/629364271 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1be80043d0 con 0x5573286e7bd0 2024-04-21T08:53:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bd3fff640 1 -- 172.21.15.82:0/629364271 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1be00048d0 con 0x5573286e7bd0 2024-04-21T08:53:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bd3fff640 1 -- 172.21.15.82:0/629364271 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1be005d050 con 0x5573286e7bd0 2024-04-21T08:53:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bfddab740 10 client.17583 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.935+0000 7f1bfddab740 1 -- 172.21.15.82:0/629364271 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55732831e0f0 con 0x5573286e7bd0 2024-04-21T08:53:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.936+0000 7f1bd3fff640 1 -- 172.21.15.82:0/629364271 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1be80043d0 con 0x5573286e7bd0 2024-04-21T08:53:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.936+0000 7f1bd3fff640 1 -- 172.21.15.82:0/629364271 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f1be0003bc0 con 0x5573286e7bd0 2024-04-21T08:53:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.936+0000 7f1bfddab740 10 client.17583 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.936+0000 7f1bfddab740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.936+0000 7f1bfddab740 20 client.17583 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.936+0000 7f1bfddab740 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557328704ee0 0x557328707300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.936+0000 7f1bfddab740 4 client.17583 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.936+0000 7f1bfddab740 1 -- 172.21.15.82:0/629364271 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557328659c00 con 0x557328704ee0 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.936+0000 7f1bf8e7c640 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557328704ee0 0x557328707300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.936+0000 7f1bd3fff640 10 client.17583 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.936+0000 7f1bf8e7c640 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557328704ee0 0x557328707300 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9fa3b5eceb9926b server_cookie=d47ee4f0dbd8eb1f in_seq=0 out_seq=0 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bd3fff640 1 -- 172.21.15.82:0/629364271 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557328659c00 con 0x557328704ee0 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bd3fff640 10 client.17583 handle_command_reply: tid=0 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bfddab740 1 client.17583 shutdown 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bfddab740 1 -- 172.21.15.82:0/629364271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 msgr2=0x5573286f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bfddab740 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286f23b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1be00058d0 tx=0x7f1be002bc30 comp rx=0 tx=0).stop 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bfddab740 1 -- 172.21.15.82:0/629364271 shutdown_connections 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bfddab740 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557328704ee0 0x557328707300 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bfddab740 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286f4e90 0x5573286f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bfddab740 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286f23b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bd3fff640 0 client.17583 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:55.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bfddab740 1 --2- 172.21.15.82:0/629364271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286e72b0 0x5573286f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bfddab740 1 -- 172.21.15.82:0/629364271 >> 172.21.15.82:0/629364271 conn(0x5573286e1cb0 msgr2=0x5573286e3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:55.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bfddab740 1 -- 172.21.15.82:0/629364271 shutdown_connections 2024-04-21T08:53:55.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bfddab740 1 -- 172.21.15.82:0/629364271 wait complete. 2024-04-21T08:53:55.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.937+0000 7f1bfddab740 20 client.17583 trim_cache size 0 max 16384 2024-04-21T08:53:55.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.955+0000 7f1bfddab740 1 Processor -- start 2024-04-21T08:53:55.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.955+0000 7f1bfddab740 1 -- start start 2024-04-21T08:53:55.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.955+0000 7f1bfddab740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e72b0 0x5573286f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.955+0000 7f1bfddab740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286e7bd0 0x557328700b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.955+0000 7f1bfddab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286f4e90 0x5573286f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.955+0000 7f1bfddab740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41ad2a0 con 0x5573286e72b0 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.955+0000 7f1bfddab740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41aeed0 con 0x5573286e7bd0 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.955+0000 7f1bfddab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41ad970 con 0x5573286f4e90 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.955+0000 7f1bf8e7c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e72b0 0x5573286f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.955+0000 7f1bf8e7c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e72b0 0x5573286f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44752/0 (socket says 172.21.15.82:44752) 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.955+0000 7f1bf8e7c640 1 -- 172.21.15.82:0/977832337 learned_addr learned my addr 172.21.15.82:0/977832337 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.955+0000 7f1bf3fff640 1 --2- 172.21.15.82:0/977832337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286e7bd0 0x557328700b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.956+0000 7f1bf8e7c640 1 -- 172.21.15.82:0/977832337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286f4e90 msgr2=0x5573286f0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.956+0000 7f1bf8e7c640 1 --2- 172.21.15.82:0/977832337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286f4e90 0x5573286f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.956+0000 7f1bf8e7c640 1 -- 172.21.15.82:0/977832337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286e7bd0 msgr2=0x557328700b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.956+0000 7f1bf8e7c640 1 --2- 172.21.15.82:0/977832337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286e7bd0 0x557328700b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.956+0000 7f1bf8e7c640 1 -- 172.21.15.82:0/977832337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573286e82d0 con 0x5573286e72b0 2024-04-21T08:53:55.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.956+0000 7f1bf8e7c640 1 --2- 172.21.15.82:0/977832337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e72b0 0x5573286f45a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1be800a010 tx=0x7f1be8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508e748eaed3f5eb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.956+0000 7f1bde7fc640 1 -- 172.21.15.82:0/977832337 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be802f020 con 0x5573286e72b0 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.956+0000 7f1bde7fc640 1 -- 172.21.15.82:0/977832337 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1be802bd50 con 0x5573286e72b0 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.956+0000 7f1bde7fc640 1 -- 172.21.15.82:0/977832337 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be802b820 con 0x5573286e72b0 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 -- 172.21.15.82:0/977832337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e72b0 msgr2=0x5573286f45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 --2- 172.21.15.82:0/977832337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e72b0 0x5573286f45a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1be800a010 tx=0x7f1be8011810 comp rx=0 tx=0).stop 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 -- 172.21.15.82:0/977832337 shutdown_connections 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 --2- 172.21.15.82:0/977832337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286f4e90 0x5573286f0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 --2- 172.21.15.82:0/977832337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286e7bd0 0x557328700b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 --2- 172.21.15.82:0/977832337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e72b0 0x5573286f45a0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 -- 172.21.15.82:0/977832337 >> 172.21.15.82:0/977832337 conn(0x5573286e1cb0 msgr2=0x5573286e3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 -- 172.21.15.82:0/977832337 shutdown_connections 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 -- 172.21.15.82:0/977832337 wait complete. 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 -- start start 2024-04-21T08:53:55.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 Processor -- start 2024-04-21T08:53:55.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286e72b0 0x5573286eef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286f4e90 0x55732876a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41ae7d0 con 0x5573286e7bd0 2024-04-21T08:53:55.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41ae0d0 con 0x5573286f4e90 2024-04-21T08:53:55.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41ab250 con 0x5573286e72b0 2024-04-21T08:53:55.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bfddab740 10 client.0 _ll_register_callbacks cb 0x7f1be0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:55.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bf3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286ef4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44756/0 (socket says 172.21.15.82:44756) 2024-04-21T08:53:55.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.957+0000 7f1bf3fff640 1 -- 172.21.15.82:0/587510084 learned_addr learned my addr 172.21.15.82:0/587510084 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:55.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bf967d640 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286f4e90 0x55732876a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bf8e7c640 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286e72b0 0x5573286eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bf3fff640 1 -- 172.21.15.82:0/587510084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286e72b0 msgr2=0x5573286eef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bf3fff640 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286e72b0 0x5573286eef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bf3fff640 1 -- 172.21.15.82:0/587510084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286f4e90 msgr2=0x55732876a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:55.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bf3fff640 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286f4e90 0x55732876a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bf3fff640 1 -- 172.21.15.82:0/587510084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573286efd80 con 0x5573286e7bd0 2024-04-21T08:53:55.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bf3fff640 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286ef4d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x5573286f16d0 tx=0x7f1be00056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679ce026ba0e3127 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:55.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:55.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bd3fff640 1 -- 172.21.15.82:0/587510084 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0067070 con 0x5573286e7bd0 2024-04-21T08:53:55.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bd3fff640 1 -- 172.21.15.82:0/587510084 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1be0004ce0 con 0x5573286e7bd0 2024-04-21T08:53:55.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bfddab740 1 -- 172.21.15.82:0/587510084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1be802bd50 con 0x5573286e7bd0 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bd3fff640 1 -- 172.21.15.82:0/587510084 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1be006f050 con 0x5573286e7bd0 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bfddab740 10 client.17589 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bfddab740 1 -- 172.21.15.82:0/587510084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55732876ba70 con 0x5573286e7bd0 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.958+0000 7f1bd3fff640 1 -- 172.21.15.82:0/587510084 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1be802bd50 con 0x5573286e7bd0 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bd3fff640 1 -- 172.21.15.82:0/587510084 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f1be006a030 con 0x5573286e7bd0 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 10 client.17589 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 10 client.17589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 20 client.17589 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55732876c1f0 0x55732876e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 4 client.17589 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 1 -- 172.21.15.82:0/587510084 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557328653a70 con 0x55732876c1f0 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bf8e7c640 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55732876c1f0 0x55732876e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bf8e7c640 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55732876c1f0 0x55732876e610 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3bd45664f675da server_cookie=edcf749666a47d13 in_seq=0 out_seq=0 2024-04-21T08:53:55.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bd3fff640 10 client.17589 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bd3fff640 1 -- 172.21.15.82:0/587510084 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557328653a70 con 0x55732876c1f0 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bd3fff640 10 client.17589 handle_command_reply: tid=0 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 1 client.17589 shutdown 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 1 -- 172.21.15.82:0/587510084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 msgr2=0x5573286ef4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286ef4d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x5573286f16d0 tx=0x7f1be00056d0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 1 -- 172.21.15.82:0/587510084 shutdown_connections 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55732876c1f0 0x55732876e610 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5573286f4e90 0x55732876a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5573286e7bd0 0x5573286ef4d0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bd3fff640 0 client.17589 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 1 --2- 172.21.15.82:0/587510084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5573286e72b0 0x5573286eef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.959+0000 7f1bfddab740 1 -- 172.21.15.82:0/587510084 >> 172.21.15.82:0/587510084 conn(0x5573286e1cb0 msgr2=0x5573286e04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfddab740 1 -- 172.21.15.82:0/587510084 shutdown_connections 2024-04-21T08:53:55.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfddab740 1 -- 172.21.15.82:0/587510084 wait complete. 2024-04-21T08:53:55.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfddab740 20 client.17589 trim_cache size 0 max 16384 2024-04-21T08:53:55.967 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:53:55.967 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:53:55.967 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:53:55.967 INFO:teuthology.orchestra.run.smithi082.stdout: "277adcaf-6bde-442b-a789-7fe7f9670fe1": { 2024-04-21T08:53:55.967 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:53:55.967 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "277adcaf-6bde-442b-a789-7fe7f9670fe1", 2024-04-21T08:53:55.967 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:53:55.967 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:53:55.967 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:53:55.968 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:53:55.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1187130385 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1be40a3da0 msgr2=0x7f1be40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfb0e0640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1be40a3da0 0x7f1be40a6260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009230 tx=0x7f1be0005e40 comp rx=0 tx=0).stop 2024-04-21T08:53:55.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1187130385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bf419a080 msgr2=0x7f1bf41a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:55.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfb0e0640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bf419a080 0x7f1bf41a4630 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00eaa0 tx=0x7f1bec00ef60 comp rx=0 tx=0).stop 2024-04-21T08:53:55.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1187130385 shutdown_connections 2024-04-21T08:53:55.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfb0e0640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1be40a3da0 0x7f1be40a6260 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfb0e0640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bf419a080 0x7f1bf41a4630 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfb0e0640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1bf4106150 0x7f1bf4199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfb0e0640 1 --2- 172.21.15.82:0/1187130385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bf4101080 0x7f1bf41995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:55.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1187130385 >> 172.21.15.82:0/1187130385 conn(0x7f1bf4078000 msgr2=0x7f1bf40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:55.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1187130385 shutdown_connections 2024-04-21T08:53:55.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:53:55.960+0000 7f1bfb0e0640 1 -- 172.21.15.82:0/1187130385 wait complete. 2024-04-21T08:53:55.971 INFO:tasks.cephfs.filesystem:scrub status for tag:277adcaf-6bde-442b-a789-7fe7f9670fe1 - {'path': '/', 'tag': '277adcaf-6bde-442b-a789-7fe7f9670fe1', 'options': 'recursive,force'} 2024-04-21T08:53:56.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:56 smithi082 ceph-mon[27967]: pgmap v210: 129 pgs: 129 active+clean; 889 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 589 op/s 2024-04-21T08:53:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:56 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:56.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:56 smithi106 ceph-mon[31072]: pgmap v210: 129 pgs: 129 active+clean; 889 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 589 op/s 2024-04-21T08:53:56.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:56 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.679+0000 7f186ccf7640 1 -- 172.21.15.82:0/2434548186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1868101fc0 msgr2=0x7f18681023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.679+0000 7f186ccf7640 1 --2- 172.21.15.82:0/2434548186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1868101fc0 0x7f18681023a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1858009900 tx=0x7f185802bc50 comp rx=0 tx=0).stop 2024-04-21T08:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.681+0000 7f186ccf7640 1 -- 172.21.15.82:0/2434548186 shutdown_connections 2024-04-21T08:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.681+0000 7f186ccf7640 1 --2- 172.21.15.82:0/2434548186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1868107090 0x7f186810ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.681+0000 7f186ccf7640 1 --2- 172.21.15.82:0/2434548186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1868102970 0x7f18681069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.681+0000 7f186ccf7640 1 --2- 172.21.15.82:0/2434548186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1868101fc0 0x7f18681023a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.681+0000 7f186ccf7640 1 -- 172.21.15.82:0/2434548186 >> 172.21.15.82:0/2434548186 conn(0x7f18680fd8c0 msgr2=0x7f18680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.682+0000 7f186ccf7640 1 -- 172.21.15.82:0/2434548186 shutdown_connections 2024-04-21T08:53:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.682+0000 7f186ccf7640 1 -- 172.21.15.82:0/2434548186 wait complete. 2024-04-21T08:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.682+0000 7f186ccf7640 1 Processor -- start 2024-04-21T08:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.683+0000 7f186ccf7640 1 -- start start 2024-04-21T08:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.683+0000 7f186ccf7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1868101fc0 0x7f18681963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.683+0000 7f186ccf7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1868107090 0x7f1868196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.683+0000 7f186ccf7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f186819ac80 0x7f186819b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.683+0000 7f186ccf7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f186810fea0 con 0x7f1868107090 2024-04-21T08:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.684+0000 7f18677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1868101fc0 0x7f18681963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.684+0000 7f18677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1868101fc0 0x7f18681963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:44674/0 (socket says 172.21.15.82:44674) 2024-04-21T08:53:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.684+0000 7f18677fe640 1 -- 172.21.15.82:0/18943099 learned_addr learned my addr 172.21.15.82:0/18943099 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.684+0000 7f186ccf7640 1 -- 172.21.15.82:0/18943099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f186810fd20 con 0x7f186819ac80 2024-04-21T08:53:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.684+0000 7f186ccf7640 1 -- 172.21.15.82:0/18943099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1868110020 con 0x7f1868101fc0 2024-04-21T08:53:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.684+0000 7f1866ffd640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1868107090 0x7f1868196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.684+0000 7f1866ffd640 1 -- 172.21.15.82:0/18943099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1868101fc0 msgr2=0x7f18681963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.684+0000 7f1866ffd640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1868101fc0 0x7f18681963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.684+0000 7f1866ffd640 1 -- 172.21.15.82:0/18943099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f186819ac80 msgr2=0x7f186819b130 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:53:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.684+0000 7f1866ffd640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f186819ac80 0x7f186819b130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.684+0000 7f1866ffd640 1 -- 172.21.15.82:0/18943099 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186819b770 con 0x7f1868107090 2024-04-21T08:53:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.685+0000 7f1866ffd640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1868107090 0x7f1868196900 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f185c00b3c0 tx=0x7f185c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.685+0000 7f1864ff9640 1 -- 172.21.15.82:0/18943099 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c011830 con 0x7f1868107090 2024-04-21T08:53:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.685+0000 7f1864ff9640 1 -- 172.21.15.82:0/18943099 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f185c016070 con 0x7f1868107090 2024-04-21T08:53:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.685+0000 7f1864ff9640 1 -- 172.21.15.82:0/18943099 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c010e10 con 0x7f1868107090 2024-04-21T08:53:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.685+0000 7f186ccf7640 1 -- 172.21.15.82:0/18943099 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186819ba60 con 0x7f1868107090 2024-04-21T08:53:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.685+0000 7f186ccf7640 1 -- 172.21.15.82:0/18943099 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18681a31a0 con 0x7f1868107090 2024-04-21T08:53:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.689+0000 7f1870c2c740 1 Processor -- start 2024-04-21T08:53:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.690+0000 7f1864ff9640 1 -- 172.21.15.82:0/18943099 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f185c0119d0 con 0x7f1868107090 2024-04-21T08:53:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.692+0000 7f1864ff9640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18500a3e10 0x7f18500a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.692+0000 7f1864ff9640 1 -- 172.21.15.82:0/18943099 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f185c0d0060 con 0x7f1868107090 2024-04-21T08:53:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.692+0000 7f1870c2c740 1 -- start start 2024-04-21T08:53:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.693+0000 7f1870c2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae941f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.693+0000 7f1870c2c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 0x557ae941f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.693+0000 7f1870c2c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae942aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.693+0000 7f1870c2c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557ae937b8d0 con 0x557ae9428ae0 2024-04-21T08:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.693+0000 7f1870c2c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557ae9318e80 con 0x557ae941f560 2024-04-21T08:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.693+0000 7f1870c2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557ae9377890 con 0x557ae94202d0 2024-04-21T08:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.693+0000 7f1867fff640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18500a3e10 0x7f18500a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.693+0000 7f18677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae941f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.693+0000 7f18677fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae941f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:44682/0 (socket says 172.21.15.82:44682) 2024-04-21T08:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.693+0000 7f18677fe640 1 -- 172.21.15.82:0/2373077318 learned_addr learned my addr 172.21.15.82:0/2373077318 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.693+0000 7f1866ffd640 1 --2- 172.21.15.82:0/2373077318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 0x557ae941f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.694+0000 7f1867fff640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18500a3e10 0x7f18500a62d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1860005fd0 tx=0x7f1860009990 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.694+0000 7f18677fe640 1 -- 172.21.15.82:0/2373077318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 msgr2=0x557ae941f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.694+0000 7f18677fe640 1 --2- 172.21.15.82:0/2373077318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 0x557ae941f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.694+0000 7f1867fff640 1 --2- 172.21.15.82:0/2373077318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae942aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.694+0000 7f18677fe640 1 -- 172.21.15.82:0/2373077318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 msgr2=0x557ae942aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.694+0000 7f18677fe640 1 --2- 172.21.15.82:0/2373077318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae942aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.694+0000 7f18677fe640 1 -- 172.21.15.82:0/2373077318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae933a320 con 0x557ae94202d0 2024-04-21T08:53:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.694+0000 7f1867fff640 1 --2- 172.21.15.82:0/2373077318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae942aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:53:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.694+0000 7f18677fe640 1 --2- 172.21.15.82:0/2373077318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae941f020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1858009430 tx=0x7f1858005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=367cac0d0c151df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.695+0000 7f184dffb640 1 -- 172.21.15.82:0/2373077318 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858043070 con 0x557ae94202d0 2024-04-21T08:53:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.695+0000 7f184dffb640 1 -- 172.21.15.82:0/2373077318 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18580053c0 con 0x557ae94202d0 2024-04-21T08:53:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.695+0000 7f184dffb640 1 -- 172.21.15.82:0/2373077318 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858005220 con 0x557ae94202d0 2024-04-21T08:53:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.696+0000 7f1870c2c740 1 -- 172.21.15.82:0/2373077318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 msgr2=0x557ae941f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.696+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2373077318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae941f020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1858009430 tx=0x7f1858005e00 comp rx=0 tx=0).stop 2024-04-21T08:53:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.697+0000 7f1870c2c740 1 -- 172.21.15.82:0/2373077318 shutdown_connections 2024-04-21T08:53:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.697+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2373077318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae942aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.697+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2373077318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 0x557ae941f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.697+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2373077318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae941f020 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.697+0000 7f1870c2c740 1 -- 172.21.15.82:0/2373077318 >> 172.21.15.82:0/2373077318 conn(0x557ae93ddc90 msgr2=0x557ae941a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.697+0000 7f1870c2c740 1 -- 172.21.15.82:0/2373077318 shutdown_connections 2024-04-21T08:53:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.698+0000 7f1870c2c740 1 -- 172.21.15.82:0/2373077318 wait complete. 2024-04-21T08:53:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.698+0000 7f1870c2c740 1 -- start start 2024-04-21T08:53:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.698+0000 7f1870c2c740 1 Processor -- start 2024-04-21T08:53:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.699+0000 7f1870c2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae941f560 0x557ae94259f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.699+0000 7f1870c2c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae94202d0 0x557ae9425f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.699+0000 7f1870c2c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae9426470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.699+0000 7f1870c2c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557ae94116b0 con 0x557ae9428ae0 2024-04-21T08:53:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.699+0000 7f1870c2c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557ae9303eb0 con 0x557ae94202d0 2024-04-21T08:53:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.699+0000 7f1870c2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557ae9411830 con 0x557ae941f560 2024-04-21T08:53:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.699+0000 7f1870c2c740 10 client.0 _ll_register_callbacks cb 0x557ae9330b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.699+0000 7f1867fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae9426470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.699+0000 7f1867fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae9426470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44806/0 (socket says 172.21.15.82:44806) 2024-04-21T08:53:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.699+0000 7f1867fff640 1 -- 172.21.15.82:0/2468439644 learned_addr learned my addr 172.21.15.82:0/2468439644 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.699+0000 7f18677fe640 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae941f560 0x557ae94259f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.700+0000 7f1867fff640 1 -- 172.21.15.82:0/2468439644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae941f560 msgr2=0x557ae94259f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.700+0000 7f1867fff640 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae941f560 0x557ae94259f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.700+0000 7f1866ffd640 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae94202d0 0x557ae9425f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.700+0000 7f1867fff640 1 -- 172.21.15.82:0/2468439644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae94202d0 msgr2=0x557ae9425f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.700+0000 7f1867fff640 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae94202d0 0x557ae9425f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.700+0000 7f1867fff640 1 -- 172.21.15.82:0/2468439644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae908e210 con 0x557ae9428ae0 2024-04-21T08:53:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.700+0000 7f18677fe640 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae941f560 0x557ae94259f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:53:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.700+0000 7f1866ffd640 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae94202d0 0x557ae9425f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:53:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.700+0000 7f1867fff640 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae9426470 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1860012bf0 tx=0x7f1860004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f6bdcafc7412a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.700+0000 7f1846ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.701+0000 7f1846ffd640 1 -- 172.21.15.82:0/2468439644 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860022070 con 0x557ae9428ae0 2024-04-21T08:53:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.701+0000 7f1846ffd640 1 -- 172.21.15.82:0/2468439644 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1860011cf0 con 0x557ae9428ae0 2024-04-21T08:53:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.701+0000 7f1870c2c740 1 -- 172.21.15.82:0/2468439644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18580053c0 con 0x557ae9428ae0 2024-04-21T08:53:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.701+0000 7f1846ffd640 1 -- 172.21.15.82:0/2468439644 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1860027460 con 0x557ae9428ae0 2024-04-21T08:53:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.701+0000 7f1870c2c740 10 client.17598 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.701+0000 7f1870c2c740 1 -- 172.21.15.82:0/2468439644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557ae915a620 con 0x557ae9428ae0 2024-04-21T08:53:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.702+0000 7f1846ffd640 1 -- 172.21.15.82:0/2468439644 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18600318d0 con 0x557ae9428ae0 2024-04-21T08:53:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.702+0000 7f1846ffd640 1 -- 172.21.15.82:0/2468439644 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f186001e730 con 0x557ae9428ae0 2024-04-21T08:53:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.702+0000 7f1870c2c740 10 client.17598 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.702+0000 7f1870c2c740 10 client.17598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.702+0000 7f1870c2c740 20 client.17598 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.702+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ae9441710 0x557ae9443b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.702+0000 7f1870c2c740 4 client.17598 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:53:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.702+0000 7f1870c2c740 1 -- 172.21.15.82:0/2468439644 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f18580053c0 con 0x557ae9441710 2024-04-21T08:53:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.703+0000 7f18677fe640 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ae9441710 0x557ae9443b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.704+0000 7f18677fe640 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ae9441710 0x557ae9443b30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582c8789c63c2cef server_cookie=f3f4f88de0044f6c in_seq=0 out_seq=0 2024-04-21T08:53:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.704+0000 7f1846ffd640 10 client.17598 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.705+0000 7f1846ffd640 1 -- 172.21.15.82:0/2468439644 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f18580053c0 con 0x557ae9441710 2024-04-21T08:53:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.705+0000 7f1846ffd640 10 client.17598 handle_command_reply: tid=0 2024-04-21T08:53:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.705+0000 7f1870c2c740 1 client.17598 shutdown 2024-04-21T08:53:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.707+0000 7f1870c2c740 1 -- 172.21.15.82:0/2468439644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 msgr2=0x557ae9426470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.708+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae9426470 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1860012bf0 tx=0x7f1860004a40 comp rx=0 tx=0).stop 2024-04-21T08:53:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.709+0000 7f1870c2c740 1 -- 172.21.15.82:0/2468439644 shutdown_connections 2024-04-21T08:53:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.709+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ae9441710 0x557ae9443b30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.709+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae9426470 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.709+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae94202d0 0x557ae9425f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.709+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2468439644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae941f560 0x557ae94259f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.709+0000 7f1870c2c740 1 -- 172.21.15.82:0/2468439644 >> 172.21.15.82:0/2468439644 conn(0x557ae93ddc90 msgr2=0x557ae941bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.709+0000 7f1846ffd640 0 client.17598 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.709+0000 7f1870c2c740 1 -- 172.21.15.82:0/2468439644 shutdown_connections 2024-04-21T08:53:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.709+0000 7f1870c2c740 1 -- 172.21.15.82:0/2468439644 wait complete. 2024-04-21T08:53:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.709+0000 7f1870c2c740 20 client.17598 trim_cache size 0 max 16384 2024-04-21T08:53:56.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:56 smithi134 ceph-mon[31772]: pgmap v210: 129 pgs: 129 active+clean; 889 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 589 op/s 2024-04-21T08:53:56.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:56 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.736+0000 7f1870c2c740 1 Processor -- start 2024-04-21T08:53:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.737+0000 7f1870c2c740 1 -- start start 2024-04-21T08:53:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.737+0000 7f1870c2c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 0x557ae9421150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.737+0000 7f1870c2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae9426290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.737+0000 7f1870c2c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae941bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f1870c2c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f18681ad140 con 0x557ae9428ae0 2024-04-21T08:53:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f1870c2c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18681aed70 con 0x557ae941f560 2024-04-21T08:53:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f1870c2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18681ad810 con 0x557ae94202d0 2024-04-21T08:53:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f1867fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae941bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f1867fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae941bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44822/0 (socket says 172.21.15.82:44822) 2024-04-21T08:53:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f1867fff640 1 -- 172.21.15.82:0/2650801989 learned_addr learned my addr 172.21.15.82:0/2650801989 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f1866ffd640 1 --2- 172.21.15.82:0/2650801989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae9426290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f18677fe640 1 --2- 172.21.15.82:0/2650801989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 0x557ae9421150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f1867fff640 1 -- 172.21.15.82:0/2650801989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 msgr2=0x557ae9426290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f1867fff640 1 --2- 172.21.15.82:0/2650801989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae9426290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f1867fff640 1 -- 172.21.15.82:0/2650801989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 msgr2=0x557ae9421150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f1867fff640 1 --2- 172.21.15.82:0/2650801989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 0x557ae9421150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.738+0000 7f1867fff640 1 -- 172.21.15.82:0/2650801989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae9411820 con 0x557ae9428ae0 2024-04-21T08:53:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.739+0000 7f1867fff640 1 --2- 172.21.15.82:0/2650801989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae941bcc0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f186001ebd0 tx=0x7f1860011830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac92485ebd540505 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.739+0000 7f184d7fa640 1 -- 172.21.15.82:0/2650801989 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186003e070 con 0x557ae9428ae0 2024-04-21T08:53:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.739+0000 7f184d7fa640 1 -- 172.21.15.82:0/2650801989 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f186001e420 con 0x557ae9428ae0 2024-04-21T08:53:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.739+0000 7f184d7fa640 1 -- 172.21.15.82:0/2650801989 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860022030 con 0x557ae9428ae0 2024-04-21T08:53:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.740+0000 7f1870c2c740 1 -- 172.21.15.82:0/2650801989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 msgr2=0x557ae941bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.740+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2650801989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae941bcc0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f186001ebd0 tx=0x7f1860011830 comp rx=0 tx=0).stop 2024-04-21T08:53:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.741+0000 7f1870c2c740 1 -- 172.21.15.82:0/2650801989 shutdown_connections 2024-04-21T08:53:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.741+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2650801989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae941bcc0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.741+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2650801989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae9426290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.741+0000 7f1870c2c740 1 --2- 172.21.15.82:0/2650801989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 0x557ae9421150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.741+0000 7f1870c2c740 1 -- 172.21.15.82:0/2650801989 >> 172.21.15.82:0/2650801989 conn(0x557ae93ddc90 msgr2=0x557ae941b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.742+0000 7f1870c2c740 1 -- 172.21.15.82:0/2650801989 shutdown_connections 2024-04-21T08:53:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.742+0000 7f1870c2c740 1 -- 172.21.15.82:0/2650801989 wait complete. 2024-04-21T08:53:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.742+0000 7f1870c2c740 1 -- start start 2024-04-21T08:53:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.743+0000 7f1870c2c740 1 Processor -- start 2024-04-21T08:53:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.743+0000 7f1870c2c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 0x557ae9423160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.743+0000 7f1870c2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae94236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.743+0000 7f1870c2c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae9421dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.743+0000 7f1870c2c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f18681ae670 con 0x557ae9428ae0 2024-04-21T08:53:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.743+0000 7f1870c2c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18681adf70 con 0x557ae941f560 2024-04-21T08:53:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.743+0000 7f1870c2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18681ab0f0 con 0x557ae94202d0 2024-04-21T08:53:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.743+0000 7f1870c2c740 10 client.0 _ll_register_callbacks cb 0x7f1860016050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:53:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.743+0000 7f1867fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae9421dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.743+0000 7f1867fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae9421dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44832/0 (socket says 172.21.15.82:44832) 2024-04-21T08:53:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.743+0000 7f1867fff640 1 -- 172.21.15.82:0/3044115421 learned_addr learned my addr 172.21.15.82:0/3044115421 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:53:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.744+0000 7f18677fe640 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 0x557ae9423160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.744+0000 7f1866ffd640 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae94236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.744+0000 7f1867fff640 1 -- 172.21.15.82:0/3044115421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 msgr2=0x557ae94236a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.744+0000 7f1867fff640 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae94236a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.744+0000 7f1867fff640 1 -- 172.21.15.82:0/3044115421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 msgr2=0x557ae9423160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:53:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.744+0000 7f1867fff640 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 0x557ae9423160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.744+0000 7f1867fff640 1 -- 172.21.15.82:0/3044115421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae942b240 con 0x557ae9428ae0 2024-04-21T08:53:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.744+0000 7f1866ffd640 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae94236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:53:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.744+0000 7f1867fff640 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae9421dc0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1860009ce0 tx=0x7f186001e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4204ab99792a7130 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:53:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.744+0000 7f1846ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:53:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.744+0000 7f1846ffd640 1 -- 172.21.15.82:0/3044115421 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860049060 con 0x557ae9428ae0 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1870c2c740 1 -- 172.21.15.82:0/3044115421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f186001e420 con 0x557ae9428ae0 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1846ffd640 1 -- 172.21.15.82:0/3044115421 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9474+0+0 (secure 0 0 0) 0x7f1860030630 con 0x557ae9428ae0 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1846ffd640 1 -- 172.21.15.82:0/3044115421 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f186003e070 con 0x557ae9428ae0 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1870c2c740 10 client.17604 fetch_fsmap learned FSMap version 40 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1870c2c740 1 -- 172.21.15.82:0/3044115421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557ae9418570 con 0x557ae9428ae0 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1846ffd640 1 -- 172.21.15.82:0/3044115421 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1860030d40 con 0x557ae9428ae0 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1846ffd640 1 -- 172.21.15.82:0/3044115421 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f1860027930 con 0x557ae9428ae0 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1870c2c740 10 client.17604 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1870c2c740 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1870c2c740 20 client.17604 populate_metadata read hostname 'smithi082' 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1870c2c740 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ae94a7dc0 0x557ae94aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1870c2c740 4 client.17604 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.745+0000 7f1870c2c740 1 -- 172.21.15.82:0/3044115421 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f186001e420 con 0x557ae94a7dc0 2024-04-21T08:53:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.746+0000 7f18677fe640 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ae94a7dc0 0x557ae94aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:53:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.746+0000 7f1846ffd640 10 client.17604 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.746+0000 7f18677fe640 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ae94a7dc0 0x557ae94aa1e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=110a1a86987ed182 server_cookie=5b701b1ecc4a185a in_seq=0 out_seq=0 2024-04-21T08:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1846ffd640 1 -- 172.21.15.82:0/3044115421 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f186001e420 con 0x557ae94a7dc0 2024-04-21T08:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1846ffd640 10 client.17604 handle_command_reply: tid=0 2024-04-21T08:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1870c2c740 1 client.17604 shutdown 2024-04-21T08:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1870c2c740 1 -- 172.21.15.82:0/3044115421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 msgr2=0x557ae9421dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1870c2c740 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae9421dc0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1860009ce0 tx=0x7f186001e800 comp rx=0 tx=0).stop 2024-04-21T08:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1870c2c740 1 -- 172.21.15.82:0/3044115421 shutdown_connections 2024-04-21T08:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1870c2c740 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x557ae94a7dc0 0x557ae94aa1e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1846ffd640 0 client.17604 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1870c2c740 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ae9428ae0 0x557ae9421dc0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1870c2c740 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ae94202d0 0x557ae94236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1870c2c740 1 --2- 172.21.15.82:0/3044115421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ae941f560 0x557ae9423160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1870c2c740 1 -- 172.21.15.82:0/3044115421 >> 172.21.15.82:0/3044115421 conn(0x557ae93ddc90 msgr2=0x557ae9420fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1870c2c740 1 -- 172.21.15.82:0/3044115421 shutdown_connections 2024-04-21T08:53:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.860+0000 7f1870c2c740 1 -- 172.21.15.82:0/3044115421 wait complete. 2024-04-21T08:53:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.861+0000 7f1870c2c740 20 client.17604 trim_cache size 0 max 16384 2024-04-21T08:53:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:53:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:53:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:53:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:53:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.861+0000 7f186ccf7640 1 -- 172.21.15.82:0/18943099 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18500a3e10 msgr2=0x7f18500a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.861+0000 7f186ccf7640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18500a3e10 0x7f18500a62d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1860005fd0 tx=0x7f1860009990 comp rx=0 tx=0).stop 2024-04-21T08:53:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.861+0000 7f186ccf7640 1 -- 172.21.15.82:0/18943099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1868107090 msgr2=0x7f1868196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:53:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.861+0000 7f186ccf7640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1868107090 0x7f1868196900 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f185c00b3c0 tx=0x7f185c00b880 comp rx=0 tx=0).stop 2024-04-21T08:53:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.861+0000 7f186ccf7640 1 -- 172.21.15.82:0/18943099 shutdown_connections 2024-04-21T08:53:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.861+0000 7f186ccf7640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f186819ac80 0x7f186819b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.861+0000 7f186ccf7640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18500a3e10 0x7f18500a62d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.861+0000 7f186ccf7640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1868107090 0x7f1868196900 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.861+0000 7f186ccf7640 1 --2- 172.21.15.82:0/18943099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1868101fc0 0x7f18681963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:53:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.861+0000 7f186ccf7640 1 -- 172.21.15.82:0/18943099 >> 172.21.15.82:0/18943099 conn(0x7f18680fd8c0 msgr2=0x7f18680feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:53:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.862+0000 7f186ccf7640 1 -- 172.21.15.82:0/18943099 shutdown_connections 2024-04-21T08:53:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:53:56.862+0000 7f186ccf7640 1 -- 172.21.15.82:0/18943099 wait complete. 2024-04-21T08:53:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:53:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:57.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:57.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:57.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:57.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:57.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:57 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:53:57.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:57.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:57.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:57.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:57.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:57 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:53:57.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:53:57.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:57.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:57.728 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:57 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:53:58.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:58 smithi082 ceph-mon[27967]: pgmap v211: 129 pgs: 129 active+clean; 891 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 745 op/s 2024-04-21T08:53:58.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:53:58 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:58.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:58 smithi106 ceph-mon[31072]: pgmap v211: 129 pgs: 129 active+clean; 891 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 745 op/s 2024-04-21T08:53:58.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:53:58 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:53:58.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:58 smithi134 ceph-mon[31772]: pgmap v211: 129 pgs: 129 active+clean; 891 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 745 op/s 2024-04-21T08:53:58.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:53:58 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:00.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:00 smithi082 ceph-mon[27967]: pgmap v212: 129 pgs: 129 active+clean; 894 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 752 op/s 2024-04-21T08:54:00.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:00 smithi106 ceph-mon[31072]: pgmap v212: 129 pgs: 129 active+clean; 894 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 752 op/s 2024-04-21T08:54:00.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:00 smithi134 ceph-mon[31772]: pgmap v212: 129 pgs: 129 active+clean; 894 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 752 op/s 2024-04-21T08:54:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:02 smithi082 ceph-mon[27967]: pgmap v213: 129 pgs: 129 active+clean; 894 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 605 op/s 2024-04-21T08:54:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:02 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:02 smithi106 ceph-mon[31072]: pgmap v213: 129 pgs: 129 active+clean; 894 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 605 op/s 2024-04-21T08:54:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:02 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:02 smithi134 ceph-mon[31772]: pgmap v213: 129 pgs: 129 active+clean; 894 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 605 op/s 2024-04-21T08:54:02.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:02 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:04 smithi082 ceph-mon[27967]: pgmap v214: 129 pgs: 129 active+clean; 859 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 1.27k op/s 2024-04-21T08:54:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:04 smithi106 ceph-mon[31072]: pgmap v214: 129 pgs: 129 active+clean; 859 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 1.27k op/s 2024-04-21T08:54:04.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:04 smithi134 ceph-mon[31772]: pgmap v214: 129 pgs: 129 active+clean; 859 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 1.27k op/s 2024-04-21T08:54:06.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:06 smithi082 ceph-mon[27967]: pgmap v215: 129 pgs: 129 active+clean; 859 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 976 op/s 2024-04-21T08:54:06.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:06 smithi106 ceph-mon[31072]: pgmap v215: 129 pgs: 129 active+clean; 859 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 976 op/s 2024-04-21T08:54:06.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:06 smithi134 ceph-mon[31772]: pgmap v215: 129 pgs: 129 active+clean; 859 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 976 op/s 2024-04-21T08:54:08.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:08 smithi082 ceph-mon[27967]: pgmap v216: 129 pgs: 129 active+clean; 867 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 1.08k op/s 2024-04-21T08:54:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:08 smithi106 ceph-mon[31072]: pgmap v216: 129 pgs: 129 active+clean; 867 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 1.08k op/s 2024-04-21T08:54:08.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:08 smithi134 ceph-mon[31772]: pgmap v216: 129 pgs: 129 active+clean; 867 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 1.08k op/s 2024-04-21T08:54:09.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:09 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:54:09.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:09 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:54:09.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:09 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:54:09.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:09 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:54:09.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:09 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:54:09.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:09 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:54:09.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:09 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:54:09.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:09 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:54:09.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:09 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:54:09.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:09 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:54:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:09 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:54:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:09 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:54:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:09 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:54:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:09 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:54:09.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:09 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:54:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.003+0000 7f192bed5640 1 -- 172.21.15.82:0/114805718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1924109490 msgr2=0x7f1924111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.003+0000 7f192bed5640 1 --2- 172.21.15.82:0/114805718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1924109490 0x7f1924111060 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f191c009ca0 tx=0x7f191c02c120 comp rx=0 tx=0).stop 2024-04-21T08:54:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.003+0000 7f192bed5640 1 -- 172.21.15.82:0/114805718 shutdown_connections 2024-04-21T08:54:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.003+0000 7f192bed5640 1 --2- 172.21.15.82:0/114805718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1924109490 0x7f1924111060 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.003+0000 7f192bed5640 1 --2- 172.21.15.82:0/114805718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1924104d70 0x7f1924108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.003+0000 7f192bed5640 1 --2- 172.21.15.82:0/114805718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19241043c0 0x7f19241047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.003+0000 7f192bed5640 1 -- 172.21.15.82:0/114805718 >> 172.21.15.82:0/114805718 conn(0x7f19240ffc80 msgr2=0x7f19241020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.004+0000 7f192bed5640 1 -- 172.21.15.82:0/114805718 shutdown_connections 2024-04-21T08:54:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.004+0000 7f192bed5640 1 -- 172.21.15.82:0/114805718 wait complete. 2024-04-21T08:54:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.004+0000 7f192bed5640 1 Processor -- start 2024-04-21T08:54:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.005+0000 7f192bed5640 1 -- start start 2024-04-21T08:54:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.005+0000 7f192bed5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19241043c0 0x7f19241a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.005+0000 7f192bed5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1924109490 0x7f19241a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.005+0000 7f192bed5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f192410fa70 0x7f192410ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.005+0000 7f192bed5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1924077590 con 0x7f1924109490 2024-04-21T08:54:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.005+0000 7f192bed5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1924077410 con 0x7f19241043c0 2024-04-21T08:54:10.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.005+0000 7f192bed5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1924077710 con 0x7f192410fa70 2024-04-21T08:54:10.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.005+0000 7f192a472640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f192410fa70 0x7f192410ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.006+0000 7f192a472640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f192410fa70 0x7f192410ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:53986/0 (socket says 172.21.15.82:53986) 2024-04-21T08:54:10.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.006+0000 7f192a472640 1 -- 172.21.15.82:0/3777937471 learned_addr learned my addr 172.21.15.82:0/3777937471 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.007+0000 7f192a472640 1 -- 172.21.15.82:0/3777937471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19241043c0 msgr2=0x7f19241a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.007+0000 7f1929c71640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19241043c0 0x7f19241a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.007+0000 7f192a472640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19241043c0 0x7f19241a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.008+0000 7f192a472640 1 -- 172.21.15.82:0/3777937471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1924109490 msgr2=0x7f19241a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.008+0000 7f192a472640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1924109490 0x7f19241a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.008+0000 7f192a472640 1 -- 172.21.15.82:0/3777937471 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1924110630 con 0x7f192410fa70 2024-04-21T08:54:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.009+0000 7f192a472640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f192410fa70 0x7f192410ff00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f191c02c6b0 tx=0x7f191c035ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.009+0000 7f1929c71640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19241043c0 0x7f19241a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.009+0000 7f191affd640 1 -- 172.21.15.82:0/3777937471 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c039040 con 0x7f192410fa70 2024-04-21T08:54:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.009+0000 7f191affd640 1 -- 172.21.15.82:0/3777937471 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f191c044070 con 0x7f192410fa70 2024-04-21T08:54:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.009+0000 7f191affd640 1 -- 172.21.15.82:0/3777937471 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c0042a0 con 0x7f192410fa70 2024-04-21T08:54:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.010+0000 7f192bed5640 1 -- 172.21.15.82:0/3777937471 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192406d980 con 0x7f192410fa70 2024-04-21T08:54:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.010+0000 7f192bed5640 1 -- 172.21.15.82:0/3777937471 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f192406de20 con 0x7f192410fa70 2024-04-21T08:54:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.014+0000 7f192eb9c740 1 Processor -- start 2024-04-21T08:54:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.014+0000 7f191affd640 1 -- 172.21.15.82:0/3777937471 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f191c004440 con 0x7f192410fa70 2024-04-21T08:54:10.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.015+0000 7f192eb9c740 1 -- start start 2024-04-21T08:54:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.015+0000 7f192eb9c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2ea88d0 0x55afc2ea7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.015+0000 7f192eb9c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea7b60 0x55afc2ea7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.015+0000 7f192eb9c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 0x55afc2eb34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.015+0000 7f192eb9c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55afc2e0c8d0 con 0x55afc2ea7b60 2024-04-21T08:54:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.015+0000 7f192eb9c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55afc2da9e80 con 0x55afc2eb10e0 2024-04-21T08:54:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.015+0000 7f192eb9c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55afc2e08890 con 0x55afc2ea88d0 2024-04-21T08:54:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.016+0000 7f1929470640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2ea88d0 0x55afc2ea7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.016+0000 7f191affd640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19100a3e90 0x7f19100a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.016+0000 7f1929470640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2ea88d0 0x55afc2ea7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:53996/0 (socket says 172.21.15.82:53996) 2024-04-21T08:54:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.016+0000 7f1929470640 1 -- 172.21.15.82:0/1260860389 learned_addr learned my addr 172.21.15.82:0/1260860389 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.016+0000 7f1929470640 1 --2- 172.21.15.82:0/1260860389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 0x55afc2eb34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.016+0000 7f191affd640 1 -- 172.21.15.82:0/3777937471 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9650+0+0 (secure 0 0 0) 0x7f191c0f29e0 con 0x7f192410fa70 2024-04-21T08:54:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.017+0000 7f192a472640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19100a3e90 0x7f19100a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.018+0000 7f1929c71640 1 --2- 172.21.15.82:0/1260860389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea7b60 0x55afc2ea7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.019+0000 7f1929c71640 1 -- 172.21.15.82:0/1260860389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2ea88d0 msgr2=0x55afc2ea7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.019+0000 7f1929c71640 1 --2- 172.21.15.82:0/1260860389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2ea88d0 0x55afc2ea7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.019+0000 7f1929c71640 1 -- 172.21.15.82:0/1260860389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 msgr2=0x55afc2eb34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.019+0000 7f1929c71640 1 --2- 172.21.15.82:0/1260860389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 0x55afc2eb34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.019+0000 7f1929c71640 1 -- 172.21.15.82:0/1260860389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc2dcb320 con 0x55afc2ea7b60 2024-04-21T08:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.019+0000 7f1929c71640 1 --2- 172.21.15.82:0/1260860389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea7b60 0x55afc2ea7fe0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1920009b90 tx=0x7f192000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68326fbdedf013f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.020+0000 7f190ffff640 1 -- 172.21.15.82:0/1260860389 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920016070 con 0x55afc2ea7b60 2024-04-21T08:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.020+0000 7f190ffff640 1 -- 172.21.15.82:0/1260860389 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19200054d0 con 0x55afc2ea7b60 2024-04-21T08:54:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.020+0000 7f190ffff640 1 -- 172.21.15.82:0/1260860389 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920016070 con 0x55afc2ea7b60 2024-04-21T08:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.022+0000 7f192eb9c740 1 -- 172.21.15.82:0/1260860389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea7b60 msgr2=0x55afc2ea7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.022+0000 7f192eb9c740 1 --2- 172.21.15.82:0/1260860389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea7b60 0x55afc2ea7fe0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1920009b90 tx=0x7f192000e6c0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.022+0000 7f192a472640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19100a3e90 0x7f19100a6350 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f191c006010 tx=0x7f191c0900f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.022+0000 7f192eb9c740 1 -- 172.21.15.82:0/1260860389 shutdown_connections 2024-04-21T08:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.022+0000 7f192eb9c740 1 --2- 172.21.15.82:0/1260860389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 0x55afc2eb34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.022+0000 7f192eb9c740 1 --2- 172.21.15.82:0/1260860389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea7b60 0x55afc2ea7fe0 secure :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1920009b90 tx=0x7f192000e6c0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.022+0000 7f192eb9c740 1 --2- 172.21.15.82:0/1260860389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2ea88d0 0x55afc2ea7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.022+0000 7f192eb9c740 1 -- 172.21.15.82:0/1260860389 >> 172.21.15.82:0/1260860389 conn(0x55afc2e6ec90 msgr2=0x55afc2ea2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.024+0000 7f192eb9c740 1 -- 172.21.15.82:0/1260860389 shutdown_connections 2024-04-21T08:54:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.025+0000 7f192eb9c740 1 -- 172.21.15.82:0/1260860389 wait complete. 2024-04-21T08:54:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.025+0000 7f192eb9c740 1 -- start start 2024-04-21T08:54:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.025+0000 7f192eb9c740 1 Processor -- start 2024-04-21T08:54:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.026+0000 7f192eb9c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2ea88d0 0x55afc2eadee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.026+0000 7f192eb9c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2eb10e0 0x55afc2eae420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.026+0000 7f192eb9c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2ebd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.026+0000 7f192eb9c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55afc2eb3e10 con 0x55afc2eb10e0 2024-04-21T08:54:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.026+0000 7f192eb9c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55afc2d94eb0 con 0x55afc2ea88d0 2024-04-21T08:54:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.026+0000 7f192eb9c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55afc2eb3f90 con 0x55afc2eae960 2024-04-21T08:54:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.026+0000 7f192eb9c740 10 client.0 _ll_register_callbacks cb 0x55afc2dc1b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.028+0000 7f1929470640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2ea88d0 0x55afc2eadee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.028+0000 7f1929470640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2ebd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.029+0000 7f1929470640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2ebd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:54006/0 (socket says 172.21.15.82:54006) 2024-04-21T08:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.029+0000 7f1929470640 1 -- 172.21.15.82:0/395271591 learned_addr learned my addr 172.21.15.82:0/395271591 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.029+0000 7f1929c71640 1 --2- 172.21.15.82:0/395271591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2eb10e0 0x55afc2eae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.030+0000 7f1929c71640 1 -- 172.21.15.82:0/395271591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 msgr2=0x55afc2ebd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.030+0000 7f1929c71640 1 --2- 172.21.15.82:0/395271591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2ebd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.030+0000 7f1929c71640 1 -- 172.21.15.82:0/395271591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2ea88d0 msgr2=0x55afc2eadee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.030+0000 7f1929c71640 1 --2- 172.21.15.82:0/395271591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2ea88d0 0x55afc2eadee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.030+0000 7f1929c71640 1 -- 172.21.15.82:0/395271591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc2b1f210 con 0x55afc2eb10e0 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.033+0000 7f1929c71640 1 --2- 172.21.15.82:0/395271591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2eb10e0 0x55afc2eae420 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x55afc2ea8fa0 tx=0x7f1920004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a6876c57710c65 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.033+0000 7f190d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.033+0000 7f190d7fa640 1 -- 172.21.15.82:0/395271591 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920016070 con 0x55afc2eb10e0 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.033+0000 7f192eb9c740 1 -- 172.21.15.82:0/395271591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19200054d0 con 0x55afc2eb10e0 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.033+0000 7f190d7fa640 1 -- 172.21.15.82:0/395271591 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9650+0+0 (secure 0 0 0) 0x7f1920009c80 con 0x55afc2eb10e0 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.034+0000 7f190d7fa640 1 -- 172.21.15.82:0/395271591 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f192002e050 con 0x55afc2eb10e0 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.034+0000 7f192eb9c740 10 client.17610 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.034+0000 7f192eb9c740 1 -- 172.21.15.82:0/395271591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55afc2beb620 con 0x55afc2eb10e0 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.034+0000 7f190d7fa640 1 -- 172.21.15.82:0/395271591 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19200054d0 con 0x55afc2eb10e0 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.034+0000 7f190d7fa640 1 -- 172.21.15.82:0/395271591 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f192000f040 con 0x55afc2eb10e0 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.034+0000 7f192eb9c740 10 client.17610 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.034+0000 7f192eb9c740 10 client.17610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.034+0000 7f192eb9c740 20 client.17610 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.034+0000 7f192eb9c740 1 --2- 172.21.15.82:0/395271591 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55afc2ec1940 0x55afc2ec3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.034+0000 7f192eb9c740 4 client.17610 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.034+0000 7f192eb9c740 1 -- 172.21.15.82:0/395271591 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afc2e20880 con 0x55afc2ec1940 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.035+0000 7f1929470640 1 --2- 172.21.15.82:0/395271591 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55afc2ec1940 0x55afc2ec3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.036+0000 7f1929470640 1 --2- 172.21.15.82:0/395271591 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55afc2ec1940 0x55afc2ec3e00 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=243fecc430a36523 server_cookie=1def7511b0206d90 in_seq=0 out_seq=0 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.036+0000 7f190d7fa640 10 client.17610 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.037+0000 7f190d7fa640 1 -- 172.21.15.82:0/395271591 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afc2e20880 con 0x55afc2ec1940 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.037+0000 7f190d7fa640 10 client.17610 handle_command_reply: tid=0 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.037+0000 7f192eb9c740 1 client.17610 shutdown 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.038+0000 7f192eb9c740 1 -- 172.21.15.82:0/395271591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2eb10e0 msgr2=0x55afc2eae420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.038+0000 7f192eb9c740 1 --2- 172.21.15.82:0/395271591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2eb10e0 0x55afc2eae420 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x55afc2ea8fa0 tx=0x7f1920004ec0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.039+0000 7f192eb9c740 1 -- 172.21.15.82:0/395271591 shutdown_connections 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.039+0000 7f192eb9c740 1 --2- 172.21.15.82:0/395271591 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55afc2ec1940 0x55afc2ec3e00 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.039+0000 7f192eb9c740 1 --2- 172.21.15.82:0/395271591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2ebd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.039+0000 7f192eb9c740 1 --2- 172.21.15.82:0/395271591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2eb10e0 0x55afc2eae420 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.039+0000 7f192eb9c740 1 --2- 172.21.15.82:0/395271591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2ea88d0 0x55afc2eadee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.039+0000 7f192eb9c740 1 -- 172.21.15.82:0/395271591 >> 172.21.15.82:0/395271591 conn(0x55afc2e6ec90 msgr2=0x55afc2ea45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.039+0000 7f190d7fa640 0 client.17610 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.040+0000 7f192eb9c740 1 -- 172.21.15.82:0/395271591 shutdown_connections 2024-04-21T08:54:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.040+0000 7f192eb9c740 1 -- 172.21.15.82:0/395271591 wait complete. 2024-04-21T08:54:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.040+0000 7f192eb9c740 20 client.17610 trim_cache size 0 max 16384 2024-04-21T08:54:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.068+0000 7f192eb9c740 1 Processor -- start 2024-04-21T08:54:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.068+0000 7f192eb9c740 1 -- start start 2024-04-21T08:54:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.068+0000 7f192eb9c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea88d0 0x55afc2ec9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.068+0000 7f192eb9c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2ec9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.068+0000 7f192eb9c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 0x55afc2ebcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f192eb9c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f19241be5a0 con 0x55afc2ea88d0 2024-04-21T08:54:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f192eb9c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19241c01d0 con 0x55afc2eb10e0 2024-04-21T08:54:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f192eb9c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19241bec70 con 0x55afc2eae960 2024-04-21T08:54:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f1929470640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea88d0 0x55afc2ec9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f1929c71640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2ec9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f1929470640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea88d0 0x55afc2ec9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35916/0 (socket says 172.21.15.82:35916) 2024-04-21T08:54:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f1929470640 1 -- 172.21.15.82:0/678701858 learned_addr learned my addr 172.21.15.82:0/678701858 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f1929c71640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2ec9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:54008/0 (socket says 172.21.15.82:54008) 2024-04-21T08:54:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f1929470640 1 -- 172.21.15.82:0/678701858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 msgr2=0x55afc2ec9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f1929470640 1 --2- 172.21.15.82:0/678701858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2ec9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f1929470640 1 -- 172.21.15.82:0/678701858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 msgr2=0x55afc2ebcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f1929470640 1 --2- 172.21.15.82:0/678701858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 0x55afc2ebcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f1929470640 1 -- 172.21.15.82:0/678701858 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc2e9cce0 con 0x55afc2ea88d0 2024-04-21T08:54:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f1929470640 1 --2- 172.21.15.82:0/678701858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea88d0 0x55afc2ec9710 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1914009fa0 tx=0x7f191400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621ea8a48d2a99f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.069+0000 7f190f7fe640 1 -- 172.21.15.82:0/678701858 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914012040 con 0x55afc2ea88d0 2024-04-21T08:54:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.070+0000 7f190f7fe640 1 -- 172.21.15.82:0/678701858 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1914008e30 con 0x55afc2ea88d0 2024-04-21T08:54:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.070+0000 7f190f7fe640 1 -- 172.21.15.82:0/678701858 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191401e0e0 con 0x55afc2ea88d0 2024-04-21T08:54:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.070+0000 7f192eb9c740 1 -- 172.21.15.82:0/678701858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea88d0 msgr2=0x55afc2ec9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.071+0000 7f192eb9c740 1 --2- 172.21.15.82:0/678701858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea88d0 0x55afc2ec9710 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1914009fa0 tx=0x7f191400b040 comp rx=0 tx=0).stop 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.071+0000 7f192eb9c740 1 -- 172.21.15.82:0/678701858 shutdown_connections 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.071+0000 7f192eb9c740 1 --2- 172.21.15.82:0/678701858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 0x55afc2ebcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.071+0000 7f192eb9c740 1 --2- 172.21.15.82:0/678701858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2ec9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.071+0000 7f192eb9c740 1 --2- 172.21.15.82:0/678701858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea88d0 0x55afc2ec9710 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.071+0000 7f192eb9c740 1 -- 172.21.15.82:0/678701858 >> 172.21.15.82:0/678701858 conn(0x55afc2e6ec90 msgr2=0x55afc2eac7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.071+0000 7f192eb9c740 1 -- 172.21.15.82:0/678701858 shutdown_connections 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.071+0000 7f192eb9c740 1 -- 172.21.15.82:0/678701858 wait complete. 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.071+0000 7f192eb9c740 1 -- start start 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.071+0000 7f192eb9c740 1 Processor -- start 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.072+0000 7f192eb9c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea88d0 0x55afc2eb95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.072+0000 7f192eb9c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2eab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.072+0000 7f192eb9c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 0x55afc2eab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.072+0000 7f192eb9c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f19241bfad0 con 0x55afc2ea88d0 2024-04-21T08:54:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.072+0000 7f1929c71640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2eab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.072+0000 7f1929c71640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2eab160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:54016/0 (socket says 172.21.15.82:54016) 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.072+0000 7f1929c71640 1 -- 172.21.15.82:0/1620014126 learned_addr learned my addr 172.21.15.82:0/1620014126 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.072+0000 7f192eb9c740 1 -- 172.21.15.82:0/1620014126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19241bf3d0 con 0x55afc2eb10e0 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.072+0000 7f1929470640 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea88d0 0x55afc2eb95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.072+0000 7f192eb9c740 1 -- 172.21.15.82:0/1620014126 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1924110800 con 0x55afc2eae960 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f192eb9c740 10 client.0 _ll_register_callbacks cb 0x7f1914001900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f1929470640 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 0x55afc2eab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f1929c71640 1 -- 172.21.15.82:0/1620014126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 msgr2=0x55afc2eab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f1929c71640 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 0x55afc2eab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f1929c71640 1 -- 172.21.15.82:0/1620014126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea88d0 msgr2=0x55afc2eb95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f1929c71640 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea88d0 0x55afc2eb95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f1929c71640 1 -- 172.21.15.82:0/1620014126 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc2ea6e30 con 0x55afc2eae960 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f1929470640 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 0x55afc2eab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f1929c71640 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2eab160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55afc2ead740 tx=0x7f1920000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f8e8b47b07b93d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f190d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f190d7fa640 1 -- 172.21.15.82:0/1620014126 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920016070 con 0x55afc2eae960 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f190d7fa640 1 -- 172.21.15.82:0/1620014126 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9650+0+0 (secure 0 0 0) 0x7f1920009c80 con 0x55afc2eae960 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.073+0000 7f192eb9c740 1 -- 172.21.15.82:0/1620014126 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1914008e30 con 0x55afc2eae960 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.074+0000 7f190d7fa640 1 -- 172.21.15.82:0/1620014126 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f192001e070 con 0x55afc2eae960 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.074+0000 7f192eb9c740 10 client.26611 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.074+0000 7f192eb9c740 1 -- 172.21.15.82:0/1620014126 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55afc2eb2280 con 0x55afc2eae960 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.074+0000 7f190d7fa640 1 -- 172.21.15.82:0/1620014126 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1914008e30 con 0x55afc2eae960 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.074+0000 7f190d7fa640 1 -- 172.21.15.82:0/1620014126 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f1920005380 con 0x55afc2eae960 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.074+0000 7f192eb9c740 10 client.26611 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.074+0000 7f192eb9c740 10 client.26611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.074+0000 7f192eb9c740 20 client.26611 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.074+0000 7f192eb9c740 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55afc2eb2a00 0x55afc2f29f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.074+0000 7f192eb9c740 4 client.26611 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:54:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.074+0000 7f192eb9c740 1 -- 172.21.15.82:0/1620014126 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afc2dac3d0 con 0x55afc2eb2a00 2024-04-21T08:54:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.076+0000 7f1929470640 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55afc2eb2a00 0x55afc2f29f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.076+0000 7f1929470640 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55afc2eb2a00 0x55afc2f29f90 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f7ee7f46b78902 server_cookie=c2c703a624685844 in_seq=0 out_seq=0 2024-04-21T08:54:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.076+0000 7f190d7fa640 10 client.26611 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.105+0000 7f190d7fa640 1 -- 172.21.15.82:0/1620014126 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55afc2dac3d0 con 0x55afc2eb2a00 2024-04-21T08:54:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.105+0000 7f190d7fa640 10 client.26611 handle_command_reply: tid=0 2024-04-21T08:54:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.105+0000 7f192eb9c740 1 client.26611 shutdown 2024-04-21T08:54:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.106+0000 7f192eb9c740 1 -- 172.21.15.82:0/1620014126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 msgr2=0x55afc2eab160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.106+0000 7f192eb9c740 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2eab160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55afc2ead740 tx=0x7f1920000bc0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.106+0000 7f192eb9c740 1 -- 172.21.15.82:0/1620014126 shutdown_connections 2024-04-21T08:54:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.106+0000 7f192eb9c740 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55afc2eb2a00 0x55afc2f29f90 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.106+0000 7f192eb9c740 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55afc2eb10e0 0x55afc2eab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.106+0000 7f192eb9c740 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55afc2eae960 0x55afc2eab160 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.106+0000 7f190d7fa640 0 client.26611 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.106+0000 7f192eb9c740 1 --2- 172.21.15.82:0/1620014126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55afc2ea88d0 0x55afc2eb95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.106+0000 7f192eb9c740 1 -- 172.21.15.82:0/1620014126 >> 172.21.15.82:0/1620014126 conn(0x55afc2e6ec90 msgr2=0x55afc2eac710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.106+0000 7f192eb9c740 1 -- 172.21.15.82:0/1620014126 shutdown_connections 2024-04-21T08:54:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.106+0000 7f192eb9c740 1 -- 172.21.15.82:0/1620014126 wait complete. 2024-04-21T08:54:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.106+0000 7f192eb9c740 20 client.26611 trim_cache size 0 max 16384 2024-04-21T08:54:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:54:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:54:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:54:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:54:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.107+0000 7f192bed5640 1 -- 172.21.15.82:0/3777937471 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19100a3e90 msgr2=0x7f19100a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.107+0000 7f192bed5640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19100a3e90 0x7f19100a6350 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f191c006010 tx=0x7f191c0900f0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.107+0000 7f192bed5640 1 -- 172.21.15.82:0/3777937471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f192410fa70 msgr2=0x7f192410ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.107+0000 7f192bed5640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f192410fa70 0x7f192410ff00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f191c02c6b0 tx=0x7f191c035ce0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.107+0000 7f192bed5640 1 -- 172.21.15.82:0/3777937471 shutdown_connections 2024-04-21T08:54:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.107+0000 7f192bed5640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f192410fa70 0x7f192410ff00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.107+0000 7f192bed5640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19100a3e90 0x7f19100a6350 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.107+0000 7f192bed5640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1924109490 0x7f19241a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.107+0000 7f192bed5640 1 --2- 172.21.15.82:0/3777937471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19241043c0 0x7f19241a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.107+0000 7f192bed5640 1 -- 172.21.15.82:0/3777937471 >> 172.21.15.82:0/3777937471 conn(0x7f19240ffc80 msgr2=0x7f1924100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.107+0000 7f192bed5640 1 -- 172.21.15.82:0/3777937471 shutdown_connections 2024-04-21T08:54:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:10.107+0000 7f192bed5640 1 -- 172.21.15.82:0/3777937471 wait complete. 2024-04-21T08:54:10.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:10 smithi082 ceph-mon[27967]: pgmap v217: 129 pgs: 129 active+clean; 869 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 1.02k op/s 2024-04-21T08:54:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:10 smithi082 ceph-mon[27967]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T08:54:10.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:10 smithi106 ceph-mon[31072]: pgmap v217: 129 pgs: 129 active+clean; 869 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 1.02k op/s 2024-04-21T08:54:10.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:10 smithi106 ceph-mon[31072]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T08:54:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:10 smithi134 ceph-mon[31772]: pgmap v217: 129 pgs: 129 active+clean; 869 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 1.02k op/s 2024-04-21T08:54:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:10 smithi134 ceph-mon[31772]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T08:54:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:12 smithi082 ceph-mon[27967]: pgmap v219: 129 pgs: 129 active+clean; 869 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.04k op/s 2024-04-21T08:54:12.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:12 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:54:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:12 smithi106 ceph-mon[31072]: pgmap v219: 129 pgs: 129 active+clean; 869 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.04k op/s 2024-04-21T08:54:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:12 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:54:12.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:12 smithi134 ceph-mon[31772]: pgmap v219: 129 pgs: 129 active+clean; 869 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 1.04k op/s 2024-04-21T08:54:12.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:12 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:54:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:14 smithi106 ceph-mon[31072]: pgmap v220: 129 pgs: 129 active+clean; 877 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 476 op/s 2024-04-21T08:54:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:14 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:54:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:14 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:54:14.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:14 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:54:14.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:14 smithi134 ceph-mon[31772]: pgmap v220: 129 pgs: 129 active+clean; 877 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 476 op/s 2024-04-21T08:54:14.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:14 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:54:14.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:14 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:54:14.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:14 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:54:14.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:14 smithi082 ceph-mon[27967]: pgmap v220: 129 pgs: 129 active+clean; 877 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 476 op/s 2024-04-21T08:54:14.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:14 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:54:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:14 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:54:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:14 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:54:16.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:16 smithi106 ceph-mon[31072]: pgmap v221: 129 pgs: 129 active+clean; 877 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 476 op/s 2024-04-21T08:54:16.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:16 smithi106 ceph-mon[31072]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T08:54:16.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:16 smithi134 ceph-mon[31772]: pgmap v221: 129 pgs: 129 active+clean; 877 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 476 op/s 2024-04-21T08:54:16.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:16 smithi134 ceph-mon[31772]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T08:54:16.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:16 smithi082 ceph-mon[27967]: pgmap v221: 129 pgs: 129 active+clean; 877 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 476 op/s 2024-04-21T08:54:16.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:16 smithi082 ceph-mon[27967]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T08:54:18.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:18 smithi106 ceph-mon[31072]: pgmap v223: 129 pgs: 129 active+clean; 883 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 489 op/s 2024-04-21T08:54:18.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:18 smithi134 ceph-mon[31772]: pgmap v223: 129 pgs: 129 active+clean; 883 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 489 op/s 2024-04-21T08:54:18.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:18 smithi082 ceph-mon[27967]: pgmap v223: 129 pgs: 129 active+clean; 883 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 489 op/s 2024-04-21T08:54:20.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:20 smithi106 ceph-mon[31072]: pgmap v224: 129 pgs: 129 active+clean; 893 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 590 op/s 2024-04-21T08:54:20.411 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:20 smithi082 ceph-mon[27967]: pgmap v224: 129 pgs: 129 active+clean; 893 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 590 op/s 2024-04-21T08:54:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:20 smithi134 ceph-mon[31772]: pgmap v224: 129 pgs: 129 active+clean; 893 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 590 op/s 2024-04-21T08:54:22.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:22 smithi106 ceph-mon[31072]: pgmap v225: 129 pgs: 129 active+clean; 893 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 555 op/s 2024-04-21T08:54:22.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:22 smithi134 ceph-mon[31772]: pgmap v225: 129 pgs: 129 active+clean; 893 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 555 op/s 2024-04-21T08:54:22.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:22 smithi082 ceph-mon[27967]: pgmap v225: 129 pgs: 129 active+clean; 893 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 555 op/s 2024-04-21T08:54:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.200+0000 7f18ae7fc640 1 -- 172.21.15.82:0/2592918748 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80328c0 con 0x7f18b0106170 2024-04-21T08:54:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.201+0000 7f18b5c55640 1 -- 172.21.15.82:0/2592918748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18b0106170 msgr2=0x7f18b010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.201+0000 7f18b5c55640 1 --2- 172.21.15.82:0/2592918748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18b0106170 0x7f18b010dd40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009ca0 tx=0x7f18a802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.201+0000 7f18b5c55640 1 -- 172.21.15.82:0/2592918748 shutdown_connections 2024-04-21T08:54:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.201+0000 7f18b5c55640 1 --2- 172.21.15.82:0/2592918748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18b0106170 0x7f18b010dd40 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.201+0000 7f18b5c55640 1 --2- 172.21.15.82:0/2592918748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18b0101aa0 0x7f18b0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.201+0000 7f18b5c55640 1 --2- 172.21.15.82:0/2592918748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18b01010f0 0x7f18b01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.201+0000 7f18b5c55640 1 -- 172.21.15.82:0/2592918748 >> 172.21.15.82:0/2592918748 conn(0x7f18b0077fd0 msgr2=0x7f18b00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.201+0000 7f18b5c55640 1 -- 172.21.15.82:0/2592918748 shutdown_connections 2024-04-21T08:54:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.201+0000 7f18b5c55640 1 -- 172.21.15.82:0/2592918748 wait complete. 2024-04-21T08:54:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18b5c55640 1 Processor -- start 2024-04-21T08:54:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18b5c55640 1 -- start start 2024-04-21T08:54:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18b5c55640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18b01010f0 0x7f18b01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18b5c55640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18b0106170 0x7f18b0198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18b5c55640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18b019cf80 0x7f18b019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18b5c55640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01110e0 con 0x7f18b0106170 2024-04-21T08:54:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18b5c55640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0110f60 con 0x7f18b01010f0 2024-04-21T08:54:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18b5c55640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0111260 con 0x7f18b019cf80 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18b0106170 0x7f18b0198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18b0106170 0x7f18b0198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45716/0 (socket says 172.21.15.82:45716) 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18aeffd640 1 -- 172.21.15.82:0/2168050436 learned_addr learned my addr 172.21.15.82:0/2168050436 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18af7fe640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18b01010f0 0x7f18b01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18affff640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18b019cf80 0x7f18b019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18aeffd640 1 -- 172.21.15.82:0/2168050436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18b019cf80 msgr2=0x7f18b019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18aeffd640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18b019cf80 0x7f18b019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18aeffd640 1 -- 172.21.15.82:0/2168050436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18b01010f0 msgr2=0x7f18b01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18aeffd640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18b01010f0 0x7f18b01986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.202+0000 7f18aeffd640 1 -- 172.21.15.82:0/2168050436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b019da70 con 0x7f18b0106170 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.203+0000 7f18aeffd640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18b0106170 0x7f18b0198c00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f18a400e9f0 tx=0x7f18a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.203+0000 7f18acff9640 1 -- 172.21.15.82:0/2168050436 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a400ccc0 con 0x7f18b0106170 2024-04-21T08:54:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.203+0000 7f18acff9640 1 -- 172.21.15.82:0/2168050436 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a40054d0 con 0x7f18b0106170 2024-04-21T08:54:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.203+0000 7f18acff9640 1 -- 172.21.15.82:0/2168050436 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4010430 con 0x7f18b0106170 2024-04-21T08:54:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.203+0000 7f18b5c55640 1 -- 172.21.15.82:0/2168050436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b019dd60 con 0x7f18b0106170 2024-04-21T08:54:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.203+0000 7f18b5c55640 1 -- 172.21.15.82:0/2168050436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b01a5500 con 0x7f18b0106170 2024-04-21T08:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.205+0000 7f18acff9640 1 -- 172.21.15.82:0/2168050436 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18a40046c0 con 0x7f18b0106170 2024-04-21T08:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.205+0000 7f18b884a740 1 Processor -- start 2024-04-21T08:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.205+0000 7f18b884a740 1 -- start start 2024-04-21T08:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18b884a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cb4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18b884a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cb4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18acff9640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18980a3dc0 0x7f18980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18b884a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cc04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18b884a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e8b8c198d0 con 0x55e8b8cb58d0 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18b884a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e8b8bb6e80 con 0x55e8b8cb4b60 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18b884a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e8b8c15890 con 0x55e8b8cbe0e0 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cb4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18af7fe640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18980a3dc0 0x7f18980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cb4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45724/0 (socket says 172.21.15.82:45724) 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18affff640 1 -- 172.21.15.82:0/2859915965 learned_addr learned my addr 172.21.15.82:0/2859915965 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18aeffd640 1 --2- 172.21.15.82:0/2859915965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cb4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18affff640 1 --2- 172.21.15.82:0/2859915965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cc04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18acff9640 1 -- 172.21.15.82:0/2168050436 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f18a40d0f40 con 0x7f18b0106170 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18affff640 1 -- 172.21.15.82:0/2859915965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cbe0e0 msgr2=0x55e8b8cc04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18affff640 1 --2- 172.21.15.82:0/2859915965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cc04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18affff640 1 -- 172.21.15.82:0/2859915965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cb4b60 msgr2=0x55e8b8cb4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18affff640 1 --2- 172.21.15.82:0/2859915965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cb4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18affff640 1 -- 172.21.15.82:0/2859915965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8b8bd8320 con 0x55e8b8cb58d0 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18af7fe640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18980a3dc0 0x7f18980a6280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009bd0 tx=0x7f18a0005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f18affff640 1 --2- 172.21.15.82:0/2859915965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cb4620 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009890 tx=0x7f18a8009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7fee6347f93acb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.206+0000 7f1895ffb640 1 -- 172.21.15.82:0/2859915965 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8039040 con 0x55e8b8cb58d0 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.207+0000 7f1895ffb640 1 -- 172.21.15.82:0/2859915965 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a8032970 con 0x55e8b8cb58d0 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.207+0000 7f1895ffb640 1 -- 172.21.15.82:0/2859915965 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8004230 con 0x55e8b8cb58d0 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.207+0000 7f18b884a740 1 -- 172.21.15.82:0/2859915965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb58d0 msgr2=0x55e8b8cb4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.207+0000 7f18b884a740 1 --2- 172.21.15.82:0/2859915965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cb4620 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009890 tx=0x7f18a8009820 comp rx=0 tx=0).stop 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.207+0000 7f18b884a740 1 -- 172.21.15.82:0/2859915965 shutdown_connections 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.207+0000 7f18b884a740 1 --2- 172.21.15.82:0/2859915965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cc04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.207+0000 7f18b884a740 1 --2- 172.21.15.82:0/2859915965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cb4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.207+0000 7f18b884a740 1 --2- 172.21.15.82:0/2859915965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cb4620 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.207+0000 7f18b884a740 1 -- 172.21.15.82:0/2859915965 >> 172.21.15.82:0/2859915965 conn(0x55e8b8c7bc90 msgr2=0x55e8b8caf850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.207+0000 7f18b884a740 1 -- 172.21.15.82:0/2859915965 shutdown_connections 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.207+0000 7f18b884a740 1 -- 172.21.15.82:0/2859915965 wait complete. 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18b884a740 1 -- start start 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18b884a740 1 Processor -- start 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18b884a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cbd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18b884a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cbadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18b884a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cbb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18b884a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e8b8cc0db0 con 0x55e8b8cbe0e0 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18b884a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e8b8ba1eb0 con 0x55e8b8cb58d0 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18b884a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e8b8cc0f30 con 0x55e8b8cb4b60 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18b884a740 10 client.0 _ll_register_callbacks cb 0x55e8b8bceb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cbb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cbb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45732/0 (socket says 172.21.15.82:45732) 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18affff640 1 -- 172.21.15.82:0/1874508051 learned_addr learned my addr 172.21.15.82:0/1874508051 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18af7fe640 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cbd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18aeffd640 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cbadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18affff640 1 -- 172.21.15.82:0/1874508051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb4b60 msgr2=0x55e8b8cbd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18affff640 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cbd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18affff640 1 -- 172.21.15.82:0/1874508051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cb58d0 msgr2=0x55e8b8cbadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18affff640 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cbadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18affff640 1 -- 172.21.15.82:0/1874508051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8b892c210 con 0x55e8b8cbe0e0 2024-04-21T08:54:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.208+0000 7f18aeffd640 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cbadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f18affff640 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cbb330 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009620 tx=0x7f18a8032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0e513b8fb7ab60 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f188effd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f188effd640 1 -- 172.21.15.82:0/1874508051 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8052060 con 0x55e8b8cbe0e0 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f188effd640 1 -- 172.21.15.82:0/1874508051 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f18a80326c0 con 0x55e8b8cbe0e0 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f18b884a740 1 -- 172.21.15.82:0/1874508051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a8032970 con 0x55e8b8cbe0e0 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f188effd640 1 -- 172.21.15.82:0/1874508051 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f18a8059050 con 0x55e8b8cbe0e0 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f18b884a740 10 client.17628 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f18b884a740 1 -- 172.21.15.82:0/1874508051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e8b89f8620 con 0x55e8b8cbe0e0 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f188effd640 1 -- 172.21.15.82:0/1874508051 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a8032970 con 0x55e8b8cbe0e0 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f188effd640 1 -- 172.21.15.82:0/1874508051 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f18a803c020 con 0x55e8b8cbe0e0 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f18b884a740 10 client.17628 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f18b884a740 10 client.17628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f18b884a740 20 client.17628 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f18b884a740 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e8b8ccdee0 0x55e8b8cd02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f18b884a740 4 client.17628 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.209+0000 7f18b884a740 1 -- 172.21.15.82:0/1874508051 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8b8c2d880 con 0x55e8b8ccdee0 2024-04-21T08:54:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.210+0000 7f18af7fe640 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e8b8ccdee0 0x55e8b8cd02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.210+0000 7f188effd640 10 client.17628 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.210+0000 7f18af7fe640 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e8b8ccdee0 0x55e8b8cd02d0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51993ca60055d448 server_cookie=6392c23e61d38e4d in_seq=0 out_seq=0 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f188effd640 1 -- 172.21.15.82:0/1874508051 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e8b8c2d880 con 0x55e8b8ccdee0 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f188effd640 10 client.17628 handle_command_reply: tid=0 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f18b884a740 1 client.17628 shutdown 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f18b884a740 1 -- 172.21.15.82:0/1874508051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cbe0e0 msgr2=0x55e8b8cbb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f18b884a740 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cbb330 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009620 tx=0x7f18a8032420 comp rx=0 tx=0).stop 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f18b884a740 1 -- 172.21.15.82:0/1874508051 shutdown_connections 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f18b884a740 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e8b8ccdee0 0x55e8b8cd02d0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f18b884a740 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cbb330 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f18b884a740 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cbadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f188effd640 0 client.17628 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f18b884a740 1 --2- 172.21.15.82:0/1874508051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cbd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f18b884a740 1 -- 172.21.15.82:0/1874508051 >> 172.21.15.82:0/1874508051 conn(0x55e8b8c7bc90 msgr2=0x55e8b8cb15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f18b884a740 1 -- 172.21.15.82:0/1874508051 shutdown_connections 2024-04-21T08:54:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f18b884a740 1 -- 172.21.15.82:0/1874508051 wait complete. 2024-04-21T08:54:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.211+0000 7f18b884a740 20 client.17628 trim_cache size 0 max 16384 2024-04-21T08:54:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18b884a740 1 Processor -- start 2024-04-21T08:54:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18b884a740 1 -- start start 2024-04-21T08:54:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18b884a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cb4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18b884a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cd32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18b884a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cd3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18b884a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01af440 con 0x55e8b8cb4b60 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18b884a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b1070 con 0x55e8b8cbe0e0 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18b884a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01afb10 con 0x55e8b8cb58d0 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cb4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cb4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45748/0 (socket says 172.21.15.82:45748) 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18af7fe640 1 -- 172.21.15.82:0/546672604 learned_addr learned my addr 172.21.15.82:0/546672604 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18affff640 1 --2- 172.21.15.82:0/546672604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cd3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18aeffd640 1 --2- 172.21.15.82:0/546672604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cd32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18af7fe640 1 -- 172.21.15.82:0/546672604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb58d0 msgr2=0x55e8b8cd32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18af7fe640 1 --2- 172.21.15.82:0/546672604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cd32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18af7fe640 1 -- 172.21.15.82:0/546672604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cbe0e0 msgr2=0x55e8b8cd3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18af7fe640 1 --2- 172.21.15.82:0/546672604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cd3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18af7fe640 1 -- 172.21.15.82:0/546672604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8b8cb3e30 con 0x55e8b8cb4b60 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18affff640 1 --2- 172.21.15.82:0/546672604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cd3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18af7fe640 1 --2- 172.21.15.82:0/546672604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cb4080 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x55e8b8cc0f80 tx=0x7f18a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2c57278b6cd2a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18957fa640 1 -- 172.21.15.82:0/546672604 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a001f9c0 con 0x55e8b8cb4b60 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18957fa640 1 -- 172.21.15.82:0/546672604 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a002f070 con 0x55e8b8cb4b60 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.226+0000 7f18957fa640 1 -- 172.21.15.82:0/546672604 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a001d410 con 0x55e8b8cb4b60 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.227+0000 7f18b884a740 1 -- 172.21.15.82:0/546672604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 msgr2=0x55e8b8cb4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.227+0000 7f18b884a740 1 --2- 172.21.15.82:0/546672604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cb4080 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x55e8b8cc0f80 tx=0x7f18a0014000 comp rx=0 tx=0).stop 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.227+0000 7f18b884a740 1 -- 172.21.15.82:0/546672604 shutdown_connections 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.227+0000 7f18b884a740 1 --2- 172.21.15.82:0/546672604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cd3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.227+0000 7f18b884a740 1 --2- 172.21.15.82:0/546672604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cd32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.227+0000 7f18b884a740 1 --2- 172.21.15.82:0/546672604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cb4080 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.227+0000 7f18b884a740 1 -- 172.21.15.82:0/546672604 >> 172.21.15.82:0/546672604 conn(0x55e8b8c7bc90 msgr2=0x55e8b8cb96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.227+0000 7f18b884a740 1 -- 172.21.15.82:0/546672604 shutdown_connections 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.227+0000 7f18b884a740 1 -- 172.21.15.82:0/546672604 wait complete. 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.227+0000 7f18b884a740 1 -- start start 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18b884a740 1 Processor -- start 2024-04-21T08:54:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18b884a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cd3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18b884a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cd08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18b884a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cd0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18b884a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b0970 con 0x55e8b8cb4b60 2024-04-21T08:54:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18b884a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b0270 con 0x55e8b8cbe0e0 2024-04-21T08:54:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18b884a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01ad410 con 0x55e8b8cb58d0 2024-04-21T08:54:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18b884a740 10 client.0 _ll_register_callbacks cb 0x7f18a4004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cd3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cd3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45760/0 (socket says 172.21.15.82:45760) 2024-04-21T08:54:23.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18af7fe640 1 -- 172.21.15.82:0/3322126884 learned_addr learned my addr 172.21.15.82:0/3322126884 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18aeffd640 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cd08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18affff640 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cd0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18af7fe640 1 -- 172.21.15.82:0/3322126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb58d0 msgr2=0x55e8b8cd08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18af7fe640 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cd08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18af7fe640 1 -- 172.21.15.82:0/3322126884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cbe0e0 msgr2=0x55e8b8cd0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18af7fe640 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cd0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18af7fe640 1 -- 172.21.15.82:0/3322126884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8b8cd1350 con 0x55e8b8cb4b60 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f18af7fe640 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cd3310 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f18a0004620 tx=0x7f18a0029630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddab9f08e6386b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.228+0000 7f188effd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f188effd640 1 -- 172.21.15.82:0/3322126884 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0029250 con 0x55e8b8cb4b60 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f188effd640 1 -- 172.21.15.82:0/3322126884 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f18a001f9c0 con 0x55e8b8cb4b60 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f18b884a740 1 -- 172.21.15.82:0/3322126884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a002f070 con 0x55e8b8cb4b60 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f188effd640 1 -- 172.21.15.82:0/3322126884 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f18a0046050 con 0x55e8b8cb4b60 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f18b884a740 10 client.17634 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f18b884a740 1 -- 172.21.15.82:0/3322126884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e8b8d33de0 con 0x55e8b8cb4b60 2024-04-21T08:54:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f188effd640 1 -- 172.21.15.82:0/3322126884 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a002f070 con 0x55e8b8cb4b60 2024-04-21T08:54:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f188effd640 1 -- 172.21.15.82:0/3322126884 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f18a001dc60 con 0x55e8b8cb4b60 2024-04-21T08:54:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f18b884a740 10 client.17634 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f18b884a740 10 client.17634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f18b884a740 20 client.17634 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f18b884a740 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e8b8d345e0 0x55e8b8d36a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f18b884a740 4 client.17634 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:54:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f18b884a740 1 -- 172.21.15.82:0/3322126884 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8b8cb4850 con 0x55e8b8d345e0 2024-04-21T08:54:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.229+0000 7f18aeffd640 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e8b8d345e0 0x55e8b8d36a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.230+0000 7f18aeffd640 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e8b8d345e0 0x55e8b8d36a00 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7a9a2264529f79 server_cookie=7ec7e45f6fa6ac65 in_seq=0 out_seq=0 2024-04-21T08:54:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.230+0000 7f188effd640 10 client.17634 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f188effd640 1 -- 172.21.15.82:0/3322126884 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8b8cb4850 con 0x55e8b8d345e0 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f188effd640 10 client.17634 handle_command_reply: tid=0 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f18b884a740 1 client.17634 shutdown 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f18b884a740 1 -- 172.21.15.82:0/3322126884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 msgr2=0x55e8b8cd3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f18b884a740 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cd3310 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f18a0004620 tx=0x7f18a0029630 comp rx=0 tx=0).stop 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f18b884a740 1 -- 172.21.15.82:0/3322126884 shutdown_connections 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f18b884a740 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e8b8d345e0 0x55e8b8d36a00 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f18b884a740 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e8b8cbe0e0 0x55e8b8cd0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f18b884a740 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e8b8cb58d0 0x55e8b8cd08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f18b884a740 1 --2- 172.21.15.82:0/3322126884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e8b8cb4b60 0x55e8b8cd3310 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f18b884a740 1 -- 172.21.15.82:0/3322126884 >> 172.21.15.82:0/3322126884 conn(0x55e8b8c7bc90 msgr2=0x55e8b8cb9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f188effd640 0 client.17634 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f18b884a740 1 -- 172.21.15.82:0/3322126884 shutdown_connections 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f18b884a740 1 -- 172.21.15.82:0/3322126884 wait complete. 2024-04-21T08:54:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.238+0000 7f18b884a740 20 client.17634 trim_cache size 0 max 16384 2024-04-21T08:54:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:54:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:54:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:54:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:54:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.239+0000 7f18b5c55640 1 -- 172.21.15.82:0/2168050436 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18980a3dc0 msgr2=0x7f18980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.239+0000 7f18b5c55640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18980a3dc0 0x7f18980a6280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009bd0 tx=0x7f18a0005f90 comp rx=0 tx=0).stop 2024-04-21T08:54:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.239+0000 7f18b5c55640 1 -- 172.21.15.82:0/2168050436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18b0106170 msgr2=0x7f18b0198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.239+0000 7f18b5c55640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18b0106170 0x7f18b0198c00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f18a400e9f0 tx=0x7f18a400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.239+0000 7f18b5c55640 1 -- 172.21.15.82:0/2168050436 shutdown_connections 2024-04-21T08:54:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.239+0000 7f18b5c55640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18b019cf80 0x7f18b019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.239+0000 7f18b5c55640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f18980a3dc0 0x7f18980a6280 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.239+0000 7f18b5c55640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18b0106170 0x7f18b0198c00 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.239+0000 7f18b5c55640 1 --2- 172.21.15.82:0/2168050436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18b01010f0 0x7f18b01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.239+0000 7f18b5c55640 1 -- 172.21.15.82:0/2168050436 >> 172.21.15.82:0/2168050436 conn(0x7f18b0077fd0 msgr2=0x7f18b00698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.239+0000 7f18b5c55640 1 -- 172.21.15.82:0/2168050436 shutdown_connections 2024-04-21T08:54:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:23.239+0000 7f18b5c55640 1 -- 172.21.15.82:0/2168050436 wait complete. 2024-04-21T08:54:24.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:24 smithi106 ceph-mon[31072]: pgmap v226: 129 pgs: 129 active+clean; 867 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 538 op/s 2024-04-21T08:54:24.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:24 smithi134 ceph-mon[31772]: pgmap v226: 129 pgs: 129 active+clean; 867 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 538 op/s 2024-04-21T08:54:24.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:24 smithi082 ceph-mon[27967]: pgmap v226: 129 pgs: 129 active+clean; 867 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 538 op/s 2024-04-21T08:54:25.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:54:25.972 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:54:26.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.075+0000 7fc98f92c640 1 -- 172.21.15.82:0/4100799267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc988101a30 msgr2=0x7fc988105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.075+0000 7fc98f92c640 1 --2- 172.21.15.82:0/4100799267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc988101a30 0x7fc988105a70 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc978009900 tx=0x7fc97802bc30 comp rx=0 tx=0).stop 2024-04-21T08:54:26.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.075+0000 7fc98f92c640 1 -- 172.21.15.82:0/4100799267 shutdown_connections 2024-04-21T08:54:26.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.075+0000 7fc98f92c640 1 --2- 172.21.15.82:0/4100799267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc988106150 0x7fc98810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.075+0000 7fc98f92c640 1 --2- 172.21.15.82:0/4100799267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc988101a30 0x7fc988105a70 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.075+0000 7fc98f92c640 1 --2- 172.21.15.82:0/4100799267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc988101080 0x7fc988101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.075+0000 7fc98f92c640 1 -- 172.21.15.82:0/4100799267 >> 172.21.15.82:0/4100799267 conn(0x7fc988078000 msgr2=0x7fc988078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.075+0000 7fc98f92c640 1 -- 172.21.15.82:0/4100799267 shutdown_connections 2024-04-21T08:54:26.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.075+0000 7fc98f92c640 1 -- 172.21.15.82:0/4100799267 wait complete. 2024-04-21T08:54:26.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98f92c640 1 Processor -- start 2024-04-21T08:54:26.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98f92c640 1 -- start start 2024-04-21T08:54:26.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98f92c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc988101080 0x7fc988199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98f92c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc988106150 0x7fc9881999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98f92c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc988199f60 0x7fc9881a4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98f92c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc988111070 con 0x7fc988199f60 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98f92c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc988110ef0 con 0x7fc988106150 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98f92c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9881111f0 con 0x7fc988101080 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98dec9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc988199f60 0x7fc9881a4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98dec9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc988199f60 0x7fc9881a4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45778/0 (socket says 172.21.15.82:45778) 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98cec7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc988106150 0x7fc9881999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98dec9640 1 -- 172.21.15.82:0/808033256 learned_addr learned my addr 172.21.15.82:0/808033256 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98d6c8640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc988101080 0x7fc988199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98dec9640 1 -- 172.21.15.82:0/808033256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc988101080 msgr2=0x7fc988199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98dec9640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc988101080 0x7fc988199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98dec9640 1 -- 172.21.15.82:0/808033256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc988106150 msgr2=0x7fc9881999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98dec9640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc988106150 0x7fc9881999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98dec9640 1 -- 172.21.15.82:0/808033256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9881a4ce0 con 0x7fc988199f60 2024-04-21T08:54:26.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.076+0000 7fc98cec7640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc988106150 0x7fc9881999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:26.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.077+0000 7fc98dec9640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc988199f60 0x7fc9881a4510 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc98000eb10 tx=0x7fc98000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.077+0000 7fc97e7fc640 1 -- 172.21.15.82:0/808033256 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98000ce20 con 0x7fc988199f60 2024-04-21T08:54:26.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.077+0000 7fc98f92c640 1 -- 172.21.15.82:0/808033256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9881a4fd0 con 0x7fc988199f60 2024-04-21T08:54:26.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.077+0000 7fc97e7fc640 1 -- 172.21.15.82:0/808033256 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc980005020 con 0x7fc988199f60 2024-04-21T08:54:26.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.077+0000 7fc97e7fc640 1 -- 172.21.15.82:0/808033256 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980010690 con 0x7fc988199f60 2024-04-21T08:54:26.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.077+0000 7fc98f92c640 1 -- 172.21.15.82:0/808033256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9881a54a0 con 0x7fc988199f60 2024-04-21T08:54:26.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.079+0000 7fc97e7fc640 1 -- 172.21.15.82:0/808033256 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc980005510 con 0x7fc988199f60 2024-04-21T08:54:26.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.079+0000 7fc9925f0740 1 Processor -- start 2024-04-21T08:54:26.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.079+0000 7fc9925f0740 1 -- start start 2024-04-21T08:54:26.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc9925f0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 0x55d0b342c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc9925f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342cea0 0x55d0b3435540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc9925f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b3435a80 0x55d0b3437e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc9925f0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d0b33911e0 con 0x55d0b342c580 2024-04-21T08:54:26.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc9925f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d0b332e390 con 0x55d0b3435a80 2024-04-21T08:54:26.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc9925f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0b338d1a0 con 0x55d0b342cea0 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98cec7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 0x55d0b342c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98cec7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 0x55d0b342c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45794/0 (socket says 172.21.15.82:45794) 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98cec7640 1 -- 172.21.15.82:0/1247914347 learned_addr learned my addr 172.21.15.82:0/1247914347 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc97e7fc640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc9740a3d20 0x7fc9740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98cec7640 1 --2- 172.21.15.82:0/1247914347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b3435a80 0x55d0b3437e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98d6c8640 1 --2- 172.21.15.82:0/1247914347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342cea0 0x55d0b3435540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98dec9640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc9740a3d20 0x7fc9740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98cec7640 1 -- 172.21.15.82:0/1247914347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342cea0 msgr2=0x55d0b3435540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98cec7640 1 --2- 172.21.15.82:0/1247914347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342cea0 0x55d0b3435540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98cec7640 1 -- 172.21.15.82:0/1247914347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b3435a80 msgr2=0x55d0b3437e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98cec7640 1 --2- 172.21.15.82:0/1247914347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b3435a80 0x55d0b3437e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98cec7640 1 -- 172.21.15.82:0/1247914347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b316d7d0 con 0x55d0b342c580 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98d6c8640 1 --2- 172.21.15.82:0/1247914347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342cea0 0x55d0b3435540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:26.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc97e7fc640 1 -- 172.21.15.82:0/808033256 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fc9800d1100 con 0x7fc988199f60 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98cec7640 1 --2- 172.21.15.82:0/1247914347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 0x55d0b342c960 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc9780096c0 tx=0x7fc978005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5951caf42c812a65 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc98dec9640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc9740a3d20 0x7fc9740a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc9800ce4f0 tx=0x7fc98006e6d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc9737fe640 1 -- 172.21.15.82:0/1247914347 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97802c890 con 0x55d0b342c580 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc9737fe640 1 -- 172.21.15.82:0/1247914347 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc97802ca30 con 0x55d0b342c580 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.080+0000 7fc9737fe640 1 -- 172.21.15.82:0/1247914347 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9780436a0 con 0x55d0b342c580 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.081+0000 7fc9925f0740 1 -- 172.21.15.82:0/1247914347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 msgr2=0x55d0b342c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.081+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1247914347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 0x55d0b342c960 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc9780096c0 tx=0x7fc978005e40 comp rx=0 tx=0).stop 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.081+0000 7fc9925f0740 1 -- 172.21.15.82:0/1247914347 shutdown_connections 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.081+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1247914347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b3435a80 0x55d0b3437e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.081+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1247914347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342cea0 0x55d0b3435540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.081+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1247914347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 0x55d0b342c960 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.081+0000 7fc9925f0740 1 -- 172.21.15.82:0/1247914347 >> 172.21.15.82:0/1247914347 conn(0x55d0b3426cb0 msgr2=0x55d0b3427090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.081+0000 7fc9925f0740 1 -- 172.21.15.82:0/1247914347 shutdown_connections 2024-04-21T08:54:26.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.081+0000 7fc9925f0740 1 -- 172.21.15.82:0/1247914347 wait complete. 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc9925f0740 1 -- start start 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc9925f0740 1 Processor -- start 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc9925f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342c580 0x55d0b3432aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc9925f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b342cea0 0x55d0b3432fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc9925f0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 0x55d0b3433520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc9925f0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d0b3438810 con 0x55d0b3435a80 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc9925f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d0b334a2f0 con 0x55d0b342cea0 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc9925f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0b3438990 con 0x55d0b342c580 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc9925f0740 10 client.0 _ll_register_callbacks cb 0x55d0b3357580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc98cec7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 0x55d0b3433520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc98cec7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 0x55d0b3433520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45804/0 (socket says 172.21.15.82:45804) 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc98cec7640 1 -- 172.21.15.82:0/4085930520 learned_addr learned my addr 172.21.15.82:0/4085930520 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc98d6c8640 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b342cea0 0x55d0b3432fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc98cec7640 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342c580 0x55d0b3432aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc98cec7640 1 -- 172.21.15.82:0/4085930520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342c580 msgr2=0x55d0b3432aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc98cec7640 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342c580 0x55d0b3432aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc98cec7640 1 -- 172.21.15.82:0/4085930520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b342cea0 msgr2=0x55d0b3432fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc98cec7640 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b342cea0 0x55d0b3432fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc98cec7640 1 -- 172.21.15.82:0/4085930520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b30a33f0 con 0x55d0b3435a80 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.082+0000 7fc98d6c8640 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b342cea0 0x55d0b3432fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc98cec7640 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 0x55d0b3433520 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc97802c170 tx=0x7fc978003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7110fdaae0b990a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc970ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc970ff9640 1 -- 172.21.15.82:0/4085930520 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97802cbe0 con 0x55d0b3435a80 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc970ff9640 1 -- 172.21.15.82:0/4085930520 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fc978043440 con 0x55d0b3435a80 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc9925f0740 1 -- 172.21.15.82:0/4085930520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc97802ca30 con 0x55d0b3435a80 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc970ff9640 1 -- 172.21.15.82:0/4085930520 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc978043640 con 0x55d0b3435a80 2024-04-21T08:54:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc9925f0740 10 client.17646 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc9925f0740 1 -- 172.21.15.82:0/4085930520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d0b30630f0 con 0x55d0b3435a80 2024-04-21T08:54:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc970ff9640 1 -- 172.21.15.82:0/4085930520 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc97802ca30 con 0x55d0b3435a80 2024-04-21T08:54:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc970ff9640 1 -- 172.21.15.82:0/4085930520 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7fc978034770 con 0x55d0b3435a80 2024-04-21T08:54:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc9925f0740 10 client.17646 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc9925f0740 10 client.17646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc9925f0740 20 client.17646 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc9925f0740 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0b3445890 0x55d0b3447c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc9925f0740 4 client.17646 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc9925f0740 1 -- 172.21.15.82:0/4085930520 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0b339ec00 con 0x55d0b3445890 2024-04-21T08:54:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.083+0000 7fc98d6c8640 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0b3445890 0x55d0b3447c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.084+0000 7fc970ff9640 10 client.17646 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.084+0000 7fc98d6c8640 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0b3445890 0x55d0b3447c80 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fce15ef26cd4e9a server_cookie=4811775e4522ebb1 in_seq=0 out_seq=0 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc970ff9640 1 -- 172.21.15.82:0/4085930520 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0b339ec00 con 0x55d0b3445890 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc970ff9640 10 client.17646 handle_command_reply: tid=0 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc9925f0740 1 client.17646 shutdown 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc9925f0740 1 -- 172.21.15.82:0/4085930520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 msgr2=0x55d0b3433520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc9925f0740 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 0x55d0b3433520 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc97802c170 tx=0x7fc978003da0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc9925f0740 1 -- 172.21.15.82:0/4085930520 shutdown_connections 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc9925f0740 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0b3445890 0x55d0b3447c80 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc9925f0740 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 0x55d0b3433520 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc970ff9640 0 client.17646 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc9925f0740 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b342cea0 0x55d0b3432fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc9925f0740 1 --2- 172.21.15.82:0/4085930520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342c580 0x55d0b3432aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc9925f0740 1 -- 172.21.15.82:0/4085930520 >> 172.21.15.82:0/4085930520 conn(0x55d0b3426cb0 msgr2=0x55d0b3428a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc9925f0740 1 -- 172.21.15.82:0/4085930520 shutdown_connections 2024-04-21T08:54:26.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc9925f0740 1 -- 172.21.15.82:0/4085930520 wait complete. 2024-04-21T08:54:26.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.085+0000 7fc9925f0740 20 client.17646 trim_cache size 0 max 16384 2024-04-21T08:54:26.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.103+0000 7fc9925f0740 1 Processor -- start 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.103+0000 7fc9925f0740 1 -- start start 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.103+0000 7fc9925f0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 0x55d0b34351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.103+0000 7fc9925f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342cea0 0x55d0b3441700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.103+0000 7fc9925f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b3435a80 0x55d0b3425bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.103+0000 7fc9925f0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc9881af430 con 0x55d0b342c580 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.103+0000 7fc9925f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9881b1060 con 0x55d0b3435a80 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.103+0000 7fc9925f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9881afb00 con 0x55d0b342cea0 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc98cec7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 0x55d0b34351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc98cec7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 0x55d0b34351d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45816/0 (socket says 172.21.15.82:45816) 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc98cec7640 1 -- 172.21.15.82:0/1593180332 learned_addr learned my addr 172.21.15.82:0/1593180332 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc98d6c8640 1 --2- 172.21.15.82:0/1593180332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342cea0 0x55d0b3441700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc98cec7640 1 -- 172.21.15.82:0/1593180332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342cea0 msgr2=0x55d0b3441700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc98cec7640 1 --2- 172.21.15.82:0/1593180332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342cea0 0x55d0b3441700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc98cec7640 1 -- 172.21.15.82:0/1593180332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b3435a80 msgr2=0x55d0b3425bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc98cec7640 1 --2- 172.21.15.82:0/1593180332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b3435a80 0x55d0b3425bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc98cec7640 1 -- 172.21.15.82:0/1593180332 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b3437e70 con 0x55d0b342c580 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc98cec7640 1 --2- 172.21.15.82:0/1593180332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 0x55d0b34351d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc978005ec0 tx=0x7fc9780043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21eb410f30827ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc972ffd640 1 -- 172.21.15.82:0/1593180332 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978009520 con 0x55d0b342c580 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc972ffd640 1 -- 172.21.15.82:0/1593180332 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc978052070 con 0x55d0b342c580 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.104+0000 7fc972ffd640 1 -- 172.21.15.82:0/1593180332 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978003e10 con 0x55d0b342c580 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.105+0000 7fc9925f0740 1 -- 172.21.15.82:0/1593180332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 msgr2=0x55d0b34351d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.105+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1593180332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 0x55d0b34351d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc978005ec0 tx=0x7fc9780043e0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.105+0000 7fc9925f0740 1 -- 172.21.15.82:0/1593180332 shutdown_connections 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.105+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1593180332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b3435a80 0x55d0b3425bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.105+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1593180332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342cea0 0x55d0b3441700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.105+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1593180332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b342c580 0x55d0b34351d0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.105+0000 7fc9925f0740 1 -- 172.21.15.82:0/1593180332 >> 172.21.15.82:0/1593180332 conn(0x55d0b3426cb0 msgr2=0x55d0b34254e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.105+0000 7fc9925f0740 1 -- 172.21.15.82:0/1593180332 shutdown_connections 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.105+0000 7fc9925f0740 1 -- 172.21.15.82:0/1593180332 wait complete. 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.105+0000 7fc9925f0740 1 -- start start 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.105+0000 7fc9925f0740 1 Processor -- start 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc9925f0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342c580 0x55d0b343dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc9925f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b342cea0 0x55d0b343e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc9925f0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 0x55d0b3439030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc9925f0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc9881b0960 con 0x55d0b3435a80 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc9925f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9881b0260 con 0x55d0b342cea0 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc9925f0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9881ad400 con 0x55d0b342c580 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc9925f0740 10 client.0 _ll_register_callbacks cb 0x7fc9780042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc98cec7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 0x55d0b3439030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc98cec7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 0x55d0b3439030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45818/0 (socket says 172.21.15.82:45818) 2024-04-21T08:54:26.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc98cec7640 1 -- 172.21.15.82:0/1115825602 learned_addr learned my addr 172.21.15.82:0/1115825602 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc98cec7640 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342c580 0x55d0b343dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc98d6c8640 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b342cea0 0x55d0b343e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc98cec7640 1 -- 172.21.15.82:0/1115825602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342c580 msgr2=0x55d0b343dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc98cec7640 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342c580 0x55d0b343dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc98cec7640 1 -- 172.21.15.82:0/1115825602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b342cea0 msgr2=0x55d0b343e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc98cec7640 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b342cea0 0x55d0b343e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc98cec7640 1 -- 172.21.15.82:0/1115825602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b3439860 con 0x55d0b3435a80 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc98d6c8640 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b342cea0 0x55d0b343e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc98cec7640 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 0x55d0b3439030 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc97802c170 tx=0x7fc97802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b7009e82389666 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc970ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc970ff9640 1 -- 172.21.15.82:0/1115825602 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97802c9d0 con 0x55d0b3435a80 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc970ff9640 1 -- 172.21.15.82:0/1115825602 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fc97806c070 con 0x55d0b3435a80 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc9925f0740 1 -- 172.21.15.82:0/1115825602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc978052070 con 0x55d0b3435a80 2024-04-21T08:54:26.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc970ff9640 1 -- 172.21.15.82:0/1115825602 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc97802cb70 con 0x55d0b3435a80 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.106+0000 7fc9925f0740 10 client.17652 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.107+0000 7fc9925f0740 1 -- 172.21.15.82:0/1115825602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d0b34ab6a0 con 0x55d0b3435a80 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.107+0000 7fc970ff9640 1 -- 172.21.15.82:0/1115825602 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc978052070 con 0x55d0b3435a80 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.107+0000 7fc970ff9640 1 -- 172.21.15.82:0/1115825602 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7fc9780437e0 con 0x55d0b3435a80 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.107+0000 7fc9925f0740 10 client.17652 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.107+0000 7fc9925f0740 10 client.17652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.107+0000 7fc9925f0740 20 client.17652 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.107+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0b3445890 0x55d0b34adc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.107+0000 7fc9925f0740 4 client.17652 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.107+0000 7fc9925f0740 1 -- 172.21.15.82:0/1115825602 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d0b3398a70 con 0x55d0b3445890 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.107+0000 7fc98d6c8640 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0b3445890 0x55d0b34adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.107+0000 7fc98d6c8640 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0b3445890 0x55d0b34adc40 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a5401e1993f8dfd server_cookie=9f17342ddc6b0e07 in_seq=0 out_seq=0 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.107+0000 7fc970ff9640 10 client.17652 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc970ff9640 1 -- 172.21.15.82:0/1115825602 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d0b3398a70 con 0x55d0b3445890 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc970ff9640 10 client.17652 handle_command_reply: tid=0 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc9925f0740 1 client.17652 shutdown 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc9925f0740 1 -- 172.21.15.82:0/1115825602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 msgr2=0x55d0b3439030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 0x55d0b3439030 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc97802c170 tx=0x7fc97802bc30 comp rx=0 tx=0).stop 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc9925f0740 1 -- 172.21.15.82:0/1115825602 shutdown_connections 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d0b3445890 0x55d0b34adc40 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0b3435a80 0x55d0b3439030 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d0b342cea0 0x55d0b343e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc970ff9640 0 client.17652 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc9925f0740 1 --2- 172.21.15.82:0/1115825602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0b342c580 0x55d0b343dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc9925f0740 1 -- 172.21.15.82:0/1115825602 >> 172.21.15.82:0/1115825602 conn(0x55d0b3426cb0 msgr2=0x55d0b3425d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc9925f0740 1 -- 172.21.15.82:0/1115825602 shutdown_connections 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc9925f0740 1 -- 172.21.15.82:0/1115825602 wait complete. 2024-04-21T08:54:26.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc9925f0740 20 client.17652 trim_cache size 0 max 16384 2024-04-21T08:54:26.116 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:54:26.116 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:54:26.116 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:54:26.116 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:54:26.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc98f92c640 1 -- 172.21.15.82:0/808033256 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc9740a3d20 msgr2=0x7fc9740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc98f92c640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc9740a3d20 0x7fc9740a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc9800ce4f0 tx=0x7fc98006e6d0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc98f92c640 1 -- 172.21.15.82:0/808033256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc988199f60 msgr2=0x7fc9881a4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.108+0000 7fc98f92c640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc988199f60 0x7fc9881a4510 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc98000eb10 tx=0x7fc98000ef10 comp rx=0 tx=0).stop 2024-04-21T08:54:26.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.109+0000 7fc98f92c640 1 -- 172.21.15.82:0/808033256 shutdown_connections 2024-04-21T08:54:26.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.109+0000 7fc98f92c640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc9740a3d20 0x7fc9740a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.109+0000 7fc98f92c640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc988199f60 0x7fc9881a4510 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.109+0000 7fc98f92c640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc988106150 0x7fc9881999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.109+0000 7fc98f92c640 1 --2- 172.21.15.82:0/808033256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc988101080 0x7fc988199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.109+0000 7fc98f92c640 1 -- 172.21.15.82:0/808033256 >> 172.21.15.82:0/808033256 conn(0x7fc988078000 msgr2=0x7fc9880fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.109+0000 7fc98f92c640 1 -- 172.21.15.82:0/808033256 shutdown_connections 2024-04-21T08:54:26.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.109+0000 7fc98f92c640 1 -- 172.21.15.82:0/808033256 wait complete. 2024-04-21T08:54:26.119 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:54:26.119 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:54:26.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.225+0000 7f1e5bc06640 1 -- 172.21.15.82:0/4017299385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54101b60 msgr2=0x7f1e5410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.225+0000 7f1e5bc06640 1 --2- 172.21.15.82:0/4017299385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54101b60 0x7f1e5410dc90 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009cd0 tx=0x7f1e4c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 -- 172.21.15.82:0/4017299385 shutdown_connections 2024-04-21T08:54:26.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 --2- 172.21.15.82:0/4017299385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e54101b60 0x7f1e5410dc90 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 --2- 172.21.15.82:0/4017299385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e541011a0 0x7f1e54101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 --2- 172.21.15.82:0/4017299385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e54105e30 0x7f1e54106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 -- 172.21.15.82:0/4017299385 >> 172.21.15.82:0/4017299385 conn(0x7f1e54077e30 msgr2=0x7f1e54078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 -- 172.21.15.82:0/4017299385 shutdown_connections 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 -- 172.21.15.82:0/4017299385 wait complete. 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 Processor -- start 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 -- start start 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e541011a0 0x7f1e5419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e54101b60 0x7f1e5419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e54105e30 0x7f1e54198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1e541110e0 con 0x7f1e541011a0 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e54110f60 con 0x7f1e54105e30 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5bc06640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e54111260 con 0x7f1e54101b60 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e599a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e541011a0 0x7f1e5419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e599a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e541011a0 0x7f1e5419e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45844/0 (socket says 172.21.15.82:45844) 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e599a2640 1 -- 172.21.15.82:0/3962483775 learned_addr learned my addr 172.21.15.82:0/3962483775 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.226+0000 7f1e5a1a3640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e54105e30 0x7f1e54198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.227+0000 7f1e591a1640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e54101b60 0x7f1e5419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.227+0000 7f1e599a2640 1 -- 172.21.15.82:0/3962483775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e54101b60 msgr2=0x7f1e5419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.227+0000 7f1e599a2640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e54101b60 0x7f1e5419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.227+0000 7f1e599a2640 1 -- 172.21.15.82:0/3962483775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e54105e30 msgr2=0x7f1e54198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.227+0000 7f1e599a2640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e54105e30 0x7f1e54198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.227+0000 7f1e599a2640 1 -- 172.21.15.82:0/3962483775 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e54198d10 con 0x7f1e541011a0 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.227+0000 7f1e599a2640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e541011a0 0x7f1e5419e3e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000e9f0 tx=0x7f1e5000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.227+0000 7f1e4affd640 1 -- 172.21.15.82:0/3962483775 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5000cd80 con 0x7f1e541011a0 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.227+0000 7f1e5bc06640 1 -- 172.21.15.82:0/3962483775 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e54199000 con 0x7f1e541011a0 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.227+0000 7f1e4affd640 1 -- 172.21.15.82:0/3962483775 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e500054d0 con 0x7f1e541011a0 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.227+0000 7f1e4affd640 1 -- 172.21.15.82:0/3962483775 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50010690 con 0x7f1e541011a0 2024-04-21T08:54:26.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.227+0000 7f1e5bc06640 1 -- 172.21.15.82:0/3962483775 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e541a52a0 con 0x7f1e541011a0 2024-04-21T08:54:26.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.228+0000 7f1e5bc06640 1 -- 172.21.15.82:0/3962483775 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e54068de0 con 0x7f1e541011a0 2024-04-21T08:54:26.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.229+0000 7f1e4affd640 1 -- 172.21.15.82:0/3962483775 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1e50004e90 con 0x7f1e541011a0 2024-04-21T08:54:26.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.230+0000 7f1e4affd640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1e400a3d20 0x7f1e400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.230+0000 7f1e591a1640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1e400a3d20 0x7f1e400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.230+0000 7f1e4affd640 1 -- 172.21.15.82:0/3962483775 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f1e500d1040 con 0x7f1e541011a0 2024-04-21T08:54:26.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.230+0000 7f1e591a1640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1e400a3d20 0x7f1e400a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e54105690 tx=0x7f1e44005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.234+0000 7f1e4affd640 1 -- 172.21.15.82:0/3962483775 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e5009d2d0 con 0x7f1e541011a0 2024-04-21T08:54:26.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:26 smithi082 ceph-mon[27967]: pgmap v227: 129 pgs: 129 active+clean; 867 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 538 op/s 2024-04-21T08:54:26.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:26 smithi134 ceph-mon[31772]: pgmap v227: 129 pgs: 129 active+clean; 867 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 538 op/s 2024-04-21T08:54:26.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.485+0000 7f1e5bc06640 1 -- 172.21.15.82:0/3962483775 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e54199ac0 con 0x7f1e541011a0 2024-04-21T08:54:26.487 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:54:26.487 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:53:29.845577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:54:26.487 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 40 2024-04-21T08:54:26.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.487+0000 7f1e4affd640 1 -- 172.21.15.82:0/3962483775 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11953 (secure 0 0 0) 0x7f1e5009cbd0 con 0x7f1e541011a0 2024-04-21T08:54:26.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.490+0000 7f1e5bc06640 1 -- 172.21.15.82:0/3962483775 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1e400a3d20 msgr2=0x7f1e400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.490+0000 7f1e5bc06640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1e400a3d20 0x7f1e400a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e54105690 tx=0x7f1e44005f90 comp rx=0 tx=0).stop 2024-04-21T08:54:26.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.490+0000 7f1e5bc06640 1 -- 172.21.15.82:0/3962483775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e541011a0 msgr2=0x7f1e5419e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.490+0000 7f1e5bc06640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e541011a0 0x7f1e5419e3e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000e9f0 tx=0x7f1e5000edf0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.490+0000 7f1e5bc06640 1 -- 172.21.15.82:0/3962483775 shutdown_connections 2024-04-21T08:54:26.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.490+0000 7f1e5bc06640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e54105e30 0x7f1e54198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.490+0000 7f1e5bc06640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e54101b60 0x7f1e5419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.490+0000 7f1e5bc06640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f1e400a3d20 0x7f1e400a61e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.490+0000 7f1e5bc06640 1 --2- 172.21.15.82:0/3962483775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e541011a0 0x7f1e5419e3e0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.490+0000 7f1e5bc06640 1 -- 172.21.15.82:0/3962483775 >> 172.21.15.82:0/3962483775 conn(0x7f1e54077e30 msgr2=0x7f1e54069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.490+0000 7f1e5bc06640 1 -- 172.21.15.82:0/3962483775 shutdown_connections 2024-04-21T08:54:26.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.490+0000 7f1e5bc06640 1 -- 172.21.15.82:0/3962483775 wait complete. 2024-04-21T08:54:26.510 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:54:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:26 smithi106 ceph-mon[31072]: pgmap v227: 129 pgs: 129 active+clean; 867 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 538 op/s 2024-04-21T08:54:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.674+0000 7ff42fe93640 1 -- 172.21.15.82:0/173565592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff428101090 msgr2=0x7ff428101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.674+0000 7ff42fe93640 1 --2- 172.21.15.82:0/173565592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff428101090 0x7ff428101470 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff420009860 tx=0x7ff42002bb70 comp rx=0 tx=0).stop 2024-04-21T08:54:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.675+0000 7ff42fe93640 1 -- 172.21.15.82:0/173565592 shutdown_connections 2024-04-21T08:54:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.675+0000 7ff42fe93640 1 --2- 172.21.15.82:0/173565592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff428106160 0x7ff42810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.675+0000 7ff42fe93640 1 --2- 172.21.15.82:0/173565592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff428101a40 0x7ff428105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.675+0000 7ff42fe93640 1 --2- 172.21.15.82:0/173565592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff428101090 0x7ff428101470 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.675+0000 7ff42fe93640 1 -- 172.21.15.82:0/173565592 >> 172.21.15.82:0/173565592 conn(0x7ff428077f50 msgr2=0x7ff428078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.677+0000 7ff42fe93640 1 -- 172.21.15.82:0/173565592 shutdown_connections 2024-04-21T08:54:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.678+0000 7ff42fe93640 1 -- 172.21.15.82:0/173565592 wait complete. 2024-04-21T08:54:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.679+0000 7ff42fe93640 1 Processor -- start 2024-04-21T08:54:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.679+0000 7ff42fe93640 1 -- start start 2024-04-21T08:54:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.680+0000 7ff42fe93640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff428101090 0x7ff4280738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.680+0000 7ff42fe93640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff428106160 0x7ff42806d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.680+0000 7ff42fe93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff42806deb0 0x7ff42806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.680+0000 7ff42fe93640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff42806c230 con 0x7ff428106160 2024-04-21T08:54:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.680+0000 7ff42fe93640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff42806c0b0 con 0x7ff428101090 2024-04-21T08:54:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.680+0000 7ff42fe93640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff42806c3b0 con 0x7ff42806deb0 2024-04-21T08:54:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.680+0000 7ff42e430640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff42806deb0 0x7ff42806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.680+0000 7ff42e430640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff42806deb0 0x7ff42806e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:43268/0 (socket says 172.21.15.82:43268) 2024-04-21T08:54:26.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.680+0000 7ff42e430640 1 -- 172.21.15.82:0/520254752 learned_addr learned my addr 172.21.15.82:0/520254752 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.681+0000 7ff42dc2f640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff428101090 0x7ff4280738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.682+0000 7ff42d42e640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff428106160 0x7ff42806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.683+0000 7ff42d42e640 1 -- 172.21.15.82:0/520254752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff42806deb0 msgr2=0x7ff42806e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.683+0000 7ff42d42e640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff42806deb0 0x7ff42806e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.683+0000 7ff42d42e640 1 -- 172.21.15.82:0/520254752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff428101090 msgr2=0x7ff4280738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.683+0000 7ff42d42e640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff428101090 0x7ff4280738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.683+0000 7ff42d42e640 1 -- 172.21.15.82:0/520254752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4281ab420 con 0x7ff428106160 2024-04-21T08:54:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.683+0000 7ff42d42e640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff428106160 0x7ff42806d970 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff41801e990 tx=0x7ff41801ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.685+0000 7ff41effd640 1 -- 172.21.15.82:0/520254752 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418028070 con 0x7ff428106160 2024-04-21T08:54:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.685+0000 7ff41effd640 1 -- 172.21.15.82:0/520254752 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff418021470 con 0x7ff428106160 2024-04-21T08:54:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.685+0000 7ff41effd640 1 -- 172.21.15.82:0/520254752 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418020630 con 0x7ff428106160 2024-04-21T08:54:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.687+0000 7ff42fe93640 1 -- 172.21.15.82:0/520254752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4281ab710 con 0x7ff428106160 2024-04-21T08:54:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.687+0000 7ff42fe93640 1 -- 172.21.15.82:0/520254752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4281abbe0 con 0x7ff428106160 2024-04-21T08:54:26.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.691+0000 7ff432b65740 1 Processor -- start 2024-04-21T08:54:26.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.692+0000 7ff432b65740 1 -- start start 2024-04-21T08:54:26.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.692+0000 7ff41effd640 1 -- 172.21.15.82:0/520254752 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff41802b070 con 0x7ff428106160 2024-04-21T08:54:26.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.692+0000 7ff432b65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 0x564d082868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.692+0000 7ff432b65740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d08286e80 0x564d0828f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.692+0000 7ff432b65740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828fa00 0x564d08291e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.692+0000 7ff432b65740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564d081eb1f0 con 0x564d08286e80 2024-04-21T08:54:26.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.692+0000 7ff432b65740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564d081884b0 con 0x564d0828fa00 2024-04-21T08:54:26.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.692+0000 7ff432b65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564d081e71b0 con 0x564d082864d0 2024-04-21T08:54:26.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.692+0000 7ff42dc2f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d08286e80 0x564d0828f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.692+0000 7ff42dc2f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d08286e80 0x564d0828f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45878/0 (socket says 172.21.15.82:45878) 2024-04-21T08:54:26.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.692+0000 7ff42dc2f640 1 -- 172.21.15.82:0/2188749575 learned_addr learned my addr 172.21.15.82:0/2188749575 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.693+0000 7ff42d42e640 1 --2- 172.21.15.82:0/2188749575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828fa00 0x564d08291e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.693+0000 7ff41effd640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff4140a3ef0 0x7ff4140a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.694+0000 7ff41effd640 1 -- 172.21.15.82:0/520254752 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ff4180e1390 con 0x7ff428106160 2024-04-21T08:54:26.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.694+0000 7ff42dc2f640 1 -- 172.21.15.82:0/2188749575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 msgr2=0x564d082868b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:54:26.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.694+0000 7ff42dc2f640 1 --2- 172.21.15.82:0/2188749575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 0x564d082868b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.694+0000 7ff42dc2f640 1 -- 172.21.15.82:0/2188749575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828fa00 msgr2=0x564d08291e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.694+0000 7ff42dc2f640 1 --2- 172.21.15.82:0/2188749575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828fa00 0x564d08291e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.694+0000 7ff42dc2f640 1 -- 172.21.15.82:0/2188749575 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0817a1b0 con 0x564d08286e80 2024-04-21T08:54:26.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.695+0000 7ff42dc2f640 1 --2- 172.21.15.82:0/2188749575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d08286e80 0x564d0828f430 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff420000c00 tx=0x7ff420034570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b1c9d1ffac8dac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.695+0000 7ff42e430640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff4140a3ef0 0x7ff4140a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.696+0000 7ff413fff640 1 -- 172.21.15.82:0/2188749575 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420039040 con 0x564d08286e80 2024-04-21T08:54:26.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.696+0000 7ff413fff640 1 -- 172.21.15.82:0/2188749575 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff420043070 con 0x564d08286e80 2024-04-21T08:54:26.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.696+0000 7ff413fff640 1 -- 172.21.15.82:0/2188749575 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4200053c0 con 0x564d08286e80 2024-04-21T08:54:26.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.696+0000 7ff42e430640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff4140a3ef0 0x7ff4140a63b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff424005cc0 tx=0x7ff424005bd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.697+0000 7ff432b65740 1 -- 172.21.15.82:0/2188749575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d08286e80 msgr2=0x564d0828f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.697+0000 7ff432b65740 1 --2- 172.21.15.82:0/2188749575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d08286e80 0x564d0828f430 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff420000c00 tx=0x7ff420034570 comp rx=0 tx=0).stop 2024-04-21T08:54:26.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.697+0000 7ff432b65740 1 -- 172.21.15.82:0/2188749575 shutdown_connections 2024-04-21T08:54:26.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.697+0000 7ff432b65740 1 --2- 172.21.15.82:0/2188749575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828fa00 0x564d08291e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.697+0000 7ff432b65740 1 --2- 172.21.15.82:0/2188749575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d08286e80 0x564d0828f430 secure :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff420000c00 tx=0x7ff420034570 comp rx=0 tx=0).stop 2024-04-21T08:54:26.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.697+0000 7ff432b65740 1 --2- 172.21.15.82:0/2188749575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 0x564d082868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.697+0000 7ff432b65740 1 -- 172.21.15.82:0/2188749575 >> 172.21.15.82:0/2188749575 conn(0x564d08280ba0 msgr2=0x564d08280f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.698+0000 7ff432b65740 1 -- 172.21.15.82:0/2188749575 shutdown_connections 2024-04-21T08:54:26.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.699+0000 7ff432b65740 1 -- 172.21.15.82:0/2188749575 wait complete. 2024-04-21T08:54:26.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.699+0000 7ff432b65740 1 -- start start 2024-04-21T08:54:26.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.699+0000 7ff432b65740 1 Processor -- start 2024-04-21T08:54:26.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.700+0000 7ff432b65740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d082864d0 0x564d0828c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.700+0000 7ff432b65740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 0x564d0828ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.700+0000 7ff432b65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0828d230 0x564d0829bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.700+0000 7ff432b65740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564d08292910 con 0x564d0828fa00 2024-04-21T08:54:26.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.700+0000 7ff432b65740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564d081a4500 con 0x564d082864d0 2024-04-21T08:54:26.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.700+0000 7ff432b65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564d08292a90 con 0x564d0828d230 2024-04-21T08:54:26.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.700+0000 7ff432b65740 10 client.0 _ll_register_callbacks cb 0x564d08172b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:26.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.701+0000 7ff42d42e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 0x564d0828ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.701+0000 7ff42d42e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 0x564d0828ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45884/0 (socket says 172.21.15.82:45884) 2024-04-21T08:54:26.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.701+0000 7ff42d42e640 1 -- 172.21.15.82:0/571367715 learned_addr learned my addr 172.21.15.82:0/571367715 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.702+0000 7ff42d42e640 1 -- 172.21.15.82:0/571367715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0828d230 msgr2=0x564d0829bd80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.702+0000 7ff42d42e640 1 --2- 172.21.15.82:0/571367715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0828d230 0x564d0829bd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.702+0000 7ff42d42e640 1 -- 172.21.15.82:0/571367715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d082864d0 msgr2=0x564d0828c7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.702+0000 7ff42d42e640 1 --2- 172.21.15.82:0/571367715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d082864d0 0x564d0828c7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.702+0000 7ff42d42e640 1 -- 172.21.15.82:0/571367715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d07efd3f0 con 0x564d0828fa00 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.703+0000 7ff42d42e640 1 --2- 172.21.15.82:0/571367715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 0x564d0828ccf0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff4180a9910 tx=0x7ff4180ab450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b1c430e8ca8ead server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.703+0000 7ff4117fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.703+0000 7ff4117fa640 1 -- 172.21.15.82:0/571367715 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4180ac8a0 con 0x564d0828fa00 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.703+0000 7ff4117fa640 1 -- 172.21.15.82:0/571367715 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ff418021920 con 0x564d0828fa00 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.703+0000 7ff432b65740 1 -- 172.21.15.82:0/571367715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff420043070 con 0x564d0828fa00 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.703+0000 7ff4117fa640 1 -- 172.21.15.82:0/571367715 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff4180c6050 con 0x564d0828fa00 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.704+0000 7ff432b65740 10 client.17670 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.704+0000 7ff432b65740 1 -- 172.21.15.82:0/571367715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564d07fc77d0 con 0x564d0828fa00 2024-04-21T08:54:26.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.704+0000 7ff4117fa640 1 -- 172.21.15.82:0/571367715 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff420043070 con 0x564d0828fa00 2024-04-21T08:54:26.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.705+0000 7ff4117fa640 1 -- 172.21.15.82:0/571367715 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7ff41801f040 con 0x564d0828fa00 2024-04-21T08:54:26.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.705+0000 7ff432b65740 10 client.17670 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:26.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.705+0000 7ff432b65740 10 client.17670 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:54:26.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.705+0000 7ff432b65740 20 client.17670 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:26.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.705+0000 7ff432b65740 1 --2- 172.21.15.82:0/571367715 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x564d082a0540 0x564d082a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.705+0000 7ff432b65740 4 client.17670 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:26.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.705+0000 7ff432b65740 1 -- 172.21.15.82:0/571367715 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d0818a930 con 0x564d082a0540 2024-04-21T08:54:26.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.705+0000 7ff42dc2f640 1 --2- 172.21.15.82:0/571367715 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x564d082a0540 0x564d082a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.706+0000 7ff42dc2f640 1 --2- 172.21.15.82:0/571367715 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x564d082a0540 0x564d082a29c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=872495994922df64 server_cookie=a47e0db28d750304 in_seq=0 out_seq=0 2024-04-21T08:54:26.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.706+0000 7ff4117fa640 10 client.17670 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.707+0000 7ff4117fa640 1 -- 172.21.15.82:0/571367715 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d0818a930 con 0x564d082a0540 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.707+0000 7ff4117fa640 10 client.17670 handle_command_reply: tid=0 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.707+0000 7ff432b65740 1 client.17670 shutdown 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.707+0000 7ff432b65740 1 -- 172.21.15.82:0/571367715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 msgr2=0x564d0828ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.707+0000 7ff432b65740 1 --2- 172.21.15.82:0/571367715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 0x564d0828ccf0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff4180a9910 tx=0x7ff4180ab450 comp rx=0 tx=0).stop 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.708+0000 7ff432b65740 1 -- 172.21.15.82:0/571367715 shutdown_connections 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.708+0000 7ff432b65740 1 --2- 172.21.15.82:0/571367715 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x564d082a0540 0x564d082a29c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.708+0000 7ff432b65740 1 --2- 172.21.15.82:0/571367715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d0828d230 0x564d0829bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.708+0000 7ff432b65740 1 --2- 172.21.15.82:0/571367715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 0x564d0828ccf0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.708+0000 7ff432b65740 1 --2- 172.21.15.82:0/571367715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d082864d0 0x564d0828c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.708+0000 7ff432b65740 1 -- 172.21.15.82:0/571367715 >> 172.21.15.82:0/571367715 conn(0x564d08280ba0 msgr2=0x564d082828f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.708+0000 7ff4117fa640 0 client.17670 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:54:26.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.708+0000 7ff432b65740 1 -- 172.21.15.82:0/571367715 shutdown_connections 2024-04-21T08:54:26.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.708+0000 7ff432b65740 1 -- 172.21.15.82:0/571367715 wait complete. 2024-04-21T08:54:26.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.709+0000 7ff432b65740 20 client.17670 trim_cache size 0 max 16384 2024-04-21T08:54:26.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.738+0000 7ff432b65740 1 Processor -- start 2024-04-21T08:54:26.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.739+0000 7ff432b65740 1 -- start start 2024-04-21T08:54:26.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.739+0000 7ff432b65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 0x564d082a82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.739+0000 7ff432b65740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828d230 0x564d082a8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.739+0000 7ff432b65740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 0x564d0829b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff42d42e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828d230 0x564d082a8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff42d42e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828d230 0x564d082a8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:41728/0 (socket says 172.21.15.82:41728) 2024-04-21T08:54:26.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff42d42e640 1 -- 172.21.15.82:0/1778893213 learned_addr learned my addr 172.21.15.82:0/1778893213 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff432b65740 1 -- 172.21.15.82:0/1778893213 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281b5c70 con 0x564d0828fa00 2024-04-21T08:54:26.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff42e430640 1 --2- 172.21.15.82:0/1778893213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 0x564d0829b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff432b65740 1 -- 172.21.15.82:0/1778893213 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281b78a0 con 0x564d0828d230 2024-04-21T08:54:26.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff42dc2f640 1 --2- 172.21.15.82:0/1778893213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 0x564d082a82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff432b65740 1 -- 172.21.15.82:0/1778893213 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281b6340 con 0x564d082864d0 2024-04-21T08:54:26.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff42e430640 1 -- 172.21.15.82:0/1778893213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 msgr2=0x564d082a82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff42e430640 1 --2- 172.21.15.82:0/1778893213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 0x564d082a82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff42e430640 1 -- 172.21.15.82:0/1778893213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828d230 msgr2=0x564d082a8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff42e430640 1 --2- 172.21.15.82:0/1778893213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828d230 0x564d082a8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff42e430640 1 -- 172.21.15.82:0/1778893213 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d08286af0 con 0x564d0828fa00 2024-04-21T08:54:26.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.740+0000 7ff42d42e640 1 --2- 172.21.15.82:0/1778893213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828d230 0x564d082a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:26.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.741+0000 7ff42e430640 1 --2- 172.21.15.82:0/1778893213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 0x564d0829b7d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff424007f90 tx=0x7ff424011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96259b20a544ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.742+0000 7ff4137fe640 1 -- 172.21.15.82:0/1778893213 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424021070 con 0x564d0828fa00 2024-04-21T08:54:26.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.742+0000 7ff4137fe640 1 -- 172.21.15.82:0/1778893213 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff42401db00 con 0x564d0828fa00 2024-04-21T08:54:26.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.742+0000 7ff4137fe640 1 -- 172.21.15.82:0/1778893213 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42401cbb0 con 0x564d0828fa00 2024-04-21T08:54:26.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.743+0000 7ff432b65740 1 -- 172.21.15.82:0/1778893213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 msgr2=0x564d0829b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.743+0000 7ff432b65740 1 --2- 172.21.15.82:0/1778893213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 0x564d0829b7d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff424007f90 tx=0x7ff424011000 comp rx=0 tx=0).stop 2024-04-21T08:54:26.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.743+0000 7ff432b65740 1 -- 172.21.15.82:0/1778893213 shutdown_connections 2024-04-21T08:54:26.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.743+0000 7ff432b65740 1 --2- 172.21.15.82:0/1778893213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828fa00 0x564d0829b7d0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.743+0000 7ff432b65740 1 --2- 172.21.15.82:0/1778893213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828d230 0x564d082a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.743+0000 7ff432b65740 1 --2- 172.21.15.82:0/1778893213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 0x564d082a82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.743+0000 7ff432b65740 1 -- 172.21.15.82:0/1778893213 >> 172.21.15.82:0/1778893213 conn(0x564d08280ba0 msgr2=0x564d0828c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.744+0000 7ff432b65740 1 -- 172.21.15.82:0/1778893213 shutdown_connections 2024-04-21T08:54:26.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.745+0000 7ff432b65740 1 -- 172.21.15.82:0/1778893213 wait complete. 2024-04-21T08:54:26.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.746+0000 7ff432b65740 1 -- start start 2024-04-21T08:54:26.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.746+0000 7ff432b65740 1 Processor -- start 2024-04-21T08:54:26.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.748+0000 7ff432b65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 0x564d0828a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.748+0000 7ff432b65740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828d230 0x564d08297ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.748+0000 7ff432b65740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828fa00 0x564d08288a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.748+0000 7ff432b65740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281b71a0 con 0x564d0828d230 2024-04-21T08:54:26.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.748+0000 7ff432b65740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281b6aa0 con 0x564d0828fa00 2024-04-21T08:54:26.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.749+0000 7ff432b65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff4281b3c40 con 0x564d082864d0 2024-04-21T08:54:26.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.749+0000 7ff42d42e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828d230 0x564d08297ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.749+0000 7ff42e430640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828fa00 0x564d08288a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.749+0000 7ff42e430640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828fa00 0x564d08288a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:41740/0 (socket says 172.21.15.82:41740) 2024-04-21T08:54:26.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.749+0000 7ff42dc2f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 0x564d0828a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.749+0000 7ff42e430640 1 -- 172.21.15.82:0/1677073837 learned_addr learned my addr 172.21.15.82:0/1677073837 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.749+0000 7ff432b65740 10 client.0 _ll_register_callbacks cb 0x7ff4180e4ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:26.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.749+0000 7ff42e430640 1 -- 172.21.15.82:0/1677073837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 msgr2=0x564d0828a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.749+0000 7ff42e430640 1 --2- 172.21.15.82:0/1677073837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 0x564d0828a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.749+0000 7ff42e430640 1 -- 172.21.15.82:0/1677073837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828d230 msgr2=0x564d08297ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.749+0000 7ff42e430640 1 --2- 172.21.15.82:0/1677073837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828d230 0x564d08297ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.749+0000 7ff42e430640 1 -- 172.21.15.82:0/1677073837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d08288f60 con 0x564d0828fa00 2024-04-21T08:54:26.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.750+0000 7ff42e430640 1 --2- 172.21.15.82:0/1677073837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828fa00 0x564d08288a20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff42401da00 tx=0x7ff42401da30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f77518b1f380096b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.750+0000 7ff4117fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:54:26.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.750+0000 7ff4117fa640 1 -- 172.21.15.82:0/1677073837 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424035060 con 0x564d0828fa00 2024-04-21T08:54:26.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.750+0000 7ff4117fa640 1 -- 172.21.15.82:0/1677073837 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ff42401c9c0 con 0x564d0828fa00 2024-04-21T08:54:26.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.750+0000 7ff432b65740 1 -- 172.21.15.82:0/1677073837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff42401db00 con 0x564d0828fa00 2024-04-21T08:54:26.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.751+0000 7ff4117fa640 1 -- 172.21.15.82:0/1677073837 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff424021050 con 0x564d0828fa00 2024-04-21T08:54:26.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.753+0000 7ff432b65740 10 client.26996 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:26.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.753+0000 7ff432b65740 1 -- 172.21.15.82:0/1677073837 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564d083061c0 con 0x564d0828fa00 2024-04-21T08:54:26.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.754+0000 7ff4117fa640 1 -- 172.21.15.82:0/1677073837 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff42401cbe0 con 0x564d0828fa00 2024-04-21T08:54:26.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.754+0000 7ff4117fa640 1 -- 172.21.15.82:0/1677073837 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7ff424026450 con 0x564d0828fa00 2024-04-21T08:54:26.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.754+0000 7ff432b65740 10 client.26996 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:26.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.754+0000 7ff432b65740 10 client.26996 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:54:26.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.754+0000 7ff432b65740 20 client.26996 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:26.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.754+0000 7ff432b65740 1 --2- 172.21.15.82:0/1677073837 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x564d08306790 0x564d08308bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.754+0000 7ff432b65740 4 client.26996 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:54:26.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.754+0000 7ff432b65740 1 -- 172.21.15.82:0/1677073837 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff42401db00 con 0x564d08306790 2024-04-21T08:54:26.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.755+0000 7ff42dc2f640 1 --2- 172.21.15.82:0/1677073837 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x564d08306790 0x564d08308bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.756+0000 7ff42dc2f640 1 --2- 172.21.15.82:0/1677073837 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x564d08306790 0x564d08308bb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=393e64b17990add3 server_cookie=99d0524f9058e2cc in_seq=0 out_seq=0 2024-04-21T08:54:26.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.756+0000 7ff4117fa640 10 client.26996 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:54:26.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.756+0000 7ff4117fa640 1 -- 172.21.15.82:0/1677073837 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff42401db00 con 0x564d08306790 2024-04-21T08:54:26.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.756+0000 7ff4117fa640 10 client.26996 handle_command_reply: tid=0 2024-04-21T08:54:26.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.756+0000 7ff432b65740 1 client.26996 shutdown 2024-04-21T08:54:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.757+0000 7ff432b65740 1 -- 172.21.15.82:0/1677073837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828fa00 msgr2=0x564d08288a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.757+0000 7ff432b65740 1 --2- 172.21.15.82:0/1677073837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828fa00 0x564d08288a20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff42401da00 tx=0x7ff42401da30 comp rx=0 tx=0).stop 2024-04-21T08:54:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.757+0000 7ff432b65740 1 -- 172.21.15.82:0/1677073837 shutdown_connections 2024-04-21T08:54:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.757+0000 7ff432b65740 1 --2- 172.21.15.82:0/1677073837 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x564d08306790 0x564d08308bb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.757+0000 7ff432b65740 1 --2- 172.21.15.82:0/1677073837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d0828fa00 0x564d08288a20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.757+0000 7ff4117fa640 0 client.26996 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:54:26.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.757+0000 7ff432b65740 1 --2- 172.21.15.82:0/1677073837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0828d230 0x564d08297ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.757+0000 7ff432b65740 1 --2- 172.21.15.82:0/1677073837 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d082864d0 0x564d0828a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.757+0000 7ff432b65740 1 -- 172.21.15.82:0/1677073837 >> 172.21.15.82:0/1677073837 conn(0x564d08280ba0 msgr2=0x564d08281f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.757+0000 7ff432b65740 1 -- 172.21.15.82:0/1677073837 shutdown_connections 2024-04-21T08:54:26.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.757+0000 7ff432b65740 1 -- 172.21.15.82:0/1677073837 wait complete. 2024-04-21T08:54:26.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.757+0000 7ff432b65740 20 client.26996 trim_cache size 0 max 16384 2024-04-21T08:54:26.758 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:54:26.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.760+0000 7ff42fe93640 1 -- 172.21.15.82:0/520254752 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff4140a3ef0 msgr2=0x7ff4140a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.760+0000 7ff42fe93640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff4140a3ef0 0x7ff4140a63b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff424005cc0 tx=0x7ff424005bd0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.760+0000 7ff42fe93640 1 -- 172.21.15.82:0/520254752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff428106160 msgr2=0x7ff42806d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.760+0000 7ff42fe93640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff428106160 0x7ff42806d970 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff41801e990 tx=0x7ff41801ee50 comp rx=0 tx=0).stop 2024-04-21T08:54:26.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.761+0000 7ff42fe93640 1 -- 172.21.15.82:0/520254752 shutdown_connections 2024-04-21T08:54:26.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.761+0000 7ff42fe93640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff42806deb0 0x7ff42806e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.761+0000 7ff42fe93640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff4140a3ef0 0x7ff4140a63b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.761+0000 7ff42fe93640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff428106160 0x7ff42806d970 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.761+0000 7ff42fe93640 1 --2- 172.21.15.82:0/520254752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff428101090 0x7ff4280738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.761+0000 7ff42fe93640 1 -- 172.21.15.82:0/520254752 >> 172.21.15.82:0/520254752 conn(0x7ff428077f50 msgr2=0x7ff4280ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.762+0000 7ff42fe93640 1 -- 172.21.15.82:0/520254752 shutdown_connections 2024-04-21T08:54:26.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.762+0000 7ff42fe93640 1 -- 172.21.15.82:0/520254752 wait complete. 2024-04-21T08:54:26.780 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:54:26.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.921+0000 7f4c36911640 1 -- 172.21.15.82:0/2392614179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c30106160 msgr2=0x7f4c3010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.921+0000 7f4c36911640 1 --2- 172.21.15.82:0/2392614179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c30106160 0x7f4c3010dd30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009ca0 tx=0x7f4c2802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.921+0000 7f4c36911640 1 -- 172.21.15.82:0/2392614179 shutdown_connections 2024-04-21T08:54:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.921+0000 7f4c36911640 1 --2- 172.21.15.82:0/2392614179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c30106160 0x7f4c3010dd30 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.921+0000 7f4c36911640 1 --2- 172.21.15.82:0/2392614179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c30101a40 0x7f4c30105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.921+0000 7f4c36911640 1 --2- 172.21.15.82:0/2392614179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c30101090 0x7f4c30101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.921+0000 7f4c36911640 1 -- 172.21.15.82:0/2392614179 >> 172.21.15.82:0/2392614179 conn(0x7f4c30077f50 msgr2=0x7f4c30078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.921+0000 7f4c36911640 1 -- 172.21.15.82:0/2392614179 shutdown_connections 2024-04-21T08:54:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.921+0000 7f4c36911640 1 -- 172.21.15.82:0/2392614179 wait complete. 2024-04-21T08:54:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c36911640 1 Processor -- start 2024-04-21T08:54:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c36911640 1 -- start start 2024-04-21T08:54:26.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c36911640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c30101090 0x7f4c301986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c36911640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c30106160 0x7f4c30198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c36911640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c3019cfb0 0x7f4c3019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c36911640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301110b0 con 0x7f4c30106160 2024-04-21T08:54:26.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c36911640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4c30110f30 con 0x7f4c3019cfb0 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c36911640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4c30111230 con 0x7f4c30101090 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c2f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c30106160 0x7f4c30198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c2f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c30106160 0x7f4c30198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45906/0 (socket says 172.21.15.82:45906) 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c2f7fe640 1 -- 172.21.15.82:0/2363282689 learned_addr learned my addr 172.21.15.82:0/2363282689 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c34eae640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c3019cfb0 0x7f4c3019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.922+0000 7f4c2ffff640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c30101090 0x7f4c301986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.923+0000 7f4c2f7fe640 1 -- 172.21.15.82:0/2363282689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c30101090 msgr2=0x7f4c301986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.923+0000 7f4c2f7fe640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c30101090 0x7f4c301986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.923+0000 7f4c2f7fe640 1 -- 172.21.15.82:0/2363282689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c3019cfb0 msgr2=0x7f4c3019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.923+0000 7f4c2f7fe640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c3019cfb0 0x7f4c3019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.923+0000 7f4c2f7fe640 1 -- 172.21.15.82:0/2363282689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3019daa0 con 0x7f4c30106160 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.923+0000 7f4c2f7fe640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c30106160 0x7f4c30198c30 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400e9c0 tx=0x7f4c2400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.923+0000 7f4c2d7fa640 1 -- 172.21.15.82:0/2363282689 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400cd80 con 0x7f4c30106160 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.923+0000 7f4c2d7fa640 1 -- 172.21.15.82:0/2363282689 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c240054d0 con 0x7f4c30106160 2024-04-21T08:54:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.923+0000 7f4c36911640 1 -- 172.21.15.82:0/2363282689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3019dd90 con 0x7f4c30106160 2024-04-21T08:54:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.923+0000 7f4c2d7fa640 1 -- 172.21.15.82:0/2363282689 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24010430 con 0x7f4c30106160 2024-04-21T08:54:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.923+0000 7f4c36911640 1 -- 172.21.15.82:0/2363282689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c301a5580 con 0x7f4c30106160 2024-04-21T08:54:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.925+0000 7f4c2d7fa640 1 -- 172.21.15.82:0/2363282689 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4c24004fd0 con 0x7f4c30106160 2024-04-21T08:54:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.925+0000 7f4c395d6740 1 Processor -- start 2024-04-21T08:54:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c395d6740 1 -- start start 2024-04-21T08:54:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c395d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe94d0 0x557440fe98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c395d6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440ff2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c395d6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440ff4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c395d6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557440f4e1f0 con 0x557440ff2a00 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c395d6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557440eeb4b0 con 0x557440fe9e80 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c395d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557440f4a1b0 con 0x557440fe94d0 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c2d7fa640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4c180a3d20 0x7f4c180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c2f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440ff4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c2ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440ff2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c34eae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe94d0 0x557440fe98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c2ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440ff2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:41766/0 (socket says 172.21.15.82:41766) 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c2ffff640 1 -- 172.21.15.82:0/3061481145 learned_addr learned my addr 172.21.15.82:0/3061481145 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c34eae640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4c180a3d20 0x7f4c180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c2d7fa640 1 -- 172.21.15.82:0/2363282689 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f4c240d0f80 con 0x7f4c30106160 2024-04-21T08:54:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c34eae640 1 -- 172.21.15.82:0/3061481145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 msgr2=0x557440ff2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c34eae640 1 --2- 172.21.15.82:0/3061481145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440ff2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c34eae640 1 -- 172.21.15.82:0/3061481145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 msgr2=0x557440ff4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c34eae640 1 --2- 172.21.15.82:0/3061481145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440ff4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c34eae640 1 -- 172.21.15.82:0/3061481145 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557440edd1b0 con 0x557440fe94d0 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c2ffff640 1 --2- 172.21.15.82:0/3061481145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440ff2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c34eae640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4c180a3d20 0x7f4c180a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4c28037040 tx=0x7f4c2803f000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.926+0000 7f4c34eae640 1 --2- 172.21.15.82:0/3061481145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe94d0 0x557440fe98b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009bf0 tx=0x7f4c28032ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba251956cad707d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.927+0000 7f4c167fc640 1 -- 172.21.15.82:0/3061481145 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28051070 con 0x557440fe94d0 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.927+0000 7f4c167fc640 1 -- 172.21.15.82:0/3061481145 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c28004d40 con 0x557440fe94d0 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.927+0000 7f4c167fc640 1 -- 172.21.15.82:0/3061481145 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28039040 con 0x557440fe94d0 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.927+0000 7f4c395d6740 1 -- 172.21.15.82:0/3061481145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe94d0 msgr2=0x557440fe98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.927+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3061481145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe94d0 0x557440fe98b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009bf0 tx=0x7f4c28032ee0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.927+0000 7f4c395d6740 1 -- 172.21.15.82:0/3061481145 shutdown_connections 2024-04-21T08:54:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.927+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3061481145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440ff4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.927+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3061481145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440ff2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.927+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3061481145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe94d0 0x557440fe98b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.927+0000 7f4c395d6740 1 -- 172.21.15.82:0/3061481145 >> 172.21.15.82:0/3061481145 conn(0x557440fe3ba0 msgr2=0x557440fe3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.927+0000 7f4c395d6740 1 -- 172.21.15.82:0/3061481145 shutdown_connections 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.927+0000 7f4c395d6740 1 -- 172.21.15.82:0/3061481145 wait complete. 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c395d6740 1 -- start start 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c395d6740 1 Processor -- start 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c395d6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440fe94d0 0x557440ff2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c395d6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440fef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c395d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440ff2a00 0x557440fefcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c395d6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557440ff5910 con 0x557440fe94d0 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c395d6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557440f07500 con 0x557440fe9e80 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c395d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557440ff5a90 con 0x557440ff2a00 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c395d6740 10 client.0 _ll_register_callbacks cb 0x557440ed5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:26.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c2ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440fe94d0 0x557440ff2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c2ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440fe94d0 0x557440ff2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45922/0 (socket says 172.21.15.82:45922) 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c2ffff640 1 -- 172.21.15.82:0/2752481333 learned_addr learned my addr 172.21.15.82:0/2752481333 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c34eae640 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440ff2a00 0x557440fefcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c2f7fe640 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440fef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c2ffff640 1 -- 172.21.15.82:0/2752481333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440ff2a00 msgr2=0x557440fefcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c2ffff640 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440ff2a00 0x557440fefcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c2ffff640 1 -- 172.21.15.82:0/2752481333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 msgr2=0x557440fef6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c2ffff640 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440fef6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c2ffff640 1 -- 172.21.15.82:0/2752481333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557440c603f0 con 0x557440fe94d0 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c34eae640 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440ff2a00 0x557440fefcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.928+0000 7f4c2f7fe640 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440fef6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c2ffff640 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440fe94d0 0x557440ff2330 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000bf70 tx=0x7f4c200118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430cfc8d06fe7322 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c0f7fe640 1 -- 172.21.15.82:0/2752481333 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20004020 con 0x557440fe94d0 2024-04-21T08:54:26.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c0f7fe640 1 -- 172.21.15.82:0/2752481333 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f4c200041c0 con 0x557440fe94d0 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c395d6740 1 -- 172.21.15.82:0/2752481333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c28004d40 con 0x557440fe94d0 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c0f7fe640 1 -- 172.21.15.82:0/2752481333 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4c200043c0 con 0x557440fe94d0 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c395d6740 10 client.17688 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c395d6740 1 -- 172.21.15.82:0/2752481333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557440d2a7d0 con 0x557440fe94d0 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c0f7fe640 1 -- 172.21.15.82:0/2752481333 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c28004d40 con 0x557440fe94d0 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c0f7fe640 1 -- 172.21.15.82:0/2752481333 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f4c20004ca0 con 0x557440fe94d0 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c395d6740 10 client.17688 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c395d6740 10 client.17688 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c395d6740 20 client.17688 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c395d6740 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557441002c20 0x557441005030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c395d6740 4 client.17688 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.929+0000 7f4c395d6740 1 -- 172.21.15.82:0/2752481333 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557440eed930 con 0x557441002c20 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.930+0000 7f4c2f7fe640 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557441002c20 0x557441005030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.930+0000 7f4c2f7fe640 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557441002c20 0x557441005030 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d65ea1cae429120f server_cookie=3d6e39a0a95acdda in_seq=0 out_seq=0 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.930+0000 7f4c0f7fe640 10 client.17688 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:54:26.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.930+0000 7f4c0f7fe640 1 -- 172.21.15.82:0/2752481333 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557440eed930 con 0x557441002c20 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.930+0000 7f4c0f7fe640 10 client.17688 handle_command_reply: tid=0 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c395d6740 1 client.17688 shutdown 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c395d6740 1 -- 172.21.15.82:0/2752481333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440fe94d0 msgr2=0x557440ff2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c395d6740 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440fe94d0 0x557440ff2330 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000bf70 tx=0x7f4c200118d0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c395d6740 1 -- 172.21.15.82:0/2752481333 shutdown_connections 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c395d6740 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557441002c20 0x557441005030 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c0f7fe640 0 client.17688 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c395d6740 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440ff2a00 0x557440fefcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c395d6740 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440fef6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c395d6740 1 --2- 172.21.15.82:0/2752481333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440fe94d0 0x557440ff2330 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c395d6740 1 -- 172.21.15.82:0/2752481333 >> 172.21.15.82:0/2752481333 conn(0x557440fe3ba0 msgr2=0x557440fe58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c395d6740 1 -- 172.21.15.82:0/2752481333 shutdown_connections 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c395d6740 1 -- 172.21.15.82:0/2752481333 wait complete. 2024-04-21T08:54:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.931+0000 7f4c395d6740 20 client.17688 trim_cache size 0 max 16384 2024-04-21T08:54:26.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c395d6740 1 Processor -- start 2024-04-21T08:54:26.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c395d6740 1 -- start start 2024-04-21T08:54:26.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c395d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe94d0 0x557440ffe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c395d6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440ffb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c395d6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440fee820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c395d6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301af560 con 0x557440ff2a00 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c395d6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301b1190 con 0x557440fe9e80 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c395d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301afc30 con 0x557440fe94d0 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c34eae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440fee820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c34eae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440fee820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45936/0 (socket says 172.21.15.82:45936) 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c34eae640 1 -- 172.21.15.82:0/3748305015 learned_addr learned my addr 172.21.15.82:0/3748305015 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c2f7fe640 1 --2- 172.21.15.82:0/3748305015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440ffb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.944+0000 7f4c2ffff640 1 --2- 172.21.15.82:0/3748305015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe94d0 0x557440ffe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c34eae640 1 -- 172.21.15.82:0/3748305015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe94d0 msgr2=0x557440ffe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c34eae640 1 --2- 172.21.15.82:0/3748305015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe94d0 0x557440ffe7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c34eae640 1 -- 172.21.15.82:0/3748305015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 msgr2=0x557440ffb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c34eae640 1 --2- 172.21.15.82:0/3748305015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440ffb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c34eae640 1 -- 172.21.15.82:0/3748305015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557440ff0040 con 0x557440ff2a00 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c2ffff640 1 --2- 172.21.15.82:0/3748305015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe94d0 0x557440ffe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c34eae640 1 --2- 172.21.15.82:0/3748305015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440fee820 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009bf0 tx=0x7f4c2802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca86146b27380828 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c15ffb640 1 -- 172.21.15.82:0/3748305015 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2805a070 con 0x557440ff2a00 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c15ffb640 1 -- 172.21.15.82:0/3748305015 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c28009540 con 0x557440ff2a00 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c15ffb640 1 -- 172.21.15.82:0/3748305015 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28039040 con 0x557440ff2a00 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c395d6740 1 -- 172.21.15.82:0/3748305015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 msgr2=0x557440fee820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3748305015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440fee820 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009bf0 tx=0x7f4c2802e040 comp rx=0 tx=0).stop 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c395d6740 1 -- 172.21.15.82:0/3748305015 shutdown_connections 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3748305015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440fee820 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3748305015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe9e80 0x557440ffb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3748305015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe94d0 0x557440ffe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.945+0000 7f4c395d6740 1 -- 172.21.15.82:0/3748305015 >> 172.21.15.82:0/3748305015 conn(0x557440fe3ba0 msgr2=0x557440fea3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c395d6740 1 -- 172.21.15.82:0/3748305015 shutdown_connections 2024-04-21T08:54:26.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c395d6740 1 -- 172.21.15.82:0/3748305015 wait complete. 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c395d6740 1 -- start start 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c395d6740 1 Processor -- start 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c395d6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe94d0 0x557441065610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c395d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe9e80 0x557441065b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c395d6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440ff1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c395d6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301b0a90 con 0x557440ff2a00 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c395d6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301b0390 con 0x557440fe94d0 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c395d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4c301ad530 con 0x557440fe9e80 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c395d6740 10 client.0 _ll_register_callbacks cb 0x7f4c2000ba70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c34eae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440ff1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c2ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe94d0 0x557441065610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c2ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe94d0 0x557441065610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:41786/0 (socket says 172.21.15.82:41786) 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c2ffff640 1 -- 172.21.15.82:0/3123834294 learned_addr learned my addr 172.21.15.82:0/3123834294 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:26.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.946+0000 7f4c2f7fe640 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe9e80 0x557441065b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c2ffff640 1 -- 172.21.15.82:0/3123834294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe9e80 msgr2=0x557441065b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c2ffff640 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe9e80 0x557441065b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c2ffff640 1 -- 172.21.15.82:0/3123834294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 msgr2=0x557440ff1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c2ffff640 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440ff1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c2ffff640 1 -- 172.21.15.82:0/3123834294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557440ff18a0 con 0x557440fe94d0 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c34eae640 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440ff1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c2ffff640 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe94d0 0x557441065610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4c20010910 tx=0x7f4c20005370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f842d68ffcef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c0f7fe640 1 -- 172.21.15.82:0/3123834294 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2000b480 con 0x557440fe94d0 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c395d6740 1 -- 172.21.15.82:0/3123834294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c28009540 con 0x557440fe94d0 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c0f7fe640 1 -- 172.21.15.82:0/3123834294 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f4c20004660 con 0x557440fe94d0 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c0f7fe640 1 -- 172.21.15.82:0/3123834294 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4c20005850 con 0x557440fe94d0 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c395d6740 10 client.27014 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:26.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c395d6740 1 -- 172.21.15.82:0/3123834294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55744106ce90 con 0x557440fe94d0 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.947+0000 7f4c0f7fe640 1 -- 172.21.15.82:0/3123834294 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c28009540 con 0x557440fe94d0 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c0f7fe640 1 -- 172.21.15.82:0/3123834294 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f4c20029770 con 0x557440fe94d0 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 10 client.27014 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 10 client.27014 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 20 client.27014 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557441002c20 0x55744106f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 4 client.27014 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 1 -- 172.21.15.82:0/3123834294 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557440f5bc10 con 0x557441002c20 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c2f7fe640 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557441002c20 0x55744106f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c0f7fe640 10 client.27014 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c2f7fe640 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557441002c20 0x55744106f650 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4259a5428d7c1761 server_cookie=82608412137db47b in_seq=0 out_seq=0 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c0f7fe640 1 -- 172.21.15.82:0/3123834294 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557440f5bc10 con 0x557441002c20 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c0f7fe640 10 client.27014 handle_command_reply: tid=0 2024-04-21T08:54:26.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 1 client.27014 shutdown 2024-04-21T08:54:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 1 -- 172.21.15.82:0/3123834294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe94d0 msgr2=0x557441065610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe94d0 0x557441065610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4c20010910 tx=0x7f4c20005370 comp rx=0 tx=0).stop 2024-04-21T08:54:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 1 -- 172.21.15.82:0/3123834294 shutdown_connections 2024-04-21T08:54:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x557441002c20 0x55744106f650 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557440ff2a00 0x557440ff1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c0f7fe640 0 client.27014 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:54:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557440fe9e80 0x557441065b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 1 --2- 172.21.15.82:0/3123834294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557440fe94d0 0x557441065610 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.948+0000 7f4c395d6740 1 -- 172.21.15.82:0/3123834294 >> 172.21.15.82:0/3123834294 conn(0x557440fe3ba0 msgr2=0x557440fe2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c395d6740 1 -- 172.21.15.82:0/3123834294 shutdown_connections 2024-04-21T08:54:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c395d6740 1 -- 172.21.15.82:0/3123834294 wait complete. 2024-04-21T08:54:26.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c395d6740 20 client.27014 trim_cache size 0 max 16384 2024-04-21T08:54:26.956 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:54:26.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c36911640 1 -- 172.21.15.82:0/2363282689 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4c180a3d20 msgr2=0x7f4c180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c36911640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4c180a3d20 0x7f4c180a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4c28037040 tx=0x7f4c2803f000 comp rx=0 tx=0).stop 2024-04-21T08:54:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c36911640 1 -- 172.21.15.82:0/2363282689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c30106160 msgr2=0x7f4c30198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c36911640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c30106160 0x7f4c30198c30 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400e9c0 tx=0x7f4c2400ee80 comp rx=0 tx=0).stop 2024-04-21T08:54:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c36911640 1 -- 172.21.15.82:0/2363282689 shutdown_connections 2024-04-21T08:54:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c36911640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c3019cfb0 0x7f4c3019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c36911640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f4c180a3d20 0x7f4c180a61e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c36911640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c30106160 0x7f4c30198c30 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c36911640 1 --2- 172.21.15.82:0/2363282689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c30101090 0x7f4c301986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c36911640 1 -- 172.21.15.82:0/2363282689 >> 172.21.15.82:0/2363282689 conn(0x7f4c30077f50 msgr2=0x7f4c30069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c36911640 1 -- 172.21.15.82:0/2363282689 shutdown_connections 2024-04-21T08:54:26.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:26.949+0000 7f4c36911640 1 -- 172.21.15.82:0/2363282689 wait complete. 2024-04-21T08:54:26.960 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:54:27.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.064+0000 7f008e72b640 1 -- 172.21.15.82:0/2254244169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0088101a50 msgr2=0x7f0088105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.064+0000 7f008e72b640 1 --2- 172.21.15.82:0/2254244169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0088101a50 0x7f0088105a90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f007c009900 tx=0x7f007c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:54:27.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.064+0000 7f008e72b640 1 -- 172.21.15.82:0/2254244169 shutdown_connections 2024-04-21T08:54:27.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.064+0000 7f008e72b640 1 --2- 172.21.15.82:0/2254244169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0088106170 0x7f008810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.064+0000 7f008e72b640 1 --2- 172.21.15.82:0/2254244169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0088101a50 0x7f0088105a90 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.064+0000 7f008e72b640 1 --2- 172.21.15.82:0/2254244169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00881010a0 0x7f0088101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.064+0000 7f008e72b640 1 -- 172.21.15.82:0/2254244169 >> 172.21.15.82:0/2254244169 conn(0x7f0088077eb0 msgr2=0x7f00880782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.064+0000 7f008e72b640 1 -- 172.21.15.82:0/2254244169 shutdown_connections 2024-04-21T08:54:27.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.064+0000 7f008e72b640 1 -- 172.21.15.82:0/2254244169 wait complete. 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f008e72b640 1 Processor -- start 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f008e72b640 1 -- start start 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f008e72b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00881010a0 0x7f0088197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f008e72b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0088106170 0x7f00881978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f008e72b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0088197e00 0x7f00881a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f008e72b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0088111090 con 0x7f00881010a0 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f008e72b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0088110f10 con 0x7f0088197e00 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f008e72b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0088111210 con 0x7f0088106170 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f0087fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00881010a0 0x7f0088197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f0087fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00881010a0 0x7f0088197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45954/0 (socket says 172.21.15.82:45954) 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f0087fff640 1 -- 172.21.15.82:0/2913499588 learned_addr learned my addr 172.21.15.82:0/2913499588 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f008ccc8640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0088197e00 0x7f00881a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f00877fe640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0088106170 0x7f00881978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f0087fff640 1 -- 172.21.15.82:0/2913499588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0088106170 msgr2=0x7f00881978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f0087fff640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0088106170 0x7f00881978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.065+0000 7f0087fff640 1 -- 172.21.15.82:0/2913499588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0088197e00 msgr2=0x7f00881a6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.066+0000 7f0087fff640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0088197e00 0x7f00881a6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.066+0000 7f0087fff640 1 -- 172.21.15.82:0/2913499588 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00881a6fc0 con 0x7f00881010a0 2024-04-21T08:54:27.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.066+0000 7f0087fff640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00881010a0 0x7f0088197380 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f007800ea20 tx=0x7f007800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.066+0000 7f00857fa640 1 -- 172.21.15.82:0/2913499588 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007800cd20 con 0x7f00881010a0 2024-04-21T08:54:27.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.066+0000 7f008e72b640 1 -- 172.21.15.82:0/2913499588 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00881a72b0 con 0x7f00881010a0 2024-04-21T08:54:27.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.066+0000 7f00857fa640 1 -- 172.21.15.82:0/2913499588 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00780108e0 con 0x7f00881010a0 2024-04-21T08:54:27.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.066+0000 7f00857fa640 1 -- 172.21.15.82:0/2913499588 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00780187a0 con 0x7f00881010a0 2024-04-21T08:54:27.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.066+0000 7f008e72b640 1 -- 172.21.15.82:0/2913499588 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00881a7820 con 0x7f00881010a0 2024-04-21T08:54:27.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.068+0000 7f00857fa640 1 -- 172.21.15.82:0/2913499588 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0078010430 con 0x7f00881010a0 2024-04-21T08:54:27.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.068+0000 7f00913f9740 1 Processor -- start 2024-04-21T08:54:27.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.068+0000 7f00913f9740 1 -- start start 2024-04-21T08:54:27.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.068+0000 7f00913f9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c2164d0 0x56479c2168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.068+0000 7f00913f9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c216e80 0x56479c21f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.068+0000 7f00913f9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 0x56479c221e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.068+0000 7f00913f9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56479c17b1f0 con 0x56479c21fa00 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.068+0000 7f00913f9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56479c1184b0 con 0x56479c2164d0 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f00913f9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56479c1771b0 con 0x56479c216e80 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f00877fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 0x56479c221e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f008ccc8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c2164d0 0x56479c2168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f008ccc8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c2164d0 0x56479c2168b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:41816/0 (socket says 172.21.15.82:41816) 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f00877fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 0x56479c221e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45968/0 (socket says 172.21.15.82:45968) 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f008ccc8640 1 -- 172.21.15.82:0/3567478134 learned_addr learned my addr 172.21.15.82:0/3567478134 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f0087fff640 1 --2- 172.21.15.82:0/3567478134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c216e80 0x56479c21f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f00857fa640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f00700a3d20 0x7f00700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f00877fe640 1 -- 172.21.15.82:0/3567478134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c216e80 msgr2=0x56479c21f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f00877fe640 1 --2- 172.21.15.82:0/3567478134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c216e80 0x56479c21f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f00877fe640 1 -- 172.21.15.82:0/3567478134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c2164d0 msgr2=0x56479c2168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f00877fe640 1 --2- 172.21.15.82:0/3567478134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c2164d0 0x56479c2168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f00877fe640 1 -- 172.21.15.82:0/3567478134 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56479c10a1b0 con 0x56479c21fa00 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f0087fff640 1 --2- 172.21.15.82:0/3567478134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c216e80 0x56479c21f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f008ccc8640 1 --2- 172.21.15.82:0/3567478134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c2164d0 0x56479c2168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f008ccc8640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f00700a3d20 0x7f00700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f00857fa640 1 -- 172.21.15.82:0/2913499588 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f00780d0f40 con 0x7f00881010a0 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f00877fe640 1 --2- 172.21.15.82:0/3567478134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 0x56479c221e50 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f007c036040 tx=0x7f007c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b549527e6c6b5b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f006e7fc640 1 -- 172.21.15.82:0/3567478134 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c045020 con 0x56479c21fa00 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f006e7fc640 1 -- 172.21.15.82:0/3567478134 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f007c02c7e0 con 0x56479c21fa00 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f006e7fc640 1 -- 172.21.15.82:0/3567478134 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c0046c0 con 0x56479c21fa00 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.069+0000 7f008ccc8640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f00700a3d20 0x7f00700a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f00800125f0 tx=0x7f0080004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.070+0000 7f00913f9740 1 -- 172.21.15.82:0/3567478134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 msgr2=0x56479c221e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.070+0000 7f00913f9740 1 --2- 172.21.15.82:0/3567478134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 0x56479c221e50 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f007c036040 tx=0x7f007c02c6c0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.070+0000 7f00913f9740 1 -- 172.21.15.82:0/3567478134 shutdown_connections 2024-04-21T08:54:27.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.070+0000 7f00913f9740 1 --2- 172.21.15.82:0/3567478134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 0x56479c221e50 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.070+0000 7f00913f9740 1 --2- 172.21.15.82:0/3567478134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c216e80 0x56479c21f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.070+0000 7f00913f9740 1 --2- 172.21.15.82:0/3567478134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c2164d0 0x56479c2168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.070+0000 7f00913f9740 1 -- 172.21.15.82:0/3567478134 >> 172.21.15.82:0/3567478134 conn(0x56479c210ba0 msgr2=0x56479c210f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.070+0000 7f00913f9740 1 -- 172.21.15.82:0/3567478134 shutdown_connections 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.070+0000 7f00913f9740 1 -- 172.21.15.82:0/3567478134 wait complete. 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.070+0000 7f00913f9740 1 -- start start 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00913f9740 1 Processor -- start 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00913f9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c2164d0 0x56479c21f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00913f9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c216e80 0x56479c21c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00913f9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c21fa00 0x56479c21cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00913f9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56479c222910 con 0x56479c216e80 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00913f9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56479c134500 con 0x56479c21fa00 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00913f9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56479c222a90 con 0x56479c2164d0 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00913f9740 10 client.0 _ll_register_callbacks cb 0x56479c102b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00877fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c216e80 0x56479c21c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00877fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c216e80 0x56479c21c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45982/0 (socket says 172.21.15.82:45982) 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00877fe640 1 -- 172.21.15.82:0/2121696409 learned_addr learned my addr 172.21.15.82:0/2121696409 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f008ccc8640 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c21fa00 0x56479c21cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f0087fff640 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c2164d0 0x56479c21f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00877fe640 1 -- 172.21.15.82:0/2121696409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c2164d0 msgr2=0x56479c21f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00877fe640 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c2164d0 0x56479c21f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00877fe640 1 -- 172.21.15.82:0/2121696409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c21fa00 msgr2=0x56479c21cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00877fe640 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c21fa00 0x56479c21cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00877fe640 1 -- 172.21.15.82:0/2121696409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56479be8d3f0 con 0x56479c216e80 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00877fe640 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c216e80 0x56479c21c650 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f007c009900 tx=0x7f007c02cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53749abd4ae50f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.071+0000 7f00677fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00677fe640 1 -- 172.21.15.82:0/2121696409 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c045070 con 0x56479c216e80 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00677fe640 1 -- 172.21.15.82:0/2121696409 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f007c0035f0 con 0x56479c216e80 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00913f9740 1 -- 172.21.15.82:0/2121696409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f007c02c7e0 con 0x56479c216e80 2024-04-21T08:54:27.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00677fe640 1 -- 172.21.15.82:0/2121696409 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f007c059050 con 0x56479c216e80 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00913f9740 10 client.17706 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00913f9740 1 -- 172.21.15.82:0/2121696409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56479bf577d0 con 0x56479c216e80 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00677fe640 1 -- 172.21.15.82:0/2121696409 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f007c02c7e0 con 0x56479c216e80 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00677fe640 1 -- 172.21.15.82:0/2121696409 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f007c054030 con 0x56479c216e80 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00913f9740 10 client.17706 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00913f9740 10 client.17706 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00913f9740 20 client.17706 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00913f9740 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56479c22fa00 0x56479c231df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00913f9740 4 client.17706 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f00913f9740 1 -- 172.21.15.82:0/2121696409 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56479c11a930 con 0x56479c22fa00 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.072+0000 7f0087fff640 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56479c22fa00 0x56479c231df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.073+0000 7f0087fff640 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56479c22fa00 0x56479c231df0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c122dbfdaaa9a660 server_cookie=8ea28b34683a7887 in_seq=0 out_seq=0 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.073+0000 7f00677fe640 10 client.17706 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00677fe640 1 -- 172.21.15.82:0/2121696409 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56479c11a930 con 0x56479c22fa00 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00677fe640 10 client.17706 handle_command_reply: tid=0 2024-04-21T08:54:27.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00913f9740 1 client.17706 shutdown 2024-04-21T08:54:27.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00913f9740 1 -- 172.21.15.82:0/2121696409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c216e80 msgr2=0x56479c21c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00913f9740 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c216e80 0x56479c21c650 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f007c009900 tx=0x7f007c02cbc0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00913f9740 1 -- 172.21.15.82:0/2121696409 shutdown_connections 2024-04-21T08:54:27.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00913f9740 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56479c22fa00 0x56479c231df0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00913f9740 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c21fa00 0x56479c21cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00913f9740 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c216e80 0x56479c21c650 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00913f9740 1 --2- 172.21.15.82:0/2121696409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c2164d0 0x56479c21f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00913f9740 1 -- 172.21.15.82:0/2121696409 >> 172.21.15.82:0/2121696409 conn(0x56479c210ba0 msgr2=0x56479c2128f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00677fe640 0 client.17706 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:54:27.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00913f9740 1 -- 172.21.15.82:0/2121696409 shutdown_connections 2024-04-21T08:54:27.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00913f9740 1 -- 172.21.15.82:0/2121696409 wait complete. 2024-04-21T08:54:27.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.074+0000 7f00913f9740 20 client.17706 trim_cache size 0 max 16384 2024-04-21T08:54:27.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00913f9740 1 Processor -- start 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00913f9740 1 -- start start 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00913f9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c2164d0 0x56479c22b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00913f9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 0x56479c231a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00913f9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c21ed80 0x56479c21f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00913f9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f00881b1600 con 0x56479c21fa00 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00913f9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f00881b3230 con 0x56479c2164d0 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00913f9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00881b1cd0 con 0x56479c21ed80 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00877fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 0x56479c231a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00877fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 0x56479c231a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45998/0 (socket says 172.21.15.82:45998) 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00877fe640 1 -- 172.21.15.82:0/405569602 learned_addr learned my addr 172.21.15.82:0/405569602 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f0087fff640 1 --2- 172.21.15.82:0/405569602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c2164d0 0x56479c22b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f008ccc8640 1 --2- 172.21.15.82:0/405569602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c21ed80 0x56479c21f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00877fe640 1 -- 172.21.15.82:0/405569602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c21ed80 msgr2=0x56479c21f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00877fe640 1 --2- 172.21.15.82:0/405569602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c21ed80 0x56479c21f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00877fe640 1 -- 172.21.15.82:0/405569602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c2164d0 msgr2=0x56479c22b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.088+0000 7f00877fe640 1 --2- 172.21.15.82:0/405569602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c2164d0 0x56479c22b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f00877fe640 1 -- 172.21.15.82:0/405569602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56479c21cf20 con 0x56479c21fa00 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f008ccc8640 1 --2- 172.21.15.82:0/405569602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c21ed80 0x56479c21f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f00877fe640 1 --2- 172.21.15.82:0/405569602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 0x56479c231a10 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f007c036040 tx=0x7f007c02c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a2f28d2f3af8cc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f006dffb640 1 -- 172.21.15.82:0/405569602 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c045040 con 0x56479c21fa00 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f006dffb640 1 -- 172.21.15.82:0/405569602 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f007c02c7e0 con 0x56479c21fa00 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f006dffb640 1 -- 172.21.15.82:0/405569602 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c041cc0 con 0x56479c21fa00 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f00913f9740 1 -- 172.21.15.82:0/405569602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 msgr2=0x56479c231a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f00913f9740 1 --2- 172.21.15.82:0/405569602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 0x56479c231a10 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f007c036040 tx=0x7f007c02c3b0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f00913f9740 1 -- 172.21.15.82:0/405569602 shutdown_connections 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f00913f9740 1 --2- 172.21.15.82:0/405569602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c21ed80 0x56479c21f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f00913f9740 1 --2- 172.21.15.82:0/405569602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c21fa00 0x56479c231a10 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f00913f9740 1 --2- 172.21.15.82:0/405569602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c2164d0 0x56479c22b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f00913f9740 1 -- 172.21.15.82:0/405569602 >> 172.21.15.82:0/405569602 conn(0x56479c210ba0 msgr2=0x56479c20fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f00913f9740 1 -- 172.21.15.82:0/405569602 shutdown_connections 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.089+0000 7f00913f9740 1 -- 172.21.15.82:0/405569602 wait complete. 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f00913f9740 1 -- start start 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f00913f9740 1 Processor -- start 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f00913f9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c2164d0 0x56479c223640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f00913f9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c21ed80 0x56479c223b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f00913f9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c21fa00 0x56479c2240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f00913f9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f00881b2b30 con 0x56479c2164d0 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f00913f9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f00881b2430 con 0x56479c21ed80 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f00913f9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00881af5d0 con 0x56479c21fa00 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f00913f9740 10 client.0 _ll_register_callbacks cb 0x7f007c0056c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f0087fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c2164d0 0x56479c223640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f0087fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c2164d0 0x56479c223640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46006/0 (socket says 172.21.15.82:46006) 2024-04-21T08:54:27.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f0087fff640 1 -- 172.21.15.82:0/2225042324 learned_addr learned my addr 172.21.15.82:0/2225042324 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f00877fe640 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c21ed80 0x56479c223b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.090+0000 7f008ccc8640 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c21fa00 0x56479c2240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f0087fff640 1 -- 172.21.15.82:0/2225042324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c21fa00 msgr2=0x56479c2240c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f0087fff640 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c21fa00 0x56479c2240c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f0087fff640 1 -- 172.21.15.82:0/2225042324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c21ed80 msgr2=0x56479c223b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f0087fff640 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c21ed80 0x56479c223b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f0087fff640 1 -- 172.21.15.82:0/2225042324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56479c294d80 con 0x56479c2164d0 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f0087fff640 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c2164d0 0x56479c223640 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0078099550 tx=0x7f00780d3450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2315e576d8c09f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f00677fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f00677fe640 1 -- 172.21.15.82:0/2225042324 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078010d70 con 0x56479c2164d0 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f00677fe640 1 -- 172.21.15.82:0/2225042324 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f007809ae80 con 0x56479c2164d0 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f00913f9740 1 -- 172.21.15.82:0/2225042324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f007c02c7e0 con 0x56479c2164d0 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f00677fe640 1 -- 172.21.15.82:0/2225042324 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0078010a90 con 0x56479c2164d0 2024-04-21T08:54:27.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f00913f9740 10 client.17712 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:27.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f00913f9740 1 -- 172.21.15.82:0/2225042324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56479c295f90 con 0x56479c2164d0 2024-04-21T08:54:27.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f00677fe640 1 -- 172.21.15.82:0/2225042324 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f007c02c7e0 con 0x56479c2164d0 2024-04-21T08:54:27.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.091+0000 7f00677fe640 1 -- 172.21.15.82:0/2225042324 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f00780d30b0 con 0x56479c2164d0 2024-04-21T08:54:27.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.092+0000 7f00913f9740 10 client.17712 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:27.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.092+0000 7f00913f9740 10 client.17712 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:54:27.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.092+0000 7f00913f9740 20 client.17712 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:27.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.092+0000 7f00913f9740 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56479c2967c0 0x56479c298bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.092+0000 7f00913f9740 4 client.17712 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:54:27.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.092+0000 7f00913f9740 1 -- 172.21.15.82:0/2225042324 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56479c188c10 con 0x56479c2967c0 2024-04-21T08:54:27.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.092+0000 7f00877fe640 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56479c2967c0 0x56479c298bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.092+0000 7f00877fe640 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56479c2967c0 0x56479c298bb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=891763a818c41eba server_cookie=4e2037df91e8dbd7 in_seq=0 out_seq=0 2024-04-21T08:54:27.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.092+0000 7f00677fe640 10 client.17712 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.093+0000 7f00677fe640 1 -- 172.21.15.82:0/2225042324 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56479c188c10 con 0x56479c2967c0 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.093+0000 7f00677fe640 10 client.17712 handle_command_reply: tid=0 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.093+0000 7f00913f9740 1 client.17712 shutdown 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f00913f9740 1 -- 172.21.15.82:0/2225042324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c2164d0 msgr2=0x56479c223640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f00913f9740 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c2164d0 0x56479c223640 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0078099550 tx=0x7f00780d3450 comp rx=0 tx=0).stop 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f00913f9740 1 -- 172.21.15.82:0/2225042324 shutdown_connections 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f00913f9740 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56479c2967c0 0x56479c298bb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f00913f9740 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56479c21fa00 0x56479c2240c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f00913f9740 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56479c21ed80 0x56479c223b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f00677fe640 0 client.17712 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f00913f9740 1 --2- 172.21.15.82:0/2225042324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56479c2164d0 0x56479c223640 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f00913f9740 1 -- 172.21.15.82:0/2225042324 >> 172.21.15.82:0/2225042324 conn(0x56479c210ba0 msgr2=0x56479c21dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f00913f9740 1 -- 172.21.15.82:0/2225042324 shutdown_connections 2024-04-21T08:54:27.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f00913f9740 1 -- 172.21.15.82:0/2225042324 wait complete. 2024-04-21T08:54:27.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f00913f9740 20 client.17712 trim_cache size 0 max 16384 2024-04-21T08:54:27.100 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:54:27.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f008e72b640 1 -- 172.21.15.82:0/2913499588 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f00700a3d20 msgr2=0x7f00700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f008e72b640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f00700a3d20 0x7f00700a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f00800125f0 tx=0x7f0080004a40 comp rx=0 tx=0).stop 2024-04-21T08:54:27.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f008e72b640 1 -- 172.21.15.82:0/2913499588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00881010a0 msgr2=0x7f0088197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f008e72b640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00881010a0 0x7f0088197380 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f007800ea20 tx=0x7f007800eee0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f008e72b640 1 -- 172.21.15.82:0/2913499588 shutdown_connections 2024-04-21T08:54:27.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f008e72b640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0088197e00 0x7f00881a6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f008e72b640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0088106170 0x7f00881978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f008e72b640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f00700a3d20 0x7f00700a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f008e72b640 1 --2- 172.21.15.82:0/2913499588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00881010a0 0x7f0088197380 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f008e72b640 1 -- 172.21.15.82:0/2913499588 >> 172.21.15.82:0/2913499588 conn(0x7f0088077eb0 msgr2=0x7f00880ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.094+0000 7f008e72b640 1 -- 172.21.15.82:0/2913499588 shutdown_connections 2024-04-21T08:54:27.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.095+0000 7f008e72b640 1 -- 172.21.15.82:0/2913499588 wait complete. 2024-04-21T08:54:27.105 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:54:27.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.214+0000 7f2b89159640 1 -- 172.21.15.82:0/1764033499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b841021c0 msgr2=0x7f2b841025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.214+0000 7f2b89159640 1 --2- 172.21.15.82:0/1764033499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b841021c0 0x7f2b841025a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009900 tx=0x7f2b7402bc30 comp rx=0 tx=0).stop 2024-04-21T08:54:27.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.214+0000 7f2b89159640 1 -- 172.21.15.82:0/1764033499 shutdown_connections 2024-04-21T08:54:27.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.214+0000 7f2b89159640 1 --2- 172.21.15.82:0/1764033499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b84107290 0x7f2b8410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.214+0000 7f2b89159640 1 --2- 172.21.15.82:0/1764033499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b84102b70 0x7f2b84106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.214+0000 7f2b89159640 1 --2- 172.21.15.82:0/1764033499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b841021c0 0x7f2b841025a0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.214+0000 7f2b89159640 1 -- 172.21.15.82:0/1764033499 >> 172.21.15.82:0/1764033499 conn(0x7f2b840793e0 msgr2=0x7f2b840797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.214+0000 7f2b89159640 1 -- 172.21.15.82:0/1764033499 shutdown_connections 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.214+0000 7f2b89159640 1 -- 172.21.15.82:0/1764033499 wait complete. 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b89159640 1 Processor -- start 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b89159640 1 -- start start 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b89159640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b841021c0 0x7f2b8419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b89159640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b84107290 0x7f2b8419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b89159640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84198610 0x7f2b84198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b89159640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841121d0 con 0x7f2b84198610 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b89159640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2b84112050 con 0x7f2b84107290 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b89159640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b84112350 con 0x7f2b841021c0 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b8359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84198610 0x7f2b84198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b8359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84198610 0x7f2b84198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46024/0 (socket says 172.21.15.82:46024) 2024-04-21T08:54:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b8359e640 1 -- 172.21.15.82:0/4001870538 learned_addr learned my addr 172.21.15.82:0/4001870538 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b841021c0 0x7f2b8419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b7bfff640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b84107290 0x7f2b8419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b8359e640 1 -- 172.21.15.82:0/4001870538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b841021c0 msgr2=0x7f2b8419e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b8359e640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b841021c0 0x7f2b8419e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b8359e640 1 -- 172.21.15.82:0/4001870538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b84107290 msgr2=0x7f2b8419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b8359e640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b84107290 0x7f2b8419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b8359e640 1 -- 172.21.15.82:0/4001870538 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b84199130 con 0x7f2b84198610 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b8359e640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84198610 0x7f2b84198ac0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00ea60 tx=0x7f2b7c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b80d99640 1 -- 172.21.15.82:0/4001870538 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c00ce40 con 0x7f2b84198610 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b89159640 1 -- 172.21.15.82:0/4001870538 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b84199420 con 0x7f2b84198610 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b80d99640 1 -- 172.21.15.82:0/4001870538 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b7c005020 con 0x7f2b84198610 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.215+0000 7f2b80d99640 1 -- 172.21.15.82:0/4001870538 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c010670 con 0x7f2b84198610 2024-04-21T08:54:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.216+0000 7f2b89159640 1 -- 172.21.15.82:0/4001870538 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b841a5670 con 0x7f2b84198610 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.217+0000 7f2b80d99640 1 -- 172.21.15.82:0/4001870538 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2b7c0054b0 con 0x7f2b84198610 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.218+0000 7f2b8be20740 1 Processor -- start 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.218+0000 7f2b8be20740 1 -- start start 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.218+0000 7f2b80d99640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2b700a3d20 0x7f2b700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.218+0000 7f2b80d99640 1 -- 172.21.15.82:0/4001870538 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f2b7c0d1070 con 0x7f2b84198610 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.218+0000 7f2b7bfff640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2b700a3d20 0x7f2b700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.218+0000 7f2b8be20740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.218+0000 7f2b8be20740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.218+0000 7f2b8be20740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b475cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.218+0000 7f2b8be20740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562b475281f0 con 0x562b475c34d0 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.218+0000 7f2b8be20740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562b474c54b0 con 0x562b475c3e80 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.218+0000 7f2b8be20740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562b475241b0 con 0x562b475cca00 2024-04-21T08:54:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.218+0000 7f2b82d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b7bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b82d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475c38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46036/0 (socket says 172.21.15.82:46036) 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b82d9d640 1 -- 172.21.15.82:0/3850989810 learned_addr learned my addr 172.21.15.82:0/3850989810 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b8359e640 1 --2- 172.21.15.82:0/3850989810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b475cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b82d9d640 1 -- 172.21.15.82:0/3850989810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 msgr2=0x562b475cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b7bfff640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2b700a3d20 0x7f2b700a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2b84101920 tx=0x7f2b6c011920 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/3850989810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b475cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b82d9d640 1 -- 172.21.15.82:0/3850989810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 msgr2=0x562b475cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/3850989810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b82d9d640 1 -- 172.21.15.82:0/3850989810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b474b71b0 con 0x562b475c34d0 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b8359e640 1 --2- 172.21.15.82:0/3850989810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b475cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/3850989810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475c38b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009900 tx=0x7f2b740095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4421305d2d58de5d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b797fa640 1 -- 172.21.15.82:0/3850989810 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74045020 con 0x562b475c34d0 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b797fa640 1 -- 172.21.15.82:0/3850989810 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b7402c7e0 con 0x562b475c34d0 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.219+0000 7f2b797fa640 1 -- 172.21.15.82:0/3850989810 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b740054c0 con 0x562b475c34d0 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.220+0000 7f2b8be20740 1 -- 172.21.15.82:0/3850989810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 msgr2=0x562b475c38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.220+0000 7f2b8be20740 1 --2- 172.21.15.82:0/3850989810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475c38b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009900 tx=0x7f2b740095f0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.220+0000 7f2b8be20740 1 -- 172.21.15.82:0/3850989810 shutdown_connections 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.220+0000 7f2b8be20740 1 --2- 172.21.15.82:0/3850989810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b475cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.220+0000 7f2b8be20740 1 --2- 172.21.15.82:0/3850989810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.220+0000 7f2b8be20740 1 --2- 172.21.15.82:0/3850989810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475c38b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.220+0000 7f2b8be20740 1 -- 172.21.15.82:0/3850989810 >> 172.21.15.82:0/3850989810 conn(0x562b475bdba0 msgr2=0x562b475bdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.220+0000 7f2b8be20740 1 -- 172.21.15.82:0/3850989810 shutdown_connections 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.220+0000 7f2b8be20740 1 -- 172.21.15.82:0/3850989810 wait complete. 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.220+0000 7f2b8be20740 1 -- start start 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.220+0000 7f2b8be20740 1 Processor -- start 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b8be20740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b8be20740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b8be20740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b475c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b8be20740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562b475cf910 con 0x562b475c34d0 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b8be20740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562b474e1500 con 0x562b475c3e80 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b8be20740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562b475cfa90 con 0x562b475cca00 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b82d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b8be20740 10 client.0 _ll_register_callbacks cb 0x562b474afb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b82d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46044/0 (socket says 172.21.15.82:46044) 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b82d9d640 1 -- 172.21.15.82:0/1388871691 learned_addr learned my addr 172.21.15.82:0/1388871691 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b7bfff640 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b8359e640 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b475c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b82d9d640 1 -- 172.21.15.82:0/1388871691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 msgr2=0x562b475c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b475c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b82d9d640 1 -- 172.21.15.82:0/1388871691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 msgr2=0x562b475c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b82d9d640 1 -- 172.21.15.82:0/1388871691 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4723a3f0 con 0x562b475c34d0 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b7bfff640 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475cc330 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2b74036040 tx=0x7f2b7402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60dc8a715a998343 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b6a7fc640 1 -- 172.21.15.82:0/1388871691 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74004460 con 0x562b475c34d0 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b6a7fc640 1 -- 172.21.15.82:0/1388871691 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f2b740046c0 con 0x562b475c34d0 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b8be20740 1 -- 172.21.15.82:0/1388871691 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b7402c7e0 con 0x562b475c34d0 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.221+0000 7f2b6a7fc640 1 -- 172.21.15.82:0/1388871691 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2b74058050 con 0x562b475c34d0 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.222+0000 7f2b8be20740 10 client.17724 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.222+0000 7f2b8be20740 1 -- 172.21.15.82:0/1388871691 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562b473047d0 con 0x562b475c34d0 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.222+0000 7f2b6a7fc640 1 -- 172.21.15.82:0/1388871691 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b7402c7e0 con 0x562b475c34d0 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.222+0000 7f2b6a7fc640 1 -- 172.21.15.82:0/1388871691 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f2b74003a30 con 0x562b475c34d0 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.222+0000 7f2b8be20740 10 client.17724 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.222+0000 7f2b8be20740 10 client.17724 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.222+0000 7f2b8be20740 20 client.17724 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.222+0000 7f2b8be20740 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562b475dc9d0 0x562b475dedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.222+0000 7f2b8be20740 4 client.17724 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.222+0000 7f2b8be20740 1 -- 172.21.15.82:0/1388871691 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b474c7930 con 0x562b475dc9d0 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.222+0000 7f2b7bfff640 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562b475dc9d0 0x562b475dedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b6a7fc640 10 client.17724 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b7bfff640 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562b475dc9d0 0x562b475dedc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f82f6c2802118c46 server_cookie=413ba385dfebde99 in_seq=0 out_seq=0 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b6a7fc640 1 -- 172.21.15.82:0/1388871691 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b474c7930 con 0x562b475dc9d0 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b6a7fc640 10 client.17724 handle_command_reply: tid=0 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b8be20740 1 client.17724 shutdown 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b8be20740 1 -- 172.21.15.82:0/1388871691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 msgr2=0x562b475cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b8be20740 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475cc330 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2b74036040 tx=0x7f2b7402bc30 comp rx=0 tx=0).stop 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b8be20740 1 -- 172.21.15.82:0/1388871691 shutdown_connections 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b8be20740 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562b475dc9d0 0x562b475dedc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b8be20740 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b475c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b8be20740 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b8be20740 1 --2- 172.21.15.82:0/1388871691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475cc330 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b6a7fc640 0 client.17724 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:54:27.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b8be20740 1 -- 172.21.15.82:0/1388871691 >> 172.21.15.82:0/1388871691 conn(0x562b475bdba0 msgr2=0x562b475bf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b8be20740 1 -- 172.21.15.82:0/1388871691 shutdown_connections 2024-04-21T08:54:27.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.223+0000 7f2b8be20740 1 -- 172.21.15.82:0/1388871691 wait complete. 2024-04-21T08:54:27.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.224+0000 7f2b8be20740 20 client.17724 trim_cache size 0 max 16384 2024-04-21T08:54:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.237+0000 7f2b8be20740 1 Processor -- start 2024-04-21T08:54:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.237+0000 7f2b8be20740 1 -- start start 2024-04-21T08:54:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b8be20740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475d87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b8be20740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475c3e80 0x562b475c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b8be20740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475cca00 0x562b475bca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b8be20740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841af490 con 0x562b475c34d0 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b8be20740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841b10c0 con 0x562b475cca00 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b8be20740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841afb60 con 0x562b475c3e80 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b82d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475d87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b82d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475d87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46052/0 (socket says 172.21.15.82:46052) 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b82d9d640 1 -- 172.21.15.82:0/439519621 learned_addr learned my addr 172.21.15.82:0/439519621 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b8359e640 1 --2- 172.21.15.82:0/439519621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475cca00 0x562b475bca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b82d9d640 1 -- 172.21.15.82:0/439519621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475c3e80 msgr2=0x562b475c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b7bfff640 1 --2- 172.21.15.82:0/439519621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475c3e80 0x562b475c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/439519621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475c3e80 0x562b475c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b82d9d640 1 -- 172.21.15.82:0/439519621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475cca00 msgr2=0x562b475bca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/439519621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475cca00 0x562b475bca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b82d9d640 1 -- 172.21.15.82:0/439519621 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b475b9470 con 0x562b475c34d0 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/439519621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475d87d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2b7402ce10 tx=0x7f2b74000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51869945715838d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.238+0000 7f2b78ff9640 1 -- 172.21.15.82:0/439519621 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74045040 con 0x562b475c34d0 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.239+0000 7f2b78ff9640 1 -- 172.21.15.82:0/439519621 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b7402c7e0 con 0x562b475c34d0 2024-04-21T08:54:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.239+0000 7f2b78ff9640 1 -- 172.21.15.82:0/439519621 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b740044c0 con 0x562b475c34d0 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.239+0000 7f2b8be20740 1 -- 172.21.15.82:0/439519621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 msgr2=0x562b475d87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.239+0000 7f2b8be20740 1 --2- 172.21.15.82:0/439519621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475d87d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2b7402ce10 tx=0x7f2b74000bc0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.239+0000 7f2b8be20740 1 -- 172.21.15.82:0/439519621 shutdown_connections 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.239+0000 7f2b8be20740 1 --2- 172.21.15.82:0/439519621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475cca00 0x562b475bca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.239+0000 7f2b8be20740 1 --2- 172.21.15.82:0/439519621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475c3e80 0x562b475c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.239+0000 7f2b8be20740 1 --2- 172.21.15.82:0/439519621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475d87d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.239+0000 7f2b8be20740 1 -- 172.21.15.82:0/439519621 >> 172.21.15.82:0/439519621 conn(0x562b475bdba0 msgr2=0x562b475bc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.239+0000 7f2b8be20740 1 -- 172.21.15.82:0/439519621 shutdown_connections 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.239+0000 7f2b8be20740 1 -- 172.21.15.82:0/439519621 wait complete. 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b8be20740 1 -- start start 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b8be20740 1 Processor -- start 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b8be20740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b8be20740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b8be20740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b47640c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b8be20740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841b09c0 con 0x562b475c34d0 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b8be20740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841b02c0 con 0x562b475c3e80 2024-04-21T08:54:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b8be20740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b841ad460 con 0x562b475cca00 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b8be20740 10 client.0 _ll_register_callbacks cb 0x7f2b7402ccc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b82d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b82d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46056/0 (socket says 172.21.15.82:46056) 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b82d9d640 1 -- 172.21.15.82:0/3275095848 learned_addr learned my addr 172.21.15.82:0/3275095848 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b7bfff640 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b8359e640 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b47640c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b82d9d640 1 -- 172.21.15.82:0/3275095848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 msgr2=0x562b47640c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b47640c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b82d9d640 1 -- 172.21.15.82:0/3275095848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 msgr2=0x562b475d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b82d9d640 1 -- 172.21.15.82:0/3275095848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b476411c0 con 0x562b475c34d0 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b8359e640 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b47640c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.240+0000 7f2b7bfff640 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:27.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b82d9d640 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475cfe90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2b740059d0 tx=0x7f2b740056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b35720716cee55c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b6a7fc640 1 -- 172.21.15.82:0/3275095848 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74045040 con 0x562b475c34d0 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b6a7fc640 1 -- 172.21.15.82:0/3275095848 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f2b740035f0 con 0x562b475c34d0 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b8be20740 1 -- 172.21.15.82:0/3275095848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b7402c7e0 con 0x562b475c34d0 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b6a7fc640 1 -- 172.21.15.82:0/3275095848 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2b7406e050 con 0x562b475c34d0 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b8be20740 10 client.17730 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b8be20740 1 -- 172.21.15.82:0/3275095848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562b476424a0 con 0x562b475c34d0 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b6a7fc640 1 -- 172.21.15.82:0/3275095848 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b7402c7e0 con 0x562b475c34d0 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b6a7fc640 1 -- 172.21.15.82:0/3275095848 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f2b740046c0 con 0x562b475c34d0 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b8be20740 10 client.17730 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b8be20740 10 client.17730 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b8be20740 20 client.17730 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b8be20740 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562b47642b30 0x562b47644f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b8be20740 4 client.17730 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:54:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b8be20740 1 -- 172.21.15.82:0/3275095848 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b47535c10 con 0x562b47642b30 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.241+0000 7f2b7bfff640 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562b47642b30 0x562b47644f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b7bfff640 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562b47642b30 0x562b47644f50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e00a638513f18b64 server_cookie=84937dd7cc09fa53 in_seq=0 out_seq=0 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b6a7fc640 10 client.17730 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b6a7fc640 1 -- 172.21.15.82:0/3275095848 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b47535c10 con 0x562b47642b30 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b6a7fc640 10 client.17730 handle_command_reply: tid=0 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b8be20740 1 client.17730 shutdown 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b8be20740 1 -- 172.21.15.82:0/3275095848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 msgr2=0x562b475cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b8be20740 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475cfe90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2b740059d0 tx=0x7f2b740056a0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b8be20740 1 -- 172.21.15.82:0/3275095848 shutdown_connections 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b8be20740 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x562b47642b30 0x562b47644f50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b8be20740 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562b475cca00 0x562b47640c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b8be20740 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b475c3e80 0x562b475d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b6a7fc640 0 client.17730 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b8be20740 1 --2- 172.21.15.82:0/3275095848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562b475c34d0 0x562b475cfe90 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b8be20740 1 -- 172.21.15.82:0/3275095848 >> 172.21.15.82:0/3275095848 conn(0x562b475bdba0 msgr2=0x562b475befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b8be20740 1 -- 172.21.15.82:0/3275095848 shutdown_connections 2024-04-21T08:54:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b8be20740 1 -- 172.21.15.82:0/3275095848 wait complete. 2024-04-21T08:54:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.242+0000 7f2b8be20740 20 client.17730 trim_cache size 0 max 16384 2024-04-21T08:54:27.249 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:54:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.243+0000 7f2b89159640 1 -- 172.21.15.82:0/4001870538 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2b700a3d20 msgr2=0x7f2b700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.243+0000 7f2b89159640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2b700a3d20 0x7f2b700a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2b84101920 tx=0x7f2b6c011920 comp rx=0 tx=0).stop 2024-04-21T08:54:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.243+0000 7f2b89159640 1 -- 172.21.15.82:0/4001870538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84198610 msgr2=0x7f2b84198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.243+0000 7f2b89159640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84198610 0x7f2b84198ac0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00ea60 tx=0x7f2b7c00ef20 comp rx=0 tx=0).stop 2024-04-21T08:54:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.243+0000 7f2b89159640 1 -- 172.21.15.82:0/4001870538 shutdown_connections 2024-04-21T08:54:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.243+0000 7f2b89159640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2b700a3d20 0x7f2b700a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.243+0000 7f2b89159640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b84198610 0x7f2b84198ac0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.243+0000 7f2b89159640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b84107290 0x7f2b8419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.243+0000 7f2b89159640 1 --2- 172.21.15.82:0/4001870538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b841021c0 0x7f2b8419e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.243+0000 7f2b89159640 1 -- 172.21.15.82:0/4001870538 >> 172.21.15.82:0/4001870538 conn(0x7f2b840793e0 msgr2=0x7f2b84100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.243+0000 7f2b89159640 1 -- 172.21.15.82:0/4001870538 shutdown_connections 2024-04-21T08:54:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.243+0000 7f2b89159640 1 -- 172.21.15.82:0/4001870538 wait complete. 2024-04-21T08:54:27.254 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:54:27.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.357+0000 7f14d3cd9640 1 -- 172.21.15.82:0/1891959920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc101a40 msgr2=0x7f14cc105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.357+0000 7f14d3cd9640 1 --2- 172.21.15.82:0/1891959920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc101a40 0x7f14cc105a80 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009900 tx=0x7f14bc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.357+0000 7f14d3cd9640 1 -- 172.21.15.82:0/1891959920 shutdown_connections 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.357+0000 7f14d3cd9640 1 --2- 172.21.15.82:0/1891959920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc106160 0x7f14cc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.357+0000 7f14d3cd9640 1 --2- 172.21.15.82:0/1891959920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc101a40 0x7f14cc105a80 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.357+0000 7f14d3cd9640 1 --2- 172.21.15.82:0/1891959920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14cc101090 0x7f14cc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.357+0000 7f14d3cd9640 1 -- 172.21.15.82:0/1891959920 >> 172.21.15.82:0/1891959920 conn(0x7f14cc077f50 msgr2=0x7f14cc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.357+0000 7f14d3cd9640 1 -- 172.21.15.82:0/1891959920 shutdown_connections 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.357+0000 7f14d3cd9640 1 -- 172.21.15.82:0/1891959920 wait complete. 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.357+0000 7f14d3cd9640 1 Processor -- start 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.357+0000 7f14d3cd9640 1 -- start start 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d3cd9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14cc101090 0x7f14cc197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d3cd9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc106160 0x7f14cc197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d3cd9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc19bf20 0x7f14cc19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d3cd9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1110a0 con 0x7f14cc19bf20 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d3cd9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc110f20 con 0x7f14cc101090 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d3cd9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc111220 con 0x7f14cc106160 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d2276640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc19bf20 0x7f14cc19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d1a75640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14cc101090 0x7f14cc197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d2276640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc19bf20 0x7f14cc19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46084/0 (socket says 172.21.15.82:46084) 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d2276640 1 -- 172.21.15.82:0/2906006623 learned_addr learned my addr 172.21.15.82:0/2906006623 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d1274640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc106160 0x7f14cc197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d1a75640 1 -- 172.21.15.82:0/2906006623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc106160 msgr2=0x7f14cc197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d1a75640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc106160 0x7f14cc197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d1a75640 1 -- 172.21.15.82:0/2906006623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc19bf20 msgr2=0x7f14cc19c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d1a75640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc19bf20 0x7f14cc19c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d1a75640 1 -- 172.21.15.82:0/2906006623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14cc19ca10 con 0x7f14cc101090 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d2276640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc19bf20 0x7f14cc19c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d1a75640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14cc101090 0x7f14cc197660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f14c800e9f0 tx=0x7f14c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14c2ffd640 1 -- 172.21.15.82:0/2906006623 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c800ccc0 con 0x7f14cc101090 2024-04-21T08:54:27.361 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:27 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3962483775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:54:27.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14c2ffd640 1 -- 172.21.15.82:0/2906006623 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c80108e0 con 0x7f14cc101090 2024-04-21T08:54:27.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14d3cd9640 1 -- 172.21.15.82:0/2906006623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14cc19cd00 con 0x7f14cc101090 2024-04-21T08:54:27.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.358+0000 7f14c2ffd640 1 -- 172.21.15.82:0/2906006623 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8018880 con 0x7f14cc101090 2024-04-21T08:54:27.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.359+0000 7f14d3cd9640 1 -- 172.21.15.82:0/2906006623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14cc1a4510 con 0x7f14cc101090 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.361+0000 7f14c2ffd640 1 -- 172.21.15.82:0/2906006623 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14c8010430 con 0x7f14cc101090 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.361+0000 7f14d69aa740 1 Processor -- start 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.361+0000 7f14d69aa740 1 -- start start 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.361+0000 7f14d69aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e330f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.361+0000 7f14d69aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3318430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.361+0000 7f14d69aa740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 0x55d2e331ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.361+0000 7f14d69aa740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d2e32741f0 con 0x55d2e3318a00 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.361+0000 7f14d69aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d2e32114b0 con 0x55d2e330fe80 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.361+0000 7f14d69aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d2e32701b0 con 0x55d2e330f4d0 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d1a75640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3318430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d1274640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 0x55d2e331ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d1a75640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3318430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:41900/0 (socket says 172.21.15.82:41900) 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d1a75640 1 -- 172.21.15.82:0/477562458 learned_addr learned my addr 172.21.15.82:0/477562458 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d2276640 1 --2- 172.21.15.82:0/477562458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e330f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14c2ffd640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f14b80a3da0 0x7f14b80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d1274640 1 -- 172.21.15.82:0/477562458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 msgr2=0x55d2e330f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d1274640 1 --2- 172.21.15.82:0/477562458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e330f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d1274640 1 -- 172.21.15.82:0/477562458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 msgr2=0x55d2e3318430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d1274640 1 --2- 172.21.15.82:0/477562458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3318430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d1274640 1 -- 172.21.15.82:0/477562458 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2e32031b0 con 0x55d2e3318a00 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d1a75640 1 --2- 172.21.15.82:0/477562458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3318430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d2276640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f14b80a3da0 0x7f14b80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14c2ffd640 1 -- 172.21.15.82:0/2906006623 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f14c80d0ef0 con 0x7f14cc101090 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14d1274640 1 --2- 172.21.15.82:0/477562458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 0x55d2e331ae50 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f14bc036040 tx=0x7f14bc0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f240947913f7cc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14b7fff640 1 -- 172.21.15.82:0/477562458 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc045020 con 0x55d2e3318a00 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14b7fff640 1 -- 172.21.15.82:0/477562458 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14bc02c7e0 con 0x55d2e3318a00 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.362+0000 7f14b7fff640 1 -- 172.21.15.82:0/477562458 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc0054c0 con 0x55d2e3318a00 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.363+0000 7f14d69aa740 1 -- 172.21.15.82:0/477562458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 msgr2=0x55d2e331ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.363+0000 7f14d2276640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f14b80a3da0 0x7f14b80a6260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f14c40049d0 tx=0x7f14c4004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.363+0000 7f14d69aa740 1 --2- 172.21.15.82:0/477562458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 0x55d2e331ae50 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f14bc036040 tx=0x7f14bc0095f0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.363+0000 7f14d69aa740 1 -- 172.21.15.82:0/477562458 shutdown_connections 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.363+0000 7f14d69aa740 1 --2- 172.21.15.82:0/477562458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 0x55d2e331ae50 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.363+0000 7f14d69aa740 1 --2- 172.21.15.82:0/477562458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3318430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.363+0000 7f14d69aa740 1 --2- 172.21.15.82:0/477562458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e330f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.363+0000 7f14d69aa740 1 -- 172.21.15.82:0/477562458 >> 172.21.15.82:0/477562458 conn(0x55d2e3309ba0 msgr2=0x55d2e3309f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.363+0000 7f14d69aa740 1 -- 172.21.15.82:0/477562458 shutdown_connections 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.363+0000 7f14d69aa740 1 -- 172.21.15.82:0/477562458 wait complete. 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d69aa740 1 -- start start 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d69aa740 1 Processor -- start 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d69aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e33207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d69aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3320d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d69aa740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 0x55d2e3317850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d69aa740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d2e331b910 con 0x55d2e3318a00 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d69aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d2e322d500 con 0x55d2e330fe80 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d69aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d2e331ba90 con 0x55d2e330f4d0 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d69aa740 10 client.0 _ll_register_callbacks cb 0x55d2e31fbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d2276640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 0x55d2e3317850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d1274640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3320d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d2276640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 0x55d2e3317850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46104/0 (socket says 172.21.15.82:46104) 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d2276640 1 -- 172.21.15.82:0/2506790765 learned_addr learned my addr 172.21.15.82:0/2506790765 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d1a75640 1 --2- 172.21.15.82:0/2506790765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e33207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d2276640 1 -- 172.21.15.82:0/2506790765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 msgr2=0x55d2e33207f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d2276640 1 --2- 172.21.15.82:0/2506790765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e33207f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d2276640 1 -- 172.21.15.82:0/2506790765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 msgr2=0x55d2e3320d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d2276640 1 --2- 172.21.15.82:0/2506790765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3320d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.364+0000 7f14d2276640 1 -- 172.21.15.82:0/2506790765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2e2f863f0 con 0x55d2e3318a00 2024-04-21T08:54:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14d2276640 1 --2- 172.21.15.82:0/2506790765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 0x55d2e3317850 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f14c4005c20 tx=0x7f14c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b9c7e81b0027fb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14b57fa640 1 -- 172.21.15.82:0/2506790765 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c401e830 con 0x55d2e3318a00 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14b57fa640 1 -- 172.21.15.82:0/2506790765 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f14c401e9d0 con 0x55d2e3318a00 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14d69aa740 1 -- 172.21.15.82:0/2506790765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14bc02c7e0 con 0x55d2e3318a00 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14b57fa640 1 -- 172.21.15.82:0/2506790765 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f14c4030050 con 0x55d2e3318a00 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14d69aa740 10 client.17742 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14d69aa740 1 -- 172.21.15.82:0/2506790765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d2e30507d0 con 0x55d2e3318a00 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14b57fa640 1 -- 172.21.15.82:0/2506790765 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14bc02c7e0 con 0x55d2e3318a00 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14b57fa640 1 -- 172.21.15.82:0/2506790765 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f14c4028ad0 con 0x55d2e3318a00 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14d69aa740 10 client.17742 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14d69aa740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14d69aa740 20 client.17742 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14d69aa740 1 --2- 172.21.15.82:0/2506790765 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2e332aed0 0x55d2e332d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14d69aa740 4 client.17742 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:27.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.365+0000 7f14d69aa740 1 -- 172.21.15.82:0/2506790765 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2e3213930 con 0x55d2e332aed0 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.366+0000 7f14d1a75640 1 --2- 172.21.15.82:0/2506790765 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2e332aed0 0x55d2e332d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.366+0000 7f14b57fa640 10 client.17742 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.366+0000 7f14d1a75640 1 --2- 172.21.15.82:0/2506790765 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2e332aed0 0x55d2e332d320 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ceb5c6d4c92ecc server_cookie=4954b11a891d12fa in_seq=0 out_seq=0 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.366+0000 7f14b57fa640 1 -- 172.21.15.82:0/2506790765 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2e3213930 con 0x55d2e332aed0 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.366+0000 7f14b57fa640 10 client.17742 handle_command_reply: tid=0 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14d69aa740 1 client.17742 shutdown 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14d69aa740 1 -- 172.21.15.82:0/2506790765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 msgr2=0x55d2e3317850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14d69aa740 1 --2- 172.21.15.82:0/2506790765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 0x55d2e3317850 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f14c4005c20 tx=0x7f14c4004ab0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14d69aa740 1 -- 172.21.15.82:0/2506790765 shutdown_connections 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14d69aa740 1 --2- 172.21.15.82:0/2506790765 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2e332aed0 0x55d2e332d320 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14d69aa740 1 --2- 172.21.15.82:0/2506790765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e3318a00 0x55d2e3317850 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14d69aa740 1 --2- 172.21.15.82:0/2506790765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3320d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14d69aa740 1 --2- 172.21.15.82:0/2506790765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e33207f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14b57fa640 0 client.17742 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14d69aa740 1 -- 172.21.15.82:0/2506790765 >> 172.21.15.82:0/2506790765 conn(0x55d2e3309ba0 msgr2=0x55d2e330b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14d69aa740 1 -- 172.21.15.82:0/2506790765 shutdown_connections 2024-04-21T08:54:27.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14d69aa740 1 -- 172.21.15.82:0/2506790765 wait complete. 2024-04-21T08:54:27.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.367+0000 7f14d69aa740 20 client.17742 trim_cache size 0 max 16384 2024-04-21T08:54:27.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d69aa740 1 Processor -- start 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d69aa740 1 -- start start 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d69aa740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330f4d0 0x55d2e3320a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d69aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3332c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d69aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e3318a00 0x55d2e3333170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d69aa740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1ad320 con 0x55d2e330f4d0 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d69aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1aef50 con 0x55d2e330fe80 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d69aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1ad9f0 con 0x55d2e3318a00 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d1a75640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330f4d0 0x55d2e3320a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d1a75640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330f4d0 0x55d2e3320a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46118/0 (socket says 172.21.15.82:46118) 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d1a75640 1 -- 172.21.15.82:0/2426070035 learned_addr learned my addr 172.21.15.82:0/2426070035 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d1274640 1 --2- 172.21.15.82:0/2426070035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3332c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d1a75640 1 -- 172.21.15.82:0/2426070035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e3318a00 msgr2=0x55d2e3333170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d1a75640 1 --2- 172.21.15.82:0/2426070035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e3318a00 0x55d2e3333170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d1a75640 1 -- 172.21.15.82:0/2426070035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 msgr2=0x55d2e3332c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d1a75640 1 --2- 172.21.15.82:0/2426070035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3332c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.381+0000 7f14d1a75640 1 -- 172.21.15.82:0/2426070035 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2e330e820 con 0x55d2e330f4d0 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14d1a75640 1 --2- 172.21.15.82:0/2426070035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330f4d0 0x55d2e3320a50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f14c809c7d0 tx=0x7f14c80bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e149ca8861fbdc89 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14b77fe640 1 -- 172.21.15.82:0/2426070035 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c809cad0 con 0x55d2e330f4d0 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14b77fe640 1 -- 172.21.15.82:0/2426070035 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c80a0dd0 con 0x55d2e330f4d0 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14b77fe640 1 -- 172.21.15.82:0/2426070035 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8099140 con 0x55d2e330f4d0 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14d69aa740 1 -- 172.21.15.82:0/2426070035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330f4d0 msgr2=0x55d2e3320a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14d69aa740 1 --2- 172.21.15.82:0/2426070035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330f4d0 0x55d2e3320a50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f14c809c7d0 tx=0x7f14c80bd040 comp rx=0 tx=0).stop 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14d69aa740 1 -- 172.21.15.82:0/2426070035 shutdown_connections 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14d69aa740 1 --2- 172.21.15.82:0/2426070035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e3318a00 0x55d2e3333170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14d69aa740 1 --2- 172.21.15.82:0/2426070035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e330fe80 0x55d2e3332c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14d69aa740 1 --2- 172.21.15.82:0/2426070035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330f4d0 0x55d2e3320a50 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14d69aa740 1 -- 172.21.15.82:0/2426070035 >> 172.21.15.82:0/2426070035 conn(0x55d2e3309ba0 msgr2=0x55d2e3320340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14d69aa740 1 -- 172.21.15.82:0/2426070035 shutdown_connections 2024-04-21T08:54:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.382+0000 7f14d69aa740 1 -- 172.21.15.82:0/2426070035 wait complete. 2024-04-21T08:54:27.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d69aa740 1 -- start start 2024-04-21T08:54:27.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d69aa740 1 Processor -- start 2024-04-21T08:54:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d69aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e3327080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d69aa740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330fe80 0x55d2e3324410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d69aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e3318a00 0x55d2e3324950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d69aa740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1ae850 con 0x55d2e330fe80 2024-04-21T08:54:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d69aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1ae150 con 0x55d2e3318a00 2024-04-21T08:54:27.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d69aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc1ab2f0 con 0x55d2e330f4d0 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d69aa740 10 client.0 _ll_register_callbacks cb 0x7f14c4013b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d1274640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330fe80 0x55d2e3324410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d1274640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330fe80 0x55d2e3324410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46128/0 (socket says 172.21.15.82:46128) 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d1274640 1 -- 172.21.15.82:0/1901416443 learned_addr learned my addr 172.21.15.82:0/1901416443 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d2276640 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e3318a00 0x55d2e3324950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.383+0000 7f14d1a75640 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e3327080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d1274640 1 -- 172.21.15.82:0/1901416443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 msgr2=0x55d2e3327080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d1274640 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e3327080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d1274640 1 -- 172.21.15.82:0/1901416443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e3318a00 msgr2=0x55d2e3324950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d1274640 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e3318a00 0x55d2e3324950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d1274640 1 -- 172.21.15.82:0/1901416443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2e3325150 con 0x55d2e330fe80 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d1a75640 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e3327080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d2276640 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e3318a00 0x55d2e3324950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d1274640 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330fe80 0x55d2e3324410 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009900 tx=0x7f14bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0527d5eee0340a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14b57fa640 1 -- 172.21.15.82:0/1901416443 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc045040 con 0x55d2e330fe80 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d69aa740 1 -- 172.21.15.82:0/1901416443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c80a0dd0 con 0x55d2e330fe80 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14b57fa640 1 -- 172.21.15.82:0/1901416443 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f14bc005350 con 0x55d2e330fe80 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14b57fa640 1 -- 172.21.15.82:0/1901416443 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f14bc05c050 con 0x55d2e330fe80 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d69aa740 10 client.17748 fetch_fsmap learned FSMap version 40 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d69aa740 1 -- 172.21.15.82:0/1901416443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d2e3391420 con 0x55d2e330fe80 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14b57fa640 1 -- 172.21.15.82:0/1901416443 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c80a0dd0 con 0x55d2e330fe80 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14b57fa640 1 -- 172.21.15.82:0/1901416443 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) ==== 8390+0+0 (secure 0 0 0) 0x7f14bc0046c0 con 0x55d2e330fe80 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d69aa740 10 client.17748 fetch_fsmap finished waiting for FSMap version 40 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d69aa740 10 client.17748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d69aa740 20 client.17748 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d69aa740 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2e3391800 0x55d2e3393c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d69aa740 4 client.17748 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.384+0000 7f14d69aa740 1 -- 172.21.15.82:0/1901416443 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2e3281c10 con 0x55d2e3391800 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d1a75640 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2e3391800 0x55d2e3393c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d1a75640 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2e3391800 0x55d2e3393c20 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5104e392ebdf5b32 server_cookie=52918616d5afe2f1 in_seq=0 out_seq=0 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14b57fa640 10 client.17748 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14b57fa640 1 -- 172.21.15.82:0/1901416443 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2e3281c10 con 0x55d2e3391800 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14b57fa640 10 client.17748 handle_command_reply: tid=0 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d69aa740 1 client.17748 shutdown 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d69aa740 1 -- 172.21.15.82:0/1901416443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330fe80 msgr2=0x55d2e3324410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d69aa740 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330fe80 0x55d2e3324410 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009900 tx=0x7f14bc004270 comp rx=0 tx=0).stop 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d69aa740 1 -- 172.21.15.82:0/1901416443 shutdown_connections 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d69aa740 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55d2e3391800 0x55d2e3393c20 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d69aa740 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2e3318a00 0x55d2e3324950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d69aa740 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e330fe80 0x55d2e3324410 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14b57fa640 0 client.17748 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d69aa740 1 --2- 172.21.15.82:0/1901416443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2e330f4d0 0x55d2e3327080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d69aa740 1 -- 172.21.15.82:0/1901416443 >> 172.21.15.82:0/1901416443 conn(0x55d2e3309ba0 msgr2=0x55d2e3310260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d69aa740 1 -- 172.21.15.82:0/1901416443 shutdown_connections 2024-04-21T08:54:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d69aa740 1 -- 172.21.15.82:0/1901416443 wait complete. 2024-04-21T08:54:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.385+0000 7f14d69aa740 20 client.17748 trim_cache size 0 max 16384 2024-04-21T08:54:27.392 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:54:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.386+0000 7f14d3cd9640 1 -- 172.21.15.82:0/2906006623 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f14b80a3da0 msgr2=0x7f14b80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.386+0000 7f14d3cd9640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f14b80a3da0 0x7f14b80a6260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f14c40049d0 tx=0x7f14c4004a40 comp rx=0 tx=0).stop 2024-04-21T08:54:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.386+0000 7f14d3cd9640 1 -- 172.21.15.82:0/2906006623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14cc101090 msgr2=0x7f14cc197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.386+0000 7f14d3cd9640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14cc101090 0x7f14cc197660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f14c800e9f0 tx=0x7f14c800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.386+0000 7f14d3cd9640 1 -- 172.21.15.82:0/2906006623 shutdown_connections 2024-04-21T08:54:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.386+0000 7f14d3cd9640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f14b80a3da0 0x7f14b80a6260 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.386+0000 7f14d3cd9640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14cc19bf20 0x7f14cc19c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.386+0000 7f14d3cd9640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14cc106160 0x7f14cc197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.386+0000 7f14d3cd9640 1 --2- 172.21.15.82:0/2906006623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14cc101090 0x7f14cc197660 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.386+0000 7f14d3cd9640 1 -- 172.21.15.82:0/2906006623 >> 172.21.15.82:0/2906006623 conn(0x7f14cc077f50 msgr2=0x7f14cc0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.386+0000 7f14d3cd9640 1 -- 172.21.15.82:0/2906006623 shutdown_connections 2024-04-21T08:54:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:27.386+0000 7f14d3cd9640 1 -- 172.21.15.82:0/2906006623 wait complete. 2024-04-21T08:54:27.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:27 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3962483775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:54:27.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:27 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3962483775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:54:28.398 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:54:28.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:54:28.399 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:54:28.419 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:28 smithi082 ceph-mon[27967]: pgmap v228: 129 pgs: 129 active+clean; 874 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 560 op/s 2024-04-21T08:54:28.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:28 smithi134 ceph-mon[31772]: pgmap v228: 129 pgs: 129 active+clean; 874 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 560 op/s 2024-04-21T08:54:28.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.560+0000 7f81828d2640 1 -- 172.21.15.82:0/1669421952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c10e7d0 msgr2=0x7f817c10ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.560+0000 7f81828d2640 1 --2- 172.21.15.82:0/1669421952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c10e7d0 0x7f817c10ec50 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8170009890 tx=0x7f817002bb90 comp rx=0 tx=0).stop 2024-04-21T08:54:28.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.561+0000 7f81828d2640 1 -- 172.21.15.82:0/1669421952 shutdown_connections 2024-04-21T08:54:28.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.561+0000 7f81828d2640 1 --2- 172.21.15.82:0/1669421952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c11b730 0x7f817c11db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.561+0000 7f81828d2640 1 --2- 172.21.15.82:0/1669421952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c10e7d0 0x7f817c10ec50 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.561+0000 7f81828d2640 1 --2- 172.21.15.82:0/1669421952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c1107a0 0x7f817c10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.561+0000 7f81828d2640 1 -- 172.21.15.82:0/1669421952 >> 172.21.15.82:0/1669421952 conn(0x7f817c06d060 msgr2=0x7f817c06d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:28.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.561+0000 7f81828d2640 1 -- 172.21.15.82:0/1669421952 shutdown_connections 2024-04-21T08:54:28.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.561+0000 7f81828d2640 1 -- 172.21.15.82:0/1669421952 wait complete. 2024-04-21T08:54:28.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.562+0000 7f81828d2640 1 Processor -- start 2024-04-21T08:54:28.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.562+0000 7f81828d2640 1 -- start start 2024-04-21T08:54:28.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.562+0000 7f81828d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c10e7d0 0x7f817c1a7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.562+0000 7f81828d2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c1107a0 0x7f817c1a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.562+0000 7f81828d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c11b730 0x7f817c1abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.562+0000 7f81828d2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f817c120fc0 con 0x7f817c1107a0 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.562+0000 7f81828d2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f817c120e40 con 0x7f817c10e7d0 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.562+0000 7f81828d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f817c121140 con 0x7f817c11b730 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.563+0000 7f8180e6f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c11b730 0x7f817c1abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.563+0000 7f8180e6f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c11b730 0x7f817c1abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:43512/0 (socket says 172.21.15.82:43512) 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.563+0000 7f8180e6f640 1 -- 172.21.15.82:0/3396956988 learned_addr learned my addr 172.21.15.82:0/3396956988 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.563+0000 7f817b7fe640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c1107a0 0x7f817c1a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.564+0000 7f817bfff640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c10e7d0 0x7f817c1a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.564+0000 7f8180e6f640 1 -- 172.21.15.82:0/3396956988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c10e7d0 msgr2=0x7f817c1a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.564+0000 7f8180e6f640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c10e7d0 0x7f817c1a7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.564+0000 7f8180e6f640 1 -- 172.21.15.82:0/3396956988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c1107a0 msgr2=0x7f817c1a7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.564+0000 7f8180e6f640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c1107a0 0x7f817c1a7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.564+0000 7f8180e6f640 1 -- 172.21.15.82:0/3396956988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817c1ac650 con 0x7f817c11b730 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.564+0000 7f817bfff640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c10e7d0 0x7f817c1a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:28.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.564+0000 7f8180e6f640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c11b730 0x7f817c1abf20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8174009fd0 tx=0x7f817400b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:28.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.565+0000 7f81797fa640 1 -- 172.21.15.82:0/3396956988 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174011830 con 0x7f817c11b730 2024-04-21T08:54:28.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.565+0000 7f81797fa640 1 -- 172.21.15.82:0/3396956988 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8174016070 con 0x7f817c11b730 2024-04-21T08:54:28.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.566+0000 7f81797fa640 1 -- 172.21.15.82:0/3396956988 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174010de0 con 0x7f817c11b730 2024-04-21T08:54:28.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.567+0000 7f81828d2640 1 -- 172.21.15.82:0/3396956988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817c1ac940 con 0x7f817c11b730 2024-04-21T08:54:28.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.567+0000 7f81828d2640 1 -- 172.21.15.82:0/3396956988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817c1b41d0 con 0x7f817c11b730 2024-04-21T08:54:28.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.570+0000 7f81855a6740 1 Processor -- start 2024-04-21T08:54:28.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f81797fa640 1 -- 172.21.15.82:0/3396956988 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8174010380 con 0x7f817c11b730 2024-04-21T08:54:28.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f81855a6740 1 -- start start 2024-04-21T08:54:28.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f81855a6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x555940845cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f81855a6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f81855a6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x555940851260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f81855a6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5559407a9f00 con 0x555940846290 2024-04-21T08:54:28.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f81855a6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555940747590 con 0x55594084ee10 2024-04-21T08:54:28.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f81855a6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5559407a5ec0 con 0x5559408458e0 2024-04-21T08:54:28.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x555940845cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x555940845cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:43524/0 (socket says 172.21.15.82:43524) 2024-04-21T08:54:28.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f817b7fe640 1 -- 172.21.15.82:0/873821031 learned_addr learned my addr 172.21.15.82:0/873821031 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:28.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f817b7fe640 1 --2- 172.21.15.82:0/873821031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x555940851260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f817bfff640 1 --2- 172.21.15.82:0/873821031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.571+0000 7f81797fa640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f81640a3c40 0x7f81640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.572+0000 7f817bfff640 1 -- 172.21.15.82:0/873821031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 msgr2=0x555940845cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.572+0000 7f817bfff640 1 --2- 172.21.15.82:0/873821031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x555940845cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.572+0000 7f817bfff640 1 -- 172.21.15.82:0/873821031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 msgr2=0x555940851260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.572+0000 7f817bfff640 1 --2- 172.21.15.82:0/873821031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x555940851260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.572+0000 7f81797fa640 1 -- 172.21.15.82:0/3396956988 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f81740cfb70 con 0x7f817c11b730 2024-04-21T08:54:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.572+0000 7f817bfff640 1 -- 172.21.15.82:0/873821031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559405847e0 con 0x555940846290 2024-04-21T08:54:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.572+0000 7f8180e6f640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f81640a3c40 0x7f81640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.572+0000 7f817bfff640 1 --2- 172.21.15.82:0/873821031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084e840 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f816c010590 tx=0x7f816c010a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936156f869a6b00d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.572+0000 7f8180e6f640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f81640a3c40 0x7f81640a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f817400bde0 tx=0x7f817401af10 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.572+0000 7f81627fc640 1 -- 172.21.15.82:0/873821031 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c01d070 con 0x555940846290 2024-04-21T08:54:28.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.573+0000 7f81627fc640 1 -- 172.21.15.82:0/873821031 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f816c003e20 con 0x555940846290 2024-04-21T08:54:28.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.573+0000 7f81627fc640 1 -- 172.21.15.82:0/873821031 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c004340 con 0x555940846290 2024-04-21T08:54:28.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.574+0000 7f81855a6740 1 -- 172.21.15.82:0/873821031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 msgr2=0x55594084e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.574+0000 7f81855a6740 1 --2- 172.21.15.82:0/873821031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084e840 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f816c010590 tx=0x7f816c010a50 comp rx=0 tx=0).stop 2024-04-21T08:54:28.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.574+0000 7f81855a6740 1 -- 172.21.15.82:0/873821031 shutdown_connections 2024-04-21T08:54:28.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.574+0000 7f81855a6740 1 --2- 172.21.15.82:0/873821031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x555940851260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.574+0000 7f81855a6740 1 --2- 172.21.15.82:0/873821031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084e840 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.574+0000 7f81855a6740 1 --2- 172.21.15.82:0/873821031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x555940845cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.574+0000 7f81855a6740 1 -- 172.21.15.82:0/873821031 >> 172.21.15.82:0/873821031 conn(0x55594083ffb0 msgr2=0x555940840390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:28.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.574+0000 7f81855a6740 1 -- 172.21.15.82:0/873821031 shutdown_connections 2024-04-21T08:54:28.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.574+0000 7f81855a6740 1 -- 172.21.15.82:0/873821031 wait complete. 2024-04-21T08:54:28.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.575+0000 7f81855a6740 1 -- start start 2024-04-21T08:54:28.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.575+0000 7f81855a6740 1 Processor -- start 2024-04-21T08:54:28.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.576+0000 7f81855a6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x55594084bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.576+0000 7f81855a6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.576+0000 7f81855a6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x55594084c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.576+0000 7f81855a6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555940851d80 con 0x555940846290 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.576+0000 7f81855a6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555940763580 con 0x55594084ee10 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.576+0000 7f81855a6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555940851f00 con 0x5559408458e0 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.576+0000 7f81855a6740 10 client.0 _ll_register_callbacks cb 0x555940769650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.577+0000 7f817bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.577+0000 7f817bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46150/0 (socket says 172.21.15.82:46150) 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.577+0000 7f817bfff640 1 -- 172.21.15.82:0/2964201451 learned_addr learned my addr 172.21.15.82:0/2964201451 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.577+0000 7f817bfff640 1 -- 172.21.15.82:0/2964201451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 msgr2=0x55594084bdb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.577+0000 7f817bfff640 1 --2- 172.21.15.82:0/2964201451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x55594084bdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.577+0000 7f817bfff640 1 -- 172.21.15.82:0/2964201451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 msgr2=0x55594084c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.577+0000 7f817bfff640 1 --2- 172.21.15.82:0/2964201451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x55594084c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.577+0000 7f817bfff640 1 -- 172.21.15.82:0/2964201451 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559405e4f70 con 0x555940846290 2024-04-21T08:54:28.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.578+0000 7f817bfff640 1 --2- 172.21.15.82:0/2964201451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084c2f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f816c000f80 tx=0x7f816c004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424bbc5d6b49547b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:28.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.578+0000 7f815b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:28.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.578+0000 7f815b7fe640 1 -- 172.21.15.82:0/2964201451 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c01d040 con 0x555940846290 2024-04-21T08:54:28.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.578+0000 7f81855a6740 1 -- 172.21.15.82:0/2964201451 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f816c003e20 con 0x555940846290 2024-04-21T08:54:28.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.578+0000 7f815b7fe640 1 -- 172.21.15.82:0/2964201451 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f816c004d60 con 0x555940846290 2024-04-21T08:54:28.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.579+0000 7f815b7fe640 1 -- 172.21.15.82:0/2964201451 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f816c030050 con 0x555940846290 2024-04-21T08:54:28.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.579+0000 7f81855a6740 10 client.17760 fetch_fsmap learned FSMap version 41 2024-04-21T08:54:28.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.579+0000 7f81855a6740 1 -- 172.21.15.82:0/2964201451 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55594047c170 con 0x555940846290 2024-04-21T08:54:28.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.579+0000 7f815b7fe640 1 -- 172.21.15.82:0/2964201451 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f816c003e20 con 0x555940846290 2024-04-21T08:54:28.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.580+0000 7f815b7fe640 1 -- 172.21.15.82:0/2964201451 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f816c00b040 con 0x555940846290 2024-04-21T08:54:28.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.580+0000 7f81855a6740 10 client.17760 fetch_fsmap finished waiting for FSMap version 41 2024-04-21T08:54:28.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.580+0000 7f81855a6740 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:28.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.580+0000 7f81855a6740 20 client.17760 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:28.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.580+0000 7f81855a6740 1 --2- 172.21.15.82:0/2964201451 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55594085f670 0x555940861af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.580+0000 7f81855a6740 4 client.17760 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:28.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.580+0000 7f81855a6740 1 -- 172.21.15.82:0/2964201451 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559407b7920 con 0x55594085f670 2024-04-21T08:54:28.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.580+0000 7f817b7fe640 1 --2- 172.21.15.82:0/2964201451 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55594085f670 0x555940861af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.581+0000 7f817b7fe640 1 --2- 172.21.15.82:0/2964201451 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55594085f670 0x555940861af0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8bc334694a6a026 server_cookie=fdb6174a04916fc5 in_seq=0 out_seq=0 2024-04-21T08:54:28.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.581+0000 7f815b7fe640 10 client.17760 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:28.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.582+0000 7f815b7fe640 1 -- 172.21.15.82:0/2964201451 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559407b7920 con 0x55594085f670 2024-04-21T08:54:28.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.582+0000 7f815b7fe640 10 client.17760 handle_command_reply: tid=0 2024-04-21T08:54:28.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.582+0000 7f81855a6740 1 client.17760 shutdown 2024-04-21T08:54:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.582+0000 7f81855a6740 1 -- 172.21.15.82:0/2964201451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 msgr2=0x55594084c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.582+0000 7f81855a6740 1 --2- 172.21.15.82:0/2964201451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084c2f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f816c000f80 tx=0x7f816c004c70 comp rx=0 tx=0).stop 2024-04-21T08:54:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.582+0000 7f81855a6740 1 -- 172.21.15.82:0/2964201451 shutdown_connections 2024-04-21T08:54:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.582+0000 7f81855a6740 1 --2- 172.21.15.82:0/2964201451 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55594085f670 0x555940861af0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.582+0000 7f81855a6740 1 --2- 172.21.15.82:0/2964201451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x55594084c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.582+0000 7f81855a6740 1 --2- 172.21.15.82:0/2964201451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084c2f0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.582+0000 7f81855a6740 1 --2- 172.21.15.82:0/2964201451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x55594084bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.582+0000 7f81855a6740 1 -- 172.21.15.82:0/2964201451 >> 172.21.15.82:0/2964201451 conn(0x55594083ffb0 msgr2=0x555940841d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.583+0000 7f815b7fe640 0 client.17760 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.583+0000 7f81855a6740 1 -- 172.21.15.82:0/2964201451 shutdown_connections 2024-04-21T08:54:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.583+0000 7f81855a6740 1 -- 172.21.15.82:0/2964201451 wait complete. 2024-04-21T08:54:28.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.583+0000 7f81855a6740 20 client.17760 trim_cache size 0 max 16384 2024-04-21T08:54:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.616+0000 7f81855a6740 1 Processor -- start 2024-04-21T08:54:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.616+0000 7f81855a6740 1 -- start start 2024-04-21T08:54:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.616+0000 7f81855a6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x55594085aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.616+0000 7f81855a6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.616+0000 7f81855a6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x555940867400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f81855a6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1be020 con 0x555940846290 2024-04-21T08:54:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x55594085aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x55594085aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:43552/0 (socket says 172.21.15.82:43552) 2024-04-21T08:54:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f817b7fe640 1 -- 172.21.15.82:0/3030998609 learned_addr learned my addr 172.21.15.82:0/3030998609 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f81855a6740 1 -- 172.21.15.82:0/3030998609 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1bfc20 con 0x55594084ee10 2024-04-21T08:54:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f81855a6740 1 -- 172.21.15.82:0/3030998609 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1be6f0 con 0x5559408458e0 2024-04-21T08:54:28.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f817bfff640 1 --2- 172.21.15.82:0/3030998609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f817b7fe640 1 --2- 172.21.15.82:0/3030998609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x555940867400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f817bfff640 1 -- 172.21.15.82:0/3030998609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 msgr2=0x55594085aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f817bfff640 1 --2- 172.21.15.82:0/3030998609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x55594085aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f817bfff640 1 -- 172.21.15.82:0/3030998609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 msgr2=0x555940867400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f817bfff640 1 --2- 172.21.15.82:0/3030998609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x555940867400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.617+0000 7f817bfff640 1 -- 172.21.15.82:0/3030998609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55594084ad60 con 0x555940846290 2024-04-21T08:54:28.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.618+0000 7f817bfff640 1 --2- 172.21.15.82:0/3030998609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084c650 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f816c009600 tx=0x7f816c010540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c0c079aa382770 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:28.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.618+0000 7f8161ffb640 1 -- 172.21.15.82:0/3030998609 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c015070 con 0x555940846290 2024-04-21T08:54:28.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.618+0000 7f8161ffb640 1 -- 172.21.15.82:0/3030998609 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f816c027070 con 0x555940846290 2024-04-21T08:54:28.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.618+0000 7f8161ffb640 1 -- 172.21.15.82:0/3030998609 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c01d070 con 0x555940846290 2024-04-21T08:54:28.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.619+0000 7f81855a6740 1 -- 172.21.15.82:0/3030998609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 msgr2=0x55594084c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.619+0000 7f81855a6740 1 --2- 172.21.15.82:0/3030998609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084c650 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f816c009600 tx=0x7f816c010540 comp rx=0 tx=0).stop 2024-04-21T08:54:28.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.619+0000 7f81855a6740 1 -- 172.21.15.82:0/3030998609 shutdown_connections 2024-04-21T08:54:28.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.619+0000 7f81855a6740 1 --2- 172.21.15.82:0/3030998609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x555940867400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.619+0000 7f81855a6740 1 --2- 172.21.15.82:0/3030998609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555940846290 0x55594084c650 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.619+0000 7f81855a6740 1 --2- 172.21.15.82:0/3030998609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5559408458e0 0x55594085aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.619+0000 7f81855a6740 1 -- 172.21.15.82:0/3030998609 >> 172.21.15.82:0/3030998609 conn(0x55594083ffb0 msgr2=0x55594083e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:28.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.620+0000 7f81855a6740 1 -- 172.21.15.82:0/3030998609 shutdown_connections 2024-04-21T08:54:28.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.620+0000 7f81855a6740 1 -- 172.21.15.82:0/3030998609 wait complete. 2024-04-21T08:54:28.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.620+0000 7f81855a6740 1 -- start start 2024-04-21T08:54:28.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.621+0000 7f81855a6740 1 Processor -- start 2024-04-21T08:54:28.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.621+0000 7f81855a6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559408458e0 0x555940852170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.621+0000 7f81855a6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555940846290 0x5559408526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.621+0000 7f81855a6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x555940852bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.621+0000 7f81855a6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1bf520 con 0x5559408458e0 2024-04-21T08:54:28.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.621+0000 7f81855a6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1bee20 con 0x55594084ee10 2024-04-21T08:54:28.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.621+0000 7f81855a6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1bc080 con 0x555940846290 2024-04-21T08:54:28.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.622+0000 7f81855a6740 10 client.0 _ll_register_callbacks cb 0x7f816c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:28.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.622+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559408458e0 0x555940852170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.622+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559408458e0 0x555940852170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46158/0 (socket says 172.21.15.82:46158) 2024-04-21T08:54:28.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.622+0000 7f817b7fe640 1 -- 172.21.15.82:0/4182585250 learned_addr learned my addr 172.21.15.82:0/4182585250 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:28.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.622+0000 7f817b7fe640 1 --2- 172.21.15.82:0/4182585250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x555940852bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.622+0000 7f817b7fe640 1 -- 172.21.15.82:0/4182585250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555940846290 msgr2=0x5559408526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.622+0000 7f817b7fe640 1 --2- 172.21.15.82:0/4182585250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555940846290 0x5559408526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.622+0000 7f817b7fe640 1 -- 172.21.15.82:0/4182585250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 msgr2=0x555940852bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.622+0000 7f817b7fe640 1 --2- 172.21.15.82:0/4182585250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x555940852bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.622+0000 7f817b7fe640 1 -- 172.21.15.82:0/4182585250 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559408c3d60 con 0x5559408458e0 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.623+0000 7f817b7fe640 1 --2- 172.21.15.82:0/4182585250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559408458e0 0x555940852170 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8170034840 tx=0x7f816c003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a865ff86dff097 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.623+0000 7f815b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.623+0000 7f815b7fe640 1 -- 172.21.15.82:0/4182585250 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8170039040 con 0x5559408458e0 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.623+0000 7f815b7fe640 1 -- 172.21.15.82:0/4182585250 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f81700053f0 con 0x5559408458e0 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.623+0000 7f81855a6740 1 -- 172.21.15.82:0/4182585250 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f816c027070 con 0x5559408458e0 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.623+0000 7f815b7fe640 1 -- 172.21.15.82:0/4182585250 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f81700055f0 con 0x5559408458e0 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.624+0000 7f81855a6740 10 client.17766 fetch_fsmap learned FSMap version 41 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.624+0000 7f81855a6740 1 -- 172.21.15.82:0/4182585250 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5559408c5110 con 0x5559408458e0 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.624+0000 7f815b7fe640 1 -- 172.21.15.82:0/4182585250 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81700044d0 con 0x5559408458e0 2024-04-21T08:54:28.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.624+0000 7f815b7fe640 1 -- 172.21.15.82:0/4182585250 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f817003f040 con 0x5559408458e0 2024-04-21T08:54:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.624+0000 7f81855a6740 10 client.17766 fetch_fsmap finished waiting for FSMap version 41 2024-04-21T08:54:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.624+0000 7f81855a6740 10 client.17766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.624+0000 7f81855a6740 20 client.17766 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.624+0000 7f81855a6740 1 --2- 172.21.15.82:0/4182585250 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5559408c5880 0x5559408c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.624+0000 7f81855a6740 4 client.17766 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:54:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.624+0000 7f81855a6740 1 -- 172.21.15.82:0/4182585250 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f816c027070 con 0x5559408c5880 2024-04-21T08:54:28.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.625+0000 7f817bfff640 1 --2- 172.21.15.82:0/4182585250 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5559408c5880 0x5559408c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.625+0000 7f815b7fe640 10 client.17766 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:28.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.626+0000 7f817bfff640 1 --2- 172.21.15.82:0/4182585250 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5559408c5880 0x5559408c7ca0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90439e904db226cb server_cookie=e08c4ec9b4e39b36 in_seq=0 out_seq=0 2024-04-21T08:54:28.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.627+0000 7f815b7fe640 1 -- 172.21.15.82:0/4182585250 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f816c027070 con 0x5559408c5880 2024-04-21T08:54:28.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.627+0000 7f815b7fe640 10 client.17766 handle_command_reply: tid=0 2024-04-21T08:54:28.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.627+0000 7f81855a6740 1 client.17766 shutdown 2024-04-21T08:54:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:28 smithi106 ceph-mon[31072]: pgmap v228: 129 pgs: 129 active+clean; 874 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 560 op/s 2024-04-21T08:54:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.629+0000 7f81855a6740 1 -- 172.21.15.82:0/4182585250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559408458e0 msgr2=0x555940852170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.629+0000 7f81855a6740 1 --2- 172.21.15.82:0/4182585250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559408458e0 0x555940852170 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8170034840 tx=0x7f816c003d10 comp rx=0 tx=0).stop 2024-04-21T08:54:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.629+0000 7f81855a6740 1 -- 172.21.15.82:0/4182585250 shutdown_connections 2024-04-21T08:54:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.629+0000 7f81855a6740 1 --2- 172.21.15.82:0/4182585250 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5559408c5880 0x5559408c7ca0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.629+0000 7f81855a6740 1 --2- 172.21.15.82:0/4182585250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594084ee10 0x555940852bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.629+0000 7f81855a6740 1 --2- 172.21.15.82:0/4182585250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555940846290 0x5559408526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.629+0000 7f81855a6740 1 --2- 172.21.15.82:0/4182585250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559408458e0 0x555940852170 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.629+0000 7f81855a6740 1 -- 172.21.15.82:0/4182585250 >> 172.21.15.82:0/4182585250 conn(0x55594083ffb0 msgr2=0x55594083f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:28.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.629+0000 7f815b7fe640 0 client.17766 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:28.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.629+0000 7f81855a6740 1 -- 172.21.15.82:0/4182585250 shutdown_connections 2024-04-21T08:54:28.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.631+0000 7f81855a6740 1 -- 172.21.15.82:0/4182585250 wait complete. 2024-04-21T08:54:28.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.631+0000 7f81855a6740 20 client.17766 trim_cache size 0 max 16384 2024-04-21T08:54:28.632 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:54:28.632 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:54:28.632 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "77545521-e780-48f5-abcf-1a3d951b744d", 2024-04-21T08:54:28.632 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:54:28.632 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:54:28.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.633+0000 7f81828d2640 1 -- 172.21.15.82:0/3396956988 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f81640a3c40 msgr2=0x7f81640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.633+0000 7f81828d2640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f81640a3c40 0x7f81640a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f817400bde0 tx=0x7f817401af10 comp rx=0 tx=0).stop 2024-04-21T08:54:28.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.633+0000 7f81828d2640 1 -- 172.21.15.82:0/3396956988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c11b730 msgr2=0x7f817c1abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.634+0000 7f81828d2640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c11b730 0x7f817c1abf20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8174009fd0 tx=0x7f817400b8e0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.635+0000 7f81828d2640 1 -- 172.21.15.82:0/3396956988 shutdown_connections 2024-04-21T08:54:28.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.635+0000 7f81828d2640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f817c11b730 0x7f817c1abf20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.635+0000 7f81828d2640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f81640a3c40 0x7f81640a6100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.636+0000 7f81828d2640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f817c1107a0 0x7f817c1a7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.636+0000 7f81828d2640 1 --2- 172.21.15.82:0/3396956988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c10e7d0 0x7f817c1a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.636+0000 7f81828d2640 1 -- 172.21.15.82:0/3396956988 >> 172.21.15.82:0/3396956988 conn(0x7f817c06d060 msgr2=0x7f817c11a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.636+0000 7f81828d2640 1 -- 172.21.15.82:0/3396956988 shutdown_connections 2024-04-21T08:54:28.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.636+0000 7f81828d2640 1 -- 172.21.15.82:0/3396956988 wait complete. 2024-04-21T08:54:28.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:54:28.654 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:54:28.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.798+0000 7fb8951f4640 1 -- 172.21.15.82:0/3605521164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb890107310 msgr2=0x7fb89010ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.798+0000 7fb8951f4640 1 --2- 172.21.15.82:0/3605521164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb890107310 0x7fb89010ee90 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fb888009ca0 tx=0x7fb88802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.798+0000 7fb8951f4640 1 -- 172.21.15.82:0/3605521164 shutdown_connections 2024-04-21T08:54:28.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.798+0000 7fb8951f4640 1 --2- 172.21.15.82:0/3605521164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb890107310 0x7fb89010ee90 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.798+0000 7fb8951f4640 1 --2- 172.21.15.82:0/3605521164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8901010b0 0x7fb890106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.798+0000 7fb8951f4640 1 --2- 172.21.15.82:0/3605521164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb890100700 0x7fb890100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.798+0000 7fb8951f4640 1 -- 172.21.15.82:0/3605521164 >> 172.21.15.82:0/3605521164 conn(0x7fb8900773a0 msgr2=0x7fb8900fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:28.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.799+0000 7fb8951f4640 1 -- 172.21.15.82:0/3605521164 shutdown_connections 2024-04-21T08:54:28.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.799+0000 7fb8951f4640 1 -- 172.21.15.82:0/3605521164 wait complete. 2024-04-21T08:54:28.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.799+0000 7fb8951f4640 1 Processor -- start 2024-04-21T08:54:28.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.799+0000 7fb8951f4640 1 -- start start 2024-04-21T08:54:28.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb8951f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb890100700 0x7fb89019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb8951f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb890107310 0x7fb89019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb8951f4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8901985f0 0x7fb890198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb8951f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb890112230 con 0x7fb890100700 2024-04-21T08:54:28.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb8951f4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901120b0 con 0x7fb8901985f0 2024-04-21T08:54:28.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb8951f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901123b0 con 0x7fb890107310 2024-04-21T08:54:28.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb88ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb890100700 0x7fb89019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb88e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb890107310 0x7fb89019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb88ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb890100700 0x7fb89019e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46170/0 (socket says 172.21.15.82:46170) 2024-04-21T08:54:28.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb88ed9d640 1 -- 172.21.15.82:0/3159273596 learned_addr learned my addr 172.21.15.82:0/3159273596 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:28.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb88f59e640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8901985f0 0x7fb890198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb88ed9d640 1 -- 172.21.15.82:0/3159273596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb890107310 msgr2=0x7fb89019e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb88ed9d640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb890107310 0x7fb89019e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb88ed9d640 1 -- 172.21.15.82:0/3159273596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8901985f0 msgr2=0x7fb890198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb88ed9d640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8901985f0 0x7fb890198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb88ed9d640 1 -- 172.21.15.82:0/3159273596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb890199110 con 0x7fb890100700 2024-04-21T08:54:28.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb88f59e640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8901985f0 0x7fb890198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:28.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb88ed9d640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb890100700 0x7fb89019e440 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e9f0 tx=0x7fb88000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:28.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb877fff640 1 -- 172.21.15.82:0/3159273596 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88000cd80 con 0x7fb890100700 2024-04-21T08:54:28.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb8951f4640 1 -- 172.21.15.82:0/3159273596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb890199400 con 0x7fb890100700 2024-04-21T08:54:28.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb8951f4640 1 -- 172.21.15.82:0/3159273596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8901a56a0 con 0x7fb890100700 2024-04-21T08:54:28.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb877fff640 1 -- 172.21.15.82:0/3159273596 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8800108e0 con 0x7fb890100700 2024-04-21T08:54:28.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.800+0000 7fb877fff640 1 -- 172.21.15.82:0/3159273596 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800188e0 con 0x7fb890100700 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.802+0000 7fb877fff640 1 -- 172.21.15.82:0/3159273596 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb880010430 con 0x7fb890100700 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb897de8740 1 Processor -- start 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb897de8740 1 -- start start 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb897de8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f5e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb897de8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5eea0 0x5563c2f67540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb897de8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f67a80 0x5563c2f69e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb897de8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5563c2ec31e0 con 0x5563c2f5e580 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb897de8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5563c2e60390 con 0x5563c2f67a80 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb877fff640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb8780a3d20 0x7fb8780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb897de8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5563c2ebf1a0 con 0x5563c2f5eea0 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb88f59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f5e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb88f59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f5e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46180/0 (socket says 172.21.15.82:46180) 2024-04-21T08:54:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb88f59e640 1 -- 172.21.15.82:0/61671273 learned_addr learned my addr 172.21.15.82:0/61671273 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb88e59c640 1 --2- 172.21.15.82:0/61671273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f67a80 0x5563c2f69e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb88ed9d640 1 --2- 172.21.15.82:0/61671273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5eea0 0x5563c2f67540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb88f59e640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb8780a3d20 0x7fb8780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb877fff640 1 -- 172.21.15.82:0/3159273596 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fb8800d0e10 con 0x7fb890100700 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb88f59e640 1 -- 172.21.15.82:0/61671273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5eea0 msgr2=0x5563c2f67540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb88f59e640 1 --2- 172.21.15.82:0/61671273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5eea0 0x5563c2f67540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb88f59e640 1 -- 172.21.15.82:0/61671273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f67a80 msgr2=0x5563c2f69e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb88f59e640 1 --2- 172.21.15.82:0/61671273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f67a80 0x5563c2f69e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.803+0000 7fb88f59e640 1 -- 172.21.15.82:0/61671273 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c2c9f7d0 con 0x5563c2f5e580 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.804+0000 7fb88f59e640 1 --2- 172.21.15.82:0/61671273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f5e960 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb888037040 tx=0x7fb8880328b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e11e02a76f4fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.804+0000 7fb88f59e640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb8780a3d20 0x7fb8780a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb888009ca0 tx=0x7fb888042000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.804+0000 7fb8757fa640 1 -- 172.21.15.82:0/61671273 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888051070 con 0x5563c2f5e580 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.804+0000 7fb8757fa640 1 -- 172.21.15.82:0/61671273 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb888004d40 con 0x5563c2f5e580 2024-04-21T08:54:28.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.804+0000 7fb8757fa640 1 -- 172.21.15.82:0/61671273 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888039040 con 0x5563c2f5e580 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.804+0000 7fb897de8740 1 -- 172.21.15.82:0/61671273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 msgr2=0x5563c2f5e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.804+0000 7fb897de8740 1 --2- 172.21.15.82:0/61671273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f5e960 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb888037040 tx=0x7fb8880328b0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.804+0000 7fb897de8740 1 -- 172.21.15.82:0/61671273 shutdown_connections 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.804+0000 7fb897de8740 1 --2- 172.21.15.82:0/61671273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f67a80 0x5563c2f69e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.804+0000 7fb897de8740 1 --2- 172.21.15.82:0/61671273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5eea0 0x5563c2f67540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.804+0000 7fb897de8740 1 --2- 172.21.15.82:0/61671273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f5e960 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.804+0000 7fb897de8740 1 -- 172.21.15.82:0/61671273 >> 172.21.15.82:0/61671273 conn(0x5563c2f58cb0 msgr2=0x5563c2f59090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb897de8740 1 -- 172.21.15.82:0/61671273 shutdown_connections 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb897de8740 1 -- 172.21.15.82:0/61671273 wait complete. 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb897de8740 1 -- start start 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb897de8740 1 Processor -- start 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb897de8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f64a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb897de8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f5eea0 0x5563c2f64fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb897de8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f67a80 0x5563c2f654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb897de8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5563c2f6a810 con 0x5563c2f5e580 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb897de8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5563c2e7c2f0 con 0x5563c2f5eea0 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb897de8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5563c2f6a990 con 0x5563c2f67a80 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb897de8740 10 client.0 _ll_register_callbacks cb 0x5563c2e89580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb88ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f64a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb88ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f64a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46190/0 (socket says 172.21.15.82:46190) 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb88ed9d640 1 -- 172.21.15.82:0/702906280 learned_addr learned my addr 172.21.15.82:0/702906280 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb88e59c640 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f5eea0 0x5563c2f64fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb88f59e640 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f67a80 0x5563c2f654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb88ed9d640 1 -- 172.21.15.82:0/702906280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f67a80 msgr2=0x5563c2f654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb88ed9d640 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f67a80 0x5563c2f654f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb88ed9d640 1 -- 172.21.15.82:0/702906280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f5eea0 msgr2=0x5563c2f64fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb88ed9d640 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f5eea0 0x5563c2f64fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.805+0000 7fb88ed9d640 1 -- 172.21.15.82:0/702906280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c2bd53f0 con 0x5563c2f5e580 2024-04-21T08:54:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb88ed9d640 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f64a70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb880098730 tx=0x7fb88006c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56732ccded5eab server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb872ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb872ff5640 1 -- 172.21.15.82:0/702906280 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800a5380 con 0x5563c2f5e580 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb872ff5640 1 -- 172.21.15.82:0/702906280 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fb8800bbb60 con 0x5563c2f5e580 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb897de8740 1 -- 172.21.15.82:0/702906280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb888004d40 con 0x5563c2f5e580 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb872ff5640 1 -- 172.21.15.82:0/702906280 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb880010720 con 0x5563c2f5e580 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb897de8740 10 client.17778 fetch_fsmap learned FSMap version 41 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb897de8740 1 -- 172.21.15.82:0/702906280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5563c2b950f0 con 0x5563c2f5e580 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb872ff5640 1 -- 172.21.15.82:0/702906280 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb888004d40 con 0x5563c2f5e580 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb872ff5640 1 -- 172.21.15.82:0/702906280 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fb88006b190 con 0x5563c2f5e580 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb897de8740 10 client.17778 fetch_fsmap finished waiting for FSMap version 41 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb897de8740 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb897de8740 20 client.17778 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb897de8740 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5563c2f779c0 0x5563c2f79e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb897de8740 4 client.17778 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.806+0000 7fb897de8740 1 -- 172.21.15.82:0/702906280 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563c2ed0c00 con 0x5563c2f779c0 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.807+0000 7fb88e59c640 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5563c2f779c0 0x5563c2f79e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.807+0000 7fb88e59c640 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5563c2f779c0 0x5563c2f79e40 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce64d69142f6a3c server_cookie=bb86545ff204514b in_seq=0 out_seq=0 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.807+0000 7fb872ff5640 10 client.17778 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb872ff5640 1 -- 172.21.15.82:0/702906280 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5563c2ed0c00 con 0x5563c2f779c0 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb872ff5640 10 client.17778 handle_command_reply: tid=0 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb897de8740 1 client.17778 shutdown 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb897de8740 1 -- 172.21.15.82:0/702906280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 msgr2=0x5563c2f64a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb897de8740 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f64a70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb880098730 tx=0x7fb88006c3f0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb897de8740 1 -- 172.21.15.82:0/702906280 shutdown_connections 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb897de8740 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5563c2f779c0 0x5563c2f79e40 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb897de8740 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f67a80 0x5563c2f654f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb872ff5640 0 client.17778 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb897de8740 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f5eea0 0x5563c2f64fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb897de8740 1 --2- 172.21.15.82:0/702906280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f64a70 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb897de8740 1 -- 172.21.15.82:0/702906280 >> 172.21.15.82:0/702906280 conn(0x5563c2f58cb0 msgr2=0x5563c2f5aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:28.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb897de8740 1 -- 172.21.15.82:0/702906280 shutdown_connections 2024-04-21T08:54:28.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb897de8740 1 -- 172.21.15.82:0/702906280 wait complete. 2024-04-21T08:54:28.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.808+0000 7fb897de8740 20 client.17778 trim_cache size 0 max 16384 2024-04-21T08:54:28.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.825+0000 7fb897de8740 1 Processor -- start 2024-04-21T08:54:28.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.825+0000 7fb897de8740 1 -- start start 2024-04-21T08:54:28.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb897de8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5e580 0x5563c2f671a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb897de8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f5eea0 0x5563c2f73730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb897de8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f67a80 0x5563c2f639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb897de8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c010070 con 0x5563c2f67a80 2024-04-21T08:54:28.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb897de8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c011c70 con 0x5563c2f5eea0 2024-04-21T08:54:28.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb897de8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c010740 con 0x5563c2f5e580 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb88f59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f67a80 0x5563c2f639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb88f59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f67a80 0x5563c2f639b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46200/0 (socket says 172.21.15.82:46200) 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb88f59e640 1 -- 172.21.15.82:0/4058164745 learned_addr learned my addr 172.21.15.82:0/4058164745 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb88e59c640 1 --2- 172.21.15.82:0/4058164745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f5eea0 0x5563c2f73730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb88ed9d640 1 --2- 172.21.15.82:0/4058164745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5e580 0x5563c2f671a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb88f59e640 1 -- 172.21.15.82:0/4058164745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5e580 msgr2=0x5563c2f671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb88f59e640 1 --2- 172.21.15.82:0/4058164745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5e580 0x5563c2f671a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb88f59e640 1 -- 172.21.15.82:0/4058164745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f5eea0 msgr2=0x5563c2f73730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb88f59e640 1 --2- 172.21.15.82:0/4058164745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f5eea0 0x5563c2f73730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb88f59e640 1 -- 172.21.15.82:0/4058164745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c2f5ecd0 con 0x5563c2f67a80 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb88f59e640 1 --2- 172.21.15.82:0/4058164745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f67a80 0x5563c2f639b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb88802c7e0 tx=0x7fb888005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70284da58b63dda1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb874ff9640 1 -- 172.21.15.82:0/4058164745 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888051040 con 0x5563c2f67a80 2024-04-21T08:54:28.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb874ff9640 1 -- 172.21.15.82:0/4058164745 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb888004830 con 0x5563c2f67a80 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.826+0000 7fb874ff9640 1 -- 172.21.15.82:0/4058164745 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888039040 con 0x5563c2f67a80 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 -- 172.21.15.82:0/4058164745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f67a80 msgr2=0x5563c2f639b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 --2- 172.21.15.82:0/4058164745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f67a80 0x5563c2f639b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb88802c7e0 tx=0x7fb888005c70 comp rx=0 tx=0).stop 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 -- 172.21.15.82:0/4058164745 shutdown_connections 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 --2- 172.21.15.82:0/4058164745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f67a80 0x5563c2f639b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 --2- 172.21.15.82:0/4058164745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f5eea0 0x5563c2f73730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 --2- 172.21.15.82:0/4058164745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5e580 0x5563c2f671a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 -- 172.21.15.82:0/4058164745 >> 172.21.15.82:0/4058164745 conn(0x5563c2f58cb0 msgr2=0x5563c2f57d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 -- 172.21.15.82:0/4058164745 shutdown_connections 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 -- 172.21.15.82:0/4058164745 wait complete. 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 -- start start 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 Processor -- start 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f6b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.827+0000 7fb897de8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5eea0 0x5563c2f6bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb897de8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f67a80 0x5563c2f6c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb897de8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c011570 con 0x5563c2f5e580 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb897de8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c010e70 con 0x5563c2f67a80 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb897de8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c00e040 con 0x5563c2f5eea0 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb897de8740 10 client.0 _ll_register_callbacks cb 0x7fb88000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb88ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f6b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb88ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f6b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46210/0 (socket says 172.21.15.82:46210) 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb88ed9d640 1 -- 172.21.15.82:0/857777756 learned_addr learned my addr 172.21.15.82:0/857777756 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb88e59c640 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5eea0 0x5563c2f6bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb88f59e640 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f67a80 0x5563c2f6c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb88ed9d640 1 -- 172.21.15.82:0/857777756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5eea0 msgr2=0x5563c2f6bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb88ed9d640 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5eea0 0x5563c2f6bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb88ed9d640 1 -- 172.21.15.82:0/857777756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f67a80 msgr2=0x5563c2f6c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:28.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb88ed9d640 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f67a80 0x5563c2f6c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb88ed9d640 1 -- 172.21.15.82:0/857777756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c2fdd1d0 con 0x5563c2f5e580 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb88ed9d640 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f6b880 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb88009c750 tx=0x7fb88000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635fdf7ac913f716 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb872ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb872ff5640 1 -- 172.21.15.82:0/857777756 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88009c590 con 0x5563c2f5e580 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb872ff5640 1 -- 172.21.15.82:0/857777756 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fb88009c0b0 con 0x5563c2f5e580 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb897de8740 1 -- 172.21.15.82:0/857777756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb888004830 con 0x5563c2f5e580 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb872ff5640 1 -- 172.21.15.82:0/857777756 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb880098cf0 con 0x5563c2f5e580 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb897de8740 10 client.17784 fetch_fsmap learned FSMap version 41 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb897de8740 1 -- 172.21.15.82:0/857777756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5563c2fde4a0 con 0x5563c2f5e580 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.828+0000 7fb872ff5640 1 -- 172.21.15.82:0/857777756 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb888004830 con 0x5563c2f5e580 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb872ff5640 1 -- 172.21.15.82:0/857777756 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fb8800987a0 con 0x5563c2f5e580 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb897de8740 10 client.17784 fetch_fsmap finished waiting for FSMap version 41 2024-04-21T08:54:28.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb897de8740 10 client.17784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb897de8740 20 client.17784 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb897de8740 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5563c2fdeb50 0x5563c2fe0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb897de8740 4 client.17784 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb897de8740 1 -- 172.21.15.82:0/857777756 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563c2ecaa70 con 0x5563c2fdeb50 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb88e59c640 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5563c2fdeb50 0x5563c2fe0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb872ff5640 10 client.17784 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb88e59c640 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5563c2fdeb50 0x5563c2fe0f70 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d645f14dbada80f server_cookie=96be8f52966f0b84 in_seq=0 out_seq=0 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb872ff5640 1 -- 172.21.15.82:0/857777756 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5563c2ecaa70 con 0x5563c2fdeb50 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb872ff5640 10 client.17784 handle_command_reply: tid=0 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb897de8740 1 client.17784 shutdown 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.829+0000 7fb897de8740 1 -- 172.21.15.82:0/857777756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 msgr2=0x5563c2f6b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb897de8740 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f6b880 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb88009c750 tx=0x7fb88000c7d0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb897de8740 1 -- 172.21.15.82:0/857777756 shutdown_connections 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb897de8740 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5563c2fdeb50 0x5563c2fe0f70 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb897de8740 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5563c2f67a80 0x5563c2f6c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb897de8740 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5563c2f5eea0 0x5563c2f6bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb897de8740 1 --2- 172.21.15.82:0/857777756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563c2f5e580 0x5563c2f6b880 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb872ff5640 0 client.17784 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:28.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb897de8740 1 -- 172.21.15.82:0/857777756 >> 172.21.15.82:0/857777756 conn(0x5563c2f58cb0 msgr2=0x5563c2f59fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:28.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb897de8740 1 -- 172.21.15.82:0/857777756 shutdown_connections 2024-04-21T08:54:28.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb897de8740 1 -- 172.21.15.82:0/857777756 wait complete. 2024-04-21T08:54:28.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb897de8740 20 client.17784 trim_cache size 0 max 16384 2024-04-21T08:54:28.837 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:54:28.837 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:54:28.838 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:54:28.838 INFO:teuthology.orchestra.run.smithi082.stdout: "77545521-e780-48f5-abcf-1a3d951b744d": { 2024-04-21T08:54:28.838 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:54:28.838 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "77545521-e780-48f5-abcf-1a3d951b744d", 2024-04-21T08:54:28.838 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:54:28.838 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:54:28.838 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:54:28.838 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:54:28.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb8951f4640 1 -- 172.21.15.82:0/3159273596 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb8780a3d20 msgr2=0x7fb8780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb8951f4640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb8780a3d20 0x7fb8780a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb888009ca0 tx=0x7fb888042000 comp rx=0 tx=0).stop 2024-04-21T08:54:28.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb8951f4640 1 -- 172.21.15.82:0/3159273596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb890100700 msgr2=0x7fb89019e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:28.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb8951f4640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb890100700 0x7fb89019e440 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e9f0 tx=0x7fb88000eeb0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb8951f4640 1 -- 172.21.15.82:0/3159273596 shutdown_connections 2024-04-21T08:54:28.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb8951f4640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8901985f0 0x7fb890198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb8951f4640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb890107310 0x7fb89019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb8951f4640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb8780a3d20 0x7fb8780a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb8951f4640 1 --2- 172.21.15.82:0/3159273596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb890100700 0x7fb89019e440 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:28.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb8951f4640 1 -- 172.21.15.82:0/3159273596 >> 172.21.15.82:0/3159273596 conn(0x7fb8900773a0 msgr2=0x7fb890105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:28.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb8951f4640 1 -- 172.21.15.82:0/3159273596 shutdown_connections 2024-04-21T08:54:28.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:28.830+0000 7fb8951f4640 1 -- 172.21.15.82:0/3159273596 wait complete. 2024-04-21T08:54:28.841 INFO:tasks.cephfs.filesystem:scrub status for tag:77545521-e780-48f5-abcf-1a3d951b744d - {'path': '/', 'tag': '77545521-e780-48f5-abcf-1a3d951b744d', 'options': 'recursive,force'} 2024-04-21T08:54:29.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:29 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:54:29.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:29 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:54:29.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:29 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:29.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:29 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:54:29.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:29 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:54:29.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:29 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:29 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:54:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:29 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:54:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:29 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:30.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:30 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:54:30.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:30 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:30.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:30 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:54:30.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:30 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:54:30.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:30 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:30.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:30 smithi134 ceph-mon[31772]: pgmap v229: 129 pgs: 129 active+clean; 879 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 495 op/s 2024-04-21T08:54:30.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:30 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:54:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:30 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:30 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:54:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:30 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:54:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:30 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:30 smithi082 ceph-mon[27967]: pgmap v229: 129 pgs: 129 active+clean; 879 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 495 op/s 2024-04-21T08:54:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:30 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:54:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:30 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:30 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:54:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:30 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:54:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:30 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:30 smithi106 ceph-mon[31072]: pgmap v229: 129 pgs: 129 active+clean; 879 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 495 op/s 2024-04-21T08:54:32.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:32 smithi134 ceph-mon[31772]: pgmap v230: 129 pgs: 129 active+clean; 879 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 359 op/s 2024-04-21T08:54:32.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:32 smithi082 ceph-mon[27967]: pgmap v230: 129 pgs: 129 active+clean; 879 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 359 op/s 2024-04-21T08:54:32.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:32 smithi106 ceph-mon[31072]: pgmap v230: 129 pgs: 129 active+clean; 879 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 359 op/s 2024-04-21T08:54:34.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:34 smithi134 ceph-mon[31772]: pgmap v231: 129 pgs: 129 active+clean; 891 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 706 op/s 2024-04-21T08:54:34.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:34 smithi082 ceph-mon[27967]: pgmap v231: 129 pgs: 129 active+clean; 891 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 706 op/s 2024-04-21T08:54:34.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:34 smithi106 ceph-mon[31072]: pgmap v231: 129 pgs: 129 active+clean; 891 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 706 op/s 2024-04-21T08:54:35.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:35 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:35.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:35 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:35.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:35 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:54:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.331+0000 7fa768606640 1 -- 172.21.15.82:0/2183399887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7601080e0 msgr2=0x7fa7601084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.331+0000 7fa768606640 1 --2- 172.21.15.82:0/2183399887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7601080e0 0x7fa7601084c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009900 tx=0x7fa75c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 -- 172.21.15.82:0/2183399887 shutdown_connections 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 --2- 172.21.15.82:0/2183399887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa76006b120 0x7fa76010edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 --2- 172.21.15.82:0/2183399887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa76006a780 0x7fa76006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 --2- 172.21.15.82:0/2183399887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7601080e0 0x7fa7601084c0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 -- 172.21.15.82:0/2183399887 >> 172.21.15.82:0/2183399887 conn(0x7fa760076c50 msgr2=0x7fa760077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 -- 172.21.15.82:0/2183399887 shutdown_connections 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 -- 172.21.15.82:0/2183399887 wait complete. 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 Processor -- start 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 -- start start 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76006a780 0x7fa760198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa76006b120 0x7fa760198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7601080e0 0x7fa76019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa760112190 con 0x7fa76006a780 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa760112010 con 0x7fa7601080e0 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa768606640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa760112310 con 0x7fa76006b120 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa7663a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76006a780 0x7fa760198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa7663a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76006a780 0x7fa760198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56884/0 (socket says 172.21.15.82:56884) 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.332+0000 7fa7663a2640 1 -- 172.21.15.82:0/1514568563 learned_addr learned my addr 172.21.15.82:0/1514568563 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa766ba3640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7601080e0 0x7fa76019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa765ba1640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa76006b120 0x7fa760198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa7663a2640 1 -- 172.21.15.82:0/1514568563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa76006b120 msgr2=0x7fa760198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa7663a2640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa76006b120 0x7fa760198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa7663a2640 1 -- 172.21.15.82:0/1514568563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7601080e0 msgr2=0x7fa76019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa7663a2640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7601080e0 0x7fa76019ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa7663a2640 1 -- 172.21.15.82:0/1514568563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76019d600 con 0x7fa76006a780 2024-04-21T08:54:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa765ba1640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa76006b120 0x7fa760198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa7663a2640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76006a780 0x7fa760198610 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa75c005940 tx=0x7fa75c02d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa7577fe640 1 -- 172.21.15.82:0/1514568563 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c043020 con 0x7fa76006a780 2024-04-21T08:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa7577fe640 1 -- 172.21.15.82:0/1514568563 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa75c02dc90 con 0x7fa76006a780 2024-04-21T08:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa768606640 1 -- 172.21.15.82:0/1514568563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76019d890 con 0x7fa76006a780 2024-04-21T08:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa7577fe640 1 -- 172.21.15.82:0/1514568563 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c003ab0 con 0x7fa76006a780 2024-04-21T08:54:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.333+0000 7fa768606640 1 -- 172.21.15.82:0/1514568563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7601a5180 con 0x7fa76006a780 2024-04-21T08:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.335+0000 7fa7577fe640 1 -- 172.21.15.82:0/1514568563 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa75c046020 con 0x7fa76006a780 2024-04-21T08:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.336+0000 7fa76b1f4740 1 Processor -- start 2024-04-21T08:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.336+0000 7fa76b1f4740 1 -- start start 2024-04-21T08:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.336+0000 7fa76b1f4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc152958d0 0x55fc15294620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.336+0000 7fa76b1f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc15294fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.336+0000 7fa76b1f4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 0x55fc152a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.336+0000 7fa76b1f4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55fc151f98d0 con 0x55fc1529e0e0 2024-04-21T08:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.336+0000 7fa76b1f4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fc15196e80 con 0x55fc152958d0 2024-04-21T08:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.336+0000 7fa76b1f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fc151f5890 con 0x55fc15294b60 2024-04-21T08:54:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.336+0000 7fa7577fe640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa74c0a3dc0 0x7fa74c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.336+0000 7fa765ba1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 0x55fc152a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa765ba1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 0x55fc152a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56892/0 (socket says 172.21.15.82:56892) 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa765ba1640 1 -- 172.21.15.82:0/2154199771 learned_addr learned my addr 172.21.15.82:0/2154199771 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa766ba3640 1 --2- 172.21.15.82:0/2154199771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc152958d0 0x55fc15294620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa7663a2640 1 --2- 172.21.15.82:0/2154199771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc15294fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa766ba3640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa74c0a3dc0 0x7fa74c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa7577fe640 1 -- 172.21.15.82:0/1514568563 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fa75c0f2c60 con 0x7fa76006a780 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa765ba1640 1 -- 172.21.15.82:0/2154199771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 msgr2=0x55fc15294fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa765ba1640 1 --2- 172.21.15.82:0/2154199771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc15294fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa765ba1640 1 -- 172.21.15.82:0/2154199771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc152958d0 msgr2=0x55fc15294620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa765ba1640 1 --2- 172.21.15.82:0/2154199771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc152958d0 0x55fc15294620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa765ba1640 1 -- 172.21.15.82:0/2154199771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc151b8320 con 0x55fc1529e0e0 2024-04-21T08:54:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa766ba3640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa74c0a3dc0 0x7fa74c0a6280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa75800a780 tx=0x7fa75800a4d0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa765ba1640 1 --2- 172.21.15.82:0/2154199771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 0x55fc152a04d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa75000bd30 tx=0x7fa75000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53a4ab63718c9ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa754bfe640 1 -- 172.21.15.82:0/2154199771 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750005220 con 0x55fc1529e0e0 2024-04-21T08:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa754bfe640 1 -- 172.21.15.82:0/2154199771 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa75000be20 con 0x55fc1529e0e0 2024-04-21T08:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.337+0000 7fa754bfe640 1 -- 172.21.15.82:0/2154199771 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7500116a0 con 0x55fc1529e0e0 2024-04-21T08:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.338+0000 7fa76b1f4740 1 -- 172.21.15.82:0/2154199771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 msgr2=0x55fc152a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.338+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/2154199771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 0x55fc152a04d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa75000bd30 tx=0x7fa75000f730 comp rx=0 tx=0).stop 2024-04-21T08:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.338+0000 7fa76b1f4740 1 -- 172.21.15.82:0/2154199771 shutdown_connections 2024-04-21T08:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.338+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/2154199771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 0x55fc152a04d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.338+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/2154199771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc15294fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.338+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/2154199771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc152958d0 0x55fc15294620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.338+0000 7fa76b1f4740 1 -- 172.21.15.82:0/2154199771 >> 172.21.15.82:0/2154199771 conn(0x55fc1525bc90 msgr2=0x55fc1528f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.338+0000 7fa76b1f4740 1 -- 172.21.15.82:0/2154199771 shutdown_connections 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.338+0000 7fa76b1f4740 1 -- 172.21.15.82:0/2154199771 wait complete. 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.338+0000 7fa76b1f4740 1 -- start start 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.338+0000 7fa76b1f4740 1 Processor -- start 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa76b1f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc1529d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa76b1f4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 0x55fc1529ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa76b1f4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc1529e0e0 0x55fc1529b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa76b1f4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55fc152a0e10 con 0x55fc152958d0 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa76b1f4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fc15181eb0 con 0x55fc1529e0e0 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa76b1f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fc152a0f90 con 0x55fc15294b60 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa76b1f4740 10 client.0 _ll_register_callbacks cb 0x55fc151aeb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa765ba1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 0x55fc1529ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa765ba1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 0x55fc1529ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56908/0 (socket says 172.21.15.82:56908) 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa765ba1640 1 -- 172.21.15.82:0/3232585370 learned_addr learned my addr 172.21.15.82:0/3232585370 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa766ba3640 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc1529e0e0 0x55fc1529b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa7663a2640 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc1529d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa765ba1640 1 -- 172.21.15.82:0/3232585370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 msgr2=0x55fc1529d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa765ba1640 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc1529d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa765ba1640 1 -- 172.21.15.82:0/3232585370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc1529e0e0 msgr2=0x55fc1529b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa765ba1640 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc1529e0e0 0x55fc1529b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa765ba1640 1 -- 172.21.15.82:0/3232585370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc14f0c210 con 0x55fc152958d0 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa7663a2640 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc1529d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa765ba1640 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 0x55fc1529ad80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa7500054a0 tx=0x7fa75000ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51cba27ad936098 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa749ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa749ffb640 1 -- 172.21.15.82:0/3232585370 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75002b060 con 0x55fc152958d0 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa749ffb640 1 -- 172.21.15.82:0/3232585370 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fa7500046c0 con 0x55fc152958d0 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.339+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3232585370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa75000be20 con 0x55fc152958d0 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.340+0000 7fa749ffb640 1 -- 172.21.15.82:0/3232585370 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa750032050 con 0x55fc152958d0 2024-04-21T08:54:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.340+0000 7fa76b1f4740 10 client.17796 fetch_fsmap learned FSMap version 41 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.340+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3232585370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fc14fd8620 con 0x55fc152958d0 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.340+0000 7fa749ffb640 1 -- 172.21.15.82:0/3232585370 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa75000be20 con 0x55fc152958d0 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.340+0000 7fa749ffb640 1 -- 172.21.15.82:0/3232585370 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fa750011ab0 con 0x55fc152958d0 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.340+0000 7fa76b1f4740 10 client.17796 fetch_fsmap finished waiting for FSMap version 41 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.340+0000 7fa76b1f4740 10 client.17796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.340+0000 7fa76b1f4740 20 client.17796 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.340+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fc152adfd0 0x55fc152b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.340+0000 7fa76b1f4740 4 client.17796 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.340+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3232585370 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc1520d880 con 0x55fc152adfd0 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.340+0000 7fa7663a2640 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fc152adfd0 0x55fc152b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.341+0000 7fa749ffb640 10 client.17796 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.341+0000 7fa7663a2640 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fc152adfd0 0x55fc152b03c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2ebdd74ea83a78a server_cookie=b836836225002c1f in_seq=0 out_seq=0 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.341+0000 7fa749ffb640 1 -- 172.21.15.82:0/3232585370 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc1520d880 con 0x55fc152adfd0 2024-04-21T08:54:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.341+0000 7fa749ffb640 10 client.17796 handle_command_reply: tid=0 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.341+0000 7fa76b1f4740 1 client.17796 shutdown 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.341+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3232585370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 msgr2=0x55fc1529ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.341+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 0x55fc1529ad80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa7500054a0 tx=0x7fa75000ba70 comp rx=0 tx=0).stop 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.342+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3232585370 shutdown_connections 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.342+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fc152adfd0 0x55fc152b03c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.342+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc1529e0e0 0x55fc1529b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.342+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 0x55fc1529ad80 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.342+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/3232585370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc1529d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.342+0000 7fa749ffb640 0 client.17796 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.342+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3232585370 >> 172.21.15.82:0/3232585370 conn(0x55fc1525bc90 msgr2=0x55fc152915a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.342+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3232585370 shutdown_connections 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.342+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3232585370 wait complete. 2024-04-21T08:54:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.342+0000 7fa76b1f4740 20 client.17796 trim_cache size 0 max 16384 2024-04-21T08:54:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa76b1f4740 1 Processor -- start 2024-04-21T08:54:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa76b1f4740 1 -- start start 2024-04-21T08:54:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa76b1f4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc15294b60 0x55fc15296630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa76b1f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc152958d0 0x55fc152a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa76b1f4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 0x55fc1529b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa76b1f4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601af410 con 0x55fc1529e0e0 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa76b1f4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601b1010 con 0x55fc15294b60 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa76b1f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601afae0 con 0x55fc152958d0 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa766ba3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 0x55fc1529b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa766ba3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 0x55fc1529b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56912/0 (socket says 172.21.15.82:56912) 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa766ba3640 1 -- 172.21.15.82:0/1573168017 learned_addr learned my addr 172.21.15.82:0/1573168017 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa7663a2640 1 --2- 172.21.15.82:0/1573168017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc15294b60 0x55fc15296630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa766ba3640 1 -- 172.21.15.82:0/1573168017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc152958d0 msgr2=0x55fc152a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa766ba3640 1 --2- 172.21.15.82:0/1573168017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc152958d0 0x55fc152a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa766ba3640 1 -- 172.21.15.82:0/1573168017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc15294b60 msgr2=0x55fc15296630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.357+0000 7fa766ba3640 1 --2- 172.21.15.82:0/1573168017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc15294b60 0x55fc15296630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.358+0000 7fa766ba3640 1 -- 172.21.15.82:0/1573168017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc15293e30 con 0x55fc1529e0e0 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.358+0000 7fa766ba3640 1 --2- 172.21.15.82:0/1573168017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 0x55fc1529b1a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fa758005cb0 tx=0x7fa75800a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadeb85cbcbc52e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.358+0000 7fa74bfff640 1 -- 172.21.15.82:0/1573168017 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758004d20 con 0x55fc1529e0e0 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.358+0000 7fa74bfff640 1 -- 172.21.15.82:0/1573168017 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa758005020 con 0x55fc1529e0e0 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.358+0000 7fa74bfff640 1 -- 172.21.15.82:0/1573168017 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758021670 con 0x55fc1529e0e0 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.358+0000 7fa76b1f4740 1 -- 172.21.15.82:0/1573168017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 msgr2=0x55fc1529b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.358+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/1573168017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 0x55fc1529b1a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fa758005cb0 tx=0x7fa75800a890 comp rx=0 tx=0).stop 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.358+0000 7fa76b1f4740 1 -- 172.21.15.82:0/1573168017 shutdown_connections 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.358+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/1573168017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc1529e0e0 0x55fc1529b1a0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.358+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/1573168017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc152958d0 0x55fc152a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.358+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/1573168017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc15294b60 0x55fc15296630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.358+0000 7fa76b1f4740 1 -- 172.21.15.82:0/1573168017 >> 172.21.15.82:0/1573168017 conn(0x55fc1525bc90 msgr2=0x55fc15299690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa76b1f4740 1 -- 172.21.15.82:0/1573168017 shutdown_connections 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa76b1f4740 1 -- 172.21.15.82:0/1573168017 wait complete. 2024-04-21T08:54:36.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa76b1f4740 1 -- start start 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa76b1f4740 1 Processor -- start 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa76b1f4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc152a6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa76b1f4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 0x55fc15297650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa76b1f4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc1529e0e0 0x55fc15297b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa76b1f4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601b0910 con 0x55fc152958d0 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa76b1f4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601b0210 con 0x55fc1529e0e0 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa76b1f4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7601ad3e0 con 0x55fc15294b60 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa76b1f4740 10 client.0 _ll_register_callbacks cb 0x7fa750015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa765ba1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 0x55fc15297650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa765ba1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 0x55fc15297650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56920/0 (socket says 172.21.15.82:56920) 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa765ba1640 1 -- 172.21.15.82:0/3905359179 learned_addr learned my addr 172.21.15.82:0/3905359179 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa766ba3640 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc1529e0e0 0x55fc15297b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa7663a2640 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc152a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa765ba1640 1 -- 172.21.15.82:0/3905359179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 msgr2=0x55fc152a6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa765ba1640 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc152a6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa765ba1640 1 -- 172.21.15.82:0/3905359179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc1529e0e0 msgr2=0x55fc15297b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa765ba1640 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc1529e0e0 0x55fc15297b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa765ba1640 1 -- 172.21.15.82:0/3905359179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc152980d0 con 0x55fc152958d0 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.359+0000 7fa7663a2640 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc152a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa765ba1640 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 0x55fc15297650 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa7500059d0 tx=0x7fa75001b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cbf9e42bd79ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa749ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa749ffb640 1 -- 172.21.15.82:0/3905359179 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75002b030 con 0x55fc152958d0 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa749ffb640 1 -- 172.21.15.82:0/3905359179 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fa75000be20 con 0x55fc152958d0 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3905359179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa758005020 con 0x55fc152958d0 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa749ffb640 1 -- 172.21.15.82:0/3905359179 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa75000b5a0 con 0x55fc152958d0 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa76b1f4740 10 client.17802 fetch_fsmap learned FSMap version 41 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3905359179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fc15313fb0 con 0x55fc152958d0 2024-04-21T08:54:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa749ffb640 1 -- 172.21.15.82:0/3905359179 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa758005020 con 0x55fc152958d0 2024-04-21T08:54:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa749ffb640 1 -- 172.21.15.82:0/3905359179 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fa750003cc0 con 0x55fc152958d0 2024-04-21T08:54:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa76b1f4740 10 client.17802 fetch_fsmap finished waiting for FSMap version 41 2024-04-21T08:54:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa76b1f4740 10 client.17802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa76b1f4740 20 client.17802 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fc15314660 0x55fc15316a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa76b1f4740 4 client.17802 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:54:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3905359179 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc151993d0 con 0x55fc15314660 2024-04-21T08:54:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.360+0000 7fa7663a2640 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fc15314660 0x55fc15316a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.361+0000 7fa7663a2640 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fc15314660 0x55fc15316a80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e13dd51fc2a759 server_cookie=aff4f91cb84c596a in_seq=0 out_seq=0 2024-04-21T08:54:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.361+0000 7fa749ffb640 10 client.17802 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa749ffb640 1 -- 172.21.15.82:0/3905359179 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc151993d0 con 0x55fc15314660 2024-04-21T08:54:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa749ffb640 10 client.17802 handle_command_reply: tid=0 2024-04-21T08:54:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa76b1f4740 1 client.17802 shutdown 2024-04-21T08:54:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3905359179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 msgr2=0x55fc15297650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 0x55fc15297650 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa7500059d0 tx=0x7fa75001b540 comp rx=0 tx=0).stop 2024-04-21T08:54:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3905359179 shutdown_connections 2024-04-21T08:54:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55fc15314660 0x55fc15316a80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fc1529e0e0 0x55fc15297b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc152958d0 0x55fc15297650 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa76b1f4740 1 --2- 172.21.15.82:0/3905359179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fc15294b60 0x55fc152a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa749ffb640 0 client.17802 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3905359179 >> 172.21.15.82:0/3905359179 conn(0x55fc1525bc90 msgr2=0x55fc1528e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3905359179 shutdown_connections 2024-04-21T08:54:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.370+0000 7fa76b1f4740 1 -- 172.21.15.82:0/3905359179 wait complete. 2024-04-21T08:54:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa76b1f4740 20 client.17802 trim_cache size 0 max 16384 2024-04-21T08:54:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:54:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:54:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:54:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa768606640 1 -- 172.21.15.82:0/1514568563 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa74c0a3dc0 msgr2=0x7fa74c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa768606640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa74c0a3dc0 0x7fa74c0a6280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa75800a780 tx=0x7fa75800a4d0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa768606640 1 -- 172.21.15.82:0/1514568563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76006a780 msgr2=0x7fa760198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa768606640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76006a780 0x7fa760198610 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa75c005940 tx=0x7fa75c02d890 comp rx=0 tx=0).stop 2024-04-21T08:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa768606640 1 -- 172.21.15.82:0/1514568563 shutdown_connections 2024-04-21T08:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa768606640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7601080e0 0x7fa76019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa768606640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa76006b120 0x7fa760198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa768606640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa74c0a3dc0 0x7fa74c0a6280 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa768606640 1 --2- 172.21.15.82:0/1514568563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76006a780 0x7fa760198610 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa768606640 1 -- 172.21.15.82:0/1514568563 >> 172.21.15.82:0/1514568563 conn(0x7fa760076c50 msgr2=0x7fa7600ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa768606640 1 -- 172.21.15.82:0/1514568563 shutdown_connections 2024-04-21T08:54:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:36.371+0000 7fa768606640 1 -- 172.21.15.82:0/1514568563 wait complete. 2024-04-21T08:54:36.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:36 smithi134 ceph-mon[31772]: pgmap v232: 129 pgs: 129 active+clean; 891 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 527 op/s 2024-04-21T08:54:36.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:36 smithi082 ceph-mon[27967]: pgmap v232: 129 pgs: 129 active+clean; 891 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 527 op/s 2024-04-21T08:54:36.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:36 smithi106 ceph-mon[31072]: pgmap v232: 129 pgs: 129 active+clean; 891 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 527 op/s 2024-04-21T08:54:38.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:38 smithi134 ceph-mon[31772]: pgmap v233: 129 pgs: 129 active+clean; 899 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 682 op/s 2024-04-21T08:54:38.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:38 smithi082 ceph-mon[27967]: pgmap v233: 129 pgs: 129 active+clean; 899 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 682 op/s 2024-04-21T08:54:38.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:38 smithi106 ceph-mon[31072]: pgmap v233: 129 pgs: 129 active+clean; 899 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 682 op/s 2024-04-21T08:54:40.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:40 smithi134 ceph-mon[31772]: pgmap v234: 129 pgs: 129 active+clean; 899 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 720 op/s 2024-04-21T08:54:40.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:40 smithi082 ceph-mon[27967]: pgmap v234: 129 pgs: 129 active+clean; 899 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 720 op/s 2024-04-21T08:54:40.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:40 smithi106 ceph-mon[31072]: pgmap v234: 129 pgs: 129 active+clean; 899 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 720 op/s 2024-04-21T08:54:42.429 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:42 smithi106 ceph-mon[31072]: pgmap v235: 129 pgs: 129 active+clean; 899 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 635 op/s 2024-04-21T08:54:42.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:42 smithi134 ceph-mon[31772]: pgmap v235: 129 pgs: 129 active+clean; 899 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 635 op/s 2024-04-21T08:54:42.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:42 smithi082 ceph-mon[27967]: pgmap v235: 129 pgs: 129 active+clean; 899 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 635 op/s 2024-04-21T08:54:44.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:44 smithi134 ceph-mon[31772]: pgmap v236: 129 pgs: 129 active+clean; 873 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 901 op/s 2024-04-21T08:54:44.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:44 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:54:44.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:44 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:54:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:44 smithi082 ceph-mon[27967]: pgmap v236: 129 pgs: 129 active+clean; 873 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 901 op/s 2024-04-21T08:54:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:44 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:54:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:44 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:54:44.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:44 smithi106 ceph-mon[31072]: pgmap v236: 129 pgs: 129 active+clean; 873 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 901 op/s 2024-04-21T08:54:44.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:44 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:54:44.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:44 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:54:45.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:45 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:54:45.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:45 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:54:45.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:45 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:54:46.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:46 smithi134 ceph-mon[31772]: pgmap v237: 129 pgs: 129 active+clean; 873 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 554 op/s 2024-04-21T08:54:46.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:46 smithi082 ceph-mon[27967]: pgmap v237: 129 pgs: 129 active+clean; 873 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 554 op/s 2024-04-21T08:54:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:46 smithi106 ceph-mon[31072]: pgmap v237: 129 pgs: 129 active+clean; 873 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 554 op/s 2024-04-21T08:54:48.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:48 smithi134 ceph-mon[31772]: pgmap v238: 129 pgs: 129 active+clean; 873 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 645 op/s 2024-04-21T08:54:48.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:48 smithi082 ceph-mon[27967]: pgmap v238: 129 pgs: 129 active+clean; 873 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 645 op/s 2024-04-21T08:54:48.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:48 smithi106 ceph-mon[31072]: pgmap v238: 129 pgs: 129 active+clean; 873 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 645 op/s 2024-04-21T08:54:48.903 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-04-21T08:54:48.903 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-21T08:54:49.172 DEBUG:teuthology.parallel:result is None 2024-04-21T08:54:49.172 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-21T08:54:49.201 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-21T08:54:49.201 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-21T08:54:49.212 INFO:tasks.quiescer:joining Quiescers 2024-04-21T08:54:49.213 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:54:49.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.342+0000 7f7ac1214640 1 -- 172.21.15.82:0/3730827015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7abc105e30 msgr2=0x7f7abc106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.342+0000 7f7ac1214640 1 --2- 172.21.15.82:0/3730827015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7abc105e30 0x7f7abc106210 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009930 tx=0x7f7ab402bc30 comp rx=0 tx=0).stop 2024-04-21T08:54:49.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.342+0000 7f7ac1214640 1 -- 172.21.15.82:0/3730827015 shutdown_connections 2024-04-21T08:54:49.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.342+0000 7f7ac1214640 1 --2- 172.21.15.82:0/3730827015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7abc101b60 0x7f7abc10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.342+0000 7f7ac1214640 1 --2- 172.21.15.82:0/3730827015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc1011a0 0x7f7abc101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.342+0000 7f7ac1214640 1 --2- 172.21.15.82:0/3730827015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7abc105e30 0x7f7abc106210 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.342+0000 7f7ac1214640 1 -- 172.21.15.82:0/3730827015 >> 172.21.15.82:0/3730827015 conn(0x7f7abc077e30 msgr2=0x7f7abc078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.342+0000 7f7ac1214640 1 -- 172.21.15.82:0/3730827015 shutdown_connections 2024-04-21T08:54:49.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.342+0000 7f7ac1214640 1 -- 172.21.15.82:0/3730827015 wait complete. 2024-04-21T08:54:49.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.343+0000 7f7ac1214640 1 Processor -- start 2024-04-21T08:54:49.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.343+0000 7f7ac1214640 1 -- start start 2024-04-21T08:54:49.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.343+0000 7f7ac1214640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7abc1011a0 0x7f7abc1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.343+0000 7f7ac1214640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7abc101b60 0x7f7abc198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.343+0000 7f7ac1214640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc105e30 0x7f7abc19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7ac1214640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc1110e0 con 0x7f7abc1011a0 2024-04-21T08:54:49.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7ac1214640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc110f60 con 0x7f7abc105e30 2024-04-21T08:54:49.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7ac1214640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc111260 con 0x7f7abc101b60 2024-04-21T08:54:49.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7abad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7abc1011a0 0x7f7abc1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7abad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7abc1011a0 0x7f7abc1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42250/0 (socket says 172.21.15.82:42250) 2024-04-21T08:54:49.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7abad9d640 1 -- 172.21.15.82:0/979667221 learned_addr learned my addr 172.21.15.82:0/979667221 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:49.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7abb59e640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc105e30 0x7f7abc19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7aba59c640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7abc101b60 0x7f7abc198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7abad9d640 1 -- 172.21.15.82:0/979667221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7abc101b60 msgr2=0x7f7abc198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7abad9d640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7abc101b60 0x7f7abc198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7abad9d640 1 -- 172.21.15.82:0/979667221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc105e30 msgr2=0x7f7abc19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7abad9d640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc105e30 0x7f7abc19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7abad9d640 1 -- 172.21.15.82:0/979667221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc19d750 con 0x7f7abc1011a0 2024-04-21T08:54:49.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7abad9d640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7abc1011a0 0x7f7abc1987b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009230 tx=0x7f7ab402c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7aa3fff640 1 -- 172.21.15.82:0/979667221 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4043020 con 0x7f7abc1011a0 2024-04-21T08:54:49.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7ac1214640 1 -- 172.21.15.82:0/979667221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7abc19d9e0 con 0x7f7abc1011a0 2024-04-21T08:54:49.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7aa3fff640 1 -- 172.21.15.82:0/979667221 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab40043d0 con 0x7f7abc1011a0 2024-04-21T08:54:49.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7aa3fff640 1 -- 172.21.15.82:0/979667221 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4003bc0 con 0x7f7abc1011a0 2024-04-21T08:54:49.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.344+0000 7f7ac1214640 1 -- 172.21.15.82:0/979667221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7abc1a52d0 con 0x7f7abc1011a0 2024-04-21T08:54:49.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.345+0000 7f7ac1214640 1 -- 172.21.15.82:0/979667221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7abc068de0 con 0x7f7abc1011a0 2024-04-21T08:54:49.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.346+0000 7f7aa3fff640 1 -- 172.21.15.82:0/979667221 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7ab4046020 con 0x7f7abc1011a0 2024-04-21T08:54:49.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.347+0000 7f7aa3fff640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7aa40a3d20 0x7f7aa40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.347+0000 7f7aa3fff640 1 -- 172.21.15.82:0/979667221 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f7ab40f30f0 con 0x7f7abc1011a0 2024-04-21T08:54:49.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.347+0000 7f7aba59c640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7aa40a3d20 0x7f7aa40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.347+0000 7f7aba59c640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7aa40a3d20 0x7f7aa40a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0005c40 tx=0x7f7ab0005bd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.351+0000 7f7aa3fff640 1 -- 172.21.15.82:0/979667221 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ab404d020 con 0x7f7abc1011a0 2024-04-21T08:54:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.459+0000 7ffa37f84640 1 -- 172.21.15.82:0/2819194610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa30101680 msgr2=0x7ffa3010e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.459+0000 7ffa37f84640 1 --2- 172.21.15.82:0/2819194610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa30101680 0x7ffa3010e3d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ffa20009900 tx=0x7ffa2002bc30 comp rx=0 tx=0).stop 2024-04-21T08:54:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.459+0000 7ffa37f84640 1 -- 172.21.15.82:0/2819194610 shutdown_connections 2024-04-21T08:54:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.459+0000 7ffa37f84640 1 --2- 172.21.15.82:0/2819194610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa3010e910 0x7ffa30110d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.459+0000 7ffa37f84640 1 --2- 172.21.15.82:0/2819194610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa30101680 0x7ffa3010e3d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.459+0000 7ffa37f84640 1 --2- 172.21.15.82:0/2819194610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa30100d60 0x7ffa30101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.459+0000 7ffa37f84640 1 -- 172.21.15.82:0/2819194610 >> 172.21.15.82:0/2819194610 conn(0x7ffa30077fd0 msgr2=0x7ffa300783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.459+0000 7ffa37f84640 1 -- 172.21.15.82:0/2819194610 shutdown_connections 2024-04-21T08:54:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.459+0000 7ffa37f84640 1 -- 172.21.15.82:0/2819194610 wait complete. 2024-04-21T08:54:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa37f84640 1 Processor -- start 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa37f84640 1 -- start start 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa37f84640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa30100d60 0x7ffa301a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa37f84640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa30101680 0x7ffa301a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa37f84640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa3010e910 0x7ffa3019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa37f84640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301143e0 con 0x7ffa30100d60 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa37f84640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa30114260 con 0x7ffa30101680 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa37f84640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa30114560 con 0x7ffa3010e910 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa35d20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa30100d60 0x7ffa301a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa35d20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa30100d60 0x7ffa301a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42266/0 (socket says 172.21.15.82:42266) 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa35d20640 1 -- 172.21.15.82:0/136360545 learned_addr learned my addr 172.21.15.82:0/136360545 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa3551f640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa30101680 0x7ffa301a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa36521640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa3010e910 0x7ffa3019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa35d20640 1 -- 172.21.15.82:0/136360545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa3010e910 msgr2=0x7ffa3019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa35d20640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa3010e910 0x7ffa3019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa35d20640 1 -- 172.21.15.82:0/136360545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa30101680 msgr2=0x7ffa301a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa35d20640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa30101680 0x7ffa301a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa35d20640 1 -- 172.21.15.82:0/136360545 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3019afa0 con 0x7ffa30100d60 2024-04-21T08:54:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa36521640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa3010e910 0x7ffa3019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa35d20640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa30100d60 0x7ffa301a0620 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00ea60 tx=0x7ffa2c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.460+0000 7ffa26ffd640 1 -- 172.21.15.82:0/136360545 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c00cde0 con 0x7ffa30100d60 2024-04-21T08:54:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.461+0000 7ffa37f84640 1 -- 172.21.15.82:0/136360545 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3019b290 con 0x7ffa30100d60 2024-04-21T08:54:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.461+0000 7ffa26ffd640 1 -- 172.21.15.82:0/136360545 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa2c0035f0 con 0x7ffa30100d60 2024-04-21T08:54:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.461+0000 7ffa26ffd640 1 -- 172.21.15.82:0/136360545 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c010690 con 0x7ffa30100d60 2024-04-21T08:54:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.461+0000 7ffa37f84640 1 -- 172.21.15.82:0/136360545 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa301a75a0 con 0x7ffa30100d60 2024-04-21T08:54:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.463+0000 7ffa26ffd640 1 -- 172.21.15.82:0/136360545 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa2c0037a0 con 0x7ffa30100d60 2024-04-21T08:54:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.463+0000 7ffa3ab7b740 1 Processor -- start 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.463+0000 7ffa3ab7b740 1 -- start start 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.463+0000 7ffa3ab7b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 0x563ea8a02620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.463+0000 7ffa3ab7b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a02fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.463+0000 7ffa3ab7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a0c0e0 0x563ea8a0e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.463+0000 7ffa3ab7b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563ea89678d0 con 0x563ea8a038d0 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.463+0000 7ffa3ab7b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563ea8904e80 con 0x563ea8a02b60 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.463+0000 7ffa3ab7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563ea8963890 con 0x563ea8a0c0e0 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.463+0000 7ffa36521640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 0x563ea8a02620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa36521640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 0x563ea8a02620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42274/0 (socket says 172.21.15.82:42274) 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa36521640 1 -- 172.21.15.82:0/2282737976 learned_addr learned my addr 172.21.15.82:0/2282737976 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa35d20640 1 --2- 172.21.15.82:0/2282737976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a02fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa3551f640 1 --2- 172.21.15.82:0/2282737976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a0c0e0 0x563ea8a0e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa26ffd640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa1c0a3dc0 0x7ffa1c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa36521640 1 -- 172.21.15.82:0/2282737976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a0c0e0 msgr2=0x563ea8a0e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa36521640 1 --2- 172.21.15.82:0/2282737976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a0c0e0 0x563ea8a0e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa36521640 1 -- 172.21.15.82:0/2282737976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 msgr2=0x563ea8a02fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa36521640 1 --2- 172.21.15.82:0/2282737976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a02fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa36521640 1 -- 172.21.15.82:0/2282737976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ea8926320 con 0x563ea8a038d0 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa35d20640 1 --2- 172.21.15.82:0/2282737976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a02fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa26ffd640 1 -- 172.21.15.82:0/136360545 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ffa2c0d0fc0 con 0x7ffa30100d60 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa36521640 1 --2- 172.21.15.82:0/2282737976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 0x563ea8a02620 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009e10 tx=0x7ffa2800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17456ca740dcb50 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa36521640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa1c0a3dc0 0x7ffa1c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa1bfff640 1 -- 172.21.15.82:0/2282737976 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2801a680 con 0x563ea8a038d0 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa1bfff640 1 -- 172.21.15.82:0/2282737976 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa28011440 con 0x563ea8a038d0 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.464+0000 7ffa1bfff640 1 -- 172.21.15.82:0/2282737976 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa280195e0 con 0x563ea8a038d0 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.465+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/2282737976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 msgr2=0x563ea8a02620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.465+0000 7ffa36521640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa1c0a3dc0 0x7ffa1c0a6280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffa28019000 tx=0x7ffa28019cb0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.465+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/2282737976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 0x563ea8a02620 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009e10 tx=0x7ffa2800f730 comp rx=0 tx=0).stop 2024-04-21T08:54:49.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.465+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/2282737976 shutdown_connections 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.465+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/2282737976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a0c0e0 0x563ea8a0e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.465+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/2282737976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a02fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.465+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/2282737976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 0x563ea8a02620 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.465+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/2282737976 >> 172.21.15.82:0/2282737976 conn(0x563ea89c9c90 msgr2=0x563ea89fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.465+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/2282737976 shutdown_connections 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.465+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/2282737976 wait complete. 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.465+0000 7ffa3ab7b740 1 -- start start 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa3ab7b740 1 Processor -- start 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa3ab7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a02b60 0x563ea8a0ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa3ab7b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a038d0 0x563ea8a08db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa3ab7b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 0x563ea8a092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa3ab7b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563ea8a0ee10 con 0x563ea8a0c0e0 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa3ab7b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563ea88efeb0 con 0x563ea8a038d0 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa3ab7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563ea8a0ef90 con 0x563ea8a02b60 2024-04-21T08:54:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa3ab7b740 10 client.0 _ll_register_callbacks cb 0x563ea891cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa36521640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 0x563ea8a092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa36521640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 0x563ea8a092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42280/0 (socket says 172.21.15.82:42280) 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa36521640 1 -- 172.21.15.82:0/711047690 learned_addr learned my addr 172.21.15.82:0/711047690 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa35d20640 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a02b60 0x563ea8a0ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa3551f640 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a038d0 0x563ea8a08db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa36521640 1 -- 172.21.15.82:0/711047690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a02b60 msgr2=0x563ea8a0ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa36521640 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a02b60 0x563ea8a0ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa36521640 1 -- 172.21.15.82:0/711047690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a038d0 msgr2=0x563ea8a08db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa36521640 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a038d0 0x563ea8a08db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa36521640 1 -- 172.21.15.82:0/711047690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ea867a210 con 0x563ea8a0c0e0 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.466+0000 7ffa3551f640 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a038d0 0x563ea8a08db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa36521640 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 0x563ea8a092f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ffa28011bb0 tx=0x7ffa2801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc05e6cb7ed23b17 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa197fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa197fa640 1 -- 172.21.15.82:0/711047690 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa280195f0 con 0x563ea8a0c0e0 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa197fa640 1 -- 172.21.15.82:0/711047690 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ffa280115f0 con 0x563ea8a0c0e0 2024-04-21T08:54:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/711047690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa28011440 con 0x563ea8a0c0e0 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa197fa640 1 -- 172.21.15.82:0/711047690 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ffa2803d050 con 0x563ea8a0c0e0 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa3ab7b740 10 client.17820 fetch_fsmap learned FSMap version 41 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/711047690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563ea8746620 con 0x563ea8a0c0e0 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa197fa640 1 -- 172.21.15.82:0/711047690 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa28011440 con 0x563ea8a0c0e0 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa197fa640 1 -- 172.21.15.82:0/711047690 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7ffa28010a00 con 0x563ea8a0c0e0 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa3ab7b740 10 client.17820 fetch_fsmap finished waiting for FSMap version 41 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa3ab7b740 10 client.17820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa3ab7b740 20 client.17820 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563ea8a1c030 0x563ea8a1e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa3ab7b740 4 client.17820 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.467+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/711047690 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ea897b880 con 0x563ea8a1c030 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.468+0000 7ffa35d20640 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563ea8a1c030 0x563ea8a1e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.468+0000 7ffa197fa640 10 client.17820 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.468+0000 7ffa35d20640 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563ea8a1c030 0x563ea8a1e420 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f9cf747b5edae8 server_cookie=938bb3d3236a1be in_seq=0 out_seq=0 2024-04-21T08:54:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.468+0000 7ffa197fa640 1 -- 172.21.15.82:0/711047690 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ea897b880 con 0x563ea8a1c030 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.468+0000 7ffa197fa640 10 client.17820 handle_command_reply: tid=0 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa3ab7b740 1 client.17820 shutdown 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/711047690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 msgr2=0x563ea8a092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 0x563ea8a092f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ffa28011bb0 tx=0x7ffa2801a000 comp rx=0 tx=0).stop 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/711047690 shutdown_connections 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563ea8a1c030 0x563ea8a1e420 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 0x563ea8a092f0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa197fa640 0 client.17820 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a038d0 0x563ea8a08db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/711047690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a02b60 0x563ea8a0ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/711047690 >> 172.21.15.82:0/711047690 conn(0x563ea89c9c90 msgr2=0x563ea89ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/711047690 shutdown_connections 2024-04-21T08:54:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/711047690 wait complete. 2024-04-21T08:54:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.469+0000 7ffa3ab7b740 20 client.17820 trim_cache size 0 max 16384 2024-04-21T08:54:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.483+0000 7ffa3ab7b740 1 Processor -- start 2024-04-21T08:54:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.483+0000 7ffa3ab7b740 1 -- start start 2024-04-21T08:54:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa3ab7b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a17d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa3ab7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a038d0 0x563ea8a0b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa3ab7b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 0x563ea8a0ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa3ab7b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301b1760 con 0x563ea8a0c0e0 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa3ab7b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301b3360 con 0x563ea8a02b60 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa3ab7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301b1e30 con 0x563ea8a038d0 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa36521640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 0x563ea8a0ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa36521640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 0x563ea8a0ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42286/0 (socket says 172.21.15.82:42286) 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa36521640 1 -- 172.21.15.82:0/831663431 learned_addr learned my addr 172.21.15.82:0/831663431 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa3551f640 1 --2- 172.21.15.82:0/831663431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a038d0 0x563ea8a0b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa35d20640 1 --2- 172.21.15.82:0/831663431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a17d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa36521640 1 -- 172.21.15.82:0/831663431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a038d0 msgr2=0x563ea8a0b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa36521640 1 --2- 172.21.15.82:0/831663431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a038d0 0x563ea8a0b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa36521640 1 -- 172.21.15.82:0/831663431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 msgr2=0x563ea8a17d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa36521640 1 --2- 172.21.15.82:0/831663431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a17d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa36521640 1 -- 172.21.15.82:0/831663431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ea8a1e880 con 0x563ea8a0c0e0 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa35d20640 1 --2- 172.21.15.82:0/831663431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a17d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa36521640 1 --2- 172.21.15.82:0/831663431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 0x563ea8a0ba20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009d30 tx=0x7ffa28005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fbe0eb0cea07a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa1b7fe640 1 -- 172.21.15.82:0/831663431 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa28045030 con 0x563ea8a0c0e0 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa1b7fe640 1 -- 172.21.15.82:0/831663431 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa28034070 con 0x563ea8a0c0e0 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.484+0000 7ffa1b7fe640 1 -- 172.21.15.82:0/831663431 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa28011de0 con 0x563ea8a0c0e0 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.485+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/831663431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 msgr2=0x563ea8a0ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.485+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/831663431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 0x563ea8a0ba20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009d30 tx=0x7ffa28005fd0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.485+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/831663431 shutdown_connections 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.485+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/831663431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a0c0e0 0x563ea8a0ba20 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.485+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/831663431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a038d0 0x563ea8a0b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.485+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/831663431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a17d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.485+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/831663431 >> 172.21.15.82:0/831663431 conn(0x563ea89c9c90 msgr2=0x563ea89fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.485+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/831663431 shutdown_connections 2024-04-21T08:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.485+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/831663431 wait complete. 2024-04-21T08:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.485+0000 7ffa3ab7b740 1 -- start start 2024-04-21T08:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.485+0000 7ffa3ab7b740 1 Processor -- start 2024-04-21T08:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3ab7b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a13fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3ab7b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 0x563ea8a144f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3ab7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a0c0e0 0x563ea8a050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3ab7b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301b2c60 con 0x563ea8a038d0 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3ab7b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301b2560 con 0x563ea8a02b60 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3ab7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301af730 con 0x563ea8a0c0e0 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3ab7b740 10 client.0 _ll_register_callbacks cb 0x7ffa28009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3551f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 0x563ea8a144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa35d20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a13fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3551f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 0x563ea8a144f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42296/0 (socket says 172.21.15.82:42296) 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3551f640 1 -- 172.21.15.82:0/846093085 learned_addr learned my addr 172.21.15.82:0/846093085 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa36521640 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a0c0e0 0x563ea8a050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3551f640 1 -- 172.21.15.82:0/846093085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a0c0e0 msgr2=0x563ea8a050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3551f640 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a0c0e0 0x563ea8a050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3551f640 1 -- 172.21.15.82:0/846093085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 msgr2=0x563ea8a13fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3551f640 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a13fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3551f640 1 -- 172.21.15.82:0/846093085 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ea8a06eb0 con 0x563ea8a038d0 2024-04-21T08:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa36521640 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a0c0e0 0x563ea8a050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa3551f640 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 0x563ea8a144f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ffa2002c170 tx=0x7ffa20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c23446aee3548ea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa197fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.486+0000 7ffa197fa640 1 -- 172.21.15.82:0/846093085 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2002c610 con 0x563ea8a038d0 2024-04-21T08:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/846093085 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa28034070 con 0x563ea8a038d0 2024-04-21T08:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa197fa640 1 -- 172.21.15.82:0/846093085 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ffa2003a040 con 0x563ea8a038d0 2024-04-21T08:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa197fa640 1 -- 172.21.15.82:0/846093085 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ffa20005030 con 0x563ea8a038d0 2024-04-21T08:54:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa3ab7b740 10 client.17826 fetch_fsmap learned FSMap version 41 2024-04-21T08:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/846093085 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563ea8a81930 con 0x563ea8a038d0 2024-04-21T08:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa197fa640 1 -- 172.21.15.82:0/846093085 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa28034070 con 0x563ea8a038d0 2024-04-21T08:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa197fa640 1 -- 172.21.15.82:0/846093085 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7ffa200035f0 con 0x563ea8a038d0 2024-04-21T08:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa3ab7b740 10 client.17826 fetch_fsmap finished waiting for FSMap version 41 2024-04-21T08:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa3ab7b740 10 client.17826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa3ab7b740 20 client.17826 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563ea8a1c030 0x563ea8a841a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa3ab7b740 4 client.17826 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/846093085 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ea89073d0 con 0x563ea8a1c030 2024-04-21T08:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.487+0000 7ffa35d20640 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563ea8a1c030 0x563ea8a841a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.488+0000 7ffa197fa640 10 client.17826 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.488+0000 7ffa35d20640 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563ea8a1c030 0x563ea8a841a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf70914570e36026 server_cookie=ded25118084e6fa2 in_seq=0 out_seq=0 2024-04-21T08:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa197fa640 1 -- 172.21.15.82:0/846093085 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ea89073d0 con 0x563ea8a1c030 2024-04-21T08:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa197fa640 10 client.17826 handle_command_reply: tid=0 2024-04-21T08:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa3ab7b740 1 client.17826 shutdown 2024-04-21T08:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/846093085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 msgr2=0x563ea8a144f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 0x563ea8a144f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ffa2002c170 tx=0x7ffa20004270 comp rx=0 tx=0).stop 2024-04-21T08:54:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/846093085 shutdown_connections 2024-04-21T08:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563ea8a1c030 0x563ea8a841a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ea8a0c0e0 0x563ea8a050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea8a038d0 0x563ea8a144f0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa197fa640 0 client.17826 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa3ab7b740 1 --2- 172.21.15.82:0/846093085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ea8a02b60 0x563ea8a13fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/846093085 >> 172.21.15.82:0/846093085 conn(0x563ea89c9c90 msgr2=0x563ea8a03ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/846093085 shutdown_connections 2024-04-21T08:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa3ab7b740 1 -- 172.21.15.82:0/846093085 wait complete. 2024-04-21T08:54:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.503+0000 7ffa3ab7b740 20 client.17826 trim_cache size 0 max 16384 2024-04-21T08:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-21T08:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-21T08:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-21T08:54:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-21T08:54:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.504+0000 7ffa37f84640 1 -- 172.21.15.82:0/136360545 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa1c0a3dc0 msgr2=0x7ffa1c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.504+0000 7ffa37f84640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa1c0a3dc0 0x7ffa1c0a6280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffa28019000 tx=0x7ffa28019cb0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.504+0000 7ffa37f84640 1 -- 172.21.15.82:0/136360545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa30100d60 msgr2=0x7ffa301a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.504+0000 7ffa37f84640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa30100d60 0x7ffa301a0620 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00ea60 tx=0x7ffa2c00ef20 comp rx=0 tx=0).stop 2024-04-21T08:54:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.504+0000 7ffa37f84640 1 -- 172.21.15.82:0/136360545 shutdown_connections 2024-04-21T08:54:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.504+0000 7ffa37f84640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa3010e910 0x7ffa3019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.504+0000 7ffa37f84640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa30101680 0x7ffa301a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.504+0000 7ffa37f84640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa1c0a3dc0 0x7ffa1c0a6280 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.504+0000 7ffa37f84640 1 --2- 172.21.15.82:0/136360545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa30100d60 0x7ffa301a0620 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.504+0000 7ffa37f84640 1 -- 172.21.15.82:0/136360545 >> 172.21.15.82:0/136360545 conn(0x7ffa30077fd0 msgr2=0x7ffa3010fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.504+0000 7ffa37f84640 1 -- 172.21.15.82:0/136360545 shutdown_connections 2024-04-21T08:54:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-21T08:54:49.504+0000 7ffa37f84640 1 -- 172.21.15.82:0/136360545 wait complete. 2024-04-21T08:54:49.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.611+0000 7f7ac1214640 1 -- 172.21.15.82:0/979667221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7abc10a340 con 0x7f7abc1011a0 2024-04-21T08:54:49.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.614+0000 7f7aa3fff640 1 -- 172.21.15.82:0/979667221 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11951 (secure 0 0 0) 0x7f7ab40bf5b0 con 0x7f7abc1011a0 2024-04-21T08:54:49.614 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:54:49.614 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:54:27.210249+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24481,24476,24407,24409]},"id":1}]} 2024-04-21T08:54:49.615 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 41 2024-04-21T08:54:49.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.617+0000 7f7ac1214640 1 -- 172.21.15.82:0/979667221 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7aa40a3d20 msgr2=0x7f7aa40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.617+0000 7f7ac1214640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7aa40a3d20 0x7f7aa40a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0005c40 tx=0x7f7ab0005bd0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.618+0000 7f7ac1214640 1 -- 172.21.15.82:0/979667221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7abc1011a0 msgr2=0x7f7abc1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.618+0000 7f7ac1214640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7abc1011a0 0x7f7abc1987b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009230 tx=0x7f7ab402c710 comp rx=0 tx=0).stop 2024-04-21T08:54:49.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.618+0000 7f7ac1214640 1 -- 172.21.15.82:0/979667221 shutdown_connections 2024-04-21T08:54:49.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.618+0000 7f7ac1214640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7abc105e30 0x7f7abc19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.618+0000 7f7ac1214640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7abc101b60 0x7f7abc198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.618+0000 7f7ac1214640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f7aa40a3d20 0x7f7aa40a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.618+0000 7f7ac1214640 1 --2- 172.21.15.82:0/979667221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7abc1011a0 0x7f7abc1987b0 secure :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009230 tx=0x7f7ab402c710 comp rx=0 tx=0).stop 2024-04-21T08:54:49.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.618+0000 7f7ac1214640 1 -- 172.21.15.82:0/979667221 >> 172.21.15.82:0/979667221 conn(0x7f7abc077e30 msgr2=0x7f7abc0694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.619+0000 7f7ac1214640 1 -- 172.21.15.82:0/979667221 shutdown_connections 2024-04-21T08:54:49.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.619+0000 7f7ac1214640 1 -- 172.21.15.82:0/979667221 wait complete. 2024-04-21T08:54:49.638 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --cancel --all' 2024-04-21T08:54:49.638 DEBUG:teuthology.orchestra.run.smithi082:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --cancel --all 2024-04-21T08:54:49.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.800+0000 7f9226f75640 1 -- 172.21.15.82:0/3628769514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220102b40 msgr2=0x7f9220106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.800+0000 7f9226f75640 1 --2- 172.21.15.82:0/3628769514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220102b40 0x7f9220106b80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f920c009900 tx=0x7f920c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:54:49.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.802+0000 7f9226f75640 1 -- 172.21.15.82:0/3628769514 shutdown_connections 2024-04-21T08:54:49.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.802+0000 7f9226f75640 1 --2- 172.21.15.82:0/3628769514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9220107260 0x7f922010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.802+0000 7f9226f75640 1 --2- 172.21.15.82:0/3628769514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220102b40 0x7f9220106b80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.802+0000 7f9226f75640 1 --2- 172.21.15.82:0/3628769514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9220102190 0x7f9220102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.802+0000 7f9226f75640 1 -- 172.21.15.82:0/3628769514 >> 172.21.15.82:0/3628769514 conn(0x7f92200793a0 msgr2=0x7f92200797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.802+0000 7f9226f75640 1 -- 172.21.15.82:0/3628769514 shutdown_connections 2024-04-21T08:54:49.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.802+0000 7f9226f75640 1 -- 172.21.15.82:0/3628769514 wait complete. 2024-04-21T08:54:49.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.804+0000 7f9226f75640 1 Processor -- start 2024-04-21T08:54:49.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.804+0000 7f9226f75640 1 -- start start 2024-04-21T08:54:49.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.804+0000 7f9226f75640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9220102190 0x7f922019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.804+0000 7f9226f75640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220107260 0x7f922019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.804+0000 7f9226f75640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9220198590 0x7f9220198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.804+0000 7f9226f75640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f92201121f0 con 0x7f9220102190 2024-04-21T08:54:49.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.805+0000 7f9226f75640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9220112070 con 0x7f9220198590 2024-04-21T08:54:49.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.805+0000 7f9226f75640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9220112370 con 0x7f9220107260 2024-04-21T08:54:49.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.805+0000 7f9224d11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9220102190 0x7f922019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.805+0000 7f9225512640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9220198590 0x7f9220198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.805+0000 7f9225512640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9220198590 0x7f9220198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:60596/0 (socket says 172.21.15.82:60596) 2024-04-21T08:54:49.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.805+0000 7f9225512640 1 -- 172.21.15.82:0/1139601220 learned_addr learned my addr 172.21.15.82:0/1139601220 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:49.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.805+0000 7f921ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220107260 0x7f922019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.805+0000 7f9224d11640 1 -- 172.21.15.82:0/1139601220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220107260 msgr2=0x7f922019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.805+0000 7f9224d11640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220107260 0x7f922019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.805+0000 7f9224d11640 1 -- 172.21.15.82:0/1139601220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9220198590 msgr2=0x7f9220198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.805+0000 7f9224d11640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9220198590 0x7f9220198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.806+0000 7f9224d11640 1 -- 172.21.15.82:0/1139601220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92201990b0 con 0x7f9220102190 2024-04-21T08:54:49.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.806+0000 7f9225512640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9220198590 0x7f9220198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:49.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.806+0000 7f921ffff640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220107260 0x7f922019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:49.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.806+0000 7f9224d11640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9220102190 0x7f922019e3e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f921400e9f0 tx=0x7f921400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.806+0000 7f921dffb640 1 -- 172.21.15.82:0/1139601220 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921400ccc0 con 0x7f9220102190 2024-04-21T08:54:49.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.807+0000 7f921dffb640 1 -- 172.21.15.82:0/1139601220 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92140108e0 con 0x7f9220102190 2024-04-21T08:54:49.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.807+0000 7f921dffb640 1 -- 172.21.15.82:0/1139601220 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92140188e0 con 0x7f9220102190 2024-04-21T08:54:49.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.808+0000 7f9226f75640 1 -- 172.21.15.82:0/1139601220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92201993a0 con 0x7f9220102190 2024-04-21T08:54:49.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.809+0000 7f9226f75640 1 -- 172.21.15.82:0/1139601220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92201a55a0 con 0x7f9220102190 2024-04-21T08:54:49.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.814+0000 7f9229b67740 1 Processor -- start 2024-04-21T08:54:49.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.814+0000 7f9229b67740 1 -- start start 2024-04-21T08:54:49.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.814+0000 7f9229b67740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.814+0000 7f9229b67740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993b76c0 0x5633993bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.814+0000 7f9229b67740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993c0240 0x5633993c2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.814+0000 7f9229b67740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56339931c000 con 0x5633993b6d10 2024-04-21T08:54:49.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.814+0000 7f9229b67740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5633992b8b20 con 0x5633993b76c0 2024-04-21T08:54:49.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.814+0000 7f9229b67740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563399317fc0 con 0x5633993c0240 2024-04-21T08:54:49.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.815+0000 7f921dffb640 1 -- 172.21.15.82:0/1139601220 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f921401f020 con 0x7f9220102190 2024-04-21T08:54:49.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.815+0000 7f9224d11640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993b76c0 0x5633993bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.815+0000 7f9224d11640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993b76c0 0x5633993bfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:60610/0 (socket says 172.21.15.82:60610) 2024-04-21T08:54:49.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.816+0000 7f9224d11640 1 -- 172.21.15.82:0/761954958 learned_addr learned my addr 172.21.15.82:0/761954958 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:49.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.816+0000 7f921dffb640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f92100a3e70 0x7f92100a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.816+0000 7f921dffb640 1 -- 172.21.15.82:0/1139601220 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f92140d0c30 con 0x7f9220102190 2024-04-21T08:54:49.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.816+0000 7f9224d11640 1 -- 172.21.15.82:0/761954958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993c0240 msgr2=0x5633993c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.816+0000 7f9225512640 1 --2- 172.21.15.82:0/761954958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.816+0000 7f9224d11640 1 --2- 172.21.15.82:0/761954958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993c0240 0x5633993c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.816+0000 7f9224d11640 1 -- 172.21.15.82:0/761954958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 msgr2=0x5633993b70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.816+0000 7f9224d11640 1 --2- 172.21.15.82:0/761954958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993b70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.816+0000 7f9224d11640 1 -- 172.21.15.82:0/761954958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633992d2ea0 con 0x5633993b76c0 2024-04-21T08:54:49.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.816+0000 7f9225512640 1 --2- 172.21.15.82:0/761954958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:49.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.816+0000 7f9224d11640 1 --2- 172.21.15.82:0/761954958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993b76c0 0x5633993bfc70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f921400e9c0 tx=0x7f921400cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e3f05cd7e62bb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.817+0000 7f920affd640 1 -- 172.21.15.82:0/761954958 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921409c7c0 con 0x5633993b76c0 2024-04-21T08:54:49.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.817+0000 7f920affd640 1 -- 172.21.15.82:0/761954958 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92140a0b70 con 0x5633993b76c0 2024-04-21T08:54:49.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.817+0000 7f920affd640 1 -- 172.21.15.82:0/761954958 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214098e30 con 0x5633993b76c0 2024-04-21T08:54:49.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.817+0000 7f9225512640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f92100a3e70 0x7f92100a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.818+0000 7f9229b67740 1 -- 172.21.15.82:0/761954958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993b76c0 msgr2=0x5633993bfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.818+0000 7f9229b67740 1 --2- 172.21.15.82:0/761954958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993b76c0 0x5633993bfc70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f921400e9c0 tx=0x7f921400cf60 comp rx=0 tx=0).stop 2024-04-21T08:54:49.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.818+0000 7f9225512640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f92100a3e70 0x7f92100a6330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f92180049d0 tx=0x7f9218004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.818+0000 7f9229b67740 1 -- 172.21.15.82:0/761954958 shutdown_connections 2024-04-21T08:54:49.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.818+0000 7f9229b67740 1 --2- 172.21.15.82:0/761954958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993c0240 0x5633993c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.818+0000 7f9229b67740 1 --2- 172.21.15.82:0/761954958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993b76c0 0x5633993bfc70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.818+0000 7f9229b67740 1 --2- 172.21.15.82:0/761954958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.818+0000 7f9229b67740 1 -- 172.21.15.82:0/761954958 >> 172.21.15.82:0/761954958 conn(0x5633993b13e0 msgr2=0x5633993b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.820+0000 7f9229b67740 1 -- 172.21.15.82:0/761954958 shutdown_connections 2024-04-21T08:54:49.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.821+0000 7f9229b67740 1 -- 172.21.15.82:0/761954958 wait complete. 2024-04-21T08:54:49.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.821+0000 7f9229b67740 1 -- start start 2024-04-21T08:54:49.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.822+0000 7f9229b67740 1 Processor -- start 2024-04-21T08:54:49.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.822+0000 7f9229b67740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b6d10 0x5633993bd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.822+0000 7f9229b67740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b76c0 0x5633993bd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.822+0000 7f9229b67740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 0x5633993bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.822+0000 7f921ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b76c0 0x5633993bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f9229b67740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5633993c3000 con 0x5633993b76c0 2024-04-21T08:54:49.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f9224d11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b6d10 0x5633993bd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f9229b67740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5633992b6510 con 0x5633993c0240 2024-04-21T08:54:49.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f9229b67740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5633993c3180 con 0x5633993b6d10 2024-04-21T08:54:49.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f921ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b76c0 0x5633993bd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42348/0 (socket says 172.21.15.82:42348) 2024-04-21T08:54:49.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f921ffff640 1 -- 172.21.15.82:0/116703116 learned_addr learned my addr 172.21.15.82:0/116703116 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:49.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f9229b67740 10 client.0 _ll_register_callbacks cb 0x5633992b5f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:49.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f9225512640 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 0x5633993bdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f9224d11640 1 -- 172.21.15.82:0/116703116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 msgr2=0x5633993bdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f9224d11640 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 0x5633993bdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f9224d11640 1 -- 172.21.15.82:0/116703116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b76c0 msgr2=0x5633993bd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f9224d11640 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b76c0 0x5633993bd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f9224d11640 1 -- 172.21.15.82:0/116703116 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633990faa20 con 0x5633993b6d10 2024-04-21T08:54:49.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.823+0000 7f921ffff640 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b76c0 0x5633993bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:54:49.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.824+0000 7f9224d11640 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b6d10 0x5633993bd240 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9214099190 tx=0x7f9214098a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c6448ad262e39d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.824+0000 7f91fffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:54:49.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.824+0000 7f91fffff640 1 -- 172.21.15.82:0/116703116 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214010a90 con 0x5633993b6d10 2024-04-21T08:54:49.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.824+0000 7f91fffff640 1 -- 172.21.15.82:0/116703116 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f921409ca20 con 0x5633993b6d10 2024-04-21T08:54:49.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.825+0000 7f9229b67740 1 -- 172.21.15.82:0/116703116 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92140a0b70 con 0x5633993b6d10 2024-04-21T08:54:49.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.826+0000 7f91fffff640 1 -- 172.21.15.82:0/116703116 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f921409cca0 con 0x5633993b6d10 2024-04-21T08:54:49.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.826+0000 7f9229b67740 10 client.26770 fetch_fsmap learned FSMap version 41 2024-04-21T08:54:49.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.826+0000 7f9229b67740 1 -- 172.21.15.82:0/116703116 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56339915b1b0 con 0x5633993b6d10 2024-04-21T08:54:49.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.826+0000 7f91fffff640 1 -- 172.21.15.82:0/116703116 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92140a0b70 con 0x5633993b6d10 2024-04-21T08:54:49.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.827+0000 7f91fffff640 1 -- 172.21.15.82:0/116703116 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f921409bda0 con 0x5633993b6d10 2024-04-21T08:54:49.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.827+0000 7f9229b67740 10 client.26770 fetch_fsmap finished waiting for FSMap version 41 2024-04-21T08:54:49.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.827+0000 7f9229b67740 10 client.26770 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-21T08:54:49.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.827+0000 7f9229b67740 20 client.26770 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:49.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.827+0000 7f9229b67740 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5633993d0500 0x5633993d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.827+0000 7f9229b67740 4 client.26770 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:49.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.827+0000 7f9229b67740 1 -- 172.21.15.82:0/116703116 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563399329a20 con 0x5633993d0500 2024-04-21T08:54:49.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.830+0000 7f921ffff640 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5633993d0500 0x5633993d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.832+0000 7f921ffff640 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5633993d0500 0x5633993d28f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de4bf22c9ee47c85 server_cookie=e33a8e6625cf16b3 in_seq=0 out_seq=0 2024-04-21T08:54:49.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.832+0000 7f91fffff640 10 client.26770 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:49.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.833+0000 7f91fffff640 1 -- 172.21.15.82:0/116703116 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563399329a20 con 0x5633993d0500 2024-04-21T08:54:49.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.833+0000 7f91fffff640 10 client.26770 handle_command_reply: tid=0 2024-04-21T08:54:49.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.833+0000 7f9229b67740 1 client.26770 shutdown 2024-04-21T08:54:49.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.833+0000 7f9229b67740 1 -- 172.21.15.82:0/116703116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b6d10 msgr2=0x5633993bd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.833+0000 7f9229b67740 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b6d10 0x5633993bd240 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9214099190 tx=0x7f9214098a60 comp rx=0 tx=0).stop 2024-04-21T08:54:49.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.834+0000 7f9229b67740 1 -- 172.21.15.82:0/116703116 shutdown_connections 2024-04-21T08:54:49.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.834+0000 7f9229b67740 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5633993d0500 0x5633993d28f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.834+0000 7f9229b67740 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 0x5633993bdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.834+0000 7f9229b67740 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b76c0 0x5633993bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.834+0000 7f9229b67740 1 --2- 172.21.15.82:0/116703116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b6d10 0x5633993bd240 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.834+0000 7f9229b67740 1 -- 172.21.15.82:0/116703116 >> 172.21.15.82:0/116703116 conn(0x5633993b13e0 msgr2=0x5633993b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.834+0000 7f91fffff640 0 client.26770 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:49.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.834+0000 7f9229b67740 1 -- 172.21.15.82:0/116703116 shutdown_connections 2024-04-21T08:54:49.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.834+0000 7f9229b67740 1 -- 172.21.15.82:0/116703116 wait complete. 2024-04-21T08:54:49.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.836+0000 7f9229b67740 20 client.26770 trim_cache size 0 max 16384 2024-04-21T08:54:49.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.871+0000 7f9229b67740 1 Processor -- start 2024-04-21T08:54:49.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.871+0000 7f9229b67740 1 -- start start 2024-04-21T08:54:49.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.872+0000 7f9229b67740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993bf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.872+0000 7f9229b67740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b76c0 0x5633993d5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.872+0000 7f9229b67740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 0x5633993d5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.872+0000 7f9229b67740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f92201ad490 con 0x5633993b6d10 2024-04-21T08:54:49.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.872+0000 7f9229b67740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f92201af090 con 0x5633993c0240 2024-04-21T08:54:49.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.872+0000 7f9229b67740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92201adb60 con 0x5633993b76c0 2024-04-21T08:54:49.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.873+0000 7f9224d11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993bf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.873+0000 7f9224d11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993bf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42362/0 (socket says 172.21.15.82:42362) 2024-04-21T08:54:49.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.873+0000 7f9224d11640 1 -- 172.21.15.82:0/4268917956 learned_addr learned my addr 172.21.15.82:0/4268917956 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:49.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.873+0000 7f921ffff640 1 --2- 172.21.15.82:0/4268917956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b76c0 0x5633993d5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.874+0000 7f9225512640 1 --2- 172.21.15.82:0/4268917956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 0x5633993d5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.874+0000 7f921ffff640 1 -- 172.21.15.82:0/4268917956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 msgr2=0x5633993d5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.874+0000 7f921ffff640 1 --2- 172.21.15.82:0/4268917956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 0x5633993d5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.874+0000 7f921ffff640 1 -- 172.21.15.82:0/4268917956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 msgr2=0x5633993bf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.874+0000 7f921ffff640 1 --2- 172.21.15.82:0/4268917956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993bf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.874+0000 7f921ffff640 1 -- 172.21.15.82:0/4268917956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633993b7460 con 0x5633993b76c0 2024-04-21T08:54:49.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.874+0000 7f921ffff640 1 --2- 172.21.15.82:0/4268917956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b76c0 0x5633993d5530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f920c006010 tx=0x7f920c005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d6c4f8e0e055af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.874+0000 7f920a7fc640 1 -- 172.21.15.82:0/4268917956 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c0044d0 con 0x5633993b76c0 2024-04-21T08:54:49.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.874+0000 7f920a7fc640 1 -- 172.21.15.82:0/4268917956 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f920c050070 con 0x5633993b76c0 2024-04-21T08:54:49.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.874+0000 7f920a7fc640 1 -- 172.21.15.82:0/4268917956 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c04a780 con 0x5633993b76c0 2024-04-21T08:54:49.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.876+0000 7f9229b67740 1 -- 172.21.15.82:0/4268917956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b76c0 msgr2=0x5633993d5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.876+0000 7f9229b67740 1 --2- 172.21.15.82:0/4268917956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b76c0 0x5633993d5530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f920c006010 tx=0x7f920c005a20 comp rx=0 tx=0).stop 2024-04-21T08:54:49.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.878+0000 7f9229b67740 1 -- 172.21.15.82:0/4268917956 shutdown_connections 2024-04-21T08:54:49.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.878+0000 7f9229b67740 1 --2- 172.21.15.82:0/4268917956 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 0x5633993d5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.878+0000 7f9229b67740 1 --2- 172.21.15.82:0/4268917956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b76c0 0x5633993d5530 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.878+0000 7f9229b67740 1 --2- 172.21.15.82:0/4268917956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993bf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.878+0000 7f9229b67740 1 -- 172.21.15.82:0/4268917956 >> 172.21.15.82:0/4268917956 conn(0x5633993b13e0 msgr2=0x5633993b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.880+0000 7f9229b67740 1 -- 172.21.15.82:0/4268917956 shutdown_connections 2024-04-21T08:54:49.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.880+0000 7f9229b67740 1 -- 172.21.15.82:0/4268917956 wait complete. 2024-04-21T08:54:49.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.880+0000 7f9229b67740 1 -- start start 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.881+0000 7f9229b67740 1 Processor -- start 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.881+0000 7f9229b67740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.881+0000 7f9229b67740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b76c0 0x5633993d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.882+0000 7f9229b67740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 0x5633993d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.882+0000 7f9229b67740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f92201ae990 con 0x5633993b6d10 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.882+0000 7f9229b67740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f92201ae290 con 0x5633993c0240 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.882+0000 7f9229b67740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92201ab460 con 0x5633993b76c0 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.882+0000 7f9229b67740 10 client.0 _ll_register_callbacks cb 0x7f92140a5430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.882+0000 7f9224d11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.882+0000 7f9224d11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993d54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42368/0 (socket says 172.21.15.82:42368) 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.882+0000 7f9224d11640 1 -- 172.21.15.82:0/2696990642 learned_addr learned my addr 172.21.15.82:0/2696990642 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.882+0000 7f921ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b76c0 0x5633993d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.883+0000 7f9225512640 1 --2- 172.21.15.82:0/2696990642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 0x5633993d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.883+0000 7f9224d11640 1 -- 172.21.15.82:0/2696990642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b76c0 msgr2=0x5633993d5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.883+0000 7f9224d11640 1 --2- 172.21.15.82:0/2696990642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b76c0 0x5633993d5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.883+0000 7f9224d11640 1 -- 172.21.15.82:0/2696990642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 msgr2=0x5633993d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.883+0000 7f9224d11640 1 --2- 172.21.15.82:0/2696990642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 0x5633993d2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.883+0000 7f9224d11640 1 -- 172.21.15.82:0/2696990642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563398fee310 con 0x5633993b6d10 2024-04-21T08:54:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.883+0000 7f9224d11640 1 --2- 172.21.15.82:0/2696990642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993d54e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9214009230 tx=0x7f92140d0f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4acb14ab8a4a57b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.883+0000 7f91fffff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.883+0000 7f91fffff640 1 -- 172.21.15.82:0/2696990642 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92140a0b70 con 0x5633993b6d10 2024-04-21T08:54:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.884+0000 7f91fffff640 1 -- 172.21.15.82:0/2696990642 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f92140a4fa0 con 0x5633993b6d10 2024-04-21T08:54:49.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.884+0000 7f9229b67740 1 -- 172.21.15.82:0/2696990642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f920c050070 con 0x5633993b6d10 2024-04-21T08:54:49.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.885+0000 7f91fffff640 1 -- 172.21.15.82:0/2696990642 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f921409c7c0 con 0x5633993b6d10 2024-04-21T08:54:49.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.885+0000 7f9229b67740 10 client.17841 fetch_fsmap learned FSMap version 41 2024-04-21T08:54:49.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.885+0000 7f9229b67740 1 -- 172.21.15.82:0/2696990642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56339943e950 con 0x5633993b6d10 2024-04-21T08:54:49.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.887+0000 7f91fffff640 1 -- 172.21.15.82:0/2696990642 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92140a51e0 con 0x5633993b6d10 2024-04-21T08:54:49.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.887+0000 7f91fffff640 1 -- 172.21.15.82:0/2696990642 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f921409bc10 con 0x5633993b6d10 2024-04-21T08:54:49.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.888+0000 7f9229b67740 10 client.17841 fetch_fsmap finished waiting for FSMap version 41 2024-04-21T08:54:49.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.888+0000 7f9229b67740 10 client.17841 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-21T08:54:49.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.888+0000 7f9229b67740 20 client.17841 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:49.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.888+0000 7f9229b67740 1 --2- 172.21.15.82:0/2696990642 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56339943eec0 0x5633994412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:49.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.888+0000 7f9229b67740 4 client.17841 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-21T08:54:49.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.888+0000 7f9229b67740 1 -- 172.21.15.82:0/2696990642 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x7f920c050070 con 0x56339943eec0 2024-04-21T08:54:49.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.891+0000 7f921ffff640 1 --2- 172.21.15.82:0/2696990642 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56339943eec0 0x5633994412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:49.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.893+0000 7f921ffff640 1 --2- 172.21.15.82:0/2696990642 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56339943eec0 0x5633994412e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ceee1165a471b1 server_cookie=7edd6e7d7b8df327 in_seq=0 out_seq=0 2024-04-21T08:54:49.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.893+0000 7f91fffff640 10 client.17841 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:49.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.893+0000 7f91fffff640 1 -- 172.21.15.82:0/2696990642 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x7f920c050070 con 0x56339943eec0 2024-04-21T08:54:49.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.893+0000 7f91fffff640 10 client.17841 handle_command_reply: tid=0 2024-04-21T08:54:49.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.893+0000 7f9229b67740 1 client.17841 shutdown 2024-04-21T08:54:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.894+0000 7f9229b67740 1 -- 172.21.15.82:0/2696990642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 msgr2=0x5633993d54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.894+0000 7f9229b67740 1 --2- 172.21.15.82:0/2696990642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993d54e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9214009230 tx=0x7f92140d0f30 comp rx=0 tx=0).stop 2024-04-21T08:54:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.894+0000 7f9229b67740 1 -- 172.21.15.82:0/2696990642 shutdown_connections 2024-04-21T08:54:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.894+0000 7f9229b67740 1 --2- 172.21.15.82:0/2696990642 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x56339943eec0 0x5633994412e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.894+0000 7f91fffff640 0 client.17841 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.895+0000 7f9229b67740 1 --2- 172.21.15.82:0/2696990642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5633993c0240 0x5633993d2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.895+0000 7f9229b67740 1 --2- 172.21.15.82:0/2696990642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5633993b76c0 0x5633993d5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.895+0000 7f9229b67740 1 --2- 172.21.15.82:0/2696990642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633993b6d10 0x5633993d54e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.895+0000 7f9229b67740 1 -- 172.21.15.82:0/2696990642 >> 172.21.15.82:0/2696990642 conn(0x5633993b13e0 msgr2=0x5633993b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.895+0000 7f9229b67740 1 -- 172.21.15.82:0/2696990642 shutdown_connections 2024-04-21T08:54:49.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.895+0000 7f9229b67740 1 -- 172.21.15.82:0/2696990642 wait complete. 2024-04-21T08:54:49.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.895+0000 7f9229b67740 20 client.17841 trim_cache size 0 max 16384 2024-04-21T08:54:49.896 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:54:49.896 INFO:teuthology.orchestra.run.smithi082.stdout: "epoch": 41, 2024-04-21T08:54:49.896 INFO:teuthology.orchestra.run.smithi082.stdout: "leader": 24509, 2024-04-21T08:54:49.896 INFO:teuthology.orchestra.run.smithi082.stdout: "set_version": 3, 2024-04-21T08:54:49.896 INFO:teuthology.orchestra.run.smithi082.stdout: "sets": {} 2024-04-21T08:54:49.896 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:54:49.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.898+0000 7f9226f75640 1 -- 172.21.15.82:0/1139601220 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f92100a3e70 msgr2=0x7f92100a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.898+0000 7f9226f75640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f92100a3e70 0x7f92100a6330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f92180049d0 tx=0x7f9218004a40 comp rx=0 tx=0).stop 2024-04-21T08:54:49.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.898+0000 7f9226f75640 1 -- 172.21.15.82:0/1139601220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9220102190 msgr2=0x7f922019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:49.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.898+0000 7f9226f75640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9220102190 0x7f922019e3e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f921400e9f0 tx=0x7f921400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.899+0000 7f9226f75640 1 -- 172.21.15.82:0/1139601220 shutdown_connections 2024-04-21T08:54:49.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.899+0000 7f9226f75640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9220198590 0x7f9220198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.899+0000 7f9226f75640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220107260 0x7f922019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.899+0000 7f9226f75640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f92100a3e70 0x7f92100a6330 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.899+0000 7f9226f75640 1 --2- 172.21.15.82:0/1139601220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9220102190 0x7f922019e3e0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:49.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.899+0000 7f9226f75640 1 -- 172.21.15.82:0/1139601220 >> 172.21.15.82:0/1139601220 conn(0x7f92200793a0 msgr2=0x7f9220100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:49.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.899+0000 7f9226f75640 1 -- 172.21.15.82:0/1139601220 shutdown_connections 2024-04-21T08:54:49.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:49.899+0000 7f9226f75640 1 -- 172.21.15.82:0/1139601220 wait complete. 2024-04-21T08:54:49.911 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/quiescer.py", line 263, in stop_all_quiescers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: Error releasing root: -124 (Unknown) 2024-04-21T08:54:49.912 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-21T08:54:49.926 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi082.front.sepia.ceph.com 2024-04-21T08:54:49.935 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-21T08:54:49.947 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-21T08:54:49.947 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T08:54:49.947 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-04-21T08:54:49.947 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-21T08:54:50.000 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-21T08:54:50.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:50 smithi134 ceph-mon[31772]: pgmap v239: 129 pgs: 129 active+clean; 888 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 571 op/s 2024-04-21T08:54:50.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:50 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/979667221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:54:50.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:50 smithi082 ceph-mon[27967]: pgmap v239: 129 pgs: 129 active+clean; 888 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 571 op/s 2024-04-21T08:54:50.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:50 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/979667221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:54:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:50 smithi106 ceph-mon[31072]: pgmap v239: 129 pgs: 129 active+clean; 888 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 571 op/s 2024-04-21T08:54:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:50 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/979667221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:54:52.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:52 smithi134 ceph-mon[31772]: pgmap v240: 129 pgs: 129 active+clean; 888 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 438 op/s 2024-04-21T08:54:52.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:52 smithi082 ceph-mon[27967]: pgmap v240: 129 pgs: 129 active+clean; 888 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 438 op/s 2024-04-21T08:54:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:52 smithi106 ceph-mon[31072]: pgmap v240: 129 pgs: 129 active+clean; 888 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 438 op/s 2024-04-21T08:54:54.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:54 smithi134 ceph-mon[31772]: pgmap v241: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 601 op/s 2024-04-21T08:54:54.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:54 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:54:54.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:54 smithi082 ceph-mon[27967]: pgmap v241: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 601 op/s 2024-04-21T08:54:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:54 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:54:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:54 smithi106 ceph-mon[31072]: pgmap v241: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 601 op/s 2024-04-21T08:54:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:54 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:54:56.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:56 smithi134 ceph-mon[31772]: pgmap v242: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 335 op/s 2024-04-21T08:54:56.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:56 smithi082 ceph-mon[27967]: pgmap v242: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 335 op/s 2024-04-21T08:54:56.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:56 smithi106 ceph-mon[31072]: pgmap v242: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 335 op/s 2024-04-21T08:54:58.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:54:58 smithi134 ceph-mon[31772]: pgmap v243: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 337 op/s 2024-04-21T08:54:58.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:54:58 smithi082 ceph-mon[27967]: pgmap v243: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 337 op/s 2024-04-21T08:54:58.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:54:58 smithi106 ceph-mon[31072]: pgmap v243: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 337 op/s 2024-04-21T08:54:58.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:54:58.842 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:54:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.954+0000 7f2d9ba77640 1 -- 172.21.15.82:0/250611577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d941031b0 msgr2=0x7f2d94103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.954+0000 7f2d9ba77640 1 --2- 172.21.15.82:0/250611577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d941031b0 0x7f2d94103590 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009900 tx=0x7f2d8802bcd0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.954+0000 7f2d9ba77640 1 -- 172.21.15.82:0/250611577 shutdown_connections 2024-04-21T08:54:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.954+0000 7f2d9ba77640 1 --2- 172.21.15.82:0/250611577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9410c580 0x7f2d9410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.954+0000 7f2d9ba77640 1 --2- 172.21.15.82:0/250611577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d94103ad0 0x7f2d9410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.954+0000 7f2d9ba77640 1 --2- 172.21.15.82:0/250611577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d941031b0 0x7f2d94103590 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.954+0000 7f2d9ba77640 1 -- 172.21.15.82:0/250611577 >> 172.21.15.82:0/250611577 conn(0x7f2d940771d0 msgr2=0x7f2d940775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.954+0000 7f2d9ba77640 1 -- 172.21.15.82:0/250611577 shutdown_connections 2024-04-21T08:54:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.954+0000 7f2d9ba77640 1 -- 172.21.15.82:0/250611577 wait complete. 2024-04-21T08:54:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d9ba77640 1 Processor -- start 2024-04-21T08:54:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d9ba77640 1 -- start start 2024-04-21T08:54:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d9ba77640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d941031b0 0x7f2d9419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d9ba77640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d94103ad0 0x7f2d9419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d9ba77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9410c580 0x7f2d94198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d9ba77640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94112220 con 0x7f2d94103ad0 2024-04-21T08:54:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d9ba77640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941120a0 con 0x7f2d941031b0 2024-04-21T08:54:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d9ba77640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941123a0 con 0x7f2d9410c580 2024-04-21T08:54:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d99012640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d94103ad0 0x7f2d9419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d99012640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d94103ad0 0x7f2d9419eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42970/0 (socket says 172.21.15.82:42970) 2024-04-21T08:54:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d99012640 1 -- 172.21.15.82:0/3879248551 learned_addr learned my addr 172.21.15.82:0/3879248551 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:58.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d99813640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d941031b0 0x7f2d9419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d9a014640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9410c580 0x7f2d94198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d99012640 1 -- 172.21.15.82:0/3879248551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9410c580 msgr2=0x7f2d94198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:58.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d99012640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9410c580 0x7f2d94198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d99012640 1 -- 172.21.15.82:0/3879248551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d941031b0 msgr2=0x7f2d9419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:58.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d99012640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d941031b0 0x7f2d9419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d99012640 1 -- 172.21.15.82:0/3879248551 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d94198f00 con 0x7f2d94103ad0 2024-04-21T08:54:58.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.955+0000 7f2d99813640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d941031b0 0x7f2d9419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:58.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.956+0000 7f2d99012640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d94103ad0 0x7f2d9419eac0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000d950 tx=0x7f2d9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:58.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.956+0000 7f2d86ffd640 1 -- 172.21.15.82:0/3879248551 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90017070 con 0x7f2d94103ad0 2024-04-21T08:54:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.956+0000 7f2d9ba77640 1 -- 172.21.15.82:0/3879248551 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d941991f0 con 0x7f2d94103ad0 2024-04-21T08:54:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.956+0000 7f2d86ffd640 1 -- 172.21.15.82:0/3879248551 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d90009c30 con 0x7f2d94103ad0 2024-04-21T08:54:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.956+0000 7f2d86ffd640 1 -- 172.21.15.82:0/3879248551 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9000f620 con 0x7f2d94103ad0 2024-04-21T08:54:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.956+0000 7f2d9ba77640 1 -- 172.21.15.82:0/3879248551 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d941a5500 con 0x7f2d94103ad0 2024-04-21T08:54:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.958+0000 7f2d86ffd640 1 -- 172.21.15.82:0/3879248551 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2d90010460 con 0x7f2d94103ad0 2024-04-21T08:54:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.958+0000 7f2d9e660740 1 Processor -- start 2024-04-21T08:54:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9e660740 1 -- start start 2024-04-21T08:54:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9e660740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18bfe960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9e660740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18bfeea0 0x55db18c07540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9e660740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18c07a80 0x55db18c09e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9e660740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55db18b631e0 con 0x55db18bfe580 2024-04-21T08:54:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9e660740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55db18b00390 con 0x55db18bfeea0 2024-04-21T08:54:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9e660740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55db18b5f1a0 con 0x55db18c07a80 2024-04-21T08:54:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9a014640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18bfe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9a014640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18bfe960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42976/0 (socket says 172.21.15.82:42976) 2024-04-21T08:54:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9a014640 1 -- 172.21.15.82:0/2462421091 learned_addr learned my addr 172.21.15.82:0/2462421091 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d99813640 1 --2- 172.21.15.82:0/2462421091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18bfeea0 0x55db18c07540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d86ffd640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2d800a3d20 0x7f2d800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d99012640 1 --2- 172.21.15.82:0/2462421091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18c07a80 0x55db18c09e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9a014640 1 -- 172.21.15.82:0/2462421091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18c07a80 msgr2=0x55db18c09e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9a014640 1 --2- 172.21.15.82:0/2462421091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18c07a80 0x55db18c09e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9a014640 1 -- 172.21.15.82:0/2462421091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18bfeea0 msgr2=0x55db18c07540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9a014640 1 --2- 172.21.15.82:0/2462421091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18bfeea0 0x55db18c07540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9a014640 1 -- 172.21.15.82:0/2462421091 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db1893f7d0 con 0x55db18bfe580 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d99012640 1 --2- 172.21.15.82:0/2462421091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18c07a80 0x55db18c09e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d86ffd640 1 -- 172.21.15.82:0/3879248551 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f2d900d0920 con 0x7f2d94103ad0 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.959+0000 7f2d9a014640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2d800a3d20 0x7f2d800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.960+0000 7f2d9a014640 1 --2- 172.21.15.82:0/2462421091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18bfe960 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c007ee0 tx=0x7f2d8c00c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e1baec61ea881a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.960+0000 7f2d7ffff640 1 -- 172.21.15.82:0/2462421091 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c017040 con 0x55db18bfe580 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.960+0000 7f2d7ffff640 1 -- 172.21.15.82:0/2462421091 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d8c01edd0 con 0x55db18bfe580 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.960+0000 7f2d7ffff640 1 -- 172.21.15.82:0/2462421091 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c01dab0 con 0x55db18bfe580 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9e660740 1 -- 172.21.15.82:0/2462421091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 msgr2=0x55db18bfe960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2462421091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18bfe960 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c007ee0 tx=0x7f2d8c00c560 comp rx=0 tx=0).stop 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9a014640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2d800a3d20 0x7f2d800a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c01d000 tx=0x7f2d8c01e000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9e660740 1 -- 172.21.15.82:0/2462421091 shutdown_connections 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2462421091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18c07a80 0x55db18c09e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2462421091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18bfeea0 0x55db18c07540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2462421091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18bfe960 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9e660740 1 -- 172.21.15.82:0/2462421091 >> 172.21.15.82:0/2462421091 conn(0x55db18bf8cb0 msgr2=0x55db18bf9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9e660740 1 -- 172.21.15.82:0/2462421091 shutdown_connections 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9e660740 1 -- 172.21.15.82:0/2462421091 wait complete. 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9e660740 1 -- start start 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9e660740 1 Processor -- start 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.961+0000 7f2d9e660740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18c04ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d9e660740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18bfeea0 0x55db18c05010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d9e660740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18c07a80 0x55db18c05550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d9e660740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55db18c0a870 con 0x55db18bfe580 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d9e660740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55db18b1c2f0 con 0x55db18bfeea0 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d9e660740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55db18c0a9f0 con 0x55db18c07a80 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d9e660740 10 client.0 _ll_register_callbacks cb 0x55db18b29580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d99813640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18c04ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d99813640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18c04ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42978/0 (socket says 172.21.15.82:42978) 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d99813640 1 -- 172.21.15.82:0/2719628541 learned_addr learned my addr 172.21.15.82:0/2719628541 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d9a014640 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18c07a80 0x55db18c05550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d99012640 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18bfeea0 0x55db18c05010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d99813640 1 -- 172.21.15.82:0/2719628541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18c07a80 msgr2=0x55db18c05550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d99813640 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18c07a80 0x55db18c05550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d99813640 1 -- 172.21.15.82:0/2719628541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18bfeea0 msgr2=0x55db18c05010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d99813640 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18bfeea0 0x55db18c05010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d99813640 1 -- 172.21.15.82:0/2719628541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db188753f0 con 0x55db18bfe580 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d99813640 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18c04ad0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009900 tx=0x7f2d88005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe176fcd710a0f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d7d7fa640 1 -- 172.21.15.82:0/2719628541 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88043070 con 0x55db18bfe580 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d9e660740 1 -- 172.21.15.82:0/2719628541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d8c01edd0 con 0x55db18bfe580 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d7d7fa640 1 -- 172.21.15.82:0/2719628541 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f2d8802ca80 con 0x55db18bfe580 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d7d7fa640 1 -- 172.21.15.82:0/2719628541 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2d8804f050 con 0x55db18bfe580 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d9e660740 10 client.17853 fetch_fsmap learned FSMap version 42 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.962+0000 7f2d9e660740 1 -- 172.21.15.82:0/2719628541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55db188350f0 con 0x55db18bfe580 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.963+0000 7f2d7d7fa640 1 -- 172.21.15.82:0/2719628541 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d8c01edd0 con 0x55db18bfe580 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.963+0000 7f2d7d7fa640 1 -- 172.21.15.82:0/2719628541 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7f2d880035f0 con 0x55db18bfe580 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.963+0000 7f2d9e660740 10 client.17853 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.963+0000 7f2d9e660740 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.963+0000 7f2d9e660740 20 client.17853 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.963+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55db18c179d0 0x55db18c19e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.963+0000 7f2d9e660740 4 client.17853 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.963+0000 7f2d9e660740 1 -- 172.21.15.82:0/2719628541 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db18b70c00 con 0x55db18c179d0 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.963+0000 7f2d99012640 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55db18c179d0 0x55db18c19e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.963+0000 7f2d99012640 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55db18c179d0 0x55db18c19e70 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57bd125c9d0e04d5 server_cookie=2d4cc943e330ab6c in_seq=0 out_seq=0 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.963+0000 7f2d7d7fa640 10 client.17853 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.964+0000 7f2d7d7fa640 1 -- 172.21.15.82:0/2719628541 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db18b70c00 con 0x55db18c179d0 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.964+0000 7f2d7d7fa640 10 client.17853 handle_command_reply: tid=0 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.964+0000 7f2d9e660740 1 client.17853 shutdown 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.964+0000 7f2d9e660740 1 -- 172.21.15.82:0/2719628541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 msgr2=0x55db18c04ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.964+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18c04ad0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009900 tx=0x7f2d88005250 comp rx=0 tx=0).stop 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.964+0000 7f2d9e660740 1 -- 172.21.15.82:0/2719628541 shutdown_connections 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.964+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55db18c179d0 0x55db18c19e70 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.964+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18c07a80 0x55db18c05550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.964+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18bfeea0 0x55db18c05010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.964+0000 7f2d7d7fa640 0 client.17853 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:58.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.964+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2719628541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18c04ad0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.964+0000 7f2d9e660740 1 -- 172.21.15.82:0/2719628541 >> 172.21.15.82:0/2719628541 conn(0x55db18bf8cb0 msgr2=0x55db18bfaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:58.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.965+0000 7f2d9e660740 1 -- 172.21.15.82:0/2719628541 shutdown_connections 2024-04-21T08:54:58.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.965+0000 7f2d9e660740 1 -- 172.21.15.82:0/2719628541 wait complete. 2024-04-21T08:54:58.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.965+0000 7f2d9e660740 20 client.17853 trim_cache size 0 max 16384 2024-04-21T08:54:58.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.982+0000 7f2d9e660740 1 Processor -- start 2024-04-21T08:54:58.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.982+0000 7f2d9e660740 1 -- start start 2024-04-21T08:54:58.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.982+0000 7f2d9e660740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18c07200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.982+0000 7f2d9e660740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18bfeea0 0x55db18c03a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.982+0000 7f2d9e660740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18c07a80 0x55db18c13760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.982+0000 7f2d9e660740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941af600 con 0x55db18bfe580 2024-04-21T08:54:58.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.982+0000 7f2d9e660740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941b1230 con 0x55db18c07a80 2024-04-21T08:54:58.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.982+0000 7f2d9e660740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941afcd0 con 0x55db18bfeea0 2024-04-21T08:54:58.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.982+0000 7f2d99813640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18c07200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.982+0000 7f2d99813640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18c07200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42988/0 (socket says 172.21.15.82:42988) 2024-04-21T08:54:58.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.982+0000 7f2d99813640 1 -- 172.21.15.82:0/3043112997 learned_addr learned my addr 172.21.15.82:0/3043112997 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d99813640 1 -- 172.21.15.82:0/3043112997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18bfeea0 msgr2=0x55db18c03a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d99012640 1 --2- 172.21.15.82:0/3043112997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18bfeea0 0x55db18c03a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d99813640 1 --2- 172.21.15.82:0/3043112997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18bfeea0 0x55db18c03a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d99813640 1 -- 172.21.15.82:0/3043112997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18c07a80 msgr2=0x55db18c13760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d99813640 1 --2- 172.21.15.82:0/3043112997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18c07a80 0x55db18c13760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d99813640 1 -- 172.21.15.82:0/3043112997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db18c03020 con 0x55db18bfe580 2024-04-21T08:54:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d99012640 1 --2- 172.21.15.82:0/3043112997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18bfeea0 0x55db18c03a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d99813640 1 --2- 172.21.15.82:0/3043112997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18c07200 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2d8802f020 tx=0x7f2d88005920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f76c850a0d81ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d7f7fe640 1 -- 172.21.15.82:0/3043112997 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88043040 con 0x55db18bfe580 2024-04-21T08:54:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d7f7fe640 1 -- 172.21.15.82:0/3043112997 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d88003d40 con 0x55db18bfe580 2024-04-21T08:54:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d7f7fe640 1 -- 172.21.15.82:0/3043112997 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d880046c0 con 0x55db18bfe580 2024-04-21T08:54:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d9e660740 1 -- 172.21.15.82:0/3043112997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 msgr2=0x55db18c07200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d9e660740 1 --2- 172.21.15.82:0/3043112997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18c07200 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2d8802f020 tx=0x7f2d88005920 comp rx=0 tx=0).stop 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d9e660740 1 -- 172.21.15.82:0/3043112997 shutdown_connections 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d9e660740 1 --2- 172.21.15.82:0/3043112997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18c07a80 0x55db18c13760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d9e660740 1 --2- 172.21.15.82:0/3043112997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18bfeea0 0x55db18c03a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.983+0000 7f2d9e660740 1 --2- 172.21.15.82:0/3043112997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfe580 0x55db18c07200 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d9e660740 1 -- 172.21.15.82:0/3043112997 >> 172.21.15.82:0/3043112997 conn(0x55db18bf8cb0 msgr2=0x55db18bfa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d9e660740 1 -- 172.21.15.82:0/3043112997 shutdown_connections 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d9e660740 1 -- 172.21.15.82:0/3043112997 wait complete. 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d9e660740 1 -- start start 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d9e660740 1 Processor -- start 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d9e660740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18bfe580 0x55db18c024f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d9e660740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfeea0 0x55db18c00d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d9e660740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18c07a80 0x55db18c01280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d9e660740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941b0b30 con 0x55db18bfeea0 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d9e660740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941b0430 con 0x55db18c07a80 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d9e660740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941ad5d0 con 0x55db18bfe580 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d9e660740 10 client.0 _ll_register_callbacks cb 0x7f2d88004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d99012640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfeea0 0x55db18c00d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d99012640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfeea0 0x55db18c00d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43000/0 (socket says 172.21.15.82:43000) 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.984+0000 7f2d99012640 1 -- 172.21.15.82:0/2711025532 learned_addr learned my addr 172.21.15.82:0/2711025532 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d99813640 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18bfe580 0x55db18c024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d99012640 1 -- 172.21.15.82:0/2711025532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18bfe580 msgr2=0x55db18c024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d99012640 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18bfe580 0x55db18c024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d99012640 1 -- 172.21.15.82:0/2711025532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18c07a80 msgr2=0x55db18c01280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d99012640 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18c07a80 0x55db18c01280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d99012640 1 -- 172.21.15.82:0/2711025532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db18c018f0 con 0x55db18bfeea0 2024-04-21T08:54:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d99813640 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18bfe580 0x55db18c024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d99012640 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfeea0 0x55db18c00d40 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2d90010cf0 tx=0x7f2d9006ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3434b05e0ef1b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d7d7fa640 1 -- 172.21.15.82:0/2711025532 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90020040 con 0x55db18bfeea0 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d7d7fa640 1 -- 172.21.15.82:0/2711025532 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f2d900cdee0 con 0x55db18bfeea0 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d9e660740 1 -- 172.21.15.82:0/2711025532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d88003d40 con 0x55db18bfeea0 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d7d7fa640 1 -- 172.21.15.82:0/2711025532 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2d90009de0 con 0x55db18bfeea0 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d9e660740 10 client.17859 fetch_fsmap learned FSMap version 42 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d9e660740 1 -- 172.21.15.82:0/2711025532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55db18c82e20 con 0x55db18bfeea0 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d7d7fa640 1 -- 172.21.15.82:0/2711025532 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d88003d40 con 0x55db18bfeea0 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d7d7fa640 1 -- 172.21.15.82:0/2711025532 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7f2d90098b20 con 0x55db18bfeea0 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d9e660740 10 client.17859 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d9e660740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d9e660740 20 client.17859 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:58.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.985+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55db18c835d0 0x55db18c859f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.986+0000 7f2d9e660740 4 client.17859 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.986+0000 7f2d9e660740 1 -- 172.21.15.82:0/2711025532 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db18b6aa70 con 0x55db18c835d0 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.986+0000 7f2d99813640 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55db18c835d0 0x55db18c859f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.986+0000 7f2d99813640 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55db18c835d0 0x55db18c859f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cd90d43a888635f server_cookie=9fde8f5f99b60148 in_seq=0 out_seq=0 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.986+0000 7f2d7d7fa640 10 client.17859 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.986+0000 7f2d7d7fa640 1 -- 172.21.15.82:0/2711025532 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55db18b6aa70 con 0x55db18c835d0 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.986+0000 7f2d7d7fa640 10 client.17859 handle_command_reply: tid=0 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.986+0000 7f2d9e660740 1 client.17859 shutdown 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.986+0000 7f2d9e660740 1 -- 172.21.15.82:0/2711025532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfeea0 msgr2=0x55db18c00d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.986+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfeea0 0x55db18c00d40 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2d90010cf0 tx=0x7f2d9006ae60 comp rx=0 tx=0).stop 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.986+0000 7f2d9e660740 1 -- 172.21.15.82:0/2711025532 shutdown_connections 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55db18c835d0 0x55db18c859f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55db18c07a80 0x55db18c01280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db18bfeea0 0x55db18c00d40 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9e660740 1 --2- 172.21.15.82:0/2711025532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55db18bfe580 0x55db18c024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d7d7fa640 0 client.17859 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:54:58.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9e660740 1 -- 172.21.15.82:0/2711025532 >> 172.21.15.82:0/2711025532 conn(0x55db18bf8cb0 msgr2=0x55db18bf74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:58.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9e660740 1 -- 172.21.15.82:0/2711025532 shutdown_connections 2024-04-21T08:54:58.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9e660740 1 -- 172.21.15.82:0/2711025532 wait complete. 2024-04-21T08:54:58.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9e660740 20 client.17859 trim_cache size 0 max 16384 2024-04-21T08:54:58.994 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:54:58.994 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:54:58.994 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:54:58.995 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:54:58.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9ba77640 1 -- 172.21.15.82:0/3879248551 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2d800a3d20 msgr2=0x7f2d800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:58.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9ba77640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2d800a3d20 0x7f2d800a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c01d000 tx=0x7f2d8c01e000 comp rx=0 tx=0).stop 2024-04-21T08:54:58.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9ba77640 1 -- 172.21.15.82:0/3879248551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d94103ad0 msgr2=0x7f2d9419eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:58.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9ba77640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d94103ad0 0x7f2d9419eac0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000d950 tx=0x7f2d9000de10 comp rx=0 tx=0).stop 2024-04-21T08:54:58.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9ba77640 1 -- 172.21.15.82:0/3879248551 shutdown_connections 2024-04-21T08:54:58.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9ba77640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d9410c580 0x7f2d94198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9ba77640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2d800a3d20 0x7f2d800a61e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9ba77640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d94103ad0 0x7f2d9419eac0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9ba77640 1 --2- 172.21.15.82:0/3879248551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d941031b0 0x7f2d9419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:58.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9ba77640 1 -- 172.21.15.82:0/3879248551 >> 172.21.15.82:0/3879248551 conn(0x7f2d940771d0 msgr2=0x7f2d940ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:58.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9ba77640 1 -- 172.21.15.82:0/3879248551 shutdown_connections 2024-04-21T08:54:58.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:58.987+0000 7f2d9ba77640 1 -- 172.21.15.82:0/3879248551 wait complete. 2024-04-21T08:54:58.998 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:54:58.998 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:54:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.096+0000 7f24709ef640 1 -- 172.21.15.82:0/2108884178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f246c06a8d0 msgr2=0x7f246c06acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.096+0000 7f24709ef640 1 --2- 172.21.15.82:0/2108884178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f246c06a8d0 0x7f246c06acb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f245c009ca0 tx=0x7f245c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.096+0000 7f24709ef640 1 -- 172.21.15.82:0/2108884178 shutdown_connections 2024-04-21T08:54:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.096+0000 7f24709ef640 1 --2- 172.21.15.82:0/2108884178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f246c107260 0x7f246c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.096+0000 7f24709ef640 1 --2- 172.21.15.82:0/2108884178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f246c06b280 0x7f246c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.096+0000 7f24709ef640 1 --2- 172.21.15.82:0/2108884178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f246c06a8d0 0x7f246c06acb0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.096+0000 7f24709ef640 1 -- 172.21.15.82:0/2108884178 >> 172.21.15.82:0/2108884178 conn(0x7f246c077c20 msgr2=0x7f246c078030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.096+0000 7f24709ef640 1 -- 172.21.15.82:0/2108884178 shutdown_connections 2024-04-21T08:54:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.096+0000 7f24709ef640 1 -- 172.21.15.82:0/2108884178 wait complete. 2024-04-21T08:54:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.097+0000 7f24709ef640 1 Processor -- start 2024-04-21T08:54:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.097+0000 7f24709ef640 1 -- start start 2024-04-21T08:54:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.097+0000 7f24709ef640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f246c06a8d0 0x7f246c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.097+0000 7f24709ef640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f246c107260 0x7f246c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.097+0000 7f24709ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f246c1985a0 0x7f246c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.097+0000 7f24709ef640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f246c112220 con 0x7f246c06a8d0 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.097+0000 7f24709ef640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f246c1120a0 con 0x7f246c107260 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.097+0000 7f24709ef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f246c1123a0 con 0x7f246c1985a0 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.097+0000 7f246a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f246c06a8d0 0x7f246c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.097+0000 7f246a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f246c06a8d0 0x7f246c19e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43016/0 (socket says 172.21.15.82:43016) 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.097+0000 7f246a59c640 1 -- 172.21.15.82:0/549897278 learned_addr learned my addr 172.21.15.82:0/549897278 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f2469d9b640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f246c107260 0x7f246c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f246ad9d640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f246c1985a0 0x7f246c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f246a59c640 1 -- 172.21.15.82:0/549897278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f246c1985a0 msgr2=0x7f246c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f246a59c640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f246c1985a0 0x7f246c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f246a59c640 1 -- 172.21.15.82:0/549897278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f246c107260 msgr2=0x7f246c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f246a59c640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f246c107260 0x7f246c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f246a59c640 1 -- 172.21.15.82:0/549897278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246c1990c0 con 0x7f246c06a8d0 2024-04-21T08:54:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f246ad9d640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f246c1985a0 0x7f246c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f246a59c640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f246c06a8d0 0x7f246c19e3f0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f245c005e30 tx=0x7f245c034780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f245b7fe640 1 -- 172.21.15.82:0/549897278 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c039040 con 0x7f246c06a8d0 2024-04-21T08:54:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f24709ef640 1 -- 172.21.15.82:0/549897278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246c199350 con 0x7f246c06a8d0 2024-04-21T08:54:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f245b7fe640 1 -- 172.21.15.82:0/549897278 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f245c044070 con 0x7f246c06a8d0 2024-04-21T08:54:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f245b7fe640 1 -- 172.21.15.82:0/549897278 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c003e60 con 0x7f246c06a8d0 2024-04-21T08:54:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.098+0000 7f24709ef640 1 -- 172.21.15.82:0/549897278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246c1a55a0 con 0x7f246c06a8d0 2024-04-21T08:54:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.099+0000 7f24709ef640 1 -- 172.21.15.82:0/549897278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f246c069230 con 0x7f246c06a8d0 2024-04-21T08:54:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.100+0000 7f245b7fe640 1 -- 172.21.15.82:0/549897278 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f245c0324c0 con 0x7f246c06a8d0 2024-04-21T08:54:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.101+0000 7f245b7fe640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f24540a3d20 0x7f24540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.101+0000 7f2469d9b640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f24540a3d20 0x7f24540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.101+0000 7f245b7fe640 1 -- 172.21.15.82:0/549897278 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f245c0f2e30 con 0x7f246c06a8d0 2024-04-21T08:54:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.101+0000 7f2469d9b640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f24540a3d20 0x7f24540a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f246c100db0 tx=0x7f246000b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.106+0000 7f245b7fe640 1 -- 172.21.15.82:0/549897278 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f245c0bf1d0 con 0x7f246c06a8d0 2024-04-21T08:54:59.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.323+0000 7f24709ef640 1 -- 172.21.15.82:0/549897278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f246c06b280 con 0x7f246c06a8d0 2024-04-21T08:54:59.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.324+0000 7f245b7fe640 1 -- 172.21.15.82:0/549897278 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11953 (secure 0 0 0) 0x7f245c0bead0 con 0x7f246c06a8d0 2024-04-21T08:54:59.324 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:54:59.324 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:54:52.215593+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24409,24407,24476,24509,24481]},"id":1}]} 2024-04-21T08:54:59.324 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 42 2024-04-21T08:54:59.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.326+0000 7f24709ef640 1 -- 172.21.15.82:0/549897278 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f24540a3d20 msgr2=0x7f24540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.326+0000 7f24709ef640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f24540a3d20 0x7f24540a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f246c100db0 tx=0x7f246000b390 comp rx=0 tx=0).stop 2024-04-21T08:54:59.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.326+0000 7f24709ef640 1 -- 172.21.15.82:0/549897278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f246c06a8d0 msgr2=0x7f246c19e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.326+0000 7f24709ef640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f246c06a8d0 0x7f246c19e3f0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f245c005e30 tx=0x7f245c034780 comp rx=0 tx=0).stop 2024-04-21T08:54:59.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.326+0000 7f24709ef640 1 -- 172.21.15.82:0/549897278 shutdown_connections 2024-04-21T08:54:59.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.326+0000 7f24709ef640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f246c1985a0 0x7f246c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.326+0000 7f24709ef640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f246c107260 0x7f246c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.326+0000 7f24709ef640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f24540a3d20 0x7f24540a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.326+0000 7f24709ef640 1 --2- 172.21.15.82:0/549897278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f246c06a8d0 0x7f246c19e3f0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.326+0000 7f24709ef640 1 -- 172.21.15.82:0/549897278 >> 172.21.15.82:0/549897278 conn(0x7f246c077c20 msgr2=0x7f246c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.326+0000 7f24709ef640 1 -- 172.21.15.82:0/549897278 shutdown_connections 2024-04-21T08:54:59.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.326+0000 7f24709ef640 1 -- 172.21.15.82:0/549897278 wait complete. 2024-04-21T08:54:59.338 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:54:59.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.438+0000 7fb39cec1640 1 -- 172.21.15.82:0/3480906917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb398101090 msgr2=0x7fb398101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.438+0000 7fb39cec1640 1 --2- 172.21.15.82:0/3480906917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb398101090 0x7fb398101470 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fb390009ca0 tx=0x7fb39002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.438+0000 7fb39cec1640 1 -- 172.21.15.82:0/3480906917 shutdown_connections 2024-04-21T08:54:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.438+0000 7fb39cec1640 1 --2- 172.21.15.82:0/3480906917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb398106160 0x7fb39810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.438+0000 7fb39cec1640 1 --2- 172.21.15.82:0/3480906917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb398101a40 0x7fb398105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.438+0000 7fb39cec1640 1 --2- 172.21.15.82:0/3480906917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb398101090 0x7fb398101470 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.438+0000 7fb39cec1640 1 -- 172.21.15.82:0/3480906917 >> 172.21.15.82:0/3480906917 conn(0x7fb398077f50 msgr2=0x7fb398078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.438+0000 7fb39cec1640 1 -- 172.21.15.82:0/3480906917 shutdown_connections 2024-04-21T08:54:59.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.438+0000 7fb39cec1640 1 -- 172.21.15.82:0/3480906917 wait complete. 2024-04-21T08:54:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb39cec1640 1 Processor -- start 2024-04-21T08:54:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb39cec1640 1 -- start start 2024-04-21T08:54:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb39cec1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb398101090 0x7fb39819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb39cec1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb398106160 0x7fb39819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb39cec1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb398198610 0x7fb398198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb39cec1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981110c0 con 0x7fb398198610 2024-04-21T08:54:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb39cec1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb398110f40 con 0x7fb398101090 2024-04-21T08:54:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb39cec1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb398111240 con 0x7fb398106160 2024-04-21T08:54:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb396d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb398198610 0x7fb398198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb396d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb398198610 0x7fb398198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43034/0 (socket says 172.21.15.82:43034) 2024-04-21T08:54:59.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb396d9d640 1 -- 172.21.15.82:0/3185665126 learned_addr learned my addr 172.21.15.82:0/3185665126 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb395d9b640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb398106160 0x7fb39819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb39659c640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb398101090 0x7fb39819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb396d9d640 1 -- 172.21.15.82:0/3185665126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb398106160 msgr2=0x7fb39819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb396d9d640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb398106160 0x7fb39819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb396d9d640 1 -- 172.21.15.82:0/3185665126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb398101090 msgr2=0x7fb39819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb396d9d640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb398101090 0x7fb39819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.439+0000 7fb396d9d640 1 -- 172.21.15.82:0/3185665126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb398199130 con 0x7fb398198610 2024-04-21T08:54:59.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.440+0000 7fb396d9d640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb398198610 0x7fb398198ac0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d950 tx=0x7fb38800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.440+0000 7fb38f7fe640 1 -- 172.21.15.82:0/3185665126 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388017030 con 0x7fb398198610 2024-04-21T08:54:59.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.440+0000 7fb38f7fe640 1 -- 172.21.15.82:0/3185665126 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb388004020 con 0x7fb398198610 2024-04-21T08:54:59.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.440+0000 7fb39cec1640 1 -- 172.21.15.82:0/3185665126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb398199420 con 0x7fb398198610 2024-04-21T08:54:59.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.440+0000 7fb38f7fe640 1 -- 172.21.15.82:0/3185665126 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388003b50 con 0x7fb398198610 2024-04-21T08:54:59.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.440+0000 7fb39cec1640 1 -- 172.21.15.82:0/3185665126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3981a55d0 con 0x7fb398198610 2024-04-21T08:54:59.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.442+0000 7fb38f7fe640 1 -- 172.21.15.82:0/3185665126 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb38801a070 con 0x7fb398198610 2024-04-21T08:54:59.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.442+0000 7fb39fb91740 1 Processor -- start 2024-04-21T08:54:59.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39fb91740 1 -- start start 2024-04-21T08:54:59.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39fb91740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea6a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39fb91740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea73430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39fb91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea75e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39fb91740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556f3e9cf1f0 con 0x556f3ea6ae80 2024-04-21T08:54:59.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39fb91740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556f3e96c4b0 con 0x556f3ea6a4d0 2024-04-21T08:54:59.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39fb91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556f3e9cb1b0 con 0x556f3ea73a00 2024-04-21T08:54:59.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea73430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea73430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43036/0 (socket says 172.21.15.82:43036) 2024-04-21T08:54:59.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39659c640 1 -- 172.21.15.82:0/1676787137 learned_addr learned my addr 172.21.15.82:0/1676787137 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb38f7fe640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb3840a3d20 0x7fb3840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb395d9b640 1 --2- 172.21.15.82:0/1676787137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea6a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb395d9b640 1 --2- 172.21.15.82:0/1676787137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea75e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb396d9d640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb3840a3d20 0x7fb3840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39659c640 1 -- 172.21.15.82:0/1676787137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 msgr2=0x556f3ea75e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39659c640 1 --2- 172.21.15.82:0/1676787137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea75e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39659c640 1 -- 172.21.15.82:0/1676787137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 msgr2=0x556f3ea6a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39659c640 1 --2- 172.21.15.82:0/1676787137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea6a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39659c640 1 -- 172.21.15.82:0/1676787137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f3e95e1b0 con 0x556f3ea6ae80 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb395d9b640 1 --2- 172.21.15.82:0/1676787137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea75e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb38f7fe640 1 -- 172.21.15.82:0/3185665126 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fb3880d0370 con 0x7fb398198610 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.443+0000 7fb39659c640 1 --2- 172.21.15.82:0/1676787137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea73430 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb390037040 tx=0x7fb390005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a68775ef7497162 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.444+0000 7fb396d9d640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb3840a3d20 0x7fb3840a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb388098360 tx=0x7fb388097a80 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.444+0000 7fb38cff9640 1 -- 172.21.15.82:0/1676787137 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390039050 con 0x556f3ea6ae80 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.444+0000 7fb38cff9640 1 -- 172.21.15.82:0/1676787137 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb390045070 con 0x556f3ea6ae80 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.444+0000 7fb38cff9640 1 -- 172.21.15.82:0/1676787137 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390004040 con 0x556f3ea6ae80 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.444+0000 7fb39fb91740 1 -- 172.21.15.82:0/1676787137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 msgr2=0x556f3ea73430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.444+0000 7fb39fb91740 1 --2- 172.21.15.82:0/1676787137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea73430 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb390037040 tx=0x7fb390005ef0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.444+0000 7fb39fb91740 1 -- 172.21.15.82:0/1676787137 shutdown_connections 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.444+0000 7fb39fb91740 1 --2- 172.21.15.82:0/1676787137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea75e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.444+0000 7fb39fb91740 1 --2- 172.21.15.82:0/1676787137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea73430 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.444+0000 7fb39fb91740 1 --2- 172.21.15.82:0/1676787137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea6a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.444+0000 7fb39fb91740 1 -- 172.21.15.82:0/1676787137 >> 172.21.15.82:0/1676787137 conn(0x556f3ea64ba0 msgr2=0x556f3ea64f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb39fb91740 1 -- 172.21.15.82:0/1676787137 shutdown_connections 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb39fb91740 1 -- 172.21.15.82:0/1676787137 wait complete. 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb39fb91740 1 -- start start 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb39fb91740 1 Processor -- start 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb39fb91740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6a4d0 0x556f3ea73320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb39fb91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea6ae80 0x556f3ea706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb39fb91740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea73a00 0x556f3ea70c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb39fb91740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556f3ea76910 con 0x556f3ea6a4d0 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb39fb91740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556f3e988500 con 0x556f3ea73a00 2024-04-21T08:54:59.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb39fb91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556f3ea76a90 con 0x556f3ea6ae80 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb39fb91740 10 client.0 _ll_register_callbacks cb 0x556f3e956b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb395d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6a4d0 0x556f3ea73320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb395d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6a4d0 0x556f3ea73320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43052/0 (socket says 172.21.15.82:43052) 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb395d9b640 1 -- 172.21.15.82:0/846455760 learned_addr learned my addr 172.21.15.82:0/846455760 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb395d9b640 1 -- 172.21.15.82:0/846455760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea6ae80 msgr2=0x556f3ea706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb395d9b640 1 --2- 172.21.15.82:0/846455760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea6ae80 0x556f3ea706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb395d9b640 1 -- 172.21.15.82:0/846455760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea73a00 msgr2=0x556f3ea70c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb395d9b640 1 --2- 172.21.15.82:0/846455760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea73a00 0x556f3ea70c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.445+0000 7fb395d9b640 1 -- 172.21.15.82:0/846455760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3e6e13f0 con 0x556f3ea6a4d0 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb395d9b640 1 --2- 172.21.15.82:0/846455760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6a4d0 0x556f3ea73320 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb38000b620 tx=0x7fb380012b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178a65109fb90211 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb37dffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb37dffb640 1 -- 172.21.15.82:0/846455760 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380004230 con 0x556f3ea6a4d0 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb39fb91740 1 -- 172.21.15.82:0/846455760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb390045070 con 0x556f3ea6a4d0 2024-04-21T08:54:59.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb37dffb640 1 -- 172.21.15.82:0/846455760 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fb3800043d0 con 0x556f3ea6a4d0 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb37dffb640 1 -- 172.21.15.82:0/846455760 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb3800119a0 con 0x556f3ea6a4d0 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb39fb91740 10 client.17877 fetch_fsmap learned FSMap version 42 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb39fb91740 1 -- 172.21.15.82:0/846455760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556f3e7ab7d0 con 0x556f3ea6a4d0 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb37dffb640 1 -- 172.21.15.82:0/846455760 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb390045070 con 0x556f3ea6a4d0 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb37dffb640 1 -- 172.21.15.82:0/846455760 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7fb38001d430 con 0x556f3ea6a4d0 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb39fb91740 10 client.17877 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb39fb91740 10 client.17877 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb39fb91740 20 client.17877 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb39fb91740 1 --2- 172.21.15.82:0/846455760 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556f3ea83c10 0x556f3ea86060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb39fb91740 4 client.17877 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.446+0000 7fb39fb91740 1 -- 172.21.15.82:0/846455760 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f3e96e930 con 0x556f3ea83c10 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.447+0000 7fb39659c640 1 --2- 172.21.15.82:0/846455760 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556f3ea83c10 0x556f3ea86060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.447+0000 7fb37dffb640 10 client.17877 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.447+0000 7fb39659c640 1 --2- 172.21.15.82:0/846455760 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556f3ea83c10 0x556f3ea86060 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b1527c91f7997b48 server_cookie=467381a1f8baadb1 in_seq=0 out_seq=0 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb37dffb640 1 -- 172.21.15.82:0/846455760 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f3e96e930 con 0x556f3ea83c10 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb37dffb640 10 client.17877 handle_command_reply: tid=0 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb39fb91740 1 client.17877 shutdown 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb39fb91740 1 -- 172.21.15.82:0/846455760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6a4d0 msgr2=0x556f3ea73320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb39fb91740 1 --2- 172.21.15.82:0/846455760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6a4d0 0x556f3ea73320 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb38000b620 tx=0x7fb380012b10 comp rx=0 tx=0).stop 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb39fb91740 1 -- 172.21.15.82:0/846455760 shutdown_connections 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb39fb91740 1 --2- 172.21.15.82:0/846455760 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556f3ea83c10 0x556f3ea86060 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb39fb91740 1 --2- 172.21.15.82:0/846455760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea73a00 0x556f3ea70c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb39fb91740 1 --2- 172.21.15.82:0/846455760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea6ae80 0x556f3ea706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb39fb91740 1 --2- 172.21.15.82:0/846455760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6a4d0 0x556f3ea73320 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb39fb91740 1 -- 172.21.15.82:0/846455760 >> 172.21.15.82:0/846455760 conn(0x556f3ea64ba0 msgr2=0x556f3ea668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb37dffb640 0 client.17877 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb39fb91740 1 -- 172.21.15.82:0/846455760 shutdown_connections 2024-04-21T08:54:59.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb39fb91740 1 -- 172.21.15.82:0/846455760 wait complete. 2024-04-21T08:54:59.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.448+0000 7fb39fb91740 20 client.17877 trim_cache size 0 max 16384 2024-04-21T08:54:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.461+0000 7fb39fb91740 1 Processor -- start 2024-04-21T08:54:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.461+0000 7fb39fb91740 1 -- start start 2024-04-21T08:54:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.461+0000 7fb39fb91740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea7f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.461+0000 7fb39fb91740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea5fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.461+0000 7fb39fb91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea66700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.461+0000 7fb39fb91740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981af5e0 con 0x556f3ea6ae80 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.461+0000 7fb39fb91740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981b1210 con 0x556f3ea6a4d0 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.461+0000 7fb39fb91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981afcb0 con 0x556f3ea73a00 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.461+0000 7fb39659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea5fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb39659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea5fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43056/0 (socket says 172.21.15.82:43056) 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb39659c640 1 -- 172.21.15.82:0/3704430320 learned_addr learned my addr 172.21.15.82:0/3704430320 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb395d9b640 1 --2- 172.21.15.82:0/3704430320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea66700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb395d9b640 1 --2- 172.21.15.82:0/3704430320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea7f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb39659c640 1 -- 172.21.15.82:0/3704430320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 msgr2=0x556f3ea66700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb39659c640 1 --2- 172.21.15.82:0/3704430320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea66700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb39659c640 1 -- 172.21.15.82:0/3704430320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 msgr2=0x556f3ea7f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb39659c640 1 --2- 172.21.15.82:0/3704430320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea7f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb39659c640 1 -- 172.21.15.82:0/3704430320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f3ea6ac20 con 0x556f3ea6ae80 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb395d9b640 1 --2- 172.21.15.82:0/3704430320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea7f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb39659c640 1 --2- 172.21.15.82:0/3704430320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea5fa60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb390009bf0 tx=0x7fb390009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c799ef457a824c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb37ffff640 1 -- 172.21.15.82:0/3704430320 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390039040 con 0x556f3ea6ae80 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb37ffff640 1 -- 172.21.15.82:0/3704430320 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb390045040 con 0x556f3ea6ae80 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.462+0000 7fb37ffff640 1 -- 172.21.15.82:0/3704430320 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3900406b0 con 0x556f3ea6ae80 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 -- 172.21.15.82:0/3704430320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 msgr2=0x556f3ea5fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 --2- 172.21.15.82:0/3704430320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea5fa60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb390009bf0 tx=0x7fb390009580 comp rx=0 tx=0).stop 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 -- 172.21.15.82:0/3704430320 shutdown_connections 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 --2- 172.21.15.82:0/3704430320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea66700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 --2- 172.21.15.82:0/3704430320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea5fa60 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 --2- 172.21.15.82:0/3704430320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea7f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 -- 172.21.15.82:0/3704430320 >> 172.21.15.82:0/3704430320 conn(0x556f3ea64ba0 msgr2=0x556f3ea65ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 -- 172.21.15.82:0/3704430320 shutdown_connections 2024-04-21T08:54:59.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 -- 172.21.15.82:0/3704430320 wait complete. 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 -- start start 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 Processor -- start 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea7bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea6cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea6d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981b0b10 con 0x556f3ea6ae80 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981b0410 con 0x556f3ea6a4d0 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981ad5b0 con 0x556f3ea73a00 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea6cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39fb91740 10 client.0 _ll_register_callbacks cb 0x7fb38000bee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea6cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43066/0 (socket says 172.21.15.82:43066) 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb39659c640 1 -- 172.21.15.82:0/340526937 learned_addr learned my addr 172.21.15.82:0/340526937 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.463+0000 7fb395d9b640 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea7bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb395d9b640 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea6d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39659c640 1 -- 172.21.15.82:0/340526937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 msgr2=0x556f3ea6d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39659c640 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea6d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39659c640 1 -- 172.21.15.82:0/340526937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 msgr2=0x556f3ea7bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39659c640 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea7bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39659c640 1 -- 172.21.15.82:0/340526937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3ea6da70 con 0x556f3ea6ae80 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb395d9b640 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea6d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39659c640 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea6cf40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb390006010 tx=0x7fb390000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba5ec16c5c71632 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb37dffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb37dffb640 1 -- 172.21.15.82:0/340526937 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39005b030 con 0x556f3ea6ae80 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb37dffb640 1 -- 172.21.15.82:0/340526937 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fb3900324c0 con 0x556f3ea6ae80 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39fb91740 1 -- 172.21.15.82:0/340526937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb390045040 con 0x556f3ea6ae80 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb37dffb640 1 -- 172.21.15.82:0/340526937 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb390065050 con 0x556f3ea6ae80 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39fb91740 10 client.17883 fetch_fsmap learned FSMap version 42 2024-04-21T08:54:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39fb91740 1 -- 172.21.15.82:0/340526937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556f3eae9f00 con 0x556f3ea6ae80 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb37dffb640 1 -- 172.21.15.82:0/340526937 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb390045040 con 0x556f3ea6ae80 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb37dffb640 1 -- 172.21.15.82:0/340526937 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7fb390040a50 con 0x556f3ea6ae80 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39fb91740 10 client.17883 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39fb91740 10 client.17883 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39fb91740 20 client.17883 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39fb91740 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556f3eaea1c0 0x556f3eaec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39fb91740 4 client.17883 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.464+0000 7fb39fb91740 1 -- 172.21.15.82:0/340526937 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f3e9dcc10 con 0x556f3eaea1c0 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb395d9b640 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556f3eaea1c0 0x556f3eaec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb395d9b640 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556f3eaea1c0 0x556f3eaec5b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6127aad961e619a0 server_cookie=f47acd7e02aaad55 in_seq=0 out_seq=0 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb37dffb640 10 client.17883 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb37dffb640 1 -- 172.21.15.82:0/340526937 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f3e9dcc10 con 0x556f3eaea1c0 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb37dffb640 10 client.17883 handle_command_reply: tid=0 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb39fb91740 1 client.17883 shutdown 2024-04-21T08:54:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb39fb91740 1 -- 172.21.15.82:0/340526937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 msgr2=0x556f3ea6cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb39fb91740 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea6cf40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb390006010 tx=0x7fb390000f40 comp rx=0 tx=0).stop 2024-04-21T08:54:59.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb39fb91740 1 -- 172.21.15.82:0/340526937 shutdown_connections 2024-04-21T08:54:59.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb39fb91740 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x556f3eaea1c0 0x556f3eaec5b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb39fb91740 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f3ea73a00 0x556f3ea6d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb39fb91740 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556f3ea6ae80 0x556f3ea6cf40 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb39fb91740 1 --2- 172.21.15.82:0/340526937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f3ea6a4d0 0x556f3ea7bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb37dffb640 0 client.17883 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:54:59.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb39fb91740 1 -- 172.21.15.82:0/340526937 >> 172.21.15.82:0/340526937 conn(0x556f3ea64ba0 msgr2=0x556f3ea63c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb39fb91740 1 -- 172.21.15.82:0/340526937 shutdown_connections 2024-04-21T08:54:59.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb39fb91740 1 -- 172.21.15.82:0/340526937 wait complete. 2024-04-21T08:54:59.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.465+0000 7fb39fb91740 20 client.17883 trim_cache size 0 max 16384 2024-04-21T08:54:59.473 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:54:59.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.466+0000 7fb39cec1640 1 -- 172.21.15.82:0/3185665126 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb3840a3d20 msgr2=0x7fb3840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.466+0000 7fb39cec1640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb3840a3d20 0x7fb3840a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb388098360 tx=0x7fb388097a80 comp rx=0 tx=0).stop 2024-04-21T08:54:59.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.466+0000 7fb39cec1640 1 -- 172.21.15.82:0/3185665126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb398198610 msgr2=0x7fb398198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.466+0000 7fb39cec1640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb398198610 0x7fb398198ac0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d950 tx=0x7fb38800de10 comp rx=0 tx=0).stop 2024-04-21T08:54:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.466+0000 7fb39cec1640 1 -- 172.21.15.82:0/3185665126 shutdown_connections 2024-04-21T08:54:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.466+0000 7fb39cec1640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb3840a3d20 0x7fb3840a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.466+0000 7fb39cec1640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb398198610 0x7fb398198ac0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.466+0000 7fb39cec1640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb398106160 0x7fb39819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.466+0000 7fb39cec1640 1 --2- 172.21.15.82:0/3185665126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb398101090 0x7fb39819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.466+0000 7fb39cec1640 1 -- 172.21.15.82:0/3185665126 >> 172.21.15.82:0/3185665126 conn(0x7fb398077f50 msgr2=0x7fb398104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.466+0000 7fb39cec1640 1 -- 172.21.15.82:0/3185665126 shutdown_connections 2024-04-21T08:54:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.466+0000 7fb39cec1640 1 -- 172.21.15.82:0/3185665126 wait complete. 2024-04-21T08:54:59.476 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:54:59.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.576+0000 7fd11d1f4640 1 -- 172.21.15.82:0/1070454517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1181072e0 msgr2=0x7fd11810eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.576+0000 7fd11d1f4640 1 --2- 172.21.15.82:0/1070454517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1181072e0 0x7fd11810eeb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fd110009900 tx=0x7fd11002bc30 comp rx=0 tx=0).stop 2024-04-21T08:54:59.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.576+0000 7fd11d1f4640 1 -- 172.21.15.82:0/1070454517 shutdown_connections 2024-04-21T08:54:59.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.576+0000 7fd11d1f4640 1 --2- 172.21.15.82:0/1070454517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1181072e0 0x7fd11810eeb0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.576+0000 7fd11d1f4640 1 --2- 172.21.15.82:0/1070454517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd118102bc0 0x7fd118106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.576+0000 7fd11d1f4640 1 --2- 172.21.15.82:0/1070454517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd118102210 0x7fd1181025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.576+0000 7fd11d1f4640 1 -- 172.21.15.82:0/1070454517 >> 172.21.15.82:0/1070454517 conn(0x7fd1180fdab0 msgr2=0x7fd1180ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.576+0000 7fd11d1f4640 1 -- 172.21.15.82:0/1070454517 shutdown_connections 2024-04-21T08:54:59.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.576+0000 7fd11d1f4640 1 -- 172.21.15.82:0/1070454517 wait complete. 2024-04-21T08:54:59.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.577+0000 7fd11d1f4640 1 Processor -- start 2024-04-21T08:54:59.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.577+0000 7fd11d1f4640 1 -- start start 2024-04-21T08:54:59.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.577+0000 7fd11d1f4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd118102210 0x7fd11819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.577+0000 7fd11d1f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1181072e0 0x7fd11819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.577+0000 7fd11d1f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd118198630 0x7fd118198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.577+0000 7fd11d1f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd118112250 con 0x7fd1181072e0 2024-04-21T08:54:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.577+0000 7fd11d1f4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181120d0 con 0x7fd118102210 2024-04-21T08:54:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.577+0000 7fd11d1f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181123d0 con 0x7fd118198630 2024-04-21T08:54:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd11659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1181072e0 0x7fd11819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd11659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1181072e0 0x7fd11819e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43096/0 (socket says 172.21.15.82:43096) 2024-04-21T08:54:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd11659c640 1 -- 172.21.15.82:0/727409868 learned_addr learned my addr 172.21.15.82:0/727409868 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd116d9d640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd118102210 0x7fd11819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd11759e640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd118198630 0x7fd118198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd11659c640 1 -- 172.21.15.82:0/727409868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd118198630 msgr2=0x7fd118198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd11659c640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd118198630 0x7fd118198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd11659c640 1 -- 172.21.15.82:0/727409868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd118102210 msgr2=0x7fd11819e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd11659c640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd118102210 0x7fd11819e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd11659c640 1 -- 172.21.15.82:0/727409868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd118199150 con 0x7fd1181072e0 2024-04-21T08:54:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd11659c640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1181072e0 0x7fd11819e970 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00d770 tx=0x7fd10c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd0fffff640 1 -- 172.21.15.82:0/727409868 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c017070 con 0x7fd1181072e0 2024-04-21T08:54:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd11d1f4640 1 -- 172.21.15.82:0/727409868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd118199440 con 0x7fd1181072e0 2024-04-21T08:54:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd0fffff640 1 -- 172.21.15.82:0/727409868 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd10c009c30 con 0x7fd1181072e0 2024-04-21T08:54:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd0fffff640 1 -- 172.21.15.82:0/727409868 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c00f620 con 0x7fd1181072e0 2024-04-21T08:54:59.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.578+0000 7fd11d1f4640 1 -- 172.21.15.82:0/727409868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1181a5690 con 0x7fd1181072e0 2024-04-21T08:54:59.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.581+0000 7fd0fffff640 1 -- 172.21.15.82:0/727409868 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd10c01a070 con 0x7fd1181072e0 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.581+0000 7fd11fde4740 1 Processor -- start 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.581+0000 7fd11fde4740 1 -- start start 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.581+0000 7fd11fde4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e030474d0 0x555e030478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.581+0000 7fd11fde4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e03047e80 0x555e03050430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.581+0000 7fd11fde4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 0x555e03052e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.581+0000 7fd11fde4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555e02fac1f0 con 0x555e030474d0 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.581+0000 7fd11fde4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555e02f494b0 con 0x555e03047e80 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.581+0000 7fd11fde4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e02fa81b0 con 0x555e03050a00 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.581+0000 7fd0fffff640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd1000a3d20 0x7fd1000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.581+0000 7fd11759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e030474d0 0x555e030478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd11759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e030474d0 0x555e030478b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43100/0 (socket says 172.21.15.82:43100) 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd11759e640 1 -- 172.21.15.82:0/4034645157 learned_addr learned my addr 172.21.15.82:0/4034645157 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd116d9d640 1 --2- 172.21.15.82:0/4034645157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e03047e80 0x555e03050430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd11759e640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd1000a3d20 0x7fd1000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd0fffff640 1 -- 172.21.15.82:0/727409868 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fd10c0d0640 con 0x7fd1181072e0 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd11659c640 1 --2- 172.21.15.82:0/4034645157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 0x555e03052e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd11759e640 1 -- 172.21.15.82:0/4034645157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 msgr2=0x555e03052e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd11759e640 1 --2- 172.21.15.82:0/4034645157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 0x555e03052e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd11759e640 1 -- 172.21.15.82:0/4034645157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e03047e80 msgr2=0x555e03050430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd11759e640 1 --2- 172.21.15.82:0/4034645157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e03047e80 0x555e03050430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd11759e640 1 -- 172.21.15.82:0/4034645157 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e02f3b1b0 con 0x555e030474d0 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd116d9d640 1 --2- 172.21.15.82:0/4034645157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e03047e80 0x555e03050430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd11759e640 1 --2- 172.21.15.82:0/4034645157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e030474d0 0x555e030478b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd110006010 tx=0x7fd1100043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec4ad57498ea57a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd0fd7fa640 1 -- 172.21.15.82:0/4034645157 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11002cc70 con 0x555e030474d0 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd0fd7fa640 1 -- 172.21.15.82:0/4034645157 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1100054f0 con 0x555e030474d0 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd0fd7fa640 1 -- 172.21.15.82:0/4034645157 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110004dd0 con 0x555e030474d0 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.582+0000 7fd11759e640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd1000a3d20 0x7fd1000a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd11004e000 tx=0x7fd11004a000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 -- 172.21.15.82:0/4034645157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e030474d0 msgr2=0x555e030478b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 --2- 172.21.15.82:0/4034645157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e030474d0 0x555e030478b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd110006010 tx=0x7fd1100043d0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 -- 172.21.15.82:0/4034645157 shutdown_connections 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 --2- 172.21.15.82:0/4034645157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 0x555e03052e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 --2- 172.21.15.82:0/4034645157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e03047e80 0x555e03050430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 --2- 172.21.15.82:0/4034645157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e030474d0 0x555e030478b0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 -- 172.21.15.82:0/4034645157 >> 172.21.15.82:0/4034645157 conn(0x555e03041ba0 msgr2=0x555e03041f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 -- 172.21.15.82:0/4034645157 shutdown_connections 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 -- 172.21.15.82:0/4034645157 wait complete. 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 -- start start 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 Processor -- start 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 0x555e03050350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03047e80 0x555e0304d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03050a00 0x555e0304dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555e03053970 con 0x555e03050a00 2024-04-21T08:54:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555e02f65500 con 0x555e030474d0 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.583+0000 7fd11fde4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e03053af0 con 0x555e03047e80 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11fde4740 10 client.0 _ll_register_callbacks cb 0x555e02f33b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03050a00 0x555e0304dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03050a00 0x555e0304dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43112/0 (socket says 172.21.15.82:43112) 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11759e640 1 -- 172.21.15.82:0/3812200381 learned_addr learned my addr 172.21.15.82:0/3812200381 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11659c640 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03047e80 0x555e0304d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd116d9d640 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 0x555e03050350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11759e640 1 -- 172.21.15.82:0/3812200381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03047e80 msgr2=0x555e0304d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11759e640 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03047e80 0x555e0304d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11759e640 1 -- 172.21.15.82:0/3812200381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 msgr2=0x555e03050350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11759e640 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 0x555e03050350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11759e640 1 -- 172.21.15.82:0/3812200381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e02cbe3f0 con 0x555e03050a00 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11759e640 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03050a00 0x555e0304dcf0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd11004f040 tx=0x7fd1100040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11162c8fc936e4ce server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd0faff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd0faff5640 1 -- 172.21.15.82:0/3812200381 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11005c070 con 0x555e03050a00 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd0faff5640 1 -- 172.21.15.82:0/3812200381 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fd110005000 con 0x555e03050a00 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11fde4740 1 -- 172.21.15.82:0/3812200381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1100054f0 con 0x555e03050a00 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd0faff5640 1 -- 172.21.15.82:0/3812200381 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd110068050 con 0x555e03050a00 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11fde4740 10 client.17895 fetch_fsmap learned FSMap version 42 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.584+0000 7fd11fde4740 1 -- 172.21.15.82:0/3812200381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555e02d887d0 con 0x555e03050a00 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.585+0000 7fd0faff5640 1 -- 172.21.15.82:0/3812200381 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1100054f0 con 0x555e03050a00 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.585+0000 7fd0faff5640 1 -- 172.21.15.82:0/3812200381 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7fd110040040 con 0x555e03050a00 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.585+0000 7fd11fde4740 10 client.17895 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.585+0000 7fd11fde4740 10 client.17895 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.585+0000 7fd11fde4740 20 client.17895 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.585+0000 7fd11fde4740 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x555e03060ae0 0x555e03062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.585+0000 7fd11fde4740 4 client.17895 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.585+0000 7fd11fde4740 1 -- 172.21.15.82:0/3812200381 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e02f4b930 con 0x555e03060ae0 2024-04-21T08:54:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.585+0000 7fd116d9d640 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x555e03060ae0 0x555e03062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.585+0000 7fd0faff5640 10 client.17895 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.585+0000 7fd116d9d640 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x555e03060ae0 0x555e03062ed0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8bb5ef0a7c308e server_cookie=fa70614e664b1f37 in_seq=0 out_seq=0 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd0faff5640 1 -- 172.21.15.82:0/3812200381 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e02f4b930 con 0x555e03060ae0 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd0faff5640 10 client.17895 handle_command_reply: tid=0 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd11fde4740 1 client.17895 shutdown 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd11fde4740 1 -- 172.21.15.82:0/3812200381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03050a00 msgr2=0x555e0304dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd11fde4740 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03050a00 0x555e0304dcf0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd11004f040 tx=0x7fd1100040b0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd11fde4740 1 -- 172.21.15.82:0/3812200381 shutdown_connections 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd11fde4740 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x555e03060ae0 0x555e03062ed0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd11fde4740 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03050a00 0x555e0304dcf0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd0faff5640 0 client.17895 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd11fde4740 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03047e80 0x555e0304d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd11fde4740 1 --2- 172.21.15.82:0/3812200381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 0x555e03050350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd11fde4740 1 -- 172.21.15.82:0/3812200381 >> 172.21.15.82:0/3812200381 conn(0x555e03041ba0 msgr2=0x555e030438f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd11fde4740 1 -- 172.21.15.82:0/3812200381 shutdown_connections 2024-04-21T08:54:59.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd11fde4740 1 -- 172.21.15.82:0/3812200381 wait complete. 2024-04-21T08:54:59.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.586+0000 7fd11fde4740 20 client.17895 trim_cache size 0 max 16384 2024-04-21T08:54:59.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.599+0000 7fd11fde4740 1 Processor -- start 2024-04-21T08:54:59.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.599+0000 7fd11fde4740 1 -- start start 2024-04-21T08:54:59.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11fde4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 0x555e0304fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11fde4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 0x555e03050350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11fde4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 0x555e03065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11fde4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181af530 con 0x555e03047e80 2024-04-21T08:54:59.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11fde4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181b1160 con 0x555e030474d0 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11fde4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181afc00 con 0x555e03050a00 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 0x555e03050350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 0x555e03050350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43124/0 (socket says 172.21.15.82:43124) 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11659c640 1 -- 172.21.15.82:0/3915836781 learned_addr learned my addr 172.21.15.82:0/3915836781 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd116d9d640 1 --2- 172.21.15.82:0/3915836781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 0x555e0304fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11659c640 1 -- 172.21.15.82:0/3915836781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 msgr2=0x555e03065ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11659c640 1 --2- 172.21.15.82:0/3915836781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 0x555e03065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11659c640 1 -- 172.21.15.82:0/3915836781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 msgr2=0x555e0304fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11659c640 1 --2- 172.21.15.82:0/3915836781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 0x555e0304fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11659c640 1 -- 172.21.15.82:0/3915836781 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e030424e0 con 0x555e03047e80 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd11659c640 1 --2- 172.21.15.82:0/3915836781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 0x555e03050350 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009610 tx=0x7fd10c00df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2beaa3e1c3da40c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd0fcff9640 1 -- 172.21.15.82:0/3915836781 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c021040 con 0x555e03047e80 2024-04-21T08:54:59.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd0fcff9640 1 -- 172.21.15.82:0/3915836781 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd10c0a4e40 con 0x555e03047e80 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.600+0000 7fd0fcff9640 1 -- 172.21.15.82:0/3915836781 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c0a0170 con 0x555e03047e80 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 -- 172.21.15.82:0/3915836781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 msgr2=0x555e03050350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 --2- 172.21.15.82:0/3915836781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 0x555e03050350 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009610 tx=0x7fd10c00df10 comp rx=0 tx=0).stop 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 -- 172.21.15.82:0/3915836781 shutdown_connections 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 --2- 172.21.15.82:0/3915836781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 0x555e03065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 --2- 172.21.15.82:0/3915836781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 0x555e03050350 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 --2- 172.21.15.82:0/3915836781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 0x555e0304fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 -- 172.21.15.82:0/3915836781 >> 172.21.15.82:0/3915836781 conn(0x555e03041ba0 msgr2=0x555e0304c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 -- 172.21.15.82:0/3915836781 shutdown_connections 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 -- 172.21.15.82:0/3915836781 wait complete. 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 -- start start 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 Processor -- start 2024-04-21T08:54:59.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 0x555e03064030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 0x555e03064570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 0x555e030c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181b0a60 con 0x555e03047e80 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181b0360 con 0x555e030474d0 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181ad500 con 0x555e03050a00 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11fde4740 10 client.0 _ll_register_callbacks cb 0x7fd110005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 0x555e03064570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 0x555e03064570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43138/0 (socket says 172.21.15.82:43138) 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.601+0000 7fd11659c640 1 -- 172.21.15.82:0/2703699572 learned_addr learned my addr 172.21.15.82:0/2703699572 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd116d9d640 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 0x555e03064030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11759e640 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 0x555e030c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11659c640 1 -- 172.21.15.82:0/2703699572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 msgr2=0x555e030c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11659c640 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 0x555e030c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11659c640 1 -- 172.21.15.82:0/2703699572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 msgr2=0x555e03064030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11659c640 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 0x555e03064030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11659c640 1 -- 172.21.15.82:0/2703699572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e030c6200 con 0x555e03047e80 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11759e640 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 0x555e030c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11659c640 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 0x555e03064570 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd10c09c210 tx=0x7fd10c00de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1fa3e50805f9db server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd0faff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd0faff5640 1 -- 172.21.15.82:0/2703699572 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c021040 con 0x555e03047e80 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd0faff5640 1 -- 172.21.15.82:0/2703699572 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fd10c098960 con 0x555e03047e80 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11fde4740 1 -- 172.21.15.82:0/2703699572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd10c0a4e40 con 0x555e03047e80 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd0faff5640 1 -- 172.21.15.82:0/2703699572 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd10c0cdc00 con 0x555e03047e80 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11fde4740 10 client.17901 fetch_fsmap learned FSMap version 42 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11fde4740 1 -- 172.21.15.82:0/2703699572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555e030c7430 con 0x555e03047e80 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd0faff5640 1 -- 172.21.15.82:0/2703699572 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd10c0a4e40 con 0x555e03047e80 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd0faff5640 1 -- 172.21.15.82:0/2703699572 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7fd10c0d02d0 con 0x555e03047e80 2024-04-21T08:54:59.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11fde4740 10 client.17901 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11fde4740 10 client.17901 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.602+0000 7fd11fde4740 20 client.17901 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd11fde4740 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x555e030c7b20 0x555e030c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd11fde4740 4 client.17901 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd11fde4740 1 -- 172.21.15.82:0/2703699572 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e02fb9c10 con 0x555e030c7b20 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd116d9d640 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x555e030c7b20 0x555e030c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd0faff5640 10 client.17901 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd116d9d640 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x555e030c7b20 0x555e030c9f40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ee003673e7f0dfcc server_cookie=b92572682e6156d0 in_seq=0 out_seq=0 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd0faff5640 1 -- 172.21.15.82:0/2703699572 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e02fb9c10 con 0x555e030c7b20 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd0faff5640 10 client.17901 handle_command_reply: tid=0 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd11fde4740 1 client.17901 shutdown 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd11fde4740 1 -- 172.21.15.82:0/2703699572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 msgr2=0x555e03064570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd11fde4740 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 0x555e03064570 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd10c09c210 tx=0x7fd10c00de80 comp rx=0 tx=0).stop 2024-04-21T08:54:59.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd11fde4740 1 -- 172.21.15.82:0/2703699572 shutdown_connections 2024-04-21T08:54:59.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd11fde4740 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x555e030c7b20 0x555e030c9f40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd11fde4740 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e03050a00 0x555e030c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd11fde4740 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e03047e80 0x555e03064570 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd0faff5640 0 client.17901 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:54:59.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd11fde4740 1 --2- 172.21.15.82:0/2703699572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e030474d0 0x555e03064030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.603+0000 7fd11fde4740 1 -- 172.21.15.82:0/2703699572 >> 172.21.15.82:0/2703699572 conn(0x555e03041ba0 msgr2=0x555e03040370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11fde4740 1 -- 172.21.15.82:0/2703699572 shutdown_connections 2024-04-21T08:54:59.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11fde4740 1 -- 172.21.15.82:0/2703699572 wait complete. 2024-04-21T08:54:59.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11fde4740 20 client.17901 trim_cache size 0 max 16384 2024-04-21T08:54:59.611 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:54:59.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11d1f4640 1 -- 172.21.15.82:0/727409868 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd1000a3d20 msgr2=0x7fd1000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11d1f4640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd1000a3d20 0x7fd1000a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd11004e000 tx=0x7fd11004a000 comp rx=0 tx=0).stop 2024-04-21T08:54:59.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11d1f4640 1 -- 172.21.15.82:0/727409868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1181072e0 msgr2=0x7fd11819e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11d1f4640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1181072e0 0x7fd11819e970 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00d770 tx=0x7fd10c00dc30 comp rx=0 tx=0).stop 2024-04-21T08:54:59.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11d1f4640 1 -- 172.21.15.82:0/727409868 shutdown_connections 2024-04-21T08:54:59.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11d1f4640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd118198630 0x7fd118198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11d1f4640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd1000a3d20 0x7fd1000a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11d1f4640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1181072e0 0x7fd11819e970 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11d1f4640 1 --2- 172.21.15.82:0/727409868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd118102210 0x7fd11819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11d1f4640 1 -- 172.21.15.82:0/727409868 >> 172.21.15.82:0/727409868 conn(0x7fd1180fdab0 msgr2=0x7fd1180ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11d1f4640 1 -- 172.21.15.82:0/727409868 shutdown_connections 2024-04-21T08:54:59.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.604+0000 7fd11d1f4640 1 -- 172.21.15.82:0/727409868 wait complete. 2024-04-21T08:54:59.615 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:54:59.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.715+0000 7ffaa9b92640 1 -- 172.21.15.82:0/4092381857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaa406b170 msgr2=0x7ffaa4106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.715+0000 7ffaa9b92640 1 --2- 172.21.15.82:0/4092381857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaa406b170 0x7ffaa4106ba0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009900 tx=0x7ffa9802bc30 comp rx=0 tx=0).stop 2024-04-21T08:54:59.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.715+0000 7ffaa9b92640 1 -- 172.21.15.82:0/4092381857 shutdown_connections 2024-04-21T08:54:59.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.715+0000 7ffaa9b92640 1 --2- 172.21.15.82:0/4092381857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa4107280 0x7ffaa410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.715+0000 7ffaa9b92640 1 --2- 172.21.15.82:0/4092381857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaa406b170 0x7ffaa4106ba0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.715+0000 7ffaa9b92640 1 --2- 172.21.15.82:0/4092381857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa406a7c0 0x7ffaa406aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.715+0000 7ffaa9b92640 1 -- 172.21.15.82:0/4092381857 >> 172.21.15.82:0/4092381857 conn(0x7ffaa4076c50 msgr2=0x7ffaa4077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.715+0000 7ffaa9b92640 1 -- 172.21.15.82:0/4092381857 shutdown_connections 2024-04-21T08:54:59.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.715+0000 7ffaa9b92640 1 -- 172.21.15.82:0/4092381857 wait complete. 2024-04-21T08:54:59.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa9b92640 1 Processor -- start 2024-04-21T08:54:59.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa9b92640 1 -- start start 2024-04-21T08:54:59.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa9b92640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaa406a7c0 0x7ffaa4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa9b92640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4107280 0x7ffaa4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa9b92640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa419cfe0 0x7ffaa419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa9b92640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41121a0 con 0x7ffaa406a7c0 2024-04-21T08:54:59.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa9b92640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa4112020 con 0x7ffaa4107280 2024-04-21T08:54:59.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa9b92640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa4112320 con 0x7ffaa419cfe0 2024-04-21T08:54:59.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaa406a7c0 0x7ffaa4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaa406a7c0 0x7ffaa4198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43156/0 (socket says 172.21.15.82:43156) 2024-04-21T08:54:59.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa37fe640 1 -- 172.21.15.82:0/2870707673 learned_addr learned my addr 172.21.15.82:0/2870707673 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa37fe640 1 -- 172.21.15.82:0/2870707673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa419cfe0 msgr2=0x7ffaa419d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa3fff640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa419cfe0 0x7ffaa419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa37fe640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa419cfe0 0x7ffaa419d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa37fe640 1 -- 172.21.15.82:0/2870707673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4107280 msgr2=0x7ffaa4198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa2ffd640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4107280 0x7ffaa4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa37fe640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4107280 0x7ffaa4198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa37fe640 1 -- 172.21.15.82:0/2870707673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa419dad0 con 0x7ffaa406a7c0 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.716+0000 7ffaa2ffd640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4107280 0x7ffaa4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.717+0000 7ffaa37fe640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaa406a7c0 0x7ffaa4198720 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400d950 tx=0x7ffa9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.717+0000 7ffaa0ff9640 1 -- 172.21.15.82:0/2870707673 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94017030 con 0x7ffaa406a7c0 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.717+0000 7ffaa0ff9640 1 -- 172.21.15.82:0/2870707673 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa94004020 con 0x7ffaa406a7c0 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.717+0000 7ffaa9b92640 1 -- 172.21.15.82:0/2870707673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa419ddc0 con 0x7ffaa406a7c0 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.717+0000 7ffaa0ff9640 1 -- 172.21.15.82:0/2870707673 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa940047a0 con 0x7ffaa406a7c0 2024-04-21T08:54:59.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.717+0000 7ffaa9b92640 1 -- 172.21.15.82:0/2870707673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa41a55b0 con 0x7ffaa406a7c0 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaa0ff9640 1 -- 172.21.15.82:0/2870707673 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa94009a30 con 0x7ffaa406a7c0 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaac778740 1 Processor -- start 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaac778740 1 -- start start 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaac778740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed434d0 0x55a72ed438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaac778740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed43e80 0x55a72ed4c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaac778740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed4ca00 0x55a72ed4ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaac778740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a72eca81f0 con 0x55a72ed434d0 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaac778740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a72ec454b0 con 0x55a72ed4ca00 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaac778740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a72eca41b0 con 0x55a72ed43e80 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaa3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed434d0 0x55a72ed438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaa3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed434d0 0x55a72ed438b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43158/0 (socket says 172.21.15.82:43158) 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaa3fff640 1 -- 172.21.15.82:0/2459010794 learned_addr learned my addr 172.21.15.82:0/2459010794 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.719+0000 7ffaa0ff9640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa8c0a3d20 0x7ffa8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffaa2ffd640 1 --2- 172.21.15.82:0/2459010794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed4ca00 0x55a72ed4ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffaa37fe640 1 --2- 172.21.15.82:0/2459010794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed43e80 0x55a72ed4c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffaa3fff640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa8c0a3d20 0x7ffa8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffaa3fff640 1 -- 172.21.15.82:0/2459010794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed43e80 msgr2=0x55a72ed4c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffaa3fff640 1 --2- 172.21.15.82:0/2459010794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed43e80 0x55a72ed4c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffaa0ff9640 1 -- 172.21.15.82:0/2870707673 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ffa940d07c0 con 0x7ffaa406a7c0 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffaa3fff640 1 -- 172.21.15.82:0/2459010794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed4ca00 msgr2=0x55a72ed4ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffaa3fff640 1 --2- 172.21.15.82:0/2459010794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed4ca00 0x55a72ed4ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffaa3fff640 1 -- 172.21.15.82:0/2459010794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a72ec371b0 con 0x55a72ed434d0 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffaa3fff640 1 --2- 172.21.15.82:0/2459010794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed434d0 0x55a72ed438b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0148c0 tx=0x7ffa9c008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a7c24410e9ba27 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffa89ffb640 1 -- 172.21.15.82:0/2459010794 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c021070 con 0x55a72ed434d0 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffa89ffb640 1 -- 172.21.15.82:0/2459010794 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa9c008db0 con 0x55a72ed434d0 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.720+0000 7ffa89ffb640 1 -- 172.21.15.82:0/2459010794 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c00b370 con 0x55a72ed434d0 2024-04-21T08:54:59.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.721+0000 7ffaac778740 1 -- 172.21.15.82:0/2459010794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed434d0 msgr2=0x55a72ed438b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.721+0000 7ffaa3fff640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa8c0a3d20 0x7ffa8c0a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c01e000 tx=0x7ffa9c01a000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.721+0000 7ffaac778740 1 --2- 172.21.15.82:0/2459010794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed434d0 0x55a72ed438b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0148c0 tx=0x7ffa9c008f90 comp rx=0 tx=0).stop 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.721+0000 7ffaac778740 1 -- 172.21.15.82:0/2459010794 shutdown_connections 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.721+0000 7ffaac778740 1 --2- 172.21.15.82:0/2459010794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed4ca00 0x55a72ed4ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.721+0000 7ffaac778740 1 --2- 172.21.15.82:0/2459010794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed43e80 0x55a72ed4c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.721+0000 7ffaac778740 1 --2- 172.21.15.82:0/2459010794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed434d0 0x55a72ed438b0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.721+0000 7ffaac778740 1 -- 172.21.15.82:0/2459010794 >> 172.21.15.82:0/2459010794 conn(0x55a72ed3dba0 msgr2=0x55a72ed3df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.721+0000 7ffaac778740 1 -- 172.21.15.82:0/2459010794 shutdown_connections 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.721+0000 7ffaac778740 1 -- 172.21.15.82:0/2459010794 wait complete. 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.721+0000 7ffaac778740 1 -- start start 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.721+0000 7ffaac778740 1 Processor -- start 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaac778740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 0x55a72ed4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaac778740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed49650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaac778740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 0x55a72ed49bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaac778740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a72ed4f970 con 0x55a72ed43e80 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaac778740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a72ec61500 con 0x55a72ed434d0 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaac778740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a72ed4faf0 con 0x55a72ed4ca00 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaac778740 10 client.0 _ll_register_callbacks cb 0x55a72ec2fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed49650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed49650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43166/0 (socket says 172.21.15.82:43166) 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaa2ffd640 1 -- 172.21.15.82:0/1509595047 learned_addr learned my addr 172.21.15.82:0/1509595047 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaa37fe640 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 0x55a72ed4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaa3fff640 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 0x55a72ed49bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaa2ffd640 1 -- 172.21.15.82:0/1509595047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 msgr2=0x55a72ed49bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaa2ffd640 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 0x55a72ed49bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaa2ffd640 1 -- 172.21.15.82:0/1509595047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 msgr2=0x55a72ed4c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaa2ffd640 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 0x55a72ed4c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaa2ffd640 1 -- 172.21.15.82:0/1509595047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a72e9ba3f0 con 0x55a72ed43e80 2024-04-21T08:54:59.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaa2ffd640 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed49650 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffaa41033d0 tx=0x7ffa9802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb622f3c0890bc74 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffa82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffa82ffd640 1 -- 172.21.15.82:0/1509595047 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98005030 con 0x55a72ed43e80 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffa82ffd640 1 -- 172.21.15.82:0/1509595047 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ffa980051d0 con 0x55a72ed43e80 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.722+0000 7ffaac778740 1 -- 172.21.15.82:0/1509595047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa9c008db0 con 0x55a72ed43e80 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.723+0000 7ffa82ffd640 1 -- 172.21.15.82:0/1509595047 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ffa98050050 con 0x55a72ed43e80 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.723+0000 7ffaac778740 10 client.17913 fetch_fsmap learned FSMap version 42 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.723+0000 7ffaac778740 1 -- 172.21.15.82:0/1509595047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a72ea847d0 con 0x55a72ed43e80 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.723+0000 7ffa82ffd640 1 -- 172.21.15.82:0/1509595047 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa9c008db0 con 0x55a72ed43e80 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.723+0000 7ffa82ffd640 1 -- 172.21.15.82:0/1509595047 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7ffa980035f0 con 0x55a72ed43e80 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.723+0000 7ffaac778740 10 client.17913 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.723+0000 7ffaac778740 10 client.17913 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.723+0000 7ffaac778740 20 client.17913 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.723+0000 7ffaac778740 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a72ed5cc70 0x55a72ed5f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.723+0000 7ffaac778740 4 client.17913 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:59.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.723+0000 7ffaac778740 1 -- 172.21.15.82:0/1509595047 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a72ec47930 con 0x55a72ed5cc70 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.723+0000 7ffaa37fe640 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a72ed5cc70 0x55a72ed5f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.724+0000 7ffa82ffd640 10 client.17913 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.724+0000 7ffaa37fe640 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a72ed5cc70 0x55a72ed5f0f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d27486ec76bd6e3a server_cookie=dc15a597b202182e in_seq=0 out_seq=0 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.724+0000 7ffa82ffd640 1 -- 172.21.15.82:0/1509595047 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a72ec47930 con 0x55a72ed5cc70 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.724+0000 7ffa82ffd640 10 client.17913 handle_command_reply: tid=0 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.724+0000 7ffaac778740 1 client.17913 shutdown 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.725+0000 7ffaac778740 1 -- 172.21.15.82:0/1509595047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 msgr2=0x55a72ed49650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.725+0000 7ffaac778740 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed49650 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffaa41033d0 tx=0x7ffa9802ca10 comp rx=0 tx=0).stop 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.725+0000 7ffaac778740 1 -- 172.21.15.82:0/1509595047 shutdown_connections 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.725+0000 7ffaac778740 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a72ed5cc70 0x55a72ed5f0f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.725+0000 7ffaac778740 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 0x55a72ed49bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.725+0000 7ffaac778740 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed49650 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.725+0000 7ffa82ffd640 0 client.17913 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:54:59.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.725+0000 7ffaac778740 1 --2- 172.21.15.82:0/1509595047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 0x55a72ed4c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.725+0000 7ffaac778740 1 -- 172.21.15.82:0/1509595047 >> 172.21.15.82:0/1509595047 conn(0x55a72ed3dba0 msgr2=0x55a72ed3f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.725+0000 7ffaac778740 1 -- 172.21.15.82:0/1509595047 shutdown_connections 2024-04-21T08:54:59.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.725+0000 7ffaac778740 1 -- 172.21.15.82:0/1509595047 wait complete. 2024-04-21T08:54:59.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.725+0000 7ffaac778740 20 client.17913 trim_cache size 0 max 16384 2024-04-21T08:54:59.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaac778740 1 Processor -- start 2024-04-21T08:54:59.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaac778740 1 -- start start 2024-04-21T08:54:59.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaac778740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 0x55a72ed58830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaac778740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed4bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaac778740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 0x55a72ed4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaac778740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41af5c0 con 0x55a72ed43e80 2024-04-21T08:54:59.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaac778740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41b11f0 con 0x55a72ed434d0 2024-04-21T08:54:59.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaac778740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41afc90 con 0x55a72ed4ca00 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed4bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed4bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43180/0 (socket says 172.21.15.82:43180) 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaa2ffd640 1 -- 172.21.15.82:0/4167931066 learned_addr learned my addr 172.21.15.82:0/4167931066 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaa2ffd640 1 -- 172.21.15.82:0/4167931066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 msgr2=0x55a72ed4c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaa2ffd640 1 --2- 172.21.15.82:0/4167931066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 0x55a72ed4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaa2ffd640 1 -- 172.21.15.82:0/4167931066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 msgr2=0x55a72ed58830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaa2ffd640 1 --2- 172.21.15.82:0/4167931066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 0x55a72ed58830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.738+0000 7ffaa2ffd640 1 -- 172.21.15.82:0/4167931066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a72ed4a1b0 con 0x55a72ed43e80 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffaa2ffd640 1 --2- 172.21.15.82:0/4167931066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed4bd80 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ffa98003a40 tx=0x7ffa9802c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644b10c4e749156b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffa897fa640 1 -- 172.21.15.82:0/4167931066 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98009520 con 0x55a72ed43e80 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffa897fa640 1 -- 172.21.15.82:0/4167931066 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa98005480 con 0x55a72ed43e80 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffa897fa640 1 -- 172.21.15.82:0/4167931066 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98004a40 con 0x55a72ed43e80 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffaac778740 1 -- 172.21.15.82:0/4167931066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 msgr2=0x55a72ed4bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffaac778740 1 --2- 172.21.15.82:0/4167931066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed4bd80 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ffa98003a40 tx=0x7ffa9802c350 comp rx=0 tx=0).stop 2024-04-21T08:54:59.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffaac778740 1 -- 172.21.15.82:0/4167931066 shutdown_connections 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffaac778740 1 --2- 172.21.15.82:0/4167931066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 0x55a72ed4c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffaac778740 1 --2- 172.21.15.82:0/4167931066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed4bd80 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffaac778740 1 --2- 172.21.15.82:0/4167931066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 0x55a72ed58830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffaac778740 1 -- 172.21.15.82:0/4167931066 >> 172.21.15.82:0/4167931066 conn(0x55a72ed3dba0 msgr2=0x55a72ed3cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffaac778740 1 -- 172.21.15.82:0/4167931066 shutdown_connections 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.739+0000 7ffaac778740 1 -- 172.21.15.82:0/4167931066 wait complete. 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaac778740 1 -- start start 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaac778740 1 Processor -- start 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaac778740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 0x55a72ed46690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaac778740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed46bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaac778740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 0x55a72edc2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaac778740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41b0af0 con 0x55a72ed43e80 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaac778740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41b03f0 con 0x55a72ed434d0 2024-04-21T08:54:59.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaac778740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41ad590 con 0x55a72ed4ca00 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaac778740 10 client.0 _ll_register_callbacks cb 0x7ffa980043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed46bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed46bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43192/0 (socket says 172.21.15.82:43192) 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaa2ffd640 1 -- 172.21.15.82:0/3650635776 learned_addr learned my addr 172.21.15.82:0/3650635776 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaa3fff640 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 0x55a72edc2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaa37fe640 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 0x55a72ed46690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaa2ffd640 1 -- 172.21.15.82:0/3650635776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 msgr2=0x55a72edc2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.740+0000 7ffaa2ffd640 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 0x55a72edc2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaa2ffd640 1 -- 172.21.15.82:0/3650635776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 msgr2=0x55a72ed46690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaa2ffd640 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 0x55a72ed46690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaa2ffd640 1 -- 172.21.15.82:0/3650635776 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a72edc2770 con 0x55a72ed43e80 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaa37fe640 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 0x55a72ed46690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaa2ffd640 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed46bd0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ffa980092a0 tx=0x7ffa98003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858e5c10744562cf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffa82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffa82ffd640 1 -- 172.21.15.82:0/3650635776 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98062060 con 0x55a72ed43e80 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffa82ffd640 1 -- 172.21.15.82:0/3650635776 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ffa980051d0 con 0x55a72ed43e80 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaac778740 1 -- 172.21.15.82:0/3650635776 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa98005480 con 0x55a72ed43e80 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffa82ffd640 1 -- 172.21.15.82:0/3650635776 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ffa98069050 con 0x55a72ed43e80 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaac778740 10 client.17919 fetch_fsmap learned FSMap version 42 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaac778740 1 -- 172.21.15.82:0/3650635776 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a72edc38b0 con 0x55a72ed43e80 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffa82ffd640 1 -- 172.21.15.82:0/3650635776 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa98005480 con 0x55a72ed43e80 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffa82ffd640 1 -- 172.21.15.82:0/3650635776 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7ffa9805d5f0 con 0x55a72ed43e80 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaac778740 10 client.17919 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaac778740 10 client.17919 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaac778740 20 client.17919 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaac778740 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a72edc4030 0x55a72edc6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaac778740 4 client.17919 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:54:59.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.741+0000 7ffaac778740 1 -- 172.21.15.82:0/3650635776 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a72ecb5c10 con 0x55a72edc4030 2024-04-21T08:54:59.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffaa37fe640 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a72edc4030 0x55a72edc6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffaa37fe640 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a72edc4030 0x55a72edc6450 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=151888b5a70cbc41 server_cookie=d8af507f58e1f21c in_seq=0 out_seq=0 2024-04-21T08:54:59.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffa82ffd640 10 client.17919 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:54:59.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffa82ffd640 1 -- 172.21.15.82:0/3650635776 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a72ecb5c10 con 0x55a72edc4030 2024-04-21T08:54:59.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffa82ffd640 10 client.17919 handle_command_reply: tid=0 2024-04-21T08:54:59.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffaac778740 1 client.17919 shutdown 2024-04-21T08:54:59.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffaac778740 1 -- 172.21.15.82:0/3650635776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 msgr2=0x55a72ed46bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffaac778740 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed46bd0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ffa980092a0 tx=0x7ffa98003ee0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffaac778740 1 -- 172.21.15.82:0/3650635776 shutdown_connections 2024-04-21T08:54:59.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffaac778740 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55a72edc4030 0x55a72edc6450 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffaac778740 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a72ed4ca00 0x55a72edc2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffa82ffd640 0 client.17919 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:54:59.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffaac778740 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a72ed43e80 0x55a72ed46bd0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffaac778740 1 --2- 172.21.15.82:0/3650635776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a72ed434d0 0x55a72ed46690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffaac778740 1 -- 172.21.15.82:0/3650635776 >> 172.21.15.82:0/3650635776 conn(0x55a72ed3dba0 msgr2=0x55a72ed3ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.742+0000 7ffaac778740 1 -- 172.21.15.82:0/3650635776 shutdown_connections 2024-04-21T08:54:59.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaac778740 1 -- 172.21.15.82:0/3650635776 wait complete. 2024-04-21T08:54:59.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaac778740 20 client.17919 trim_cache size 0 max 16384 2024-04-21T08:54:59.750 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:54:59.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaa9b92640 1 -- 172.21.15.82:0/2870707673 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa8c0a3d20 msgr2=0x7ffa8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaa9b92640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa8c0a3d20 0x7ffa8c0a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c01e000 tx=0x7ffa9c01a000 comp rx=0 tx=0).stop 2024-04-21T08:54:59.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaa9b92640 1 -- 172.21.15.82:0/2870707673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaa406a7c0 msgr2=0x7ffaa4198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaa9b92640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaa406a7c0 0x7ffaa4198720 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400d950 tx=0x7ffa9400de10 comp rx=0 tx=0).stop 2024-04-21T08:54:59.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaa9b92640 1 -- 172.21.15.82:0/2870707673 shutdown_connections 2024-04-21T08:54:59.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaa9b92640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaa419cfe0 0x7ffaa419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaa9b92640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4107280 0x7ffaa4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaa9b92640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ffa8c0a3d20 0x7ffa8c0a61e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaa9b92640 1 --2- 172.21.15.82:0/2870707673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaa406a7c0 0x7ffaa4198720 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaa9b92640 1 -- 172.21.15.82:0/2870707673 >> 172.21.15.82:0/2870707673 conn(0x7ffaa4076c50 msgr2=0x7ffaa40ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaa9b92640 1 -- 172.21.15.82:0/2870707673 shutdown_connections 2024-04-21T08:54:59.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.743+0000 7ffaa9b92640 1 -- 172.21.15.82:0/2870707673 wait complete. 2024-04-21T08:54:59.754 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:54:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.855+0000 7ff3edda4640 1 -- 172.21.15.82:0/4115364173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3e8106160 msgr2=0x7ff3e810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.855+0000 7ff3edda4640 1 --2- 172.21.15.82:0/4115364173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3e8106160 0x7ff3e810dd30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009ca0 tx=0x7ff3e002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.855+0000 7ff3edda4640 1 -- 172.21.15.82:0/4115364173 shutdown_connections 2024-04-21T08:54:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.855+0000 7ff3edda4640 1 --2- 172.21.15.82:0/4115364173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3e8106160 0x7ff3e810dd30 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.855+0000 7ff3edda4640 1 --2- 172.21.15.82:0/4115364173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3e8101a40 0x7ff3e8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.855+0000 7ff3edda4640 1 --2- 172.21.15.82:0/4115364173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3e8101090 0x7ff3e8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.855+0000 7ff3edda4640 1 -- 172.21.15.82:0/4115364173 >> 172.21.15.82:0/4115364173 conn(0x7ff3e8077f50 msgr2=0x7ff3e8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.855+0000 7ff3edda4640 1 -- 172.21.15.82:0/4115364173 shutdown_connections 2024-04-21T08:54:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.855+0000 7ff3edda4640 1 -- 172.21.15.82:0/4115364173 wait complete. 2024-04-21T08:54:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.855+0000 7ff3edda4640 1 Processor -- start 2024-04-21T08:54:59.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3edda4640 1 -- start start 2024-04-21T08:54:59.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3edda4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3e8101090 0x7ff3e8102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3edda4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3e8106160 0x7ff3e8103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3edda4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3e81035a0 0x7ff3e81a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3edda4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81110b0 con 0x7ff3e8106160 2024-04-21T08:54:59.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3edda4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e8110f30 con 0x7ff3e81035a0 2024-04-21T08:54:59.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3edda4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e8111230 con 0x7ff3e8101090 2024-04-21T08:54:59.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3e8106160 0x7ff3e8103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3e8106160 0x7ff3e8103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43212/0 (socket says 172.21.15.82:43212) 2024-04-21T08:54:59.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3e6ffd640 1 -- 172.21.15.82:0/1862169348 learned_addr learned my addr 172.21.15.82:0/1862169348 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3e7fff640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3e81035a0 0x7ff3e81a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3e6ffd640 1 -- 172.21.15.82:0/1862169348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3e8101090 msgr2=0x7ff3e8102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3e8101090 0x7ff3e8102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3e6ffd640 1 -- 172.21.15.82:0/1862169348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3e81035a0 msgr2=0x7ff3e81a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3e81035a0 0x7ff3e81a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3e6ffd640 1 -- 172.21.15.82:0/1862169348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e81a6fc0 con 0x7ff3e8106160 2024-04-21T08:54:59.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3e7fff640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3e81035a0 0x7ff3e81a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:54:59.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3e8106160 0x7ff3e8103060 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c970 tx=0x7ff3d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.856+0000 7ff3e4ff9640 1 -- 172.21.15.82:0/1862169348 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8007bb0 con 0x7ff3e8106160 2024-04-21T08:54:59.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.857+0000 7ff3e4ff9640 1 -- 172.21.15.82:0/1862169348 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3d8007d50 con 0x7ff3e8106160 2024-04-21T08:54:59.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.857+0000 7ff3e4ff9640 1 -- 172.21.15.82:0/1862169348 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d800f450 con 0x7ff3e8106160 2024-04-21T08:54:59.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.857+0000 7ff3edda4640 1 -- 172.21.15.82:0/1862169348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e81a72b0 con 0x7ff3e8106160 2024-04-21T08:54:59.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.857+0000 7ff3edda4640 1 -- 172.21.15.82:0/1862169348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e81a7850 con 0x7ff3e8106160 2024-04-21T08:54:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.859+0000 7ff3e4ff9640 1 -- 172.21.15.82:0/1862169348 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff3d8004f20 con 0x7ff3e8106160 2024-04-21T08:54:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.859+0000 7ff3f0999740 1 Processor -- start 2024-04-21T08:54:59.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3f0999740 1 -- start start 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3f0999740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 0x55a246bb38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3f0999740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bb3e80 0x55a246bbc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3f0999740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bbca00 0x55a246bbee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3f0999740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a246b181f0 con 0x55a246bb34d0 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3f0999740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a246ab54b0 con 0x55a246bbca00 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3f0999740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a246b141b0 con 0x55a246bb3e80 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 0x55a246bb38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 0x55a246bb38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43218/0 (socket says 172.21.15.82:43218) 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e7fff640 1 -- 172.21.15.82:0/4067533596 learned_addr learned my addr 172.21.15.82:0/4067533596 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e4ff9640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff3d00a3dd0 0x7ff3d00a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e7fff640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff3d00a3dd0 0x7ff3d00a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e4ff9640 1 -- 172.21.15.82:0/1862169348 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ff3d80d0870 con 0x7ff3e8106160 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/4067533596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bbca00 0x55a246bbee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e7fff640 1 -- 172.21.15.82:0/4067533596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bb3e80 msgr2=0x55a246bbc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e7fff640 1 --2- 172.21.15.82:0/4067533596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bb3e80 0x55a246bbc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e7fff640 1 -- 172.21.15.82:0/4067533596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bbca00 msgr2=0x55a246bbee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e7fff640 1 --2- 172.21.15.82:0/4067533596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bbca00 0x55a246bbee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e7fff640 1 -- 172.21.15.82:0/4067533596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a246aa71b0 con 0x55a246bb34d0 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/4067533596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bbca00 0x55a246bbee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.860+0000 7ff3e7fff640 1 --2- 172.21.15.82:0/4067533596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 0x55a246bb38b0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009ca0 tx=0x7ff3e0032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1942c701f923e824 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3cdffb640 1 -- 172.21.15.82:0/4067533596 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0039040 con 0x55a246bb34d0 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3cdffb640 1 -- 172.21.15.82:0/4067533596 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3e0032960 con 0x55a246bb34d0 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3cdffb640 1 -- 172.21.15.82:0/4067533596 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0034680 con 0x55a246bb34d0 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3f0999740 1 -- 172.21.15.82:0/4067533596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 msgr2=0x55a246bb38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3e7fff640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff3d00a3dd0 0x7ff3d00a6290 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff3e004c000 tx=0x7ff3e0048000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3f0999740 1 --2- 172.21.15.82:0/4067533596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 0x55a246bb38b0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009ca0 tx=0x7ff3e0032d70 comp rx=0 tx=0).stop 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3f0999740 1 -- 172.21.15.82:0/4067533596 shutdown_connections 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3f0999740 1 --2- 172.21.15.82:0/4067533596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bbca00 0x55a246bbee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3f0999740 1 --2- 172.21.15.82:0/4067533596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bb3e80 0x55a246bbc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3f0999740 1 --2- 172.21.15.82:0/4067533596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 0x55a246bb38b0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3f0999740 1 -- 172.21.15.82:0/4067533596 >> 172.21.15.82:0/4067533596 conn(0x55a246badba0 msgr2=0x55a246badf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3f0999740 1 -- 172.21.15.82:0/4067533596 shutdown_connections 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.861+0000 7ff3f0999740 1 -- 172.21.15.82:0/4067533596 wait complete. 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3f0999740 1 -- start start 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3f0999740 1 Processor -- start 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3f0999740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 0x55a246bbc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3f0999740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bb3e80 0x55a246bb9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3f0999740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bbca00 0x55a246bb9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3f0999740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a246bbf970 con 0x55a246bb34d0 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3f0999740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a246ad1500 con 0x55a246bbca00 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3f0999740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a246bbfaf0 con 0x55a246bb3e80 2024-04-21T08:54:59.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3f0999740 10 client.0 _ll_register_callbacks cb 0x55a246a9fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3e77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 0x55a246bbc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3e77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 0x55a246bbc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43220/0 (socket says 172.21.15.82:43220) 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3e77fe640 1 -- 172.21.15.82:0/70960536 learned_addr learned my addr 172.21.15.82:0/70960536 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3e77fe640 1 -- 172.21.15.82:0/70960536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bb3e80 msgr2=0x55a246bb9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3e77fe640 1 --2- 172.21.15.82:0/70960536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bb3e80 0x55a246bb9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3e77fe640 1 -- 172.21.15.82:0/70960536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bbca00 msgr2=0x55a246bb9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3e77fe640 1 --2- 172.21.15.82:0/70960536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bbca00 0x55a246bb9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.862+0000 7ff3e77fe640 1 -- 172.21.15.82:0/70960536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24682a3f0 con 0x55a246bb34d0 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3e77fe640 1 --2- 172.21.15.82:0/70960536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 0x55a246bbc3a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00c2d0 tx=0x7ff3dc00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6e97eeedb9030b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3c6ffd640 1 -- 172.21.15.82:0/70960536 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc00f860 con 0x55a246bb34d0 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3c6ffd640 1 -- 172.21.15.82:0/70960536 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ff3dc00fa00 con 0x55a246bb34d0 2024-04-21T08:54:59.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3f0999740 1 -- 172.21.15.82:0/70960536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a246ab7930 con 0x55a246bb34d0 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3c6ffd640 1 -- 172.21.15.82:0/70960536 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff3dc021050 con 0x55a246bb34d0 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3f0999740 10 client.17931 fetch_fsmap learned FSMap version 42 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3f0999740 1 -- 172.21.15.82:0/70960536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a2468f47d0 con 0x55a246bb34d0 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3c6ffd640 1 -- 172.21.15.82:0/70960536 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a246ab7930 con 0x55a246bb34d0 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3c6ffd640 1 -- 172.21.15.82:0/70960536 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7ff3dc00e3b0 con 0x55a246bb34d0 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3f0999740 10 client.17931 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3f0999740 10 client.17931 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3f0999740 20 client.17931 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3f0999740 1 --2- 172.21.15.82:0/70960536 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a246bccc50 0x55a246bcf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3f0999740 4 client.17931 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.863+0000 7ff3f0999740 1 -- 172.21.15.82:0/70960536 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a246b25c10 con 0x55a246bccc50 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.864+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/70960536 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a246bccc50 0x55a246bcf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.864+0000 7ff3c6ffd640 10 client.17931 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:54:59.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.864+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/70960536 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a246bccc50 0x55a246bcf110 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48f1fddfeade542c server_cookie=f54eaee52c41cc9a in_seq=0 out_seq=0 2024-04-21T08:54:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3c6ffd640 1 -- 172.21.15.82:0/70960536 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a246b25c10 con 0x55a246bccc50 2024-04-21T08:54:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3c6ffd640 10 client.17931 handle_command_reply: tid=0 2024-04-21T08:54:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3f0999740 1 client.17931 shutdown 2024-04-21T08:54:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3f0999740 1 -- 172.21.15.82:0/70960536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 msgr2=0x55a246bbc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3f0999740 1 --2- 172.21.15.82:0/70960536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 0x55a246bbc3a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00c2d0 tx=0x7ff3dc00c790 comp rx=0 tx=0).stop 2024-04-21T08:54:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3f0999740 1 -- 172.21.15.82:0/70960536 shutdown_connections 2024-04-21T08:54:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3f0999740 1 --2- 172.21.15.82:0/70960536 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a246bccc50 0x55a246bcf110 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3f0999740 1 --2- 172.21.15.82:0/70960536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bbca00 0x55a246bb9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3f0999740 1 --2- 172.21.15.82:0/70960536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bb3e80 0x55a246bb9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3f0999740 1 --2- 172.21.15.82:0/70960536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb34d0 0x55a246bbc3a0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3c6ffd640 0 client.17931 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:54:59.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3f0999740 1 -- 172.21.15.82:0/70960536 >> 172.21.15.82:0/70960536 conn(0x55a246badba0 msgr2=0x55a246baf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3f0999740 1 -- 172.21.15.82:0/70960536 shutdown_connections 2024-04-21T08:54:59.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3f0999740 1 -- 172.21.15.82:0/70960536 wait complete. 2024-04-21T08:54:59.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.865+0000 7ff3f0999740 20 client.17931 trim_cache size 0 max 16384 2024-04-21T08:54:59.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3f0999740 1 Processor -- start 2024-04-21T08:54:59.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3f0999740 1 -- start start 2024-04-21T08:54:59.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3f0999740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bb34d0 0x55a246bc8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3f0999740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 0x55a246bd4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3f0999740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bbca00 0x55a246bd4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3f0999740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81b17c0 con 0x55a246bb3e80 2024-04-21T08:54:59.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3f0999740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81b33f0 con 0x55a246bb34d0 2024-04-21T08:54:59.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3f0999740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81b1e90 con 0x55a246bbca00 2024-04-21T08:54:59.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 0x55a246bd4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 0x55a246bd4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43232/0 (socket says 172.21.15.82:43232) 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3e6ffd640 1 -- 172.21.15.82:0/287291053 learned_addr learned my addr 172.21.15.82:0/287291053 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3e6ffd640 1 -- 172.21.15.82:0/287291053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bbca00 msgr2=0x55a246bd4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/287291053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bbca00 0x55a246bd4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3e6ffd640 1 -- 172.21.15.82:0/287291053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bb34d0 msgr2=0x55a246bc8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/287291053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bb34d0 0x55a246bc8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.878+0000 7ff3e6ffd640 1 -- 172.21.15.82:0/287291053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a246bb3bf0 con 0x55a246bb3e80 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/287291053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 0x55a246bd4a30 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800afd0 tx=0x7ff3d8098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a85a41ced9d9a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3cd7fa640 1 -- 172.21.15.82:0/287291053 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8085020 con 0x55a246bb3e80 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3cd7fa640 1 -- 172.21.15.82:0/287291053 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3d80a5070 con 0x55a246bb3e80 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3cd7fa640 1 -- 172.21.15.82:0/287291053 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d801d040 con 0x55a246bb3e80 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3f0999740 1 -- 172.21.15.82:0/287291053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 msgr2=0x55a246bd4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3f0999740 1 --2- 172.21.15.82:0/287291053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 0x55a246bd4a30 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800afd0 tx=0x7ff3d8098040 comp rx=0 tx=0).stop 2024-04-21T08:54:59.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3f0999740 1 -- 172.21.15.82:0/287291053 shutdown_connections 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3f0999740 1 --2- 172.21.15.82:0/287291053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bbca00 0x55a246bd4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3f0999740 1 --2- 172.21.15.82:0/287291053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 0x55a246bd4a30 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3f0999740 1 --2- 172.21.15.82:0/287291053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bb34d0 0x55a246bc8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3f0999740 1 -- 172.21.15.82:0/287291053 >> 172.21.15.82:0/287291053 conn(0x55a246badba0 msgr2=0x55a246bacc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3f0999740 1 -- 172.21.15.82:0/287291053 shutdown_connections 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.879+0000 7ff3f0999740 1 -- 172.21.15.82:0/287291053 wait complete. 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3f0999740 1 -- start start 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3f0999740 1 Processor -- start 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3f0999740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bb34d0 0x55a246bb6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3f0999740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 0x55a246bb6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3f0999740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bbca00 0x55a246c322a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3f0999740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81b2cf0 con 0x55a246bb3e80 2024-04-21T08:54:59.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3f0999740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81b25f0 con 0x55a246bb34d0 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3f0999740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e81af790 con 0x55a246bbca00 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3f0999740 10 client.0 _ll_register_callbacks cb 0x7ff3dc00ab90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 0x55a246bb6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 0x55a246bb6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43236/0 (socket says 172.21.15.82:43236) 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3e6ffd640 1 -- 172.21.15.82:0/1954661190 learned_addr learned my addr 172.21.15.82:0/1954661190 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3e6ffd640 1 -- 172.21.15.82:0/1954661190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bbca00 msgr2=0x55a246c322a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bbca00 0x55a246c322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3e77fe640 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bb34d0 0x55a246bb6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3e6ffd640 1 -- 172.21.15.82:0/1954661190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bb34d0 msgr2=0x55a246bb6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bb34d0 0x55a246bb6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3e6ffd640 1 -- 172.21.15.82:0/1954661190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a246c327e0 con 0x55a246bb3e80 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.880+0000 7ff3e77fe640 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bb34d0 0x55a246bb6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:54:59.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3e6ffd640 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 0x55a246bb6f90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff3d809c100 tx=0x7ff3d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d335d7bc1af8f97b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3c6ffd640 1 -- 172.21.15.82:0/1954661190 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d801d030 con 0x55a246bb3e80 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3c6ffd640 1 -- 172.21.15.82:0/1954661190 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7ff3d80cde30 con 0x55a246bb3e80 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3f0999740 1 -- 172.21.15.82:0/1954661190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3d80a5070 con 0x55a246bb3e80 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3c6ffd640 1 -- 172.21.15.82:0/1954661190 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff3d8085040 con 0x55a246bb3e80 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3f0999740 10 client.17937 fetch_fsmap learned FSMap version 42 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3f0999740 1 -- 172.21.15.82:0/1954661190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a246c339e0 con 0x55a246bb3e80 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3c6ffd640 1 -- 172.21.15.82:0/1954661190 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3d80a5070 con 0x55a246bb3e80 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3c6ffd640 1 -- 172.21.15.82:0/1954661190 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7ff3d80b5b80 con 0x55a246bb3e80 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3f0999740 10 client.17937 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3f0999740 10 client.17937 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3f0999740 20 client.17937 populate_metadata read hostname 'smithi082' 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3f0999740 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a246c34210 0x55a246c36630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3f0999740 4 client.17937 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3f0999740 1 -- 172.21.15.82:0/1954661190 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a246bc94e0 con 0x55a246c34210 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.881+0000 7ff3e77fe640 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a246c34210 0x55a246c36630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3c6ffd640 10 client.17937 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3e77fe640 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a246c34210 0x55a246c36630 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0c15bd669e6c689 server_cookie=30902e22d149f43b in_seq=0 out_seq=0 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3c6ffd640 1 -- 172.21.15.82:0/1954661190 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a246bc94e0 con 0x55a246c34210 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3c6ffd640 10 client.17937 handle_command_reply: tid=0 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3f0999740 1 client.17937 shutdown 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3f0999740 1 -- 172.21.15.82:0/1954661190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 msgr2=0x55a246bb6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3f0999740 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 0x55a246bb6f90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff3d809c100 tx=0x7ff3d80046c0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3f0999740 1 -- 172.21.15.82:0/1954661190 shutdown_connections 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3f0999740 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55a246c34210 0x55a246c36630 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3f0999740 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a246bbca00 0x55a246c322a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3f0999740 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a246bb3e80 0x55a246bb6f90 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3c6ffd640 0 client.17937 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:54:59.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3f0999740 1 --2- 172.21.15.82:0/1954661190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a246bb34d0 0x55a246bb6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3f0999740 1 -- 172.21.15.82:0/1954661190 >> 172.21.15.82:0/1954661190 conn(0x55a246badba0 msgr2=0x55a246bb43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3f0999740 1 -- 172.21.15.82:0/1954661190 shutdown_connections 2024-04-21T08:54:59.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3f0999740 1 -- 172.21.15.82:0/1954661190 wait complete. 2024-04-21T08:54:59.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.882+0000 7ff3f0999740 20 client.17937 trim_cache size 0 max 16384 2024-04-21T08:54:59.890 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:54:59.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.883+0000 7ff3edda4640 1 -- 172.21.15.82:0/1862169348 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff3d00a3dd0 msgr2=0x7ff3d00a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.883+0000 7ff3edda4640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff3d00a3dd0 0x7ff3d00a6290 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff3e004c000 tx=0x7ff3e0048000 comp rx=0 tx=0).stop 2024-04-21T08:54:59.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.883+0000 7ff3edda4640 1 -- 172.21.15.82:0/1862169348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3e8106160 msgr2=0x7ff3e8103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.883+0000 7ff3edda4640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3e8106160 0x7ff3e8103060 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c970 tx=0x7ff3d800ce30 comp rx=0 tx=0).stop 2024-04-21T08:54:59.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.883+0000 7ff3edda4640 1 -- 172.21.15.82:0/1862169348 shutdown_connections 2024-04-21T08:54:59.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.883+0000 7ff3edda4640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff3e81035a0 0x7ff3e81a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.883+0000 7ff3edda4640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff3d00a3dd0 0x7ff3d00a6290 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.883+0000 7ff3edda4640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3e8106160 0x7ff3e8103060 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.883+0000 7ff3edda4640 1 --2- 172.21.15.82:0/1862169348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3e8101090 0x7ff3e8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.883+0000 7ff3edda4640 1 -- 172.21.15.82:0/1862169348 >> 172.21.15.82:0/1862169348 conn(0x7ff3e8077f50 msgr2=0x7ff3e80694d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.883+0000 7ff3edda4640 1 -- 172.21.15.82:0/1862169348 shutdown_connections 2024-04-21T08:54:59.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.883+0000 7ff3edda4640 1 -- 172.21.15.82:0/1862169348 wait complete. 2024-04-21T08:54:59.894 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:54:59.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.995+0000 7fdf39f15640 1 -- 172.21.15.82:0/117568726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf3406a570 msgr2=0x7fdf3406a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.995+0000 7fdf39f15640 1 --2- 172.21.15.82:0/117568726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf3406a570 0x7fdf3406a9f0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009930 tx=0x7fdf1c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:54:59.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.995+0000 7fdf39f15640 1 -- 172.21.15.82:0/117568726 shutdown_connections 2024-04-21T08:54:59.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.995+0000 7fdf39f15640 1 --2- 172.21.15.82:0/117568726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf3406af30 0x7fdf3410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.995+0000 7fdf39f15640 1 --2- 172.21.15.82:0/117568726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf3406a570 0x7fdf3406a9f0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.995+0000 7fdf39f15640 1 --2- 172.21.15.82:0/117568726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf341080f0 0x7fdf341084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.995+0000 7fdf39f15640 1 -- 172.21.15.82:0/117568726 >> 172.21.15.82:0/117568726 conn(0x7fdf34077e30 msgr2=0x7fdf34078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:54:59.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.995+0000 7fdf39f15640 1 -- 172.21.15.82:0/117568726 shutdown_connections 2024-04-21T08:54:59.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.995+0000 7fdf39f15640 1 -- 172.21.15.82:0/117568726 wait complete. 2024-04-21T08:54:59.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf39f15640 1 Processor -- start 2024-04-21T08:54:59.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf39f15640 1 -- start start 2024-04-21T08:54:59.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf39f15640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf3406a570 0x7fdf34198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf39f15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf3406af30 0x7fdf34198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf39f15640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf341080f0 0x7fdf3419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf39f15640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341121d0 con 0x7fdf341080f0 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf39f15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34112050 con 0x7fdf3406af30 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf39f15640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34112350 con 0x7fdf3406a570 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf33fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf341080f0 0x7fdf3419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf33fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf341080f0 0x7fdf3419cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43242/0 (socket says 172.21.15.82:43242) 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf33fff640 1 -- 172.21.15.82:0/2719617097 learned_addr learned my addr 172.21.15.82:0/2719617097 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf3406af30 0x7fdf34198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf337fe640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf3406a570 0x7fdf34198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf33fff640 1 -- 172.21.15.82:0/2719617097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf3406a570 msgr2=0x7fdf34198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf33fff640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf3406a570 0x7fdf34198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf33fff640 1 -- 172.21.15.82:0/2719617097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf3406af30 msgr2=0x7fdf34198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf33fff640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf3406af30 0x7fdf34198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.996+0000 7fdf33fff640 1 -- 172.21.15.82:0/2719617097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3419d720 con 0x7fdf341080f0 2024-04-21T08:54:59.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.997+0000 7fdf33fff640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf341080f0 0x7fdf3419cff0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00eaa0 tx=0x7fdf2c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:00.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.997+0000 7fdf317fa640 1 -- 172.21.15.82:0/2719617097 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c00cdb0 con 0x7fdf341080f0 2024-04-21T08:55:00.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.997+0000 7fdf317fa640 1 -- 172.21.15.82:0/2719617097 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf2c005020 con 0x7fdf341080f0 2024-04-21T08:55:00.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.997+0000 7fdf39f15640 1 -- 172.21.15.82:0/2719617097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3419da10 con 0x7fdf341080f0 2024-04-21T08:55:00.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.997+0000 7fdf317fa640 1 -- 172.21.15.82:0/2719617097 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0116d0 con 0x7fdf341080f0 2024-04-21T08:55:00.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.997+0000 7fdf39f15640 1 -- 172.21.15.82:0/2719617097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf341a52a0 con 0x7fdf341080f0 2024-04-21T08:55:00.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.999+0000 7fdf317fa640 1 -- 172.21.15.82:0/2719617097 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdf2c0051d0 con 0x7fdf341080f0 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.999+0000 7fdf3cbe7740 1 Processor -- start 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.999+0000 7fdf3cbe7740 1 -- start start 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.999+0000 7fdf3cbe7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c08f9330 0x55e0c08f9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.999+0000 7fdf3cbe7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 0x55e0c0902820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.999+0000 7fdf317fa640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdf200a3d20 0x7fdf200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.999+0000 7fdf3cbe7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c0902e40 0x55e0c09032c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.999+0000 7fdf3cbe7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e0c085e1f0 con 0x55e0c08f9ce0 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.999+0000 7fdf3cbe7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e0c07fb4b0 con 0x55e0c08f9330 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:54:59.999+0000 7fdf3cbe7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e0c085a1b0 con 0x55e0c0902e40 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf337fe640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdf200a3d20 0x7fdf200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf2bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 0x55e0c0902820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf2bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 0x55e0c0902820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43258/0 (socket says 172.21.15.82:43258) 2024-04-21T08:55:00.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf2bfff640 1 -- 172.21.15.82:0/2381570975 learned_addr learned my addr 172.21.15.82:0/2381570975 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/2381570975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c08f9330 0x55e0c08f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf317fa640 1 -- 172.21.15.82:0/2719617097 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fdf2c0d0e20 con 0x7fdf341080f0 2024-04-21T08:55:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf33fff640 1 --2- 172.21.15.82:0/2381570975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c0902e40 0x55e0c09032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf2bfff640 1 -- 172.21.15.82:0/2381570975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c0902e40 msgr2=0x55e0c09032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/2381570975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c0902e40 0x55e0c09032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf2bfff640 1 -- 172.21.15.82:0/2381570975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c08f9330 msgr2=0x55e0c08f9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/2381570975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c08f9330 0x55e0c08f9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf2bfff640 1 -- 172.21.15.82:0/2381570975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c07ed1b0 con 0x55e0c08f9ce0 2024-04-21T08:55:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf33fff640 1 --2- 172.21.15.82:0/2381570975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c0902e40 0x55e0c09032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:55:00.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf337fe640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdf200a3d20 0x7fdf200a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdf24005fd0 tx=0x7fdf24005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/2381570975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 0x55e0c0902820 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c036040 tx=0x7fdf1c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89bf3a40ac4138c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf29ffb640 1 -- 172.21.15.82:0/2381570975 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c04a070 con 0x55e0c08f9ce0 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf29ffb640 1 -- 172.21.15.82:0/2381570975 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1c02cc90 con 0x55e0c08f9ce0 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.000+0000 7fdf29ffb640 1 -- 172.21.15.82:0/2381570975 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c003dc0 con 0x55e0c08f9ce0 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.001+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2381570975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 msgr2=0x55e0c0902820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.001+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/2381570975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 0x55e0c0902820 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c036040 tx=0x7fdf1c005d90 comp rx=0 tx=0).stop 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.001+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2381570975 shutdown_connections 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.001+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/2381570975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c0902e40 0x55e0c09032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.001+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/2381570975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 0x55e0c0902820 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.001+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/2381570975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c08f9330 0x55e0c08f9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.001+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2381570975 >> 172.21.15.82:0/2381570975 conn(0x55e0c08f3ba0 msgr2=0x55e0c08f3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.001+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2381570975 shutdown_connections 2024-04-21T08:55:00.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.001+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2381570975 wait complete. 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.001+0000 7fdf3cbe7740 1 -- start start 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf3cbe7740 1 Processor -- start 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf3cbe7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c08f9330 0x55e0c0909260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf3cbe7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 0x55e0c09065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf3cbe7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c0902e40 0x55e0c0906b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf3cbe7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e0c0909d60 con 0x55e0c08f9ce0 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf3cbe7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e0c0817500 con 0x55e0c08f9330 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf3cbe7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e0c0909ee0 con 0x55e0c0902e40 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf3cbe7740 10 client.0 _ll_register_callbacks cb 0x55e0c07e5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf2bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 0x55e0c09065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf2bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 0x55e0c09065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43262/0 (socket says 172.21.15.82:43262) 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf2bfff640 1 -- 172.21.15.82:0/2839801596 learned_addr learned my addr 172.21.15.82:0/2839801596 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf33fff640 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c0902e40 0x55e0c0906b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf2bfff640 1 -- 172.21.15.82:0/2839801596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c0902e40 msgr2=0x55e0c0906b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c0902e40 0x55e0c0906b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf2bfff640 1 -- 172.21.15.82:0/2839801596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c08f9330 msgr2=0x55e0c0909260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c08f9330 0x55e0c0909260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf2bfff640 1 -- 172.21.15.82:0/2839801596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c05703f0 con 0x55e0c08f9ce0 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf33fff640 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c0902e40 0x55e0c0906b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 0x55e0c09065f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c02c170 tx=0x7fdf1c02cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3eff3824cfbf794 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf12ffd640 1 -- 172.21.15.82:0/2839801596 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c053060 con 0x55e0c08f9ce0 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.002+0000 7fdf12ffd640 1 -- 172.21.15.82:0/2839801596 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fdf1c005070 con 0x55e0c08f9ce0 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2839801596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf1c02cc90 con 0x55e0c08f9ce0 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf12ffd640 1 -- 172.21.15.82:0/2839801596 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fdf1c04a070 con 0x55e0c08f9ce0 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf3cbe7740 10 client.17949 fetch_fsmap learned FSMap version 42 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2839801596 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e0c063a7d0 con 0x55e0c08f9ce0 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf12ffd640 1 -- 172.21.15.82:0/2839801596 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1c02cc90 con 0x55e0c08f9ce0 2024-04-21T08:55:00.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf12ffd640 1 -- 172.21.15.82:0/2839801596 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7fdf1c004aa0 con 0x55e0c08f9ce0 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf3cbe7740 10 client.17949 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf3cbe7740 10 client.17949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf3cbe7740 20 client.17949 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e0c0916eb0 0x55e0c09192a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf3cbe7740 4 client.17949 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2839801596 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0c07fd930 con 0x55e0c0916eb0 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.003+0000 7fdf337fe640 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e0c0916eb0 0x55e0c09192a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.004+0000 7fdf337fe640 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e0c0916eb0 0x55e0c09192a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f92353eb188c86 server_cookie=54eb38f36a246e97 in_seq=0 out_seq=0 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.004+0000 7fdf12ffd640 10 client.17949 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.004+0000 7fdf12ffd640 1 -- 172.21.15.82:0/2839801596 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0c07fd930 con 0x55e0c0916eb0 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.004+0000 7fdf12ffd640 10 client.17949 handle_command_reply: tid=0 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.004+0000 7fdf3cbe7740 1 client.17949 shutdown 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.005+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2839801596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 msgr2=0x55e0c09065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:00.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.005+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 0x55e0c09065f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c02c170 tx=0x7fdf1c02cb10 comp rx=0 tx=0).stop 2024-04-21T08:55:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.005+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2839801596 shutdown_connections 2024-04-21T08:55:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.005+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e0c0916eb0 0x55e0c09192a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.005+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c0902e40 0x55e0c0906b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.005+0000 7fdf12ffd640 0 client.17949 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.005+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9ce0 0x55e0c09065f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.005+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/2839801596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c08f9330 0x55e0c0909260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.005+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2839801596 >> 172.21.15.82:0/2839801596 conn(0x55e0c08f3ba0 msgr2=0x55e0c08f58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.005+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2839801596 shutdown_connections 2024-04-21T08:55:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.005+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/2839801596 wait complete. 2024-04-21T08:55:00.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.005+0000 7fdf3cbe7740 20 client.17949 trim_cache size 0 max 16384 2024-04-21T08:55:00.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf3cbe7740 1 Processor -- start 2024-04-21T08:55:00.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf3cbe7740 1 -- start start 2024-04-21T08:55:00.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf3cbe7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 0x55e0c0904230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf3cbe7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c08f9ce0 0x55e0c0901800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf3cbe7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c0902e40 0x55e0c0905750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf3cbe7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341ad2a0 con 0x55e0c08f9330 2024-04-21T08:55:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf3cbe7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341aeed0 con 0x55e0c0902e40 2024-04-21T08:55:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf3cbe7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341ad970 con 0x55e0c08f9ce0 2024-04-21T08:55:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 0x55e0c0904230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 0x55e0c0904230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43276/0 (socket says 172.21.15.82:43276) 2024-04-21T08:55:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf337fe640 1 -- 172.21.15.82:0/4065462062 learned_addr learned my addr 172.21.15.82:0/4065462062 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf33fff640 1 --2- 172.21.15.82:0/4065462062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c0902e40 0x55e0c0905750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf337fe640 1 -- 172.21.15.82:0/4065462062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c08f9ce0 msgr2=0x55e0c0901800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/4065462062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c08f9ce0 0x55e0c0901800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf337fe640 1 --2- 172.21.15.82:0/4065462062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c08f9ce0 0x55e0c0901800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf337fe640 1 -- 172.21.15.82:0/4065462062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c0902e40 msgr2=0x55e0c0905750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf337fe640 1 --2- 172.21.15.82:0/4065462062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c0902e40 0x55e0c0905750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf337fe640 1 -- 172.21.15.82:0/4065462062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c091f0e0 con 0x55e0c08f9330 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf33fff640 1 --2- 172.21.15.82:0/4065462062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c0902e40 0x55e0c0905750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.018+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/4065462062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c08f9ce0 0x55e0c0901800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf337fe640 1 --2- 172.21.15.82:0/4065462062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 0x55e0c0904230 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x55e0c0909f40 tx=0x7fdf24012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f45764495a6f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf297fa640 1 -- 172.21.15.82:0/4065462062 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24025b50 con 0x55e0c08f9330 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf297fa640 1 -- 172.21.15.82:0/4065462062 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf24025cf0 con 0x55e0c08f9330 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf297fa640 1 -- 172.21.15.82:0/4065462062 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2401e9e0 con 0x55e0c08f9330 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4065462062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 msgr2=0x55e0c0904230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/4065462062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 0x55e0c0904230 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x55e0c0909f40 tx=0x7fdf24012000 comp rx=0 tx=0).stop 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4065462062 shutdown_connections 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/4065462062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c0902e40 0x55e0c0905750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/4065462062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c08f9ce0 0x55e0c0901800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/4065462062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 0x55e0c0904230 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4065462062 >> 172.21.15.82:0/4065462062 conn(0x55e0c08f3ba0 msgr2=0x55e0c08f2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4065462062 shutdown_connections 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.019+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4065462062 wait complete. 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf3cbe7740 1 -- start start 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf3cbe7740 1 Processor -- start 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf3cbe7740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 0x55e0c091c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf3cbe7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c08f9ce0 0x55e0c091c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf3cbe7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c0902e40 0x55e0c09198b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf3cbe7740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341ae7d0 con 0x55e0c08f9330 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf3cbe7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341ae0d0 con 0x55e0c0902e40 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf3cbe7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341ab250 con 0x55e0c08f9ce0 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf3cbe7740 10 client.0 _ll_register_callbacks cb 0x7fdf1c004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:00.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 0x55e0c091c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 0x55e0c091c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43280/0 (socket says 172.21.15.82:43280) 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf337fe640 1 -- 172.21.15.82:0/4117876359 learned_addr learned my addr 172.21.15.82:0/4117876359 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf33fff640 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c0902e40 0x55e0c09198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c08f9ce0 0x55e0c091c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf337fe640 1 -- 172.21.15.82:0/4117876359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c08f9ce0 msgr2=0x55e0c091c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf337fe640 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c08f9ce0 0x55e0c091c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf337fe640 1 -- 172.21.15.82:0/4117876359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c0902e40 msgr2=0x55e0c09198b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf337fe640 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c0902e40 0x55e0c09198b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.020+0000 7fdf337fe640 1 -- 172.21.15.82:0/4117876359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c091a0b0 con 0x55e0c08f9330 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c08f9ce0 0x55e0c091c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf337fe640 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 0x55e0c091c320 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400bac0 tx=0x7fdf24011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6243e65d29f644 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf12ffd640 1 -- 172.21.15.82:0/4117876359 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24005510 con 0x55e0c08f9330 2024-04-21T08:55:00.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf12ffd640 1 -- 172.21.15.82:0/4117876359 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fdf2401e760 con 0x55e0c08f9330 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4117876359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf24025cf0 con 0x55e0c08f9330 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf12ffd640 1 -- 172.21.15.82:0/4117876359 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fdf2401e960 con 0x55e0c08f9330 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf3cbe7740 10 client.17955 fetch_fsmap learned FSMap version 42 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4117876359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e0c097cc90 con 0x55e0c08f9330 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf12ffd640 1 -- 172.21.15.82:0/4117876359 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf24025cf0 con 0x55e0c08f9330 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf12ffd640 1 -- 172.21.15.82:0/4117876359 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7fdf24034070 con 0x55e0c08f9330 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf3cbe7740 10 client.17955 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf3cbe7740 10 client.17955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf3cbe7740 20 client.17955 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e0c097d380 0x55e0c097f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf3cbe7740 4 client.17955 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.021+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4117876359 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0c086bc10 con 0x55e0c097d380 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e0c097d380 0x55e0c097f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:00.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf12ffd640 10 client.17955 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf2bfff640 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e0c097d380 0x55e0c097f7a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a54f5fcde6fa90a8 server_cookie=f65dd3793a5a9fd in_seq=0 out_seq=0 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf12ffd640 1 -- 172.21.15.82:0/4117876359 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0c086bc10 con 0x55e0c097d380 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf12ffd640 10 client.17955 handle_command_reply: tid=0 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf3cbe7740 1 client.17955 shutdown 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4117876359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 msgr2=0x55e0c091c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 0x55e0c091c320 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400bac0 tx=0x7fdf24011de0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4117876359 shutdown_connections 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55e0c097d380 0x55e0c097f7a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0c0902e40 0x55e0c09198b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0c08f9ce0 0x55e0c091c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf12ffd640 0 client.17955 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf3cbe7740 1 --2- 172.21.15.82:0/4117876359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0c08f9330 0x55e0c091c320 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4117876359 >> 172.21.15.82:0/4117876359 conn(0x55e0c08f3ba0 msgr2=0x55e0c0912a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:00.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4117876359 shutdown_connections 2024-04-21T08:55:00.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf3cbe7740 1 -- 172.21.15.82:0/4117876359 wait complete. 2024-04-21T08:55:00.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.022+0000 7fdf3cbe7740 20 client.17955 trim_cache size 0 max 16384 2024-04-21T08:55:00.031 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:55:00.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.023+0000 7fdf39f15640 1 -- 172.21.15.82:0/2719617097 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdf200a3d20 msgr2=0x7fdf200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:00.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.023+0000 7fdf39f15640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdf200a3d20 0x7fdf200a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdf24005fd0 tx=0x7fdf24005850 comp rx=0 tx=0).stop 2024-04-21T08:55:00.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.023+0000 7fdf39f15640 1 -- 172.21.15.82:0/2719617097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf341080f0 msgr2=0x7fdf3419cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:00.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.023+0000 7fdf39f15640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf341080f0 0x7fdf3419cff0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00eaa0 tx=0x7fdf2c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:55:00.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.023+0000 7fdf39f15640 1 -- 172.21.15.82:0/2719617097 shutdown_connections 2024-04-21T08:55:00.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.023+0000 7fdf39f15640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdf200a3d20 0x7fdf200a61e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.023+0000 7fdf39f15640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf341080f0 0x7fdf3419cff0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.023+0000 7fdf39f15640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf3406af30 0x7fdf34198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.023+0000 7fdf39f15640 1 --2- 172.21.15.82:0/2719617097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf3406a570 0x7fdf34198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:00.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.023+0000 7fdf39f15640 1 -- 172.21.15.82:0/2719617097 >> 172.21.15.82:0/2719617097 conn(0x7fdf34077e30 msgr2=0x7fdf340fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:00.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.023+0000 7fdf39f15640 1 -- 172.21.15.82:0/2719617097 shutdown_connections 2024-04-21T08:55:00.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:00.023+0000 7fdf39f15640 1 -- 172.21.15.82:0/2719617097 wait complete. 2024-04-21T08:55:00.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:00 smithi134 ceph-mon[31772]: pgmap v244: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 250 op/s 2024-04-21T08:55:00.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:00 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/549897278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:55:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:00 smithi082 ceph-mon[27967]: pgmap v244: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 250 op/s 2024-04-21T08:55:00.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:00 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/549897278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:55:00.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:00 smithi106 ceph-mon[31072]: pgmap v244: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 250 op/s 2024-04-21T08:55:00.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:00 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/549897278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:55:01.035 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:55:01.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:55:01.035 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:55:01.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.143+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/300923094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d8101090 msgr2=0x7fe7d8101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.143+0000 7fe7dfa8e640 1 --2- 172.21.15.82:0/300923094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d8101090 0x7fe7d8101470 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009900 tx=0x7fe7c802bc30 comp rx=0 tx=0).stop 2024-04-21T08:55:01.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.143+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/300923094 shutdown_connections 2024-04-21T08:55:01.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.143+0000 7fe7dfa8e640 1 --2- 172.21.15.82:0/300923094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7d8106160 0x7fe7d810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.143+0000 7fe7dfa8e640 1 --2- 172.21.15.82:0/300923094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8101a40 0x7fe7d8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.143+0000 7fe7dfa8e640 1 --2- 172.21.15.82:0/300923094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d8101090 0x7fe7d8101470 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.143+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/300923094 >> 172.21.15.82:0/300923094 conn(0x7fe7d8077f50 msgr2=0x7fe7d8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:01.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.143+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/300923094 shutdown_connections 2024-04-21T08:55:01.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.143+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/300923094 wait complete. 2024-04-21T08:55:01.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.143+0000 7fe7dfa8e640 1 Processor -- start 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.143+0000 7fe7dfa8e640 1 -- start start 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.143+0000 7fe7dfa8e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8101090 0x7fe7d8197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7dfa8e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7d8106160 0x7fe7d8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7dfa8e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d819bf00 0x7fe7d819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7dfa8e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8111120 con 0x7fe7d819bf00 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7dfa8e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8110fa0 con 0x7fe7d8106160 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7dfa8e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81112a0 con 0x7fe7d8101090 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7de02b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d819bf00 0x7fe7d819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7de02b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d819bf00 0x7fe7d819c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43310/0 (socket says 172.21.15.82:43310) 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7de02b640 1 -- 172.21.15.82:0/1807629442 learned_addr learned my addr 172.21.15.82:0/1807629442 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7dd029640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7d8106160 0x7fe7d8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7dd82a640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8101090 0x7fe7d8197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7de02b640 1 -- 172.21.15.82:0/1807629442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8101090 msgr2=0x7fe7d8197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7de02b640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8101090 0x7fe7d8197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7de02b640 1 -- 172.21.15.82:0/1807629442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7d8106160 msgr2=0x7fe7d8197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7de02b640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7d8106160 0x7fe7d8197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7de02b640 1 -- 172.21.15.82:0/1807629442 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d819c9f0 con 0x7fe7d819bf00 2024-04-21T08:55:01.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7dd82a640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8101090 0x7fe7d8197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:01.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7de02b640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d819bf00 0x7fe7d819c3b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000b970 tx=0x7fe7d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:01.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7ceffd640 1 -- 172.21.15.82:0/1807629442 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0019070 con 0x7fe7d819bf00 2024-04-21T08:55:01.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7ceffd640 1 -- 172.21.15.82:0/1807629442 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d0014440 con 0x7fe7d819bf00 2024-04-21T08:55:01.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7ceffd640 1 -- 172.21.15.82:0/1807629442 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00075d0 con 0x7fe7d819bf00 2024-04-21T08:55:01.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/1807629442 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d819cce0 con 0x7fe7d819bf00 2024-04-21T08:55:01.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.144+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/1807629442 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d81a4480 con 0x7fe7d819bf00 2024-04-21T08:55:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.146+0000 7fe7ceffd640 1 -- 172.21.15.82:0/1807629442 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe7d001c070 con 0x7fe7d819bf00 2024-04-21T08:55:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7e267d740 1 Processor -- start 2024-04-21T08:55:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7e267d740 1 -- start start 2024-04-21T08:55:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7ceffd640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7c40a3d20 0x7fe7c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7e267d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 0x5645140b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7e267d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b4290 0x5645140bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7e267d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140bf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7e267d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564514017f00 con 0x5645140b38e0 2024-04-21T08:55:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7e267d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564513fb5590 con 0x5645140b4290 2024-04-21T08:55:01.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7e267d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564514013ec0 con 0x5645140bce10 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7dd82a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 0x5645140b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7dd029640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7c40a3d20 0x7fe7c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7dd82a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 0x5645140b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43312/0 (socket says 172.21.15.82:43312) 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7dd82a640 1 -- 172.21.15.82:0/2886249108 learned_addr learned my addr 172.21.15.82:0/2886249108 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.147+0000 7fe7ceffd640 1 -- 172.21.15.82:0/1807629442 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fe7d00d0140 con 0x7fe7d819bf00 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.148+0000 7fe7de02b640 1 --2- 172.21.15.82:0/2886249108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140bf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.148+0000 7fe7dd82a640 1 -- 172.21.15.82:0/2886249108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 msgr2=0x5645140bf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.148+0000 7fe7dd82a640 1 --2- 172.21.15.82:0/2886249108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140bf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.148+0000 7fe7dd82a640 1 -- 172.21.15.82:0/2886249108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b4290 msgr2=0x5645140bc840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.148+0000 7fe7dd82a640 1 --2- 172.21.15.82:0/2886249108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b4290 0x5645140bc840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.148+0000 7fe7dd82a640 1 -- 172.21.15.82:0/2886249108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564513df27e0 con 0x5645140b38e0 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.148+0000 7fe7dd029640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7c40a3d20 0x7fe7c40a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005e10 tx=0x7fe7d4005da0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:01.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.148+0000 7fe7dd82a640 1 --2- 172.21.15.82:0/2886249108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 0x5645140b3cc0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8036040 tx=0x7fe7c8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5e2bcc4fa3b1e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.148+0000 7fe7c3fff640 1 -- 172.21.15.82:0/2886249108 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8043020 con 0x5645140b38e0 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.148+0000 7fe7c3fff640 1 -- 172.21.15.82:0/2886249108 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c802cc90 con 0x5645140b38e0 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.148+0000 7fe7c3fff640 1 -- 172.21.15.82:0/2886249108 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80046c0 con 0x5645140b38e0 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.149+0000 7fe7e267d740 1 -- 172.21.15.82:0/2886249108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 msgr2=0x5645140b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.149+0000 7fe7e267d740 1 --2- 172.21.15.82:0/2886249108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 0x5645140b3cc0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8036040 tx=0x7fe7c8005e80 comp rx=0 tx=0).stop 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.149+0000 7fe7e267d740 1 -- 172.21.15.82:0/2886249108 shutdown_connections 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.149+0000 7fe7e267d740 1 --2- 172.21.15.82:0/2886249108 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140bf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.149+0000 7fe7e267d740 1 --2- 172.21.15.82:0/2886249108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b4290 0x5645140bc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.149+0000 7fe7e267d740 1 --2- 172.21.15.82:0/2886249108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 0x5645140b3cc0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.149+0000 7fe7e267d740 1 -- 172.21.15.82:0/2886249108 >> 172.21.15.82:0/2886249108 conn(0x5645140adfb0 msgr2=0x5645140ae390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.149+0000 7fe7e267d740 1 -- 172.21.15.82:0/2886249108 shutdown_connections 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.149+0000 7fe7e267d740 1 -- 172.21.15.82:0/2886249108 wait complete. 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.149+0000 7fe7e267d740 1 -- start start 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.149+0000 7fe7e267d740 1 Processor -- start 2024-04-21T08:55:01.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7e267d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b38e0 0x5645140b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7e267d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b4290 0x5645140ba360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7e267d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140ba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7e267d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5645140bfd20 con 0x5645140b4290 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7e267d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564513fd1580 con 0x5645140b38e0 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7e267d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5645140bfea0 con 0x5645140bce10 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7e267d740 10 client.0 _ll_register_callbacks cb 0x564513fd7650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7dd029640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b4290 0x5645140ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7dd029640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b4290 0x5645140ba360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43318/0 (socket says 172.21.15.82:43318) 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7dd029640 1 -- 172.21.15.82:0/4120272275 learned_addr learned my addr 172.21.15.82:0/4120272275 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7dd82a640 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b38e0 0x5645140b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7de02b640 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140ba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7dd029640 1 -- 172.21.15.82:0/4120272275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 msgr2=0x5645140ba8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7dd029640 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140ba8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7dd029640 1 -- 172.21.15.82:0/4120272275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b38e0 msgr2=0x5645140b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7dd029640 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b38e0 0x5645140b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7dd029640 1 -- 172.21.15.82:0/4120272275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564513e52f70 con 0x5645140b4290 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7dd82a640 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b38e0 0x5645140b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7dd029640 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b4290 0x5645140ba360 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe7d806b6e0 tx=0x7fe7d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44054f4a61505f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7c17fa640 1 -- 172.21.15.82:0/4120272275 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4028060 con 0x5645140b4290 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7c17fa640 1 -- 172.21.15.82:0/4120272275 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fe7d4010930 con 0x5645140b4290 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.150+0000 7fe7e267d740 1 -- 172.21.15.82:0/4120272275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c802cc90 con 0x5645140b4290 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.151+0000 7fe7c17fa640 1 -- 172.21.15.82:0/4120272275 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe7d402f050 con 0x5645140b4290 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.151+0000 7fe7e267d740 10 client.17967 fetch_fsmap learned FSMap version 42 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.151+0000 7fe7e267d740 1 -- 172.21.15.82:0/4120272275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564513cea170 con 0x5645140b4290 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.151+0000 7fe7c17fa640 1 -- 172.21.15.82:0/4120272275 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c802cc90 con 0x5645140b4290 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.151+0000 7fe7c17fa640 1 -- 172.21.15.82:0/4120272275 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7fe7d40189c0 con 0x5645140b4290 2024-04-21T08:55:01.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.151+0000 7fe7e267d740 10 client.17967 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.151+0000 7fe7e267d740 10 client.17967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.151+0000 7fe7e267d740 20 client.17967 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.151+0000 7fe7e267d740 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5645140cd370 0x5645140cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.151+0000 7fe7e267d740 4 client.17967 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.151+0000 7fe7e267d740 1 -- 172.21.15.82:0/4120272275 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564514025920 con 0x5645140cd370 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.151+0000 7fe7dd82a640 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5645140cd370 0x5645140cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.152+0000 7fe7c17fa640 10 client.17967 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.152+0000 7fe7dd82a640 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5645140cd370 0x5645140cf810 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ce7d376fc710a7 server_cookie=2ca4a2ce26e72aa3 in_seq=0 out_seq=0 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.152+0000 7fe7c17fa640 1 -- 172.21.15.82:0/4120272275 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564514025920 con 0x5645140cd370 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.152+0000 7fe7c17fa640 10 client.17967 handle_command_reply: tid=0 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.152+0000 7fe7e267d740 1 client.17967 shutdown 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.152+0000 7fe7e267d740 1 -- 172.21.15.82:0/4120272275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b4290 msgr2=0x5645140ba360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.152+0000 7fe7e267d740 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b4290 0x5645140ba360 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe7d806b6e0 tx=0x7fe7d4004730 comp rx=0 tx=0).stop 2024-04-21T08:55:01.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.153+0000 7fe7e267d740 1 -- 172.21.15.82:0/4120272275 shutdown_connections 2024-04-21T08:55:01.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.153+0000 7fe7e267d740 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5645140cd370 0x5645140cf810 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.153+0000 7fe7e267d740 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140ba8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.153+0000 7fe7e267d740 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b4290 0x5645140ba360 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.153+0000 7fe7c17fa640 0 client.17967 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:01.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.153+0000 7fe7e267d740 1 --2- 172.21.15.82:0/4120272275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b38e0 0x5645140b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.153+0000 7fe7e267d740 1 -- 172.21.15.82:0/4120272275 >> 172.21.15.82:0/4120272275 conn(0x5645140adfb0 msgr2=0x5645140afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:01.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.153+0000 7fe7e267d740 1 -- 172.21.15.82:0/4120272275 shutdown_connections 2024-04-21T08:55:01.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.153+0000 7fe7e267d740 1 -- 172.21.15.82:0/4120272275 wait complete. 2024-04-21T08:55:01.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.153+0000 7fe7e267d740 20 client.17967 trim_cache size 0 max 16384 2024-04-21T08:55:01.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.170+0000 7fe7e267d740 1 Processor -- start 2024-04-21T08:55:01.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.170+0000 7fe7e267d740 1 -- start start 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.170+0000 7fe7e267d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b38e0 0x5645140bc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.170+0000 7fe7e267d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140b4290 0x5645140b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.170+0000 7fe7e267d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140bce10 0x5645140c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.170+0000 7fe7e267d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81ad330 con 0x5645140bce10 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.170+0000 7fe7e267d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81aef60 con 0x5645140b38e0 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.170+0000 7fe7e267d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81ada00 con 0x5645140b4290 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.170+0000 7fe7de02b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140bce10 0x5645140c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.170+0000 7fe7de02b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140bce10 0x5645140c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43330/0 (socket says 172.21.15.82:43330) 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.170+0000 7fe7de02b640 1 -- 172.21.15.82:0/1226168081 learned_addr learned my addr 172.21.15.82:0/1226168081 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7de02b640 1 -- 172.21.15.82:0/1226168081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140b4290 msgr2=0x5645140b8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7de02b640 1 --2- 172.21.15.82:0/1226168081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140b4290 0x5645140b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7de02b640 1 -- 172.21.15.82:0/1226168081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b38e0 msgr2=0x5645140bc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7de02b640 1 --2- 172.21.15.82:0/1226168081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b38e0 0x5645140bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7de02b640 1 -- 172.21.15.82:0/1226168081 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645140b8370 con 0x5645140bce10 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7de02b640 1 --2- 172.21.15.82:0/1226168081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140bce10 0x5645140c8be0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0098240 tx=0x7fe7d00bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4b8f37a4242c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7c37fe640 1 -- 172.21.15.82:0/1226168081 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0065de0 con 0x5645140bce10 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7c37fe640 1 -- 172.21.15.82:0/1226168081 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d009bde0 con 0x5645140bce10 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7c37fe640 1 -- 172.21.15.82:0/1226168081 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00cd700 con 0x5645140bce10 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7e267d740 1 -- 172.21.15.82:0/1226168081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140bce10 msgr2=0x5645140c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7e267d740 1 --2- 172.21.15.82:0/1226168081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140bce10 0x5645140c8be0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0098240 tx=0x7fe7d00bc040 comp rx=0 tx=0).stop 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7e267d740 1 -- 172.21.15.82:0/1226168081 shutdown_connections 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7e267d740 1 --2- 172.21.15.82:0/1226168081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140bce10 0x5645140c8be0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7e267d740 1 --2- 172.21.15.82:0/1226168081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140b4290 0x5645140b8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7e267d740 1 --2- 172.21.15.82:0/1226168081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b38e0 0x5645140bc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.171+0000 7fe7e267d740 1 -- 172.21.15.82:0/1226168081 >> 172.21.15.82:0/1226168081 conn(0x5645140adfb0 msgr2=0x5645140ba670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:01.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7e267d740 1 -- 172.21.15.82:0/1226168081 shutdown_connections 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7e267d740 1 -- 172.21.15.82:0/1226168081 wait complete. 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7e267d740 1 -- start start 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7e267d740 1 Processor -- start 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7e267d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 0x5645140c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7e267d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b4290 0x5645140c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7e267d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7e267d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81ae860 con 0x5645140b38e0 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7e267d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81ae160 con 0x5645140b4290 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7e267d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d81ab300 con 0x5645140bce10 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7e267d740 10 client.0 _ll_register_callbacks cb 0x7fe7d4014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7dd82a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 0x5645140c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7dd82a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 0x5645140c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43346/0 (socket says 172.21.15.82:43346) 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.172+0000 7fe7dd82a640 1 -- 172.21.15.82:0/4124773894 learned_addr learned my addr 172.21.15.82:0/4124773894 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7de02b640 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7dd82a640 1 -- 172.21.15.82:0/4124773894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 msgr2=0x5645140c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7dd029640 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b4290 0x5645140c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7dd82a640 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7dd82a640 1 -- 172.21.15.82:0/4124773894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b4290 msgr2=0x5645140c15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7dd82a640 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b4290 0x5645140c15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7dd82a640 1 -- 172.21.15.82:0/4124773894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645140c20e0 con 0x5645140b38e0 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7de02b640 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7dd029640 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b4290 0x5645140c15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7dd82a640 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 0x5645140c5260 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8005dc0 tx=0x7fe7c802c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed39f4a73afeec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7c17fa640 1 -- 172.21.15.82:0/4124773894 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c804f030 con 0x5645140b38e0 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7c17fa640 1 -- 172.21.15.82:0/4124773894 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7fe7c802c7e0 con 0x5645140b38e0 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7e267d740 1 -- 172.21.15.82:0/4124773894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7d009bde0 con 0x5645140b38e0 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7c17fa640 1 -- 172.21.15.82:0/4124773894 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe7c805b050 con 0x5645140b38e0 2024-04-21T08:55:01.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7e267d740 10 client.17973 fetch_fsmap learned FSMap version 42 2024-04-21T08:55:01.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7e267d740 1 -- 172.21.15.82:0/4124773894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564514134790 con 0x5645140b38e0 2024-04-21T08:55:01.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7c17fa640 1 -- 172.21.15.82:0/4124773894 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7d009bde0 con 0x5645140b38e0 2024-04-21T08:55:01.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7c17fa640 1 -- 172.21.15.82:0/4124773894 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7fe7c8003d10 con 0x5645140b38e0 2024-04-21T08:55:01.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7e267d740 10 client.17973 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:55:01.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7e267d740 10 client.17973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:01.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7e267d740 20 client.17973 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:01.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7e267d740 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5645140cd370 0x564514136a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7e267d740 4 client.17973 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:55:01.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.173+0000 7fe7e267d740 1 -- 172.21.15.82:0/4124773894 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56451401f790 con 0x5645140cd370 2024-04-21T08:55:01.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.174+0000 7fe7dd029640 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5645140cd370 0x564514136a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.174+0000 7fe7c17fa640 10 client.17973 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:01.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.174+0000 7fe7dd029640 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5645140cd370 0x564514136a40 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76440632fed7a54c server_cookie=98a59f37605bd4aa in_seq=0 out_seq=0 2024-04-21T08:55:01.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7c17fa640 1 -- 172.21.15.82:0/4124773894 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56451401f790 con 0x5645140cd370 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7c17fa640 10 client.17973 handle_command_reply: tid=0 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7e267d740 1 client.17973 shutdown 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7e267d740 1 -- 172.21.15.82:0/4124773894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 msgr2=0x5645140c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7e267d740 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 0x5645140c5260 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8005dc0 tx=0x7fe7c802c720 comp rx=0 tx=0).stop 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7e267d740 1 -- 172.21.15.82:0/4124773894 shutdown_connections 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7e267d740 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5645140cd370 0x564514136a40 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7e267d740 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645140bce10 0x5645140c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7e267d740 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5645140b4290 0x5645140c15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7e267d740 1 --2- 172.21.15.82:0/4124773894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645140b38e0 0x5645140c5260 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7e267d740 1 -- 172.21.15.82:0/4124773894 >> 172.21.15.82:0/4124773894 conn(0x5645140adfb0 msgr2=0x5645140af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7c17fa640 0 client.17973 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7e267d740 1 -- 172.21.15.82:0/4124773894 shutdown_connections 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.175+0000 7fe7e267d740 1 -- 172.21.15.82:0/4124773894 wait complete. 2024-04-21T08:55:01.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7e267d740 20 client.17973 trim_cache size 0 max 16384 2024-04-21T08:55:01.182 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:55:01.182 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:55:01.182 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "e11808de-5d86-4148-86bf-1bda3f6c9296", 2024-04-21T08:55:01.182 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:55:01.182 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:55:01.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/1807629442 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7c40a3d20 msgr2=0x7fe7c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7dfa8e640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7c40a3d20 0x7fe7c40a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005e10 tx=0x7fe7d4005da0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/1807629442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d819bf00 msgr2=0x7fe7d819c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7dfa8e640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d819bf00 0x7fe7d819c3b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000b970 tx=0x7fe7d000be30 comp rx=0 tx=0).stop 2024-04-21T08:55:01.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/1807629442 shutdown_connections 2024-04-21T08:55:01.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7dfa8e640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7c40a3d20 0x7fe7c40a61e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7dfa8e640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d819bf00 0x7fe7d819c3b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7dfa8e640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7d8106160 0x7fe7d8197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7dfa8e640 1 --2- 172.21.15.82:0/1807629442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7d8101090 0x7fe7d8197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/1807629442 >> 172.21.15.82:0/1807629442 conn(0x7fe7d8077f50 msgr2=0x7fe7d80fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:01.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/1807629442 shutdown_connections 2024-04-21T08:55:01.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.176+0000 7fe7dfa8e640 1 -- 172.21.15.82:0/1807629442 wait complete. 2024-04-21T08:55:01.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:55:01.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:55:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.287+0000 7f83ac17e640 1 -- 172.21.15.82:0/2096488120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83a41011a0 msgr2=0x7f83a4101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.287+0000 7f83ac17e640 1 --2- 172.21.15.82:0/2096488120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83a41011a0 0x7f83a4101600 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8394009900 tx=0x7f839402bc30 comp rx=0 tx=0).stop 2024-04-21T08:55:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.287+0000 7f83ac17e640 1 -- 172.21.15.82:0/2096488120 shutdown_connections 2024-04-21T08:55:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.287+0000 7f83ac17e640 1 --2- 172.21.15.82:0/2096488120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a4101b40 0x7f83a410dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.287+0000 7f83ac17e640 1 --2- 172.21.15.82:0/2096488120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83a41011a0 0x7f83a4101600 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.287+0000 7f83ac17e640 1 --2- 172.21.15.82:0/2096488120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83a4106f90 0x7f83a4107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.287+0000 7f83ac17e640 1 -- 172.21.15.82:0/2096488120 >> 172.21.15.82:0/2096488120 conn(0x7f83a4076c50 msgr2=0x7f83a4077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.287+0000 7f83ac17e640 1 -- 172.21.15.82:0/2096488120 shutdown_connections 2024-04-21T08:55:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.287+0000 7f83ac17e640 1 -- 172.21.15.82:0/2096488120 wait complete. 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83ac17e640 1 Processor -- start 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83ac17e640 1 -- start start 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83ac17e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83a41011a0 0x7f83a4199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83ac17e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83a4101b40 0x7f83a4199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83ac17e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a4106f90 0x7f83a41a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83ac17e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41110e0 con 0x7f83a4101b40 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83ac17e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83a4110f60 con 0x7f83a4106f90 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83ac17e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83a4111260 con 0x7f83a41011a0 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83a9719640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83a4101b40 0x7f83a4199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83a9719640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83a4101b40 0x7f83a4199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43362/0 (socket says 172.21.15.82:43362) 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83a9719640 1 -- 172.21.15.82:0/265951929 learned_addr learned my addr 172.21.15.82:0/265951929 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83aa71b640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a4106f90 0x7f83a41a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83a9f1a640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83a41011a0 0x7f83a4199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83a9719640 1 -- 172.21.15.82:0/265951929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83a41011a0 msgr2=0x7f83a4199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83a9719640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83a41011a0 0x7f83a4199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83a9719640 1 -- 172.21.15.82:0/265951929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a4106f90 msgr2=0x7f83a41a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83a9719640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a4106f90 0x7f83a41a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83a9719640 1 -- 172.21.15.82:0/265951929 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a41a4bc0 con 0x7f83a4101b40 2024-04-21T08:55:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f83a9719640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83a4101b40 0x7f83a4199b40 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8394005940 tx=0x7f839402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.288+0000 7f839affd640 1 -- 172.21.15.82:0/265951929 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394043020 con 0x7f83a4101b40 2024-04-21T08:55:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.289+0000 7f839affd640 1 -- 172.21.15.82:0/265951929 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f839402cbe0 con 0x7f83a4101b40 2024-04-21T08:55:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.289+0000 7f83ac17e640 1 -- 172.21.15.82:0/265951929 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a41a4d90 con 0x7f83a4101b40 2024-04-21T08:55:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.289+0000 7f839affd640 1 -- 172.21.15.82:0/265951929 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394003bf0 con 0x7f83a4101b40 2024-04-21T08:55:01.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.289+0000 7f83ac17e640 1 -- 172.21.15.82:0/265951929 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a41a51f0 con 0x7f83a4101b40 2024-04-21T08:55:01.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f839affd640 1 -- 172.21.15.82:0/265951929 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8394046020 con 0x7f83a4101b40 2024-04-21T08:55:01.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f83aed6a740 1 Processor -- start 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f83aed6a740 1 -- start start 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f83aed6a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649d230 0x5637b649d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f83aed6a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649db50 0x5637b64aa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f839affd640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f83900a3d20 0x7f83900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f83aed6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b64aae90 0x5637b64ad280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f83aed6a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5637b64021e0 con 0x5637b649d230 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f83aed6a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5637b639f390 con 0x5637b649db50 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f83aed6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5637b63fe1a0 con 0x5637b64aae90 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f83aa71b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649d230 0x5637b649d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f83a9f1a640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f83900a3d20 0x7f83900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f83aa71b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649d230 0x5637b649d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43364/0 (socket says 172.21.15.82:43364) 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.291+0000 7f83aa71b640 1 -- 172.21.15.82:0/2106016401 learned_addr learned my addr 172.21.15.82:0/2106016401 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:01.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f83a9719640 1 --2- 172.21.15.82:0/2106016401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649db50 0x5637b64aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f839affd640 1 -- 172.21.15.82:0/265951929 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f83940f2e00 con 0x7f83a4101b40 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f83aa71b640 1 --2- 172.21.15.82:0/2106016401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b64aae90 0x5637b64ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f83aa71b640 1 -- 172.21.15.82:0/2106016401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b64aae90 msgr2=0x5637b64ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f83aa71b640 1 --2- 172.21.15.82:0/2106016401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b64aae90 0x5637b64ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f83aa71b640 1 -- 172.21.15.82:0/2106016401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649db50 msgr2=0x5637b64aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f83aa71b640 1 --2- 172.21.15.82:0/2106016401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649db50 0x5637b64aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f83aa71b640 1 -- 172.21.15.82:0/2106016401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b61de7d0 con 0x5637b649d230 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f83a9719640 1 --2- 172.21.15.82:0/2106016401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649db50 0x5637b64aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f83a9f1a640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f83900a3d20 0x7f83900a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f839c005fd0 tx=0x7f839c009200 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f83aa71b640 1 --2- 172.21.15.82:0/2106016401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649d230 0x5637b649d610 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f83a000cfd0 tx=0x7f83a000c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0f6fcc76af5a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f838ffff640 1 -- 172.21.15.82:0/2106016401 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0004020 con 0x5637b649d230 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f838ffff640 1 -- 172.21.15.82:0/2106016401 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a00041c0 con 0x5637b649d230 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.292+0000 7f838ffff640 1 -- 172.21.15.82:0/2106016401 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0003850 con 0x5637b649d230 2024-04-21T08:55:01.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.293+0000 7f83aed6a740 1 -- 172.21.15.82:0/2106016401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649d230 msgr2=0x5637b649d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.293+0000 7f83aed6a740 1 --2- 172.21.15.82:0/2106016401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649d230 0x5637b649d610 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f83a000cfd0 tx=0x7f83a000c530 comp rx=0 tx=0).stop 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.293+0000 7f83aed6a740 1 -- 172.21.15.82:0/2106016401 shutdown_connections 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.293+0000 7f83aed6a740 1 --2- 172.21.15.82:0/2106016401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b64aae90 0x5637b64ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.293+0000 7f83aed6a740 1 --2- 172.21.15.82:0/2106016401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649db50 0x5637b64aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.293+0000 7f83aed6a740 1 --2- 172.21.15.82:0/2106016401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649d230 0x5637b649d610 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.293+0000 7f83aed6a740 1 -- 172.21.15.82:0/2106016401 >> 172.21.15.82:0/2106016401 conn(0x5637b6497cb0 msgr2=0x5637b6498090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.293+0000 7f83aed6a740 1 -- 172.21.15.82:0/2106016401 shutdown_connections 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.293+0000 7f83aed6a740 1 -- 172.21.15.82:0/2106016401 wait complete. 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.293+0000 7f83aed6a740 1 -- start start 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.293+0000 7f83aed6a740 1 Processor -- start 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aed6a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 0x5637b64a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aed6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b649db50 0x5637b64a8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aed6a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 0x5637b64a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aed6a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5637b64adcd0 con 0x5637b64aae90 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aed6a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5637b63bb2f0 con 0x5637b649d230 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aed6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5637b64ade50 con 0x5637b649db50 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aed6a740 10 client.0 _ll_register_callbacks cb 0x5637b63c8580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aa71b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 0x5637b64a88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aa71b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 0x5637b64a88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43370/0 (socket says 172.21.15.82:43370) 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aa71b640 1 -- 172.21.15.82:0/3537237979 learned_addr learned my addr 172.21.15.82:0/3537237979 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83a9719640 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b649db50 0x5637b64a8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83a9f1a640 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 0x5637b64a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aa71b640 1 -- 172.21.15.82:0/3537237979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b649db50 msgr2=0x5637b64a8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aa71b640 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b649db50 0x5637b64a8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aa71b640 1 -- 172.21.15.82:0/3537237979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 msgr2=0x5637b64a7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aa71b640 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 0x5637b64a7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aa71b640 1 -- 172.21.15.82:0/3537237979 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b61143f0 con 0x5637b64aae90 2024-04-21T08:55:01.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83a9719640 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b649db50 0x5637b64a8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aa71b640 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 0x5637b64a88b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f83a0005d50 tx=0x7f83a00045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74f2c08c06ec2f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f838d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f838d7fa640 1 -- 172.21.15.82:0/3537237979 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0005460 con 0x5637b64aae90 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f838d7fa640 1 -- 172.21.15.82:0/3537237979 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f83a00046c0 con 0x5637b64aae90 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aed6a740 1 -- 172.21.15.82:0/3537237979 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a00041c0 con 0x5637b64aae90 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f838d7fa640 1 -- 172.21.15.82:0/3537237979 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f83a002f050 con 0x5637b64aae90 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aed6a740 10 client.17985 fetch_fsmap learned FSMap version 42 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.294+0000 7f83aed6a740 1 -- 172.21.15.82:0/3537237979 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5637b60d40f0 con 0x5637b64aae90 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.295+0000 7f838d7fa640 1 -- 172.21.15.82:0/3537237979 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a00041c0 con 0x5637b64aae90 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.295+0000 7f838d7fa640 1 -- 172.21.15.82:0/3537237979 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7f83a0003680 con 0x5637b64aae90 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.295+0000 7f83aed6a740 10 client.17985 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.295+0000 7f83aed6a740 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.295+0000 7f83aed6a740 20 client.17985 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:01.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.295+0000 7f83aed6a740 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5637b64badf0 0x5637b64bd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.295+0000 7f83aed6a740 4 client.17985 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.295+0000 7f83aed6a740 1 -- 172.21.15.82:0/3537237979 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637b640fc00 con 0x5637b64badf0 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.295+0000 7f83a9f1a640 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5637b64badf0 0x5637b64bd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f83a9f1a640 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5637b64badf0 0x5637b64bd1e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f91141afd43904 server_cookie=51e3df58472a33ae in_seq=0 out_seq=0 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f838d7fa640 10 client.17985 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f838d7fa640 1 -- 172.21.15.82:0/3537237979 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637b640fc00 con 0x5637b64badf0 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f838d7fa640 10 client.17985 handle_command_reply: tid=0 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f83aed6a740 1 client.17985 shutdown 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f83aed6a740 1 -- 172.21.15.82:0/3537237979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 msgr2=0x5637b64a88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f83aed6a740 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 0x5637b64a88b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f83a0005d50 tx=0x7f83a00045a0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f83aed6a740 1 -- 172.21.15.82:0/3537237979 shutdown_connections 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f83aed6a740 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5637b64badf0 0x5637b64bd1e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f83aed6a740 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 0x5637b64a88b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f838d7fa640 0 client.17985 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f83aed6a740 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b649db50 0x5637b64a8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f83aed6a740 1 --2- 172.21.15.82:0/3537237979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 0x5637b64a7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.296+0000 7f83aed6a740 1 -- 172.21.15.82:0/3537237979 >> 172.21.15.82:0/3537237979 conn(0x5637b6497cb0 msgr2=0x5637b6499a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.297+0000 7f83aed6a740 1 -- 172.21.15.82:0/3537237979 shutdown_connections 2024-04-21T08:55:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.297+0000 7f83aed6a740 1 -- 172.21.15.82:0/3537237979 wait complete. 2024-04-21T08:55:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.297+0000 7f83aed6a740 20 client.17985 trim_cache size 0 max 16384 2024-04-21T08:55:01.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aed6a740 1 Processor -- start 2024-04-21T08:55:01.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aed6a740 1 -- start start 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aed6a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 0x5637b64b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aed6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b649db50 0x5637b64a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aed6a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 0x5637b6496bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aed6a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41ad210 con 0x5637b64aae90 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aed6a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41aee40 con 0x5637b649d230 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aed6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41ad8e0 con 0x5637b649db50 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aa71b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 0x5637b6496bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aa71b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 0x5637b6496bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43382/0 (socket says 172.21.15.82:43382) 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aa71b640 1 -- 172.21.15.82:0/1054725068 learned_addr learned my addr 172.21.15.82:0/1054725068 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aa71b640 1 -- 172.21.15.82:0/1054725068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b649db50 msgr2=0x5637b64a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aa71b640 1 --2- 172.21.15.82:0/1054725068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b649db50 0x5637b64a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aa71b640 1 -- 172.21.15.82:0/1054725068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 msgr2=0x5637b64b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aa71b640 1 --2- 172.21.15.82:0/1054725068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 0x5637b64b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.314+0000 7f83aa71b640 1 -- 172.21.15.82:0/1054725068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b64bd550 con 0x5637b64aae90 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f83aa71b640 1 --2- 172.21.15.82:0/1054725068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 0x5637b6496bf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f83a0012a10 tx=0x7f83a000c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d7304b9beb3d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f838f7fe640 1 -- 172.21.15.82:0/1054725068 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0010c90 con 0x5637b64aae90 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f838f7fe640 1 -- 172.21.15.82:0/1054725068 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a00040f0 con 0x5637b64aae90 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f838f7fe640 1 -- 172.21.15.82:0/1054725068 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0005260 con 0x5637b64aae90 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f83aed6a740 1 -- 172.21.15.82:0/1054725068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 msgr2=0x5637b6496bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f83aed6a740 1 --2- 172.21.15.82:0/1054725068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 0x5637b6496bf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f83a0012a10 tx=0x7f83a000c510 comp rx=0 tx=0).stop 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f83aed6a740 1 -- 172.21.15.82:0/1054725068 shutdown_connections 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f83aed6a740 1 --2- 172.21.15.82:0/1054725068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b64aae90 0x5637b6496bf0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f83aed6a740 1 --2- 172.21.15.82:0/1054725068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b649db50 0x5637b64a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f83aed6a740 1 --2- 172.21.15.82:0/1054725068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 0x5637b64b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f83aed6a740 1 -- 172.21.15.82:0/1054725068 >> 172.21.15.82:0/1054725068 conn(0x5637b6497cb0 msgr2=0x5637b64964e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f83aed6a740 1 -- 172.21.15.82:0/1054725068 shutdown_connections 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.315+0000 7f83aed6a740 1 -- 172.21.15.82:0/1054725068 wait complete. 2024-04-21T08:55:01.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83aed6a740 1 -- start start 2024-04-21T08:55:01.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83aed6a740 1 Processor -- start 2024-04-21T08:55:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83aed6a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 0x5637b64b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83aed6a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649db50 0x5637b64b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83aed6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b64aae90 0x5637b64ae700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83aed6a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41ae740 con 0x5637b649db50 2024-04-21T08:55:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83aed6a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41ae040 con 0x5637b649d230 2024-04-21T08:55:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83aed6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41ab1e0 con 0x5637b64aae90 2024-04-21T08:55:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83aed6a740 10 client.0 _ll_register_callbacks cb 0x7f83a000ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83a9719640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649db50 0x5637b64b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83a9719640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649db50 0x5637b64b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43384/0 (socket says 172.21.15.82:43384) 2024-04-21T08:55:01.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83a9719640 1 -- 172.21.15.82:0/2839204642 learned_addr learned my addr 172.21.15.82:0/2839204642 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.316+0000 7f83aa71b640 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b64aae90 0x5637b64ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83a9719640 1 -- 172.21.15.82:0/2839204642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b64aae90 msgr2=0x5637b64ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83a9719640 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b64aae90 0x5637b64ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83a9719640 1 -- 172.21.15.82:0/2839204642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 msgr2=0x5637b64b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83a9f1a640 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 0x5637b64b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83a9719640 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 0x5637b64b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83a9719640 1 -- 172.21.15.82:0/2839204642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b64aeed0 con 0x5637b649db50 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83a9719640 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649db50 0x5637b64b36d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f83940f0300 tx=0x7f83940be5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce0a9cccfd768b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f838d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f838d7fa640 1 -- 172.21.15.82:0/2839204642 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940bf1a0 con 0x5637b649db50 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f838d7fa640 1 -- 172.21.15.82:0/2839204642 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9562+0+0 (secure 0 0 0) 0x7f83940bcda0 con 0x5637b649db50 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83aed6a740 1 -- 172.21.15.82:0/2839204642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a00040f0 con 0x5637b649db50 2024-04-21T08:55:01.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f838d7fa640 1 -- 172.21.15.82:0/2839204642 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f83940beaa0 con 0x5637b649db50 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83aed6a740 10 client.17991 fetch_fsmap learned FSMap version 42 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83aed6a740 1 -- 172.21.15.82:0/2839204642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5637b6520f30 con 0x5637b649db50 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f838d7fa640 1 -- 172.21.15.82:0/2839204642 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a00040f0 con 0x5637b649db50 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f838d7fa640 1 -- 172.21.15.82:0/2839204642 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) ==== 8390+0+0 (secure 0 0 0) 0x7f83940bb110 con 0x5637b649db50 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83aed6a740 10 client.17991 fetch_fsmap finished waiting for FSMap version 42 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83aed6a740 10 client.17991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83aed6a740 20 client.17991 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83aed6a740 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5637b65216e0 0x5637b6523b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83aed6a740 4 client.17991 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.317+0000 7f83aed6a740 1 -- 172.21.15.82:0/2839204642 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5637b6409a70 con 0x5637b65216e0 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83a9f1a640 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5637b65216e0 0x5637b6523b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f838d7fa640 10 client.17991 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83a9f1a640 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5637b65216e0 0x5637b6523b00 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39c5c13ae50bca9d server_cookie=9bcb0b946fa0c1c6 in_seq=0 out_seq=0 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f838d7fa640 1 -- 172.21.15.82:0/2839204642 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5637b6409a70 con 0x5637b65216e0 2024-04-21T08:55:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f838d7fa640 10 client.17991 handle_command_reply: tid=0 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83aed6a740 1 client.17991 shutdown 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83aed6a740 1 -- 172.21.15.82:0/2839204642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649db50 msgr2=0x5637b64b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83aed6a740 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649db50 0x5637b64b36d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f83940f0300 tx=0x7f83940be5e0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83aed6a740 1 -- 172.21.15.82:0/2839204642 shutdown_connections 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83aed6a740 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5637b65216e0 0x5637b6523b00 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83aed6a740 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5637b64aae90 0x5637b64ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83aed6a740 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637b649db50 0x5637b64b36d0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83aed6a740 1 --2- 172.21.15.82:0/2839204642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637b649d230 0x5637b64b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f838d7fa640 0 client.17991 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83aed6a740 1 -- 172.21.15.82:0/2839204642 >> 172.21.15.82:0/2839204642 conn(0x5637b6497cb0 msgr2=0x5637b6498eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83aed6a740 1 -- 172.21.15.82:0/2839204642 shutdown_connections 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83aed6a740 1 -- 172.21.15.82:0/2839204642 wait complete. 2024-04-21T08:55:01.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.318+0000 7f83aed6a740 20 client.17991 trim_cache size 0 max 16384 2024-04-21T08:55:01.325 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:55:01.325 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:55:01.325 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:55:01.325 INFO:teuthology.orchestra.run.smithi082.stdout: "e11808de-5d86-4148-86bf-1bda3f6c9296": { 2024-04-21T08:55:01.325 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:55:01.325 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "e11808de-5d86-4148-86bf-1bda3f6c9296", 2024-04-21T08:55:01.325 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:55:01.325 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:55:01.325 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:55:01.325 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:55:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.319+0000 7f83ac17e640 1 -- 172.21.15.82:0/265951929 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f83900a3d20 msgr2=0x7f83900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.319+0000 7f83ac17e640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f83900a3d20 0x7f83900a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f839c005fd0 tx=0x7f839c009200 comp rx=0 tx=0).stop 2024-04-21T08:55:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.319+0000 7f83ac17e640 1 -- 172.21.15.82:0/265951929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83a4101b40 msgr2=0x7f83a4199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.319+0000 7f83ac17e640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83a4101b40 0x7f83a4199b40 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8394005940 tx=0x7f839402c7e0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.319+0000 7f83ac17e640 1 -- 172.21.15.82:0/265951929 shutdown_connections 2024-04-21T08:55:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.319+0000 7f83ac17e640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83a4106f90 0x7f83a41a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.319+0000 7f83ac17e640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f83900a3d20 0x7f83900a61e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.319+0000 7f83ac17e640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83a4101b40 0x7f83a4199b40 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.319+0000 7f83ac17e640 1 --2- 172.21.15.82:0/265951929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83a41011a0 0x7f83a4199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.319+0000 7f83ac17e640 1 -- 172.21.15.82:0/265951929 >> 172.21.15.82:0/265951929 conn(0x7f83a4076c50 msgr2=0x7f83a4100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.319+0000 7f83ac17e640 1 -- 172.21.15.82:0/265951929 shutdown_connections 2024-04-21T08:55:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:01.319+0000 7f83ac17e640 1 -- 172.21.15.82:0/265951929 wait complete. 2024-04-21T08:55:01.330 INFO:tasks.cephfs.filesystem:scrub status for tag:e11808de-5d86-4148-86bf-1bda3f6c9296 - {'path': '/', 'tag': 'e11808de-5d86-4148-86bf-1bda3f6c9296', 'options': 'recursive,force'} 2024-04-21T08:55:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:02 smithi134 ceph-mon[31772]: pgmap v245: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 437 KiB/s wr, 168 op/s 2024-04-21T08:55:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:02 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:55:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:02 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:55:02.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:02 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:55:02.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:02 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:55:02.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:02 smithi082 ceph-mon[27967]: pgmap v245: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 437 KiB/s wr, 168 op/s 2024-04-21T08:55:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:02 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:55:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:02 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:55:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:02 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:55:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:02 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:55:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:02 smithi106 ceph-mon[31072]: pgmap v245: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 437 KiB/s wr, 168 op/s 2024-04-21T08:55:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:02 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:55:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:02 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:55:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:02 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:55:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:02 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:55:04.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:04 smithi134 ceph-mon[31772]: pgmap v246: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 485 KiB/s wr, 192 op/s 2024-04-21T08:55:04.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:04 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:55:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:04 smithi082 ceph-mon[27967]: pgmap v246: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 485 KiB/s wr, 192 op/s 2024-04-21T08:55:04.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:04 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:55:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:04 smithi106 ceph-mon[31072]: pgmap v246: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 485 KiB/s wr, 192 op/s 2024-04-21T08:55:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:04 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:55:06.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:06 smithi134 ceph-mon[31772]: pgmap v247: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-04-21T08:55:06.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:06 smithi082 ceph-mon[27967]: pgmap v247: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-04-21T08:55:06.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:06 smithi106 ceph-mon[31072]: pgmap v247: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-04-21T08:55:08.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:08 smithi134 ceph-mon[31772]: pgmap v248: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 31 op/s 2024-04-21T08:55:08.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:08 smithi082 ceph-mon[27967]: pgmap v248: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 31 op/s 2024-04-21T08:55:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:08 smithi106 ceph-mon[31072]: pgmap v248: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 31 op/s 2024-04-21T08:55:10.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:10 smithi082 ceph-mon[27967]: pgmap v249: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 30 op/s 2024-04-21T08:55:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:10 smithi082 ceph-mon[27967]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T08:55:10.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:10 smithi106 ceph-mon[31072]: pgmap v249: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 30 op/s 2024-04-21T08:55:10.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:10 smithi106 ceph-mon[31072]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T08:55:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:10 smithi134 ceph-mon[31772]: pgmap v249: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 61 KiB/s wr, 30 op/s 2024-04-21T08:55:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:10 smithi134 ceph-mon[31772]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T08:55:12.528 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:12 smithi134 ceph-mon[31772]: pgmap v251: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-04-21T08:55:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:12 smithi082 ceph-mon[27967]: pgmap v251: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-04-21T08:55:12.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:12 smithi106 ceph-mon[31072]: pgmap v251: 129 pgs: 129 active+clean; 889 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-04-21T08:55:13.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:13 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:55:13.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:13 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:55:13.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:13 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:55:14.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:14 smithi082 ceph-mon[27967]: pgmap v252: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 887 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-21T08:55:14.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:14 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:55:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:14 smithi106 ceph-mon[31072]: pgmap v252: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 887 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-21T08:55:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:14 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:55:14.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:14 smithi134 ceph-mon[31772]: pgmap v252: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 887 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-21T08:55:14.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:14 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:55:15.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:15 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:55:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:15 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:55:15.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:15 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:55:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:15 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:55:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:15 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:55:15.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:15 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:55:15.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:15 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:55:15.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:15 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:55:15.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:15 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:55:16.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:16 smithi082 ceph-mon[27967]: pgmap v253: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 887 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-21T08:55:16.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:16 smithi106 ceph-mon[31072]: pgmap v253: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 887 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-21T08:55:16.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:16 smithi134 ceph-mon[31772]: pgmap v253: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 887 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-21T08:55:18.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:18 smithi082 ceph-mon[27967]: pgmap v254: 129 pgs: 12 active+clean+snaptrim, 74 active+clean, 43 active+clean+snaptrim_wait; 886 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-21T08:55:18.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:18 smithi106 ceph-mon[31072]: pgmap v254: 129 pgs: 12 active+clean+snaptrim, 74 active+clean, 43 active+clean+snaptrim_wait; 886 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-21T08:55:18.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:18 smithi134 ceph-mon[31772]: pgmap v254: 129 pgs: 12 active+clean+snaptrim, 74 active+clean, 43 active+clean+snaptrim_wait; 886 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-21T08:55:20.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:20 smithi082 ceph-mon[27967]: pgmap v255: 129 pgs: 10 active+clean+snaptrim, 84 active+clean, 35 active+clean+snaptrim_wait; 884 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-21T08:55:20.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:20 smithi106 ceph-mon[31072]: pgmap v255: 129 pgs: 10 active+clean+snaptrim, 84 active+clean, 35 active+clean+snaptrim_wait; 884 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-21T08:55:20.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:20 smithi134 ceph-mon[31772]: pgmap v255: 129 pgs: 10 active+clean+snaptrim, 84 active+clean, 35 active+clean+snaptrim_wait; 884 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-21T08:55:22.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:22 smithi082 ceph-mon[27967]: pgmap v256: 129 pgs: 10 active+clean+snaptrim, 84 active+clean, 35 active+clean+snaptrim_wait; 884 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 797 B/s wr, 10 op/s 2024-04-21T08:55:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:22 smithi106 ceph-mon[31072]: pgmap v256: 129 pgs: 10 active+clean+snaptrim, 84 active+clean, 35 active+clean+snaptrim_wait; 884 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 797 B/s wr, 10 op/s 2024-04-21T08:55:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:22 smithi134 ceph-mon[31772]: pgmap v256: 129 pgs: 10 active+clean+snaptrim, 84 active+clean, 35 active+clean+snaptrim_wait; 884 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 797 B/s wr, 10 op/s 2024-04-21T08:55:24.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:24 smithi082 ceph-mon[27967]: pgmap v257: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 14 op/s 2024-04-21T08:55:24.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:24 smithi106 ceph-mon[31072]: pgmap v257: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 14 op/s 2024-04-21T08:55:24.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:24 smithi134 ceph-mon[31772]: pgmap v257: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 14 op/s 2024-04-21T08:55:26.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:26 smithi134 ceph-mon[31772]: pgmap v258: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-21T08:55:26.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:26 smithi134 ceph-mon[31772]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T08:55:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:26 smithi082 ceph-mon[27967]: pgmap v258: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-21T08:55:26.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:26 smithi082 ceph-mon[27967]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T08:55:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:26 smithi106 ceph-mon[31072]: pgmap v258: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-21T08:55:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:26 smithi106 ceph-mon[31072]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T08:55:28.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:28 smithi134 ceph-mon[31772]: pgmap v260: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-21T08:55:28.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:28 smithi082 ceph-mon[27967]: pgmap v260: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-21T08:55:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:28 smithi106 ceph-mon[31072]: pgmap v260: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-04-21T08:55:29.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:29 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:55:29.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:29 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:55:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:29 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:55:30.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:30 smithi134 ceph-mon[31772]: pgmap v261: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:55:30.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:30 smithi082 ceph-mon[27967]: pgmap v261: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:55:30.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:30 smithi106 ceph-mon[31072]: pgmap v261: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:55:31.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:55:31.332 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:55:31.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.443+0000 7fc8826f2640 1 -- 172.21.15.82:0/2390508985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc87c1031b0 msgr2=0x7fc87c103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.443+0000 7fc8826f2640 1 --2- 172.21.15.82:0/2390508985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc87c1031b0 0x7fc87c103590 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009900 tx=0x7fc86c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.443+0000 7fc8826f2640 1 -- 172.21.15.82:0/2390508985 shutdown_connections 2024-04-21T08:55:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.443+0000 7fc8826f2640 1 --2- 172.21.15.82:0/2390508985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c10c580 0x7fc87c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.443+0000 7fc8826f2640 1 --2- 172.21.15.82:0/2390508985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc87c103ad0 0x7fc87c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.443+0000 7fc8826f2640 1 --2- 172.21.15.82:0/2390508985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc87c1031b0 0x7fc87c103590 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.443+0000 7fc8826f2640 1 -- 172.21.15.82:0/2390508985 >> 172.21.15.82:0/2390508985 conn(0x7fc87c0771d0 msgr2=0x7fc87c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.443+0000 7fc8826f2640 1 -- 172.21.15.82:0/2390508985 shutdown_connections 2024-04-21T08:55:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.443+0000 7fc8826f2640 1 -- 172.21.15.82:0/2390508985 wait complete. 2024-04-21T08:55:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.444+0000 7fc8826f2640 1 Processor -- start 2024-04-21T08:55:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.444+0000 7fc8826f2640 1 -- start start 2024-04-21T08:55:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.444+0000 7fc8826f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c1031b0 0x7fc87c100e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.444+0000 7fc8826f2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc87c103ad0 0x7fc87c101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.444+0000 7fc8826f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc87c10c580 0x7fc87c101880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.444+0000 7fc8826f2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c112220 con 0x7fc87c10c580 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.444+0000 7fc8826f2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1120a0 con 0x7fc87c103ad0 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.444+0000 7fc8826f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1123a0 con 0x7fc87c1031b0 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.444+0000 7fc880c8f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc87c10c580 0x7fc87c101880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc880c8f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc87c10c580 0x7fc87c101880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42570/0 (socket says 172.21.15.82:42570) 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc880c8f640 1 -- 172.21.15.82:0/665368200 learned_addr learned my addr 172.21.15.82:0/665368200 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc87b7fe640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc87c103ad0 0x7fc87c101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc87bfff640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c1031b0 0x7fc87c100e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc880c8f640 1 -- 172.21.15.82:0/665368200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c1031b0 msgr2=0x7fc87c100e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc880c8f640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c1031b0 0x7fc87c100e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc880c8f640 1 -- 172.21.15.82:0/665368200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc87c103ad0 msgr2=0x7fc87c101340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc880c8f640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc87c103ad0 0x7fc87c101340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc880c8f640 1 -- 172.21.15.82:0/665368200 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c1a6d20 con 0x7fc87c10c580 2024-04-21T08:55:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc880c8f640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc87c10c580 0x7fc87c101880 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b9e0 tx=0x7fc87400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc8797fa640 1 -- 172.21.15.82:0/665368200 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874019070 con 0x7fc87c10c580 2024-04-21T08:55:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc8826f2640 1 -- 172.21.15.82:0/665368200 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87c1a7010 con 0x7fc87c10c580 2024-04-21T08:55:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc8797fa640 1 -- 172.21.15.82:0/665368200 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc874014440 con 0x7fc87c10c580 2024-04-21T08:55:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc8797fa640 1 -- 172.21.15.82:0/665368200 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8740125a0 con 0x7fc87c10c580 2024-04-21T08:55:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.445+0000 7fc8826f2640 1 -- 172.21.15.82:0/665368200 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87c1a7530 con 0x7fc87c10c580 2024-04-21T08:55:31.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.447+0000 7fc8797fa640 1 -- 172.21.15.82:0/665368200 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc87401c070 con 0x7fc87c10c580 2024-04-21T08:55:31.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc8853c3740 1 Processor -- start 2024-04-21T08:55:31.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc8853c3740 1 -- start start 2024-04-21T08:55:31.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc8797fa640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc8640a3d20 0x7fc8640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc8853c3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 0x55ed432b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc8853c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3ea0 0x55ed432bc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc8853c3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432bca80 0x55ed432bee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc8853c3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ed432181e0 con 0x55ed432b3580 2024-04-21T08:55:31.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc8853c3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ed431b5390 con 0x55ed432bca80 2024-04-21T08:55:31.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc8853c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ed432141a0 con 0x55ed432b3ea0 2024-04-21T08:55:31.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc87b7fe640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc8640a3d20 0x7fc8640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 0x55ed432b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 0x55ed432b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42584/0 (socket says 172.21.15.82:42584) 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc87bfff640 1 -- 172.21.15.82:0/3033297273 learned_addr learned my addr 172.21.15.82:0/3033297273 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc8797fa640 1 -- 172.21.15.82:0/665368200 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fc8740d0040 con 0x7fc87c10c580 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc880c8f640 1 --2- 172.21.15.82:0/3033297273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432bca80 0x55ed432bee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc87bfff640 1 -- 172.21.15.82:0/3033297273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3ea0 msgr2=0x55ed432bc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc87bfff640 1 --2- 172.21.15.82:0/3033297273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3ea0 0x55ed432bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc87bfff640 1 -- 172.21.15.82:0/3033297273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432bca80 msgr2=0x55ed432bee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc87bfff640 1 --2- 172.21.15.82:0/3033297273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432bca80 0x55ed432bee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc87bfff640 1 -- 172.21.15.82:0/3033297273 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed42ff47d0 con 0x55ed432b3580 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc880c8f640 1 --2- 172.21.15.82:0/3033297273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432bca80 0x55ed432bee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.448+0000 7fc87b7fe640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc8640a3d20 0x7fc8640a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc87c102890 tx=0x7fc87000a6f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.449+0000 7fc87bfff640 1 --2- 172.21.15.82:0/3033297273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 0x55ed432b3960 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009900 tx=0x7fc86c0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748c0f155c361c82 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.449+0000 7fc8627fc640 1 -- 172.21.15.82:0/3033297273 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0042a0 con 0x55ed432b3580 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.449+0000 7fc8627fc640 1 -- 172.21.15.82:0/3033297273 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc86c004440 con 0x55ed432b3580 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.449+0000 7fc8627fc640 1 -- 172.21.15.82:0/3033297273 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0046c0 con 0x55ed432b3580 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.449+0000 7fc8853c3740 1 -- 172.21.15.82:0/3033297273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 msgr2=0x55ed432b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.449+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3033297273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 0x55ed432b3960 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009900 tx=0x7fc86c0040f0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.449+0000 7fc8853c3740 1 -- 172.21.15.82:0/3033297273 shutdown_connections 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.449+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3033297273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432bca80 0x55ed432bee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.449+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3033297273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3ea0 0x55ed432bc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.449+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3033297273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 0x55ed432b3960 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.449+0000 7fc8853c3740 1 -- 172.21.15.82:0/3033297273 >> 172.21.15.82:0/3033297273 conn(0x55ed432adcb0 msgr2=0x55ed432ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.449+0000 7fc8853c3740 1 -- 172.21.15.82:0/3033297273 shutdown_connections 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc8853c3740 1 -- 172.21.15.82:0/3033297273 wait complete. 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc8853c3740 1 -- start start 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc8853c3740 1 Processor -- start 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc8853c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3580 0x55ed432b9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc8853c3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432b3ea0 0x55ed432b9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc8853c3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432bca80 0x55ed432ba400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc8853c3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ed432bf810 con 0x55ed432bca80 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc8853c3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ed431d12f0 con 0x55ed432b3ea0 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc8853c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ed432bf990 con 0x55ed432b3580 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc8853c3740 10 client.0 _ll_register_callbacks cb 0x55ed431de580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc880c8f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432bca80 0x55ed432ba400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc880c8f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432bca80 0x55ed432ba400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42594/0 (socket says 172.21.15.82:42594) 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.450+0000 7fc880c8f640 1 -- 172.21.15.82:0/3998186507 learned_addr learned my addr 172.21.15.82:0/3998186507 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc87bfff640 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3580 0x55ed432b9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc87b7fe640 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432b3ea0 0x55ed432b9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc880c8f640 1 -- 172.21.15.82:0/3998186507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3580 msgr2=0x55ed432b9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc880c8f640 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3580 0x55ed432b9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc880c8f640 1 -- 172.21.15.82:0/3998186507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432b3ea0 msgr2=0x55ed432b9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc880c8f640 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432b3ea0 0x55ed432b9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc880c8f640 1 -- 172.21.15.82:0/3998186507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed42f2a3f0 con 0x55ed432bca80 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc87b7fe640 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432b3ea0 0x55ed432b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc880c8f640 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432bca80 0x55ed432ba400 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b9b0 tx=0x7fc87409bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f9cdec96964aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc85b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc85b7fe640 1 -- 172.21.15.82:0/3998186507 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874014930 con 0x55ed432bca80 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc85b7fe640 1 -- 172.21.15.82:0/3998186507 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fc874095a90 con 0x55ed432bca80 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc8853c3740 1 -- 172.21.15.82:0/3998186507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc86c004440 con 0x55ed432bca80 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc85b7fe640 1 -- 172.21.15.82:0/3998186507 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc874014ad0 con 0x55ed432bca80 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc8853c3740 10 client.18003 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc8853c3740 1 -- 172.21.15.82:0/3998186507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ed42eea0f0 con 0x55ed432bca80 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.451+0000 7fc85b7fe640 1 -- 172.21.15.82:0/3998186507 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc86c004440 con 0x55ed432bca80 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.452+0000 7fc85b7fe640 1 -- 172.21.15.82:0/3998186507 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fc8740a44c0 con 0x55ed432bca80 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.452+0000 7fc8853c3740 10 client.18003 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:31.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.452+0000 7fc8853c3740 10 client.18003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.452+0000 7fc8853c3740 20 client.18003 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.452+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ed432cc8a0 0x55ed432ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.452+0000 7fc8853c3740 4 client.18003 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.452+0000 7fc8853c3740 1 -- 172.21.15.82:0/3998186507 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed43225c00 con 0x55ed432cc8a0 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.452+0000 7fc87bfff640 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ed432cc8a0 0x55ed432ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.452+0000 7fc85b7fe640 10 client.18003 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.452+0000 7fc87bfff640 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ed432cc8a0 0x55ed432ced20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8715344565e302b2 server_cookie=fa46cb2f12739b71 in_seq=0 out_seq=0 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.453+0000 7fc85b7fe640 1 -- 172.21.15.82:0/3998186507 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed43225c00 con 0x55ed432cc8a0 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.453+0000 7fc85b7fe640 10 client.18003 handle_command_reply: tid=0 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.453+0000 7fc8853c3740 1 client.18003 shutdown 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.453+0000 7fc8853c3740 1 -- 172.21.15.82:0/3998186507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432bca80 msgr2=0x55ed432ba400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.453+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432bca80 0x55ed432ba400 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b9b0 tx=0x7fc87409bd90 comp rx=0 tx=0).stop 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.453+0000 7fc8853c3740 1 -- 172.21.15.82:0/3998186507 shutdown_connections 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.453+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ed432cc8a0 0x55ed432ced20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.453+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432bca80 0x55ed432ba400 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.453+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432b3ea0 0x55ed432b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.453+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3998186507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3580 0x55ed432b9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.453+0000 7fc85b7fe640 0 client.18003 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:31.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.453+0000 7fc8853c3740 1 -- 172.21.15.82:0/3998186507 >> 172.21.15.82:0/3998186507 conn(0x55ed432adcb0 msgr2=0x55ed432afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.454+0000 7fc8853c3740 1 -- 172.21.15.82:0/3998186507 shutdown_connections 2024-04-21T08:55:31.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.454+0000 7fc8853c3740 1 -- 172.21.15.82:0/3998186507 wait complete. 2024-04-21T08:55:31.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.454+0000 7fc8853c3740 20 client.18003 trim_cache size 0 max 16384 2024-04-21T08:55:31.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc8853c3740 1 Processor -- start 2024-04-21T08:55:31.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc8853c3740 1 -- start start 2024-04-21T08:55:31.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc8853c3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 0x55ed432bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc8853c3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432b3ea0 0x55ed432c8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc8853c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432bca80 0x55ed432b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc8853c3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1b1620 con 0x55ed432b3580 2024-04-21T08:55:31.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc8853c3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1b31f0 con 0x55ed432b3ea0 2024-04-21T08:55:31.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc8853c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1b1cf0 con 0x55ed432bca80 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 0x55ed432bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 0x55ed432bc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42598/0 (socket says 172.21.15.82:42598) 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc87bfff640 1 -- 172.21.15.82:0/3330217795 learned_addr learned my addr 172.21.15.82:0/3330217795 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc87bfff640 1 -- 172.21.15.82:0/3330217795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432bca80 msgr2=0x55ed432b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc87bfff640 1 --2- 172.21.15.82:0/3330217795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432bca80 0x55ed432b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc87bfff640 1 -- 172.21.15.82:0/3330217795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432b3ea0 msgr2=0x55ed432c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc87bfff640 1 --2- 172.21.15.82:0/3330217795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432b3ea0 0x55ed432c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.471+0000 7fc87bfff640 1 -- 172.21.15.82:0/3330217795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed432b7ed0 con 0x55ed432b3580 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc87bfff640 1 --2- 172.21.15.82:0/3330217795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 0x55ed432bc0b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc86c000c00 tx=0x7fc86c02bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea60e252d3d565e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc861ffb640 1 -- 172.21.15.82:0/3330217795 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c02c900 con 0x55ed432b3580 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc861ffb640 1 -- 172.21.15.82:0/3330217795 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc86c050070 con 0x55ed432b3580 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc861ffb640 1 -- 172.21.15.82:0/3330217795 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c050480 con 0x55ed432b3580 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc8853c3740 1 -- 172.21.15.82:0/3330217795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 msgr2=0x55ed432bc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3330217795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 0x55ed432bc0b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc86c000c00 tx=0x7fc86c02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc8853c3740 1 -- 172.21.15.82:0/3330217795 shutdown_connections 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3330217795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432bca80 0x55ed432b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3330217795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432b3ea0 0x55ed432c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc8853c3740 1 --2- 172.21.15.82:0/3330217795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3580 0x55ed432bc0b0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc8853c3740 1 -- 172.21.15.82:0/3330217795 >> 172.21.15.82:0/3330217795 conn(0x55ed432adcb0 msgr2=0x55ed432b9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc8853c3740 1 -- 172.21.15.82:0/3330217795 shutdown_connections 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.472+0000 7fc8853c3740 1 -- 172.21.15.82:0/3330217795 wait complete. 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc8853c3740 1 -- start start 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc8853c3740 1 Processor -- start 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc8853c3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3580 0x55ed432c0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc8853c3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3ea0 0x55ed432c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc8853c3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432bca80 0x55ed432c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc8853c3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1b2af0 con 0x55ed432b3ea0 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc8853c3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1b23f0 con 0x55ed432bca80 2024-04-21T08:55:31.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc8853c3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1af5f0 con 0x55ed432b3580 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc8853c3740 10 client.0 _ll_register_callbacks cb 0x7fc874073c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc87b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3ea0 0x55ed432c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc87b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3ea0 0x55ed432c0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42608/0 (socket says 172.21.15.82:42608) 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc87b7fe640 1 -- 172.21.15.82:0/1511382142 learned_addr learned my addr 172.21.15.82:0/1511382142 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.473+0000 7fc880c8f640 1 --2- 172.21.15.82:0/1511382142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432bca80 0x55ed432c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc87b7fe640 1 -- 172.21.15.82:0/1511382142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3580 msgr2=0x55ed432c0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc87b7fe640 1 --2- 172.21.15.82:0/1511382142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3580 0x55ed432c0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc87b7fe640 1 -- 172.21.15.82:0/1511382142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432bca80 msgr2=0x55ed432c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc87b7fe640 1 --2- 172.21.15.82:0/1511382142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432bca80 0x55ed432c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc87b7fe640 1 -- 172.21.15.82:0/1511382142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed43331ed0 con 0x55ed432b3ea0 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc87b7fe640 1 --2- 172.21.15.82:0/1511382142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3ea0 0x55ed432c0b80 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc87000a310 tx=0x7fc8700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96a47b55b6e5c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc85b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc85b7fe640 1 -- 172.21.15.82:0/1511382142 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870018070 con 0x55ed432b3ea0 2024-04-21T08:55:31.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc85b7fe640 1 -- 172.21.15.82:0/1511382142 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fc87000eb40 con 0x55ed432b3ea0 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc8853c3740 1 -- 172.21.15.82:0/1511382142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc86c050070 con 0x55ed432b3ea0 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc85b7fe640 1 -- 172.21.15.82:0/1511382142 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc87002f050 con 0x55ed432b3ea0 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc8853c3740 10 client.18009 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc8853c3740 1 -- 172.21.15.82:0/1511382142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ed43333230 con 0x55ed432b3ea0 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc85b7fe640 1 -- 172.21.15.82:0/1511382142 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc86c050070 con 0x55ed432b3ea0 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc85b7fe640 1 -- 172.21.15.82:0/1511382142 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fc87001b070 con 0x55ed432b3ea0 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc8853c3740 10 client.18009 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc8853c3740 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc8853c3740 20 client.18009 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc8853c3740 1 --2- 172.21.15.82:0/1511382142 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ed43333a10 0x55ed43335e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc8853c3740 4 client.18009 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.474+0000 7fc8853c3740 1 -- 172.21.15.82:0/1511382142 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed4321fa70 con 0x55ed43333a10 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc87bfff640 1 --2- 172.21.15.82:0/1511382142 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ed43333a10 0x55ed43335e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc87bfff640 1 --2- 172.21.15.82:0/1511382142 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ed43333a10 0x55ed43335e30 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c233f9583ba17321 server_cookie=e62d41103163bcdd in_seq=0 out_seq=0 2024-04-21T08:55:31.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc85b7fe640 10 client.18009 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc85b7fe640 1 -- 172.21.15.82:0/1511382142 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ed4321fa70 con 0x55ed43333a10 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc85b7fe640 10 client.18009 handle_command_reply: tid=0 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc8853c3740 1 client.18009 shutdown 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc8853c3740 1 -- 172.21.15.82:0/1511382142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3ea0 msgr2=0x55ed432c0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc8853c3740 1 --2- 172.21.15.82:0/1511382142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3ea0 0x55ed432c0b80 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc87000a310 tx=0x7fc8700046c0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc8853c3740 1 -- 172.21.15.82:0/1511382142 shutdown_connections 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc8853c3740 1 --2- 172.21.15.82:0/1511382142 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55ed43333a10 0x55ed43335e30 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc8853c3740 1 --2- 172.21.15.82:0/1511382142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ed432bca80 0x55ed432c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc8853c3740 1 --2- 172.21.15.82:0/1511382142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed432b3ea0 0x55ed432c0b80 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc85b7fe640 0 client.18009 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc8853c3740 1 --2- 172.21.15.82:0/1511382142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed432b3580 0x55ed432c0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.475+0000 7fc8853c3740 1 -- 172.21.15.82:0/1511382142 >> 172.21.15.82:0/1511382142 conn(0x55ed432adcb0 msgr2=0x55ed432b8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8853c3740 1 -- 172.21.15.82:0/1511382142 shutdown_connections 2024-04-21T08:55:31.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8853c3740 1 -- 172.21.15.82:0/1511382142 wait complete. 2024-04-21T08:55:31.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8853c3740 20 client.18009 trim_cache size 0 max 16384 2024-04-21T08:55:31.482 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:55:31.482 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:55:31.482 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:55:31.482 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:55:31.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8826f2640 1 -- 172.21.15.82:0/665368200 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc8640a3d20 msgr2=0x7fc8640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8826f2640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc8640a3d20 0x7fc8640a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc87c102890 tx=0x7fc87000a6f0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8826f2640 1 -- 172.21.15.82:0/665368200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc87c10c580 msgr2=0x7fc87c101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8826f2640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc87c10c580 0x7fc87c101880 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b9e0 tx=0x7fc87400bea0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8826f2640 1 -- 172.21.15.82:0/665368200 shutdown_connections 2024-04-21T08:55:31.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8826f2640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc8640a3d20 0x7fc8640a61e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8826f2640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc87c10c580 0x7fc87c101880 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8826f2640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc87c103ad0 0x7fc87c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8826f2640 1 --2- 172.21.15.82:0/665368200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc87c1031b0 0x7fc87c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8826f2640 1 -- 172.21.15.82:0/665368200 >> 172.21.15.82:0/665368200 conn(0x7fc87c0771d0 msgr2=0x7fc87c0ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8826f2640 1 -- 172.21.15.82:0/665368200 shutdown_connections 2024-04-21T08:55:31.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.476+0000 7fc8826f2640 1 -- 172.21.15.82:0/665368200 wait complete. 2024-04-21T08:55:31.487 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:55:31.487 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:55:31.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.585+0000 7ff7701a4640 1 -- 172.21.15.82:0/2236525632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff768102190 msgr2=0x7ff768102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.585+0000 7ff7701a4640 1 --2- 172.21.15.82:0/2236525632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff768102190 0x7ff768102570 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7ff764009900 tx=0x7ff76402bc30 comp rx=0 tx=0).stop 2024-04-21T08:55:31.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.585+0000 7ff7701a4640 1 -- 172.21.15.82:0/2236525632 shutdown_connections 2024-04-21T08:55:31.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.585+0000 7ff7701a4640 1 --2- 172.21.15.82:0/2236525632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff768107260 0x7ff76810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.585+0000 7ff7701a4640 1 --2- 172.21.15.82:0/2236525632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff768102b40 0x7ff768106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.585+0000 7ff7701a4640 1 --2- 172.21.15.82:0/2236525632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff768102190 0x7ff768102570 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.585+0000 7ff7701a4640 1 -- 172.21.15.82:0/2236525632 >> 172.21.15.82:0/2236525632 conn(0x7ff7680793e0 msgr2=0x7ff7680797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.585+0000 7ff7701a4640 1 -- 172.21.15.82:0/2236525632 shutdown_connections 2024-04-21T08:55:31.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.585+0000 7ff7701a4640 1 -- 172.21.15.82:0/2236525632 wait complete. 2024-04-21T08:55:31.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff7701a4640 1 Processor -- start 2024-04-21T08:55:31.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff7701a4640 1 -- start start 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff7701a4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff768102190 0x7ff768198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff7701a4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff768107260 0x7ff768198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff7701a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff76819d020 0x7ff76819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff7701a4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681121c0 con 0x7ff768107260 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff7701a4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff768112040 con 0x7ff768102190 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff7701a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff768112340 con 0x7ff76819d020 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff76d73f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff768107260 0x7ff768198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff76d73f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff768107260 0x7ff768198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42632/0 (socket says 172.21.15.82:42632) 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff76d73f640 1 -- 172.21.15.82:0/1799312163 learned_addr learned my addr 172.21.15.82:0/1799312163 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff76df40640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff768102190 0x7ff768198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff76e741640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff76819d020 0x7ff76819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff76d73f640 1 -- 172.21.15.82:0/1799312163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff76819d020 msgr2=0x7ff76819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff76d73f640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff76819d020 0x7ff76819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff76d73f640 1 -- 172.21.15.82:0/1799312163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff768102190 msgr2=0x7ff768198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff76d73f640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff768102190 0x7ff768198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff76d73f640 1 -- 172.21.15.82:0/1799312163 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76819db10 con 0x7ff768107260 2024-04-21T08:55:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff76e741640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff76819d020 0x7ff76819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.586+0000 7ff76d73f640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff768107260 0x7ff768198ca0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff75800d740 tx=0x7ff75800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.587+0000 7ff75effd640 1 -- 172.21.15.82:0/1799312163 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758017070 con 0x7ff768107260 2024-04-21T08:55:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.587+0000 7ff7701a4640 1 -- 172.21.15.82:0/1799312163 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76819de00 con 0x7ff768107260 2024-04-21T08:55:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.587+0000 7ff75effd640 1 -- 172.21.15.82:0/1799312163 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff75800de50 con 0x7ff768107260 2024-04-21T08:55:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.587+0000 7ff75effd640 1 -- 172.21.15.82:0/1799312163 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758004ce0 con 0x7ff768107260 2024-04-21T08:55:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.587+0000 7ff7701a4640 1 -- 172.21.15.82:0/1799312163 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7681a55f0 con 0x7ff768107260 2024-04-21T08:55:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.587+0000 7ff7701a4640 1 -- 172.21.15.82:0/1799312163 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff768069230 con 0x7ff768107260 2024-04-21T08:55:31.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.589+0000 7ff75effd640 1 -- 172.21.15.82:0/1799312163 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff758004e80 con 0x7ff768107260 2024-04-21T08:55:31.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.589+0000 7ff75effd640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff7540a3d20 0x7ff7540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.589+0000 7ff76df40640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff7540a3d20 0x7ff7540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.589+0000 7ff75effd640 1 -- 172.21.15.82:0/1799312163 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7ff7580d09e0 con 0x7ff768107260 2024-04-21T08:55:31.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.590+0000 7ff76df40640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff7540a3d20 0x7ff7540a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff764006010 tx=0x7ff764005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.593+0000 7ff75effd640 1 -- 172.21.15.82:0/1799312163 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff75809cc70 con 0x7ff768107260 2024-04-21T08:55:31.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.810+0000 7ff7701a4640 1 -- 172.21.15.82:0/1799312163 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff768199ab0 con 0x7ff768107260 2024-04-21T08:55:31.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.811+0000 7ff75effd640 1 -- 172.21.15.82:0/1799312163 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11949 (secure 0 0 0) 0x7ff768199ab0 con 0x7ff768107260 2024-04-21T08:55:31.811 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:55:31.811 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:55:27.214396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:55:31.811 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 44 2024-04-21T08:55:31.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.813+0000 7ff7701a4640 1 -- 172.21.15.82:0/1799312163 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff7540a3d20 msgr2=0x7ff7540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.813+0000 7ff7701a4640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff7540a3d20 0x7ff7540a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff764006010 tx=0x7ff764005e40 comp rx=0 tx=0).stop 2024-04-21T08:55:31.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.813+0000 7ff7701a4640 1 -- 172.21.15.82:0/1799312163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff768107260 msgr2=0x7ff768198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.813+0000 7ff7701a4640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff768107260 0x7ff768198ca0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff75800d740 tx=0x7ff75800dc00 comp rx=0 tx=0).stop 2024-04-21T08:55:31.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.813+0000 7ff7701a4640 1 -- 172.21.15.82:0/1799312163 shutdown_connections 2024-04-21T08:55:31.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.813+0000 7ff7701a4640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff76819d020 0x7ff76819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.813+0000 7ff7701a4640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff7540a3d20 0x7ff7540a61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.813+0000 7ff7701a4640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff768107260 0x7ff768198ca0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.813+0000 7ff7701a4640 1 --2- 172.21.15.82:0/1799312163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff768102190 0x7ff768198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.813+0000 7ff7701a4640 1 -- 172.21.15.82:0/1799312163 >> 172.21.15.82:0/1799312163 conn(0x7ff7680793e0 msgr2=0x7ff7680fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.813+0000 7ff7701a4640 1 -- 172.21.15.82:0/1799312163 shutdown_connections 2024-04-21T08:55:31.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.813+0000 7ff7701a4640 1 -- 172.21.15.82:0/1799312163 wait complete. 2024-04-21T08:55:31.824 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:55:31.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.924+0000 7fdfd16f2640 1 -- 172.21.15.82:0/679900861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfcc102b40 msgr2=0x7fdfcc106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.924+0000 7fdfd16f2640 1 --2- 172.21.15.82:0/679900861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfcc102b40 0x7fdfcc106b80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009900 tx=0x7fdfbc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:55:31.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.924+0000 7fdfd16f2640 1 -- 172.21.15.82:0/679900861 shutdown_connections 2024-04-21T08:55:31.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.924+0000 7fdfd16f2640 1 --2- 172.21.15.82:0/679900861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfcc107260 0x7fdfcc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.924+0000 7fdfd16f2640 1 --2- 172.21.15.82:0/679900861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfcc102b40 0x7fdfcc106b80 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.924+0000 7fdfd16f2640 1 --2- 172.21.15.82:0/679900861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfcc102190 0x7fdfcc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.924+0000 7fdfd16f2640 1 -- 172.21.15.82:0/679900861 >> 172.21.15.82:0/679900861 conn(0x7fdfcc0fd990 msgr2=0x7fdfcc0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.924+0000 7fdfd16f2640 1 -- 172.21.15.82:0/679900861 shutdown_connections 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.924+0000 7fdfd16f2640 1 -- 172.21.15.82:0/679900861 wait complete. 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.924+0000 7fdfd16f2640 1 Processor -- start 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.924+0000 7fdfd16f2640 1 -- start start 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfd16f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfcc102190 0x7fdfcc074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfd16f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfcc107260 0x7fdfcc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfd16f2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfcc074cd0 0x7fdfcc1a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfd16f2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc112250 con 0x7fdfcc107260 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfd16f2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1120d0 con 0x7fdfcc074cd0 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfd16f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1123d0 con 0x7fdfcc102190 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfca7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfcc107260 0x7fdfcc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfca7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfcc107260 0x7fdfcc074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42644/0 (socket says 172.21.15.82:42644) 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfca7fc640 1 -- 172.21.15.82:0/818148718 learned_addr learned my addr 172.21.15.82:0/818148718 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:31.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfcaffd640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfcc102190 0x7fdfcc074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfcb7fe640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfcc074cd0 0x7fdfcc1a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfca7fc640 1 -- 172.21.15.82:0/818148718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfcc102190 msgr2=0x7fdfcc074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfcc102190 0x7fdfcc074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfca7fc640 1 -- 172.21.15.82:0/818148718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfcc074cd0 msgr2=0x7fdfcc1a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfcc074cd0 0x7fdfcc1a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfca7fc640 1 -- 172.21.15.82:0/818148718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc1a70e0 con 0x7fdfcc107260 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfcb7fe640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfcc074cd0 0x7fdfcc1a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfcc107260 0x7fdfcc074790 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc02c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfb3fff640 1 -- 172.21.15.82:0/818148718 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc043020 con 0x7fdfcc107260 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfb3fff640 1 -- 172.21.15.82:0/818148718 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfbc02cc60 con 0x7fdfcc107260 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfd16f2640 1 -- 172.21.15.82:0/818148718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc1a7370 con 0x7fdfcc107260 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfb3fff640 1 -- 172.21.15.82:0/818148718 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc003c30 con 0x7fdfcc107260 2024-04-21T08:55:31.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.925+0000 7fdfd16f2640 1 -- 172.21.15.82:0/818148718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc1a78c0 con 0x7fdfcc107260 2024-04-21T08:55:31.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfb3fff640 1 -- 172.21.15.82:0/818148718 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdfbc046020 con 0x7fdfcc107260 2024-04-21T08:55:31.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfd43c4740 1 Processor -- start 2024-04-21T08:55:31.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfd43c4740 1 -- start start 2024-04-21T08:55:31.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfd43c4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9c4d0 0x55ac49a9c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfd43c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9ce80 0x55ac49aa5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfd43c4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 0x55ac49aa7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfd43c4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ac49a011f0 con 0x55ac49aa5a00 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfd43c4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ac4999e4b0 con 0x55ac49a9c4d0 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfd43c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ac499fd1b0 con 0x55ac49a9ce80 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfca7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 0x55ac49aa7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfb3fff640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdfb40a3d50 0x7fdfb40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfca7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 0x55ac49aa7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42656/0 (socket says 172.21.15.82:42656) 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfca7fc640 1 -- 172.21.15.82:0/1861738322 learned_addr learned my addr 172.21.15.82:0/1861738322 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfcb7fe640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdfb40a3d50 0x7fdfb40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfcb7fe640 1 --2- 172.21.15.82:0/1861738322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9c4d0 0x55ac49a9c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfcaffd640 1 --2- 172.21.15.82:0/1861738322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9ce80 0x55ac49aa5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.928+0000 7fdfb3fff640 1 -- 172.21.15.82:0/818148718 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fdfbc0f2d30 con 0x7fdfcc107260 2024-04-21T08:55:31.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.929+0000 7fdfca7fc640 1 -- 172.21.15.82:0/1861738322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9ce80 msgr2=0x55ac49aa5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.929+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/1861738322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9ce80 0x55ac49aa5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.929+0000 7fdfca7fc640 1 -- 172.21.15.82:0/1861738322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9c4d0 msgr2=0x55ac49a9c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.929+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/1861738322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9c4d0 0x55ac49a9c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.929+0000 7fdfca7fc640 1 -- 172.21.15.82:0/1861738322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac499901b0 con 0x55ac49aa5a00 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.929+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/1861738322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 0x55ac49aa7e50 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0beb90 tx=0x7fdfbc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142cc298bc84e193 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.929+0000 7fdfcb7fe640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdfb40a3d50 0x7fdfb40a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009440 tx=0x7fdfc4009e30 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.929+0000 7fdfb17fa640 1 -- 172.21.15.82:0/1861738322 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0be7a0 con 0x55ac49aa5a00 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.929+0000 7fdfb17fa640 1 -- 172.21.15.82:0/1861738322 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfbc0055b0 con 0x55ac49aa5a00 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.929+0000 7fdfb17fa640 1 -- 172.21.15.82:0/1861738322 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0be330 con 0x55ac49aa5a00 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.929+0000 7fdfd43c4740 1 -- 172.21.15.82:0/1861738322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 msgr2=0x55ac49aa7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.929+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/1861738322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 0x55ac49aa7e50 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0beb90 tx=0x7fdfbc009260 comp rx=0 tx=0).stop 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 -- 172.21.15.82:0/1861738322 shutdown_connections 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/1861738322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 0x55ac49aa7e50 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/1861738322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9ce80 0x55ac49aa5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/1861738322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9c4d0 0x55ac49a9c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 -- 172.21.15.82:0/1861738322 >> 172.21.15.82:0/1861738322 conn(0x55ac49a96ba0 msgr2=0x55ac49a96f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 -- 172.21.15.82:0/1861738322 shutdown_connections 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 -- 172.21.15.82:0/1861738322 wait complete. 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 -- start start 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 Processor -- start 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9c4d0 0x55ac49aa52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9ce80 0x55ac49aa2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 0x55ac49aa2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ac49a011f0 con 0x55ac49aa5a00 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ac499ba500 con 0x55ac49a9ce80 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ac49aa8730 con 0x55ac49a9c4d0 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfd43c4740 10 client.0 _ll_register_callbacks cb 0x55ac49988b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:31.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfcb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 0x55ac49aa2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfcb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 0x55ac49aa2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42662/0 (socket says 172.21.15.82:42662) 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfcb7fe640 1 -- 172.21.15.82:0/223378214 learned_addr learned my addr 172.21.15.82:0/223378214 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.930+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9ce80 0x55ac49aa2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfcaffd640 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9c4d0 0x55ac49aa52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfcb7fe640 1 -- 172.21.15.82:0/223378214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9c4d0 msgr2=0x55ac49aa52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfcb7fe640 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9c4d0 0x55ac49aa52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfcb7fe640 1 -- 172.21.15.82:0/223378214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9ce80 msgr2=0x55ac49aa2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfcb7fe640 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9ce80 0x55ac49aa2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfcb7fe640 1 -- 172.21.15.82:0/223378214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac497133f0 con 0x55ac49aa5a00 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfcaffd640 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9c4d0 0x55ac49aa52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfcb7fe640 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 0x55ac49aa2c00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400aea0 tx=0x7fdfc40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5939ec666f729a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfaeff5640 1 -- 172.21.15.82:0/223378214 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc401f040 con 0x55ac49aa5a00 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfaeff5640 1 -- 172.21.15.82:0/223378214 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fdfc4019e00 con 0x55ac49aa5a00 2024-04-21T08:55:31.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfd43c4740 1 -- 172.21.15.82:0/223378214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ac499a0930 con 0x55ac49aa5a00 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfaeff5640 1 -- 172.21.15.82:0/223378214 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdfc4013040 con 0x55ac49aa5a00 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfd43c4740 10 client.18027 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfd43c4740 1 -- 172.21.15.82:0/223378214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ac497dd7d0 con 0x55ac49aa5a00 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfaeff5640 1 -- 172.21.15.82:0/223378214 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ac499a0930 con 0x55ac49aa5a00 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfaeff5640 1 -- 172.21.15.82:0/223378214 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fdfc402c4d0 con 0x55ac49aa5a00 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfd43c4740 10 client.18027 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfd43c4740 10 client.18027 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfd43c4740 20 client.18027 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55ac49ab5980 0x55ac49ab7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfd43c4740 4 client.18027 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.931+0000 7fdfd43c4740 1 -- 172.21.15.82:0/223378214 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac49a0ec10 con 0x55ac49ab5980 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.932+0000 7fdfcaffd640 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55ac49ab5980 0x55ac49ab7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.932+0000 7fdfcaffd640 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55ac49ab5980 0x55ac49ab7d70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4a43cf36aaa41c07 server_cookie=afe7fa4b8e25bdf3 in_seq=0 out_seq=0 2024-04-21T08:55:31.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.932+0000 7fdfaeff5640 10 client.18027 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfaeff5640 1 -- 172.21.15.82:0/223378214 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac49a0ec10 con 0x55ac49ab5980 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfaeff5640 10 client.18027 handle_command_reply: tid=0 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfd43c4740 1 client.18027 shutdown 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfd43c4740 1 -- 172.21.15.82:0/223378214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 msgr2=0x55ac49aa2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 0x55ac49aa2c00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400aea0 tx=0x7fdfc40093d0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfd43c4740 1 -- 172.21.15.82:0/223378214 shutdown_connections 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55ac49ab5980 0x55ac49ab7d70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49aa5a00 0x55ac49aa2c00 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfaeff5640 0 client.18027 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9ce80 0x55ac49aa2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/223378214 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9c4d0 0x55ac49aa52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfd43c4740 1 -- 172.21.15.82:0/223378214 >> 172.21.15.82:0/223378214 conn(0x55ac49a96ba0 msgr2=0x55ac49a988f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfd43c4740 1 -- 172.21.15.82:0/223378214 shutdown_connections 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfd43c4740 1 -- 172.21.15.82:0/223378214 wait complete. 2024-04-21T08:55:31.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.933+0000 7fdfd43c4740 20 client.18027 trim_cache size 0 max 16384 2024-04-21T08:55:31.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfd43c4740 1 Processor -- start 2024-04-21T08:55:31.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfd43c4740 1 -- start start 2024-04-21T08:55:31.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfd43c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9c4d0 0x55ac49ab14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfd43c4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 0x55ac49aa17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfd43c4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49aa5a00 0x55ac49a98700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfd43c4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1b1730 con 0x55ac49a9ce80 2024-04-21T08:55:31.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfd43c4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1b3300 con 0x55ac49aa5a00 2024-04-21T08:55:31.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfd43c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1b1e00 con 0x55ac49a9c4d0 2024-04-21T08:55:31.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfca7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 0x55ac49aa17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfca7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 0x55ac49aa17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42676/0 (socket says 172.21.15.82:42676) 2024-04-21T08:55:31.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfca7fc640 1 -- 172.21.15.82:0/3923858833 learned_addr learned my addr 172.21.15.82:0/3923858833 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:31.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfcaffd640 1 --2- 172.21.15.82:0/3923858833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9c4d0 0x55ac49ab14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.946+0000 7fdfca7fc640 1 -- 172.21.15.82:0/3923858833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9c4d0 msgr2=0x55ac49ab14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/3923858833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9c4d0 0x55ac49ab14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfca7fc640 1 -- 172.21.15.82:0/3923858833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49aa5a00 msgr2=0x55ac49a98700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfcb7fe640 1 --2- 172.21.15.82:0/3923858833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49aa5a00 0x55ac49a98700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/3923858833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49aa5a00 0x55ac49a98700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfca7fc640 1 -- 172.21.15.82:0/3923858833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac49ab83f0 con 0x55ac49a9ce80 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/3923858833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 0x55ac49aa17e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0f0670 tx=0x7fdfbc0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb29e0e8685278b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfb0ff9640 1 -- 172.21.15.82:0/3923858833 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0be510 con 0x55ac49a9ce80 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfb0ff9640 1 -- 172.21.15.82:0/3923858833 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfbc0be9d0 con 0x55ac49a9ce80 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfb0ff9640 1 -- 172.21.15.82:0/3923858833 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0bccd0 con 0x55ac49a9ce80 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3923858833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 msgr2=0x55ac49aa17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/3923858833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 0x55ac49aa17e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0f0670 tx=0x7fdfbc0bb040 comp rx=0 tx=0).stop 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3923858833 shutdown_connections 2024-04-21T08:55:31.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/3923858833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49aa5a00 0x55ac49a98700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/3923858833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 0x55ac49aa17e0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.947+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/3923858833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49a9c4d0 0x55ac49ab14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3923858833 >> 172.21.15.82:0/3923858833 conn(0x55ac49a96ba0 msgr2=0x55ac49a97ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3923858833 shutdown_connections 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3923858833 wait complete. 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfd43c4740 1 -- start start 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfd43c4740 1 Processor -- start 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfd43c4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9c4d0 0x55ac49aad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfd43c4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 0x55ac49aadf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfd43c4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49aa5a00 0x55ac49a9ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfd43c4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1b2c00 con 0x55ac49a9ce80 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfd43c4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1b2500 con 0x55ac49a9c4d0 2024-04-21T08:55:31.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfd43c4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1af700 con 0x55ac49aa5a00 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfd43c4740 10 client.0 _ll_register_callbacks cb 0x7fdfc4009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfca7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 0x55ac49aadf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfca7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 0x55ac49aadf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42678/0 (socket says 172.21.15.82:42678) 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfca7fc640 1 -- 172.21.15.82:0/3580084860 learned_addr learned my addr 172.21.15.82:0/3580084860 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfcb7fe640 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49aa5a00 0x55ac49a9ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfcaffd640 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9c4d0 0x55ac49aad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfca7fc640 1 -- 172.21.15.82:0/3580084860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49aa5a00 msgr2=0x55ac49a9ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49aa5a00 0x55ac49a9ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfca7fc640 1 -- 172.21.15.82:0/3580084860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9c4d0 msgr2=0x55ac49aad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9c4d0 0x55ac49aad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.948+0000 7fdfca7fc640 1 -- 172.21.15.82:0/3580084860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac49a9f550 con 0x55ac49a9ce80 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfca7fc640 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 0x55ac49aadf20 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0092a0 tx=0x7fdfbc0f2f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04d40273e2be66d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:31.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfaeff5640 1 -- 172.21.15.82:0/3580084860 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0dc020 con 0x55ac49a9ce80 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfaeff5640 1 -- 172.21.15.82:0/3580084860 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fdfbc0f0260 con 0x55ac49a9ce80 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3580084860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ac49a08a80 con 0x55ac49a9ce80 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfaeff5640 1 -- 172.21.15.82:0/3580084860 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdfbc0be7a0 con 0x55ac49a9ce80 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfd43c4740 10 client.18033 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3580084860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ac49b1b9b0 con 0x55ac49a9ce80 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfaeff5640 1 -- 172.21.15.82:0/3580084860 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ac49a08a80 con 0x55ac49a9ce80 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfaeff5640 1 -- 172.21.15.82:0/3580084860 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fdfbc0ddb90 con 0x55ac49a9ce80 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfd43c4740 10 client.18033 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfd43c4740 10 client.18033 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfd43c4740 20 client.18033 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55ac49b1be50 0x55ac49b1e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfd43c4740 4 client.18033 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3580084860 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac499846e0 con 0x55ac49b1be50 2024-04-21T08:55:31.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.949+0000 7fdfcaffd640 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55ac49b1be50 0x55ac49b1e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfaeff5640 10 client.18033 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfcaffd640 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55ac49b1be50 0x55ac49b1e240 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c358987ecdbae15 server_cookie=cab00d7aadcd370 in_seq=0 out_seq=0 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfaeff5640 1 -- 172.21.15.82:0/3580084860 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac499846e0 con 0x55ac49b1be50 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfaeff5640 10 client.18033 handle_command_reply: tid=0 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfd43c4740 1 client.18033 shutdown 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3580084860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 msgr2=0x55ac49aadf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 0x55ac49aadf20 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0092a0 tx=0x7fdfbc0f2f30 comp rx=0 tx=0).stop 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3580084860 shutdown_connections 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x55ac49b1be50 0x55ac49b1e240 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ac49aa5a00 0x55ac49a9ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfaeff5640 0 client.18033 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ac49a9ce80 0x55ac49aadf20 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfd43c4740 1 --2- 172.21.15.82:0/3580084860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ac49a9c4d0 0x55ac49aad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3580084860 >> 172.21.15.82:0/3580084860 conn(0x55ac49a96ba0 msgr2=0x55ac49a95c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3580084860 shutdown_connections 2024-04-21T08:55:31.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfd43c4740 1 -- 172.21.15.82:0/3580084860 wait complete. 2024-04-21T08:55:31.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.950+0000 7fdfd43c4740 20 client.18033 trim_cache size 0 max 16384 2024-04-21T08:55:31.958 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:55:31.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.951+0000 7fdfd16f2640 1 -- 172.21.15.82:0/818148718 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdfb40a3d50 msgr2=0x7fdfb40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.951+0000 7fdfd16f2640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdfb40a3d50 0x7fdfb40a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009440 tx=0x7fdfc4009e30 comp rx=0 tx=0).stop 2024-04-21T08:55:31.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.951+0000 7fdfd16f2640 1 -- 172.21.15.82:0/818148718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfcc107260 msgr2=0x7fdfcc074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:31.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.951+0000 7fdfd16f2640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfcc107260 0x7fdfcc074790 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc02c860 comp rx=0 tx=0).stop 2024-04-21T08:55:31.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.951+0000 7fdfd16f2640 1 -- 172.21.15.82:0/818148718 shutdown_connections 2024-04-21T08:55:31.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.951+0000 7fdfd16f2640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfcc074cd0 0x7fdfcc1a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.951+0000 7fdfd16f2640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdfb40a3d50 0x7fdfb40a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.951+0000 7fdfd16f2640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfcc107260 0x7fdfcc074790 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.951+0000 7fdfd16f2640 1 --2- 172.21.15.82:0/818148718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfcc102190 0x7fdfcc074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:31.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.951+0000 7fdfd16f2640 1 -- 172.21.15.82:0/818148718 >> 172.21.15.82:0/818148718 conn(0x7fdfcc0fd990 msgr2=0x7fdfcc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:31.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.951+0000 7fdfd16f2640 1 -- 172.21.15.82:0/818148718 shutdown_connections 2024-04-21T08:55:31.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:31.951+0000 7fdfd16f2640 1 -- 172.21.15.82:0/818148718 wait complete. 2024-04-21T08:55:31.961 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:55:32.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 -- 172.21.15.82:0/3821282165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9c106f50 msgr2=0x7f6c9c107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 --2- 172.21.15.82:0/3821282165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9c106f50 0x7f6c9c107330 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009cd0 tx=0x7f6c9402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 -- 172.21.15.82:0/3821282165 shutdown_connections 2024-04-21T08:55:32.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 --2- 172.21.15.82:0/3821282165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c9c100e80 0x7f6c9c10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 --2- 172.21.15.82:0/3821282165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c9c1004c0 0x7f6c9c100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 --2- 172.21.15.82:0/3821282165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9c106f50 0x7f6c9c107330 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 -- 172.21.15.82:0/3821282165 >> 172.21.15.82:0/3821282165 conn(0x7f6c9c077450 msgr2=0x7f6c9c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 -- 172.21.15.82:0/3821282165 shutdown_connections 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 -- 172.21.15.82:0/3821282165 wait complete. 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 Processor -- start 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 -- start start 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9c1004c0 0x7f6c9c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c9c100e80 0x7f6c9c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c9c106f50 0x7f6c9c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c112260 con 0x7f6c9c1004c0 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1120e0 con 0x7f6c9c106f50 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca2f04640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1123e0 con 0x7f6c9c100e80 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.061+0000 7f6ca0ca0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9c1004c0 0x7f6c9c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6ca0ca0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9c1004c0 0x7f6c9c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42702/0 (socket says 172.21.15.82:42702) 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6ca0ca0640 1 -- 172.21.15.82:0/2467199489 learned_addr learned my addr 172.21.15.82:0/2467199489 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6c9bfff640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c9c100e80 0x7f6c9c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6ca0ca0640 1 -- 172.21.15.82:0/2467199489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c9c100e80 msgr2=0x7f6c9c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6ca14a1640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c9c106f50 0x7f6c9c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6ca0ca0640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c9c100e80 0x7f6c9c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6ca0ca0640 1 -- 172.21.15.82:0/2467199489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c9c106f50 msgr2=0x7f6c9c19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6ca0ca0640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c9c106f50 0x7f6c9c19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6ca0ca0640 1 -- 172.21.15.82:0/2467199489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9c19afe0 con 0x7f6c9c1004c0 2024-04-21T08:55:32.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6ca14a1640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c9c106f50 0x7f6c9c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:32.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6ca0ca0640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9c1004c0 0x7f6c9c1a06b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f6c9402c700 tx=0x7f6c940346e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6c99ffb640 1 -- 172.21.15.82:0/2467199489 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94041020 con 0x7f6c9c1004c0 2024-04-21T08:55:32.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6c99ffb640 1 -- 172.21.15.82:0/2467199489 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c94032960 con 0x7f6c9c1004c0 2024-04-21T08:55:32.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6c99ffb640 1 -- 172.21.15.82:0/2467199489 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94039040 con 0x7f6c9c1004c0 2024-04-21T08:55:32.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6ca2f04640 1 -- 172.21.15.82:0/2467199489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c9c19b270 con 0x7f6c9c1004c0 2024-04-21T08:55:32.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.062+0000 7f6ca2f04640 1 -- 172.21.15.82:0/2467199489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c9c1a7570 con 0x7f6c9c1004c0 2024-04-21T08:55:32.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.064+0000 7f6c99ffb640 1 -- 172.21.15.82:0/2467199489 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6c94044020 con 0x7f6c9c1004c0 2024-04-21T08:55:32.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6ca5bd5740 1 Processor -- start 2024-04-21T08:55:32.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6ca5bd5740 1 -- start start 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6ca5bd5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d284d0 0x560fb0d288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6ca5bd5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d28e80 0x560fb0d31430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6ca5bd5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 0x560fb0d33e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6ca5bd5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560fb0c8d1f0 con 0x560fb0d31a00 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6ca5bd5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560fb0c2a4b0 con 0x560fb0d284d0 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6ca5bd5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560fb0c891b0 con 0x560fb0d28e80 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6c99ffb640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c880a3d20 0x7f6c880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 0x560fb0d33e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 0x560fb0d33e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42718/0 (socket says 172.21.15.82:42718) 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6c9bfff640 1 -- 172.21.15.82:0/488039112 learned_addr learned my addr 172.21.15.82:0/488039112 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6ca0ca0640 1 --2- 172.21.15.82:0/488039112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d28e80 0x560fb0d31430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6ca14a1640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c880a3d20 0x7f6c880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6ca14a1640 1 --2- 172.21.15.82:0/488039112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d284d0 0x560fb0d288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6c99ffb640 1 -- 172.21.15.82:0/2467199489 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f6c940f2900 con 0x7f6c9c1004c0 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6c9bfff640 1 -- 172.21.15.82:0/488039112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d28e80 msgr2=0x560fb0d31430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6c9bfff640 1 --2- 172.21.15.82:0/488039112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d28e80 0x560fb0d31430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6c9bfff640 1 -- 172.21.15.82:0/488039112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d284d0 msgr2=0x560fb0d288b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6c9bfff640 1 --2- 172.21.15.82:0/488039112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d284d0 0x560fb0d288b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6c9bfff640 1 -- 172.21.15.82:0/488039112 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb0c1c1b0 con 0x560fb0d31a00 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.065+0000 7f6ca14a1640 1 --2- 172.21.15.82:0/488039112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d284d0 0x560fb0d288b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.066+0000 7f6c9bfff640 1 --2- 172.21.15.82:0/488039112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 0x560fb0d33e50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00bc40 tx=0x7f6c8c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173916986d10eb0e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.066+0000 7f6ca14a1640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c880a3d20 0x7f6c880a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6c84005fd0 tx=0x7f6c84005da0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.066+0000 7f6c92ffd640 1 -- 172.21.15.82:0/488039112 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c015070 con 0x560fb0d31a00 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.066+0000 7f6c92ffd640 1 -- 172.21.15.82:0/488039112 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c8c01a910 con 0x560fb0d31a00 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.066+0000 7f6c92ffd640 1 -- 172.21.15.82:0/488039112 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c011860 con 0x560fb0d31a00 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/488039112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 msgr2=0x560fb0d33e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/488039112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 0x560fb0d33e50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00bc40 tx=0x7f6c8c00f730 comp rx=0 tx=0).stop 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/488039112 shutdown_connections 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/488039112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 0x560fb0d33e50 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/488039112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d28e80 0x560fb0d31430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/488039112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d284d0 0x560fb0d288b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/488039112 >> 172.21.15.82:0/488039112 conn(0x560fb0d22ba0 msgr2=0x560fb0d22f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/488039112 shutdown_connections 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/488039112 wait complete. 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 -- start start 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 Processor -- start 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d284d0 0x560fb0d312e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d28e80 0x560fb0d2e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 0x560fb0d2ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560fb0d34970 con 0x560fb0d31a00 2024-04-21T08:55:32.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560fb0c46500 con 0x560fb0d28e80 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560fb0d34af0 con 0x560fb0d284d0 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.067+0000 7f6ca5bd5740 10 client.0 _ll_register_callbacks cb 0x560fb0c14b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca14a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 0x560fb0d2ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca0ca0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d284d0 0x560fb0d312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca0ca0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d284d0 0x560fb0d312e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:41612/0 (socket says 172.21.15.82:41612) 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca14a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 0x560fb0d2ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42720/0 (socket says 172.21.15.82:42720) 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca0ca0640 1 -- 172.21.15.82:0/2690206080 learned_addr learned my addr 172.21.15.82:0/2690206080 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6c9bfff640 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d28e80 0x560fb0d2e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca14a1640 1 -- 172.21.15.82:0/2690206080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d284d0 msgr2=0x560fb0d312e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca14a1640 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d284d0 0x560fb0d312e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca14a1640 1 -- 172.21.15.82:0/2690206080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d28e80 msgr2=0x560fb0d2e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca14a1640 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d28e80 0x560fb0d2e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca14a1640 1 -- 172.21.15.82:0/2690206080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb099f3f0 con 0x560fb0d31a00 2024-04-21T08:55:32.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca0ca0640 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d284d0 0x560fb0d312e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca14a1640 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 0x560fb0d2ebd0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400abf0 tx=0x7f6c840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862ed820c2257013 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6c7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6c7bfff640 1 -- 172.21.15.82:0/2690206080 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84010a60 con 0x560fb0d31a00 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6c7bfff640 1 -- 172.21.15.82:0/2690206080 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f6c84010c00 con 0x560fb0d31a00 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/2690206080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c8c01a910 con 0x560fb0d31a00 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6c7bfff640 1 -- 172.21.15.82:0/2690206080 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6c84010e00 con 0x560fb0d31a00 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca5bd5740 10 client.18045 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.068+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/2690206080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560fb0a697d0 con 0x560fb0d31a00 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.069+0000 7f6c7bfff640 1 -- 172.21.15.82:0/2690206080 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c8c01a910 con 0x560fb0d31a00 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.069+0000 7f6c7bfff640 1 -- 172.21.15.82:0/2690206080 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f6c8401e480 con 0x560fb0d31a00 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.069+0000 7f6ca5bd5740 10 client.18045 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.069+0000 7f6ca5bd5740 10 client.18045 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.069+0000 7f6ca5bd5740 20 client.18045 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.069+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x560fb0d41c30 0x560fb0d440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.069+0000 7f6ca5bd5740 4 client.18045 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.069+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/2690206080 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fb0c2c930 con 0x560fb0d41c30 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.069+0000 7f6ca0ca0640 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x560fb0d41c30 0x560fb0d440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6c7bfff640 10 client.18045 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6ca0ca0640 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x560fb0d41c30 0x560fb0d440a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f22e80a44d49279b server_cookie=61142755dff5cb09 in_seq=0 out_seq=0 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6c7bfff640 1 -- 172.21.15.82:0/2690206080 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560fb0c2c930 con 0x560fb0d41c30 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6c7bfff640 10 client.18045 handle_command_reply: tid=0 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6ca5bd5740 1 client.18045 shutdown 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/2690206080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 msgr2=0x560fb0d2ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 0x560fb0d2ebd0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400abf0 tx=0x7f6c840046c0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/2690206080 shutdown_connections 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x560fb0d41c30 0x560fb0d440a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d31a00 0x560fb0d2ebd0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6c7bfff640 0 client.18045 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:55:32.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d28e80 0x560fb0d2e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/2690206080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d284d0 0x560fb0d312e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.070+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/2690206080 >> 172.21.15.82:0/2690206080 conn(0x560fb0d22ba0 msgr2=0x560fb0d248f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.071+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/2690206080 shutdown_connections 2024-04-21T08:55:32.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.071+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/2690206080 wait complete. 2024-04-21T08:55:32.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.071+0000 7f6ca5bd5740 20 client.18045 trim_cache size 0 max 16384 2024-04-21T08:55:32.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6ca5bd5740 1 Processor -- start 2024-04-21T08:55:32.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6ca5bd5740 1 -- start start 2024-04-21T08:55:32.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6ca5bd5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d284d0 0x560fb0d3d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6ca5bd5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d28e80 0x560fb0d21a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6ca5bd5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d31a00 0x560fb0d30da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6ca5bd5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1b1740 con 0x560fb0d28e80 2024-04-21T08:55:32.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6ca5bd5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1b3310 con 0x560fb0d284d0 2024-04-21T08:55:32.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6ca5bd5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1b1e10 con 0x560fb0d31a00 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d28e80 0x560fb0d21a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d28e80 0x560fb0d21a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42734/0 (socket says 172.21.15.82:42734) 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6c9bfff640 1 -- 172.21.15.82:0/3254205463 learned_addr learned my addr 172.21.15.82:0/3254205463 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6ca14a1640 1 --2- 172.21.15.82:0/3254205463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d31a00 0x560fb0d30da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6ca0ca0640 1 --2- 172.21.15.82:0/3254205463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d284d0 0x560fb0d3d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6c9bfff640 1 -- 172.21.15.82:0/3254205463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d31a00 msgr2=0x560fb0d30da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6c9bfff640 1 --2- 172.21.15.82:0/3254205463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d31a00 0x560fb0d30da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6c9bfff640 1 -- 172.21.15.82:0/3254205463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d284d0 msgr2=0x560fb0d3d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6c9bfff640 1 --2- 172.21.15.82:0/3254205463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d284d0 0x560fb0d3d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6c9bfff640 1 -- 172.21.15.82:0/3254205463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb0d2cbc0 con 0x560fb0d28e80 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6ca0ca0640 1 --2- 172.21.15.82:0/3254205463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d284d0 0x560fb0d3d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6c9bfff640 1 --2- 172.21.15.82:0/3254205463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d28e80 0x560fb0d21a80 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00bb60 tx=0x7f6c8c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725e36fa455fd178 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.084+0000 7f6c927fc640 1 -- 172.21.15.82:0/3254205463 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c015040 con 0x560fb0d28e80 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6c927fc640 1 -- 172.21.15.82:0/3254205463 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c8c005040 con 0x560fb0d28e80 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6c927fc640 1 -- 172.21.15.82:0/3254205463 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c004aa0 con 0x560fb0d28e80 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3254205463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d28e80 msgr2=0x560fb0d21a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/3254205463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d28e80 0x560fb0d21a80 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00bb60 tx=0x7f6c8c00b8e0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3254205463 shutdown_connections 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/3254205463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d31a00 0x560fb0d30da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/3254205463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d28e80 0x560fb0d21a80 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/3254205463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d284d0 0x560fb0d3d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3254205463 >> 172.21.15.82:0/3254205463 conn(0x560fb0d22ba0 msgr2=0x560fb0d21370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3254205463 shutdown_connections 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3254205463 wait complete. 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6ca5bd5740 1 -- start start 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.085+0000 7f6ca5bd5740 1 Processor -- start 2024-04-21T08:55:32.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca5bd5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d284d0 0x560fb0d358d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca5bd5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d28e80 0x560fb0d35e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca5bd5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d31a00 0x560fb0d36350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca5bd5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1b2c10 con 0x560fb0d284d0 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca5bd5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1b2510 con 0x560fb0d28e80 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca5bd5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c1af710 con 0x560fb0d31a00 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca5bd5740 10 client.0 _ll_register_callbacks cb 0x7f6c8400dd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca0ca0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d284d0 0x560fb0d358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca0ca0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d284d0 0x560fb0d358d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42740/0 (socket says 172.21.15.82:42740) 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca0ca0640 1 -- 172.21.15.82:0/3508256953 learned_addr learned my addr 172.21.15.82:0/3508256953 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca14a1640 1 --2- 172.21.15.82:0/3508256953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d31a00 0x560fb0d36350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca0ca0640 1 -- 172.21.15.82:0/3508256953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d31a00 msgr2=0x560fb0d36350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca0ca0640 1 --2- 172.21.15.82:0/3508256953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d31a00 0x560fb0d36350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca0ca0640 1 -- 172.21.15.82:0/3508256953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d28e80 msgr2=0x560fb0d35e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca0ca0640 1 --2- 172.21.15.82:0/3508256953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d28e80 0x560fb0d35e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca0ca0640 1 -- 172.21.15.82:0/3508256953 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb0da73d0 con 0x560fb0d284d0 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca0ca0640 1 --2- 172.21.15.82:0/3508256953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d284d0 0x560fb0d358d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f6c940baf70 tx=0x7f6c9408ffe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee86d145f0166c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6c7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6c7bfff640 1 -- 172.21.15.82:0/3508256953 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c940890b0 con 0x560fb0d284d0 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6c7bfff640 1 -- 172.21.15.82:0/3508256953 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f6c94003970 con 0x560fb0d284d0 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.086+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3508256953 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c8c005040 con 0x560fb0d284d0 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.087+0000 7f6c7bfff640 1 -- 172.21.15.82:0/3508256953 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6c940beca0 con 0x560fb0d284d0 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.087+0000 7f6ca5bd5740 10 client.18051 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.087+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3508256953 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560fb0da8660 con 0x560fb0d284d0 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.087+0000 7f6c7bfff640 1 -- 172.21.15.82:0/3508256953 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c8c005040 con 0x560fb0d284d0 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.087+0000 7f6c7bfff640 1 -- 172.21.15.82:0/3508256953 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f6c9403c030 con 0x560fb0d284d0 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.087+0000 7f6ca5bd5740 10 client.18051 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.087+0000 7f6ca5bd5740 10 client.18051 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.087+0000 7f6ca5bd5740 20 client.18051 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:32.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.087+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/3508256953 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x560fb0da8d40 0x560fb0dab180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.087+0000 7f6ca5bd5740 4 client.18051 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.087+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3508256953 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fb0c9ac10 con 0x560fb0da8d40 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.087+0000 7f6c9bfff640 1 --2- 172.21.15.82:0/3508256953 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x560fb0da8d40 0x560fb0dab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6c9bfff640 1 --2- 172.21.15.82:0/3508256953 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x560fb0da8d40 0x560fb0dab180 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=84145ed21a6b31ba server_cookie=bdc6e4fca2ae77fd in_seq=0 out_seq=0 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6c7bfff640 10 client.18051 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6c7bfff640 1 -- 172.21.15.82:0/3508256953 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fb0c9ac10 con 0x560fb0da8d40 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6c7bfff640 10 client.18051 handle_command_reply: tid=0 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6ca5bd5740 1 client.18051 shutdown 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3508256953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d284d0 msgr2=0x560fb0d358d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/3508256953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d284d0 0x560fb0d358d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f6c940baf70 tx=0x7f6c9408ffe0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3508256953 shutdown_connections 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/3508256953 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x560fb0da8d40 0x560fb0dab180 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/3508256953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560fb0d31a00 0x560fb0d36350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/3508256953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560fb0d28e80 0x560fb0d35e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6ca5bd5740 1 --2- 172.21.15.82:0/3508256953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fb0d284d0 0x560fb0d358d0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6c7bfff640 0 client.18051 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:55:32.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3508256953 >> 172.21.15.82:0/3508256953 conn(0x560fb0d22ba0 msgr2=0x560fb0d21c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3508256953 shutdown_connections 2024-04-21T08:55:32.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6ca5bd5740 1 -- 172.21.15.82:0/3508256953 wait complete. 2024-04-21T08:55:32.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.088+0000 7f6ca5bd5740 20 client.18051 trim_cache size 0 max 16384 2024-04-21T08:55:32.095 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:55:32.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.089+0000 7f6ca2f04640 1 -- 172.21.15.82:0/2467199489 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c880a3d20 msgr2=0x7f6c880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.089+0000 7f6ca2f04640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c880a3d20 0x7f6c880a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6c84005fd0 tx=0x7f6c84005da0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.089+0000 7f6ca2f04640 1 -- 172.21.15.82:0/2467199489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9c1004c0 msgr2=0x7f6c9c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.089+0000 7f6ca2f04640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9c1004c0 0x7f6c9c1a06b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f6c9402c700 tx=0x7f6c940346e0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.089+0000 7f6ca2f04640 1 -- 172.21.15.82:0/2467199489 shutdown_connections 2024-04-21T08:55:32.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.089+0000 7f6ca2f04640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c9c106f50 0x7f6c9c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.089+0000 7f6ca2f04640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c9c100e80 0x7f6c9c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.089+0000 7f6ca2f04640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6c880a3d20 0x7f6c880a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.089+0000 7f6ca2f04640 1 --2- 172.21.15.82:0/2467199489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9c1004c0 0x7f6c9c1a06b0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.089+0000 7f6ca2f04640 1 -- 172.21.15.82:0/2467199489 >> 172.21.15.82:0/2467199489 conn(0x7f6c9c077450 msgr2=0x7f6c9c105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.089+0000 7f6ca2f04640 1 -- 172.21.15.82:0/2467199489 shutdown_connections 2024-04-21T08:55:32.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.089+0000 7f6ca2f04640 1 -- 172.21.15.82:0/2467199489 wait complete. 2024-04-21T08:55:32.099 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:55:32.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.199+0000 7fbc94585640 1 -- 172.21.15.82:0/1384782567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc8c107260 msgr2=0x7fbc8c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.199+0000 7fbc94585640 1 --2- 172.21.15.82:0/1384782567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc8c107260 0x7fbc8c10ee30 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009ca0 tx=0x7fbc8402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.199+0000 7fbc94585640 1 -- 172.21.15.82:0/1384782567 shutdown_connections 2024-04-21T08:55:32.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.199+0000 7fbc94585640 1 --2- 172.21.15.82:0/1384782567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc8c107260 0x7fbc8c10ee30 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.199+0000 7fbc94585640 1 --2- 172.21.15.82:0/1384782567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc8c102b40 0x7fbc8c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.199+0000 7fbc94585640 1 --2- 172.21.15.82:0/1384782567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc8c102190 0x7fbc8c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.199+0000 7fbc94585640 1 -- 172.21.15.82:0/1384782567 >> 172.21.15.82:0/1384782567 conn(0x7fbc8c0793a0 msgr2=0x7fbc8c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.199+0000 7fbc94585640 1 -- 172.21.15.82:0/1384782567 shutdown_connections 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.199+0000 7fbc94585640 1 -- 172.21.15.82:0/1384782567 wait complete. 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc94585640 1 Processor -- start 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc94585640 1 -- start start 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc94585640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc8c102190 0x7fbc8c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc94585640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc8c107260 0x7fbc8c198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc94585640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc8c19cf90 0x7fbc8c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc94585640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1121d0 con 0x7fbc8c102190 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc94585640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c112050 con 0x7fbc8c19cf90 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc94585640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c112350 con 0x7fbc8c107260 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc92321640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc8c102190 0x7fbc8c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc92321640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc8c102190 0x7fbc8c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42754/0 (socket says 172.21.15.82:42754) 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc92321640 1 -- 172.21.15.82:0/2950686253 learned_addr learned my addr 172.21.15.82:0/2950686253 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc92321640 1 -- 172.21.15.82:0/2950686253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc8c107260 msgr2=0x7fbc8c198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc92321640 1 --2- 172.21.15.82:0/2950686253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc8c107260 0x7fbc8c198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc92321640 1 -- 172.21.15.82:0/2950686253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc8c19cf90 msgr2=0x7fbc8c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc92321640 1 --2- 172.21.15.82:0/2950686253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc8c19cf90 0x7fbc8c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc92321640 1 -- 172.21.15.82:0/2950686253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c19da80 con 0x7fbc8c102190 2024-04-21T08:55:32.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc92321640 1 --2- 172.21.15.82:0/2950686253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc8c102190 0x7fbc8c1986d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800c920 tx=0x7fbc8800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.200+0000 7fbc837fe640 1 -- 172.21.15.82:0/2950686253 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88007c70 con 0x7fbc8c102190 2024-04-21T08:55:32.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.201+0000 7fbc837fe640 1 -- 172.21.15.82:0/2950686253 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc88007e10 con 0x7fbc8c102190 2024-04-21T08:55:32.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.201+0000 7fbc94585640 1 -- 172.21.15.82:0/2950686253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c19dd70 con 0x7fbc8c102190 2024-04-21T08:55:32.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.201+0000 7fbc837fe640 1 -- 172.21.15.82:0/2950686253 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88018820 con 0x7fbc8c102190 2024-04-21T08:55:32.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.201+0000 7fbc94585640 1 -- 172.21.15.82:0/2950686253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c1a5560 con 0x7fbc8c102190 2024-04-21T08:55:32.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.203+0000 7fbc837fe640 1 -- 172.21.15.82:0/2950686253 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbc8800f450 con 0x7fbc8c102190 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.203+0000 7fbc97174740 1 Processor -- start 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.203+0000 7fbc97174740 1 -- start start 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.203+0000 7fbc97174740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 0x55e7306bc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.203+0000 7fbc97174740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bcd60 0x55e7306c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.203+0000 7fbc97174740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306c5e60 0x55e7306c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.203+0000 7fbc837fe640 1 --2- 172.21.15.82:0/2950686253 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbc780a3d20 0x7fbc780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.203+0000 7fbc97174740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e7306211f0 con 0x55e7306bc3b0 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc97174740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e7305be4b0 con 0x55e7306bcd60 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc97174740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e73061d1b0 con 0x55e7306c5e60 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92b22640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 0x55e7306bc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92b22640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 0x55e7306bc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42768/0 (socket says 172.21.15.82:42768) 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92b22640 1 -- 172.21.15.82:0/458441185 learned_addr learned my addr 172.21.15.82:0/458441185 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92b22640 1 --2- 172.21.15.82:0/2950686253 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbc780a3d20 0x7fbc780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc837fe640 1 -- 172.21.15.82:0/2950686253 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fbc880d0de0 con 0x7fbc8c102190 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92321640 1 --2- 172.21.15.82:0/458441185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bcd60 0x55e7306c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc91b20640 1 --2- 172.21.15.82:0/458441185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306c5e60 0x55e7306c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92b22640 1 -- 172.21.15.82:0/458441185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306c5e60 msgr2=0x55e7306c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92b22640 1 --2- 172.21.15.82:0/458441185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306c5e60 0x55e7306c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92b22640 1 -- 172.21.15.82:0/458441185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bcd60 msgr2=0x55e7306c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92b22640 1 --2- 172.21.15.82:0/458441185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bcd60 0x55e7306c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92b22640 1 -- 172.21.15.82:0/458441185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7305b01b0 con 0x55e7306bc3b0 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92321640 1 --2- 172.21.15.82:0/458441185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bcd60 0x55e7306c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92b22640 1 --2- 172.21.15.82:0/458441185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 0x55e7306bc790 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fbc84004af0 tx=0x7fbc84032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7e741c71fad38 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc92b22640 1 --2- 172.21.15.82:0/2950686253 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbc780a3d20 0x7fbc780a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbc84005cc0 tx=0x7fbc84045000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc80bfe640 1 -- 172.21.15.82:0/458441185 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84039040 con 0x55e7306bc3b0 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc80bfe640 1 -- 172.21.15.82:0/458441185 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc8404f070 con 0x55e7306bc3b0 2024-04-21T08:55:32.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.204+0000 7fbc80bfe640 1 -- 172.21.15.82:0/458441185 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84034d70 con 0x55e7306bc3b0 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.205+0000 7fbc97174740 1 -- 172.21.15.82:0/458441185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 msgr2=0x55e7306bc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.205+0000 7fbc97174740 1 --2- 172.21.15.82:0/458441185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 0x55e7306bc790 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fbc84004af0 tx=0x7fbc84032eb0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.205+0000 7fbc97174740 1 -- 172.21.15.82:0/458441185 shutdown_connections 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.205+0000 7fbc97174740 1 --2- 172.21.15.82:0/458441185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306c5e60 0x55e7306c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.205+0000 7fbc97174740 1 --2- 172.21.15.82:0/458441185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bcd60 0x55e7306c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.205+0000 7fbc97174740 1 --2- 172.21.15.82:0/458441185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 0x55e7306bc790 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.205+0000 7fbc97174740 1 -- 172.21.15.82:0/458441185 >> 172.21.15.82:0/458441185 conn(0x55e7306b6ba0 msgr2=0x55e7306b6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.205+0000 7fbc97174740 1 -- 172.21.15.82:0/458441185 shutdown_connections 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.205+0000 7fbc97174740 1 -- 172.21.15.82:0/458441185 wait complete. 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc97174740 1 -- start start 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc97174740 1 Processor -- start 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc97174740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bc3b0 0x55e7306cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc97174740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306bcd60 0x55e7306c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc97174740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306c5e60 0x55e7306c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc97174740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e7306ccd40 con 0x55e7306c5e60 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc97174740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e7305da500 con 0x55e7306bc3b0 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc97174740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e7306ccec0 con 0x55e7306bcd60 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc97174740 10 client.0 _ll_register_callbacks cb 0x55e7305a8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc92b22640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306c5e60 0x55e7306c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc92b22640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306c5e60 0x55e7306c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42778/0 (socket says 172.21.15.82:42778) 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc92b22640 1 -- 172.21.15.82:0/2073004624 learned_addr learned my addr 172.21.15.82:0/2073004624 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc92b22640 1 -- 172.21.15.82:0/2073004624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306bcd60 msgr2=0x55e7306c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc92b22640 1 --2- 172.21.15.82:0/2073004624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306bcd60 0x55e7306c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc92b22640 1 -- 172.21.15.82:0/2073004624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bc3b0 msgr2=0x55e7306cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc92b22640 1 --2- 172.21.15.82:0/2073004624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bc3b0 0x55e7306cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc92b22640 1 -- 172.21.15.82:0/2073004624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7303333f0 con 0x55e7306c5e60 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc92b22640 1 --2- 172.21.15.82:0/2073004624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306c5e60 0x55e7306c9cc0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fbc84005c90 tx=0x7fbc84009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ddbd90f41b4e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.206+0000 7fbc75ffb640 1 -- 172.21.15.82:0/2073004624 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84039050 con 0x55e7306c5e60 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc75ffb640 1 -- 172.21.15.82:0/2073004624 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fbc84035d50 con 0x55e7306c5e60 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc97174740 1 -- 172.21.15.82:0/2073004624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc8404f070 con 0x55e7306c5e60 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc75ffb640 1 -- 172.21.15.82:0/2073004624 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbc84061050 con 0x55e7306c5e60 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc97174740 10 client.18063 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc97174740 1 -- 172.21.15.82:0/2073004624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e7303fd7d0 con 0x55e7306c5e60 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc75ffb640 1 -- 172.21.15.82:0/2073004624 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc8404f070 con 0x55e7306c5e60 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc75ffb640 1 -- 172.21.15.82:0/2073004624 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fbc8403f050 con 0x55e7306c5e60 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc97174740 10 client.18063 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc97174740 10 client.18063 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc97174740 20 client.18063 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc97174740 1 --2- 172.21.15.82:0/2073004624 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55e7306d9e60 0x55e7306dc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc97174740 4 client.18063 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc97174740 1 -- 172.21.15.82:0/2073004624 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7305c0930 con 0x55e7306d9e60 2024-04-21T08:55:32.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.207+0000 7fbc92321640 1 --2- 172.21.15.82:0/2073004624 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55e7306d9e60 0x55e7306dc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.208+0000 7fbc75ffb640 10 client.18063 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.208+0000 7fbc92321640 1 --2- 172.21.15.82:0/2073004624 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55e7306d9e60 0x55e7306dc250 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58ac6411d90d920 server_cookie=d92631ee509ae9ac in_seq=0 out_seq=0 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.208+0000 7fbc75ffb640 1 -- 172.21.15.82:0/2073004624 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7305c0930 con 0x55e7306d9e60 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.208+0000 7fbc75ffb640 10 client.18063 handle_command_reply: tid=0 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.208+0000 7fbc97174740 1 client.18063 shutdown 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.209+0000 7fbc97174740 1 -- 172.21.15.82:0/2073004624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306c5e60 msgr2=0x55e7306c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.209+0000 7fbc97174740 1 --2- 172.21.15.82:0/2073004624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306c5e60 0x55e7306c9cc0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fbc84005c90 tx=0x7fbc84009640 comp rx=0 tx=0).stop 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.209+0000 7fbc97174740 1 -- 172.21.15.82:0/2073004624 shutdown_connections 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.209+0000 7fbc97174740 1 --2- 172.21.15.82:0/2073004624 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55e7306d9e60 0x55e7306dc250 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.209+0000 7fbc97174740 1 --2- 172.21.15.82:0/2073004624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306c5e60 0x55e7306c9cc0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.209+0000 7fbc75ffb640 0 client.18063 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.209+0000 7fbc97174740 1 --2- 172.21.15.82:0/2073004624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306bcd60 0x55e7306c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.209+0000 7fbc97174740 1 --2- 172.21.15.82:0/2073004624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bc3b0 0x55e7306cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.209+0000 7fbc97174740 1 -- 172.21.15.82:0/2073004624 >> 172.21.15.82:0/2073004624 conn(0x55e7306b6ba0 msgr2=0x55e7306b88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.209+0000 7fbc97174740 1 -- 172.21.15.82:0/2073004624 shutdown_connections 2024-04-21T08:55:32.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.209+0000 7fbc97174740 1 -- 172.21.15.82:0/2073004624 wait complete. 2024-04-21T08:55:32.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.209+0000 7fbc97174740 20 client.18063 trim_cache size 0 max 16384 2024-04-21T08:55:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.222+0000 7fbc97174740 1 Processor -- start 2024-04-21T08:55:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.222+0000 7fbc97174740 1 -- start start 2024-04-21T08:55:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.222+0000 7fbc97174740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bc3b0 0x55e7306d5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.222+0000 7fbc97174740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bcd60 0x55e7306c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.222+0000 7fbc97174740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306c5e60 0x55e7306cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.222+0000 7fbc97174740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1ad2b0 con 0x55e7306bcd60 2024-04-21T08:55:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.222+0000 7fbc97174740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1aee80 con 0x55e7306bc3b0 2024-04-21T08:55:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.222+0000 7fbc97174740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1ad980 con 0x55e7306c5e60 2024-04-21T08:55:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.222+0000 7fbc91b20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bcd60 0x55e7306c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.222+0000 7fbc91b20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bcd60 0x55e7306c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42792/0 (socket says 172.21.15.82:42792) 2024-04-21T08:55:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.222+0000 7fbc91b20640 1 -- 172.21.15.82:0/3355438355 learned_addr learned my addr 172.21.15.82:0/3355438355 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc92321640 1 --2- 172.21.15.82:0/3355438355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bc3b0 0x55e7306d5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc91b20640 1 -- 172.21.15.82:0/3355438355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306c5e60 msgr2=0x55e7306cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc92b22640 1 --2- 172.21.15.82:0/3355438355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306c5e60 0x55e7306cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc91b20640 1 --2- 172.21.15.82:0/3355438355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306c5e60 0x55e7306cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc91b20640 1 -- 172.21.15.82:0/3355438355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bc3b0 msgr2=0x55e7306d5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc91b20640 1 --2- 172.21.15.82:0/3355438355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bc3b0 0x55e7306d5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc91b20640 1 -- 172.21.15.82:0/3355438355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7306b2470 con 0x55e7306bcd60 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc92321640 1 --2- 172.21.15.82:0/3355438355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bc3b0 0x55e7306d5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc92b22640 1 --2- 172.21.15.82:0/3355438355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306c5e60 0x55e7306cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc91b20640 1 --2- 172.21.15.82:0/3355438355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bcd60 0x55e7306c7250 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c06b820 tx=0x7fbc7c00ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f45a3436cad8036 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc77fff640 1 -- 172.21.15.82:0/3355438355 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c004220 con 0x55e7306bcd60 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc77fff640 1 -- 172.21.15.82:0/3355438355 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc7c00fdf0 con 0x55e7306bcd60 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc77fff640 1 -- 172.21.15.82:0/3355438355 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c004a00 con 0x55e7306bcd60 2024-04-21T08:55:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.223+0000 7fbc97174740 1 -- 172.21.15.82:0/3355438355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bcd60 msgr2=0x55e7306c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 --2- 172.21.15.82:0/3355438355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bcd60 0x55e7306c7250 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c06b820 tx=0x7fbc7c00ddb0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 -- 172.21.15.82:0/3355438355 shutdown_connections 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 --2- 172.21.15.82:0/3355438355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306c5e60 0x55e7306cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 --2- 172.21.15.82:0/3355438355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bcd60 0x55e7306c7250 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 --2- 172.21.15.82:0/3355438355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306bc3b0 0x55e7306d5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 -- 172.21.15.82:0/3355438355 >> 172.21.15.82:0/3355438355 conn(0x55e7306b6ba0 msgr2=0x55e7306b8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 -- 172.21.15.82:0/3355438355 shutdown_connections 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 -- 172.21.15.82:0/3355438355 wait complete. 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 -- start start 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 Processor -- start 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 0x55e7306cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306bcd60 0x55e7306cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306c5e60 0x55e73073f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1ae780 con 0x55e7306bc3b0 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1ae080 con 0x55e7306c5e60 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c1ab280 con 0x55e7306bcd60 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc97174740 10 client.0 _ll_register_callbacks cb 0x7fbc84032bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc92321640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 0x55e7306cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc92321640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 0x55e7306cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42802/0 (socket says 172.21.15.82:42802) 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc92321640 1 -- 172.21.15.82:0/2366273231 learned_addr learned my addr 172.21.15.82:0/2366273231 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc92b22640 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306c5e60 0x55e73073f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc91b20640 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306bcd60 0x55e7306cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc92321640 1 -- 172.21.15.82:0/2366273231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306bcd60 msgr2=0x55e7306cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc92321640 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306bcd60 0x55e7306cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc92321640 1 -- 172.21.15.82:0/2366273231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306c5e60 msgr2=0x55e73073f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.224+0000 7fbc92321640 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306c5e60 0x55e73073f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc92321640 1 -- 172.21.15.82:0/2366273231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7306cc410 con 0x55e7306bc3b0 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc92321640 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 0x55e7306cb600 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fbc880ce180 tx=0x7fbc8809d130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e03aa62cd41b31f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc75ffb640 1 -- 172.21.15.82:0/2366273231 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880a55e0 con 0x55e7306bc3b0 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc75ffb640 1 -- 172.21.15.82:0/2366273231 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fbc8800fa90 con 0x55e7306bc3b0 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc97174740 1 -- 172.21.15.82:0/2366273231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc7c00fdf0 con 0x55e7306bc3b0 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc75ffb640 1 -- 172.21.15.82:0/2366273231 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbc8800f740 con 0x55e7306bc3b0 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc97174740 10 client.18069 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc97174740 1 -- 172.21.15.82:0/2366273231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e730740780 con 0x55e7306bc3b0 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc75ffb640 1 -- 172.21.15.82:0/2366273231 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc7c00fdf0 con 0x55e7306bc3b0 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc75ffb640 1 -- 172.21.15.82:0/2366273231 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fbc880acd60 con 0x55e7306bc3b0 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc97174740 10 client.18069 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc97174740 10 client.18069 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc97174740 20 client.18069 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc97174740 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55e730740e50 0x55e730743270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc97174740 4 client.18069 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc97174740 1 -- 172.21.15.82:0/2366273231 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e73062ec10 con 0x55e730740e50 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.225+0000 7fbc91b20640 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55e730740e50 0x55e730743270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc91b20640 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55e730740e50 0x55e730743270 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=817ff080670f5560 server_cookie=e414b0af04685bad in_seq=0 out_seq=0 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc75ffb640 10 client.18069 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc75ffb640 1 -- 172.21.15.82:0/2366273231 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e73062ec10 con 0x55e730740e50 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc75ffb640 10 client.18069 handle_command_reply: tid=0 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc97174740 1 client.18069 shutdown 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc97174740 1 -- 172.21.15.82:0/2366273231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 msgr2=0x55e7306cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc97174740 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 0x55e7306cb600 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fbc880ce180 tx=0x7fbc8809d130 comp rx=0 tx=0).stop 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc97174740 1 -- 172.21.15.82:0/2366273231 shutdown_connections 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc97174740 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x55e730740e50 0x55e730743270 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc97174740 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7306c5e60 0x55e73073f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc97174740 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7306bcd60 0x55e7306cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc75ffb640 0 client.18069 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:55:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc97174740 1 --2- 172.21.15.82:0/2366273231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7306bc3b0 0x55e7306cb600 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc97174740 1 -- 172.21.15.82:0/2366273231 >> 172.21.15.82:0/2366273231 conn(0x55e7306b6ba0 msgr2=0x55e7306b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc97174740 1 -- 172.21.15.82:0/2366273231 shutdown_connections 2024-04-21T08:55:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc97174740 1 -- 172.21.15.82:0/2366273231 wait complete. 2024-04-21T08:55:32.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.226+0000 7fbc97174740 20 client.18069 trim_cache size 0 max 16384 2024-04-21T08:55:32.234 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:55:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.227+0000 7fbc94585640 1 -- 172.21.15.82:0/2950686253 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbc780a3d20 msgr2=0x7fbc780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.227+0000 7fbc94585640 1 --2- 172.21.15.82:0/2950686253 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbc780a3d20 0x7fbc780a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbc84005cc0 tx=0x7fbc84045000 comp rx=0 tx=0).stop 2024-04-21T08:55:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.227+0000 7fbc94585640 1 -- 172.21.15.82:0/2950686253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc8c102190 msgr2=0x7fbc8c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.227+0000 7fbc94585640 1 --2- 172.21.15.82:0/2950686253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc8c102190 0x7fbc8c1986d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800c920 tx=0x7fbc8800cde0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.227+0000 7fbc94585640 1 -- 172.21.15.82:0/2950686253 shutdown_connections 2024-04-21T08:55:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.227+0000 7fbc94585640 1 --2- 172.21.15.82:0/2950686253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc8c19cf90 0x7fbc8c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.227+0000 7fbc94585640 1 --2- 172.21.15.82:0/2950686253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc8c107260 0x7fbc8c198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.227+0000 7fbc94585640 1 --2- 172.21.15.82:0/2950686253 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbc780a3d20 0x7fbc780a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.227+0000 7fbc94585640 1 --2- 172.21.15.82:0/2950686253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc8c102190 0x7fbc8c1986d0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.227+0000 7fbc94585640 1 -- 172.21.15.82:0/2950686253 >> 172.21.15.82:0/2950686253 conn(0x7fbc8c0793a0 msgr2=0x7fbc8c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.227+0000 7fbc94585640 1 -- 172.21.15.82:0/2950686253 shutdown_connections 2024-04-21T08:55:32.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.227+0000 7fbc94585640 1 -- 172.21.15.82:0/2950686253 wait complete. 2024-04-21T08:55:32.237 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:55:32.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.338+0000 7f07be477640 1 -- 172.21.15.82:0/2548182908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07b8107270 msgr2=0x7f07b810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.338+0000 7f07be477640 1 --2- 172.21.15.82:0/2548182908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07b8107270 0x7f07b810ee40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009ca0 tx=0x7f07b002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.338+0000 7f07be477640 1 -- 172.21.15.82:0/2548182908 shutdown_connections 2024-04-21T08:55:32.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.338+0000 7f07be477640 1 --2- 172.21.15.82:0/2548182908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07b8107270 0x7f07b810ee40 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.338+0000 7f07be477640 1 --2- 172.21.15.82:0/2548182908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07b806b230 0x7f07b8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.338+0000 7f07be477640 1 --2- 172.21.15.82:0/2548182908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07b806a880 0x7f07b806ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.338+0000 7f07be477640 1 -- 172.21.15.82:0/2548182908 >> 172.21.15.82:0/2548182908 conn(0x7f07b8077eb0 msgr2=0x7f07b80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.338+0000 7f07be477640 1 -- 172.21.15.82:0/2548182908 shutdown_connections 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.338+0000 7f07be477640 1 -- 172.21.15.82:0/2548182908 wait complete. 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.338+0000 7f07be477640 1 Processor -- start 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.338+0000 7f07be477640 1 -- start start 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07be477640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07b806a880 0x7f07b81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07be477640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07b8107270 0x7f07b81a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07be477640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07b819a730 0x7f07b819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07be477640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81121b0 con 0x7f07b8107270 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07be477640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8112030 con 0x7f07b806a880 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07be477640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8112330 con 0x7f07b819a730 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07b77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07b8107270 0x7f07b81a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07b77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07b8107270 0x7f07b81a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42818/0 (socket says 172.21.15.82:42818) 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07b77fe640 1 -- 172.21.15.82:0/4127419468 learned_addr learned my addr 172.21.15.82:0/4127419468 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07bca14640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07b819a730 0x7f07b819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07b7fff640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07b806a880 0x7f07b81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07b77fe640 1 -- 172.21.15.82:0/4127419468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07b819a730 msgr2=0x7f07b819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07b77fe640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07b819a730 0x7f07b819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07b77fe640 1 -- 172.21.15.82:0/4127419468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07b806a880 msgr2=0x7f07b81a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07b77fe640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07b806a880 0x7f07b81a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07b77fe640 1 -- 172.21.15.82:0/4127419468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b819b310 con 0x7f07b8107270 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07bca14640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07b819a730 0x7f07b819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07b7fff640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07b806a880 0x7f07b81a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07b77fe640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07b8107270 0x7f07b81a0b20 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00c970 tx=0x7f07ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.339+0000 7f07b57fa640 1 -- 172.21.15.82:0/4127419468 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac007bb0 con 0x7f07b8107270 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.340+0000 7f07b57fa640 1 -- 172.21.15.82:0/4127419468 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07ac007d50 con 0x7f07b8107270 2024-04-21T08:55:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.340+0000 7f07b57fa640 1 -- 172.21.15.82:0/4127419468 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac00f450 con 0x7f07b8107270 2024-04-21T08:55:32.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.340+0000 7f07be477640 1 -- 172.21.15.82:0/4127419468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b81a73f0 con 0x7f07b8107270 2024-04-21T08:55:32.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.340+0000 7f07be477640 1 -- 172.21.15.82:0/4127419468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b81a77f0 con 0x7f07b8107270 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07b57fa640 1 -- 172.21.15.82:0/4127419468 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f07ac004f20 con 0x7f07b8107270 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07c1061740 1 Processor -- start 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07c1061740 1 -- start start 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07b57fa640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f07a00a3da0 0x7f07a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07c1061740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 0x561b19f058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07c1061740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f05e80 0x561b19f0e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07c1061740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f0ea00 0x561b19f10e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07c1061740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561b19e6a1f0 con 0x561b19f054d0 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07c1061740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561b19e074b0 con 0x561b19f0ea00 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07c1061740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561b19e661b0 con 0x561b19f05e80 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07b7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 0x561b19f058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07bca14640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f07a00a3da0 0x7f07a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07b7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 0x561b19f058b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42832/0 (socket says 172.21.15.82:42832) 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07b57fa640 1 -- 172.21.15.82:0/4127419468 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f07ac0d0b90 con 0x7f07b8107270 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.342+0000 7f07b7fff640 1 -- 172.21.15.82:0/1882485357 learned_addr learned my addr 172.21.15.82:0/1882485357 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.343+0000 7f07b77fe640 1 --2- 172.21.15.82:0/1882485357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f05e80 0x561b19f0e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.343+0000 7f07b7fff640 1 -- 172.21.15.82:0/1882485357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f05e80 msgr2=0x561b19f0e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.343+0000 7f07b7fff640 1 --2- 172.21.15.82:0/1882485357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f05e80 0x561b19f0e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.343+0000 7f07b7fff640 1 -- 172.21.15.82:0/1882485357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f0ea00 msgr2=0x561b19f10e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.343+0000 7f07b7fff640 1 --2- 172.21.15.82:0/1882485357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f0ea00 0x561b19f10e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.343+0000 7f07b7fff640 1 -- 172.21.15.82:0/1882485357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b19df91b0 con 0x561b19f054d0 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.343+0000 7f07b7fff640 1 --2- 172.21.15.82:0/1882485357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 0x561b19f058b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f07a800bc20 tx=0x7f07a800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5ae69f964daab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.343+0000 7f07bca14640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f07a00a3da0 0x7f07a00a6260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f07b0006010 tx=0x7f07b0005fa0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.343+0000 7f079e7fc640 1 -- 172.21.15.82:0/1882485357 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8017620 con 0x561b19f054d0 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.343+0000 7f079e7fc640 1 -- 172.21.15.82:0/1882485357 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a8017c70 con 0x561b19f054d0 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.343+0000 7f079e7fc640 1 -- 172.21.15.82:0/1882485357 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a800fb40 con 0x561b19f054d0 2024-04-21T08:55:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.344+0000 7f07c1061740 1 -- 172.21.15.82:0/1882485357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 msgr2=0x561b19f058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.344+0000 7f07c1061740 1 --2- 172.21.15.82:0/1882485357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 0x561b19f058b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f07a800bc20 tx=0x7f07a800d710 comp rx=0 tx=0).stop 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.344+0000 7f07c1061740 1 -- 172.21.15.82:0/1882485357 shutdown_connections 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.344+0000 7f07c1061740 1 --2- 172.21.15.82:0/1882485357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f0ea00 0x561b19f10e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.344+0000 7f07c1061740 1 --2- 172.21.15.82:0/1882485357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f05e80 0x561b19f0e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.344+0000 7f07c1061740 1 --2- 172.21.15.82:0/1882485357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 0x561b19f058b0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.344+0000 7f07c1061740 1 -- 172.21.15.82:0/1882485357 >> 172.21.15.82:0/1882485357 conn(0x561b19effba0 msgr2=0x561b19efff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.344+0000 7f07c1061740 1 -- 172.21.15.82:0/1882485357 shutdown_connections 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.344+0000 7f07c1061740 1 -- 172.21.15.82:0/1882485357 wait complete. 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.344+0000 7f07c1061740 1 -- start start 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.344+0000 7f07c1061740 1 Processor -- start 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.344+0000 7f07c1061740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f054d0 0x561b19f0e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07c1061740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f05e80 0x561b19f0b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07c1061740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f0ea00 0x561b19f0bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07c1061740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561b19f11970 con 0x561b19f0ea00 2024-04-21T08:55:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07c1061740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561b19e23500 con 0x561b19f05e80 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07c1061740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561b19f11af0 con 0x561b19f054d0 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07c1061740 10 client.0 _ll_register_callbacks cb 0x561b19df1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07bca14640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f0ea00 0x561b19f0bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07bca14640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f0ea00 0x561b19f0bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42846/0 (socket says 172.21.15.82:42846) 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07bca14640 1 -- 172.21.15.82:0/937593904 learned_addr learned my addr 172.21.15.82:0/937593904 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07b7fff640 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f054d0 0x561b19f0e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07b77fe640 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f05e80 0x561b19f0b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07bca14640 1 -- 172.21.15.82:0/937593904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f054d0 msgr2=0x561b19f0e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07bca14640 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f054d0 0x561b19f0e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07bca14640 1 -- 172.21.15.82:0/937593904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f05e80 msgr2=0x561b19f0b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07bca14640 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f05e80 0x561b19f0b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07bca14640 1 -- 172.21.15.82:0/937593904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b19b7c3f0 con 0x561b19f0ea00 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07bca14640 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f0ea00 0x561b19f0bbc0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009ca0 tx=0x7f07b00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e6140506c425c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07977fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07977fe640 1 -- 172.21.15.82:0/937593904 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0051060 con 0x561b19f0ea00 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07977fe640 1 -- 172.21.15.82:0/937593904 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f07b0004510 con 0x561b19f0ea00 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07c1061740 1 -- 172.21.15.82:0/937593904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a8017c70 con 0x561b19f0ea00 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07977fe640 1 -- 172.21.15.82:0/937593904 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f07b0058050 con 0x561b19f0ea00 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07c1061740 10 client.18081 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.345+0000 7f07c1061740 1 -- 172.21.15.82:0/937593904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561b19c467d0 con 0x561b19f0ea00 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.346+0000 7f07977fe640 1 -- 172.21.15.82:0/937593904 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a8017c70 con 0x561b19f0ea00 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.346+0000 7f07977fe640 1 -- 172.21.15.82:0/937593904 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f07b0040ba0 con 0x561b19f0ea00 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.346+0000 7f07c1061740 10 client.18081 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.346+0000 7f07c1061740 10 client.18081 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.346+0000 7f07c1061740 20 client.18081 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.346+0000 7f07c1061740 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x561b19f1ebd0 0x561b19f21050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.346+0000 7f07c1061740 4 client.18081 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.346+0000 7f07c1061740 1 -- 172.21.15.82:0/937593904 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b19e09930 con 0x561b19f1ebd0 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.346+0000 7f07b7fff640 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x561b19f1ebd0 0x561b19f21050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.346+0000 7f07977fe640 10 client.18081 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:55:32.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.346+0000 7f07b7fff640 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x561b19f1ebd0 0x561b19f21050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2206198474d2925a server_cookie=78f69f98aaf7318d in_seq=0 out_seq=0 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07977fe640 1 -- 172.21.15.82:0/937593904 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b19e09930 con 0x561b19f1ebd0 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07977fe640 10 client.18081 handle_command_reply: tid=0 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07c1061740 1 client.18081 shutdown 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07c1061740 1 -- 172.21.15.82:0/937593904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f0ea00 msgr2=0x561b19f0bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07c1061740 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f0ea00 0x561b19f0bbc0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009ca0 tx=0x7f07b00095c0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07c1061740 1 -- 172.21.15.82:0/937593904 shutdown_connections 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07c1061740 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x561b19f1ebd0 0x561b19f21050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07c1061740 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f0ea00 0x561b19f0bbc0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07c1061740 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f05e80 0x561b19f0b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07977fe640 0 client.18081 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07c1061740 1 --2- 172.21.15.82:0/937593904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f054d0 0x561b19f0e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07c1061740 1 -- 172.21.15.82:0/937593904 >> 172.21.15.82:0/937593904 conn(0x561b19effba0 msgr2=0x561b19f018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.347+0000 7f07c1061740 1 -- 172.21.15.82:0/937593904 shutdown_connections 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.348+0000 7f07c1061740 1 -- 172.21.15.82:0/937593904 wait complete. 2024-04-21T08:55:32.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.348+0000 7f07c1061740 20 client.18081 trim_cache size 0 max 16384 2024-04-21T08:55:32.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:32 smithi082 ceph-mon[27967]: pgmap v262: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:55:32.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:32 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1799312163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:55:32.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07c1061740 1 Processor -- start 2024-04-21T08:55:32.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07c1061740 1 -- start start 2024-04-21T08:55:32.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07c1061740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 0x561b19f1a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07c1061740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f05e80 0x561b19efea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07c1061740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f0ea00 0x561b19f0dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07c1061740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b1670 con 0x561b19f054d0 2024-04-21T08:55:32.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07c1061740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b3240 con 0x561b19f05e80 2024-04-21T08:55:32.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07c1061740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b1d40 con 0x561b19f0ea00 2024-04-21T08:55:32.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07b7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 0x561b19f1a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07b7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 0x561b19f1a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42850/0 (socket says 172.21.15.82:42850) 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07b7fff640 1 -- 172.21.15.82:0/3977069224 learned_addr learned my addr 172.21.15.82:0/3977069224 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07bca14640 1 --2- 172.21.15.82:0/3977069224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f0ea00 0x561b19f0dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07b77fe640 1 --2- 172.21.15.82:0/3977069224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f05e80 0x561b19efea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07b7fff640 1 -- 172.21.15.82:0/3977069224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f0ea00 msgr2=0x561b19f0dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07b7fff640 1 --2- 172.21.15.82:0/3977069224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f0ea00 0x561b19f0dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07b7fff640 1 -- 172.21.15.82:0/3977069224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f05e80 msgr2=0x561b19efea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07b7fff640 1 --2- 172.21.15.82:0/3977069224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f05e80 0x561b19efea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.361+0000 7f07b7fff640 1 -- 172.21.15.82:0/3977069224 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b19efb470 con 0x561b19f054d0 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f07b7fff640 1 --2- 172.21.15.82:0/3977069224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 0x561b19f1a830 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f07a800bbf0 tx=0x7f07a800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31059e6a0bebb96 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f079dffb640 1 -- 172.21.15.82:0/3977069224 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a802d070 con 0x561b19f054d0 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f079dffb640 1 -- 172.21.15.82:0/3977069224 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a800b750 con 0x561b19f054d0 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f079dffb640 1 -- 172.21.15.82:0/3977069224 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8005040 con 0x561b19f054d0 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f07c1061740 1 -- 172.21.15.82:0/3977069224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 msgr2=0x561b19f1a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f07c1061740 1 --2- 172.21.15.82:0/3977069224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 0x561b19f1a830 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f07a800bbf0 tx=0x7f07a800d710 comp rx=0 tx=0).stop 2024-04-21T08:55:32.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f07c1061740 1 -- 172.21.15.82:0/3977069224 shutdown_connections 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f07c1061740 1 --2- 172.21.15.82:0/3977069224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f0ea00 0x561b19f0dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f07c1061740 1 --2- 172.21.15.82:0/3977069224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f05e80 0x561b19efea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f07c1061740 1 --2- 172.21.15.82:0/3977069224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f054d0 0x561b19f1a830 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f07c1061740 1 -- 172.21.15.82:0/3977069224 >> 172.21.15.82:0/3977069224 conn(0x561b19effba0 msgr2=0x561b19efe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f07c1061740 1 -- 172.21.15.82:0/3977069224 shutdown_connections 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.362+0000 7f07c1061740 1 -- 172.21.15.82:0/3977069224 wait complete. 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07c1061740 1 -- start start 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07c1061740 1 Processor -- start 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07c1061740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f054d0 0x561b19f16f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07c1061740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f05e80 0x561b19f12340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07c1061740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f0ea00 0x561b19f12880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07c1061740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b2b40 con 0x561b19f05e80 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07c1061740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b2440 con 0x561b19f0ea00 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07c1061740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81af640 con 0x561b19f054d0 2024-04-21T08:55:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07c1061740 10 client.0 _ll_register_callbacks cb 0x7f07b0032860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07b77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f05e80 0x561b19f12340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07b77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f05e80 0x561b19f12340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42852/0 (socket says 172.21.15.82:42852) 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07b77fe640 1 -- 172.21.15.82:0/221692306 learned_addr learned my addr 172.21.15.82:0/221692306 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07b7fff640 1 --2- 172.21.15.82:0/221692306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f054d0 0x561b19f16f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07b77fe640 1 -- 172.21.15.82:0/221692306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f054d0 msgr2=0x561b19f16f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07b77fe640 1 --2- 172.21.15.82:0/221692306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f054d0 0x561b19f16f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07b77fe640 1 -- 172.21.15.82:0/221692306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f0ea00 msgr2=0x561b19f12880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07b77fe640 1 --2- 172.21.15.82:0/221692306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f0ea00 0x561b19f12880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07b77fe640 1 -- 172.21.15.82:0/221692306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b19f12ef0 con 0x561b19f05e80 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07b77fe640 1 --2- 172.21.15.82:0/221692306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f05e80 0x561b19f12340 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0cdf30 tx=0x7f07ac09cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975d63cc0b12a3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.363+0000 7f07977fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07977fe640 1 -- 172.21.15.82:0/221692306 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac01d040 con 0x561b19f05e80 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07977fe640 1 -- 172.21.15.82:0/221692306 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f07ac0046c0 con 0x561b19f05e80 2024-04-21T08:55:32.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07c1061740 1 -- 172.21.15.82:0/221692306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a800b750 con 0x561b19f05e80 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07977fe640 1 -- 172.21.15.82:0/221692306 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f07ac0a5390 con 0x561b19f05e80 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07c1061740 10 client.18087 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07c1061740 1 -- 172.21.15.82:0/221692306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561b19f85080 con 0x561b19f05e80 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07977fe640 1 -- 172.21.15.82:0/221692306 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a800b750 con 0x561b19f05e80 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07977fe640 1 -- 172.21.15.82:0/221692306 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f07ac005210 con 0x561b19f05e80 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07c1061740 10 client.18087 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07c1061740 10 client.18087 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07c1061740 20 client.18087 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07c1061740 1 --2- 172.21.15.82:0/221692306 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x561b19f85860 0x561b19f87c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07c1061740 4 client.18087 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07c1061740 1 -- 172.21.15.82:0/221692306 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b19e77c10 con 0x561b19f85860 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.364+0000 7f07b7fff640 1 --2- 172.21.15.82:0/221692306 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x561b19f85860 0x561b19f87c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07b7fff640 1 --2- 172.21.15.82:0/221692306 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x561b19f85860 0x561b19f87c50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44a3d616adeab302 server_cookie=a5593c49f5a80230 in_seq=0 out_seq=0 2024-04-21T08:55:32.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07977fe640 10 client.18087 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07977fe640 1 -- 172.21.15.82:0/221692306 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b19e77c10 con 0x561b19f85860 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07977fe640 10 client.18087 handle_command_reply: tid=0 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07c1061740 1 client.18087 shutdown 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07c1061740 1 -- 172.21.15.82:0/221692306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f05e80 msgr2=0x561b19f12340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07c1061740 1 --2- 172.21.15.82:0/221692306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f05e80 0x561b19f12340 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0cdf30 tx=0x7f07ac09cee0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07c1061740 1 -- 172.21.15.82:0/221692306 shutdown_connections 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07c1061740 1 --2- 172.21.15.82:0/221692306 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x561b19f85860 0x561b19f87c50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07c1061740 1 --2- 172.21.15.82:0/221692306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561b19f0ea00 0x561b19f12880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07c1061740 1 --2- 172.21.15.82:0/221692306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b19f05e80 0x561b19f12340 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07977fe640 0 client.18087 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07c1061740 1 --2- 172.21.15.82:0/221692306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b19f054d0 0x561b19f16f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07c1061740 1 -- 172.21.15.82:0/221692306 >> 172.21.15.82:0/221692306 conn(0x561b19effba0 msgr2=0x561b19efec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07c1061740 1 -- 172.21.15.82:0/221692306 shutdown_connections 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07c1061740 1 -- 172.21.15.82:0/221692306 wait complete. 2024-04-21T08:55:32.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.365+0000 7f07c1061740 20 client.18087 trim_cache size 0 max 16384 2024-04-21T08:55:32.372 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:55:32.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.366+0000 7f07be477640 1 -- 172.21.15.82:0/4127419468 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f07a00a3da0 msgr2=0x7f07a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.366+0000 7f07be477640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f07a00a3da0 0x7f07a00a6260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f07b0006010 tx=0x7f07b0005fa0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.366+0000 7f07be477640 1 -- 172.21.15.82:0/4127419468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07b8107270 msgr2=0x7f07b81a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.366+0000 7f07be477640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07b8107270 0x7f07b81a0b20 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00c970 tx=0x7f07ac00ce30 comp rx=0 tx=0).stop 2024-04-21T08:55:32.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.366+0000 7f07be477640 1 -- 172.21.15.82:0/4127419468 shutdown_connections 2024-04-21T08:55:32.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.366+0000 7f07be477640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07b819a730 0x7f07b819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.366+0000 7f07be477640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f07a00a3da0 0x7f07a00a6260 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.366+0000 7f07be477640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07b8107270 0x7f07b81a0b20 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.366+0000 7f07be477640 1 --2- 172.21.15.82:0/4127419468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07b806a880 0x7f07b81a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.366+0000 7f07be477640 1 -- 172.21.15.82:0/4127419468 >> 172.21.15.82:0/4127419468 conn(0x7f07b8077eb0 msgr2=0x7f07b810d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.366+0000 7f07be477640 1 -- 172.21.15.82:0/4127419468 shutdown_connections 2024-04-21T08:55:32.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.366+0000 7f07be477640 1 -- 172.21.15.82:0/4127419468 wait complete. 2024-04-21T08:55:32.376 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:55:32.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.476+0000 7ff926a3f640 1 -- 172.21.15.82:0/1394093239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff920107030 msgr2=0x7ff920107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.476+0000 7ff926a3f640 1 --2- 172.21.15.82:0/1394093239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff920107030 0x7ff920107410 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ff908009930 tx=0x7ff90802bc30 comp rx=0 tx=0).stop 2024-04-21T08:55:32.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.476+0000 7ff926a3f640 1 -- 172.21.15.82:0/1394093239 shutdown_connections 2024-04-21T08:55:32.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.476+0000 7ff926a3f640 1 --2- 172.21.15.82:0/1394093239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff920101b40 0x7ff92010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.476+0000 7ff926a3f640 1 --2- 172.21.15.82:0/1394093239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9201011a0 0x7ff920101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.476+0000 7ff926a3f640 1 --2- 172.21.15.82:0/1394093239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff920107030 0x7ff920107410 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.476+0000 7ff926a3f640 1 -- 172.21.15.82:0/1394093239 >> 172.21.15.82:0/1394093239 conn(0x7ff920076c50 msgr2=0x7ff920077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.476+0000 7ff926a3f640 1 -- 172.21.15.82:0/1394093239 shutdown_connections 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.476+0000 7ff926a3f640 1 -- 172.21.15.82:0/1394093239 wait complete. 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.476+0000 7ff926a3f640 1 Processor -- start 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.476+0000 7ff926a3f640 1 -- start start 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff926a3f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9201011a0 0x7ff92019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff926a3f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff920101b40 0x7ff92019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff926a3f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff920107030 0x7ff9201985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff926a3f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff920112220 con 0x7ff920107030 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff926a3f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201120a0 con 0x7ff920101b40 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff926a3f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201123a0 con 0x7ff9201011a0 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff924fdc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff920107030 0x7ff9201985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff924fdc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff920107030 0x7ff9201985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42878/0 (socket says 172.21.15.82:42878) 2024-04-21T08:55:32.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff924fdc640 1 -- 172.21.15.82:0/664303871 learned_addr learned my addr 172.21.15.82:0/664303871 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff91ffff640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9201011a0 0x7ff92019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff920101b40 0x7ff92019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff924fdc640 1 -- 172.21.15.82:0/664303871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9201011a0 msgr2=0x7ff92019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff924fdc640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9201011a0 0x7ff92019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff924fdc640 1 -- 172.21.15.82:0/664303871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff920101b40 msgr2=0x7ff92019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff924fdc640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff920101b40 0x7ff92019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff924fdc640 1 -- 172.21.15.82:0/664303871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff920198d40 con 0x7ff920107030 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff91ffff640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9201011a0 0x7ff92019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff924fdc640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff920107030 0x7ff9201985c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff91800cad0 tx=0x7ff91800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff91d7fa640 1 -- 172.21.15.82:0/664303871 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91800ae20 con 0x7ff920107030 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff91d7fa640 1 -- 172.21.15.82:0/664303871 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff918005020 con 0x7ff920107030 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff926a3f640 1 -- 172.21.15.82:0/664303871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff920199030 con 0x7ff920107030 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff91d7fa640 1 -- 172.21.15.82:0/664303871 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91800f6b0 con 0x7ff920107030 2024-04-21T08:55:32.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.477+0000 7ff926a3f640 1 -- 172.21.15.82:0/664303871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9201a52d0 con 0x7ff920107030 2024-04-21T08:55:32.481 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:32 smithi134 ceph-mon[31772]: pgmap v262: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:55:32.481 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:32 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1799312163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:55:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff91d7fa640 1 -- 172.21.15.82:0/664303871 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff9180051d0 con 0x7ff920107030 2024-04-21T08:55:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff92962d740 1 Processor -- start 2024-04-21T08:55:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff92962d740 1 -- start start 2024-04-21T08:55:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff92962d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e698434d0 0x555e698438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff92962d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e69843e80 0x555e6984c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff92962d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 0x555e6984ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff92962d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555e697a81f0 con 0x555e6984ca00 2024-04-21T08:55:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff92962d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555e697454b0 con 0x555e698434d0 2024-04-21T08:55:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff92962d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e697a41b0 con 0x555e69843e80 2024-04-21T08:55:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff91d7fa640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff90c0a3d20 0x7ff90c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff91f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 0x555e6984ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff91f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 0x555e6984ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42894/0 (socket says 172.21.15.82:42894) 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff91f7fe640 1 -- 172.21.15.82:0/644683742 learned_addr learned my addr 172.21.15.82:0/644683742 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff924fdc640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff90c0a3d20 0x7ff90c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/644683742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e698434d0 0x555e698438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.480+0000 7ff91d7fa640 1 -- 172.21.15.82:0/664303871 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7ff9180d0d70 con 0x7ff920107030 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.481+0000 7ff91ffff640 1 --2- 172.21.15.82:0/644683742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e69843e80 0x555e6984c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.481+0000 7ff91f7fe640 1 -- 172.21.15.82:0/644683742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e69843e80 msgr2=0x555e6984c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.481+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/644683742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e69843e80 0x555e6984c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.481+0000 7ff91f7fe640 1 -- 172.21.15.82:0/644683742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e698434d0 msgr2=0x555e698438b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.481+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/644683742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e698434d0 0x555e698438b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.481+0000 7ff91f7fe640 1 -- 172.21.15.82:0/644683742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e697371b0 con 0x555e6984ca00 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.481+0000 7ff924fdc640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff90c0a3d20 0x7ff90c0a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff918098d60 tx=0x7ff91806f3f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.481+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/644683742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 0x555e6984ee50 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ff910013910 tx=0x7ff91000e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992b1395d17bd404 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.481+0000 7ff9167fc640 1 -- 172.21.15.82:0/644683742 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91000b8f0 con 0x555e6984ca00 2024-04-21T08:55:32.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.481+0000 7ff9167fc640 1 -- 172.21.15.82:0/644683742 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff910005020 con 0x555e6984ca00 2024-04-21T08:55:32.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.481+0000 7ff9167fc640 1 -- 172.21.15.82:0/644683742 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9100049a0 con 0x555e6984ca00 2024-04-21T08:55:32.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 -- 172.21.15.82:0/644683742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 msgr2=0x555e6984ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 --2- 172.21.15.82:0/644683742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 0x555e6984ee50 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ff910013910 tx=0x7ff91000e430 comp rx=0 tx=0).stop 2024-04-21T08:55:32.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 -- 172.21.15.82:0/644683742 shutdown_connections 2024-04-21T08:55:32.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 --2- 172.21.15.82:0/644683742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 0x555e6984ee50 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 --2- 172.21.15.82:0/644683742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e69843e80 0x555e6984c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 --2- 172.21.15.82:0/644683742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e698434d0 0x555e698438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 -- 172.21.15.82:0/644683742 >> 172.21.15.82:0/644683742 conn(0x555e6983dba0 msgr2=0x555e6983df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 -- 172.21.15.82:0/644683742 shutdown_connections 2024-04-21T08:55:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 -- 172.21.15.82:0/644683742 wait complete. 2024-04-21T08:55:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 -- start start 2024-04-21T08:55:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 Processor -- start 2024-04-21T08:55:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e698434d0 0x555e6984c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e69843e80 0x555e69849690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e6984ca00 0x555e69849c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555e6984f970 con 0x555e698434d0 2024-04-21T08:55:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555e69761500 con 0x555e69843e80 2024-04-21T08:55:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.482+0000 7ff92962d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e6984faf0 con 0x555e6984ca00 2024-04-21T08:55:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff92962d740 10 client.0 _ll_register_callbacks cb 0x555e6972fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff91f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e698434d0 0x555e6984c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff91f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e698434d0 0x555e6984c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42902/0 (socket says 172.21.15.82:42902) 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff91f7fe640 1 -- 172.21.15.82:0/1074834868 learned_addr learned my addr 172.21.15.82:0/1074834868 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/1074834868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e6984ca00 0x555e69849c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff91f7fe640 1 -- 172.21.15.82:0/1074834868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e6984ca00 msgr2=0x555e69849c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/1074834868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e6984ca00 0x555e69849c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff91f7fe640 1 -- 172.21.15.82:0/1074834868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e69843e80 msgr2=0x555e69849690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/1074834868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e69843e80 0x555e69849690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff91f7fe640 1 -- 172.21.15.82:0/1074834868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e694ba3f0 con 0x555e698434d0 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/1074834868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e698434d0 0x555e6984c300 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff91000e7a0 tx=0x7ff9100135b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97aa6ffe0ed180a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff8ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff8ff7fe640 1 -- 172.21.15.82:0/1074834868 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91000bdc0 con 0x555e698434d0 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff8ff7fe640 1 -- 172.21.15.82:0/1074834868 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7ff910004020 con 0x555e698434d0 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff92962d740 1 -- 172.21.15.82:0/1074834868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff910005020 con 0x555e698434d0 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff8ff7fe640 1 -- 172.21.15.82:0/1074834868 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff91002f050 con 0x555e698434d0 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff92962d740 10 client.18099 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff92962d740 1 -- 172.21.15.82:0/1074834868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555e695847d0 con 0x555e698434d0 2024-04-21T08:55:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.483+0000 7ff8ff7fe640 1 -- 172.21.15.82:0/1074834868 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff910005020 con 0x555e698434d0 2024-04-21T08:55:32.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.484+0000 7ff8ff7fe640 1 -- 172.21.15.82:0/1074834868 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7ff9100035f0 con 0x555e698434d0 2024-04-21T08:55:32.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.484+0000 7ff92962d740 10 client.18099 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:32.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.484+0000 7ff92962d740 10 client.18099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:32.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.484+0000 7ff92962d740 20 client.18099 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:32.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.484+0000 7ff92962d740 1 --2- 172.21.15.82:0/1074834868 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555e6985cb90 0x555e6985ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.484+0000 7ff92962d740 4 client.18099 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:55:32.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.484+0000 7ff92962d740 1 -- 172.21.15.82:0/1074834868 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e69747930 con 0x555e6985cb90 2024-04-21T08:55:32.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.484+0000 7ff91ffff640 1 --2- 172.21.15.82:0/1074834868 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555e6985cb90 0x555e6985ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.484+0000 7ff91ffff640 1 --2- 172.21.15.82:0/1074834868 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555e6985cb90 0x555e6985ef80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc33099d6170 server_cookie=4861598ebcf91f88 in_seq=0 out_seq=0 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.484+0000 7ff8ff7fe640 10 client.18099 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff8ff7fe640 1 -- 172.21.15.82:0/1074834868 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e69747930 con 0x555e6985cb90 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff8ff7fe640 10 client.18099 handle_command_reply: tid=0 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff92962d740 1 client.18099 shutdown 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff92962d740 1 -- 172.21.15.82:0/1074834868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e698434d0 msgr2=0x555e6984c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff92962d740 1 --2- 172.21.15.82:0/1074834868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e698434d0 0x555e6984c300 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff91000e7a0 tx=0x7ff9100135b0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff92962d740 1 -- 172.21.15.82:0/1074834868 shutdown_connections 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff92962d740 1 --2- 172.21.15.82:0/1074834868 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555e6985cb90 0x555e6985ef80 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff92962d740 1 --2- 172.21.15.82:0/1074834868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e6984ca00 0x555e69849c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff8ff7fe640 0 client.18099 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff92962d740 1 --2- 172.21.15.82:0/1074834868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e69843e80 0x555e69849690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff92962d740 1 --2- 172.21.15.82:0/1074834868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e698434d0 0x555e6984c300 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff92962d740 1 -- 172.21.15.82:0/1074834868 >> 172.21.15.82:0/1074834868 conn(0x555e6983dba0 msgr2=0x555e6983f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff92962d740 1 -- 172.21.15.82:0/1074834868 shutdown_connections 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.485+0000 7ff92962d740 1 -- 172.21.15.82:0/1074834868 wait complete. 2024-04-21T08:55:32.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.486+0000 7ff92962d740 20 client.18099 trim_cache size 0 max 16384 2024-04-21T08:55:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.498+0000 7ff92962d740 1 Processor -- start 2024-04-21T08:55:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.498+0000 7ff92962d740 1 -- start start 2024-04-21T08:55:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff92962d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e698434d0 0x555e6984bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff92962d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e69843e80 0x555e6984c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff92962d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e6984ca00 0x555e698645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff92962d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201af4a0 con 0x555e69843e80 2024-04-21T08:55:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff92962d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201b1070 con 0x555e6984ca00 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff92962d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201afb70 con 0x555e698434d0 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff91ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e69843e80 0x555e6984c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff91ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e69843e80 0x555e6984c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42908/0 (socket says 172.21.15.82:42908) 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff91ffff640 1 -- 172.21.15.82:0/2642232454 learned_addr learned my addr 172.21.15.82:0/2642232454 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff91ffff640 1 -- 172.21.15.82:0/2642232454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e698434d0 msgr2=0x555e6984bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff91ffff640 1 --2- 172.21.15.82:0/2642232454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e698434d0 0x555e6984bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff91ffff640 1 -- 172.21.15.82:0/2642232454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e6984ca00 msgr2=0x555e698645e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff91ffff640 1 --2- 172.21.15.82:0/2642232454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e6984ca00 0x555e698645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff91ffff640 1 -- 172.21.15.82:0/2642232454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e69843c20 con 0x555e69843e80 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff91ffff640 1 --2- 172.21.15.82:0/2642232454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e69843e80 0x555e6984c300 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff908009930 tx=0x7ff908009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258ac07a5a9abb12 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff915ffb640 1 -- 172.21.15.82:0/2642232454 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908049070 con 0x555e69843e80 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff915ffb640 1 -- 172.21.15.82:0/2642232454 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff908004460 con 0x555e69843e80 2024-04-21T08:55:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.499+0000 7ff915ffb640 1 -- 172.21.15.82:0/2642232454 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908005450 con 0x555e69843e80 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.500+0000 7ff92962d740 1 -- 172.21.15.82:0/2642232454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e69843e80 msgr2=0x555e6984c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.500+0000 7ff92962d740 1 --2- 172.21.15.82:0/2642232454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e69843e80 0x555e6984c300 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff908009930 tx=0x7ff908009260 comp rx=0 tx=0).stop 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.500+0000 7ff92962d740 1 -- 172.21.15.82:0/2642232454 shutdown_connections 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.500+0000 7ff92962d740 1 --2- 172.21.15.82:0/2642232454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e6984ca00 0x555e698645e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.500+0000 7ff92962d740 1 --2- 172.21.15.82:0/2642232454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e69843e80 0x555e6984c300 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.500+0000 7ff92962d740 1 --2- 172.21.15.82:0/2642232454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e698434d0 0x555e6984bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.500+0000 7ff92962d740 1 -- 172.21.15.82:0/2642232454 >> 172.21.15.82:0/2642232454 conn(0x555e6983dba0 msgr2=0x555e69844af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.500+0000 7ff92962d740 1 -- 172.21.15.82:0/2642232454 shutdown_connections 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.500+0000 7ff92962d740 1 -- 172.21.15.82:0/2642232454 wait complete. 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.500+0000 7ff92962d740 1 -- start start 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.500+0000 7ff92962d740 1 Processor -- start 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff92962d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e698434d0 0x555e69854d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff92962d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e69843e80 0x555e69855280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff92962d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 0x555e69845cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff92962d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201b0970 con 0x555e6984ca00 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff92962d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201b0270 con 0x555e69843e80 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff92962d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201ad470 con 0x555e698434d0 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff92962d740 10 client.0 _ll_register_callbacks cb 0x7ff9100051d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff91f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 0x555e69845cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff91f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 0x555e69845cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42910/0 (socket says 172.21.15.82:42910) 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff91f7fe640 1 -- 172.21.15.82:0/3643038558 learned_addr learned my addr 172.21.15.82:0/3643038558 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff91ffff640 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e69843e80 0x555e69855280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e698434d0 0x555e69854d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff91f7fe640 1 -- 172.21.15.82:0/3643038558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e698434d0 msgr2=0x555e69854d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e698434d0 0x555e69854d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff91f7fe640 1 -- 172.21.15.82:0/3643038558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e69843e80 msgr2=0x555e69855280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e69843e80 0x555e69855280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff91f7fe640 1 -- 172.21.15.82:0/3643038558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e69846500 con 0x555e6984ca00 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff91ffff640 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e69843e80 0x555e69855280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff91f7fe640 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 0x555e69845cd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff910005ad0 tx=0x7ff91000e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb607734870bd90 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff8ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff8ff7fe640 1 -- 172.21.15.82:0/3643038558 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91000e8f0 con 0x555e6984ca00 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff8ff7fe640 1 -- 172.21.15.82:0/3643038558 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7ff91000bbc0 con 0x555e6984ca00 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff92962d740 1 -- 172.21.15.82:0/3643038558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff908004460 con 0x555e6984ca00 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff8ff7fe640 1 -- 172.21.15.82:0/3643038558 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff91000bdc0 con 0x555e6984ca00 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff92962d740 10 client.18105 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.501+0000 7ff92962d740 1 -- 172.21.15.82:0/3643038558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555e698c28f0 con 0x555e6984ca00 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff8ff7fe640 1 -- 172.21.15.82:0/3643038558 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff91003a070 con 0x555e6984ca00 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff8ff7fe640 1 -- 172.21.15.82:0/3643038558 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7ff91002c4e0 con 0x555e6984ca00 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff92962d740 10 client.18105 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff92962d740 10 client.18105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:32.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff92962d740 20 client.18105 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff92962d740 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555e698c2ea0 0x555e698c5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff92962d740 4 client.18105 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff92962d740 1 -- 172.21.15.82:0/3643038558 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e697b5c10 con 0x555e698c2ea0 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff91ffff640 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555e698c2ea0 0x555e698c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff8ff7fe640 10 client.18105 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff91ffff640 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555e698c2ea0 0x555e698c5290 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a03abbabdf05b39e server_cookie=afae618f84576360 in_seq=0 out_seq=0 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff8ff7fe640 1 -- 172.21.15.82:0/3643038558 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e697b5c10 con 0x555e698c2ea0 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff8ff7fe640 10 client.18105 handle_command_reply: tid=0 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.502+0000 7ff92962d740 1 client.18105 shutdown 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff92962d740 1 -- 172.21.15.82:0/3643038558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 msgr2=0x555e69845cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff92962d740 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 0x555e69845cd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff910005ad0 tx=0x7ff91000e430 comp rx=0 tx=0).stop 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff92962d740 1 -- 172.21.15.82:0/3643038558 shutdown_connections 2024-04-21T08:55:32.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff92962d740 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x555e698c2ea0 0x555e698c5290 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff92962d740 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e6984ca00 0x555e69845cd0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff8ff7fe640 0 client.18105 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:32.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff92962d740 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e69843e80 0x555e69855280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff92962d740 1 --2- 172.21.15.82:0/3643038558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e698434d0 0x555e69854d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff92962d740 1 -- 172.21.15.82:0/3643038558 >> 172.21.15.82:0/3643038558 conn(0x555e6983dba0 msgr2=0x555e6983c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff92962d740 1 -- 172.21.15.82:0/3643038558 shutdown_connections 2024-04-21T08:55:32.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff92962d740 1 -- 172.21.15.82:0/3643038558 wait complete. 2024-04-21T08:55:32.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff92962d740 20 client.18105 trim_cache size 0 max 16384 2024-04-21T08:55:32.511 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:55:32.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff926a3f640 1 -- 172.21.15.82:0/664303871 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff90c0a3d20 msgr2=0x7ff90c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff926a3f640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff90c0a3d20 0x7ff90c0a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff918098d60 tx=0x7ff91806f3f0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff926a3f640 1 -- 172.21.15.82:0/664303871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff920107030 msgr2=0x7ff9201985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:32.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff926a3f640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff920107030 0x7ff9201985c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff91800cad0 tx=0x7ff91800cf90 comp rx=0 tx=0).stop 2024-04-21T08:55:32.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff926a3f640 1 -- 172.21.15.82:0/664303871 shutdown_connections 2024-04-21T08:55:32.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff926a3f640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff90c0a3d20 0x7ff90c0a61e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff926a3f640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff920107030 0x7ff9201985c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff926a3f640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff920101b40 0x7ff92019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff926a3f640 1 --2- 172.21.15.82:0/664303871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9201011a0 0x7ff92019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:32.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff926a3f640 1 -- 172.21.15.82:0/664303871 >> 172.21.15.82:0/664303871 conn(0x7ff920076c50 msgr2=0x7ff9200fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:32.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.503+0000 7ff926a3f640 1 -- 172.21.15.82:0/664303871 shutdown_connections 2024-04-21T08:55:32.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:32.504+0000 7ff926a3f640 1 -- 172.21.15.82:0/664303871 wait complete. 2024-04-21T08:55:32.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:32 smithi106 ceph-mon[31072]: pgmap v262: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:55:32.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:32 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1799312163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:55:33.515 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:55:33.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:55:33.515 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:55:33.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.625+0000 7fce02c3a640 1 -- 172.21.15.82:0/1752208437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc106f30 msgr2=0x7fcdfc107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.625+0000 7fce02c3a640 1 --2- 172.21.15.82:0/1752208437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc106f30 0x7fcdfc107310 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009930 tx=0x7fcdf002bc20 comp rx=0 tx=0).stop 2024-04-21T08:55:33.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.625+0000 7fce02c3a640 1 -- 172.21.15.82:0/1752208437 shutdown_connections 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.625+0000 7fce02c3a640 1 --2- 172.21.15.82:0/1752208437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc100e10 0x7fcdfc10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.625+0000 7fce02c3a640 1 --2- 172.21.15.82:0/1752208437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdfc100450 0x7fcdfc1008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.625+0000 7fce02c3a640 1 --2- 172.21.15.82:0/1752208437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc106f30 0x7fcdfc107310 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.625+0000 7fce02c3a640 1 -- 172.21.15.82:0/1752208437 >> 172.21.15.82:0/1752208437 conn(0x7fcdfc077410 msgr2=0x7fcdfc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.625+0000 7fce02c3a640 1 -- 172.21.15.82:0/1752208437 shutdown_connections 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.625+0000 7fce02c3a640 1 -- 172.21.15.82:0/1752208437 wait complete. 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce02c3a640 1 Processor -- start 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce02c3a640 1 -- start start 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce02c3a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc100450 0x7fcdfc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce02c3a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdfc100e10 0x7fcdfc198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce02c3a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc106f30 0x7fcdfc19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce02c3a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc112270 con 0x7fcdfc106f30 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce02c3a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1120f0 con 0x7fcdfc100450 2024-04-21T08:55:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce02c3a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1123f0 con 0x7fcdfc100e10 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce011d7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc106f30 0x7fcdfc19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce011d7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc106f30 0x7fcdfc19cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47976/0 (socket says 172.21.15.82:47976) 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce011d7640 1 -- 172.21.15.82:0/331102321 learned_addr learned my addr 172.21.15.82:0/331102321 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fcdfbfff640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdfc100e10 0x7fcdfc198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce009d6640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc100450 0x7fcdfc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce011d7640 1 -- 172.21.15.82:0/331102321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdfc100e10 msgr2=0x7fcdfc198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce011d7640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdfc100e10 0x7fcdfc198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce011d7640 1 -- 172.21.15.82:0/331102321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc100450 msgr2=0x7fcdfc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce011d7640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc100450 0x7fcdfc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce011d7640 1 -- 172.21.15.82:0/331102321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdfc19d720 con 0x7fcdfc106f30 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fce011d7640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc106f30 0x7fcdfc19cff0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400cce0 tx=0x7fcdf400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.626+0000 7fcdf9ffb640 1 -- 172.21.15.82:0/331102321 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf400f450 con 0x7fcdfc106f30 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.627+0000 7fcdf9ffb640 1 -- 172.21.15.82:0/331102321 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf400faa0 con 0x7fcdfc106f30 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.627+0000 7fce02c3a640 1 -- 172.21.15.82:0/331102321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdfc19da10 con 0x7fcdfc106f30 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.627+0000 7fcdf9ffb640 1 -- 172.21.15.82:0/331102321 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4018a80 con 0x7fcdfc106f30 2024-04-21T08:55:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.627+0000 7fce02c3a640 1 -- 172.21.15.82:0/331102321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdfc1a52a0 con 0x7fcdfc106f30 2024-04-21T08:55:33.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.629+0000 7fcdf9ffb640 1 -- 172.21.15.82:0/331102321 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcdf400fc50 con 0x7fcdfc106f30 2024-04-21T08:55:33.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.629+0000 7fcdf9ffb640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcdec0a3da0 0x7fcdec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.629+0000 7fce05822740 1 Processor -- start 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.629+0000 7fce05822740 1 -- start start 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.629+0000 7fce009d6640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcdec0a3da0 0x7fcdec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.629+0000 7fcdf9ffb640 1 -- 172.21.15.82:0/331102321 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fcdf40d1080 con 0x7fcdfc106f30 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.629+0000 7fce05822740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30963cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.629+0000 7fce05822740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f30964290 0x559f3096c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.629+0000 7fce05822740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 0x559f3096f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.629+0000 7fce05822740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559f308c7f00 con 0x559f309638e0 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.629+0000 7fce05822740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559f30865590 con 0x559f3096ce10 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.629+0000 7fce05822740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f308c3ec0 con 0x559f30964290 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fce009d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30963cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fce009d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30963cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47992/0 (socket says 172.21.15.82:47992) 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fce009d6640 1 -- 172.21.15.82:0/2846111400 learned_addr learned my addr 172.21.15.82:0/2846111400 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:33.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fcdfbfff640 1 --2- 172.21.15.82:0/2846111400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f30964290 0x559f3096c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fce011d7640 1 --2- 172.21.15.82:0/2846111400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 0x559f3096f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fce009d6640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcdec0a3da0 0x7fcdec0a6260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005c40 tx=0x7fcdf0004370 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fce009d6640 1 -- 172.21.15.82:0/2846111400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f30964290 msgr2=0x559f3096c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fce009d6640 1 --2- 172.21.15.82:0/2846111400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f30964290 0x559f3096c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fce009d6640 1 -- 172.21.15.82:0/2846111400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 msgr2=0x559f3096f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fce009d6640 1 --2- 172.21.15.82:0/2846111400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 0x559f3096f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fce009d6640 1 -- 172.21.15.82:0/2846111400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f306a27e0 con 0x559f309638e0 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fcdfbfff640 1 --2- 172.21.15.82:0/2846111400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f30964290 0x559f3096c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fce011d7640 1 --2- 172.21.15.82:0/2846111400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 0x559f3096f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fce009d6640 1 --2- 172.21.15.82:0/2846111400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30963cc0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0036040 tx=0x7fcdf0049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690a2b1810ff70ed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fcde6ffd640 1 -- 172.21.15.82:0/2846111400 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0005240 con 0x559f309638e0 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fcde6ffd640 1 -- 172.21.15.82:0/2846111400 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf0051070 con 0x559f309638e0 2024-04-21T08:55:33.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.630+0000 7fcde6ffd640 1 -- 172.21.15.82:0/2846111400 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0003850 con 0x559f309638e0 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.631+0000 7fce05822740 1 -- 172.21.15.82:0/2846111400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 msgr2=0x559f30963cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.631+0000 7fce05822740 1 --2- 172.21.15.82:0/2846111400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30963cc0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0036040 tx=0x7fcdf0049000 comp rx=0 tx=0).stop 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.631+0000 7fce05822740 1 -- 172.21.15.82:0/2846111400 shutdown_connections 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.631+0000 7fce05822740 1 --2- 172.21.15.82:0/2846111400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 0x559f3096f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.631+0000 7fce05822740 1 --2- 172.21.15.82:0/2846111400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f30964290 0x559f3096c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.631+0000 7fce05822740 1 --2- 172.21.15.82:0/2846111400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30963cc0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.631+0000 7fce05822740 1 -- 172.21.15.82:0/2846111400 >> 172.21.15.82:0/2846111400 conn(0x559f3095dfb0 msgr2=0x559f3095e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.631+0000 7fce05822740 1 -- 172.21.15.82:0/2846111400 shutdown_connections 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.631+0000 7fce05822740 1 -- 172.21.15.82:0/2846111400 wait complete. 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.631+0000 7fce05822740 1 -- start start 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.631+0000 7fce05822740 1 Processor -- start 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce05822740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30969de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce05822740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f30964290 0x559f3096a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce05822740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 0x559f3096a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce05822740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559f3096fd20 con 0x559f309638e0 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce05822740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559f30881580 con 0x559f3096ce10 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce05822740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f3096fea0 con 0x559f30964290 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce05822740 10 client.0 _ll_register_callbacks cb 0x559f30887650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce009d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30969de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce009d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30969de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48008/0 (socket says 172.21.15.82:48008) 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce009d6640 1 -- 172.21.15.82:0/3192320341 learned_addr learned my addr 172.21.15.82:0/3192320341 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fcdfbfff640 1 --2- 172.21.15.82:0/3192320341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f30964290 0x559f3096a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce009d6640 1 -- 172.21.15.82:0/3192320341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f30964290 msgr2=0x559f3096a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce009d6640 1 --2- 172.21.15.82:0/3192320341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f30964290 0x559f3096a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce009d6640 1 -- 172.21.15.82:0/3192320341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 msgr2=0x559f3096a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce009d6640 1 --2- 172.21.15.82:0/3192320341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 0x559f3096a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce009d6640 1 -- 172.21.15.82:0/3192320341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f30702f70 con 0x559f309638e0 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fce009d6640 1 --2- 172.21.15.82:0/3192320341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30969de0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0004e30 tx=0x7fcdf00049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84517201ed0d974 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:33.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.632+0000 7fcddbfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fcddbfff640 1 -- 172.21.15.82:0/3192320341 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0059030 con 0x559f309638e0 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fcddbfff640 1 -- 172.21.15.82:0/3192320341 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fcdf00050e0 con 0x559f309638e0 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fce05822740 1 -- 172.21.15.82:0/3192320341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf0051070 con 0x559f309638e0 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fcddbfff640 1 -- 172.21.15.82:0/3192320341 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcdf0066050 con 0x559f309638e0 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fce05822740 10 client.18117 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fce05822740 1 -- 172.21.15.82:0/3192320341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559f3059a170 con 0x559f309638e0 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fcddbfff640 1 -- 172.21.15.82:0/3192320341 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf0051070 con 0x559f309638e0 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fcddbfff640 1 -- 172.21.15.82:0/3192320341 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fcdf0003b90 con 0x559f309638e0 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fce05822740 10 client.18117 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fce05822740 10 client.18117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fce05822740 20 client.18117 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fce05822740 1 --2- 172.21.15.82:0/3192320341 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f3097d110 0x559f3097f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fce05822740 4 client.18117 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:55:33.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fce05822740 1 -- 172.21.15.82:0/3192320341 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f308d5920 con 0x559f3097d110 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.633+0000 7fcdfbfff640 1 --2- 172.21.15.82:0/3192320341 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f3097d110 0x559f3097f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.634+0000 7fcddbfff640 10 client.18117 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.634+0000 7fcdfbfff640 1 --2- 172.21.15.82:0/3192320341 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f3097d110 0x559f3097f500 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97036a28e652114c server_cookie=298f31c1c9db145c in_seq=0 out_seq=0 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fcddbfff640 1 -- 172.21.15.82:0/3192320341 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f308d5920 con 0x559f3097d110 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fcddbfff640 10 client.18117 handle_command_reply: tid=0 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fce05822740 1 client.18117 shutdown 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fce05822740 1 -- 172.21.15.82:0/3192320341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 msgr2=0x559f30969de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fce05822740 1 --2- 172.21.15.82:0/3192320341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30969de0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0004e30 tx=0x7fcdf00049f0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fce05822740 1 -- 172.21.15.82:0/3192320341 shutdown_connections 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fce05822740 1 --2- 172.21.15.82:0/3192320341 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f3097d110 0x559f3097f500 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fce05822740 1 --2- 172.21.15.82:0/3192320341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 0x559f3096a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fce05822740 1 --2- 172.21.15.82:0/3192320341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f30964290 0x559f3096a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fcddbfff640 0 client.18117 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:33.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fce05822740 1 --2- 172.21.15.82:0/3192320341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30969de0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fce05822740 1 -- 172.21.15.82:0/3192320341 >> 172.21.15.82:0/3192320341 conn(0x559f3095dfb0 msgr2=0x559f3095fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:33.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fce05822740 1 -- 172.21.15.82:0/3192320341 shutdown_connections 2024-04-21T08:55:33.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fce05822740 1 -- 172.21.15.82:0/3192320341 wait complete. 2024-04-21T08:55:33.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.635+0000 7fce05822740 20 client.18117 trim_cache size 0 max 16384 2024-04-21T08:55:33.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.652+0000 7fce05822740 1 Processor -- start 2024-04-21T08:55:33.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.652+0000 7fce05822740 1 -- start start 2024-04-21T08:55:33.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.652+0000 7fce05822740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30978c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.652+0000 7fce05822740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f30964290 0x559f3095fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.652+0000 7fce05822740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3096ce10 0x559f30984b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.652+0000 7fce05822740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1af450 con 0x559f309638e0 2024-04-21T08:55:33.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.652+0000 7fce05822740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1b1020 con 0x559f30964290 2024-04-21T08:55:33.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.652+0000 7fce05822740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1afb20 con 0x559f3096ce10 2024-04-21T08:55:33.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.652+0000 7fce009d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30978c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.652+0000 7fce009d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30978c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48020/0 (socket says 172.21.15.82:48020) 2024-04-21T08:55:33.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.652+0000 7fce009d6640 1 -- 172.21.15.82:0/499480721 learned_addr learned my addr 172.21.15.82:0/499480721 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:33.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.653+0000 7fce009d6640 1 -- 172.21.15.82:0/499480721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3096ce10 msgr2=0x559f30984b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.653+0000 7fce009d6640 1 --2- 172.21.15.82:0/499480721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3096ce10 0x559f30984b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.653+0000 7fce009d6640 1 -- 172.21.15.82:0/499480721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f30964290 msgr2=0x559f3095fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.653+0000 7fce009d6640 1 --2- 172.21.15.82:0/499480721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f30964290 0x559f3095fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.653+0000 7fce009d6640 1 -- 172.21.15.82:0/499480721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f30964030 con 0x559f309638e0 2024-04-21T08:55:33.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.653+0000 7fce009d6640 1 --2- 172.21.15.82:0/499480721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30978c10 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcdf002c160 tx=0x7fcdf0004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edfb75f87829a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:33.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.653+0000 7fcde67fc640 1 -- 172.21.15.82:0/499480721 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0059070 con 0x559f309638e0 2024-04-21T08:55:33.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.653+0000 7fcde67fc640 1 -- 172.21.15.82:0/499480721 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf0051070 con 0x559f309638e0 2024-04-21T08:55:33.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.653+0000 7fcde67fc640 1 -- 172.21.15.82:0/499480721 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0005340 con 0x559f309638e0 2024-04-21T08:55:33.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.653+0000 7fce05822740 1 -- 172.21.15.82:0/499480721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 msgr2=0x559f30978c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.653+0000 7fce05822740 1 --2- 172.21.15.82:0/499480721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30978c10 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcdf002c160 tx=0x7fcdf0004e60 comp rx=0 tx=0).stop 2024-04-21T08:55:33.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 -- 172.21.15.82:0/499480721 shutdown_connections 2024-04-21T08:55:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 --2- 172.21.15.82:0/499480721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f3096ce10 0x559f30984b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 --2- 172.21.15.82:0/499480721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f30964290 0x559f3095fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 --2- 172.21.15.82:0/499480721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f309638e0 0x559f30978c10 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 -- 172.21.15.82:0/499480721 >> 172.21.15.82:0/499480721 conn(0x559f3095dfb0 msgr2=0x559f3095f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 -- 172.21.15.82:0/499480721 shutdown_connections 2024-04-21T08:55:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 -- 172.21.15.82:0/499480721 wait complete. 2024-04-21T08:55:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 -- start start 2024-04-21T08:55:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 Processor -- start 2024-04-21T08:55:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f309638e0 0x559f30971c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f30964290 0x559f309721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 0x559f309726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1b0920 con 0x559f30964290 2024-04-21T08:55:33.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1b0220 con 0x559f3096ce10 2024-04-21T08:55:33.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1ad420 con 0x559f309638e0 2024-04-21T08:55:33.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fce05822740 10 client.0 _ll_register_callbacks cb 0x7fcdf0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:33.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fcdfbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f30964290 0x559f309721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fcdfbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f30964290 0x559f309721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48028/0 (socket says 172.21.15.82:48028) 2024-04-21T08:55:33.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.654+0000 7fcdfbfff640 1 -- 172.21.15.82:0/641116890 learned_addr learned my addr 172.21.15.82:0/641116890 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:33.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fce009d6640 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f309638e0 0x559f30971c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fce011d7640 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 0x559f309726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fcdfbfff640 1 -- 172.21.15.82:0/641116890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f309638e0 msgr2=0x559f30971c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fcdfbfff640 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f309638e0 0x559f30971c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fcdfbfff640 1 -- 172.21.15.82:0/641116890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 msgr2=0x559f309726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fcdfbfff640 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 0x559f309726e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fcdfbfff640 1 -- 172.21.15.82:0/641116890 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f309ebc90 con 0x559f30964290 2024-04-21T08:55:33.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fcdfbfff640 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f30964290 0x559f309721a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x559f30964690 tx=0x7fcde80138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f30ad41ecbc4351 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:33.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fcddbfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:33.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fcddbfff640 1 -- 172.21.15.82:0/641116890 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde801d630 con 0x559f30964290 2024-04-21T08:55:33.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fcddbfff640 1 -- 172.21.15.82:0/641116890 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fcde801d7d0 con 0x559f30964290 2024-04-21T08:55:33.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fce05822740 1 -- 172.21.15.82:0/641116890 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf0051070 con 0x559f30964290 2024-04-21T08:55:33.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fcddbfff640 1 -- 172.21.15.82:0/641116890 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcde802b050 con 0x559f30964290 2024-04-21T08:55:33.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fce05822740 10 client.18123 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:33.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fce05822740 1 -- 172.21.15.82:0/641116890 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559f309ed060 con 0x559f30964290 2024-04-21T08:55:33.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fcddbfff640 1 -- 172.21.15.82:0/641116890 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf0051070 con 0x559f30964290 2024-04-21T08:55:33.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fcddbfff640 1 -- 172.21.15.82:0/641116890 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fcde800f420 con 0x559f30964290 2024-04-21T08:55:33.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fce05822740 10 client.18123 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:33.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fce05822740 10 client.18123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:33.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fce05822740 20 client.18123 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:33.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fce05822740 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f309ed840 0x559f309efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fce05822740 4 client.18123 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:55:33.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.655+0000 7fce05822740 1 -- 172.21.15.82:0/641116890 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f308cf790 con 0x559f309ed840 2024-04-21T08:55:33.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.656+0000 7fce009d6640 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f309ed840 0x559f309efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.656+0000 7fce009d6640 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f309ed840 0x559f309efc60 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35ad2d2ba22d0b6e server_cookie=51406405efcd55ad in_seq=0 out_seq=0 2024-04-21T08:55:33.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.656+0000 7fcddbfff640 10 client.18123 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:33.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fcddbfff640 1 -- 172.21.15.82:0/641116890 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f308cf790 con 0x559f309ed840 2024-04-21T08:55:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fcddbfff640 10 client.18123 handle_command_reply: tid=0 2024-04-21T08:55:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fce05822740 1 client.18123 shutdown 2024-04-21T08:55:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fce05822740 1 -- 172.21.15.82:0/641116890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f30964290 msgr2=0x559f309721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fce05822740 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f30964290 0x559f309721a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x559f30964690 tx=0x7fcde80138f0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fce05822740 1 -- 172.21.15.82:0/641116890 shutdown_connections 2024-04-21T08:55:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fce05822740 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x559f309ed840 0x559f309efc60 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fce05822740 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559f3096ce10 0x559f309726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fce05822740 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f30964290 0x559f309721a0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fcddbfff640 0 client.18123 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:33.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fce05822740 1 --2- 172.21.15.82:0/641116890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f309638e0 0x559f30971c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fce05822740 1 -- 172.21.15.82:0/641116890 >> 172.21.15.82:0/641116890 conn(0x559f3095dfb0 msgr2=0x559f3096a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:33.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fce05822740 1 -- 172.21.15.82:0/641116890 shutdown_connections 2024-04-21T08:55:33.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.657+0000 7fce05822740 1 -- 172.21.15.82:0/641116890 wait complete. 2024-04-21T08:55:33.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce05822740 20 client.18123 trim_cache size 0 max 16384 2024-04-21T08:55:33.673 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:55:33.674 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:55:33.674 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "16186bb7-9603-4aff-8b40-116ed394aec2", 2024-04-21T08:55:33.674 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:55:33.674 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:55:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce02c3a640 1 -- 172.21.15.82:0/331102321 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcdec0a3da0 msgr2=0x7fcdec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce02c3a640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcdec0a3da0 0x7fcdec0a6260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005c40 tx=0x7fcdf0004370 comp rx=0 tx=0).stop 2024-04-21T08:55:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce02c3a640 1 -- 172.21.15.82:0/331102321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc106f30 msgr2=0x7fcdfc19cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce02c3a640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc106f30 0x7fcdfc19cff0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400cce0 tx=0x7fcdf400a620 comp rx=0 tx=0).stop 2024-04-21T08:55:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce02c3a640 1 -- 172.21.15.82:0/331102321 shutdown_connections 2024-04-21T08:55:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce02c3a640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fcdec0a3da0 0x7fcdec0a6260 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce02c3a640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc106f30 0x7fcdfc19cff0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce02c3a640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdfc100e10 0x7fcdfc198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce02c3a640 1 --2- 172.21.15.82:0/331102321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc100450 0x7fcdfc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce02c3a640 1 -- 172.21.15.82:0/331102321 >> 172.21.15.82:0/331102321 conn(0x7fcdfc077410 msgr2=0x7fcdfc104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce02c3a640 1 -- 172.21.15.82:0/331102321 shutdown_connections 2024-04-21T08:55:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.658+0000 7fce02c3a640 1 -- 172.21.15.82:0/331102321 wait complete. 2024-04-21T08:55:33.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:55:33.678 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:55:33.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.778+0000 7f9ccab59640 1 -- 172.21.15.82:0/2427849199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cc4107270 msgr2=0x7f9cc410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.778+0000 7f9ccab59640 1 --2- 172.21.15.82:0/2427849199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cc4107270 0x7f9cc410ee40 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009ca0 tx=0x7f9cbc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.778+0000 7f9ccab59640 1 -- 172.21.15.82:0/2427849199 shutdown_connections 2024-04-21T08:55:33.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.778+0000 7f9ccab59640 1 --2- 172.21.15.82:0/2427849199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cc4107270 0x7f9cc410ee40 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.778+0000 7f9ccab59640 1 --2- 172.21.15.82:0/2427849199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc4102b50 0x7f9cc4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.778+0000 7f9ccab59640 1 --2- 172.21.15.82:0/2427849199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cc41021a0 0x7f9cc4102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.778+0000 7f9ccab59640 1 -- 172.21.15.82:0/2427849199 >> 172.21.15.82:0/2427849199 conn(0x7f9cc40fda40 msgr2=0x7f9cc40ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:33.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.778+0000 7f9ccab59640 1 -- 172.21.15.82:0/2427849199 shutdown_connections 2024-04-21T08:55:33.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.778+0000 7f9ccab59640 1 -- 172.21.15.82:0/2427849199 wait complete. 2024-04-21T08:55:33.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.778+0000 7f9ccab59640 1 Processor -- start 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9ccab59640 1 -- start start 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9ccab59640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc41021a0 0x7f9cc41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9ccab59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cc4107270 0x7f9cc4198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9ccab59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cc419cf60 0x7f9cc419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9ccab59640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc41121e0 con 0x7f9cc4107270 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9ccab59640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc4112060 con 0x7f9cc41021a0 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9ccab59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc4112360 con 0x7f9cc419cf60 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cc4107270 0x7f9cc4198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cc4107270 0x7f9cc4198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48060/0 (socket says 172.21.15.82:48060) 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cbbfff640 1 -- 172.21.15.82:0/2114267187 learned_addr learned my addr 172.21.15.82:0/2114267187 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc41021a0 0x7f9cc41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cc90f6640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cc419cf60 0x7f9cc419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cbbfff640 1 -- 172.21.15.82:0/2114267187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cc419cf60 msgr2=0x7f9cc419d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cc419cf60 0x7f9cc419d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cbbfff640 1 -- 172.21.15.82:0/2114267187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc41021a0 msgr2=0x7f9cc41986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc41021a0 0x7f9cc41986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cbbfff640 1 -- 172.21.15.82:0/2114267187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc419da50 con 0x7f9cc4107270 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc41021a0 0x7f9cc41986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cc4107270 0x7f9cc4198be0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400c970 tx=0x7f9cb400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cb9ffb640 1 -- 172.21.15.82:0/2114267187 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4014070 con 0x7f9cc4107270 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cb9ffb640 1 -- 172.21.15.82:0/2114267187 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb4004440 con 0x7f9cc4107270 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9ccab59640 1 -- 172.21.15.82:0/2114267187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc419dd40 con 0x7f9cc4107270 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.779+0000 7f9cb9ffb640 1 -- 172.21.15.82:0/2114267187 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4003e50 con 0x7f9cc4107270 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.780+0000 7f9ccab59640 1 -- 172.21.15.82:0/2114267187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc41a5530 con 0x7f9cc4107270 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9cb9ffb640 1 -- 172.21.15.82:0/2114267187 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9cb4017070 con 0x7f9cc4107270 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9ccd82c740 1 Processor -- start 2024-04-21T08:55:33.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9ccd82c740 1 -- start start 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9ccd82c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018580 0x55617b018960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9ccd82c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b018ea0 0x55617b021540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9ccd82c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b021a80 0x55617b023e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9ccd82c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55617af7d1e0 con 0x55617b021a80 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9ccd82c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55617af1a390 con 0x55617b018ea0 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9ccd82c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55617af791a0 con 0x55617b018580 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9cb9ffb640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9cac0a3ec0 0x7f9cac0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b021a80 0x55617b023e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b021a80 0x55617b023e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48070/0 (socket says 172.21.15.82:48070) 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9cbbfff640 1 -- 172.21.15.82:0/1005692018 learned_addr learned my addr 172.21.15.82:0/1005692018 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9cc90f6640 1 --2- 172.21.15.82:0/1005692018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018580 0x55617b018960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9cc90f6640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9cac0a3ec0 0x7f9cac0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.782+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/1005692018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b018ea0 0x55617b021540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.783+0000 7f9cb9ffb640 1 -- 172.21.15.82:0/2114267187 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f9cb40d07c0 con 0x7f9cc4107270 2024-04-21T08:55:33.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.783+0000 7f9cbbfff640 1 -- 172.21.15.82:0/1005692018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018580 msgr2=0x55617b018960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.783+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/1005692018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018580 0x55617b018960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.783+0000 7f9cbbfff640 1 -- 172.21.15.82:0/1005692018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b018ea0 msgr2=0x55617b021540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.783+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/1005692018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b018ea0 0x55617b021540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.783+0000 7f9cbbfff640 1 -- 172.21.15.82:0/1005692018 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617ad597d0 con 0x55617b021a80 2024-04-21T08:55:33.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.783+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/1005692018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b018ea0 0x55617b021540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:55:33.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.783+0000 7f9cc90f6640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9cac0a3ec0 0x7f9cac0a6380 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc004af0 tx=0x7f9cbc004a40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:33.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.783+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/1005692018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b021a80 0x55617b023e70 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40cda70 tx=0x7f9cb40045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145383901c770e20 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:33.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.783+0000 7f9caaffd640 1 -- 172.21.15.82:0/1005692018 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb409c350 con 0x55617b021a80 2024-04-21T08:55:33.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.783+0000 7f9caaffd640 1 -- 172.21.15.82:0/1005692018 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb40a0700 con 0x55617b021a80 2024-04-21T08:55:33.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.783+0000 7f9caaffd640 1 -- 172.21.15.82:0/1005692018 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb40980a0 con 0x55617b021a80 2024-04-21T08:55:33.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 -- 172.21.15.82:0/1005692018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b021a80 msgr2=0x55617b023e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/1005692018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b021a80 0x55617b023e70 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40cda70 tx=0x7f9cb40045f0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 -- 172.21.15.82:0/1005692018 shutdown_connections 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/1005692018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b021a80 0x55617b023e70 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/1005692018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b018ea0 0x55617b021540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/1005692018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018580 0x55617b018960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 -- 172.21.15.82:0/1005692018 >> 172.21.15.82:0/1005692018 conn(0x55617b012cb0 msgr2=0x55617b013090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 -- 172.21.15.82:0/1005692018 shutdown_connections 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 -- 172.21.15.82:0/1005692018 wait complete. 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 -- start start 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 Processor -- start 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b01eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018ea0 0x55617b01f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b021a80 0x55617b01f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55617b024870 con 0x55617b018580 2024-04-21T08:55:33.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.784+0000 7f9ccd82c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55617af362f0 con 0x55617b021a80 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9ccd82c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55617b0249f0 con 0x55617b018ea0 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9ccd82c740 10 client.0 _ll_register_callbacks cb 0x55617af43580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9cc88f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b01eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9cc88f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b01eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48082/0 (socket says 172.21.15.82:48082) 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9cc88f5640 1 -- 172.21.15.82:0/3771477838 learned_addr learned my addr 172.21.15.82:0/3771477838 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018ea0 0x55617b01f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9cc90f6640 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b021a80 0x55617b01f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9cc88f5640 1 -- 172.21.15.82:0/3771477838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018ea0 msgr2=0x55617b01f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018ea0 0x55617b01f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9cc88f5640 1 -- 172.21.15.82:0/3771477838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b021a80 msgr2=0x55617b01f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b021a80 0x55617b01f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9cc88f5640 1 -- 172.21.15.82:0/3771477838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617ac8f3f0 con 0x55617b018580 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9cc90f6640 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b021a80 0x55617b01f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:55:33.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b01eb00 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0010670 tx=0x7f9cc0010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b150e521aa4ef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9ca3fff640 1 -- 172.21.15.82:0/3771477838 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000ead0 con 0x55617b018580 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9ca3fff640 1 -- 172.21.15.82:0/3771477838 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f9cc000ec70 con 0x55617b018580 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9ccd82c740 1 -- 172.21.15.82:0/3771477838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb40a0700 con 0x55617b018580 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9ca3fff640 1 -- 172.21.15.82:0/3771477838 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9cc0025050 con 0x55617b018580 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9ccd82c740 10 client.18135 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9ccd82c740 1 -- 172.21.15.82:0/3771477838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55617ac4f0f0 con 0x55617b018580 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.785+0000 7f9ca3fff640 1 -- 172.21.15.82:0/3771477838 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb40a0700 con 0x55617b018580 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.786+0000 7f9ca3fff640 1 -- 172.21.15.82:0/3771477838 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f9cc000f040 con 0x55617b018580 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.786+0000 7f9ccd82c740 10 client.18135 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.786+0000 7f9ccd82c740 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.786+0000 7f9ccd82c740 20 client.18135 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.786+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55617b031b20 0x55617b033fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.786+0000 7f9ccd82c740 4 client.18135 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:55:33.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.786+0000 7f9ccd82c740 1 -- 172.21.15.82:0/3771477838 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55617af8ac00 con 0x55617b031b20 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.786+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55617b031b20 0x55617b033fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.786+0000 7f9ca3fff640 10 client.18135 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.786+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55617b031b20 0x55617b033fc0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62d6e8fe8b5b42b2 server_cookie=3b064855c6d5aab4 in_seq=0 out_seq=0 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ca3fff640 1 -- 172.21.15.82:0/3771477838 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55617af8ac00 con 0x55617b031b20 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ca3fff640 10 client.18135 handle_command_reply: tid=0 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ccd82c740 1 client.18135 shutdown 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ccd82c740 1 -- 172.21.15.82:0/3771477838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 msgr2=0x55617b01eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b01eb00 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0010670 tx=0x7f9cc0010b30 comp rx=0 tx=0).stop 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ccd82c740 1 -- 172.21.15.82:0/3771477838 shutdown_connections 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55617b031b20 0x55617b033fc0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b021a80 0x55617b01f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018ea0 0x55617b01f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/3771477838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b01eb00 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ca3fff640 0 client.18135 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:33.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ccd82c740 1 -- 172.21.15.82:0/3771477838 >> 172.21.15.82:0/3771477838 conn(0x55617b012cb0 msgr2=0x55617b014a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:33.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ccd82c740 1 -- 172.21.15.82:0/3771477838 shutdown_connections 2024-04-21T08:55:33.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ccd82c740 1 -- 172.21.15.82:0/3771477838 wait complete. 2024-04-21T08:55:33.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.787+0000 7f9ccd82c740 20 client.18135 trim_cache size 0 max 16384 2024-04-21T08:55:33.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9ccd82c740 1 Processor -- start 2024-04-21T08:55:33.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9ccd82c740 1 -- start start 2024-04-21T08:55:33.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9ccd82c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b021230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9ccd82c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018ea0 0x55617b02d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9ccd82c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b021a80 0x55617b014810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9ccd82c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc41af570 con 0x55617b018580 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9ccd82c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc41b1140 con 0x55617b021a80 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9ccd82c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc41afc40 con 0x55617b018ea0 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9cc88f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b021230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9cc88f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b021230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48096/0 (socket says 172.21.15.82:48096) 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9cc88f5640 1 -- 172.21.15.82:0/1383366187 learned_addr learned my addr 172.21.15.82:0/1383366187 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9cc90f6640 1 --2- 172.21.15.82:0/1383366187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b021a80 0x55617b014810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/1383366187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018ea0 0x55617b02d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9cc88f5640 1 -- 172.21.15.82:0/1383366187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018ea0 msgr2=0x55617b02d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/1383366187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018ea0 0x55617b02d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9cc88f5640 1 -- 172.21.15.82:0/1383366187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b021a80 msgr2=0x55617b014810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/1383366187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b021a80 0x55617b014810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9cc88f5640 1 -- 172.21.15.82:0/1383366187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617b013ee0 con 0x55617b018580 2024-04-21T08:55:33.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/1383366187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b021230 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009ca0 tx=0x7f9cc0009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af38fecad0de489b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9caa7fc640 1 -- 172.21.15.82:0/1383366187 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc002d020 con 0x55617b018580 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9caa7fc640 1 -- 172.21.15.82:0/1383366187 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc00095b0 con 0x55617b018580 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.805+0000 7f9caa7fc640 1 -- 172.21.15.82:0/1383366187 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000e8d0 con 0x55617b018580 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.806+0000 7f9ccd82c740 1 -- 172.21.15.82:0/1383366187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 msgr2=0x55617b021230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.806+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/1383366187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b021230 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009ca0 tx=0x7f9cc0009d40 comp rx=0 tx=0).stop 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.806+0000 7f9ccd82c740 1 -- 172.21.15.82:0/1383366187 shutdown_connections 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.806+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/1383366187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b021a80 0x55617b014810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.806+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/1383366187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b018ea0 0x55617b02d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.806+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/1383366187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b021230 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.806+0000 7f9ccd82c740 1 -- 172.21.15.82:0/1383366187 >> 172.21.15.82:0/1383366187 conn(0x55617b012cb0 msgr2=0x55617b014100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.806+0000 7f9ccd82c740 1 -- 172.21.15.82:0/1383366187 shutdown_connections 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.806+0000 7f9ccd82c740 1 -- 172.21.15.82:0/1383366187 wait complete. 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.806+0000 7f9ccd82c740 1 -- start start 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.806+0000 7f9ccd82c740 1 Processor -- start 2024-04-21T08:55:33.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9ccd82c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b025910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9ccd82c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b018ea0 0x55617b025e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9ccd82c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b021a80 0x55617b0263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9ccd82c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc41b0a40 con 0x55617b018580 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9ccd82c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc41b0340 con 0x55617b018ea0 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9ccd82c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc41ad540 con 0x55617b021a80 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9ccd82c740 10 client.0 _ll_register_callbacks cb 0x7f9cc00093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9cc88f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b025910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9cc88f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b025910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48112/0 (socket says 172.21.15.82:48112) 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9cc88f5640 1 -- 172.21.15.82:0/2427066794 learned_addr learned my addr 172.21.15.82:0/2427066794 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/2427066794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b018ea0 0x55617b025e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9cc88f5640 1 -- 172.21.15.82:0/2427066794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b021a80 msgr2=0x55617b0263b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/2427066794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b021a80 0x55617b0263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9cc88f5640 1 -- 172.21.15.82:0/2427066794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b018ea0 msgr2=0x55617b025e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:55:33.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/2427066794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b018ea0 0x55617b025e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9cc88f5640 1 -- 172.21.15.82:0/2427066794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617b097420 con 0x55617b018580 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9cc88f5640 1 --2- 172.21.15.82:0/2427066794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b025910 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0010b70 tx=0x7f9cc0010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bb319ce0816a43 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.807+0000 7f9ca3fff640 1 -- 172.21.15.82:0/2427066794 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0036070 con 0x55617b018580 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ca3fff640 1 -- 172.21.15.82:0/2427066794 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f9cc000eda0 con 0x55617b018580 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ccd82c740 1 -- 172.21.15.82:0/2427066794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cc00095b0 con 0x55617b018580 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ca3fff640 1 -- 172.21.15.82:0/2427066794 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9cc0042050 con 0x55617b018580 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ccd82c740 10 client.18141 fetch_fsmap learned FSMap version 44 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ccd82c740 1 -- 172.21.15.82:0/2427066794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55617b0986d0 con 0x55617b018580 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ca3fff640 1 -- 172.21.15.82:0/2427066794 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc00095b0 con 0x55617b018580 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ca3fff640 1 -- 172.21.15.82:0/2427066794 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f9cc00127c0 con 0x55617b018580 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ccd82c740 10 client.18141 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ccd82c740 10 client.18141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:55:33.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ccd82c740 20 client.18141 populate_metadata read hostname 'smithi082' 2024-04-21T08:55:33.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/2427066794 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55617b098c80 0x55617b09b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:55:33.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ccd82c740 4 client.18141 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:55:33.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9ccd82c740 1 -- 172.21.15.82:0/2427066794 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55617af84a70 con 0x55617b098c80 2024-04-21T08:55:33.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.808+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/2427066794 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55617b098c80 0x55617b09b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:55:33.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9cbbfff640 1 --2- 172.21.15.82:0/2427066794 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55617b098c80 0x55617b09b0f0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a71e314ec1f824cd server_cookie=c3b82eaa5321d463 in_seq=0 out_seq=0 2024-04-21T08:55:33.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ca3fff640 10 client.18141 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:33.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ca3fff640 1 -- 172.21.15.82:0/2427066794 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55617af84a70 con 0x55617b098c80 2024-04-21T08:55:33.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ca3fff640 10 client.18141 handle_command_reply: tid=0 2024-04-21T08:55:33.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ccd82c740 1 client.18141 shutdown 2024-04-21T08:55:33.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ccd82c740 1 -- 172.21.15.82:0/2427066794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 msgr2=0x55617b025910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/2427066794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b025910 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0010b70 tx=0x7f9cc0010e70 comp rx=0 tx=0).stop 2024-04-21T08:55:33.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ccd82c740 1 -- 172.21.15.82:0/2427066794 shutdown_connections 2024-04-21T08:55:33.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/2427066794 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55617b098c80 0x55617b09b0f0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/2427066794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55617b021a80 0x55617b0263b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ca3fff640 0 client.18141 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:55:33.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/2427066794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55617b018ea0 0x55617b025e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ccd82c740 1 --2- 172.21.15.82:0/2427066794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55617b018580 0x55617b025910 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ccd82c740 1 -- 172.21.15.82:0/2427066794 >> 172.21.15.82:0/2427066794 conn(0x55617b012cb0 msgr2=0x55617b011d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:33.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ccd82c740 1 -- 172.21.15.82:0/2427066794 shutdown_connections 2024-04-21T08:55:33.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ccd82c740 1 -- 172.21.15.82:0/2427066794 wait complete. 2024-04-21T08:55:33.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.809+0000 7f9ccd82c740 20 client.18141 trim_cache size 0 max 16384 2024-04-21T08:55:33.816 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:55:33.817 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:55:33.817 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:55:33.817 INFO:teuthology.orchestra.run.smithi082.stdout: "16186bb7-9603-4aff-8b40-116ed394aec2": { 2024-04-21T08:55:33.817 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:55:33.817 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "16186bb7-9603-4aff-8b40-116ed394aec2", 2024-04-21T08:55:33.817 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:55:33.817 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:55:33.817 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:55:33.817 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:55:33.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.810+0000 7f9ccab59640 1 -- 172.21.15.82:0/2114267187 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9cac0a3ec0 msgr2=0x7f9cac0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.810+0000 7f9ccab59640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9cac0a3ec0 0x7f9cac0a6380 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc004af0 tx=0x7f9cbc004a40 comp rx=0 tx=0).stop 2024-04-21T08:55:33.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.810+0000 7f9ccab59640 1 -- 172.21.15.82:0/2114267187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cc4107270 msgr2=0x7f9cc4198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:55:33.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.810+0000 7f9ccab59640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cc4107270 0x7f9cc4198be0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400c970 tx=0x7f9cb400ce30 comp rx=0 tx=0).stop 2024-04-21T08:55:33.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.810+0000 7f9ccab59640 1 -- 172.21.15.82:0/2114267187 shutdown_connections 2024-04-21T08:55:33.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.810+0000 7f9ccab59640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cc419cf60 0x7f9cc419d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.810+0000 7f9ccab59640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9cac0a3ec0 0x7f9cac0a6380 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.810+0000 7f9ccab59640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cc4107270 0x7f9cc4198be0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.810+0000 7f9ccab59640 1 --2- 172.21.15.82:0/2114267187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc41021a0 0x7f9cc41986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:55:33.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.810+0000 7f9ccab59640 1 -- 172.21.15.82:0/2114267187 >> 172.21.15.82:0/2114267187 conn(0x7f9cc40fda40 msgr2=0x7f9cc40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:55:33.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.810+0000 7f9ccab59640 1 -- 172.21.15.82:0/2114267187 shutdown_connections 2024-04-21T08:55:33.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:55:33.810+0000 7f9ccab59640 1 -- 172.21.15.82:0/2114267187 wait complete. 2024-04-21T08:55:33.820 INFO:tasks.cephfs.filesystem:scrub status for tag:16186bb7-9603-4aff-8b40-116ed394aec2 - {'path': '/', 'tag': '16186bb7-9603-4aff-8b40-116ed394aec2', 'options': 'recursive,force'} 2024-04-21T08:55:34.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:34 smithi134 ceph-mon[31772]: pgmap v263: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:55:34.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:34 smithi082 ceph-mon[27967]: pgmap v263: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:55:34.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:34 smithi106 ceph-mon[31072]: pgmap v263: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:55:35.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:35 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:55:35.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:35 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:55:35.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:35 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:55:35.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:35 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:55:35.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:35 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:55:35.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:35 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:55:35.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:35 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:55:35.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:35 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:55:35.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:35 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:55:35.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:35 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:55:35.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:35 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:55:35.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:35 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:55:36.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:36 smithi134 ceph-mon[31772]: pgmap v264: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:55:36.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:36 smithi082 ceph-mon[27967]: pgmap v264: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:55:36.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:36 smithi106 ceph-mon[31072]: pgmap v264: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:55:38.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:38 smithi134 ceph-mon[31772]: pgmap v265: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-21T08:55:38.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:38 smithi082 ceph-mon[27967]: pgmap v265: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-21T08:55:38.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:38 smithi106 ceph-mon[31072]: pgmap v265: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-21T08:55:39.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:39 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:55:39.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:39 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:55:39.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:39 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:55:40.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:40 smithi134 ceph-mon[31772]: pgmap v266: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-21T08:55:40.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:40 smithi082 ceph-mon[27967]: pgmap v266: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-21T08:55:40.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:40 smithi106 ceph-mon[31072]: pgmap v266: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-21T08:55:42.431 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:42 smithi106 ceph-mon[31072]: pgmap v267: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:55:42.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:42 smithi134 ceph-mon[31772]: pgmap v267: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:55:42.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:42 smithi082 ceph-mon[27967]: pgmap v267: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:55:44.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:44 smithi134 ceph-mon[31772]: pgmap v268: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 14 op/s 2024-04-21T08:55:44.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:44 smithi082 ceph-mon[27967]: pgmap v268: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 14 op/s 2024-04-21T08:55:44.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:44 smithi106 ceph-mon[31072]: pgmap v268: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 14 op/s 2024-04-21T08:55:46.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:46 smithi134 ceph-mon[31772]: pgmap v269: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:55:46.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:46 smithi082 ceph-mon[27967]: pgmap v269: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:55:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:46 smithi106 ceph-mon[31072]: pgmap v269: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:55:48.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:48 smithi134 ceph-mon[31772]: pgmap v270: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-04-21T08:55:48.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:48 smithi082 ceph-mon[27967]: pgmap v270: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-04-21T08:55:48.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:48 smithi106 ceph-mon[31072]: pgmap v270: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 op/s 2024-04-21T08:55:50.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:50 smithi134 ceph-mon[31772]: pgmap v271: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-21T08:55:50.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:50 smithi082 ceph-mon[27967]: pgmap v271: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-21T08:55:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:50 smithi106 ceph-mon[31072]: pgmap v271: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-21T08:55:52.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:52 smithi134 ceph-mon[31772]: pgmap v272: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:55:52.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:52 smithi082 ceph-mon[27967]: pgmap v272: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:55:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:52 smithi106 ceph-mon[31072]: pgmap v272: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:55:54.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:54 smithi134 ceph-mon[31772]: pgmap v273: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-21T08:55:54.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:54 smithi082 ceph-mon[27967]: pgmap v273: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-21T08:55:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:54 smithi106 ceph-mon[31072]: pgmap v273: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-21T08:55:56.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:56 smithi134 ceph-mon[31772]: pgmap v274: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:55:56.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:56 smithi082 ceph-mon[27967]: pgmap v274: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:55:56.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:56 smithi106 ceph-mon[31072]: pgmap v274: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:55:58.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:55:58 smithi134 ceph-mon[31772]: pgmap v275: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-21T08:55:58.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:55:58 smithi082 ceph-mon[27967]: pgmap v275: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-21T08:55:58.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:55:58 smithi106 ceph-mon[31072]: pgmap v275: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-21T08:56:00.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:00 smithi134 ceph-mon[31772]: pgmap v276: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-21T08:56:00.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:00 smithi082 ceph-mon[27967]: pgmap v276: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-21T08:56:00.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:00 smithi106 ceph-mon[31072]: pgmap v276: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-21T08:56:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:02 smithi134 ceph-mon[31772]: pgmap v277: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-21T08:56:02.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:02 smithi082 ceph-mon[27967]: pgmap v277: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-21T08:56:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:02 smithi106 ceph-mon[31072]: pgmap v277: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-21T08:56:03.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:56:03.821 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:56:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.932+0000 7f0da0dd5640 1 -- 172.21.15.82:0/261111734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c06af70 msgr2=0x7f0d9c10edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.932+0000 7f0da0dd5640 1 --2- 172.21.15.82:0/261111734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c06af70 0x7f0d9c10edb0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009ca0 tx=0x7f0d9402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.932+0000 7f0da0dd5640 1 -- 172.21.15.82:0/261111734 shutdown_connections 2024-04-21T08:56:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.932+0000 7f0da0dd5640 1 --2- 172.21.15.82:0/261111734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c06af70 0x7f0d9c10edb0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.932+0000 7f0da0dd5640 1 --2- 172.21.15.82:0/261111734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d9c06a5b0 0x7f0d9c06aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.932+0000 7f0da0dd5640 1 --2- 172.21.15.82:0/261111734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d9c106f20 0x7f0d9c107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.932+0000 7f0da0dd5640 1 -- 172.21.15.82:0/261111734 >> 172.21.15.82:0/261111734 conn(0x7f0d9c077c40 msgr2=0x7f0d9c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.932+0000 7f0da0dd5640 1 -- 172.21.15.82:0/261111734 shutdown_connections 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.932+0000 7f0da0dd5640 1 -- 172.21.15.82:0/261111734 wait complete. 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.933+0000 7f0da0dd5640 1 Processor -- start 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.933+0000 7f0da0dd5640 1 -- start start 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.933+0000 7f0da0dd5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d9c06a5b0 0x7f0d9c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.933+0000 7f0da0dd5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c06af70 0x7f0d9c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.933+0000 7f0da0dd5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d9c106f20 0x7f0d9c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.933+0000 7f0da0dd5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c112210 con 0x7f0d9c06af70 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.933+0000 7f0da0dd5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c112090 con 0x7f0d9c106f20 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.933+0000 7f0da0dd5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c112390 con 0x7f0d9c06a5b0 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.933+0000 7f0d99d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c06af70 0x7f0d9c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d99d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c06af70 0x7f0d9c198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38616/0 (socket says 172.21.15.82:38616) 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d99d9b640 1 -- 172.21.15.82:0/209435308 learned_addr learned my addr 172.21.15.82:0/209435308 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d9a59c640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d9c06a5b0 0x7f0d9c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d9c106f20 0x7f0d9c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d99d9b640 1 -- 172.21.15.82:0/209435308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d9c06a5b0 msgr2=0x7f0d9c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d99d9b640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d9c06a5b0 0x7f0d9c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d99d9b640 1 -- 172.21.15.82:0/209435308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d9c106f20 msgr2=0x7f0d9c19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d99d9b640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d9c106f20 0x7f0d9c19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d99d9b640 1 -- 172.21.15.82:0/209435308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c19d600 con 0x7f0d9c06af70 2024-04-21T08:56:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d99d9b640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c06af70 0x7f0d9c198b50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c920 tx=0x7f0d9000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d8b7fe640 1 -- 172.21.15.82:0/209435308 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90007bb0 con 0x7f0d9c06af70 2024-04-21T08:56:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0da0dd5640 1 -- 172.21.15.82:0/209435308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c19d8f0 con 0x7f0d9c06af70 2024-04-21T08:56:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d8b7fe640 1 -- 172.21.15.82:0/209435308 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d90007d50 con 0x7f0d9c06af70 2024-04-21T08:56:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0d8b7fe640 1 -- 172.21.15.82:0/209435308 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90018730 con 0x7f0d9c06af70 2024-04-21T08:56:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.934+0000 7f0da0dd5640 1 -- 172.21.15.82:0/209435308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c1a5180 con 0x7f0d9c06af70 2024-04-21T08:56:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.936+0000 7f0d8b7fe640 1 -- 172.21.15.82:0/209435308 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0d9000f450 con 0x7f0d9c06af70 2024-04-21T08:56:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.936+0000 7f0da39c8740 1 Processor -- start 2024-04-21T08:56:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0da39c8740 1 -- start start 2024-04-21T08:56:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0da39c8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 0x5600ea875690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0da39c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea875bd0 0x5600ea882950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0da39c8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea882e90 0x5600ea885280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0da39c8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5600ea7da1e0 con 0x5600ea8752b0 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0da39c8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5600ea777390 con 0x5600ea882e90 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0da39c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5600ea7d61a0 con 0x5600ea875bd0 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0d8b7fe640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0d840a3d20 0x7f0d840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0d9ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 0x5600ea875690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0d9ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 0x5600ea875690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38620/0 (socket says 172.21.15.82:38620) 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0d9ad9d640 1 -- 172.21.15.82:0/475899802 learned_addr learned my addr 172.21.15.82:0/475899802 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0d840a3d20 0x7f0d840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0d9a59c640 1 --2- 172.21.15.82:0/475899802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea875bd0 0x5600ea882950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0d8b7fe640 1 -- 172.21.15.82:0/209435308 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f0d900d0d50 con 0x7f0d9c06af70 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.937+0000 7f0d99d9b640 1 --2- 172.21.15.82:0/475899802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea882e90 0x5600ea885280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.938+0000 7f0d9ad9d640 1 -- 172.21.15.82:0/475899802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea875bd0 msgr2=0x5600ea882950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.938+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/475899802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea875bd0 0x5600ea882950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.938+0000 7f0d9ad9d640 1 -- 172.21.15.82:0/475899802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea882e90 msgr2=0x5600ea885280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:03.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.938+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/475899802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea882e90 0x5600ea885280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.938+0000 7f0d9ad9d640 1 -- 172.21.15.82:0/475899802 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ea5b67d0 con 0x5600ea8752b0 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.938+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/475899802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 0x5600ea875690 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f0d94005fe0 tx=0x7f0d94032640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112e070204e45fda server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.938+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0d840a3d20 0x7f0d840a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d94032450 tx=0x7f0d94047000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.938+0000 7f0d88ff9640 1 -- 172.21.15.82:0/475899802 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94051070 con 0x5600ea8752b0 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.938+0000 7f0d88ff9640 1 -- 172.21.15.82:0/475899802 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d94048070 con 0x5600ea8752b0 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.938+0000 7f0d88ff9640 1 -- 172.21.15.82:0/475899802 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94039040 con 0x5600ea8752b0 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 -- 172.21.15.82:0/475899802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 msgr2=0x5600ea875690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 --2- 172.21.15.82:0/475899802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 0x5600ea875690 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f0d94005fe0 tx=0x7f0d94032640 comp rx=0 tx=0).stop 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 -- 172.21.15.82:0/475899802 shutdown_connections 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 --2- 172.21.15.82:0/475899802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea882e90 0x5600ea885280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 --2- 172.21.15.82:0/475899802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea875bd0 0x5600ea882950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 --2- 172.21.15.82:0/475899802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 0x5600ea875690 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 -- 172.21.15.82:0/475899802 >> 172.21.15.82:0/475899802 conn(0x5600ea86fcb0 msgr2=0x5600ea870090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:03.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 -- 172.21.15.82:0/475899802 shutdown_connections 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 -- 172.21.15.82:0/475899802 wait complete. 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 -- start start 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 Processor -- start 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea8752b0 0x5600ea87fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea875bd0 0x5600ea880420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 0x5600ea880960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.939+0000 7f0da39c8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5600ea885bf0 con 0x5600ea882e90 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0da39c8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5600ea7932f0 con 0x5600ea875bd0 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0da39c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5600ea885d70 con 0x5600ea8752b0 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0da39c8740 10 client.0 _ll_register_callbacks cb 0x5600ea7a0580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d9ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 0x5600ea880960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d9ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 0x5600ea880960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38622/0 (socket says 172.21.15.82:38622) 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d9ad9d640 1 -- 172.21.15.82:0/3102546751 learned_addr learned my addr 172.21.15.82:0/3102546751 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d9a59c640 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea8752b0 0x5600ea87fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d99d9b640 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea875bd0 0x5600ea880420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d9ad9d640 1 -- 172.21.15.82:0/3102546751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea8752b0 msgr2=0x5600ea87fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea8752b0 0x5600ea87fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d9ad9d640 1 -- 172.21.15.82:0/3102546751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea875bd0 msgr2=0x5600ea880420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea875bd0 0x5600ea880420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d9ad9d640 1 -- 172.21.15.82:0/3102546751 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ea4ec3f0 con 0x5600ea882e90 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d99d9b640 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea875bd0 0x5600ea880420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 0x5600ea880960 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0d9402c630 tx=0x7f0d9402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bf97fbac2b08d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d81ffb640 1 -- 172.21.15.82:0/3102546751 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94039070 con 0x5600ea882e90 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d81ffb640 1 -- 172.21.15.82:0/3102546751 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f0d94004040 con 0x5600ea882e90 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0da39c8740 1 -- 172.21.15.82:0/3102546751 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d94048070 con 0x5600ea882e90 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d81ffb640 1 -- 172.21.15.82:0/3102546751 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0d94051070 con 0x5600ea882e90 2024-04-21T08:56:03.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0da39c8740 10 client.18153 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0da39c8740 1 -- 172.21.15.82:0/3102546751 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5600ea4ac0f0 con 0x5600ea882e90 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.940+0000 7f0d81ffb640 1 -- 172.21.15.82:0/3102546751 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d94048070 con 0x5600ea882e90 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.941+0000 7f0d81ffb640 1 -- 172.21.15.82:0/3102546751 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f0d9404c810 con 0x5600ea882e90 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.941+0000 7f0da39c8740 10 client.18153 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.941+0000 7f0da39c8740 10 client.18153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.941+0000 7f0da39c8740 20 client.18153 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.941+0000 7f0da39c8740 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5600ea892ce0 0x5600ea8950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.941+0000 7f0da39c8740 4 client.18153 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.941+0000 7f0da39c8740 1 -- 172.21.15.82:0/3102546751 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600ea7e7c00 con 0x5600ea892ce0 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.941+0000 7f0d9a59c640 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5600ea892ce0 0x5600ea8950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.941+0000 7f0d81ffb640 10 client.18153 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.941+0000 7f0d9a59c640 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5600ea892ce0 0x5600ea8950d0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e291878bf696bc1 server_cookie=397922f819e9c1f0 in_seq=0 out_seq=0 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0d81ffb640 1 -- 172.21.15.82:0/3102546751 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600ea7e7c00 con 0x5600ea892ce0 2024-04-21T08:56:03.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0d81ffb640 10 client.18153 handle_command_reply: tid=0 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0da39c8740 1 client.18153 shutdown 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0da39c8740 1 -- 172.21.15.82:0/3102546751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 msgr2=0x5600ea880960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0da39c8740 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 0x5600ea880960 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0d9402c630 tx=0x7f0d9402c8b0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0da39c8740 1 -- 172.21.15.82:0/3102546751 shutdown_connections 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0da39c8740 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5600ea892ce0 0x5600ea8950d0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0da39c8740 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 0x5600ea880960 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0da39c8740 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea875bd0 0x5600ea880420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0da39c8740 1 --2- 172.21.15.82:0/3102546751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea8752b0 0x5600ea87fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0d81ffb640 0 client.18153 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0da39c8740 1 -- 172.21.15.82:0/3102546751 >> 172.21.15.82:0/3102546751 conn(0x5600ea86fcb0 msgr2=0x5600ea871a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0da39c8740 1 -- 172.21.15.82:0/3102546751 shutdown_connections 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0da39c8740 1 -- 172.21.15.82:0/3102546751 wait complete. 2024-04-21T08:56:03.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.942+0000 7f0da39c8740 20 client.18153 trim_cache size 0 max 16384 2024-04-21T08:56:03.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.959+0000 7f0da39c8740 1 Processor -- start 2024-04-21T08:56:03.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.959+0000 7f0da39c8740 1 -- start start 2024-04-21T08:56:03.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0da39c8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 0x5600ea88eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0da39c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea875bd0 0x5600ea87f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0da39c8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea882e90 0x5600ea880780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0da39c8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1ad230 con 0x5600ea8752b0 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0da39c8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1aee00 con 0x5600ea882e90 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0da39c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1ad900 con 0x5600ea875bd0 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d9a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 0x5600ea88eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d9a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 0x5600ea88eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38632/0 (socket says 172.21.15.82:38632) 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d9a59c640 1 -- 172.21.15.82:0/2845036539 learned_addr learned my addr 172.21.15.82:0/2845036539 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/2845036539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea882e90 0x5600ea880780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d9a59c640 1 -- 172.21.15.82:0/2845036539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea875bd0 msgr2=0x5600ea87f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d9a59c640 1 --2- 172.21.15.82:0/2845036539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea875bd0 0x5600ea87f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d9a59c640 1 -- 172.21.15.82:0/2845036539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea882e90 msgr2=0x5600ea880780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d9a59c640 1 --2- 172.21.15.82:0/2845036539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea882e90 0x5600ea880780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d9a59c640 1 -- 172.21.15.82:0/2845036539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ea875a00 con 0x5600ea8752b0 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/2845036539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea882e90 0x5600ea880780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d9a59c640 1 --2- 172.21.15.82:0/2845036539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 0x5600ea88eaf0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c103410 tx=0x7f0d8c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22819be0cc3b9716 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d83fff640 1 -- 172.21.15.82:0/2845036539 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c021070 con 0x5600ea8752b0 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d83fff640 1 -- 172.21.15.82:0/2845036539 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d8c016cf0 con 0x5600ea8752b0 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.960+0000 7f0d83fff640 1 -- 172.21.15.82:0/2845036539 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c015620 con 0x5600ea8752b0 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.961+0000 7f0da39c8740 1 -- 172.21.15.82:0/2845036539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 msgr2=0x5600ea88eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.961+0000 7f0da39c8740 1 --2- 172.21.15.82:0/2845036539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 0x5600ea88eaf0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c103410 tx=0x7f0d8c008040 comp rx=0 tx=0).stop 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.961+0000 7f0da39c8740 1 -- 172.21.15.82:0/2845036539 shutdown_connections 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.961+0000 7f0da39c8740 1 --2- 172.21.15.82:0/2845036539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea882e90 0x5600ea880780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.961+0000 7f0da39c8740 1 --2- 172.21.15.82:0/2845036539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea875bd0 0x5600ea87f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.961+0000 7f0da39c8740 1 --2- 172.21.15.82:0/2845036539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea8752b0 0x5600ea88eaf0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.961+0000 7f0da39c8740 1 -- 172.21.15.82:0/2845036539 >> 172.21.15.82:0/2845036539 conn(0x5600ea86fcb0 msgr2=0x5600ea871090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.961+0000 7f0da39c8740 1 -- 172.21.15.82:0/2845036539 shutdown_connections 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.961+0000 7f0da39c8740 1 -- 172.21.15.82:0/2845036539 wait complete. 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.961+0000 7f0da39c8740 1 -- start start 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.961+0000 7f0da39c8740 1 Processor -- start 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0da39c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea8752b0 0x5600ea88b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0da39c8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea875bd0 0x5600ea87c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0da39c8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 0x5600ea87cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0da39c8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1ae700 con 0x5600ea882e90 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0da39c8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1ae000 con 0x5600ea875bd0 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0da39c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c1ab200 con 0x5600ea8752b0 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0da39c8740 10 client.0 _ll_register_callbacks cb 0x7f0d9404f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d9ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 0x5600ea87cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d9ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 0x5600ea87cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38640/0 (socket says 172.21.15.82:38640) 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d9ad9d640 1 -- 172.21.15.82:0/1375406058 learned_addr learned my addr 172.21.15.82:0/1375406058 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d9a59c640 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea8752b0 0x5600ea88b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d99d9b640 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea875bd0 0x5600ea87c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d9ad9d640 1 -- 172.21.15.82:0/1375406058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea8752b0 msgr2=0x5600ea88b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea8752b0 0x5600ea88b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d9ad9d640 1 -- 172.21.15.82:0/1375406058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea875bd0 msgr2=0x5600ea87c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:03.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea875bd0 0x5600ea87c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d9ad9d640 1 -- 172.21.15.82:0/1375406058 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ea87d6f0 con 0x5600ea882e90 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d99d9b640 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea875bd0 0x5600ea87c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d9ad9d640 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 0x5600ea87cec0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0d9404cda0 tx=0x7f0d94005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910f521cc574f3b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d81ffb640 1 -- 172.21.15.82:0/1375406058 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94063060 con 0x5600ea882e90 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d81ffb640 1 -- 172.21.15.82:0/1375406058 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f0d94004040 con 0x5600ea882e90 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0da39c8740 1 -- 172.21.15.82:0/1375406058 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d8c016cf0 con 0x5600ea882e90 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0d81ffb640 1 -- 172.21.15.82:0/1375406058 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0d94039040 con 0x5600ea882e90 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0da39c8740 10 client.18159 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.962+0000 7f0da39c8740 1 -- 172.21.15.82:0/1375406058 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5600ea8f8f30 con 0x5600ea882e90 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.963+0000 7f0d81ffb640 1 -- 172.21.15.82:0/1375406058 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d8c016cf0 con 0x5600ea882e90 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.963+0000 7f0d81ffb640 1 -- 172.21.15.82:0/1375406058 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f0d94034440 con 0x5600ea882e90 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.963+0000 7f0da39c8740 10 client.18159 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:03.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.963+0000 7f0da39c8740 10 client.18159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.963+0000 7f0da39c8740 20 client.18159 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.963+0000 7f0da39c8740 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5600ea8f9700 0x5600ea8fbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.963+0000 7f0da39c8740 4 client.18159 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.963+0000 7f0da39c8740 1 -- 172.21.15.82:0/1375406058 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5600ea7e1a70 con 0x5600ea8f9700 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.963+0000 7f0d9a59c640 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5600ea8f9700 0x5600ea8fbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.963+0000 7f0d81ffb640 10 client.18159 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.963+0000 7f0d9a59c640 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5600ea8f9700 0x5600ea8fbb20 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cdddcf30032e890 server_cookie=a69a441319701e93 in_seq=0 out_seq=0 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0d81ffb640 1 -- 172.21.15.82:0/1375406058 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5600ea7e1a70 con 0x5600ea8f9700 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0d81ffb640 10 client.18159 handle_command_reply: tid=0 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da39c8740 1 client.18159 shutdown 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da39c8740 1 -- 172.21.15.82:0/1375406058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 msgr2=0x5600ea87cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da39c8740 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 0x5600ea87cec0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0d9404cda0 tx=0x7f0d94005df0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da39c8740 1 -- 172.21.15.82:0/1375406058 shutdown_connections 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da39c8740 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5600ea8f9700 0x5600ea8fbb20 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da39c8740 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600ea882e90 0x5600ea87cec0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0d81ffb640 0 client.18159 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:03.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da39c8740 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5600ea875bd0 0x5600ea87c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da39c8740 1 --2- 172.21.15.82:0/1375406058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5600ea8752b0 0x5600ea88b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da39c8740 1 -- 172.21.15.82:0/1375406058 >> 172.21.15.82:0/1375406058 conn(0x5600ea86fcb0 msgr2=0x5600ea86ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:03.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da39c8740 1 -- 172.21.15.82:0/1375406058 shutdown_connections 2024-04-21T08:56:03.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da39c8740 1 -- 172.21.15.82:0/1375406058 wait complete. 2024-04-21T08:56:03.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da39c8740 20 client.18159 trim_cache size 0 max 16384 2024-04-21T08:56:03.972 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:56:03.972 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:56:03.972 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:56:03.973 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:56:03.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da0dd5640 1 -- 172.21.15.82:0/209435308 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0d840a3d20 msgr2=0x7f0d840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:03.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da0dd5640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0d840a3d20 0x7f0d840a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d94032450 tx=0x7f0d94047000 comp rx=0 tx=0).stop 2024-04-21T08:56:03.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da0dd5640 1 -- 172.21.15.82:0/209435308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c06af70 msgr2=0x7f0d9c198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:03.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da0dd5640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c06af70 0x7f0d9c198b50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000c920 tx=0x7f0d9000cd20 comp rx=0 tx=0).stop 2024-04-21T08:56:03.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da0dd5640 1 -- 172.21.15.82:0/209435308 shutdown_connections 2024-04-21T08:56:03.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da0dd5640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d9c106f20 0x7f0d9c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da0dd5640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f0d840a3d20 0x7f0d840a61e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da0dd5640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c06af70 0x7f0d9c198b50 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da0dd5640 1 --2- 172.21.15.82:0/209435308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d9c06a5b0 0x7f0d9c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:03.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da0dd5640 1 -- 172.21.15.82:0/209435308 >> 172.21.15.82:0/209435308 conn(0x7f0d9c077c40 msgr2=0x7f0d9c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:03.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da0dd5640 1 -- 172.21.15.82:0/209435308 shutdown_connections 2024-04-21T08:56:03.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:03.964+0000 7f0da0dd5640 1 -- 172.21.15.82:0/209435308 wait complete. 2024-04-21T08:56:03.976 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:56:03.976 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:56:04.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.074+0000 7f5b14f59640 1 -- 172.21.15.82:0/2311500717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b10105e30 msgr2=0x7f5b10106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.074+0000 7f5b14f59640 1 --2- 172.21.15.82:0/2311500717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b10105e30 0x7f5b10106210 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009930 tx=0x7f5b0002bc30 comp rx=0 tx=0).stop 2024-04-21T08:56:04.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.074+0000 7f5b14f59640 1 -- 172.21.15.82:0/2311500717 shutdown_connections 2024-04-21T08:56:04.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.074+0000 7f5b14f59640 1 --2- 172.21.15.82:0/2311500717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b10101b60 0x7f5b1010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.074+0000 7f5b14f59640 1 --2- 172.21.15.82:0/2311500717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b101011a0 0x7f5b10101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.074+0000 7f5b14f59640 1 --2- 172.21.15.82:0/2311500717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b10105e30 0x7f5b10106210 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.074+0000 7f5b14f59640 1 -- 172.21.15.82:0/2311500717 >> 172.21.15.82:0/2311500717 conn(0x7f5b10077e30 msgr2=0x7f5b10078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.075+0000 7f5b14f59640 1 -- 172.21.15.82:0/2311500717 shutdown_connections 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.075+0000 7f5b14f59640 1 -- 172.21.15.82:0/2311500717 wait complete. 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.075+0000 7f5b14f59640 1 Processor -- start 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.075+0000 7f5b14f59640 1 -- start start 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.075+0000 7f5b14f59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b101011a0 0x7f5b101973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.075+0000 7f5b14f59640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b10101b60 0x7f5b101978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.075+0000 7f5b14f59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b10105e30 0x7f5b10197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.075+0000 7f5b14f59640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5b101110c0 con 0x7f5b10105e30 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.075+0000 7f5b14f59640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5b10110f40 con 0x7f5b10101b60 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.075+0000 7f5b14f59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5b10111240 con 0x7f5b101011a0 2024-04-21T08:56:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b0ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b10105e30 0x7f5b10197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b0ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b10105e30 0x7f5b10197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38662/0 (socket says 172.21.15.82:38662) 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b0ed9d640 1 -- 172.21.15.82:0/1098705623 learned_addr learned my addr 172.21.15.82:0/1098705623 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b0e59c640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b101011a0 0x7f5b101973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b07fff640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b10101b60 0x7f5b101978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b0ed9d640 1 -- 172.21.15.82:0/1098705623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b101011a0 msgr2=0x7f5b101973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b0ed9d640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b101011a0 0x7f5b101973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b0ed9d640 1 -- 172.21.15.82:0/1098705623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b10101b60 msgr2=0x7f5b101978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b0ed9d640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b10101b60 0x7f5b101978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b0ed9d640 1 -- 172.21.15.82:0/1098705623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b101a6c20 con 0x7f5b10105e30 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b07fff640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b10101b60 0x7f5b101978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b0ed9d640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b10105e30 0x7f5b10197e30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800cad0 tx=0x7f5b0800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b077fe640 1 -- 172.21.15.82:0/1098705623 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800ae20 con 0x7f5b10105e30 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b077fe640 1 -- 172.21.15.82:0/1098705623 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b08005020 con 0x7f5b10105e30 2024-04-21T08:56:04.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b14f59640 1 -- 172.21.15.82:0/1098705623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b101a6f10 con 0x7f5b10105e30 2024-04-21T08:56:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b077fe640 1 -- 172.21.15.82:0/1098705623 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800f6b0 con 0x7f5b10105e30 2024-04-21T08:56:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.076+0000 7f5b14f59640 1 -- 172.21.15.82:0/1098705623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b101a73e0 con 0x7f5b10105e30 2024-04-21T08:56:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.077+0000 7f5b14f59640 1 -- 172.21.15.82:0/1098705623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b10068de0 con 0x7f5b10105e30 2024-04-21T08:56:04.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.078+0000 7f5b077fe640 1 -- 172.21.15.82:0/1098705623 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5b080051d0 con 0x7f5b10105e30 2024-04-21T08:56:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.079+0000 7f5b077fe640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5afc0a3d20 0x7f5afc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.079+0000 7f5b0e59c640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5afc0a3d20 0x7f5afc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.079+0000 7f5b077fe640 1 -- 172.21.15.82:0/1098705623 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f5b080d0ca0 con 0x7f5b10105e30 2024-04-21T08:56:04.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.079+0000 7f5b0e59c640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5afc0a3d20 0x7f5afc0a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5b00005ab0 tx=0x7f5b00005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.084+0000 7f5b077fe640 1 -- 172.21.15.82:0/1098705623 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b08014030 con 0x7f5b10105e30 2024-04-21T08:56:04.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.301+0000 7f5b14f59640 1 -- 172.21.15.82:0/1098705623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b1002d5c0 con 0x7f5b10105e30 2024-04-21T08:56:04.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.301+0000 7f5b077fe640 1 -- 172.21.15.82:0/1098705623 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11949 (secure 0 0 0) 0x7f5b0809cf30 con 0x7f5b10105e30 2024-04-21T08:56:04.302 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:56:04.303 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:55:27.214396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:56:04.303 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 44 2024-04-21T08:56:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.303+0000 7f5b14f59640 1 -- 172.21.15.82:0/1098705623 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5afc0a3d20 msgr2=0x7f5afc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.303+0000 7f5b14f59640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5afc0a3d20 0x7f5afc0a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5b00005ab0 tx=0x7f5b00005fd0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.303+0000 7f5b14f59640 1 -- 172.21.15.82:0/1098705623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b10105e30 msgr2=0x7f5b10197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.303+0000 7f5b14f59640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b10105e30 0x7f5b10197e30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800cad0 tx=0x7f5b0800cf90 comp rx=0 tx=0).stop 2024-04-21T08:56:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.303+0000 7f5b14f59640 1 -- 172.21.15.82:0/1098705623 shutdown_connections 2024-04-21T08:56:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.303+0000 7f5b14f59640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5afc0a3d20 0x7f5afc0a61e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.303+0000 7f5b14f59640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b10105e30 0x7f5b10197e30 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.303+0000 7f5b14f59640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b10101b60 0x7f5b101978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.303+0000 7f5b14f59640 1 --2- 172.21.15.82:0/1098705623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b101011a0 0x7f5b101973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.303+0000 7f5b14f59640 1 -- 172.21.15.82:0/1098705623 >> 172.21.15.82:0/1098705623 conn(0x7f5b10077e30 msgr2=0x7f5b100fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.303+0000 7f5b14f59640 1 -- 172.21.15.82:0/1098705623 shutdown_connections 2024-04-21T08:56:04.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.303+0000 7f5b14f59640 1 -- 172.21.15.82:0/1098705623 wait complete. 2024-04-21T08:56:04.315 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:56:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.414+0000 7fc318806640 1 -- 172.21.15.82:0/1114085113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc314101570 msgr2=0x7fc3141019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.414+0000 7fc318806640 1 --2- 172.21.15.82:0/1114085113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc314101570 0x7fc3141019f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fc304009900 tx=0x7fc30402bc30 comp rx=0 tx=0).stop 2024-04-21T08:56:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.414+0000 7fc318806640 1 -- 172.21.15.82:0/1114085113 shutdown_connections 2024-04-21T08:56:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.414+0000 7fc318806640 1 --2- 172.21.15.82:0/1114085113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc314101f30 0x7fc31410dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.414+0000 7fc318806640 1 --2- 172.21.15.82:0/1114085113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc314101570 0x7fc3141019f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.414+0000 7fc318806640 1 --2- 172.21.15.82:0/1114085113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc314105e50 0x7fc314106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.414+0000 7fc318806640 1 -- 172.21.15.82:0/1114085113 >> 172.21.15.82:0/1114085113 conn(0x7fc3140773f0 msgr2=0x7fc3140fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.414+0000 7fc318806640 1 -- 172.21.15.82:0/1114085113 shutdown_connections 2024-04-21T08:56:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.414+0000 7fc318806640 1 -- 172.21.15.82:0/1114085113 wait complete. 2024-04-21T08:56:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.415+0000 7fc318806640 1 Processor -- start 2024-04-21T08:56:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.415+0000 7fc318806640 1 -- start start 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.415+0000 7fc318806640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc314101570 0x7fc3141986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.415+0000 7fc318806640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc314101f30 0x7fc314198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.415+0000 7fc318806640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc314105e50 0x7fc31419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.415+0000 7fc318806640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc314111060 con 0x7fc314101570 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.415+0000 7fc318806640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc314110ee0 con 0x7fc314105e50 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.415+0000 7fc318806640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141111e0 con 0x7fc314101f30 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.415+0000 7fc31259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc314101570 0x7fc3141986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.415+0000 7fc31259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc314101570 0x7fc3141986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38684/0 (socket says 172.21.15.82:38684) 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.415+0000 7fc31259c640 1 -- 172.21.15.82:0/3112555204 learned_addr learned my addr 172.21.15.82:0/3112555204 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.415+0000 7fc311d9b640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc314101f30 0x7fc314198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.416+0000 7fc312d9d640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc314105e50 0x7fc31419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.416+0000 7fc31259c640 1 -- 172.21.15.82:0/3112555204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc314101f30 msgr2=0x7fc314198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.416+0000 7fc31259c640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc314101f30 0x7fc314198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.416+0000 7fc31259c640 1 -- 172.21.15.82:0/3112555204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc314105e50 msgr2=0x7fc31419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.416+0000 7fc31259c640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc314105e50 0x7fc31419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.416+0000 7fc31259c640 1 -- 172.21.15.82:0/3112555204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31419d6c0 con 0x7fc314101570 2024-04-21T08:56:04.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.416+0000 7fc31259c640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc314101570 0x7fc3141986d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00d950 tx=0x7fc2fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.416+0000 7fc30b7fe640 1 -- 172.21.15.82:0/3112555204 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc017030 con 0x7fc314101570 2024-04-21T08:56:04.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.416+0000 7fc30b7fe640 1 -- 172.21.15.82:0/3112555204 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2fc004020 con 0x7fc314101570 2024-04-21T08:56:04.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.416+0000 7fc318806640 1 -- 172.21.15.82:0/3112555204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31419d9b0 con 0x7fc314101570 2024-04-21T08:56:04.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.416+0000 7fc30b7fe640 1 -- 172.21.15.82:0/3112555204 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc0047a0 con 0x7fc314101570 2024-04-21T08:56:04.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.416+0000 7fc318806640 1 -- 172.21.15.82:0/3112555204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3141a5240 con 0x7fc314101570 2024-04-21T08:56:04.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.418+0000 7fc30b7fe640 1 -- 172.21.15.82:0/3112555204 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc2fc009a30 con 0x7fc314101570 2024-04-21T08:56:04.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.418+0000 7fc31b3f6740 1 Processor -- start 2024-04-21T08:56:04.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31b3f6740 1 -- start start 2024-04-21T08:56:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31b3f6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b644d0 0x562c72b648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31b3f6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 0x562c72b6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31b3f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b6fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31b3f6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562c72ac91f0 con 0x562c72b64e80 2024-04-21T08:56:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31b3f6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562c72a664b0 con 0x562c72b644d0 2024-04-21T08:56:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31b3f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562c72ac51b0 con 0x562c72b6da00 2024-04-21T08:56:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 0x562c72b6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc30b7fe640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc3000a3d20 0x7fc3000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 0x562c72b6d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38694/0 (socket says 172.21.15.82:38694) 2024-04-21T08:56:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31259c640 1 -- 172.21.15.82:0/1000899657 learned_addr learned my addr 172.21.15.82:0/1000899657 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc312d9d640 1 --2- 172.21.15.82:0/1000899657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b644d0 0x562c72b648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc311d9b640 1 --2- 172.21.15.82:0/1000899657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b6fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc312d9d640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc3000a3d20 0x7fc3000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc30b7fe640 1 -- 172.21.15.82:0/3112555204 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fc2fc0d0720 con 0x7fc314101570 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31259c640 1 -- 172.21.15.82:0/1000899657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 msgr2=0x562c72b6fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31259c640 1 --2- 172.21.15.82:0/1000899657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b6fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31259c640 1 -- 172.21.15.82:0/1000899657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b644d0 msgr2=0x562c72b648b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31259c640 1 --2- 172.21.15.82:0/1000899657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b644d0 0x562c72b648b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc31259c640 1 -- 172.21.15.82:0/1000899657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c72a581b0 con 0x562c72b64e80 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.419+0000 7fc311d9b640 1 --2- 172.21.15.82:0/1000899657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b6fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.420+0000 7fc31259c640 1 --2- 172.21.15.82:0/1000899657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 0x562c72b6d430 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0cda00 tx=0x7fc2fc004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3e8ead371ad5f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.420+0000 7fc312d9d640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc3000a3d20 0x7fc3000a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc30c005c40 tx=0x7fc30c009500 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.420+0000 7fc308ff9640 1 -- 172.21.15.82:0/1000899657 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc09c9b0 con 0x562c72b64e80 2024-04-21T08:56:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.420+0000 7fc308ff9640 1 -- 172.21.15.82:0/1000899657 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2fc0a4f20 con 0x562c72b64e80 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.420+0000 7fc308ff9640 1 -- 172.21.15.82:0/1000899657 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc098920 con 0x562c72b64e80 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.420+0000 7fc31b3f6740 1 -- 172.21.15.82:0/1000899657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 msgr2=0x562c72b6d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.420+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/1000899657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 0x562c72b6d430 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0cda00 tx=0x7fc2fc004580 comp rx=0 tx=0).stop 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 -- 172.21.15.82:0/1000899657 shutdown_connections 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/1000899657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b6fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/1000899657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 0x562c72b6d430 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/1000899657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b644d0 0x562c72b648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 -- 172.21.15.82:0/1000899657 >> 172.21.15.82:0/1000899657 conn(0x562c72b5eba0 msgr2=0x562c72b5ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 -- 172.21.15.82:0/1000899657 shutdown_connections 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 -- 172.21.15.82:0/1000899657 wait complete. 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 -- start start 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 Processor -- start 2024-04-21T08:56:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b644d0 0x562c72b6d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b64e80 0x562c72b6a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b6da00 0x562c72b6acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562c72b70910 con 0x562c72b6da00 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562c72a82500 con 0x562c72b64e80 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562c72b70a90 con 0x562c72b644d0 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31b3f6740 10 client.0 _ll_register_callbacks cb 0x562c72a50b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc312d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b6da00 0x562c72b6acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc312d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b6da00 0x562c72b6acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38708/0 (socket says 172.21.15.82:38708) 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc312d9d640 1 -- 172.21.15.82:0/4273652057 learned_addr learned my addr 172.21.15.82:0/4273652057 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.421+0000 7fc31259c640 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b644d0 0x562c72b6d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc311d9b640 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b64e80 0x562c72b6a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc312d9d640 1 -- 172.21.15.82:0/4273652057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b644d0 msgr2=0x562c72b6d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc312d9d640 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b644d0 0x562c72b6d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc312d9d640 1 -- 172.21.15.82:0/4273652057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b64e80 msgr2=0x562c72b6a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc312d9d640 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b64e80 0x562c72b6a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc312d9d640 1 -- 172.21.15.82:0/4273652057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c727db3f0 con 0x562c72b6da00 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc31259c640 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b644d0 0x562c72b6d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc312d9d640 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b6da00 0x562c72b6acd0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00dec0 tx=0x7fc30c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ad5b3879cba650 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc2f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc2f9ffb640 1 -- 172.21.15.82:0/4273652057 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c0047c0 con 0x562c72b6da00 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc2f9ffb640 1 -- 172.21.15.82:0/4273652057 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fc30c012050 con 0x562c72b6da00 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc31b3f6740 1 -- 172.21.15.82:0/4273652057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2fc0a4f20 con 0x562c72b6da00 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc2f9ffb640 1 -- 172.21.15.82:0/4273652057 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc30c012250 con 0x562c72b6da00 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc31b3f6740 10 client.18177 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc31b3f6740 1 -- 172.21.15.82:0/4273652057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562c728a57d0 con 0x562c72b6da00 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc2f9ffb640 1 -- 172.21.15.82:0/4273652057 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2fc0a4f20 con 0x562c72b6da00 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc2f9ffb640 1 -- 172.21.15.82:0/4273652057 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fc30c011440 con 0x562c72b6da00 2024-04-21T08:56:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc31b3f6740 10 client.18177 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc31b3f6740 10 client.18177 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.422+0000 7fc31b3f6740 20 client.18177 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.423+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562c72b7dc00 0x562c72b80080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.423+0000 7fc31b3f6740 4 client.18177 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.423+0000 7fc31b3f6740 1 -- 172.21.15.82:0/4273652057 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c72a68930 con 0x562c72b7dc00 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.423+0000 7fc31259c640 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562c72b7dc00 0x562c72b80080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.423+0000 7fc31259c640 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562c72b7dc00 0x562c72b80080 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=51c730761161be97 server_cookie=857f454c37424072 in_seq=0 out_seq=0 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.423+0000 7fc2f9ffb640 10 client.18177 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc2f9ffb640 1 -- 172.21.15.82:0/4273652057 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c72a68930 con 0x562c72b7dc00 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc2f9ffb640 10 client.18177 handle_command_reply: tid=0 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc31b3f6740 1 client.18177 shutdown 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc31b3f6740 1 -- 172.21.15.82:0/4273652057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b6da00 msgr2=0x562c72b6acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b6da00 0x562c72b6acd0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00dec0 tx=0x7fc30c004a40 comp rx=0 tx=0).stop 2024-04-21T08:56:04.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc31b3f6740 1 -- 172.21.15.82:0/4273652057 shutdown_connections 2024-04-21T08:56:04.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562c72b7dc00 0x562c72b80080 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b6da00 0x562c72b6acd0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc2f9ffb640 0 client.18177 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:56:04.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b64e80 0x562c72b6a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/4273652057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b644d0 0x562c72b6d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc31b3f6740 1 -- 172.21.15.82:0/4273652057 >> 172.21.15.82:0/4273652057 conn(0x562c72b5eba0 msgr2=0x562c72b608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc31b3f6740 1 -- 172.21.15.82:0/4273652057 shutdown_connections 2024-04-21T08:56:04.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc31b3f6740 1 -- 172.21.15.82:0/4273652057 wait complete. 2024-04-21T08:56:04.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.424+0000 7fc31b3f6740 20 client.18177 trim_cache size 0 max 16384 2024-04-21T08:56:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.437+0000 7fc31b3f6740 1 Processor -- start 2024-04-21T08:56:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.437+0000 7fc31b3f6740 1 -- start start 2024-04-21T08:56:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc31b3f6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b644d0 0x562c72b79800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc31b3f6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 0x562c72b69710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc31b3f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b6cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc31b3f6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141af540 con 0x562c72b64e80 2024-04-21T08:56:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc31b3f6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141b1110 con 0x562c72b644d0 2024-04-21T08:56:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc31b3f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141afc10 con 0x562c72b6da00 2024-04-21T08:56:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc311d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 0x562c72b69710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc311d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 0x562c72b69710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38724/0 (socket says 172.21.15.82:38724) 2024-04-21T08:56:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc311d9b640 1 -- 172.21.15.82:0/2554891901 learned_addr learned my addr 172.21.15.82:0/2554891901 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc312d9d640 1 --2- 172.21.15.82:0/2554891901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b6cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc311d9b640 1 -- 172.21.15.82:0/2554891901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 msgr2=0x562c72b6cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc311d9b640 1 --2- 172.21.15.82:0/2554891901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b6cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc311d9b640 1 -- 172.21.15.82:0/2554891901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b644d0 msgr2=0x562c72b79800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc31259c640 1 --2- 172.21.15.82:0/2554891901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b644d0 0x562c72b79800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc311d9b640 1 --2- 172.21.15.82:0/2554891901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b644d0 0x562c72b79800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc311d9b640 1 -- 172.21.15.82:0/2554891901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c72b6b040 con 0x562c72b64e80 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc312d9d640 1 --2- 172.21.15.82:0/2554891901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b6cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc311d9b640 1 --2- 172.21.15.82:0/2554891901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 0x562c72b69710 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc30402c720 tx=0x7fc304005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61a0d605863bbe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc2fbfff640 1 -- 172.21.15.82:0/2554891901 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30402ca00 con 0x562c72b64e80 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc2fbfff640 1 -- 172.21.15.82:0/2554891901 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30402cba0 con 0x562c72b64e80 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.438+0000 7fc2fbfff640 1 -- 172.21.15.82:0/2554891901 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304004920 con 0x562c72b64e80 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 -- 172.21.15.82:0/2554891901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 msgr2=0x562c72b69710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/2554891901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 0x562c72b69710 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc30402c720 tx=0x7fc304005d50 comp rx=0 tx=0).stop 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 -- 172.21.15.82:0/2554891901 shutdown_connections 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/2554891901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b6cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/2554891901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b64e80 0x562c72b69710 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/2554891901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b644d0 0x562c72b79800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 -- 172.21.15.82:0/2554891901 >> 172.21.15.82:0/2554891901 conn(0x562c72b5eba0 msgr2=0x562c72b60890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 -- 172.21.15.82:0/2554891901 shutdown_connections 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 -- 172.21.15.82:0/2554891901 wait complete. 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 -- start start 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 Processor -- start 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b644d0 0x562c72b75d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.439+0000 7fc31b3f6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b64e80 0x562c72b76250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31b3f6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b66950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31b3f6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141b0a10 con 0x562c72b644d0 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31b3f6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141b0310 con 0x562c72b64e80 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31b3f6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3141ad510 con 0x562c72b6da00 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31b3f6740 10 client.0 _ll_register_callbacks cb 0x7fc30c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b644d0 0x562c72b75d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b644d0 0x562c72b75d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38738/0 (socket says 172.21.15.82:38738) 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31259c640 1 -- 172.21.15.82:0/3258538637 learned_addr learned my addr 172.21.15.82:0/3258538637 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc311d9b640 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b64e80 0x562c72b76250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31259c640 1 -- 172.21.15.82:0/3258538637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 msgr2=0x562c72b66950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31259c640 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b66950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31259c640 1 -- 172.21.15.82:0/3258538637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b64e80 msgr2=0x562c72b76250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31259c640 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b64e80 0x562c72b76250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31259c640 1 -- 172.21.15.82:0/3258538637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c72b66f60 con 0x562c72b644d0 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc311d9b640 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b64e80 0x562c72b76250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31259c640 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b644d0 0x562c72b75d10 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005fd0 tx=0x7fc2fc00d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732785f682ff324 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc2f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc2f9ffb640 1 -- 172.21.15.82:0/3258538637 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc09c9b0 con 0x562c72b644d0 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc2f9ffb640 1 -- 172.21.15.82:0/3258538637 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fc2fc009d20 con 0x562c72b644d0 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31b3f6740 1 -- 172.21.15.82:0/3258538637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc30402cba0 con 0x562c72b644d0 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc2f9ffb640 1 -- 172.21.15.82:0/3258538637 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc0a4f20 con 0x562c72b644d0 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31b3f6740 10 client.18183 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc31b3f6740 1 -- 172.21.15.82:0/3258538637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562c72be3a00 con 0x562c72b644d0 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.440+0000 7fc2f9ffb640 1 -- 172.21.15.82:0/3258538637 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30402cba0 con 0x562c72b644d0 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc2f9ffb640 1 -- 172.21.15.82:0/3258538637 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fc2fc081be0 con 0x562c72b644d0 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc31b3f6740 10 client.18183 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc31b3f6740 10 client.18183 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc31b3f6740 20 client.18183 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562c72be3db0 0x562c72be61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc31b3f6740 4 client.18183 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:56:04.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc31b3f6740 1 -- 172.21.15.82:0/3258538637 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c72ad6c10 con 0x562c72be3db0 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc311d9b640 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562c72be3db0 0x562c72be61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc2f9ffb640 10 client.18183 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc311d9b640 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562c72be3db0 0x562c72be61a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de81c7083a6056af server_cookie=ae83fdc87d07b5a8 in_seq=0 out_seq=0 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc2f9ffb640 1 -- 172.21.15.82:0/3258538637 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c72ad6c10 con 0x562c72be3db0 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc2f9ffb640 10 client.18183 handle_command_reply: tid=0 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.441+0000 7fc31b3f6740 1 client.18183 shutdown 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc31b3f6740 1 -- 172.21.15.82:0/3258538637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b644d0 msgr2=0x562c72b75d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b644d0 0x562c72b75d10 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005fd0 tx=0x7fc2fc00d8d0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc31b3f6740 1 -- 172.21.15.82:0/3258538637 shutdown_connections 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562c72be3db0 0x562c72be61a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562c72b6da00 0x562c72b66950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c72b64e80 0x562c72b76250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc31b3f6740 1 --2- 172.21.15.82:0/3258538637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c72b644d0 0x562c72b75d10 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc2f9ffb640 0 client.18183 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:56:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc31b3f6740 1 -- 172.21.15.82:0/3258538637 >> 172.21.15.82:0/3258538637 conn(0x562c72b5eba0 msgr2=0x562c72b69710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc31b3f6740 1 -- 172.21.15.82:0/3258538637 shutdown_connections 2024-04-21T08:56:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc31b3f6740 1 -- 172.21.15.82:0/3258538637 wait complete. 2024-04-21T08:56:04.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc31b3f6740 20 client.18183 trim_cache size 0 max 16384 2024-04-21T08:56:04.451 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:56:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc318806640 1 -- 172.21.15.82:0/3112555204 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc3000a3d20 msgr2=0x7fc3000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc318806640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc3000a3d20 0x7fc3000a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc30c005c40 tx=0x7fc30c009500 comp rx=0 tx=0).stop 2024-04-21T08:56:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc318806640 1 -- 172.21.15.82:0/3112555204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc314101570 msgr2=0x7fc3141986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc318806640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc314101570 0x7fc3141986d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00d950 tx=0x7fc2fc00de10 comp rx=0 tx=0).stop 2024-04-21T08:56:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc318806640 1 -- 172.21.15.82:0/3112555204 shutdown_connections 2024-04-21T08:56:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc318806640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc314105e50 0x7fc31419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc318806640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc314101f30 0x7fc314198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc318806640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fc3000a3d20 0x7fc3000a61e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc318806640 1 --2- 172.21.15.82:0/3112555204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc314101570 0x7fc3141986d0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc318806640 1 -- 172.21.15.82:0/3112555204 >> 172.21.15.82:0/3112555204 conn(0x7fc3140773f0 msgr2=0x7fc314100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc318806640 1 -- 172.21.15.82:0/3112555204 shutdown_connections 2024-04-21T08:56:04.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.442+0000 7fc318806640 1 -- 172.21.15.82:0/3112555204 wait complete. 2024-04-21T08:56:04.454 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:56:04.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:04 smithi082 ceph-mon[27967]: pgmap v278: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-04-21T08:56:04.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:04 smithi134 ceph-mon[31772]: pgmap v278: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-04-21T08:56:04.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.553+0000 7f0622b15640 1 -- 172.21.15.82:0/2072075418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061c1072b0 msgr2=0x7f061c10ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.553+0000 7f0622b15640 1 --2- 172.21.15.82:0/2072075418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061c1072b0 0x7f061c10ee80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0614009ca0 tx=0x7f061402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.553+0000 7f0622b15640 1 -- 172.21.15.82:0/2072075418 shutdown_connections 2024-04-21T08:56:04.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.553+0000 7f0622b15640 1 --2- 172.21.15.82:0/2072075418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061c1072b0 0x7f061c10ee80 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.553+0000 7f0622b15640 1 --2- 172.21.15.82:0/2072075418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f061c102b90 0x7f061c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.553+0000 7f0622b15640 1 --2- 172.21.15.82:0/2072075418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f061c1021e0 0x7f061c1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.553+0000 7f0622b15640 1 -- 172.21.15.82:0/2072075418 >> 172.21.15.82:0/2072075418 conn(0x7f061c0fda60 msgr2=0x7f061c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.553+0000 7f0622b15640 1 -- 172.21.15.82:0/2072075418 shutdown_connections 2024-04-21T08:56:04.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.553+0000 7f0622b15640 1 -- 172.21.15.82:0/2072075418 wait complete. 2024-04-21T08:56:04.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f0622b15640 1 Processor -- start 2024-04-21T08:56:04.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f0622b15640 1 -- start start 2024-04-21T08:56:04.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f0622b15640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f061c1021e0 0x7f061c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f0622b15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f061c1072b0 0x7f061c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f0622b15640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061c19cef0 0x7f061c19d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f0622b15640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f061c112200 con 0x7f061c19cef0 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f0622b15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f061c112080 con 0x7f061c1072b0 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f0622b15640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f061c112380 con 0x7f061c1021e0 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f06210b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061c19cef0 0x7f061c19d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f06210b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061c19cef0 0x7f061c19d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38762/0 (socket says 172.21.15.82:38762) 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f06210b2640 1 -- 172.21.15.82:0/1065620730 learned_addr learned my addr 172.21.15.82:0/1065620730 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f061bfff640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f061c1072b0 0x7f061c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f06208b1640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f061c1021e0 0x7f061c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f06210b2640 1 -- 172.21.15.82:0/1065620730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f061c1021e0 msgr2=0x7f061c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f06210b2640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f061c1021e0 0x7f061c198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.554+0000 7f06210b2640 1 -- 172.21.15.82:0/1065620730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f061c1072b0 msgr2=0x7f061c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.555+0000 7f06210b2640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f061c1072b0 0x7f061c198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.555+0000 7f06210b2640 1 -- 172.21.15.82:0/1065620730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061c19d9e0 con 0x7f061c19cef0 2024-04-21T08:56:04.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.555+0000 7f061bfff640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f061c1072b0 0x7f061c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:04.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.555+0000 7f06210b2640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061c19cef0 0x7f061c19d3a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f06140098d0 tx=0x7f0614035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.555+0000 7f0619ffb640 1 -- 172.21.15.82:0/1065620730 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614039040 con 0x7f061c19cef0 2024-04-21T08:56:04.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.555+0000 7f0619ffb640 1 -- 172.21.15.82:0/1065620730 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0614044070 con 0x7f061c19cef0 2024-04-21T08:56:04.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.555+0000 7f0622b15640 1 -- 172.21.15.82:0/1065620730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f061c19dc70 con 0x7f061c19cef0 2024-04-21T08:56:04.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.555+0000 7f0619ffb640 1 -- 172.21.15.82:0/1065620730 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06140042a0 con 0x7f061c19cef0 2024-04-21T08:56:04.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.555+0000 7f0622b15640 1 -- 172.21.15.82:0/1065620730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f061c1a5490 con 0x7f061c19cef0 2024-04-21T08:56:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.557+0000 7f0619ffb640 1 -- 172.21.15.82:0/1065620730 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0614004440 con 0x7f061c19cef0 2024-04-21T08:56:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f06257e4740 1 Processor -- start 2024-04-21T08:56:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f06257e4740 1 -- start start 2024-04-21T08:56:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f06257e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb904d0 0x561c8bb908b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f06257e4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb90e80 0x561c8bb99430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f06257e4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bb9be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f06257e4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561c8baf51f0 con 0x561c8bb99a00 2024-04-21T08:56:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f06257e4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561c8ba924b0 con 0x561c8bb90e80 2024-04-21T08:56:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f06257e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c8baf11b0 con 0x561c8bb904d0 2024-04-21T08:56:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f0619ffb640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f06040a3da0 0x7f06040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f061bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bb9be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f061bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bb9be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38776/0 (socket says 172.21.15.82:38776) 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f061bfff640 1 -- 172.21.15.82:0/2653824753 learned_addr learned my addr 172.21.15.82:0/2653824753 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f06210b2640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f06040a3da0 0x7f06040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f061bfff640 1 --2- 172.21.15.82:0/2653824753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb904d0 0x561c8bb908b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f06208b1640 1 --2- 172.21.15.82:0/2653824753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb90e80 0x561c8bb99430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f0619ffb640 1 -- 172.21.15.82:0/1065620730 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f06140f3100 con 0x7f061c19cef0 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f061bfff640 1 -- 172.21.15.82:0/2653824753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb904d0 msgr2=0x561c8bb908b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f061bfff640 1 --2- 172.21.15.82:0/2653824753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb904d0 0x561c8bb908b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f061bfff640 1 -- 172.21.15.82:0/2653824753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb90e80 msgr2=0x561c8bb99430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f061bfff640 1 --2- 172.21.15.82:0/2653824753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb90e80 0x561c8bb99430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.558+0000 7f061bfff640 1 -- 172.21.15.82:0/2653824753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c8ba841b0 con 0x561c8bb99a00 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.559+0000 7f061bfff640 1 --2- 172.21.15.82:0/2653824753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bb9be50 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f0610012800 tx=0x7f061000d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d74fa572010d39 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.559+0000 7f06210b2640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f06040a3da0 0x7f06040a6260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f06140becd0 tx=0x7f0614032ab0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.559+0000 7f0602ffd640 1 -- 172.21.15.82:0/2653824753 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610005220 con 0x561c8bb99a00 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.559+0000 7f0602ffd640 1 -- 172.21.15.82:0/2653824753 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0610004c50 con 0x561c8bb99a00 2024-04-21T08:56:04.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.559+0000 7f0602ffd640 1 -- 172.21.15.82:0/2653824753 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061000a990 con 0x561c8bb99a00 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.559+0000 7f06257e4740 1 -- 172.21.15.82:0/2653824753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 msgr2=0x561c8bb9be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.559+0000 7f06257e4740 1 --2- 172.21.15.82:0/2653824753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bb9be50 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f0610012800 tx=0x7f061000d3a0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 -- 172.21.15.82:0/2653824753 shutdown_connections 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 --2- 172.21.15.82:0/2653824753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bb9be50 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 --2- 172.21.15.82:0/2653824753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb90e80 0x561c8bb99430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 --2- 172.21.15.82:0/2653824753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb904d0 0x561c8bb908b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 -- 172.21.15.82:0/2653824753 >> 172.21.15.82:0/2653824753 conn(0x561c8bb8aba0 msgr2=0x561c8bb8af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 -- 172.21.15.82:0/2653824753 shutdown_connections 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 -- 172.21.15.82:0/2653824753 wait complete. 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 -- start start 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 Processor -- start 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb904d0 0x561c8bb992f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 0x561c8bb96680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb99a00 0x561c8bb96c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561c8bba0d80 con 0x561c8bb904d0 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561c8baae500 con 0x561c8bb99a00 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561c8bba0f00 con 0x561c8bb90e80 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f06257e4740 10 client.0 _ll_register_callbacks cb 0x561c8ba7cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f061bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb904d0 0x561c8bb992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f061bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb904d0 0x561c8bb992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38792/0 (socket says 172.21.15.82:38792) 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.560+0000 7f061bfff640 1 -- 172.21.15.82:0/13072892 learned_addr learned my addr 172.21.15.82:0/13072892 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f061bfff640 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb99a00 0x561c8bb96c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f06208b1640 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 0x561c8bb96680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f061bfff640 1 -- 172.21.15.82:0/13072892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 msgr2=0x561c8bb96680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f061bfff640 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 0x561c8bb96680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f061bfff640 1 -- 172.21.15.82:0/13072892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb99a00 msgr2=0x561c8bb96c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f061bfff640 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb99a00 0x561c8bb96c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f061bfff640 1 -- 172.21.15.82:0/13072892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c8b8073f0 con 0x561c8bb904d0 2024-04-21T08:56:04.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f06208b1640 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 0x561c8bb96680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f061bfff640 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb904d0 0x561c8bb992f0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f061000d370 tx=0x7f0610005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c085b86615cdc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f05fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f05fbfff640 1 -- 172.21.15.82:0/13072892 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610012240 con 0x561c8bb904d0 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f05fbfff640 1 -- 172.21.15.82:0/13072892 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f06100123e0 con 0x561c8bb904d0 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f06257e4740 1 -- 172.21.15.82:0/13072892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0610004c50 con 0x561c8bb904d0 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f05fbfff640 1 -- 172.21.15.82:0/13072892 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0610032050 con 0x561c8bb904d0 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f06257e4740 10 client.18195 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f06257e4740 1 -- 172.21.15.82:0/13072892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561c8b8d17d0 con 0x561c8bb904d0 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.561+0000 7f05fbfff640 1 -- 172.21.15.82:0/13072892 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0610004c50 con 0x561c8bb904d0 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.562+0000 7f05fbfff640 1 -- 172.21.15.82:0/13072892 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f061000ab90 con 0x561c8bb904d0 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.562+0000 7f06257e4740 10 client.18195 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.562+0000 7f06257e4740 10 client.18195 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.562+0000 7f06257e4740 20 client.18195 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.562+0000 7f06257e4740 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561c8bbadec0 0x561c8bbb02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.562+0000 7f06257e4740 4 client.18195 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.562+0000 7f06257e4740 1 -- 172.21.15.82:0/13072892 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c8ba94930 con 0x561c8bbadec0 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.562+0000 7f06208b1640 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561c8bbadec0 0x561c8bbb02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.562+0000 7f06208b1640 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561c8bbadec0 0x561c8bbb02b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=416b73a2e53cde2c server_cookie=6291483855f53589 in_seq=0 out_seq=0 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.562+0000 7f05fbfff640 10 client.18195 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f05fbfff640 1 -- 172.21.15.82:0/13072892 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c8ba94930 con 0x561c8bbadec0 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f05fbfff640 10 client.18195 handle_command_reply: tid=0 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f06257e4740 1 client.18195 shutdown 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f06257e4740 1 -- 172.21.15.82:0/13072892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb904d0 msgr2=0x561c8bb992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f06257e4740 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb904d0 0x561c8bb992f0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f061000d370 tx=0x7f0610005850 comp rx=0 tx=0).stop 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f06257e4740 1 -- 172.21.15.82:0/13072892 shutdown_connections 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f06257e4740 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561c8bbadec0 0x561c8bbb02b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f06257e4740 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb99a00 0x561c8bb96c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f06257e4740 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 0x561c8bb96680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f05fbfff640 0 client.18195 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:56:04.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f06257e4740 1 --2- 172.21.15.82:0/13072892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb904d0 0x561c8bb992f0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f06257e4740 1 -- 172.21.15.82:0/13072892 >> 172.21.15.82:0/13072892 conn(0x561c8bb8aba0 msgr2=0x561c8bb8c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f06257e4740 1 -- 172.21.15.82:0/13072892 shutdown_connections 2024-04-21T08:56:04.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.563+0000 7f06257e4740 1 -- 172.21.15.82:0/13072892 wait complete. 2024-04-21T08:56:04.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.564+0000 7f06257e4740 20 client.18195 trim_cache size 0 max 16384 2024-04-21T08:56:04.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.576+0000 7f06257e4740 1 Processor -- start 2024-04-21T08:56:04.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f06257e4740 1 -- start start 2024-04-21T08:56:04.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f06257e4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb904d0 0x561c8bba9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f06257e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 0x561c8bb89c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f06257e4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bb98db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f06257e4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f061c1ad0c0 con 0x561c8bb99a00 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f06257e4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f061c1aec90 con 0x561c8bb904d0 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f06257e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f061c1ad790 con 0x561c8bb90e80 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f061bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bb98db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f061bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bb98db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38804/0 (socket says 172.21.15.82:38804) 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f061bfff640 1 -- 172.21.15.82:0/3200380222 learned_addr learned my addr 172.21.15.82:0/3200380222 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f061bfff640 1 -- 172.21.15.82:0/3200380222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 msgr2=0x561c8bb89c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f061bfff640 1 --2- 172.21.15.82:0/3200380222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 0x561c8bb89c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f061bfff640 1 -- 172.21.15.82:0/3200380222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb904d0 msgr2=0x561c8bba9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f061bfff640 1 --2- 172.21.15.82:0/3200380222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb904d0 0x561c8bba9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f061bfff640 1 -- 172.21.15.82:0/3200380222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c8bb85070 con 0x561c8bb99a00 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f061bfff640 1 --2- 172.21.15.82:0/3200380222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bb98db0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f0610005940 tx=0x7f061000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213827b3d68cf9c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f06027fc640 1 -- 172.21.15.82:0/3200380222 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610004c50 con 0x561c8bb99a00 2024-04-21T08:56:04.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f06027fc640 1 -- 172.21.15.82:0/3200380222 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061000d840 con 0x561c8bb99a00 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.577+0000 7f06027fc640 1 -- 172.21.15.82:0/3200380222 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061000a8f0 con 0x561c8bb99a00 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.578+0000 7f06257e4740 1 -- 172.21.15.82:0/3200380222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 msgr2=0x561c8bb98db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.578+0000 7f06257e4740 1 --2- 172.21.15.82:0/3200380222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bb98db0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f0610005940 tx=0x7f061000df80 comp rx=0 tx=0).stop 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.578+0000 7f06257e4740 1 -- 172.21.15.82:0/3200380222 shutdown_connections 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.578+0000 7f06257e4740 1 --2- 172.21.15.82:0/3200380222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bb98db0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.578+0000 7f06257e4740 1 --2- 172.21.15.82:0/3200380222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 0x561c8bb89c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.578+0000 7f06257e4740 1 --2- 172.21.15.82:0/3200380222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb904d0 0x561c8bba9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.578+0000 7f06257e4740 1 -- 172.21.15.82:0/3200380222 >> 172.21.15.82:0/3200380222 conn(0x561c8bb8aba0 msgr2=0x561c8bb956d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.578+0000 7f06257e4740 1 -- 172.21.15.82:0/3200380222 shutdown_connections 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.578+0000 7f06257e4740 1 -- 172.21.15.82:0/3200380222 wait complete. 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.578+0000 7f06257e4740 1 -- start start 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.578+0000 7f06257e4740 1 Processor -- start 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f06257e4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb904d0 0x561c8bc0c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f06257e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 0x561c8bc0ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f06257e4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bba5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f06257e4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f061c1ae590 con 0x561c8bb99a00 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f06257e4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f061c1ade90 con 0x561c8bb904d0 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f06257e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f061c1ab090 con 0x561c8bb90e80 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f06257e4740 10 client.0 _ll_register_callbacks cb 0x7f0610004680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f061bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bba5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f061bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bba5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38818/0 (socket says 172.21.15.82:38818) 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f061bfff640 1 -- 172.21.15.82:0/1650342649 learned_addr learned my addr 172.21.15.82:0/1650342649 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f061bfff640 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb904d0 0x561c8bc0c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f06208b1640 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 0x561c8bc0ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f061bfff640 1 -- 172.21.15.82:0/1650342649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 msgr2=0x561c8bc0ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f061bfff640 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 0x561c8bc0ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f061bfff640 1 -- 172.21.15.82:0/1650342649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb904d0 msgr2=0x561c8bc0c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f061bfff640 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb904d0 0x561c8bc0c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f061bfff640 1 -- 172.21.15.82:0/1650342649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c8bba61a0 con 0x561c8bb99a00 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f061bfff640 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bba5b30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x561c8bbaa1d0 tx=0x7f06100055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ab1b9d3e98f073 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f05fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f05fbfff640 1 -- 172.21.15.82:0/1650342649 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06100051a0 con 0x561c8bb99a00 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f05fbfff640 1 -- 172.21.15.82:0/1650342649 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f0610005340 con 0x561c8bb99a00 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f06257e4740 1 -- 172.21.15.82:0/1650342649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f061000d840 con 0x561c8bb99a00 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f05fbfff640 1 -- 172.21.15.82:0/1650342649 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f061000a650 con 0x561c8bb99a00 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f06257e4740 10 client.18201 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.579+0000 7f06257e4740 1 -- 172.21.15.82:0/1650342649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561c8bc13df0 con 0x561c8bb99a00 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.580+0000 7f05fbfff640 1 -- 172.21.15.82:0/1650342649 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061000d840 con 0x561c8bb99a00 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.580+0000 7f05fbfff640 1 -- 172.21.15.82:0/1650342649 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f061001d700 con 0x561c8bb99a00 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.580+0000 7f06257e4740 10 client.18201 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.580+0000 7f06257e4740 10 client.18201 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.580+0000 7f06257e4740 20 client.18201 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.580+0000 7f06257e4740 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561c8bc14450 0x561c8bc16870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.580+0000 7f06257e4740 4 client.18201 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.580+0000 7f06257e4740 1 -- 172.21.15.82:0/1650342649 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c8bb02c10 con 0x561c8bc14450 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.580+0000 7f06208b1640 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561c8bc14450 0x561c8bc16870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.580+0000 7f06208b1640 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561c8bc14450 0x561c8bc16870 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f1cd831b8f82c29d server_cookie=5f96e925daef3fb in_seq=0 out_seq=0 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.580+0000 7f05fbfff640 10 client.18201 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f05fbfff640 1 -- 172.21.15.82:0/1650342649 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c8bb02c10 con 0x561c8bc14450 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f05fbfff640 10 client.18201 handle_command_reply: tid=0 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f06257e4740 1 client.18201 shutdown 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f06257e4740 1 -- 172.21.15.82:0/1650342649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 msgr2=0x561c8bba5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f06257e4740 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bba5b30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x561c8bbaa1d0 tx=0x7f06100055c0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f06257e4740 1 -- 172.21.15.82:0/1650342649 shutdown_connections 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f06257e4740 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x561c8bc14450 0x561c8bc16870 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f06257e4740 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c8bb99a00 0x561c8bba5b30 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f06257e4740 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c8bb90e80 0x561c8bc0ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f06257e4740 1 --2- 172.21.15.82:0/1650342649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561c8bb904d0 0x561c8bc0c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f05fbfff640 0 client.18201 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:56:04.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f06257e4740 1 -- 172.21.15.82:0/1650342649 >> 172.21.15.82:0/1650342649 conn(0x561c8bb8aba0 msgr2=0x561c8bb91b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f06257e4740 1 -- 172.21.15.82:0/1650342649 shutdown_connections 2024-04-21T08:56:04.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f06257e4740 1 -- 172.21.15.82:0/1650342649 wait complete. 2024-04-21T08:56:04.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f06257e4740 20 client.18201 trim_cache size 0 max 16384 2024-04-21T08:56:04.588 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:56:04.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f0622b15640 1 -- 172.21.15.82:0/1065620730 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f06040a3da0 msgr2=0x7f06040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f0622b15640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f06040a3da0 0x7f06040a6260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f06140becd0 tx=0x7f0614032ab0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f0622b15640 1 -- 172.21.15.82:0/1065620730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061c19cef0 msgr2=0x7f061c19d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f0622b15640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061c19cef0 0x7f061c19d3a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f06140098d0 tx=0x7f0614035d60 comp rx=0 tx=0).stop 2024-04-21T08:56:04.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.581+0000 7f0622b15640 1 -- 172.21.15.82:0/1065620730 shutdown_connections 2024-04-21T08:56:04.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.582+0000 7f0622b15640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f06040a3da0 0x7f06040a6260 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.582+0000 7f0622b15640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f061c19cef0 0x7f061c19d3a0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.582+0000 7f0622b15640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f061c1072b0 0x7f061c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.582+0000 7f0622b15640 1 --2- 172.21.15.82:0/1065620730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f061c1021e0 0x7f061c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.582+0000 7f0622b15640 1 -- 172.21.15.82:0/1065620730 >> 172.21.15.82:0/1065620730 conn(0x7f061c0fda60 msgr2=0x7f061c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.582+0000 7f0622b15640 1 -- 172.21.15.82:0/1065620730 shutdown_connections 2024-04-21T08:56:04.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.582+0000 7f0622b15640 1 -- 172.21.15.82:0/1065620730 wait complete. 2024-04-21T08:56:04.593 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:56:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:04 smithi106 ceph-mon[31072]: pgmap v278: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-04-21T08:56:04.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.733+0000 7f2b60881640 1 -- 172.21.15.82:0/2885830924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b5c1031d0 msgr2=0x7f2b5c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.733+0000 7f2b60881640 1 --2- 172.21.15.82:0/2885830924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b5c1031d0 0x7f2b5c1035b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009900 tx=0x7f2b5002bd60 comp rx=0 tx=0).stop 2024-04-21T08:56:04.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.733+0000 7f2b60881640 1 -- 172.21.15.82:0/2885830924 shutdown_connections 2024-04-21T08:56:04.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.733+0000 7f2b60881640 1 --2- 172.21.15.82:0/2885830924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b5c10c510 0x7f2b5c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.733+0000 7f2b60881640 1 --2- 172.21.15.82:0/2885830924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b5c103af0 0x7f2b5c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.733+0000 7f2b60881640 1 --2- 172.21.15.82:0/2885830924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b5c1031d0 0x7f2b5c1035b0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.733+0000 7f2b60881640 1 -- 172.21.15.82:0/2885830924 >> 172.21.15.82:0/2885830924 conn(0x7f2b5c077f50 msgr2=0x7f2b5c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.733+0000 7f2b60881640 1 -- 172.21.15.82:0/2885830924 shutdown_connections 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.733+0000 7f2b60881640 1 -- 172.21.15.82:0/2885830924 wait complete. 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.733+0000 7f2b60881640 1 Processor -- start 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.733+0000 7f2b60881640 1 -- start start 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b60881640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b5c1031d0 0x7f2b5c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b60881640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b5c103af0 0x7f2b5c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b60881640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b5c10c510 0x7f2b5c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b60881640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c112180 con 0x7f2b5c10c510 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b60881640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c112000 con 0x7f2b5c1031d0 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b60881640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c112300 con 0x7f2b5c103af0 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b5ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b5c10c510 0x7f2b5c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b5ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b5c10c510 0x7f2b5c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38840/0 (socket says 172.21.15.82:38840) 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b5ad9d640 1 -- 172.21.15.82:0/1852407549 learned_addr learned my addr 172.21.15.82:0/1852407549 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b5c1031d0 0x7f2b5c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b5c103af0 0x7f2b5c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b5ad9d640 1 -- 172.21.15.82:0/1852407549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b5c103af0 msgr2=0x7f2b5c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b5ad9d640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b5c103af0 0x7f2b5c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b5ad9d640 1 -- 172.21.15.82:0/1852407549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b5c1031d0 msgr2=0x7f2b5c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b5ad9d640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b5c1031d0 0x7f2b5c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b5ad9d640 1 -- 172.21.15.82:0/1852407549 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5c198d20 con 0x7f2b5c10c510 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b5c1031d0 0x7f2b5c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b5ad9d640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b5c10c510 0x7f2b5c1985a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400cad0 tx=0x7f2b5400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b4b7fe640 1 -- 172.21.15.82:0/1852407549 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400ae20 con 0x7f2b5c10c510 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b60881640 1 -- 172.21.15.82:0/1852407549 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5c199010 con 0x7f2b5c10c510 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b4b7fe640 1 -- 172.21.15.82:0/1852407549 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b54005020 con 0x7f2b5c10c510 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b4b7fe640 1 -- 172.21.15.82:0/1852407549 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400f6b0 con 0x7f2b5c10c510 2024-04-21T08:56:04.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.734+0000 7f2b60881640 1 -- 172.21.15.82:0/1852407549 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5c1a52b0 con 0x7f2b5c10c510 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.736+0000 7f2b4b7fe640 1 -- 172.21.15.82:0/1852407549 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2b540051d0 con 0x7f2b5c10c510 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b63474740 1 Processor -- start 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b63474740 1 -- start start 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b63474740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x5605012658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b63474740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x56050126e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b63474740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x560501270e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b63474740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5605011ca1f0 con 0x560501265e80 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b63474740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5605011674b0 con 0x5605012654d0 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b63474740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5605011c61b0 con 0x56050126ea00 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b4b7fe640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2b440a3d20 0x7f2b440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b5a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x56050126e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b5a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x56050126e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38842/0 (socket says 172.21.15.82:38842) 2024-04-21T08:56:04.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b5a59c640 1 -- 172.21.15.82:0/3885269264 learned_addr learned my addr 172.21.15.82:0/3885269264 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b5ad9d640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2b440a3d20 0x7f2b440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/3885269264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x560501270e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b4b7fe640 1 -- 172.21.15.82:0/1852407549 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f2b540d0d00 con 0x7f2b5c10c510 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.737+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/3885269264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x5605012658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.738+0000 7f2b5a59c640 1 -- 172.21.15.82:0/3885269264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 msgr2=0x560501270e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.738+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/3885269264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x560501270e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.738+0000 7f2b5a59c640 1 -- 172.21.15.82:0/3885269264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 msgr2=0x5605012658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.738+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/3885269264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x5605012658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.738+0000 7f2b5a59c640 1 -- 172.21.15.82:0/3885269264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605011591b0 con 0x560501265e80 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.738+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/3885269264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x560501270e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.738+0000 7f2b5ad9d640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2b440a3d20 0x7f2b440a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b540ce0f0 tx=0x7f2b5406b240 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.738+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/3885269264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x56050126e430 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2b50000c00 tx=0x7f2b50004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79eac4da60c3007 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.738+0000 7f2b48ff9640 1 -- 172.21.15.82:0/3885269264 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50004430 con 0x560501265e80 2024-04-21T08:56:04.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.738+0000 7f2b48ff9640 1 -- 172.21.15.82:0/3885269264 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5002cdc0 con 0x560501265e80 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.738+0000 7f2b48ff9640 1 -- 172.21.15.82:0/3885269264 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50004970 con 0x560501265e80 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 -- 172.21.15.82:0/3885269264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 msgr2=0x56050126e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 --2- 172.21.15.82:0/3885269264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x56050126e430 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2b50000c00 tx=0x7f2b50004040 comp rx=0 tx=0).stop 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 -- 172.21.15.82:0/3885269264 shutdown_connections 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 --2- 172.21.15.82:0/3885269264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x560501270e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 --2- 172.21.15.82:0/3885269264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x56050126e430 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 --2- 172.21.15.82:0/3885269264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x5605012658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 -- 172.21.15.82:0/3885269264 >> 172.21.15.82:0/3885269264 conn(0x56050125fba0 msgr2=0x56050125ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 -- 172.21.15.82:0/3885269264 shutdown_connections 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 -- 172.21.15.82:0/3885269264 wait complete. 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 -- start start 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 Processor -- start 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605012654d0 0x56050126e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560501265e80 0x56050126b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56050126ea00 0x56050126bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560501271910 con 0x56050126ea00 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560501183500 con 0x560501265e80 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560501271a90 con 0x5605012654d0 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b63474740 10 client.0 _ll_register_callbacks cb 0x560501151b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b59d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56050126ea00 0x56050126bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b59d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56050126ea00 0x56050126bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38850/0 (socket says 172.21.15.82:38850) 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.739+0000 7f2b59d9b640 1 -- 172.21.15.82:0/1658638305 learned_addr learned my addr 172.21.15.82:0/1658638305 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560501265e80 0x56050126b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605012654d0 0x56050126e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b59d9b640 1 -- 172.21.15.82:0/1658638305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605012654d0 msgr2=0x56050126e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605012654d0 0x56050126e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b59d9b640 1 -- 172.21.15.82:0/1658638305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560501265e80 msgr2=0x56050126b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560501265e80 0x56050126b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b59d9b640 1 -- 172.21.15.82:0/1658638305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560500edc3f0 con 0x56050126ea00 2024-04-21T08:56:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560501265e80 0x56050126b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56050126ea00 0x56050126bbd0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0058c0 tx=0x7f2b4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5247e746942348e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b41ffb640 1 -- 172.21.15.82:0/1658638305 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c013110 con 0x56050126ea00 2024-04-21T08:56:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b41ffb640 1 -- 172.21.15.82:0/1658638305 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f2b4c004020 con 0x56050126ea00 2024-04-21T08:56:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b63474740 1 -- 172.21.15.82:0/1658638305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5002cdc0 con 0x56050126ea00 2024-04-21T08:56:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b41ffb640 1 -- 172.21.15.82:0/1658638305 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2b4c025050 con 0x56050126ea00 2024-04-21T08:56:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b63474740 10 client.18213 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b63474740 1 -- 172.21.15.82:0/1658638305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560500fa67d0 con 0x56050126ea00 2024-04-21T08:56:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.740+0000 7f2b41ffb640 1 -- 172.21.15.82:0/1658638305 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5002cdc0 con 0x56050126ea00 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.741+0000 7f2b41ffb640 1 -- 172.21.15.82:0/1658638305 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f2b4c004830 con 0x56050126ea00 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.741+0000 7f2b63474740 10 client.18213 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.741+0000 7f2b63474740 10 client.18213 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.741+0000 7f2b63474740 20 client.18213 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.741+0000 7f2b63474740 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56050127ec00 0x560501281060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.741+0000 7f2b63474740 4 client.18213 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.741+0000 7f2b63474740 1 -- 172.21.15.82:0/1658638305 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560501169930 con 0x56050127ec00 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.741+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56050127ec00 0x560501281060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.741+0000 7f2b41ffb640 10 client.18213 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.741+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56050127ec00 0x560501281060 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a8aa7bbbfec2ae server_cookie=29a15d1b1b4a68ce in_seq=0 out_seq=0 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b41ffb640 1 -- 172.21.15.82:0/1658638305 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560501169930 con 0x56050127ec00 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b41ffb640 10 client.18213 handle_command_reply: tid=0 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b63474740 1 client.18213 shutdown 2024-04-21T08:56:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b63474740 1 -- 172.21.15.82:0/1658638305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56050126ea00 msgr2=0x56050126bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b63474740 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56050126ea00 0x56050126bbd0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0058c0 tx=0x7f2b4c0046c0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b63474740 1 -- 172.21.15.82:0/1658638305 shutdown_connections 2024-04-21T08:56:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b63474740 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x56050127ec00 0x560501281060 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b63474740 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56050126ea00 0x56050126bbd0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b63474740 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560501265e80 0x56050126b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b41ffb640 0 client.18213 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:56:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b63474740 1 --2- 172.21.15.82:0/1658638305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5605012654d0 0x56050126e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b63474740 1 -- 172.21.15.82:0/1658638305 >> 172.21.15.82:0/1658638305 conn(0x56050125fba0 msgr2=0x5605012618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b63474740 1 -- 172.21.15.82:0/1658638305 shutdown_connections 2024-04-21T08:56:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b63474740 1 -- 172.21.15.82:0/1658638305 wait complete. 2024-04-21T08:56:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.742+0000 7f2b63474740 20 client.18213 trim_cache size 0 max 16384 2024-04-21T08:56:04.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.755+0000 7f2b63474740 1 Processor -- start 2024-04-21T08:56:04.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.755+0000 7f2b63474740 1 -- start start 2024-04-21T08:56:04.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.755+0000 7f2b63474740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x56050127a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.755+0000 7f2b63474740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x56050126a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.755+0000 7f2b63474740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x56050125ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b63474740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1af5e0 con 0x560501265e80 2024-04-21T08:56:04.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b63474740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1b11b0 con 0x5605012654d0 2024-04-21T08:56:04.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b63474740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1afcb0 con 0x56050126ea00 2024-04-21T08:56:04.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b5a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x56050126a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b5a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x56050126a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38862/0 (socket says 172.21.15.82:38862) 2024-04-21T08:56:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b5a59c640 1 -- 172.21.15.82:0/116180782 learned_addr learned my addr 172.21.15.82:0/116180782 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/116180782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x56050127a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b5a59c640 1 -- 172.21.15.82:0/116180782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 msgr2=0x56050125ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/116180782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x56050125ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b5a59c640 1 -- 172.21.15.82:0/116180782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 msgr2=0x56050127a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/116180782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x56050127a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b5a59c640 1 -- 172.21.15.82:0/116180782 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560501265c20 con 0x560501265e80 2024-04-21T08:56:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/116180782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x56050127a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/116180782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x56050126a7d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x560501264fa0 tx=0x7f2b50005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c8bfd21033b7fa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b43fff640 1 -- 172.21.15.82:0/116180782 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b500041b0 con 0x560501265e80 2024-04-21T08:56:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b43fff640 1 -- 172.21.15.82:0/116180782 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b50004350 con 0x560501265e80 2024-04-21T08:56:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.756+0000 7f2b43fff640 1 -- 172.21.15.82:0/116180782 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50052080 con 0x560501265e80 2024-04-21T08:56:04.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 -- 172.21.15.82:0/116180782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 msgr2=0x56050126a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 --2- 172.21.15.82:0/116180782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x56050126a7d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x560501264fa0 tx=0x7f2b50005fd0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 -- 172.21.15.82:0/116180782 shutdown_connections 2024-04-21T08:56:04.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 --2- 172.21.15.82:0/116180782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x56050125ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 --2- 172.21.15.82:0/116180782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x56050126a7d0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 --2- 172.21.15.82:0/116180782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x56050127a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 -- 172.21.15.82:0/116180782 >> 172.21.15.82:0/116180782 conn(0x56050125fba0 msgr2=0x56050125e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 -- 172.21.15.82:0/116180782 shutdown_connections 2024-04-21T08:56:04.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 -- 172.21.15.82:0/116180782 wait complete. 2024-04-21T08:56:04.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 -- start start 2024-04-21T08:56:04.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 Processor -- start 2024-04-21T08:56:04.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x560501276e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x5605012722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x560501272830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1b0ab0 con 0x560501265e80 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1b03b0 con 0x5605012654d0 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1ad5b0 con 0x56050126ea00 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b63474740 10 client.0 _ll_register_callbacks cb 0x7f2b4c0138b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b5a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x5605012722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b5a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x5605012722f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38876/0 (socket says 172.21.15.82:38876) 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.757+0000 7f2b5a59c640 1 -- 172.21.15.82:0/1513482636 learned_addr learned my addr 172.21.15.82:0/1513482636 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x560501276e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x560501272830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b5a59c640 1 -- 172.21.15.82:0/1513482636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 msgr2=0x560501272830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x560501272830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b5a59c640 1 -- 172.21.15.82:0/1513482636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 msgr2=0x560501276e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x560501276e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b5a59c640 1 -- 172.21.15.82:0/1513482636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560501272da0 con 0x560501265e80 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x560501272830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b5a59c640 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x5605012722f0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f2b5002cb20 tx=0x7f2b50005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47498489ba13fbf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b41ffb640 1 -- 172.21.15.82:0/1513482636 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50052290 con 0x560501265e80 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b41ffb640 1 -- 172.21.15.82:0/1513482636 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f2b50052430 con 0x560501265e80 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b63474740 1 -- 172.21.15.82:0/1513482636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b50004350 con 0x560501265e80 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b41ffb640 1 -- 172.21.15.82:0/1513482636 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2b50063050 con 0x560501265e80 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b63474740 10 client.18219 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b63474740 1 -- 172.21.15.82:0/1513482636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5605012e4e90 con 0x560501265e80 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.758+0000 7f2b41ffb640 1 -- 172.21.15.82:0/1513482636 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b50004350 con 0x560501265e80 2024-04-21T08:56:04.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b41ffb640 1 -- 172.21.15.82:0/1513482636 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f2b50059030 con 0x560501265e80 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 10 client.18219 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 10 client.18219 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 20 client.18219 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5605012e5110 0x5605012e7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 4 client.18219 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 1 -- 172.21.15.82:0/1513482636 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605011d7c10 con 0x5605012e5110 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5605012e5110 0x5605012e7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b59d9b640 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5605012e5110 0x5605012e7500 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aed79038591f5a5e server_cookie=a2cf7f76be372ff2 in_seq=0 out_seq=0 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b41ffb640 10 client.18219 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b41ffb640 1 -- 172.21.15.82:0/1513482636 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605011d7c10 con 0x5605012e5110 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b41ffb640 10 client.18219 handle_command_reply: tid=0 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 1 client.18219 shutdown 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 1 -- 172.21.15.82:0/1513482636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 msgr2=0x5605012722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x5605012722f0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f2b5002cb20 tx=0x7f2b50005650 comp rx=0 tx=0).stop 2024-04-21T08:56:04.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 1 -- 172.21.15.82:0/1513482636 shutdown_connections 2024-04-21T08:56:04.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x5605012e5110 0x5605012e7500 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050126ea00 0x560501272830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b41ffb640 0 client.18219 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:56:04.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.759+0000 7f2b63474740 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560501265e80 0x5605012722f0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b63474740 1 --2- 172.21.15.82:0/1513482636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5605012654d0 0x560501276e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b63474740 1 -- 172.21.15.82:0/1513482636 >> 172.21.15.82:0/1513482636 conn(0x56050125fba0 msgr2=0x56050125e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b63474740 1 -- 172.21.15.82:0/1513482636 shutdown_connections 2024-04-21T08:56:04.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b63474740 1 -- 172.21.15.82:0/1513482636 wait complete. 2024-04-21T08:56:04.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b63474740 20 client.18219 trim_cache size 0 max 16384 2024-04-21T08:56:04.768 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:56:04.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b60881640 1 -- 172.21.15.82:0/1852407549 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2b440a3d20 msgr2=0x7f2b440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b60881640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2b440a3d20 0x7f2b440a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b540ce0f0 tx=0x7f2b5406b240 comp rx=0 tx=0).stop 2024-04-21T08:56:04.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b60881640 1 -- 172.21.15.82:0/1852407549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b5c10c510 msgr2=0x7f2b5c1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b60881640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b5c10c510 0x7f2b5c1985a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400cad0 tx=0x7f2b5400cf90 comp rx=0 tx=0).stop 2024-04-21T08:56:04.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b60881640 1 -- 172.21.15.82:0/1852407549 shutdown_connections 2024-04-21T08:56:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b60881640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2b440a3d20 0x7f2b440a61e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b60881640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b5c10c510 0x7f2b5c1985a0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b60881640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b5c103af0 0x7f2b5c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b60881640 1 --2- 172.21.15.82:0/1852407549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b5c1031d0 0x7f2b5c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b60881640 1 -- 172.21.15.82:0/1852407549 >> 172.21.15.82:0/1852407549 conn(0x7f2b5c077f50 msgr2=0x7f2b5c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b60881640 1 -- 172.21.15.82:0/1852407549 shutdown_connections 2024-04-21T08:56:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.760+0000 7f2b60881640 1 -- 172.21.15.82:0/1852407549 wait complete. 2024-04-21T08:56:04.772 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:56:04.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.871+0000 7fa772366640 1 -- 172.21.15.82:0/2976434585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76c107270 msgr2=0x7fa76c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.871+0000 7fa772366640 1 --2- 172.21.15.82:0/2976434585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76c107270 0x7fa76c10ee40 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fa764009ca0 tx=0x7fa76402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.871+0000 7fa772366640 1 -- 172.21.15.82:0/2976434585 shutdown_connections 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.871+0000 7fa772366640 1 --2- 172.21.15.82:0/2976434585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76c107270 0x7fa76c10ee40 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.871+0000 7fa772366640 1 --2- 172.21.15.82:0/2976434585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa76c06b1f0 0x7fa76c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.871+0000 7fa772366640 1 --2- 172.21.15.82:0/2976434585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa76c06a840 0x7fa76c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.871+0000 7fa772366640 1 -- 172.21.15.82:0/2976434585 >> 172.21.15.82:0/2976434585 conn(0x7fa76c077eb0 msgr2=0x7fa76c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.871+0000 7fa772366640 1 -- 172.21.15.82:0/2976434585 shutdown_connections 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.871+0000 7fa772366640 1 -- 172.21.15.82:0/2976434585 wait complete. 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa772366640 1 Processor -- start 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa772366640 1 -- start start 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa772366640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa76c06a840 0x7fa76c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa772366640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76c107270 0x7fa76c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa772366640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa76c19ceb0 0x7fa76c19d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa772366640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1121b0 con 0x7fa76c107270 2024-04-21T08:56:04.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa772366640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c112030 con 0x7fa76c19ceb0 2024-04-21T08:56:04.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa772366640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c112330 con 0x7fa76c06a840 2024-04-21T08:56:04.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa76b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76c107270 0x7fa76c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa76b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76c107270 0x7fa76c198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38908/0 (socket says 172.21.15.82:38908) 2024-04-21T08:56:04.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa76b7fe640 1 -- 172.21.15.82:0/2844447432 learned_addr learned my addr 172.21.15.82:0/2844447432 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa76b7fe640 1 -- 172.21.15.82:0/2844447432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa76c06a840 msgr2=0x7fa76c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa76c06a840 0x7fa76c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa76b7fe640 1 -- 172.21.15.82:0/2844447432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa76c19ceb0 msgr2=0x7fa76c19d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.872+0000 7fa770903640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa76c19ceb0 0x7fa76c19d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.873+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa76c19ceb0 0x7fa76c19d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.873+0000 7fa76b7fe640 1 -- 172.21.15.82:0/2844447432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c19d9a0 con 0x7fa76c107270 2024-04-21T08:56:04.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.873+0000 7fa770903640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa76c19ceb0 0x7fa76c19d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:04.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.873+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76c107270 0x7fa76c198b30 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa76000d770 tx=0x7fa76000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.873+0000 7fa7697fa640 1 -- 172.21.15.82:0/2844447432 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760017070 con 0x7fa76c107270 2024-04-21T08:56:04.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.873+0000 7fa7697fa640 1 -- 172.21.15.82:0/2844447432 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa760009c30 con 0x7fa76c107270 2024-04-21T08:56:04.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.873+0000 7fa772366640 1 -- 172.21.15.82:0/2844447432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76c19dc90 con 0x7fa76c107270 2024-04-21T08:56:04.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.873+0000 7fa7697fa640 1 -- 172.21.15.82:0/2844447432 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76000f620 con 0x7fa76c107270 2024-04-21T08:56:04.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.873+0000 7fa772366640 1 -- 172.21.15.82:0/2844447432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76c1a5480 con 0x7fa76c107270 2024-04-21T08:56:04.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.875+0000 7fa775031740 1 Processor -- start 2024-04-21T08:56:04.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.875+0000 7fa7697fa640 1 -- 172.21.15.82:0/2844447432 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa76001a070 con 0x7fa76c107270 2024-04-21T08:56:04.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.875+0000 7fa775031740 1 -- start start 2024-04-21T08:56:04.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.875+0000 7fa775031740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d84ef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa775031740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84efe80 0x5614d84f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa775031740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84f8a00 0x5614d84fae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa775031740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5614d84541f0 con 0x5614d84f8a00 2024-04-21T08:56:04.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa775031740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5614d83f14b0 con 0x5614d84ef4d0 2024-04-21T08:56:04.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa775031740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5614d84501b0 con 0x5614d84efe80 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa76b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84f8a00 0x5614d84fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa76b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84f8a00 0x5614d84fae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38920/0 (socket says 172.21.15.82:38920) 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa76b7fe640 1 -- 172.21.15.82:0/4217312448 learned_addr learned my addr 172.21.15.82:0/4217312448 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa770903640 1 --2- 172.21.15.82:0/4217312448 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d84ef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa76bfff640 1 --2- 172.21.15.82:0/4217312448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84efe80 0x5614d84f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa76b7fe640 1 -- 172.21.15.82:0/4217312448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84efe80 msgr2=0x5614d84f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa7697fa640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa7540a3d20 0x7fa7540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/4217312448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84efe80 0x5614d84f8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa76b7fe640 1 -- 172.21.15.82:0/4217312448 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 msgr2=0x5614d84ef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/4217312448 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d84ef8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa76b7fe640 1 -- 172.21.15.82:0/4217312448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614d83e31b0 con 0x5614d84f8a00 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa770903640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa7540a3d20 0x7fa7540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa7697fa640 1 -- 172.21.15.82:0/2844447432 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fa7600d07a0 con 0x7fa76c107270 2024-04-21T08:56:04.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/4217312448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84f8a00 0x5614d84fae50 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa7600cda50 tx=0x7fa760098400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce542d7db79fc2e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.876+0000 7fa7527fc640 1 -- 172.21.15.82:0/4217312448 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760021070 con 0x5614d84f8a00 2024-04-21T08:56:04.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.877+0000 7fa7527fc640 1 -- 172.21.15.82:0/4217312448 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa76009ca30 con 0x5614d84f8a00 2024-04-21T08:56:04.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.877+0000 7fa7527fc640 1 -- 172.21.15.82:0/4217312448 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76009bc90 con 0x5614d84f8a00 2024-04-21T08:56:04.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.877+0000 7fa775031740 1 -- 172.21.15.82:0/4217312448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84f8a00 msgr2=0x5614d84fae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.877+0000 7fa770903640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa7540a3d20 0x7fa7540a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa764009430 tx=0x7fa7640095c0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.877+0000 7fa775031740 1 --2- 172.21.15.82:0/4217312448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84f8a00 0x5614d84fae50 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa7600cda50 tx=0x7fa760098400 comp rx=0 tx=0).stop 2024-04-21T08:56:04.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.877+0000 7fa775031740 1 -- 172.21.15.82:0/4217312448 shutdown_connections 2024-04-21T08:56:04.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.877+0000 7fa775031740 1 --2- 172.21.15.82:0/4217312448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84f8a00 0x5614d84fae50 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.877+0000 7fa775031740 1 --2- 172.21.15.82:0/4217312448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84efe80 0x5614d84f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.877+0000 7fa775031740 1 --2- 172.21.15.82:0/4217312448 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d84ef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.877+0000 7fa775031740 1 -- 172.21.15.82:0/4217312448 >> 172.21.15.82:0/4217312448 conn(0x5614d84e9ba0 msgr2=0x5614d84e9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.877+0000 7fa775031740 1 -- 172.21.15.82:0/4217312448 shutdown_connections 2024-04-21T08:56:04.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.877+0000 7fa775031740 1 -- 172.21.15.82:0/4217312448 wait complete. 2024-04-21T08:56:04.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa775031740 1 -- start start 2024-04-21T08:56:04.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa775031740 1 Processor -- start 2024-04-21T08:56:04.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa775031740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84ef4d0 0x5614d84f8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa775031740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84efe80 0x5614d84f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa775031740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84f8a00 0x5614d84f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa775031740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5614d84fb970 con 0x5614d84ef4d0 2024-04-21T08:56:04.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa775031740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5614d840d500 con 0x5614d84f8a00 2024-04-21T08:56:04.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa775031740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5614d84fbaf0 con 0x5614d84efe80 2024-04-21T08:56:04.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa775031740 10 client.0 _ll_register_callbacks cb 0x5614d83dbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:04.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84ef4d0 0x5614d84f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84ef4d0 0x5614d84f8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38934/0 (socket says 172.21.15.82:38934) 2024-04-21T08:56:04.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa76bfff640 1 -- 172.21.15.82:0/3868366131 learned_addr learned my addr 172.21.15.82:0/3868366131 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/3868366131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84efe80 0x5614d84f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa76bfff640 1 -- 172.21.15.82:0/3868366131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84efe80 msgr2=0x5614d84f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa76bfff640 1 --2- 172.21.15.82:0/3868366131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84efe80 0x5614d84f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa76bfff640 1 -- 172.21.15.82:0/3868366131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84f8a00 msgr2=0x5614d84f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa76bfff640 1 --2- 172.21.15.82:0/3868366131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84f8a00 0x5614d84f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.878+0000 7fa76bfff640 1 -- 172.21.15.82:0/3868366131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614d81663f0 con 0x5614d84ef4d0 2024-04-21T08:56:04.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa76bfff640 1 --2- 172.21.15.82:0/3868366131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84ef4d0 0x5614d84f8320 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00f640 tx=0x7fa75c00fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19ac2bda8669a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa74b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:04.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa74b7fe640 1 -- 172.21.15.82:0/3868366131 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c01d060 con 0x5614d84ef4d0 2024-04-21T08:56:04.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa74b7fe640 1 -- 172.21.15.82:0/3868366131 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fa75c00da80 con 0x5614d84ef4d0 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa775031740 1 -- 172.21.15.82:0/3868366131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa76009ca30 con 0x5614d84ef4d0 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa74b7fe640 1 -- 172.21.15.82:0/3868366131 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa75c024050 con 0x5614d84ef4d0 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa775031740 10 client.18231 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa775031740 1 -- 172.21.15.82:0/3868366131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5614d82307d0 con 0x5614d84ef4d0 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa74b7fe640 1 -- 172.21.15.82:0/3868366131 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa76009ca30 con 0x5614d84ef4d0 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa74b7fe640 1 -- 172.21.15.82:0/3868366131 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fa75c016340 con 0x5614d84ef4d0 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa775031740 10 client.18231 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa775031740 10 client.18231 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa775031740 20 client.18231 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa775031740 1 --2- 172.21.15.82:0/3868366131 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5614d8508cc0 0x5614d850b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa775031740 4 client.18231 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa775031740 1 -- 172.21.15.82:0/3868366131 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614d83f3930 con 0x5614d8508cc0 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.879+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/3868366131 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5614d8508cc0 0x5614d850b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.880+0000 7fa74b7fe640 10 client.18231 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.880+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/3868366131 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5614d8508cc0 0x5614d850b180 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=881ef6b53134467d server_cookie=dc7202d5906bc679 in_seq=0 out_seq=0 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.880+0000 7fa74b7fe640 1 -- 172.21.15.82:0/3868366131 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614d83f3930 con 0x5614d8508cc0 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa74b7fe640 10 client.18231 handle_command_reply: tid=0 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa775031740 1 client.18231 shutdown 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa775031740 1 -- 172.21.15.82:0/3868366131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84ef4d0 msgr2=0x5614d84f8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa775031740 1 --2- 172.21.15.82:0/3868366131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84ef4d0 0x5614d84f8320 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00f640 tx=0x7fa75c00fb00 comp rx=0 tx=0).stop 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa775031740 1 -- 172.21.15.82:0/3868366131 shutdown_connections 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa775031740 1 --2- 172.21.15.82:0/3868366131 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5614d8508cc0 0x5614d850b180 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa775031740 1 --2- 172.21.15.82:0/3868366131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84f8a00 0x5614d84f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa74b7fe640 0 client.18231 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa775031740 1 --2- 172.21.15.82:0/3868366131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84efe80 0x5614d84f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa775031740 1 --2- 172.21.15.82:0/3868366131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84ef4d0 0x5614d84f8320 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa775031740 1 -- 172.21.15.82:0/3868366131 >> 172.21.15.82:0/3868366131 conn(0x5614d84e9ba0 msgr2=0x5614d84eb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa775031740 1 -- 172.21.15.82:0/3868366131 shutdown_connections 2024-04-21T08:56:04.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa775031740 1 -- 172.21.15.82:0/3868366131 wait complete. 2024-04-21T08:56:04.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.881+0000 7fa775031740 20 client.18231 trim_cache size 0 max 16384 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.894+0000 7fa775031740 1 Processor -- start 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.894+0000 7fa775031740 1 -- start start 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.894+0000 7fa775031740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d8504830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.894+0000 7fa775031740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 0x5614d84e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.894+0000 7fa775031740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84f8a00 0x5614d84f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.894+0000 7fa775031740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1af430 con 0x5614d84efe80 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.894+0000 7fa775031740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1b1000 con 0x5614d84ef4d0 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.894+0000 7fa775031740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1afb00 con 0x5614d84f8a00 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.894+0000 7fa76b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 0x5614d84e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.894+0000 7fa76b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 0x5614d84e8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38950/0 (socket says 172.21.15.82:38950) 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.894+0000 7fa76b7fe640 1 -- 172.21.15.82:0/963209295 learned_addr learned my addr 172.21.15.82:0/963209295 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa76bfff640 1 --2- 172.21.15.82:0/963209295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d8504830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa76b7fe640 1 -- 172.21.15.82:0/963209295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84f8a00 msgr2=0x5614d84f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/963209295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84f8a00 0x5614d84f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa76b7fe640 1 -- 172.21.15.82:0/963209295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 msgr2=0x5614d8504830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/963209295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d8504830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa76b7fe640 1 -- 172.21.15.82:0/963209295 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614d84f4660 con 0x5614d84efe80 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa76bfff640 1 --2- 172.21.15.82:0/963209295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d8504830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/963209295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 0x5614d84e8a80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa76009c030 tx=0x7fa76001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6aed393a4d09d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa751ffb640 1 -- 172.21.15.82:0/963209295 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760021040 con 0x5614d84efe80 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa751ffb640 1 -- 172.21.15.82:0/963209295 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa76009ca30 con 0x5614d84efe80 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa751ffb640 1 -- 172.21.15.82:0/963209295 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7600a4fa0 con 0x5614d84efe80 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa775031740 1 -- 172.21.15.82:0/963209295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 msgr2=0x5614d84e8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa775031740 1 --2- 172.21.15.82:0/963209295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 0x5614d84e8a80 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa76009c030 tx=0x7fa76001e040 comp rx=0 tx=0).stop 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa775031740 1 -- 172.21.15.82:0/963209295 shutdown_connections 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa775031740 1 --2- 172.21.15.82:0/963209295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84f8a00 0x5614d84f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa775031740 1 --2- 172.21.15.82:0/963209295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 0x5614d84e8a80 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa775031740 1 --2- 172.21.15.82:0/963209295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d8504830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa775031740 1 -- 172.21.15.82:0/963209295 >> 172.21.15.82:0/963209295 conn(0x5614d84e9ba0 msgr2=0x5614d84e8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.895+0000 7fa775031740 1 -- 172.21.15.82:0/963209295 shutdown_connections 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa775031740 1 -- 172.21.15.82:0/963209295 wait complete. 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa775031740 1 -- start start 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa775031740 1 Processor -- start 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa775031740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d84fc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa775031740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 0x5614d84fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa775031740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84f8a00 0x5614d84fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa775031740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1b0900 con 0x5614d84efe80 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa775031740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1b0200 con 0x5614d84ef4d0 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa775031740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1ad400 con 0x5614d84f8a00 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa775031740 10 client.0 _ll_register_callbacks cb 0x7fa75c00ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa76b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 0x5614d84fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa76b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 0x5614d84fcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38960/0 (socket says 172.21.15.82:38960) 2024-04-21T08:56:04.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.896+0000 7fa76b7fe640 1 -- 172.21.15.82:0/383358394 learned_addr learned my addr 172.21.15.82:0/383358394 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa76bfff640 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d84fc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa76b7fe640 1 -- 172.21.15.82:0/383358394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84f8a00 msgr2=0x5614d84fd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84f8a00 0x5614d84fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa76b7fe640 1 -- 172.21.15.82:0/383358394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 msgr2=0x5614d84fc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d84fc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa76b7fe640 1 -- 172.21.15.82:0/383358394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614d84efc20 con 0x5614d84efe80 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa76bfff640 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d84fc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa76b7fe640 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 0x5614d84fcd60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fa7600ce0e0 tx=0x7fa760010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2d3e9b76210e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa74b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa74b7fe640 1 -- 172.21.15.82:0/383358394 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7600b5060 con 0x5614d84efe80 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa74b7fe640 1 -- 172.21.15.82:0/383358394 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7fa7600b0a10 con 0x5614d84efe80 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa775031740 1 -- 172.21.15.82:0/383358394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa76009ca30 con 0x5614d84efe80 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa74b7fe640 1 -- 172.21.15.82:0/383358394 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa760021070 con 0x5614d84efe80 2024-04-21T08:56:04.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa775031740 10 client.18237 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa775031740 1 -- 172.21.15.82:0/383358394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5614d856f410 con 0x5614d84efe80 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa74b7fe640 1 -- 172.21.15.82:0/383358394 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa76009ca30 con 0x5614d84efe80 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa74b7fe640 1 -- 172.21.15.82:0/383358394 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fa7600cdd60 con 0x5614d84efe80 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa775031740 10 client.18237 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa775031740 10 client.18237 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa775031740 20 client.18237 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa775031740 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5614d856f690 0x5614d8571ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa775031740 4 client.18237 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.897+0000 7fa775031740 1 -- 172.21.15.82:0/383358394 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614d8461c10 con 0x5614d856f690 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa76bfff640 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5614d856f690 0x5614d8571ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa74b7fe640 10 client.18237 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa76bfff640 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5614d856f690 0x5614d8571ab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=925d279f7a367448 server_cookie=7ae6676f6d8ccee in_seq=0 out_seq=0 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa74b7fe640 1 -- 172.21.15.82:0/383358394 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614d8461c10 con 0x5614d856f690 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa74b7fe640 10 client.18237 handle_command_reply: tid=0 2024-04-21T08:56:04.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa775031740 1 client.18237 shutdown 2024-04-21T08:56:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa775031740 1 -- 172.21.15.82:0/383358394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 msgr2=0x5614d84fcd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa775031740 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 0x5614d84fcd60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fa7600ce0e0 tx=0x7fa760010c00 comp rx=0 tx=0).stop 2024-04-21T08:56:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa775031740 1 -- 172.21.15.82:0/383358394 shutdown_connections 2024-04-21T08:56:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa775031740 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x5614d856f690 0x5614d8571ab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa775031740 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614d84f8a00 0x5614d84fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa74b7fe640 0 client.18237 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:56:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa775031740 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614d84efe80 0x5614d84fcd60 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa775031740 1 --2- 172.21.15.82:0/383358394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5614d84ef4d0 0x5614d84fc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa775031740 1 -- 172.21.15.82:0/383358394 >> 172.21.15.82:0/383358394 conn(0x5614d84e9ba0 msgr2=0x5614d84e8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.898+0000 7fa775031740 1 -- 172.21.15.82:0/383358394 shutdown_connections 2024-04-21T08:56:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa775031740 1 -- 172.21.15.82:0/383358394 wait complete. 2024-04-21T08:56:04.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa775031740 20 client.18237 trim_cache size 0 max 16384 2024-04-21T08:56:04.907 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:56:04.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa772366640 1 -- 172.21.15.82:0/2844447432 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa7540a3d20 msgr2=0x7fa7540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa772366640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa7540a3d20 0x7fa7540a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa764009430 tx=0x7fa7640095c0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa772366640 1 -- 172.21.15.82:0/2844447432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76c107270 msgr2=0x7fa76c198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa772366640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76c107270 0x7fa76c198b30 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa76000d770 tx=0x7fa76000dc30 comp rx=0 tx=0).stop 2024-04-21T08:56:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa772366640 1 -- 172.21.15.82:0/2844447432 shutdown_connections 2024-04-21T08:56:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa772366640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa76c19ceb0 0x7fa76c19d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa772366640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa7540a3d20 0x7fa7540a61e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa772366640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa76c107270 0x7fa76c198b30 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa772366640 1 --2- 172.21.15.82:0/2844447432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa76c06a840 0x7fa76c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa772366640 1 -- 172.21.15.82:0/2844447432 >> 172.21.15.82:0/2844447432 conn(0x7fa76c077eb0 msgr2=0x7fa76c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa772366640 1 -- 172.21.15.82:0/2844447432 shutdown_connections 2024-04-21T08:56:04.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:04.899+0000 7fa772366640 1 -- 172.21.15.82:0/2844447432 wait complete. 2024-04-21T08:56:04.911 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:56:05.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.009+0000 7f75d953d640 1 -- 172.21.15.82:0/3520244514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d41072b0 msgr2=0x7f75d410eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.009+0000 7f75d953d640 1 --2- 172.21.15.82:0/3520244514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d41072b0 0x7f75d410eec0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009ca0 tx=0x7f75cc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.009+0000 7f75d953d640 1 -- 172.21.15.82:0/3520244514 shutdown_connections 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.009+0000 7f75d953d640 1 --2- 172.21.15.82:0/3520244514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d41072b0 0x7f75d410eec0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.009+0000 7f75d953d640 1 --2- 172.21.15.82:0/3520244514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75d41014a0 0x7f75d4106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.009+0000 7f75d953d640 1 --2- 172.21.15.82:0/3520244514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75d4100af0 0x7f75d4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.009+0000 7f75d953d640 1 -- 172.21.15.82:0/3520244514 >> 172.21.15.82:0/3520244514 conn(0x7f75d40760e0 msgr2=0x7f75d40764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.009+0000 7f75d953d640 1 -- 172.21.15.82:0/3520244514 shutdown_connections 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.009+0000 7f75d953d640 1 -- 172.21.15.82:0/3520244514 wait complete. 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.010+0000 7f75d953d640 1 Processor -- start 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.010+0000 7f75d953d640 1 -- start start 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.010+0000 7f75d953d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75d4100af0 0x7f75d4101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.010+0000 7f75d953d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75d41072b0 0x7f75d4101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.010+0000 7f75d953d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d41024e0 0x7f75d41a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.010+0000 7f75d953d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4112250 con 0x7f75d41024e0 2024-04-21T08:56:05.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.010+0000 7f75d953d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41120d0 con 0x7f75d4100af0 2024-04-21T08:56:05.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.010+0000 7f75d953d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41123d0 con 0x7f75d41072b0 2024-04-21T08:56:05.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.010+0000 7f75d37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d41024e0 0x7f75d41a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.010+0000 7f75d37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d41024e0 0x7f75d41a6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38982/0 (socket says 172.21.15.82:38982) 2024-04-21T08:56:05.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.010+0000 7f75d37fe640 1 -- 172.21.15.82:0/4116792526 learned_addr learned my addr 172.21.15.82:0/4116792526 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.011+0000 7f75d27fc640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75d41072b0 0x7f75d4101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.011+0000 7f75d37fe640 1 -- 172.21.15.82:0/4116792526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75d41072b0 msgr2=0x7f75d4101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.011+0000 7f75d37fe640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75d41072b0 0x7f75d4101fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.011+0000 7f75d37fe640 1 -- 172.21.15.82:0/4116792526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75d4100af0 msgr2=0x7f75d4101a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.011+0000 7f75d37fe640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75d4100af0 0x7f75d4101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.011+0000 7f75d37fe640 1 -- 172.21.15.82:0/4116792526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d41a7080 con 0x7f75d41024e0 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.011+0000 7f75d37fe640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d41024e0 0x7f75d41a6980 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f75cc005e30 tx=0x7f75cc032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.011+0000 7f75bbfff640 1 -- 172.21.15.82:0/4116792526 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc041020 con 0x7f75d41024e0 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.011+0000 7f75bbfff640 1 -- 172.21.15.82:0/4116792526 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75cc032970 con 0x7f75d41024e0 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.011+0000 7f75d953d640 1 -- 172.21.15.82:0/4116792526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d41a7310 con 0x7f75d41024e0 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.011+0000 7f75bbfff640 1 -- 172.21.15.82:0/4116792526 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc039040 con 0x7f75d41024e0 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.011+0000 7f75d953d640 1 -- 172.21.15.82:0/4116792526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d41a7850 con 0x7f75d41024e0 2024-04-21T08:56:05.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.013+0000 7f75bbfff640 1 -- 172.21.15.82:0/4116792526 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f75cc044020 con 0x7f75d41024e0 2024-04-21T08:56:05.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75dc20a740 1 Processor -- start 2024-04-21T08:56:05.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75bbfff640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f75bc0a3dd0 0x7f75bc0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75dc20a740 1 -- start start 2024-04-21T08:56:05.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75d27fc640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f75bc0a3dd0 0x7f75bc0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75bbfff640 1 -- 172.21.15.82:0/4116792526 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f75cc0f2920 con 0x7f75d41024e0 2024-04-21T08:56:05.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75dc20a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfff4d0 0x558cfcfff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75dc20a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfffe80 0x558cfd008430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75dc20a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd00ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75dc20a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558cfcf641f0 con 0x558cfcfff4d0 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75dc20a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558cfcf014b0 con 0x558cfd008a00 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75dc20a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558cfcf601b0 con 0x558cfcfffe80 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75d2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfff4d0 0x558cfcfff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75d2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfff4d0 0x558cfcfff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38996/0 (socket says 172.21.15.82:38996) 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.014+0000 7f75d2ffd640 1 -- 172.21.15.82:0/3670389061 learned_addr learned my addr 172.21.15.82:0/3670389061 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75d27fc640 1 --2- 172.21.15.82:0/3670389061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfffe80 0x558cfd008430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75d27fc640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f75bc0a3dd0 0x7f75bc0a6290 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f75c8005e00 tx=0x7f75c8005c50 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75d37fe640 1 --2- 172.21.15.82:0/3670389061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd00ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75d2ffd640 1 -- 172.21.15.82:0/3670389061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfffe80 msgr2=0x558cfd008430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75d2ffd640 1 --2- 172.21.15.82:0/3670389061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfffe80 0x558cfd008430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75d2ffd640 1 -- 172.21.15.82:0/3670389061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 msgr2=0x558cfd00ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75d2ffd640 1 --2- 172.21.15.82:0/3670389061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd00ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75d2ffd640 1 -- 172.21.15.82:0/3670389061 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfcef31b0 con 0x558cfcfff4d0 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75d27fc640 1 --2- 172.21.15.82:0/3670389061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfffe80 0x558cfd008430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75d2ffd640 1 --2- 172.21.15.82:0/3670389061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfff4d0 0x558cfcfff8b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009c40 tx=0x7f75c400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d7f39f12188a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75b97fa640 1 -- 172.21.15.82:0/3670389061 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4005020 con 0x558cfcfff4d0 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75b97fa640 1 -- 172.21.15.82:0/3670389061 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c40047a0 con 0x558cfcfff4d0 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.015+0000 7f75b97fa640 1 -- 172.21.15.82:0/3670389061 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4010690 con 0x558cfcfff4d0 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 -- 172.21.15.82:0/3670389061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfff4d0 msgr2=0x558cfcfff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3670389061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfff4d0 0x558cfcfff8b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009c40 tx=0x7f75c400e720 comp rx=0 tx=0).stop 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 -- 172.21.15.82:0/3670389061 shutdown_connections 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3670389061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd00ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3670389061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfffe80 0x558cfd008430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3670389061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfff4d0 0x558cfcfff8b0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 -- 172.21.15.82:0/3670389061 >> 172.21.15.82:0/3670389061 conn(0x558cfcff9ba0 msgr2=0x558cfcff9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 -- 172.21.15.82:0/3670389061 shutdown_connections 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 -- 172.21.15.82:0/3670389061 wait complete. 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 -- start start 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 Processor -- start 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfff4d0 0x558cfd0082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfd005680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558cfd00b970 con 0x558cfcfffe80 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558cfcf1d500 con 0x558cfd008a00 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558cfd00baf0 con 0x558cfcfff4d0 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.016+0000 7f75dc20a740 10 client.0 _ll_register_callbacks cb 0x558cfceebb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75d27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfd005680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75d27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfd005680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39006/0 (socket says 172.21.15.82:39006) 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75d27fc640 1 -- 172.21.15.82:0/3417632746 learned_addr learned my addr 172.21.15.82:0/3417632746 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75d37fe640 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75d2ffd640 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfff4d0 0x558cfd0082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75d27fc640 1 -- 172.21.15.82:0/3417632746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfff4d0 msgr2=0x558cfd0082f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75d27fc640 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfff4d0 0x558cfd0082f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75d27fc640 1 -- 172.21.15.82:0/3417632746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 msgr2=0x558cfd005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75d27fc640 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75d27fc640 1 -- 172.21.15.82:0/3417632746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfcc763f0 con 0x558cfcfffe80 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75d2ffd640 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfff4d0 0x558cfd0082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75d27fc640 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfd005680 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f75c8005fd0 tx=0x7f75c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43962072dc014485 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75b6ff5640 1 -- 172.21.15.82:0/3417632746 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c801f840 con 0x558cfcfffe80 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75b6ff5640 1 -- 172.21.15.82:0/3417632746 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f75c801f9e0 con 0x558cfcfffe80 2024-04-21T08:56:05.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75dc20a740 1 -- 172.21.15.82:0/3417632746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c40047a0 con 0x558cfcfffe80 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75b6ff5640 1 -- 172.21.15.82:0/3417632746 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f75c8030050 con 0x558cfcfffe80 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75dc20a740 10 client.18249 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75dc20a740 1 -- 172.21.15.82:0/3417632746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558cfcd407d0 con 0x558cfcfffe80 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.017+0000 7f75b6ff5640 1 -- 172.21.15.82:0/3417632746 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c40047a0 con 0x558cfcfffe80 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.018+0000 7f75b6ff5640 1 -- 172.21.15.82:0/3417632746 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f75c8010b20 con 0x558cfcfffe80 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.018+0000 7f75dc20a740 10 client.18249 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.018+0000 7f75dc20a740 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.018+0000 7f75dc20a740 20 client.18249 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.018+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558cfd018bf0 0x558cfd01b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.018+0000 7f75dc20a740 4 client.18249 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.018+0000 7f75dc20a740 1 -- 172.21.15.82:0/3417632746 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cfcf03930 con 0x558cfd018bf0 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.018+0000 7f75d2ffd640 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558cfd018bf0 0x558cfd01b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.018+0000 7f75b6ff5640 10 client.18249 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:05.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.018+0000 7f75d2ffd640 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558cfd018bf0 0x558cfd01b050 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3b9678d13cccfc server_cookie=a37ac8b71638ef5 in_seq=0 out_seq=0 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75b6ff5640 1 -- 172.21.15.82:0/3417632746 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558cfcf03930 con 0x558cfd018bf0 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75b6ff5640 10 client.18249 handle_command_reply: tid=0 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75dc20a740 1 client.18249 shutdown 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75dc20a740 1 -- 172.21.15.82:0/3417632746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 msgr2=0x558cfd005680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfd005680 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f75c8005fd0 tx=0x7f75c80046c0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75dc20a740 1 -- 172.21.15.82:0/3417632746 shutdown_connections 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558cfd018bf0 0x558cfd01b050 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd005c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfd005680 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3417632746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfff4d0 0x558cfd0082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75b6ff5640 0 client.18249 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75dc20a740 1 -- 172.21.15.82:0/3417632746 >> 172.21.15.82:0/3417632746 conn(0x558cfcff9ba0 msgr2=0x558cfcffb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75dc20a740 1 -- 172.21.15.82:0/3417632746 shutdown_connections 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75dc20a740 1 -- 172.21.15.82:0/3417632746 wait complete. 2024-04-21T08:56:05.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.019+0000 7f75dc20a740 20 client.18249 trim_cache size 0 max 16384 2024-04-21T08:56:05.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.032+0000 7f75dc20a740 1 Processor -- start 2024-04-21T08:56:05.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.032+0000 7f75dc20a740 1 -- start start 2024-04-21T08:56:05.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.032+0000 7f75dc20a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfcfff4d0 0x558cfd014860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.032+0000 7f75dc20a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfcff8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.032+0000 7f75dc20a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfd008a00 0x558cfd007db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.032+0000 7f75dc20a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41b16f0 con 0x558cfcfffe80 2024-04-21T08:56:05.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.032+0000 7f75dc20a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41b32c0 con 0x558cfcfff4d0 2024-04-21T08:56:05.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.032+0000 7f75dc20a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41b1dc0 con 0x558cfd008a00 2024-04-21T08:56:05.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75d27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfcff8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75d27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfcff8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39010/0 (socket says 172.21.15.82:39010) 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75d27fc640 1 -- 172.21.15.82:0/3402209528 learned_addr learned my addr 172.21.15.82:0/3402209528 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75d37fe640 1 --2- 172.21.15.82:0/3402209528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfd008a00 0x558cfd007db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75d27fc640 1 -- 172.21.15.82:0/3402209528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfd008a00 msgr2=0x558cfd007db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75d27fc640 1 --2- 172.21.15.82:0/3402209528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfd008a00 0x558cfd007db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75d27fc640 1 -- 172.21.15.82:0/3402209528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfcfff4d0 msgr2=0x558cfd014860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75d27fc640 1 --2- 172.21.15.82:0/3402209528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfcfff4d0 0x558cfd014860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75d27fc640 1 -- 172.21.15.82:0/3402209528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfd005f70 con 0x558cfcfffe80 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75d27fc640 1 --2- 172.21.15.82:0/3402209528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfcff8a80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f75c8010f70 tx=0x7f75c801e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47405a50870dd94 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75b8ff9640 1 -- 172.21.15.82:0/3402209528 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8039020 con 0x558cfcfffe80 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75b8ff9640 1 -- 172.21.15.82:0/3402209528 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c801ebb0 con 0x558cfcfffe80 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.033+0000 7f75b8ff9640 1 -- 172.21.15.82:0/3402209528 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c80103d0 con 0x558cfcfffe80 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 -- 172.21.15.82:0/3402209528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 msgr2=0x558cfcff8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:05.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3402209528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfcff8a80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f75c8010f70 tx=0x7f75c801e740 comp rx=0 tx=0).stop 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 -- 172.21.15.82:0/3402209528 shutdown_connections 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3402209528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfd008a00 0x558cfd007db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3402209528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfcff8a80 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 --2- 172.21.15.82:0/3402209528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfcfff4d0 0x558cfd014860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 -- 172.21.15.82:0/3402209528 >> 172.21.15.82:0/3402209528 conn(0x558cfcff9ba0 msgr2=0x558cfcff8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 -- 172.21.15.82:0/3402209528 shutdown_connections 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 -- 172.21.15.82:0/3402209528 wait complete. 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 -- start start 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 Processor -- start 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfff4d0 0x558cfd0112b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfd002220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd002960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41b2bc0 con 0x558cfcfffe80 2024-04-21T08:56:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41b24c0 con 0x558cfd008a00 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41af6c0 con 0x558cfcfff4d0 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75dc20a740 10 client.0 _ll_register_callbacks cb 0x7f75c800fc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75d27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfd002220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75d27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfd002220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39026/0 (socket says 172.21.15.82:39026) 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75d27fc640 1 -- 172.21.15.82:0/1289670284 learned_addr learned my addr 172.21.15.82:0/1289670284 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75d2ffd640 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfff4d0 0x558cfd0112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.034+0000 7f75d37fe640 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd002960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75d27fc640 1 -- 172.21.15.82:0/1289670284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfff4d0 msgr2=0x558cfd0112b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75d27fc640 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfff4d0 0x558cfd0112b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75d27fc640 1 -- 172.21.15.82:0/1289670284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 msgr2=0x558cfd002960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75d27fc640 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd002960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75d27fc640 1 -- 172.21.15.82:0/1289670284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfd07e190 con 0x558cfcfffe80 2024-04-21T08:56:05.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75d37fe640 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd002960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75d27fc640 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfd002220 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f75c8010ef0 tx=0x7f75c801eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc87c2133796663 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75b6ff5640 1 -- 172.21.15.82:0/1289670284 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8039030 con 0x558cfcfffe80 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75b6ff5640 1 -- 172.21.15.82:0/1289670284 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f75c8004680 con 0x558cfcfffe80 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75dc20a740 1 -- 172.21.15.82:0/1289670284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c801ebb0 con 0x558cfcfffe80 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75b6ff5640 1 -- 172.21.15.82:0/1289670284 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f75c804e050 con 0x558cfcfffe80 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75dc20a740 10 client.18255 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75dc20a740 1 -- 172.21.15.82:0/1289670284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558cfd07f330 con 0x558cfcfffe80 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75b6ff5640 1 -- 172.21.15.82:0/1289670284 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c801ebb0 con 0x558cfcfffe80 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75b6ff5640 1 -- 172.21.15.82:0/1289670284 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f75c8033040 con 0x558cfcfffe80 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75dc20a740 10 client.18255 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75dc20a740 10 client.18255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75dc20a740 20 client.18255 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:05.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75dc20a740 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558cfd07f990 0x558cfd081db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75dc20a740 4 client.18255 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.035+0000 7f75dc20a740 1 -- 172.21.15.82:0/1289670284 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558cfcf71c10 con 0x558cfd07f990 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75d2ffd640 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558cfd07f990 0x558cfd081db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75d2ffd640 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558cfd07f990 0x558cfd081db0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b77304bce2aedeff server_cookie=38db3eb6a0788385 in_seq=0 out_seq=0 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75b6ff5640 10 client.18255 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75b6ff5640 1 -- 172.21.15.82:0/1289670284 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558cfcf71c10 con 0x558cfd07f990 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75b6ff5640 10 client.18255 handle_command_reply: tid=0 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75dc20a740 1 client.18255 shutdown 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75dc20a740 1 -- 172.21.15.82:0/1289670284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 msgr2=0x558cfd002220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75dc20a740 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfd002220 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f75c8010ef0 tx=0x7f75c801eb40 comp rx=0 tx=0).stop 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75dc20a740 1 -- 172.21.15.82:0/1289670284 shutdown_connections 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75dc20a740 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558cfd07f990 0x558cfd081db0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75dc20a740 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558cfd008a00 0x558cfd002960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75b6ff5640 0 client.18255 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:05.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75dc20a740 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558cfcfffe80 0x558cfd002220 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75dc20a740 1 --2- 172.21.15.82:0/1289670284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558cfcfff4d0 0x558cfd0112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75dc20a740 1 -- 172.21.15.82:0/1289670284 >> 172.21.15.82:0/1289670284 conn(0x558cfcff9ba0 msgr2=0x558cfd000af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:05.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75dc20a740 1 -- 172.21.15.82:0/1289670284 shutdown_connections 2024-04-21T08:56:05.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75dc20a740 1 -- 172.21.15.82:0/1289670284 wait complete. 2024-04-21T08:56:05.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.036+0000 7f75dc20a740 20 client.18255 trim_cache size 0 max 16384 2024-04-21T08:56:05.047 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:56:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.037+0000 7f75d953d640 1 -- 172.21.15.82:0/4116792526 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f75bc0a3dd0 msgr2=0x7f75bc0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.037+0000 7f75d953d640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f75bc0a3dd0 0x7f75bc0a6290 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f75c8005e00 tx=0x7f75c8005c50 comp rx=0 tx=0).stop 2024-04-21T08:56:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.037+0000 7f75d953d640 1 -- 172.21.15.82:0/4116792526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d41024e0 msgr2=0x7f75d41a6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.037+0000 7f75d953d640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d41024e0 0x7f75d41a6980 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f75cc005e30 tx=0x7f75cc032420 comp rx=0 tx=0).stop 2024-04-21T08:56:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.037+0000 7f75d953d640 1 -- 172.21.15.82:0/4116792526 shutdown_connections 2024-04-21T08:56:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.037+0000 7f75d953d640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f75bc0a3dd0 0x7f75bc0a6290 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.037+0000 7f75d953d640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75d41024e0 0x7f75d41a6980 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.037+0000 7f75d953d640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75d41072b0 0x7f75d4101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.037+0000 7f75d953d640 1 --2- 172.21.15.82:0/4116792526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75d4100af0 0x7f75d4101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.037+0000 7f75d953d640 1 -- 172.21.15.82:0/4116792526 >> 172.21.15.82:0/4116792526 conn(0x7f75d40760e0 msgr2=0x7f75d40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.037+0000 7f75d953d640 1 -- 172.21.15.82:0/4116792526 shutdown_connections 2024-04-21T08:56:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:05.037+0000 7f75d953d640 1 -- 172.21.15.82:0/4116792526 wait complete. 2024-04-21T08:56:05.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:05 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1098705623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:56:05.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:05 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1098705623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:56:05.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:05 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1098705623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:56:06.051 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:56:06.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:56:06.051 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:56:06.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.164+0000 7f5782b67640 1 -- 172.21.15.82:0/705454626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f577c101a40 msgr2=0x7f577c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.164+0000 7f5782b67640 1 --2- 172.21.15.82:0/705454626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f577c101a40 0x7f577c105a80 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5768009900 tx=0x7f576802bc30 comp rx=0 tx=0).stop 2024-04-21T08:56:06.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.164+0000 7f5782b67640 1 -- 172.21.15.82:0/705454626 shutdown_connections 2024-04-21T08:56:06.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.164+0000 7f5782b67640 1 --2- 172.21.15.82:0/705454626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f577c106160 0x7f577c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.164+0000 7f5782b67640 1 --2- 172.21.15.82:0/705454626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f577c101a40 0x7f577c105a80 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.164+0000 7f5782b67640 1 --2- 172.21.15.82:0/705454626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f577c101090 0x7f577c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.164+0000 7f5782b67640 1 -- 172.21.15.82:0/705454626 >> 172.21.15.82:0/705454626 conn(0x7f577c077f50 msgr2=0x7f577c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:06.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.164+0000 7f5782b67640 1 -- 172.21.15.82:0/705454626 shutdown_connections 2024-04-21T08:56:06.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.164+0000 7f5782b67640 1 -- 172.21.15.82:0/705454626 wait complete. 2024-04-21T08:56:06.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.164+0000 7f5782b67640 1 Processor -- start 2024-04-21T08:56:06.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.164+0000 7f5782b67640 1 -- start start 2024-04-21T08:56:06.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5782b67640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f577c101090 0x7f577c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5782b67640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f577c106160 0x7f577c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5782b67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f577c1985d0 0x7f577c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5782b67640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f577c111080 con 0x7f577c101090 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5782b67640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f577c110f00 con 0x7f577c106160 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5782b67640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f577c111200 con 0x7f577c1985d0 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5780903640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f577c101090 0x7f577c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5780903640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f577c101090 0x7f577c19e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39046/0 (socket says 172.21.15.82:39046) 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5780903640 1 -- 172.21.15.82:0/418072241 learned_addr learned my addr 172.21.15.82:0/418072241 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f577bfff640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f577c106160 0x7f577c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5780903640 1 -- 172.21.15.82:0/418072241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f577c1985d0 msgr2=0x7f577c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5781104640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f577c1985d0 0x7f577c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5780903640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f577c1985d0 0x7f577c198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5780903640 1 -- 172.21.15.82:0/418072241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f577c106160 msgr2=0x7f577c19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5780903640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f577c106160 0x7f577c19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5780903640 1 -- 172.21.15.82:0/418072241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c1990f0 con 0x7f577c101090 2024-04-21T08:56:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5780903640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f577c101090 0x7f577c19e420 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f577000d950 tx=0x7f577000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5779ffb640 1 -- 172.21.15.82:0/418072241 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5770017030 con 0x7f577c101090 2024-04-21T08:56:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5779ffb640 1 -- 172.21.15.82:0/418072241 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5770004020 con 0x7f577c101090 2024-04-21T08:56:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5779ffb640 1 -- 172.21.15.82:0/418072241 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700047a0 con 0x7f577c101090 2024-04-21T08:56:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.165+0000 7f5782b67640 1 -- 172.21.15.82:0/418072241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577c1993e0 con 0x7f577c101090 2024-04-21T08:56:06.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.166+0000 7f5782b67640 1 -- 172.21.15.82:0/418072241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577c1a5630 con 0x7f577c101090 2024-04-21T08:56:06.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.168+0000 7f5779ffb640 1 -- 172.21.15.82:0/418072241 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5770009a30 con 0x7f577c101090 2024-04-21T08:56:06.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.168+0000 7f5785838740 1 Processor -- start 2024-04-21T08:56:06.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5785838740 1 -- start start 2024-04-21T08:56:06.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5779ffb640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f576c0a3d20 0x7f576c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5785838740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3b47c0 0x5583eb3b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5785838740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5785838740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 0x5583eb3be6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5785838740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5583eb318f00 con 0x5583eb3be270 2024-04-21T08:56:06.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5785838740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5583eb2b6590 con 0x5583eb3b47c0 2024-04-21T08:56:06.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5785838740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5583eb314ec0 con 0x5583eb3b5170 2024-04-21T08:56:06.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5781104640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f576c0a3d20 0x7f576c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5779ffb640 1 -- 172.21.15.82:0/418072241 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f57700d06f0 con 0x7f577c101090 2024-04-21T08:56:06.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5781104640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 0x5583eb3be6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5781104640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 0x5583eb3be6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39048/0 (socket says 172.21.15.82:39048) 2024-04-21T08:56:06.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5781104640 1 -- 172.21.15.82:0/320113802 learned_addr learned my addr 172.21.15.82:0/320113802 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:06.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5780903640 1 --2- 172.21.15.82:0/320113802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3b47c0 0x5583eb3b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f577bfff640 1 --2- 172.21.15.82:0/320113802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5781104640 1 -- 172.21.15.82:0/320113802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 msgr2=0x5583eb3bdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5781104640 1 --2- 172.21.15.82:0/320113802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3bdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5781104640 1 -- 172.21.15.82:0/320113802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3b47c0 msgr2=0x5583eb3b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5781104640 1 --2- 172.21.15.82:0/320113802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3b47c0 0x5583eb3b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5781104640 1 -- 172.21.15.82:0/320113802 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583eb0f37e0 con 0x5583eb3be270 2024-04-21T08:56:06.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.169+0000 7f5781104640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f576c0a3d20 0x7f576c0a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5774014a40 tx=0x7f5774009c70 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:06.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5781104640 1 --2- 172.21.15.82:0/320113802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 0x5583eb3be6d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5774011c80 tx=0x7f577401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370d7f5e5214e896 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:06.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5766ffd640 1 -- 172.21.15.82:0/320113802 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774011d70 con 0x5583eb3be270 2024-04-21T08:56:06.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5766ffd640 1 -- 172.21.15.82:0/320113802 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577400fc00 con 0x5583eb3be270 2024-04-21T08:56:06.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5766ffd640 1 -- 172.21.15.82:0/320113802 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774003b70 con 0x5583eb3be270 2024-04-21T08:56:06.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5785838740 1 -- 172.21.15.82:0/320113802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 msgr2=0x5583eb3be6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5785838740 1 --2- 172.21.15.82:0/320113802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 0x5583eb3be6d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5774011c80 tx=0x7f577401b000 comp rx=0 tx=0).stop 2024-04-21T08:56:06.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5785838740 1 -- 172.21.15.82:0/320113802 shutdown_connections 2024-04-21T08:56:06.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5785838740 1 --2- 172.21.15.82:0/320113802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 0x5583eb3be6d0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5785838740 1 --2- 172.21.15.82:0/320113802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3bdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5785838740 1 --2- 172.21.15.82:0/320113802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3b47c0 0x5583eb3b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5785838740 1 -- 172.21.15.82:0/320113802 >> 172.21.15.82:0/320113802 conn(0x5583eb3aefb0 msgr2=0x5583eb3af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:06.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5785838740 1 -- 172.21.15.82:0/320113802 shutdown_connections 2024-04-21T08:56:06.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.170+0000 7f5785838740 1 -- 172.21.15.82:0/320113802 wait complete. 2024-04-21T08:56:06.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5785838740 1 -- start start 2024-04-21T08:56:06.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5785838740 1 Processor -- start 2024-04-21T08:56:06.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5785838740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 0x5583eb3c1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5785838740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5785838740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3be270 0x5583eb3c27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5785838740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5583eb3c5150 con 0x5583eb3b47c0 2024-04-21T08:56:06.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5785838740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5583eb2d2580 con 0x5583eb3be270 2024-04-21T08:56:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5785838740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5583eb3c52d0 con 0x5583eb3b5170 2024-04-21T08:56:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5785838740 10 client.0 _ll_register_callbacks cb 0x5583eb2d8650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5780903640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 0x5583eb3c1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5780903640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 0x5583eb3c1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39062/0 (socket says 172.21.15.82:39062) 2024-04-21T08:56:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5780903640 1 -- 172.21.15.82:0/1334389981 learned_addr learned my addr 172.21.15.82:0/1334389981 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.171+0000 7f5781104640 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3be270 0x5583eb3c27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f577bfff640 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f5780903640 1 -- 172.21.15.82:0/1334389981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 msgr2=0x5583eb3c22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f5780903640 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3c22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f5780903640 1 -- 172.21.15.82:0/1334389981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3be270 msgr2=0x5583eb3c27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f5780903640 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3be270 0x5583eb3c27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f5780903640 1 -- 172.21.15.82:0/1334389981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583eb153f70 con 0x5583eb3b47c0 2024-04-21T08:56:06.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f5781104640 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3be270 0x5583eb3c27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:06.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f5780903640 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 0x5583eb3c1d70 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f577009bf20 tx=0x7f577009c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626aafb18ac04b01 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:06.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f575bfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:06.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f575bfff640 1 -- 172.21.15.82:0/1334389981 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700a4e90 con 0x5583eb3b47c0 2024-04-21T08:56:06.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f575bfff640 1 -- 172.21.15.82:0/1334389981 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f57700041d0 con 0x5583eb3b47c0 2024-04-21T08:56:06.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f5785838740 1 -- 172.21.15.82:0/1334389981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f577400fc00 con 0x5583eb3b47c0 2024-04-21T08:56:06.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f575bfff640 1 -- 172.21.15.82:0/1334389981 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5770098890 con 0x5583eb3b47c0 2024-04-21T08:56:06.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f5785838740 10 client.18267 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f5785838740 1 -- 172.21.15.82:0/1334389981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5583eafeb170 con 0x5583eb3b47c0 2024-04-21T08:56:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f575bfff640 1 -- 172.21.15.82:0/1334389981 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577400fc00 con 0x5583eb3b47c0 2024-04-21T08:56:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f575bfff640 1 -- 172.21.15.82:0/1334389981 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f57700a0120 con 0x5583eb3b47c0 2024-04-21T08:56:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.172+0000 7f5785838740 10 client.18267 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.173+0000 7f5785838740 10 client.18267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.173+0000 7f5785838740 20 client.18267 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.173+0000 7f5785838740 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5583eb3d2700 0x5583eb3d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.173+0000 7f5785838740 4 client.18267 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.173+0000 7f5785838740 1 -- 172.21.15.82:0/1334389981 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583eb326920 con 0x5583eb3d2700 2024-04-21T08:56:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.173+0000 7f577bfff640 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5583eb3d2700 0x5583eb3d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.173+0000 7f577bfff640 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5583eb3d2700 0x5583eb3d4b30 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd03e868c3120a7d server_cookie=c079cf36f8421f2c in_seq=0 out_seq=0 2024-04-21T08:56:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.173+0000 7f575bfff640 10 client.18267 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f575bfff640 1 -- 172.21.15.82:0/1334389981 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583eb326920 con 0x5583eb3d2700 2024-04-21T08:56:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f575bfff640 10 client.18267 handle_command_reply: tid=0 2024-04-21T08:56:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f5785838740 1 client.18267 shutdown 2024-04-21T08:56:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f5785838740 1 -- 172.21.15.82:0/1334389981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 msgr2=0x5583eb3c1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f5785838740 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 0x5583eb3c1d70 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f577009bf20 tx=0x7f577009c2e0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f5785838740 1 -- 172.21.15.82:0/1334389981 shutdown_connections 2024-04-21T08:56:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f5785838740 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5583eb3d2700 0x5583eb3d4b30 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f5785838740 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3be270 0x5583eb3c27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f575bfff640 0 client.18267 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f5785838740 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3c22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f5785838740 1 --2- 172.21.15.82:0/1334389981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 0x5583eb3c1d70 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f5785838740 1 -- 172.21.15.82:0/1334389981 >> 172.21.15.82:0/1334389981 conn(0x5583eb3aefb0 msgr2=0x5583eb3b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:06.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f5785838740 1 -- 172.21.15.82:0/1334389981 shutdown_connections 2024-04-21T08:56:06.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f5785838740 1 -- 172.21.15.82:0/1334389981 wait complete. 2024-04-21T08:56:06.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.174+0000 7f5785838740 20 client.18267 trim_cache size 0 max 16384 2024-04-21T08:56:06.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5785838740 1 Processor -- start 2024-04-21T08:56:06.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5785838740 1 -- start start 2024-04-21T08:56:06.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5785838740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 0x5583eb3ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5785838740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5785838740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3be270 0x5583eb3bf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5785838740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1ad360 con 0x5583eb3b47c0 2024-04-21T08:56:06.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5785838740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1aef30 con 0x5583eb3be270 2024-04-21T08:56:06.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5785838740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1ada30 con 0x5583eb3b5170 2024-04-21T08:56:06.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5780903640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 0x5583eb3ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5780903640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 0x5583eb3ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39068/0 (socket says 172.21.15.82:39068) 2024-04-21T08:56:06.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5780903640 1 -- 172.21.15.82:0/1657264555 learned_addr learned my addr 172.21.15.82:0/1657264555 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:06.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5780903640 1 -- 172.21.15.82:0/1657264555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 msgr2=0x5583eb3c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5781104640 1 --2- 172.21.15.82:0/1657264555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3be270 0x5583eb3bf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5780903640 1 --2- 172.21.15.82:0/1657264555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5780903640 1 -- 172.21.15.82:0/1657264555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3be270 msgr2=0x5583eb3bf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5780903640 1 --2- 172.21.15.82:0/1657264555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3be270 0x5583eb3bf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5780903640 1 -- 172.21.15.82:0/1657264555 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583eb3b3970 con 0x5583eb3b47c0 2024-04-21T08:56:06.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f5780903640 1 --2- 172.21.15.82:0/1657264555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 0x5583eb3ce0a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f577009c160 tx=0x7f577009a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdb712e025b9d53 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:06.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f57667fc640 1 -- 172.21.15.82:0/1657264555 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700b5070 con 0x5583eb3b47c0 2024-04-21T08:56:06.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f57667fc640 1 -- 172.21.15.82:0/1657264555 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5770009d20 con 0x5583eb3b47c0 2024-04-21T08:56:06.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.192+0000 7f57667fc640 1 -- 172.21.15.82:0/1657264555 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700c88c0 con 0x5583eb3b47c0 2024-04-21T08:56:06.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.193+0000 7f5785838740 1 -- 172.21.15.82:0/1657264555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 msgr2=0x5583eb3ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.193+0000 7f5785838740 1 --2- 172.21.15.82:0/1657264555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 0x5583eb3ce0a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f577009c160 tx=0x7f577009a730 comp rx=0 tx=0).stop 2024-04-21T08:56:06.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.193+0000 7f5785838740 1 -- 172.21.15.82:0/1657264555 shutdown_connections 2024-04-21T08:56:06.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.193+0000 7f5785838740 1 --2- 172.21.15.82:0/1657264555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3be270 0x5583eb3bf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.193+0000 7f5785838740 1 --2- 172.21.15.82:0/1657264555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.193+0000 7f5785838740 1 --2- 172.21.15.82:0/1657264555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3b47c0 0x5583eb3ce0a0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.193+0000 7f5785838740 1 -- 172.21.15.82:0/1657264555 >> 172.21.15.82:0/1657264555 conn(0x5583eb3aefb0 msgr2=0x5583eb3b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.193+0000 7f5785838740 1 -- 172.21.15.82:0/1657264555 shutdown_connections 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.193+0000 7f5785838740 1 -- 172.21.15.82:0/1657264555 wait complete. 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.193+0000 7f5785838740 1 -- start start 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.193+0000 7f5785838740 1 Processor -- start 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5785838740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3b47c0 0x5583eb3ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5785838740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5785838740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 0x5583eb3b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5785838740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1ae830 con 0x5583eb3be270 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5785838740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1ae130 con 0x5583eb3b47c0 2024-04-21T08:56:06.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5785838740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1ab330 con 0x5583eb3b5170 2024-04-21T08:56:06.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5785838740 10 client.0 _ll_register_callbacks cb 0x7f577009bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:06.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5781104640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 0x5583eb3b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5781104640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 0x5583eb3b9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39084/0 (socket says 172.21.15.82:39084) 2024-04-21T08:56:06.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5781104640 1 -- 172.21.15.82:0/2716903442 learned_addr learned my addr 172.21.15.82:0/2716903442 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:06.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5780903640 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3b47c0 0x5583eb3ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f577bfff640 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5781104640 1 -- 172.21.15.82:0/2716903442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 msgr2=0x5583eb3caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5781104640 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5781104640 1 -- 172.21.15.82:0/2716903442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3b47c0 msgr2=0x5583eb3ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5781104640 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3b47c0 0x5583eb3ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5781104640 1 -- 172.21.15.82:0/2716903442 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583eb3ba3f0 con 0x5583eb3be270 2024-04-21T08:56:06.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f5781104640 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 0x5583eb3b9bf0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5774000f80 tx=0x7f57740095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3abd040fb3ecefd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:06.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f575bfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:06.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.194+0000 7f575bfff640 1 -- 172.21.15.82:0/2716903442 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577402c070 con 0x5583eb3be270 2024-04-21T08:56:06.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f575bfff640 1 -- 172.21.15.82:0/2716903442 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f57740140c0 con 0x5583eb3be270 2024-04-21T08:56:06.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f5785838740 1 -- 172.21.15.82:0/2716903442 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5770009d20 con 0x5583eb3be270 2024-04-21T08:56:06.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f575bfff640 1 -- 172.21.15.82:0/2716903442 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f577403a050 con 0x5583eb3be270 2024-04-21T08:56:06.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f5785838740 10 client.18273 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:06.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f5785838740 1 -- 172.21.15.82:0/2716903442 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5583eb3bb6f0 con 0x5583eb3be270 2024-04-21T08:56:06.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f575bfff640 1 -- 172.21.15.82:0/2716903442 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5770009d20 con 0x5583eb3be270 2024-04-21T08:56:06.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f575bfff640 1 -- 172.21.15.82:0/2716903442 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f577400f050 con 0x5583eb3be270 2024-04-21T08:56:06.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f5785838740 10 client.18273 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:06.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f5785838740 10 client.18273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f5785838740 20 client.18273 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f5785838740 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5583eb439cd0 0x5583eb43c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f5785838740 4 client.18273 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f5785838740 1 -- 172.21.15.82:0/2716903442 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5583eb320790 con 0x5583eb439cd0 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.195+0000 7f5780903640 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5583eb439cd0 0x5583eb43c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.196+0000 7f575bfff640 10 client.18273 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.196+0000 7f5780903640 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5583eb439cd0 0x5583eb43c0c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19d5217987bd35c5 server_cookie=4374b0eb3644b0b8 in_seq=0 out_seq=0 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f575bfff640 1 -- 172.21.15.82:0/2716903442 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5583eb320790 con 0x5583eb439cd0 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f575bfff640 10 client.18273 handle_command_reply: tid=0 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f5785838740 1 client.18273 shutdown 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f5785838740 1 -- 172.21.15.82:0/2716903442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 msgr2=0x5583eb3b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f5785838740 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 0x5583eb3b9bf0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f5774000f80 tx=0x7f57740095f0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f5785838740 1 -- 172.21.15.82:0/2716903442 shutdown_connections 2024-04-21T08:56:06.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f5785838740 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5583eb439cd0 0x5583eb43c0c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f5785838740 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583eb3be270 0x5583eb3b9bf0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f575bfff640 0 client.18273 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:06.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f5785838740 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583eb3b5170 0x5583eb3caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f5785838740 1 --2- 172.21.15.82:0/2716903442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5583eb3b47c0 0x5583eb3ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f5785838740 1 -- 172.21.15.82:0/2716903442 >> 172.21.15.82:0/2716903442 conn(0x5583eb3aefb0 msgr2=0x5583eb3ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:06.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f5785838740 1 -- 172.21.15.82:0/2716903442 shutdown_connections 2024-04-21T08:56:06.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f5785838740 1 -- 172.21.15.82:0/2716903442 wait complete. 2024-04-21T08:56:06.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.197+0000 7f5785838740 20 client.18273 trim_cache size 0 max 16384 2024-04-21T08:56:06.205 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:56:06.206 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:56:06.206 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "26275d45-e29c-4066-8739-3d5eaf545924", 2024-04-21T08:56:06.206 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:56:06.206 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:56:06.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.198+0000 7f5782b67640 1 -- 172.21.15.82:0/418072241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f576c0a3d20 msgr2=0x7f576c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.198+0000 7f5782b67640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f576c0a3d20 0x7f576c0a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5774014a40 tx=0x7f5774009c70 comp rx=0 tx=0).stop 2024-04-21T08:56:06.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.198+0000 7f5782b67640 1 -- 172.21.15.82:0/418072241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f577c101090 msgr2=0x7f577c19e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.198+0000 7f5782b67640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f577c101090 0x7f577c19e420 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f577000d950 tx=0x7f577000de10 comp rx=0 tx=0).stop 2024-04-21T08:56:06.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.198+0000 7f5782b67640 1 -- 172.21.15.82:0/418072241 shutdown_connections 2024-04-21T08:56:06.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.198+0000 7f5782b67640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f577c1985d0 0x7f577c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.198+0000 7f5782b67640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f577c106160 0x7f577c19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.198+0000 7f5782b67640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f576c0a3d20 0x7f576c0a61e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.198+0000 7f5782b67640 1 --2- 172.21.15.82:0/418072241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f577c101090 0x7f577c19e420 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.198+0000 7f5782b67640 1 -- 172.21.15.82:0/418072241 >> 172.21.15.82:0/418072241 conn(0x7f577c077f50 msgr2=0x7f577c104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:06.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.198+0000 7f5782b67640 1 -- 172.21.15.82:0/418072241 shutdown_connections 2024-04-21T08:56:06.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.198+0000 7f5782b67640 1 -- 172.21.15.82:0/418072241 wait complete. 2024-04-21T08:56:06.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:56:06.210 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:56:06.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.309+0000 7f5332081640 1 -- 172.21.15.82:0/2981606183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f532c1021a0 msgr2=0x7f532c102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.309+0000 7f5332081640 1 --2- 172.21.15.82:0/2981606183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f532c1021a0 0x7f532c102580 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f531c009900 tx=0x7f531c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:56:06.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.309+0000 7f5332081640 1 -- 172.21.15.82:0/2981606183 shutdown_connections 2024-04-21T08:56:06.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.309+0000 7f5332081640 1 --2- 172.21.15.82:0/2981606183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f532c107270 0x7f532c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.309+0000 7f5332081640 1 --2- 172.21.15.82:0/2981606183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f532c102b50 0x7f532c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.309+0000 7f5332081640 1 --2- 172.21.15.82:0/2981606183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f532c1021a0 0x7f532c102580 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.309+0000 7f5332081640 1 -- 172.21.15.82:0/2981606183 >> 172.21.15.82:0/2981606183 conn(0x7f532c0793e0 msgr2=0x7f532c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.309+0000 7f5332081640 1 -- 172.21.15.82:0/2981606183 shutdown_connections 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.309+0000 7f5332081640 1 -- 172.21.15.82:0/2981606183 wait complete. 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f5332081640 1 Processor -- start 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f5332081640 1 -- start start 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f5332081640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f532c1021a0 0x7f532c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f5332081640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f532c107270 0x7f532c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f5332081640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f532c074cd0 0x7f532c1a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f5332081640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f532c112250 con 0x7f532c074cd0 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f5332081640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f532c1120d0 con 0x7f532c107270 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f5332081640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f532c1123d0 con 0x7f532c1021a0 2024-04-21T08:56:06.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f532bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f532c074cd0 0x7f532c1a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f532bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f532c074cd0 0x7f532c1a6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39108/0 (socket says 172.21.15.82:39108) 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f532bfff640 1 -- 172.21.15.82:0/2261746021 learned_addr learned my addr 172.21.15.82:0/2261746021 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f532b7fe640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f532c1021a0 0x7f532c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f532affd640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f532c107270 0x7f532c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f532bfff640 1 -- 172.21.15.82:0/2261746021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f532c1021a0 msgr2=0x7f532c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f532bfff640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f532c1021a0 0x7f532c074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f532bfff640 1 -- 172.21.15.82:0/2261746021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f532c107270 msgr2=0x7f532c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f532bfff640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f532c107270 0x7f532c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.310+0000 7f532bfff640 1 -- 172.21.15.82:0/2261746021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532c1a7060 con 0x7f532c074cd0 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.311+0000 7f532bfff640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f532c074cd0 0x7f532c1a6960 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f532400c970 tx=0x7f532400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.311+0000 7f5328ff9640 1 -- 172.21.15.82:0/2261746021 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324007bb0 con 0x7f532c074cd0 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.311+0000 7f5328ff9640 1 -- 172.21.15.82:0/2261746021 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5324007d50 con 0x7f532c074cd0 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.311+0000 7f5332081640 1 -- 172.21.15.82:0/2261746021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532c1a7350 con 0x7f532c074cd0 2024-04-21T08:56:06.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.311+0000 7f5328ff9640 1 -- 172.21.15.82:0/2261746021 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324004e60 con 0x7f532c074cd0 2024-04-21T08:56:06.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.311+0000 7f5332081640 1 -- 172.21.15.82:0/2261746021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f532c1a78a0 con 0x7f532c074cd0 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.313+0000 7f5328ff9640 1 -- 172.21.15.82:0/2261746021 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5324005020 con 0x7f532c074cd0 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.313+0000 7f5328ff9640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f53140a3d50 0x7f53140a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.313+0000 7f5334c7b740 1 Processor -- start 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.313+0000 7f5334c7b740 1 -- start start 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532b7fe640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f53140a3d50 0x7f53140a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f5328ff9640 1 -- 172.21.15.82:0/2261746021 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f53240d0910 con 0x7f532c074cd0 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f5334c7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4580 0x562a8edd4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f5334c7b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8edd4ea0 0x562a8eddd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f5334c7b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8eddda80 0x562a8eddfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f5334c7b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562a8ed391e0 con 0x562a8eddda80 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f5334c7b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562a8ecd6390 con 0x562a8edd4ea0 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f5334c7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562a8ed351a0 con 0x562a8edd4580 2024-04-21T08:56:06.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8eddda80 0x562a8eddfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8eddda80 0x562a8eddfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39118/0 (socket says 172.21.15.82:39118) 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532bfff640 1 -- 172.21.15.82:0/3695108692 learned_addr learned my addr 172.21.15.82:0/3695108692 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532affd640 1 --2- 172.21.15.82:0/3695108692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8edd4ea0 0x562a8eddd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532b7fe640 1 --2- 172.21.15.82:0/3695108692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4580 0x562a8edd4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532bfff640 1 -- 172.21.15.82:0/3695108692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4580 msgr2=0x562a8edd4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532bfff640 1 --2- 172.21.15.82:0/3695108692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4580 0x562a8edd4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532bfff640 1 -- 172.21.15.82:0/3695108692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8edd4ea0 msgr2=0x562a8eddd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532b7fe640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f53140a3d50 0x7f53140a6210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f531c036040 tx=0x7f531c02c710 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532bfff640 1 --2- 172.21.15.82:0/3695108692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8edd4ea0 0x562a8eddd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532bfff640 1 -- 172.21.15.82:0/3695108692 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a8eb157d0 con 0x562a8eddda80 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f532bfff640 1 --2- 172.21.15.82:0/3695108692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8eddda80 0x562a8eddfe70 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5324098900 tx=0x7f532409cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329b18337e381098 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f5311ffb640 1 -- 172.21.15.82:0/3695108692 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532401d070 con 0x562a8eddda80 2024-04-21T08:56:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f5311ffb640 1 -- 172.21.15.82:0/3695108692 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f532409a7a0 con 0x562a8eddda80 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.314+0000 7f5311ffb640 1 -- 172.21.15.82:0/3695108692 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53240a4c80 con 0x562a8eddda80 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.315+0000 7f5334c7b740 1 -- 172.21.15.82:0/3695108692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8eddda80 msgr2=0x562a8eddfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.315+0000 7f5334c7b740 1 --2- 172.21.15.82:0/3695108692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8eddda80 0x562a8eddfe70 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5324098900 tx=0x7f532409cce0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.315+0000 7f5334c7b740 1 -- 172.21.15.82:0/3695108692 shutdown_connections 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.315+0000 7f5334c7b740 1 --2- 172.21.15.82:0/3695108692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8eddda80 0x562a8eddfe70 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.315+0000 7f5334c7b740 1 --2- 172.21.15.82:0/3695108692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8edd4ea0 0x562a8eddd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.315+0000 7f5334c7b740 1 --2- 172.21.15.82:0/3695108692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4580 0x562a8edd4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.315+0000 7f5334c7b740 1 -- 172.21.15.82:0/3695108692 >> 172.21.15.82:0/3695108692 conn(0x562a8edcecb0 msgr2=0x562a8edcf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.315+0000 7f5334c7b740 1 -- 172.21.15.82:0/3695108692 shutdown_connections 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.315+0000 7f5334c7b740 1 -- 172.21.15.82:0/3695108692 wait complete. 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.315+0000 7f5334c7b740 1 -- start start 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f5334c7b740 1 Processor -- start 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f5334c7b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 0x562a8eddab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f5334c7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4ea0 0x562a8eddb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f5334c7b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 0x562a8eddb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f5334c7b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562a8ede0870 con 0x562a8edd4580 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f5334c7b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562a8ecf22f0 con 0x562a8eddda80 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f5334c7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562a8ede09f0 con 0x562a8edd4ea0 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f5334c7b740 10 client.0 _ll_register_callbacks cb 0x562a8ecff580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f532b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 0x562a8eddab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f532b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 0x562a8eddab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39126/0 (socket says 172.21.15.82:39126) 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f532b7fe640 1 -- 172.21.15.82:0/2825643427 learned_addr learned my addr 172.21.15.82:0/2825643427 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f532affd640 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4ea0 0x562a8eddb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f532bfff640 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 0x562a8eddb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f532b7fe640 1 -- 172.21.15.82:0/2825643427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4ea0 msgr2=0x562a8eddb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f532b7fe640 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4ea0 0x562a8eddb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f532b7fe640 1 -- 172.21.15.82:0/2825643427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 msgr2=0x562a8eddb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f532b7fe640 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 0x562a8eddb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f532b7fe640 1 -- 172.21.15.82:0/2825643427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a8ea4b3f0 con 0x562a8edd4580 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f532b7fe640 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 0x562a8eddab20 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f531c035000 tx=0x7f531c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d8cbdd2eff5497 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.316+0000 7f530affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f530affd640 1 -- 172.21.15.82:0/2825643427 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c04e030 con 0x562a8edd4580 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f530affd640 1 -- 172.21.15.82:0/2825643427 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f531c03a040 con 0x562a8edd4580 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f5334c7b740 1 -- 172.21.15.82:0/2825643427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f532409a7a0 con 0x562a8edd4580 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f530affd640 1 -- 172.21.15.82:0/2825643427 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f531c05b050 con 0x562a8edd4580 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f5334c7b740 10 client.18285 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f5334c7b740 1 -- 172.21.15.82:0/2825643427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562a8ea0b0f0 con 0x562a8edd4580 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f530affd640 1 -- 172.21.15.82:0/2825643427 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f532409a7a0 con 0x562a8edd4580 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f530affd640 1 -- 172.21.15.82:0/2825643427 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f531c003a30 con 0x562a8edd4580 2024-04-21T08:56:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f5334c7b740 10 client.18285 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f5334c7b740 10 client.18285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f5334c7b740 20 client.18285 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f5334c7b740 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562a8ededb90 0x562a8edf0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f5334c7b740 4 client.18285 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f5334c7b740 1 -- 172.21.15.82:0/2825643427 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a8ed46c00 con 0x562a8ededb90 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.317+0000 7f532affd640 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562a8ededb90 0x562a8edf0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.318+0000 7f530affd640 10 client.18285 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.318+0000 7f532affd640 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562a8ededb90 0x562a8edf0010 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7f7e52187ec30ac server_cookie=e12c857c5f30c2f3 in_seq=0 out_seq=0 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f530affd640 1 -- 172.21.15.82:0/2825643427 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a8ed46c00 con 0x562a8ededb90 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f530affd640 10 client.18285 handle_command_reply: tid=0 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f5334c7b740 1 client.18285 shutdown 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f5334c7b740 1 -- 172.21.15.82:0/2825643427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 msgr2=0x562a8eddab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f5334c7b740 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 0x562a8eddab20 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f531c035000 tx=0x7f531c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:56:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f5334c7b740 1 -- 172.21.15.82:0/2825643427 shutdown_connections 2024-04-21T08:56:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f5334c7b740 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562a8ededb90 0x562a8edf0010 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f5334c7b740 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 0x562a8eddb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f5334c7b740 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4ea0 0x562a8eddb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f530affd640 0 client.18285 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f5334c7b740 1 --2- 172.21.15.82:0/2825643427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 0x562a8eddab20 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f5334c7b740 1 -- 172.21.15.82:0/2825643427 >> 172.21.15.82:0/2825643427 conn(0x562a8edcecb0 msgr2=0x562a8edd0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f5334c7b740 1 -- 172.21.15.82:0/2825643427 shutdown_connections 2024-04-21T08:56:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f5334c7b740 1 -- 172.21.15.82:0/2825643427 wait complete. 2024-04-21T08:56:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.319+0000 7f5334c7b740 20 client.18285 trim_cache size 0 max 16384 2024-04-21T08:56:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.336+0000 7f5334c7b740 1 Processor -- start 2024-04-21T08:56:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.336+0000 7f5334c7b740 1 -- start start 2024-04-21T08:56:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.336+0000 7f5334c7b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 0x562a8eddd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.336+0000 7f5334c7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4ea0 0x562a8ede9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.336+0000 7f5334c7b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 0x562a8edd08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.336+0000 7f5334c7b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f532c1b1770 con 0x562a8edd4580 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.336+0000 7f5334c7b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f532c1b3340 con 0x562a8eddda80 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.336+0000 7f5334c7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f532c1b1e40 con 0x562a8edd4ea0 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.336+0000 7f532b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 0x562a8eddd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.336+0000 7f532b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 0x562a8eddd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39134/0 (socket says 172.21.15.82:39134) 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.336+0000 7f532b7fe640 1 -- 172.21.15.82:0/2267534694 learned_addr learned my addr 172.21.15.82:0/2267534694 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f532b7fe640 1 -- 172.21.15.82:0/2267534694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4ea0 msgr2=0x562a8ede9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f532b7fe640 1 --2- 172.21.15.82:0/2267534694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4ea0 0x562a8ede9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f532b7fe640 1 -- 172.21.15.82:0/2267534694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 msgr2=0x562a8edd08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f532b7fe640 1 --2- 172.21.15.82:0/2267534694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 0x562a8edd08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f532b7fe640 1 -- 172.21.15.82:0/2267534694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a8edd4cd0 con 0x562a8edd4580 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f532b7fe640 1 --2- 172.21.15.82:0/2267534694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 0x562a8eddd250 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f531c02c170 tx=0x7f531c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7875c75e17127efa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f53117fa640 1 -- 172.21.15.82:0/2267534694 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c04e040 con 0x562a8edd4580 2024-04-21T08:56:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f53117fa640 1 -- 172.21.15.82:0/2267534694 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f531c003aa0 con 0x562a8edd4580 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f53117fa640 1 -- 172.21.15.82:0/2267534694 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c045070 con 0x562a8edd4580 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f5334c7b740 1 -- 172.21.15.82:0/2267534694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 msgr2=0x562a8eddd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f5334c7b740 1 --2- 172.21.15.82:0/2267534694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 0x562a8eddd250 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f531c02c170 tx=0x7f531c0095f0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f5334c7b740 1 -- 172.21.15.82:0/2267534694 shutdown_connections 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f5334c7b740 1 --2- 172.21.15.82:0/2267534694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 0x562a8edd08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.337+0000 7f5334c7b740 1 --2- 172.21.15.82:0/2267534694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4ea0 0x562a8ede9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 1 --2- 172.21.15.82:0/2267534694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4580 0x562a8eddd250 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 1 -- 172.21.15.82:0/2267534694 >> 172.21.15.82:0/2267534694 conn(0x562a8edcecb0 msgr2=0x562a8edd9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 1 -- 172.21.15.82:0/2267534694 shutdown_connections 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 1 -- 172.21.15.82:0/2267534694 wait complete. 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 1 -- start start 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 1 Processor -- start 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4580 0x562a8ede1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4ea0 0x562a8ede2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 0x562a8ee53150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f532c1b2c40 con 0x562a8edd4ea0 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f532c1b2540 con 0x562a8eddda80 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f532c1af740 con 0x562a8edd4580 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f5334c7b740 10 client.0 _ll_register_callbacks cb 0x7f531c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f532affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4ea0 0x562a8ede2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f532affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4ea0 0x562a8ede2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39146/0 (socket says 172.21.15.82:39146) 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f532affd640 1 -- 172.21.15.82:0/3300844371 learned_addr learned my addr 172.21.15.82:0/3300844371 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.338+0000 7f532b7fe640 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4580 0x562a8ede1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f532bfff640 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 0x562a8ee53150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f532affd640 1 -- 172.21.15.82:0/3300844371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4580 msgr2=0x562a8ede1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f532affd640 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4580 0x562a8ede1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f532affd640 1 -- 172.21.15.82:0/3300844371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 msgr2=0x562a8ee53150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f532affd640 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 0x562a8ee53150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f532affd640 1 -- 172.21.15.82:0/3300844371 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a8ee53690 con 0x562a8edd4ea0 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f532b7fe640 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4580 0x562a8ede1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f532affd640 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4ea0 0x562a8ede2250 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5320007dd0 tx=0x7f5320014c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3cfe7d69df9eac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f530affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f530affd640 1 -- 172.21.15.82:0/3300844371 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532001f020 con 0x562a8edd4ea0 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f530affd640 1 -- 172.21.15.82:0/3300844371 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9562+0+0 (secure 0 0 0) 0x7f5320013830 con 0x562a8edd4ea0 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f5334c7b740 1 -- 172.21.15.82:0/3300844371 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f531c003aa0 con 0x562a8edd4ea0 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f530affd640 1 -- 172.21.15.82:0/3300844371 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f532002a050 con 0x562a8edd4ea0 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f5334c7b740 10 client.18291 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f5334c7b740 1 -- 172.21.15.82:0/3300844371 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562a8ee54970 con 0x562a8edd4ea0 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f530affd640 1 -- 172.21.15.82:0/3300844371 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f531c003aa0 con 0x562a8edd4ea0 2024-04-21T08:56:06.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f530affd640 1 -- 172.21.15.82:0/3300844371 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f5320011040 con 0x562a8edd4ea0 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f5334c7b740 10 client.18291 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f5334c7b740 10 client.18291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f5334c7b740 20 client.18291 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f5334c7b740 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562a8ee55150 0x562a8ee57570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f5334c7b740 4 client.18291 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.339+0000 7f5334c7b740 1 -- 172.21.15.82:0/3300844371 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562a8ed40a70 con 0x562a8ee55150 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f532b7fe640 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562a8ee55150 0x562a8ee57570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f532b7fe640 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562a8ee55150 0x562a8ee57570 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ecb87606d1cdad1 server_cookie=b2272e9b0c2f8dbc in_seq=0 out_seq=0 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f530affd640 10 client.18291 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f530affd640 1 -- 172.21.15.82:0/3300844371 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562a8ed40a70 con 0x562a8ee55150 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f530affd640 10 client.18291 handle_command_reply: tid=0 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f5334c7b740 1 client.18291 shutdown 2024-04-21T08:56:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f5334c7b740 1 -- 172.21.15.82:0/3300844371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4ea0 msgr2=0x562a8ede2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f5334c7b740 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4ea0 0x562a8ede2250 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5320007dd0 tx=0x7f5320014c50 comp rx=0 tx=0).stop 2024-04-21T08:56:06.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f5334c7b740 1 -- 172.21.15.82:0/3300844371 shutdown_connections 2024-04-21T08:56:06.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f5334c7b740 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x562a8ee55150 0x562a8ee57570 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f5334c7b740 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a8eddda80 0x562a8ee53150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f5334c7b740 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8edd4ea0 0x562a8ede2250 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f530affd640 0 client.18291 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:06.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f5334c7b740 1 --2- 172.21.15.82:0/3300844371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562a8edd4580 0x562a8ede1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f5334c7b740 1 -- 172.21.15.82:0/3300844371 >> 172.21.15.82:0/3300844371 conn(0x562a8edcecb0 msgr2=0x562a8edd08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:06.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f5334c7b740 1 -- 172.21.15.82:0/3300844371 shutdown_connections 2024-04-21T08:56:06.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.340+0000 7f5334c7b740 1 -- 172.21.15.82:0/3300844371 wait complete. 2024-04-21T08:56:06.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5334c7b740 20 client.18291 trim_cache size 0 max 16384 2024-04-21T08:56:06.350 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:56:06.350 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:56:06.350 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:56:06.350 INFO:teuthology.orchestra.run.smithi082.stdout: "26275d45-e29c-4066-8739-3d5eaf545924": { 2024-04-21T08:56:06.350 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:56:06.350 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "26275d45-e29c-4066-8739-3d5eaf545924", 2024-04-21T08:56:06.350 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:56:06.350 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:56:06.350 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:56:06.350 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:56:06.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5332081640 1 -- 172.21.15.82:0/2261746021 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f53140a3d50 msgr2=0x7f53140a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5332081640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f53140a3d50 0x7f53140a6210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f531c036040 tx=0x7f531c02c710 comp rx=0 tx=0).stop 2024-04-21T08:56:06.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5332081640 1 -- 172.21.15.82:0/2261746021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f532c074cd0 msgr2=0x7f532c1a6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:06.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5332081640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f532c074cd0 0x7f532c1a6960 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f532400c970 tx=0x7f532400ce30 comp rx=0 tx=0).stop 2024-04-21T08:56:06.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5332081640 1 -- 172.21.15.82:0/2261746021 shutdown_connections 2024-04-21T08:56:06.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5332081640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f53140a3d50 0x7f53140a6210 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5332081640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f532c074cd0 0x7f532c1a6960 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5332081640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f532c107270 0x7f532c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5332081640 1 --2- 172.21.15.82:0/2261746021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f532c1021a0 0x7f532c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:06.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5332081640 1 -- 172.21.15.82:0/2261746021 >> 172.21.15.82:0/2261746021 conn(0x7f532c0793e0 msgr2=0x7f532c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:06.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5332081640 1 -- 172.21.15.82:0/2261746021 shutdown_connections 2024-04-21T08:56:06.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:06.341+0000 7f5332081640 1 -- 172.21.15.82:0/2261746021 wait complete. 2024-04-21T08:56:06.352 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:06 smithi082 ceph-mon[27967]: pgmap v279: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-21T08:56:06.353 INFO:tasks.cephfs.filesystem:scrub status for tag:26275d45-e29c-4066-8739-3d5eaf545924 - {'path': '/', 'tag': '26275d45-e29c-4066-8739-3d5eaf545924', 'options': 'recursive,force'} 2024-04-21T08:56:06.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:06 smithi134 ceph-mon[31772]: pgmap v279: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-21T08:56:06.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:06 smithi106 ceph-mon[31072]: pgmap v279: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-21T08:56:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:07 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:56:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:07 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:56:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:07 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:56:07.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:07 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:56:07.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:07 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:56:07.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:07 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:56:07.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:07 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:56:07.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:07 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:56:07.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:07 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:56:07.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:07 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:56:07.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:07 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:56:07.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:07 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:56:08.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:08 smithi134 ceph-mon[31772]: pgmap v280: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-21T08:56:08.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:08 smithi082 ceph-mon[27967]: pgmap v280: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-21T08:56:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:08 smithi106 ceph-mon[31072]: pgmap v280: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-21T08:56:09.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:09 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:56:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:09 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:56:09.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:09 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:56:10.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:10 smithi082 ceph-mon[27967]: pgmap v281: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-21T08:56:10.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:10 smithi082 ceph-mon[27967]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T08:56:10.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:10 smithi106 ceph-mon[31072]: pgmap v281: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-21T08:56:10.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:10 smithi106 ceph-mon[31072]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T08:56:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:10 smithi134 ceph-mon[31772]: pgmap v281: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-21T08:56:10.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:10 smithi134 ceph-mon[31772]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T08:56:12.528 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:12 smithi134 ceph-mon[31772]: pgmap v283: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-04-21T08:56:12.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:12 smithi082 ceph-mon[27967]: pgmap v283: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-04-21T08:56:12.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:12 smithi106 ceph-mon[31072]: pgmap v283: 129 pgs: 129 active+clean; 883 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-04-21T08:56:14.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:14 smithi082 ceph-mon[27967]: pgmap v284: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 883 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 204 B/s wr, 12 op/s 2024-04-21T08:56:14.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:14 smithi106 ceph-mon[31072]: pgmap v284: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 883 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 204 B/s wr, 12 op/s 2024-04-21T08:56:14.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:14 smithi134 ceph-mon[31772]: pgmap v284: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 883 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 204 B/s wr, 12 op/s 2024-04-21T08:56:16.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:16 smithi082 ceph-mon[27967]: pgmap v285: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 883 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 204 B/s wr, 12 op/s 2024-04-21T08:56:16.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:16 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:56:16.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:16 smithi106 ceph-mon[31072]: pgmap v285: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 883 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 204 B/s wr, 12 op/s 2024-04-21T08:56:16.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:16 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:56:16.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:16 smithi134 ceph-mon[31772]: pgmap v285: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 883 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 204 B/s wr, 12 op/s 2024-04-21T08:56:16.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:16 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:56:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:17 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:56:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:17 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:56:17.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:17 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:56:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:17 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:56:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:17 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:56:17.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:17 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:56:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:17 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:56:17.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:17 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:56:17.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:17 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:56:18.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:18 smithi082 ceph-mon[27967]: pgmap v286: 129 pgs: 20 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 44 active+clean; 883 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-21T08:56:18.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:18 smithi106 ceph-mon[31072]: pgmap v286: 129 pgs: 20 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 44 active+clean; 883 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-21T08:56:18.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:18 smithi134 ceph-mon[31772]: pgmap v286: 129 pgs: 20 active+clean+snaptrim, 65 active+clean+snaptrim_wait, 44 active+clean; 883 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-21T08:56:20.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:20 smithi082 ceph-mon[27967]: pgmap v287: 129 pgs: 19 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 52 active+clean; 883 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-21T08:56:20.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:20 smithi106 ceph-mon[31072]: pgmap v287: 129 pgs: 19 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 52 active+clean; 883 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-21T08:56:20.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:20 smithi134 ceph-mon[31772]: pgmap v287: 129 pgs: 19 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 52 active+clean; 883 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-21T08:56:22.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:22 smithi082 ceph-mon[27967]: pgmap v288: 129 pgs: 19 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 52 active+clean; 883 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 177 B/s wr, 9 op/s 2024-04-21T08:56:22.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:22 smithi106 ceph-mon[31072]: pgmap v288: 129 pgs: 19 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 52 active+clean; 883 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 177 B/s wr, 9 op/s 2024-04-21T08:56:22.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:22 smithi134 ceph-mon[31772]: pgmap v288: 129 pgs: 19 active+clean+snaptrim, 58 active+clean+snaptrim_wait, 52 active+clean; 883 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 177 B/s wr, 9 op/s 2024-04-21T08:56:24.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:24 smithi082 ceph-mon[27967]: pgmap v289: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-21T08:56:24.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:24 smithi106 ceph-mon[31072]: pgmap v289: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-21T08:56:24.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:24 smithi134 ceph-mon[31772]: pgmap v289: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-21T08:56:26.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:26 smithi082 ceph-mon[27967]: pgmap v290: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-21T08:56:26.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:26 smithi106 ceph-mon[31072]: pgmap v290: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-21T08:56:26.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:26 smithi134 ceph-mon[31772]: pgmap v290: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-21T08:56:28.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:28 smithi082 ceph-mon[27967]: pgmap v291: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-21T08:56:28.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:28 smithi106 ceph-mon[31072]: pgmap v291: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-21T08:56:28.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:28 smithi134 ceph-mon[31772]: pgmap v291: 129 pgs: 4 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 112 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-21T08:56:30.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:30 smithi082 ceph-mon[27967]: pgmap v292: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-21T08:56:30.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:30 smithi082 ceph-mon[27967]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T08:56:30.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:30 smithi106 ceph-mon[31072]: pgmap v292: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-21T08:56:30.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:30 smithi106 ceph-mon[31072]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T08:56:30.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:30 smithi134 ceph-mon[31772]: pgmap v292: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-21T08:56:30.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:30 smithi134 ceph-mon[31772]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T08:56:32.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:32 smithi082 ceph-mon[27967]: pgmap v294: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:32.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:32 smithi106 ceph-mon[31072]: pgmap v294: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:32.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:32 smithi134 ceph-mon[31772]: pgmap v294: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:34.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:34 smithi082 ceph-mon[27967]: pgmap v295: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:34 smithi106 ceph-mon[31072]: pgmap v295: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:34.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:34 smithi134 ceph-mon[31772]: pgmap v295: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:36.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:56:36.354 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:56:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.466+0000 7f9ce3032640 1 -- 172.21.15.82:0/1441648302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cdc1031c0 msgr2=0x7f9cdc1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.466+0000 7f9ce3032640 1 --2- 172.21.15.82:0/1441648302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cdc1031c0 0x7f9cdc1035a0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009900 tx=0x7f9cd002bd60 comp rx=0 tx=0).stop 2024-04-21T08:56:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.466+0000 7f9ce3032640 1 -- 172.21.15.82:0/1441648302 shutdown_connections 2024-04-21T08:56:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.466+0000 7f9ce3032640 1 --2- 172.21.15.82:0/1441648302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cdc10c500 0x7f9cdc10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.466+0000 7f9ce3032640 1 --2- 172.21.15.82:0/1441648302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cdc103ae0 0x7f9cdc10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.466+0000 7f9ce3032640 1 --2- 172.21.15.82:0/1441648302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cdc1031c0 0x7f9cdc1035a0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.466+0000 7f9ce3032640 1 -- 172.21.15.82:0/1441648302 >> 172.21.15.82:0/1441648302 conn(0x7f9cdc078000 msgr2=0x7f9cdc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.466+0000 7f9ce3032640 1 -- 172.21.15.82:0/1441648302 shutdown_connections 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.466+0000 7f9ce3032640 1 -- 172.21.15.82:0/1441648302 wait complete. 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.466+0000 7f9ce3032640 1 Processor -- start 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.466+0000 7f9ce3032640 1 -- start start 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce3032640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cdc1031c0 0x7f9cdc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce3032640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cdc103ae0 0x7f9cdc198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce3032640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cdc10c500 0x7f9cdc19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce3032640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc112170 con 0x7f9cdc1031c0 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce3032640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc111ff0 con 0x7f9cdc103ae0 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce3032640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc1122f0 con 0x7f9cdc10c500 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce0dce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cdc1031c0 0x7f9cdc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce0dce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cdc1031c0 0x7f9cdc198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59006/0 (socket says 172.21.15.82:59006) 2024-04-21T08:56:36.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce0dce640 1 -- 172.21.15.82:0/4084005762 learned_addr learned my addr 172.21.15.82:0/4084005762 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9cdbfff640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cdc103ae0 0x7f9cdc198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce0dce640 1 -- 172.21.15.82:0/4084005762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cdc10c500 msgr2=0x7f9cdc19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce15cf640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cdc10c500 0x7f9cdc19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cdc10c500 0x7f9cdc19cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce0dce640 1 -- 172.21.15.82:0/4084005762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cdc103ae0 msgr2=0x7f9cdc198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cdc103ae0 0x7f9cdc198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce0dce640 1 -- 172.21.15.82:0/4084005762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cdc19d680 con 0x7f9cdc1031c0 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9cdbfff640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cdc103ae0 0x7f9cdc198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cdc1031c0 0x7f9cdc198690 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0005940 tx=0x7f9cd002d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9cd9ffb640 1 -- 172.21.15.82:0/4084005762 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0043020 con 0x7f9cdc1031c0 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9cd9ffb640 1 -- 172.21.15.82:0/4084005762 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd002dd20 con 0x7f9cdc1031c0 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9ce3032640 1 -- 172.21.15.82:0/4084005762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cdc19d910 con 0x7f9cdc1031c0 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.467+0000 7f9cd9ffb640 1 -- 172.21.15.82:0/4084005762 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0003b80 con 0x7f9cdc1031c0 2024-04-21T08:56:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.468+0000 7f9ce3032640 1 -- 172.21.15.82:0/4084005762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cdc1a5200 con 0x7f9cdc1031c0 2024-04-21T08:56:36.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.470+0000 7f9cd9ffb640 1 -- 172.21.15.82:0/4084005762 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9cd0046020 con 0x7f9cdc1031c0 2024-04-21T08:56:36.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce5c19740 1 Processor -- start 2024-04-21T08:56:36.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce5c19740 1 -- start start 2024-04-21T08:56:36.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce5c19740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358580 0x55cb1a358960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce5c19740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 0x55cb1a361540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9cd9ffb640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9ccc0a3d20 0x7f9ccc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce5c19740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a361a80 0x55cb1a363e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce5c19740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a2bd1e0 con 0x55cb1a358ea0 2024-04-21T08:56:36.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce5c19740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a25a390 con 0x55cb1a361a80 2024-04-21T08:56:36.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce5c19740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a2b91a0 con 0x55cb1a358580 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce0dce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 0x55cb1a361540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce15cf640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9ccc0a3d20 0x7f9ccc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce0dce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 0x55cb1a361540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59014/0 (socket says 172.21.15.82:59014) 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce0dce640 1 -- 172.21.15.82:0/4134391855 learned_addr learned my addr 172.21.15.82:0/4134391855 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9cd9ffb640 1 -- 172.21.15.82:0/4084005762 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7f9cd00f2d60 con 0x7f9cdc1031c0 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce15cf640 1 --2- 172.21.15.82:0/4134391855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358580 0x55cb1a358960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9cdbfff640 1 --2- 172.21.15.82:0/4134391855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a361a80 0x55cb1a363e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce0dce640 1 -- 172.21.15.82:0/4134391855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358580 msgr2=0x55cb1a358960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/4134391855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358580 0x55cb1a358960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce0dce640 1 -- 172.21.15.82:0/4134391855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a361a80 msgr2=0x55cb1a363e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/4134391855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a361a80 0x55cb1a363e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9ce0dce640 1 -- 172.21.15.82:0/4134391855 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb1a0997d0 con 0x55cb1a358ea0 2024-04-21T08:56:36.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.471+0000 7f9cdbfff640 1 --2- 172.21.15.82:0/4134391855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a361a80 0x55cb1a363e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.472+0000 7f9ce15cf640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9ccc0a3d20 0x7f9ccc0a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4014580 tx=0x7f9cd4014980 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.472+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/4134391855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 0x55cb1a361540 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00be990 tx=0x7f9cd002bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d07de02cb1bb7d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.472+0000 7f9cc6ffd640 1 -- 172.21.15.82:0/4134391855 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd00be540 con 0x55cb1a358ea0 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.472+0000 7f9cc6ffd640 1 -- 172.21.15.82:0/4134391855 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd00bf100 con 0x55cb1a358ea0 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.472+0000 7f9cc6ffd640 1 -- 172.21.15.82:0/4134391855 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0039240 con 0x55cb1a358ea0 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.472+0000 7f9ce5c19740 1 -- 172.21.15.82:0/4134391855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 msgr2=0x55cb1a361540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.472+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/4134391855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 0x55cb1a361540 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00be990 tx=0x7f9cd002bd60 comp rx=0 tx=0).stop 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.472+0000 7f9ce5c19740 1 -- 172.21.15.82:0/4134391855 shutdown_connections 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.472+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/4134391855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a361a80 0x55cb1a363e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.472+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/4134391855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 0x55cb1a361540 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.472+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/4134391855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358580 0x55cb1a358960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9ce5c19740 1 -- 172.21.15.82:0/4134391855 >> 172.21.15.82:0/4134391855 conn(0x55cb1a352cb0 msgr2=0x55cb1a353090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9ce5c19740 1 -- 172.21.15.82:0/4134391855 shutdown_connections 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9ce5c19740 1 -- 172.21.15.82:0/4134391855 wait complete. 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9ce5c19740 1 -- start start 2024-04-21T08:56:36.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9ce5c19740 1 Processor -- start 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9ce5c19740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a358580 0x55cb1a35eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9ce5c19740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 0x55cb1a35f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9ce5c19740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a361a80 0x55cb1a35f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9ce5c19740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a364630 con 0x55cb1a358ea0 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9ce5c19740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a2762f0 con 0x55cb1a358580 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9ce5c19740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a3647b0 con 0x55cb1a361a80 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9ce5c19740 10 client.0 _ll_register_callbacks cb 0x55cb1a283580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.473+0000 7f9cdbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 0x55cb1a35f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cdbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 0x55cb1a35f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59028/0 (socket says 172.21.15.82:59028) 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cdbfff640 1 -- 172.21.15.82:0/2377874484 learned_addr learned my addr 172.21.15.82:0/2377874484 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9ce15cf640 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a361a80 0x55cb1a35f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a358580 0x55cb1a35eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cdbfff640 1 -- 172.21.15.82:0/2377874484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a361a80 msgr2=0x55cb1a35f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cdbfff640 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a361a80 0x55cb1a35f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cdbfff640 1 -- 172.21.15.82:0/2377874484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a358580 msgr2=0x55cb1a35eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cdbfff640 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a358580 0x55cb1a35eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cdbfff640 1 -- 172.21.15.82:0/2377874484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb19fcf3f0 con 0x55cb1a358ea0 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a358580 0x55cb1a35eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cdbfff640 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 0x55cb1a35f070 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80106a0 tx=0x7f9cc8010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ba146cf63f6e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cbbfff640 1 -- 172.21.15.82:0/2377874484 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8010ea0 con 0x55cb1a358ea0 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cbbfff640 1 -- 172.21.15.82:0/2377874484 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7f9cc800ead0 con 0x55cb1a358ea0 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9ce5c19740 1 -- 172.21.15.82:0/2377874484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd00bf100 con 0x55cb1a358ea0 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cbbfff640 1 -- 172.21.15.82:0/2377874484 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9cc8024050 con 0x55cb1a358ea0 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9ce5c19740 10 client.18303 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9ce5c19740 1 -- 172.21.15.82:0/2377874484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cb19f8f0f0 con 0x55cb1a358ea0 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.474+0000 7f9cbbfff640 1 -- 172.21.15.82:0/2377874484 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd00bf100 con 0x55cb1a358ea0 2024-04-21T08:56:36.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.475+0000 7f9cbbfff640 1 -- 172.21.15.82:0/2377874484 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f9cc800f040 con 0x55cb1a358ea0 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.475+0000 7f9ce5c19740 10 client.18303 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.475+0000 7f9ce5c19740 10 client.18303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.475+0000 7f9ce5c19740 20 client.18303 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.475+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55cb1a3718e0 0x55cb1a373d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.475+0000 7f9ce5c19740 4 client.18303 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.475+0000 7f9ce5c19740 1 -- 172.21.15.82:0/2377874484 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb1a2cac00 con 0x55cb1a3718e0 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.475+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55cb1a3718e0 0x55cb1a373d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.475+0000 7f9cbbfff640 10 client.18303 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.475+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55cb1a3718e0 0x55cb1a373d60 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e3e619d29fd374c server_cookie=50cf6053fa3e2dc5 in_seq=0 out_seq=0 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9cbbfff640 1 -- 172.21.15.82:0/2377874484 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb1a2cac00 con 0x55cb1a3718e0 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9cbbfff640 10 client.18303 handle_command_reply: tid=0 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9ce5c19740 1 client.18303 shutdown 2024-04-21T08:56:36.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9ce5c19740 1 -- 172.21.15.82:0/2377874484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 msgr2=0x55cb1a35f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 0x55cb1a35f070 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80106a0 tx=0x7f9cc8010b60 comp rx=0 tx=0).stop 2024-04-21T08:56:36.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9ce5c19740 1 -- 172.21.15.82:0/2377874484 shutdown_connections 2024-04-21T08:56:36.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55cb1a3718e0 0x55cb1a373d60 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a361a80 0x55cb1a35f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9cbbfff640 0 client.18303 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:36.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358ea0 0x55cb1a35f070 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/2377874484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a358580 0x55cb1a35eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9ce5c19740 1 -- 172.21.15.82:0/2377874484 >> 172.21.15.82:0/2377874484 conn(0x55cb1a352cb0 msgr2=0x55cb1a354a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9ce5c19740 1 -- 172.21.15.82:0/2377874484 shutdown_connections 2024-04-21T08:56:36.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9ce5c19740 1 -- 172.21.15.82:0/2377874484 wait complete. 2024-04-21T08:56:36.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.476+0000 7f9ce5c19740 20 client.18303 trim_cache size 0 max 16384 2024-04-21T08:56:36.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce5c19740 1 Processor -- start 2024-04-21T08:56:36.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce5c19740 1 -- start start 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce5c19740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a358580 0x55cb1a361260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce5c19740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358ea0 0x55cb1a36d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce5c19740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a361a80 0x55cb1a351bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce5c19740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc1af4a0 con 0x55cb1a361a80 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce5c19740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc1b1070 con 0x55cb1a358580 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce5c19740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc1afb70 con 0x55cb1a358ea0 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce15cf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a361a80 0x55cb1a351bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce15cf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a361a80 0x55cb1a351bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59042/0 (socket says 172.21.15.82:59042) 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce15cf640 1 -- 172.21.15.82:0/3435430729 learned_addr learned my addr 172.21.15.82:0/3435430729 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/3435430729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a358580 0x55cb1a361260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9cdbfff640 1 --2- 172.21.15.82:0/3435430729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358ea0 0x55cb1a36d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce15cf640 1 -- 172.21.15.82:0/3435430729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358ea0 msgr2=0x55cb1a36d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce15cf640 1 --2- 172.21.15.82:0/3435430729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358ea0 0x55cb1a36d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce15cf640 1 -- 172.21.15.82:0/3435430729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a358580 msgr2=0x55cb1a361260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce15cf640 1 --2- 172.21.15.82:0/3435430729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a358580 0x55cb1a361260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce15cf640 1 -- 172.21.15.82:0/3435430729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb1a35d080 con 0x55cb1a361a80 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.494+0000 7f9ce15cf640 1 --2- 172.21.15.82:0/3435430729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a361a80 0x55cb1a351bf0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4014070 tx=0x7f9cd401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754c2c71e2c4ef84 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9cc67fc640 1 -- 172.21.15.82:0/3435430729 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4021680 con 0x55cb1a361a80 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9cc67fc640 1 -- 172.21.15.82:0/3435430729 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd4021cd0 con 0x55cb1a361a80 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9cc67fc640 1 -- 172.21.15.82:0/3435430729 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4012d70 con 0x55cb1a361a80 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9ce5c19740 1 -- 172.21.15.82:0/3435430729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a361a80 msgr2=0x55cb1a351bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/3435430729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a361a80 0x55cb1a351bf0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4014070 tx=0x7f9cd401a000 comp rx=0 tx=0).stop 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9ce5c19740 1 -- 172.21.15.82:0/3435430729 shutdown_connections 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/3435430729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a361a80 0x55cb1a351bf0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/3435430729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358ea0 0x55cb1a36d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/3435430729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a358580 0x55cb1a361260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9ce5c19740 1 -- 172.21.15.82:0/3435430729 >> 172.21.15.82:0/3435430729 conn(0x55cb1a352cb0 msgr2=0x55cb1a3514e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9ce5c19740 1 -- 172.21.15.82:0/3435430729 shutdown_connections 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9ce5c19740 1 -- 172.21.15.82:0/3435430729 wait complete. 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.495+0000 7f9ce5c19740 1 -- start start 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce5c19740 1 Processor -- start 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce5c19740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358580 0x55cb1a369d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce5c19740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358ea0 0x55cb1a35b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce5c19740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a361a80 0x55cb1a35bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce5c19740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc1b0970 con 0x55cb1a358580 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce5c19740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc1b0270 con 0x55cb1a361a80 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce5c19740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc1ad470 con 0x55cb1a358ea0 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce5c19740 10 client.0 _ll_register_callbacks cb 0x7f9cc8009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce0dce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358580 0x55cb1a369d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce0dce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358580 0x55cb1a369d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59052/0 (socket says 172.21.15.82:59052) 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce0dce640 1 -- 172.21.15.82:0/711098296 learned_addr learned my addr 172.21.15.82:0/711098296 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:36.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9cdbfff640 1 --2- 172.21.15.82:0/711098296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358ea0 0x55cb1a35b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce0dce640 1 -- 172.21.15.82:0/711098296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358ea0 msgr2=0x55cb1a35b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/711098296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358ea0 0x55cb1a35b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce0dce640 1 -- 172.21.15.82:0/711098296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a361a80 msgr2=0x55cb1a35bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/711098296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a361a80 0x55cb1a35bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce0dce640 1 -- 172.21.15.82:0/711098296 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb1a35c3c0 con 0x55cb1a358580 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9ce0dce640 1 --2- 172.21.15.82:0/711098296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358580 0x55cb1a369d30 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00be700 tx=0x7f9cd00df1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3c4a0b8d524d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.496+0000 7f9cbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9cbbfff640 1 -- 172.21.15.82:0/711098296 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd00be360 con 0x55cb1a358580 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9cbbfff640 1 -- 172.21.15.82:0/711098296 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7f9cd00bcd00 con 0x55cb1a358580 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9ce5c19740 1 -- 172.21.15.82:0/711098296 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd4021cd0 con 0x55cb1a358580 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9cbbfff640 1 -- 172.21.15.82:0/711098296 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9cd00bf100 con 0x55cb1a358580 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9ce5c19740 10 client.18309 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9ce5c19740 1 -- 172.21.15.82:0/711098296 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cb1a3d7ff0 con 0x55cb1a358580 2024-04-21T08:56:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9cbbfff640 1 -- 172.21.15.82:0/711098296 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd4021cd0 con 0x55cb1a358580 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9cbbfff640 1 -- 172.21.15.82:0/711098296 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f9cd00bb070 con 0x55cb1a358580 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9ce5c19740 10 client.18309 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9ce5c19740 10 client.18309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9ce5c19740 20 client.18309 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/711098296 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55cb1a3d8540 0x55cb1a3da960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9ce5c19740 4 client.18309 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9ce5c19740 1 -- 172.21.15.82:0/711098296 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cb1a2c4a70 con 0x55cb1a3d8540 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.497+0000 7f9cdbfff640 1 --2- 172.21.15.82:0/711098296 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55cb1a3d8540 0x55cb1a3da960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9cbbfff640 10 client.18309 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9cdbfff640 1 --2- 172.21.15.82:0/711098296 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55cb1a3d8540 0x55cb1a3da960 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf2decdc63947e6 server_cookie=2f0beec134a07a58 in_seq=0 out_seq=0 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9cbbfff640 1 -- 172.21.15.82:0/711098296 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cb1a2c4a70 con 0x55cb1a3d8540 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9cbbfff640 10 client.18309 handle_command_reply: tid=0 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9ce5c19740 1 client.18309 shutdown 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9ce5c19740 1 -- 172.21.15.82:0/711098296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358580 msgr2=0x55cb1a369d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/711098296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358580 0x55cb1a369d30 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00be700 tx=0x7f9cd00df1b0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9ce5c19740 1 -- 172.21.15.82:0/711098296 shutdown_connections 2024-04-21T08:56:36.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/711098296 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x55cb1a3d8540 0x55cb1a3da960 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/711098296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a361a80 0x55cb1a35bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/711098296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb1a358ea0 0x55cb1a35b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9cbbfff640 0 client.18309 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:36.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9ce5c19740 1 --2- 172.21.15.82:0/711098296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb1a358580 0x55cb1a369d30 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9ce5c19740 1 -- 172.21.15.82:0/711098296 >> 172.21.15.82:0/711098296 conn(0x55cb1a352cb0 msgr2=0x55cb1a351d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9ce5c19740 1 -- 172.21.15.82:0/711098296 shutdown_connections 2024-04-21T08:56:36.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9ce5c19740 1 -- 172.21.15.82:0/711098296 wait complete. 2024-04-21T08:56:36.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.498+0000 7f9ce5c19740 20 client.18309 trim_cache size 0 max 16384 2024-04-21T08:56:36.505 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:56:36.506 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:56:36.506 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:56:36.506 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:56:36.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.499+0000 7f9ce3032640 1 -- 172.21.15.82:0/4084005762 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9ccc0a3d20 msgr2=0x7f9ccc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.499+0000 7f9ce3032640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9ccc0a3d20 0x7f9ccc0a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4014580 tx=0x7f9cd4014980 comp rx=0 tx=0).stop 2024-04-21T08:56:36.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.499+0000 7f9ce3032640 1 -- 172.21.15.82:0/4084005762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cdc1031c0 msgr2=0x7f9cdc198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.499+0000 7f9ce3032640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cdc1031c0 0x7f9cdc198690 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0005940 tx=0x7f9cd002d920 comp rx=0 tx=0).stop 2024-04-21T08:56:36.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.499+0000 7f9ce3032640 1 -- 172.21.15.82:0/4084005762 shutdown_connections 2024-04-21T08:56:36.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.499+0000 7f9ce3032640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cdc10c500 0x7f9cdc19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.499+0000 7f9ce3032640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cdc103ae0 0x7f9cdc198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.499+0000 7f9ce3032640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f9ccc0a3d20 0x7f9ccc0a61e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.499+0000 7f9ce3032640 1 --2- 172.21.15.82:0/4084005762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cdc1031c0 0x7f9cdc198690 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.499+0000 7f9ce3032640 1 -- 172.21.15.82:0/4084005762 >> 172.21.15.82:0/4084005762 conn(0x7f9cdc078000 msgr2=0x7f9cdc0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.499+0000 7f9ce3032640 1 -- 172.21.15.82:0/4084005762 shutdown_connections 2024-04-21T08:56:36.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.499+0000 7f9ce3032640 1 -- 172.21.15.82:0/4084005762 wait complete. 2024-04-21T08:56:36.509 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:56:36.510 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:56:36.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.607+0000 7fe796b3d640 1 -- 172.21.15.82:0/3347759110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7901011a0 msgr2=0x7fe790101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.607+0000 7fe796b3d640 1 --2- 172.21.15.82:0/3347759110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7901011a0 0x7fe790101620 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009900 tx=0x7fe77c02bc80 comp rx=0 tx=0).stop 2024-04-21T08:56:36.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.607+0000 7fe796b3d640 1 -- 172.21.15.82:0/3347759110 shutdown_connections 2024-04-21T08:56:36.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.607+0000 7fe796b3d640 1 --2- 172.21.15.82:0/3347759110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe790101b60 0x7fe79010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.607+0000 7fe796b3d640 1 --2- 172.21.15.82:0/3347759110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7901011a0 0x7fe790101620 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.607+0000 7fe796b3d640 1 --2- 172.21.15.82:0/3347759110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe790105ee0 0x7fe7901062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.607+0000 7fe796b3d640 1 -- 172.21.15.82:0/3347759110 >> 172.21.15.82:0/3347759110 conn(0x7fe790077c20 msgr2=0x7fe790078030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.607+0000 7fe796b3d640 1 -- 172.21.15.82:0/3347759110 shutdown_connections 2024-04-21T08:56:36.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.607+0000 7fe796b3d640 1 -- 172.21.15.82:0/3347759110 wait complete. 2024-04-21T08:56:36.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe796b3d640 1 Processor -- start 2024-04-21T08:56:36.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe796b3d640 1 -- start start 2024-04-21T08:56:36.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe796b3d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7901011a0 0x7fe79019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe796b3d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe790101b60 0x7fe79019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe796b3d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe790105ee0 0x7fe790198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe796b3d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe790111150 con 0x7fe7901011a0 2024-04-21T08:56:36.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe796b3d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe790110fd0 con 0x7fe790101b60 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe796b3d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7901112d0 con 0x7fe790105ee0 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe7948d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7901011a0 0x7fe79019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe7948d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7901011a0 0x7fe79019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59074/0 (socket says 172.21.15.82:59074) 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe7948d9640 1 -- 172.21.15.82:0/1775435696 learned_addr learned my addr 172.21.15.82:0/1775435696 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe7948d9640 1 -- 172.21.15.82:0/1775435696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe790105ee0 msgr2=0x7fe790198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe7950da640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe790105ee0 0x7fe790198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.608+0000 7fe783fff640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe790101b60 0x7fe79019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.609+0000 7fe7948d9640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe790105ee0 0x7fe790198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.609+0000 7fe7948d9640 1 -- 172.21.15.82:0/1775435696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe790101b60 msgr2=0x7fe79019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.609+0000 7fe7948d9640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe790101b60 0x7fe79019e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.609+0000 7fe7948d9640 1 -- 172.21.15.82:0/1775435696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe790198d80 con 0x7fe7901011a0 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.609+0000 7fe7950da640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe790105ee0 0x7fe790198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.609+0000 7fe7948d9640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7901011a0 0x7fe79019e400 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe78400eab0 tx=0x7fe78400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.609+0000 7fe78e7fc640 1 -- 172.21.15.82:0/1775435696 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78400ce40 con 0x7fe7901011a0 2024-04-21T08:56:36.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.609+0000 7fe78e7fc640 1 -- 172.21.15.82:0/1775435696 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe784011920 con 0x7fe7901011a0 2024-04-21T08:56:36.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.609+0000 7fe796b3d640 1 -- 172.21.15.82:0/1775435696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe790199070 con 0x7fe7901011a0 2024-04-21T08:56:36.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.609+0000 7fe78e7fc640 1 -- 172.21.15.82:0/1775435696 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784010a30 con 0x7fe7901011a0 2024-04-21T08:56:36.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.609+0000 7fe796b3d640 1 -- 172.21.15.82:0/1775435696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7901a52c0 con 0x7fe7901011a0 2024-04-21T08:56:36.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.610+0000 7fe796b3d640 1 -- 172.21.15.82:0/1775435696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe790069230 con 0x7fe7901011a0 2024-04-21T08:56:36.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.611+0000 7fe78e7fc640 1 -- 172.21.15.82:0/1775435696 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe784010bd0 con 0x7fe7901011a0 2024-04-21T08:56:36.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.612+0000 7fe78e7fc640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7740a3d20 0x7fe7740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.612+0000 7fe783fff640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7740a3d20 0x7fe7740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.612+0000 7fe78e7fc640 1 -- 172.21.15.82:0/1775435696 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7fe7840d12a0 con 0x7fe7901011a0 2024-04-21T08:56:36.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.612+0000 7fe783fff640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7740a3d20 0x7fe7740a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe77c036040 tx=0x7fe77c000bc0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.615+0000 7fe78e7fc640 1 -- 172.21.15.82:0/1775435696 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe784016030 con 0x7fe7901011a0 2024-04-21T08:56:36.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.833+0000 7fe796b3d640 1 -- 172.21.15.82:0/1775435696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe790199ae0 con 0x7fe7901011a0 2024-04-21T08:56:36.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.833+0000 7fe78e7fc640 1 -- 172.21.15.82:0/1775435696 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11949 (secure 0 0 0) 0x7fe784021350 con 0x7fe7901011a0 2024-04-21T08:56:36.834 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:56:36.834 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:55:27.214396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24409,24476,24481,24509]},"id":1}]} 2024-04-21T08:56:36.834 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 44 2024-04-21T08:56:36.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.836+0000 7fe796b3d640 1 -- 172.21.15.82:0/1775435696 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7740a3d20 msgr2=0x7fe7740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.836+0000 7fe796b3d640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7740a3d20 0x7fe7740a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe77c036040 tx=0x7fe77c000bc0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.836+0000 7fe796b3d640 1 -- 172.21.15.82:0/1775435696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7901011a0 msgr2=0x7fe79019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.836+0000 7fe796b3d640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7901011a0 0x7fe79019e400 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe78400eab0 tx=0x7fe78400ef70 comp rx=0 tx=0).stop 2024-04-21T08:56:36.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.836+0000 7fe796b3d640 1 -- 172.21.15.82:0/1775435696 shutdown_connections 2024-04-21T08:56:36.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.836+0000 7fe796b3d640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe790105ee0 0x7fe790198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.836+0000 7fe796b3d640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe790101b60 0x7fe79019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.836+0000 7fe796b3d640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe7740a3d20 0x7fe7740a61e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.836+0000 7fe796b3d640 1 --2- 172.21.15.82:0/1775435696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7901011a0 0x7fe79019e400 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.836+0000 7fe796b3d640 1 -- 172.21.15.82:0/1775435696 >> 172.21.15.82:0/1775435696 conn(0x7fe790077c20 msgr2=0x7fe790104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.836+0000 7fe796b3d640 1 -- 172.21.15.82:0/1775435696 shutdown_connections 2024-04-21T08:56:36.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.836+0000 7fe796b3d640 1 -- 172.21.15.82:0/1775435696 wait complete. 2024-04-21T08:56:36.845 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:36 smithi082 ceph-mon[27967]: pgmap v296: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:36.847 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:56:36.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:36 smithi106 ceph-mon[31072]: pgmap v296: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:36.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.947+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/3333849361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea81016d0 msgr2=0x7f6ea810be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.947+0000 7f6eaf3f4640 1 --2- 172.21.15.82:0/3333849361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea81016d0 0x7f6ea810be30 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009900 tx=0x7f6e9802bc30 comp rx=0 tx=0).stop 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.947+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/3333849361 shutdown_connections 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.947+0000 7f6eaf3f4640 1 --2- 172.21.15.82:0/3333849361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ea810c4f0 0x7f6ea810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.947+0000 7f6eaf3f4640 1 --2- 172.21.15.82:0/3333849361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea81016d0 0x7f6ea810be30 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.947+0000 7f6eaf3f4640 1 --2- 172.21.15.82:0/3333849361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ea8100d20 0x7f6ea8101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.947+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/3333849361 >> 172.21.15.82:0/3333849361 conn(0x7f6ea8076150 msgr2=0x7f6ea8076560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.947+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/3333849361 shutdown_connections 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.947+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/3333849361 wait complete. 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.947+0000 7f6eaf3f4640 1 Processor -- start 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.947+0000 7f6eaf3f4640 1 -- start start 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6eaf3f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea8100d20 0x7f6ea8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6eaf3f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ea810c4f0 0x7f6ea8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6eaf3f4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ea819d000 0x7f6ea819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6eaf3f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8112180 con 0x7f6ea8100d20 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6eaf3f4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8112000 con 0x7f6ea819d000 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6eaf3f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8112300 con 0x7f6ea810c4f0 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6ead190640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea8100d20 0x7f6ea8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6ead190640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea8100d20 0x7f6ea8198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59094/0 (socket says 172.21.15.82:59094) 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6ead190640 1 -- 172.21.15.82:0/1883830000 learned_addr learned my addr 172.21.15.82:0/1883830000 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6eac98f640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ea810c4f0 0x7f6ea8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6ead991640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ea819d000 0x7f6ea819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6ead190640 1 -- 172.21.15.82:0/1883830000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ea810c4f0 msgr2=0x7f6ea8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6ead190640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ea810c4f0 0x7f6ea8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6ead190640 1 -- 172.21.15.82:0/1883830000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ea819d000 msgr2=0x7f6ea819d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6ead190640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ea819d000 0x7f6ea819d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6ead190640 1 -- 172.21.15.82:0/1883830000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea819daf0 con 0x7f6ea8100d20 2024-04-21T08:56:36.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6ead991640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ea819d000 0x7f6ea819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:36.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6ead190640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea8100d20 0x7f6ea8198740 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400d950 tx=0x7f6ea400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6e9e7fc640 1 -- 172.21.15.82:0/1883830000 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4017070 con 0x7f6ea8100d20 2024-04-21T08:56:36.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6e9e7fc640 1 -- 172.21.15.82:0/1883830000 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea4009c30 con 0x7f6ea8100d20 2024-04-21T08:56:36.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/1883830000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea819dde0 con 0x7f6ea8100d20 2024-04-21T08:56:36.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6e9e7fc640 1 -- 172.21.15.82:0/1883830000 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400f620 con 0x7f6ea8100d20 2024-04-21T08:56:36.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.948+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/1883830000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea81a55a0 con 0x7f6ea8100d20 2024-04-21T08:56:36.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.950+0000 7f6e9e7fc640 1 -- 172.21.15.82:0/1883830000 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ea4010460 con 0x7f6ea8100d20 2024-04-21T08:56:36.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6eb1fec740 1 Processor -- start 2024-04-21T08:56:36.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6eb1fec740 1 -- start start 2024-04-21T08:56:36.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6eb1fec740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b7c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6eb1fec740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 0x562155b85820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6eb1fec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b85e40 0x562155b862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6eb1fec740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562155ae11f0 con 0x562155b7cce0 2024-04-21T08:56:36.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6eb1fec740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562155a7e4b0 con 0x562155b7c330 2024-04-21T08:56:36.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6eb1fec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562155add1b0 con 0x562155b85e40 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6e9e7fc640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6e940a3d20 0x7f6e940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6ead190640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 0x562155b85820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6ead190640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 0x562155b85820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59106/0 (socket says 172.21.15.82:59106) 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6ead190640 1 -- 172.21.15.82:0/3859550293 learned_addr learned my addr 172.21.15.82:0/3859550293 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6ead991640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6e940a3d20 0x7f6e940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6eac98f640 1 --2- 172.21.15.82:0/3859550293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b85e40 0x562155b862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.951+0000 7f6ead991640 1 --2- 172.21.15.82:0/3859550293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b7c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.952+0000 7f6e9e7fc640 1 -- 172.21.15.82:0/1883830000 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7f6ea40d0850 con 0x7f6ea8100d20 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.952+0000 7f6ead190640 1 -- 172.21.15.82:0/3859550293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b85e40 msgr2=0x562155b862c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.952+0000 7f6ead190640 1 --2- 172.21.15.82:0/3859550293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b85e40 0x562155b862c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.952+0000 7f6ead190640 1 -- 172.21.15.82:0/3859550293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 msgr2=0x562155b7c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.952+0000 7f6ead190640 1 --2- 172.21.15.82:0/3859550293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b7c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.952+0000 7f6ead190640 1 -- 172.21.15.82:0/3859550293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562155a701b0 con 0x562155b7cce0 2024-04-21T08:56:36.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.952+0000 7f6eac98f640 1 --2- 172.21.15.82:0/3859550293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b85e40 0x562155b862c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.952+0000 7f6ead991640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6e940a3d20 0x7f6e940a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0005fd0 tx=0x7f6ea0009b10 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.952+0000 7f6ead190640 1 --2- 172.21.15.82:0/3859550293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 0x562155b85820 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40987f0 tx=0x7f6ea4010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d87e35312d6487 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.952+0000 7f6e937fe640 1 -- 172.21.15.82:0/3859550293 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4020070 con 0x562155b7cce0 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.952+0000 7f6e937fe640 1 -- 172.21.15.82:0/3859550293 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea409cae0 con 0x562155b7cce0 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.952+0000 7f6e937fe640 1 -- 172.21.15.82:0/3859550293 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4098a50 con 0x562155b7cce0 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.953+0000 7f6eb1fec740 1 -- 172.21.15.82:0/3859550293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 msgr2=0x562155b85820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.953+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/3859550293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 0x562155b85820 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40987f0 tx=0x7f6ea4010cb0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.953+0000 7f6eb1fec740 1 -- 172.21.15.82:0/3859550293 shutdown_connections 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.953+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/3859550293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b85e40 0x562155b862c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.953+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/3859550293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 0x562155b85820 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.953+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/3859550293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b7c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.953+0000 7f6eb1fec740 1 -- 172.21.15.82:0/3859550293 >> 172.21.15.82:0/3859550293 conn(0x562155b76ba0 msgr2=0x562155b76f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.953+0000 7f6eb1fec740 1 -- 172.21.15.82:0/3859550293 shutdown_connections 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.953+0000 7f6eb1fec740 1 -- 172.21.15.82:0/3859550293 wait complete. 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.953+0000 7f6eb1fec740 1 -- start start 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.953+0000 7f6eb1fec740 1 Processor -- start 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6eb1fec740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b8c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6eb1fec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b7cce0 0x562155b895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6eb1fec740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 0x562155b89b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6eb1fec740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562155b8cd00 con 0x562155b85e40 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6eb1fec740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562155a9a500 con 0x562155b7c330 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6eb1fec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562155b8ce80 con 0x562155b7cce0 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6eb1fec740 10 client.0 _ll_register_callbacks cb 0x562155a68b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6ead991640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 0x562155b89b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6ead991640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 0x562155b89b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59116/0 (socket says 172.21.15.82:59116) 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6ead991640 1 -- 172.21.15.82:0/1569085730 learned_addr learned my addr 172.21.15.82:0/1569085730 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6eac98f640 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b7cce0 0x562155b895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6ead190640 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b8c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6ead991640 1 -- 172.21.15.82:0/1569085730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b7cce0 msgr2=0x562155b895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6ead991640 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b7cce0 0x562155b895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6ead991640 1 -- 172.21.15.82:0/1569085730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 msgr2=0x562155b8c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6ead991640 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b8c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6ead991640 1 -- 172.21.15.82:0/1569085730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621557f33f0 con 0x562155b85e40 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6ead991640 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 0x562155b89b10 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0011f60 tx=0x7f6ea0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9284f53a6a67e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6e90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6e90ff9640 1 -- 172.21.15.82:0/1569085730 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0025070 con 0x562155b85e40 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6e90ff9640 1 -- 172.21.15.82:0/1569085730 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7f6ea0004d20 con 0x562155b85e40 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1569085730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ea409cae0 con 0x562155b85e40 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6e90ff9640 1 -- 172.21.15.82:0/1569085730 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6ea0030050 con 0x562155b85e40 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6eb1fec740 10 client.18327 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.954+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1569085730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5621558bd7d0 con 0x562155b85e40 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.955+0000 7f6e90ff9640 1 -- 172.21.15.82:0/1569085730 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea409cae0 con 0x562155b85e40 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.955+0000 7f6e90ff9640 1 -- 172.21.15.82:0/1569085730 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f6ea001c050 con 0x562155b85e40 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.955+0000 7f6eb1fec740 10 client.18327 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.955+0000 7f6eb1fec740 10 client.18327 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.955+0000 7f6eb1fec740 20 client.18327 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.955+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562155b99f90 0x562155b9c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.955+0000 7f6eb1fec740 4 client.18327 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.955+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1569085730 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562155a80930 con 0x562155b99f90 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.955+0000 7f6ead190640 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562155b99f90 0x562155b9c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6e90ff9640 10 client.18327 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6ead190640 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562155b99f90 0x562155b9c430 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=17e1365c25eb6663 server_cookie=fc9278dff63565f6 in_seq=0 out_seq=0 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6e90ff9640 1 -- 172.21.15.82:0/1569085730 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562155a80930 con 0x562155b99f90 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6e90ff9640 10 client.18327 handle_command_reply: tid=0 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6eb1fec740 1 client.18327 shutdown 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1569085730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 msgr2=0x562155b89b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 0x562155b89b10 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0011f60 tx=0x7f6ea0019000 comp rx=0 tx=0).stop 2024-04-21T08:56:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1569085730 shutdown_connections 2024-04-21T08:56:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562155b99f90 0x562155b9c430 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 0x562155b89b10 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6e90ff9640 0 client.18327 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:56:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b7cce0 0x562155b895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/1569085730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b8c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.956+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1569085730 >> 172.21.15.82:0/1569085730 conn(0x562155b76ba0 msgr2=0x562155b788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.957+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1569085730 shutdown_connections 2024-04-21T08:56:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.957+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1569085730 wait complete. 2024-04-21T08:56:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.957+0000 7f6eb1fec740 20 client.18327 trim_cache size 0 max 16384 2024-04-21T08:56:36.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6eb1fec740 1 Processor -- start 2024-04-21T08:56:36.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6eb1fec740 1 -- start start 2024-04-21T08:56:36.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6eb1fec740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b87230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6eb1fec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b7cce0 0x562155b95c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6eb1fec740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 0x562155b8bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6eb1fec740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ad2d0 con 0x562155b85e40 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6eb1fec740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81aeea0 con 0x562155b7c330 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6eb1fec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ad9a0 con 0x562155b7cce0 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6ead991640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 0x562155b8bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6ead991640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 0x562155b8bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59122/0 (socket says 172.21.15.82:59122) 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6ead991640 1 -- 172.21.15.82:0/74403834 learned_addr learned my addr 172.21.15.82:0/74403834 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6ead991640 1 -- 172.21.15.82:0/74403834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b7cce0 msgr2=0x562155b95c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6ead991640 1 --2- 172.21.15.82:0/74403834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b7cce0 0x562155b95c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6ead991640 1 -- 172.21.15.82:0/74403834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 msgr2=0x562155b87230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6ead991640 1 --2- 172.21.15.82:0/74403834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b87230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6ead991640 1 -- 172.21.15.82:0/74403834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562155b89e80 con 0x562155b85e40 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.970+0000 7f6ead991640 1 --2- 172.21.15.82:0/74403834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 0x562155b8bce0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009bf0 tx=0x7f6ea0011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175b016252042795 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6e92ffd640 1 -- 172.21.15.82:0/74403834 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea002d070 con 0x562155b85e40 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6e92ffd640 1 -- 172.21.15.82:0/74403834 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea000fb70 con 0x562155b85e40 2024-04-21T08:56:36.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6e92ffd640 1 -- 172.21.15.82:0/74403834 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0025070 con 0x562155b85e40 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6eb1fec740 1 -- 172.21.15.82:0/74403834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 msgr2=0x562155b8bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/74403834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 0x562155b8bce0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009bf0 tx=0x7f6ea0011d70 comp rx=0 tx=0).stop 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6eb1fec740 1 -- 172.21.15.82:0/74403834 shutdown_connections 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/74403834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b85e40 0x562155b8bce0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/74403834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b7cce0 0x562155b95c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/74403834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b87230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6eb1fec740 1 -- 172.21.15.82:0/74403834 >> 172.21.15.82:0/74403834 conn(0x562155b76ba0 msgr2=0x562155b75c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6eb1fec740 1 -- 172.21.15.82:0/74403834 shutdown_connections 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6eb1fec740 1 -- 172.21.15.82:0/74403834 wait complete. 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.971+0000 7f6eb1fec740 1 -- start start 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eb1fec740 1 Processor -- start 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eb1fec740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b8b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eb1fec740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 0x562155b8b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eb1fec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b85e40 0x562155b8be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eb1fec740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ae7a0 con 0x562155b7cce0 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eb1fec740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ae0a0 con 0x562155b7c330 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eb1fec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ab2a0 con 0x562155b85e40 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eb1fec740 10 client.0 _ll_register_callbacks cb 0x7f6ea0011e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eac98f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 0x562155b8b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eac98f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 0x562155b8b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59130/0 (socket says 172.21.15.82:59130) 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eac98f640 1 -- 172.21.15.82:0/1643643730 learned_addr learned my addr 172.21.15.82:0/1643643730 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6ead991640 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b85e40 0x562155b8be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6ead190640 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b8b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eac98f640 1 -- 172.21.15.82:0/1643643730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b85e40 msgr2=0x562155b8be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eac98f640 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b85e40 0x562155b8be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eac98f640 1 -- 172.21.15.82:0/1643643730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 msgr2=0x562155b8b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:36.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eac98f640 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b8b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eac98f640 1 -- 172.21.15.82:0/1643643730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562155bff770 con 0x562155b7cce0 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6eac98f640 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 0x562155b8b8e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009900 tx=0x7f6e9802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e8783602b9b900 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.972+0000 7f6e90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6e90ff9640 1 -- 172.21.15.82:0/1643643730 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e980035f0 con 0x562155b7cce0 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6e90ff9640 1 -- 172.21.15.82:0/1643643730 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7f6e98003790 con 0x562155b7cce0 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1643643730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ea000fb70 con 0x562155b7cce0 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6e90ff9640 1 -- 172.21.15.82:0/1643643730 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6e98054050 con 0x562155b7cce0 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6eb1fec740 10 client.18333 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1643643730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562155c00a30 con 0x562155b7cce0 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6e90ff9640 1 -- 172.21.15.82:0/1643643730 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea000fb70 con 0x562155b7cce0 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6e90ff9640 1 -- 172.21.15.82:0/1643643730 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f6e98034b40 con 0x562155b7cce0 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6eb1fec740 10 client.18333 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6eb1fec740 10 client.18333 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6eb1fec740 20 client.18333 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:36.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562155c01240 0x562155c03660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6eb1fec740 4 client.18333 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.973+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1643643730 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562155aeec10 con 0x562155c01240 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6ead190640 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562155c01240 0x562155c03660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6e90ff9640 10 client.18333 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6ead190640 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562155c01240 0x562155c03660 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1cf3bdd498993dec server_cookie=c4f28b190b4d5abc in_seq=0 out_seq=0 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6e90ff9640 1 -- 172.21.15.82:0/1643643730 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562155aeec10 con 0x562155c01240 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6e90ff9640 10 client.18333 handle_command_reply: tid=0 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6eb1fec740 1 client.18333 shutdown 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1643643730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 msgr2=0x562155b8b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 0x562155b8b8e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009900 tx=0x7f6e9802ca10 comp rx=0 tx=0).stop 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1643643730 shutdown_connections 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562155c01240 0x562155c03660 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562155b85e40 0x562155b8be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562155b7cce0 0x562155b8b8e0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6e90ff9640 0 client.18333 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:56:36.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6eb1fec740 1 --2- 172.21.15.82:0/1643643730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562155b7c330 0x562155b8b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1643643730 >> 172.21.15.82:0/1643643730 conn(0x562155b76ba0 msgr2=0x562155b77d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1643643730 shutdown_connections 2024-04-21T08:56:36.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6eb1fec740 1 -- 172.21.15.82:0/1643643730 wait complete. 2024-04-21T08:56:36.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.974+0000 7f6eb1fec740 20 client.18333 trim_cache size 0 max 16384 2024-04-21T08:56:36.981 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:56:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.975+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/1883830000 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6e940a3d20 msgr2=0x7f6e940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.975+0000 7f6eaf3f4640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6e940a3d20 0x7f6e940a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0005fd0 tx=0x7f6ea0009b10 comp rx=0 tx=0).stop 2024-04-21T08:56:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.975+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/1883830000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea8100d20 msgr2=0x7f6ea8198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.975+0000 7f6eaf3f4640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea8100d20 0x7f6ea8198740 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400d950 tx=0x7f6ea400de10 comp rx=0 tx=0).stop 2024-04-21T08:56:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.975+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/1883830000 shutdown_connections 2024-04-21T08:56:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.975+0000 7f6eaf3f4640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ea819d000 0x7f6ea819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.975+0000 7f6eaf3f4640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ea810c4f0 0x7f6ea8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.975+0000 7f6eaf3f4640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f6e940a3d20 0x7f6e940a61e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.975+0000 7f6eaf3f4640 1 --2- 172.21.15.82:0/1883830000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea8100d20 0x7f6ea8198740 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:36.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.975+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/1883830000 >> 172.21.15.82:0/1883830000 conn(0x7f6ea8076150 msgr2=0x7f6ea80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:36.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.975+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/1883830000 shutdown_connections 2024-04-21T08:56:36.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:36.975+0000 7f6eaf3f4640 1 -- 172.21.15.82:0/1883830000 wait complete. 2024-04-21T08:56:36.983 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:36 smithi134 ceph-mon[31772]: pgmap v296: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:36.985 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:56:37.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.085+0000 7fa24f152640 1 -- 172.21.15.82:0/2903295022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa248102190 msgr2=0x7fa248102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.085+0000 7fa24f152640 1 --2- 172.21.15.82:0/2903295022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa248102190 0x7fa248102570 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009900 tx=0x7fa23c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.085+0000 7fa24f152640 1 -- 172.21.15.82:0/2903295022 shutdown_connections 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.085+0000 7fa24f152640 1 --2- 172.21.15.82:0/2903295022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa248107260 0x7fa24810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.085+0000 7fa24f152640 1 --2- 172.21.15.82:0/2903295022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa248102b40 0x7fa248106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.085+0000 7fa24f152640 1 --2- 172.21.15.82:0/2903295022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa248102190 0x7fa248102570 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.085+0000 7fa24f152640 1 -- 172.21.15.82:0/2903295022 >> 172.21.15.82:0/2903295022 conn(0x7fa2480793a0 msgr2=0x7fa2480797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.085+0000 7fa24f152640 1 -- 172.21.15.82:0/2903295022 shutdown_connections 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.085+0000 7fa24f152640 1 -- 172.21.15.82:0/2903295022 wait complete. 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24f152640 1 Processor -- start 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24f152640 1 -- start start 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24f152640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa248102190 0x7fa24819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24f152640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa248107260 0x7fa24819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24f152640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2481985a0 0x7fa248198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24f152640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa248112220 con 0x7fa2481985a0 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24f152640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481120a0 con 0x7fa248102190 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24f152640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481123a0 con 0x7fa248107260 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24d6ef640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2481985a0 0x7fa248198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24d6ef640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2481985a0 0x7fa248198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59158/0 (socket says 172.21.15.82:59158) 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24d6ef640 1 -- 172.21.15.82:0/2654952993 learned_addr learned my addr 172.21.15.82:0/2654952993 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24ceee640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa248102190 0x7fa24819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24d6ef640 1 -- 172.21.15.82:0/2654952993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa248107260 msgr2=0x7fa24819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24d6ef640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa248107260 0x7fa24819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24d6ef640 1 -- 172.21.15.82:0/2654952993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa248102190 msgr2=0x7fa24819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24d6ef640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa248102190 0x7fa24819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24d6ef640 1 -- 172.21.15.82:0/2654952993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2481990c0 con 0x7fa2481985a0 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.086+0000 7fa24d6ef640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2481985a0 0x7fa248198a50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa24000cce0 tx=0x7fa24000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.087+0000 7fa245ffb640 1 -- 172.21.15.82:0/2654952993 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24000f450 con 0x7fa2481985a0 2024-04-21T08:56:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.087+0000 7fa245ffb640 1 -- 172.21.15.82:0/2654952993 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa24000faa0 con 0x7fa2481985a0 2024-04-21T08:56:37.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.087+0000 7fa24f152640 1 -- 172.21.15.82:0/2654952993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2481993b0 con 0x7fa2481985a0 2024-04-21T08:56:37.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.087+0000 7fa245ffb640 1 -- 172.21.15.82:0/2654952993 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240018b70 con 0x7fa2481985a0 2024-04-21T08:56:37.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.087+0000 7fa24f152640 1 -- 172.21.15.82:0/2654952993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2481a5600 con 0x7fa2481985a0 2024-04-21T08:56:37.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.089+0000 7fa245ffb640 1 -- 172.21.15.82:0/2654952993 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa240018d10 con 0x7fa2481985a0 2024-04-21T08:56:37.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa251e23740 1 Processor -- start 2024-04-21T08:56:37.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa251e23740 1 -- start start 2024-04-21T08:56:37.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa251e23740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b4944d0 0x55ba9b4948b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa251e23740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b494e80 0x55ba9b49d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa251e23740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b49da00 0x55ba9b49fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa251e23740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ba9b3f91f0 con 0x55ba9b4944d0 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa251e23740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ba9b3964b0 con 0x55ba9b49da00 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa251e23740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ba9b3f51b0 con 0x55ba9b494e80 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa245ffb640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa2380a3da0 0x7fa2380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa247fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b4944d0 0x55ba9b4948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa247fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b4944d0 0x55ba9b4948b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59174/0 (socket says 172.21.15.82:59174) 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa247fff640 1 -- 172.21.15.82:0/1941142395 learned_addr learned my addr 172.21.15.82:0/1941142395 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa24d6ef640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa2380a3da0 0x7fa2380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa245ffb640 1 -- 172.21.15.82:0/2654952993 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7fa2400d10d0 con 0x7fa2481985a0 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa247fff640 1 -- 172.21.15.82:0/1941142395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b494e80 msgr2=0x55ba9b49d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.090+0000 7fa24ceee640 1 --2- 172.21.15.82:0/1941142395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b494e80 0x55ba9b49d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.091+0000 7fa247fff640 1 --2- 172.21.15.82:0/1941142395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b494e80 0x55ba9b49d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.091+0000 7fa247fff640 1 -- 172.21.15.82:0/1941142395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b49da00 msgr2=0x55ba9b49fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.091+0000 7fa247fff640 1 --2- 172.21.15.82:0/1941142395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b49da00 0x55ba9b49fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.091+0000 7fa247fff640 1 -- 172.21.15.82:0/1941142395 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba9b3881b0 con 0x55ba9b4944d0 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.091+0000 7fa247fff640 1 --2- 172.21.15.82:0/1941142395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b4944d0 0x55ba9b4948b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa234009890 tx=0x7fa234009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae10f104fc3d76f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.091+0000 7fa24d6ef640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa2380a3da0 0x7fa2380a6260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa24009cb90 tx=0x7fa2400bd040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.091+0000 7fa232ffd640 1 -- 172.21.15.82:0/1941142395 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23400f070 con 0x55ba9b4944d0 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.091+0000 7fa232ffd640 1 -- 172.21.15.82:0/1941142395 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa234017070 con 0x55ba9b4944d0 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.091+0000 7fa232ffd640 1 -- 172.21.15.82:0/1941142395 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234011350 con 0x55ba9b4944d0 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 -- 172.21.15.82:0/1941142395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b4944d0 msgr2=0x55ba9b4948b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 --2- 172.21.15.82:0/1941142395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b4944d0 0x55ba9b4948b0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa234009890 tx=0x7fa234009d50 comp rx=0 tx=0).stop 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 -- 172.21.15.82:0/1941142395 shutdown_connections 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 --2- 172.21.15.82:0/1941142395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b49da00 0x55ba9b49fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 --2- 172.21.15.82:0/1941142395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b494e80 0x55ba9b49d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 --2- 172.21.15.82:0/1941142395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b4944d0 0x55ba9b4948b0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 -- 172.21.15.82:0/1941142395 >> 172.21.15.82:0/1941142395 conn(0x55ba9b48eba0 msgr2=0x55ba9b48ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 -- 172.21.15.82:0/1941142395 shutdown_connections 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 -- 172.21.15.82:0/1941142395 wait complete. 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 -- start start 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 Processor -- start 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b4944d0 0x55ba9b49d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b494e80 0x55ba9b49a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 0x55ba9b49ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55ba9b4a0970 con 0x55ba9b49da00 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ba9b3b2500 con 0x55ba9b494e80 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ba9b4a0af0 con 0x55ba9b4944d0 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa251e23740 10 client.0 _ll_register_callbacks cb 0x55ba9b380b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.092+0000 7fa247fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 0x55ba9b49ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa247fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 0x55ba9b49ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59176/0 (socket says 172.21.15.82:59176) 2024-04-21T08:56:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa247fff640 1 -- 172.21.15.82:0/464464644 learned_addr learned my addr 172.21.15.82:0/464464644 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa24ceee640 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b494e80 0x55ba9b49a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa247fff640 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b4944d0 0x55ba9b49d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa247fff640 1 -- 172.21.15.82:0/464464644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b4944d0 msgr2=0x55ba9b49d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa247fff640 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b4944d0 0x55ba9b49d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa247fff640 1 -- 172.21.15.82:0/464464644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b494e80 msgr2=0x55ba9b49a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa247fff640 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b494e80 0x55ba9b49a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa247fff640 1 -- 172.21.15.82:0/464464644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9b10b3f0 con 0x55ba9b49da00 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa24ceee640 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b494e80 0x55ba9b49a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa247fff640 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 0x55ba9b49ae70 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa2340048d0 tx=0x7fa234012a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16ca24e20783bda server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa227fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa227fff640 1 -- 172.21.15.82:0/464464644 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234022020 con 0x55ba9b49da00 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa227fff640 1 -- 172.21.15.82:0/464464644 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7fa23401d640 con 0x55ba9b49da00 2024-04-21T08:56:37.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa251e23740 1 -- 172.21.15.82:0/464464644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa234017070 con 0x55ba9b49da00 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa227fff640 1 -- 172.21.15.82:0/464464644 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa23402d050 con 0x55ba9b49da00 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa251e23740 10 client.18345 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa251e23740 1 -- 172.21.15.82:0/464464644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ba9b1d57d0 con 0x55ba9b49da00 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.093+0000 7fa227fff640 1 -- 172.21.15.82:0/464464644 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa234017070 con 0x55ba9b49da00 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.094+0000 7fa227fff640 1 -- 172.21.15.82:0/464464644 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fa23401e870 con 0x55ba9b49da00 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.094+0000 7fa251e23740 10 client.18345 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.094+0000 7fa251e23740 10 client.18345 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.094+0000 7fa251e23740 20 client.18345 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.094+0000 7fa251e23740 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ba9b4adc20 0x55ba9b4b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.094+0000 7fa251e23740 4 client.18345 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.094+0000 7fa251e23740 1 -- 172.21.15.82:0/464464644 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba9b398930 con 0x55ba9b4adc20 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.094+0000 7fa24ceee640 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ba9b4adc20 0x55ba9b4b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.094+0000 7fa227fff640 10 client.18345 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:56:37.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.094+0000 7fa24ceee640 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ba9b4adc20 0x55ba9b4b0010 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3fd1b5aa4295878c server_cookie=6dee34acabd1385e in_seq=0 out_seq=0 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa227fff640 1 -- 172.21.15.82:0/464464644 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba9b398930 con 0x55ba9b4adc20 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa227fff640 10 client.18345 handle_command_reply: tid=0 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa251e23740 1 client.18345 shutdown 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa251e23740 1 -- 172.21.15.82:0/464464644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 msgr2=0x55ba9b49ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa251e23740 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 0x55ba9b49ae70 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa2340048d0 tx=0x7fa234012a80 comp rx=0 tx=0).stop 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa251e23740 1 -- 172.21.15.82:0/464464644 shutdown_connections 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa251e23740 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ba9b4adc20 0x55ba9b4b0010 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa251e23740 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 0x55ba9b49ae70 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa251e23740 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b494e80 0x55ba9b49a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa227fff640 0 client.18345 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa251e23740 1 --2- 172.21.15.82:0/464464644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b4944d0 0x55ba9b49d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa251e23740 1 -- 172.21.15.82:0/464464644 >> 172.21.15.82:0/464464644 conn(0x55ba9b48eba0 msgr2=0x55ba9b4908f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.095+0000 7fa251e23740 1 -- 172.21.15.82:0/464464644 shutdown_connections 2024-04-21T08:56:37.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.096+0000 7fa251e23740 1 -- 172.21.15.82:0/464464644 wait complete. 2024-04-21T08:56:37.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.096+0000 7fa251e23740 20 client.18345 trim_cache size 0 max 16384 2024-04-21T08:56:37.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa251e23740 1 Processor -- start 2024-04-21T08:56:37.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa251e23740 1 -- start start 2024-04-21T08:56:37.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa251e23740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b4944d0 0x55ba9b499ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa251e23740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b494e80 0x55ba9b4b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa251e23740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 0x55ba9b4b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa251e23740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481af520 con 0x55ba9b49da00 2024-04-21T08:56:37.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa251e23740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481b10f0 con 0x55ba9b494e80 2024-04-21T08:56:37.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa251e23740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481afbf0 con 0x55ba9b4944d0 2024-04-21T08:56:37.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa247fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 0x55ba9b4b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa247fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 0x55ba9b4b5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59186/0 (socket says 172.21.15.82:59186) 2024-04-21T08:56:37.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa247fff640 1 -- 172.21.15.82:0/429056062 learned_addr learned my addr 172.21.15.82:0/429056062 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa247fff640 1 -- 172.21.15.82:0/429056062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b4944d0 msgr2=0x55ba9b499ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa247fff640 1 --2- 172.21.15.82:0/429056062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b4944d0 0x55ba9b499ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa247fff640 1 -- 172.21.15.82:0/429056062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b494e80 msgr2=0x55ba9b4b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa247fff640 1 --2- 172.21.15.82:0/429056062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b494e80 0x55ba9b4b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa247fff640 1 -- 172.21.15.82:0/429056062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba9b48a470 con 0x55ba9b49da00 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa247fff640 1 --2- 172.21.15.82:0/429056062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 0x55ba9b4b5b50 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa23401ec10 tx=0x7fa234012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee67962febe799e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.109+0000 7fa2327fc640 1 -- 172.21.15.82:0/429056062 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234022070 con 0x55ba9b49da00 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.110+0000 7fa2327fc640 1 -- 172.21.15.82:0/429056062 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa234017070 con 0x55ba9b49da00 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.110+0000 7fa2327fc640 1 -- 172.21.15.82:0/429056062 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23400f050 con 0x55ba9b49da00 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.110+0000 7fa251e23740 1 -- 172.21.15.82:0/429056062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 msgr2=0x55ba9b4b5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.110+0000 7fa251e23740 1 --2- 172.21.15.82:0/429056062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 0x55ba9b4b5b50 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa23401ec10 tx=0x7fa234012bb0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.110+0000 7fa251e23740 1 -- 172.21.15.82:0/429056062 shutdown_connections 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.110+0000 7fa251e23740 1 --2- 172.21.15.82:0/429056062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b49da00 0x55ba9b4b5b50 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.110+0000 7fa251e23740 1 --2- 172.21.15.82:0/429056062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b494e80 0x55ba9b4b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.110+0000 7fa251e23740 1 --2- 172.21.15.82:0/429056062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b4944d0 0x55ba9b499ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.110+0000 7fa251e23740 1 -- 172.21.15.82:0/429056062 >> 172.21.15.82:0/429056062 conn(0x55ba9b48eba0 msgr2=0x55ba9b495b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.110+0000 7fa251e23740 1 -- 172.21.15.82:0/429056062 shutdown_connections 2024-04-21T08:56:37.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.110+0000 7fa251e23740 1 -- 172.21.15.82:0/429056062 wait complete. 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.110+0000 7fa251e23740 1 -- start start 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa251e23740 1 Processor -- start 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa251e23740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b4944d0 0x55ba9b4a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa251e23740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b494e80 0x55ba9b4a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa251e23740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b49da00 0x55ba9b4a1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa251e23740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481b09f0 con 0x55ba9b494e80 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa251e23740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481b02f0 con 0x55ba9b4944d0 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa251e23740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481ad4f0 con 0x55ba9b49da00 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa251e23740 10 client.0 _ll_register_callbacks cb 0x7fa2340093b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa24ceee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b494e80 0x55ba9b4a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa24ceee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b494e80 0x55ba9b4a61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59202/0 (socket says 172.21.15.82:59202) 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa24ceee640 1 -- 172.21.15.82:0/1396038262 learned_addr learned my addr 172.21.15.82:0/1396038262 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa24ceee640 1 -- 172.21.15.82:0/1396038262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b49da00 msgr2=0x55ba9b4a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa24ceee640 1 --2- 172.21.15.82:0/1396038262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b49da00 0x55ba9b4a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa24ceee640 1 -- 172.21.15.82:0/1396038262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b4944d0 msgr2=0x55ba9b4a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa24ceee640 1 --2- 172.21.15.82:0/1396038262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b4944d0 0x55ba9b4a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa24ceee640 1 -- 172.21.15.82:0/1396038262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9b4a19c0 con 0x55ba9b494e80 2024-04-21T08:56:37.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa24ceee640 1 --2- 172.21.15.82:0/1396038262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b494e80 0x55ba9b4a61c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009230 tx=0x7fa23c0041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceef104aa67439c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa227fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:37.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.111+0000 7fa227fff640 1 -- 172.21.15.82:0/1396038262 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c02c9e0 con 0x55ba9b494e80 2024-04-21T08:56:37.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa227fff640 1 -- 172.21.15.82:0/1396038262 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7fa23c02cb80 con 0x55ba9b494e80 2024-04-21T08:56:37.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa251e23740 1 -- 172.21.15.82:0/1396038262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa234017070 con 0x55ba9b494e80 2024-04-21T08:56:37.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa227fff640 1 -- 172.21.15.82:0/1396038262 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa23c052050 con 0x55ba9b494e80 2024-04-21T08:56:37.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa251e23740 10 client.18351 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:37.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa251e23740 1 -- 172.21.15.82:0/1396038262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ba9b513a70 con 0x55ba9b494e80 2024-04-21T08:56:37.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa227fff640 1 -- 172.21.15.82:0/1396038262 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa234017070 con 0x55ba9b494e80 2024-04-21T08:56:37.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa227fff640 1 -- 172.21.15.82:0/1396038262 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fa23c043630 con 0x55ba9b494e80 2024-04-21T08:56:37.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa251e23740 10 client.18351 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:37.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa251e23740 10 client.18351 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:56:37.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa251e23740 20 client.18351 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:37.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa251e23740 1 --2- 172.21.15.82:0/1396038262 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ba9b5142a0 0x55ba9b5166c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa251e23740 4 client.18351 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa251e23740 1 -- 172.21.15.82:0/1396038262 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba9b406c10 con 0x55ba9b5142a0 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.112+0000 7fa247fff640 1 --2- 172.21.15.82:0/1396038262 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ba9b5142a0 0x55ba9b5166c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa227fff640 10 client.18351 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa247fff640 1 --2- 172.21.15.82:0/1396038262 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ba9b5142a0 0x55ba9b5166c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3f5437c066e06367 server_cookie=26b018a4cb83f808 in_seq=0 out_seq=0 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa227fff640 1 -- 172.21.15.82:0/1396038262 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba9b406c10 con 0x55ba9b5142a0 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa227fff640 10 client.18351 handle_command_reply: tid=0 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa251e23740 1 client.18351 shutdown 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa251e23740 1 -- 172.21.15.82:0/1396038262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b494e80 msgr2=0x55ba9b4a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa251e23740 1 --2- 172.21.15.82:0/1396038262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b494e80 0x55ba9b4a61c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009230 tx=0x7fa23c0041a0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa251e23740 1 -- 172.21.15.82:0/1396038262 shutdown_connections 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa251e23740 1 --2- 172.21.15.82:0/1396038262 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55ba9b5142a0 0x55ba9b5166c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa251e23740 1 --2- 172.21.15.82:0/1396038262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba9b49da00 0x55ba9b4a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa251e23740 1 --2- 172.21.15.82:0/1396038262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ba9b494e80 0x55ba9b4a61c0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa227fff640 0 client.18351 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:56:37.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa251e23740 1 --2- 172.21.15.82:0/1396038262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ba9b4944d0 0x55ba9b4a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa251e23740 1 -- 172.21.15.82:0/1396038262 >> 172.21.15.82:0/1396038262 conn(0x55ba9b48eba0 msgr2=0x55ba9b48d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa251e23740 1 -- 172.21.15.82:0/1396038262 shutdown_connections 2024-04-21T08:56:37.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa251e23740 1 -- 172.21.15.82:0/1396038262 wait complete. 2024-04-21T08:56:37.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.113+0000 7fa251e23740 20 client.18351 trim_cache size 0 max 16384 2024-04-21T08:56:37.121 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:56:37.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.114+0000 7fa24f152640 1 -- 172.21.15.82:0/2654952993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa2380a3da0 msgr2=0x7fa2380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.114+0000 7fa24f152640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa2380a3da0 0x7fa2380a6260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa24009cb90 tx=0x7fa2400bd040 comp rx=0 tx=0).stop 2024-04-21T08:56:37.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.114+0000 7fa24f152640 1 -- 172.21.15.82:0/2654952993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2481985a0 msgr2=0x7fa248198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.114+0000 7fa24f152640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2481985a0 0x7fa248198a50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa24000cce0 tx=0x7fa24000a620 comp rx=0 tx=0).stop 2024-04-21T08:56:37.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.114+0000 7fa24f152640 1 -- 172.21.15.82:0/2654952993 shutdown_connections 2024-04-21T08:56:37.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.114+0000 7fa24f152640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa2380a3da0 0x7fa2380a6260 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.114+0000 7fa24f152640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2481985a0 0x7fa248198a50 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.114+0000 7fa24f152640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa248107260 0x7fa24819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.114+0000 7fa24f152640 1 --2- 172.21.15.82:0/2654952993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa248102190 0x7fa24819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.114+0000 7fa24f152640 1 -- 172.21.15.82:0/2654952993 >> 172.21.15.82:0/2654952993 conn(0x7fa2480793a0 msgr2=0x7fa2480fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.114+0000 7fa24f152640 1 -- 172.21.15.82:0/2654952993 shutdown_connections 2024-04-21T08:56:37.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.114+0000 7fa24f152640 1 -- 172.21.15.82:0/2654952993 wait complete. 2024-04-21T08:56:37.124 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:56:37.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.225+0000 7fac614ba640 1 -- 172.21.15.82:0/961367192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac5c101b40 msgr2=0x7fac5c10dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.225+0000 7fac614ba640 1 --2- 172.21.15.82:0/961367192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac5c101b40 0x7fac5c10dc80 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fac54009cd0 tx=0x7fac5402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.225+0000 7fac614ba640 1 -- 172.21.15.82:0/961367192 shutdown_connections 2024-04-21T08:56:37.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.225+0000 7fac614ba640 1 --2- 172.21.15.82:0/961367192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac5c101b40 0x7fac5c10dc80 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.225+0000 7fac614ba640 1 --2- 172.21.15.82:0/961367192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac5c1011a0 0x7fac5c101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.225+0000 7fac614ba640 1 --2- 172.21.15.82:0/961367192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac5c107030 0x7fac5c107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.225+0000 7fac614ba640 1 -- 172.21.15.82:0/961367192 >> 172.21.15.82:0/961367192 conn(0x7fac5c076c50 msgr2=0x7fac5c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.225+0000 7fac614ba640 1 -- 172.21.15.82:0/961367192 shutdown_connections 2024-04-21T08:56:37.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.225+0000 7fac614ba640 1 -- 172.21.15.82:0/961367192 wait complete. 2024-04-21T08:56:37.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.225+0000 7fac614ba640 1 Processor -- start 2024-04-21T08:56:37.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.225+0000 7fac614ba640 1 -- start start 2024-04-21T08:56:37.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac614ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac5c1011a0 0x7fac5c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac614ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac5c101b40 0x7fac5c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac614ba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac5c107030 0x7fac5c1a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac614ba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1110e0 con 0x7fac5c107030 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac614ba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c110f60 con 0x7fac5c101b40 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac614ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c111260 con 0x7fac5c1011a0 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac5b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac5c107030 0x7fac5c1a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac5b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac5c107030 0x7fac5c1a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59228/0 (socket says 172.21.15.82:59228) 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac5b7fe640 1 -- 172.21.15.82:0/2224847138 learned_addr learned my addr 172.21.15.82:0/2224847138 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac53fff640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac5c101b40 0x7fac5c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac5affd640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac5c1011a0 0x7fac5c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac5b7fe640 1 -- 172.21.15.82:0/2224847138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac5c1011a0 msgr2=0x7fac5c1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac5b7fe640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac5c1011a0 0x7fac5c1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac5b7fe640 1 -- 172.21.15.82:0/2224847138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac5c101b40 msgr2=0x7fac5c199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac5b7fe640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac5c101b40 0x7fac5c199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac5b7fe640 1 -- 172.21.15.82:0/2224847138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5c1a4b80 con 0x7fac5c107030 2024-04-21T08:56:37.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac53fff640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac5c101b40 0x7fac5c199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:37.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac5b7fe640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac5c107030 0x7fac5c1a4640 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fac540098d0 tx=0x7fac54032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac58ff9640 1 -- 172.21.15.82:0/2224847138 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54041020 con 0x7fac5c107030 2024-04-21T08:56:37.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac58ff9640 1 -- 172.21.15.82:0/2224847138 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac54032970 con 0x7fac5c107030 2024-04-21T08:56:37.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac614ba640 1 -- 172.21.15.82:0/2224847138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5c1a4d50 con 0x7fac5c107030 2024-04-21T08:56:37.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac58ff9640 1 -- 172.21.15.82:0/2224847138 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54039040 con 0x7fac5c107030 2024-04-21T08:56:37.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.226+0000 7fac614ba640 1 -- 172.21.15.82:0/2224847138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac5c1a51b0 con 0x7fac5c107030 2024-04-21T08:56:37.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.228+0000 7fac58ff9640 1 -- 172.21.15.82:0/2224847138 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fac54044020 con 0x7fac5c107030 2024-04-21T08:56:37.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.229+0000 7fac64186740 1 Processor -- start 2024-04-21T08:56:37.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.229+0000 7fac64186740 1 -- start start 2024-04-21T08:56:37.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.229+0000 7fac64186740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c5694d0 0x560b9c5698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.229+0000 7fac64186740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c569e80 0x560b9c572430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.229+0000 7fac64186740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 0x560b9c574e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.229+0000 7fac64186740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560b9c4ce1f0 con 0x560b9c572a00 2024-04-21T08:56:37.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.229+0000 7fac64186740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560b9c46b4b0 con 0x560b9c569e80 2024-04-21T08:56:37.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.229+0000 7fac64186740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560b9c4ca1b0 con 0x560b9c5694d0 2024-04-21T08:56:37.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.229+0000 7fac53fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 0x560b9c574e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.229+0000 7fac58ff9640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fac480a3d20 0x7fac480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.229+0000 7fac53fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 0x560b9c574e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59234/0 (socket says 172.21.15.82:59234) 2024-04-21T08:56:37.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.229+0000 7fac53fff640 1 -- 172.21.15.82:0/65379169 learned_addr learned my addr 172.21.15.82:0/65379169 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac53fff640 1 --2- 172.21.15.82:0/65379169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c5694d0 0x560b9c5698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac5b7fe640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fac480a3d20 0x7fac480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac5affd640 1 --2- 172.21.15.82:0/65379169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c569e80 0x560b9c572430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac53fff640 1 -- 172.21.15.82:0/65379169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c5694d0 msgr2=0x560b9c5698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac53fff640 1 --2- 172.21.15.82:0/65379169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c5694d0 0x560b9c5698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac53fff640 1 -- 172.21.15.82:0/65379169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c569e80 msgr2=0x560b9c572430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac53fff640 1 --2- 172.21.15.82:0/65379169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c569e80 0x560b9c572430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac58ff9640 1 -- 172.21.15.82:0/2224847138 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7fac540f2980 con 0x7fac5c107030 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac53fff640 1 -- 172.21.15.82:0/65379169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9c45d1b0 con 0x560b9c572a00 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac5affd640 1 --2- 172.21.15.82:0/65379169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c569e80 0x560b9c572430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac53fff640 1 --2- 172.21.15.82:0/65379169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 0x560b9c574e50 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fac440124c0 tx=0x7fac4400c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea0399377cae60b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac5b7fe640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fac480a3d20 0x7fac480a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac540be2c0 tx=0x7fac54090000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac517fa640 1 -- 172.21.15.82:0/65379169 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44018030 con 0x560b9c572a00 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac517fa640 1 -- 172.21.15.82:0/65379169 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac44005020 con 0x560b9c572a00 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.230+0000 7fac517fa640 1 -- 172.21.15.82:0/65379169 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44004a80 con 0x560b9c572a00 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.231+0000 7fac64186740 1 -- 172.21.15.82:0/65379169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 msgr2=0x560b9c574e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.231+0000 7fac64186740 1 --2- 172.21.15.82:0/65379169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 0x560b9c574e50 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fac440124c0 tx=0x7fac4400c420 comp rx=0 tx=0).stop 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.231+0000 7fac64186740 1 -- 172.21.15.82:0/65379169 shutdown_connections 2024-04-21T08:56:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.231+0000 7fac64186740 1 --2- 172.21.15.82:0/65379169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 0x560b9c574e50 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.231+0000 7fac64186740 1 --2- 172.21.15.82:0/65379169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c569e80 0x560b9c572430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.231+0000 7fac64186740 1 --2- 172.21.15.82:0/65379169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c5694d0 0x560b9c5698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.231+0000 7fac64186740 1 -- 172.21.15.82:0/65379169 >> 172.21.15.82:0/65379169 conn(0x560b9c563ba0 msgr2=0x560b9c563f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.231+0000 7fac64186740 1 -- 172.21.15.82:0/65379169 shutdown_connections 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.231+0000 7fac64186740 1 -- 172.21.15.82:0/65379169 wait complete. 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.231+0000 7fac64186740 1 -- start start 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.231+0000 7fac64186740 1 Processor -- start 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac64186740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c5694d0 0x560b9c572270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac64186740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c569e80 0x560b9c56f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac64186740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 0x560b9c56fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac64186740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560b9c575970 con 0x560b9c572a00 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac64186740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560b9c487500 con 0x560b9c5694d0 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac64186740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560b9c575af0 con 0x560b9c569e80 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac64186740 10 client.0 _ll_register_callbacks cb 0x560b9c455b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac53fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 0x560b9c56fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac53fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 0x560b9c56fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59242/0 (socket says 172.21.15.82:59242) 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac53fff640 1 -- 172.21.15.82:0/1172263838 learned_addr learned my addr 172.21.15.82:0/1172263838 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac53fff640 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c5694d0 0x560b9c572270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac5affd640 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c569e80 0x560b9c56f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac53fff640 1 -- 172.21.15.82:0/1172263838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c569e80 msgr2=0x560b9c56f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac53fff640 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c569e80 0x560b9c56f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac53fff640 1 -- 172.21.15.82:0/1172263838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c5694d0 msgr2=0x560b9c572270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac53fff640 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c5694d0 0x560b9c572270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac53fff640 1 -- 172.21.15.82:0/1172263838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9c1e03f0 con 0x560b9c572a00 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac53fff640 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 0x560b9c56fb40 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fac44001eb0 tx=0x7fac44004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d62d018af42fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac427fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac427fc640 1 -- 172.21.15.82:0/1172263838 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44018040 con 0x560b9c572a00 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac427fc640 1 -- 172.21.15.82:0/1172263838 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7fac440051d0 con 0x560b9c572a00 2024-04-21T08:56:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac64186740 1 -- 172.21.15.82:0/1172263838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac44005020 con 0x560b9c572a00 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac427fc640 1 -- 172.21.15.82:0/1172263838 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fac44034050 con 0x560b9c572a00 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac64186740 10 client.18363 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.232+0000 7fac64186740 1 -- 172.21.15.82:0/1172263838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560b9c2aa7d0 con 0x560b9c572a00 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.233+0000 7fac427fc640 1 -- 172.21.15.82:0/1172263838 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac44005020 con 0x560b9c572a00 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.233+0000 7fac427fc640 1 -- 172.21.15.82:0/1172263838 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fac44029020 con 0x560b9c572a00 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.233+0000 7fac64186740 10 client.18363 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.233+0000 7fac64186740 10 client.18363 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.233+0000 7fac64186740 20 client.18363 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.233+0000 7fac64186740 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x560b9c582ae0 0x560b9c584ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.233+0000 7fac64186740 4 client.18363 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.233+0000 7fac64186740 1 -- 172.21.15.82:0/1172263838 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b9c46d930 con 0x560b9c582ae0 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.233+0000 7fac5affd640 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x560b9c582ae0 0x560b9c584ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.233+0000 7fac5affd640 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x560b9c582ae0 0x560b9c584ed0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6a8ad859d9041a4 server_cookie=77201cf2de1a79ae in_seq=0 out_seq=0 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.233+0000 7fac427fc640 10 client.18363 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:56:37.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac427fc640 1 -- 172.21.15.82:0/1172263838 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b9c46d930 con 0x560b9c582ae0 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac427fc640 10 client.18363 handle_command_reply: tid=0 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac64186740 1 client.18363 shutdown 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac64186740 1 -- 172.21.15.82:0/1172263838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 msgr2=0x560b9c56fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac64186740 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 0x560b9c56fb40 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fac44001eb0 tx=0x7fac44004570 comp rx=0 tx=0).stop 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac64186740 1 -- 172.21.15.82:0/1172263838 shutdown_connections 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac64186740 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x560b9c582ae0 0x560b9c584ed0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac64186740 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c572a00 0x560b9c56fb40 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac427fc640 0 client.18363 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac64186740 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c569e80 0x560b9c56f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac64186740 1 --2- 172.21.15.82:0/1172263838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c5694d0 0x560b9c572270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac64186740 1 -- 172.21.15.82:0/1172263838 >> 172.21.15.82:0/1172263838 conn(0x560b9c563ba0 msgr2=0x560b9c5658f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac64186740 1 -- 172.21.15.82:0/1172263838 shutdown_connections 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac64186740 1 -- 172.21.15.82:0/1172263838 wait complete. 2024-04-21T08:56:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.234+0000 7fac64186740 20 client.18363 trim_cache size 0 max 16384 2024-04-21T08:56:37.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.247+0000 7fac64186740 1 Processor -- start 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.247+0000 7fac64186740 1 -- start start 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac64186740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c5694d0 0x560b9c57e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac64186740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c569e80 0x560b9c571d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac64186740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c572a00 0x560b9c572270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac64186740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1af470 con 0x560b9c569e80 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac64186740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1b1040 con 0x560b9c5694d0 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac64186740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1afb40 con 0x560b9c572a00 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac5affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c569e80 0x560b9c571d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac5affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c569e80 0x560b9c571d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59258/0 (socket says 172.21.15.82:59258) 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac5affd640 1 -- 172.21.15.82:0/2596583336 learned_addr learned my addr 172.21.15.82:0/2596583336 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac53fff640 1 --2- 172.21.15.82:0/2596583336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c5694d0 0x560b9c57e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac5affd640 1 -- 172.21.15.82:0/2596583336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c572a00 msgr2=0x560b9c572270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac5affd640 1 --2- 172.21.15.82:0/2596583336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c572a00 0x560b9c572270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac5affd640 1 -- 172.21.15.82:0/2596583336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c5694d0 msgr2=0x560b9c57e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac5affd640 1 --2- 172.21.15.82:0/2596583336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c5694d0 0x560b9c57e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac5affd640 1 -- 172.21.15.82:0/2596583336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9c56fe70 con 0x560b9c569e80 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac5affd640 1 --2- 172.21.15.82:0/2596583336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c569e80 0x560b9c571d30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x560b9c564c10 tx=0x7fac4c013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5275b08c8f5e494b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac50ff9640 1 -- 172.21.15.82:0/2596583336 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c01d430 con 0x560b9c569e80 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac50ff9640 1 -- 172.21.15.82:0/2596583336 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4c011440 con 0x560b9c569e80 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.248+0000 7fac50ff9640 1 -- 172.21.15.82:0/2596583336 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00f680 con 0x560b9c569e80 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.249+0000 7fac64186740 1 -- 172.21.15.82:0/2596583336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c569e80 msgr2=0x560b9c571d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.249+0000 7fac64186740 1 --2- 172.21.15.82:0/2596583336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c569e80 0x560b9c571d30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x560b9c564c10 tx=0x7fac4c013830 comp rx=0 tx=0).stop 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.249+0000 7fac64186740 1 -- 172.21.15.82:0/2596583336 shutdown_connections 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.249+0000 7fac64186740 1 --2- 172.21.15.82:0/2596583336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c572a00 0x560b9c572270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.249+0000 7fac64186740 1 --2- 172.21.15.82:0/2596583336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c569e80 0x560b9c571d30 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.249+0000 7fac64186740 1 --2- 172.21.15.82:0/2596583336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c5694d0 0x560b9c57e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.249+0000 7fac64186740 1 -- 172.21.15.82:0/2596583336 >> 172.21.15.82:0/2596583336 conn(0x560b9c563ba0 msgr2=0x560b9c56ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.249+0000 7fac64186740 1 -- 172.21.15.82:0/2596583336 shutdown_connections 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.249+0000 7fac64186740 1 -- 172.21.15.82:0/2596583336 wait complete. 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.249+0000 7fac64186740 1 -- start start 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.249+0000 7fac64186740 1 Processor -- start 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac64186740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c5694d0 0x560b9c57ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac64186740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c569e80 0x560b9c57b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac64186740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c572a00 0x560b9c576370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac64186740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1b0940 con 0x560b9c5694d0 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac64186740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1b0240 con 0x560b9c569e80 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac64186740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1ad440 con 0x560b9c572a00 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac64186740 10 client.0 _ll_register_callbacks cb 0x7fac44004db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac53fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c5694d0 0x560b9c57ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac53fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c5694d0 0x560b9c57ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59262/0 (socket says 172.21.15.82:59262) 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac53fff640 1 -- 172.21.15.82:0/792542720 learned_addr learned my addr 172.21.15.82:0/792542720 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac5affd640 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c569e80 0x560b9c57b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac53fff640 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c572a00 0x560b9c576370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac53fff640 1 -- 172.21.15.82:0/792542720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c572a00 msgr2=0x560b9c576370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac53fff640 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c572a00 0x560b9c576370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac53fff640 1 -- 172.21.15.82:0/792542720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c569e80 msgr2=0x560b9c57b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac53fff640 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c569e80 0x560b9c57b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac53fff640 1 -- 172.21.15.82:0/792542720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9c576b70 con 0x560b9c5694d0 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac53fff640 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c5694d0 0x560b9c57ad70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fac440124c0 tx=0x7fac44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d0cd3d1b63e17d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac427fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac427fc640 1 -- 172.21.15.82:0/792542720 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44018040 con 0x560b9c5694d0 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac427fc640 1 -- 172.21.15.82:0/792542720 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7fac4401c5e0 con 0x560b9c5694d0 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.250+0000 7fac64186740 1 -- 172.21.15.82:0/792542720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac4c011440 con 0x560b9c5694d0 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac427fc640 1 -- 172.21.15.82:0/792542720 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fac44041050 con 0x560b9c5694d0 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac64186740 10 client.18369 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac64186740 1 -- 172.21.15.82:0/792542720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560b9c5f1370 con 0x560b9c5694d0 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac427fc640 1 -- 172.21.15.82:0/792542720 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4c011440 con 0x560b9c5694d0 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac427fc640 1 -- 172.21.15.82:0/792542720 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fac44029050 con 0x560b9c5694d0 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac64186740 10 client.18369 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac64186740 10 client.18369 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:56:37.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac64186740 20 client.18369 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac64186740 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x560b9c5f1ad0 0x560b9c5f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac64186740 4 client.18369 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac64186740 1 -- 172.21.15.82:0/792542720 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b9c4dbc10 con 0x560b9c5f1ad0 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac5affd640 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x560b9c5f1ad0 0x560b9c5f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac5affd640 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x560b9c5f1ad0 0x560b9c5f3ef0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6fb07165930972a6 server_cookie=2b2f04978af189e3 in_seq=0 out_seq=0 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.251+0000 7fac427fc640 10 client.18369 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac427fc640 1 -- 172.21.15.82:0/792542720 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b9c4dbc10 con 0x560b9c5f1ad0 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac427fc640 10 client.18369 handle_command_reply: tid=0 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac64186740 1 client.18369 shutdown 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac64186740 1 -- 172.21.15.82:0/792542720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c5694d0 msgr2=0x560b9c57ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac64186740 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c5694d0 0x560b9c57ad70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fac440124c0 tx=0x7fac44000bc0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac64186740 1 -- 172.21.15.82:0/792542720 shutdown_connections 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac64186740 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x560b9c5f1ad0 0x560b9c5f3ef0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac64186740 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b9c572a00 0x560b9c576370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac427fc640 0 client.18369 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:56:37.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac64186740 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9c569e80 0x560b9c57b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac64186740 1 --2- 172.21.15.82:0/792542720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b9c5694d0 0x560b9c57ad70 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac64186740 1 -- 172.21.15.82:0/792542720 >> 172.21.15.82:0/792542720 conn(0x560b9c563ba0 msgr2=0x560b9c56a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac64186740 1 -- 172.21.15.82:0/792542720 shutdown_connections 2024-04-21T08:56:37.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac64186740 1 -- 172.21.15.82:0/792542720 wait complete. 2024-04-21T08:56:37.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac64186740 20 client.18369 trim_cache size 0 max 16384 2024-04-21T08:56:37.258 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:56:37.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac614ba640 1 -- 172.21.15.82:0/2224847138 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fac480a3d20 msgr2=0x7fac480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac614ba640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fac480a3d20 0x7fac480a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac540be2c0 tx=0x7fac54090000 comp rx=0 tx=0).stop 2024-04-21T08:56:37.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac614ba640 1 -- 172.21.15.82:0/2224847138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac5c107030 msgr2=0x7fac5c1a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac614ba640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac5c107030 0x7fac5c1a4640 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fac540098d0 tx=0x7fac54032420 comp rx=0 tx=0).stop 2024-04-21T08:56:37.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac614ba640 1 -- 172.21.15.82:0/2224847138 shutdown_connections 2024-04-21T08:56:37.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac614ba640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fac480a3d20 0x7fac480a61e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac614ba640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac5c107030 0x7fac5c1a4640 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac614ba640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac5c101b40 0x7fac5c199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac614ba640 1 --2- 172.21.15.82:0/2224847138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac5c1011a0 0x7fac5c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.252+0000 7fac614ba640 1 -- 172.21.15.82:0/2224847138 >> 172.21.15.82:0/2224847138 conn(0x7fac5c076c50 msgr2=0x7fac5c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.253+0000 7fac614ba640 1 -- 172.21.15.82:0/2224847138 shutdown_connections 2024-04-21T08:56:37.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.253+0000 7fac614ba640 1 -- 172.21.15.82:0/2224847138 wait complete. 2024-04-21T08:56:37.263 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:56:37.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.363+0000 7fb63869a640 1 -- 172.21.15.82:0/3225705182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb630102190 msgr2=0x7fb630102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.363+0000 7fb63869a640 1 --2- 172.21.15.82:0/3225705182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb630102190 0x7fb630102570 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009900 tx=0x7fb62c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:56:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.363+0000 7fb63869a640 1 -- 172.21.15.82:0/3225705182 shutdown_connections 2024-04-21T08:56:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.363+0000 7fb63869a640 1 --2- 172.21.15.82:0/3225705182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb630107260 0x7fb63010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.363+0000 7fb63869a640 1 --2- 172.21.15.82:0/3225705182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb630102b40 0x7fb630106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.363+0000 7fb63869a640 1 --2- 172.21.15.82:0/3225705182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb630102190 0x7fb630102570 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.363+0000 7fb63869a640 1 -- 172.21.15.82:0/3225705182 >> 172.21.15.82:0/3225705182 conn(0x7fb6300793a0 msgr2=0x7fb6300797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.363+0000 7fb63869a640 1 -- 172.21.15.82:0/3225705182 shutdown_connections 2024-04-21T08:56:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.363+0000 7fb63869a640 1 -- 172.21.15.82:0/3225705182 wait complete. 2024-04-21T08:56:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb63869a640 1 Processor -- start 2024-04-21T08:56:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb63869a640 1 -- start start 2024-04-21T08:56:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb63869a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb630102190 0x7fb6301986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb63869a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb630107260 0x7fb630198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb63869a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb63019cf80 0x7fb63019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb63869a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301121a0 con 0x7fb630107260 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb63869a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb630112020 con 0x7fb630102190 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb63869a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb630112320 con 0x7fb63019cf80 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb635c35640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb630107260 0x7fb630198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb635c35640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb630107260 0x7fb630198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59282/0 (socket says 172.21.15.82:59282) 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb635c35640 1 -- 172.21.15.82:0/85125723 learned_addr learned my addr 172.21.15.82:0/85125723 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb636436640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb630102190 0x7fb6301986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb636c37640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb63019cf80 0x7fb63019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb635c35640 1 -- 172.21.15.82:0/85125723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb63019cf80 msgr2=0x7fb63019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb635c35640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb63019cf80 0x7fb63019d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb635c35640 1 -- 172.21.15.82:0/85125723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb630102190 msgr2=0x7fb6301986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb635c35640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb630102190 0x7fb6301986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb635c35640 1 -- 172.21.15.82:0/85125723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63019da70 con 0x7fb630107260 2024-04-21T08:56:37.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.364+0000 7fb636436640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb630102190 0x7fb6301986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:37.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.365+0000 7fb635c35640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb630107260 0x7fb630198c00 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb62000d770 tx=0x7fb62000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.365+0000 7fb6277fe640 1 -- 172.21.15.82:0/85125723 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620017070 con 0x7fb630107260 2024-04-21T08:56:37.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.365+0000 7fb63869a640 1 -- 172.21.15.82:0/85125723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63019dd60 con 0x7fb630107260 2024-04-21T08:56:37.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.365+0000 7fb6277fe640 1 -- 172.21.15.82:0/85125723 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb620009c30 con 0x7fb630107260 2024-04-21T08:56:37.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.365+0000 7fb6277fe640 1 -- 172.21.15.82:0/85125723 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62000f620 con 0x7fb630107260 2024-04-21T08:56:37.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.365+0000 7fb63869a640 1 -- 172.21.15.82:0/85125723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6301a5550 con 0x7fb630107260 2024-04-21T08:56:37.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.367+0000 7fb6277fe640 1 -- 172.21.15.82:0/85125723 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb62001a070 con 0x7fb630107260 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.367+0000 7fb63b36f740 1 Processor -- start 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.367+0000 7fb63b36f740 1 -- start start 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb63b36f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 0x563a33390850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb63b36f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390e20 0x563a3339d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb63b36f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a3339de10 0x563a333a0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb63b36f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563a332f51f0 con 0x563a33390470 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb63b36f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563a332924b0 con 0x563a33390e20 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb63b36f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563a332f11b0 con 0x563a3339de10 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb636c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 0x563a33390850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb636c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 0x563a33390850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59290/0 (socket says 172.21.15.82:59290) 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb636c37640 1 -- 172.21.15.82:0/3959820278 learned_addr learned my addr 172.21.15.82:0/3959820278 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb6277fe640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb61c0a3d20 0x7fb61c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb636436640 1 --2- 172.21.15.82:0/3959820278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390e20 0x563a3339d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb635c35640 1 --2- 172.21.15.82:0/3959820278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a3339de10 0x563a333a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb636c37640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb61c0a3d20 0x7fb61c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb636c37640 1 -- 172.21.15.82:0/3959820278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a3339de10 msgr2=0x563a333a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb636c37640 1 --2- 172.21.15.82:0/3959820278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a3339de10 0x563a333a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb636c37640 1 -- 172.21.15.82:0/3959820278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390e20 msgr2=0x563a3339d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb636c37640 1 --2- 172.21.15.82:0/3959820278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390e20 0x563a3339d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb636c37640 1 -- 172.21.15.82:0/3959820278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a332841b0 con 0x563a33390470 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb635c35640 1 --2- 172.21.15.82:0/3959820278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a3339de10 0x563a333a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb6277fe640 1 -- 172.21.15.82:0/85125723 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7fb6200d0830 con 0x7fb630107260 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.368+0000 7fb636c37640 1 --2- 172.21.15.82:0/3959820278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 0x563a33390850 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb628005fd0 tx=0x7fb628014dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707704f3e9f20623 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.369+0000 7fb624ff9640 1 -- 172.21.15.82:0/3959820278 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628021070 con 0x563a33390470 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.369+0000 7fb624ff9640 1 -- 172.21.15.82:0/3959820278 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb62800e8e0 con 0x563a33390470 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.369+0000 7fb624ff9640 1 -- 172.21.15.82:0/3959820278 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62800f630 con 0x563a33390470 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.369+0000 7fb636c37640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb61c0a3d20 0x7fb61c0a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb62801e000 tx=0x7fb62801a000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.369+0000 7fb63b36f740 1 -- 172.21.15.82:0/3959820278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 msgr2=0x563a33390850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.369+0000 7fb63b36f740 1 --2- 172.21.15.82:0/3959820278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 0x563a33390850 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb628005fd0 tx=0x7fb628014dc0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.369+0000 7fb63b36f740 1 -- 172.21.15.82:0/3959820278 shutdown_connections 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.369+0000 7fb63b36f740 1 --2- 172.21.15.82:0/3959820278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a3339de10 0x563a333a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.369+0000 7fb63b36f740 1 --2- 172.21.15.82:0/3959820278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390e20 0x563a3339d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.369+0000 7fb63b36f740 1 --2- 172.21.15.82:0/3959820278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 0x563a33390850 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.369+0000 7fb63b36f740 1 -- 172.21.15.82:0/3959820278 >> 172.21.15.82:0/3959820278 conn(0x563a3338aba0 msgr2=0x563a3338af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb63b36f740 1 -- 172.21.15.82:0/3959820278 shutdown_connections 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb63b36f740 1 -- 172.21.15.82:0/3959820278 wait complete. 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb63b36f740 1 -- start start 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb63b36f740 1 Processor -- start 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb63b36f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a33390470 0x563a3339d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb63b36f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390e20 0x563a3339aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb63b36f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a3339de10 0x563a3339b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb63b36f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x563a333a0d40 con 0x563a33390e20 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb63b36f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563a332ae500 con 0x563a3339de10 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb63b36f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563a333a0ec0 con 0x563a33390470 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb63b36f740 10 client.0 _ll_register_callbacks cb 0x563a3327cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb635c35640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390e20 0x563a3339aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb635c35640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390e20 0x563a3339aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59292/0 (socket says 172.21.15.82:59292) 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb635c35640 1 -- 172.21.15.82:0/1987178683 learned_addr learned my addr 172.21.15.82:0/1987178683 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb636436640 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a33390470 0x563a3339d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb635c35640 1 -- 172.21.15.82:0/1987178683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a33390470 msgr2=0x563a3339d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb635c35640 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a33390470 0x563a3339d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb635c35640 1 -- 172.21.15.82:0/1987178683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a3339de10 msgr2=0x563a3339b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb636c37640 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a3339de10 0x563a3339b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb635c35640 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a3339de10 0x563a3339b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb635c35640 1 -- 172.21.15.82:0/1987178683 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a330073f0 con 0x563a33390e20 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.370+0000 7fb636c37640 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a3339de10 0x563a3339b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb635c35640 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390e20 0x563a3339aae0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb62000d740 tx=0x7fb62006bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e9c4773f1c4a33 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb619ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb619ffb640 1 -- 172.21.15.82:0/1987178683 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620021050 con 0x563a33390e20 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb619ffb640 1 -- 172.21.15.82:0/1987178683 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7fb6200d03b0 con 0x563a33390e20 2024-04-21T08:56:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb63b36f740 1 -- 172.21.15.82:0/1987178683 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb62800e8e0 con 0x563a33390e20 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb619ffb640 1 -- 172.21.15.82:0/1987178683 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb62009cac0 con 0x563a33390e20 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb63b36f740 10 client.18381 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb63b36f740 1 -- 172.21.15.82:0/1987178683 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563a330d17d0 con 0x563a33390e20 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb619ffb640 1 -- 172.21.15.82:0/1987178683 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb62800e8e0 con 0x563a33390e20 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb619ffb640 1 -- 172.21.15.82:0/1987178683 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fb6200d0ce0 con 0x563a33390e20 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb63b36f740 10 client.18381 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb63b36f740 10 client.18381 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb63b36f740 20 client.18381 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb63b36f740 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x563a333ae040 0x563a333b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb63b36f740 4 client.18381 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.371+0000 7fb63b36f740 1 -- 172.21.15.82:0/1987178683 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a33294930 con 0x563a333ae040 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.372+0000 7fb636436640 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x563a333ae040 0x563a333b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.372+0000 7fb619ffb640 10 client.18381 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.372+0000 7fb636436640 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x563a333ae040 0x563a333b0430 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ffe8474855a28d17 server_cookie=84e777d6dd3dbb5f in_seq=0 out_seq=0 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb619ffb640 1 -- 172.21.15.82:0/1987178683 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a33294930 con 0x563a333ae040 2024-04-21T08:56:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb619ffb640 10 client.18381 handle_command_reply: tid=0 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb63b36f740 1 client.18381 shutdown 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb63b36f740 1 -- 172.21.15.82:0/1987178683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390e20 msgr2=0x563a3339aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb63b36f740 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390e20 0x563a3339aae0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb62000d740 tx=0x7fb62006bca0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb63b36f740 1 -- 172.21.15.82:0/1987178683 shutdown_connections 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb63b36f740 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x563a333ae040 0x563a333b0430 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb63b36f740 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a3339de10 0x563a3339b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb619ffb640 0 client.18381 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb63b36f740 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390e20 0x563a3339aae0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb63b36f740 1 --2- 172.21.15.82:0/1987178683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a33390470 0x563a3339d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb63b36f740 1 -- 172.21.15.82:0/1987178683 >> 172.21.15.82:0/1987178683 conn(0x563a3338aba0 msgr2=0x563a3338c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb63b36f740 1 -- 172.21.15.82:0/1987178683 shutdown_connections 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb63b36f740 1 -- 172.21.15.82:0/1987178683 wait complete. 2024-04-21T08:56:37.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.373+0000 7fb63b36f740 20 client.18381 trim_cache size 0 max 16384 2024-04-21T08:56:37.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb63b36f740 1 Processor -- start 2024-04-21T08:56:37.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb63b36f740 1 -- start start 2024-04-21T08:56:37.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb63b36f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 0x563a333a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb63b36f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390e20 0x563a3338c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb63b36f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a3339de10 0x563a3339d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb63b36f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301ad2a0 con 0x563a33390470 2024-04-21T08:56:37.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb63b36f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301aee70 con 0x563a33390e20 2024-04-21T08:56:37.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb63b36f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301ad970 con 0x563a3339de10 2024-04-21T08:56:37.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb636436640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 0x563a333a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb636436640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 0x563a333a9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59294/0 (socket says 172.21.15.82:59294) 2024-04-21T08:56:37.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb636436640 1 -- 172.21.15.82:0/963881477 learned_addr learned my addr 172.21.15.82:0/963881477 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb636436640 1 -- 172.21.15.82:0/963881477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a3339de10 msgr2=0x563a3339d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb636436640 1 --2- 172.21.15.82:0/963881477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a3339de10 0x563a3339d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb636436640 1 -- 172.21.15.82:0/963881477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390e20 msgr2=0x563a3338c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb636436640 1 --2- 172.21.15.82:0/963881477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390e20 0x563a3338c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.386+0000 7fb636436640 1 -- 172.21.15.82:0/963881477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a33399a90 con 0x563a33390470 2024-04-21T08:56:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb636436640 1 --2- 172.21.15.82:0/963881477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 0x563a333a9c40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb62c02c170 tx=0x7fb62c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60ca4cef800485a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb61bfff640 1 -- 172.21.15.82:0/963881477 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c047050 con 0x563a33390470 2024-04-21T08:56:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb61bfff640 1 -- 172.21.15.82:0/963881477 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb62c02cc90 con 0x563a33390470 2024-04-21T08:56:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb61bfff640 1 -- 172.21.15.82:0/963881477 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c005410 con 0x563a33390470 2024-04-21T08:56:37.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb63b36f740 1 -- 172.21.15.82:0/963881477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 msgr2=0x563a333a9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb63b36f740 1 --2- 172.21.15.82:0/963881477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 0x563a333a9c40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb62c02c170 tx=0x7fb62c0058b0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb63b36f740 1 -- 172.21.15.82:0/963881477 shutdown_connections 2024-04-21T08:56:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb63b36f740 1 --2- 172.21.15.82:0/963881477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a3339de10 0x563a3339d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb63b36f740 1 --2- 172.21.15.82:0/963881477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390e20 0x563a3338c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb63b36f740 1 --2- 172.21.15.82:0/963881477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a33390470 0x563a333a9c40 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb63b36f740 1 -- 172.21.15.82:0/963881477 >> 172.21.15.82:0/963881477 conn(0x563a3338aba0 msgr2=0x563a3338af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb63b36f740 1 -- 172.21.15.82:0/963881477 shutdown_connections 2024-04-21T08:56:37.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb63b36f740 1 -- 172.21.15.82:0/963881477 wait complete. 2024-04-21T08:56:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.387+0000 7fb63b36f740 1 -- start start 2024-04-21T08:56:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb63b36f740 1 Processor -- start 2024-04-21T08:56:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb63b36f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390470 0x563a333a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb63b36f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a33390e20 0x563a333a2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb63b36f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a3339de10 0x563a333a2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb63b36f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301ae770 con 0x563a3339de10 2024-04-21T08:56:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb63b36f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301ae070 con 0x563a33390470 2024-04-21T08:56:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb63b36f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6301ab270 con 0x563a33390e20 2024-04-21T08:56:37.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb63b36f740 10 client.0 _ll_register_callbacks cb 0x7fb62009c190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb636c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a3339de10 0x563a333a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb636c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a3339de10 0x563a333a2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59308/0 (socket says 172.21.15.82:59308) 2024-04-21T08:56:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb636c37640 1 -- 172.21.15.82:0/1888603088 learned_addr learned my addr 172.21.15.82:0/1888603088 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb636436640 1 --2- 172.21.15.82:0/1888603088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390470 0x563a333a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb636c37640 1 -- 172.21.15.82:0/1888603088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a33390e20 msgr2=0x563a333a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb636c37640 1 --2- 172.21.15.82:0/1888603088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a33390e20 0x563a333a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb636c37640 1 -- 172.21.15.82:0/1888603088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390470 msgr2=0x563a333a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb636c37640 1 --2- 172.21.15.82:0/1888603088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390470 0x563a333a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb636c37640 1 -- 172.21.15.82:0/1888603088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a3339b460 con 0x563a3339de10 2024-04-21T08:56:37.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb636c37640 1 --2- 172.21.15.82:0/1888603088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a3339de10 0x563a333a2790 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb628008ce0 tx=0x7fb628005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511d4c1cf06c312f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb619ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:37.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb619ffb640 1 -- 172.21.15.82:0/1888603088 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628021050 con 0x563a3339de10 2024-04-21T08:56:37.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb619ffb640 1 -- 172.21.15.82:0/1888603088 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7fb62800f950 con 0x563a3339de10 2024-04-21T08:56:37.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.388+0000 7fb63b36f740 1 -- 172.21.15.82:0/1888603088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb62c02cc90 con 0x563a3339de10 2024-04-21T08:56:37.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.389+0000 7fb619ffb640 1 -- 172.21.15.82:0/1888603088 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb628011e70 con 0x563a3339de10 2024-04-21T08:56:37.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.389+0000 7fb63b36f740 10 client.18387 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:37.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.389+0000 7fb63b36f740 1 -- 172.21.15.82:0/1888603088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563a33414830 con 0x563a3339de10 2024-04-21T08:56:37.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.389+0000 7fb619ffb640 1 -- 172.21.15.82:0/1888603088 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb62c02cc90 con 0x563a3339de10 2024-04-21T08:56:37.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.389+0000 7fb619ffb640 1 -- 172.21.15.82:0/1888603088 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fb628005340 con 0x563a3339de10 2024-04-21T08:56:37.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.389+0000 7fb63b36f740 10 client.18387 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:37.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.389+0000 7fb63b36f740 10 client.18387 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:56:37.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.389+0000 7fb63b36f740 20 client.18387 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:37.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.389+0000 7fb63b36f740 1 --2- 172.21.15.82:0/1888603088 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x563a33414e40 0x563a33417260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.389+0000 7fb63b36f740 4 client.18387 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:56:37.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.389+0000 7fb63b36f740 1 -- 172.21.15.82:0/1888603088 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a33302c10 con 0x563a33414e40 2024-04-21T08:56:37.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.389+0000 7fb636436640 1 --2- 172.21.15.82:0/1888603088 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x563a33414e40 0x563a33417260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb636436640 1 --2- 172.21.15.82:0/1888603088 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x563a33414e40 0x563a33417260 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6bc166789d58ff1b server_cookie=fdfd600c1367ffc8 in_seq=0 out_seq=0 2024-04-21T08:56:37.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb619ffb640 10 client.18387 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:56:37.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb619ffb640 1 -- 172.21.15.82:0/1888603088 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a33302c10 con 0x563a33414e40 2024-04-21T08:56:37.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb619ffb640 10 client.18387 handle_command_reply: tid=0 2024-04-21T08:56:37.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb63b36f740 1 client.18387 shutdown 2024-04-21T08:56:37.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb63b36f740 1 -- 172.21.15.82:0/1888603088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a3339de10 msgr2=0x563a333a2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb63b36f740 1 --2- 172.21.15.82:0/1888603088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a3339de10 0x563a333a2790 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb628008ce0 tx=0x7fb628005e90 comp rx=0 tx=0).stop 2024-04-21T08:56:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb63b36f740 1 -- 172.21.15.82:0/1888603088 shutdown_connections 2024-04-21T08:56:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb63b36f740 1 --2- 172.21.15.82:0/1888603088 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x563a33414e40 0x563a33417260 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb63b36f740 1 --2- 172.21.15.82:0/1888603088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a3339de10 0x563a333a2790 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb619ffb640 0 client.18387 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:56:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb63b36f740 1 --2- 172.21.15.82:0/1888603088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a33390e20 0x563a333a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb63b36f740 1 --2- 172.21.15.82:0/1888603088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563a33390470 0x563a333a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb63b36f740 1 -- 172.21.15.82:0/1888603088 >> 172.21.15.82:0/1888603088 conn(0x563a3338aba0 msgr2=0x563a33396070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb63b36f740 1 -- 172.21.15.82:0/1888603088 shutdown_connections 2024-04-21T08:56:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb63b36f740 1 -- 172.21.15.82:0/1888603088 wait complete. 2024-04-21T08:56:37.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.390+0000 7fb63b36f740 20 client.18387 trim_cache size 0 max 16384 2024-04-21T08:56:37.401 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:56:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.391+0000 7fb63869a640 1 -- 172.21.15.82:0/85125723 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb61c0a3d20 msgr2=0x7fb61c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.391+0000 7fb63869a640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb61c0a3d20 0x7fb61c0a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb62801e000 tx=0x7fb62801a000 comp rx=0 tx=0).stop 2024-04-21T08:56:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.391+0000 7fb63869a640 1 -- 172.21.15.82:0/85125723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb630107260 msgr2=0x7fb630198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.391+0000 7fb63869a640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb630107260 0x7fb630198c00 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb62000d770 tx=0x7fb62000dc30 comp rx=0 tx=0).stop 2024-04-21T08:56:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.391+0000 7fb63869a640 1 -- 172.21.15.82:0/85125723 shutdown_connections 2024-04-21T08:56:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.391+0000 7fb63869a640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb63019cf80 0x7fb63019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.391+0000 7fb63869a640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fb61c0a3d20 0x7fb61c0a61e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.391+0000 7fb63869a640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb630107260 0x7fb630198c00 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.391+0000 7fb63869a640 1 --2- 172.21.15.82:0/85125723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb630102190 0x7fb6301986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.391+0000 7fb63869a640 1 -- 172.21.15.82:0/85125723 >> 172.21.15.82:0/85125723 conn(0x7fb6300793a0 msgr2=0x7fb6300fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.391+0000 7fb63869a640 1 -- 172.21.15.82:0/85125723 shutdown_connections 2024-04-21T08:56:37.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.391+0000 7fb63869a640 1 -- 172.21.15.82:0/85125723 wait complete. 2024-04-21T08:56:37.403 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:56:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.503+0000 7ff043aa5640 1 -- 172.21.15.82:0/1941170785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff03c06a820 msgr2=0x7ff03c06ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.503+0000 7ff043aa5640 1 --2- 172.21.15.82:0/1941170785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff03c06a820 0x7ff03c06ac00 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff038009900 tx=0x7ff03802bc30 comp rx=0 tx=0).stop 2024-04-21T08:56:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.503+0000 7ff043aa5640 1 -- 172.21.15.82:0/1941170785 shutdown_connections 2024-04-21T08:56:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.503+0000 7ff043aa5640 1 --2- 172.21.15.82:0/1941170785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff03c107270 0x7ff03c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.503+0000 7ff043aa5640 1 --2- 172.21.15.82:0/1941170785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff03c06b1d0 0x7ff03c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.503+0000 7ff043aa5640 1 --2- 172.21.15.82:0/1941170785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff03c06a820 0x7ff03c06ac00 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.503+0000 7ff043aa5640 1 -- 172.21.15.82:0/1941170785 >> 172.21.15.82:0/1941170785 conn(0x7ff03c077eb0 msgr2=0x7ff03c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.503+0000 7ff043aa5640 1 -- 172.21.15.82:0/1941170785 shutdown_connections 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.503+0000 7ff043aa5640 1 -- 172.21.15.82:0/1941170785 wait complete. 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.503+0000 7ff043aa5640 1 Processor -- start 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff043aa5640 1 -- start start 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff043aa5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff03c06a820 0x7ff03c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff043aa5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff03c107270 0x7ff03c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff043aa5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff03c19cfd0 0x7ff03c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff043aa5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1121d0 con 0x7ff03c19cfd0 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff043aa5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c112050 con 0x7ff03c107270 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff043aa5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c112350 con 0x7ff03c06a820 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff042042640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff03c19cfd0 0x7ff03c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff042042640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff03c19cfd0 0x7ff03c19d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59336/0 (socket says 172.21.15.82:59336) 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff042042640 1 -- 172.21.15.82:0/314119559 learned_addr learned my addr 172.21.15.82:0/314119559 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff041841640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff03c06a820 0x7ff03c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff041040640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff03c107270 0x7ff03c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff042042640 1 -- 172.21.15.82:0/314119559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff03c06a820 msgr2=0x7ff03c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff042042640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff03c06a820 0x7ff03c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff042042640 1 -- 172.21.15.82:0/314119559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff03c107270 msgr2=0x7ff03c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff042042640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff03c107270 0x7ff03c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff042042640 1 -- 172.21.15.82:0/314119559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c19dac0 con 0x7ff03c19cfd0 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff041040640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff03c107270 0x7ff03c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff042042640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff03c19cfd0 0x7ff03c19d480 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff03400eb10 tx=0x7ff03400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff032ffd640 1 -- 172.21.15.82:0/314119559 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034004b20 con 0x7ff03c19cfd0 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.504+0000 7ff032ffd640 1 -- 172.21.15.82:0/314119559 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff034004cc0 con 0x7ff03c19cfd0 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.505+0000 7ff043aa5640 1 -- 172.21.15.82:0/314119559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03c19ddb0 con 0x7ff03c19cfd0 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.505+0000 7ff032ffd640 1 -- 172.21.15.82:0/314119559 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034010670 con 0x7ff03c19cfd0 2024-04-21T08:56:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.505+0000 7ff043aa5640 1 -- 172.21.15.82:0/314119559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03c1a5550 con 0x7ff03c19cfd0 2024-04-21T08:56:37.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.507+0000 7ff032ffd640 1 -- 172.21.15.82:0/314119559 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff034005020 con 0x7ff03c19cfd0 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.507+0000 7ff046773740 1 Processor -- start 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.507+0000 7ff046773740 1 -- start start 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff046773740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3250 0x5603c26e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff046773740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 0x5603c26f0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff046773740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26f0e10 0x5603c26f3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff046773740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5603c26481f0 con 0x5603c26e3c00 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff046773740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5603c25e54b0 con 0x5603c26f0e10 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff046773740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5603c26441b0 con 0x5603c26e3250 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff041841640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 0x5603c26f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff041841640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 0x5603c26f0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59346/0 (socket says 172.21.15.82:59346) 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff041841640 1 -- 172.21.15.82:0/826950994 learned_addr learned my addr 172.21.15.82:0/826950994 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff032ffd640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff0280a3d20 0x7ff0280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff041040640 1 --2- 172.21.15.82:0/826950994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26f0e10 0x5603c26f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff041040640 1 --2- 172.21.15.82:0/826950994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3250 0x5603c26e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff042042640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff0280a3d20 0x7ff0280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff041841640 1 -- 172.21.15.82:0/826950994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3250 msgr2=0x5603c26e3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff041841640 1 --2- 172.21.15.82:0/826950994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3250 0x5603c26e3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff041841640 1 -- 172.21.15.82:0/826950994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26f0e10 msgr2=0x5603c26f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff041841640 1 --2- 172.21.15.82:0/826950994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26f0e10 0x5603c26f3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff041841640 1 -- 172.21.15.82:0/826950994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c25d71b0 con 0x5603c26e3c00 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff041040640 1 --2- 172.21.15.82:0/826950994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3250 0x5603c26e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff032ffd640 1 -- 172.21.15.82:0/314119559 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7ff0340d11c0 con 0x7ff03c19cfd0 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff041841640 1 --2- 172.21.15.82:0/826950994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 0x5603c26f0840 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff038009900 tx=0x7ff03802c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5601684414c44e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.508+0000 7ff042042640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff0280a3d20 0x7ff0280a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff0340991b0 tx=0x7ff0340c7e60 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.509+0000 7ff027fff640 1 -- 172.21.15.82:0/826950994 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038045020 con 0x5603c26e3c00 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.510+0000 7ff027fff640 1 -- 172.21.15.82:0/826950994 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03802c7e0 con 0x5603c26e3c00 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.510+0000 7ff027fff640 1 -- 172.21.15.82:0/826950994 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0380046c0 con 0x5603c26e3c00 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.510+0000 7ff046773740 1 -- 172.21.15.82:0/826950994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 msgr2=0x5603c26f0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.510+0000 7ff046773740 1 --2- 172.21.15.82:0/826950994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 0x5603c26f0840 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff038009900 tx=0x7ff03802c610 comp rx=0 tx=0).stop 2024-04-21T08:56:37.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.510+0000 7ff046773740 1 -- 172.21.15.82:0/826950994 shutdown_connections 2024-04-21T08:56:37.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.510+0000 7ff046773740 1 --2- 172.21.15.82:0/826950994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26f0e10 0x5603c26f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.510+0000 7ff046773740 1 --2- 172.21.15.82:0/826950994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 0x5603c26f0840 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.510+0000 7ff046773740 1 --2- 172.21.15.82:0/826950994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3250 0x5603c26e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.510+0000 7ff046773740 1 -- 172.21.15.82:0/826950994 >> 172.21.15.82:0/826950994 conn(0x5603c26ddba0 msgr2=0x5603c26ddf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.510+0000 7ff046773740 1 -- 172.21.15.82:0/826950994 shutdown_connections 2024-04-21T08:56:37.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.510+0000 7ff046773740 1 -- 172.21.15.82:0/826950994 wait complete. 2024-04-21T08:56:37.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff046773740 1 -- start start 2024-04-21T08:56:37.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff046773740 1 Processor -- start 2024-04-21T08:56:37.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff046773740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 0x5603c26f06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff046773740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3c00 0x5603c26eda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff046773740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 0x5603c26edfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff046773740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5603c26f3ce0 con 0x5603c26f0e10 2024-04-21T08:56:37.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff046773740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5603c2601500 con 0x5603c26e3250 2024-04-21T08:56:37.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff046773740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5603c26f3e60 con 0x5603c26e3c00 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff046773740 10 client.0 _ll_register_callbacks cb 0x5603c25cfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff041040640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 0x5603c26edfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff041040640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 0x5603c26edfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59354/0 (socket says 172.21.15.82:59354) 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff041040640 1 -- 172.21.15.82:0/940606005 learned_addr learned my addr 172.21.15.82:0/940606005 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff041040640 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 0x5603c26f06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.511+0000 7ff041841640 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3c00 0x5603c26eda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.512+0000 7ff041040640 1 -- 172.21.15.82:0/940606005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3c00 msgr2=0x5603c26eda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.512+0000 7ff041040640 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3c00 0x5603c26eda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.512+0000 7ff041040640 1 -- 172.21.15.82:0/940606005 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 msgr2=0x5603c26f06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.512+0000 7ff041040640 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 0x5603c26f06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.512+0000 7ff041040640 1 -- 172.21.15.82:0/940606005 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c235a3f0 con 0x5603c26f0e10 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.512+0000 7ff041040640 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 0x5603c26edfe0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff02c013fa0 tx=0x7ff02c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f666677da1ea10 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.512+0000 7ff0257fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.512+0000 7ff0257fa640 1 -- 172.21.15.82:0/940606005 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c022440 con 0x5603c26f0e10 2024-04-21T08:56:37.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.512+0000 7ff0257fa640 1 -- 172.21.15.82:0/940606005 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7ff02c0225e0 con 0x5603c26f0e10 2024-04-21T08:56:37.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.512+0000 7ff046773740 1 -- 172.21.15.82:0/940606005 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff03802c7e0 con 0x5603c26f0e10 2024-04-21T08:56:37.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.517+0000 7ff0257fa640 1 -- 172.21.15.82:0/940606005 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff02c0227e0 con 0x5603c26f0e10 2024-04-21T08:56:37.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.517+0000 7ff046773740 10 client.18399 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:37.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.517+0000 7ff046773740 1 -- 172.21.15.82:0/940606005 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5603c24247d0 con 0x5603c26f0e10 2024-04-21T08:56:37.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.517+0000 7ff0257fa640 1 -- 172.21.15.82:0/940606005 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03802c7e0 con 0x5603c26f0e10 2024-04-21T08:56:37.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.517+0000 7ff0257fa640 1 -- 172.21.15.82:0/940606005 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7ff02c020020 con 0x5603c26f0e10 2024-04-21T08:56:37.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.517+0000 7ff046773740 10 client.18399 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:37.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.517+0000 7ff046773740 10 client.18399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:37.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.517+0000 7ff046773740 20 client.18399 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:37.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.517+0000 7ff046773740 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5603c2701030 0x5603c2703440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.517+0000 7ff046773740 4 client.18399 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:37.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.517+0000 7ff046773740 1 -- 172.21.15.82:0/940606005 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603c25e7930 con 0x5603c2701030 2024-04-21T08:56:37.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.518+0000 7ff041841640 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5603c2701030 0x5603c2703440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.518+0000 7ff0257fa640 10 client.18399 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:37.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.518+0000 7ff041841640 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5603c2701030 0x5603c2703440 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d05dfc8a536945c2 server_cookie=bfee89598b3f9415 in_seq=0 out_seq=0 2024-04-21T08:56:37.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff0257fa640 1 -- 172.21.15.82:0/940606005 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603c25e7930 con 0x5603c2701030 2024-04-21T08:56:37.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff0257fa640 10 client.18399 handle_command_reply: tid=0 2024-04-21T08:56:37.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff046773740 1 client.18399 shutdown 2024-04-21T08:56:37.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff046773740 1 -- 172.21.15.82:0/940606005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 msgr2=0x5603c26edfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff046773740 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 0x5603c26edfe0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff02c013fa0 tx=0x7ff02c0046c0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff046773740 1 -- 172.21.15.82:0/940606005 shutdown_connections 2024-04-21T08:56:37.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff046773740 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5603c2701030 0x5603c2703440 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff046773740 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 0x5603c26edfe0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff046773740 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3c00 0x5603c26eda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff0257fa640 0 client.18399 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:37.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff046773740 1 --2- 172.21.15.82:0/940606005 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 0x5603c26f06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff046773740 1 -- 172.21.15.82:0/940606005 >> 172.21.15.82:0/940606005 conn(0x5603c26ddba0 msgr2=0x5603c26df8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff046773740 1 -- 172.21.15.82:0/940606005 shutdown_connections 2024-04-21T08:56:37.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff046773740 1 -- 172.21.15.82:0/940606005 wait complete. 2024-04-21T08:56:37.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.519+0000 7ff046773740 20 client.18399 trim_cache size 0 max 16384 2024-04-21T08:56:37.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.532+0000 7ff046773740 1 Processor -- start 2024-04-21T08:56:37.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.532+0000 7ff046773740 1 -- start start 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.532+0000 7ff046773740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 0x5603c26fcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.532+0000 7ff046773740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 0x5603c26ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.532+0000 7ff046773740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26f0e10 0x5603c26dca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.532+0000 7ff046773740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1ad1c0 con 0x5603c26e3c00 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.532+0000 7ff046773740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1aed90 con 0x5603c26e3250 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.532+0000 7ff046773740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1ad890 con 0x5603c26f0e10 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff041841640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 0x5603c26ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff041841640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 0x5603c26ecbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59370/0 (socket says 172.21.15.82:59370) 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff041841640 1 -- 172.21.15.82:0/2003550931 learned_addr learned my addr 172.21.15.82:0/2003550931 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff041040640 1 --2- 172.21.15.82:0/2003550931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 0x5603c26fcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff041841640 1 -- 172.21.15.82:0/2003550931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26f0e10 msgr2=0x5603c26dca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff041841640 1 --2- 172.21.15.82:0/2003550931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26f0e10 0x5603c26dca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff041841640 1 -- 172.21.15.82:0/2003550931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 msgr2=0x5603c26fcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff041841640 1 --2- 172.21.15.82:0/2003550931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 0x5603c26fcbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff041841640 1 -- 172.21.15.82:0/2003550931 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c26e2d90 con 0x5603c26e3c00 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff041841640 1 --2- 172.21.15.82:0/2003550931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 0x5603c26ecbe0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7ff038005e80 tx=0x7ff038005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7c3ae4180e60a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff0277fe640 1 -- 172.21.15.82:0/2003550931 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038045040 con 0x5603c26e3c00 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff0277fe640 1 -- 172.21.15.82:0/2003550931 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03802c7e0 con 0x5603c26e3c00 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.533+0000 7ff0277fe640 1 -- 172.21.15.82:0/2003550931 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038003970 con 0x5603c26e3c00 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.534+0000 7ff046773740 1 -- 172.21.15.82:0/2003550931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 msgr2=0x5603c26ecbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.534+0000 7ff046773740 1 --2- 172.21.15.82:0/2003550931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 0x5603c26ecbe0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7ff038005e80 tx=0x7ff038005250 comp rx=0 tx=0).stop 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.534+0000 7ff046773740 1 -- 172.21.15.82:0/2003550931 shutdown_connections 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.534+0000 7ff046773740 1 --2- 172.21.15.82:0/2003550931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26f0e10 0x5603c26dca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.534+0000 7ff046773740 1 --2- 172.21.15.82:0/2003550931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26e3c00 0x5603c26ecbe0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.534+0000 7ff046773740 1 --2- 172.21.15.82:0/2003550931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 0x5603c26fcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.534+0000 7ff046773740 1 -- 172.21.15.82:0/2003550931 >> 172.21.15.82:0/2003550931 conn(0x5603c26ddba0 msgr2=0x5603c26dc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.534+0000 7ff046773740 1 -- 172.21.15.82:0/2003550931 shutdown_connections 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.534+0000 7ff046773740 1 -- 172.21.15.82:0/2003550931 wait complete. 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.534+0000 7ff046773740 1 -- start start 2024-04-21T08:56:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.534+0000 7ff046773740 1 Processor -- start 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff046773740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 0x5603c26f4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff046773740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3c00 0x5603c26f4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff046773740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 0x5603c275f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff046773740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1ae690 con 0x5603c26f0e10 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff046773740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1adf90 con 0x5603c26e3250 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff046773740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1ab190 con 0x5603c26e3c00 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff046773740 10 client.0 _ll_register_callbacks cb 0x7ff02c013dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff041040640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 0x5603c275f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff041040640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 0x5603c275f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59384/0 (socket says 172.21.15.82:59384) 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff041040640 1 -- 172.21.15.82:0/2569164753 learned_addr learned my addr 172.21.15.82:0/2569164753 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff041040640 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 0x5603c26f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff041841640 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3c00 0x5603c26f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff041040640 1 -- 172.21.15.82:0/2569164753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3c00 msgr2=0x5603c26f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff041040640 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3c00 0x5603c26f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff041040640 1 -- 172.21.15.82:0/2569164753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 msgr2=0x5603c26f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:37.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff041040640 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 0x5603c26f4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff041040640 1 -- 172.21.15.82:0/2569164753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c275fb80 con 0x5603c26f0e10 2024-04-21T08:56:37.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff041841640 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3c00 0x5603c26f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:37.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff041040640 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 0x5603c275f540 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff02c005fd0 tx=0x7ff02c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e915c19b82a68f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:37.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff0257fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:37.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff0257fa640 1 -- 172.21.15.82:0/2569164753 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c034060 con 0x5603c26f0e10 2024-04-21T08:56:37.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff046773740 1 -- 172.21.15.82:0/2569164753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff03802c7e0 con 0x5603c26f0e10 2024-04-21T08:56:37.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff0257fa640 1 -- 172.21.15.82:0/2569164753 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7ff02c011440 con 0x5603c26f0e10 2024-04-21T08:56:37.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff0257fa640 1 -- 172.21.15.82:0/2569164753 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff02c011640 con 0x5603c26f0e10 2024-04-21T08:56:37.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff046773740 10 client.18405 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:37.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.535+0000 7ff046773740 1 -- 172.21.15.82:0/2569164753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5603c2766ef0 con 0x5603c26f0e10 2024-04-21T08:56:37.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.536+0000 7ff0257fa640 1 -- 172.21.15.82:0/2569164753 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03802c7e0 con 0x5603c26f0e10 2024-04-21T08:56:37.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.536+0000 7ff0257fa640 1 -- 172.21.15.82:0/2569164753 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7ff02c020040 con 0x5603c26f0e10 2024-04-21T08:56:37.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.536+0000 7ff046773740 10 client.18405 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:37.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.536+0000 7ff046773740 10 client.18405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:37.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.536+0000 7ff046773740 20 client.18405 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:37.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.536+0000 7ff046773740 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5603c2701030 0x5603c2769620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:37.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.536+0000 7ff046773740 4 client.18405 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:56:37.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.536+0000 7ff046773740 1 -- 172.21.15.82:0/2569164753 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603c2655c10 con 0x5603c2701030 2024-04-21T08:56:37.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.536+0000 7ff041841640 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5603c2701030 0x5603c2769620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.536+0000 7ff041841640 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5603c2701030 0x5603c2769620 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c180ec6be842b08c server_cookie=38b5e9ea7b495ac1 in_seq=0 out_seq=0 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.536+0000 7ff0257fa640 10 client.18405 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff0257fa640 1 -- 172.21.15.82:0/2569164753 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603c2655c10 con 0x5603c2701030 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff0257fa640 10 client.18405 handle_command_reply: tid=0 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff046773740 1 client.18405 shutdown 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff046773740 1 -- 172.21.15.82:0/2569164753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 msgr2=0x5603c275f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff046773740 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 0x5603c275f540 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff02c005fd0 tx=0x7ff02c001e50 comp rx=0 tx=0).stop 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff046773740 1 -- 172.21.15.82:0/2569164753 shutdown_connections 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff046773740 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x5603c2701030 0x5603c2769620 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff046773740 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5603c26f0e10 0x5603c275f540 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff0257fa640 0 client.18405 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff046773740 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603c26e3c00 0x5603c26f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff046773740 1 --2- 172.21.15.82:0/2569164753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603c26e3250 0x5603c26f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff046773740 1 -- 172.21.15.82:0/2569164753 >> 172.21.15.82:0/2569164753 conn(0x5603c26ddba0 msgr2=0x5603c26dc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff046773740 1 -- 172.21.15.82:0/2569164753 shutdown_connections 2024-04-21T08:56:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff046773740 1 -- 172.21.15.82:0/2569164753 wait complete. 2024-04-21T08:56:37.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff046773740 20 client.18405 trim_cache size 0 max 16384 2024-04-21T08:56:37.545 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:56:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff043aa5640 1 -- 172.21.15.82:0/314119559 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff0280a3d20 msgr2=0x7ff0280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff043aa5640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff0280a3d20 0x7ff0280a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff0340991b0 tx=0x7ff0340c7e60 comp rx=0 tx=0).stop 2024-04-21T08:56:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff043aa5640 1 -- 172.21.15.82:0/314119559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff03c19cfd0 msgr2=0x7ff03c19d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.537+0000 7ff043aa5640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff03c19cfd0 0x7ff03c19d480 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff03400eb10 tx=0x7ff03400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.538+0000 7ff043aa5640 1 -- 172.21.15.82:0/314119559 shutdown_connections 2024-04-21T08:56:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.538+0000 7ff043aa5640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff0280a3d20 0x7ff0280a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.538+0000 7ff043aa5640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff03c19cfd0 0x7ff03c19d480 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.538+0000 7ff043aa5640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff03c107270 0x7ff03c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.538+0000 7ff043aa5640 1 --2- 172.21.15.82:0/314119559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff03c06a820 0x7ff03c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:37.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.538+0000 7ff043aa5640 1 -- 172.21.15.82:0/314119559 >> 172.21.15.82:0/314119559 conn(0x7ff03c077eb0 msgr2=0x7ff03c108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:37.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.538+0000 7ff043aa5640 1 -- 172.21.15.82:0/314119559 shutdown_connections 2024-04-21T08:56:37.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:37.538+0000 7ff043aa5640 1 -- 172.21.15.82:0/314119559 wait complete. 2024-04-21T08:56:37.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:37 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1775435696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:56:37.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:37 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1775435696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:56:37.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:37 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1775435696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:56:38.548 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:56:38.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:56:38.549 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:56:38.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.665+0000 7f38b53bb640 1 -- 172.21.15.82:0/3159450776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b0074b40 msgr2=0x7f38b010e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.665+0000 7f38b53bb640 1 --2- 172.21.15.82:0/3159450776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b0074b40 0x7f38b010e2f0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3898009900 tx=0x7f389802bcd0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.665+0000 7f38b53bb640 1 -- 172.21.15.82:0/3159450776 shutdown_connections 2024-04-21T08:56:38.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.665+0000 7f38b53bb640 1 --2- 172.21.15.82:0/3159450776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b010e830 0x7f38b0110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.665+0000 7f38b53bb640 1 --2- 172.21.15.82:0/3159450776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b0074b40 0x7f38b010e2f0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.665+0000 7f38b53bb640 1 --2- 172.21.15.82:0/3159450776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38b0074220 0x7f38b0074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.665+0000 7f38b53bb640 1 -- 172.21.15.82:0/3159450776 >> 172.21.15.82:0/3159450776 conn(0x7f38b00793a0 msgr2=0x7f38b00797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:38.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.665+0000 7f38b53bb640 1 -- 172.21.15.82:0/3159450776 shutdown_connections 2024-04-21T08:56:38.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.665+0000 7f38b53bb640 1 -- 172.21.15.82:0/3159450776 wait complete. 2024-04-21T08:56:38.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38b53bb640 1 Processor -- start 2024-04-21T08:56:38.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38b53bb640 1 -- start start 2024-04-21T08:56:38.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38b53bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b0074220 0x7f38b01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38b53bb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38b0074b40 0x7f38b01a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38b53bb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b010e830 0x7f38b019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38b53bb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01143f0 con 0x7f38b010e830 2024-04-21T08:56:38.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38b53bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0114270 con 0x7f38b0074220 2024-04-21T08:56:38.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38b53bb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0114570 con 0x7f38b0074b40 2024-04-21T08:56:38.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b010e830 0x7f38b019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38af7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b010e830 0x7f38b019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59402/0 (socket says 172.21.15.82:59402) 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38af7fe640 1 -- 172.21.15.82:0/2726424846 learned_addr learned my addr 172.21.15.82:0/2726424846 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38a67fc640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38b0074b40 0x7f38b01a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38aeffd640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b0074220 0x7f38b01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.666+0000 7f38af7fe640 1 -- 172.21.15.82:0/2726424846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38b0074b40 msgr2=0x7f38b01a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.667+0000 7f38af7fe640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38b0074b40 0x7f38b01a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.667+0000 7f38af7fe640 1 -- 172.21.15.82:0/2726424846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b0074220 msgr2=0x7f38b01a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.667+0000 7f38af7fe640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b0074220 0x7f38b01a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.667+0000 7f38af7fe640 1 -- 172.21.15.82:0/2726424846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b019afc0 con 0x7f38b010e830 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.667+0000 7f38a67fc640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38b0074b40 0x7f38b01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.667+0000 7f38aeffd640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b0074220 0x7f38b01a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.667+0000 7f38af7fe640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b010e830 0x7f38b019a7f0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f38a800ccb0 tx=0x7f38a800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.667+0000 7f38acff9640 1 -- 172.21.15.82:0/2726424846 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a800f450 con 0x7f38b010e830 2024-04-21T08:56:38.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.667+0000 7f38acff9640 1 -- 172.21.15.82:0/2726424846 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a800faa0 con 0x7f38b010e830 2024-04-21T08:56:38.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.667+0000 7f38b53bb640 1 -- 172.21.15.82:0/2726424846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b019b2b0 con 0x7f38b010e830 2024-04-21T08:56:38.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.667+0000 7f38acff9640 1 -- 172.21.15.82:0/2726424846 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8018b00 con 0x7f38b010e830 2024-04-21T08:56:38.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.667+0000 7f38b53bb640 1 -- 172.21.15.82:0/2726424846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b01a7480 con 0x7f38b010e830 2024-04-21T08:56:38.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.669+0000 7f38acff9640 1 -- 172.21.15.82:0/2726424846 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f38a800fc50 con 0x7f38b010e830 2024-04-21T08:56:38.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.669+0000 7f38b7fa8740 1 Processor -- start 2024-04-21T08:56:38.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.669+0000 7f38b7fa8740 1 -- start start 2024-04-21T08:56:38.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38acff9640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f389c0a3d20 0x7f389c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38b7fa8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754e68e0 0x5636754e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38b7fa8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38b7fa8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754efe10 0x5636754f2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38b7fa8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56367544af00 con 0x5636754e7290 2024-04-21T08:56:38.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38b7fa8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5636753e8590 con 0x5636754efe10 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38b7fa8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563675446ec0 con 0x5636754e68e0 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38a67fc640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f389c0a3d20 0x7f389c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38a67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38aeffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754e68e0 0x5636754e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38acff9640 1 -- 172.21.15.82:0/2726424846 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7f38a80d13a0 con 0x7f38b010e830 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38a67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754ef840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59414/0 (socket says 172.21.15.82:59414) 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38aeffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754e68e0 0x5636754e6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:56912/0 (socket says 172.21.15.82:56912) 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38a67fc640 1 -- 172.21.15.82:0/2082808544 learned_addr learned my addr 172.21.15.82:0/2082808544 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38af7fe640 1 --2- 172.21.15.82:0/2082808544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754efe10 0x5636754f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38a67fc640 1 -- 172.21.15.82:0/2082808544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754e68e0 msgr2=0x5636754e6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38a67fc640 1 --2- 172.21.15.82:0/2082808544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754e68e0 0x5636754e6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38a67fc640 1 -- 172.21.15.82:0/2082808544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754efe10 msgr2=0x5636754f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38a67fc640 1 --2- 172.21.15.82:0/2082808544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754efe10 0x5636754f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.670+0000 7f38a67fc640 1 -- 172.21.15.82:0/2082808544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636752257e0 con 0x5636754e7290 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.671+0000 7f38a67fc640 1 --2- 172.21.15.82:0/2082808544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754ef840 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f3898005ab0 tx=0x7f389802cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a72d0aac5f9ea57 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.671+0000 7f38a67fc640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f389c0a3d20 0x7f389c0a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f389802c210 tx=0x7f3898039000 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.671+0000 7f38a57fa640 1 -- 172.21.15.82:0/2082808544 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898050070 con 0x5636754e7290 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.671+0000 7f38a57fa640 1 -- 172.21.15.82:0/2082808544 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3898005050 con 0x5636754e7290 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.671+0000 7f38a57fa640 1 -- 172.21.15.82:0/2082808544 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389802c6b0 con 0x5636754e7290 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.671+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2082808544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 msgr2=0x5636754ef840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.671+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/2082808544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754ef840 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f3898005ab0 tx=0x7f389802cb20 comp rx=0 tx=0).stop 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2082808544 shutdown_connections 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/2082808544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754efe10 0x5636754f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/2082808544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754ef840 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/2082808544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754e68e0 0x5636754e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2082808544 >> 172.21.15.82:0/2082808544 conn(0x5636754e0fb0 msgr2=0x5636754e1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2082808544 shutdown_connections 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2082808544 wait complete. 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 -- start start 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 Processor -- start 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e68e0 0x5636754ecea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e7290 0x5636754ed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 0x5636754ed920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5636754f2d20 con 0x5636754e68e0 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563675404580 con 0x5636754e7290 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5636754f2ea0 con 0x5636754efe10 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38b7fa8740 10 client.0 _ll_register_callbacks cb 0x56367540a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e68e0 0x5636754ecea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38aeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e68e0 0x5636754ecea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59418/0 (socket says 172.21.15.82:59418) 2024-04-21T08:56:38.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.672+0000 7f38aeffd640 1 -- 172.21.15.82:0/1439782107 learned_addr learned my addr 172.21.15.82:0/1439782107 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38af7fe640 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 0x5636754ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38aeffd640 1 -- 172.21.15.82:0/1439782107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 msgr2=0x5636754ed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38aeffd640 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 0x5636754ed920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38aeffd640 1 -- 172.21.15.82:0/1439782107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e7290 msgr2=0x5636754ed3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38a67fc640 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e7290 0x5636754ed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38aeffd640 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e7290 0x5636754ed3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38aeffd640 1 -- 172.21.15.82:0/1439782107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563675285f70 con 0x5636754e68e0 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38aeffd640 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e68e0 0x5636754ecea0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f38a0012480 tx=0x7f38a0012940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7ce5daf1bad54f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38967fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38967fc640 1 -- 172.21.15.82:0/1439782107 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a001f060 con 0x5636754e68e0 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38967fc640 1 -- 172.21.15.82:0/1439782107 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7f38a0010aa0 con 0x5636754e68e0 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38b7fa8740 1 -- 172.21.15.82:0/1439782107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3898005050 con 0x5636754e68e0 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38967fc640 1 -- 172.21.15.82:0/1439782107 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f38a0026050 con 0x5636754e68e0 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38b7fa8740 10 client.18417 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.673+0000 7f38b7fa8740 1 -- 172.21.15.82:0/1439782107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56367511d170 con 0x5636754e68e0 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.674+0000 7f38967fc640 1 -- 172.21.15.82:0/1439782107 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3898005050 con 0x5636754e68e0 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.674+0000 7f38967fc640 1 -- 172.21.15.82:0/1439782107 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f38a0019030 con 0x5636754e68e0 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.674+0000 7f38b7fa8740 10 client.18417 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.674+0000 7f38b7fa8740 10 client.18417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.674+0000 7f38b7fa8740 20 client.18417 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.674+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563675500450 0x5636755028c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.674+0000 7f38b7fa8740 4 client.18417 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.674+0000 7f38b7fa8740 1 -- 172.21.15.82:0/1439782107 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563675458920 con 0x563675500450 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.674+0000 7f38a67fc640 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563675500450 0x5636755028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.674+0000 7f38967fc640 10 client.18417 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.674+0000 7f38a67fc640 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563675500450 0x5636755028c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=548b1fa7216af897 server_cookie=bd8fd2b3e467d3aa in_seq=0 out_seq=0 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38967fc640 1 -- 172.21.15.82:0/1439782107 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563675458920 con 0x563675500450 2024-04-21T08:56:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38967fc640 10 client.18417 handle_command_reply: tid=0 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38b7fa8740 1 client.18417 shutdown 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38b7fa8740 1 -- 172.21.15.82:0/1439782107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e68e0 msgr2=0x5636754ecea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e68e0 0x5636754ecea0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f38a0012480 tx=0x7f38a0012940 comp rx=0 tx=0).stop 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38b7fa8740 1 -- 172.21.15.82:0/1439782107 shutdown_connections 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563675500450 0x5636755028c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 0x5636754ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e7290 0x5636754ed3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/1439782107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e68e0 0x5636754ecea0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38967fc640 0 client.18417 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38b7fa8740 1 -- 172.21.15.82:0/1439782107 >> 172.21.15.82:0/1439782107 conn(0x5636754e0fb0 msgr2=0x5636754e2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38b7fa8740 1 -- 172.21.15.82:0/1439782107 shutdown_connections 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38b7fa8740 1 -- 172.21.15.82:0/1439782107 wait complete. 2024-04-21T08:56:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.675+0000 7f38b7fa8740 20 client.18417 trim_cache size 0 max 16384 2024-04-21T08:56:38.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.693+0000 7f38b7fa8740 1 Processor -- start 2024-04-21T08:56:38.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.693+0000 7f38b7fa8740 1 -- start start 2024-04-21T08:56:38.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38b7fa8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e68e0 0x5636754ef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38b7fa8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754fbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38b7fa8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 0x5636755081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38b7fa8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01b1660 con 0x5636754e7290 2024-04-21T08:56:38.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38b7fa8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01b3230 con 0x5636754e68e0 2024-04-21T08:56:38.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38b7fa8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01b1d30 con 0x5636754efe10 2024-04-21T08:56:38.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38a67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754fbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38a67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754fbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59432/0 (socket says 172.21.15.82:59432) 2024-04-21T08:56:38.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38a67fc640 1 -- 172.21.15.82:0/3304376608 learned_addr learned my addr 172.21.15.82:0/3304376608 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:38.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38a67fc640 1 -- 172.21.15.82:0/3304376608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 msgr2=0x5636755081d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38a67fc640 1 --2- 172.21.15.82:0/3304376608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 0x5636755081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38aeffd640 1 --2- 172.21.15.82:0/3304376608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e68e0 0x5636754ef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38a67fc640 1 -- 172.21.15.82:0/3304376608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e68e0 msgr2=0x5636754ef5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38a67fc640 1 --2- 172.21.15.82:0/3304376608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e68e0 0x5636754ef5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38a67fc640 1 -- 172.21.15.82:0/3304376608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636754e2050 con 0x5636754e7290 2024-04-21T08:56:38.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38aeffd640 1 --2- 172.21.15.82:0/3304376608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e68e0 0x5636754ef5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:38.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38a67fc640 1 --2- 172.21.15.82:0/3304376608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754fbbe0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3898005de0 tx=0x7f38980044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121f9bfae9f4dbcc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:38.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38a4ff9640 1 -- 172.21.15.82:0/3304376608 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898050040 con 0x5636754e7290 2024-04-21T08:56:38.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38a4ff9640 1 -- 172.21.15.82:0/3304376608 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3898005500 con 0x5636754e7290 2024-04-21T08:56:38.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.694+0000 7f38a4ff9640 1 -- 172.21.15.82:0/3304376608 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898004a40 con 0x5636754e7290 2024-04-21T08:56:38.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.695+0000 7f38b7fa8740 1 -- 172.21.15.82:0/3304376608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 msgr2=0x5636754fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.695+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/3304376608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754fbbe0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3898005de0 tx=0x7f38980044d0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.695+0000 7f38b7fa8740 1 -- 172.21.15.82:0/3304376608 shutdown_connections 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.695+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/3304376608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 0x5636755081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.695+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/3304376608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754fbbe0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.695+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/3304376608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e68e0 0x5636754ef5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.695+0000 7f38b7fa8740 1 -- 172.21.15.82:0/3304376608 >> 172.21.15.82:0/3304376608 conn(0x5636754e0fb0 msgr2=0x5636754e2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.695+0000 7f38b7fa8740 1 -- 172.21.15.82:0/3304376608 shutdown_connections 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.695+0000 7f38b7fa8740 1 -- 172.21.15.82:0/3304376608 wait complete. 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38b7fa8740 1 -- start start 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38b7fa8740 1 Processor -- start 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38b7fa8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e68e0 0x5636754f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38b7fa8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38b7fa8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 0x5636754f5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38b7fa8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01b2b30 con 0x5636754e7290 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38b7fa8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01b2430 con 0x5636754e68e0 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38b7fa8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01af630 con 0x5636754efe10 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38b7fa8740 10 client.0 _ll_register_callbacks cb 0x7f38a000b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38a67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38a67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59444/0 (socket says 172.21.15.82:59444) 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38a67fc640 1 -- 172.21.15.82:0/2641216910 learned_addr learned my addr 172.21.15.82:0/2641216910 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38aeffd640 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e68e0 0x5636754f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38a67fc640 1 -- 172.21.15.82:0/2641216910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 msgr2=0x5636754f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38a67fc640 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 0x5636754f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38a67fc640 1 -- 172.21.15.82:0/2641216910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e68e0 msgr2=0x5636754f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38a67fc640 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e68e0 0x5636754f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38a67fc640 1 -- 172.21.15.82:0/2641216910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636754f5d40 con 0x5636754e7290 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38aeffd640 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e68e0 0x5636754f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:38.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38a67fc640 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754f50c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f38980039e0 tx=0x7f389802c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291fc5cc25673e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38967fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.696+0000 7f38967fc640 1 -- 172.21.15.82:0/2641216910 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898050030 con 0x5636754e7290 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2641216910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3898005500 con 0x5636754e7290 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38967fc640 1 -- 172.21.15.82:0/2641216910 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7f3898003d10 con 0x5636754e7290 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38967fc640 1 -- 172.21.15.82:0/2641216910 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3898068050 con 0x5636754e7290 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38b7fa8740 10 client.18423 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2641216910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563675567950 con 0x5636754e7290 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38967fc640 1 -- 172.21.15.82:0/2641216910 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3898005500 con 0x5636754e7290 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38967fc640 1 -- 172.21.15.82:0/2641216910 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f389804a070 con 0x5636754e7290 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38b7fa8740 10 client.18423 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38b7fa8740 10 client.18423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38b7fa8740 20 client.18423 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563675567c10 0x56367556a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38b7fa8740 4 client.18423 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2641216910 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563675452790 con 0x563675567c10 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.697+0000 7f38aeffd640 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563675567c10 0x56367556a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.698+0000 7f38aeffd640 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563675567c10 0x56367556a000 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f792e3da3b127cbf server_cookie=1411f9cb0007bad in_seq=0 out_seq=0 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.698+0000 7f38967fc640 10 client.18423 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38967fc640 1 -- 172.21.15.82:0/2641216910 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563675452790 con 0x563675567c10 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38967fc640 10 client.18423 handle_command_reply: tid=0 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38b7fa8740 1 client.18423 shutdown 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2641216910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 msgr2=0x5636754f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754f50c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f38980039e0 tx=0x7f389802c350 comp rx=0 tx=0).stop 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2641216910 shutdown_connections 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x563675567c10 0x56367556a000 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5636754efe10 0x5636754f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5636754e7290 0x5636754f50c0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38967fc640 0 client.18423 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:38.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38b7fa8740 1 --2- 172.21.15.82:0/2641216910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636754e68e0 0x5636754f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2641216910 >> 172.21.15.82:0/2641216910 conn(0x5636754e0fb0 msgr2=0x5636754ece40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:38.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2641216910 shutdown_connections 2024-04-21T08:56:38.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38b7fa8740 1 -- 172.21.15.82:0/2641216910 wait complete. 2024-04-21T08:56:38.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.699+0000 7f38b7fa8740 20 client.18423 trim_cache size 0 max 16384 2024-04-21T08:56:38.706 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:56:38.706 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-21T08:56:38.706 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "1feca02f-cab9-4961-a2e1-bc02b14ba165", 2024-04-21T08:56:38.706 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-21T08:56:38.706 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:56:38.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.700+0000 7f38b53bb640 1 -- 172.21.15.82:0/2726424846 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f389c0a3d20 msgr2=0x7f389c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.700+0000 7f38b53bb640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f389c0a3d20 0x7f389c0a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f389802c210 tx=0x7f3898039000 comp rx=0 tx=0).stop 2024-04-21T08:56:38.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.700+0000 7f38b53bb640 1 -- 172.21.15.82:0/2726424846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b010e830 msgr2=0x7f38b019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.700+0000 7f38b53bb640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b010e830 0x7f38b019a7f0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f38a800ccb0 tx=0x7f38a800a620 comp rx=0 tx=0).stop 2024-04-21T08:56:38.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.700+0000 7f38b53bb640 1 -- 172.21.15.82:0/2726424846 shutdown_connections 2024-04-21T08:56:38.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.700+0000 7f38b53bb640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f389c0a3d20 0x7f389c0a61e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.700+0000 7f38b53bb640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b010e830 0x7f38b019a7f0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.700+0000 7f38b53bb640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38b0074b40 0x7f38b01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.700+0000 7f38b53bb640 1 --2- 172.21.15.82:0/2726424846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38b0074220 0x7f38b01a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.700+0000 7f38b53bb640 1 -- 172.21.15.82:0/2726424846 >> 172.21.15.82:0/2726424846 conn(0x7f38b00793a0 msgr2=0x7f38b00fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:38.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.700+0000 7f38b53bb640 1 -- 172.21.15.82:0/2726424846 shutdown_connections 2024-04-21T08:56:38.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.700+0000 7f38b53bb640 1 -- 172.21.15.82:0/2726424846 wait complete. 2024-04-21T08:56:38.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:56:38.711 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:56:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.810+0000 7f28778eb640 1 -- 172.21.15.82:0/688484685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28701021a0 msgr2=0x7f2870102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.810+0000 7f28778eb640 1 --2- 172.21.15.82:0/688484685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28701021a0 0x7f2870102580 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f2860009900 tx=0x7f286002bc30 comp rx=0 tx=0).stop 2024-04-21T08:56:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 -- 172.21.15.82:0/688484685 shutdown_connections 2024-04-21T08:56:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 --2- 172.21.15.82:0/688484685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2870107270 0x7f287010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 --2- 172.21.15.82:0/688484685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2870102b50 0x7f2870106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 --2- 172.21.15.82:0/688484685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28701021a0 0x7f2870102580 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 -- 172.21.15.82:0/688484685 >> 172.21.15.82:0/688484685 conn(0x7f28700793e0 msgr2=0x7f28700797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 -- 172.21.15.82:0/688484685 shutdown_connections 2024-04-21T08:56:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 -- 172.21.15.82:0/688484685 wait complete. 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 Processor -- start 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 -- start start 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28701021a0 0x7f287019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2870107270 0x7f287019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28701986b0 0x7f2870198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f28701122b0 con 0x7f28701021a0 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2870112130 con 0x7f2870107270 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f28778eb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2870112430 con 0x7f28701986b0 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f2875687640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28701021a0 0x7f287019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f2875687640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28701021a0 0x7f287019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59454/0 (socket says 172.21.15.82:59454) 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.811+0000 7f2875687640 1 -- 172.21.15.82:0/3655315735 learned_addr learned my addr 172.21.15.82:0/3655315735 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f2875e88640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28701986b0 0x7f2870198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f2875687640 1 -- 172.21.15.82:0/3655315735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28701986b0 msgr2=0x7f2870198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f2875687640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28701986b0 0x7f2870198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f2874e86640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2870107270 0x7f287019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f2875687640 1 -- 172.21.15.82:0/3655315735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2870107270 msgr2=0x7f287019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f2875687640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2870107270 0x7f287019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f2875687640 1 -- 172.21.15.82:0/3655315735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28701992b0 con 0x7f28701021a0 2024-04-21T08:56:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f2874e86640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2870107270 0x7f287019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:56:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f2875687640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28701021a0 0x7f287019e5f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f2860005940 tx=0x7f286002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f28667fc640 1 -- 172.21.15.82:0/3655315735 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860043020 con 0x7f28701021a0 2024-04-21T08:56:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f28667fc640 1 -- 172.21.15.82:0/3655315735 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f286002cc60 con 0x7f28701021a0 2024-04-21T08:56:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f28778eb640 1 -- 172.21.15.82:0/3655315735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2870199540 con 0x7f28701021a0 2024-04-21T08:56:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f28667fc640 1 -- 172.21.15.82:0/3655315735 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860003c30 con 0x7f28701021a0 2024-04-21T08:56:38.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.812+0000 7f28778eb640 1 -- 172.21.15.82:0/3655315735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28701a56e0 con 0x7f28701021a0 2024-04-21T08:56:38.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.814+0000 7f28667fc640 1 -- 172.21.15.82:0/3655315735 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2860046020 con 0x7f28701021a0 2024-04-21T08:56:38.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f287a5bc740 1 Processor -- start 2024-04-21T08:56:38.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f28667fc640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f285c0a3d20 0x7f285c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f287a5bc740 1 -- start start 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f28667fc640 1 -- 172.21.15.82:0/3655315735 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7f28600f2fc0 con 0x7f28701021a0 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f2875e88640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f285c0a3d20 0x7f285c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f287a5bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41580 0x560da3e41960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f287a5bc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 0x560da3e4a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f287a5bc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 0x560da3e4ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f287a5bc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560da3da61e0 con 0x560da3e41ea0 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f287a5bc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560da3d43390 con 0x560da3e4aa80 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f287a5bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560da3da21a0 con 0x560da3e41580 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f2874e86640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 0x560da3e4a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f2874e86640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 0x560da3e4a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59458/0 (socket says 172.21.15.82:59458) 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f2874e86640 1 -- 172.21.15.82:0/3506777593 learned_addr learned my addr 172.21.15.82:0/3506777593 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f2874e86640 1 -- 172.21.15.82:0/3506777593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41580 msgr2=0x560da3e41960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f2875e88640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f285c0a3d20 0x7f285c0a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2868007dd0 tx=0x7f2868007270 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:38.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f2875687640 1 --2- 172.21.15.82:0/3506777593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41580 0x560da3e41960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f2875e88640 1 --2- 172.21.15.82:0/3506777593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 0x560da3e4ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.815+0000 7f2874e86640 1 --2- 172.21.15.82:0/3506777593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41580 0x560da3e41960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.816+0000 7f2874e86640 1 -- 172.21.15.82:0/3506777593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 msgr2=0x560da3e4ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.816+0000 7f2874e86640 1 --2- 172.21.15.82:0/3506777593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 0x560da3e4ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.816+0000 7f2874e86640 1 -- 172.21.15.82:0/3506777593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da3b827d0 con 0x560da3e41ea0 2024-04-21T08:56:38.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.816+0000 7f2874e86640 1 --2- 172.21.15.82:0/3506777593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 0x560da3e4a540 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f286c00a9a0 tx=0x7f286c00ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a64f568aac4bc9d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:38.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.816+0000 7f285b7fe640 1 -- 172.21.15.82:0/3506777593 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c015070 con 0x560da3e41ea0 2024-04-21T08:56:38.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.816+0000 7f285b7fe640 1 -- 172.21.15.82:0/3506777593 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f286c00cc00 con 0x560da3e41ea0 2024-04-21T08:56:38.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.816+0000 7f285b7fe640 1 -- 172.21.15.82:0/3506777593 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c0185c0 con 0x560da3e41ea0 2024-04-21T08:56:38.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 -- 172.21.15.82:0/3506777593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 msgr2=0x560da3e4a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 --2- 172.21.15.82:0/3506777593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 0x560da3e4a540 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f286c00a9a0 tx=0x7f286c00ae60 comp rx=0 tx=0).stop 2024-04-21T08:56:38.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 -- 172.21.15.82:0/3506777593 shutdown_connections 2024-04-21T08:56:38.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 --2- 172.21.15.82:0/3506777593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 0x560da3e4ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 --2- 172.21.15.82:0/3506777593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 0x560da3e4a540 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 --2- 172.21.15.82:0/3506777593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41580 0x560da3e41960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 -- 172.21.15.82:0/3506777593 >> 172.21.15.82:0/3506777593 conn(0x560da3e3bcb0 msgr2=0x560da3e3c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:38.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 -- 172.21.15.82:0/3506777593 shutdown_connections 2024-04-21T08:56:38.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 -- 172.21.15.82:0/3506777593 wait complete. 2024-04-21T08:56:38.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 -- start start 2024-04-21T08:56:38.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 Processor -- start 2024-04-21T08:56:38.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41580 0x560da3e47a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 0x560da3e47fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 0x560da3e48500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560da3e4d870 con 0x560da3e41ea0 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560da3d5f2f0 con 0x560da3e4aa80 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560da3e4d9f0 con 0x560da3e41580 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f287a5bc740 10 client.0 _ll_register_callbacks cb 0x560da3d6c580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.817+0000 7f2874e86640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 0x560da3e47fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2874e86640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 0x560da3e47fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59460/0 (socket says 172.21.15.82:59460) 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2874e86640 1 -- 172.21.15.82:0/4204395535 learned_addr learned my addr 172.21.15.82:0/4204395535 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2875687640 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41580 0x560da3e47a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2874e86640 1 -- 172.21.15.82:0/4204395535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41580 msgr2=0x560da3e47a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2874e86640 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41580 0x560da3e47a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2874e86640 1 -- 172.21.15.82:0/4204395535 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 msgr2=0x560da3e48500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2874e86640 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 0x560da3e48500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2874e86640 1 -- 172.21.15.82:0/4204395535 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da3ab83f0 con 0x560da3e41ea0 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2875687640 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41580 0x560da3e47a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:56:38.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2874e86640 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 0x560da3e47fc0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f286c00a1a0 tx=0x7f286c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea57dc26b333f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2858ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2858ff9640 1 -- 172.21.15.82:0/4204395535 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c020070 con 0x560da3e41ea0 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2858ff9640 1 -- 172.21.15.82:0/4204395535 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7f286c01a650 con 0x560da3e41ea0 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f287a5bc740 1 -- 172.21.15.82:0/4204395535 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f286c00cc00 con 0x560da3e41ea0 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2858ff9640 1 -- 172.21.15.82:0/4204395535 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f286c015040 con 0x560da3e41ea0 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f287a5bc740 10 client.18435 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f287a5bc740 1 -- 172.21.15.82:0/4204395535 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560da3a780f0 con 0x560da3e41ea0 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.818+0000 7f2858ff9640 1 -- 172.21.15.82:0/4204395535 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f286c00cc00 con 0x560da3e41ea0 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.819+0000 7f2858ff9640 1 -- 172.21.15.82:0/4204395535 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f286c023790 con 0x560da3e41ea0 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.819+0000 7f287a5bc740 10 client.18435 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.819+0000 7f287a5bc740 10 client.18435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.819+0000 7f287a5bc740 20 client.18435 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.819+0000 7f287a5bc740 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560da3e5a9b0 0x560da3e5cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.819+0000 7f287a5bc740 4 client.18435 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.819+0000 7f287a5bc740 1 -- 172.21.15.82:0/4204395535 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560da3db3c00 con 0x560da3e5a9b0 2024-04-21T08:56:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.819+0000 7f2875687640 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560da3e5a9b0 0x560da3e5cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.819+0000 7f2858ff9640 10 client.18435 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.819+0000 7f2875687640 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560da3e5a9b0 0x560da3e5cda0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e121c8abdd777d23 server_cookie=cb9eec912f01f644 in_seq=0 out_seq=0 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f2858ff9640 1 -- 172.21.15.82:0/4204395535 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560da3db3c00 con 0x560da3e5a9b0 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f2858ff9640 10 client.18435 handle_command_reply: tid=0 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f287a5bc740 1 client.18435 shutdown 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f287a5bc740 1 -- 172.21.15.82:0/4204395535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 msgr2=0x560da3e47fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f287a5bc740 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 0x560da3e47fc0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f286c00a1a0 tx=0x7f286c011a30 comp rx=0 tx=0).stop 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f287a5bc740 1 -- 172.21.15.82:0/4204395535 shutdown_connections 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f287a5bc740 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560da3e5a9b0 0x560da3e5cda0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f287a5bc740 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 0x560da3e48500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f287a5bc740 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41ea0 0x560da3e47fc0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f2858ff9640 0 client.18435 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f287a5bc740 1 --2- 172.21.15.82:0/4204395535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41580 0x560da3e47a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f287a5bc740 1 -- 172.21.15.82:0/4204395535 >> 172.21.15.82:0/4204395535 conn(0x560da3e3bcb0 msgr2=0x560da3e3da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:38.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f287a5bc740 1 -- 172.21.15.82:0/4204395535 shutdown_connections 2024-04-21T08:56:38.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f287a5bc740 1 -- 172.21.15.82:0/4204395535 wait complete. 2024-04-21T08:56:38.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.820+0000 7f287a5bc740 20 client.18435 trim_cache size 0 max 16384 2024-04-21T08:56:38.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f287a5bc740 1 Processor -- start 2024-04-21T08:56:38.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f287a5bc740 1 -- start start 2024-04-21T08:56:38.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f287a5bc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41580 0x560da3e56760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f287a5bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41ea0 0x560da3e624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f287a5bc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 0x560da3e62a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f287a5bc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f28701af520 con 0x560da3e41580 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f287a5bc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f28701b10f0 con 0x560da3e4aa80 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f287a5bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28701afbf0 con 0x560da3e41ea0 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f2875687640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41580 0x560da3e56760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f2875687640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41580 0x560da3e56760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59476/0 (socket says 172.21.15.82:59476) 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f2875687640 1 -- 172.21.15.82:0/3799647939 learned_addr learned my addr 172.21.15.82:0/3799647939 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f2875687640 1 -- 172.21.15.82:0/3799647939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41ea0 msgr2=0x560da3e624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f2874e86640 1 --2- 172.21.15.82:0/3799647939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41ea0 0x560da3e624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f2875687640 1 --2- 172.21.15.82:0/3799647939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41ea0 0x560da3e624c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f2875687640 1 -- 172.21.15.82:0/3799647939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 msgr2=0x560da3e62a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f2875687640 1 --2- 172.21.15.82:0/3799647939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 0x560da3e62a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f2875687640 1 -- 172.21.15.82:0/3799647939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da3e4ce70 con 0x560da3e41580 2024-04-21T08:56:38.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.838+0000 7f2875687640 1 --2- 172.21.15.82:0/3799647939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41580 0x560da3e56760 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2860006010 tx=0x7f286004a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bcf68c78cbe463 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f285affd640 1 -- 172.21.15.82:0/3799647939 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28600c3010 con 0x560da3e41580 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f285affd640 1 -- 172.21.15.82:0/3799647939 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28600c78d0 con 0x560da3e41580 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f285affd640 1 -- 172.21.15.82:0/3799647939 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28600bb2d0 con 0x560da3e41580 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f287a5bc740 1 -- 172.21.15.82:0/3799647939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41580 msgr2=0x560da3e56760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f287a5bc740 1 --2- 172.21.15.82:0/3799647939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41580 0x560da3e56760 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2860006010 tx=0x7f286004a040 comp rx=0 tx=0).stop 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f287a5bc740 1 -- 172.21.15.82:0/3799647939 shutdown_connections 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f287a5bc740 1 --2- 172.21.15.82:0/3799647939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e4aa80 0x560da3e62a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f287a5bc740 1 --2- 172.21.15.82:0/3799647939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41ea0 0x560da3e624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f287a5bc740 1 --2- 172.21.15.82:0/3799647939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e41580 0x560da3e56760 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f287a5bc740 1 -- 172.21.15.82:0/3799647939 >> 172.21.15.82:0/3799647939 conn(0x560da3e3bcb0 msgr2=0x560da3e3ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f287a5bc740 1 -- 172.21.15.82:0/3799647939 shutdown_connections 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f287a5bc740 1 -- 172.21.15.82:0/3799647939 wait complete. 2024-04-21T08:56:38.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.839+0000 7f287a5bc740 1 -- start start 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f287a5bc740 1 Processor -- start 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f287a5bc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e41580 0x560da3e52c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f287a5bc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41ea0 0x560da3e531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f287a5bc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e4aa80 0x560da3e4fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f287a5bc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f28701b09f0 con 0x560da3e4aa80 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f287a5bc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f28701b02f0 con 0x560da3e41580 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f287a5bc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28701ad4f0 con 0x560da3e41ea0 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f287a5bc740 10 client.0 _ll_register_callbacks cb 0x7f286c00a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f2875e88640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e4aa80 0x560da3e4fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f2875e88640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e4aa80 0x560da3e4fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59492/0 (socket says 172.21.15.82:59492) 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f2875e88640 1 -- 172.21.15.82:0/1807556584 learned_addr learned my addr 172.21.15.82:0/1807556584 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f2874e86640 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41ea0 0x560da3e531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f2875687640 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e41580 0x560da3e52c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f2875e88640 1 -- 172.21.15.82:0/1807556584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41ea0 msgr2=0x560da3e531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f2875e88640 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41ea0 0x560da3e531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f2875e88640 1 -- 172.21.15.82:0/1807556584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e41580 msgr2=0x560da3e52c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:56:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f2875e88640 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e41580 0x560da3e52c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f2875e88640 1 -- 172.21.15.82:0/1807556584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da3e50300 con 0x560da3e4aa80 2024-04-21T08:56:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.840+0000 7f2875e88640 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e4aa80 0x560da3e4fc40 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f28680139c0 tx=0x7f2868013dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ff7412883e554c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:56:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f2858ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:56:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f2858ff9640 1 -- 172.21.15.82:0/1807556584 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2868011d20 con 0x560da3e4aa80 2024-04-21T08:56:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f2858ff9640 1 -- 172.21.15.82:0/1807556584 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9562+0+0 (secure 0 0 0) 0x7f286801e410 con 0x560da3e4aa80 2024-04-21T08:56:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f287a5bc740 1 -- 172.21.15.82:0/1807556584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28600c78d0 con 0x560da3e4aa80 2024-04-21T08:56:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f2858ff9640 1 -- 172.21.15.82:0/1807556584 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f286802c050 con 0x560da3e4aa80 2024-04-21T08:56:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f287a5bc740 10 client.18441 fetch_fsmap learned FSMap version 44 2024-04-21T08:56:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f287a5bc740 1 -- 172.21.15.82:0/1807556584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560da3ec1770 con 0x560da3e4aa80 2024-04-21T08:56:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f2858ff9640 1 -- 172.21.15.82:0/1807556584 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28600c78d0 con 0x560da3e4aa80 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f2858ff9640 1 -- 172.21.15.82:0/1807556584 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f286801d490 con 0x560da3e4aa80 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f287a5bc740 10 client.18441 fetch_fsmap finished waiting for FSMap version 44 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f287a5bc740 10 client.18441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f287a5bc740 20 client.18441 populate_metadata read hostname 'smithi082' 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f287a5bc740 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560da3ec1f70 0x560da3ec4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f287a5bc740 4 client.18441 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f287a5bc740 1 -- 172.21.15.82:0/1807556584 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560da3dada70 con 0x560da3ec1f70 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.841+0000 7f2875687640 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560da3ec1f70 0x560da3ec4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f2858ff9640 10 client.18441 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f2875687640 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560da3ec1f70 0x560da3ec4390 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aea1716917b643ab server_cookie=d9c0b4be0e34e8c5 in_seq=0 out_seq=0 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f2858ff9640 1 -- 172.21.15.82:0/1807556584 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560da3dada70 con 0x560da3ec1f70 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f2858ff9640 10 client.18441 handle_command_reply: tid=0 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f287a5bc740 1 client.18441 shutdown 2024-04-21T08:56:38.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f287a5bc740 1 -- 172.21.15.82:0/1807556584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e4aa80 msgr2=0x560da3e4fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f287a5bc740 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e4aa80 0x560da3e4fc40 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f28680139c0 tx=0x7f2868013dc0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f287a5bc740 1 -- 172.21.15.82:0/1807556584 shutdown_connections 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f287a5bc740 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x560da3ec1f70 0x560da3ec4390 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f287a5bc740 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560da3e4aa80 0x560da3e4fc40 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f287a5bc740 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da3e41ea0 0x560da3e531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f2858ff9640 0 client.18441 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f287a5bc740 1 --2- 172.21.15.82:0/1807556584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560da3e41580 0x560da3e52c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f287a5bc740 1 -- 172.21.15.82:0/1807556584 >> 172.21.15.82:0/1807556584 conn(0x560da3e3bcb0 msgr2=0x560da3e47a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f287a5bc740 1 -- 172.21.15.82:0/1807556584 shutdown_connections 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f287a5bc740 1 -- 172.21.15.82:0/1807556584 wait complete. 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.842+0000 7f287a5bc740 20 client.18441 trim_cache size 0 max 16384 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:56:38.849 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:56:38.850 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-21T08:56:38.850 INFO:teuthology.orchestra.run.smithi082.stdout: "1feca02f-cab9-4961-a2e1-bc02b14ba165": { 2024-04-21T08:56:38.850 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-21T08:56:38.850 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "1feca02f-cab9-4961-a2e1-bc02b14ba165", 2024-04-21T08:56:38.850 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2024-04-21T08:56:38.850 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:56:38.850 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:56:38.850 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:56:38.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.843+0000 7f28778eb640 1 -- 172.21.15.82:0/3655315735 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f285c0a3d20 msgr2=0x7f285c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.843+0000 7f28778eb640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f285c0a3d20 0x7f285c0a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2868007dd0 tx=0x7f2868007270 comp rx=0 tx=0).stop 2024-04-21T08:56:38.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.843+0000 7f28778eb640 1 -- 172.21.15.82:0/3655315735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28701021a0 msgr2=0x7f287019e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:56:38.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.843+0000 7f28778eb640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28701021a0 0x7f287019e5f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f2860005940 tx=0x7f286002c860 comp rx=0 tx=0).stop 2024-04-21T08:56:38.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.843+0000 7f28778eb640 1 -- 172.21.15.82:0/3655315735 shutdown_connections 2024-04-21T08:56:38.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.843+0000 7f28778eb640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28701986b0 0x7f2870198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.843+0000 7f28778eb640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2870107270 0x7f287019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.843+0000 7f28778eb640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f285c0a3d20 0x7f285c0a61e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.843+0000 7f28778eb640 1 --2- 172.21.15.82:0/3655315735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28701021a0 0x7f287019e5f0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:56:38.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.843+0000 7f28778eb640 1 -- 172.21.15.82:0/3655315735 >> 172.21.15.82:0/3655315735 conn(0x7f28700793e0 msgr2=0x7f2870100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:56:38.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.843+0000 7f28778eb640 1 -- 172.21.15.82:0/3655315735 shutdown_connections 2024-04-21T08:56:38.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:56:38.843+0000 7f28778eb640 1 -- 172.21.15.82:0/3655315735 wait complete. 2024-04-21T08:56:38.852 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:38 smithi082 ceph-mon[27967]: pgmap v297: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:38.853 INFO:tasks.cephfs.filesystem:scrub status for tag:1feca02f-cab9-4961-a2e1-bc02b14ba165 - {'path': '/', 'tag': '1feca02f-cab9-4961-a2e1-bc02b14ba165', 'options': 'recursive,force'} 2024-04-21T08:56:38.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:38 smithi106 ceph-mon[31072]: pgmap v297: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:38.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:38 smithi134 ceph-mon[31772]: pgmap v297: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:39.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:39 smithi082 ceph-mon[27967]: scrub queued for path: / 2024-04-21T08:56:39.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:39 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:56:39.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:39 smithi082 ceph-mon[27967]: scrub summary: active paths [/] 2024-04-21T08:56:39.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:39 smithi082 ceph-mon[27967]: scrub summary: idle+waiting paths [/] 2024-04-21T08:56:39.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:39 smithi106 ceph-mon[31072]: scrub queued for path: / 2024-04-21T08:56:39.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:39 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:56:39.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:39 smithi106 ceph-mon[31072]: scrub summary: active paths [/] 2024-04-21T08:56:39.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:39 smithi106 ceph-mon[31072]: scrub summary: idle+waiting paths [/] 2024-04-21T08:56:39.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:39 smithi134 ceph-mon[31772]: scrub queued for path: / 2024-04-21T08:56:39.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:39 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:56:39.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:39 smithi134 ceph-mon[31772]: scrub summary: active paths [/] 2024-04-21T08:56:39.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:39 smithi134 ceph-mon[31772]: scrub summary: idle+waiting paths [/] 2024-04-21T08:56:40.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:40 smithi082 ceph-mon[27967]: pgmap v298: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:40.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:40 smithi106 ceph-mon[31072]: pgmap v298: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:40.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:40 smithi134 ceph-mon[31772]: pgmap v298: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:56:42.779 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:42 smithi134 ceph-mon[31772]: pgmap v299: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-21T08:56:42.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:42 smithi082 ceph-mon[27967]: pgmap v299: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-21T08:56:42.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:42 smithi106 ceph-mon[31072]: pgmap v299: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-21T08:56:43.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:43 smithi082 ceph-mon[27967]: scrub summary: idle 2024-04-21T08:56:43.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:43 smithi106 ceph-mon[31072]: scrub summary: idle 2024-04-21T08:56:43.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:43 smithi134 ceph-mon[31772]: scrub summary: idle 2024-04-21T08:56:44.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:44 smithi082 ceph-mon[27967]: pgmap v300: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 14 op/s 2024-04-21T08:56:44.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:44 smithi106 ceph-mon[31072]: pgmap v300: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 14 op/s 2024-04-21T08:56:44.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:44 smithi134 ceph-mon[31772]: pgmap v300: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 14 op/s 2024-04-21T08:56:46.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:46 smithi082 ceph-mon[27967]: pgmap v301: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:56:46.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:46 smithi106 ceph-mon[31072]: pgmap v301: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:56:46.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:46 smithi134 ceph-mon[31772]: pgmap v301: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:56:48.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:48 smithi082 ceph-mon[27967]: pgmap v302: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 11 op/s 2024-04-21T08:56:48.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:48 smithi106 ceph-mon[31072]: pgmap v302: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 11 op/s 2024-04-21T08:56:48.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:48 smithi134 ceph-mon[31772]: pgmap v302: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 11 op/s 2024-04-21T08:56:50.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:50 smithi082 ceph-mon[27967]: pgmap v303: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-21T08:56:50.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:50 smithi106 ceph-mon[31072]: pgmap v303: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-21T08:56:50.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:50 smithi134 ceph-mon[31772]: pgmap v303: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-21T08:56:52.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:52 smithi082 ceph-mon[27967]: pgmap v304: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:56:52.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:52 smithi106 ceph-mon[31072]: pgmap v304: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:56:52.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:52 smithi134 ceph-mon[31772]: pgmap v304: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:56:54.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:54 smithi082 ceph-mon[27967]: pgmap v305: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 14 op/s 2024-04-21T08:56:54.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:54 smithi106 ceph-mon[31072]: pgmap v305: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 14 op/s 2024-04-21T08:56:54.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:54 smithi134 ceph-mon[31772]: pgmap v305: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 14 op/s 2024-04-21T08:56:56.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:56 smithi082 ceph-mon[27967]: pgmap v306: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:56:56.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:56 smithi106 ceph-mon[31072]: pgmap v306: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:56:56.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:56 smithi134 ceph-mon[31772]: pgmap v306: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:56:58.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:58 smithi082 ceph-mon[27967]: pgmap v307: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:56:58.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:58 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:56:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:58 smithi106 ceph-mon[31072]: pgmap v307: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:56:58.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:58 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:56:58.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:58 smithi134 ceph-mon[31772]: pgmap v307: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:56:58.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:58 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:56:59.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:56:59 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:56:59.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:56:59 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:56:59.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:56:59 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:57:00.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:00 smithi082 ceph-mon[27967]: pgmap v308: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 op/s 2024-04-21T08:57:00.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:00 smithi106 ceph-mon[31072]: pgmap v308: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 op/s 2024-04-21T08:57:00.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:00 smithi134 ceph-mon[31772]: pgmap v308: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 op/s 2024-04-21T08:57:02.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:02 smithi082 ceph-mon[27967]: pgmap v309: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:57:02.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:02 smithi106 ceph-mon[31072]: pgmap v309: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:57:02.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:02 smithi134 ceph-mon[31772]: pgmap v309: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:57:04.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:04 smithi082 ceph-mon[27967]: pgmap v310: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-21T08:57:04.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:04 smithi106 ceph-mon[31072]: pgmap v310: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-21T08:57:04.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:04 smithi134 ceph-mon[31772]: pgmap v310: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-21T08:57:06.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:06 smithi082 ceph-mon[27967]: pgmap v311: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-21T08:57:06.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:06 smithi106 ceph-mon[31072]: pgmap v311: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-21T08:57:06.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:06 smithi134 ceph-mon[31772]: pgmap v311: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-21T08:57:08.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:57:08.854 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:57:08.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:08 smithi082 ceph-mon[27967]: pgmap v312: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-04-21T08:57:08.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:08 smithi106 ceph-mon[31072]: pgmap v312: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-04-21T08:57:08.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:08 smithi134 ceph-mon[31772]: pgmap v312: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-04-21T08:57:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.034+0000 7f166503f640 1 -- 172.21.15.82:0/935986526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f166011b730 msgr2=0x7f166011db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.034+0000 7f166503f640 1 --2- 172.21.15.82:0/935986526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f166011b730 0x7f166011db20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f165401a940 tx=0x7f165403cca0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.035+0000 7f166503f640 1 -- 172.21.15.82:0/935986526 shutdown_connections 2024-04-21T08:57:09.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.035+0000 7f166503f640 1 --2- 172.21.15.82:0/935986526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f166011b730 0x7f166011db20 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.035+0000 7f166503f640 1 --2- 172.21.15.82:0/935986526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f166010e7b0 0x7f166010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.035+0000 7f166503f640 1 --2- 172.21.15.82:0/935986526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16601106d0 0x7f166010e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.035+0000 7f166503f640 1 -- 172.21.15.82:0/935986526 >> 172.21.15.82:0/935986526 conn(0x7f166006d070 msgr2=0x7f166006d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:09.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.036+0000 7f166503f640 1 -- 172.21.15.82:0/935986526 shutdown_connections 2024-04-21T08:57:09.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.037+0000 7f166503f640 1 -- 172.21.15.82:0/935986526 wait complete. 2024-04-21T08:57:09.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.038+0000 7f166503f640 1 Processor -- start 2024-04-21T08:57:09.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.040+0000 7f166503f640 1 -- start start 2024-04-21T08:57:09.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.040+0000 7f166503f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f166010e7b0 0x7f166011af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.040+0000 7f166503f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16601106d0 0x7f1660116fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.040+0000 7f166503f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f166011b730 0x7f16601174e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.040+0000 7f166503f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1660113850 con 0x7f166011b730 2024-04-21T08:57:09.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.040+0000 7f166503f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f16601136d0 con 0x7f16601106d0 2024-04-21T08:57:09.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.040+0000 7f166503f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16601139d0 con 0x7f166010e7b0 2024-04-21T08:57:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.041+0000 7f165f59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f166011b730 0x7f16601174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.041+0000 7f165f59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f166011b730 0x7f16601174e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33254/0 (socket says 172.21.15.82:33254) 2024-04-21T08:57:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.041+0000 7f165f59e640 1 -- 172.21.15.82:0/4230468356 learned_addr learned my addr 172.21.15.82:0/4230468356 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.041+0000 7f165e59c640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16601106d0 0x7f1660116fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.041+0000 7f165ed9d640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f166010e7b0 0x7f166011af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.042+0000 7f165f59e640 1 -- 172.21.15.82:0/4230468356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f166010e7b0 msgr2=0x7f166011af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.042+0000 7f165f59e640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f166010e7b0 0x7f166011af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.042+0000 7f165f59e640 1 -- 172.21.15.82:0/4230468356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16601106d0 msgr2=0x7f1660116fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.042+0000 7f165f59e640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16601106d0 0x7f1660116fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.042+0000 7f165f59e640 1 -- 172.21.15.82:0/4230468356 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1660117c60 con 0x7f166011b730 2024-04-21T08:57:09.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.042+0000 7f165f59e640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f166011b730 0x7f16601174e0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f165401a860 tx=0x7f1654004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:09.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.042+0000 7f1647fff640 1 -- 172.21.15.82:0/4230468356 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654052020 con 0x7f166011b730 2024-04-21T08:57:09.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.042+0000 7f166503f640 1 -- 172.21.15.82:0/4230468356 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16601ab4d0 con 0x7f166011b730 2024-04-21T08:57:09.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.043+0000 7f166503f640 1 -- 172.21.15.82:0/4230468356 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16601ab990 con 0x7f166011b730 2024-04-21T08:57:09.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.044+0000 7f1647fff640 1 -- 172.21.15.82:0/4230468356 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f165403dd00 con 0x7f166011b730 2024-04-21T08:57:09.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.044+0000 7f1647fff640 1 -- 172.21.15.82:0/4230468356 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654003c60 con 0x7f166011b730 2024-04-21T08:57:09.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.047+0000 7f1667c2b740 1 Processor -- start 2024-04-21T08:57:09.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.048+0000 7f1647fff640 1 -- 172.21.15.82:0/4230468356 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1654055020 con 0x7f166011b730 2024-04-21T08:57:09.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.049+0000 7f1647fff640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f16480a3f20 0x7f16480a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.049+0000 7f1647fff640 1 -- 172.21.15.82:0/4230468356 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f16541037a0 con 0x7f166011b730 2024-04-21T08:57:09.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.049+0000 7f1667c2b740 1 -- start start 2024-04-21T08:57:09.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.049+0000 7f1667c2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf54580 0x558afdf54960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.049+0000 7f1667c2b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54ea0 0x558afdf5d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.049+0000 7f1667c2b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf5da80 0x558afdf5fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.050+0000 7f1667c2b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558afdeb91e0 con 0x558afdf54ea0 2024-04-21T08:57:09.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.050+0000 7f1667c2b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558afde56390 con 0x558afdf5da80 2024-04-21T08:57:09.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.050+0000 7f1667c2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558afdeb51a0 con 0x558afdf54580 2024-04-21T08:57:09.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.050+0000 7f165ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf54580 0x558afdf54960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.050+0000 7f165ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf54580 0x558afdf54960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52214/0 (socket says 172.21.15.82:52214) 2024-04-21T08:57:09.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.050+0000 7f165ed9d640 1 -- 172.21.15.82:0/129802077 learned_addr learned my addr 172.21.15.82:0/129802077 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:09.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.050+0000 7f165f59e640 1 --2- 172.21.15.82:0/129802077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf5da80 0x558afdf5fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.050+0000 7f165e59c640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f16480a3f20 0x7f16480a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.050+0000 7f165ed9d640 1 -- 172.21.15.82:0/129802077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf5da80 msgr2=0x558afdf5fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:09.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.051+0000 7f165ed9d640 1 --2- 172.21.15.82:0/129802077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf5da80 0x558afdf5fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.051+0000 7f165ed9d640 1 -- 172.21.15.82:0/129802077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54ea0 msgr2=0x558afdf5d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:09.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.051+0000 7f165ed9d640 1 --2- 172.21.15.82:0/129802077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54ea0 0x558afdf5d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.051+0000 7f165ed9d640 1 -- 172.21.15.82:0/129802077 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558afdc957d0 con 0x558afdf54580 2024-04-21T08:57:09.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.051+0000 7f165ed9d640 1 --2- 172.21.15.82:0/129802077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf54580 0x558afdf54960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1650009a40 tx=0x7f1650009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f54459e2dfeaa4b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:09.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.051+0000 7f16457fa640 1 -- 172.21.15.82:0/129802077 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650017070 con 0x558afdf54580 2024-04-21T08:57:09.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.051+0000 7f165e59c640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f16480a3f20 0x7f16480a63e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x558afdf54050 tx=0x7f16580099f0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:09.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.051+0000 7f16457fa640 1 -- 172.21.15.82:0/129802077 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1650013470 con 0x558afdf54580 2024-04-21T08:57:09.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.052+0000 7f16457fa640 1 -- 172.21.15.82:0/129802077 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650012630 con 0x558afdf54580 2024-04-21T08:57:09.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.053+0000 7f1667c2b740 1 -- 172.21.15.82:0/129802077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf54580 msgr2=0x558afdf54960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.053+0000 7f1667c2b740 1 --2- 172.21.15.82:0/129802077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf54580 0x558afdf54960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1650009a40 tx=0x7f1650009f00 comp rx=0 tx=0).stop 2024-04-21T08:57:09.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.056+0000 7f1667c2b740 1 -- 172.21.15.82:0/129802077 shutdown_connections 2024-04-21T08:57:09.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.056+0000 7f1667c2b740 1 --2- 172.21.15.82:0/129802077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf5da80 0x558afdf5fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.056+0000 7f1667c2b740 1 --2- 172.21.15.82:0/129802077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54ea0 0x558afdf5d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.056+0000 7f1667c2b740 1 --2- 172.21.15.82:0/129802077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf54580 0x558afdf54960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.056+0000 7f1667c2b740 1 -- 172.21.15.82:0/129802077 >> 172.21.15.82:0/129802077 conn(0x558afdf4ecb0 msgr2=0x558afdf4f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:09.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.060+0000 7f1667c2b740 1 -- 172.21.15.82:0/129802077 shutdown_connections 2024-04-21T08:57:09.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.061+0000 7f1667c2b740 1 -- 172.21.15.82:0/129802077 wait complete. 2024-04-21T08:57:09.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.062+0000 7f1667c2b740 1 -- start start 2024-04-21T08:57:09.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.062+0000 7f1667c2b740 1 Processor -- start 2024-04-21T08:57:09.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.063+0000 7f1667c2b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf5aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.063+0000 7f1667c2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf54ea0 0x558afdf5af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.063+0000 7f165ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf5aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.063+0000 7f165ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf5aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33262/0 (socket says 172.21.15.82:33262) 2024-04-21T08:57:09.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.063+0000 7f1667c2b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf5da80 0x558afdf5b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.063+0000 7f1667c2b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x558afdf60870 con 0x558afdf54580 2024-04-21T08:57:09.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.063+0000 7f1667c2b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558afde722f0 con 0x558afdf5da80 2024-04-21T08:57:09.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.063+0000 7f1667c2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558afdf609f0 con 0x558afdf54ea0 2024-04-21T08:57:09.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.063+0000 7f1667c2b740 10 client.0 _ll_register_callbacks cb 0x558afde7f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:57:09.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.063+0000 7f165ed9d640 1 -- 172.21.15.82:0/243121045 learned_addr learned my addr 172.21.15.82:0/243121045 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:09.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.063+0000 7f165f59e640 1 --2- 172.21.15.82:0/243121045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf5da80 0x558afdf5b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.064+0000 7f165ed9d640 1 -- 172.21.15.82:0/243121045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf54ea0 msgr2=0x558afdf5af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:09.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.064+0000 7f165ed9d640 1 --2- 172.21.15.82:0/243121045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf54ea0 0x558afdf5af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.064+0000 7f165ed9d640 1 -- 172.21.15.82:0/243121045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf5da80 msgr2=0x558afdf5b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:09.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.064+0000 7f165ed9d640 1 --2- 172.21.15.82:0/243121045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf5da80 0x558afdf5b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.064+0000 7f165ed9d640 1 -- 172.21.15.82:0/243121045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558afdbcb3f0 con 0x558afdf54580 2024-04-21T08:57:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.066+0000 7f165ed9d640 1 --2- 172.21.15.82:0/243121045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf5aa40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f1650013990 tx=0x7f16500139c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d1520fd381add3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.066+0000 7f1642ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:57:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.066+0000 7f1642ff5640 1 -- 172.21.15.82:0/243121045 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650017050 con 0x558afdf54580 2024-04-21T08:57:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.067+0000 7f1642ff5640 1 -- 172.21.15.82:0/243121045 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f165001c660 con 0x558afdf54580 2024-04-21T08:57:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.067+0000 7f1667c2b740 1 -- 172.21.15.82:0/243121045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1650013470 con 0x558afdf54580 2024-04-21T08:57:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.067+0000 7f1642ff5640 1 -- 172.21.15.82:0/243121045 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f165001c860 con 0x558afdf54580 2024-04-21T08:57:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.068+0000 7f1667c2b740 10 client.18450 fetch_fsmap learned FSMap version 46 2024-04-21T08:57:09.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.068+0000 7f1667c2b740 1 -- 172.21.15.82:0/243121045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558afdb8b0f0 con 0x558afdf54580 2024-04-21T08:57:09.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.069+0000 7f1642ff5640 1 -- 172.21.15.82:0/243121045 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1650030420 con 0x558afdf54580 2024-04-21T08:57:09.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.070+0000 7f1642ff5640 1 -- 172.21.15.82:0/243121045 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f1650026410 con 0x558afdf54580 2024-04-21T08:57:09.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.070+0000 7f1667c2b740 10 client.18450 fetch_fsmap finished waiting for FSMap version 46 2024-04-21T08:57:09.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.070+0000 7f1667c2b740 10 client.18450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:57:09.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.070+0000 7f1667c2b740 20 client.18450 populate_metadata read hostname 'smithi082' 2024-04-21T08:57:09.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.070+0000 7f1667c2b740 1 --2- 172.21.15.82:0/243121045 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558afdf6d6f0 0x558afdf6fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.070+0000 7f1667c2b740 4 client.18450 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:57:09.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.070+0000 7f1667c2b740 1 -- 172.21.15.82:0/243121045 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1650013470 con 0x558afdf6d6f0 2024-04-21T08:57:09.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.071+0000 7f165e59c640 1 --2- 172.21.15.82:0/243121045 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558afdf6d6f0 0x558afdf6fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.076+0000 7f165e59c640 1 --2- 172.21.15.82:0/243121045 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558afdf6d6f0 0x558afdf6fae0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6f67a58c3de55b7 server_cookie=49ba3c6a9cc14882 in_seq=0 out_seq=0 2024-04-21T08:57:09.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.076+0000 7f1642ff5640 10 client.18450 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:57:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.077+0000 7f1642ff5640 1 -- 172.21.15.82:0/243121045 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f1650013470 con 0x558afdf6d6f0 2024-04-21T08:57:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.077+0000 7f1642ff5640 10 client.18450 handle_command_reply: tid=0 2024-04-21T08:57:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.077+0000 7f1667c2b740 1 client.18450 shutdown 2024-04-21T08:57:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.077+0000 7f1667c2b740 1 -- 172.21.15.82:0/243121045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 msgr2=0x558afdf5aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.077+0000 7f1667c2b740 1 --2- 172.21.15.82:0/243121045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf5aa40 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f1650013990 tx=0x7f16500139c0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.082+0000 7f1667c2b740 1 -- 172.21.15.82:0/243121045 shutdown_connections 2024-04-21T08:57:09.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.082+0000 7f1667c2b740 1 --2- 172.21.15.82:0/243121045 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558afdf6d6f0 0x558afdf6fae0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.082+0000 7f1667c2b740 1 --2- 172.21.15.82:0/243121045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf5da80 0x558afdf5b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.082+0000 7f1667c2b740 1 --2- 172.21.15.82:0/243121045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf54ea0 0x558afdf5af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.082+0000 7f1667c2b740 1 --2- 172.21.15.82:0/243121045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf5aa40 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.082+0000 7f1667c2b740 1 -- 172.21.15.82:0/243121045 >> 172.21.15.82:0/243121045 conn(0x558afdf4ecb0 msgr2=0x558afdf50a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:09.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.082+0000 7f1642ff5640 0 client.18450 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:57:09.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.084+0000 7f1667c2b740 1 -- 172.21.15.82:0/243121045 shutdown_connections 2024-04-21T08:57:09.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.085+0000 7f1667c2b740 1 -- 172.21.15.82:0/243121045 wait complete. 2024-04-21T08:57:09.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.087+0000 7f1667c2b740 20 client.18450 trim_cache size 0 max 16384 2024-04-21T08:57:09.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.126+0000 7f1667c2b740 1 Processor -- start 2024-04-21T08:57:09.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.126+0000 7f1667c2b740 1 -- start start 2024-04-21T08:57:09.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.126+0000 7f1667c2b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf5d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.126+0000 7f1667c2b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf54ea0 0x558afdf4dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.127+0000 7f1667c2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf5da80 0x558afdf5b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.127+0000 7f1667c2b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f164c0102b0 con 0x558afdf54580 2024-04-21T08:57:09.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.127+0000 7f1667c2b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f164c011e20 con 0x558afdf54ea0 2024-04-21T08:57:09.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.127+0000 7f1667c2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f164c010980 con 0x558afdf5da80 2024-04-21T08:57:09.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.127+0000 7f165f59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf5da80 0x558afdf5b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.127+0000 7f165e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf54ea0 0x558afdf4dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.127+0000 7f165f59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf5da80 0x558afdf5b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52228/0 (socket says 172.21.15.82:52228) 2024-04-21T08:57:09.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.127+0000 7f165f59e640 1 -- 172.21.15.82:0/3831138425 learned_addr learned my addr 172.21.15.82:0/3831138425 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:09.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.128+0000 7f165ed9d640 1 --2- 172.21.15.82:0/3831138425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf5d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.130+0000 7f165ed9d640 1 -- 172.21.15.82:0/3831138425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf5da80 msgr2=0x558afdf5b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:09.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.130+0000 7f165ed9d640 1 --2- 172.21.15.82:0/3831138425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf5da80 0x558afdf5b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.130+0000 7f165ed9d640 1 -- 172.21.15.82:0/3831138425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf54ea0 msgr2=0x558afdf4dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:09.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.130+0000 7f165ed9d640 1 --2- 172.21.15.82:0/3831138425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf54ea0 0x558afdf4dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.130+0000 7f165ed9d640 1 -- 172.21.15.82:0/3831138425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558afdf538a0 con 0x558afdf54580 2024-04-21T08:57:09.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.131+0000 7f165ed9d640 1 --2- 172.21.15.82:0/3831138425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf5d170 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1650009990 tx=0x7f1650006e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8872f6984597b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:09.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.133+0000 7f1644ff9640 1 -- 172.21.15.82:0/3831138425 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650035070 con 0x558afdf54580 2024-04-21T08:57:09.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.133+0000 7f1644ff9640 1 -- 172.21.15.82:0/3831138425 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1650013470 con 0x558afdf54580 2024-04-21T08:57:09.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.134+0000 7f1667c2b740 1 -- 172.21.15.82:0/3831138425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 msgr2=0x558afdf5d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.134+0000 7f1667c2b740 1 --2- 172.21.15.82:0/3831138425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf5d170 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1650009990 tx=0x7f1650006e70 comp rx=0 tx=0).stop 2024-04-21T08:57:09.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.137+0000 7f1667c2b740 1 -- 172.21.15.82:0/3831138425 shutdown_connections 2024-04-21T08:57:09.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.137+0000 7f1667c2b740 1 --2- 172.21.15.82:0/3831138425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf5da80 0x558afdf5b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.137+0000 7f1667c2b740 1 --2- 172.21.15.82:0/3831138425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf54ea0 0x558afdf4dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.137+0000 7f1667c2b740 1 --2- 172.21.15.82:0/3831138425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf5d170 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.137+0000 7f1667c2b740 1 -- 172.21.15.82:0/3831138425 >> 172.21.15.82:0/3831138425 conn(0x558afdf4ecb0 msgr2=0x558afdf4fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:09.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.141+0000 7f1667c2b740 1 -- 172.21.15.82:0/3831138425 shutdown_connections 2024-04-21T08:57:09.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.141+0000 7f1667c2b740 1 -- 172.21.15.82:0/3831138425 wait complete. 2024-04-21T08:57:09.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.142+0000 7f1667c2b740 1 -- start start 2024-04-21T08:57:09.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.142+0000 7f1667c2b740 1 Processor -- start 2024-04-21T08:57:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.149+0000 7f1667c2b740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf65ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.149+0000 7f1667c2b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf54ea0 0x558afdf661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.149+0000 7f1667c2b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf5da80 0x558afdf62c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.149+0000 7f1667c2b740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f164c011750 con 0x558afdf54580 2024-04-21T08:57:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.149+0000 7f1667c2b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f164c011050 con 0x558afdf54ea0 2024-04-21T08:57:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.149+0000 7f1667c2b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f164c00e280 con 0x558afdf5da80 2024-04-21T08:57:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.149+0000 7f1667c2b740 10 client.0 _ll_register_callbacks cb 0x7f16500098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:57:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.150+0000 7f165ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf65ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.150+0000 7f165ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf65ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33280/0 (socket says 172.21.15.82:33280) 2024-04-21T08:57:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.150+0000 7f165ed9d640 1 -- 172.21.15.82:0/4238007123 learned_addr learned my addr 172.21.15.82:0/4238007123 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.150+0000 7f165f59e640 1 --2- 172.21.15.82:0/4238007123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf5da80 0x558afdf62c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.150+0000 7f165ed9d640 1 -- 172.21.15.82:0/4238007123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf5da80 msgr2=0x558afdf62c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.150+0000 7f165ed9d640 1 --2- 172.21.15.82:0/4238007123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf5da80 0x558afdf62c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.150+0000 7f165ed9d640 1 -- 172.21.15.82:0/4238007123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf54ea0 msgr2=0x558afdf661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.150+0000 7f165ed9d640 1 --2- 172.21.15.82:0/4238007123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf54ea0 0x558afdf661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.150+0000 7f165ed9d640 1 -- 172.21.15.82:0/4238007123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558afdf632b0 con 0x558afdf54580 2024-04-21T08:57:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.151+0000 7f165ed9d640 1 --2- 172.21.15.82:0/4238007123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf65ca0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f16500267e0 tx=0x7f16500090e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f8c0a4ce7e330a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.153+0000 7f1642ff5640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:57:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.153+0000 7f1642ff5640 1 -- 172.21.15.82:0/4238007123 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650035070 con 0x558afdf54580 2024-04-21T08:57:09.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.154+0000 7f1667c2b740 1 -- 172.21.15.82:0/4238007123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1650013470 con 0x558afdf54580 2024-04-21T08:57:09.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.154+0000 7f1642ff5640 1 -- 172.21.15.82:0/4238007123 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f1650013970 con 0x558afdf54580 2024-04-21T08:57:09.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.154+0000 7f1642ff5640 1 -- 172.21.15.82:0/4238007123 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f165003e050 con 0x558afdf54580 2024-04-21T08:57:09.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.154+0000 7f1667c2b740 10 client.18456 fetch_fsmap learned FSMap version 46 2024-04-21T08:57:09.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.154+0000 7f1667c2b740 1 -- 172.21.15.82:0/4238007123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558afdfd4340 con 0x558afdf54580 2024-04-21T08:57:09.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.155+0000 7f1642ff5640 1 -- 172.21.15.82:0/4238007123 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1650013470 con 0x558afdf54580 2024-04-21T08:57:09.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.156+0000 7f1642ff5640 1 -- 172.21.15.82:0/4238007123 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f1650038020 con 0x558afdf54580 2024-04-21T08:57:09.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.157+0000 7f1667c2b740 10 client.18456 fetch_fsmap finished waiting for FSMap version 46 2024-04-21T08:57:09.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.158+0000 7f1667c2b740 10 client.18456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:57:09.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.158+0000 7f1667c2b740 20 client.18456 populate_metadata read hostname 'smithi082' 2024-04-21T08:57:09.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.158+0000 7f1667c2b740 1 --2- 172.21.15.82:0/4238007123 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558afdfd4930 0x558afdfd6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.158+0000 7f1667c2b740 4 client.18456 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:57:09.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.158+0000 7f1667c2b740 1 -- 172.21.15.82:0/4238007123 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558afdec6c00 con 0x558afdfd4930 2024-04-21T08:57:09.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.159+0000 7f165e59c640 1 --2- 172.21.15.82:0/4238007123 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558afdfd4930 0x558afdfd6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.168+0000 7f165e59c640 1 --2- 172.21.15.82:0/4238007123 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558afdfd4930 0x558afdfd6d70 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6130df0d9a460e57 server_cookie=f8bb171422e7fab3 in_seq=0 out_seq=0 2024-04-21T08:57:09.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.168+0000 7f1642ff5640 10 client.18456 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:57:09.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.172+0000 7f1642ff5640 1 -- 172.21.15.82:0/4238007123 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558afdec6c00 con 0x558afdfd4930 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.172+0000 7f1642ff5640 10 client.18456 handle_command_reply: tid=0 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.173+0000 7f1667c2b740 1 client.18456 shutdown 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.174+0000 7f1667c2b740 1 -- 172.21.15.82:0/4238007123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 msgr2=0x558afdf65ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.174+0000 7f1667c2b740 1 --2- 172.21.15.82:0/4238007123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf65ca0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f16500267e0 tx=0x7f16500090e0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.174+0000 7f1667c2b740 1 -- 172.21.15.82:0/4238007123 shutdown_connections 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.174+0000 7f1667c2b740 1 --2- 172.21.15.82:0/4238007123 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x558afdfd4930 0x558afdfd6d70 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.174+0000 7f1667c2b740 1 --2- 172.21.15.82:0/4238007123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558afdf5da80 0x558afdf62c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.174+0000 7f1667c2b740 1 --2- 172.21.15.82:0/4238007123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558afdf54ea0 0x558afdf661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.174+0000 7f1667c2b740 1 --2- 172.21.15.82:0/4238007123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558afdf54580 0x558afdf65ca0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.174+0000 7f1667c2b740 1 -- 172.21.15.82:0/4238007123 >> 172.21.15.82:0/4238007123 conn(0x558afdf4ecb0 msgr2=0x558afdf4dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.174+0000 7f1642ff5640 0 client.18456 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.174+0000 7f1667c2b740 1 -- 172.21.15.82:0/4238007123 shutdown_connections 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.174+0000 7f1667c2b740 1 -- 172.21.15.82:0/4238007123 wait complete. 2024-04-21T08:57:09.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.174+0000 7f1667c2b740 20 client.18456 trim_cache size 0 max 16384 2024-04-21T08:57:09.176 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:57:09.176 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-21T08:57:09.176 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-21T08:57:09.176 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:57:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.177+0000 7f166503f640 1 -- 172.21.15.82:0/4230468356 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f16480a3f20 msgr2=0x7f16480a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.177+0000 7f166503f640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f16480a3f20 0x7f16480a63e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x558afdf54050 tx=0x7f16580099f0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.177+0000 7f166503f640 1 -- 172.21.15.82:0/4230468356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f166011b730 msgr2=0x7f16601174e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.177+0000 7f166503f640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f166011b730 0x7f16601174e0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f165401a860 tx=0x7f1654004040 comp rx=0 tx=0).stop 2024-04-21T08:57:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.178+0000 7f166503f640 1 -- 172.21.15.82:0/4230468356 shutdown_connections 2024-04-21T08:57:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.178+0000 7f166503f640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f16480a3f20 0x7f16480a63e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.178+0000 7f166503f640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f166011b730 0x7f16601174e0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.178+0000 7f166503f640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16601106d0 0x7f1660116fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.178+0000 7f166503f640 1 --2- 172.21.15.82:0/4230468356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f166010e7b0 0x7f166011af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.178+0000 7f166503f640 1 -- 172.21.15.82:0/4230468356 >> 172.21.15.82:0/4230468356 conn(0x7f166006d070 msgr2=0x7f166011c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:09.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.179+0000 7f166503f640 1 -- 172.21.15.82:0/4230468356 shutdown_connections 2024-04-21T08:57:09.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.179+0000 7f166503f640 1 -- 172.21.15.82:0/4230468356 wait complete. 2024-04-21T08:57:09.198 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:57:09.198 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:57:09.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.410+0000 7f433363f640 1 -- 172.21.15.82:0/639316196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f432c101130 msgr2=0x7f432c1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.410+0000 7f433363f640 1 --2- 172.21.15.82:0/639316196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f432c101130 0x7f432c1015b0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4328009900 tx=0x7f432802bc80 comp rx=0 tx=0).stop 2024-04-21T08:57:09.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.414+0000 7f433363f640 1 -- 172.21.15.82:0/639316196 shutdown_connections 2024-04-21T08:57:09.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.414+0000 7f433363f640 1 --2- 172.21.15.82:0/639316196 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f432c101af0 0x7f432c10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.414+0000 7f433363f640 1 --2- 172.21.15.82:0/639316196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f432c101130 0x7f432c1015b0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.414+0000 7f433363f640 1 --2- 172.21.15.82:0/639316196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f432c105e80 0x7f432c106260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.414+0000 7f433363f640 1 -- 172.21.15.82:0/639316196 >> 172.21.15.82:0/639316196 conn(0x7f432c077e30 msgr2=0x7f432c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:09.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.416+0000 7f433363f640 1 -- 172.21.15.82:0/639316196 shutdown_connections 2024-04-21T08:57:09.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.419+0000 7f433363f640 1 -- 172.21.15.82:0/639316196 wait complete. 2024-04-21T08:57:09.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.419+0000 7f433363f640 1 Processor -- start 2024-04-21T08:57:09.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.420+0000 7f433363f640 1 -- start start 2024-04-21T08:57:09.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.420+0000 7f433363f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f432c101130 0x7f432c196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.420+0000 7f433363f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f432c101af0 0x7f432c196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.420+0000 7f433363f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f432c105e80 0x7f432c19af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.420+0000 7f43313db640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f432c101130 0x7f432c196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.420+0000 7f43313db640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f432c101130 0x7f432c196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52238/0 (socket says 172.21.15.82:52238) 2024-04-21T08:57:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.421+0000 7f43313db640 1 -- 172.21.15.82:0/2091317854 learned_addr learned my addr 172.21.15.82:0/2091317854 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.421+0000 7f4330bda640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f432c101af0 0x7f432c196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.421+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f432c1101b0 con 0x7f432c105e80 2024-04-21T08:57:09.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.421+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f432c110030 con 0x7f432c101af0 2024-04-21T08:57:09.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.421+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f432c110330 con 0x7f432c101130 2024-04-21T08:57:09.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.422+0000 7f4331bdc640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f432c105e80 0x7f432c19af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.422+0000 7f4331bdc640 1 -- 172.21.15.82:0/2091317854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f432c101130 msgr2=0x7f432c196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.422+0000 7f4331bdc640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f432c101130 0x7f432c196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.422+0000 7f4331bdc640 1 -- 172.21.15.82:0/2091317854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f432c101af0 msgr2=0x7f432c196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.422+0000 7f4331bdc640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f432c101af0 0x7f432c196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.422+0000 7f4331bdc640 1 -- 172.21.15.82:0/2091317854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c19b640 con 0x7f432c105e80 2024-04-21T08:57:09.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.422+0000 7f4331bdc640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f432c105e80 0x7f432c19af10 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f432400e940 tx=0x7f432400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:09.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.424+0000 7f43227fc640 1 -- 172.21.15.82:0/2091317854 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324004d20 con 0x7f432c105e80 2024-04-21T08:57:09.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.424+0000 7f43227fc640 1 -- 172.21.15.82:0/2091317854 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4324005020 con 0x7f432c105e80 2024-04-21T08:57:09.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.425+0000 7f43227fc640 1 -- 172.21.15.82:0/2091317854 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43240116d0 con 0x7f432c105e80 2024-04-21T08:57:09.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.425+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c19b930 con 0x7f432c105e80 2024-04-21T08:57:09.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.425+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c1a31c0 con 0x7f432c105e80 2024-04-21T08:57:09.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.430+0000 7f43227fc640 1 -- 172.21.15.82:0/2091317854 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f432400cb20 con 0x7f432c105e80 2024-04-21T08:57:09.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.431+0000 7f43227fc640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f43180a8210 0x7f43180aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:09.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.432+0000 7f43227fc640 1 -- 172.21.15.82:0/2091317854 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f43240d0ee0 con 0x7f432c105e80 2024-04-21T08:57:09.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.432+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432c1025f0 con 0x7f432c105e80 2024-04-21T08:57:09.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.432+0000 7f43313db640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f43180a8210 0x7f43180aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:09.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.433+0000 7f43313db640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f43180a8210 0x7f43180aa6d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f431c009a30 tx=0x7f431c009190 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:09.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.441+0000 7f43227fc640 1 -- 172.21.15.82:0/2091317854 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f432401f020 con 0x7f432c105e80 2024-04-21T08:57:09.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:09 smithi082 ceph-mon[27967]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T08:57:09.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.873+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f432c068550 con 0x7f432c105e80 2024-04-21T08:57:09.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.877+0000 7f43227fc640 1 -- 172.21.15.82:0/2091317854 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11947 (secure 0 0 0) 0x7f4324016030 con 0x7f432c105e80 2024-04-21T08:57:09.877 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:57:09.878 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:56:58.435590+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24409,24407,24476,24509,24481]},"id":1}]} 2024-04-21T08:57:09.878 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 46 2024-04-21T08:57:09.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:09 smithi106 ceph-mon[31072]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T08:57:09.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.881+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f43180a8210 msgr2=0x7f43180aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.881+0000 7f433363f640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f43180a8210 0x7f43180aa6d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f431c009a30 tx=0x7f431c009190 comp rx=0 tx=0).stop 2024-04-21T08:57:09.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.881+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f432c105e80 msgr2=0x7f432c19af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:09.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.881+0000 7f433363f640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f432c105e80 0x7f432c19af10 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f432400e940 tx=0x7f432400ee00 comp rx=0 tx=0).stop 2024-04-21T08:57:09.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.883+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 shutdown_connections 2024-04-21T08:57:09.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.883+0000 7f433363f640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f43180a8210 0x7f43180aa6d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.883+0000 7f433363f640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f432c105e80 0x7f432c19af10 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.883+0000 7f433363f640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f432c101af0 0x7f432c196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.883+0000 7f433363f640 1 --2- 172.21.15.82:0/2091317854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f432c101130 0x7f432c196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:09.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.883+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 >> 172.21.15.82:0/2091317854 conn(0x7f432c077e30 msgr2=0x7f432c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:09.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.883+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 shutdown_connections 2024-04-21T08:57:09.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:09.883+0000 7f433363f640 1 -- 172.21.15.82:0/2091317854 wait complete. 2024-04-21T08:57:09.900 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:57:09.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:09 smithi134 ceph-mon[31772]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T08:57:10.072 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-21T08:57:10.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.080+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1939030306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa54101090 msgr2=0x7faa54101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.080+0000 7faa5b5f4640 1 --2- 172.21.15.82:0/1939030306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa54101090 0x7faa54101470 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009ca0 tx=0x7faa4c02c140 comp rx=0 tx=0).stop 2024-04-21T08:57:10.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.086+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1939030306 shutdown_connections 2024-04-21T08:57:10.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.086+0000 7faa5b5f4640 1 --2- 172.21.15.82:0/1939030306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa54106160 0x7faa5410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.086+0000 7faa5b5f4640 1 --2- 172.21.15.82:0/1939030306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa54101a40 0x7faa54105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.086+0000 7faa5b5f4640 1 --2- 172.21.15.82:0/1939030306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa54101090 0x7faa54101470 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.086+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1939030306 >> 172.21.15.82:0/1939030306 conn(0x7faa54077f50 msgr2=0x7faa54078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.091+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1939030306 shutdown_connections 2024-04-21T08:57:10.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.093+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1939030306 wait complete. 2024-04-21T08:57:10.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.094+0000 7faa5b5f4640 1 Processor -- start 2024-04-21T08:57:10.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.098+0000 7faa5b5f4640 1 -- start start 2024-04-21T08:57:10.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.098+0000 7faa5b5f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa54101090 0x7faa54116880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.098+0000 7faa5b5f4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa54106160 0x7faa541108f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.098+0000 7faa5b5f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa54110e30 0x7faa541112e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.098+0000 7faa5b5f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faa5410f180 con 0x7faa54101090 2024-04-21T08:57:10.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.098+0000 7faa5b5f4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faa5410f000 con 0x7faa54106160 2024-04-21T08:57:10.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.098+0000 7faa5b5f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa5410f300 con 0x7faa54110e30 2024-04-21T08:57:10.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.099+0000 7faa59b91640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa54110e30 0x7faa541112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.099+0000 7faa59b91640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa54110e30 0x7faa541112e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52262/0 (socket says 172.21.15.82:52262) 2024-04-21T08:57:10.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.099+0000 7faa59b91640 1 -- 172.21.15.82:0/1668560528 learned_addr learned my addr 172.21.15.82:0/1668560528 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.100+0000 7faa59390640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa54101090 0x7faa54116880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.100+0000 7faa59390640 1 -- 172.21.15.82:0/1668560528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa54110e30 msgr2=0x7faa541112e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.100+0000 7faa59390640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa54110e30 0x7faa541112e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.100+0000 7faa59390640 1 -- 172.21.15.82:0/1668560528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa54106160 msgr2=0x7faa541108f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:57:10.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.100+0000 7faa59390640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa54106160 0x7faa541108f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.100+0000 7faa59390640 1 -- 172.21.15.82:0/1668560528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa541b3e00 con 0x7faa54101090 2024-04-21T08:57:10.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.101+0000 7faa59390640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa54101090 0x7faa54116880 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009bf0 tx=0x7faa4c035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.102+0000 7faa4a7fc640 1 -- 172.21.15.82:0/1668560528 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c041020 con 0x7faa54101090 2024-04-21T08:57:10.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.102+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1668560528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa541b4090 con 0x7faa54101090 2024-04-21T08:57:10.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.102+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1668560528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa541b4580 con 0x7faa54101090 2024-04-21T08:57:10.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.102+0000 7faa4a7fc640 1 -- 172.21.15.82:0/1668560528 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa4c003e20 con 0x7faa54101090 2024-04-21T08:57:10.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.102+0000 7faa4a7fc640 1 -- 172.21.15.82:0/1668560528 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c039040 con 0x7faa54101090 2024-04-21T08:57:10.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.106+0000 7faa5e1e5740 1 Processor -- start 2024-04-21T08:57:10.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.106+0000 7faa5e1e5740 1 -- start start 2024-04-21T08:57:10.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.107+0000 7faa4a7fc640 1 -- 172.21.15.82:0/1668560528 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa4c044020 con 0x7faa54101090 2024-04-21T08:57:10.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.107+0000 7faa4a7fc640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faa400a3f20 0x7faa400a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.108+0000 7faa59b91640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faa400a3f20 0x7faa400a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.108+0000 7faa4a7fc640 1 -- 172.21.15.82:0/1668560528 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7faa4c0f3640 con 0x7faa54101090 2024-04-21T08:57:10.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.109+0000 7faa59b91640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faa400a3f20 0x7faa400a63e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa440099b0 tx=0x7faa44009110 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.111+0000 7faa5e1e5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fe4d0 0x562d4b7fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.111+0000 7faa5e1e5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fee80 0x562d4b807430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.111+0000 7faa5e1e5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b807a00 0x562d4b809e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.111+0000 7faa5e1e5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562d4b7631f0 con 0x562d4b7fe4d0 2024-04-21T08:57:10.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.111+0000 7faa5e1e5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562d4b7004b0 con 0x562d4b807a00 2024-04-21T08:57:10.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.111+0000 7faa5e1e5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562d4b75f1b0 con 0x562d4b7fee80 2024-04-21T08:57:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.113+0000 7faa59390640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fe4d0 0x562d4b7fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.113+0000 7faa59390640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fe4d0 0x562d4b7fe8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33338/0 (socket says 172.21.15.82:33338) 2024-04-21T08:57:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.113+0000 7faa59390640 1 -- 172.21.15.82:0/3740273298 learned_addr learned my addr 172.21.15.82:0/3740273298 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.115+0000 7faa59b91640 1 --2- 172.21.15.82:0/3740273298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b807a00 0x562d4b809e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.117+0000 7faa59b91640 1 -- 172.21.15.82:0/3740273298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fee80 msgr2=0x562d4b807430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.117+0000 7faa59b91640 1 --2- 172.21.15.82:0/3740273298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fee80 0x562d4b807430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.117+0000 7faa59b91640 1 -- 172.21.15.82:0/3740273298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fe4d0 msgr2=0x562d4b7fe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.117+0000 7faa59b91640 1 --2- 172.21.15.82:0/3740273298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fe4d0 0x562d4b7fe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.117+0000 7faa59b91640 1 -- 172.21.15.82:0/3740273298 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4b6f21b0 con 0x562d4b807a00 2024-04-21T08:57:10.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.119+0000 7faa59b91640 1 --2- 172.21.15.82:0/3740273298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b807a00 0x562d4b809e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa4400e5a0 tx=0x7faa44011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=883b17b59e388f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.119+0000 7faa3f7fe640 1 -- 172.21.15.82:0/3740273298 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4401b040 con 0x562d4b807a00 2024-04-21T08:57:10.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.120+0000 7faa3f7fe640 1 -- 172.21.15.82:0/3740273298 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa44024070 con 0x562d4b807a00 2024-04-21T08:57:10.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.120+0000 7faa3f7fe640 1 -- 172.21.15.82:0/3740273298 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4401f3f0 con 0x562d4b807a00 2024-04-21T08:57:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.120+0000 7faa5e1e5740 1 -- 172.21.15.82:0/3740273298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b807a00 msgr2=0x562d4b809e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.120+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/3740273298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b807a00 0x562d4b809e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa4400e5a0 tx=0x7faa44011000 comp rx=0 tx=0).stop 2024-04-21T08:57:10.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.121+0000 7faa5e1e5740 1 -- 172.21.15.82:0/3740273298 shutdown_connections 2024-04-21T08:57:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.121+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/3740273298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b807a00 0x562d4b809e50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.121+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/3740273298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fee80 0x562d4b807430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.121+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/3740273298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fe4d0 0x562d4b7fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.121+0000 7faa5e1e5740 1 -- 172.21.15.82:0/3740273298 >> 172.21.15.82:0/3740273298 conn(0x562d4b7f8ba0 msgr2=0x562d4b7f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.125+0000 7faa5e1e5740 1 -- 172.21.15.82:0/3740273298 shutdown_connections 2024-04-21T08:57:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.130+0000 7faa5e1e5740 1 -- 172.21.15.82:0/3740273298 wait complete. 2024-04-21T08:57:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.130+0000 7faa5e1e5740 1 -- start start 2024-04-21T08:57:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.130+0000 7faa5e1e5740 1 Processor -- start 2024-04-21T08:57:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.131+0000 7faa5e1e5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fe4d0 0x562d4b807260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.131+0000 7faa5e1e5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fee80 0x562d4b8045f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.131+0000 7faa5e1e5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b807a00 0x562d4b804b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.131+0000 7faa5e1e5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x562d4b80a970 con 0x562d4b7fee80 2024-04-21T08:57:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.131+0000 7faa5e1e5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562d4b71c500 con 0x562d4b807a00 2024-04-21T08:57:10.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.131+0000 7faa5e1e5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562d4b80aaf0 con 0x562d4b7fe4d0 2024-04-21T08:57:10.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.131+0000 7faa5e1e5740 10 client.0 _ll_register_callbacks cb 0x562d4b6eab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:57:10.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.133+0000 7faa58b8f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fee80 0x562d4b8045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.133+0000 7faa58b8f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fee80 0x562d4b8045f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33340/0 (socket says 172.21.15.82:33340) 2024-04-21T08:57:10.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.133+0000 7faa58b8f640 1 -- 172.21.15.82:0/1326473501 learned_addr learned my addr 172.21.15.82:0/1326473501 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.133+0000 7faa59b91640 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b807a00 0x562d4b804b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.133+0000 7faa59390640 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fe4d0 0x562d4b807260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.133+0000 7faa59b91640 1 -- 172.21.15.82:0/1326473501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fe4d0 msgr2=0x562d4b807260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.133+0000 7faa59b91640 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fe4d0 0x562d4b807260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.133+0000 7faa59b91640 1 -- 172.21.15.82:0/1326473501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fee80 msgr2=0x562d4b8045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.133+0000 7faa59b91640 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fee80 0x562d4b8045f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.133+0000 7faa59b91640 1 -- 172.21.15.82:0/1326473501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4b4753f0 con 0x562d4b807a00 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.134+0000 7faa59b91640 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b807a00 0x562d4b804b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faa44023000 tx=0x7faa4400e8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38c817074da03c36 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.135+0000 7faa3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.135+0000 7faa3cff9640 1 -- 172.21.15.82:0/1326473501 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4402d070 con 0x562d4b807a00 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.135+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1326473501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa44024070 con 0x562d4b807a00 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.135+0000 7faa3cff9640 1 -- 172.21.15.82:0/1326473501 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7faa44013650 con 0x562d4b807a00 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.135+0000 7faa3cff9640 1 -- 172.21.15.82:0/1326473501 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7faa4403b050 con 0x562d4b807a00 2024-04-21T08:57:10.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.135+0000 7faa5e1e5740 10 client.27473 fetch_fsmap learned FSMap version 46 2024-04-21T08:57:10.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.135+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1326473501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562d4b53f7d0 con 0x562d4b807a00 2024-04-21T08:57:10.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.136+0000 7faa3cff9640 1 -- 172.21.15.82:0/1326473501 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa44024070 con 0x562d4b807a00 2024-04-21T08:57:10.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.137+0000 7faa3cff9640 1 -- 172.21.15.82:0/1326473501 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7faa4401e030 con 0x562d4b807a00 2024-04-21T08:57:10.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.137+0000 7faa5e1e5740 10 client.27473 fetch_fsmap finished waiting for FSMap version 46 2024-04-21T08:57:10.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.137+0000 7faa5e1e5740 10 client.27473 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:57:10.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.137+0000 7faa5e1e5740 20 client.27473 populate_metadata read hostname 'smithi082' 2024-04-21T08:57:10.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.137+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562d4b83a030 0x562d4b83c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.137+0000 7faa5e1e5740 4 client.27473 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:57:10.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.137+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1326473501 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d4b702930 con 0x562d4b83a030 2024-04-21T08:57:10.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.142+0000 7faa59390640 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562d4b83a030 0x562d4b83c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.153+0000 7faa59390640 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562d4b83a030 0x562d4b83c4b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d45d172585c44ba9 server_cookie=ae529977286b0166 in_seq=0 out_seq=0 2024-04-21T08:57:10.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.153+0000 7faa3cff9640 10 client.27473 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:57:10.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.155+0000 7faa3cff9640 1 -- 172.21.15.82:0/1326473501 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7faa4c0bf920 con 0x562d4b83a030 2024-04-21T08:57:10.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.155+0000 7faa3cff9640 10 client.27473 handle_command_reply: tid=0 2024-04-21T08:57:10.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.155+0000 7faa5e1e5740 1 client.27473 shutdown 2024-04-21T08:57:10.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.156+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1326473501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b807a00 msgr2=0x562d4b804b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.156+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b807a00 0x562d4b804b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faa44023000 tx=0x7faa4400e8e0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.157+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1326473501 shutdown_connections 2024-04-21T08:57:10.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.157+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562d4b83a030 0x562d4b83c4b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.157+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b807a00 0x562d4b804b30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.157+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fee80 0x562d4b8045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.157+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/1326473501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fe4d0 0x562d4b807260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.157+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1326473501 >> 172.21.15.82:0/1326473501 conn(0x562d4b7f8ba0 msgr2=0x562d4b7fa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.160+0000 7faa3cff9640 0 client.27473 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:57:10.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.166+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1326473501 shutdown_connections 2024-04-21T08:57:10.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.166+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1326473501 wait complete. 2024-04-21T08:57:10.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.166+0000 7faa5e1e5740 20 client.27473 trim_cache size 0 max 16384 2024-04-21T08:57:10.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.192+0000 7faa5e1e5740 1 Processor -- start 2024-04-21T08:57:10.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.195+0000 7faa5e1e5740 1 -- start start 2024-04-21T08:57:10.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.195+0000 7faa5e1e5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fe4d0 0x562d4b8138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.195+0000 7faa5e1e5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b7fee80 0x562d4b806d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.195+0000 7faa5e1e5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b807a00 0x562d4b807260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.195+0000 7faa5e1e5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faa38021100 con 0x562d4b7fe4d0 2024-04-21T08:57:10.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.195+0000 7faa5e1e5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faa38022c70 con 0x562d4b7fee80 2024-04-21T08:57:10.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.195+0000 7faa5e1e5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa380217d0 con 0x562d4b807a00 2024-04-21T08:57:10.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.199+0000 7faa58b8f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b7fee80 0x562d4b806d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.199+0000 7faa58b8f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b7fee80 0x562d4b806d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:59590/0 (socket says 172.21.15.82:59590) 2024-04-21T08:57:10.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.199+0000 7faa58b8f640 1 -- 172.21.15.82:0/1208354297 learned_addr learned my addr 172.21.15.82:0/1208354297 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.199+0000 7faa59b91640 1 --2- 172.21.15.82:0/1208354297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b807a00 0x562d4b807260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.200+0000 7faa59390640 1 --2- 172.21.15.82:0/1208354297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fe4d0 0x562d4b8138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.200+0000 7faa59b91640 1 -- 172.21.15.82:0/1208354297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b7fee80 msgr2=0x562d4b806d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.200+0000 7faa59b91640 1 --2- 172.21.15.82:0/1208354297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b7fee80 0x562d4b806d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.200+0000 7faa59b91640 1 -- 172.21.15.82:0/1208354297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fe4d0 msgr2=0x562d4b8138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.200+0000 7faa59b91640 1 --2- 172.21.15.82:0/1208354297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fe4d0 0x562d4b8138c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.200+0000 7faa59b91640 1 -- 172.21.15.82:0/1208354297 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4b7fec20 con 0x562d4b807a00 2024-04-21T08:57:10.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.204+0000 7faa59b91640 1 --2- 172.21.15.82:0/1208354297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b807a00 0x562d4b807260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa44010010 tx=0x7faa4400e620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4910b160a9cd07a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.215+0000 7faa3effd640 1 -- 172.21.15.82:0/1208354297 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4402d070 con 0x562d4b807a00 2024-04-21T08:57:10.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.215+0000 7faa3effd640 1 -- 172.21.15.82:0/1208354297 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa44024040 con 0x562d4b807a00 2024-04-21T08:57:10.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.215+0000 7faa3effd640 1 -- 172.21.15.82:0/1208354297 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa440326c0 con 0x562d4b807a00 2024-04-21T08:57:10.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.217+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1208354297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b807a00 msgr2=0x562d4b807260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.217+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/1208354297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b807a00 0x562d4b807260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa44010010 tx=0x7faa4400e620 comp rx=0 tx=0).stop 2024-04-21T08:57:10.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.222+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1208354297 shutdown_connections 2024-04-21T08:57:10.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.222+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/1208354297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b807a00 0x562d4b807260 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.222+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/1208354297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b7fee80 0x562d4b806d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.222+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/1208354297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b7fe4d0 0x562d4b8138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.222+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1208354297 >> 172.21.15.82:0/1208354297 conn(0x562d4b7f8ba0 msgr2=0x562d4b7ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.223+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1208354297 shutdown_connections 2024-04-21T08:57:10.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.225+0000 7faa5e1e5740 1 -- 172.21.15.82:0/1208354297 wait complete. 2024-04-21T08:57:10.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.226+0000 7faa5e1e5740 1 -- start start 2024-04-21T08:57:10.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.226+0000 7faa5e1e5740 1 Processor -- start 2024-04-21T08:57:10.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.227+0000 7faa5e1e5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fe4d0 0x562d4b810030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.227+0000 7faa5e1e5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b7fee80 0x562d4b800e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.227+0000 7faa5e1e5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b807a00 0x562d4b8013a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.227+0000 7faa5e1e5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faa380225a0 con 0x562d4b807a00 2024-04-21T08:57:10.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.227+0000 7faa5e1e5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faa38021ea0 con 0x562d4b7fee80 2024-04-21T08:57:10.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.227+0000 7faa5e1e5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa3801f160 con 0x562d4b7fe4d0 2024-04-21T08:57:10.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.227+0000 7faa5e1e5740 10 client.0 _ll_register_callbacks cb 0x7faa44004700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:57:10.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.228+0000 7faa59390640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fe4d0 0x562d4b810030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.228+0000 7faa59390640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fe4d0 0x562d4b810030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52314/0 (socket says 172.21.15.82:52314) 2024-04-21T08:57:10.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.228+0000 7faa59390640 1 -- 172.21.15.82:0/2860369548 learned_addr learned my addr 172.21.15.82:0/2860369548 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.230+0000 7faa59b91640 1 --2- 172.21.15.82:0/2860369548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b807a00 0x562d4b8013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.232+0000 7faa59b91640 1 -- 172.21.15.82:0/2860369548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fe4d0 msgr2=0x562d4b810030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.232+0000 7faa59b91640 1 --2- 172.21.15.82:0/2860369548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fe4d0 0x562d4b810030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.232+0000 7faa59b91640 1 -- 172.21.15.82:0/2860369548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b7fee80 msgr2=0x562d4b800e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:57:10.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.232+0000 7faa59b91640 1 --2- 172.21.15.82:0/2860369548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b7fee80 0x562d4b800e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.232+0000 7faa59b91640 1 -- 172.21.15.82:0/2860369548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4b801a10 con 0x562d4b807a00 2024-04-21T08:57:10.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.232+0000 7faa59b91640 1 --2- 172.21.15.82:0/2860369548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b807a00 0x562d4b8013a0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7faa440047b0 tx=0x7faa44029ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e526104abaf0a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.232+0000 7faa3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:57:10.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.235+0000 7faa3cff9640 1 -- 172.21.15.82:0/2860369548 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44045070 con 0x562d4b807a00 2024-04-21T08:57:10.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.235+0000 7faa5e1e5740 1 -- 172.21.15.82:0/2860369548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa44024040 con 0x562d4b807a00 2024-04-21T08:57:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.236+0000 7faa3cff9640 1 -- 172.21.15.82:0/2860369548 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7faa44032660 con 0x562d4b807a00 2024-04-21T08:57:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.236+0000 7faa3cff9640 1 -- 172.21.15.82:0/2860369548 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7faa4402d040 con 0x562d4b807a00 2024-04-21T08:57:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.237+0000 7faa5e1e5740 10 client.18480 fetch_fsmap learned FSMap version 46 2024-04-21T08:57:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.237+0000 7faa5e1e5740 1 -- 172.21.15.82:0/2860369548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562d4b8a0480 con 0x562d4b807a00 2024-04-21T08:57:10.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.240+0000 7faa3cff9640 1 -- 172.21.15.82:0/2860369548 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa44024040 con 0x562d4b807a00 2024-04-21T08:57:10.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.240+0000 7faa3cff9640 1 -- 172.21.15.82:0/2860369548 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7faa4403e070 con 0x562d4b807a00 2024-04-21T08:57:10.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.241+0000 7faa5e1e5740 10 client.18480 fetch_fsmap finished waiting for FSMap version 46 2024-04-21T08:57:10.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.241+0000 7faa5e1e5740 10 client.18480 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-21T08:57:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.241+0000 7faa5e1e5740 20 client.18480 populate_metadata read hostname 'smithi082' 2024-04-21T08:57:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.241+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/2860369548 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562d4b8a0ae0 0x562d4b8a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.241+0000 7faa5e1e5740 4 client.18480 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:57:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.241+0000 7faa5e1e5740 1 -- 172.21.15.82:0/2860369548 --> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d4b770c10 con 0x562d4b8a0ae0 2024-04-21T08:57:10.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.242+0000 7faa59390640 1 --2- 172.21.15.82:0/2860369548 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562d4b8a0ae0 0x562d4b8a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.253+0000 7faa59390640 1 --2- 172.21.15.82:0/2860369548 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562d4b8a0ae0 0x562d4b8a2f00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=376054664d1627 server_cookie=9dc7d5d95e348ac2 in_seq=0 out_seq=0 2024-04-21T08:57:10.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.253+0000 7faa3cff9640 10 client.18480 ms_handle_connect on v2:172.21.15.106:6832/2583766346 2024-04-21T08:57:10.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.255+0000 7faa3cff9640 1 -- 172.21.15.82:0/2860369548 <== mds.4 v2:172.21.15.106:6832/2583766346 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d4b770c10 con 0x562d4b8a0ae0 2024-04-21T08:57:10.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.255+0000 7faa3cff9640 10 client.18480 handle_command_reply: tid=0 2024-04-21T08:57:10.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.256+0000 7faa5e1e5740 1 client.18480 shutdown 2024-04-21T08:57:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.266+0000 7faa5e1e5740 1 -- 172.21.15.82:0/2860369548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b807a00 msgr2=0x562d4b8013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.266+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/2860369548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b807a00 0x562d4b8013a0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7faa440047b0 tx=0x7faa44029ad0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.268+0000 7faa5e1e5740 1 -- 172.21.15.82:0/2860369548 shutdown_connections 2024-04-21T08:57:10.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.268+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/2860369548 >> [v2:172.21.15.106:6832/2583766346,v1:172.21.15.106:6833/2583766346] conn(0x562d4b8a0ae0 0x562d4b8a2f00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.268+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/2860369548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4b807a00 0x562d4b8013a0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.268+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/2860369548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562d4b7fee80 0x562d4b800e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.268+0000 7faa5e1e5740 1 --2- 172.21.15.82:0/2860369548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562d4b7fe4d0 0x562d4b810030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.268+0000 7faa5e1e5740 1 -- 172.21.15.82:0/2860369548 >> 172.21.15.82:0/2860369548 conn(0x562d4b7f8ba0 msgr2=0x562d4b7f7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.270+0000 7faa3cff9640 0 client.18480 ms_handle_reset on v2:172.21.15.106:6832/2583766346 2024-04-21T08:57:10.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.271+0000 7faa5e1e5740 1 -- 172.21.15.82:0/2860369548 shutdown_connections 2024-04-21T08:57:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.274+0000 7faa5e1e5740 1 -- 172.21.15.82:0/2860369548 wait complete. 2024-04-21T08:57:10.276 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:57:10.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.276+0000 7faa5e1e5740 20 client.18480 trim_cache size 0 max 16384 2024-04-21T08:57:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.278+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1668560528 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faa400a3f20 msgr2=0x7faa400a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.278+0000 7faa5b5f4640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faa400a3f20 0x7faa400a63e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa440099b0 tx=0x7faa44009110 comp rx=0 tx=0).stop 2024-04-21T08:57:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.279+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1668560528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa54101090 msgr2=0x7faa54116880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.279+0000 7faa5b5f4640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa54101090 0x7faa54116880 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009bf0 tx=0x7faa4c035ce0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.279+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1668560528 shutdown_connections 2024-04-21T08:57:10.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.279+0000 7faa5b5f4640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa54110e30 0x7faa541112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.279+0000 7faa5b5f4640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa54106160 0x7faa541108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.279+0000 7faa5b5f4640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7faa400a3f20 0x7faa400a63e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.279+0000 7faa5b5f4640 1 --2- 172.21.15.82:0/1668560528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa54101090 0x7faa54116880 secure :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009bf0 tx=0x7faa4c035ce0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.279+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1668560528 >> 172.21.15.82:0/1668560528 conn(0x7faa54077f50 msgr2=0x7faa540ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.279+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1668560528 shutdown_connections 2024-04-21T08:57:10.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.279+0000 7faa5b5f4640 1 -- 172.21.15.82:0/1668560528 wait complete. 2024-04-21T08:57:10.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.280+0000 7f1994515640 1 -- 172.21.15.82:0/4186279031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c1104c0 msgr2=0x7f198c1108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.280+0000 7f1994515640 1 --2- 172.21.15.82:0/4186279031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c1104c0 0x7f198c1108a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f198801acb0 tx=0x7f198803d170 comp rx=0 tx=0).stop 2024-04-21T08:57:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.284+0000 7f1994515640 1 -- 172.21.15.82:0/4186279031 shutdown_connections 2024-04-21T08:57:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.284+0000 7f1994515640 1 --2- 172.21.15.82:0/4186279031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f198c10ecc0 0x7f198c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.284+0000 7f1994515640 1 --2- 172.21.15.82:0/4186279031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f198c10e220 0x7f198c10e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.284+0000 7f1994515640 1 --2- 172.21.15.82:0/4186279031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c1104c0 0x7f198c1108a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.285+0000 7f1994515640 1 -- 172.21.15.82:0/4186279031 >> 172.21.15.82:0/4186279031 conn(0x7f198c06cf00 msgr2=0x7f198c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.290+0000 7f1994515640 1 -- 172.21.15.82:0/4186279031 shutdown_connections 2024-04-21T08:57:10.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.292+0000 7f1994515640 1 -- 172.21.15.82:0/4186279031 wait complete. 2024-04-21T08:57:10.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.293+0000 7f1994515640 1 Processor -- start 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.293+0000 7f1994515640 1 -- start start 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.293+0000 7f1994515640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f198c10e220 0x7f198c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.293+0000 7f1994515640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c10ecc0 0x7f198c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.293+0000 7f1994515640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f198c1104c0 0x7f198c1a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.293+0000 7f1994515640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f198c112720 con 0x7f198c1104c0 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.293+0000 7f1994515640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f198c1125a0 con 0x7f198c10e220 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.294+0000 7f19922b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f198c10e220 0x7f198c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.294+0000 7f19922b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f198c10e220 0x7f198c19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:59630/0 (socket says 172.21.15.82:59630) 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.294+0000 7f19922b1640 1 -- 172.21.15.82:0/3662335682 learned_addr learned my addr 172.21.15.82:0/3662335682 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.293+0000 7f1994515640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f198c1128a0 con 0x7f198c10ecc0 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.294+0000 7f1991ab0640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c10ecc0 0x7f198c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.294+0000 7f19922b1640 1 -- 172.21.15.82:0/3662335682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c10ecc0 msgr2=0x7f198c19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.294+0000 7f19922b1640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c10ecc0 0x7f198c19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.294+0000 7f19922b1640 1 -- 172.21.15.82:0/3662335682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f198c1104c0 msgr2=0x7f198c1a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.294+0000 7f19922b1640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f198c1104c0 0x7f198c1a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.294+0000 7f19922b1640 1 -- 172.21.15.82:0/3662335682 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c1a3d10 con 0x7f198c10e220 2024-04-21T08:57:10.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.295+0000 7f19922b1640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f198c10e220 0x7f198c19ed20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f198803d700 tx=0x7f1988003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.302+0000 7f19837fe640 1 -- 172.21.15.82:0/3662335682 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198804a040 con 0x7f198c10e220 2024-04-21T08:57:10.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.302+0000 7f19837fe640 1 -- 172.21.15.82:0/3662335682 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1988053070 con 0x7f198c10e220 2024-04-21T08:57:10.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.302+0000 7f19837fe640 1 -- 172.21.15.82:0/3662335682 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19880456a0 con 0x7f198c10e220 2024-04-21T08:57:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.303+0000 7f1994515640 1 -- 172.21.15.82:0/3662335682 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198c1a3fa0 con 0x7f198c10e220 2024-04-21T08:57:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.303+0000 7f1994515640 1 -- 172.21.15.82:0/3662335682 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198c1ab890 con 0x7f198c10e220 2024-04-21T08:57:10.306 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:57:10.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.308+0000 7f1994515640 1 -- 172.21.15.82:0/3662335682 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1970005290 con 0x7f198c10e220 2024-04-21T08:57:10.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.313+0000 7f19837fe640 1 -- 172.21.15.82:0/3662335682 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1988003b60 con 0x7f198c10e220 2024-04-21T08:57:10.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.313+0000 7f19837fe640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19780a3f20 0x7f19780a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.314+0000 7f19837fe640 1 -- 172.21.15.82:0/3662335682 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f1988103660 con 0x7f198c10e220 2024-04-21T08:57:10.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.315+0000 7f1991ab0640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19780a3f20 0x7f19780a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.319+0000 7f1991ab0640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19780a3f20 0x7f19780a63e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f198c06d8c0 tx=0x7f19840090e0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.322+0000 7f19837fe640 1 -- 172.21.15.82:0/3662335682 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19880cf940 con 0x7f198c10e220 2024-04-21T08:57:10.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.557+0000 7fe9bce6a640 1 -- 172.21.15.82:0/1558998513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b81062e0 msgr2=0x7fe9b8106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.557+0000 7fe9bce6a640 1 --2- 172.21.15.82:0/1558998513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b81062e0 0x7fe9b8106770 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009ca0 tx=0x7fe9b002c250 comp rx=0 tx=0).stop 2024-04-21T08:57:10.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.560+0000 7fe9bce6a640 1 -- 172.21.15.82:0/1558998513 shutdown_connections 2024-04-21T08:57:10.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.560+0000 7fe9bce6a640 1 --2- 172.21.15.82:0/1558998513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b81062e0 0x7fe9b8106770 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.560+0000 7fe9bce6a640 1 --2- 172.21.15.82:0/1558998513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9b81018e0 0x7fe9b8105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.560+0000 7fe9bce6a640 1 --2- 172.21.15.82:0/1558998513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9b8100fc0 0x7fe9b81013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.560+0000 7fe9bce6a640 1 -- 172.21.15.82:0/1558998513 >> 172.21.15.82:0/1558998513 conn(0x7fe9b8077f50 msgr2=0x7fe9b8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.563+0000 7fe9bce6a640 1 -- 172.21.15.82:0/1558998513 shutdown_connections 2024-04-21T08:57:10.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.567+0000 7fe9bce6a640 1 -- 172.21.15.82:0/1558998513 wait complete. 2024-04-21T08:57:10.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.568+0000 7fe9bce6a640 1 Processor -- start 2024-04-21T08:57:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.569+0000 7fe9bce6a640 1 -- start start 2024-04-21T08:57:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.570+0000 7fe9bce6a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b8100fc0 0x7fe9b8072950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.570+0000 7fe9bce6a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9b81018e0 0x7fe9b8072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.570+0000 7fe9bce6a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9b81062e0 0x7fe9b810a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.570+0000 7fe9bce6a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8118910 con 0x7fe9b8100fc0 2024-04-21T08:57:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.570+0000 7fe9bce6a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8118790 con 0x7fe9b81062e0 2024-04-21T08:57:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.570+0000 7fe9bce6a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8118a90 con 0x7fe9b81018e0 2024-04-21T08:57:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.570+0000 7fe9b659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b8100fc0 0x7fe9b8072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.570+0000 7fe9b659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b8100fc0 0x7fe9b8072950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33404/0 (socket says 172.21.15.82:33404) 2024-04-21T08:57:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.570+0000 7fe9b659c640 1 -- 172.21.15.82:0/870341093 learned_addr learned my addr 172.21.15.82:0/870341093 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.571+0000 7fe9b659c640 1 -- 172.21.15.82:0/870341093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9b81018e0 msgr2=0x7fe9b8072e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:57:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.571+0000 7fe9b659c640 1 --2- 172.21.15.82:0/870341093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9b81018e0 0x7fe9b8072e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.571+0000 7fe9b659c640 1 -- 172.21.15.82:0/870341093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9b81062e0 msgr2=0x7fe9b810a440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:57:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.571+0000 7fe9b659c640 1 --2- 172.21.15.82:0/870341093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9b81062e0 0x7fe9b810a440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.571+0000 7fe9b659c640 1 -- 172.21.15.82:0/870341093 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b810ab70 con 0x7fe9b8100fc0 2024-04-21T08:57:10.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.572+0000 7fe9b659c640 1 --2- 172.21.15.82:0/870341093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b8100fc0 0x7fe9b8072950 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800b450 tx=0x7fe9a800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.573+0000 7fe9af7fe640 1 -- 172.21.15.82:0/870341093 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8015020 con 0x7fe9b8100fc0 2024-04-21T08:57:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.573+0000 7fe9bce6a640 1 -- 172.21.15.82:0/870341093 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b810ae60 con 0x7fe9b8100fc0 2024-04-21T08:57:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.573+0000 7fe9bce6a640 1 -- 172.21.15.82:0/870341093 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b8112730 con 0x7fe9b8100fc0 2024-04-21T08:57:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.573+0000 7fe9af7fe640 1 -- 172.21.15.82:0/870341093 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a8011d40 con 0x7fe9b8100fc0 2024-04-21T08:57:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.573+0000 7fe9af7fe640 1 -- 172.21.15.82:0/870341093 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a800f340 con 0x7fe9b8100fc0 2024-04-21T08:57:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.579+0000 7fe9bfa5d740 1 Processor -- start 2024-04-21T08:57:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.579+0000 7fe9af7fe640 1 -- 172.21.15.82:0/870341093 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe9a8018020 con 0x7fe9b8100fc0 2024-04-21T08:57:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.580+0000 7fe9bfa5d740 1 -- start start 2024-04-21T08:57:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.580+0000 7fe9bfa5d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754ab4d0 0x5578754ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.580+0000 7fe9bfa5d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754abe80 0x5578754b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.580+0000 7fe9bfa5d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754b4a00 0x5578754b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.580+0000 7fe9bfa5d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5578754101f0 con 0x5578754ab4d0 2024-04-21T08:57:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.580+0000 7fe9bfa5d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5578753ad4b0 con 0x5578754b4a00 2024-04-21T08:57:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.580+0000 7fe9bfa5d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55787540c1b0 con 0x5578754abe80 2024-04-21T08:57:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.580+0000 7fe9b5d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754b4a00 0x5578754b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.580+0000 7fe9b5d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754b4a00 0x5578754b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:59646/0 (socket says 172.21.15.82:59646) 2024-04-21T08:57:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.580+0000 7fe9b5d9b640 1 -- 172.21.15.82:0/2427950514 learned_addr learned my addr 172.21.15.82:0/2427950514 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.581+0000 7fe9af7fe640 1 --2- 172.21.15.82:0/870341093 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9a40a3f20 0x7fe9a40a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.581+0000 7fe9b5d9b640 1 -- 172.21.15.82:0/2427950514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754abe80 msgr2=0x5578754b4430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:57:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.581+0000 7fe9b5d9b640 1 --2- 172.21.15.82:0/2427950514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754abe80 0x5578754b4430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.581+0000 7fe9b5d9b640 1 -- 172.21.15.82:0/2427950514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754ab4d0 msgr2=0x5578754ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.581+0000 7fe9b5d9b640 1 --2- 172.21.15.82:0/2427950514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754ab4d0 0x5578754ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.581+0000 7fe9b5d9b640 1 -- 172.21.15.82:0/2427950514 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787539f1b0 con 0x5578754b4a00 2024-04-21T08:57:10.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.581+0000 7fe9af7fe640 1 -- 172.21.15.82:0/870341093 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7fe9a80d05c0 con 0x7fe9b8100fc0 2024-04-21T08:57:10.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.586+0000 7fe9b6d9d640 1 --2- 172.21.15.82:0/870341093 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9a40a3f20 0x7fe9a40a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.590+0000 7fe9b6d9d640 1 --2- 172.21.15.82:0/870341093 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9a40a3f20 0x7fe9a40a63e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009a60 tx=0x7fe9b002c770 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.591+0000 7fe9b5d9b640 1 --2- 172.21.15.82:0/2427950514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754b4a00 0x5578754b6e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009740 tx=0x7fe9a0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aab51ad105156fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.597+0000 7fe9acff9640 1 -- 172.21.15.82:0/2427950514 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0015020 con 0x5578754b4a00 2024-04-21T08:57:10.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.597+0000 7fe9acff9640 1 -- 172.21.15.82:0/2427950514 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a0009e50 con 0x5578754b4a00 2024-04-21T08:57:10.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.597+0000 7fe9acff9640 1 -- 172.21.15.82:0/2427950514 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0010380 con 0x5578754b4a00 2024-04-21T08:57:10.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.599+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2427950514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754b4a00 msgr2=0x5578754b6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.599+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2427950514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754b4a00 0x5578754b6e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009740 tx=0x7fe9a0009c00 comp rx=0 tx=0).stop 2024-04-21T08:57:10.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.602+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2427950514 shutdown_connections 2024-04-21T08:57:10.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.602+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2427950514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754b4a00 0x5578754b6e50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.602+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2427950514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754abe80 0x5578754b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.602+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2427950514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754ab4d0 0x5578754ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.602+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2427950514 >> 172.21.15.82:0/2427950514 conn(0x5578754a5ba0 msgr2=0x5578754a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.603+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2427950514 shutdown_connections 2024-04-21T08:57:10.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.603+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2427950514 wait complete. 2024-04-21T08:57:10.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.603+0000 7fe9bfa5d740 1 -- start start 2024-04-21T08:57:10.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.604+0000 7fe9bfa5d740 1 Processor -- start 2024-04-21T08:57:10.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.604+0000 7fe9bfa5d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754ab4d0 0x5578754b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.604+0000 7fe9bfa5d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754abe80 0x5578754b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.605+0000 7fe9b659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754ab4d0 0x5578754b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.605+0000 7fe9b659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754ab4d0 0x5578754b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33418/0 (socket says 172.21.15.82:33418) 2024-04-21T08:57:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.605+0000 7fe9b659c640 1 -- 172.21.15.82:0/2443300744 learned_addr learned my addr 172.21.15.82:0/2443300744 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.605+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754b4a00 0x5578754b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.605+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2443300744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5578754b7970 con 0x5578754ab4d0 2024-04-21T08:57:10.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.605+0000 7fe9b5d9b640 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754abe80 0x5578754b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.605+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2443300744 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5578753c9500 con 0x5578754abe80 2024-04-21T08:57:10.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.605+0000 7fe9b6d9d640 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754b4a00 0x5578754b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.605+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2443300744 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578754b7af0 con 0x5578754b4a00 2024-04-21T08:57:10.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.605+0000 7fe9bfa5d740 10 client.0 _ll_register_callbacks cb 0x557875397b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:57:10.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.607+0000 7fe9b5d9b640 1 -- 172.21.15.82:0/2443300744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754b4a00 msgr2=0x5578754b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.607+0000 7fe9b5d9b640 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754b4a00 0x5578754b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.607+0000 7fe9b5d9b640 1 -- 172.21.15.82:0/2443300744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754ab4d0 msgr2=0x5578754b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.607+0000 7fe9b5d9b640 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754ab4d0 0x5578754b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.607+0000 7fe9b5d9b640 1 -- 172.21.15.82:0/2443300744 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578751223f0 con 0x5578754abe80 2024-04-21T08:57:10.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.607+0000 7fe9b5d9b640 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754abe80 0x5578754b1680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009740 tx=0x7fe9a0012cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3fb34eb74473 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.608+0000 7fe99dffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-21T08:57:10.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.608+0000 7fe99dffb640 1 -- 172.21.15.82:0/2443300744 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000bce0 con 0x5578754abe80 2024-04-21T08:57:10.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.608+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2443300744 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9a0009e50 con 0x5578754abe80 2024-04-21T08:57:10.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.608+0000 7fe99dffb640 1 -- 172.21.15.82:0/2443300744 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7fe9a0004040 con 0x5578754abe80 2024-04-21T08:57:10.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.616+0000 7fe99dffb640 1 -- 172.21.15.82:0/2443300744 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe9a0012e40 con 0x5578754abe80 2024-04-21T08:57:10.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.619+0000 7fe9bfa5d740 10 client.27485 fetch_fsmap learned FSMap version 46 2024-04-21T08:57:10.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.619+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2443300744 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5578751ec7d0 con 0x5578754abe80 2024-04-21T08:57:10.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.624+0000 7fe99dffb640 1 -- 172.21.15.82:0/2443300744 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a0009e50 con 0x5578754abe80 2024-04-21T08:57:10.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.624+0000 7fe99dffb640 1 -- 172.21.15.82:0/2443300744 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fe9a001a420 con 0x5578754abe80 2024-04-21T08:57:10.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.626+0000 7fe9bfa5d740 10 client.27485 fetch_fsmap finished waiting for FSMap version 46 2024-04-21T08:57:10.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.627+0000 7fe9bfa5d740 10 client.27485 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:57:10.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.627+0000 7fe9bfa5d740 20 client.27485 populate_metadata read hostname 'smithi082' 2024-04-21T08:57:10.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.627+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5578754c48a0 0x5578754c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.627+0000 7fe9bfa5d740 4 client.27485 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:57:10.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.627+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2443300744 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578753af930 con 0x5578754c48a0 2024-04-21T08:57:10.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.632+0000 7fe9b659c640 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5578754c48a0 0x5578754c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.636+0000 7fe9b659c640 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5578754c48a0 0x5578754c6c90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ee9c9bb99cf8b703 server_cookie=275e2cc61222eeea in_seq=0 out_seq=0 2024-04-21T08:57:10.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.637+0000 7fe99dffb640 10 client.27485 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:57:10.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.644+0000 7fe99dffb640 1 -- 172.21.15.82:0/2443300744 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578753af930 con 0x5578754c48a0 2024-04-21T08:57:10.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.644+0000 7fe99dffb640 10 client.27485 handle_command_reply: tid=0 2024-04-21T08:57:10.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.644+0000 7fe9bfa5d740 1 client.27485 shutdown 2024-04-21T08:57:10.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.649+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2443300744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754abe80 msgr2=0x5578754b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.649+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754abe80 0x5578754b1680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009740 tx=0x7fe9a0012cd0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.649+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2443300744 shutdown_connections 2024-04-21T08:57:10.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.649+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x5578754c48a0 0x5578754c6c90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.649+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754b4a00 0x5578754b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.649+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754abe80 0x5578754b1680 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009740 tx=0x7fe9a0012cd0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.649+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2443300744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754ab4d0 0x5578754b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.649+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2443300744 >> 172.21.15.82:0/2443300744 conn(0x5578754a5ba0 msgr2=0x5578754a78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.649+0000 7fe99dffb640 0 client.27485 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:57:10.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.652+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2443300744 shutdown_connections 2024-04-21T08:57:10.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.652+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2443300744 wait complete. 2024-04-21T08:57:10.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.653+0000 7fe9bfa5d740 20 client.27485 trim_cache size 0 max 16384 2024-04-21T08:57:10.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.686+0000 7fe9bfa5d740 1 Processor -- start 2024-04-21T08:57:10.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.686+0000 7fe9bfa5d740 1 -- start start 2024-04-21T08:57:10.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.686+0000 7fe9bfa5d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754ab4d0 0x5578754c0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.686+0000 7fe9bfa5d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754b4a00 0x5578754b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.686+0000 7fe9bfa5d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754c6fc0 0x5578754b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.686+0000 7fe9bfa5d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe998010260 con 0x5578754c6fc0 2024-04-21T08:57:10.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.686+0000 7fe9bfa5d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe998011dd0 con 0x5578754ab4d0 2024-04-21T08:57:10.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.686+0000 7fe9bfa5d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe998010930 con 0x5578754b4a00 2024-04-21T08:57:10.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.689+0000 7fe9b5d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754b4a00 0x5578754b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.689+0000 7fe9b5d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754b4a00 0x5578754b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52330/0 (socket says 172.21.15.82:52330) 2024-04-21T08:57:10.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.689+0000 7fe9b5d9b640 1 -- 172.21.15.82:0/2863521658 learned_addr learned my addr 172.21.15.82:0/2863521658 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.689+0000 7fe9b6d9d640 1 --2- 172.21.15.82:0/2863521658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754c6fc0 0x5578754b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.690+0000 7fe9b6d9d640 1 -- 172.21.15.82:0/2863521658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754b4a00 msgr2=0x5578754b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.690+0000 7fe9b6d9d640 1 --2- 172.21.15.82:0/2863521658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754b4a00 0x5578754b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.690+0000 7fe9b6d9d640 1 -- 172.21.15.82:0/2863521658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754ab4d0 msgr2=0x5578754c0860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:57:10.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.690+0000 7fe9b6d9d640 1 --2- 172.21.15.82:0/2863521658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754ab4d0 0x5578754c0860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.690+0000 7fe9b6d9d640 1 -- 172.21.15.82:0/2863521658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578754aa7f0 con 0x5578754c6fc0 2024-04-21T08:57:10.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.690+0000 7fe9b6d9d640 1 --2- 172.21.15.82:0/2863521658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754c6fc0 0x5578754b42f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe9b002c860 tx=0x7fe9b003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b1ff7b83327565 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.693+0000 7fe99ffff640 1 -- 172.21.15.82:0/2863521658 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0039040 con 0x5578754c6fc0 2024-04-21T08:57:10.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.693+0000 7fe99ffff640 1 -- 172.21.15.82:0/2863521658 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b0050070 con 0x5578754c6fc0 2024-04-21T08:57:10.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.693+0000 7fe99ffff640 1 -- 172.21.15.82:0/2863521658 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00041b0 con 0x5578754c6fc0 2024-04-21T08:57:10.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.696+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2863521658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754c6fc0 msgr2=0x5578754b42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.696+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2863521658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754c6fc0 0x5578754b42f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe9b002c860 tx=0x7fe9b003e000 comp rx=0 tx=0).stop 2024-04-21T08:57:10.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.698+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2863521658 shutdown_connections 2024-04-21T08:57:10.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.698+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2863521658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754c6fc0 0x5578754b42f0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.698+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2863521658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754b4a00 0x5578754b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.698+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/2863521658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754ab4d0 0x5578754c0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.698+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2863521658 >> 172.21.15.82:0/2863521658 conn(0x5578754a5ba0 msgr2=0x5578754c68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.700+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2863521658 shutdown_connections 2024-04-21T08:57:10.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.700+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/2863521658 wait complete. 2024-04-21T08:57:10.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.700+0000 7fe9bfa5d740 1 -- start start 2024-04-21T08:57:10.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.700+0000 7fe9bfa5d740 1 Processor -- start 2024-04-21T08:57:10.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.701+0000 7fe9bfa5d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754ab4d0 0x5578754bd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.701+0000 7fe9bfa5d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754b4a00 0x5578754ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.701+0000 7fe9bfa5d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754c6fc0 0x5578754ae980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.701+0000 7fe9bfa5d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe998011700 con 0x5578754b4a00 2024-04-21T08:57:10.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.701+0000 7fe9bfa5d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe998011000 con 0x5578754c6fc0 2024-04-21T08:57:10.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.702+0000 7fe9bfa5d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe99800e230 con 0x5578754ab4d0 2024-04-21T08:57:10.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.702+0000 7fe9bfa5d740 10 client.0 _ll_register_callbacks cb 0x7fe9a0011650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:57:10.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.702+0000 7fe9b5d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754b4a00 0x5578754ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.702+0000 7fe9b5d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754b4a00 0x5578754ae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33434/0 (socket says 172.21.15.82:33434) 2024-04-21T08:57:10.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.702+0000 7fe9b5d9b640 1 -- 172.21.15.82:0/1441962083 learned_addr learned my addr 172.21.15.82:0/1441962083 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.703+0000 7fe9b6d9d640 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754c6fc0 0x5578754ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.703+0000 7fe9b659c640 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754ab4d0 0x5578754bd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.703+0000 7fe9b5d9b640 1 -- 172.21.15.82:0/1441962083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754ab4d0 msgr2=0x5578754bd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.703+0000 7fe9b5d9b640 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754ab4d0 0x5578754bd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.703+0000 7fe9b5d9b640 1 -- 172.21.15.82:0/1441962083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754c6fc0 msgr2=0x5578754ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.703+0000 7fe9b5d9b640 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754c6fc0 0x5578754ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.703+0000 7fe9b5d9b640 1 -- 172.21.15.82:0/1441962083 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557875529d00 con 0x5578754b4a00 2024-04-21T08:57:10.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.703+0000 7fe9b5d9b640 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754b4a00 0x5578754ae240 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009c40 tx=0x7fe9a00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fbafddefc2dd2e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.706+0000 7fe99dffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:57:10.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.706+0000 7fe99dffb640 1 -- 172.21.15.82:0/1441962083 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0015040 con 0x5578754b4a00 2024-04-21T08:57:10.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.706+0000 7fe99dffb640 1 -- 172.21.15.82:0/1441962083 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7fe9a001ac70 con 0x5578754b4a00 2024-04-21T08:57:10.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.706+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/1441962083 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b0050070 con 0x5578754b4a00 2024-04-21T08:57:10.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.711+0000 7fe99dffb640 1 -- 172.21.15.82:0/1441962083 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe9a0037050 con 0x5578754b4a00 2024-04-21T08:57:10.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.711+0000 7fe9bfa5d740 10 client.18495 fetch_fsmap learned FSMap version 46 2024-04-21T08:57:10.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.711+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/1441962083 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55787552aec0 con 0x5578754b4a00 2024-04-21T08:57:10.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.711+0000 7fe99dffb640 1 -- 172.21.15.82:0/1441962083 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b0050070 con 0x5578754b4a00 2024-04-21T08:57:10.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.712+0000 7fe99dffb640 1 -- 172.21.15.82:0/1441962083 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fe9a0003740 con 0x5578754b4a00 2024-04-21T08:57:10.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.712+0000 7fe9bfa5d740 10 client.18495 fetch_fsmap finished waiting for FSMap version 46 2024-04-21T08:57:10.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.712+0000 7fe9bfa5d740 10 client.18495 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-04-21T08:57:10.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.712+0000 7fe9bfa5d740 20 client.18495 populate_metadata read hostname 'smithi082' 2024-04-21T08:57:10.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.712+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55787552b5c0 0x55787552d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.712+0000 7fe9bfa5d740 4 client.18495 mds_command: new command op to 24409 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:57:10.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.712+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/1441962083 --> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55787541dc10 con 0x55787552b5c0 2024-04-21T08:57:10.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.713+0000 7fe9b659c640 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55787552b5c0 0x55787552d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.713+0000 7fe9b659c640 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55787552b5c0 0x55787552d9e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a53933d0080f20cb server_cookie=24e9c810ccf4ef6e in_seq=0 out_seq=0 2024-04-21T08:57:10.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.713+0000 7fe99dffb640 10 client.18495 ms_handle_connect on v2:172.21.15.134:6832/2942732132 2024-04-21T08:57:10.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.714+0000 7fe99dffb640 1 -- 172.21.15.82:0/1441962083 <== mds.3 v2:172.21.15.134:6832/2942732132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55787541dc10 con 0x55787552b5c0 2024-04-21T08:57:10.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.714+0000 7fe99dffb640 10 client.18495 handle_command_reply: tid=0 2024-04-21T08:57:10.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.715+0000 7fe9bfa5d740 1 client.18495 shutdown 2024-04-21T08:57:10.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.721+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/1441962083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754b4a00 msgr2=0x5578754ae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.721+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754b4a00 0x5578754ae240 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009c40 tx=0x7fe9a00096c0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.725+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/1441962083 shutdown_connections 2024-04-21T08:57:10.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.725+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.134:6832/2942732132,v1:172.21.15.134:6833/2942732132] conn(0x55787552b5c0 0x55787552d9e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.725+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578754c6fc0 0x5578754ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.725+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578754b4a00 0x5578754ae240 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.725+0000 7fe9bfa5d740 1 --2- 172.21.15.82:0/1441962083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578754ab4d0 0x5578754bd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.725+0000 7fe99dffb640 0 client.18495 ms_handle_reset on v2:172.21.15.134:6832/2942732132 2024-04-21T08:57:10.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.725+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/1441962083 >> 172.21.15.82:0/1441962083 conn(0x5578754a5ba0 msgr2=0x5578754a6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.731+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/1441962083 shutdown_connections 2024-04-21T08:57:10.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.731+0000 7fe9bfa5d740 1 -- 172.21.15.82:0/1441962083 wait complete. 2024-04-21T08:57:10.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.732+0000 7fe9bfa5d740 20 client.18495 trim_cache size 0 max 16384 2024-04-21T08:57:10.736 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:57:10.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.741+0000 7fe9bce6a640 1 -- 172.21.15.82:0/870341093 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9a40a3f20 msgr2=0x7fe9a40a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.741+0000 7fe9bce6a640 1 --2- 172.21.15.82:0/870341093 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9a40a3f20 0x7fe9a40a63e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009a60 tx=0x7fe9b002c770 comp rx=0 tx=0).stop 2024-04-21T08:57:10.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.741+0000 7fe9bce6a640 1 -- 172.21.15.82:0/870341093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b8100fc0 msgr2=0x7fe9b8072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.741+0000 7fe9bce6a640 1 --2- 172.21.15.82:0/870341093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b8100fc0 0x7fe9b8072950 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800b450 tx=0x7fe9a800b910 comp rx=0 tx=0).stop 2024-04-21T08:57:10.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.746+0000 7fe9bce6a640 1 -- 172.21.15.82:0/870341093 shutdown_connections 2024-04-21T08:57:10.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.746+0000 7fe9bce6a640 1 --2- 172.21.15.82:0/870341093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9b81062e0 0x7fe9b810a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.746+0000 7fe9bce6a640 1 --2- 172.21.15.82:0/870341093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9b81018e0 0x7fe9b8072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.746+0000 7fe9bce6a640 1 --2- 172.21.15.82:0/870341093 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fe9a40a3f20 0x7fe9a40a63e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.746+0000 7fe9bce6a640 1 --2- 172.21.15.82:0/870341093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9b8100fc0 0x7fe9b8072950 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.746+0000 7fe9bce6a640 1 -- 172.21.15.82:0/870341093 >> 172.21.15.82:0/870341093 conn(0x7fe9b8077f50 msgr2=0x7fe9b8103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.747+0000 7fe9bce6a640 1 -- 172.21.15.82:0/870341093 shutdown_connections 2024-04-21T08:57:10.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.747+0000 7fe9bce6a640 1 -- 172.21.15.82:0/870341093 wait complete. 2024-04-21T08:57:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.758+0000 7f1994515640 1 -- 172.21.15.82:0/3662335682 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1970000bc0 con 0x7f19780a3f20 2024-04-21T08:57:10.771 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:57:10.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.781+0000 7f19837fe640 1 -- 172.21.15.82:0/3662335682 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f1970000bc0 con 0x7f19780a3f20 2024-04-21T08:57:10.781 INFO:teuthology.orchestra.run.smithi082.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-21T00:00:00", "created": "2024-04-21T08:47:08", "first": "2024-04-21T08:48:00", "last": "2024-04-21T08:57:00", "last_pruned": "2024-04-21T08:57:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-21T08:57:10.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.785+0000 7f1994515640 1 -- 172.21.15.82:0/3662335682 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19780a3f20 msgr2=0x7f19780a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.785+0000 7f1994515640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19780a3f20 0x7f19780a63e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f198c06d8c0 tx=0x7f19840090e0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.785+0000 7f1994515640 1 -- 172.21.15.82:0/3662335682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f198c10e220 msgr2=0x7f198c19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.785+0000 7f1994515640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f198c10e220 0x7f198c19ed20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f198803d700 tx=0x7f1988003970 comp rx=0 tx=0).stop 2024-04-21T08:57:10.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.789+0000 7f1994515640 1 -- 172.21.15.82:0/3662335682 shutdown_connections 2024-04-21T08:57:10.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.789+0000 7f1994515640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f19780a3f20 0x7f19780a63e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.789+0000 7f1994515640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f198c1104c0 0x7f198c1a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.789+0000 7f1994515640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f198c10ecc0 0x7f198c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.789+0000 7f1994515640 1 --2- 172.21.15.82:0/3662335682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f198c10e220 0x7f198c19ed20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.789+0000 7f1994515640 1 -- 172.21.15.82:0/3662335682 >> 172.21.15.82:0/3662335682 conn(0x7f198c06cf00 msgr2=0x7f198c10a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.792+0000 7f1994515640 1 -- 172.21.15.82:0/3662335682 shutdown_connections 2024-04-21T08:57:10.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.794+0000 7f1994515640 1 -- 172.21.15.82:0/3662335682 wait complete. 2024-04-21T08:57:10.818 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-21T08:57:10.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:10 smithi106 ceph-mon[31072]: pgmap v314: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-21T08:57:10.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:10 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2091317854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:57:10.910 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:10 smithi082 ceph-mon[27967]: pgmap v314: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-21T08:57:10.910 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:10 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2091317854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:57:10.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.972+0000 7f56015a4640 1 -- 172.21.15.82:0/2419319262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc11b730 msgr2=0x7f55fc11db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.972+0000 7f56015a4640 1 --2- 172.21.15.82:0/2419319262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc11b730 0x7f55fc11db20 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009c10 tx=0x7f55ec02bef0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.975+0000 7f56015a4640 1 -- 172.21.15.82:0/2419319262 shutdown_connections 2024-04-21T08:57:10.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.975+0000 7f56015a4640 1 --2- 172.21.15.82:0/2419319262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc11b730 0x7f55fc11db20 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.975+0000 7f56015a4640 1 --2- 172.21.15.82:0/2419319262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc10e7d0 0x7f55fc10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.975+0000 7f56015a4640 1 --2- 172.21.15.82:0/2419319262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55fc1107a0 0x7f55fc10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.975+0000 7f56015a4640 1 -- 172.21.15.82:0/2419319262 >> 172.21.15.82:0/2419319262 conn(0x7f55fc06d060 msgr2=0x7f55fc06d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:10 smithi134 ceph-mon[31772]: pgmap v314: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-21T08:57:10.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:10 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2091317854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:57:10.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.978+0000 7f56015a4640 1 -- 172.21.15.82:0/2419319262 shutdown_connections 2024-04-21T08:57:10.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.978+0000 7f56015a4640 1 -- 172.21.15.82:0/2419319262 wait complete. 2024-04-21T08:57:10.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.979+0000 7f56015a4640 1 Processor -- start 2024-04-21T08:57:10.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.980+0000 7f56015a4640 1 -- start start 2024-04-21T08:57:10.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.980+0000 7f56015a4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc10e7d0 0x7f55fc1a7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.980+0000 7f56015a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc1107a0 0x7f55fc1a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.980+0000 7f56015a4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55fc11b730 0x7f55fc1abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.980+0000 7f56015a4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc120f40 con 0x7f55fc10e7d0 2024-04-21T08:57:10.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.980+0000 7f56015a4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc120dc0 con 0x7f55fc11b730 2024-04-21T08:57:10.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.980+0000 7f56015a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc1210c0 con 0x7f55fc1107a0 2024-04-21T08:57:10.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.980+0000 7f55faffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc10e7d0 0x7f55fc1a7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.980+0000 7f55faffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc10e7d0 0x7f55fc1a7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33454/0 (socket says 172.21.15.82:33454) 2024-04-21T08:57:10.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.980+0000 7f55faffd640 1 -- 172.21.15.82:0/2505947538 learned_addr learned my addr 172.21.15.82:0/2505947538 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.981+0000 7f55fb7fe640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55fc11b730 0x7f55fc1abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.981+0000 7f55fb7fe640 1 -- 172.21.15.82:0/2505947538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc1107a0 msgr2=0x7f55fc1a7ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:57:10.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.981+0000 7f55fb7fe640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc1107a0 0x7f55fc1a7ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.981+0000 7f55fb7fe640 1 -- 172.21.15.82:0/2505947538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc10e7d0 msgr2=0x7f55fc1a7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.981+0000 7f55fb7fe640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc10e7d0 0x7f55fc1a7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.981+0000 7f55fb7fe640 1 -- 172.21.15.82:0/2505947538 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc1ac560 con 0x7f55fc11b730 2024-04-21T08:57:10.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.982+0000 7f55fb7fe640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55fc11b730 0x7f55fc1abe30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009be0 tx=0x7f55ec033970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.982+0000 7f55f3fff640 1 -- 172.21.15.82:0/2505947538 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec043020 con 0x7f55fc11b730 2024-04-21T08:57:10.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.982+0000 7f55faffd640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc10e7d0 0x7f55fc1a7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:57:10.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.982+0000 7f56015a4640 1 -- 172.21.15.82:0/2505947538 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55fc1ac7f0 con 0x7f55fc11b730 2024-04-21T08:57:10.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.982+0000 7f56015a4640 1 -- 172.21.15.82:0/2505947538 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55fc1b40e0 con 0x7f55fc11b730 2024-04-21T08:57:10.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.982+0000 7f55f3fff640 1 -- 172.21.15.82:0/2505947538 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55ec0333e0 con 0x7f55fc11b730 2024-04-21T08:57:10.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.984+0000 7f55f3fff640 1 -- 172.21.15.82:0/2505947538 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec02ca90 con 0x7f55fc11b730 2024-04-21T08:57:10.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.987+0000 7f5604194740 1 Processor -- start 2024-04-21T08:57:10.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.987+0000 7f5604194740 1 -- start start 2024-04-21T08:57:10.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.987+0000 7f55f3fff640 1 -- 172.21.15.82:0/2505947538 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f55ec046020 con 0x7f55fc11b730 2024-04-21T08:57:10.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.988+0000 7f5604194740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ebc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.988+0000 7f5604194740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.988+0000 7f5604194740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 0x556972ec7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.988+0000 7f55fa7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ebc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.988+0000 7f55f3fff640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f55e80a3c80 0x7f55e80a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.988+0000 7f55fa7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ebc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52368/0 (socket says 172.21.15.82:52368) 2024-04-21T08:57:10.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.989+0000 7f55fa7fc640 1 -- 172.21.15.82:0/1296840156 learned_addr learned my addr 172.21.15.82:0/1296840156 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.989+0000 7f55fb7fe640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f55e80a3c80 0x7f55e80a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.989+0000 7f55faffd640 1 --2- 172.21.15.82:0/1296840156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.989+0000 7f55f3fff640 1 -- 172.21.15.82:0/2505947538 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f55ec0f3140 con 0x7f55fc11b730 2024-04-21T08:57:10.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.989+0000 7f5604194740 1 -- 172.21.15.82:0/1296840156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556972e211f0 con 0x556972ec5a00 2024-04-21T08:57:10.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.989+0000 7f5604194740 1 -- 172.21.15.82:0/1296840156 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556972dbe4b0 con 0x556972ebce80 2024-04-21T08:57:10.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.989+0000 7f5604194740 1 -- 172.21.15.82:0/1296840156 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556972e1d1b0 con 0x556972ebc4d0 2024-04-21T08:57:10.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.991+0000 7f55fa7fc640 1 --2- 172.21.15.82:0/1296840156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 0x556972ec7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.991+0000 7f55faffd640 1 -- 172.21.15.82:0/1296840156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 msgr2=0x556972ebc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.991+0000 7f55faffd640 1 --2- 172.21.15.82:0/1296840156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ebc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.991+0000 7f55faffd640 1 -- 172.21.15.82:0/1296840156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 msgr2=0x556972ec7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:10.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.991+0000 7f55faffd640 1 --2- 172.21.15.82:0/1296840156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 0x556972ec7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.991+0000 7f55faffd640 1 -- 172.21.15.82:0/1296840156 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556972db01b0 con 0x556972ebce80 2024-04-21T08:57:10.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.992+0000 7f55faffd640 1 --2- 172.21.15.82:0/1296840156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec5430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f55f4011440 tx=0x7f55f4011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff3d7d123ed09e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.993+0000 7f55f17fa640 1 -- 172.21.15.82:0/1296840156 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4004030 con 0x556972ebce80 2024-04-21T08:57:10.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.993+0000 7f55f17fa640 1 -- 172.21.15.82:0/1296840156 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f40041d0 con 0x556972ebce80 2024-04-21T08:57:10.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.993+0000 7f55f17fa640 1 -- 172.21.15.82:0/1296840156 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f400f860 con 0x556972ebce80 2024-04-21T08:57:10.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.994+0000 7f5604194740 1 -- 172.21.15.82:0/1296840156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 msgr2=0x556972ec5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:10.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.994+0000 7f5604194740 1 --2- 172.21.15.82:0/1296840156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec5430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f55f4011440 tx=0x7f55f4011900 comp rx=0 tx=0).stop 2024-04-21T08:57:10.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.994+0000 7f5604194740 1 -- 172.21.15.82:0/1296840156 shutdown_connections 2024-04-21T08:57:10.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.994+0000 7f5604194740 1 --2- 172.21.15.82:0/1296840156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 0x556972ec7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.994+0000 7f5604194740 1 --2- 172.21.15.82:0/1296840156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec5430 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.994+0000 7f5604194740 1 --2- 172.21.15.82:0/1296840156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ebc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:10.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.995+0000 7f5604194740 1 -- 172.21.15.82:0/1296840156 >> 172.21.15.82:0/1296840156 conn(0x556972eb6ba0 msgr2=0x556972eb6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:10.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.995+0000 7f5604194740 1 -- 172.21.15.82:0/1296840156 shutdown_connections 2024-04-21T08:57:10.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.996+0000 7f55fb7fe640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f55e80a3c80 0x7f55e80a6140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0c7460 tx=0x7f55ec005c40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:10.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.996+0000 7f5604194740 1 -- 172.21.15.82:0/1296840156 wait complete. 2024-04-21T08:57:10.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.997+0000 7f5604194740 1 -- start start 2024-04-21T08:57:10.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.997+0000 7f5604194740 1 Processor -- start 2024-04-21T08:57:10.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.998+0000 7f5604194740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ec52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.998+0000 7f5604194740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.998+0000 7f5604194740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 0x556972ec2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.998+0000 7f55fa7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ec52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.998+0000 7f55fa7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ec52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52382/0 (socket says 172.21.15.82:52382) 2024-04-21T08:57:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.998+0000 7f55fa7fc640 1 -- 172.21.15.82:0/331210343 learned_addr learned my addr 172.21.15.82:0/331210343 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.998+0000 7f55faffd640 1 --2- 172.21.15.82:0/331210343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.998+0000 7f5604194740 1 -- 172.21.15.82:0/331210343 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556972ec8970 con 0x556972ec5a00 2024-04-21T08:57:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.999+0000 7f5604194740 1 -- 172.21.15.82:0/331210343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556972dda500 con 0x556972ebce80 2024-04-21T08:57:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.999+0000 7f5604194740 1 -- 172.21.15.82:0/331210343 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556972ec8af0 con 0x556972ebc4d0 2024-04-21T08:57:10.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:10.999+0000 7f5604194740 10 client.0 _ll_register_callbacks cb 0x556972da8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:57:11.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.001+0000 7f55fa7fc640 1 -- 172.21.15.82:0/331210343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 msgr2=0x556972ec2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.001+0000 7f55fa7fc640 1 --2- 172.21.15.82:0/331210343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.001+0000 7f55fa7fc640 1 -- 172.21.15.82:0/331210343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 msgr2=0x556972ec2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.001+0000 7f55fa7fc640 1 --2- 172.21.15.82:0/331210343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 0x556972ec2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.001+0000 7f55fa7fc640 1 -- 172.21.15.82:0/331210343 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556972b333f0 con 0x556972ebc4d0 2024-04-21T08:57:11.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.005+0000 7f55fa7fc640 1 --2- 172.21.15.82:0/331210343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ec52e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f55e4012c00 tx=0x7f55e4010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80d6611f71ad3fa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.005+0000 7f55e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:57:11.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.005+0000 7f55e27fc640 1 -- 172.21.15.82:0/331210343 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4010e70 con 0x556972ebc4d0 2024-04-21T08:57:11.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.005+0000 7f55e27fc640 1 -- 172.21.15.82:0/331210343 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f55e4011440 con 0x556972ebc4d0 2024-04-21T08:57:11.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.006+0000 7f5604194740 1 -- 172.21.15.82:0/331210343 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f40041d0 con 0x556972ebc4d0 2024-04-21T08:57:11.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.006+0000 7f55e27fc640 1 -- 172.21.15.82:0/331210343 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f55e4023050 con 0x556972ebc4d0 2024-04-21T08:57:11.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.006+0000 7f5604194740 10 client.27154 fetch_fsmap learned FSMap version 46 2024-04-21T08:57:11.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.006+0000 7f5604194740 1 -- 172.21.15.82:0/331210343 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556972bfd7d0 con 0x556972ebc4d0 2024-04-21T08:57:11.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.007+0000 7f55e27fc640 1 -- 172.21.15.82:0/331210343 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f40041d0 con 0x556972ebc4d0 2024-04-21T08:57:11.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.008+0000 7f55e27fc640 1 -- 172.21.15.82:0/331210343 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f55e401e680 con 0x556972ebc4d0 2024-04-21T08:57:11.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.008+0000 7f5604194740 10 client.27154 fetch_fsmap finished waiting for FSMap version 46 2024-04-21T08:57:11.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.008+0000 7f5604194740 10 client.27154 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:57:11.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.008+0000 7f5604194740 20 client.27154 populate_metadata read hostname 'smithi082' 2024-04-21T08:57:11.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.008+0000 7f5604194740 1 --2- 172.21.15.82:0/331210343 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x556972ed5a10 0x556972ed7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.008+0000 7f5604194740 4 client.27154 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:57:11.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.008+0000 7f5604194740 1 -- 172.21.15.82:0/331210343 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556972dc0930 con 0x556972ed5a10 2024-04-21T08:57:11.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.010+0000 7f55faffd640 1 --2- 172.21.15.82:0/331210343 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x556972ed5a10 0x556972ed7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.011+0000 7f55faffd640 1 --2- 172.21.15.82:0/331210343 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x556972ed5a10 0x556972ed7e80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59d8e160f4072bcc server_cookie=c45b9944a1ebb534 in_seq=0 out_seq=0 2024-04-21T08:57:11.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.011+0000 7f55e27fc640 10 client.27154 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:57:11.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.012+0000 7f55e27fc640 1 -- 172.21.15.82:0/331210343 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556972dc0930 con 0x556972ed5a10 2024-04-21T08:57:11.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.012+0000 7f55e27fc640 10 client.27154 handle_command_reply: tid=0 2024-04-21T08:57:11.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.013+0000 7f5604194740 1 client.27154 shutdown 2024-04-21T08:57:11.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.018+0000 7f5604194740 1 -- 172.21.15.82:0/331210343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 msgr2=0x556972ec52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.018+0000 7f5604194740 1 --2- 172.21.15.82:0/331210343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ec52e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f55e4012c00 tx=0x7f55e4010800 comp rx=0 tx=0).stop 2024-04-21T08:57:11.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.026+0000 7f5604194740 1 -- 172.21.15.82:0/331210343 shutdown_connections 2024-04-21T08:57:11.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.026+0000 7f5604194740 1 --2- 172.21.15.82:0/331210343 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x556972ed5a10 0x556972ed7e80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.026+0000 7f5604194740 1 --2- 172.21.15.82:0/331210343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 0x556972ec2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.026+0000 7f5604194740 1 --2- 172.21.15.82:0/331210343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.026+0000 7f5604194740 1 --2- 172.21.15.82:0/331210343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ec52e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.026+0000 7f5604194740 1 -- 172.21.15.82:0/331210343 >> 172.21.15.82:0/331210343 conn(0x556972eb6ba0 msgr2=0x556972eb88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.027+0000 7f55e27fc640 0 client.27154 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:57:11.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.033+0000 7f5604194740 1 -- 172.21.15.82:0/331210343 shutdown_connections 2024-04-21T08:57:11.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.033+0000 7f5604194740 1 -- 172.21.15.82:0/331210343 wait complete. 2024-04-21T08:57:11.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.034+0000 7f5604194740 20 client.27154 trim_cache size 0 max 16384 2024-04-21T08:57:11.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.063+0000 7f13c64ab640 1 -- 172.21.15.82:0/1011262279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c0110610 msgr2=0x7f13c010e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.063+0000 7f13c64ab640 1 --2- 172.21.15.82:0/1011262279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c0110610 0x7f13c010e0c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f13b801acb0 tx=0x7f13b803d200 comp rx=0 tx=0).stop 2024-04-21T08:57:11.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.064+0000 7f13c64ab640 1 -- 172.21.15.82:0/1011262279 shutdown_connections 2024-04-21T08:57:11.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.064+0000 7f13c64ab640 1 --2- 172.21.15.82:0/1011262279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c010ec20 0x7f13c011d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.064+0000 7f13c64ab640 1 --2- 172.21.15.82:0/1011262279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13c01109f0 0x7f13c010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.064+0000 7f13c64ab640 1 --2- 172.21.15.82:0/1011262279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c0110610 0x7f13c010e0c0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.064+0000 7f13c64ab640 1 -- 172.21.15.82:0/1011262279 >> 172.21.15.82:0/1011262279 conn(0x7f13c006ce90 msgr2=0x7f13c006d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.071+0000 7f13c64ab640 1 -- 172.21.15.82:0/1011262279 shutdown_connections 2024-04-21T08:57:11.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.074+0000 7f5604194740 1 Processor -- start 2024-04-21T08:57:11.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.074+0000 7f13c64ab640 1 -- 172.21.15.82:0/1011262279 wait complete. 2024-04-21T08:57:11.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.075+0000 7f13c64ab640 1 Processor -- start 2024-04-21T08:57:11.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.076+0000 7f5604194740 1 -- start start 2024-04-21T08:57:11.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.075+0000 7f13c64ab640 1 -- start start 2024-04-21T08:57:11.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.075+0000 7f13c64ab640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c010ec20 0x7f13c011d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.075+0000 7f13c64ab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c0110610 0x7f13c0119360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.075+0000 7f13c64ab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13c01109f0 0x7f13c01198a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.075+0000 7f13c64ab640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f13c0113880 con 0x7f13c0110610 2024-04-21T08:57:11.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.075+0000 7f13c64ab640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f13c0113700 con 0x7f13c01109f0 2024-04-21T08:57:11.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.075+0000 7f13c64ab640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13c0113a00 con 0x7f13c010ec20 2024-04-21T08:57:11.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.075+0000 7f13c4ca8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c0110610 0x7f13c0119360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.076+0000 7f13c4ca8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c0110610 0x7f13c0119360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33500/0 (socket says 172.21.15.82:33500) 2024-04-21T08:57:11.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.076+0000 7f13c4ca8640 1 -- 172.21.15.82:0/3990542873 learned_addr learned my addr 172.21.15.82:0/3990542873 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.077+0000 7f5604194740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ed18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.077+0000 7f5604194740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.077+0000 7f5604194740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 0x556972ec52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.077+0000 7f5604194740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc0102b0 con 0x556972ec5a00 2024-04-21T08:57:11.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.077+0000 7f5604194740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc011e20 con 0x556972ebce80 2024-04-21T08:57:11.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.077+0000 7f5604194740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc010980 con 0x556972ebc4d0 2024-04-21T08:57:11.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.076+0000 7f13c54a9640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c010ec20 0x7f13c011d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.076+0000 7f13c5caa640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13c01109f0 0x7f13c01198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.076+0000 7f13c4ca8640 1 -- 172.21.15.82:0/3990542873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c010ec20 msgr2=0x7f13c011d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.076+0000 7f13c4ca8640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c010ec20 0x7f13c011d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.076+0000 7f13c4ca8640 1 -- 172.21.15.82:0/3990542873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13c01109f0 msgr2=0x7f13c01198a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.076+0000 7f13c4ca8640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13c01109f0 0x7f13c01198a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.076+0000 7f13c4ca8640 1 -- 172.21.15.82:0/3990542873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c0119e60 con 0x7f13c0110610 2024-04-21T08:57:11.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.077+0000 7f13c54a9640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c010ec20 0x7f13c011d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:57:11.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.078+0000 7f55fa7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ed18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.078+0000 7f55fa7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ed18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52410/0 (socket says 172.21.15.82:52410) 2024-04-21T08:57:11.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.078+0000 7f55fa7fc640 1 -- 172.21.15.82:0/2440388756 learned_addr learned my addr 172.21.15.82:0/2440388756 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.078+0000 7f55fa7fc640 1 --2- 172.21.15.82:0/2440388756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 0x556972ec52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.081+0000 7f13c4ca8640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c0110610 0x7f13c0119360 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00b450 tx=0x7f13bc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.081+0000 7f13b67fc640 1 -- 172.21.15.82:0/3990542873 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc015020 con 0x7f13c0110610 2024-04-21T08:57:11.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.082+0000 7f13c64ab640 1 -- 172.21.15.82:0/3990542873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c011a150 con 0x7f13c0110610 2024-04-21T08:57:11.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.082+0000 7f13c64ab640 1 -- 172.21.15.82:0/3990542873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c007a4e0 con 0x7f13c0110610 2024-04-21T08:57:11.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.082+0000 7f55faffd640 1 --2- 172.21.15.82:0/2440388756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.082+0000 7f55fa7fc640 1 -- 172.21.15.82:0/2440388756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 msgr2=0x556972ec4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.082+0000 7f55fa7fc640 1 --2- 172.21.15.82:0/2440388756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.082+0000 7f55fa7fc640 1 -- 172.21.15.82:0/2440388756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 msgr2=0x556972ec52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.082+0000 7f55fa7fc640 1 --2- 172.21.15.82:0/2440388756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 0x556972ec52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.082+0000 7f55fa7fc640 1 -- 172.21.15.82:0/2440388756 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556972eb2470 con 0x556972ebc4d0 2024-04-21T08:57:11.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.087+0000 7f55fa7fc640 1 --2- 172.21.15.82:0/2440388756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ed18c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f55e4010d80 tx=0x7f55e4009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df380f6fb61e3bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.087+0000 7f55f0ff9640 1 -- 172.21.15.82:0/2440388756 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4012260 con 0x556972ebc4d0 2024-04-21T08:57:11.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.087+0000 7f55f0ff9640 1 -- 172.21.15.82:0/2440388756 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55e4016030 con 0x556972ebc4d0 2024-04-21T08:57:11.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.087+0000 7f55f0ff9640 1 -- 172.21.15.82:0/2440388756 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4004a00 con 0x556972ebc4d0 2024-04-21T08:57:11.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.088+0000 7f5604194740 1 -- 172.21.15.82:0/2440388756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 msgr2=0x556972ed18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.088+0000 7f5604194740 1 --2- 172.21.15.82:0/2440388756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ed18c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f55e4010d80 tx=0x7f55e4009600 comp rx=0 tx=0).stop 2024-04-21T08:57:11.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.088+0000 7f5604194740 1 -- 172.21.15.82:0/2440388756 shutdown_connections 2024-04-21T08:57:11.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.088+0000 7f5604194740 1 --2- 172.21.15.82:0/2440388756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ec5a00 0x556972ec52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.088+0000 7f5604194740 1 --2- 172.21.15.82:0/2440388756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ebce80 0x556972ec4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.088+0000 7f5604194740 1 --2- 172.21.15.82:0/2440388756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ebc4d0 0x556972ed18c0 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f55e4010d80 tx=0x7f55e4009600 comp rx=0 tx=0).stop 2024-04-21T08:57:11.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.088+0000 7f5604194740 1 -- 172.21.15.82:0/2440388756 >> 172.21.15.82:0/2440388756 conn(0x556972eb6ba0 msgr2=0x556972eb5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.082+0000 7f13b67fc640 1 -- 172.21.15.82:0/3990542873 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13bc011d40 con 0x7f13c0110610 2024-04-21T08:57:11.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.083+0000 7f13b67fc640 1 -- 172.21.15.82:0/3990542873 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc01ade0 con 0x7f13c0110610 2024-04-21T08:57:11.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.085+0000 7f13c64ab640 1 -- 172.21.15.82:0/3990542873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a4005290 con 0x7f13c0110610 2024-04-21T08:57:11.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.086+0000 7f13b67fc640 1 -- 172.21.15.82:0/3990542873 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f13bc018020 con 0x7f13c0110610 2024-04-21T08:57:11.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.087+0000 7f13b67fc640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f13ac0a3de0 0x7f13ac0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.087+0000 7f13b67fc640 1 -- 172.21.15.82:0/3990542873 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f13bc0d0710 con 0x7f13c0110610 2024-04-21T08:57:11.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.089+0000 7f13c54a9640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f13ac0a3de0 0x7f13ac0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.089+0000 7f13c54a9640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f13ac0a3de0 0x7f13ac0a62a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f13b8048040 tx=0x7f13b803f040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.095+0000 7f5604194740 1 -- 172.21.15.82:0/2440388756 shutdown_connections 2024-04-21T08:57:11.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.098+0000 7f5604194740 1 -- 172.21.15.82:0/2440388756 wait complete. 2024-04-21T08:57:11.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.098+0000 7f5604194740 1 -- start start 2024-04-21T08:57:11.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.098+0000 7f5604194740 1 Processor -- start 2024-04-21T08:57:11.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.098+0000 7f5604194740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ebce80 0x556972ece090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.098+0000 7f5604194740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ec5a00 0x556972ec9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.099+0000 7f5604194740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ec98d0 0x556972ec9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.099+0000 7f5604194740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc011750 con 0x556972ebce80 2024-04-21T08:57:11.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.099+0000 7f5604194740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc011050 con 0x556972ec5a00 2024-04-21T08:57:11.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.099+0000 7f5604194740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc00e280 con 0x556972ec98d0 2024-04-21T08:57:11.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.099+0000 7f5604194740 10 client.0 _ll_register_callbacks cb 0x7f55e40128b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:57:11.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.100+0000 7f13b67fc640 1 -- 172.21.15.82:0/3990542873 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13bc01f020 con 0x7f13c0110610 2024-04-21T08:57:11.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.101+0000 7f55fa7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ec98d0 0x556972ec9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.101+0000 7f55fa7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ec98d0 0x556972ec9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52422/0 (socket says 172.21.15.82:52422) 2024-04-21T08:57:11.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.101+0000 7f55fa7fc640 1 -- 172.21.15.82:0/2654493891 learned_addr learned my addr 172.21.15.82:0/2654493891 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.101+0000 7f55fa7fc640 1 -- 172.21.15.82:0/2654493891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ec5a00 msgr2=0x556972ec9390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:57:11.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.101+0000 7f55fa7fc640 1 --2- 172.21.15.82:0/2654493891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ec5a00 0x556972ec9390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.101+0000 7f55fa7fc640 1 -- 172.21.15.82:0/2654493891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ebce80 msgr2=0x556972ece090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.101+0000 7f55fa7fc640 1 --2- 172.21.15.82:0/2654493891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ebce80 0x556972ece090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.101+0000 7f55fa7fc640 1 -- 172.21.15.82:0/2654493891 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556972f4bfd0 con 0x556972ec98d0 2024-04-21T08:57:11.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.101+0000 7f55fa7fc640 1 --2- 172.21.15.82:0/2654493891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ec98d0 0x556972ec9d50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55e402f000 tx=0x7f55e4012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa93eceb2102d1be server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.103+0000 7f55e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:57:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.106+0000 7f55e27fc640 1 -- 172.21.15.82:0/2654493891 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e403a060 con 0x556972ec98d0 2024-04-21T08:57:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.106+0000 7f55e27fc640 1 -- 172.21.15.82:0/2654493891 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f55e4012260 con 0x556972ec98d0 2024-04-21T08:57:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.106+0000 7f5604194740 1 -- 172.21.15.82:0/2654493891 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55e4016030 con 0x556972ec98d0 2024-04-21T08:57:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.110+0000 7f55e27fc640 1 -- 172.21.15.82:0/2654493891 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f55e4004030 con 0x556972ec98d0 2024-04-21T08:57:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.110+0000 7f5604194740 10 client.27160 fetch_fsmap learned FSMap version 46 2024-04-21T08:57:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.110+0000 7f5604194740 1 -- 172.21.15.82:0/2654493891 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556972f4d2b0 con 0x556972ec98d0 2024-04-21T08:57:11.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.121+0000 7f55e27fc640 1 -- 172.21.15.82:0/2654493891 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55e4016030 con 0x556972ec98d0 2024-04-21T08:57:11.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.121+0000 7f55e27fc640 1 -- 172.21.15.82:0/2654493891 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7f55e4005040 con 0x556972ec98d0 2024-04-21T08:57:11.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.121+0000 7f5604194740 10 client.27160 fetch_fsmap finished waiting for FSMap version 46 2024-04-21T08:57:11.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.121+0000 7f5604194740 10 client.27160 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:57:11.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.121+0000 7f5604194740 20 client.27160 populate_metadata read hostname 'smithi082' 2024-04-21T08:57:11.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.121+0000 7f5604194740 1 --2- 172.21.15.82:0/2654493891 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x556972f4d7c0 0x556972f4fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.121+0000 7f5604194740 4 client.27160 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:57:11.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.121+0000 7f5604194740 1 -- 172.21.15.82:0/2654493891 --> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556972e2ec10 con 0x556972f4d7c0 2024-04-21T08:57:11.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.123+0000 7f55faffd640 1 --2- 172.21.15.82:0/2654493891 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x556972f4d7c0 0x556972f4fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.136+0000 7f55faffd640 1 --2- 172.21.15.82:0/2654493891 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x556972f4d7c0 0x556972f4fbe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1ee2756acfb55cb server_cookie=f48b485b764d89ed in_seq=0 out_seq=0 2024-04-21T08:57:11.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.136+0000 7f55e27fc640 10 client.27160 ms_handle_connect on v2:172.21.15.106:6836/3731760515 2024-04-21T08:57:11.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.138+0000 7f55e27fc640 1 -- 172.21.15.82:0/2654493891 <== mds.2 v2:172.21.15.106:6836/3731760515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556972e2ec10 con 0x556972f4d7c0 2024-04-21T08:57:11.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.138+0000 7f55e27fc640 10 client.27160 handle_command_reply: tid=0 2024-04-21T08:57:11.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.139+0000 7f5604194740 1 client.27160 shutdown 2024-04-21T08:57:11.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.139+0000 7f5604194740 1 -- 172.21.15.82:0/2654493891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ec98d0 msgr2=0x556972ec9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.139+0000 7f5604194740 1 --2- 172.21.15.82:0/2654493891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ec98d0 0x556972ec9d50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55e402f000 tx=0x7f55e4012b90 comp rx=0 tx=0).stop 2024-04-21T08:57:11.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.140+0000 7f5604194740 1 -- 172.21.15.82:0/2654493891 shutdown_connections 2024-04-21T08:57:11.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.140+0000 7f5604194740 1 --2- 172.21.15.82:0/2654493891 >> [v2:172.21.15.106:6836/3731760515,v1:172.21.15.106:6837/3731760515] conn(0x556972f4d7c0 0x556972f4fbe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.140+0000 7f5604194740 1 --2- 172.21.15.82:0/2654493891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556972ec98d0 0x556972ec9d50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.140+0000 7f5604194740 1 --2- 172.21.15.82:0/2654493891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556972ec5a00 0x556972ec9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.140+0000 7f5604194740 1 --2- 172.21.15.82:0/2654493891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556972ebce80 0x556972ece090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.140+0000 7f5604194740 1 -- 172.21.15.82:0/2654493891 >> 172.21.15.82:0/2654493891 conn(0x556972eb6ba0 msgr2=0x556972ebd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.140+0000 7f55e27fc640 0 client.27160 ms_handle_reset on v2:172.21.15.106:6836/3731760515 2024-04-21T08:57:11.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.143+0000 7f5604194740 1 -- 172.21.15.82:0/2654493891 shutdown_connections 2024-04-21T08:57:11.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.145+0000 7f5604194740 1 -- 172.21.15.82:0/2654493891 wait complete. 2024-04-21T08:57:11.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.147+0000 7f5604194740 20 client.27160 trim_cache size 0 max 16384 2024-04-21T08:57:11.147 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:57:11.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.153+0000 7f56015a4640 1 -- 172.21.15.82:0/2505947538 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f55e80a3c80 msgr2=0x7f55e80a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.153+0000 7f56015a4640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f55e80a3c80 0x7f55e80a6140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0c7460 tx=0x7f55ec005c40 comp rx=0 tx=0).stop 2024-04-21T08:57:11.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.153+0000 7f56015a4640 1 -- 172.21.15.82:0/2505947538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55fc11b730 msgr2=0x7f55fc1abe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.153+0000 7f56015a4640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55fc11b730 0x7f55fc1abe30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009be0 tx=0x7f55ec033970 comp rx=0 tx=0).stop 2024-04-21T08:57:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.155+0000 7f56015a4640 1 -- 172.21.15.82:0/2505947538 shutdown_connections 2024-04-21T08:57:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.155+0000 7f56015a4640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55fc11b730 0x7f55fc1abe30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.155+0000 7f56015a4640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55fc1107a0 0x7f55fc1a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.155+0000 7f56015a4640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f55e80a3c80 0x7f55e80a6140 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.155+0000 7f56015a4640 1 --2- 172.21.15.82:0/2505947538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55fc10e7d0 0x7f55fc1a7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.155+0000 7f56015a4640 1 -- 172.21.15.82:0/2505947538 >> 172.21.15.82:0/2505947538 conn(0x7f55fc06d060 msgr2=0x7f55fc072a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.157+0000 7f56015a4640 1 -- 172.21.15.82:0/2505947538 shutdown_connections 2024-04-21T08:57:11.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.157+0000 7f56015a4640 1 -- 172.21.15.82:0/2505947538 wait complete. 2024-04-21T08:57:11.189 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:57:11.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.404+0000 7fbbdb385640 1 -- 172.21.15.82:0/4208838742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbd411b730 msgr2=0x7fbbd411db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.404+0000 7fbbdb385640 1 --2- 172.21.15.82:0/4208838742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbd411b730 0x7fbbd411db20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009ca0 tx=0x7fbbcc02c140 comp rx=0 tx=0).stop 2024-04-21T08:57:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.409+0000 7fbbdb385640 1 -- 172.21.15.82:0/4208838742 shutdown_connections 2024-04-21T08:57:11.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.409+0000 7fbbdb385640 1 --2- 172.21.15.82:0/4208838742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbd411b730 0x7fbbd411db20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.409+0000 7fbbdb385640 1 --2- 172.21.15.82:0/4208838742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbd410e7d0 0x7fbbd410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.409+0000 7fbbdb385640 1 --2- 172.21.15.82:0/4208838742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd41107a0 0x7fbbd410e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.409+0000 7fbbdb385640 1 -- 172.21.15.82:0/4208838742 >> 172.21.15.82:0/4208838742 conn(0x7fbbd406d060 msgr2=0x7fbbd406d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.411+0000 7fbbdb385640 1 -- 172.21.15.82:0/4208838742 shutdown_connections 2024-04-21T08:57:11.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.413+0000 7fbbdb385640 1 -- 172.21.15.82:0/4208838742 wait complete. 2024-04-21T08:57:11.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.414+0000 7fbbdb385640 1 Processor -- start 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.415+0000 7fbbdb385640 1 -- start start 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.415+0000 7fbbdb385640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd410e7d0 0x7fbbd4118d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.415+0000 7fbbdb385640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbd41107a0 0x7fbbd4111e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.415+0000 7fbbdb385640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbd411b730 0x7fbbd4112360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.415+0000 7fbbdb385640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd4121030 con 0x7fbbd41107a0 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.415+0000 7fbbdb385640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd4120eb0 con 0x7fbbd410e7d0 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.415+0000 7fbbdb385640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd41211b0 con 0x7fbbd411b730 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.415+0000 7fbbd9121640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd410e7d0 0x7fbbd4118d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.415+0000 7fbbd9121640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd410e7d0 0x7fbbd4118d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:59748/0 (socket says 172.21.15.82:59748) 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.415+0000 7fbbd9121640 1 -- 172.21.15.82:0/3111773303 learned_addr learned my addr 172.21.15.82:0/3111773303 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.415+0000 7fbbd8920640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbd41107a0 0x7fbbd4111e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.416+0000 7fbbd9121640 1 -- 172.21.15.82:0/3111773303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbd411b730 msgr2=0x7fbbd4112360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.416+0000 7fbbd9121640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbd411b730 0x7fbbd4112360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.416+0000 7fbbd9121640 1 -- 172.21.15.82:0/3111773303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbd41107a0 msgr2=0x7fbbd4111e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.416+0000 7fbbd9121640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbd41107a0 0x7fbbd4111e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.416+0000 7fbbd9121640 1 -- 172.21.15.82:0/3111773303 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd4112970 con 0x7fbbd410e7d0 2024-04-21T08:57:11.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.416+0000 7fbbd9121640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd410e7d0 0x7fbbd4118d50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400b450 tx=0x7fbbc400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.420+0000 7fbbca7fc640 1 -- 172.21.15.82:0/3111773303 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4015020 con 0x7fbbd410e7d0 2024-04-21T08:57:11.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.420+0000 7fbbdb385640 1 -- 172.21.15.82:0/3111773303 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd4112c60 con 0x7fbbd410e7d0 2024-04-21T08:57:11.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.421+0000 7fbbdb385640 1 -- 172.21.15.82:0/3111773303 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd4116bf0 con 0x7fbbd410e7d0 2024-04-21T08:57:11.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.421+0000 7fbbca7fc640 1 -- 172.21.15.82:0/3111773303 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc4011d40 con 0x7fbbd410e7d0 2024-04-21T08:57:11.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.421+0000 7fbbca7fc640 1 -- 172.21.15.82:0/3111773303 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4010e30 con 0x7fbbd410e7d0 2024-04-21T08:57:11.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.424+0000 7fbbca7fc640 1 -- 172.21.15.82:0/3111773303 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbbc4018020 con 0x7fbbd410e7d0 2024-04-21T08:57:11.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.425+0000 7fbbca7fc640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbbc00a3eb0 0x7fbbc00a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.425+0000 7fbbca7fc640 1 -- 172.21.15.82:0/3111773303 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7fbbc40d1440 con 0x7fbbd410e7d0 2024-04-21T08:57:11.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.425+0000 7fbbd8920640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbbc00a3eb0 0x7fbbc00a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.426+0000 7fbbd8920640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbbc00a3eb0 0x7fbbc00a6370 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4113590 tx=0x7fbbd0005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.426+0000 7fbbddf73740 1 Processor -- start 2024-04-21T08:57:11.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.427+0000 7fbbddf73740 1 -- start start 2024-04-21T08:57:11.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.427+0000 7fbbddf73740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30af4d0 0x55add30af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.427+0000 7fbbddf73740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30afe80 0x55add30b8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbddf73740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30bae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbddf73740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55add30141f0 con 0x55add30b8a00 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbddf73740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55add2fb14b0 con 0x55add30af4d0 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbddf73740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55add30101b0 con 0x55add30afe80 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbd9121640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30af4d0 0x55add30af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbd9121640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30af4d0 0x55add30af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:59762/0 (socket says 172.21.15.82:59762) 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbd9121640 1 -- 172.21.15.82:0/1248703901 learned_addr learned my addr 172.21.15.82:0/1248703901 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbd9121640 1 -- 172.21.15.82:0/1248703901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30afe80 msgr2=0x55add30b8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbd9121640 1 --2- 172.21.15.82:0/1248703901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30afe80 0x55add30b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbd9121640 1 -- 172.21.15.82:0/1248703901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 msgr2=0x55add30bae50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbd9121640 1 --2- 172.21.15.82:0/1248703901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30bae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbd9121640 1 -- 172.21.15.82:0/1248703901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55add2fa31b0 con 0x55add30af4d0 2024-04-21T08:57:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.428+0000 7fbbd9121640 1 --2- 172.21.15.82:0/1248703901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30af4d0 0x55add30af8b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40994e0 tx=0x7fbbc409d8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1a8b2226dee8d05 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.432+0000 7fbbbf7fe640 1 -- 172.21.15.82:0/1248703901 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40a5cf0 con 0x55add30af4d0 2024-04-21T08:57:11.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.432+0000 7fbbbf7fe640 1 -- 172.21.15.82:0/1248703901 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc409b380 con 0x55add30af4d0 2024-04-21T08:57:11.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.432+0000 7fbbbf7fe640 1 -- 172.21.15.82:0/1248703901 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40996f0 con 0x55add30af4d0 2024-04-21T08:57:11.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.435+0000 7fbbddf73740 1 -- 172.21.15.82:0/1248703901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30af4d0 msgr2=0x55add30af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.435+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1248703901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30af4d0 0x55add30af8b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40994e0 tx=0x7fbbc409d8c0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.437+0000 7fbbddf73740 1 -- 172.21.15.82:0/1248703901 shutdown_connections 2024-04-21T08:57:11.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.437+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1248703901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30bae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.437+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1248703901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30afe80 0x55add30b8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.437+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1248703901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30af4d0 0x55add30af8b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.437+0000 7fbbddf73740 1 -- 172.21.15.82:0/1248703901 >> 172.21.15.82:0/1248703901 conn(0x55add30a9ba0 msgr2=0x55add30a9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.443+0000 7fbbddf73740 1 -- 172.21.15.82:0/1248703901 shutdown_connections 2024-04-21T08:57:11.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.444+0000 7fbbddf73740 1 -- 172.21.15.82:0/1248703901 wait complete. 2024-04-21T08:57:11.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.445+0000 7fbbddf73740 1 -- start start 2024-04-21T08:57:11.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.445+0000 7fbbddf73740 1 Processor -- start 2024-04-21T08:57:11.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.445+0000 7fbbddf73740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30b8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.445+0000 7fbbddf73740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.445+0000 7fbbddf73740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.445+0000 7fbbddf73740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55add30bb8b0 con 0x55add30b8a00 2024-04-21T08:57:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.445+0000 7fbbddf73740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55add2fcd500 con 0x55add30afe80 2024-04-21T08:57:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.445+0000 7fbbddf73740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55add30bba30 con 0x55add30af4d0 2024-04-21T08:57:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.445+0000 7fbbddf73740 10 client.0 _ll_register_callbacks cb 0x55add2f9bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:57:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.446+0000 7fbbd9121640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.446+0000 7fbbd9121640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30b8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52462/0 (socket says 172.21.15.82:52462) 2024-04-21T08:57:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.446+0000 7fbbd9121640 1 -- 172.21.15.82:0/4096685709 learned_addr learned my addr 172.21.15.82:0/4096685709 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.446+0000 7fbbd8920640 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.446+0000 7fbbd9922640 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.446+0000 7fbbd9121640 1 -- 172.21.15.82:0/4096685709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 msgr2=0x55add30b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.446+0000 7fbbd9121640 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.446+0000 7fbbd9121640 1 -- 172.21.15.82:0/4096685709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 msgr2=0x55add30b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.446+0000 7fbbd9121640 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.446+0000 7fbbd9121640 1 -- 172.21.15.82:0/4096685709 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add2d263f0 con 0x55add30af4d0 2024-04-21T08:57:11.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.446+0000 7fbbd9121640 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30b8360 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4099a50 tx=0x7fbbc40d1160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6055f315543dae21 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.446+0000 7fbbbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:57:11.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.447+0000 7fbbbcff9640 1 -- 172.21.15.82:0/4096685709 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40ba070 con 0x55add30af4d0 2024-04-21T08:57:11.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.447+0000 7fbbbcff9640 1 -- 172.21.15.82:0/4096685709 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7fbbc4011890 con 0x55add30af4d0 2024-04-21T08:57:11.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.447+0000 7fbbddf73740 1 -- 172.21.15.82:0/4096685709 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc409b380 con 0x55add30af4d0 2024-04-21T08:57:11.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.448+0000 7fbbbcff9640 1 -- 172.21.15.82:0/4096685709 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fbbc40af050 con 0x55add30af4d0 2024-04-21T08:57:11.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.449+0000 7fbbddf73740 10 client.27166 fetch_fsmap learned FSMap version 46 2024-04-21T08:57:11.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.449+0000 7fbbddf73740 1 -- 172.21.15.82:0/4096685709 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55add2df07d0 con 0x55add30af4d0 2024-04-21T08:57:11.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.452+0000 7fbbbcff9640 1 -- 172.21.15.82:0/4096685709 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc409b380 con 0x55add30af4d0 2024-04-21T08:57:11.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.452+0000 7fbbbcff9640 1 -- 172.21.15.82:0/4096685709 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fbbc4098dd0 con 0x55add30af4d0 2024-04-21T08:57:11.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.452+0000 7fbbddf73740 10 client.27166 fetch_fsmap finished waiting for FSMap version 46 2024-04-21T08:57:11.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.452+0000 7fbbddf73740 10 client.27166 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:57:11.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.452+0000 7fbbddf73740 20 client.27166 populate_metadata read hostname 'smithi082' 2024-04-21T08:57:11.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.452+0000 7fbbddf73740 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55add30c8b00 0x55add30caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.452+0000 7fbbddf73740 4 client.27166 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:57:11.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.452+0000 7fbbddf73740 1 -- 172.21.15.82:0/4096685709 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55add2fb3930 con 0x55add30c8b00 2024-04-21T08:57:11.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.454+0000 7fbbd8920640 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55add30c8b00 0x55add30caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.454+0000 7fbbd8920640 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55add30c8b00 0x55add30caf20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7950a796d7147b55 server_cookie=751b0687ae293bae in_seq=0 out_seq=0 2024-04-21T08:57:11.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.455+0000 7fbbbcff9640 10 client.27166 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:57:11.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.455+0000 7fbbbcff9640 1 -- 172.21.15.82:0/4096685709 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55add2fb3930 con 0x55add30c8b00 2024-04-21T08:57:11.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.455+0000 7fbbbcff9640 10 client.27166 handle_command_reply: tid=0 2024-04-21T08:57:11.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.455+0000 7fbbddf73740 1 client.27166 shutdown 2024-04-21T08:57:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.460+0000 7fbbddf73740 1 -- 172.21.15.82:0/4096685709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 msgr2=0x55add30b8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.460+0000 7fbbddf73740 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30b8360 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4099a50 tx=0x7fbbc40d1160 comp rx=0 tx=0).stop 2024-04-21T08:57:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.460+0000 7fbbddf73740 1 -- 172.21.15.82:0/4096685709 shutdown_connections 2024-04-21T08:57:11.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.460+0000 7fbbddf73740 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55add30c8b00 0x55add30caf20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.460+0000 7fbbddf73740 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.460+0000 7fbbddf73740 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.460+0000 7fbbddf73740 1 --2- 172.21.15.82:0/4096685709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30b8360 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.460+0000 7fbbddf73740 1 -- 172.21.15.82:0/4096685709 >> 172.21.15.82:0/4096685709 conn(0x55add30a9ba0 msgr2=0x55add30ab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.463+0000 7fbbbcff9640 0 client.27166 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:57:11.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.463+0000 7fbbddf73740 1 -- 172.21.15.82:0/4096685709 shutdown_connections 2024-04-21T08:57:11.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.466+0000 7fbbddf73740 1 -- 172.21.15.82:0/4096685709 wait complete. 2024-04-21T08:57:11.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.466+0000 7fbbddf73740 20 client.27166 trim_cache size 0 max 16384 2024-04-21T08:57:11.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.498+0000 7fbbddf73740 1 Processor -- start 2024-04-21T08:57:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.499+0000 7fbbddf73740 1 -- start start 2024-04-21T08:57:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.499+0000 7fbbddf73740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30b5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.499+0000 7fbbddf73740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.499+0000 7fbbddf73740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.499+0000 7fbbddf73740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb8010300 con 0x55add30b8a00 2024-04-21T08:57:11.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.499+0000 7fbbddf73740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb8011e70 con 0x55add30afe80 2024-04-21T08:57:11.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.499+0000 7fbbddf73740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb80109d0 con 0x55add30af4d0 2024-04-21T08:57:11.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.499+0000 7fbbd9922640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.499+0000 7fbbd9922640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33542/0 (socket says 172.21.15.82:33542) 2024-04-21T08:57:11.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.499+0000 7fbbd9922640 1 -- 172.21.15.82:0/1747873806 learned_addr learned my addr 172.21.15.82:0/1747873806 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.499+0000 7fbbd8920640 1 --2- 172.21.15.82:0/1747873806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.500+0000 7fbbd8920640 1 -- 172.21.15.82:0/1747873806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 msgr2=0x55add30b5700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:57:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.500+0000 7fbbd8920640 1 --2- 172.21.15.82:0/1747873806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30b5700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.500+0000 7fbbd8920640 1 -- 172.21.15.82:0/1747873806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 msgr2=0x55add30b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.500+0000 7fbbd8920640 1 --2- 172.21.15.82:0/1747873806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.500+0000 7fbbd8920640 1 -- 172.21.15.82:0/1747873806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55add30aa4e0 con 0x55add30afe80 2024-04-21T08:57:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.500+0000 7fbbd8920640 1 --2- 172.21.15.82:0/1747873806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30c47d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009730 tx=0x7fbbd0013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1b3e661ccee9f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.504+0000 7fbbbeffd640 1 -- 172.21.15.82:0/1747873806 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd002f070 con 0x55add30afe80 2024-04-21T08:57:11.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.504+0000 7fbbbeffd640 1 -- 172.21.15.82:0/1747873806 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd001fc90 con 0x55add30afe80 2024-04-21T08:57:11.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.505+0000 7fbbddf73740 1 -- 172.21.15.82:0/1747873806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 msgr2=0x55add30c47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.505+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1747873806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30c47d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009730 tx=0x7fbbd0013040 comp rx=0 tx=0).stop 2024-04-21T08:57:11.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.506+0000 7fbbddf73740 1 -- 172.21.15.82:0/1747873806 shutdown_connections 2024-04-21T08:57:11.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.506+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1747873806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.506+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1747873806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30c47d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.506+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1747873806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.506+0000 7fbbddf73740 1 -- 172.21.15.82:0/1747873806 >> 172.21.15.82:0/1747873806 conn(0x55add30a9ba0 msgr2=0x55add30b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.509+0000 7fbbddf73740 1 -- 172.21.15.82:0/1747873806 shutdown_connections 2024-04-21T08:57:11.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.509+0000 7fbbddf73740 1 -- 172.21.15.82:0/1747873806 wait complete. 2024-04-21T08:57:11.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.510+0000 7fbbddf73740 1 -- start start 2024-04-21T08:57:11.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.510+0000 7fbbddf73740 1 Processor -- start 2024-04-21T08:57:11.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.511+0000 7fbbddf73740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30c0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.511+0000 7fbbddf73740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.511+0000 7fbbddf73740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.511+0000 7fbbddf73740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb80117a0 con 0x55add30b8a00 2024-04-21T08:57:11.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.511+0000 7fbbddf73740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb80110a0 con 0x55add30afe80 2024-04-21T08:57:11.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.511+0000 7fbbddf73740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb800e2d0 con 0x55add30af4d0 2024-04-21T08:57:11.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.511+0000 7fbbddf73740 10 client.0 _ll_register_callbacks cb 0x7fbbc40a5dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:57:11.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.511+0000 7fbbd9121640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30c0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.511+0000 7fbbd9121640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30c0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52478/0 (socket says 172.21.15.82:52478) 2024-04-21T08:57:11.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.511+0000 7fbbd9121640 1 -- 172.21.15.82:0/1777425738 learned_addr learned my addr 172.21.15.82:0/1777425738 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.511+0000 7fbbd9922640 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.511+0000 7fbbd8920640 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.512+0000 7fbbd9922640 1 -- 172.21.15.82:0/1777425738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 msgr2=0x55add30c0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.512+0000 7fbbd9922640 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30c0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.512+0000 7fbbd9922640 1 -- 172.21.15.82:0/1777425738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 msgr2=0x55add30b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.512+0000 7fbbd9922640 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.512+0000 7fbbd9922640 1 -- 172.21.15.82:0/1777425738 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add30b2a90 con 0x55add30b8a00 2024-04-21T08:57:11.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.512+0000 7fbbd9922640 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b24a0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009ca0 tx=0x7fbbcc032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0cfcac970dfed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.512+0000 7fbbd9121640 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30c0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:57:11.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.513+0000 7fbbbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:57:11.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.513+0000 7fbbbcff9640 1 -- 172.21.15.82:0/1777425738 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc045070 con 0x55add30b8a00 2024-04-21T08:57:11.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.513+0000 7fbbbcff9640 1 -- 172.21.15.82:0/1777425738 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7fbbcc032650 con 0x55add30b8a00 2024-04-21T08:57:11.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.515+0000 7fbbddf73740 1 -- 172.21.15.82:0/1777425738 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd001fc90 con 0x55add30b8a00 2024-04-21T08:57:11.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.516+0000 7fbbbcff9640 1 -- 172.21.15.82:0/1777425738 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fbbcc04e050 con 0x55add30b8a00 2024-04-21T08:57:11.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.518+0000 7fbbddf73740 10 client.18528 fetch_fsmap learned FSMap version 46 2024-04-21T08:57:11.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.518+0000 7fbbddf73740 1 -- 172.21.15.82:0/1777425738 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55add312ee60 con 0x55add30b8a00 2024-04-21T08:57:11.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.519+0000 7fbbbcff9640 1 -- 172.21.15.82:0/1777425738 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd001fc90 con 0x55add30b8a00 2024-04-21T08:57:11.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.519+0000 7fbbbcff9640 1 -- 172.21.15.82:0/1777425738 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7fbbcc0493f0 con 0x55add30b8a00 2024-04-21T08:57:11.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.519+0000 7fbbddf73740 10 client.18528 fetch_fsmap finished waiting for FSMap version 46 2024-04-21T08:57:11.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.519+0000 7fbbddf73740 10 client.18528 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:57:11.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.519+0000 7fbbddf73740 20 client.18528 populate_metadata read hostname 'smithi082' 2024-04-21T08:57:11.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.519+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55add312f5b0 0x55add31319d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.519+0000 7fbbddf73740 4 client.18528 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:57:11.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.519+0000 7fbbddf73740 1 -- 172.21.15.82:0/1777425738 --> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55add3021c10 con 0x55add312f5b0 2024-04-21T08:57:11.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.520+0000 7fbbd9121640 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55add312f5b0 0x55add31319d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.536+0000 7fbbd9121640 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55add312f5b0 0x55add31319d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4dcce336d580294d server_cookie=61cb69125b9241c2 in_seq=0 out_seq=0 2024-04-21T08:57:11.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.537+0000 7fbbbcff9640 10 client.18528 ms_handle_connect on v2:172.21.15.134:6838/394037083 2024-04-21T08:57:11.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.540+0000 7fbbbcff9640 1 -- 172.21.15.82:0/1777425738 <== mds.1 v2:172.21.15.134:6838/394037083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55add3021c10 con 0x55add312f5b0 2024-04-21T08:57:11.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.540+0000 7fbbbcff9640 10 client.18528 handle_command_reply: tid=0 2024-04-21T08:57:11.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.540+0000 7fbbddf73740 1 client.18528 shutdown 2024-04-21T08:57:11.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.547+0000 7fbbddf73740 1 -- 172.21.15.82:0/1777425738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 msgr2=0x55add30b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.547+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b24a0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009ca0 tx=0x7fbbcc032be0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.549+0000 7fbbddf73740 1 -- 172.21.15.82:0/1777425738 shutdown_connections 2024-04-21T08:57:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.549+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.134:6838/394037083,v1:172.21.15.134:6839/394037083] conn(0x55add312f5b0 0x55add31319d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.549+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55add30b8a00 0x55add30b24a0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.549+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55add30afe80 0x55add30b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.549+0000 7fbbddf73740 1 --2- 172.21.15.82:0/1777425738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55add30af4d0 0x55add30c0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.549+0000 7fbbddf73740 1 -- 172.21.15.82:0/1777425738 >> 172.21.15.82:0/1777425738 conn(0x55add30a9ba0 msgr2=0x55add30a8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.549+0000 7fbbbcff9640 0 client.18528 ms_handle_reset on v2:172.21.15.134:6838/394037083 2024-04-21T08:57:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.551+0000 7fbbddf73740 1 -- 172.21.15.82:0/1777425738 shutdown_connections 2024-04-21T08:57:11.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.553+0000 7fbbddf73740 1 -- 172.21.15.82:0/1777425738 wait complete. 2024-04-21T08:57:11.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.554+0000 7f13c64ab640 1 -- 172.21.15.82:0/3990542873 --> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f13a4000bc0 con 0x7f13ac0a3de0 2024-04-21T08:57:11.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.556+0000 7fbbddf73740 20 client.18528 trim_cache size 0 max 16384 2024-04-21T08:57:11.562 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:57:11.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.565+0000 7fbbdb385640 1 -- 172.21.15.82:0/3111773303 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbbc00a3eb0 msgr2=0x7fbbc00a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.565+0000 7fbbdb385640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbbc00a3eb0 0x7fbbc00a6370 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4113590 tx=0x7fbbd0005f90 comp rx=0 tx=0).stop 2024-04-21T08:57:11.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.565+0000 7fbbdb385640 1 -- 172.21.15.82:0/3111773303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd410e7d0 msgr2=0x7fbbd4118d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.565+0000 7fbbdb385640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd410e7d0 0x7fbbd4118d50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400b450 tx=0x7fbbc400b910 comp rx=0 tx=0).stop 2024-04-21T08:57:11.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.571+0000 7fbbdb385640 1 -- 172.21.15.82:0/3111773303 shutdown_connections 2024-04-21T08:57:11.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.571+0000 7fbbdb385640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbd411b730 0x7fbbd4112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.571+0000 7fbbdb385640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fbbc00a3eb0 0x7fbbc00a6370 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.571+0000 7fbbdb385640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbd41107a0 0x7fbbd4111e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.571+0000 7fbbdb385640 1 --2- 172.21.15.82:0/3111773303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd410e7d0 0x7fbbd4118d50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.571+0000 7fbbdb385640 1 -- 172.21.15.82:0/3111773303 >> 172.21.15.82:0/3111773303 conn(0x7fbbd406d060 msgr2=0x7fbbd4072a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.572+0000 7fbbdb385640 1 -- 172.21.15.82:0/3111773303 shutdown_connections 2024-04-21T08:57:11.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.572+0000 7fbbdb385640 1 -- 172.21.15.82:0/3111773303 wait complete. 2024-04-21T08:57:11.592 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:57:11.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.599+0000 7f13b67fc640 1 -- 172.21.15.82:0/3990542873 <== mgr.15546 v2:172.21.15.82:6800/3178848282 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f13a4000bc0 con 0x7f13ac0a3de0 2024-04-21T08:57:11.602 INFO:teuthology.orchestra.run.smithi082.stdout:/ 1m 6m3h 2024-04-21T08:57:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.604+0000 7f13c64ab640 1 -- 172.21.15.82:0/3990542873 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f13ac0a3de0 msgr2=0x7f13ac0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.604+0000 7f13c64ab640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f13ac0a3de0 0x7f13ac0a62a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f13b8048040 tx=0x7f13b803f040 comp rx=0 tx=0).stop 2024-04-21T08:57:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.604+0000 7f13c64ab640 1 -- 172.21.15.82:0/3990542873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c0110610 msgr2=0x7f13c0119360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.604+0000 7f13c64ab640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c0110610 0x7f13c0119360 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00b450 tx=0x7f13bc00b910 comp rx=0 tx=0).stop 2024-04-21T08:57:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.605+0000 7f13c64ab640 1 -- 172.21.15.82:0/3990542873 shutdown_connections 2024-04-21T08:57:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.605+0000 7f13c64ab640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f13c01109f0 0x7f13c01198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.605+0000 7f13c64ab640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f13ac0a3de0 0x7f13ac0a62a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.605+0000 7f13c64ab640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13c0110610 0x7f13c0119360 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.605+0000 7f13c64ab640 1 --2- 172.21.15.82:0/3990542873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13c010ec20 0x7f13c011d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.605+0000 7f13c64ab640 1 -- 172.21.15.82:0/3990542873 >> 172.21.15.82:0/3990542873 conn(0x7f13c006ce90 msgr2=0x7f13c010d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.608+0000 7f13c64ab640 1 -- 172.21.15.82:0/3990542873 shutdown_connections 2024-04-21T08:57:11.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.609+0000 7f13c64ab640 1 -- 172.21.15.82:0/3990542873 wait complete. 2024-04-21T08:57:11.637 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-21T08:57:11.648 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-21T08:57:11.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.781+0000 7ff123cb2640 1 -- 172.21.15.82:0/2035578964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff11c11b730 msgr2=0x7ff11c11db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.781+0000 7ff123cb2640 1 --2- 172.21.15.82:0/2035578964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff11c11b730 0x7ff11c11db20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff108009860 tx=0x7ff10802bb90 comp rx=0 tx=0).stop 2024-04-21T08:57:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.784+0000 7ff123cb2640 1 -- 172.21.15.82:0/2035578964 shutdown_connections 2024-04-21T08:57:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.784+0000 7ff123cb2640 1 --2- 172.21.15.82:0/2035578964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff11c11b730 0x7ff11c11db20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.784+0000 7ff123cb2640 1 --2- 172.21.15.82:0/2035578964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff11c10e7d0 0x7ff11c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.784+0000 7ff123cb2640 1 --2- 172.21.15.82:0/2035578964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff11c1107a0 0x7ff11c10e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.784+0000 7ff123cb2640 1 -- 172.21.15.82:0/2035578964 >> 172.21.15.82:0/2035578964 conn(0x7ff11c06d060 msgr2=0x7ff11c06d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.786+0000 7ff123cb2640 1 -- 172.21.15.82:0/2035578964 shutdown_connections 2024-04-21T08:57:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.786+0000 7ff123cb2640 1 -- 172.21.15.82:0/2035578964 wait complete. 2024-04-21T08:57:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.787+0000 7ff123cb2640 1 Processor -- start 2024-04-21T08:57:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.788+0000 7ff123cb2640 1 -- start start 2024-04-21T08:57:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.788+0000 7ff123cb2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff11c10e7d0 0x7ff11c118db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.788+0000 7ff123cb2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff11c1107a0 0x7ff11c111e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.788+0000 7ff123cb2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff11c11b730 0x7ff11c112370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.788+0000 7ff123cb2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c120f40 con 0x7ff11c11b730 2024-04-21T08:57:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.788+0000 7ff123cb2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c120dc0 con 0x7ff11c10e7d0 2024-04-21T08:57:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.788+0000 7ff123cb2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c1210c0 con 0x7ff11c1107a0 2024-04-21T08:57:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.789+0000 7ff121a4e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff11c10e7d0 0x7ff11c118db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.789+0000 7ff121a4e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff11c10e7d0 0x7ff11c118db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:59804/0 (socket says 172.21.15.82:59804) 2024-04-21T08:57:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.789+0000 7ff121a4e640 1 -- 172.21.15.82:0/3559270948 learned_addr learned my addr 172.21.15.82:0/3559270948 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.790+0000 7ff121a4e640 1 -- 172.21.15.82:0/3559270948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff11c1107a0 msgr2=0x7ff11c111e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:57:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.790+0000 7ff121a4e640 1 --2- 172.21.15.82:0/3559270948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff11c1107a0 0x7ff11c111e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.790+0000 7ff121a4e640 1 -- 172.21.15.82:0/3559270948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff11c11b730 msgr2=0x7ff11c112370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:57:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.790+0000 7ff121a4e640 1 --2- 172.21.15.82:0/3559270948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff11c11b730 0x7ff11c112370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.790+0000 7ff121a4e640 1 -- 172.21.15.82:0/3559270948 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11c1129e0 con 0x7ff11c10e7d0 2024-04-21T08:57:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.792+0000 7ff121a4e640 1 --2- 172.21.15.82:0/3559270948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff11c10e7d0 0x7ff11c118db0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff11000ed80 tx=0x7ff11000c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.792+0000 7ff11affd640 1 -- 172.21.15.82:0/3559270948 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11001c070 con 0x7ff11c10e7d0 2024-04-21T08:57:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.792+0000 7ff123cb2640 1 -- 172.21.15.82:0/3559270948 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11c1168d0 con 0x7ff11c10e7d0 2024-04-21T08:57:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.792+0000 7ff123cb2640 1 -- 172.21.15.82:0/3559270948 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff11c116e70 con 0x7ff11c10e7d0 2024-04-21T08:57:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.793+0000 7ff11affd640 1 -- 172.21.15.82:0/3559270948 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff110017af0 con 0x7ff11c10e7d0 2024-04-21T08:57:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.793+0000 7ff11affd640 1 -- 172.21.15.82:0/3559270948 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11000f9d0 con 0x7ff11c10e7d0 2024-04-21T08:57:11.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.798+0000 7ff11affd640 1 -- 172.21.15.82:0/3559270948 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff11001f070 con 0x7ff11c10e7d0 2024-04-21T08:57:11.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.799+0000 7ff11affd640 1 --2- 172.21.15.82:0/3559270948 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff10c0a3e60 0x7ff10c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.799+0000 7ff11affd640 1 -- 172.21.15.82:0/3559270948 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7ff1100d1b10 con 0x7ff11c10e7d0 2024-04-21T08:57:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.800+0000 7ff126977740 1 Processor -- start 2024-04-21T08:57:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.800+0000 7ff126977740 1 -- start start 2024-04-21T08:57:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.800+0000 7ff126977740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb44d0 0x564f8beb48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.800+0000 7ff126977740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb4e80 0x564f8bebd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.800+0000 7ff126977740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8bebda00 0x564f8bebfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.800+0000 7ff126977740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564f8be191f0 con 0x564f8beb44d0 2024-04-21T08:57:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.800+0000 7ff126977740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564f8bdb64b0 con 0x564f8bebda00 2024-04-21T08:57:11.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.800+0000 7ff126977740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564f8be151b0 con 0x564f8beb4e80 2024-04-21T08:57:11.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.801+0000 7ff121a4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb44d0 0x564f8beb48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.801+0000 7ff121a4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb44d0 0x564f8beb48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33566/0 (socket says 172.21.15.82:33566) 2024-04-21T08:57:11.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.801+0000 7ff121a4e640 1 -- 172.21.15.82:0/2262888625 learned_addr learned my addr 172.21.15.82:0/2262888625 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.801+0000 7ff121a4e640 1 -- 172.21.15.82:0/2262888625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb4e80 msgr2=0x564f8bebd430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:57:11.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.801+0000 7ff121a4e640 1 --2- 172.21.15.82:0/2262888625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb4e80 0x564f8bebd430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.801+0000 7ff121a4e640 1 -- 172.21.15.82:0/2262888625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8bebda00 msgr2=0x564f8bebfe50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:57:11.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.801+0000 7ff121a4e640 1 --2- 172.21.15.82:0/2262888625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8bebda00 0x564f8bebfe50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.801+0000 7ff121a4e640 1 -- 172.21.15.82:0/2262888625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f8bda81b0 con 0x564f8beb44d0 2024-04-21T08:57:11.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.802+0000 7ff121a4e640 1 --2- 172.21.15.82:0/2262888625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb44d0 0x564f8beb48b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7ff1100a5e30 tx=0x7ff110000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2946aca918f4267 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.802+0000 7ff107fff640 1 -- 172.21.15.82:0/2262888625 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1100880b0 con 0x564f8beb44d0 2024-04-21T08:57:11.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.803+0000 7ff107fff640 1 -- 172.21.15.82:0/2262888625 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff11009d6f0 con 0x564f8beb44d0 2024-04-21T08:57:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.803+0000 7ff107fff640 1 -- 172.21.15.82:0/2262888625 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1100cf010 con 0x564f8beb44d0 2024-04-21T08:57:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.804+0000 7ff12124d640 1 --2- 172.21.15.82:0/3559270948 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff10c0a3e60 0x7ff10c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.804+0000 7ff126977740 1 -- 172.21.15.82:0/2262888625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb44d0 msgr2=0x564f8beb48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.804+0000 7ff126977740 1 --2- 172.21.15.82:0/2262888625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb44d0 0x564f8beb48b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7ff1100a5e30 tx=0x7ff110000f40 comp rx=0 tx=0).stop 2024-04-21T08:57:11.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.805+0000 7ff12124d640 1 --2- 172.21.15.82:0/3559270948 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff10c0a3e60 0x7ff10c0a6320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x564f8beafc10 tx=0x7ff114005b80 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.810+0000 7ff126977740 1 -- 172.21.15.82:0/2262888625 shutdown_connections 2024-04-21T08:57:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.810+0000 7ff126977740 1 --2- 172.21.15.82:0/2262888625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8bebda00 0x564f8bebfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.810+0000 7ff126977740 1 --2- 172.21.15.82:0/2262888625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb4e80 0x564f8bebd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.810+0000 7ff126977740 1 --2- 172.21.15.82:0/2262888625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb44d0 0x564f8beb48b0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.810+0000 7ff126977740 1 -- 172.21.15.82:0/2262888625 >> 172.21.15.82:0/2262888625 conn(0x564f8beaeba0 msgr2=0x564f8beaef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.816+0000 7ff126977740 1 -- 172.21.15.82:0/2262888625 shutdown_connections 2024-04-21T08:57:11.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.820+0000 7ff126977740 1 -- 172.21.15.82:0/2262888625 wait complete. 2024-04-21T08:57:11.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.820+0000 7ff126977740 1 -- start start 2024-04-21T08:57:11.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.820+0000 7ff126977740 1 Processor -- start 2024-04-21T08:57:11.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.823+0000 7ff126977740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8beb44d0 0x564f8bebd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.823+0000 7ff126977740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb4e80 0x564f8beba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.823+0000 7ff126977740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8bebda00 0x564f8bebab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.823+0000 7ff126977740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564f8bec09c0 con 0x564f8bebda00 2024-04-21T08:57:11.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.823+0000 7ff126977740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564f8bdd2500 con 0x564f8beb44d0 2024-04-21T08:57:11.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.823+0000 7ff126977740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564f8bec0b40 con 0x564f8beb4e80 2024-04-21T08:57:11.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.823+0000 7ff126977740 10 client.0 _ll_register_callbacks cb 0x564f8bda0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:57:11.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.824+0000 7ff12124d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb4e80 0x564f8beba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.824+0000 7ff12224f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8bebda00 0x564f8bebab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.824+0000 7ff12224f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8bebda00 0x564f8bebab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33582/0 (socket says 172.21.15.82:33582) 2024-04-21T08:57:11.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.824+0000 7ff12224f640 1 -- 172.21.15.82:0/13155950 learned_addr learned my addr 172.21.15.82:0/13155950 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.824+0000 7ff121a4e640 1 --2- 172.21.15.82:0/13155950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8beb44d0 0x564f8bebd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.828+0000 7ff12224f640 1 -- 172.21.15.82:0/13155950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb4e80 msgr2=0x564f8beba5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.828+0000 7ff12224f640 1 --2- 172.21.15.82:0/13155950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb4e80 0x564f8beba5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.828+0000 7ff12224f640 1 -- 172.21.15.82:0/13155950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8beb44d0 msgr2=0x564f8bebd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.828+0000 7ff12224f640 1 --2- 172.21.15.82:0/13155950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8beb44d0 0x564f8bebd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.828+0000 7ff12224f640 1 -- 172.21.15.82:0/13155950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f8bb2b3f0 con 0x564f8bebda00 2024-04-21T08:57:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.829+0000 7ff12224f640 1 --2- 172.21.15.82:0/13155950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8bebda00 0x564f8bebab10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7ff108037040 tx=0x7ff108004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d0f6b8fca8b6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.829+0000 7ff1057fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-21T08:57:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.830+0000 7ff1057fa640 1 -- 172.21.15.82:0/13155950 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108039040 con 0x564f8bebda00 2024-04-21T08:57:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.830+0000 7ff126977740 1 -- 172.21.15.82:0/13155950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff11009d6f0 con 0x564f8bebda00 2024-04-21T08:57:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.830+0000 7ff1057fa640 1 -- 172.21.15.82:0/13155950 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7ff108034970 con 0x564f8bebda00 2024-04-21T08:57:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.831+0000 7ff1057fa640 1 -- 172.21.15.82:0/13155950 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff10804b050 con 0x564f8bebda00 2024-04-21T08:57:11.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.831+0000 7ff126977740 10 client.18537 fetch_fsmap learned FSMap version 46 2024-04-21T08:57:11.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.831+0000 7ff126977740 1 -- 172.21.15.82:0/13155950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x564f8bbf57d0 con 0x564f8bebda00 2024-04-21T08:57:11.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.835+0000 7ff1057fa640 1 -- 172.21.15.82:0/13155950 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff11009d6f0 con 0x564f8bebda00 2024-04-21T08:57:11.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.835+0000 7ff1057fa640 1 -- 172.21.15.82:0/13155950 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7ff10803c070 con 0x564f8bebda00 2024-04-21T08:57:11.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.835+0000 7ff126977740 10 client.18537 fetch_fsmap finished waiting for FSMap version 46 2024-04-21T08:57:11.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.835+0000 7ff126977740 10 client.18537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:57:11.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.835+0000 7ff126977740 20 client.18537 populate_metadata read hostname 'smithi082' 2024-04-21T08:57:11.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.835+0000 7ff126977740 1 --2- 172.21.15.82:0/13155950 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564f8bece1a0 0x564f8bed0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.835+0000 7ff126977740 4 client.18537 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:57:11.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.835+0000 7ff126977740 1 -- 172.21.15.82:0/13155950 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f8bdb8930 con 0x564f8bece1a0 2024-04-21T08:57:11.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.836+0000 7ff121a4e640 1 --2- 172.21.15.82:0/13155950 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564f8bece1a0 0x564f8bed0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.847+0000 7ff121a4e640 1 --2- 172.21.15.82:0/13155950 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564f8bece1a0 0x564f8bed0620 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d851db586e772d server_cookie=89a349b6d7ee0afa in_seq=0 out_seq=0 2024-04-21T08:57:11.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.847+0000 7ff1057fa640 10 client.18537 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:57:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.849+0000 7ff1057fa640 1 -- 172.21.15.82:0/13155950 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f8bdb8930 con 0x564f8bece1a0 2024-04-21T08:57:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.849+0000 7ff1057fa640 10 client.18537 handle_command_reply: tid=0 2024-04-21T08:57:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.850+0000 7ff126977740 1 client.18537 shutdown 2024-04-21T08:57:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.856+0000 7ff126977740 1 -- 172.21.15.82:0/13155950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8bebda00 msgr2=0x564f8bebab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.856+0000 7ff126977740 1 --2- 172.21.15.82:0/13155950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8bebda00 0x564f8bebab10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7ff108037040 tx=0x7ff108004270 comp rx=0 tx=0).stop 2024-04-21T08:57:11.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.859+0000 7ff126977740 1 -- 172.21.15.82:0/13155950 shutdown_connections 2024-04-21T08:57:11.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.859+0000 7ff126977740 1 --2- 172.21.15.82:0/13155950 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564f8bece1a0 0x564f8bed0620 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.859+0000 7ff126977740 1 --2- 172.21.15.82:0/13155950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8bebda00 0x564f8bebab10 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.859+0000 7ff126977740 1 --2- 172.21.15.82:0/13155950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb4e80 0x564f8beba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.859+0000 7ff126977740 1 --2- 172.21.15.82:0/13155950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8beb44d0 0x564f8bebd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.859+0000 7ff126977740 1 -- 172.21.15.82:0/13155950 >> 172.21.15.82:0/13155950 conn(0x564f8beaeba0 msgr2=0x564f8beb08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.859+0000 7ff1057fa640 0 client.18537 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:57:11.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.862+0000 7ff126977740 1 -- 172.21.15.82:0/13155950 shutdown_connections 2024-04-21T08:57:11.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.863+0000 7ff126977740 1 -- 172.21.15.82:0/13155950 wait complete. 2024-04-21T08:57:11.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.864+0000 7ff126977740 20 client.18537 trim_cache size 0 max 16384 2024-04-21T08:57:11.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.894+0000 7ff126977740 1 Processor -- start 2024-04-21T08:57:11.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.894+0000 7ff126977740 1 -- start start 2024-04-21T08:57:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.895+0000 7ff126977740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb44d0 0x564f8bec9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.895+0000 7ff126977740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8beb4e80 0x564f8bebcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.895+0000 7ff126977740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8bebda00 0x564f8bebd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.895+0000 7ff126977740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff1000102b0 con 0x564f8beb44d0 2024-04-21T08:57:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.895+0000 7ff126977740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff100011e20 con 0x564f8beb4e80 2024-04-21T08:57:11.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.895+0000 7ff126977740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff100010980 con 0x564f8bebda00 2024-04-21T08:57:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.896+0000 7ff12224f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8bebda00 0x564f8bebd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.896+0000 7ff12224f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8bebda00 0x564f8bebd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52506/0 (socket says 172.21.15.82:52506) 2024-04-21T08:57:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.896+0000 7ff12224f640 1 -- 172.21.15.82:0/2181118738 learned_addr learned my addr 172.21.15.82:0/2181118738 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.897+0000 7ff121a4e640 1 --2- 172.21.15.82:0/2181118738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb44d0 0x564f8bec9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.899+0000 7ff12224f640 1 -- 172.21.15.82:0/2181118738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8beb4e80 msgr2=0x564f8bebcd00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:57:11.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.899+0000 7ff12224f640 1 --2- 172.21.15.82:0/2181118738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8beb4e80 0x564f8bebcd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.899+0000 7ff12224f640 1 -- 172.21.15.82:0/2181118738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb44d0 msgr2=0x564f8bec9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.899+0000 7ff12224f640 1 --2- 172.21.15.82:0/2181118738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb44d0 0x564f8bec9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.899+0000 7ff12224f640 1 -- 172.21.15.82:0/2181118738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f8beae380 con 0x564f8bebda00 2024-04-21T08:57:11.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.904+0000 7ff12224f640 1 --2- 172.21.15.82:0/2181118738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8bebda00 0x564f8bebd240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff10802bbd0 tx=0x7ff108009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e97fbd08dcac0140 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.908+0000 7ff1077fe640 1 -- 172.21.15.82:0/2181118738 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108054070 con 0x564f8bebda00 2024-04-21T08:57:11.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.908+0000 7ff1077fe640 1 -- 172.21.15.82:0/2181118738 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff108042070 con 0x564f8bebda00 2024-04-21T08:57:11.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.908+0000 7ff1077fe640 1 -- 172.21.15.82:0/2181118738 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108039040 con 0x564f8bebda00 2024-04-21T08:57:11.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.909+0000 7ff126977740 1 -- 172.21.15.82:0/2181118738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8bebda00 msgr2=0x564f8bebd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.909+0000 7ff126977740 1 --2- 172.21.15.82:0/2181118738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8bebda00 0x564f8bebd240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff10802bbd0 tx=0x7ff108009390 comp rx=0 tx=0).stop 2024-04-21T08:57:11.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.910+0000 7ff126977740 1 -- 172.21.15.82:0/2181118738 shutdown_connections 2024-04-21T08:57:11.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.910+0000 7ff126977740 1 --2- 172.21.15.82:0/2181118738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8bebda00 0x564f8bebd240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.910+0000 7ff126977740 1 --2- 172.21.15.82:0/2181118738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8beb4e80 0x564f8bebcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.910+0000 7ff126977740 1 --2- 172.21.15.82:0/2181118738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb44d0 0x564f8bec9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.910+0000 7ff126977740 1 -- 172.21.15.82:0/2181118738 >> 172.21.15.82:0/2181118738 conn(0x564f8beaeba0 msgr2=0x564f8beaef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.915+0000 7ff126977740 1 -- 172.21.15.82:0/2181118738 shutdown_connections 2024-04-21T08:57:11.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.917+0000 7ff126977740 1 -- 172.21.15.82:0/2181118738 wait complete. 2024-04-21T08:57:11.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.917+0000 7ff126977740 1 -- start start 2024-04-21T08:57:11.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.918+0000 7ff126977740 1 Processor -- start 2024-04-21T08:57:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.921+0000 7ff126977740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb44d0 0x564f8bec60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.921+0000 7ff126977740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb4e80 0x564f8beb6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.921+0000 7ff126977740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8bebda00 0x564f8beb73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.921+0000 7ff126977740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff100011750 con 0x564f8beb4e80 2024-04-21T08:57:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.921+0000 7ff126977740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff100011050 con 0x564f8bebda00 2024-04-21T08:57:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.921+0000 7ff126977740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff10000e280 con 0x564f8beb44d0 2024-04-21T08:57:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.921+0000 7ff126977740 10 client.0 _ll_register_callbacks cb 0x7ff108034890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:57:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.923+0000 7ff121a4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb44d0 0x564f8bec60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.923+0000 7ff121a4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb44d0 0x564f8bec60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52522/0 (socket says 172.21.15.82:52522) 2024-04-21T08:57:11.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.923+0000 7ff121a4e640 1 -- 172.21.15.82:0/2707769908 learned_addr learned my addr 172.21.15.82:0/2707769908 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:11.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.925+0000 7ff12224f640 1 --2- 172.21.15.82:0/2707769908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8bebda00 0x564f8beb73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.925+0000 7ff121a4e640 1 -- 172.21.15.82:0/2707769908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8bebda00 msgr2=0x564f8beb73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.925+0000 7ff121a4e640 1 --2- 172.21.15.82:0/2707769908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8bebda00 0x564f8beb73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.925+0000 7ff121a4e640 1 -- 172.21.15.82:0/2707769908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb4e80 msgr2=0x564f8beb6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:57:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.925+0000 7ff121a4e640 1 --2- 172.21.15.82:0/2707769908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb4e80 0x564f8beb6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.925+0000 7ff121a4e640 1 -- 172.21.15.82:0/2707769908 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f8beb7a50 con 0x564f8beb44d0 2024-04-21T08:57:11.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.931+0000 7ff121a4e640 1 --2- 172.21.15.82:0/2707769908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb44d0 0x564f8bec60e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff11009d3f0 tx=0x7ff110000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da3343694e53e94e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.931+0000 7ff1057fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-21T08:57:11.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.933+0000 7ff1057fa640 1 -- 172.21.15.82:0/2707769908 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11009d6f0 con 0x564f8beb44d0 2024-04-21T08:57:11.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.933+0000 7ff1057fa640 1 -- 172.21.15.82:0/2707769908 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7ff110099d60 con 0x564f8beb44d0 2024-04-21T08:57:11.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.934+0000 7ff126977740 1 -- 172.21.15.82:0/2707769908 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff108042070 con 0x564f8beb44d0 2024-04-21T08:57:11.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.939+0000 7ff1057fa640 1 -- 172.21.15.82:0/2707769908 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff11009d950 con 0x564f8beb44d0 2024-04-21T08:57:11.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.939+0000 7ff126977740 10 client.27181 fetch_fsmap learned FSMap version 46 2024-04-21T08:57:11.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.939+0000 7ff126977740 1 -- 172.21.15.82:0/2707769908 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x564f8bf345f0 con 0x564f8beb44d0 2024-04-21T08:57:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.941+0000 7ff1057fa640 1 -- 172.21.15.82:0/2707769908 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff108042070 con 0x564f8beb44d0 2024-04-21T08:57:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.941+0000 7ff1057fa640 1 -- 172.21.15.82:0/2707769908 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 46) ==== 8374+0+0 (secure 0 0 0) 0x7ff11000b030 con 0x564f8beb44d0 2024-04-21T08:57:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.941+0000 7ff126977740 10 client.27181 fetch_fsmap finished waiting for FSMap version 46 2024-04-21T08:57:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.941+0000 7ff126977740 10 client.27181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:57:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.941+0000 7ff126977740 20 client.27181 populate_metadata read hostname 'smithi082' 2024-04-21T08:57:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.941+0000 7ff126977740 1 --2- 172.21.15.82:0/2707769908 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564f8bf34d20 0x564f8bf37160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.941+0000 7ff126977740 4 client.27181 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:57:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.941+0000 7ff126977740 1 -- 172.21.15.82:0/2707769908 --> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f8be26c10 con 0x564f8bf34d20 2024-04-21T08:57:11.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.944+0000 7ff12124d640 1 --2- 172.21.15.82:0/2707769908 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564f8bf34d20 0x564f8bf37160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:11.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.949+0000 7ff12124d640 1 --2- 172.21.15.82:0/2707769908 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564f8bf34d20 0x564f8bf37160 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec057bc707f853e server_cookie=bce86f410c213ac0 in_seq=0 out_seq=0 2024-04-21T08:57:11.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.952+0000 7ff1057fa640 10 client.27181 ms_handle_connect on v2:172.21.15.106:6838/3796875133 2024-04-21T08:57:11.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.953+0000 7ff1057fa640 1 -- 172.21.15.82:0/2707769908 <== mds.0 v2:172.21.15.106:6838/3796875133 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f8be26c10 con 0x564f8bf34d20 2024-04-21T08:57:11.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.953+0000 7ff1057fa640 10 client.27181 handle_command_reply: tid=0 2024-04-21T08:57:11.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.956+0000 7ff126977740 1 client.27181 shutdown 2024-04-21T08:57:11.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.960+0000 7ff126977740 1 -- 172.21.15.82:0/2707769908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb44d0 msgr2=0x564f8bec60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.960+0000 7ff126977740 1 --2- 172.21.15.82:0/2707769908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb44d0 0x564f8bec60e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff11009d3f0 tx=0x7ff110000f40 comp rx=0 tx=0).stop 2024-04-21T08:57:11.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.962+0000 7ff126977740 1 -- 172.21.15.82:0/2707769908 shutdown_connections 2024-04-21T08:57:11.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.962+0000 7ff126977740 1 --2- 172.21.15.82:0/2707769908 >> [v2:172.21.15.106:6838/3796875133,v1:172.21.15.106:6839/3796875133] conn(0x564f8bf34d20 0x564f8bf37160 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.962+0000 7ff126977740 1 --2- 172.21.15.82:0/2707769908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f8bebda00 0x564f8beb73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.962+0000 7ff126977740 1 --2- 172.21.15.82:0/2707769908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f8beb4e80 0x564f8beb6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.962+0000 7ff126977740 1 --2- 172.21.15.82:0/2707769908 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f8beb44d0 0x564f8bec60e0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff11009d3f0 tx=0x7ff110000f40 comp rx=0 tx=0).stop 2024-04-21T08:57:11.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.962+0000 7ff126977740 1 -- 172.21.15.82:0/2707769908 >> 172.21.15.82:0/2707769908 conn(0x564f8beaeba0 msgr2=0x564f8beb52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.962+0000 7ff1057fa640 0 client.27181 ms_handle_reset on v2:172.21.15.106:6838/3796875133 2024-04-21T08:57:11.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.963+0000 7ff126977740 1 -- 172.21.15.82:0/2707769908 shutdown_connections 2024-04-21T08:57:11.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.963+0000 7ff126977740 1 -- 172.21.15.82:0/2707769908 wait complete. 2024-04-21T08:57:11.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.963+0000 7ff126977740 20 client.27181 trim_cache size 0 max 16384 2024-04-21T08:57:11.965 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-21T08:57:11.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.967+0000 7ff123cb2640 1 -- 172.21.15.82:0/3559270948 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff10c0a3e60 msgr2=0x7ff10c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.967+0000 7ff123cb2640 1 --2- 172.21.15.82:0/3559270948 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff10c0a3e60 0x7ff10c0a6320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x564f8beafc10 tx=0x7ff114005b80 comp rx=0 tx=0).stop 2024-04-21T08:57:11.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.967+0000 7ff123cb2640 1 -- 172.21.15.82:0/3559270948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff11c10e7d0 msgr2=0x7ff11c118db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:11.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.967+0000 7ff123cb2640 1 --2- 172.21.15.82:0/3559270948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff11c10e7d0 0x7ff11c118db0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff11000ed80 tx=0x7ff11000c6e0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.970+0000 7ff123cb2640 1 -- 172.21.15.82:0/3559270948 shutdown_connections 2024-04-21T08:57:11.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.970+0000 7ff123cb2640 1 --2- 172.21.15.82:0/3559270948 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7ff10c0a3e60 0x7ff10c0a6320 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.970+0000 7ff123cb2640 1 --2- 172.21.15.82:0/3559270948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff11c11b730 0x7ff11c112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.970+0000 7ff123cb2640 1 --2- 172.21.15.82:0/3559270948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff11c1107a0 0x7ff11c111e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.970+0000 7ff123cb2640 1 --2- 172.21.15.82:0/3559270948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff11c10e7d0 0x7ff11c118db0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:11.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.970+0000 7ff123cb2640 1 -- 172.21.15.82:0/3559270948 >> 172.21.15.82:0/3559270948 conn(0x7ff11c06d060 msgr2=0x7ff11c072a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:11.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.972+0000 7ff123cb2640 1 -- 172.21.15.82:0/3559270948 shutdown_connections 2024-04-21T08:57:11.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:11.973+0000 7ff123cb2640 1 -- 172.21.15.82:0/3559270948 wait complete. 2024-04-21T08:57:11.993 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/thrasher.py", line 43, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-04-21T08:57:11.994 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 152, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/quiescer.py", line 263, in stop_all_quiescers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: Error releasing root: -124 (Unknown) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: 2024-04-21T08:57:11.994 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-21T08:57:12.007 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:57:12.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.183+0000 7f74df8ba640 1 -- 172.21.15.82:0/3715601963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d8110b30 msgr2=0x7f74d8109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:12.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.183+0000 7f74df8ba640 1 --2- 172.21.15.82:0/3715601963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d8110b30 0x7f74d8109490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009900 tx=0x7f74c802bc60 comp rx=0 tx=0).stop 2024-04-21T08:57:12.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.184+0000 7f74df8ba640 1 -- 172.21.15.82:0/3715601963 shutdown_connections 2024-04-21T08:57:12.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.184+0000 7f74df8ba640 1 --2- 172.21.15.82:0/3715601963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d8109b70 0x7f74d811d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.184+0000 7f74df8ba640 1 --2- 172.21.15.82:0/3715601963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d8110b30 0x7f74d8109490 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.184+0000 7f74df8ba640 1 --2- 172.21.15.82:0/3715601963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d81106c0 0x7f74d8108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.184+0000 7f74df8ba640 1 -- 172.21.15.82:0/3715601963 >> 172.21.15.82:0/3715601963 conn(0x7f74d806e410 msgr2=0x7f74d80719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:12.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.191+0000 7f74df8ba640 1 -- 172.21.15.82:0/3715601963 shutdown_connections 2024-04-21T08:57:12.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.191+0000 7f74df8ba640 1 -- 172.21.15.82:0/3715601963 wait complete. 2024-04-21T08:57:12.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.192+0000 7f74df8ba640 1 Processor -- start 2024-04-21T08:57:12.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.192+0000 7f74df8ba640 1 -- start start 2024-04-21T08:57:12.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.192+0000 7f74df8ba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d81106c0 0x7f74d8118d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:12.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.192+0000 7f74df8ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d8110b30 0x7f74d8111de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:12.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.192+0000 7f74df8ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d8112320 0x7f74d81127d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:12.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.192+0000 7f74df8ba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f74d8121060 con 0x7f74d81106c0 2024-04-21T08:57:12.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.192+0000 7f74df8ba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f74d8120ee0 con 0x7f74d8110b30 2024-04-21T08:57:12.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.192+0000 7f74df8ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f74d81211e0 con 0x7f74d8112320 2024-04-21T08:57:12.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.193+0000 7f74dde57640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d8112320 0x7f74d81127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.193+0000 7f74dde57640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d8112320 0x7f74d81127d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:52542/0 (socket says 172.21.15.82:52542) 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.193+0000 7f74dde57640 1 -- 172.21.15.82:0/2387734828 learned_addr learned my addr 172.21.15.82:0/2387734828 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.194+0000 7f74dce55640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d8110b30 0x7f74d8111de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.195+0000 7f74dd656640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d81106c0 0x7f74d8118d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.195+0000 7f74dd656640 1 -- 172.21.15.82:0/2387734828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d8112320 msgr2=0x7f74d81127d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.195+0000 7f74dd656640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d8112320 0x7f74d81127d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.195+0000 7f74dd656640 1 -- 172.21.15.82:0/2387734828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d8110b30 msgr2=0x7f74d8111de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.195+0000 7f74dd656640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d8110b30 0x7f74d8111de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.195+0000 7f74dd656640 1 -- 172.21.15.82:0/2387734828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d8116880 con 0x7f74d81106c0 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.195+0000 7f74dd656640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d81106c0 0x7f74d8118d60 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f74d400c970 tx=0x7f74d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.196+0000 7f74ce7fc640 1 -- 172.21.15.82:0/2387734828 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4007bb0 con 0x7f74d81106c0 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.196+0000 7f74ce7fc640 1 -- 172.21.15.82:0/2387734828 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d4007d50 con 0x7f74d81106c0 2024-04-21T08:57:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.196+0000 7f74ce7fc640 1 -- 172.21.15.82:0/2387734828 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4018790 con 0x7f74d81106c0 2024-04-21T08:57:12.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.197+0000 7f74df8ba640 1 -- 172.21.15.82:0/2387734828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d8116b70 con 0x7f74d81106c0 2024-04-21T08:57:12.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.197+0000 7f74df8ba640 1 -- 172.21.15.82:0/2387734828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d8117040 con 0x7f74d81106c0 2024-04-21T08:57:12.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.200+0000 7f74df8ba640 1 -- 172.21.15.82:0/2387734828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d8073e40 con 0x7f74d81106c0 2024-04-21T08:57:12.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.209+0000 7f74ce7fc640 1 -- 172.21.15.82:0/2387734828 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f74d400fa90 con 0x7f74d81106c0 2024-04-21T08:57:12.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.210+0000 7f74ce7fc640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f74c40a3f20 0x7f74c40a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:12.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.210+0000 7f74ce7fc640 1 -- 172.21.15.82:0/2387734828 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f74d40d1150 con 0x7f74d81106c0 2024-04-21T08:57:12.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.216+0000 7f74ce7fc640 1 -- 172.21.15.82:0/2387734828 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74d4014030 con 0x7f74d81106c0 2024-04-21T08:57:12.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.220+0000 7f74dce55640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f74c40a3f20 0x7f74c40a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:12.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.221+0000 7f74dce55640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f74c40a3f20 0x7f74c40a63e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f74c802c270 tx=0x7f74c8005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:12.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.565+0000 7f74df8ba640 1 -- 172.21.15.82:0/2387734828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74d8109b70 con 0x7f74d81106c0 2024-04-21T08:57:12.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.568+0000 7f74ce7fc640 1 -- 172.21.15.82:0/2387734828 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+25568 (secure 0 0 0) 0x7f74d409d420 con 0x7f74d81106c0 2024-04-21T08:57:12.569 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:57:12.569 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":101,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:57:08.582068+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2119539425":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6800/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/539366680":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/2275507980":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2883183315":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:57:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.572+0000 7f74df8ba640 1 -- 172.21.15.82:0/2387734828 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f74c40a3f20 msgr2=0x7f74c40a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.572+0000 7f74df8ba640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f74c40a3f20 0x7f74c40a63e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f74c802c270 tx=0x7f74c8005fd0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.572+0000 7f74df8ba640 1 -- 172.21.15.82:0/2387734828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d81106c0 msgr2=0x7f74d8118d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.572+0000 7f74df8ba640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d81106c0 0x7f74d8118d60 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f74d400c970 tx=0x7f74d400ce30 comp rx=0 tx=0).stop 2024-04-21T08:57:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.572+0000 7f74df8ba640 1 -- 172.21.15.82:0/2387734828 shutdown_connections 2024-04-21T08:57:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.572+0000 7f74df8ba640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74d8112320 0x7f74d81127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.572+0000 7f74df8ba640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74d8110b30 0x7f74d8111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.572+0000 7f74df8ba640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f74c40a3f20 0x7f74c40a63e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.572+0000 7f74df8ba640 1 --2- 172.21.15.82:0/2387734828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74d81106c0 0x7f74d8118d60 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.572+0000 7f74df8ba640 1 -- 172.21.15.82:0/2387734828 >> 172.21.15.82:0/2387734828 conn(0x7f74d806e410 msgr2=0x7f74d8119a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:12.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.572+0000 7f74df8ba640 1 -- 172.21.15.82:0/2387734828 shutdown_connections 2024-04-21T08:57:12.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.572+0000 7f74df8ba640 1 -- 172.21.15.82:0/2387734828 wait complete. 2024-04-21T08:57:12.590 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:57:12.590 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:57:12.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.776+0000 7f05af8a5640 1 -- 172.21.15.82:0/1429415167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05a810e7d0 msgr2=0x7f05a810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:12.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.776+0000 7f05af8a5640 1 --2- 172.21.15.82:0/1429415167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05a810e7d0 0x7f05a810ec50 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009890 tx=0x7f05a002bbb0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.776+0000 7f05af8a5640 1 -- 172.21.15.82:0/1429415167 shutdown_connections 2024-04-21T08:57:12.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.776+0000 7f05af8a5640 1 --2- 172.21.15.82:0/1429415167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a811b730 0x7f05a811db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.776+0000 7f05af8a5640 1 --2- 172.21.15.82:0/1429415167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05a810e7d0 0x7f05a810ec50 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.776+0000 7f05af8a5640 1 --2- 172.21.15.82:0/1429415167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05a81107a0 0x7f05a810e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.776+0000 7f05af8a5640 1 -- 172.21.15.82:0/1429415167 >> 172.21.15.82:0/1429415167 conn(0x7f05a806d060 msgr2=0x7f05a806d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:12.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.783+0000 7f05af8a5640 1 -- 172.21.15.82:0/1429415167 shutdown_connections 2024-04-21T08:57:12.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.783+0000 7f05af8a5640 1 -- 172.21.15.82:0/1429415167 wait complete. 2024-04-21T08:57:12.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.784+0000 7f05af8a5640 1 Processor -- start 2024-04-21T08:57:12.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.786+0000 7f05af8a5640 1 -- start start 2024-04-21T08:57:12.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.786+0000 7f05af8a5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05a810e7d0 0x7f05a81a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:12.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.786+0000 7f05af8a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a81107a0 0x7f05a81a7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:12.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.786+0000 7f05af8a5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05a811b730 0x7f05a81abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:12.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.786+0000 7f05af8a5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f05a8121030 con 0x7f05a810e7d0 2024-04-21T08:57:12.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.786+0000 7f05af8a5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f05a8120eb0 con 0x7f05a81107a0 2024-04-21T08:57:12.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.786+0000 7f05af8a5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81211b0 con 0x7f05a811b730 2024-04-21T08:57:12.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.787+0000 7f05ad641640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05a810e7d0 0x7f05a81a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:12.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.787+0000 7f05ad641640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05a810e7d0 0x7f05a81a75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59478/0 (socket says 172.21.15.82:59478) 2024-04-21T08:57:12.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.787+0000 7f05ad641640 1 -- 172.21.15.82:0/1669034821 learned_addr learned my addr 172.21.15.82:0/1669034821 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:12.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.787+0000 7f05ade42640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05a811b730 0x7f05a81abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:12.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.790+0000 7f05ace40640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a81107a0 0x7f05a81a7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:12.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.792+0000 7f05ad641640 1 -- 172.21.15.82:0/1669034821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05a811b730 msgr2=0x7f05a81abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:12.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.792+0000 7f05ad641640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05a811b730 0x7f05a81abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.792+0000 7f05ad641640 1 -- 172.21.15.82:0/1669034821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a81107a0 msgr2=0x7f05a81a7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:12.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.792+0000 7f05ad641640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a81107a0 0x7f05a81a7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:12.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.792+0000 7f05ad641640 1 -- 172.21.15.82:0/1669034821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a81ac5c0 con 0x7f05a810e7d0 2024-04-21T08:57:12.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.792+0000 7f05ad641640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05a810e7d0 0x7f05a81a75d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f059800efb0 tx=0x7f059800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:12.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.798+0000 7f05a67fc640 1 -- 172.21.15.82:0/1669034821 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059801b070 con 0x7f05a810e7d0 2024-04-21T08:57:12.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.798+0000 7f05a67fc640 1 -- 172.21.15.82:0/1669034821 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059800ba20 con 0x7f05a810e7d0 2024-04-21T08:57:12.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.798+0000 7f05a67fc640 1 -- 172.21.15.82:0/1669034821 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598016c30 con 0x7f05a810e7d0 2024-04-21T08:57:12.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.799+0000 7f05af8a5640 1 -- 172.21.15.82:0/1669034821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a81ac8b0 con 0x7f05a810e7d0 2024-04-21T08:57:12.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.800+0000 7f05af8a5640 1 -- 172.21.15.82:0/1669034821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a81b4140 con 0x7f05a810e7d0 2024-04-21T08:57:12.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.804+0000 7f05a67fc640 1 -- 172.21.15.82:0/1669034821 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f059801e070 con 0x7f05a810e7d0 2024-04-21T08:57:12.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.804+0000 7f05a67fc640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f05900a3f30 0x7f05900a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:12.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.805+0000 7f05a67fc640 1 -- 172.21.15.82:0/1669034821 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f05980d0ad0 con 0x7f05a810e7d0 2024-04-21T08:57:12.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.805+0000 7f05ace40640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f05900a3f30 0x7f05900a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:12.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.806+0000 7f05ace40640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f05900a3f30 0x7f05900a63f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f05a811db70 tx=0x7f05a0035e70 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:12.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.806+0000 7f05af8a5640 1 -- 172.21.15.82:0/1669034821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a806e300 con 0x7f05a810e7d0 2024-04-21T08:57:12.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:12.818+0000 7f05a67fc640 1 -- 172.21.15.82:0/1669034821 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f059809c650 con 0x7f05a810e7d0 2024-04-21T08:57:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:12 smithi082 ceph-mon[27967]: from='client.27479 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:57:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:12 smithi082 ceph-mon[27967]: pgmap v315: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-21T08:57:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:12 smithi082 ceph-mon[27967]: from='client.18510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:57:12.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:12 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2387734828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:57:12.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:12 smithi106 ceph-mon[31072]: from='client.27479 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:57:12.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:12 smithi106 ceph-mon[31072]: pgmap v315: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-21T08:57:12.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:12 smithi106 ceph-mon[31072]: from='client.18510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:57:12.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:12 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2387734828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:57:12.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:12 smithi134 ceph-mon[31772]: from='client.27479 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:57:12.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:12 smithi134 ceph-mon[31772]: pgmap v315: 129 pgs: 129 active+clean; 883 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-21T08:57:12.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:12 smithi134 ceph-mon[31772]: from='client.18510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:57:12.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:12 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2387734828' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:57:13.171 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:57:13.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.171+0000 7f05af8a5640 1 -- 172.21.15.82:0/1669034821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05a81a8950 con 0x7f05a810e7d0 2024-04-21T08:57:13.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.171+0000 7f05a67fc640 1 -- 172.21.15.82:0/1669034821 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f05a81a8950 con 0x7f05a810e7d0 2024-04-21T08:57:13.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.175+0000 7f05af8a5640 1 -- 172.21.15.82:0/1669034821 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f05900a3f30 msgr2=0x7f05900a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:13.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.175+0000 7f05af8a5640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f05900a3f30 0x7f05900a63f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f05a811db70 tx=0x7f05a0035e70 comp rx=0 tx=0).stop 2024-04-21T08:57:13.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.175+0000 7f05af8a5640 1 -- 172.21.15.82:0/1669034821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05a810e7d0 msgr2=0x7f05a81a75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:13.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.175+0000 7f05af8a5640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05a810e7d0 0x7f05a81a75d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f059800efb0 tx=0x7f059800c800 comp rx=0 tx=0).stop 2024-04-21T08:57:13.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.175+0000 7f05af8a5640 1 -- 172.21.15.82:0/1669034821 shutdown_connections 2024-04-21T08:57:13.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.175+0000 7f05af8a5640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05a811b730 0x7f05a81abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.175+0000 7f05af8a5640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05a81107a0 0x7f05a81a7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.175+0000 7f05af8a5640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f05900a3f30 0x7f05900a63f0 secure :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f05a811db70 tx=0x7f05a0035e70 comp rx=0 tx=0).stop 2024-04-21T08:57:13.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.175+0000 7f05af8a5640 1 --2- 172.21.15.82:0/1669034821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05a810e7d0 0x7f05a81a75d0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.175+0000 7f05af8a5640 1 -- 172.21.15.82:0/1669034821 >> 172.21.15.82:0/1669034821 conn(0x7f05a806d060 msgr2=0x7f05a8072a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:13.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.175+0000 7f05af8a5640 1 -- 172.21.15.82:0/1669034821 shutdown_connections 2024-04-21T08:57:13.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.175+0000 7f05af8a5640 1 -- 172.21.15.82:0/1669034821 wait complete. 2024-04-21T08:57:13.193 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:57:13.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.371+0000 7f93ef53b640 1 -- 172.21.15.82:0/999774465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93e8100e60 msgr2=0x7f93e810d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:13.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.371+0000 7f93ef53b640 1 --2- 172.21.15.82:0/999774465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93e8100e60 0x7f93e810d320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009be0 tx=0x7f93e002bf30 comp rx=0 tx=0).stop 2024-04-21T08:57:13.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.372+0000 7f93ef53b640 1 -- 172.21.15.82:0/999774465 shutdown_connections 2024-04-21T08:57:13.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.372+0000 7f93ef53b640 1 --2- 172.21.15.82:0/999774465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93e8100e60 0x7f93e810d320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.372+0000 7f93ef53b640 1 --2- 172.21.15.82:0/999774465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93e81004a0 0x7f93e8100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.372+0000 7f93ef53b640 1 --2- 172.21.15.82:0/999774465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e81064f0 0x7f93e81068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.372+0000 7f93ef53b640 1 -- 172.21.15.82:0/999774465 >> 172.21.15.82:0/999774465 conn(0x7f93e80772e0 msgr2=0x7f93e80fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:13.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.375+0000 7f93ef53b640 1 -- 172.21.15.82:0/999774465 shutdown_connections 2024-04-21T08:57:13.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.378+0000 7f93ef53b640 1 -- 172.21.15.82:0/999774465 wait complete. 2024-04-21T08:57:13.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.379+0000 7f93ef53b640 1 Processor -- start 2024-04-21T08:57:13.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.380+0000 7f93ef53b640 1 -- start start 2024-04-21T08:57:13.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.380+0000 7f93ef53b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93e81004a0 0x7f93e819d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:13.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.380+0000 7f93ef53b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93e8100e60 0x7f93e819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:13.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.380+0000 7f93ef53b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e81064f0 0x7f93e81963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:13.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.380+0000 7f93ef53b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810ff60 con 0x7f93e81004a0 2024-04-21T08:57:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.380+0000 7f93ef53b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810fde0 con 0x7f93e8100e60 2024-04-21T08:57:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.380+0000 7f93ef53b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81100e0 con 0x7f93e81064f0 2024-04-21T08:57:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.380+0000 7f93edd38640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93e8100e60 0x7f93e819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.380+0000 7f93edd38640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93e8100e60 0x7f93e819d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:47194/0 (socket says 172.21.15.82:47194) 2024-04-21T08:57:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.380+0000 7f93edd38640 1 -- 172.21.15.82:0/461837919 learned_addr learned my addr 172.21.15.82:0/461837919 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.380+0000 7f93eed3a640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e81064f0 0x7f93e81963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.381+0000 7f93edd38640 1 -- 172.21.15.82:0/461837919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e81064f0 msgr2=0x7f93e81963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.381+0000 7f93edd38640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e81064f0 0x7f93e81963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.381+0000 7f93edd38640 1 -- 172.21.15.82:0/461837919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93e81004a0 msgr2=0x7f93e819d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.381+0000 7f93edd38640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93e81004a0 0x7f93e819d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.381+0000 7f93edd38640 1 -- 172.21.15.82:0/461837919 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e8196c90 con 0x7f93e8100e60 2024-04-21T08:57:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.381+0000 7f93edd38640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93e8100e60 0x7f93e819d6b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f93e400d950 tx=0x7f93e400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:13.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.382+0000 7f93df7fe640 1 -- 172.21.15.82:0/461837919 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4017070 con 0x7f93e8100e60 2024-04-21T08:57:13.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.382+0000 7f93df7fe640 1 -- 172.21.15.82:0/461837919 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e4009c30 con 0x7f93e8100e60 2024-04-21T08:57:13.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.383+0000 7f93df7fe640 1 -- 172.21.15.82:0/461837919 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e400f620 con 0x7f93e8100e60 2024-04-21T08:57:13.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.383+0000 7f93ef53b640 1 -- 172.21.15.82:0/461837919 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e8196f80 con 0x7f93e8100e60 2024-04-21T08:57:13.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.383+0000 7f93ef53b640 1 -- 172.21.15.82:0/461837919 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e819aed0 con 0x7f93e8100e60 2024-04-21T08:57:13.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.387+0000 7f93ef53b640 1 -- 172.21.15.82:0/461837919 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e8101b90 con 0x7f93e8100e60 2024-04-21T08:57:13.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.387+0000 7f93df7fe640 1 -- 172.21.15.82:0/461837919 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f93e4010460 con 0x7f93e8100e60 2024-04-21T08:57:13.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.388+0000 7f93df7fe640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f93d40a3e00 0x7f93d40a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:13.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.388+0000 7f93df7fe640 1 -- 172.21.15.82:0/461837919 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f93e40d0600 con 0x7f93e8100e60 2024-04-21T08:57:13.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.391+0000 7f93ee539640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f93d40a3e00 0x7f93d40a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:13.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.395+0000 7f93ee539640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f93d40a3e00 0x7f93d40a62c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f93d80096f0 tx=0x7f93d8006c40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:13.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.398+0000 7f93df7fe640 1 -- 172.21.15.82:0/461837919 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93e4020070 con 0x7f93e8100e60 2024-04-21T08:57:13.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.753+0000 7f93ef53b640 1 -- 172.21.15.82:0/461837919 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f93e8197720 con 0x7f93e8100e60 2024-04-21T08:57:13.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.753+0000 7f93df7fe640 1 -- 172.21.15.82:0/461837919 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f93e409c850 con 0x7f93e8100e60 2024-04-21T08:57:13.753 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:57:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.757+0000 7f93ef53b640 1 -- 172.21.15.82:0/461837919 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f93d40a3e00 msgr2=0x7f93d40a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.757+0000 7f93ef53b640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f93d40a3e00 0x7f93d40a62c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f93d80096f0 tx=0x7f93d8006c40 comp rx=0 tx=0).stop 2024-04-21T08:57:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.757+0000 7f93ef53b640 1 -- 172.21.15.82:0/461837919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93e8100e60 msgr2=0x7f93e819d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.757+0000 7f93ef53b640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93e8100e60 0x7f93e819d6b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f93e400d950 tx=0x7f93e400de10 comp rx=0 tx=0).stop 2024-04-21T08:57:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.757+0000 7f93ef53b640 1 -- 172.21.15.82:0/461837919 shutdown_connections 2024-04-21T08:57:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.757+0000 7f93ef53b640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e81064f0 0x7f93e81963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.757+0000 7f93ef53b640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93e8100e60 0x7f93e819d6b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.757+0000 7f93ef53b640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f93d40a3e00 0x7f93d40a62c0 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f93d80096f0 tx=0x7f93d8006c40 comp rx=0 tx=0).stop 2024-04-21T08:57:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.757+0000 7f93ef53b640 1 --2- 172.21.15.82:0/461837919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93e81004a0 0x7f93e819d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.757+0000 7f93ef53b640 1 -- 172.21.15.82:0/461837919 >> 172.21.15.82:0/461837919 conn(0x7f93e80772e0 msgr2=0x7f93e8103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.757+0000 7f93ef53b640 1 -- 172.21.15.82:0/461837919 shutdown_connections 2024-04-21T08:57:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.757+0000 7f93ef53b640 1 -- 172.21.15.82:0/461837919 wait complete. 2024-04-21T08:57:13.776 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:57:13.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:13 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1669034821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:57:13.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.957+0000 7fba9cd32640 1 -- 172.21.15.82:0/695793464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9811b730 msgr2=0x7fba9811db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:13.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.957+0000 7fba9cd32640 1 --2- 172.21.15.82:0/695793464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9811b730 0x7fba9811db20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fba90009cd0 tx=0x7fba9002c120 comp rx=0 tx=0).stop 2024-04-21T08:57:13.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.961+0000 7fba9cd32640 1 -- 172.21.15.82:0/695793464 shutdown_connections 2024-04-21T08:57:13.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.961+0000 7fba9cd32640 1 --2- 172.21.15.82:0/695793464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9811b730 0x7fba9811db20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.961+0000 7fba9cd32640 1 --2- 172.21.15.82:0/695793464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9810e7d0 0x7fba9810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.961+0000 7fba9cd32640 1 --2- 172.21.15.82:0/695793464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba981107a0 0x7fba9810e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.961+0000 7fba9cd32640 1 -- 172.21.15.82:0/695793464 >> 172.21.15.82:0/695793464 conn(0x7fba9806d060 msgr2=0x7fba9806d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:13.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.965+0000 7fba9cd32640 1 -- 172.21.15.82:0/695793464 shutdown_connections 2024-04-21T08:57:13.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.969+0000 7fba9cd32640 1 -- 172.21.15.82:0/695793464 wait complete. 2024-04-21T08:57:13.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.970+0000 7fba9cd32640 1 Processor -- start 2024-04-21T08:57:13.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.974+0000 7fba9cd32640 1 -- start start 2024-04-21T08:57:13.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.974+0000 7fba9cd32640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9810e7d0 0x7fba981a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:13.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.974+0000 7fba9cd32640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba981107a0 0x7fba981a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:13.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.974+0000 7fba9cd32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9811b730 0x7fba981abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:13.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.974+0000 7fba9cd32640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fba98120fc0 con 0x7fba981107a0 2024-04-21T08:57:13.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.974+0000 7fba9cd32640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fba98120e40 con 0x7fba9810e7d0 2024-04-21T08:57:13.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.974+0000 7fba9cd32640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fba98121140 con 0x7fba9811b730 2024-04-21T08:57:13.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:13 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1669034821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:57:13.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.977+0000 7fba8ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba981107a0 0x7fba981a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:13.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.977+0000 7fba9659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9810e7d0 0x7fba981a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:13.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.977+0000 7fba9659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9810e7d0 0x7fba981a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.82:47210/0 (socket says 172.21.15.82:47210) 2024-04-21T08:57:13.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.977+0000 7fba9659c640 1 -- 172.21.15.82:0/28532159 learned_addr learned my addr 172.21.15.82:0/28532159 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:13.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.979+0000 7fba9659c640 1 -- 172.21.15.82:0/28532159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9811b730 msgr2=0x7fba981abe80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:57:13.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.979+0000 7fba9659c640 1 --2- 172.21.15.82:0/28532159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9811b730 0x7fba981abe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.979+0000 7fba9659c640 1 -- 172.21.15.82:0/28532159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba981107a0 msgr2=0x7fba981a7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:13.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.979+0000 7fba9659c640 1 --2- 172.21.15.82:0/28532159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba981107a0 0x7fba981a7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:13.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.979+0000 7fba9659c640 1 -- 172.21.15.82:0/28532159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba981ac5b0 con 0x7fba9810e7d0 2024-04-21T08:57:13.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.981+0000 7fba9659c640 1 --2- 172.21.15.82:0/28532159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9810e7d0 0x7fba981a75c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba8000b4a0 tx=0x7fba8000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.983+0000 7fba8f7fe640 1 -- 172.21.15.82:0/28532159 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80014070 con 0x7fba9810e7d0 2024-04-21T08:57:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.984+0000 7fba8f7fe640 1 -- 172.21.15.82:0/28532159 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba80010ac0 con 0x7fba9810e7d0 2024-04-21T08:57:13.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.984+0000 7fba8f7fe640 1 -- 172.21.15.82:0/28532159 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80017b60 con 0x7fba9810e7d0 2024-04-21T08:57:13.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.985+0000 7fba9cd32640 1 -- 172.21.15.82:0/28532159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba981ac8a0 con 0x7fba9810e7d0 2024-04-21T08:57:13.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.985+0000 7fba9cd32640 1 -- 172.21.15.82:0/28532159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba981b4130 con 0x7fba9810e7d0 2024-04-21T08:57:13.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.988+0000 7fba9cd32640 1 -- 172.21.15.82:0/28532159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba98068d70 con 0x7fba9810e7d0 2024-04-21T08:57:13.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.993+0000 7fba8f7fe640 1 -- 172.21.15.82:0/28532159 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba80017070 con 0x7fba9810e7d0 2024-04-21T08:57:13.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.994+0000 7fba8f7fe640 1 --2- 172.21.15.82:0/28532159 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fba840a3f20 0x7fba840a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:13.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.995+0000 7fba8f7fe640 1 -- 172.21.15.82:0/28532159 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7fba800d0390 con 0x7fba9810e7d0 2024-04-21T08:57:13.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.996+0000 7fba8ffff640 1 --2- 172.21.15.82:0/28532159 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fba840a3f20 0x7fba840a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:13.997+0000 7fba8ffff640 1 --2- 172.21.15.82:0/28532159 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fba840a3f20 0x7fba840a63e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba88005fd0 tx=0x7fba88005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:13.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.000+0000 7fba8f7fe640 1 -- 172.21.15.82:0/28532159 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba8009c6f0 con 0x7fba9810e7d0 2024-04-21T08:57:14.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:13 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1669034821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:57:14.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.358+0000 7fba9cd32640 1 -- 172.21.15.82:0/28532159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fba981b44c0 con 0x7fba9810e7d0 2024-04-21T08:57:14.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.361+0000 7fba8f7fe640 1 -- 172.21.15.82:0/28532159 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7fba8009bff0 con 0x7fba9810e7d0 2024-04-21T08:57:14.361 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:57:14.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.365+0000 7fba9cd32640 1 -- 172.21.15.82:0/28532159 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fba840a3f20 msgr2=0x7fba840a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:14.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.365+0000 7fba9cd32640 1 --2- 172.21.15.82:0/28532159 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fba840a3f20 0x7fba840a63e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba88005fd0 tx=0x7fba88005850 comp rx=0 tx=0).stop 2024-04-21T08:57:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.365+0000 7fba9cd32640 1 -- 172.21.15.82:0/28532159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9810e7d0 msgr2=0x7fba981a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.365+0000 7fba9cd32640 1 --2- 172.21.15.82:0/28532159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9810e7d0 0x7fba981a75c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba8000b4a0 tx=0x7fba8000b960 comp rx=0 tx=0).stop 2024-04-21T08:57:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.366+0000 7fba9cd32640 1 -- 172.21.15.82:0/28532159 shutdown_connections 2024-04-21T08:57:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.366+0000 7fba9cd32640 1 --2- 172.21.15.82:0/28532159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9811b730 0x7fba981abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.366+0000 7fba9cd32640 1 --2- 172.21.15.82:0/28532159 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fba840a3f20 0x7fba840a63e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.366+0000 7fba9cd32640 1 --2- 172.21.15.82:0/28532159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba981107a0 0x7fba981a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.366+0000 7fba9cd32640 1 --2- 172.21.15.82:0/28532159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba9810e7d0 0x7fba981a75c0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba8000b4a0 tx=0x7fba8000b960 comp rx=0 tx=0).stop 2024-04-21T08:57:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.366+0000 7fba9cd32640 1 -- 172.21.15.82:0/28532159 >> 172.21.15.82:0/28532159 conn(0x7fba9806d060 msgr2=0x7fba9811a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.367+0000 7fba9cd32640 1 -- 172.21.15.82:0/28532159 shutdown_connections 2024-04-21T08:57:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.367+0000 7fba9cd32640 1 -- 172.21.15.82:0/28532159 wait complete. 2024-04-21T08:57:14.384 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:57:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.562+0000 7f45f6081640 1 -- 172.21.15.82:0/589328850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45f01062f0 msgr2=0x7f45f0106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.562+0000 7f45f6081640 1 --2- 172.21.15.82:0/589328850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45f01062f0 0x7f45f0106780 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009ca0 tx=0x7f45e802c250 comp rx=0 tx=0).stop 2024-04-21T08:57:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.562+0000 7f45f6081640 1 -- 172.21.15.82:0/589328850 shutdown_connections 2024-04-21T08:57:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.562+0000 7f45f6081640 1 --2- 172.21.15.82:0/589328850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45f01062f0 0x7f45f0106780 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.562+0000 7f45f6081640 1 --2- 172.21.15.82:0/589328850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45f01018f0 0x7f45f0105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.562+0000 7f45f6081640 1 --2- 172.21.15.82:0/589328850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f0100fd0 0x7f45f01013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.562+0000 7f45f6081640 1 -- 172.21.15.82:0/589328850 >> 172.21.15.82:0/589328850 conn(0x7f45f00771d0 msgr2=0x7f45f00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.565+0000 7f45f6081640 1 -- 172.21.15.82:0/589328850 shutdown_connections 2024-04-21T08:57:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.565+0000 7f45f6081640 1 -- 172.21.15.82:0/589328850 wait complete. 2024-04-21T08:57:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.566+0000 7f45f6081640 1 Processor -- start 2024-04-21T08:57:14.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.568+0000 7f45f6081640 1 -- start start 2024-04-21T08:57:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.568+0000 7f45f6081640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f0100fd0 0x7f45f0073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.568+0000 7f45f6081640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45f01018f0 0x7f45f006ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.568+0000 7f45f6081640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45f01062f0 0x7f45f006f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.568+0000 7f45f6081640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f45f006d500 con 0x7f45f0100fd0 2024-04-21T08:57:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.568+0000 7f45f6081640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f45f006d380 con 0x7f45f01062f0 2024-04-21T08:57:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.568+0000 7f45f6081640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45f006d680 con 0x7f45f01018f0 2024-04-21T08:57:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.568+0000 7f45ef7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f0100fd0 0x7f45f0073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.568+0000 7f45ef7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f0100fd0 0x7f45f0073b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59524/0 (socket says 172.21.15.82:59524) 2024-04-21T08:57:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.568+0000 7f45ef7fe640 1 -- 172.21.15.82:0/534720376 learned_addr learned my addr 172.21.15.82:0/534720376 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.568+0000 7f45effff640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45f01062f0 0x7f45f006f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.569+0000 7f45effff640 1 -- 172.21.15.82:0/534720376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45f01018f0 msgr2=0x7f45f006ec60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:57:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.569+0000 7f45effff640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45f01018f0 0x7f45f006ec60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.569+0000 7f45effff640 1 -- 172.21.15.82:0/534720376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f0100fd0 msgr2=0x7f45f0073b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.569+0000 7f45effff640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f0100fd0 0x7f45f0073b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.569+0000 7f45effff640 1 -- 172.21.15.82:0/534720376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f01b3a00 con 0x7f45f01062f0 2024-04-21T08:57:14.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.572+0000 7f45effff640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45f01062f0 0x7f45f006f200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f45e8037040 tx=0x7f45e80327f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:14.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.573+0000 7f45ecff9640 1 -- 172.21.15.82:0/534720376 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8041020 con 0x7f45f01062f0 2024-04-21T08:57:14.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.573+0000 7f45f6081640 1 -- 172.21.15.82:0/534720376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f01b3bd0 con 0x7f45f01062f0 2024-04-21T08:57:14.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.573+0000 7f45f6081640 1 -- 172.21.15.82:0/534720376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f01b4030 con 0x7f45f01062f0 2024-04-21T08:57:14.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.573+0000 7f45ecff9640 1 -- 172.21.15.82:0/534720376 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e8034e10 con 0x7f45f01062f0 2024-04-21T08:57:14.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.573+0000 7f45ecff9640 1 -- 172.21.15.82:0/534720376 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8039040 con 0x7f45f01062f0 2024-04-21T08:57:14.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.575+0000 7f45f6081640 1 -- 172.21.15.82:0/534720376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45dc005290 con 0x7f45f01062f0 2024-04-21T08:57:14.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.579+0000 7f45ecff9640 1 -- 172.21.15.82:0/534720376 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f45e8044020 con 0x7f45f01062f0 2024-04-21T08:57:14.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.580+0000 7f45ecff9640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f45d80a3e20 0x7f45d80a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:14.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.580+0000 7f45ecff9640 1 -- 172.21.15.82:0/534720376 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f45e80f28b0 con 0x7f45f01062f0 2024-04-21T08:57:14.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.580+0000 7f45ef7fe640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f45d80a3e20 0x7f45d80a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.582+0000 7f45ef7fe640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f45d80a3e20 0x7f45d80a62e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f45e0005fd0 tx=0x7f45e000b480 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.592+0000 7f45ecff9640 1 -- 172.21.15.82:0/534720376 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45e804b020 con 0x7f45f01062f0 2024-04-21T08:57:14.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:14 smithi082 ceph-mon[27967]: pgmap v316: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 878 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-21T08:57:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:14 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/461837919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:57:14.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:14 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/28532159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:57:14.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.945+0000 7f45f6081640 1 -- 172.21.15.82:0/534720376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f45dc005850 con 0x7f45f01062f0 2024-04-21T08:57:14.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.946+0000 7f45ecff9640 1 -- 172.21.15.82:0/534720376 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v101) ==== 103+0+11 (secure 0 0 0) 0x7f45e80beb90 con 0x7f45f01062f0 2024-04-21T08:57:14.946 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:57:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.951+0000 7f45f6081640 1 -- 172.21.15.82:0/534720376 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f45d80a3e20 msgr2=0x7f45d80a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.951+0000 7f45f6081640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f45d80a3e20 0x7f45d80a62e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f45e0005fd0 tx=0x7f45e000b480 comp rx=0 tx=0).stop 2024-04-21T08:57:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.951+0000 7f45f6081640 1 -- 172.21.15.82:0/534720376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45f01062f0 msgr2=0x7f45f006f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.951+0000 7f45f6081640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45f01062f0 0x7f45f006f200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f45e8037040 tx=0x7f45e80327f0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.951+0000 7f45f6081640 1 -- 172.21.15.82:0/534720376 shutdown_connections 2024-04-21T08:57:14.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.951+0000 7f45f6081640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45f01062f0 0x7f45f006f200 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.951+0000 7f45f6081640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45f01018f0 0x7f45f006ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.951+0000 7f45f6081640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f45d80a3e20 0x7f45d80a62e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.951+0000 7f45f6081640 1 --2- 172.21.15.82:0/534720376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f0100fd0 0x7f45f0073b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:14.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.951+0000 7f45f6081640 1 -- 172.21.15.82:0/534720376 >> 172.21.15.82:0/534720376 conn(0x7f45f00771d0 msgr2=0x7f45f0103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:14.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.952+0000 7f45f6081640 1 -- 172.21.15.82:0/534720376 shutdown_connections 2024-04-21T08:57:14.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:14.952+0000 7f45f6081640 1 -- 172.21.15.82:0/534720376 wait complete. 2024-04-21T08:57:14.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-21T08:57:14.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:14 smithi134 ceph-mon[31772]: pgmap v316: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 878 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-21T08:57:14.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:14 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/461837919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:57:14.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:14 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/28532159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:57:15.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:14 smithi106 ceph-mon[31072]: pgmap v316: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 878 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-21T08:57:15.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:14 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/461837919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:57:15.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:14 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/28532159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:57:15.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.140+0000 7f2bbe77e640 1 -- 172.21.15.82:0/3350126639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bb81031e0 msgr2=0x7f2bb81035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:15.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.140+0000 7f2bbe77e640 1 --2- 172.21.15.82:0/3350126639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bb81031e0 0x7f2bb81035c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009ca0 tx=0x7f2bb002c210 comp rx=0 tx=0).stop 2024-04-21T08:57:15.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.140+0000 7f2bbe77e640 1 -- 172.21.15.82:0/3350126639 shutdown_connections 2024-04-21T08:57:15.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.140+0000 7f2bbe77e640 1 --2- 172.21.15.82:0/3350126639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bb810c520 0x7f2bb810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:15.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.140+0000 7f2bbe77e640 1 --2- 172.21.15.82:0/3350126639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb8103b00 0x7f2bb810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:15.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.140+0000 7f2bbe77e640 1 --2- 172.21.15.82:0/3350126639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bb81031e0 0x7f2bb81035c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:15.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.140+0000 7f2bbe77e640 1 -- 172.21.15.82:0/3350126639 >> 172.21.15.82:0/3350126639 conn(0x7f2bb8077f50 msgr2=0x7f2bb8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:15.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.145+0000 7f2bbe77e640 1 -- 172.21.15.82:0/3350126639 shutdown_connections 2024-04-21T08:57:15.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.147+0000 7f2bbe77e640 1 -- 172.21.15.82:0/3350126639 wait complete. 2024-04-21T08:57:15.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.148+0000 7f2bbe77e640 1 Processor -- start 2024-04-21T08:57:15.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.151+0000 7f2bbe77e640 1 -- start start 2024-04-21T08:57:15.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.151+0000 7f2bbe77e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb81031e0 0x7f2bb81092c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:15.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.151+0000 7f2bbe77e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bb8103b00 0x7f2bb8109800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:15.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.151+0000 7f2bbe77e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bb810c520 0x7f2bb8109d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:15.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.151+0000 7f2bbe77e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8102990 con 0x7f2bb81031e0 2024-04-21T08:57:15.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.151+0000 7f2bbe77e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8102810 con 0x7f2bb8103b00 2024-04-21T08:57:15.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.151+0000 7f2bbe77e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8102b10 con 0x7f2bb810c520 2024-04-21T08:57:15.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.152+0000 7f2bbcd1b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bb810c520 0x7f2bb8109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:15.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.152+0000 7f2bb77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bb8103b00 0x7f2bb8109800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:15.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.152+0000 7f2bbcd1b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bb810c520 0x7f2bb8109d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.82:37534/0 (socket says 172.21.15.82:37534) 2024-04-21T08:57:15.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.152+0000 7f2bbcd1b640 1 -- 172.21.15.82:0/261303532 learned_addr learned my addr 172.21.15.82:0/261303532 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:15.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.153+0000 7f2bb7fff640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb81031e0 0x7f2bb81092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.154+0000 7f2bb7fff640 1 -- 172.21.15.82:0/261303532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bb810c520 msgr2=0x7f2bb8109d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.154+0000 7f2bb7fff640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bb810c520 0x7f2bb8109d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.154+0000 7f2bb7fff640 1 -- 172.21.15.82:0/261303532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bb8103b00 msgr2=0x7f2bb8109800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.154+0000 7f2bb7fff640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bb8103b00 0x7f2bb8109800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.154+0000 7f2bb7fff640 1 -- 172.21.15.82:0/261303532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb811bbf0 con 0x7f2bb81031e0 2024-04-21T08:57:15.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.155+0000 7f2bb7fff640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb81031e0 0x7f2bb81092c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f2bb002c820 tx=0x7f2bb0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.156+0000 7f2bb57fa640 1 -- 172.21.15.82:0/261303532 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0039040 con 0x7f2bb81031e0 2024-04-21T08:57:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.156+0000 7f2bb57fa640 1 -- 172.21.15.82:0/261303532 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb0035e20 con 0x7f2bb81031e0 2024-04-21T08:57:15.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.156+0000 7f2bb57fa640 1 -- 172.21.15.82:0/261303532 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0032830 con 0x7f2bb81031e0 2024-04-21T08:57:15.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.157+0000 7f2bbe77e640 1 -- 172.21.15.82:0/261303532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb811be80 con 0x7f2bb81031e0 2024-04-21T08:57:15.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.157+0000 7f2bbe77e640 1 -- 172.21.15.82:0/261303532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb811c320 con 0x7f2bb81031e0 2024-04-21T08:57:15.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.160+0000 7f2bbe77e640 1 -- 172.21.15.82:0/261303532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb80698e0 con 0x7f2bb81031e0 2024-04-21T08:57:15.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.164+0000 7f2bb57fa640 1 -- 172.21.15.82:0/261303532 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2bb00329d0 con 0x7f2bb81031e0 2024-04-21T08:57:15.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.165+0000 7f2bb57fa640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2ba00a3f20 0x7f2ba00a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:15.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.165+0000 7f2bb57fa640 1 -- 172.21.15.82:0/261303532 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9650+0+0 (secure 0 0 0) 0x7f2bb00f2c20 con 0x7f2bb81031e0 2024-04-21T08:57:15.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.170+0000 7f2bb77fe640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2ba00a3f20 0x7f2ba00a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:15.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.175+0000 7f2bb57fa640 1 -- 172.21.15.82:0/261303532 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bb00bef80 con 0x7f2bb81031e0 2024-04-21T08:57:15.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.178+0000 7f2bb77fe640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2ba00a3f20 0x7f2ba00a63e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2bb810aa30 tx=0x7f2bac008040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.626+0000 7f2bbe77e640 1 -- 172.21.15.82:0/261303532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f2bb811c640 con 0x7f2bb81031e0 2024-04-21T08:57:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.640+0000 7f2bb57fa640 1 -- 172.21.15.82:0/261303532 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746063 (secure 0 0 0) 0x7f2bb00be880 con 0x7f2bb81031e0 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout: "epoch": 26, 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout: "active_gid": 15546, 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout: "active_name": "x", 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout: "active_addrs": { 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:6800", 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3178848282 2024-04-21T08:57:15.653 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:6801", 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3178848282 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: "active_addr": "172.21.15.82:6801/3178848282", 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: "active_change": "2024-04-21T08:47:00.617581+0000", 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: "active_mgr_features": 4540701547738038271, 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: "available": true, 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: "standbys": [ 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.654 INFO:teuthology.orchestra.run.smithi082.stdout: "gid": 24946, 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "z", 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr_features": 4540701547738038271, 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "available_modules": [ 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alerts", 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:15.655 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.656 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.657 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.658 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.659 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.660 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_destination": { 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_destination", 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.661 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email address to send alerts to", 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_from_name": { 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_from_name", 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "Ceph", 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.662 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email From: name", 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_host": { 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_host", 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP server", 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.663 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_password": { 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_password", 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Password to authenticate with", 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.664 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_port": { 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_port", 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "465", 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP port", 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.665 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_sender": { 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_sender", 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP envelope sender", 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.666 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_ssl": { 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_ssl", 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.667 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_user": { 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_user", 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "User to authenticate as", 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.668 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "balancer", 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: "active": { 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "active", 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.669 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_time": { 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_time", 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.670 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0000", 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_weekday": { 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_weekday", 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:15.671 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_max_iterations": { 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_max_iterations", 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "25", 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-21T08:57:15.672 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "250", 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_metrics": { 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_metrics", 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.673 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_step": { 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_step", 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.5", 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0.001", 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "0.999", 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.674 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "aggressiveness of optimization", 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "end_time": { 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_time", 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2359", 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T08:57:15.675 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "end_weekday": { 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_weekday", 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:57:15.676 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.677 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.678 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.679 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.680 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.681 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: "min_score": { 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_score", 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T08:57:15.682 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "upmap", 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "crush-compat", 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "none", 2024-04-21T08:57:15.683 INFO:teuthology.orchestra.run.smithi082.stdout: "read", 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap", 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap-read" 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Balancer mode", 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_ids": { 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_ids", 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.684 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.685 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_deviation": { 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_deviation", 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_optimizations": { 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_optimizations", 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.687 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm", 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.688 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_down_multiplier": { 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_down_multiplier", 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.689 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_refresh_rate": { 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_refresh_rate", 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "20", 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T08:57:15.690 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_starting_port": { 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_starting_port", 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4721", 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.691 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_ptrace": { 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_ptrace", 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.692 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T08:57:15.693 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T08:57:15.693 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.693 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.693 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.693 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_interval": { 2024-04-21T08:57:15.693 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_interval", 2024-04-21T08:57:15.693 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.693 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.693 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.693 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:15.693 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.693 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_memory_target_ratio": { 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.7", 2024-04-21T08:57:15.694 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm_log_destination": { 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm_log_destination", 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.695 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "file", 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "file,syslog", 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "syslog" 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "cgroups_split": { 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cgroups_split", 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.696 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "config_checks_enabled": { 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_checks_enabled", 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.697 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "config_dashboard": { 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_dashboard", 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.698 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_alertmanager": { 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_alertmanager", 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.699 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_base": { 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_base", 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.700 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Container image name, without the tag", 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_elasticsearch": { 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_elasticsearch", 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.701 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "elasticsearch container image", 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_grafana": { 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_grafana", 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.702 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_haproxy": { 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_haproxy", 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.703 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "HAproxy container image", 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_agent": { 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_agent", 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T08:57:15.704 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger agent container image", 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_collector": { 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_collector", 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T08:57:15.705 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger collector container image", 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_query": { 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_query", 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.706 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger query container image", 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_keepalived": { 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_keepalived", 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.707 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Keepalived container image", 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_loki": { 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_loki", 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T08:57:15.708 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Loki container image", 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_node_exporter": { 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_node_exporter", 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T08:57:15.709 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_nvmeof": { 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_nvmeof", 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T08:57:15.710 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Nvme-of container image", 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_prometheus": { 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_prometheus", 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.711 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_promtail": { 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_promtail", 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.712 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Promtail container image", 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_samba": { 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_samba", 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.713 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Samba/SMB container image", 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_snmp_gateway": { 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_snmp_gateway", 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.714 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SNMP Gateway container image", 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: "container_init": { 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_init", 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.715 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "daemon_cache_timeout": { 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "daemon_cache_timeout", 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:15.716 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "default_cephadm_command_timeout": { 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "900", 2024-04-21T08:57:15.717 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: "default_registry": { 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_registry", 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.718 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io", 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "device_cache_timeout": { 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_cache_timeout", 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.719 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1800", 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache device inventory", 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "device_enhanced_scan": { 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_enhanced_scan", 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.720 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "facts_cache_timeout": { 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "facts_cache_timeout", 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.721 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache host facts data", 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "grafana_dashboards_path": { 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "grafana_dashboards_path", 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.722 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "host_check_interval": { 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "host_check_interval", 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.723 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to perform a host check", 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "hw_monitoring": { 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hw_monitoring", 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.724 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "inventory_list_all": { 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "inventory_list_all", 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.725 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.726 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.727 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: "log_refresh_metadata": { 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_refresh_metadata", 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.728 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.729 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.730 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.731 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.732 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.733 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "*", 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "max_count_per_host": { 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_count_per_host", 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.734 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of daemons per service per host", 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "max_osd_draining_count": { 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_osd_draining_count", 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:15.735 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "migration_current": { 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "migration_current", 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.736 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "internal - do not modify", 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.737 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "root", 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm-package", 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: "root" 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: "oob_default_addr": { 2024-04-21T08:57:15.738 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "oob_default_addr", 2024-04-21T08:57:15.741 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.741 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.742 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.742 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "169.254.1.1", 2024-04-21T08:57:15.742 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.742 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.742 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.742 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T08:57:15.742 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.742 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.742 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_alerts_path": { 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_alerts_path", 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.743 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_insecure": { 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_insecure", 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.744 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_password": { 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_password", 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.745 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_url": { 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_url", 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T08:57:15.746 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_username": { 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_username", 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.747 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "secure_monitoring_stack": { 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "secure_monitoring_stack", 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T08:57:15.748 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "service_discovery_port": { 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "service_discovery_port", 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8765", 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "cephadm service discovery port", 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.749 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: "ssh_config_file": { 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssh_config_file", 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T08:57:15.750 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "use_agent": { 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_agent", 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.751 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: "use_repo_digest": { 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_repo_digest", 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T08:57:15.752 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_failed_host_check": { 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_failed_host_check", 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T08:57:15.753 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.754 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.754 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.754 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.754 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_daemons": { 2024-04-21T08:57:15.754 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_daemons", 2024-04-21T08:57:15.754 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.754 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.754 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.754 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.754 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.754 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.754 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_hosts": { 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_hosts", 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.755 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crash", 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.756 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.757 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.758 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.759 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.760 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.761 INFO:teuthology.orchestra.run.smithi082.stdout: "retain_interval": { 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retain_interval", 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "31536000", 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.762 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_recent_interval": { 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_recent_interval", 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1209600", 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.763 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dashboard", 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.764 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.765 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.766 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_ENABLED": { 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.767 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.768 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.769 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.770 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.770 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.770 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.770 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T08:57:15.770 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T08:57:15.770 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.770 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.770 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.770 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.770 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.770 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.770 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.771 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.771 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.771 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.771 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.771 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.771 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T08:57:15.771 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T08:57:15.771 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.771 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.771 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.771 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.771 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.772 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.773 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.774 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.775 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.776 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_URL": { 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_URL", 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.781 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.781 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.781 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.781 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.781 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.781 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.781 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.781 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.781 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T08:57:15.781 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T08:57:15.781 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.781 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.782 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.782 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-21T08:57:15.782 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.782 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.782 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.782 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.782 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.782 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.782 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.782 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.782 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T08:57:15.782 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.783 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.784 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.785 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.786 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "[]", 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.787 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.788 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.788 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.788 INFO:teuthology.orchestra.run.smithi082.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T08:57:15.788 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T08:57:15.788 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.788 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.788 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.788 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "{}", 2024-04-21T08:57:15.788 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.788 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.788 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.788 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.789 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.790 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.790 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.790 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.790 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.790 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.790 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T08:57:15.790 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T08:57:15.790 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.790 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.790 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.790 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.790 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.791 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.792 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.793 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.794 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.795 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.796 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.797 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.798 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.799 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.800 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8", 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.801 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "45", 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.802 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.803 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.804 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.805 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.806 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.807 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.808 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.809 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.810 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "cross_origin_url": { 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cross_origin_url", 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.811 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "crt_file": { 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crt_file", 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.812 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "debug": { 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "debug", 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable/disable debug options", 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.813 INFO:teuthology.orchestra.run.smithi082.stdout: "jwt_token_ttl": { 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "jwt_token_ttl", 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "28800", 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.814 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.815 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.816 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.817 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.818 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.818 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.818 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.818 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.818 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.818 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.818 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.818 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.818 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.818 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.818 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.818 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.819 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.820 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "motd": { 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "motd", 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "The message of the day", 2024-04-21T08:57:15.821 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect_resolve_ip_addr": { 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.822 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.823 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.824 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.824 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.824 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.824 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.824 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-21T08:57:15.824 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-21T08:57:15.824 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.824 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.824 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.824 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8080", 2024-04-21T08:57:15.824 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.824 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.825 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl_server_port": { 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl_server_port", 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8443", 2024-04-21T08:57:15.826 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "redirect", 2024-04-21T08:57:15.827 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect" 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.828 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "url_prefix": { 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url_prefix", 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.829 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.830 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_monitoring": { 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_monitoring", 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "monitor device health metrics", 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.831 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.832 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.833 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.835 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.836 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "mark_out_threshold": { 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mark_out_threshold", 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2419200", 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.837 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_name": { 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_name", 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "device_health_metrics", 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.838 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "retention_period": { 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retention_period", 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15552000", 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain device health metrics", 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.839 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_frequency": { 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_frequency", 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.840 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "self_heal": { 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "self_heal", 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.841 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.842 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_threshold": { 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_threshold", 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7257600", 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.843 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "diskprediction_local", 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.844 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.845 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.846 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.847 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.848 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.848 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.848 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.848 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.848 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.848 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.848 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.848 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.848 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.848 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.848 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.848 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "predict_interval": { 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predict_interval", 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.849 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-04-21T08:57:15.850 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.850 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.850 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.850 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.850 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.850 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.850 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.850 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.850 INFO:teuthology.orchestra.run.smithi082.stdout: "predictor_model": { 2024-04-21T08:57:15.850 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predictor_model", 2024-04-21T08:57:15.850 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.850 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "prophetstor", 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.851 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.852 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "influx", 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": false, 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "influxdb python module not found", 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "batch_size": { 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "batch_size", 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5000", 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T08:57:15.853 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: "database": { 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "database", 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "ceph", 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.854 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": { 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hostname", 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server hostname", 2024-04-21T08:57:15.855 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "30", 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "5", 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.857 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.858 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.859 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.860 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "password": { 2024-04-21T08:57:15.861 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "password", 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "password of InfluxDB server user", 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "port": { 2024-04-21T08:57:15.862 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "port", 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8086", 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server port", 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-04-21T08:57:15.863 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "false", 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "threads": { 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threads", 2024-04-21T08:57:15.864 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "32", 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "username": { 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "username", 2024-04-21T08:57:15.865 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "username of InfluxDB server user", 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "verify_ssl": { 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "verify_ssl", 2024-04-21T08:57:15.866 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "true", 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.867 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "insights", 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.868 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.869 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.870 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.871 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "iostat", 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.873 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.874 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.875 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.876 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.877 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "k8sevents", 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.878 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_event_retention_days": { 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ceph_event_retention_days", 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7", 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.879 INFO:teuthology.orchestra.run.smithi082.stdout: "config_check_secs": { 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_check_secs", 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "10", 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.880 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.881 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.882 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.883 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.884 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.885 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "localpool", 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: "failure_domain": { 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "failure_domain", 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.886 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "host", 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "failure domain for any created local pool", 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.887 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.888 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.889 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.890 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.891 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.891 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.891 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.891 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.891 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.891 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.891 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.891 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.891 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.891 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.891 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.891 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.892 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.892 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.892 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.892 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.892 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.892 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.892 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.892 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.892 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.892 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.892 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "min_size": { 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_size", 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default min_size for any created local pool", 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.893 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi082.stdout: "num_rep": { 2024-04-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "num_rep", 2024-04-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3", 2024-04-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default replica count for any created local pool", 2024-04-21T08:57:15.894 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_num": { 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_num", 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "128", 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T08:57:15.895 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "prefix": { 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prefix", 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name prefix for any created local pool", 2024-04-21T08:57:15.896 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.897 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.897 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.897 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.897 INFO:teuthology.orchestra.run.smithi082.stdout: "subtree": { 2024-04-21T08:57:15.897 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "subtree", 2024-04-21T08:57:15.897 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.897 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.897 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.897 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "rack", 2024-04-21T08:57:15.897 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.897 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.897 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mds_autoscaler", 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.898 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.899 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.900 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.900 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.900 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.900 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.900 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.900 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.900 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.900 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.900 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.900 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.900 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.900 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.901 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.902 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.902 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.902 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.902 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.902 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.902 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.902 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.902 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.902 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.902 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.902 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.902 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.903 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirroring", 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.904 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.905 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.906 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.906 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.906 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.906 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.906 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.906 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.906 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.906 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.906 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.906 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.906 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.906 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.907 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.908 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "nfs", 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.910 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.911 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.912 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.913 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.914 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-04-21T08:57:15.915 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "fail_fs": { 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "fail_fs", 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T08:57:15.916 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.917 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.918 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.919 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.920 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.921 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator": { 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.922 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: "rook", 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: "test_orchestrator" 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Orchestrator backend", 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_perf_query", 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.923 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.924 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.925 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.926 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.927 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.928 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_support", 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.929 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.930 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.931 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.933 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.934 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_autoscaler", 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.935 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.937 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.938 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.939 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.940 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "threshold": { 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threshold", 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1.0", 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "scaling threshold", 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.941 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "progress", 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_pg_recovery_event": { 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_pg_recovery_event", 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.942 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.943 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.944 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.945 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.946 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.946 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.946 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.946 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.946 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.946 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.946 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.946 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.946 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.946 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.946 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.946 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.947 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.948 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "max_completed_events": { 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_completed_events", 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "50", 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.949 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "number of past completed events to remember", 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-21T08:57:15.950 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long the module is going to sleep", 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus", 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.951 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "cache": { 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cache", 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.952 INFO:teuthology.orchestra.run.smithi082.stdout: "exclude_perf_counters": { 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "exclude_perf_counters", 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.953 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.954 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.955 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.956 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.957 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.958 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools": { 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools", 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.959 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300", 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.960 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_interval": { 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_interval", 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15.0", 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.961 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.962 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "9283", 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.963 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "stale_cache_strategy": { 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stale_cache_strategy", 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "log", 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.964 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "default", 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "default", 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "error" 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.965 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.966 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.967 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.968 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.969 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.970 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.971 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.972 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.972 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.972 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.972 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.972 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.972 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.972 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.972 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_snap_create": { 2024-04-21T08:57:15.972 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_snap_create", 2024-04-21T08:57:15.972 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:15.972 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.972 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: "mirror_snapshot_schedule": { 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.973 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.974 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.974 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.974 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.974 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.974 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.974 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.974 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.974 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.974 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.974 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.974 INFO:teuthology.orchestra.run.smithi082.stdout: "trash_purge_schedule": { 2024-04-21T08:57:15.975 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "trash_purge_schedule", 2024-04-21T08:57:15.975 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.975 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.975 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.975 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.975 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.975 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.975 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.975 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.975 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.975 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "restful", 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_auth": { 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_auth", 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.976 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.977 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.978 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.979 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.980 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.981 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.982 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rgw", 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.988 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.989 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.989 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.989 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.989 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.989 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.989 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.989 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.989 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.989 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.989 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.989 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.989 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.990 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rook", 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.991 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.992 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.992 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.992 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.992 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.992 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.992 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.992 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.992 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.992 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.992 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.992 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.992 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.993 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.994 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.995 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "storage_class": { 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "storage_class", 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "local", 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:15.997 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "selftest", 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:15.998 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:15.999 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.000 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.001 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.002 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "roption1": { 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption1", 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.003 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "roption2": { 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption2", 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.004 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "xyz", 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption1": { 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption1", 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.005 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption2": { 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption2", 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.006 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption3": { 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption3", 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.007 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption4": { 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption4", 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.008 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption5": { 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption5", 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.009 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption6": { 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption6", 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.010 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption7": { 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption7", 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.011 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-21T08:57:16.012 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "42", 2024-04-21T08:57:16.012 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.012 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.012 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.012 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.012 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.012 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.012 INFO:teuthology.orchestra.run.smithi082.stdout: "testkey": { 2024-04-21T08:57:16.012 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testkey", 2024-04-21T08:57:16.012 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.012 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.012 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: "testlkey": { 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testlkey", 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.013 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: "testnewline": { 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testnewline", 2024-04-21T08:57:16.014 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.015 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_m_granularity": { 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_m_granularity", 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.016 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: "dump_on_update": { 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dump_on_update", 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.017 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "dump database to debug log on update", 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.018 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.019 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.020 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.020 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.020 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.020 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.020 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.020 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.020 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.020 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.020 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.020 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.020 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.020 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.021 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.022 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stats", 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.023 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.024 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.025 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.026 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.026 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.026 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.026 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.026 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.026 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.026 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.026 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.026 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.026 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.026 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.026 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "status", 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.030 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.031 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.032 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.033 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.034 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telegraf", 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: "address": { 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "address", 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.035 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15", 2024-04-21T08:57:16.036 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.037 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.038 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.039 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.040 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.041 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telemetry", 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.042 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_basic": { 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_basic", 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.043 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_crash": { 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_crash", 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_device": { 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_device", 2024-04-21T08:57:16.044 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_ident": { 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_ident", 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.045 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_perf": { 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_perf", 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.046 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "contact": { 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "contact", 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.047 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: "description": { 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "description", 2024-04-21T08:57:16.048 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "device_url": { 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_url", 2024-04-21T08:57:16.049 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-04-21T08:57:16.050 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.051 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "24", 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "8", 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "last_opt_revision": { 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "last_opt_revision", 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.052 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1", 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard": { 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard", 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.053 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard_description": { 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard_description", 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.054 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.055 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.056 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.057 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.058 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.059 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "organization": { 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "organization", 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.060 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "proxy": { 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "proxy", 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.061 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "url": { 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url", 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.062 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "test_orchestrator", 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.063 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.064 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.065 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.066 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.067 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.068 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.069 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.070 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.071 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.072 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.073 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_clones": { 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_clones", 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4", 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.074 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "periodic_async_work": { 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "periodic_async_work", 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.075 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Periodically check for async work", 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_delay": { 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_delay", 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.076 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_no_wait": { 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.077 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix", 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.078 INFO:teuthology.orchestra.run.smithi082.stdout: "discovery_interval": { 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "discovery_interval", 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "100", 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.079 INFO:teuthology.orchestra.run.smithi082.stdout: "identifier": { 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "identifier", 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.080 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.081 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.082 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.083 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.084 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.085 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_host": { 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_host", 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.086 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_port": { 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_port", 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10051", 2024-04-21T08:57:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_sender": { 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_sender", 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T08:57:16.088 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.089 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.089 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.089 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.089 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.089 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.089 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.089 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.089 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.089 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.089 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-21T08:57:16.089 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.089 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.090 INFO:teuthology.orchestra.run.smithi082.stdout: "gid": 25064, 2024-04-21T08:57:16.090 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "y", 2024-04-21T08:57:16.090 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr_features": 4540701547738038271, 2024-04-21T08:57:16.090 INFO:teuthology.orchestra.run.smithi082.stdout: "available_modules": [ 2024-04-21T08:57:16.090 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.090 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alerts", 2024-04-21T08:57:16.090 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.090 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.090 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.090 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.090 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.090 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.091 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.092 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.092 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.092 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.092 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.092 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.092 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.092 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.092 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.092 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.092 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.092 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.093 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.094 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.095 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.096 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_destination": { 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_destination", 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email address to send alerts to", 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.097 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_from_name": { 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_from_name", 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "Ceph", 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email From: name", 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.098 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_host": { 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_host", 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP server", 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.099 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_password": { 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_password", 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Password to authenticate with", 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.100 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_port": { 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_port", 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "465", 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP port", 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_sender": { 2024-04-21T08:57:16.101 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_sender", 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP envelope sender", 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.102 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_ssl": { 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_ssl", 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.103 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_user": { 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_user", 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "User to authenticate as", 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.104 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "balancer", 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "active": { 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "active", 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T08:57:16.105 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_time": { 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_time", 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0000", 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T08:57:16.106 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_weekday": { 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_weekday", 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T08:57:16.107 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_max_iterations": { 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_max_iterations", 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "25", 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "250", 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T08:57:16.108 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_metrics": { 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_metrics", 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T08:57:16.109 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_step": { 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_step", 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.5", 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0.001", 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "0.999", 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "aggressiveness of optimization", 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T08:57:16.110 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "end_time": { 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_time", 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2359", 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.111 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "end_weekday": { 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_weekday", 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.112 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.113 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.114 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.115 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.116 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: "min_score": { 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_score", 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.117 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.118 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "upmap", 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "crush-compat", 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "none", 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "read", 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap", 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap-read" 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Balancer mode", 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.119 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_ids": { 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_ids", 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.120 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.121 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_deviation": { 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_deviation", 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T08:57:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_optimizations": { 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_optimizations", 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T08:57:16.123 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm", 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_down_multiplier": { 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_down_multiplier", 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.124 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_refresh_rate": { 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_refresh_rate", 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.125 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "20", 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_starting_port": { 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_starting_port", 2024-04-21T08:57:16.126 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4721", 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_ptrace": { 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_ptrace", 2024-04-21T08:57:16.127 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_interval": { 2024-04-21T08:57:16.128 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_interval", 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_memory_target_ratio": { 2024-04-21T08:57:16.129 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.7", 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm_log_destination": { 2024-04-21T08:57:16.130 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm_log_destination", 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "file", 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "file,syslog", 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "syslog" 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.131 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: "cgroups_split": { 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cgroups_split", 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.132 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "config_checks_enabled": { 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_checks_enabled", 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.133 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: "config_dashboard": { 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_dashboard", 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.134 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_alertmanager": { 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_alertmanager", 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.135 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_base": { 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_base", 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Container image name, without the tag", 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.136 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_elasticsearch": { 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_elasticsearch", 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "elasticsearch container image", 2024-04-21T08:57:16.137 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_grafana": { 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_grafana", 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-21T08:57:16.138 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_haproxy": { 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_haproxy", 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "HAproxy container image", 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.139 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_agent": { 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_agent", 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger agent container image", 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.140 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_collector": { 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_collector", 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger collector container image", 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.141 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_query": { 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_query", 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger query container image", 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_keepalived": { 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_keepalived", 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Keepalived container image", 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.143 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_loki": { 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_loki", 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Loki container image", 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.144 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_node_exporter": { 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_node_exporter", 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.145 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_nvmeof": { 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_nvmeof", 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Nvme-of container image", 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.146 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_prometheus": { 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_prometheus", 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.147 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_promtail": { 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_promtail", 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Promtail container image", 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.148 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_samba": { 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_samba", 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Samba/SMB container image", 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.149 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_snmp_gateway": { 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_snmp_gateway", 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SNMP Gateway container image", 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.150 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.151 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.151 INFO:teuthology.orchestra.run.smithi082.stdout: "container_init": { 2024-04-21T08:57:16.151 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_init", 2024-04-21T08:57:16.151 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.151 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.151 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.151 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.151 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.151 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.151 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.151 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T08:57:16.151 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: "daemon_cache_timeout": { 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "daemon_cache_timeout", 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.152 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "default_cephadm_command_timeout": { 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "900", 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T08:57:16.153 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "default_registry": { 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_registry", 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io", 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T08:57:16.154 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "device_cache_timeout": { 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_cache_timeout", 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1800", 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache device inventory", 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.155 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "device_enhanced_scan": { 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_enhanced_scan", 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "facts_cache_timeout": { 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "facts_cache_timeout", 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache host facts data", 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.157 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "grafana_dashboards_path": { 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "grafana_dashboards_path", 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.158 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: "host_check_interval": { 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "host_check_interval", 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to perform a host check", 2024-04-21T08:57:16.159 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "hw_monitoring": { 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hw_monitoring", 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.160 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.161 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.161 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.161 INFO:teuthology.orchestra.run.smithi082.stdout: "inventory_list_all": { 2024-04-21T08:57:16.161 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "inventory_list_all", 2024-04-21T08:57:16.161 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.161 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.161 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.161 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.161 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.161 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.161 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.161 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.162 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: "log_refresh_metadata": { 2024-04-21T08:57:16.163 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_refresh_metadata", 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.164 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.165 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.166 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.168 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "*", 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.169 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: "max_count_per_host": { 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_count_per_host", 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.170 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of daemons per service per host", 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "max_osd_draining_count": { 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_osd_draining_count", 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.171 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "migration_current": { 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "migration_current", 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.172 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "internal - do not modify", 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "root", 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.173 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.174 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.174 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm-package", 2024-04-21T08:57:16.174 INFO:teuthology.orchestra.run.smithi082.stdout: "root" 2024-04-21T08:57:16.174 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.174 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T08:57:16.174 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.174 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.174 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.175 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.175 INFO:teuthology.orchestra.run.smithi082.stdout: "oob_default_addr": { 2024-04-21T08:57:16.175 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "oob_default_addr", 2024-04-21T08:57:16.175 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.175 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.175 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.175 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "169.254.1.1", 2024-04-21T08:57:16.175 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.175 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_alerts_path": { 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_alerts_path", 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.176 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_insecure": { 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_insecure", 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.177 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_password": { 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_password", 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.178 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_url": { 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_url", 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T08:57:16.179 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_username": { 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_username", 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T08:57:16.180 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "secure_monitoring_stack": { 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "secure_monitoring_stack", 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.181 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "service_discovery_port": { 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "service_discovery_port", 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8765", 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "cephadm service discovery port", 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.182 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "ssh_config_file": { 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssh_config_file", 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.183 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "use_agent": { 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_agent", 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.184 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "use_repo_digest": { 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_repo_digest", 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.185 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_failed_host_check": { 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_failed_host_check", 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.186 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_daemons": { 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_daemons", 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.187 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_hosts": { 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_hosts", 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.188 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crash", 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.189 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.190 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.191 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.192 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.193 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "retain_interval": { 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retain_interval", 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "31536000", 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.194 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_recent_interval": { 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_recent_interval", 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1209600", 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.195 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dashboard", 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.196 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.197 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.198 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_ENABLED": { 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.199 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.200 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.201 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.202 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.203 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.203 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.203 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.203 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.203 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T08:57:16.203 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T08:57:16.203 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.203 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.203 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.203 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.203 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.203 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.204 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.204 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.204 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.204 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.204 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.204 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.204 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T08:57:16.204 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T08:57:16.204 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.204 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.204 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.204 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.206 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.207 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.207 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.207 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.207 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.207 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.207 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.207 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.207 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.207 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.207 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T08:57:16.207 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T08:57:16.207 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T08:57:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T08:57:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.212 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_URL": { 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_URL", 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.213 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.214 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.215 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.216 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.217 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.218 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "[]", 2024-04-21T08:57:16.219 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "{}", 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.220 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.221 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.222 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.223 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.224 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.225 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.226 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.227 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.228 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.229 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T08:57:16.230 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.231 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.232 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8", 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T08:57:16.233 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "45", 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T08:57:16.234 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.235 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.236 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.237 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.238 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.239 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.240 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-21T08:57:16.241 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: "cross_origin_url": { 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cross_origin_url", 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.242 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: "crt_file": { 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crt_file", 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.243 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "debug": { 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "debug", 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.244 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable/disable debug options", 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "jwt_token_ttl": { 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "jwt_token_ttl", 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "28800", 2024-04-21T08:57:16.245 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.246 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.247 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.248 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.249 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.250 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.251 INFO:teuthology.orchestra.run.smithi082.stdout: "motd": { 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "motd", 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "The message of the day", 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect_resolve_ip_addr": { 2024-04-21T08:57:16.252 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-21T08:57:16.253 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-21T08:57:16.254 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8080", 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.255 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl_server_port": { 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl_server_port", 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.256 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8443", 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "redirect", 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect" 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-04-21T08:57:16.258 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "url_prefix": { 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url_prefix", 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.259 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2024-04-21T08:57:16.260 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_monitoring": { 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_monitoring", 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "monitor device health metrics", 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.261 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.262 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.263 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.264 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.265 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.266 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.266 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.266 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.266 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.266 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.266 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.266 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.266 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.266 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.266 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.266 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.266 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: "mark_out_threshold": { 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mark_out_threshold", 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2419200", 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T08:57:16.267 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_name": { 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_name", 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "device_health_metrics", 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.268 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T08:57:16.269 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.269 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.269 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.269 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.269 INFO:teuthology.orchestra.run.smithi082.stdout: "retention_period": { 2024-04-21T08:57:16.269 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retention_period", 2024-04-21T08:57:16.269 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.269 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.269 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.269 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15552000", 2024-04-21T08:57:16.269 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.269 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain device health metrics", 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_frequency": { 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_frequency", 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-04-21T08:57:16.270 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "self_heal": { 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "self_heal", 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.271 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.272 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_threshold": { 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_threshold", 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7257600", 2024-04-21T08:57:16.273 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "diskprediction_local", 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.274 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.275 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.276 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.277 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.278 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "predict_interval": { 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predict_interval", 2024-04-21T08:57:16.279 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "predictor_model": { 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predictor_model", 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.280 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "prophetstor", 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.281 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.282 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "influx", 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": false, 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "influxdb python module not found", 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "batch_size": { 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "batch_size", 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5000", 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.283 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "database": { 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "database", 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "ceph", 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.284 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": { 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hostname", 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server hostname", 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.285 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "30", 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "5", 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.286 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.287 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.288 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.289 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.290 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "password": { 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "password", 2024-04-21T08:57:16.291 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "password of InfluxDB server user", 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: "port": { 2024-04-21T08:57:16.292 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "port", 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8086", 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server port", 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-04-21T08:57:16.293 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "false", 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.294 INFO:teuthology.orchestra.run.smithi082.stdout: "threads": { 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threads", 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "32", 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.295 INFO:teuthology.orchestra.run.smithi082.stdout: "username": { 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "username", 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "username of InfluxDB server user", 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.296 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "verify_ssl": { 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "verify_ssl", 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "true", 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.297 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "insights", 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.298 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.299 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.300 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.301 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.302 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "iostat", 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.303 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.304 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.305 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.306 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.307 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.308 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "k8sevents", 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_event_retention_days": { 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ceph_event_retention_days", 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7", 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.309 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "config_check_secs": { 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_check_secs", 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "10", 2024-04-21T08:57:16.310 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.311 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.312 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.313 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.314 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.315 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "localpool", 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: "failure_domain": { 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "failure_domain", 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.316 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "host", 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "failure domain for any created local pool", 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.317 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.318 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.319 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.319 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.319 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.319 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.319 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.319 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.319 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.319 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.319 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.320 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.321 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.322 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.322 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.322 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.322 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.322 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.322 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.322 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.322 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.322 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.322 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.322 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.322 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "min_size": { 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_size", 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default min_size for any created local pool", 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T08:57:16.323 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "num_rep": { 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "num_rep", 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3", 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default replica count for any created local pool", 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.324 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_num": { 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_num", 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "128", 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.325 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "prefix": { 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prefix", 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name prefix for any created local pool", 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.326 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "subtree": { 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "subtree", 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "rack", 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.327 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mds_autoscaler", 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.328 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.329 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.330 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.331 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.332 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirroring", 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.333 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.334 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.335 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.336 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.337 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "nfs", 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.338 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.339 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.340 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.341 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.342 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.343 INFO:teuthology.orchestra.run.smithi082.stdout: "fail_fs": { 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "fail_fs", 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.344 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.345 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.346 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.347 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.348 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator": { 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.349 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: "rook", 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: "test_orchestrator" 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Orchestrator backend", 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.350 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_perf_query", 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.351 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.352 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.353 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.354 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.355 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_support", 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.356 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.357 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.358 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.359 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.360 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_autoscaler", 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.361 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.362 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.363 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.364 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.365 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.366 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "threshold": { 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threshold", 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-04-21T08:57:16.367 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1.0", 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "scaling threshold", 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "progress", 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_pg_recovery_event": { 2024-04-21T08:57:16.368 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_pg_recovery_event", 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.369 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.370 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.371 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.372 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.373 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.374 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "max_completed_events": { 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_completed_events", 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.375 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "50", 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "number of past completed events to remember", 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-21T08:57:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long the module is going to sleep", 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus", 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.377 INFO:teuthology.orchestra.run.smithi082.stdout: "cache": { 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cache", 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "exclude_perf_counters": { 2024-04-21T08:57:16.378 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "exclude_perf_counters", 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.379 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.380 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.381 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.382 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.383 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools": { 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools", 2024-04-21T08:57:16.384 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.385 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300", 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_interval": { 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_interval", 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.386 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15.0", 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.387 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "9283", 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.388 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "stale_cache_strategy": { 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stale_cache_strategy", 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "log", 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.389 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "default", 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "default", 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "error" 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-04-21T08:57:16.391 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.392 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.393 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.394 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.395 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.396 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_snap_create": { 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_snap_create", 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.397 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: "mirror_snapshot_schedule": { 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.398 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "trash_purge_schedule": { 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "trash_purge_schedule", 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.399 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "restful", 2024-04-21T08:57:16.400 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_auth": { 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_auth", 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.401 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.402 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.403 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.404 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.405 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.406 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.406 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.406 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.406 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.406 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.406 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.406 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.406 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.406 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.406 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.406 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.406 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.407 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.408 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.409 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rgw", 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.410 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.411 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.412 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.413 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.414 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.415 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rook", 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.416 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.417 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.417 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.417 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.417 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.417 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.417 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.417 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.417 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.417 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.417 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.417 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.417 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.418 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.418 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.418 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.418 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.418 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.418 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.418 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.418 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.418 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.418 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.418 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.419 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.420 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.421 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "storage_class": { 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "storage_class", 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "local", 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.422 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "selftest", 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.423 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.424 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.425 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.426 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.427 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "roption1": { 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption1", 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.428 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "roption2": { 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption2", 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.429 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "xyz", 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption1": { 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption1", 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.430 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption2": { 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption2", 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.431 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption3": { 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption3", 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.432 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption4": { 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption4", 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.433 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption5": { 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption5", 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.434 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption6": { 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption6", 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.435 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.436 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.436 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.436 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.436 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.436 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.436 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.436 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.436 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.436 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.436 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption7": { 2024-04-21T08:57:16.436 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption7", 2024-04-21T08:57:16.436 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "42", 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "testkey": { 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testkey", 2024-04-21T08:57:16.437 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: "testlkey": { 2024-04-21T08:57:16.438 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testlkey", 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "testnewline": { 2024-04-21T08:57:16.439 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testnewline", 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.440 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_m_granularity": { 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_m_granularity", 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.441 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "dump_on_update": { 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dump_on_update", 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.442 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "dump database to debug log on update", 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.443 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.444 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.445 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.446 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.447 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stats", 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.448 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.449 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.450 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.451 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.452 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "status", 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.453 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.454 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.455 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.456 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.457 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telegraf", 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.458 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "address": { 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "address", 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.459 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15", 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.460 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.461 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.462 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.463 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.464 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.464 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.464 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.464 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.464 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.464 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.464 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.464 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.464 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.464 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.464 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.464 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.465 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telemetry", 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_basic": { 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_basic", 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.466 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_crash": { 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_crash", 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.467 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_device": { 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_device", 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.468 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_ident": { 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_ident", 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.469 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_perf": { 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_perf", 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.470 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "contact": { 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "contact", 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.471 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "description": { 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "description", 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.472 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "device_url": { 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_url", 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.473 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.474 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "24", 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "8", 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.475 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "last_opt_revision": { 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "last_opt_revision", 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1", 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.476 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard": { 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard", 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.477 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard_description": { 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard_description", 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.478 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.479 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.480 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.481 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.482 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.483 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: "organization": { 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "organization", 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.484 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "proxy": { 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "proxy", 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.485 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "url": { 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url", 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.486 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "test_orchestrator", 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.487 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.489 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.490 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.491 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.492 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.493 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.494 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.495 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.496 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_clones": { 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_clones", 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.497 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4", 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "periodic_async_work": { 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "periodic_async_work", 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.498 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Periodically check for async work", 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_delay": { 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_delay", 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.499 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_no_wait": { 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.500 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix", 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.501 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "discovery_interval": { 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "discovery_interval", 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "100", 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.502 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "identifier": { 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "identifier", 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.503 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.504 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.505 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.506 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.507 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_host": { 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_host", 2024-04-21T08:57:16.509 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_port": { 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_port", 2024-04-21T08:57:16.510 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10051", 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_sender": { 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_sender", 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.511 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-21T08:57:16.512 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: "modules": [ 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: "dashboard", 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: "iostat", 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: "nfs", 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: "restful", 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: "snap_schedule" 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: "available_modules": [ 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alerts", 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.513 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.514 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.515 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.516 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.517 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.518 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.519 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_destination": { 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_destination", 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email address to send alerts to", 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.520 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_from_name": { 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_from_name", 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "Ceph", 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email From: name", 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_host": { 2024-04-21T08:57:16.521 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_host", 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP server", 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.522 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_password": { 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_password", 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Password to authenticate with", 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.523 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.524 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_port": { 2024-04-21T08:57:16.524 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_port", 2024-04-21T08:57:16.524 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.524 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.524 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.524 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "465", 2024-04-21T08:57:16.524 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.524 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.524 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.524 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP port", 2024-04-21T08:57:16.524 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.524 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_sender": { 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_sender", 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP envelope sender", 2024-04-21T08:57:16.525 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_ssl": { 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_ssl", 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.526 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_user": { 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_user", 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.527 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "User to authenticate as", 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "balancer", 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: "active": { 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "active", 2024-04-21T08:57:16.528 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_time": { 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_time", 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.529 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0000", 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_weekday": { 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_weekday", 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.530 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_max_iterations": { 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_max_iterations", 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.531 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "25", 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "250", 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_metrics": { 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_metrics", 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.532 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_step": { 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_step", 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.5", 2024-04-21T08:57:16.533 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0.001", 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "0.999", 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "aggressiveness of optimization", 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: "end_time": { 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_time", 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.534 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2359", 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "end_weekday": { 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_weekday", 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.536 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.537 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.538 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.538 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.538 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.538 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.538 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.538 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.538 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.538 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.538 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.538 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.538 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.538 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.539 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.540 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: "min_score": { 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_score", 2024-04-21T08:57:16.541 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-04-21T08:57:16.542 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "upmap", 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: "crush-compat", 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: "none", 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: "read", 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap", 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap-read" 2024-04-21T08:57:16.543 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.544 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Balancer mode", 2024-04-21T08:57:16.544 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.544 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.544 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.544 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.544 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_ids": { 2024-04-21T08:57:16.544 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_ids", 2024-04-21T08:57:16.544 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.544 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.544 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.544 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.544 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.545 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:16.546 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.546 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.546 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.546 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T08:57:16.546 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.546 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.546 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.546 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.546 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_deviation": { 2024-04-21T08:57:16.546 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_deviation", 2024-04-21T08:57:16.546 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.546 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.547 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.547 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-21T08:57:16.547 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-21T08:57:16.547 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.547 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.547 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T08:57:16.547 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T08:57:16.547 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.547 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.547 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.547 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_optimizations": { 2024-04-21T08:57:16.547 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_optimizations", 2024-04-21T08:57:16.548 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:16.548 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.548 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.548 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.548 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.548 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.548 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.548 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T08:57:16.548 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.548 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.548 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.548 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm", 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_down_multiplier": { 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_down_multiplier", 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.549 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_refresh_rate": { 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_refresh_rate", 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.550 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "20", 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_starting_port": { 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_starting_port", 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.551 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.552 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.552 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4721", 2024-04-21T08:57:16.552 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.552 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.552 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.552 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T08:57:16.552 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.552 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.552 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.552 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.552 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_ptrace": { 2024-04-21T08:57:16.552 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_ptrace", 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_interval": { 2024-04-21T08:57:16.553 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_interval", 2024-04-21T08:57:16.554 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.554 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.554 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.554 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:16.554 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.554 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.554 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.554 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T08:57:16.554 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.554 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.554 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.554 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_memory_target_ratio": { 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.7", 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.555 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm_log_destination": { 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm_log_destination", 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: "file", 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: "file,syslog", 2024-04-21T08:57:16.556 INFO:teuthology.orchestra.run.smithi082.stdout: "syslog" 2024-04-21T08:57:16.557 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.557 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T08:57:16.557 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.557 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.557 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.557 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.557 INFO:teuthology.orchestra.run.smithi082.stdout: "cgroups_split": { 2024-04-21T08:57:16.557 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cgroups_split", 2024-04-21T08:57:16.557 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.557 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.557 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.557 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.558 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.558 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.558 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.558 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T08:57:16.558 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.558 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.558 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.558 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.558 INFO:teuthology.orchestra.run.smithi082.stdout: "config_checks_enabled": { 2024-04-21T08:57:16.558 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_checks_enabled", 2024-04-21T08:57:16.558 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "config_dashboard": { 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_dashboard", 2024-04-21T08:57:16.559 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_alertmanager": { 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_alertmanager", 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.560 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_base": { 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_base", 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.561 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Container image name, without the tag", 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_elasticsearch": { 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_elasticsearch", 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.562 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "elasticsearch container image", 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_grafana": { 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_grafana", 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T08:57:16.563 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_haproxy": { 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_haproxy", 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.564 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "HAproxy container image", 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_agent": { 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_agent", 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.565 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger agent container image", 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_collector": { 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_collector", 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.566 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger collector container image", 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_query": { 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_query", 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger query container image", 2024-04-21T08:57:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_keepalived": { 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_keepalived", 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Keepalived container image", 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.568 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_loki": { 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_loki", 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Loki container image", 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.569 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_node_exporter": { 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_node_exporter", 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.570 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_nvmeof": { 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_nvmeof", 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Nvme-of container image", 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.571 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_prometheus": { 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_prometheus", 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.572 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_promtail": { 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_promtail", 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Promtail container image", 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_samba": { 2024-04-21T08:57:16.573 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_samba", 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Samba/SMB container image", 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.574 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_snmp_gateway": { 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_snmp_gateway", 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SNMP Gateway container image", 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "container_init": { 2024-04-21T08:57:16.575 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_init", 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "daemon_cache_timeout": { 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "daemon_cache_timeout", 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.576 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "default_cephadm_command_timeout": { 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.577 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "900", 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "default_registry": { 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_registry", 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.578 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io", 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "device_cache_timeout": { 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_cache_timeout", 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.579 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1800", 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache device inventory", 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "device_enhanced_scan": { 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_enhanced_scan", 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "facts_cache_timeout": { 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "facts_cache_timeout", 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.581 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache host facts data", 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: "grafana_dashboards_path": { 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "grafana_dashboards_path", 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.582 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.583 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T08:57:16.583 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.583 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.583 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.583 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T08:57:16.583 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.583 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.583 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.583 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.583 INFO:teuthology.orchestra.run.smithi082.stdout: "host_check_interval": { 2024-04-21T08:57:16.583 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "host_check_interval", 2024-04-21T08:57:16.583 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to perform a host check", 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "hw_monitoring": { 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hw_monitoring", 2024-04-21T08:57:16.584 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: "inventory_list_all": { 2024-04-21T08:57:16.585 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "inventory_list_all", 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.586 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.587 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: "log_refresh_metadata": { 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_refresh_metadata", 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.588 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.589 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.590 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.591 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.593 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "*", 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "max_count_per_host": { 2024-04-21T08:57:16.594 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_count_per_host", 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of daemons per service per host", 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "max_osd_draining_count": { 2024-04-21T08:57:16.595 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_osd_draining_count", 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "migration_current": { 2024-04-21T08:57:16.596 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "migration_current", 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "internal - do not modify", 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.597 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "root", 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm-package", 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: "root" 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.598 INFO:teuthology.orchestra.run.smithi082.stdout: "oob_default_addr": { 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "oob_default_addr", 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "169.254.1.1", 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_alerts_path": { 2024-04-21T08:57:16.599 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_alerts_path", 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_insecure": { 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_insecure", 2024-04-21T08:57:16.600 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_password": { 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_password", 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.601 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_url": { 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_url", 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.602 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_username": { 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_username", 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.603 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: "secure_monitoring_stack": { 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "secure_monitoring_stack", 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.604 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "service_discovery_port": { 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "service_discovery_port", 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.605 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8765", 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "cephadm service discovery port", 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "ssh_config_file": { 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssh_config_file", 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.606 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "use_agent": { 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_agent", 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.607 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "use_repo_digest": { 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_repo_digest", 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.608 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_failed_host_check": { 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_failed_host_check", 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.609 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_daemons": { 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_daemons", 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.610 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_hosts": { 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_hosts", 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.611 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crash", 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.612 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.613 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.614 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.615 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.616 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "retain_interval": { 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retain_interval", 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.617 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "31536000", 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_recent_interval": { 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_recent_interval", 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.618 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1209600", 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.619 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dashboard", 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.620 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.621 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.622 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_ENABLED": { 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.623 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.624 INFO:teuthology.orchestra.run.smithi082.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T08:57:16.625 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.626 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.627 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.628 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.629 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.630 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.631 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.632 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.633 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.634 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_URL": { 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_URL", 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.635 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.636 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.637 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.638 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.639 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.640 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "[]", 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.641 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "{}", 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.642 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T08:57:16.643 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T08:57:16.644 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.645 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.646 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.647 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.648 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.649 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.650 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.651 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.652 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.653 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8", 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.654 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "45", 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.655 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.656 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.657 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.658 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.660 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T08:57:16.661 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.662 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "cross_origin_url": { 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cross_origin_url", 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.663 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "crt_file": { 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crt_file", 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.664 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "debug": { 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "debug", 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.665 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable/disable debug options", 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "jwt_token_ttl": { 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "jwt_token_ttl", 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "28800", 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.666 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.667 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.668 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.669 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.670 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.671 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "motd": { 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "motd", 2024-04-21T08:57:16.672 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "The message of the day", 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect_resolve_ip_addr": { 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.673 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.674 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.675 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8080", 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.676 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl_server_port": { 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl_server_port", 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.677 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8443", 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.678 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "redirect", 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect" 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-04-21T08:57:16.679 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "url_prefix": { 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url_prefix", 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.680 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.681 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_monitoring": { 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_monitoring", 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "monitor device health metrics", 2024-04-21T08:57:16.682 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.683 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.684 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.685 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.686 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.687 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "mark_out_threshold": { 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mark_out_threshold", 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2419200", 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.688 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_name": { 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_name", 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "device_health_metrics", 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "retention_period": { 2024-04-21T08:57:16.689 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retention_period", 2024-04-21T08:57:16.690 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.690 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.690 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.690 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15552000", 2024-04-21T08:57:16.690 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.690 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.690 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.690 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain device health metrics", 2024-04-21T08:57:16.690 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.690 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_frequency": { 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_frequency", 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.691 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "self_heal": { 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "self_heal", 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.692 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.693 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_threshold": { 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_threshold", 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7257600", 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.694 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "diskprediction_local", 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.695 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.696 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.697 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.698 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.699 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "predict_interval": { 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predict_interval", 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.700 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "predictor_model": { 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predictor_model", 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "prophetstor", 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:16.701 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.702 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "influx", 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": false, 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "influxdb python module not found", 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "batch_size": { 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "batch_size", 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5000", 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.703 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "database": { 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "database", 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "ceph", 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.704 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": { 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hostname", 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server hostname", 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.705 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "30", 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "5", 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.706 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.707 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.708 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.709 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.710 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "password": { 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "password", 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.711 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "password of InfluxDB server user", 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "port": { 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "port", 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8086", 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server port", 2024-04-21T08:57:16.712 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "false", 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.713 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "threads": { 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threads", 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "32", 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.714 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "username": { 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "username", 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "username of InfluxDB server user", 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.715 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: "verify_ssl": { 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "verify_ssl", 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "true", 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.716 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "insights", 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.717 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.718 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.719 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.720 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.721 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "iostat", 2024-04-21T08:57:16.722 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.723 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.724 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.725 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.726 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "k8sevents", 2024-04-21T08:57:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_event_retention_days": { 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ceph_event_retention_days", 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7", 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.728 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "config_check_secs": { 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_check_secs", 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "10", 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.729 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.730 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.731 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.732 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.733 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.734 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "localpool", 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "failure_domain": { 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "failure_domain", 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "host", 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.735 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "failure domain for any created local pool", 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.736 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.737 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.738 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.739 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.740 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "min_size": { 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_size", 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default min_size for any created local pool", 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.741 INFO:teuthology.orchestra.run.smithi082.stdout: "num_rep": { 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "num_rep", 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3", 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default replica count for any created local pool", 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_num": { 2024-04-21T08:57:16.742 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_num", 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "128", 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "prefix": { 2024-04-21T08:57:16.743 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prefix", 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name prefix for any created local pool", 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "subtree": { 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "subtree", 2024-04-21T08:57:16.744 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "rack", 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mds_autoscaler", 2024-04-21T08:57:16.745 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.747 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.748 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.749 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirroring", 2024-04-21T08:57:16.750 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.752 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.753 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.754 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.755 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "nfs", 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.756 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.757 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.757 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.757 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.757 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.757 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.757 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.757 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.757 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.757 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.757 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.757 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.757 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.758 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.758 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.758 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.758 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.758 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.758 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.758 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.758 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.758 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.761 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.761 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.761 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.761 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.762 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.763 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.764 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.764 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.764 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.764 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.764 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.764 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.764 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.764 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.764 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-04-21T08:57:16.764 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.764 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.764 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "fail_fs": { 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "fail_fs", 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.765 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.766 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.767 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.768 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.769 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator": { 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.770 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: "rook", 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: "test_orchestrator" 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Orchestrator backend", 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_perf_query", 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.771 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.772 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.773 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.774 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.775 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_support", 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.776 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.777 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.778 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.779 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.780 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_autoscaler", 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.781 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.782 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.783 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.784 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.785 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:16.786 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "threshold": { 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threshold", 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1.0", 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.787 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "scaling threshold", 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "progress", 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_pg_recovery_event": { 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_pg_recovery_event", 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.788 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.789 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.790 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.791 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.792 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.793 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.794 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "max_completed_events": { 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_completed_events", 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.795 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "50", 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "number of past completed events to remember", 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.796 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long the module is going to sleep", 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus", 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "cache": { 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cache", 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.798 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "exclude_perf_counters": { 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "exclude_perf_counters", 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.799 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.800 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.801 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.802 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.803 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools": { 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools", 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.804 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300", 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.805 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_interval": { 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_interval", 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15.0", 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.806 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.807 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "9283", 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "stale_cache_strategy": { 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stale_cache_strategy", 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "log", 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.809 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "default", 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "default", 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "error" 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.810 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.811 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.812 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.813 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.814 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.815 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_snap_create": { 2024-04-21T08:57:16.816 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_snap_create", 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "mirror_snapshot_schedule": { 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T08:57:16.817 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "trash_purge_schedule": { 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "trash_purge_schedule", 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.818 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "restful", 2024-04-21T08:57:16.819 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_auth": { 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_auth", 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.820 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.821 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.822 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.823 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.824 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.825 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.826 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.827 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rgw", 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.828 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.829 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.830 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.831 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.832 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rook", 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.833 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.834 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.836 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.837 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "storage_class": { 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "storage_class", 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "local", 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.838 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "selftest", 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.839 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.840 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.841 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.842 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.843 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "roption1": { 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption1", 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.844 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "roption2": { 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption2", 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "xyz", 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.845 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption1": { 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption1", 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.846 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption2": { 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption2", 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.847 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption3": { 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption3", 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.848 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption4": { 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption4", 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.849 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption5": { 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption5", 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.850 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption6": { 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption6", 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.851 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption7": { 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption7", 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-21T08:57:16.852 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "42", 2024-04-21T08:57:16.853 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.853 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.853 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.853 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.853 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.853 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.853 INFO:teuthology.orchestra.run.smithi082.stdout: "testkey": { 2024-04-21T08:57:16.853 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testkey", 2024-04-21T08:57:16.853 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.853 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.853 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "testlkey": { 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testlkey", 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.854 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "testnewline": { 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testnewline", 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.855 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.856 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.856 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.856 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.856 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.856 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.856 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.856 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.856 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.856 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.856 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.856 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.856 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_m_granularity": { 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_m_granularity", 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.857 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: "dump_on_update": { 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dump_on_update", 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.858 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "dump database to debug log on update", 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.859 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.860 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.860 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.860 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.860 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.860 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.860 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.860 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.860 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.860 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.860 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.860 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.860 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.861 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.862 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.863 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.864 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stats", 2024-04-21T08:57:16.865 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.865 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.865 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.865 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.865 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.865 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.865 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.865 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.865 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.866 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.867 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.868 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.868 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.868 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.868 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.868 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.868 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.868 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.868 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.868 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.868 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.868 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.869 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.869 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.869 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.869 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.869 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.869 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.869 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.869 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.869 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.869 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.870 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.870 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.870 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.870 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.870 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.870 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.870 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.870 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.870 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.870 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.870 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.871 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.871 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.871 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.871 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.871 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.871 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.871 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.871 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.871 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "status", 2024-04-21T08:57:16.872 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.872 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.872 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.872 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.872 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.872 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.872 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.872 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.872 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.873 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.874 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.875 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.876 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telegraf", 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.877 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "address": { 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "address", 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.878 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15", 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.879 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.880 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.881 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.882 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.883 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telemetry", 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_basic": { 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_basic", 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.884 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_crash": { 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_crash", 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.885 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_device": { 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_device", 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.886 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_ident": { 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_ident", 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.887 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_perf": { 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_perf", 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.888 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "contact": { 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "contact", 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.889 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "description": { 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "description", 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.890 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "device_url": { 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_url", 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.891 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "24", 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "8", 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "last_opt_revision": { 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "last_opt_revision", 2024-04-21T08:57:16.893 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.894 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.894 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.894 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1", 2024-04-21T08:57:16.894 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.894 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.894 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.894 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.894 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.894 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.895 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.895 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.895 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard": { 2024-04-21T08:57:16.895 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard", 2024-04-21T08:57:16.895 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.895 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.895 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.895 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.895 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.895 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.895 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.896 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.896 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.896 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.896 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.896 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.896 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard_description": { 2024-04-21T08:57:16.896 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard_description", 2024-04-21T08:57:16.896 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.896 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.896 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.897 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.897 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.897 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.897 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.897 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.897 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.897 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.897 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.897 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.898 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.899 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.900 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.901 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "organization": { 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "organization", 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.902 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "proxy": { 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "proxy", 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.903 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "url": { 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url", 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.904 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "test_orchestrator", 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.905 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.906 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.907 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.908 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.909 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.910 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.911 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.912 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.913 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.914 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.915 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.915 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.915 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.915 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.915 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.915 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_clones": { 2024-04-21T08:57:16.915 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_clones", 2024-04-21T08:57:16.915 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.915 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.915 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.916 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4", 2024-04-21T08:57:16.916 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.916 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.916 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.916 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T08:57:16.916 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.916 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.916 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.916 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.916 INFO:teuthology.orchestra.run.smithi082.stdout: "periodic_async_work": { 2024-04-21T08:57:16.916 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "periodic_async_work", 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Periodically check for async work", 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_delay": { 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_delay", 2024-04-21T08:57:16.917 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_no_wait": { 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.918 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.919 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix", 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "discovery_interval": { 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "discovery_interval", 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "100", 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.920 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "identifier": { 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "identifier", 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-21T08:57:16.921 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-21T08:57:16.922 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.923 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-21T08:57:16.925 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.926 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.926 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.926 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.926 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.926 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.926 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-21T08:57:16.926 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-21T08:57:16.926 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-21T08:57:16.926 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.926 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-21T08:57:16.926 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-21T08:57:16.926 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_host": { 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_host", 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-21T08:57:16.927 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_port": { 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_port", 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10051", 2024-04-21T08:57:16.928 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.929 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.929 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.929 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.929 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.929 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.929 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.929 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.929 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_sender": { 2024-04-21T08:57:16.929 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_sender", 2024-04-21T08:57:16.929 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-21T08:57:16.929 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-21T08:57:16.929 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: "services": { 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: "dashboard": "https://172.21.15.82:8443/" 2024-04-21T08:57:16.930 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "always_on_modules": { 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "octopus": [ 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "pacific": [ 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-04-21T08:57:16.931 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "quincy": [ 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-04-21T08:57:16.932 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "reef": [ 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-04-21T08:57:16.933 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "squid": [ 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "last_failure_osd_epoch": 94, 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "active_clients": [ 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.934 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1371379600 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "libcephsqlite", 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 4034298965 2024-04-21T08:57:16.935 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 4138506066 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.936 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1161441586 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-21T08:57:16.937 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 4186693182 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1700157111 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-21T08:57:16.938 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:57:16.940 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:15 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/534720376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:57:16.940 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:15 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/534720376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:57:16.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.648+0000 7f2bbe77e640 1 -- 172.21.15.82:0/261303532 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2ba00a3f20 msgr2=0x7f2ba00a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:16.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.648+0000 7f2bbe77e640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2ba00a3f20 0x7f2ba00a63e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2bb810aa30 tx=0x7f2bac008040 comp rx=0 tx=0).stop 2024-04-21T08:57:16.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.648+0000 7f2bbe77e640 1 -- 172.21.15.82:0/261303532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb81031e0 msgr2=0x7f2bb81092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:16.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.648+0000 7f2bbe77e640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb81031e0 0x7f2bb81092c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f2bb002c820 tx=0x7f2bb0003ba0 comp rx=0 tx=0).stop 2024-04-21T08:57:16.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.648+0000 7f2bbe77e640 1 -- 172.21.15.82:0/261303532 shutdown_connections 2024-04-21T08:57:16.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.648+0000 7f2bbe77e640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bb810c520 0x7f2bb8109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:16.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.648+0000 7f2bbe77e640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bb8103b00 0x7f2bb8109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:16.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.648+0000 7f2bbe77e640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f2ba00a3f20 0x7f2ba00a63e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.648+0000 7f2bbe77e640 1 --2- 172.21.15.82:0/261303532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb81031e0 0x7f2bb81092c0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.648+0000 7f2bbe77e640 1 -- 172.21.15.82:0/261303532 >> 172.21.15.82:0/261303532 conn(0x7f2bb8077f50 msgr2=0x7f2bb80ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.648+0000 7f2bbe77e640 1 -- 172.21.15.82:0/261303532 shutdown_connections 2024-04-21T08:57:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:15.649+0000 7f2bbe77e640 1 -- 172.21.15.82:0/261303532 wait complete. 2024-04-21T08:57:16.985 DEBUG:tasks.check_counter:Getting stats from a 2024-04-21T08:57:16.986 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-21T08:57:16.986 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:15 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/534720376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:57:16.987 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:16 smithi134 ceph-mon[31772]: pgmap v317: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 878 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-21T08:57:16.987 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:16 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/261303532' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T08:57:17.029 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:16 smithi082 ceph-mon[27967]: pgmap v317: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 878 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-21T08:57:17.029 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:16 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/261303532' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T08:57:17.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:16 smithi106 ceph-mon[31072]: pgmap v317: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 878 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-21T08:57:17.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:16 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/261303532' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T08:57:17.490 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:57:17.975 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:17 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:57:17.996 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:17 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:57:18.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:17 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:57:18.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:18 smithi082 ceph-mon[27967]: pgmap v318: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 877 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-21T08:57:18.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:18 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:57:18.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:18 smithi106 ceph-mon[31072]: pgmap v318: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 877 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-21T08:57:18.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:18 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:57:18.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:18 smithi134 ceph-mon[31772]: pgmap v318: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 877 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 14 op/s 2024-04-21T08:57:18.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:18 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:57:20.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:20 smithi082 ceph-mon[27967]: pgmap v319: 129 pgs: 20 active+clean+snaptrim, 63 active+clean+snaptrim_wait, 46 active+clean; 872 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 200 B/s wr, 13 op/s 2024-04-21T08:57:20.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:20 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:20.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:20 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:20.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:20 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:20.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:20 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:20.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:20 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:20.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:20 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:20.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:20 smithi106 ceph-mon[31072]: pgmap v319: 129 pgs: 20 active+clean+snaptrim, 63 active+clean+snaptrim_wait, 46 active+clean; 872 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 200 B/s wr, 13 op/s 2024-04-21T08:57:20.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:20 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:20.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:20 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:20.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:20 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:20.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:20 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:20.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:20 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:20.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:20 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:20.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:20 smithi134 ceph-mon[31772]: pgmap v319: 129 pgs: 20 active+clean+snaptrim, 63 active+clean+snaptrim_wait, 46 active+clean; 872 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 200 B/s wr, 13 op/s 2024-04-21T08:57:20.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:20 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:20.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:20 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:20.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:20 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:20.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:20 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:20.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:20 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:20.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:20 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:21 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi134 to 176.2M 2024-04-21T08:57:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:21 smithi082 ceph-mon[27967]: Unable to set osd_memory_target on smithi134 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-21T08:57:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:21 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:21 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:21 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:21 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:21 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:21.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:21 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:21.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:21 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi134 to 176.2M 2024-04-21T08:57:21.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:21 smithi106 ceph-mon[31072]: Unable to set osd_memory_target on smithi134 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-21T08:57:21.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:21 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:21.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:21 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:21.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:21 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:21.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:21 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:21.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:21 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:21.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:21 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:21.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:21 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi134 to 176.2M 2024-04-21T08:57:21.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:21 smithi134 ceph-mon[31772]: Unable to set osd_memory_target on smithi134 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-21T08:57:21.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:21 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:21.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:21 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:21.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:21 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:21.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:21 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:21.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:21 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:21.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:21 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:22 smithi082 ceph-mon[27967]: pgmap v320: 129 pgs: 20 active+clean+snaptrim, 63 active+clean+snaptrim_wait, 46 active+clean; 872 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-21T08:57:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:22 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-21T08:57:22.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:22 smithi082 ceph-mon[27967]: Unable to set osd_memory_target on smithi106 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-21T08:57:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:22 smithi106 ceph-mon[31072]: pgmap v320: 129 pgs: 20 active+clean+snaptrim, 63 active+clean+snaptrim_wait, 46 active+clean; 872 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-21T08:57:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:22 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-21T08:57:22.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:22 smithi106 ceph-mon[31072]: Unable to set osd_memory_target on smithi106 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-21T08:57:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:22 smithi134 ceph-mon[31772]: pgmap v320: 129 pgs: 20 active+clean+snaptrim, 63 active+clean+snaptrim_wait, 46 active+clean; 872 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-21T08:57:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:22 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-21T08:57:22.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:22 smithi134 ceph-mon[31772]: Unable to set osd_memory_target on smithi106 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-04-21T08:57:23.261 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:57:23.261 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:57:23.261 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 991, 2024-04-21T08:57:23.261 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 960, 2024-04-21T08:57:23.261 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1996928, 2024-04-21T08:57:23.261 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 364742, 2024-04-21T08:57:23.261 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-04-21T08:57:23.261 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-04-21T08:57:23.261 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.128719141, 2024-04-21T08:57:23.261 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.041129740, 2024-04-21T08:57:23.262 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.058527677, 2024-04-21T08:57:23.262 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.032040834, 2024-04-21T08:57:23.262 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:23.262 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 959, 2024-04-21T08:57:23.262 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.056041386, 2024-04-21T08:57:23.262 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000058437 2024-04-21T08:57:23.262 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.262 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:23.262 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.262 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.262 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.262 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.263 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1750640, 2024-04-21T08:57:23.263 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 131648 2024-04-21T08:57:23.263 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.263 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:57:23.263 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 671, 2024-04-21T08:57:23.263 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 829, 2024-04-21T08:57:23.263 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 4710814, 2024-04-21T08:57:23.263 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 653194, 2024-04-21T08:57:23.263 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 7, 2024-04-21T08:57:23.263 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-04-21T08:57:23.263 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.070113774, 2024-04-21T08:57:23.263 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.024813240, 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.019271474, 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.019771042, 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 826, 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.037059893, 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000044866 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:57:23.264 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 440384 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 813, 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 813, 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 8601695, 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 257711, 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.076182953, 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.024251678, 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.023666861, 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.022351714, 2024-04-21T08:57:23.265 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 812, 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.043024194, 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000052985 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.266 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3749, 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.098690857, 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000026324 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-04-21T08:57:23.267 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 0, 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 0, 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.268 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 1277, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 1277, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 1277, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 0, 2024-04-21T08:57:23.269 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 1, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-04-21T08:57:23.270 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:57:23.271 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-04-21T08:57:23.272 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-04-21T08:57:23.273 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 0, 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-04-21T08:57:23.274 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 7337, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 11, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 17122596, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 17118616, 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-04-21T08:57:23.275 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 7337 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 2947, 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 2934, 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 11, 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 73, 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 62, 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 1277, 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 4311, 2024-04-21T08:57:23.276 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 3034, 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 0, 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 0, 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 59632, 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215416 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 0, 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 0, 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 0, 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-04-21T08:57:23.277 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.278 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.279 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-04-21T08:57:23.279 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.279 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.279 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.279 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.279 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-04-21T08:57:23.279 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.279 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.279 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.279 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.279 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-04-21T08:57:23.279 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-04-21T08:57:23.280 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.281 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-04-21T08:57:23.282 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.282 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.282 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.282 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.282 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-04-21T08:57:23.282 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.283 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.284 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.285 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-04-21T08:57:23.286 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-04-21T08:57:23.287 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.288 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 4, 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 3, 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 4, 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 0, 2024-04-21T08:57:23.289 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 0, 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 5265, 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:57:23.290 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:57:23.291 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:57:23.292 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 198257, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 57, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9952, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 193, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-04-21T08:57:23.293 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 957792, 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 5266, 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 2211, 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 21, 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 3, 2024-04-21T08:57:23.294 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 2211, 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2211, 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 9.203816991, 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.004162739 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2211, 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2212 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 2211, 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 2211, 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 0, 2024-04-21T08:57:23.295 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 1347, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 862, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 0, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 0, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 0, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-04-21T08:57:23.296 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 1, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-04-21T08:57:23.297 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-04-21T08:57:23.298 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 101, 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 12, 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 6, 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 6, 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 0, 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 2, 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-04-21T08:57:23.299 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-04-21T08:57:23.300 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 2471, 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 15070904, 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 2471, 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2471, 2024-04-21T08:57:23.301 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 15070904, 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-21T08:57:23.302 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:23.303 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:23.303 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2211, 2024-04-21T08:57:23.305 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 12930137, 2024-04-21T08:57:23.305 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2211, 2024-04-21T08:57:23.305 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 12930137, 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-21T08:57:23.306 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2211, 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 2211, 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2211, 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2211, 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:57:23.307 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.308 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x562f51e866e0": { 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-21T08:57:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:23.310 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:23.310 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:23.310 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:23.310 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:23.310 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:57:24.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:24 smithi082 ceph-mon[27967]: pgmap v321: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 866 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 170 B/s wr, 16 op/s 2024-04-21T08:57:24.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:24 smithi106 ceph-mon[31072]: pgmap v321: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 866 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 170 B/s wr, 16 op/s 2024-04-21T08:57:24.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:24 smithi134 ceph-mon[31772]: pgmap v321: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 866 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 170 B/s wr, 16 op/s 2024-04-21T08:57:24.857 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-21T08:57:24.857 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-21T08:57:24.857 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-21T08:57:24.857 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:57:24.857 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:57:24.857 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-04-21T08:57:24.857 DEBUG:tasks.check_counter:Getting stats from d 2024-04-21T08:57:24.858 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-21T08:57:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:57:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:26 smithi082 ceph-mon[27967]: pgmap v322: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 866 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-21T08:57:26.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:26 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:57:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:26 smithi106 ceph-mon[31072]: pgmap v322: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 866 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-21T08:57:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:26 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:57:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:26 smithi134 ceph-mon[31772]: pgmap v322: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 866 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-21T08:57:26.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:26 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:57:27.944 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:57:27.945 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:57:27.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2024-04-21T08:57:27.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2024-04-21T08:57:27.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2024-04-21T08:57:27.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2024-04-21T08:57:27.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-04-21T08:57:27.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2024-04-21T08:57:27.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.001160629, 2024-04-21T08:57:27.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T08:57:27.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000194757, 2024-04-21T08:57:27.945 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T08:57:27.946 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:27.946 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:27.946 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:27.946 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:27.946 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:27.946 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:27.946 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:27.946 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:27.946 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:27.946 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:27.946 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 2, 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 165, 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 320, 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 185152, 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.013050530, 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.009028354, 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001692777, 2024-04-21T08:57:27.947 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T08:57:27.948 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:27.948 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 162, 2024-04-21T08:57:27.948 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.019354598, 2024-04-21T08:57:27.948 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000119472 2024-04-21T08:57:27.948 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:27.948 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:27.948 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:27.948 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:27.948 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:27.948 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:27.948 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:57:27.948 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 185152 2024-04-21T08:57:27.949 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:27.949 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:57:27.949 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 231, 2024-04-21T08:57:27.949 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 218, 2024-04-21T08:57:27.949 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1594288, 2024-04-21T08:57:27.949 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 132992, 2024-04-21T08:57:27.949 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2024-04-21T08:57:27.949 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-04-21T08:57:27.949 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.040904984, 2024-04-21T08:57:27.949 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.012826694, 2024-04-21T08:57:27.949 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.033609726, 2024-04-21T08:57:27.949 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.004321203, 2024-04-21T08:57:27.950 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:27.950 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 218, 2024-04-21T08:57:27.950 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.015855840, 2024-04-21T08:57:27.950 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000072733 2024-04-21T08:57:27.950 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:27.950 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:27.950 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:27.950 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:27.950 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:27.950 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:27.950 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1594288, 2024-04-21T08:57:27.950 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 132992 2024-04-21T08:57:27.951 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:27.951 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-04-21T08:57:27.951 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2024-04-21T08:57:27.951 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-04-21T08:57:27.951 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:27.951 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-04-21T08:57:27.951 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:57:27.951 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-04-21T08:57:27.951 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:57:27.951 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:57:27.951 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:57:27.951 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-04-21T08:57:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:57:27.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:57:27.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:57:27.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:57:27.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:57:27.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:57:27.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:57:27.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-04-21T08:57:27.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:57:27.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:57:27.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:57:27.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-04-21T08:57:27.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-04-21T08:57:27.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-04-21T08:57:27.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:57:27.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:57:27.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:57:27.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:57:27.954 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 149017, 2024-04-21T08:57:27.954 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 31, 2024-04-21T08:57:27.954 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:57:27.954 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-04-21T08:57:27.954 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-04-21T08:57:27.954 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-04-21T08:57:27.954 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:57:27.955 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-04-21T08:57:27.955 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:57:27.955 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-04-21T08:57:27.955 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2024-04-21T08:57:27.955 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2024-04-21T08:57:27.955 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:57:27.955 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:57:27.955 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-04-21T08:57:27.955 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-04-21T08:57:27.955 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2024-04-21T08:57:27.955 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2024-04-21T08:57:27.955 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-04-21T08:57:27.956 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-04-21T08:57:27.956 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-04-21T08:57:27.956 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-04-21T08:57:27.956 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:27.956 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:57:27.956 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-21T08:57:27.956 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-21T08:57:27.956 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-21T08:57:27.956 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 229, 2024-04-21T08:57:27.956 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 1559274, 2024-04-21T08:57:27.956 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:27.956 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 229, 2024-04-21T08:57:27.957 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-21T08:57:27.957 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-21T08:57:27.957 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 229, 2024-04-21T08:57:27.957 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1559274, 2024-04-21T08:57:27.957 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-21T08:57:27.957 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:27.957 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:27.957 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:27.957 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:27.957 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:27.957 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:57:28.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:28 smithi082 ceph-mon[27967]: pgmap v323: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 866 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-21T08:57:28.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:28 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:28.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:28 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:28.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:28 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:28.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:28 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:28.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:28 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:28.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:28 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:28.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:28 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:57:28.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:28 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:57:28.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:28 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:28 smithi106 ceph-mon[31072]: pgmap v323: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 866 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-21T08:57:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:28 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:28 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:28 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:28 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:28.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:28 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:28.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:28 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:28.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:28 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:57:28.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:28 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:57:28.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:28 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:28.663 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-21T08:57:28.663 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-21T08:57:28.663 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-21T08:57:28.663 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-21T08:57:28.663 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-21T08:57:28.663 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-21T08:57:28.663 DEBUG:tasks.check_counter:Getting stats from g 2024-04-21T08:57:28.664 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-21T08:57:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:28 smithi134 ceph-mon[31772]: pgmap v323: 129 pgs: 12 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 82 active+clean; 866 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-21T08:57:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:28 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:28 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:28 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:28.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:28 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:28.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:28 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:28.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:28 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:57:28.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:28 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:57:28.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:28 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:57:28.727 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:28 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' 2024-04-21T08:57:28.901 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:57:29.289 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:29 smithi082 ceph-mon[27967]: Adjusting osd_memory_target on smithi082 to 176.2M 2024-04-21T08:57:29.290 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:29 smithi082 ceph-mon[27967]: Unable to set osd_memory_target on smithi082 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-21T08:57:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:29 smithi106 ceph-mon[31072]: Adjusting osd_memory_target on smithi082 to 176.2M 2024-04-21T08:57:29.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:29 smithi106 ceph-mon[31072]: Unable to set osd_memory_target on smithi082 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-21T08:57:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:29 smithi134 ceph-mon[31772]: Adjusting osd_memory_target on smithi082 to 176.2M 2024-04-21T08:57:29.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:29 smithi134 ceph-mon[31772]: Unable to set osd_memory_target on smithi082 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-04-21T08:57:30.584 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:30 smithi082 ceph-mon[27967]: pgmap v324: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-21T08:57:30.584 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:30 smithi082 ceph-mon[27967]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T08:57:30.613 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 918, 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 920, 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 4487705, 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 291925, 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.098962035, 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.037232769, 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.023594575, 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.029688190, 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 918, 2024-04-21T08:57:30.614 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.060202154, 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000065579 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1324, 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 1290, 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 16944742, 2024-04-21T08:57:30.615 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 470053, 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.132220375, 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.033347665, 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.072820762, 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.031852591, 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1290, 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.056137081, 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000043517 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.616 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1761392, 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 133552 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 190, 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 350, 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 47178, 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 503763, 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 7, 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.029878438, 2024-04-21T08:57:30.617 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.013155228, 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.006126284, 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.007000858, 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 347, 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.030838473, 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000088871 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 443968 2024-04-21T08:57:30.618 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3696, 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.090707426, 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000024542 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-04-21T08:57:30.619 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 0, 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 0, 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.620 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 1978, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 1978, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 1978, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 0, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 0, 2024-04-21T08:57:30.621 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 1, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-04-21T08:57:30.622 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:57:30.623 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-04-21T08:57:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-04-21T08:57:30.625 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 0, 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 0, 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 10432, 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-04-21T08:57:30.626 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 14, 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 23299666, 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 23295686, 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 10432 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.627 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 13, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 3898, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 3885, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 11, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 84, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 73, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 1978, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 5551, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 3573, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 0, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 0, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 60588, 2024-04-21T08:57:30.628 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 215416 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 0, 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 0, 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 0, 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.629 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.630 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-04-21T08:57:30.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:30 smithi106 ceph-mon[31072]: pgmap v324: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-21T08:57:30.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:30 smithi106 ceph-mon[31072]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T08:57:30.633 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-04-21T08:57:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-04-21T08:57:30.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-04-21T08:57:30.636 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.637 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.638 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-04-21T08:57:30.639 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 2, 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 2, 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 0, 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 0, 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 0, 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 1787, 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.640 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:57:30.641 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-04-21T08:57:30.642 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 197473, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 55, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9952, 2024-04-21T08:57:30.643 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 193, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 1452784, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 8072, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 2164, 2024-04-21T08:57:30.644 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 21, 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 3, 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 2164, 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2164, 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 14.464711907, 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.006684247 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2164, 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2165 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 2164, 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 2164, 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 0, 2024-04-21T08:57:30.645 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 1287, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 875, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 1, 2024-04-21T08:57:30.646 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-04-21T08:57:30.647 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 102, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 13, 2024-04-21T08:57:30.648 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 6, 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 6, 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 0, 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 2, 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-04-21T08:57:30.649 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 0, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 2428, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 21244763, 2024-04-21T08:57:30.650 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 2428, 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2428, 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 21244763, 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-21T08:57:30.651 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2164, 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 19106687, 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2164, 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 19106687, 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.652 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2164, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 2164, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2164, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2164, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.653 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-04-21T08:57:30.654 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x559bbc46c6e0": { 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-21T08:57:30.655 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-21T08:57:30.656 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-04-21T08:57:30.656 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-04-21T08:57:30.656 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-21T08:57:30.656 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:30.656 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:30.656 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:30.656 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:30.656 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:30.656 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:57:30.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:30 smithi134 ceph-mon[31772]: pgmap v324: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-21T08:57:30.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:30 smithi134 ceph-mon[31772]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T08:57:31.471 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-21T08:57:31.471 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-21T08:57:31.471 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-21T08:57:31.471 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:57:31.471 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:57:31.471 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-04-21T08:57:31.471 DEBUG:tasks.check_counter:Getting stats from j 2024-04-21T08:57:31.471 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-21T08:57:31.710 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:57:32.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:32 smithi082 ceph-mon[27967]: pgmap v326: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:32.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:32 smithi106 ceph-mon[31072]: pgmap v326: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:32.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:32 smithi134 ceph-mon[31772]: pgmap v326: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:33.432 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-21T08:57:33.432 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:57:33.432 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 233, 2024-04-21T08:57:33.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 220, 2024-04-21T08:57:33.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1594832, 2024-04-21T08:57:33.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 134272, 2024-04-21T08:57:33.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-04-21T08:57:33.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2024-04-21T08:57:33.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.045695546, 2024-04-21T08:57:33.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.013189140, 2024-04-21T08:57:33.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.037436444, 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.004428464, 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 220, 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.011305458, 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000051388 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1594832, 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 134272 2024-04-21T08:57:33.434 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 2, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 167, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 320, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 185504, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.012522991, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.008494890, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001727313, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 164, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.019302127, 2024-04-21T08:57:33.435 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000117695 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 185504 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2024-04-21T08:57:33.436 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.000389159, 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000329039, 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:33.437 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:57:33.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:57:33.439 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 149017, 2024-04-21T08:57:33.440 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 31, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2024-04-21T08:57:33.441 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 231, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 1559534, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 231, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-21T08:57:33.442 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-21T08:57:33.443 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 231, 2024-04-21T08:57:33.443 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1559534, 2024-04-21T08:57:33.443 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-21T08:57:33.443 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-21T08:57:33.443 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-21T08:57:33.443 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-21T08:57:33.443 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:33.443 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-21T08:57:33.443 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-21T08:57:34.371 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-21T08:57:34.371 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-21T08:57:34.371 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-21T08:57:34.371 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-21T08:57:34.371 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-21T08:57:34.371 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-21T08:57:34.371 DEBUG:tasks.check_counter:Getting stats from b 2024-04-21T08:57:34.372 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-21T08:57:34.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:34 smithi082 ceph-mon[27967]: pgmap v327: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:34.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:34 smithi106 ceph-mon[31072]: pgmap v327: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:34.682 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:57:34.726 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:34 smithi134 ceph-mon[31772]: pgmap v327: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:36.459 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:36 smithi106 ceph-mon[31072]: pgmap v328: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:36.460 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:36 smithi106 ceph-mon[31072]: osdmap e103: 12 total, 12 up, 12 in 2024-04-21T08:57:36.460 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:36 smithi106 ceph-mon[31072]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:57:36.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:36 smithi134 ceph-mon[31772]: pgmap v328: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:36.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:36 smithi134 ceph-mon[31772]: osdmap e103: 12 total, 12 up, 12 in 2024-04-21T08:57:36.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:36 smithi134 ceph-mon[31772]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:57:36.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:36 smithi082 ceph-mon[27967]: pgmap v328: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:36.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:36 smithi082 ceph-mon[27967]: osdmap e103: 12 total, 12 up, 12 in 2024-04-21T08:57:36.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:36 smithi082 ceph-mon[27967]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:57:36.787 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-21T08:57:36.787 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:57:36.787 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 4728, 2024-04-21T08:57:36.787 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 4595, 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 1969586, 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 16505550, 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 146, 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 10, 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.445727679, 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.118426554, 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.131409638, 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.162815293, 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 4591, 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1.370230664, 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000298460 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.788 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1607, 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000321647, 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000200 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 45856, 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 3344 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 31655, 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 36289, 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 11929643, 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 22780813, 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 46, 2024-04-21T08:57:36.789 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 5, 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 1.790604787, 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.765123476, 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 1.286956577, 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.141694423, 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 36289, 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 3.001515947, 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000082711 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 54789, 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.008956300, 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000163 2024-04-21T08:57:36.790 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 10362752, 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 20116448 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 11935, 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 16065, 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 6384124, 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 8707737, 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 27, 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 7, 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.980391795, 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.425872035, 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.506873635, 2024-04-21T08:57:36.791 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.103460715, 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 16064, 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1.406062976, 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000087528 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 20952, 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.003644519, 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000173 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 1820192, 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 484992 2024-04-21T08:57:36.792 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 1, 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-MDSRank": { 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 3786, 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 4.149128442, 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001095913 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-PurgeQueue": { 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-04-21T08:57:36.793 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2302, 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.257152039, 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000111708 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": { 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: "request": 16422, 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: "reply": 16361, 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: "reply_latency": { 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 16361, 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1186.128824942, 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.072497330 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.794 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_reply": 0, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "forward": 61, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_complete": 23, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_keys": 6, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_commit": 519, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_split": 3, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_merge": 0, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes": 34, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_top": 9, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_bottom": 11, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pin_tail": 14, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pinned": 21, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_expired": 2363, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_with_caps": 2, 2024-04-21T08:57:36.795 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": 2, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "subtrees": 3, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse": 32753, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_hit": 23014, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_forward": 60, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_discover": 6, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_remote_ino": 0, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_lock": 322, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "load_cent": 0, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "q": 0, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "exported": 25, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "exported_inodes": 1950, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "imported": 4, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "imported_inodes": 568, 2024-04-21T08:57:36.796 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_dir_fetch": 0, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_backtrace_fetch": 2637, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_peer_discover": 0, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rfiles": 4, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rbytes": 234, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rsnaps": 6, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_fetch": 2637, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_set_tag": 0, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_inodes": 115, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_base_inodes": 14, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dirfrag_rstats": 101, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_file_inodes": 2522, 2024-04-21T08:57:36.797 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_inode_file_caps": 166, 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_revoke": 3814, 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_grant": 3891, 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps": 5369, 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps_dirty": 1214, 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_cap_release": 181, 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "process_request_cap_release": 3991 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_cache": { 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update": 6, 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update_receipt": 9, 2024-04-21T08:57:36.798 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_try_discover": 6, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_send_discover": 12, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_handle_discover": 79, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays": 2, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_delayed": 0, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_created": 1120, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_enqueued": 1118, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_reintegrated": 0, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_migrated": 0, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_processing": 0, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_started": 0, 2024-04-21T08:57:36.799 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_completed": 0, 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_quiesce_path": 1, 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_quiesce_inode": 423, 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_enqueue_scrub": 14, 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_exportdir": 25, 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_flush": 0, 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragmentdir": 3, 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragstats": 0, 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_inodestats": 0 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_log": { 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "evlrg": 0, 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "evadd": 6765, 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "evex": 6730, 2024-04-21T08:57:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: "evtrm": 6730, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "ev": 35, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "evexg": 0, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "evexd": 7, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "segadd": 31, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "segex": 30, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "segtrm": 30, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "seg": 1, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "segmjr": 1, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "segexg": 0, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "segexd": 1, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "expos": 17209764, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "wrpos": 17285079, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "rdpos": 0, 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "jlat": { 2024-04-21T08:57:36.801 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 613, 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 6.547560245, 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.010681174 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "replayed": 0 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_mem": { 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 31, 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "ino+": 2712, 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "ino-": 2681, 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "dir": 24, 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "dir+": 89, 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "dir-": 65, 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "dn": 34, 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "dn+": 3978, 2024-04-21T08:57:36.802 INFO:teuthology.orchestra.run.smithi106.stdout: "dn-": 3944, 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "cap": 2, 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "cap+": 1856, 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "cap-": 1854, 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "rss": 77068, 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "heap": 231800 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_server": { 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_client_request": 20569, 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_server_request": 141, 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_request": 16422, 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_session": 36, 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_peer_request": 2503, 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "req_create_latency": { 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 912, 2024-04-21T08:57:36.803 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 5.470325539, 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.005998163 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getattr_latency": { 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2512, 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 25.195863241, 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.010030200 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getfilelock_latency": { 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "req_link_latency": { 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.804 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookup_latency": { 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10540, 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1139.388881712, 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.108101411 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookuphash_latency": { 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupino_latency": { 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.805 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupname_latency": { 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupparent_latency": { 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupsnap_latency": { 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.806 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.807 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.807 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.807 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lssnap_latency": { 2024-04-21T08:57:36.807 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10, 2024-04-21T08:57:36.807 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.001639985, 2024-04-21T08:57:36.807 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000163998 2024-04-21T08:57:36.807 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.807 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mkdir_latency": { 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 15, 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.018617127, 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001241141 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mknod_latency": { 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mksnap_latency": { 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10, 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 3.942331078, 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.394233107 2024-04-21T08:57:36.808 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: "req_open_latency": { 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 842, 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.862120978, 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001023896 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: "req_readdir_latency": { 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 57, 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.260018240, 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.004561723 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rename_latency": { 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 747, 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 9.502153822, 2024-04-21T08:57:36.809 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.012720420 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: "req_renamesnap_latency": { 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: "req_snapdiff_latency": { 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmdir_latency": { 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 5, 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.037608554, 2024-04-21T08:57:36.810 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.007521710 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmsnap_latency": { 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 4, 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.040342171, 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.010085542 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmxattr_latency": { 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setattr_latency": { 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2, 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.001504051, 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000752025 2024-04-21T08:57:36.811 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setdirlayout_latency": { 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setfilelock_latency": { 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setlayout_latency": { 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.812 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setxattr_latency": { 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 9, 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.025945823, 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.002882869 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: "req_symlink_latency": { 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1, 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000636765, 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000636765 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: "req_unlink_latency": { 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 695, 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1.380835856, 2024-04-21T08:57:36.813 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001986814 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getvxattr_latency": { 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_sessions": { 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: "session_count": 2, 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: "session_add": 6, 2024-04-21T08:57:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: "session_remove": 4, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_open": 2, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_stale": 0, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "total_load": 256, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "average_load": 128, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "avg_session_uptime": 603, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "mdthresh_evicted": 0 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:57:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:57:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 329499, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 131, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-04-21T08:57:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 9712, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 187, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 179237, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 1367, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-04-21T08:57:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 9103, 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 16903146, 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 37, 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 9103, 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 9103, 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 198.732303497, 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.021831517 2024-04-21T08:57:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 9103, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 13351 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 9103, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 2772, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 6331, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 473, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 2002, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 92, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 691, 2024-04-21T08:57:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 171, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 3, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 2750, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 2672, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 3023, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:57:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-04-21T08:57:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 0, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 103, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 19, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 12, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 12, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 0, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-04-21T08:57:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 1228, 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 58, 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 188 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "oft": { 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_objs": 1, 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_updates": 0, 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_removes": 0 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "osd-slow-ops": { 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_ops_count": 538 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "purge_queue": { 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops": 0, 2024-04-21T08:57:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops_high_water": 10, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing": 0, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_high_water": 6, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed_ops": 2232, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed": 1118, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_item_in_journal": 0 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 48314, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 14579313, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 48314, 2024-04-21T08:57:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 48314, 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 14579313, 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_bytes": { 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 9103, 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 17045088, 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 9103, 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 17045088, 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_ops": { 2024-04-21T08:57:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 1024, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 9103, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 9103, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 9103, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 9103, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 1118, 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 142334, 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 1118, 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 78, 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 142334, 2024-04-21T08:57:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle-0x56520b8ee500": { 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 6765, 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 13090775, 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 6765, 2024-04-21T08:57:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-21T08:57:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-21T08:57:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 613, 2024-04-21T08:57:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 13090775, 2024-04-21T08:57:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:36.831 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-21T08:57:37.652 INFO:tasks.check_counter:Daemon mds.b mds.exported=25 2024-04-21T08:57:37.652 INFO:tasks.check_counter:Daemon mds.b mds.imported=4 2024-04-21T08:57:37.652 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-21T08:57:37.652 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-21T08:57:37.652 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-21T08:57:37.652 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=3 2024-04-21T08:57:37.652 DEBUG:tasks.check_counter:Getting stats from e 2024-04-21T08:57:37.652 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-21T08:57:37.895 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:57:38.304 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:38 smithi106 ceph-mon[31072]: pgmap v330: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-04-21T08:57:38.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:38 smithi134 ceph-mon[31772]: pgmap v330: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-04-21T08:57:38.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:38 smithi082 ceph-mon[27967]: pgmap v330: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-04-21T08:57:39.767 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-21T08:57:39.767 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:57:39.767 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 1167, 2024-04-21T08:57:39.767 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 1167, 2024-04-21T08:57:39.767 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 264549070, 2024-04-21T08:57:39.768 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 370346, 2024-04-21T08:57:39.768 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 8, 2024-04-21T08:57:39.768 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 4, 2024-04-21T08:57:39.768 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.204664545, 2024-04-21T08:57:39.768 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.027185567, 2024-04-21T08:57:39.768 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.141421221, 2024-04-21T08:57:39.768 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.027405765, 2024-04-21T08:57:39.768 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:39.768 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1166, 2024-04-21T08:57:39.768 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.034264430, 2024-04-21T08:57:39.769 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000029386 2024-04-21T08:57:39.769 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.769 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:39.769 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.769 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.769 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.769 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.769 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-21T08:57:39.769 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T08:57:39.769 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.770 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:57:39.770 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 900, 2024-04-21T08:57:39.770 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 1064, 2024-04-21T08:57:39.770 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 281296500, 2024-04-21T08:57:39.770 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 731857, 2024-04-21T08:57:39.770 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 9, 2024-04-21T08:57:39.770 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 7, 2024-04-21T08:57:39.770 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.221524916, 2024-04-21T08:57:39.770 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.021961481, 2024-04-21T08:57:39.770 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.171898876, 2024-04-21T08:57:39.770 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.020082141, 2024-04-21T08:57:39.771 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:39.771 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1061, 2024-04-21T08:57:39.771 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.038433087, 2024-04-21T08:57:39.771 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000036223 2024-04-21T08:57:39.771 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.771 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:39.771 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.771 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.771 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.771 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.771 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:57:39.771 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 446288 2024-04-21T08:57:39.772 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.772 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:57:39.772 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 1610, 2024-04-21T08:57:39.772 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 1572, 2024-04-21T08:57:39.772 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 174187898, 2024-04-21T08:57:39.772 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 561650, 2024-04-21T08:57:39.772 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 5, 2024-04-21T08:57:39.772 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 4, 2024-04-21T08:57:39.772 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.251319111, 2024-04-21T08:57:39.772 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.045429825, 2024-04-21T08:57:39.772 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.159326840, 2024-04-21T08:57:39.772 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.045669540, 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1571, 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.064313428, 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000040937 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 1792656, 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 137120 2024-04-21T08:57:39.773 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.774 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-04-21T08:57:39.774 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 1, 2024-04-21T08:57:39.774 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-04-21T08:57:39.774 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.774 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-MDSRank": { 2024-04-21T08:57:39.774 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-04-21T08:57:39.774 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-04-21T08:57:39.774 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 5114, 2024-04-21T08:57:39.774 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.394365764, 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000077114 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-PurgeQueue": { 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": { 2024-04-21T08:57:39.775 INFO:teuthology.orchestra.run.smithi106.stdout: "request": 0, 2024-04-21T08:57:39.776 INFO:teuthology.orchestra.run.smithi106.stdout: "reply": 0, 2024-04-21T08:57:39.776 INFO:teuthology.orchestra.run.smithi106.stdout: "reply_latency": { 2024-04-21T08:57:39.776 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.776 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.776 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.776 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.776 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_reply": 0, 2024-04-21T08:57:39.776 INFO:teuthology.orchestra.run.smithi106.stdout: "forward": 0, 2024-04-21T08:57:39.776 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_complete": 0, 2024-04-21T08:57:39.776 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_keys": 0, 2024-04-21T08:57:39.776 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_commit": 0, 2024-04-21T08:57:39.776 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_split": 0, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_merge": 0, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes": 120542, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_top": 24720, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_bottom": 95822, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pin_tail": 0, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pinned": 85227, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_expired": 0, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_with_caps": 0, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": 0, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "subtrees": 1, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse": 0, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_hit": 0, 2024-04-21T08:57:39.777 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_forward": 0, 2024-04-21T08:57:39.778 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_discover": 0, 2024-04-21T08:57:39.778 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:57:39.778 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_remote_ino": 0, 2024-04-21T08:57:39.778 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_lock": 0, 2024-04-21T08:57:39.778 INFO:teuthology.orchestra.run.smithi106.stdout: "load_cent": 0, 2024-04-21T08:57:39.778 INFO:teuthology.orchestra.run.smithi106.stdout: "q": 0, 2024-04-21T08:57:39.778 INFO:teuthology.orchestra.run.smithi106.stdout: "exported": 0, 2024-04-21T08:57:39.778 INFO:teuthology.orchestra.run.smithi106.stdout: "exported_inodes": 0, 2024-04-21T08:57:39.778 INFO:teuthology.orchestra.run.smithi106.stdout: "imported": 0, 2024-04-21T08:57:39.778 INFO:teuthology.orchestra.run.smithi106.stdout: "imported_inodes": 0, 2024-04-21T08:57:39.778 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_dir_fetch": 0, 2024-04-21T08:57:39.779 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:57:39.779 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_peer_discover": 0, 2024-04-21T08:57:39.779 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rfiles": 0, 2024-04-21T08:57:39.779 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rbytes": 0, 2024-04-21T08:57:39.779 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rsnaps": 0, 2024-04-21T08:57:39.779 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:57:39.779 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_set_tag": 0, 2024-04-21T08:57:39.779 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:57:39.779 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:57:39.779 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_file_inodes": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_cap_release": 0, 2024-04-21T08:57:39.780 INFO:teuthology.orchestra.run.smithi106.stdout: "process_request_cap_release": 0 2024-04-21T08:57:39.781 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.781 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_cache": { 2024-04-21T08:57:39.781 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update": 0, 2024-04-21T08:57:39.781 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update_receipt": 0, 2024-04-21T08:57:39.781 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_try_discover": 0, 2024-04-21T08:57:39.781 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_send_discover": 0, 2024-04-21T08:57:39.781 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_handle_discover": 0, 2024-04-21T08:57:39.781 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays": 0, 2024-04-21T08:57:39.781 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_delayed": 0, 2024-04-21T08:57:39.781 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:57:39.781 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_created": 0, 2024-04-21T08:57:39.782 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_enqueued": 0, 2024-04-21T08:57:39.782 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_reintegrated": 0, 2024-04-21T08:57:39.782 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_migrated": 0, 2024-04-21T08:57:39.782 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_processing": 0, 2024-04-21T08:57:39.782 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:57:39.782 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:57:39.782 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_started": 0, 2024-04-21T08:57:39.782 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_completed": 0, 2024-04-21T08:57:39.782 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:57:39.782 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:57:39.782 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:57:39.782 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_exportdir": 0, 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_flush": 0, 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragstats": 0, 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_inodestats": 0 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_log": { 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: "evlrg": 0, 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: "evadd": 0, 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: "evex": 0, 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: "evtrm": 0, 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: "ev": 206977, 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: "evexg": 0, 2024-04-21T08:57:39.783 INFO:teuthology.orchestra.run.smithi106.stdout: "evexd": 0, 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "segadd": 0, 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "segex": 0, 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "segtrm": 0, 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "seg": 141, 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "segmjr": 18, 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "segexg": 0, 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "segexd": 0, 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "expos": 345668143, 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "wrpos": 721338010, 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "rdpos": 721333557, 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "jlat": { 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.784 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: "replayed": 206977 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_mem": { 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 40833, 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: "ino+": 76512, 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: "ino-": 35679, 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: "dir": 1093, 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: "dir+": 1345, 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: "dir-": 252, 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: "dn": 120542, 2024-04-21T08:57:39.785 INFO:teuthology.orchestra.run.smithi106.stdout: "dn+": 124084, 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: "dn-": 3542, 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: "cap": 0, 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: "cap+": 0, 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: "cap-": 0, 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: "rss": 264936, 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: "heap": 215416 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_server": { 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_client_request": 0, 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_server_request": 0, 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_request": 0, 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_session": 0, 2024-04-21T08:57:39.786 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_peer_request": 0, 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: "req_create_latency": { 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getattr_latency": { 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getfilelock_latency": { 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.787 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.788 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.788 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.788 INFO:teuthology.orchestra.run.smithi106.stdout: "req_link_latency": { 2024-04-21T08:57:39.788 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.788 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.788 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.788 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.788 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookup_latency": { 2024-04-21T08:57:39.788 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.788 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.788 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.788 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookuphash_latency": { 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupino_latency": { 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupname_latency": { 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.789 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.790 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.790 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupparent_latency": { 2024-04-21T08:57:39.790 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.790 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.790 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.790 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.790 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupsnap_latency": { 2024-04-21T08:57:39.790 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.790 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.790 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.790 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.790 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lssnap_latency": { 2024-04-21T08:57:39.791 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.791 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.791 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.792 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.792 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mkdir_latency": { 2024-04-21T08:57:39.792 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.792 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.792 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.792 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.792 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mknod_latency": { 2024-04-21T08:57:39.792 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.792 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.792 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.792 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.792 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mksnap_latency": { 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: "req_open_latency": { 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: "req_readdir_latency": { 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.793 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.794 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.794 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rename_latency": { 2024-04-21T08:57:39.794 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.794 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.794 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.794 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.794 INFO:teuthology.orchestra.run.smithi106.stdout: "req_renamesnap_latency": { 2024-04-21T08:57:39.794 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.794 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.794 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.794 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.794 INFO:teuthology.orchestra.run.smithi106.stdout: "req_snapdiff_latency": { 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmdir_latency": { 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmsnap_latency": { 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.795 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.796 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmxattr_latency": { 2024-04-21T08:57:39.796 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.796 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.796 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.796 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.796 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setattr_latency": { 2024-04-21T08:57:39.796 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.796 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.796 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.796 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.796 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setdirlayout_latency": { 2024-04-21T08:57:39.796 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setfilelock_latency": { 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setlayout_latency": { 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.797 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setxattr_latency": { 2024-04-21T08:57:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: "req_symlink_latency": { 2024-04-21T08:57:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: "req_unlink_latency": { 2024-04-21T08:57:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:57:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:57:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getvxattr_latency": { 2024-04-21T08:57:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_sessions": { 2024-04-21T08:57:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: "session_count": 2, 2024-04-21T08:57:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: "session_add": 5, 2024-04-21T08:57:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: "session_remove": 3, 2024-04-21T08:57:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_open": 5, 2024-04-21T08:57:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_stale": 0, 2024-04-21T08:57:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: "total_load": 0, 2024-04-21T08:57:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: "average_load": 0, 2024-04-21T08:57:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: "avg_session_uptime": 3533, 2024-04-21T08:57:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: "mdthresh_evicted": 0 2024-04-21T08:57:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-04-21T08:57:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:57:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-04-21T08:57:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:57:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:57:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:57:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:57:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:57:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:57:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:57:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:57:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:57:39.802 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:57:39.802 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:57:39.802 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:57:39.802 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:57:39.802 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-04-21T08:57:39.802 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:57:39.802 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-04-21T08:57:39.802 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:57:39.802 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:57:39.802 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:57:39.802 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:57:39.803 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:57:39.803 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:57:39.803 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:57:39.803 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-04-21T08:57:39.803 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:57:39.803 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:57:39.803 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:57:39.803 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-04-21T08:57:39.803 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-04-21T08:57:39.803 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-04-21T08:57:39.803 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 1769001, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 41419, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:57:39.804 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 9592, 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 184, 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 182710415, 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 1871710, 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-04-21T08:57:39.805 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 3396, 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 21, 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 3, 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 3396, 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 3396, 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 8.780793582, 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.002585628 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.806 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-04-21T08:57:39.807 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-04-21T08:57:39.807 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 3396, 2024-04-21T08:57:39.807 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 3397 2024-04-21T08:57:39.807 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.807 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 3396, 2024-04-21T08:57:39.807 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 3396, 2024-04-21T08:57:39.807 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 0, 2024-04-21T08:57:39.807 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-04-21T08:57:39.807 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-04-21T08:57:39.807 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 2179, 2024-04-21T08:57:39.807 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 0, 2024-04-21T08:57:39.807 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 1215, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 0, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 0, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 0, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 0, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 1, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 0, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:57:39.808 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-04-21T08:57:39.809 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-04-21T08:57:39.810 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-04-21T08:57:39.810 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-04-21T08:57:39.810 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-04-21T08:57:39.810 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-04-21T08:57:39.810 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-04-21T08:57:39.810 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 0, 2024-04-21T08:57:39.810 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-04-21T08:57:39.810 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-04-21T08:57:39.810 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-04-21T08:57:39.810 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-04-21T08:57:39.810 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 103, 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 14, 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 11, 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 11, 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 0, 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 0, 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 2, 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 0 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "oft": { 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_objs": 0, 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:57:39.811 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_updates": 0, 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_removes": 0 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: "osd-slow-ops": { 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_ops_count": 0 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: "purge_queue": { 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops": 0, 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing": 0, 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_high_water": 0, 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed_ops": 0, 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed": 0, 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_item_in_journal": 0 2024-04-21T08:57:39.812 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 3673, 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 719685934, 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 3673, 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 3673, 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 719685934, 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:39.813 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_bytes": { 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:39.814 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 3396, 2024-04-21T08:57:39.815 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 717306642, 2024-04-21T08:57:39.815 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 3396, 2024-04-21T08:57:39.815 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 717306642, 2024-04-21T08:57:39.815 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:39.815 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.815 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.815 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.815 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:39.815 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.815 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_ops": { 2024-04-21T08:57:39.815 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 1024, 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 3396, 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 3396, 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 3396, 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 3396, 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.816 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-21T08:57:39.817 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 0, 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 0, 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle-0x56013cde06e0": { 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-21T08:57:39.818 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:39.819 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:39.819 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-21T08:57:39.819 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-21T08:57:39.819 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 0, 2024-04-21T08:57:39.819 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 0, 2024-04-21T08:57:39.819 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:39.819 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:39.819 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:39.819 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:39.819 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:39.819 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:39.819 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-21T08:57:40.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:40 smithi134 ceph-mon[31772]: pgmap v331: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-21T08:57:40.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:40 smithi082 ceph-mon[27967]: pgmap v331: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-21T08:57:40.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:40 smithi106 ceph-mon[31072]: pgmap v331: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-21T08:57:41.024 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-21T08:57:41.024 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-21T08:57:41.024 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-21T08:57:41.024 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:57:41.024 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:57:41.025 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-21T08:57:41.025 DEBUG:tasks.check_counter:Getting stats from h 2024-04-21T08:57:41.025 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-21T08:57:41.291 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:57:42.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:42 smithi106 ceph-mon[31072]: pgmap v332: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:42.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:42 smithi134 ceph-mon[31772]: pgmap v332: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:42.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:42 smithi082 ceph-mon[27967]: pgmap v332: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:43.084 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-21T08:57:43.084 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:57:43.084 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 106965, 2024-04-21T08:57:43.084 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 99642, 2024-04-21T08:57:43.084 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 36641247, 2024-04-21T08:57:43.084 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 310759817, 2024-04-21T08:57:43.084 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 17, 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 5, 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 7.564306782, 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 2.912002070, 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 2.000858880, 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 2.437665760, 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 99644, 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 23.019590162, 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000231018 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 34996, 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.005059494, 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000144 2024-04-21T08:57:43.085 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 5152 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 323192, 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 481153, 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 116900306, 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 491518214, 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 9, 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 8, 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 20.252404729, 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 8.990148844, 2024-04-21T08:57:43.086 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 84.298319077, 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 2.125832258, 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 481153, 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 46.504216003, 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000096651 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 605785, 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.093462560, 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000154 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 93389888, 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 229031344 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.087 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 56439, 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 53365, 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 23382392, 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 244047511, 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 41, 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 7, 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 5.370350204, 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 2.390379186, 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 1.225205327, 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 1.632282966, 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 53358, 2024-04-21T08:57:43.088 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 17.126264263, 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000320969 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 26155, 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.004138655, 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000158 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 4784, 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 137296 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 1, 2024-04-21T08:57:43.089 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-04-21T08:57:43.090 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.090 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-MDSRank": { 2024-04-21T08:57:43.090 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-04-21T08:57:43.090 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-04-21T08:57:43.090 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 42733, 2024-04-21T08:57:43.090 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 231.663647837, 2024-04-21T08:57:43.090 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.005421188 2024-04-21T08:57:43.090 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:43.090 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.090 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-PurgeQueue": { 2024-04-21T08:57:43.090 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-04-21T08:57:43.090 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 63571, 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 25.219843924, 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000396719 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": { 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: "request": 152452, 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: "reply": 145284, 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: "reply_latency": { 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 145284, 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 11716.079840226, 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.080642602 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_reply": 0, 2024-04-21T08:57:43.091 INFO:teuthology.orchestra.run.smithi106.stdout: "forward": 7168, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_complete": 55, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_keys": 0, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_commit": 1951, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_split": 128, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_merge": 10, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes": 71101, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_top": 20542, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_bottom": 50508, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pin_tail": 51, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pinned": 41702, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_expired": 5753, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_with_caps": 0, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": 0, 2024-04-21T08:57:43.092 INFO:teuthology.orchestra.run.smithi106.stdout: "subtrees": 24, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse": 231110, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_hit": 203343, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_forward": 7124, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_discover": 151, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_dir_fetch": 1, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_remote_ino": 0, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_lock": 204, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "load_cent": 0, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "q": 0, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "exported": 26, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "exported_inodes": 3252, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "imported": 145, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "imported_inodes": 10123, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_dir_fetch": 0, 2024-04-21T08:57:43.093 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_backtrace_fetch": 82658, 2024-04-21T08:57:43.094 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_peer_discover": 0, 2024-04-21T08:57:43.094 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rfiles": 4, 2024-04-21T08:57:43.094 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rbytes": 234, 2024-04-21T08:57:43.094 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rsnaps": 6, 2024-04-21T08:57:43.094 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_fetch": 82658, 2024-04-21T08:57:43.094 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_set_tag": 0, 2024-04-21T08:57:43.094 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:57:43.094 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:57:43.094 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_inodes": 643, 2024-04-21T08:57:43.094 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:57:43.094 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dirfrag_rstats": 643, 2024-04-21T08:57:43.094 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_file_inodes": 82015, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_inode_file_caps": 42, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_revoke": 39285, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_grant": 86103, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_trunc": 6, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps": 93960, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps_dirty": 42227, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_cap_release": 6038, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "process_request_cap_release": 71836 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_cache": { 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update": 0, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update_receipt": 0, 2024-04-21T08:57:43.095 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_try_discover": 0, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_send_discover": 152, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_handle_discover": 55, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays": 151, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_delayed": 0, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_created": 34378, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_enqueued": 34227, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_reintegrated": 0, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_migrated": 0, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_processing": 0, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_started": 0, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_completed": 0, 2024-04-21T08:57:43.096 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_quiesce_path": 1, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_quiesce_inode": 9999, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_exportdir": 26, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_flush": 0, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragmentdir": 140, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragstats": 0, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_inodestats": 0 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_log": { 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "evlrg": 0, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "evadd": 206977, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "evex": 120503, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "evtrm": 110611, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "ev": 96366, 2024-04-21T08:57:43.097 INFO:teuthology.orchestra.run.smithi106.stdout: "evexg": 0, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "evexd": 9892, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "segadd": 296, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "segex": 168, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "segtrm": 155, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "seg": 141, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "segmjr": 8, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "segexg": 0, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "segexd": 13, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "expos": 345668143, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "wrpos": 721338010, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "rdpos": 0, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "jlat": { 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 7762, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 755.491801329, 2024-04-21T08:57:43.098 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.097332105 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: "replayed": 0 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_mem": { 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 40868, 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: "ino+": 76973, 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: "ino-": 36105, 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: "dir": 1129, 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: "dir+": 1377, 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: "dir-": 248, 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: "dn": 71101, 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: "dn+": 128068, 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: "dn-": 56967, 2024-04-21T08:57:43.099 INFO:teuthology.orchestra.run.smithi106.stdout: "cap": 0, 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "cap+": 40178, 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "cap-": 40178, 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "rss": 289928, 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "heap": 223608 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_server": { 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_client_request": 161336, 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_server_request": 11318, 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_request": 152452, 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_session": 54, 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_peer_request": 24055, 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "req_create_latency": { 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 35617, 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1012.870490227, 2024-04-21T08:57:43.100 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.028437838 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getattr_latency": { 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 8240, 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1253.633711405, 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.152140013 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getfilelock_latency": { 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: "req_link_latency": { 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.101 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookup_latency": { 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 34863, 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 8271.260282644, 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.237250388 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookuphash_latency": { 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupino_latency": { 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.102 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupname_latency": { 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupparent_latency": { 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupsnap_latency": { 2024-04-21T08:57:43.103 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lssnap_latency": { 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mkdir_latency": { 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 151, 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 3.894422979, 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.025790880 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.104 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mknod_latency": { 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mksnap_latency": { 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: "req_open_latency": { 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 8639, 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 227.569874800, 2024-04-21T08:57:43.105 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.026342154 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: "req_readdir_latency": { 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 998, 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1.923618554, 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001927473 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rename_latency": { 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 35400, 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 878.710795149, 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.024822338 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: "req_renamesnap_latency": { 2024-04-21T08:57:43.106 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: "req_snapdiff_latency": { 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmdir_latency": { 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 155, 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 22.343837027, 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.144153787 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.107 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmsnap_latency": { 2024-04-21T08:57:43.108 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.108 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.108 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.108 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.108 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmxattr_latency": { 2024-04-21T08:57:43.108 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.108 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.108 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.108 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.108 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setattr_latency": { 2024-04-21T08:57:43.108 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 6, 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.797749371, 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.132958228 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setdirlayout_latency": { 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setfilelock_latency": { 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setlayout_latency": { 2024-04-21T08:57:43.109 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.110 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.110 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.110 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.110 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setxattr_latency": { 2024-04-21T08:57:43.110 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.110 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.110 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.110 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.110 INFO:teuthology.orchestra.run.smithi106.stdout: "req_symlink_latency": { 2024-04-21T08:57:43.110 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2, 2024-04-21T08:57:43.110 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.001337297, 2024-04-21T08:57:43.110 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000668648 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: "req_unlink_latency": { 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 21213, 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 43.073720773, 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.002030534 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getvxattr_latency": { 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.111 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_sessions": { 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "session_count": 2, 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "session_add": 5, 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "session_remove": 3, 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_open": 2, 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_stale": 0, 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "total_load": 30642, 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "average_load": 15321, 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "avg_session_uptime": 604, 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "mdthresh_evicted": 0 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:57:43.112 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-04-21T08:57:43.113 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-04-21T08:57:43.114 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 2724475, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 33042, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 9592, 2024-04-21T08:57:43.115 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 184, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 147784537, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 1425401, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-04-21T08:57:43.116 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 183984, 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 776128420, 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 33, 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 183984, 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 183984, 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 2984.918802930, 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.016223795 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 183984, 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 219186 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 183984, 2024-04-21T08:57:43.117 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 83406, 2024-04-21T08:57:43.118 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 100578, 2024-04-21T08:57:43.118 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-04-21T08:57:43.118 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-04-21T08:57:43.118 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 779, 2024-04-21T08:57:43.118 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 20557, 2024-04-21T08:57:43.118 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 693, 2024-04-21T08:57:43.118 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 8455, 2024-04-21T08:57:43.118 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 300, 2024-04-21T08:57:43.118 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-04-21T08:57:43.118 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-04-21T08:57:43.118 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 125, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 68812, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 82665, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 30895, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-04-21T08:57:43.119 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 6, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-04-21T08:57:43.120 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 103, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 14, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 12, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 12, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 0, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 4569, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 110, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 1220 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "oft": { 2024-04-21T08:57:43.121 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_objs": 1, 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_updates": 0, 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_removes": 0 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "osd-slow-ops": { 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_ops_count": 10126 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "purge_queue": { 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops": 0, 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops_high_water": 17, 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing": 0, 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_high_water": 5, 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed_ops": 68464, 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed": 34227, 2024-04-21T08:57:43.122 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_item_in_journal": 0 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 486592, 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 121365500, 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 486592, 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 486592, 2024-04-21T08:57:43.123 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 121365500, 2024-04-21T08:57:43.124 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:43.124 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.124 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.124 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.124 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:43.124 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.124 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_bytes": { 2024-04-21T08:57:43.124 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 183984, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 780432539, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 183984, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 780432539, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.125 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_ops": { 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 1024, 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 183984, 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 183984, 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 183984, 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 183984, 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:43.126 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 34227, 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 4304999, 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 34227, 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-21T08:57:43.127 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 693, 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 4304999, 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle-0x5581721fa500": { 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 206977, 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 717143706, 2024-04-21T08:57:43.128 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:43.129 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 206977, 2024-04-21T08:57:43.129 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-21T08:57:43.129 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-21T08:57:43.129 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 7762, 2024-04-21T08:57:43.129 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 717143706, 2024-04-21T08:57:43.129 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:43.129 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:43.129 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:43.129 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:43.129 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:43.129 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:43.129 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-21T08:57:43.887 INFO:tasks.check_counter:Daemon mds.h mds.exported=26 2024-04-21T08:57:43.887 INFO:tasks.check_counter:Daemon mds.h mds.imported=145 2024-04-21T08:57:43.887 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-21T08:57:43.887 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:57:43.887 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:57:43.887 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=128 2024-04-21T08:57:43.887 DEBUG:tasks.check_counter:Getting stats from k 2024-04-21T08:57:43.887 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-21T08:57:44.126 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:57:44.471 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:44 smithi106 ceph-mon[31072]: pgmap v333: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:44.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:44 smithi134 ceph-mon[31772]: pgmap v333: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:44.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:44 smithi082 ceph-mon[27967]: pgmap v333: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:46.144 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-21T08:57:46.144 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:57:46.144 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 28359, 2024-04-21T08:57:46.144 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 36501, 2024-04-21T08:57:46.144 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 12299007, 2024-04-21T08:57:46.144 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 29014702, 2024-04-21T08:57:46.144 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 33, 2024-04-21T08:57:46.144 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 8, 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 1.728598911, 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.784435621, 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 1.412035033, 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.054513273, 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 36500, 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1.536022151, 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000042082 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 54377, 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.012845579, 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000236 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 11462688, 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 21057936 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 3552, 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 3747, 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 929896, 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 1914438, 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 20, 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 6, 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.238331151, 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.072933377, 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.060432889, 2024-04-21T08:57:46.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.077969012, 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 3744, 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.205902751, 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000054995 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 70, 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000022815, 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000325 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 3856, 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 461472 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:57:46.147 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 12025, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 16021, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 4060242, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 19553731, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 11, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 7, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.948034932, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.403315386, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.463857287, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.108597380, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 16020, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.687253590, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000042899 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 19070, 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.004102125, 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000215 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 1, 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-MDSRank": { 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-04-21T08:57:46.149 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 4032, 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 6.404525112, 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001588423 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-PurgeQueue": { 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 3917, 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.453363587, 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000115742 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": { 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: "request": 14843, 2024-04-21T08:57:46.150 INFO:teuthology.orchestra.run.smithi106.stdout: "reply": 14834, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "reply_latency": { 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 14834, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 129.766908756, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.008747937 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_reply": 0, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "forward": 9, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_complete": 8, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_keys": 0, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_commit": 47, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_split": 4, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_merge": 0, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes": 12, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_top": 1, 2024-04-21T08:57:46.151 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_bottom": 1, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pin_tail": 10, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pinned": 10, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_expired": 3553, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_with_caps": 1, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": 1, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "subtrees": 3, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse": 32086, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_hit": 25370, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_forward": 7, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_discover": 17, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_remote_ino": 0, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_lock": 458, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "load_cent": 0, 2024-04-21T08:57:46.152 INFO:teuthology.orchestra.run.smithi106.stdout: "q": 0, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "exported": 40, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "exported_inodes": 3090, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "imported": 12, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "imported_inodes": 1034, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_dir_fetch": 0, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_backtrace_fetch": 3834, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_peer_discover": 0, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rfiles": 4, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rbytes": 234, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rsnaps": 6, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_fetch": 3834, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_set_tag": 0, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:57:46.153 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_file_inodes": 3834, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_inode_file_caps": 64, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_revoke": 2373, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_grant": 4995, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps": 5287, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps_dirty": 2134, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_cap_release": 441, 2024-04-21T08:57:46.154 INFO:teuthology.orchestra.run.smithi106.stdout: "process_request_cap_release": 7345 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_cache": { 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update": 3, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update_receipt": 11, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_try_discover": 7, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_send_discover": 27, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_handle_discover": 21, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays": 0, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_delayed": 0, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_created": 1968, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_enqueued": 1968, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_reintegrated": 0, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_migrated": 0, 2024-04-21T08:57:46.155 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_processing": 0, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_started": 0, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_completed": 0, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_quiesce_path": 1, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_quiesce_inode": 515, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_exportdir": 40, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_flush": 0, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragmentdir": 4, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragstats": 0, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_inodestats": 0 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.156 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_log": { 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "evlrg": 0, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "evadd": 10432, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "evex": 0, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "evtrm": 0, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "ev": 10432, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "evexg": 0, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "evexd": 0, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "segadd": 14, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "segex": 0, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "segtrm": 0, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "seg": 14, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "segmjr": 2, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "segexg": 0, 2024-04-21T08:57:46.157 INFO:teuthology.orchestra.run.smithi106.stdout: "segexd": 0, 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: "expos": 4194304, 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: "wrpos": 23299666, 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: "rdpos": 0, 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: "jlat": { 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 894, 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 12.818686083, 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.014338575 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: "replayed": 0 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_mem": { 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 16, 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: "ino+": 3954, 2024-04-21T08:57:46.158 INFO:teuthology.orchestra.run.smithi106.stdout: "ino-": 3938, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "dir": 15, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "dir+": 86, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "dir-": 71, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "dn": 12, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "dn+": 5617, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "dn-": 5605, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "cap": 1, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "cap+": 2949, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "cap-": 2948, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "rss": 76812, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "heap": 223608 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_server": { 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_client_request": 19959, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_server_request": 335, 2024-04-21T08:57:46.159 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_request": 14843, 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_session": 17, 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_peer_request": 4565, 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "req_create_latency": { 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1519, 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 8.442109736, 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.005557675 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getattr_latency": { 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 3, 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000908579, 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000302859 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getfilelock_latency": { 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.160 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: "req_link_latency": { 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookup_latency": { 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 9571, 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 105.762597981, 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.011050318 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookuphash_latency": { 2024-04-21T08:57:46.161 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupino_latency": { 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupname_latency": { 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.162 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.163 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupparent_latency": { 2024-04-21T08:57:46.163 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.163 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.163 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.163 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.163 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupsnap_latency": { 2024-04-21T08:57:46.163 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.163 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.163 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.163 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.163 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lssnap_latency": { 2024-04-21T08:57:46.164 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.164 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.164 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.164 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.164 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mkdir_latency": { 2024-04-21T08:57:46.164 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.164 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.164 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.164 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.164 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mknod_latency": { 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mksnap_latency": { 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: "req_open_latency": { 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1419, 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.824052363, 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000580727 2024-04-21T08:57:46.165 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: "req_readdir_latency": { 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 40, 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.042645637, 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001066140 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rename_latency": { 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1519, 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 13.120390860, 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.008637518 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: "req_renamesnap_latency": { 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.166 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: "req_snapdiff_latency": { 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmdir_latency": { 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmsnap_latency": { 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.167 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmxattr_latency": { 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setattr_latency": { 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setdirlayout_latency": { 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.168 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setfilelock_latency": { 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setlayout_latency": { 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setxattr_latency": { 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.169 INFO:teuthology.orchestra.run.smithi106.stdout: "req_symlink_latency": { 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: "req_unlink_latency": { 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 763, 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1.574203600, 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.002063176 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getvxattr_latency": { 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.170 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_sessions": { 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "session_count": 2, 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "session_add": 2, 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "session_remove": 0, 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_open": 2, 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_stale": 0, 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "total_load": 2415, 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "average_load": 1207, 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "avg_session_uptime": 584, 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "mdthresh_evicted": 0 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-04-21T08:57:46.171 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:57:46.172 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:57:46.173 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 296023, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 113, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:57:46.174 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 9592, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 184, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 97016, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 419, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.175 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 9199, 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 20318063, 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 33, 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 9199, 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 9199, 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 72.013305566, 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.007828384 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-04-21T08:57:46.176 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 9199, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 9445 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 9199, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 3948, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 5251, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 0, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 10, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 106, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 1000, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 84, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-04-21T08:57:46.177 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 4, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 3952, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 3834, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 20, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-04-21T08:57:46.178 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-04-21T08:57:46.179 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 0, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 103, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 14, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 12, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 12, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 0, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 341, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 16, 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 78 2024-04-21T08:57:46.180 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "oft": { 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_objs": 1, 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_updates": 0, 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_removes": 0 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "osd-slow-ops": { 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_ops_count": 568 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "purge_queue": { 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops": 0, 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops_high_water": 10, 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing": 0, 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_high_water": 5, 2024-04-21T08:57:46.181 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed_ops": 3936, 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed": 1968, 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_item_in_journal": 0 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 43932, 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 12086472, 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 43932, 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-21T08:57:46.182 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 43932, 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 12086472, 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_bytes": { 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-21T08:57:46.183 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 9199, 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 20559607, 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 9199, 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 20559607, 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_ops": { 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 1024, 2024-04-21T08:57:46.184 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 9199, 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 9199, 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 9199, 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 9199, 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:46.185 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 1968, 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 241672, 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 1968, 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 106, 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 241672, 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:46.186 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle-0x55b3973cc500": { 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 10432, 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 19105362, 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 10432, 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-21T08:57:46.187 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 894, 2024-04-21T08:57:46.188 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 19105362, 2024-04-21T08:57:46.188 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-21T08:57:46.188 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-21T08:57:46.188 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-21T08:57:46.188 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-21T08:57:46.188 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:46.188 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-21T08:57:46.188 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-21T08:57:46.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:46 smithi134 ceph-mon[31772]: pgmap v334: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:46.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:46 smithi082 ceph-mon[27967]: pgmap v334: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:46.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:46 smithi106 ceph-mon[31072]: pgmap v334: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-21T08:57:46.987 INFO:tasks.check_counter:Daemon mds.k mds.exported=40 2024-04-21T08:57:46.988 INFO:tasks.check_counter:Daemon mds.k mds.imported=12 2024-04-21T08:57:46.988 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-04-21T08:57:46.988 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:57:46.988 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:57:46.988 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=4 2024-04-21T08:57:46.988 DEBUG:tasks.check_counter:Getting stats from c 2024-04-21T08:57:46.988 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-21T08:57:47.239 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:57:48.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:48 smithi134 ceph-mon[31772]: pgmap v335: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-04-21T08:57:48.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:48 smithi082 ceph-mon[27967]: pgmap v335: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-04-21T08:57:48.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:48 smithi106 ceph-mon[31072]: pgmap v335: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-04-21T08:57:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-21T08:57:49.202 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:57:49.202 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1322, 2024-04-21T08:57:49.202 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1324, 2024-04-21T08:57:49.202 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 4595709, 2024-04-21T08:57:49.202 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 420406, 2024-04-21T08:57:49.202 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.131457281, 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.037875335, 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.036522409, 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.044787053, 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1322, 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.058387425, 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000044165 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.203 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 609, 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 569, 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 2388871, 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 242738, 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.080561465, 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.021458509, 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.051175403, 2024-04-21T08:57:49.204 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.017054142, 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 569, 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.035139262, 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000061756 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1796080, 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 138368 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 852, 2024-04-21T08:57:49.205 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1017, 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 8602739, 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 719997, 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.075942197, 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.028371098, 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.019891634, 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.020564446, 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1014, 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.051852319, 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000051136 2024-04-21T08:57:49.206 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 449696 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-21T08:57:49.207 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4180, 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.135049020, 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000032308 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:49.208 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-04-21T08:57:49.209 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 1151, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 791, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 360, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 21, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 2, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-04-21T08:57:49.210 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-21T08:57:49.211 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 234, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:57:49.212 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-21T08:57:49.213 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-21T08:57:49.214 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:57:49.214 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-04-21T08:57:49.214 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-04-21T08:57:49.214 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-21T08:57:49.214 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-21T08:57:49.214 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-21T08:57:49.214 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:57:49.214 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:57:49.214 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-21T08:57:49.214 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-21T08:57:49.214 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:57:49.214 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 6765, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-21T08:57:49.215 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 28, 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 17209764, 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 17285079, 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 17284943, 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.216 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 6765 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 27, 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 2514, 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 2487, 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 20, 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 63, 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 43, 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 1151, 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 3735, 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 2584, 2024-04-21T08:57:49.217 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 57820, 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 215416 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.218 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.219 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.220 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-21T08:57:49.221 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.222 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.222 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.222 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.222 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-21T08:57:49.222 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.222 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.223 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.224 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.225 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.226 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.228 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 6, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 4, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 4, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 2931, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:57:49.229 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:57:49.230 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:57:49.231 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 198919, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 74, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9592, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 184, 2024-04-21T08:57:49.232 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 933878, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 5467, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 2500, 2024-04-21T08:57:49.233 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 26, 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 4, 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 2500, 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2500, 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 4.803886105, 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001921554 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2500, 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2501 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 2500, 2024-04-21T08:57:49.234 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2500, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 1530, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 967, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-21T08:57:49.235 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 2, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-21T08:57:49.236 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 103, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 14, 2024-04-21T08:57:49.237 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 6, 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 6, 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-04-21T08:57:49.238 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2779, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 15320147, 2024-04-21T08:57:49.239 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2779, 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2779, 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 15320147, 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:49.240 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2500, 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 13093219, 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2500, 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 13093219, 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:49.241 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2500, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2500, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2500, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2500, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.242 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:49.243 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x560be04146e0": { 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-21T08:57:49.244 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-21T08:57:49.245 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:49.245 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:49.245 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:49.245 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:49.245 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:49.245 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-21T08:57:49.939 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-21T08:57:49.939 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-21T08:57:49.939 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-21T08:57:49.939 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:57:49.939 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:57:49.940 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-21T08:57:49.940 DEBUG:tasks.check_counter:Getting stats from f 2024-04-21T08:57:49.940 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-21T08:57:50.191 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:57:50.446 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:50 smithi134 ceph-mon[31772]: pgmap v336: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 op/s 2024-04-21T08:57:50.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:50 smithi082 ceph-mon[27967]: pgmap v336: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 op/s 2024-04-21T08:57:50.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:50 smithi106 ceph-mon[31072]: pgmap v336: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 op/s 2024-04-21T08:57:51.957 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-21T08:57:51.957 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:57:51.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 2061, 2024-04-21T08:57:51.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2061, 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 12210866, 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 654581, 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.183073926, 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.056258313, 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.049809578, 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.063012170, 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2060, 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.078088678, 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000037907 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 264, 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 431, 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 64836, 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 536658, 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2024-04-21T08:57:51.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.032983542, 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.012692776, 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.007092869, 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.009104645, 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 428, 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.022573546, 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000052741 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 453280 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 279, 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 241, 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1793740, 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 139358, 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 3, 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.047244765, 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.013830923, 2024-04-21T08:57:51.961 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.038306649, 2024-04-21T08:57:51.962 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.005276809, 2024-04-21T08:57:51.962 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:51.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 240, 2024-04-21T08:57:51.962 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.012182006, 2024-04-21T08:57:51.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000050758 2024-04-21T08:57:51.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.962 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:51.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.962 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1793472, 2024-04-21T08:57:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 139040 2024-04-21T08:57:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-21T08:57:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-21T08:57:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-21T08:57:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-21T08:57:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-21T08:57:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-21T08:57:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3967, 2024-04-21T08:57:51.963 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.104201070, 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000026266 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-21T08:57:51.964 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 1227, 2024-04-21T08:57:51.965 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 1227, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 1227, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-21T08:57:51.966 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-21T08:57:51.967 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:57:51.968 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-04-21T08:57:51.969 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-21T08:57:51.970 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 6916, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 9, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-04-21T08:57:51.971 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-21T08:57:51.972 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-21T08:57:51.972 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-04-21T08:57:51.972 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 15811637, 2024-04-21T08:57:51.972 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 15807657, 2024-04-21T08:57:51.972 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-21T08:57:51.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.972 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.972 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 6916 2024-04-21T08:57:51.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.973 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-21T08:57:51.973 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 11, 2024-04-21T08:57:51.973 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 2638, 2024-04-21T08:57:51.973 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 2627, 2024-04-21T08:57:51.973 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 9, 2024-04-21T08:57:51.973 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 86, 2024-04-21T08:57:51.973 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 77, 2024-04-21T08:57:51.973 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 1227, 2024-04-21T08:57:51.973 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 4226, 2024-04-21T08:57:51.973 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 2999, 2024-04-21T08:57:51.973 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 57316, 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 215416 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.974 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-21T08:57:51.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-21T08:57:51.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-21T08:57:51.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.978 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.978 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-21T08:57:51.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.978 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.978 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-21T08:57:51.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.979 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.979 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-21T08:57:51.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.979 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.981 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.982 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.982 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-21T08:57:51.982 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.982 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.982 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.982 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-21T08:57:51.982 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.983 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.983 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.983 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-21T08:57:51.983 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.983 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.983 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.983 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.983 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-21T08:57:51.983 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.983 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.983 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.984 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-21T08:57:51.984 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.984 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.984 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.984 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-21T08:57:51.984 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.984 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.984 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.984 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.985 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-21T08:57:51.985 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.985 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.985 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.985 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-21T08:57:51.985 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.985 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.985 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.985 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-21T08:57:51.986 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.986 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.986 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.986 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-21T08:57:51.986 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.986 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.986 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.986 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:57:51.986 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:57:51.987 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-21T08:57:51.987 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:51.987 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:51.987 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:51.987 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:51.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.987 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-21T08:57:51.987 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-04-21T08:57:51.987 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 4, 2024-04-21T08:57:51.987 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 3, 2024-04-21T08:57:51.987 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 4, 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 5216, 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:57:51.988 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:57:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:57:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:57:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:57:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:57:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:57:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:57:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:57:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:57:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:57:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-21T08:57:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:57:51.989 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-21T08:57:51.990 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:57:51.990 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:57:51.990 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:57:51.990 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:57:51.990 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:57:51.990 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:57:51.990 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:57:51.990 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-21T08:57:51.990 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:57:51.990 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:57:51.990 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:57:51.991 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-21T08:57:51.991 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-21T08:57:51.991 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-21T08:57:51.991 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:57:51.991 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:57:51.991 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:57:51.991 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:57:51.991 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 190207, 2024-04-21T08:57:51.991 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 53, 2024-04-21T08:57:51.991 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:57:51.991 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-21T08:57:51.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9592, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 184, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 913904, 2024-04-21T08:57:51.992 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 5040, 2024-04-21T08:57:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-21T08:57:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-21T08:57:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-21T08:57:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-21T08:57:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-21T08:57:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-21T08:57:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-21T08:57:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 2320, 2024-04-21T08:57:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2024-04-21T08:57:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2024-04-21T08:57:51.993 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 2320, 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2320, 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 3.214923785, 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001385743 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2320, 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2321 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 2320, 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2320, 2024-04-21T08:57:51.994 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-04-21T08:57:51.995 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-21T08:57:51.995 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-21T08:57:51.995 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 1419, 2024-04-21T08:57:51.995 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-04-21T08:57:51.995 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 899, 2024-04-21T08:57:51.995 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-04-21T08:57:51.995 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-04-21T08:57:51.995 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-21T08:57:51.995 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-21T08:57:51.995 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-21T08:57:51.995 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-04-21T08:57:51.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-04-21T08:57:51.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-21T08:57:51.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-21T08:57:51.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-21T08:57:51.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2024-04-21T08:57:51.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-04-21T08:57:51.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:57:51.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-21T08:57:51.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:57:51.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-21T08:57:51.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-21T08:57:51.996 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-21T08:57:51.997 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:57:51.997 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-21T08:57:51.997 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:57:51.997 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-21T08:57:51.997 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-21T08:57:51.997 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-21T08:57:51.997 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-21T08:57:51.997 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-21T08:57:51.997 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-21T08:57:51.997 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-21T08:57:51.997 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-21T08:57:51.997 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-21T08:57:51.998 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-21T08:57:51.998 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-21T08:57:51.998 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-21T08:57:51.998 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-21T08:57:51.998 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-21T08:57:51.998 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-21T08:57:51.998 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-21T08:57:51.998 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-21T08:57:51.998 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 103, 2024-04-21T08:57:51.998 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-21T08:57:51.998 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 14, 2024-04-21T08:57:51.998 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 5, 2024-04-21T08:57:51.999 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 5, 2024-04-21T08:57:51.999 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-21T08:57:51.999 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-21T08:57:51.999 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-04-21T08:57:51.999 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-04-21T08:57:51.999 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-04-21T08:57:51.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:51.999 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-21T08:57:51.999 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-04-21T08:57:51.999 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:57:51.999 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-21T08:57:51.999 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-21T08:57:52.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:52.000 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-21T08:57:52.000 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-04-21T08:57:52.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:52.000 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-21T08:57:52.000 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-21T08:57:52.000 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:57:52.000 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-21T08:57:52.000 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-04-21T08:57:52.000 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-04-21T08:57:52.000 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-04-21T08:57:52.000 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-21T08:57:52.001 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:52.001 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:57:52.001 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:52.001 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-21T08:57:52.001 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:52.001 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2600, 2024-04-21T08:57:52.001 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 13818322, 2024-04-21T08:57:52.001 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:52.001 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2600, 2024-04-21T08:57:52.001 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-21T08:57:52.001 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-21T08:57:52.001 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2600, 2024-04-21T08:57:52.002 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 13818322, 2024-04-21T08:57:52.002 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:52.002 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:52.002 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:52.002 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:52.002 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:52.002 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:52.002 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-21T08:57:52.002 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:52.002 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-21T08:57:52.002 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:52.003 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-21T08:57:52.003 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-21T08:57:52.003 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:52.003 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:52.003 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2320, 2024-04-21T08:57:52.003 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 11619608, 2024-04-21T08:57:52.003 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2320, 2024-04-21T08:57:52.003 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 11619608, 2024-04-21T08:57:52.003 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:52.003 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:52.003 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:52.004 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:52.004 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:52.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:52.004 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-21T08:57:52.004 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:52.004 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-21T08:57:52.004 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:52.004 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-21T08:57:52.004 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-21T08:57:52.004 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:52.004 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:52.004 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2320, 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2320, 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2320, 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2320, 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-21T08:57:52.005 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:52.006 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-21T08:57:52.006 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-21T08:57:52.006 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:52.006 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:52.006 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-21T08:57:52.006 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-21T08:57:52.006 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-21T08:57:52.006 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-21T08:57:52.006 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:52.006 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:52.006 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:52.007 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:52.007 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:52.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:52.007 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55fe26c6c6e0": { 2024-04-21T08:57:52.007 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:52.007 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-21T08:57:52.007 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:52.007 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-21T08:57:52.007 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-21T08:57:52.007 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:52.007 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:52.007 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-21T08:57:52.008 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-21T08:57:52.008 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-21T08:57:52.008 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-21T08:57:52.008 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:52.008 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:52.008 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:52.008 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:52.008 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:52.008 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:52.008 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-21T08:57:52.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:52 smithi134 ceph-mon[31772]: pgmap v337: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:57:52.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:52 smithi082 ceph-mon[27967]: pgmap v337: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:57:52.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:52 smithi106 ceph-mon[31072]: pgmap v337: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:57:52.835 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-21T08:57:52.835 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-21T08:57:52.835 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-21T08:57:52.835 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:57:52.835 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:57:52.835 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-21T08:57:52.835 DEBUG:tasks.check_counter:Getting stats from i 2024-04-21T08:57:52.836 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-21T08:57:53.075 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:57:54.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:54 smithi134 ceph-mon[31772]: pgmap v338: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-21T08:57:54.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:54 smithi082 ceph-mon[27967]: pgmap v338: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-21T08:57:54.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:54 smithi106 ceph-mon[31072]: pgmap v338: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2024-04-21T08:57:55.046 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-21T08:57:55.046 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:57:55.046 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 8613, 2024-04-21T08:57:55.046 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 11866, 2024-04-21T08:57:55.046 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 4953917, 2024-04-21T08:57:55.046 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 17004244, 2024-04-21T08:57:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 16, 2024-04-21T08:57:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-04-21T08:57:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.687065617, 2024-04-21T08:57:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.272564982, 2024-04-21T08:57:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.415105063, 2024-04-21T08:57:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.055157434, 2024-04-21T08:57:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 11865, 2024-04-21T08:57:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.540194264, 2024-04-21T08:57:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000045528 2024-04-21T08:57:55.047 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 16363, 2024-04-21T08:57:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002713452, 2024-04-21T08:57:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000165 2024-04-21T08:57:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1807440, 2024-04-21T08:57:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 153664 2024-04-21T08:57:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:57:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 25312, 2024-04-21T08:57:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 31293, 2024-04-21T08:57:55.048 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 9023893, 2024-04-21T08:57:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 20468140, 2024-04-21T08:57:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 23, 2024-04-21T08:57:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2024-04-21T08:57:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 1.308861103, 2024-04-21T08:57:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.588599358, 2024-04-21T08:57:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 1.044681098, 2024-04-21T08:57:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.036368173, 2024-04-21T08:57:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 31290, 2024-04-21T08:57:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.533841080, 2024-04-21T08:57:55.049 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000049020 2024-04-21T08:57:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 47006, 2024-04-21T08:57:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.006546827, 2024-04-21T08:57:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000139 2024-04-21T08:57:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 8459232, 2024-04-21T08:57:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 19603920 2024-04-21T08:57:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:57:55.050 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 3322, 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 3323, 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1060112, 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 2174769, 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 27, 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 7, 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.211937768, 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.065412405, 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.058321260, 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.072530872, 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3322, 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.159312876, 2024-04-21T08:57:55.051 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000047956 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 25, 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000003213, 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000128 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3920, 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 1424 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-21T08:57:55.052 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-21T08:57:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-21T08:57:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-21T08:57:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3070, 2024-04-21T08:57:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 3.579340102, 2024-04-21T08:57:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001165908 2024-04-21T08:57:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-21T08:57:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-21T08:57:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-21T08:57:55.053 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2511, 2024-04-21T08:57:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.237143926, 2024-04-21T08:57:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000094442 2024-04-21T08:57:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-21T08:57:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 16049, 2024-04-21T08:57:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 16007, 2024-04-21T08:57:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-21T08:57:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 16007, 2024-04-21T08:57:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 262.609622853, 2024-04-21T08:57:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.016405923 2024-04-21T08:57:55.054 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 42, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 24, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 56, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 5, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 19, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 3, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 2, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 14, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 14, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 3005, 2024-04-21T08:57:55.055 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 2, 2024-04-21T08:57:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 2, 2024-04-21T08:57:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 4, 2024-04-21T08:57:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 32471, 2024-04-21T08:57:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 24858, 2024-04-21T08:57:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 41, 2024-04-21T08:57:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 12, 2024-04-21T08:57:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:57:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-21T08:57:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 291, 2024-04-21T08:57:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-04-21T08:57:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-21T08:57:55.056 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 41, 2024-04-21T08:57:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 2474, 2024-04-21T08:57:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 6, 2024-04-21T08:57:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 839, 2024-04-21T08:57:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-21T08:57:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 2678, 2024-04-21T08:57:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-21T08:57:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2024-04-21T08:57:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 234, 2024-04-21T08:57:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-04-21T08:57:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 2678, 2024-04-21T08:57:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-21T08:57:55.057 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:57:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:57:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:57:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:57:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:57:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 2678, 2024-04-21T08:57:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 95, 2024-04-21T08:57:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 2440, 2024-04-21T08:57:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 3284, 2024-04-21T08:57:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:57:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:57:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:57:55.058 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 4165, 2024-04-21T08:57:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 1499, 2024-04-21T08:57:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 130, 2024-04-21T08:57:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 4584 2024-04-21T08:57:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-21T08:57:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 3, 2024-04-21T08:57:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 11, 2024-04-21T08:57:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 7, 2024-04-21T08:57:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 22, 2024-04-21T08:57:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 8, 2024-04-21T08:57:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-21T08:57:55.059 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 1219, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 1219, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 1, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 420, 2024-04-21T08:57:55.060 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 41, 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 5, 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 6916, 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 6916, 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-21T08:57:55.061 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-21T08:57:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 9, 2024-04-21T08:57:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-21T08:57:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-21T08:57:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 9, 2024-04-21T08:57:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-04-21T08:57:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-21T08:57:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-21T08:57:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-04-21T08:57:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 15811637, 2024-04-21T08:57:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-04-21T08:57:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-21T08:57:55.062 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 690, 2024-04-21T08:57:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 7.443345268, 2024-04-21T08:57:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.010787456 2024-04-21T08:57:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-04-21T08:57:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-21T08:57:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 24, 2024-04-21T08:57:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 2779, 2024-04-21T08:57:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 2755, 2024-04-21T08:57:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 23, 2024-04-21T08:57:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 91, 2024-04-21T08:57:55.063 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 68, 2024-04-21T08:57:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 19, 2024-04-21T08:57:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 4406, 2024-04-21T08:57:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 4387, 2024-04-21T08:57:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 2, 2024-04-21T08:57:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 1897, 2024-04-21T08:57:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 1895, 2024-04-21T08:57:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 71900, 2024-04-21T08:57:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 223608 2024-04-21T08:57:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-21T08:57:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 19736, 2024-04-21T08:57:55.064 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 54, 2024-04-21T08:57:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 16049, 2024-04-21T08:57:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 21, 2024-04-21T08:57:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 2641, 2024-04-21T08:57:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-21T08:57:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 834, 2024-04-21T08:57:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 7.875400126, 2024-04-21T08:57:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.009442925 2024-04-21T08:57:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-21T08:57:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2388, 2024-04-21T08:57:55.065 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 19.051355898, 2024-04-21T08:57:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.007977954 2024-04-21T08:57:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-21T08:57:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-21T08:57:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-21T08:57:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 10322, 2024-04-21T08:57:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 227.911468893, 2024-04-21T08:57:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.022080165 2024-04-21T08:57:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-21T08:57:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-21T08:57:55.067 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000289409, 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000289409 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-21T08:57:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.069 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-21T08:57:55.070 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.070 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.070 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.070 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.070 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-21T08:57:55.071 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.071 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.071 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.071 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-21T08:57:55.071 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.071 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.071 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.071 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-21T08:57:55.071 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.072 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.072 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.072 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-21T08:57:55.072 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 793, 2024-04-21T08:57:55.072 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.500162854, 2024-04-21T08:57:55.072 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000630722 2024-04-21T08:57:55.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.072 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-21T08:57:55.072 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 41, 2024-04-21T08:57:55.072 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.068738669, 2024-04-21T08:57:55.072 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001676552 2024-04-21T08:57:55.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.073 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-21T08:57:55.073 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 836, 2024-04-21T08:57:55.073 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 5.373759869, 2024-04-21T08:57:55.073 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.006427942 2024-04-21T08:57:55.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.073 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-21T08:57:55.073 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.073 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.073 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.073 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.073 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-21T08:57:55.074 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.074 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.074 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.074 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-21T08:57:55.074 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.074 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.074 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.074 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.074 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-21T08:57:55.074 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.075 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.076 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-21T08:57:55.076 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.076 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.076 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.076 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-21T08:57:55.076 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.076 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.076 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.076 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-21T08:57:55.076 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.077 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.077 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.077 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-21T08:57:55.077 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.077 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.077 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.077 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.077 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-21T08:57:55.077 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.077 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.077 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.078 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-21T08:57:55.078 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 792, 2024-04-21T08:57:55.078 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.828447135, 2024-04-21T08:57:55.078 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002308645 2024-04-21T08:57:55.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.078 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:57:55.078 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:57:55.078 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-21T08:57:55.078 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.078 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.078 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.079 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:55.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.079 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-21T08:57:55.079 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-04-21T08:57:55.079 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 4, 2024-04-21T08:57:55.079 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 3, 2024-04-21T08:57:55.079 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-04-21T08:57:55.079 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-21T08:57:55.079 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 2617, 2024-04-21T08:57:55.079 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 2617, 2024-04-21T08:57:55.079 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 656, 2024-04-21T08:57:55.079 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:57:55.080 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:57:55.081 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:57:55.081 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:57:55.081 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:57:55.081 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:57:55.081 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-21T08:57:55.081 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:57:55.081 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-21T08:57:55.081 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:57:55.081 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:57:55.081 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:57:55.081 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:57:55.081 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:57:55.082 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:57:55.082 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:57:55.082 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-21T08:57:55.082 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:57:55.082 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:57:55.082 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:57:55.082 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-21T08:57:55.082 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-21T08:57:55.082 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-21T08:57:55.082 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:57:55.082 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:57:55.082 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:57:55.083 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:57:55.083 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 295687, 2024-04-21T08:57:55.083 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 107, 2024-04-21T08:57:55.083 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:57:55.083 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-21T08:57:55.083 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-21T08:57:55.083 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-21T08:57:55.083 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:57:55.083 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-21T08:57:55.083 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:57:55.083 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-21T08:57:55.083 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9592, 2024-04-21T08:57:55.084 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 184, 2024-04-21T08:57:55.084 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:57:55.084 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:57:55.084 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-21T08:57:55.084 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-21T08:57:55.084 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 135549, 2024-04-21T08:57:55.084 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 521, 2024-04-21T08:57:55.084 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-21T08:57:55.084 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-21T08:57:55.084 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-21T08:57:55.084 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-21T08:57:55.084 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.085 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-21T08:57:55.085 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-21T08:57:55.085 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-21T08:57:55.085 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 6291, 2024-04-21T08:57:55.085 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 12871946, 2024-04-21T08:57:55.085 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 28, 2024-04-21T08:57:55.085 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 6291, 2024-04-21T08:57:55.085 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-21T08:57:55.085 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6291, 2024-04-21T08:57:55.085 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 33.999193691, 2024-04-21T08:57:55.085 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.005404418 2024-04-21T08:57:55.085 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.086 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-21T08:57:55.086 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-21T08:57:55.086 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6291, 2024-04-21T08:57:55.086 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 6578 2024-04-21T08:57:55.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.086 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 6291, 2024-04-21T08:57:55.086 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2776, 2024-04-21T08:57:55.086 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 3515, 2024-04-21T08:57:55.086 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-21T08:57:55.086 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-21T08:57:55.086 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-04-21T08:57:55.087 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-04-21T08:57:55.087 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 74, 2024-04-21T08:57:55.087 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 764, 2024-04-21T08:57:55.087 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 69, 2024-04-21T08:57:55.087 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-21T08:57:55.087 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-21T08:57:55.087 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-21T08:57:55.087 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 5, 2024-04-21T08:57:55.087 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 2452, 2024-04-21T08:57:55.087 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-21T08:57:55.087 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-21T08:57:55.087 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-21T08:57:55.088 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 2678, 2024-04-21T08:57:55.088 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-04-21T08:57:55.088 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:57:55.088 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-21T08:57:55.088 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:57:55.088 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-21T08:57:55.088 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-21T08:57:55.088 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-21T08:57:55.088 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:57:55.088 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-21T08:57:55.088 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:57:55.089 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-21T08:57:55.089 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-21T08:57:55.089 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-21T08:57:55.089 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-21T08:57:55.089 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-21T08:57:55.089 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-21T08:57:55.089 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-21T08:57:55.089 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-21T08:57:55.089 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-21T08:57:55.089 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-21T08:57:55.089 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-21T08:57:55.089 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-21T08:57:55.090 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-21T08:57:55.090 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-21T08:57:55.090 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-21T08:57:55.090 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-21T08:57:55.090 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-21T08:57:55.090 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 103, 2024-04-21T08:57:55.090 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-21T08:57:55.090 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 14, 2024-04-21T08:57:55.090 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2024-04-21T08:57:55.090 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2024-04-21T08:57:55.090 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-21T08:57:55.090 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-21T08:57:55.091 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 370, 2024-04-21T08:57:55.091 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 48, 2024-04-21T08:57:55.091 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 88 2024-04-21T08:57:55.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.091 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-21T08:57:55.091 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-04-21T08:57:55.091 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:57:55.091 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-21T08:57:55.091 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-21T08:57:55.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.091 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-21T08:57:55.091 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 441 2024-04-21T08:57:55.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.092 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-21T08:57:55.092 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-21T08:57:55.092 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 10, 2024-04-21T08:57:55.092 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-21T08:57:55.092 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 5, 2024-04-21T08:57:55.092 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 2438, 2024-04-21T08:57:55.092 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 1219, 2024-04-21T08:57:55.092 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-21T08:57:55.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.092 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:57:55.092 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 37243, 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 10548161, 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 37243, 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 37243, 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 10548161, 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.093 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.094 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.094 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:55.094 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.094 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-21T08:57:55.094 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:55.094 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-21T08:57:55.094 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:55.094 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-21T08:57:55.094 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-21T08:57:55.094 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:55.094 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:55.095 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 6291, 2024-04-21T08:57:55.095 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 13026065, 2024-04-21T08:57:55.095 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 6291, 2024-04-21T08:57:55.095 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 13026065, 2024-04-21T08:57:55.095 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:55.095 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.095 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.095 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.095 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:55.095 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.095 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-21T08:57:55.095 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:55.096 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-21T08:57:55.096 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:55.096 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-21T08:57:55.096 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-21T08:57:55.096 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:55.096 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:55.096 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 6291, 2024-04-21T08:57:55.096 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 6291, 2024-04-21T08:57:55.096 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 6291, 2024-04-21T08:57:55.096 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 6291, 2024-04-21T08:57:55.096 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:55.096 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.097 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.097 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.097 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:55.097 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.097 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:57:55.097 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:55.097 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-21T08:57:55.097 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:55.097 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1219, 2024-04-21T08:57:55.097 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 154503, 2024-04-21T08:57:55.097 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:55.097 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1219, 2024-04-21T08:57:55.098 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-21T08:57:55.098 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-21T08:57:55.098 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 74, 2024-04-21T08:57:55.098 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 154503, 2024-04-21T08:57:55.098 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:55.098 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.098 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.098 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.098 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:55.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:55.098 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55824a66c500": { 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 6916, 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 11617333, 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 6916, 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 690, 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 11617333, 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:55.099 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:55.100 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:55.100 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:55.100 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:55.100 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:55.100 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-21T08:57:55.890 INFO:tasks.check_counter:Daemon mds.i mds.exported=41 2024-04-21T08:57:55.890 INFO:tasks.check_counter:Daemon mds.i mds.imported=6 2024-04-21T08:57:55.890 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-21T08:57:55.890 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:57:55.891 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:57:55.891 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=5 2024-04-21T08:57:55.891 DEBUG:tasks.check_counter:Getting stats from l 2024-04-21T08:57:55.891 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-21T08:57:56.131 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:57:56.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:56 smithi134 ceph-mon[31772]: pgmap v339: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:57:56.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:56 smithi082 ceph-mon[27967]: pgmap v339: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:57:56.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:56 smithi106 ceph-mon[31072]: pgmap v339: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:57:57.987 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-21T08:57:57.988 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:57:57.988 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 14338, 2024-04-21T08:57:57.988 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 17460, 2024-04-21T08:57:57.988 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 4319361, 2024-04-21T08:57:57.988 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 15597858, 2024-04-21T08:57:57.988 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 19, 2024-04-21T08:57:57.989 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-04-21T08:57:57.989 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.892245534, 2024-04-21T08:57:57.989 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.346957408, 2024-04-21T08:57:57.989 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.576227978, 2024-04-21T08:57:57.989 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.069186741, 2024-04-21T08:57:57.989 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 17458, 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.626167905, 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000035867 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 25802, 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.004346806, 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000168 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1856, 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 1952 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.990 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:57:57.991 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 28417, 2024-04-21T08:57:57.991 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 35160, 2024-04-21T08:57:57.991 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 9989722, 2024-04-21T08:57:57.991 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 26911096, 2024-04-21T08:57:57.991 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 16, 2024-04-21T08:57:57.991 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 7, 2024-04-21T08:57:57.992 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 1.492475396, 2024-04-21T08:57:57.992 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.673666929, 2024-04-21T08:57:57.992 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 1.259496933, 2024-04-21T08:57:57.992 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.072942180, 2024-04-21T08:57:57.992 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:57.992 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 35158, 2024-04-21T08:57:57.993 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.550894535, 2024-04-21T08:57:57.993 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000044112 2024-04-21T08:57:57.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.993 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:57.993 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 50453, 2024-04-21T08:57:57.993 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.007354652, 2024-04-21T08:57:57.994 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000145 2024-04-21T08:57:57.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.994 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 9249600, 2024-04-21T08:57:57.994 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 21298976 2024-04-21T08:57:57.994 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.994 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:57:57.994 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1892, 2024-04-21T08:57:57.994 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 2054, 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 2300226, 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 2008062, 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 31, 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 7, 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.189165587, 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.062897819, 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.100313050, 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.033290004, 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2053, 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.118552861, 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000057746 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.995 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1340, 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000299520, 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000223 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1797840, 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 482352 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-21T08:57:57.996 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3205, 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 3.539458758, 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001104355 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2664, 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.320677122, 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000120374 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 15292, 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 15261, 2024-04-21T08:57:57.997 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 15261, 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 214.406572349, 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.014049313 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 31, 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 55, 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 5, 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 11, 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-04-21T08:57:57.998 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 1, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 10, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 10, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 2854, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 1, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 1, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 3, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 32269, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 27367, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 31, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 11, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 375, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-21T08:57:57.999 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 39, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 2609, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 4, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 811, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 2754, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 234, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 2754, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:57:58.000 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 2754, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 161, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 3564, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 4374, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 5541, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 1612, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 191, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 4670 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 3, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 13, 2024-04-21T08:57:58.001 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 9, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 21, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 71, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 1267, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 1267, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-21T08:57:58.002 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 1, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 502, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 39, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 5, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 7337, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 7337, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-21T08:57:58.003 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 11, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 11, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 17122596, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 556, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 4.266679867, 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.007673884 2024-04-21T08:57:58.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 15, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 2946, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 2931, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 14, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 84, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 70, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 11, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 4354, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 4343, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 1, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 1958, 2024-04-21T08:57:58.005 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 1957, 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 75224, 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 223608 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 19173, 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 1355, 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 15292, 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 21, 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 5357, 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 911, 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 8.992929164, 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.009871491 2024-04-21T08:57:58.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001576971, 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001576971 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.007 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.008 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-21T08:57:58.008 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 11796, 2024-04-21T08:57:58.008 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 191.907168814, 2024-04-21T08:57:58.008 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.016268834 2024-04-21T08:57:58.008 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.008 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-21T08:57:58.008 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.008 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-21T08:57:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-04-21T08:57:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000117310, 2024-04-21T08:57:58.009 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000117310 2024-04-21T08:57:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-21T08:57:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.010 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-21T08:57:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.011 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-21T08:57:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.012 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-21T08:57:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-21T08:57:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.013 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-21T08:57:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-21T08:57:58.014 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 927, 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.927530431, 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001000572 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 39, 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.031824610, 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000816015 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-21T08:57:58.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 913, 2024-04-21T08:57:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 10.738648858, 2024-04-21T08:57:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.011761937 2024-04-21T08:57:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-21T08:57:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-21T08:57:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.016 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-21T08:57:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.017 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-21T08:57:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.018 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-21T08:57:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-21T08:57:58.019 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-21T08:57:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.020 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-21T08:57:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.021 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-21T08:57:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.022 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-21T08:57:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.023 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-21T08:57:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-21T08:57:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 673, 2024-04-21T08:57:58.024 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.806776191, 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002684660 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-04-21T08:57:58.025 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 4, 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 3, 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 2674, 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 2674, 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 640, 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:57:58.026 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:57:58.027 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:57:58.028 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 296295, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 108, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 9592, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 184, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:57:58.029 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 86488, 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 385, 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 6318, 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 14056422, 2024-04-21T08:57:58.030 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 33, 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 6318, 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6318, 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 22.887456803, 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.003622579 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6318, 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 6564 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 6318, 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2838, 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 3480, 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-21T08:57:58.031 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 84, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 640, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 76, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 4, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 2550, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 2754, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-04-21T08:57:58.032 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-21T08:57:58.033 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 103, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 14, 2024-04-21T08:57:58.034 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 349, 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 77 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.035 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 578 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 10, 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 5, 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 2534, 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 1267, 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-21T08:57:58.036 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 44643, 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 11373906, 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 44643, 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 44643, 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 11373906, 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-21T08:57:58.037 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 6318, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 14218253, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 6318, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 14218253, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.038 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 6318, 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 6318, 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 6318, 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 6318, 2024-04-21T08:57:58.039 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1267, 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 161831, 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1267, 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-21T08:57:58.040 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 84, 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 161831, 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x558ff8032500": { 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 7337, 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 12928292, 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 7337, 2024-04-21T08:57:58.041 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-21T08:57:58.042 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-21T08:57:58.042 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 556, 2024-04-21T08:57:58.042 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 12928292, 2024-04-21T08:57:58.042 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-21T08:57:58.042 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-21T08:57:58.042 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-21T08:57:58.042 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-21T08:57:58.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:58.042 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-21T08:57:58.042 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-21T08:57:58.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:58 smithi134 ceph-mon[31772]: pgmap v340: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:57:58.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:58 smithi082 ceph-mon[27967]: pgmap v340: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:57:58.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:58 smithi106 ceph-mon[31072]: pgmap v340: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-21T08:57:58.742 INFO:tasks.check_counter:Daemon mds.l mds.exported=39 2024-04-21T08:57:58.742 INFO:tasks.check_counter:Daemon mds.l mds.imported=4 2024-04-21T08:57:58.742 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=6 2024-04-21T08:57:58.742 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:57:58.742 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:57:58.742 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=5 2024-04-21T08:57:58.742 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-21T08:57:58.753 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-21T08:57:58.753 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:57:58.754 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:57:58.775 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:57:58.775 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:57:58.834 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:57:58.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.984+0000 7fa3c9d23640 1 -- 172.21.15.82:0/2362620597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3c406b2a0 msgr2=0x7fa3c4106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:58.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.984+0000 7fa3c9d23640 1 --2- 172.21.15.82:0/2362620597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3c406b2a0 0x7fa3c4106bc0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009900 tx=0x7fa3ac02bc30 comp rx=0 tx=0).stop 2024-04-21T08:57:58.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.984+0000 7fa3c9d23640 1 -- 172.21.15.82:0/2362620597 shutdown_connections 2024-04-21T08:57:58.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.984+0000 7fa3c9d23640 1 --2- 172.21.15.82:0/2362620597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c41072f0 0x7fa3c410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:58.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.984+0000 7fa3c9d23640 1 --2- 172.21.15.82:0/2362620597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3c406b2a0 0x7fa3c4106bc0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:58.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.984+0000 7fa3c9d23640 1 --2- 172.21.15.82:0/2362620597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c406a8f0 0x7fa3c406acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:58.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.984+0000 7fa3c9d23640 1 -- 172.21.15.82:0/2362620597 >> 172.21.15.82:0/2362620597 conn(0x7fa3c4077e30 msgr2=0x7fa3c4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:58.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.984+0000 7fa3c9d23640 1 -- 172.21.15.82:0/2362620597 shutdown_connections 2024-04-21T08:57:58.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.984+0000 7fa3c9d23640 1 -- 172.21.15.82:0/2362620597 wait complete. 2024-04-21T08:57:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c9d23640 1 Processor -- start 2024-04-21T08:57:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c9d23640 1 -- start start 2024-04-21T08:57:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c9d23640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c406a8f0 0x7fa3c419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c9d23640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c41072f0 0x7fa3c419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c9d23640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3c4198640 0x7fa3c4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c9d23640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4112220 con 0x7fa3c4198640 2024-04-21T08:57:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c9d23640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41120a0 con 0x7fa3c406a8f0 2024-04-21T08:57:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c9d23640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41123a0 con 0x7fa3c41072f0 2024-04-21T08:57:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3c4198640 0x7fa3c4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3c4198640 0x7fa3c4198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58638/0 (socket says 172.21.15.82:58638) 2024-04-21T08:57:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c3fff640 1 -- 172.21.15.82:0/4052677078 learned_addr learned my addr 172.21.15.82:0/4052677078 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:58.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3bbfff640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c41072f0 0x7fa3c419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c37fe640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c406a8f0 0x7fa3c419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c3fff640 1 -- 172.21.15.82:0/4052677078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c41072f0 msgr2=0x7fa3c419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c3fff640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c41072f0 0x7fa3c419eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c3fff640 1 -- 172.21.15.82:0/4052677078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c406a8f0 msgr2=0x7fa3c419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c3fff640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c406a8f0 0x7fa3c419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c3fff640 1 -- 172.21.15.82:0/4052677078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c4199240 con 0x7fa3c4198640 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.985+0000 7fa3c37fe640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c406a8f0 0x7fa3c419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.986+0000 7fa3c3fff640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3c4198640 0x7fa3c4198ac0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00cad0 tx=0x7fa3bc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.986+0000 7fa3c17fa640 1 -- 172.21.15.82:0/4052677078 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc00ae20 con 0x7fa3c4198640 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.986+0000 7fa3c17fa640 1 -- 172.21.15.82:0/4052677078 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3bc005020 con 0x7fa3c4198640 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.986+0000 7fa3c9d23640 1 -- 172.21.15.82:0/4052677078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c41a5320 con 0x7fa3c4198640 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.986+0000 7fa3c17fa640 1 -- 172.21.15.82:0/4052677078 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc00f6b0 con 0x7fa3c4198640 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.986+0000 7fa3c9d23640 1 -- 172.21.15.82:0/4052677078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c41a56d0 con 0x7fa3c4198640 2024-04-21T08:57:58.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.987+0000 7fa3c9d23640 1 -- 172.21.15.82:0/4052677078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c41036a0 con 0x7fa3c4198640 2024-04-21T08:57:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.988+0000 7fa3c17fa640 1 -- 172.21.15.82:0/4052677078 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa3bc0051d0 con 0x7fa3c4198640 2024-04-21T08:57:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.988+0000 7fa3c17fa640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa3b00a3d20 0x7fa3b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.989+0000 7fa3c37fe640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa3b00a3d20 0x7fa3b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.989+0000 7fa3c17fa640 1 -- 172.21.15.82:0/4052677078 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9518+0+0 (secure 0 0 0) 0x7fa3bc0d0d40 con 0x7fa3c4198640 2024-04-21T08:57:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.989+0000 7fa3c37fe640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa3b00a3d20 0x7fa3b00a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4006fd0 tx=0x7fa3b4008040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:58.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:58.993+0000 7fa3c17fa640 1 -- 172.21.15.82:0/4052677078 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3bc014030 con 0x7fa3c4198640 2024-04-21T08:57:59.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.173+0000 7fa3c9d23640 1 -- 172.21.15.82:0/4052677078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3c4069230 con 0x7fa3c4198640 2024-04-21T08:57:59.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.175+0000 7fa3c17fa640 1 -- 172.21.15.82:0/4052677078 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+25365 (secure 0 0 0) 0x7fa3bc09d000 con 0x7fa3c4198640 2024-04-21T08:57:59.175 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:57:59.175 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":103,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:57:35.168644+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2119539425":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6800/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/539366680":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/2275507980":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2883183315":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:57:59.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.177+0000 7fa3c9d23640 1 -- 172.21.15.82:0/4052677078 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa3b00a3d20 msgr2=0x7fa3b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.177+0000 7fa3c9d23640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa3b00a3d20 0x7fa3b00a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4006fd0 tx=0x7fa3b4008040 comp rx=0 tx=0).stop 2024-04-21T08:57:59.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.177+0000 7fa3c9d23640 1 -- 172.21.15.82:0/4052677078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3c4198640 msgr2=0x7fa3c4198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.177+0000 7fa3c9d23640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3c4198640 0x7fa3c4198ac0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00cad0 tx=0x7fa3bc00cf90 comp rx=0 tx=0).stop 2024-04-21T08:57:59.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.177+0000 7fa3c9d23640 1 -- 172.21.15.82:0/4052677078 shutdown_connections 2024-04-21T08:57:59.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.177+0000 7fa3c9d23640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa3b00a3d20 0x7fa3b00a61e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.177+0000 7fa3c9d23640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3c4198640 0x7fa3c4198ac0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.177+0000 7fa3c9d23640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3c41072f0 0x7fa3c419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.177+0000 7fa3c9d23640 1 --2- 172.21.15.82:0/4052677078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c406a8f0 0x7fa3c419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.177+0000 7fa3c9d23640 1 -- 172.21.15.82:0/4052677078 >> 172.21.15.82:0/4052677078 conn(0x7fa3c4077e30 msgr2=0x7fa3c40ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.177+0000 7fa3c9d23640 1 -- 172.21.15.82:0/4052677078 shutdown_connections 2024-04-21T08:57:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.177+0000 7fa3c9d23640 1 -- 172.21.15.82:0/4052677078 wait complete. 2024-04-21T08:57:59.188 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:39:50.316672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:44:32.331237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:44:32.865948+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:44:36.501832+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:57:59.189 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:57:59.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 -- 172.21.15.82:0/2276878938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd24106160 msgr2=0x7fdd2410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 --2- 172.21.15.82:0/2276878938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd24106160 0x7fdd2410dd30 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009ca0 tx=0x7fdd1c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 -- 172.21.15.82:0/2276878938 shutdown_connections 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 --2- 172.21.15.82:0/2276878938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd24106160 0x7fdd2410dd30 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 --2- 172.21.15.82:0/2276878938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd24101a40 0x7fdd24105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 --2- 172.21.15.82:0/2276878938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd24101090 0x7fdd24101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 -- 172.21.15.82:0/2276878938 >> 172.21.15.82:0/2276878938 conn(0x7fdd24077f50 msgr2=0x7fdd24078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 -- 172.21.15.82:0/2276878938 shutdown_connections 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 -- 172.21.15.82:0/2276878938 wait complete. 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 Processor -- start 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 -- start start 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd24101090 0x7fdd24199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd24106160 0x7fdd24199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2419a010 0x7fdd241a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdd241110d0 con 0x7fdd24106160 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24110f50 con 0x7fdd2419a010 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.290+0000 7fdd2b515640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24111250 con 0x7fdd24101090 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd28ab0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd24106160 0x7fdd24199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd28ab0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd24106160 0x7fdd24199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58656/0 (socket says 172.21.15.82:58656) 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd28ab0640 1 -- 172.21.15.82:0/552628787 learned_addr learned my addr 172.21.15.82:0/552628787 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd292b1640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd24101090 0x7fdd24199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd29ab2640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2419a010 0x7fdd241a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd28ab0640 1 -- 172.21.15.82:0/552628787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd24101090 msgr2=0x7fdd24199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd28ab0640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd24101090 0x7fdd24199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd28ab0640 1 -- 172.21.15.82:0/552628787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2419a010 msgr2=0x7fdd241a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd28ab0640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2419a010 0x7fdd241a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd28ab0640 1 -- 172.21.15.82:0/552628787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd241a4db0 con 0x7fdd24106160 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd28ab0640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd24106160 0x7fdd24199ad0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fdd20005fd0 tx=0x7fdd2000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd167fc640 1 -- 172.21.15.82:0/552628787 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20017070 con 0x7fdd24106160 2024-04-21T08:57:59.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd2b515640 1 -- 172.21.15.82:0/552628787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd241a50a0 con 0x7fdd24106160 2024-04-21T08:57:59.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd167fc640 1 -- 172.21.15.82:0/552628787 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd20009c30 con 0x7fdd24106160 2024-04-21T08:57:59.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd167fc640 1 -- 172.21.15.82:0/552628787 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20003d30 con 0x7fdd24106160 2024-04-21T08:57:59.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.291+0000 7fdd2b515640 1 -- 172.21.15.82:0/552628787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd241a55f0 con 0x7fdd24106160 2024-04-21T08:57:59.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.292+0000 7fdd2b515640 1 -- 172.21.15.82:0/552628787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd24068d70 con 0x7fdd24106160 2024-04-21T08:57:59.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.293+0000 7fdd167fc640 1 -- 172.21.15.82:0/552628787 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdd20003ed0 con 0x7fdd24106160 2024-04-21T08:57:59.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.294+0000 7fdd167fc640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdd100a3d50 0x7fdd100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.294+0000 7fdd167fc640 1 -- 172.21.15.82:0/552628787 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9518+0+0 (secure 0 0 0) 0x7fdd200d0780 con 0x7fdd24106160 2024-04-21T08:57:59.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.294+0000 7fdd292b1640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdd100a3d50 0x7fdd100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.294+0000 7fdd292b1640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdd100a3d50 0x7fdd100a6210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009bd0 tx=0x7fdd18005f90 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:59.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.298+0000 7fdd167fc640 1 -- 172.21.15.82:0/552628787 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd20020030 con 0x7fdd24106160 2024-04-21T08:57:59.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:57:59 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/4052677078' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:57:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.477+0000 7fdd2b515640 1 -- 172.21.15.82:0/552628787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdd24109420 con 0x7fdd24106160 2024-04-21T08:57:59.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.477+0000 7fdd167fc640 1 -- 172.21.15.82:0/552628787 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7fdd2009ca40 con 0x7fdd24106160 2024-04-21T08:57:59.477 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-21T08:57:59.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.479+0000 7fdd2b515640 1 -- 172.21.15.82:0/552628787 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdd100a3d50 msgr2=0x7fdd100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.479+0000 7fdd2b515640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdd100a3d50 0x7fdd100a6210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009bd0 tx=0x7fdd18005f90 comp rx=0 tx=0).stop 2024-04-21T08:57:59.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.479+0000 7fdd2b515640 1 -- 172.21.15.82:0/552628787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd24106160 msgr2=0x7fdd24199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.479+0000 7fdd2b515640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd24106160 0x7fdd24199ad0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fdd20005fd0 tx=0x7fdd2000dde0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.479+0000 7fdd2b515640 1 -- 172.21.15.82:0/552628787 shutdown_connections 2024-04-21T08:57:59.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.479+0000 7fdd2b515640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd2419a010 0x7fdd241a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.479+0000 7fdd2b515640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fdd100a3d50 0x7fdd100a6210 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.479+0000 7fdd2b515640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd24106160 0x7fdd24199ad0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.479+0000 7fdd2b515640 1 --2- 172.21.15.82:0/552628787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd24101090 0x7fdd24199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.479+0000 7fdd2b515640 1 -- 172.21.15.82:0/552628787 >> 172.21.15.82:0/552628787 conn(0x7fdd24077f50 msgr2=0x7fdd240fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:59.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.479+0000 7fdd2b515640 1 -- 172.21.15.82:0/552628787 shutdown_connections 2024-04-21T08:57:59.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.479+0000 7fdd2b515640 1 -- 172.21.15.82:0/552628787 wait complete. 2024-04-21T08:57:59.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:57:59 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/4052677078' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:57:59.490 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:57:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.591+0000 7f5b42fef640 1 -- 172.21.15.82:0/1100365701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c101090 msgr2=0x7f5b3c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.591+0000 7f5b42fef640 1 --2- 172.21.15.82:0/1100365701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c101090 0x7f5b3c101470 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009900 tx=0x7f5b3002bc30 comp rx=0 tx=0).stop 2024-04-21T08:57:59.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.591+0000 7f5b42fef640 1 -- 172.21.15.82:0/1100365701 shutdown_connections 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.591+0000 7f5b42fef640 1 --2- 172.21.15.82:0/1100365701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c106160 0x7f5b3c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 --2- 172.21.15.82:0/1100365701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c101a40 0x7f5b3c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 --2- 172.21.15.82:0/1100365701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c101090 0x7f5b3c101470 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 -- 172.21.15.82:0/1100365701 >> 172.21.15.82:0/1100365701 conn(0x7f5b3c077f50 msgr2=0x7f5b3c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 -- 172.21.15.82:0/1100365701 shutdown_connections 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 -- 172.21.15.82:0/1100365701 wait complete. 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 Processor -- start 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 -- start start 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c101090 0x7f5b3c19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c106160 0x7f5b3c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c1985f0 0x7f5b3c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c112230 con 0x7f5b3c1985f0 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1120b0 con 0x7f5b3c101090 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b42fef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1123b0 con 0x7f5b3c106160 2024-04-21T08:57:59.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b4158c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c1985f0 0x7f5b3c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b4158c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c1985f0 0x7f5b3c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58674/0 (socket says 172.21.15.82:58674) 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b4158c640 1 -- 172.21.15.82:0/1127982871 learned_addr learned my addr 172.21.15.82:0/1127982871 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.592+0000 7f5b40d8b640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c101090 0x7f5b3c19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.593+0000 7f5b3bfff640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c106160 0x7f5b3c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.593+0000 7f5b4158c640 1 -- 172.21.15.82:0/1127982871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c106160 msgr2=0x7f5b3c19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.593+0000 7f5b4158c640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c106160 0x7f5b3c19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.593+0000 7f5b4158c640 1 -- 172.21.15.82:0/1127982871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c101090 msgr2=0x7f5b3c19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.593+0000 7f5b4158c640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c101090 0x7f5b3c19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.593+0000 7f5b4158c640 1 -- 172.21.15.82:0/1127982871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c199110 con 0x7f5b3c1985f0 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.593+0000 7f5b4158c640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c1985f0 0x7f5b3c198aa0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400eaa0 tx=0x7f5b3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.593+0000 7f5b39ffb640 1 -- 172.21.15.82:0/1127982871 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400cdb0 con 0x7f5b3c1985f0 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.593+0000 7f5b39ffb640 1 -- 172.21.15.82:0/1127982871 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b34005020 con 0x7f5b3c1985f0 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.593+0000 7f5b42fef640 1 -- 172.21.15.82:0/1127982871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c199400 con 0x7f5b3c1985f0 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.593+0000 7f5b39ffb640 1 -- 172.21.15.82:0/1127982871 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b340116d0 con 0x7f5b3c1985f0 2024-04-21T08:57:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.593+0000 7f5b42fef640 1 -- 172.21.15.82:0/1127982871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c1a5600 con 0x7f5b3c1985f0 2024-04-21T08:57:59.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.594+0000 7f5b42fef640 1 -- 172.21.15.82:0/1127982871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b3c068d70 con 0x7f5b3c1985f0 2024-04-21T08:57:59.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.595+0000 7f5b39ffb640 1 -- 172.21.15.82:0/1127982871 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5b340051d0 con 0x7f5b3c1985f0 2024-04-21T08:57:59.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.596+0000 7f5b39ffb640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5b2c0a3d20 0x7f5b2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.596+0000 7f5b39ffb640 1 -- 172.21.15.82:0/1127982871 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9518+0+0 (secure 0 0 0) 0x7f5b340d11e0 con 0x7f5b3c1985f0 2024-04-21T08:57:59.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.596+0000 7f5b40d8b640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5b2c0a3d20 0x7f5b2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.596+0000 7f5b40d8b640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5b2c0a3d20 0x7f5b2c0a61e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5b30006010 tx=0x7f5b30005e40 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:59.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.599+0000 7f5b39ffb640 1 -- 172.21.15.82:0/1127982871 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b34016030 con 0x7f5b3c1985f0 2024-04-21T08:57:59.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:57:59 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/4052677078' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:57:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.776+0000 7f5b42fef640 1 -- 172.21.15.82:0/1127982871 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b3c101a40 con 0x7f5b3c1985f0 2024-04-21T08:57:59.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.777+0000 7f5b39ffb640 1 -- 172.21.15.82:0/1127982871 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7f5b3409d4a0 con 0x7f5b3c1985f0 2024-04-21T08:57:59.776 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:57:59.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.778+0000 7f5b42fef640 1 -- 172.21.15.82:0/1127982871 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5b2c0a3d20 msgr2=0x7f5b2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.778+0000 7f5b42fef640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5b2c0a3d20 0x7f5b2c0a61e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5b30006010 tx=0x7f5b30005e40 comp rx=0 tx=0).stop 2024-04-21T08:57:59.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.778+0000 7f5b42fef640 1 -- 172.21.15.82:0/1127982871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c1985f0 msgr2=0x7f5b3c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.778+0000 7f5b42fef640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c1985f0 0x7f5b3c198aa0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400eaa0 tx=0x7f5b3400ef60 comp rx=0 tx=0).stop 2024-04-21T08:57:59.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.779+0000 7f5b42fef640 1 -- 172.21.15.82:0/1127982871 shutdown_connections 2024-04-21T08:57:59.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.779+0000 7f5b42fef640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5b2c0a3d20 0x7f5b2c0a61e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.779+0000 7f5b42fef640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c1985f0 0x7f5b3c198aa0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.779+0000 7f5b42fef640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c106160 0x7f5b3c19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.779+0000 7f5b42fef640 1 --2- 172.21.15.82:0/1127982871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b3c101090 0x7f5b3c19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.779+0000 7f5b42fef640 1 -- 172.21.15.82:0/1127982871 >> 172.21.15.82:0/1127982871 conn(0x7f5b3c077f50 msgr2=0x7f5b3c0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:59.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.779+0000 7f5b42fef640 1 -- 172.21.15.82:0/1127982871 shutdown_connections 2024-04-21T08:57:59.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.779+0000 7f5b42fef640 1 -- 172.21.15.82:0/1127982871 wait complete. 2024-04-21T08:57:59.789 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:57:59.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.890+0000 7f3ad6e81640 1 -- 172.21.15.82:0/830637077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad0107030 msgr2=0x7f3ad0107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.890+0000 7f3ad6e81640 1 --2- 172.21.15.82:0/830637077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad0107030 0x7f3ad0107410 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009cd0 tx=0x7f3ac802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.890+0000 7f3ad6e81640 1 -- 172.21.15.82:0/830637077 shutdown_connections 2024-04-21T08:57:59.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.890+0000 7f3ad6e81640 1 --2- 172.21.15.82:0/830637077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ad0101b40 0x7f3ad010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.890+0000 7f3ad6e81640 1 --2- 172.21.15.82:0/830637077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad01011a0 0x7f3ad0101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.890+0000 7f3ad6e81640 1 --2- 172.21.15.82:0/830637077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad0107030 0x7f3ad0107410 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.890+0000 7f3ad6e81640 1 -- 172.21.15.82:0/830637077 >> 172.21.15.82:0/830637077 conn(0x7f3ad0076c50 msgr2=0x7f3ad0077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:57:59.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.890+0000 7f3ad6e81640 1 -- 172.21.15.82:0/830637077 shutdown_connections 2024-04-21T08:57:59.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.890+0000 7f3ad6e81640 1 -- 172.21.15.82:0/830637077 wait complete. 2024-04-21T08:57:59.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad6e81640 1 Processor -- start 2024-04-21T08:57:59.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad6e81640 1 -- start start 2024-04-21T08:57:59.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad6e81640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad01011a0 0x7f3ad0197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad6e81640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ad0101b40 0x7f3ad0197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad6e81640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0107030 0x7f3ad019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad6e81640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0111140 con 0x7f3ad01011a0 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad6e81640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0110fc0 con 0x7f3ad0107030 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad6e81640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01112c0 con 0x7f3ad0101b40 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad4c1d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad01011a0 0x7f3ad0197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad4c1d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad01011a0 0x7f3ad0197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58692/0 (socket says 172.21.15.82:58692) 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad4c1d640 1 -- 172.21.15.82:0/3468725358 learned_addr learned my addr 172.21.15.82:0/3468725358 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad541e640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0107030 0x7f3ad019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3acffff640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ad0101b40 0x7f3ad0197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad4c1d640 1 -- 172.21.15.82:0/3468725358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ad0101b40 msgr2=0x7f3ad0197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad4c1d640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ad0101b40 0x7f3ad0197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad4c1d640 1 -- 172.21.15.82:0/3468725358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0107030 msgr2=0x7f3ad019c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:57:59.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad4c1d640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0107030 0x7f3ad019c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:57:59.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad4c1d640 1 -- 172.21.15.82:0/3468725358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad019c740 con 0x7f3ad01011a0 2024-04-21T08:57:59.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3acffff640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ad0101b40 0x7f3ad0197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:57:59.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad541e640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0107030 0x7f3ad019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:57:59.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3ad4c1d640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad01011a0 0x7f3ad0197750 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f3ac802c700 tx=0x7f3ac8034700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:59.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.891+0000 7f3acdffb640 1 -- 172.21.15.82:0/3468725358 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8041020 con 0x7f3ad01011a0 2024-04-21T08:57:59.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.892+0000 7f3acdffb640 1 -- 172.21.15.82:0/3468725358 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ac8032970 con 0x7f3ad01011a0 2024-04-21T08:57:59.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.892+0000 7f3ad6e81640 1 -- 172.21.15.82:0/3468725358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad019c9d0 con 0x7f3ad01011a0 2024-04-21T08:57:59.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.892+0000 7f3acdffb640 1 -- 172.21.15.82:0/3468725358 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8039040 con 0x7f3ad01011a0 2024-04-21T08:57:59.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.892+0000 7f3ad6e81640 1 -- 172.21.15.82:0/3468725358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad01a42c0 con 0x7f3ad01011a0 2024-04-21T08:57:59.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.893+0000 7f3ad6e81640 1 -- 172.21.15.82:0/3468725358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad0102590 con 0x7f3ad01011a0 2024-04-21T08:57:59.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.894+0000 7f3acdffb640 1 -- 172.21.15.82:0/3468725358 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3ac8044020 con 0x7f3ad01011a0 2024-04-21T08:57:59.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.894+0000 7f3acdffb640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3abc0a3d20 0x7f3abc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:57:59.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.894+0000 7f3acffff640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3abc0a3d20 0x7f3abc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:57:59.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.895+0000 7f3acdffb640 1 -- 172.21.15.82:0/3468725358 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9518+0+0 (secure 0 0 0) 0x7f3ac80f2d20 con 0x7f3ad01011a0 2024-04-21T08:57:59.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.895+0000 7f3acffff640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3abc0a3d20 0x7f3abc0a61e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0106720 tx=0x7f3ac0005e50 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:57:59.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:57:59.898+0000 7f3acdffb640 1 -- 172.21.15.82:0/3468725358 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ac804b020 con 0x7f3ad01011a0 2024-04-21T08:58:00.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.073+0000 7f3ad6e81640 1 -- 172.21.15.82:0/3468725358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3ad0109340 con 0x7f3ad01011a0 2024-04-21T08:58:00.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.074+0000 7f3acdffb640 1 -- 172.21.15.82:0/3468725358 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7f3ac80bf0f0 con 0x7f3ad01011a0 2024-04-21T08:58:00.074 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-21T08:58:00.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.076+0000 7f3ad6e81640 1 -- 172.21.15.82:0/3468725358 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3abc0a3d20 msgr2=0x7f3abc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:00.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.076+0000 7f3ad6e81640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3abc0a3d20 0x7f3abc0a61e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0106720 tx=0x7f3ac0005e50 comp rx=0 tx=0).stop 2024-04-21T08:58:00.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.076+0000 7f3ad6e81640 1 -- 172.21.15.82:0/3468725358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad01011a0 msgr2=0x7f3ad0197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:00.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.076+0000 7f3ad6e81640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad01011a0 0x7f3ad0197750 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f3ac802c700 tx=0x7f3ac8034700 comp rx=0 tx=0).stop 2024-04-21T08:58:00.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.076+0000 7f3ad6e81640 1 -- 172.21.15.82:0/3468725358 shutdown_connections 2024-04-21T08:58:00.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.076+0000 7f3ad6e81640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ad0107030 0x7f3ad019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.076+0000 7f3ad6e81640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ad0101b40 0x7f3ad0197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.076+0000 7f3ad6e81640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f3abc0a3d20 0x7f3abc0a61e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.076+0000 7f3ad6e81640 1 --2- 172.21.15.82:0/3468725358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad01011a0 0x7f3ad0197750 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.076+0000 7f3ad6e81640 1 -- 172.21.15.82:0/3468725358 >> 172.21.15.82:0/3468725358 conn(0x7f3ad0076c50 msgr2=0x7f3ad00fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:00.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.076+0000 7f3ad6e81640 1 -- 172.21.15.82:0/3468725358 shutdown_connections 2024-04-21T08:58:00.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.076+0000 7f3ad6e81640 1 -- 172.21.15.82:0/3468725358 wait complete. 2024-04-21T08:58:00.087 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:58:00.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.188+0000 7f77b18d9640 1 -- 172.21.15.82:0/2071393713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77ac106170 msgr2=0x7f77ac10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:00.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.188+0000 7f77b18d9640 1 --2- 172.21.15.82:0/2071393713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77ac106170 0x7f77ac10dd40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009ca0 tx=0x7f77a402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.188+0000 7f77b18d9640 1 -- 172.21.15.82:0/2071393713 shutdown_connections 2024-04-21T08:58:00.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.188+0000 7f77b18d9640 1 --2- 172.21.15.82:0/2071393713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77ac106170 0x7f77ac10dd40 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.188+0000 7f77b18d9640 1 --2- 172.21.15.82:0/2071393713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77ac101a50 0x7f77ac105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.188+0000 7f77b18d9640 1 --2- 172.21.15.82:0/2071393713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1010a0 0x7f77ac101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.188+0000 7f77b18d9640 1 -- 172.21.15.82:0/2071393713 >> 172.21.15.82:0/2071393713 conn(0x7f77ac0771d0 msgr2=0x7f77ac0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.188+0000 7f77b18d9640 1 -- 172.21.15.82:0/2071393713 shutdown_connections 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.188+0000 7f77b18d9640 1 -- 172.21.15.82:0/2071393713 wait complete. 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77b18d9640 1 Processor -- start 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77b18d9640 1 -- start start 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77b18d9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1010a0 0x7f77ac199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77b18d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77ac106170 0x7f77ac199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77b18d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77ac19a050 0x7f77ac1a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77b18d9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1110e0 con 0x7f77ac19a050 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77b18d9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac110f60 con 0x7f77ac1010a0 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77b18d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac111260 con 0x7f77ac106170 2024-04-21T08:58:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77ab7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77ac19a050 0x7f77ac1a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:00.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77ab7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77ac19a050 0x7f77ac1a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58710/0 (socket says 172.21.15.82:58710) 2024-04-21T08:58:00.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77ab7fe640 1 -- 172.21.15.82:0/2846826044 learned_addr learned my addr 172.21.15.82:0/2846826044 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:58:00.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77aa7fc640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77ac106170 0x7f77ac199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:00.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.189+0000 7f77aaffd640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1010a0 0x7f77ac199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:00.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.190+0000 7f77ab7fe640 1 -- 172.21.15.82:0/2846826044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77ac106170 msgr2=0x7f77ac199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:00.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.190+0000 7f77ab7fe640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77ac106170 0x7f77ac199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.190+0000 7f77ab7fe640 1 -- 172.21.15.82:0/2846826044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1010a0 msgr2=0x7f77ac199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:00.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.190+0000 7f77ab7fe640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1010a0 0x7f77ac199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.190+0000 7f77ab7fe640 1 -- 172.21.15.82:0/2846826044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac1a4dd0 con 0x7f77ac19a050 2024-04-21T08:58:00.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.190+0000 7f77aa7fc640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77ac106170 0x7f77ac199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:58:00.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.190+0000 7f77ab7fe640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77ac19a050 0x7f77ac1a4600 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f77a4005e30 tx=0x7f77a4034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:00.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.190+0000 7f77b08d7640 1 -- 172.21.15.82:0/2846826044 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4039040 con 0x7f77ac19a050 2024-04-21T08:58:00.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.190+0000 7f77b08d7640 1 -- 172.21.15.82:0/2846826044 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a4044070 con 0x7f77ac19a050 2024-04-21T08:58:00.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.190+0000 7f77b18d9640 1 -- 172.21.15.82:0/2846826044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ac1a5060 con 0x7f77ac19a050 2024-04-21T08:58:00.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.190+0000 7f77b08d7640 1 -- 172.21.15.82:0/2846826044 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4003e60 con 0x7f77ac19a050 2024-04-21T08:58:00.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.190+0000 7f77b18d9640 1 -- 172.21.15.82:0/2846826044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ac1a5550 con 0x7f77ac19a050 2024-04-21T08:58:00.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.191+0000 7f77b18d9640 1 -- 172.21.15.82:0/2846826044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77ac0691c0 con 0x7f77ac19a050 2024-04-21T08:58:00.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.192+0000 7f77b08d7640 1 -- 172.21.15.82:0/2846826044 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f77a40324c0 con 0x7f77ac19a050 2024-04-21T08:58:00.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.192+0000 7f77b08d7640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f77940a3d20 0x7f77940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:00.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.193+0000 7f77aaffd640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f77940a3d20 0x7f77940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:00.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.193+0000 7f77b08d7640 1 -- 172.21.15.82:0/2846826044 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9518+0+0 (secure 0 0 0) 0x7f77a40f3900 con 0x7f77ac19a050 2024-04-21T08:58:00.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.193+0000 7f77aaffd640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f77940a3d20 0x7f77940a61e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f779c005fd0 tx=0x7f779c005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:00.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.196+0000 7f77b08d7640 1 -- 172.21.15.82:0/2846826044 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77a40bfcd0 con 0x7f77ac19a050 2024-04-21T08:58:00.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:00 smithi082 ceph-mon[27967]: pgmap v341: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 op/s 2024-04-21T08:58:00.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:00 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/552628787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:58:00.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:00 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1127982871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:58:00.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:00 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3468725358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:58:00.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.374+0000 7f77b18d9640 1 -- 172.21.15.82:0/2846826044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f77ac1093a0 con 0x7f77ac19a050 2024-04-21T08:58:00.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.374+0000 7f77b08d7640 1 -- 172.21.15.82:0/2846826044 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v103) ==== 103+0+11 (secure 0 0 0) 0x7f77a40bf5d0 con 0x7f77ac19a050 2024-04-21T08:58:00.374 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-21T08:58:00.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.376+0000 7f77b18d9640 1 -- 172.21.15.82:0/2846826044 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f77940a3d20 msgr2=0x7f77940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:00.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.376+0000 7f77b18d9640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f77940a3d20 0x7f77940a61e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f779c005fd0 tx=0x7f779c005850 comp rx=0 tx=0).stop 2024-04-21T08:58:00.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.376+0000 7f77b18d9640 1 -- 172.21.15.82:0/2846826044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77ac19a050 msgr2=0x7f77ac1a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:00.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.376+0000 7f77b18d9640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77ac19a050 0x7f77ac1a4600 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f77a4005e30 tx=0x7f77a4034760 comp rx=0 tx=0).stop 2024-04-21T08:58:00.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.376+0000 7f77b18d9640 1 -- 172.21.15.82:0/2846826044 shutdown_connections 2024-04-21T08:58:00.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.376+0000 7f77b18d9640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f77940a3d20 0x7f77940a61e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.376+0000 7f77b18d9640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77ac19a050 0x7f77ac1a4600 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.376+0000 7f77b18d9640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77ac106170 0x7f77ac199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.376+0000 7f77b18d9640 1 --2- 172.21.15.82:0/2846826044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1010a0 0x7f77ac199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.376+0000 7f77b18d9640 1 -- 172.21.15.82:0/2846826044 >> 172.21.15.82:0/2846826044 conn(0x7f77ac0771d0 msgr2=0x7f77ac0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:00.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.376+0000 7f77b18d9640 1 -- 172.21.15.82:0/2846826044 shutdown_connections 2024-04-21T08:58:00.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.376+0000 7f77b18d9640 1 -- 172.21.15.82:0/2846826044 wait complete. 2024-04-21T08:58:00.387 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T08:58:00.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:00 smithi134 ceph-mon[31772]: pgmap v341: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 op/s 2024-04-21T08:58:00.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:00 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/552628787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:58:00.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:00 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1127982871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:58:00.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:00 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3468725358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.486+0000 7f332e0f2640 1 -- 172.21.15.82:0/4002497594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f332810e900 msgr2=0x7f3328110cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.486+0000 7f332e0f2640 1 --2- 172.21.15.82:0/4002497594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f332810e900 0x7f3328110cf0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f3320009ca0 tx=0x7f332002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.486+0000 7f332e0f2640 1 -- 172.21.15.82:0/4002497594 shutdown_connections 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.486+0000 7f332e0f2640 1 --2- 172.21.15.82:0/4002497594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f332810e900 0x7f3328110cf0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.486+0000 7f332e0f2640 1 --2- 172.21.15.82:0/4002497594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3328101670 0x7f332810e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.486+0000 7f332e0f2640 1 --2- 172.21.15.82:0/4002497594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3328100d50 0x7f3328101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.486+0000 7f332e0f2640 1 -- 172.21.15.82:0/4002497594 >> 172.21.15.82:0/4002497594 conn(0x7f3328077f50 msgr2=0x7f3328078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.486+0000 7f332e0f2640 1 -- 172.21.15.82:0/4002497594 shutdown_connections 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.486+0000 7f332e0f2640 1 -- 172.21.15.82:0/4002497594 wait complete. 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.486+0000 7f332e0f2640 1 Processor -- start 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.486+0000 7f332e0f2640 1 -- start start 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f332e0f2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3328100d50 0x7f33281a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f332e0f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328101670 0x7f33281a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f332e0f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f332810e900 0x7f332819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:00.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f332e0f2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f33281143c0 con 0x7f3328101670 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f332e0f2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3328114240 con 0x7f3328100d50 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f332e0f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3328114540 con 0x7f332810e900 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3326ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328101670 0x7f33281a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3326ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328101670 0x7f33281a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58736/0 (socket says 172.21.15.82:58736) 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3326ffd640 1 -- 172.21.15.82:0/68615162 learned_addr learned my addr 172.21.15.82:0/68615162 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3327fff640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f332810e900 0x7f332819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f33277fe640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3328100d50 0x7f33281a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3326ffd640 1 -- 172.21.15.82:0/68615162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f332810e900 msgr2=0x7f332819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3326ffd640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f332810e900 0x7f332819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3326ffd640 1 -- 172.21.15.82:0/68615162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3328100d50 msgr2=0x7f33281a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3326ffd640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3328100d50 0x7f33281a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3326ffd640 1 -- 172.21.15.82:0/68615162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332819b000 con 0x7f3328101670 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3326ffd640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328101670 0x7f33281a0bc0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f331c00e9f0 tx=0x7f331c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3324ff9640 1 -- 172.21.15.82:0/68615162 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c00cd80 con 0x7f3328101670 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3324ff9640 1 -- 172.21.15.82:0/68615162 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f331c0054d0 con 0x7f3328101670 2024-04-21T08:58:00.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f332e0f2640 1 -- 172.21.15.82:0/68615162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332819b2f0 con 0x7f3328101670 2024-04-21T08:58:00.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f3324ff9640 1 -- 172.21.15.82:0/68615162 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c010690 con 0x7f3328101670 2024-04-21T08:58:00.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.487+0000 7f332e0f2640 1 -- 172.21.15.82:0/68615162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33281a7600 con 0x7f3328101670 2024-04-21T08:58:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.488+0000 7f332e0f2640 1 -- 172.21.15.82:0/68615162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33280698e0 con 0x7f3328101670 2024-04-21T08:58:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.490+0000 7f3324ff9640 1 -- 172.21.15.82:0/68615162 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f331c004e30 con 0x7f3328101670 2024-04-21T08:58:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.491+0000 7f3324ff9640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f33100a3d20 0x7f33100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.491+0000 7f33277fe640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f33100a3d20 0x7f33100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.491+0000 7f3324ff9640 1 -- 172.21.15.82:0/68615162 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9518+0+0 (secure 0 0 0) 0x7f331c0d1130 con 0x7f3328101670 2024-04-21T08:58:00.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.491+0000 7f33277fe640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f33100a3d20 0x7f33100a61e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f331800bbe0 tx=0x7f331800b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:00.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.494+0000 7f3324ff9640 1 -- 172.21.15.82:0/68615162 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f331c053ff0 con 0x7f3328101670 2024-04-21T08:58:00.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:00 smithi106 ceph-mon[31072]: pgmap v341: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 op/s 2024-04-21T08:58:00.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:00 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/552628787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:58:00.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:00 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1127982871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:58:00.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:00 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3468725358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:58:00.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.704+0000 7f332e0f2640 1 -- 172.21.15.82:0/68615162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f332819b520 con 0x7f3328101670 2024-04-21T08:58:00.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.705+0000 7f3324ff9640 1 -- 172.21.15.82:0/68615162 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v103) ==== 81+0+943 (secure 0 0 0) 0x7f331c09d3f0 con 0x7f3328101670 2024-04-21T08:58:00.705 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2119539425 2024-04-22T08:47:00.617348+0000 2024-04-21T08:58:00.705 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/2138749385 2024-04-22T08:47:00.617348+0000 2024-04-21T08:58:00.705 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/539366680 2024-04-22T08:47:00.617348+0000 2024-04-21T08:58:00.705 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2275507980 2024-04-22T08:47:00.617348+0000 2024-04-21T08:58:00.705 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/4175378061 2024-04-22T08:37:26.227727+0000 2024-04-21T08:58:00.705 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2889165689 2024-04-22T08:37:26.227727+0000 2024-04-21T08:58:00.705 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/880382736 2024-04-22T08:37:26.227727+0000 2024-04-21T08:58:00.705 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3352705806 2024-04-22T08:37:26.227727+0000 2024-04-21T08:58:00.705 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/2710226711 2024-04-22T08:36:59.578707+0000 2024-04-21T08:58:00.705 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/4224740504 2024-04-22T08:36:59.578707+0000 2024-04-21T08:58:00.705 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2873389149 2024-04-22T08:36:59.578707+0000 2024-04-21T08:58:00.706 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2883183315 2024-04-22T08:47:00.617348+0000 2024-04-21T08:58:00.706 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/2138749385 2024-04-22T08:47:00.617348+0000 2024-04-21T08:58:00.706 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/880382736 2024-04-22T08:37:26.227727+0000 2024-04-21T08:58:00.706 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/2710226711 2024-04-22T08:36:59.578707+0000 2024-04-21T08:58:00.706 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3032518359 2024-04-22T08:36:59.578707+0000 2024-04-21T08:58:00.707 INFO:teuthology.orchestra.run.smithi082.stderr:listed 16 entries 2024-04-21T08:58:00.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.707+0000 7f332e0f2640 1 -- 172.21.15.82:0/68615162 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f33100a3d20 msgr2=0x7f33100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:00.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.707+0000 7f332e0f2640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f33100a3d20 0x7f33100a61e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f331800bbe0 tx=0x7f331800b390 comp rx=0 tx=0).stop 2024-04-21T08:58:00.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.707+0000 7f332e0f2640 1 -- 172.21.15.82:0/68615162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328101670 msgr2=0x7f33281a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:00.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.707+0000 7f332e0f2640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328101670 0x7f33281a0bc0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f331c00e9f0 tx=0x7f331c00edf0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.707+0000 7f332e0f2640 1 -- 172.21.15.82:0/68615162 shutdown_connections 2024-04-21T08:58:00.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.707+0000 7f332e0f2640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f332810e900 0x7f332819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.707+0000 7f332e0f2640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f33100a3d20 0x7f33100a61e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.707+0000 7f332e0f2640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328101670 0x7f33281a0bc0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.707+0000 7f332e0f2640 1 --2- 172.21.15.82:0/68615162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3328100d50 0x7f33281a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:00.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.707+0000 7f332e0f2640 1 -- 172.21.15.82:0/68615162 >> 172.21.15.82:0/68615162 conn(0x7f3328077f50 msgr2=0x7f3328110030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:00.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.707+0000 7f332e0f2640 1 -- 172.21.15.82:0/68615162 shutdown_connections 2024-04-21T08:58:00.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:00.707+0000 7f332e0f2640 1 -- 172.21.15.82:0/68615162 wait complete. 2024-04-21T08:58:00.717 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-21T08:58:00.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:58:00.717 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-21T08:58:00.829 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-21T08:58:00.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:58:00.829 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-21T08:58:00.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:58:00.870 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-21T08:58:00.955 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-21T08:58:00.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:58:00.955 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-21T08:58:01.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:58:01.012 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-04-21T08:58:01.105 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-21T08:58:01.105 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-21T08:58:01.105 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:58:01.141 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056092,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=897 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/cb860465d9dd4389648760599048d56f382a9e05c431ddf1e93497513fc6d21f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/P74RHRWYLFPN2R5FXHWDYP27XF:/var/lib/containers/storage/overlay/l/DKJAXUDBOMAAVBHBVX3ZWG7QF2,upperdir=/var/lib/containers/storage/overlay/cb860465d9dd4389648760599048d56f382a9e05c431ddf1e93497513fc6d21f/diff,workdir=/var/lib/containers/storage/overlay/cb860465d9dd4389648760599048d56f382a9e05c431ddf1e93497513fc6d21f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9772f2d860971eb3f19d776615240978c60fc60a6b23215c3653e11e6ae9c55b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/P74RHRWYLFPN2R5FXHWDYP27XF:/var/lib/containers/storage/overlay/l/DKJAXUDBOMAAVBHBVX3ZWG7QF2,upperdir=/var/lib/containers/storage/overlay/9772f2d860971eb3f19d776615240978c60fc60a6b23215c3653e11e6ae9c55b/diff,workdir=/var/lib/containers/storage/overlay/9772f2d860971eb3f19d776615240978c60fc60a6b23215c3653e11e6ae9c55b/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/048e109f117dcb565c69f5596238cd17ca948aecad6d90f819b9eb9901bbf747/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/P74RHRWYLFPN2R5FXHWDYP27XF:/var/lib/containers/storage/overlay/l/DKJAXUDBOMAAVBHBVX3ZWG7QF2,upperdir=/var/lib/containers/storage/overlay/048e109f117dcb565c69f5596238cd17ca948aecad6d90f819b9eb9901bbf747/diff,workdir=/var/lib/containers/storage/overlay/048e109f117dcb565c69f5596238cd17ca948aecad6d90f819b9eb9901bbf747/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/47516ef441c32a032fb0d803b5341fef846267226b85f32a78f6b6a3dc257229/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/P74RHRWYLFPN2R5FXHWDYP27XF:/var/lib/containers/storage/overlay/l/DKJAXUDBOMAAVBHBVX3ZWG7QF2,upperdir=/var/lib/containers/storage/overlay/47516ef441c32a032fb0d803b5341fef846267226b85f32a78f6b6a3dc257229/diff,workdir=/var/lib/containers/storage/overlay/47516ef441c32a032fb0d803b5341fef846267226b85f32a78f6b6a3dc257229/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9e2762a086d09d81d825d10abe16e18c4b8a90db73dcf39593c3de2a6fc6ffcb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/P74RHRWYLFPN2R5FXHWDYP27XF:/var/lib/containers/storage/overlay/l/DKJAXUDBOMAAVBHBVX3ZWG7QF2,upperdir=/var/lib/containers/storage/overlay/9e2762a086d09d81d825d10abe16e18c4b8a90db73dcf39593c3de2a6fc6ffcb/diff,workdir=/var/lib/containers/storage/overlay/9e2762a086d09d81d825d10abe16e18c4b8a90db73dcf39593c3de2a6fc6ffcb/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b74c628f3b949cbbac1b6398ecaa6af4447ad840c557f71455d0594e3d49d9b1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/P74RHRWYLFPN2R5FXHWDYP27XF:/var/lib/containers/storage/overlay/l/DKJAXUDBOMAAVBHBVX3ZWG7QF2,upperdir=/var/lib/containers/storage/overlay/b74c628f3b949cbbac1b6398ecaa6af4447ad840c557f71455d0594e3d49d9b1/diff,workdir=/var/lib/containers/storage/overlay/b74c628f3b949cbbac1b6398ecaa6af4447ad840c557f71455d0594e3d49d9b1/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fb33f6b2de769041147303cc1c096df7b0b591936b85c47279341eb5bf473394/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/P74RHRWYLFPN2R5FXHWDYP27XF:/var/lib/containers/storage/overlay/l/DKJAXUDBOMAAVBHBVX3ZWG7QF2,upperdir=/var/lib/containers/storage/overlay/fb33f6b2de769041147303cc1c096df7b0b591936b85c47279341eb5bf473394/diff,workdir=/var/lib/containers/storage/overlay/fb33f6b2de769041147303cc1c096df7b0b591936b85c47279341eb5bf473394/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7eef970bb1ea502f4def775692fe1b07e55604d335bf28438956169f65f6a7dd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/P74RHRWYLFPN2R5FXHWDYP27XF:/var/lib/containers/storage/overlay/l/DKJAXUDBOMAAVBHBVX3ZWG7QF2,upperdir=/var/lib/containers/storage/overlay/7eef970bb1ea502f4def775692fe1b07e55604d335bf28438956169f65f6a7dd/diff,workdir=/var/lib/containers/storage/overlay/7eef970bb1ea502f4def775692fe1b07e55604d335bf28438956169f65f6a7dd/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/01ed206f1ea932edd37252b72a020b8e37aff753fdc6ffbf35ef8a637e8378ff/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/P74RHRWYLFPN2R5FXHWDYP27XF:/var/lib/containers/storage/overlay/l/DKJAXUDBOMAAVBHBVX3ZWG7QF2,upperdir=/var/lib/containers/storage/overlay/01ed206f1ea932edd37252b72a020b8e37aff753fdc6ffbf35ef8a637e8378ff/diff,workdir=/var/lib/containers/storage/overlay/01ed206f1ea932edd37252b72a020b8e37aff753fdc6ffbf35ef8a637e8378ff/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6e1048e875105ee30e460d83f1beb92669ddde960c07b462a12a168ecae487f5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/P74RHRWYLFPN2R5FXHWDYP27XF:/var/lib/containers/storage/overlay/l/DKJAXUDBOMAAVBHBVX3ZWG7QF2,upperdir=/var/lib/containers/storage/overlay/6e1048e875105ee30e460d83f1beb92669ddde960c07b462a12a168ecae487f5/diff,workdir=/var/lib/containers/storage/overlay/6e1048e875105ee30e460d83f1beb92669ddde960c07b462a12a168ecae487f5/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-21T08:58:01.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:58:01.141 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-21T08:58:01.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:58:01.201 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-04-21T08:58:01.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:01 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2846826044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:58:01.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:01 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/68615162' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:58:01.269 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2024-04-21T08:58:01.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:58:01.270 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-21T08:58:01.281 INFO:tasks.cephadm:Teardown begin 2024-04-21T08:58:01.282 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-21T08:58:01.282 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:58:01.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.412+0000 7f5c82f15640 1 -- 172.21.15.82:0/268359143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c06b260 msgr2=0x7f5c7c106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:01.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.412+0000 7f5c82f15640 1 --2- 172.21.15.82:0/268359143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c06b260 0x7f5c7c106c10 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009ca0 tx=0x7f5c6c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.412+0000 7f5c82f15640 1 -- 172.21.15.82:0/268359143 shutdown_connections 2024-04-21T08:58:01.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.412+0000 7f5c82f15640 1 --2- 172.21.15.82:0/268359143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c7c107340 0x7f5c7c10eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.412+0000 7f5c82f15640 1 --2- 172.21.15.82:0/268359143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c06b260 0x7f5c7c106c10 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.412+0000 7f5c82f15640 1 --2- 172.21.15.82:0/268359143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c7c06a8b0 0x7f5c7c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.412+0000 7f5c82f15640 1 -- 172.21.15.82:0/268359143 >> 172.21.15.82:0/268359143 conn(0x7f5c7c077e30 msgr2=0x7f5c7c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.412+0000 7f5c82f15640 1 -- 172.21.15.82:0/268359143 shutdown_connections 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.412+0000 7f5c82f15640 1 -- 172.21.15.82:0/268359143 wait complete. 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.412+0000 7f5c82f15640 1 Processor -- start 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.412+0000 7f5c82f15640 1 -- start start 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c82f15640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c06a8b0 0x7f5c7c19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c82f15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c7c107340 0x7f5c7c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c82f15640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c7c198640 0x7f5c7c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c82f15640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c112210 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c82f15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c112090 con 0x7f5c7c107340 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c82f15640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c112390 con 0x7f5c7c198640 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c80cb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c06a8b0 0x7f5c7c19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c80cb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c06a8b0 0x7f5c7c19e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58750/0 (socket says 172.21.15.82:58750) 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c80cb1640 1 -- 172.21.15.82:0/2002467057 learned_addr learned my addr 172.21.15.82:0/2002467057 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:58:01.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c7bfff640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c7c107340 0x7f5c7c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:01.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c814b2640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c7c198640 0x7f5c7c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:01.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c80cb1640 1 -- 172.21.15.82:0/2002467057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c7c198640 msgr2=0x7f5c7c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:01.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c80cb1640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c7c198640 0x7f5c7c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c80cb1640 1 -- 172.21.15.82:0/2002467057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c7c107340 msgr2=0x7f5c7c19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:01.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c80cb1640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c7c107340 0x7f5c7c19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c80cb1640 1 -- 172.21.15.82:0/2002467057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c199160 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c80cb1640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c06a8b0 0x7f5c7c19e440 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400d770 tx=0x7f5c7400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:01.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c79ffb640 1 -- 172.21.15.82:0/2002467057 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74017070 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c79ffb640 1 -- 172.21.15.82:0/2002467057 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c740054d0 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c79ffb640 1 -- 172.21.15.82:0/2002467057 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74004c30 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.413+0000 7f5c82f15640 1 -- 172.21.15.82:0/2002467057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7c199450 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.414+0000 7f5c82f15640 1 -- 172.21.15.82:0/2002467057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7c1a5650 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.415+0000 7f5c82f15640 1 -- 172.21.15.82:0/2002467057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c7c069230 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.416+0000 7f5c79ffb640 1 -- 172.21.15.82:0/2002467057 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5c74004e10 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.416+0000 7f5c79ffb640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5c680a3da0 0x7f5c680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:01.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.416+0000 7f5c79ffb640 1 -- 172.21.15.82:0/2002467057 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9518+0+0 (secure 0 0 0) 0x7f5c740d04e0 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.416+0000 7f5c7bfff640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5c680a3da0 0x7f5c680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:01.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.417+0000 7f5c7bfff640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5c680a3da0 0x7f5c680a6260 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c006010 tx=0x7f5c6c005c50 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:01.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.421+0000 7f5c79ffb640 1 -- 172.21.15.82:0/2002467057 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c7409c1f0 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:01 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2846826044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:58:01.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:01 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/68615162' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:58:01.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:01 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2846826044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:58:01.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:01 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/68615162' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:58:01.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.641+0000 7f5c82f15640 1 -- 172.21.15.82:0/2002467057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c7c10a4a0 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.642+0000 7f5c79ffb640 1 -- 172.21.15.82:0/2002467057 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11944 (secure 0 0 0) 0x7f5c740a04f0 con 0x7f5c7c06a8b0 2024-04-21T08:58:01.642 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:58:01.642 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:57:34.850609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24481,24476,24407,24409]},"id":1}]} 2024-04-21T08:58:01.642 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 49 2024-04-21T08:58:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.644+0000 7f5c82f15640 1 -- 172.21.15.82:0/2002467057 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5c680a3da0 msgr2=0x7f5c680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.644+0000 7f5c82f15640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5c680a3da0 0x7f5c680a6260 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c006010 tx=0x7f5c6c005c50 comp rx=0 tx=0).stop 2024-04-21T08:58:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.644+0000 7f5c82f15640 1 -- 172.21.15.82:0/2002467057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c06a8b0 msgr2=0x7f5c7c19e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.644+0000 7f5c82f15640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c06a8b0 0x7f5c7c19e440 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400d770 tx=0x7f5c7400dc30 comp rx=0 tx=0).stop 2024-04-21T08:58:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.644+0000 7f5c82f15640 1 -- 172.21.15.82:0/2002467057 shutdown_connections 2024-04-21T08:58:01.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.644+0000 7f5c82f15640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c7c198640 0x7f5c7c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.644+0000 7f5c82f15640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c7c107340 0x7f5c7c19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.644+0000 7f5c82f15640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f5c680a3da0 0x7f5c680a6260 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.644+0000 7f5c82f15640 1 --2- 172.21.15.82:0/2002467057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c06a8b0 0x7f5c7c19e440 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.644+0000 7f5c82f15640 1 -- 172.21.15.82:0/2002467057 >> 172.21.15.82:0/2002467057 conn(0x7f5c7c077e30 msgr2=0x7f5c7c0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:01.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.644+0000 7f5c82f15640 1 -- 172.21.15.82:0/2002467057 shutdown_connections 2024-04-21T08:58:01.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.644+0000 7f5c82f15640 1 -- 172.21.15.82:0/2002467057 wait complete. 2024-04-21T08:58:01.656 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:58:01.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.757+0000 7f0630198640 1 -- 172.21.15.82:0/2611947973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0628101b20 msgr2=0x7f062810dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:01.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.757+0000 7f0630198640 1 --2- 172.21.15.82:0/2611947973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0628101b20 0x7f062810dd00 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f0620009930 tx=0x7f062002bc30 comp rx=0 tx=0).stop 2024-04-21T08:58:01.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.757+0000 7f0630198640 1 -- 172.21.15.82:0/2611947973 shutdown_connections 2024-04-21T08:58:01.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.757+0000 7f0630198640 1 --2- 172.21.15.82:0/2611947973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0628101b20 0x7f062810dd00 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.757+0000 7f0630198640 1 --2- 172.21.15.82:0/2611947973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06281011a0 0x7f06281015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.757+0000 7f0630198640 1 --2- 172.21.15.82:0/2611947973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0628105ed0 0x7f06281062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.757+0000 7f0630198640 1 -- 172.21.15.82:0/2611947973 >> 172.21.15.82:0/2611947973 conn(0x7f0628077e30 msgr2=0x7f0628078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:01.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.757+0000 7f0630198640 1 -- 172.21.15.82:0/2611947973 shutdown_connections 2024-04-21T08:58:01.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.757+0000 7f0630198640 1 -- 172.21.15.82:0/2611947973 wait complete. 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.757+0000 7f0630198640 1 Processor -- start 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.757+0000 7f0630198640 1 -- start start 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f0630198640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06281011a0 0x7f06281975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f0630198640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0628101b20 0x7f0628197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f0630198640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0628105ed0 0x7f062819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f0630198640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0628111070 con 0x7f0628101b20 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f0630198640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0628110ef0 con 0x7f06281011a0 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f0630198640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06281111f0 con 0x7f0628105ed0 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f062d733640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0628101b20 0x7f0628197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f062d733640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0628101b20 0x7f0628197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58762/0 (socket says 172.21.15.82:58762) 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f062d733640 1 -- 172.21.15.82:0/2971409572 learned_addr learned my addr 172.21.15.82:0/2971409572 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f062df34640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06281011a0 0x7f06281975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f062e735640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0628105ed0 0x7f062819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f062d733640 1 -- 172.21.15.82:0/2971409572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0628105ed0 msgr2=0x7f062819be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:01.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f062d733640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0628105ed0 0x7f062819be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f062d733640 1 -- 172.21.15.82:0/2971409572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06281011a0 msgr2=0x7f06281975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:01.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f062d733640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06281011a0 0x7f06281975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f062d733640 1 -- 172.21.15.82:0/2971409572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062819c570 con 0x7f0628101b20 2024-04-21T08:58:01.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f062d733640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0628101b20 0x7f0628197b10 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f061400d950 tx=0x7f061400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:01.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f0626ffd640 1 -- 172.21.15.82:0/2971409572 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614017030 con 0x7f0628101b20 2024-04-21T08:58:01.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f0626ffd640 1 -- 172.21.15.82:0/2971409572 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0614004020 con 0x7f0628101b20 2024-04-21T08:58:01.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f0630198640 1 -- 172.21.15.82:0/2971409572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062819c860 con 0x7f0628101b20 2024-04-21T08:58:01.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f0626ffd640 1 -- 172.21.15.82:0/2971409572 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06140047a0 con 0x7f0628101b20 2024-04-21T08:58:01.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.758+0000 7f0630198640 1 -- 172.21.15.82:0/2971409572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f062806a660 con 0x7f0628101b20 2024-04-21T08:58:01.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.759+0000 7f0630198640 1 -- 172.21.15.82:0/2971409572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0628069230 con 0x7f0628101b20 2024-04-21T08:58:01.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.760+0000 7f0626ffd640 1 -- 172.21.15.82:0/2971409572 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0614009a30 con 0x7f0628101b20 2024-04-21T08:58:01.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.761+0000 7f0626ffd640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f060c0a3da0 0x7f060c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:01.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.761+0000 7f062df34640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f060c0a3da0 0x7f060c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:01.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.761+0000 7f0626ffd640 1 -- 172.21.15.82:0/2971409572 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9518+0+0 (secure 0 0 0) 0x7f06140d05a0 con 0x7f0628101b20 2024-04-21T08:58:01.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.761+0000 7f062df34640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f060c0a3da0 0x7f060c0a6260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0618005fd0 tx=0x7f0618009680 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:01.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.766+0000 7f0626ffd640 1 -- 172.21.15.82:0/2971409572 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f061409c860 con 0x7f0628101b20 2024-04-21T08:58:01.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.982+0000 7f0630198640 1 -- 172.21.15.82:0/2971409572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f062806b170 con 0x7f0628101b20 2024-04-21T08:58:01.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.983+0000 7f0626ffd640 1 -- 172.21.15.82:0/2971409572 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11944 (secure 0 0 0) 0x7f061409c160 con 0x7f0628101b20 2024-04-21T08:58:01.983 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:58:01.983 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/467976762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":467976762},{"type":"v1","addr":"172.21.15.82:6835","nonce":467976762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3751789078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3751789078},{"type":"v1","addr":"172.21.15.82:6837","nonce":3751789078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:44:33.819176+0000","modified":"2024-04-21T08:57:34.850609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24481,"mds_2":24476,"mds_3":24409,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6839/3697606919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3697606919},{"type":"v1","addr":"172.21.15.82:6839","nonce":3697606919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6841/1327149046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1327149046},{"type":"v1","addr":"172.21.15.82:6841","nonce":1327149046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.106:6833/2583766346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2583766346},{"type":"v1","addr":"172.21.15.106:6833","nonce":2583766346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6833/2942732132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2942732132},{"type":"v1","addr":"172.21.15.134:6833","nonce":2942732132}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3901395826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3901395826},{"type":"v1","addr":"172.21.15.106:6835","nonce":3901395826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2962604291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2962604291},{"type":"v1","addr":"172.21.15.134:6835","nonce":2962604291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/1870695477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1870695477},{"type":"v1","addr":"172.21.15.134:6837","nonce":1870695477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/3731760515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":3731760515},{"type":"v1","addr":"172.21.15.106:6837","nonce":3731760515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6839/394037083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":394037083},{"type":"v1","addr":"172.21.15.134:6839","nonce":394037083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3796875133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3796875133},{"type":"v1","addr":"172.21.15.106:6839","nonce":3796875133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24509,24481,24476,24407,24409]},"id":1}]} 2024-04-21T08:58:01.983 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 49 2024-04-21T08:58:01.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.985+0000 7f0630198640 1 -- 172.21.15.82:0/2971409572 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f060c0a3da0 msgr2=0x7f060c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:01.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.985+0000 7f0630198640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f060c0a3da0 0x7f060c0a6260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0618005fd0 tx=0x7f0618009680 comp rx=0 tx=0).stop 2024-04-21T08:58:01.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.985+0000 7f0630198640 1 -- 172.21.15.82:0/2971409572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0628101b20 msgr2=0x7f0628197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:01.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.985+0000 7f0630198640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0628101b20 0x7f0628197b10 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f061400d950 tx=0x7f061400de10 comp rx=0 tx=0).stop 2024-04-21T08:58:01.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.985+0000 7f0630198640 1 -- 172.21.15.82:0/2971409572 shutdown_connections 2024-04-21T08:58:01.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.985+0000 7f0630198640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0628105ed0 0x7f062819be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.985+0000 7f0630198640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f060c0a3da0 0x7f060c0a6260 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.985+0000 7f0630198640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0628101b20 0x7f0628197b10 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.985+0000 7f0630198640 1 --2- 172.21.15.82:0/2971409572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06281011a0 0x7f06281975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:01.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.985+0000 7f0630198640 1 -- 172.21.15.82:0/2971409572 >> 172.21.15.82:0/2971409572 conn(0x7f0628077e30 msgr2=0x7f06280fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:01.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.985+0000 7f0630198640 1 -- 172.21.15.82:0/2971409572 shutdown_connections 2024-04-21T08:58:01.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:01.985+0000 7f0630198640 1 -- 172.21.15.82:0/2971409572 wait complete. 2024-04-21T08:58:01.996 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:58:02.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.095+0000 7f733e64a640 1 -- 172.21.15.82:0/2538793938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7338101ad0 msgr2=0x7f733810dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.095+0000 7f733e64a640 1 --2- 172.21.15.82:0/2538793938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7338101ad0 0x7f733810dcb0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7330009ca0 tx=0x7f733002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.095+0000 7f733e64a640 1 -- 172.21.15.82:0/2538793938 shutdown_connections 2024-04-21T08:58:02.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.095+0000 7f733e64a640 1 --2- 172.21.15.82:0/2538793938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7338101ad0 0x7f733810dcb0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.095+0000 7f733e64a640 1 --2- 172.21.15.82:0/2538793938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7338101130 0x7f7338101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.095+0000 7f733e64a640 1 --2- 172.21.15.82:0/2538793938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7338106f10 0x7f73381072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.095+0000 7f733e64a640 1 -- 172.21.15.82:0/2538793938 >> 172.21.15.82:0/2538793938 conn(0x7f7338077e30 msgr2=0x7f7338078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:02.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.095+0000 7f733e64a640 1 -- 172.21.15.82:0/2538793938 shutdown_connections 2024-04-21T08:58:02.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.095+0000 7f733e64a640 1 -- 172.21.15.82:0/2538793938 wait complete. 2024-04-21T08:58:02.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f733e64a640 1 Processor -- start 2024-04-21T08:58:02.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f733e64a640 1 -- start start 2024-04-21T08:58:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f733e64a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7338101130 0x7f7338198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f733e64a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7338101ad0 0x7f7338198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:02.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f733e64a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7338106f10 0x7f733819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f733e64a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7338111140 con 0x7f7338101130 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f733e64a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7338110fc0 con 0x7f7338106f10 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f733e64a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73381112c0 con 0x7f7338101ad0 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f7337fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7338101130 0x7f7338198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f7337fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7338101130 0x7f7338198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58780/0 (socket says 172.21.15.82:58780) 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f7337fff640 1 -- 172.21.15.82:0/814647177 learned_addr learned my addr 172.21.15.82:0/814647177 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f73377fe640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7338101ad0 0x7f7338198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f733cbe7640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7338106f10 0x7f733819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f7337fff640 1 -- 172.21.15.82:0/814647177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7338101ad0 msgr2=0x7f7338198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f7337fff640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7338101ad0 0x7f7338198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f7337fff640 1 -- 172.21.15.82:0/814647177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7338106f10 msgr2=0x7f733819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f7337fff640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7338106f10 0x7f733819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f7337fff640 1 -- 172.21.15.82:0/814647177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733819d740 con 0x7f7338101130 2024-04-21T08:58:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f733cbe7640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7338106f10 0x7f733819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:58:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.096+0000 7f7337fff640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7338101130 0x7f7338198750 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f732800c920 tx=0x7f732800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.097+0000 7f73357fa640 1 -- 172.21.15.82:0/814647177 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328007c70 con 0x7f7338101130 2024-04-21T08:58:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.097+0000 7f733e64a640 1 -- 172.21.15.82:0/814647177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733819da30 con 0x7f7338101130 2024-04-21T08:58:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.097+0000 7f73357fa640 1 -- 172.21.15.82:0/814647177 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7328007e10 con 0x7f7338101130 2024-04-21T08:58:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.097+0000 7f73357fa640 1 -- 172.21.15.82:0/814647177 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328018820 con 0x7f7338101130 2024-04-21T08:58:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.097+0000 7f733e64a640 1 -- 172.21.15.82:0/814647177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73381a52c0 con 0x7f7338101130 2024-04-21T08:58:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.098+0000 7f733e64a640 1 -- 172.21.15.82:0/814647177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7338069230 con 0x7f7338101130 2024-04-21T08:58:02.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.099+0000 7f73357fa640 1 -- 172.21.15.82:0/814647177 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f732800f450 con 0x7f7338101130 2024-04-21T08:58:02.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.099+0000 7f73357fa640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f73200a3d20 0x7f73200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:02.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.099+0000 7f73377fe640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f73200a3d20 0x7f73200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:02.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.099+0000 7f73357fa640 1 -- 172.21.15.82:0/814647177 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9518+0+0 (secure 0 0 0) 0x7f73280d0db0 con 0x7f7338101130 2024-04-21T08:58:02.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.100+0000 7f73377fe640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f73200a3d20 0x7f73200a61e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7338105d00 tx=0x7f732c008040 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:02.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.104+0000 7f73357fa640 1 -- 172.21.15.82:0/814647177 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7328014030 con 0x7f7338101130 2024-04-21T08:58:02.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.287+0000 7f733e64a640 1 -- 172.21.15.82:0/814647177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73381a54f0 con 0x7f7338101130 2024-04-21T08:58:02.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.289+0000 7f73357fa640 1 -- 172.21.15.82:0/814647177 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+25365 (secure 0 0 0) 0x7f732800de20 con 0x7f7338101130 2024-04-21T08:58:02.289 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-21T08:58:02.289 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":103,"fsid":"24d05dee-ffba-11ee-bc93-c7b262605968","created":"2024-04-21T08:36:25.047081+0000","modified":"2024-04-21T08:57:35.168644+0000","last_up_change":"2024-04-21T08:43:41.125302+0000","last_in_change":"2024-04-21T08:43:20.139166+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:39:50.316672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:44:32.331237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:44:32.865948+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:44:36.501832+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b97aa94e-52aa-4c7c-8a91-252060989aea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6803","nonce":1328184221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6805","nonce":1328184221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6809","nonce":1328184221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1328184221},{"type":"v1","addr":"172.21.15.82:6807","nonce":1328184221}]},"public_addr":"172.21.15.82:6803/1328184221","cluster_addr":"172.21.15.82:6805/1328184221","heartbeat_back_addr":"172.21.15.82:6809/1328184221","heartbeat_front_addr":"172.21.15.82:6807/1328184221","state":["exists","up"]},{"osd":1,"uuid":"209098d5-cae2-4947-8cb5-93c4c7fea11d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6801","nonce":1176470970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6803","nonce":1176470970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6807","nonce":1176470970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1176470970},{"type":"v1","addr":"172.21.15.106:6805","nonce":1176470970}]},"public_addr":"172.21.15.106:6801/1176470970","cluster_addr":"172.21.15.106:6803/1176470970","heartbeat_back_addr":"172.21.15.106:6807/1176470970","heartbeat_front_addr":"172.21.15.106:6805/1176470970","state":["exists","up"]},{"osd":2,"uuid":"dceeca7d-3295-4c47-93f3-694da1b97eff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6801","nonce":3690916351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6803","nonce":3690916351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6807","nonce":3690916351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":3690916351},{"type":"v1","addr":"172.21.15.134:6805","nonce":3690916351}]},"public_addr":"172.21.15.134:6801/3690916351","cluster_addr":"172.21.15.134:6803/3690916351","heartbeat_back_addr":"172.21.15.134:6807/3690916351","heartbeat_front_addr":"172.21.15.134:6805/3690916351","state":["exists","up"]},{"osd":3,"uuid":"db05ff77-7134-4823-a2ed-5f55bbc5d66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6811","nonce":2296521685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6813","nonce":2296521685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6817","nonce":2296521685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2296521685},{"type":"v1","addr":"172.21.15.82:6815","nonce":2296521685}]},"public_addr":"172.21.15.82:6811/2296521685","cluster_addr":"172.21.15.82:6813/2296521685","heartbeat_back_addr":"172.21.15.82:6817/2296521685","heartbeat_front_addr":"172.21.15.82:6815/2296521685","state":["exists","up"]},{"osd":4,"uuid":"56e090a3-ed81-46b8-8c14-a6fdbd57c344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6809","nonce":1658170110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6811","nonce":1658170110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6815","nonce":1658170110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":1658170110},{"type":"v1","addr":"172.21.15.106:6813","nonce":1658170110}]},"public_addr":"172.21.15.106:6809/1658170110","cluster_addr":"172.21.15.106:6811/1658170110","heartbeat_back_addr":"172.21.15.106:6815/1658170110","heartbeat_front_addr":"172.21.15.106:6813/1658170110","state":["exists","up"]},{"osd":5,"uuid":"c045cd42-40b5-4b5c-9d69-69a7ec4f0bef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6809","nonce":1529948096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6811","nonce":1529948096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6815","nonce":1529948096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":1529948096},{"type":"v1","addr":"172.21.15.134:6813","nonce":1529948096}]},"public_addr":"172.21.15.134:6809/1529948096","cluster_addr":"172.21.15.134:6811/1529948096","heartbeat_back_addr":"172.21.15.134:6815/1529948096","heartbeat_front_addr":"172.21.15.134:6813/1529948096","state":["exists","up"]},{"osd":6,"uuid":"2c1f3d11-aa46-4d37-9e6c-9b2778d3306c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6819","nonce":1243429969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6821","nonce":1243429969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6825","nonce":1243429969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1243429969},{"type":"v1","addr":"172.21.15.82:6823","nonce":1243429969}]},"public_addr":"172.21.15.82:6819/1243429969","cluster_addr":"172.21.15.82:6821/1243429969","heartbeat_back_addr":"172.21.15.82:6825/1243429969","heartbeat_front_addr":"172.21.15.82:6823/1243429969","state":["exists","up"]},{"osd":7,"uuid":"62f2af9e-02e2-436b-9041-60e4523eeeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6817","nonce":3692373036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6819","nonce":3692373036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6823","nonce":3692373036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":3692373036},{"type":"v1","addr":"172.21.15.106:6821","nonce":3692373036}]},"public_addr":"172.21.15.106:6817/3692373036","cluster_addr":"172.21.15.106:6819/3692373036","heartbeat_back_addr":"172.21.15.106:6823/3692373036","heartbeat_front_addr":"172.21.15.106:6821/3692373036","state":["exists","up"]},{"osd":8,"uuid":"e39282b2-0f53-4c57-a68c-771857b35413","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6817","nonce":1700011684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6819","nonce":1700011684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6823","nonce":1700011684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1700011684},{"type":"v1","addr":"172.21.15.134:6821","nonce":1700011684}]},"public_addr":"172.21.15.134:6817/1700011684","cluster_addr":"172.21.15.134:6819/1700011684","heartbeat_back_addr":"172.21.15.134:6823/1700011684","heartbeat_front_addr":"172.21.15.134:6821/1700011684","state":["exists","up"]},{"osd":9,"uuid":"5d8c9e2a-4cb6-442a-a767-9452c6cb7f06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6827","nonce":112604646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6829","nonce":112604646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6833","nonce":112604646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":112604646},{"type":"v1","addr":"172.21.15.82:6831","nonce":112604646}]},"public_addr":"172.21.15.82:6827/112604646","cluster_addr":"172.21.15.82:6829/112604646","heartbeat_back_addr":"172.21.15.82:6833/112604646","heartbeat_front_addr":"172.21.15.82:6831/112604646","state":["exists","up"]},{"osd":10,"uuid":"73ee0364-4255-443c-ac99-c7f6286afeee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6825","nonce":204007808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6827","nonce":204007808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6831","nonce":204007808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":204007808},{"type":"v1","addr":"172.21.15.106:6829","nonce":204007808}]},"public_addr":"172.21.15.106:6825/204007808","cluster_addr":"172.21.15.106:6827/204007808","heartbeat_back_addr":"172.21.15.106:6831/204007808","heartbeat_front_addr":"172.21.15.106:6829/204007808","state":["exists","up"]},{"osd":11,"uuid":"0f38b227-d954-4575-aced-ffd87058bf16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6825","nonce":1549345879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6827","nonce":1549345879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6831","nonce":1549345879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1549345879},{"type":"v1","addr":"172.21.15.134:6829","nonce":1549345879}]},"public_addr":"172.21.15.134:6825/1549345879","cluster_addr":"172.21.15.134:6827/1549345879","heartbeat_back_addr":"172.21.15.134:6831/1549345879","heartbeat_front_addr":"172.21.15.134:6829/1549345879","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:00.520988+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:24.200695+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:39:48.266856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:13.530985+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:40:37.914357+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:02.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:28.864723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:41:54.382639+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:19.723646+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:42:46.672628+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:12.572389+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:43:38.613254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2119539425":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6800/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/539366680":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/2275507980":"2024-04-22T08:47:00.617348+0000","172.21.15.82:0/4175378061":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/2889165689":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:0/3352705806":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6800/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/4224740504":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2873389149":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/2883183315":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/2138749385":"2024-04-22T08:47:00.617348+0000","172.21.15.82:6801/880382736":"2024-04-22T08:37:26.227727+0000","172.21.15.82:6801/2710226711":"2024-04-22T08:36:59.578707+0000","172.21.15.82:0/3032518359":"2024-04-22T08:36:59.578707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:58:02.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.291+0000 7f733e64a640 1 -- 172.21.15.82:0/814647177 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f73200a3d20 msgr2=0x7f73200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.291+0000 7f733e64a640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f73200a3d20 0x7f73200a61e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7338105d00 tx=0x7f732c008040 comp rx=0 tx=0).stop 2024-04-21T08:58:02.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.291+0000 7f733e64a640 1 -- 172.21.15.82:0/814647177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7338101130 msgr2=0x7f7338198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.291+0000 7f733e64a640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7338101130 0x7f7338198750 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f732800c920 tx=0x7f732800cde0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.291+0000 7f733e64a640 1 -- 172.21.15.82:0/814647177 shutdown_connections 2024-04-21T08:58:02.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.291+0000 7f733e64a640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7338106f10 0x7f733819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.291+0000 7f733e64a640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7338101ad0 0x7f7338198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.291+0000 7f733e64a640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f73200a3d20 0x7f73200a61e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.291+0000 7f733e64a640 1 --2- 172.21.15.82:0/814647177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7338101130 0x7f7338198750 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.291+0000 7f733e64a640 1 -- 172.21.15.82:0/814647177 >> 172.21.15.82:0/814647177 conn(0x7f7338077e30 msgr2=0x7f7338100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:02.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.291+0000 7f733e64a640 1 -- 172.21.15.82:0/814647177 shutdown_connections 2024-04-21T08:58:02.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.291+0000 7f733e64a640 1 -- 172.21.15.82:0/814647177 wait complete. 2024-04-21T08:58:02.301 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:02 smithi082 ceph-mon[27967]: pgmap v342: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:58:02.301 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:02 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2002467057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:58:02.301 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:02 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2971409572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:58:02.303 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-21T08:58:02.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.402+0000 7fd5e9d23640 1 -- 172.21.15.82:0/846612099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e4106170 msgr2=0x7fd5e410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.402+0000 7fd5e9d23640 1 --2- 172.21.15.82:0/846612099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e4106170 0x7fd5e410dd40 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009ca0 tx=0x7fd5dc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.402+0000 7fd5e9d23640 1 -- 172.21.15.82:0/846612099 shutdown_connections 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.402+0000 7fd5e9d23640 1 --2- 172.21.15.82:0/846612099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e4106170 0x7fd5e410dd40 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.402+0000 7fd5e9d23640 1 --2- 172.21.15.82:0/846612099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e4101a50 0x7fd5e4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.402+0000 7fd5e9d23640 1 --2- 172.21.15.82:0/846612099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5e41010a0 0x7fd5e4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.402+0000 7fd5e9d23640 1 -- 172.21.15.82:0/846612099 >> 172.21.15.82:0/846612099 conn(0x7fd5e40771d0 msgr2=0x7fd5e40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.402+0000 7fd5e9d23640 1 -- 172.21.15.82:0/846612099 shutdown_connections 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.402+0000 7fd5e9d23640 1 -- 172.21.15.82:0/846612099 wait complete. 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.403+0000 7fd5e9d23640 1 Processor -- start 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.403+0000 7fd5e9d23640 1 -- start start 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.403+0000 7fd5e9d23640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e41010a0 0x7fd5e4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.403+0000 7fd5e9d23640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e4106170 0x7fd5e4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:02.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.403+0000 7fd5e9d23640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5e419d010 0x7fd5e419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:02.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.403+0000 7fd5e9d23640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e41121f0 con 0x7fd5e4106170 2024-04-21T08:58:02.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.403+0000 7fd5e9d23640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4112070 con 0x7fd5e419d010 2024-04-21T08:58:02.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.403+0000 7fd5e9d23640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e4112370 con 0x7fd5e41010a0 2024-04-21T08:58:02.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.403+0000 7fd5e2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e4106170 0x7fd5e4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:02.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.403+0000 7fd5e2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e4106170 0x7fd5e4198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58810/0 (socket says 172.21.15.82:58810) 2024-04-21T08:58:02.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.403+0000 7fd5e2ffd640 1 -- 172.21.15.82:0/2182357029 learned_addr learned my addr 172.21.15.82:0/2182357029 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:58:02.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.403+0000 7fd5e3fff640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5e419d010 0x7fd5e419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:02.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e37fe640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e41010a0 0x7fd5e4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:02.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e2ffd640 1 -- 172.21.15.82:0/2182357029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e41010a0 msgr2=0x7fd5e4198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e2ffd640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e41010a0 0x7fd5e4198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e2ffd640 1 -- 172.21.15.82:0/2182357029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5e419d010 msgr2=0x7fd5e419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e2ffd640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5e419d010 0x7fd5e419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e2ffd640 1 -- 172.21.15.82:0/2182357029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e419db00 con 0x7fd5e4106170 2024-04-21T08:58:02.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e37fe640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e41010a0 0x7fd5e4198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:58:02.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e2ffd640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e4106170 0x7fd5e4198c90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800d740 tx=0x7fd5d800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:02.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e0ff9640 1 -- 172.21.15.82:0/2182357029 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8017070 con 0x7fd5e4106170 2024-04-21T08:58:02.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e0ff9640 1 -- 172.21.15.82:0/2182357029 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d800de50 con 0x7fd5e4106170 2024-04-21T08:58:02.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e9d23640 1 -- 172.21.15.82:0/2182357029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e419ddf0 con 0x7fd5e4106170 2024-04-21T08:58:02.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e0ff9640 1 -- 172.21.15.82:0/2182357029 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8004ce0 con 0x7fd5e4106170 2024-04-21T08:58:02.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.404+0000 7fd5e9d23640 1 -- 172.21.15.82:0/2182357029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e41a5590 con 0x7fd5e4106170 2024-04-21T08:58:02.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.405+0000 7fd5e9d23640 1 -- 172.21.15.82:0/2182357029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e40691c0 con 0x7fd5e4106170 2024-04-21T08:58:02.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.406+0000 7fd5e0ff9640 1 -- 172.21.15.82:0/2182357029 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd5d8004e80 con 0x7fd5e4106170 2024-04-21T08:58:02.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.407+0000 7fd5e0ff9640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5cc0a3d20 0x7fd5cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:02.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.407+0000 7fd5e37fe640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5cc0a3d20 0x7fd5cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:02.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.407+0000 7fd5e0ff9640 1 -- 172.21.15.82:0/2182357029 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9518+0+0 (secure 0 0 0) 0x7fd5d80d0950 con 0x7fd5e4106170 2024-04-21T08:58:02.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.407+0000 7fd5e37fe640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5cc0a3d20 0x7fd5cc0a61e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40058c0 tx=0x7fd5d4005850 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:02.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.411+0000 7fd5e0ff9640 1 -- 172.21.15.82:0/2182357029 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5d809cc10 con 0x7fd5e4106170 2024-04-21T08:58:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-mon[31772]: pgmap v342: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:58:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2002467057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:58:02.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2971409572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:58:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-mon[31072]: pgmap v342: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-21T08:58:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2002467057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:58:02.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2971409572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:58:02.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.632+0000 7fd5e9d23640 1 -- 172.21.15.82:0/2182357029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd5e4199aa0 con 0x7fd5e4106170 2024-04-21T08:58:02.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.635+0000 7fd5e0ff9640 1 -- 172.21.15.82:0/2182357029 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) ==== 161+0+0 (secure 0 0 0) 0x7fd5e4199aa0 con 0x7fd5e4106170 2024-04-21T08:58:02.635 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-21T08:58:02.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.639+0000 7fd5e9d23640 1 -- 172.21.15.82:0/2182357029 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5cc0a3d20 msgr2=0x7fd5cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.639+0000 7fd5e9d23640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5cc0a3d20 0x7fd5cc0a61e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40058c0 tx=0x7fd5d4005850 comp rx=0 tx=0).stop 2024-04-21T08:58:02.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.639+0000 7fd5e9d23640 1 -- 172.21.15.82:0/2182357029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e4106170 msgr2=0x7fd5e4198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.639+0000 7fd5e9d23640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e4106170 0x7fd5e4198c90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800d740 tx=0x7fd5d800dc00 comp rx=0 tx=0).stop 2024-04-21T08:58:02.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.639+0000 7fd5e9d23640 1 -- 172.21.15.82:0/2182357029 shutdown_connections 2024-04-21T08:58:02.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.639+0000 7fd5e9d23640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5e419d010 0x7fd5e419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.639+0000 7fd5e9d23640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd5cc0a3d20 0x7fd5cc0a61e0 secure :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40058c0 tx=0x7fd5d4005850 comp rx=0 tx=0).stop 2024-04-21T08:58:02.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.639+0000 7fd5e9d23640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5e4106170 0x7fd5e4198c90 secure :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800d740 tx=0x7fd5d800dc00 comp rx=0 tx=0).stop 2024-04-21T08:58:02.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.639+0000 7fd5e9d23640 1 --2- 172.21.15.82:0/2182357029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5e41010a0 0x7fd5e4198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.639+0000 7fd5e9d23640 1 -- 172.21.15.82:0/2182357029 >> 172.21.15.82:0/2182357029 conn(0x7fd5e40771d0 msgr2=0x7fd5e40feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:02.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.640+0000 7fd5e9d23640 1 -- 172.21.15.82:0/2182357029 shutdown_connections 2024-04-21T08:58:02.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.640+0000 7fd5e9d23640 1 -- 172.21.15.82:0/2182357029 wait complete. 2024-04-21T08:58:02.652 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-21T08:58:02.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.758+0000 7f05b633d640 1 -- 172.21.15.82:0/2955585301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b0101a40 msgr2=0x7f05b0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.758+0000 7f05b633d640 1 --2- 172.21.15.82:0/2955585301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b0101a40 0x7f05b0105a80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009900 tx=0x7f05a402bc30 comp rx=0 tx=0).stop 2024-04-21T08:58:02.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.758+0000 7f05b633d640 1 -- 172.21.15.82:0/2955585301 shutdown_connections 2024-04-21T08:58:02.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.758+0000 7f05b633d640 1 --2- 172.21.15.82:0/2955585301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b0106160 0x7f05b010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.758+0000 7f05b633d640 1 --2- 172.21.15.82:0/2955585301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b0101a40 0x7f05b0105a80 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.758+0000 7f05b633d640 1 --2- 172.21.15.82:0/2955585301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b0101090 0x7f05b0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.758+0000 7f05b633d640 1 -- 172.21.15.82:0/2955585301 >> 172.21.15.82:0/2955585301 conn(0x7f05b00771d0 msgr2=0x7f05b00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:02.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.758+0000 7f05b633d640 1 -- 172.21.15.82:0/2955585301 shutdown_connections 2024-04-21T08:58:02.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.758+0000 7f05b633d640 1 -- 172.21.15.82:0/2955585301 wait complete. 2024-04-21T08:58:02.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.758+0000 7f05b633d640 1 Processor -- start 2024-04-21T08:58:02.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.758+0000 7f05b633d640 1 -- start start 2024-04-21T08:58:02.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05b633d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b0101090 0x7f05b0199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:02.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05b633d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b0106160 0x7f05b0199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:02.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05b633d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b019a050 0x7f05b01a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:02.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05b633d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f05b0111080 con 0x7f05b0101090 2024-04-21T08:58:02.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05b633d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f05b0110f00 con 0x7f05b0106160 2024-04-21T08:58:02.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05b633d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f05b0111200 con 0x7f05b019a050 2024-04-21T08:58:02.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b0101090 0x7f05b0199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:02.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b0101090 0x7f05b0199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60162/0 (socket says 172.21.15.82:60162) 2024-04-21T08:58:02.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05affff640 1 -- 172.21.15.82:0/3616899665 learned_addr learned my addr 172.21.15.82:0/3616899665 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:58:02.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05af7fe640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b0106160 0x7f05b0199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:02.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05b48da640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b019a050 0x7f05b01a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:02.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05affff640 1 -- 172.21.15.82:0/3616899665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b019a050 msgr2=0x7f05b01a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05affff640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b019a050 0x7f05b01a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05affff640 1 -- 172.21.15.82:0/3616899665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b0106160 msgr2=0x7f05b0199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:02.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05affff640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b0106160 0x7f05b0199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:02.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05affff640 1 -- 172.21.15.82:0/3616899665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b01a4e00 con 0x7f05b0101090 2024-04-21T08:58:02.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05affff640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b0101090 0x7f05b0199580 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f05a000e9f0 tx=0x7f05a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:02.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05ad7fa640 1 -- 172.21.15.82:0/3616899665 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a000cd80 con 0x7f05b0101090 2024-04-21T08:58:02.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05b633d640 1 -- 172.21.15.82:0/3616899665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b01a50f0 con 0x7f05b0101090 2024-04-21T08:58:02.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05ad7fa640 1 -- 172.21.15.82:0/3616899665 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a00108e0 con 0x7f05b0101090 2024-04-21T08:58:02.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05ad7fa640 1 -- 172.21.15.82:0/3616899665 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0018980 con 0x7f05b0101090 2024-04-21T08:58:02.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.759+0000 7f05b633d640 1 -- 172.21.15.82:0/3616899665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b01a5660 con 0x7f05b0101090 2024-04-21T08:58:02.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.760+0000 7f05b633d640 1 -- 172.21.15.82:0/3616899665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b0102560 con 0x7f05b0101090 2024-04-21T08:58:02.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.761+0000 7f05ad7fa640 1 -- 172.21.15.82:0/3616899665 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05a0010430 con 0x7f05b0101090 2024-04-21T08:58:02.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.762+0000 7f05ad7fa640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f05980a3d20 0x7f05980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:02.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.762+0000 7f05af7fe640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f05980a3d20 0x7f05980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:02.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.762+0000 7f05ad7fa640 1 -- 172.21.15.82:0/3616899665 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9904+0+0 (secure 0 0 0) 0x7f05a00d15f0 con 0x7f05b0101090 2024-04-21T08:58:02.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.763+0000 7f05af7fe640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f05980a3d20 0x7f05980a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009230 tx=0x7f05a4005fd0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:02.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.765+0000 7f05ad7fa640 1 -- 172.21.15.82:0/3616899665 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05a009d730 con 0x7f05b0101090 2024-04-21T08:58:02.976 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c[62554]: ignoring --setuser ceph since I am not root 2024-04-21T08:58:02.976 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c[62554]: ignoring --setgroup ceph since I am not root 2024-04-21T08:58:02.976 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c[62554]: starting mds.c at 2024-04-21T08:58:02.977 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-l[61925]: ignoring --setuser ceph since I am not root 2024-04-21T08:58:02.977 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-l[61925]: ignoring --setgroup ceph since I am not root 2024-04-21T08:58:02.977 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-l[61925]: starting mds.l at 2024-04-21T08:58:02.978 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-i[62856]: ignoring --setuser ceph since I am not root 2024-04-21T08:58:02.978 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-i[62856]: ignoring --setgroup ceph since I am not root 2024-04-21T08:58:02.978 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-mds[62860]: -- 172.21.15.134:0/624341774 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be1d5aba40 con 0x55be1d57ec00 2024-04-21T08:58:02.978 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-i[62856]: starting mds.i at 2024-04-21T08:58:02.978 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-f[62239]: ignoring --setuser ceph since I am not root 2024-04-21T08:58:02.978 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-f[62239]: ignoring --setgroup ceph since I am not root 2024-04-21T08:58:02.978 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-mds[62243]: -- 172.21.15.134:0/2686564173 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56103a4704e0 con 0x56103a446800 2024-04-21T08:58:02.978 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-mds[62243]: -- 172.21.15.134:0/2686564173 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56103a473a40 con 0x56103a446800 2024-04-21T08:58:02.978 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 21 08:58:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-f[62239]: starting mds.f at 2024-04-21T08:58:02.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:02.986+0000 7f05b633d640 1 -- 172.21.15.82:0/3616899665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f05b0109390 con 0x7f05b0101090 2024-04-21T08:58:03.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a[27963]: 2024-04-21T08:58:02.633+0000 7fe7a25c8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T08:58:03.109 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:58:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-g[69899]: ignoring --setuser ceph since I am not root 2024-04-21T08:58:03.109 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:58:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-g[69899]: ignoring --setgroup ceph since I am not root 2024-04-21T08:58:03.109 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:58:02 smithi082 ceph-mds[69903]: -- 172.21.15.82:0/1191717410 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eed13f1a40 con 0x55eed13c4800 2024-04-21T08:58:03.109 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:58:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-g[69899]: starting mds.g at 2024-04-21T08:58:03.109 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:58:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-a[70246]: ignoring --setuser ceph since I am not root 2024-04-21T08:58:03.109 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:58:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-a[70246]: ignoring --setgroup ceph since I am not root 2024-04-21T08:58:03.109 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:58:02 smithi082 ceph-mds[70250]: -- 172.21.15.82:0/3169615276 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559fb54dfa40 con 0x559fb54b3000 2024-04-21T08:58:03.109 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:58:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-a[70246]: starting mds.a at 2024-04-21T08:58:03.128 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-e[60220]: ignoring --setuser ceph since I am not root 2024-04-21T08:58:03.128 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-e[60220]: ignoring --setgroup ceph since I am not root 2024-04-21T08:58:03.129 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-mds[60224]: -- 172.21.15.106:0/1775615629 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a55663a40 con 0x560a55636800 2024-04-21T08:58:03.129 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-e[60220]: starting mds.e at 2024-04-21T08:58:03.129 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b[60834]: ignoring --setuser ceph since I am not root 2024-04-21T08:58:03.129 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b[60834]: ignoring --setgroup ceph since I am not root 2024-04-21T08:58:03.129 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-mds[60838]: -- 172.21.15.106:0/857546394 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645d2677a40 con 0x5645d264ac00 2024-04-21T08:58:03.129 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b[60834]: starting mds.b at 2024-04-21T08:58:03.129 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-k[59900]: ignoring --setuser ceph since I am not root 2024-04-21T08:58:03.130 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-k[59900]: ignoring --setgroup ceph since I am not root 2024-04-21T08:58:03.130 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-mds[59904]: -- 172.21.15.106:0/2665411746 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b2857fa40 con 0x558b28552800 2024-04-21T08:58:03.130 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-k[59900]: starting mds.k at 2024-04-21T08:58:03.130 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-h[60529]: ignoring --setuser ceph since I am not root 2024-04-21T08:58:03.130 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-h[60529]: ignoring --setgroup ceph since I am not root 2024-04-21T08:58:03.130 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-mds[60533]: -- 172.21.15.106:0/2372978288 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560963de9a40 con 0x560963dbcc00 2024-04-21T08:58:03.130 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 21 08:58:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-h[60529]: starting mds.h at 2024-04-21T08:58:03.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:03 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/814647177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:58:03.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:03 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2182357029' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T08:58:03.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:03 smithi134 ceph-mon[31772]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T08:58:03.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:03 smithi134 ceph-mon[31772]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T08:58:03.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:03 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2182357029' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T08:58:03.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:03 smithi134 ceph-mon[31772]: osdmap e104: 12 total, 12 up, 12 in 2024-04-21T08:58:03.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:03 smithi134 ceph-mon[31772]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-21T08:58:03.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:03 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3616899665' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:58:03.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:03 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/814647177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:58:03.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:03 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2182357029' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T08:58:03.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:03 smithi082 ceph-mon[27967]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T08:58:03.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:03 smithi082 ceph-mon[27967]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T08:58:03.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:03 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2182357029' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T08:58:03.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:03 smithi082 ceph-mon[27967]: osdmap e104: 12 total, 12 up, 12 in 2024-04-21T08:58:03.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:03 smithi082 ceph-mon[27967]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-21T08:58:03.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:03 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3616899665' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:58:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:03 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/814647177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:58:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:03 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2182357029' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T08:58:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:03 smithi106 ceph-mon[31072]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T08:58:03.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:03 smithi106 ceph-mon[31072]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T08:58:03.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:03 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2182357029' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T08:58:03.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:03 smithi106 ceph-mon[31072]: osdmap e104: 12 total, 12 up, 12 in 2024-04-21T08:58:03.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:03 smithi106 ceph-mon[31072]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-21T08:58:03.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:03 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3616899665' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:58:03.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.644+0000 7f05ad7fa640 1 -- 172.21.15.82:0/3616899665 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) ==== 104+0+0 (secure 0 0 0) 0x7f05a009d030 con 0x7f05b0101090 2024-04-21T08:58:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.647+0000 7f05b633d640 1 -- 172.21.15.82:0/3616899665 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f05980a3d20 msgr2=0x7f05980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.647+0000 7f05b633d640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f05980a3d20 0x7f05980a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009230 tx=0x7f05a4005fd0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.647+0000 7f05b633d640 1 -- 172.21.15.82:0/3616899665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b0101090 msgr2=0x7f05b0199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.647+0000 7f05b633d640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b0101090 0x7f05b0199580 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f05a000e9f0 tx=0x7f05a000eeb0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.647+0000 7f05b633d640 1 -- 172.21.15.82:0/3616899665 shutdown_connections 2024-04-21T08:58:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.647+0000 7f05b633d640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05b019a050 0x7f05b01a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.647+0000 7f05b633d640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05b0106160 0x7f05b0199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.647+0000 7f05b633d640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f05980a3d20 0x7f05980a61e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.647+0000 7f05b633d640 1 --2- 172.21.15.82:0/3616899665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b0101090 0x7f05b0199580 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.647+0000 7f05b633d640 1 -- 172.21.15.82:0/3616899665 >> 172.21.15.82:0/3616899665 conn(0x7f05b00771d0 msgr2=0x7f05b00fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.647+0000 7f05b633d640 1 -- 172.21.15.82:0/3616899665 shutdown_connections 2024-04-21T08:58:03.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.647+0000 7f05b633d640 1 -- 172.21.15.82:0/3616899665 wait complete. 2024-04-21T08:58:03.664 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-21T08:58:03.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.772+0000 7fd17aff4640 1 -- 172.21.15.82:0/1047757084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd174101a30 msgr2=0x7fd174105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.772+0000 7fd17aff4640 1 --2- 172.21.15.82:0/1047757084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd174101a30 0x7fd174105a70 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd160009900 tx=0x7fd16002bc30 comp rx=0 tx=0).stop 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.772+0000 7fd17aff4640 1 -- 172.21.15.82:0/1047757084 shutdown_connections 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.772+0000 7fd17aff4640 1 --2- 172.21.15.82:0/1047757084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd174106150 0x7fd17410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.772+0000 7fd17aff4640 1 --2- 172.21.15.82:0/1047757084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd174101a30 0x7fd174105a70 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.772+0000 7fd17aff4640 1 --2- 172.21.15.82:0/1047757084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd174101080 0x7fd174101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.772+0000 7fd17aff4640 1 -- 172.21.15.82:0/1047757084 >> 172.21.15.82:0/1047757084 conn(0x7fd174078000 msgr2=0x7fd174078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.772+0000 7fd17aff4640 1 -- 172.21.15.82:0/1047757084 shutdown_connections 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.772+0000 7fd17aff4640 1 -- 172.21.15.82:0/1047757084 wait complete. 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd17aff4640 1 Processor -- start 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd17aff4640 1 -- start start 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd17aff4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd174101080 0x7fd174198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd17aff4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd174106150 0x7fd174198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:03.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd17aff4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd17419cf50 0x7fd17419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd17aff4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd174111070 con 0x7fd174106150 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd17aff4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd174110ef0 con 0x7fd17419cf50 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd17aff4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1741111f0 con 0x7fd174101080 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd173fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd174106150 0x7fd174198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd173fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd174106150 0x7fd174198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60178/0 (socket says 172.21.15.82:60178) 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd173fff640 1 -- 172.21.15.82:0/3230495217 learned_addr learned my addr 172.21.15.82:0/3230495217 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd179591640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd17419cf50 0x7fd17419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd178d90640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd174101080 0x7fd174198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd173fff640 1 -- 172.21.15.82:0/3230495217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd174101080 msgr2=0x7fd174198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd173fff640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd174101080 0x7fd174198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd173fff640 1 -- 172.21.15.82:0/3230495217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd17419cf50 msgr2=0x7fd17419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd173fff640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd17419cf50 0x7fd17419d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd173fff640 1 -- 172.21.15.82:0/3230495217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17419da40 con 0x7fd174106150 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.773+0000 7fd179591640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd17419cf50 0x7fd17419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:58:03.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.774+0000 7fd173fff640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd174106150 0x7fd174198bd0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd160036040 tx=0x7fd1600045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:03.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.774+0000 7fd171ffb640 1 -- 172.21.15.82:0/3230495217 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160043020 con 0x7fd174106150 2024-04-21T08:58:03.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.774+0000 7fd171ffb640 1 -- 172.21.15.82:0/3230495217 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd16002cb70 con 0x7fd174106150 2024-04-21T08:58:03.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.774+0000 7fd17aff4640 1 -- 172.21.15.82:0/3230495217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17419dcd0 con 0x7fd174106150 2024-04-21T08:58:03.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.774+0000 7fd171ffb640 1 -- 172.21.15.82:0/3230495217 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160003bf0 con 0x7fd174106150 2024-04-21T08:58:03.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.774+0000 7fd17aff4640 1 -- 172.21.15.82:0/3230495217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1741a54f0 con 0x7fd174106150 2024-04-21T08:58:03.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.775+0000 7fd17aff4640 1 -- 172.21.15.82:0/3230495217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd174068d70 con 0x7fd174106150 2024-04-21T08:58:03.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.777+0000 7fd171ffb640 1 -- 172.21.15.82:0/3230495217 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd160046020 con 0x7fd174106150 2024-04-21T08:58:03.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.777+0000 7fd171ffb640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd1640a3d20 0x7fd1640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:03.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.778+0000 7fd171ffb640 1 -- 172.21.15.82:0/3230495217 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9904+0+0 (secure 0 0 0) 0x7fd1600f32b0 con 0x7fd174106150 2024-04-21T08:58:03.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.778+0000 7fd178d90640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd1640a3d20 0x7fd1640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:03.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.778+0000 7fd178d90640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd1640a3d20 0x7fd1640a61e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd168005e10 tx=0x7fd168005da0 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:03.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.780+0000 7fd171ffb640 1 -- 172.21.15.82:0/3230495217 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd16004d020 con 0x7fd174106150 2024-04-21T08:58:03.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:03.966+0000 7fd17aff4640 1 -- 172.21.15.82:0/3230495217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd1741999e0 con 0x7fd174106150 2024-04-21T08:58:04.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: pgmap v344: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: Cluster is now healthy 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3616899665' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.82:6838/3793731761,v1:172.21.15.82:6840/3793731761] up:boot 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.82:6839/556720661,v1:172.21.15.82:6841/556720661] up:boot 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.134:6832/2632040663,v1:172.21.15.134:6833/2632040663] up:boot 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.134:6834/3466091121,v1:172.21.15.134:6835/3466091121] up:boot 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.134:6836/604398914,v1:172.21.15.134:6837/604398914] up:boot 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.134:6838/1079412731,v1:172.21.15.134:6839/1079412731] up:boot 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.106:6832/2861363955,v1:172.21.15.106:6834/2861363955] up:boot 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.106:6833/1957323222,v1:172.21.15.106:6835/1957323222] up:boot 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.106:6836/3248476447,v1:172.21.15.106:6838/3248476447] up:boot 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: mds.? [v2:172.21.15.106:6837/1256544519,v1:172.21.15.106:6839/1256544519] up:boot 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: fsmap 12 up:standby 2024-04-21T08:58:04.477 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:58:04.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:58:04.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:58:04.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:58:04.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:58:04.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:58:04.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:58:04.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:58:04.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:58:04.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:58:04.478 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:04 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3230495217' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:58:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: pgmap v344: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-04-21T08:58:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T08:58:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T08:58:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: Cluster is now healthy 2024-04-21T08:58:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3616899665' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:58:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.82:6838/3793731761,v1:172.21.15.82:6840/3793731761] up:boot 2024-04-21T08:58:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.82:6839/556720661,v1:172.21.15.82:6841/556720661] up:boot 2024-04-21T08:58:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.134:6832/2632040663,v1:172.21.15.134:6833/2632040663] up:boot 2024-04-21T08:58:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.134:6834/3466091121,v1:172.21.15.134:6835/3466091121] up:boot 2024-04-21T08:58:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.134:6836/604398914,v1:172.21.15.134:6837/604398914] up:boot 2024-04-21T08:58:04.609 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.134:6838/1079412731,v1:172.21.15.134:6839/1079412731] up:boot 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.106:6832/2861363955,v1:172.21.15.106:6834/2861363955] up:boot 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.106:6833/1957323222,v1:172.21.15.106:6835/1957323222] up:boot 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.106:6836/3248476447,v1:172.21.15.106:6838/3248476447] up:boot 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: mds.? [v2:172.21.15.106:6837/1256544519,v1:172.21.15.106:6839/1256544519] up:boot 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: fsmap 12 up:standby 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:58:04.610 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:58:04.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:58:04.611 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:04 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3230495217' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:58:04.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: pgmap v344: 129 pgs: 129 active+clean; 861 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 33 op/s 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: Cluster is now healthy 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3616899665' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.82:6838/3793731761,v1:172.21.15.82:6840/3793731761] up:boot 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.82:6839/556720661,v1:172.21.15.82:6841/556720661] up:boot 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.134:6832/2632040663,v1:172.21.15.134:6833/2632040663] up:boot 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.134:6834/3466091121,v1:172.21.15.134:6835/3466091121] up:boot 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.134:6836/604398914,v1:172.21.15.134:6837/604398914] up:boot 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.134:6838/1079412731,v1:172.21.15.134:6839/1079412731] up:boot 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.106:6832/2861363955,v1:172.21.15.106:6834/2861363955] up:boot 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.106:6833/1957323222,v1:172.21.15.106:6835/1957323222] up:boot 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.106:6836/3248476447,v1:172.21.15.106:6838/3248476447] up:boot 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: mds.? [v2:172.21.15.106:6837/1256544519,v1:172.21.15.106:6839/1256544519] up:boot 2024-04-21T08:58:04.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: fsmap 12 up:standby 2024-04-21T08:58:04.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:58:04.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:58:04.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:58:04.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:58:04.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:58:04.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:58:04.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:58:04.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:58:04.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:58:04.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: from='mgr.15546 172.21.15.82:0/1274786181' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:58:04.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:04 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3230495217' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:58:04.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.647+0000 7fd171ffb640 1 -- 172.21.15.82:0/3230495217 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7fd1741999e0 con 0x7fd174106150 2024-04-21T08:58:04.647 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-04-21T08:58:04.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.649+0000 7fd17aff4640 1 -- 172.21.15.82:0/3230495217 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd1640a3d20 msgr2=0x7fd1640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:04.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.649+0000 7fd17aff4640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd1640a3d20 0x7fd1640a61e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd168005e10 tx=0x7fd168005da0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.649+0000 7fd17aff4640 1 -- 172.21.15.82:0/3230495217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd174106150 msgr2=0x7fd174198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:04.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.649+0000 7fd17aff4640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd174106150 0x7fd174198bd0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd160036040 tx=0x7fd1600045e0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.650+0000 7fd17aff4640 1 -- 172.21.15.82:0/3230495217 shutdown_connections 2024-04-21T08:58:04.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.650+0000 7fd17aff4640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd17419cf50 0x7fd17419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.650+0000 7fd17aff4640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fd1640a3d20 0x7fd1640a61e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.650+0000 7fd17aff4640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd174106150 0x7fd174198bd0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.650+0000 7fd17aff4640 1 --2- 172.21.15.82:0/3230495217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd174101080 0x7fd174198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.650+0000 7fd17aff4640 1 -- 172.21.15.82:0/3230495217 >> 172.21.15.82:0/3230495217 conn(0x7fd174078000 msgr2=0x7fd1740693e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:04.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.650+0000 7fd17aff4640 1 -- 172.21.15.82:0/3230495217 shutdown_connections 2024-04-21T08:58:04.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.650+0000 7fd17aff4640 1 -- 172.21.15.82:0/3230495217 wait complete. 2024-04-21T08:58:04.663 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-21T08:58:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.768+0000 7f77fc04e640 1 -- 172.21.15.82:0/2199598588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f4101090 msgr2=0x7f77f4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.768+0000 7f77fc04e640 1 --2- 172.21.15.82:0/2199598588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f4101090 0x7f77f4101470 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009900 tx=0x7f77e402bc30 comp rx=0 tx=0).stop 2024-04-21T08:58:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.768+0000 7f77fc04e640 1 -- 172.21.15.82:0/2199598588 shutdown_connections 2024-04-21T08:58:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.768+0000 7f77fc04e640 1 --2- 172.21.15.82:0/2199598588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77f4106160 0x7f77f410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.768+0000 7f77fc04e640 1 --2- 172.21.15.82:0/2199598588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77f4101a40 0x7f77f4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.768+0000 7f77fc04e640 1 --2- 172.21.15.82:0/2199598588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f4101090 0x7f77f4101470 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.768+0000 7f77fc04e640 1 -- 172.21.15.82:0/2199598588 >> 172.21.15.82:0/2199598588 conn(0x7f77f4077f50 msgr2=0x7f77f4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.768+0000 7f77fc04e640 1 -- 172.21.15.82:0/2199598588 shutdown_connections 2024-04-21T08:58:04.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.768+0000 7f77fc04e640 1 -- 172.21.15.82:0/2199598588 wait complete. 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77fc04e640 1 Processor -- start 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77fc04e640 1 -- start start 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77fc04e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f4101090 0x7f77f4102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77fc04e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77f4106160 0x7f77f4103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77fc04e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77f41035a0 0x7f77f41a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77fc04e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41110a0 con 0x7f77f4101090 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77fc04e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4110f20 con 0x7f77f41035a0 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77fc04e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4111220 con 0x7f77f4106160 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77f9dea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f4101090 0x7f77f4102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77f9dea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f4101090 0x7f77f4102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60194/0 (socket says 172.21.15.82:60194) 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77f9dea640 1 -- 172.21.15.82:0/2640174223 learned_addr learned my addr 172.21.15.82:0/2640174223 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77fa5eb640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77f41035a0 0x7f77f41a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77f9dea640 1 -- 172.21.15.82:0/2640174223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77f4106160 msgr2=0x7f77f4103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77f9dea640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77f4106160 0x7f77f4103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77f9dea640 1 -- 172.21.15.82:0/2640174223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77f41035a0 msgr2=0x7f77f41a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:04.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77f9dea640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77f41035a0 0x7f77f41a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:04.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.769+0000 7f77f9dea640 1 -- 172.21.15.82:0/2640174223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f41a6f00 con 0x7f77f4101090 2024-04-21T08:58:04.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.770+0000 7f77f9dea640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f4101090 0x7f77f4102b20 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f77e4005940 tx=0x7f77e402c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:04.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.770+0000 7f77eaffd640 1 -- 172.21.15.82:0/2640174223 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4043020 con 0x7f77f4101090 2024-04-21T08:58:04.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.770+0000 7f77fc04e640 1 -- 172.21.15.82:0/2640174223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f41a7190 con 0x7f77f4101090 2024-04-21T08:58:04.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.770+0000 7f77fc04e640 1 -- 172.21.15.82:0/2640174223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f41a7730 con 0x7f77f4101090 2024-04-21T08:58:04.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.770+0000 7f77eaffd640 1 -- 172.21.15.82:0/2640174223 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77e402cc60 con 0x7f77f4101090 2024-04-21T08:58:04.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.770+0000 7f77eaffd640 1 -- 172.21.15.82:0/2640174223 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4003c30 con 0x7f77f4101090 2024-04-21T08:58:04.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.771+0000 7f77fc04e640 1 -- 172.21.15.82:0/2640174223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d8005290 con 0x7f77f4101090 2024-04-21T08:58:04.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.772+0000 7f77eaffd640 1 -- 172.21.15.82:0/2640174223 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f77e4046020 con 0x7f77f4101090 2024-04-21T08:58:04.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.772+0000 7f77eaffd640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f77e00a3d50 0x7f77e00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:04.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.772+0000 7f77f95e9640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f77e00a3d50 0x7f77e00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:04.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.772+0000 7f77eaffd640 1 -- 172.21.15.82:0/2640174223 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9379+0+0 (secure 0 0 0) 0x7f77e40f26f0 con 0x7f77f4101090 2024-04-21T08:58:04.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.773+0000 7f77f95e9640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f77e00a3d50 0x7f77e00a6210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f77f41045b0 tx=0x7f77f0005c70 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:04.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.777+0000 7f77eaffd640 1 -- 172.21.15.82:0/2640174223 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77e404d030 con 0x7f77f4101090 2024-04-21T08:58:04.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:04.977+0000 7f77fc04e640 1 -- 172.21.15.82:0/2640174223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f77d8005580 con 0x7f77f4101090 2024-04-21T08:58:05.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.653+0000 7f77eaffd640 1 -- 172.21.15.82:0/2640174223 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) ==== 169+0+0 (secure 0 0 0) 0x7f77e40035f0 con 0x7f77f4101090 2024-04-21T08:58:05.654 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-04-21T08:58:05.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.656+0000 7f77fc04e640 1 -- 172.21.15.82:0/2640174223 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f77e00a3d50 msgr2=0x7f77e00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:05.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.656+0000 7f77fc04e640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f77e00a3d50 0x7f77e00a6210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f77f41045b0 tx=0x7f77f0005c70 comp rx=0 tx=0).stop 2024-04-21T08:58:05.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.656+0000 7f77fc04e640 1 -- 172.21.15.82:0/2640174223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f4101090 msgr2=0x7f77f4102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:05.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.656+0000 7f77fc04e640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f4101090 0x7f77f4102b20 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f77e4005940 tx=0x7f77e402c860 comp rx=0 tx=0).stop 2024-04-21T08:58:05.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.657+0000 7f77fc04e640 1 -- 172.21.15.82:0/2640174223 shutdown_connections 2024-04-21T08:58:05.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.657+0000 7f77fc04e640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77f41035a0 0x7f77f41a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.657+0000 7f77fc04e640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77f4106160 0x7f77f4103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.657+0000 7f77fc04e640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7f77e00a3d50 0x7f77e00a6210 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.657+0000 7f77fc04e640 1 --2- 172.21.15.82:0/2640174223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77f4101090 0x7f77f4102b20 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.657+0000 7f77fc04e640 1 -- 172.21.15.82:0/2640174223 >> 172.21.15.82:0/2640174223 conn(0x7f77f4077f50 msgr2=0x7f77f40693f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:05.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.657+0000 7f77fc04e640 1 -- 172.21.15.82:0/2640174223 shutdown_connections 2024-04-21T08:58:05.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.657+0000 7f77fc04e640 1 -- 172.21.15.82:0/2640174223 wait complete. 2024-04-21T08:58:05.676 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-21T08:58:05.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.794+0000 7fa5359e6640 1 -- 172.21.15.82:0/1499650333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa53010e8f0 msgr2=0x7fa530110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:05.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.794+0000 7fa5359e6640 1 --2- 172.21.15.82:0/1499650333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa53010e8f0 0x7fa530110ce0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fa528009ca0 tx=0x7fa52802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.794+0000 7fa5359e6640 1 -- 172.21.15.82:0/1499650333 shutdown_connections 2024-04-21T08:58:05.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.794+0000 7fa5359e6640 1 --2- 172.21.15.82:0/1499650333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa53010e8f0 0x7fa530110ce0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.794+0000 7fa5359e6640 1 --2- 172.21.15.82:0/1499650333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa530101660 0x7fa53010e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.794+0000 7fa5359e6640 1 --2- 172.21.15.82:0/1499650333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa530100d40 0x7fa530101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.794+0000 7fa5359e6640 1 -- 172.21.15.82:0/1499650333 >> 172.21.15.82:0/1499650333 conn(0x7fa530078000 msgr2=0x7fa530078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:05.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.794+0000 7fa5359e6640 1 -- 172.21.15.82:0/1499650333 shutdown_connections 2024-04-21T08:58:05.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.795+0000 7fa5359e6640 1 -- 172.21.15.82:0/1499650333 wait complete. 2024-04-21T08:58:05.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.795+0000 7fa5359e6640 1 Processor -- start 2024-04-21T08:58:05.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.795+0000 7fa5359e6640 1 -- start start 2024-04-21T08:58:05.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.795+0000 7fa5359e6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530100d40 0x7fa5301a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:05.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.795+0000 7fa5359e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa530101660 0x7fa5301a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.795+0000 7fa5359e6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa53010e8f0 0x7fa53019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.795+0000 7fa5359e6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa530114390 con 0x7fa530100d40 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.795+0000 7fa5359e6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa530114210 con 0x7fa53010e8f0 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.795+0000 7fa5359e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa530114510 con 0x7fa530101660 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa52effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530100d40 0x7fa5301a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa52effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530100d40 0x7fa5301a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60212/0 (socket says 172.21.15.82:60212) 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa52effd640 1 -- 172.21.15.82:0/1101061981 learned_addr learned my addr 172.21.15.82:0/1101061981 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa52e7fc640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa530101660 0x7fa5301a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa52f7fe640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa53010e8f0 0x7fa53019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa52effd640 1 -- 172.21.15.82:0/1101061981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa530101660 msgr2=0x7fa5301a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa52effd640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa530101660 0x7fa5301a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa52effd640 1 -- 172.21.15.82:0/1101061981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa53010e8f0 msgr2=0x7fa53019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa52effd640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa53010e8f0 0x7fa53019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:05.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa52effd640 1 -- 172.21.15.82:0/1101061981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53019aff0 con 0x7fa530100d40 2024-04-21T08:58:05.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa52effd640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530100d40 0x7fa5301a0620 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fa52400d950 tx=0x7fa52400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:05.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa5349e4640 1 -- 172.21.15.82:0/1101061981 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524017070 con 0x7fa530100d40 2024-04-21T08:58:05.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa5359e6640 1 -- 172.21.15.82:0/1101061981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53019b2e0 con 0x7fa530100d40 2024-04-21T08:58:05.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa5349e4640 1 -- 172.21.15.82:0/1101061981 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa524009c30 con 0x7fa530100d40 2024-04-21T08:58:05.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa5349e4640 1 -- 172.21.15.82:0/1101061981 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52400f620 con 0x7fa530100d40 2024-04-21T08:58:05.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.796+0000 7fa5359e6640 1 -- 172.21.15.82:0/1101061981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5301a75a0 con 0x7fa530100d40 2024-04-21T08:58:05.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.797+0000 7fa5359e6640 1 -- 172.21.15.82:0/1101061981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5300698e0 con 0x7fa530100d40 2024-04-21T08:58:05.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.798+0000 7fa5349e4640 1 -- 172.21.15.82:0/1101061981 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa524010460 con 0x7fa530100d40 2024-04-21T08:58:05.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.799+0000 7fa5349e4640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5180a3d20 0x7fa5180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:58:05.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.799+0000 7fa52e7fc640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5180a3d20 0x7fa5180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:58:05.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.799+0000 7fa5349e4640 1 -- 172.21.15.82:0/1101061981 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8754+0+0 (secure 0 0 0) 0x7fa5240d0550 con 0x7fa530100d40 2024-04-21T08:58:05.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.799+0000 7fa52e7fc640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5180a3d20 0x7fa5180a61e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa53019bf30 tx=0x7fa52000b390 comp rx=0 tx=0).ready entity=mgr.15546 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:58:05.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.803+0000 7fa5349e4640 1 -- 172.21.15.82:0/1101061981 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa52409cb00 con 0x7fa530100d40 2024-04-21T08:58:05.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:05 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/3230495217' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:58:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:05 smithi134 ceph-mon[31772]: osdmap e105: 12 total, 12 up, 12 in 2024-04-21T08:58:05.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:05 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2640174223' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:58:05.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:05.983+0000 7fa5359e6640 1 -- 172.21.15.82:0/1101061981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa53019bc40 con 0x7fa530100d40 2024-04-21T08:58:06.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:05 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/3230495217' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:58:06.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:05 smithi082 ceph-mon[27967]: osdmap e105: 12 total, 12 up, 12 in 2024-04-21T08:58:06.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:05 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2640174223' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:58:06.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:05 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/3230495217' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:58:06.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:05 smithi106 ceph-mon[31072]: osdmap e105: 12 total, 12 up, 12 in 2024-04-21T08:58:06.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:05 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2640174223' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:58:06.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.663+0000 7fa5349e4640 1 -- 172.21.15.82:0/1101061981 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v107) ==== 178+0+0 (secure 0 0 0) 0x7fa53019bc40 con 0x7fa530100d40 2024-04-21T08:58:06.663 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data_ec' removed 2024-04-21T08:58:06.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.665+0000 7fa5359e6640 1 -- 172.21.15.82:0/1101061981 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5180a3d20 msgr2=0x7fa5180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:06.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.665+0000 7fa5359e6640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5180a3d20 0x7fa5180a61e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa53019bf30 tx=0x7fa52000b390 comp rx=0 tx=0).stop 2024-04-21T08:58:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.665+0000 7fa5359e6640 1 -- 172.21.15.82:0/1101061981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530100d40 msgr2=0x7fa5301a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:58:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.665+0000 7fa5359e6640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530100d40 0x7fa5301a0620 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fa52400d950 tx=0x7fa52400de10 comp rx=0 tx=0).stop 2024-04-21T08:58:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.665+0000 7fa5359e6640 1 -- 172.21.15.82:0/1101061981 shutdown_connections 2024-04-21T08:58:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.665+0000 7fa5359e6640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa53010e8f0 0x7fa53019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.665+0000 7fa5359e6640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa530101660 0x7fa5301a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.665+0000 7fa5359e6640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.82:6800/3178848282,v1:172.21.15.82:6801/3178848282] conn(0x7fa5180a3d20 0x7fa5180a61e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.665+0000 7fa5359e6640 1 --2- 172.21.15.82:0/1101061981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa530100d40 0x7fa5301a0620 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:58:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.665+0000 7fa5359e6640 1 -- 172.21.15.82:0/1101061981 >> 172.21.15.82:0/1101061981 conn(0x7fa530078000 msgr2=0x7fa53010ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:58:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.666+0000 7fa5359e6640 1 -- 172.21.15.82:0/1101061981 shutdown_connections 2024-04-21T08:58:06.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-21T08:58:06.666+0000 7fa5359e6640 1 -- 172.21.15.82:0/1101061981 wait complete. 2024-04-21T08:58:06.682 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:58:06.709 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:58:06.738 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:58:06.765 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-21T08:58:06.765 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:58:06.780 DEBUG:teuthology.orchestra.run.smithi106:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:58:06.795 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:58:06.822 INFO:tasks.cephadm:Stopping all daemons... 2024-04-21T08:58:06.822 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-21T08:58:06.822 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.a 2024-04-21T08:58:06.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:06 smithi134 ceph-mon[31772]: pgmap v346: 97 pgs: 97 active+clean; 457 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:58:06.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:06 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/2640174223' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:58:06.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:06 smithi134 ceph-mon[31772]: osdmap e106: 12 total, 12 up, 12 in 2024-04-21T08:58:06.977 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:06 smithi134 ceph-mon[31772]: from='client.? 172.21.15.82:0/1101061981' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:58:07.108 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:06 smithi082 ceph-mon[27967]: pgmap v346: 97 pgs: 97 active+clean; 457 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:58:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:06 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/2640174223' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:58:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:06 smithi082 ceph-mon[27967]: osdmap e106: 12 total, 12 up, 12 in 2024-04-21T08:58:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:06 smithi082 ceph-mon[27967]: from='client.? 172.21.15.82:0/1101061981' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:58:07.109 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:06 smithi082 systemd[1]: Stopping Ceph mon.a for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:58:07.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:06 smithi106 ceph-mon[31072]: pgmap v346: 97 pgs: 97 active+clean; 457 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:58:07.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:06 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/2640174223' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:58:07.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:06 smithi106 ceph-mon[31072]: osdmap e106: 12 total, 12 up, 12 in 2024-04-21T08:58:07.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:06 smithi106 ceph-mon[31072]: from='client.? 172.21.15.82:0/1101061981' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:58:07.580 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:07 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a[27963]: 2024-04-21T08:58:07.129+0000 7fe7a7dd3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:58:07.581 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:07 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a[27963]: 2024-04-21T08:58:07.129+0000 7fe7a7dd3700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-21T08:58:07.581 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:07 smithi082 podman[103332]: 2024-04-21 08:58:07.262908024 +0000 UTC m=+0.246823004 container died 1d8a7d05453aed00536ee3ba7a01cd7d65328a94ed433c52b1590f938055607b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-04-21T08:58:07.858 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:07 smithi082 podman[103332]: 2024-04-21 08:58:07.688807933 +0000 UTC m=+0.672722911 container cleanup 1d8a7d05453aed00536ee3ba7a01cd7d65328a94ed433c52b1590f938055607b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-21T08:58:07.859 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:07 smithi082 bash[103332]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a 2024-04-21T08:58:08.159 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.a.service' 2024-04-21T08:58:08.183 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:07 smithi082 podman[103412]: 2024-04-21 08:58:07.923853494 +0000 UTC m=+0.660872721 container remove 1d8a7d05453aed00536ee3ba7a01cd7d65328a94ed433c52b1590f938055607b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:58:08.183 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:08 smithi082 systemd[1]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-21T08:58:08.183 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:08 smithi082 systemd[1]: Stopped Ceph mon.a for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:58:08.183 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 21 08:58:08 smithi082 systemd[1]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.a.service: Consumed 20.556s CPU time. 2024-04-21T08:58:08.207 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:58:08.208 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-21T08:58:08.208 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-21T08:58:08.208 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.b 2024-04-21T08:58:08.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:08 smithi106 systemd[1]: Stopping Ceph mon.b for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:58:09.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:08 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-b[31068]: 2024-04-21T08:58:08.769+0000 7f474f782700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:58:09.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:08 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-b[31068]: 2024-04-21T08:58:08.769+0000 7f474f782700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-21T08:58:09.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:08 smithi106 podman[66558]: 2024-04-21 08:58:08.994512824 +0000 UTC m=+0.555519752 container died 1eaa36030e11e0b04af2cdd6cf84cdb282cbe2ff9adc09c02cf8284d4c0ac196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-21T08:58:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:09 smithi106 podman[66558]: 2024-04-21 08:58:09.375970476 +0000 UTC m=+0.936977406 container cleanup 1eaa36030e11e0b04af2cdd6cf84cdb282cbe2ff9adc09c02cf8284d4c0ac196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-21T08:58:09.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 21 08:58:09 smithi106 bash[66558]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-b 2024-04-21T08:58:10.047 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.b.service' 2024-04-21T08:58:10.096 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:58:10.097 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-21T08:58:10.097 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-21T08:58:10.097 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.c 2024-04-21T08:58:10.476 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:10 smithi134 systemd[1]: Stopping Ceph mon.c for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:58:10.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:10 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-c[31760]: 2024-04-21T08:58:10.680+0000 7efd75dee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:58:10.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:10 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-c[31760]: 2024-04-21T08:58:10.680+0000 7efd75dee700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-21T08:58:10.976 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:10 smithi134 podman[68576]: 2024-04-21 08:58:10.732660626 +0000 UTC m=+0.436445871 container died ae5eb50a307ae116e33b3bc779b985a087ec49a6b5e7a99633cd47afae086a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:58:11.407 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:11 smithi134 podman[68576]: 2024-04-21 08:58:11.17307555 +0000 UTC m=+0.876860779 container cleanup ae5eb50a307ae116e33b3bc779b985a087ec49a6b5e7a99633cd47afae086a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-c, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-21T08:58:11.407 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:11 smithi134 bash[68576]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-c 2024-04-21T08:58:11.654 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.c.service' 2024-04-21T08:58:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:11 smithi134 podman[68590]: 2024-04-21 08:58:11.408157379 +0000 UTC m=+0.665713459 container remove ae5eb50a307ae116e33b3bc779b985a087ec49a6b5e7a99633cd47afae086a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:58:11.674 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:11 smithi134 systemd[1]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-21T08:58:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:11 smithi134 systemd[1]: Stopped Ceph mon.c for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T08:58:11.675 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 21 08:58:11 smithi134 systemd[1]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mon.c.service: Consumed 12.857s CPU time. 2024-04-21T08:58:11.700 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:58:11.700 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-21T08:58:11.701 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-21T08:58:11.701 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.x 2024-04-21T08:58:12.109 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:58:11 smithi082 systemd[1]: Stopping Ceph mgr.x for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:58:12.560 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:58:12 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x[28197]: 2024-04-21T08:58:12.217+0000 7fe1703aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:58:12.561 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:58:12 smithi082 podman[103543]: 2024-04-21 08:58:12.249091695 +0000 UTC m=+0.352301260 container died c83669ec547c1d9eaab03192df1b4c091339eda62ea7038e5a416c42f990323e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-21T08:58:12.859 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:58:12 smithi082 podman[103543]: 2024-04-21 08:58:12.660521981 +0000 UTC m=+0.763731555 container cleanup c83669ec547c1d9eaab03192df1b4c091339eda62ea7038e5a416c42f990323e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-21T08:58:12.859 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:58:12 smithi082 bash[103543]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x 2024-04-21T08:58:13.426 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 21 08:58:13 smithi082 podman[103557]: 2024-04-21 08:58:13.154393362 +0000 UTC m=+0.902760002 container remove c83669ec547c1d9eaab03192df1b4c091339eda62ea7038e5a416c42f990323e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux ) 2024-04-21T08:58:13.441 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.x.service' 2024-04-21T08:58:13.475 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:58:13.476 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-21T08:58:13.476 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-21T08:58:13.476 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.y 2024-04-21T08:58:13.859 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:58:13 smithi106 systemd[1]: Stopping Ceph mgr.y for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:58:14.128 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:58:13 smithi106 podman[66690]: 2024-04-21 08:58:13.859813804 +0000 UTC m=+0.194363931 container died 1bf6191a46f517e10c00aaa92e8cd1458410ff0200dd6fdc1b633d9b1ebf34d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416) 2024-04-21T08:58:14.516 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:58:14 smithi106 podman[66690]: 2024-04-21 08:58:14.264903561 +0000 UTC m=+0.599453696 container cleanup 1bf6191a46f517e10c00aaa92e8cd1458410ff0200dd6fdc1b633d9b1ebf34d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:58:14.516 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 21 08:58:14 smithi106 bash[66690]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-y 2024-04-21T08:58:14.681 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.y.service' 2024-04-21T08:58:14.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:58:14.720 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-21T08:58:14.720 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-21T08:58:14.720 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.z 2024-04-21T08:58:15.091 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:58:14 smithi134 systemd[1]: Stopping Ceph mgr.z for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:58:15.387 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:58:15 smithi134 podman[68709]: 2024-04-21 08:58:15.091529457 +0000 UTC m=+0.155283942 container died c1ec677de54abb1687ebdf47fef959630771a2827ab7f72aa38845b5be97fb21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-21T08:58:15.726 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:58:15 smithi134 podman[68709]: 2024-04-21 08:58:15.462417642 +0000 UTC m=+0.526172123 container cleanup c1ec677de54abb1687ebdf47fef959630771a2827ab7f72aa38845b5be97fb21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:58:15.727 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 21 08:58:15 smithi134 bash[68709]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mgr-z 2024-04-21T08:58:15.940 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mgr.z.service' 2024-04-21T08:58:15.974 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:58:15.974 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-21T08:58:15.975 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-21T08:58:15.975 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.0 2024-04-21T08:58:16.284 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:16 smithi082 systemd[1]: Stopping Ceph osd.0 for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:58:16.608 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:16 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0[42765]: 2024-04-21T08:58:16.284+0000 7f7bf70b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:58:16.609 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:16 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0[42765]: 2024-04-21T08:58:16.284+0000 7f7bf70b7700 -1 osd.0 107 *** Got signal Terminated *** 2024-04-21T08:58:16.609 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:16 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0[42765]: 2024-04-21T08:58:16.284+0000 7f7bf70b7700 -1 osd.0 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:58:22.108 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:21 smithi082 podman[103664]: 2024-04-21 08:58:21.831121756 +0000 UTC m=+5.675276236 container died 3f35a9d8d56230ef22bb02a5be763f3bda1c3cc916ba201bb69cf89872e3f676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-21T08:58:22.497 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:22 smithi082 podman[103664]: 2024-04-21 08:58:22.25718585 +0000 UTC m=+6.101340332 container cleanup 3f35a9d8d56230ef22bb02a5be763f3bda1c3cc916ba201bb69cf89872e3f676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-21T08:58:22.497 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:22 smithi082 bash[103664]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0 2024-04-21T08:58:22.858 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:22 smithi082 podman[103681]: 2024-04-21 08:58:22.500698043 +0000 UTC m=+0.664919686 container remove 3f35a9d8d56230ef22bb02a5be763f3bda1c3cc916ba201bb69cf89872e3f676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-21T08:58:23.295 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:22 smithi082 podman[103780]: 2024-04-21 08:58:22.865730503 +0000 UTC m=+0.018513017 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:58:23.295 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:23 smithi082 podman[103780]: 2024-04-21 08:58:23.001828056 +0000 UTC m=+0.154610568 container create 5ad068f7f1a7edb8cc36f4ff782fe40a24fbaaf9fd1cb81b46b1c498b05cc291 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-21T08:58:23.599 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:23 smithi082 podman[103780]: 2024-04-21 08:58:23.29627021 +0000 UTC m=+0.449052737 container init 5ad068f7f1a7edb8cc36f4ff782fe40a24fbaaf9fd1cb81b46b1c498b05cc291 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:58:23.600 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:23 smithi082 podman[103780]: 2024-04-21 08:58:23.299526884 +0000 UTC m=+0.452309400 container start 5ad068f7f1a7edb8cc36f4ff782fe40a24fbaaf9fd1cb81b46b1c498b05cc291 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:58:23.600 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:23 smithi082 podman[103780]: 2024-04-21 08:58:23.39643437 +0000 UTC m=+0.549216884 container attach 5ad068f7f1a7edb8cc36f4ff782fe40a24fbaaf9fd1cb81b46b1c498b05cc291 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:58:23.858 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 21 08:58:23 smithi082 podman[103780]: 2024-04-21 08:58:23.600766614 +0000 UTC m=+0.753549140 container died 5ad068f7f1a7edb8cc36f4ff782fe40a24fbaaf9fd1cb81b46b1c498b05cc291 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=) 2024-04-21T08:58:24.307 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.0.service' 2024-04-21T08:58:24.344 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:58:24.345 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-21T08:58:24.345 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-21T08:58:24.345 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.1 2024-04-21T08:58:24.628 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:24 smithi106 systemd[1]: Stopping Ceph osd.1 for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:58:25.128 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:24 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1[35654]: 2024-04-21T08:58:24.652+0000 7ff8ea4d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:58:25.129 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:24 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1[35654]: 2024-04-21T08:58:24.652+0000 7ff8ea4d4700 -1 osd.1 107 *** Got signal Terminated *** 2024-04-21T08:58:25.129 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:24 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1[35654]: 2024-04-21T08:58:24.652+0000 7ff8ea4d4700 -1 osd.1 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:58:30.480 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:30 smithi106 podman[66812]: 2024-04-21 08:58:30.190269876 +0000 UTC m=+5.662548661 container died efa326034debb186498adc85b1684bec53e7063907d4fa6080e5d48d9078bcf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-21T08:58:31.120 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:30 smithi106 podman[66812]: 2024-04-21 08:58:30.841498419 +0000 UTC m=+6.313777201 container cleanup efa326034debb186498adc85b1684bec53e7063907d4fa6080e5d48d9078bcf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:58:31.121 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:30 smithi106 bash[66812]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1 2024-04-21T08:58:31.378 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:31 smithi106 podman[66829]: 2024-04-21 08:58:31.121641766 +0000 UTC m=+0.928784081 container remove efa326034debb186498adc85b1684bec53e7063907d4fa6080e5d48d9078bcf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-21T08:58:31.741 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:31 smithi106 podman[66919]: 2024-04-21 08:58:31.360660186 +0000 UTC m=+0.018489373 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:58:31.742 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:31 smithi106 podman[66919]: 2024-04-21 08:58:31.489517743 +0000 UTC m=+0.147346917 container create 05d24af153516691520867975efb9d344607539f3ce225a7e4255442cb6aa004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:58:31.742 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:31 smithi106 podman[66919]: 2024-04-21 08:58:31.742354386 +0000 UTC m=+0.400183560 container init 05d24af153516691520867975efb9d344607539f3ce225a7e4255442cb6aa004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-21T08:58:32.128 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:31 smithi106 podman[66919]: 2024-04-21 08:58:31.745558722 +0000 UTC m=+0.403387903 container start 05d24af153516691520867975efb9d344607539f3ce225a7e4255442cb6aa004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-21T08:58:32.129 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:31 smithi106 podman[66919]: 2024-04-21 08:58:31.867459046 +0000 UTC m=+0.525288221 container attach 05d24af153516691520867975efb9d344607539f3ce225a7e4255442cb6aa004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-21T08:58:32.792 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 21 08:58:32 smithi106 podman[66919]: 2024-04-21 08:58:32.514766569 +0000 UTC m=+1.172595744 container died 05d24af153516691520867975efb9d344607539f3ce225a7e4255442cb6aa004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:58:33.092 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.1.service' 2024-04-21T08:58:33.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:58:33.128 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-21T08:58:33.128 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-21T08:58:33.129 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.2 2024-04-21T08:58:33.417 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:33 smithi134 systemd[1]: Stopping Ceph osd.2 for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:58:33.726 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:33 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2[36939]: 2024-04-21T08:58:33.416+0000 7fbdbf90c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:58:33.726 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:33 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2[36939]: 2024-04-21T08:58:33.416+0000 7fbdbf90c700 -1 osd.2 107 *** Got signal Terminated *** 2024-04-21T08:58:33.727 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:33 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2[36939]: 2024-04-21T08:58:33.416+0000 7fbdbf90c700 -1 osd.2 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:58:39.260 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:39 smithi134 podman[68835]: 2024-04-21 08:58:39.009006244 +0000 UTC m=+5.687903427 container died 009f000ff1bfca0d74bc42710a04d042571c146edf9077579e9a47345c897e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-21T08:58:39.568 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:39 smithi134 podman[68835]: 2024-04-21 08:58:39.351446929 +0000 UTC m=+6.030344112 container cleanup 009f000ff1bfca0d74bc42710a04d042571c146edf9077579e9a47345c897e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-21T08:58:39.569 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:39 smithi134 bash[68835]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2 2024-04-21T08:58:39.976 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:39 smithi134 podman[68853]: 2024-04-21 08:58:39.569690331 +0000 UTC m=+0.556275383 container remove 009f000ff1bfca0d74bc42710a04d042571c146edf9077579e9a47345c897e45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-21T08:58:40.323 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:40 smithi134 podman[68948]: 2024-04-21 08:58:39.914397421 +0000 UTC m=+0.018272831 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:58:40.323 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:40 smithi134 podman[68948]: 2024-04-21 08:58:40.045955115 +0000 UTC m=+0.149830525 container create 05416fb10036732a5c8ef4a90e0d1a9fc3f557e1e7907fa40d6ce7f2a5eafaa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:58:40.323 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:40 smithi134 podman[68948]: 2024-04-21 08:58:40.324267771 +0000 UTC m=+0.428143181 container init 05416fb10036732a5c8ef4a90e0d1a9fc3f557e1e7907fa40d6ce7f2a5eafaa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True) 2024-04-21T08:58:40.620 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:40 smithi134 podman[68948]: 2024-04-21 08:58:40.327785825 +0000 UTC m=+0.431661238 container start 05416fb10036732a5c8ef4a90e0d1a9fc3f557e1e7907fa40d6ce7f2a5eafaa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-21T08:58:40.620 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:40 smithi134 podman[68948]: 2024-04-21 08:58:40.415929526 +0000 UTC m=+0.519804937 container attach 05416fb10036732a5c8ef4a90e0d1a9fc3f557e1e7907fa40d6ce7f2a5eafaa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-21T08:58:40.927 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 21 08:58:40 smithi134 podman[68948]: 2024-04-21 08:58:40.621029529 +0000 UTC m=+0.724904949 container died 05416fb10036732a5c8ef4a90e0d1a9fc3f557e1e7907fa40d6ce7f2a5eafaa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-21T08:58:41.276 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.2.service' 2024-04-21T08:58:41.316 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:58:41.316 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-21T08:58:41.316 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-21T08:58:41.316 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.3 2024-04-21T08:58:41.609 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:41 smithi082 systemd[1]: Stopping Ceph osd.3 for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:58:42.108 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:41 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3[49419]: 2024-04-21T08:58:41.700+0000 7f813f5f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:58:42.108 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:41 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3[49419]: 2024-04-21T08:58:41.700+0000 7f813f5f4700 -1 osd.3 107 *** Got signal Terminated *** 2024-04-21T08:58:42.109 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:41 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3[49419]: 2024-04-21T08:58:41.700+0000 7f813f5f4700 -1 osd.3 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:58:44.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:44 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:44.662+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:45.109 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:44 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3[49419]: 2024-04-21T08:58:44.649+0000 7f813bbe8700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.461382+0000 front 2024-04-21T08:58:19.461194+0000 (oldest deadline 2024-04-21T08:58:44.161330+0000) 2024-04-21T08:58:45.726 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:45 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:45.233+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:45.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:45 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:45.623+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:46.108 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:45 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3[49419]: 2024-04-21T08:58:45.626+0000 7f813bbe8700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.461382+0000 front 2024-04-21T08:58:19.461194+0000 (oldest deadline 2024-04-21T08:58:44.161330+0000) 2024-04-21T08:58:46.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:45 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:45.728+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:46.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:46 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:46.126+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:46.698 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:46 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:46.262+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:46.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:46 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:46.583+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:46.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:46 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:46.698+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:47.081 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:46 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3[49419]: 2024-04-21T08:58:46.668+0000 7f813bbe8700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.461382+0000 front 2024-04-21T08:58:19.461194+0000 (oldest deadline 2024-04-21T08:58:44.161330+0000) 2024-04-21T08:58:47.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:47 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:47.165+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:47.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:47 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:47.081+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:47.359 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:47 smithi082 podman[103887]: 2024-04-21 08:58:47.311301255 +0000 UTC m=+5.810898045 container died 35319ec4c4a125c98405ead880ac527c7cf64a1407dd3d1ec7e65886e8c00e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-21T08:58:47.657 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:47 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:47.241+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:47.858 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:47 smithi082 podman[103887]: 2024-04-21 08:58:47.659659954 +0000 UTC m=+6.159256748 container cleanup 35319ec4c4a125c98405ead880ac527c7cf64a1407dd3d1ec7e65886e8c00e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-21T08:58:47.859 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:47 smithi082 bash[103887]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3 2024-04-21T08:58:47.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:47 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:47.571+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:47.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:47 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:47.657+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:48.283 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:48 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:48.036+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:48.283 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:47 smithi082 podman[103906]: 2024-04-21 08:58:47.960979117 +0000 UTC m=+0.646949000 container remove 35319ec4c4a125c98405ead880ac527c7cf64a1407dd3d1ec7e65886e8c00e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-21T08:58:48.283 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:48 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:48.126+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:48.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:48 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:48.219+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:48.580 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:48 smithi082 podman[103997]: 2024-04-21 08:58:48.186450498 +0000 UTC m=+0.018000140 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:58:48.580 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:48 smithi082 podman[103997]: 2024-04-21 08:58:48.315760235 +0000 UTC m=+0.147309854 container create a4769083e3027915d37b813951317ae168c357b919a9c853fefc055ab8457400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:58:48.580 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:48 smithi082 podman[103997]: 2024-04-21 08:58:48.581197879 +0000 UTC m=+0.412747506 container init a4769083e3027915d37b813951317ae168c357b919a9c853fefc055ab8457400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-04-21T08:58:48.860 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:48 smithi082 podman[103997]: 2024-04-21 08:58:48.584487272 +0000 UTC m=+0.416036894 container start a4769083e3027915d37b813951317ae168c357b919a9c853fefc055ab8457400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:58:48.860 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:48 smithi082 podman[103997]: 2024-04-21 08:58:48.692207636 +0000 UTC m=+0.523757270 container attach a4769083e3027915d37b813951317ae168c357b919a9c853fefc055ab8457400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-04-21T08:58:48.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:48 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:48.604+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:48.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:48 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:48.699+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:49.157 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:49 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:49.076+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:49.157 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 21 08:58:48 smithi082 podman[103997]: 2024-04-21 08:58:48.883059102 +0000 UTC m=+0.714608726 container died a4769083e3027915d37b813951317ae168c357b919a9c853fefc055ab8457400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:58:49.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:49 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:49.190+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:49.509 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:49 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:49.157+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:49.527 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.3.service' 2024-04-21T08:58:49.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:58:49.565 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-21T08:58:49.565 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-21T08:58:49.565 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.4 2024-04-21T08:58:49.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:49 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:49.603+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:49.879 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:49 smithi106 systemd[1]: Stopping Ceph osd.4 for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:58:49.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:49 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:49.682+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:50.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:50 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:50.114+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:50.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:50 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:50.083+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:50.378 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:49 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4[42330]: 2024-04-21T08:58:49.891+0000 7f2bebc0a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:58:50.378 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:49 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4[42330]: 2024-04-21T08:58:49.891+0000 7f2bebc0a700 -1 osd.4 107 *** Got signal Terminated *** 2024-04-21T08:58:50.378 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:49 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4[42330]: 2024-04-21T08:58:49.891+0000 7f2bebc0a700 -1 osd.4 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:58:50.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:50 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:50.183+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:50.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:50 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:50.582+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:50.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:58:50 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:58:50.775+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:58:50.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:50 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:50.680+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:51.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:51 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:51.102+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:51.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:51 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:51.111+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:51.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:51 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:51.175+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:51.878 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:51 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4[42330]: 2024-04-21T08:58:51.726+0000 7f2be81fe700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.188064+0000 front 2024-04-21T08:58:26.188112+0000 (oldest deadline 2024-04-21T08:58:51.488328+0000) 2024-04-21T08:58:51.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:51 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:51.612+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:51.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:58:51 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:58:51.775+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:58:51.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:51 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:51.642+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:52.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:52 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:52.154+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:52.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:52 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:52.154+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:27.817843+0000 front 2024-04-21T08:58:27.817750+0000 (oldest deadline 2024-04-21T08:58:51.317800+0000) 2024-04-21T08:58:52.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:52 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:52.117+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:52.609 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:52 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:52.117+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:58:52.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:52 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:52.114+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:52.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:52 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:52.114+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:58:52.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:58:52 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:58:52.758+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:58:52.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:52 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:52.644+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:53.128 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:52 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4[42330]: 2024-04-21T08:58:52.682+0000 7f2be81fe700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.188064+0000 front 2024-04-21T08:58:26.188112+0000 (oldest deadline 2024-04-21T08:58:51.488328+0000) 2024-04-21T08:58:53.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:52 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:52.641+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:53.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:52 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:52.641+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:58:53.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:53.086+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:53.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:53.086+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:58:53.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:53.113+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:53.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:53.113+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:58:53.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:53 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:53.178+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:53.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:53 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:53.178+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:27.817843+0000 front 2024-04-21T08:58:27.817750+0000 (oldest deadline 2024-04-21T08:58:51.317800+0000) 2024-04-21T08:58:53.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:58:53 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:58:53.294+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:58:53.937 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:53 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4[42330]: 2024-04-21T08:58:53.645+0000 7f2be81fe700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.188064+0000 front 2024-04-21T08:58:26.188112+0000 (oldest deadline 2024-04-21T08:58:51.488328+0000) 2024-04-21T08:58:53.937 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:53 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:53.638+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:53.937 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:53 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:53.638+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:58:53.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:58:53 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:58:53.723+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:58:53.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:53 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:53.666+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:54.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:54 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:54.088+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:54.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:54 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:54.088+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:58:54.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:54 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:54.132+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:54.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:54 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:54.132+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:58:54.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:54 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:54.187+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:54.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:54 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:54.187+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:27.817843+0000 front 2024-04-21T08:58:27.817750+0000 (oldest deadline 2024-04-21T08:58:51.317800+0000) 2024-04-21T08:58:54.624 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:58:54 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:58:54.330+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:58:54.878 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:54 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4[42330]: 2024-04-21T08:58:54.658+0000 7f2be81fe700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.188064+0000 front 2024-04-21T08:58:26.188112+0000 (oldest deadline 2024-04-21T08:58:51.488328+0000) 2024-04-21T08:58:54.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:54 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:54.624+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:54.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:54 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:54.624+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:58:54.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:58:54 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:58:54.707+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:58:54.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:54 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:54.687+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:55.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:55.084+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:55.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:55.084+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:58:55.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:55.089+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:55.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:55.090+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:58:55.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:55 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:55.156+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:55.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:55 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:55.156+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:27.817843+0000 front 2024-04-21T08:58:27.817750+0000 (oldest deadline 2024-04-21T08:58:51.317800+0000) 2024-04-21T08:58:55.628 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:55 smithi106 podman[67042]: 2024-04-21 08:58:55.338184151 +0000 UTC m=+5.573517189 container died ef418a0520d20e4d7b62f6245fa87af297120a85e7c91cf2004d41403ca44123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-21T08:58:55.629 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:58:55 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:58:55.306+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:58:55.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:58:55 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:58:55.736+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:58:55.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:55 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:55.680+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:56.008 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:55 smithi106 podman[67042]: 2024-04-21 08:58:55.741567144 +0000 UTC m=+5.976900182 container cleanup ef418a0520d20e4d7b62f6245fa87af297120a85e7c91cf2004d41403ca44123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:58:56.009 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:55 smithi106 bash[67042]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4 2024-04-21T08:58:56.009 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:55 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:55.662+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:56.009 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:55 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:55.662+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:58:56.320 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:56 smithi106 podman[67060]: 2024-04-21 08:58:56.009701692 +0000 UTC m=+0.671568925 container remove ef418a0520d20e4d7b62f6245fa87af297120a85e7c91cf2004d41403ca44123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:58:56.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:56 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:56.057+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:56.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:56 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:56.057+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:58:56.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:56 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:56.097+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:56.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:56 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:56.097+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:58:56.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:56 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:56.126+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:56.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:56 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:56.126+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:27.817843+0000 front 2024-04-21T08:58:27.817750+0000 (oldest deadline 2024-04-21T08:58:51.317800+0000) 2024-04-21T08:58:56.613 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:58:56 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:58:56.320+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:58:56.613 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:56 smithi106 podman[67150]: 2024-04-21 08:58:56.344169583 +0000 UTC m=+0.114529924 container create 7ccfb119fb2f96b46966fd34a672c91ff5a6aef44e0fdb48ea8626aa0ae66b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-21T08:58:56.613 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:56 smithi106 podman[67150]: 2024-04-21 08:58:56.24721763 +0000 UTC m=+0.017577986 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:58:56.614 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:56 smithi106 podman[67150]: 2024-04-21 08:58:56.614502148 +0000 UTC m=+0.384862502 container init 7ccfb119fb2f96b46966fd34a672c91ff5a6aef44e0fdb48ea8626aa0ae66b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-21T08:58:56.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:56 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:56.681+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:56.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:56 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:56.681+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:58:56.879 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:56 smithi106 podman[67150]: 2024-04-21 08:58:56.617943808 +0000 UTC m=+0.388304150 container start 7ccfb119fb2f96b46966fd34a672c91ff5a6aef44e0fdb48ea8626aa0ae66b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True) 2024-04-21T08:58:56.879 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:56 smithi106 podman[67150]: 2024-04-21 08:58:56.680501248 +0000 UTC m=+0.450861610 container attach 7ccfb119fb2f96b46966fd34a672c91ff5a6aef44e0fdb48ea8626aa0ae66b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-21T08:58:56.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:58:56 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:58:56.704+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:58:56.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:56 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:56.674+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:57.199 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 21 08:58:56 smithi106 podman[67150]: 2024-04-21 08:58:56.914598332 +0000 UTC m=+0.684958675 container died 7ccfb119fb2f96b46966fd34a672c91ff5a6aef44e0fdb48ea8626aa0ae66b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-21T08:58:57.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:57 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:57.079+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:57.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:57 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:57.079+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:58:57.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:57 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:57.128+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:57.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:57 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:57.128+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:58:57.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:57 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:57.100+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:57.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:57 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:57.100+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:27.817843+0000 front 2024-04-21T08:58:27.817750+0000 (oldest deadline 2024-04-21T08:58:51.317800+0000) 2024-04-21T08:58:57.491 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:58:57 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:58:57.296+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:58:57.512 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.4.service' 2024-04-21T08:58:57.548 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:58:57.549 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-21T08:58:57.549 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-21T08:58:57.549 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.5 2024-04-21T08:58:57.847 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:58:57 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:58:57.735+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:58:57.848 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:57 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:57.720+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:57.848 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:57 smithi134 systemd[1]: Stopping Ceph osd.5 for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:58:57.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:57 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:57.714+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:57.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:57 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:57.714+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:58:58.226 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:57 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:57.847+0000 7fe70bd72700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:58:58.226 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:57 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:57.847+0000 7fe70bd72700 -1 osd.5 107 *** Got signal Terminated *** 2024-04-21T08:58:58.226 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:57 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:57.847+0000 7fe70bd72700 -1 osd.5 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:58:58.226 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:58 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:58.068+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:58.227 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:58 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:58.068+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:27.817843+0000 front 2024-04-21T08:58:27.817750+0000 (oldest deadline 2024-04-21T08:58:51.317800+0000) 2024-04-21T08:58:58.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:58.094+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:58.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:58.094+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:58:58.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:58.080+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:58.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:58.080+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:58:58.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:58:58 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:58:58.331+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:58:58.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:58:58 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:58:58.786+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:58:58.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:58 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:58.682+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:59.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:58 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:58.735+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:58:59.129 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:58 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:58.735+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:58:59.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:59 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:59.077+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:58:59.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:58:59 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:58:59.077+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:58:59.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:59 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:59.101+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:58:59.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:58:59 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:58:59.101+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:58:59.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:59 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:59.107+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:58:59.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:58:59 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:58:59.107+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:27.817843+0000 front 2024-04-21T08:58:27.817750+0000 (oldest deadline 2024-04-21T08:58:51.317800+0000) 2024-04-21T08:58:59.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:58:59 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:58:59.371+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:58:59.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:58:59 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:58:59.702+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:58:59.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:58:59 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:58:59.804+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:58:59.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:58:59 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:58:59.804+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:00.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:59 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:59.711+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:00.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:58:59 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:58:59.711+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:00.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:00.053+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:00.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:00.053+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:00.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:00.063+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:59:00.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:00.063+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:59:00.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:00 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:00.063+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:35.752848+0000 front 2024-04-21T08:58:35.752879+0000 (oldest deadline 2024-04-21T08:58:59.153523+0000) 2024-04-21T08:59:00.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:00 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:59:00.141+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:59:00.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:00 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:59:00.141+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:27.817843+0000 front 2024-04-21T08:58:27.817750+0000 (oldest deadline 2024-04-21T08:58:51.317800+0000) 2024-04-21T08:59:00.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:00 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:59:00.141+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:34.218966+0000 front 2024-04-21T08:58:34.218988+0000 (oldest deadline 2024-04-21T08:58:59.519035+0000) 2024-04-21T08:59:00.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:00 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:00.361+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:00.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:00 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:00.361+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:00.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:00 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:00.846+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:00.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:00 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:00.846+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:00.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:00 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:00.711+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:01.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:00 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:00.667+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:01.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:00 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:00.667+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:01.129 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:00 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:00.667+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:01.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:01 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:01.100+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:01.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:01 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:01.100+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:01.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:01 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:01.076+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:59:01.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:01 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:01.076+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:59:01.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:01 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:01.076+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:35.752848+0000 front 2024-04-21T08:58:35.752879+0000 (oldest deadline 2024-04-21T08:58:59.153523+0000) 2024-04-21T08:59:01.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:01 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:59:01.174+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:59:01.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:01 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:59:01.174+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:27.817843+0000 front 2024-04-21T08:58:27.817750+0000 (oldest deadline 2024-04-21T08:58:51.317800+0000) 2024-04-21T08:59:01.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:01 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:59:01.174+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:34.218966+0000 front 2024-04-21T08:58:34.218988+0000 (oldest deadline 2024-04-21T08:58:59.519035+0000) 2024-04-21T08:59:01.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:01 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:01.343+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:01.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:01 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:01.343+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:01.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:01 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:01.847+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:01.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:01 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:01.847+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:01.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:01 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:01.669+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:02.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:01 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:01.660+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:02.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:01 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:01.660+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:02.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:01 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:01.660+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:02.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:02.078+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:02.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:02.078+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:02.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:02.078+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:02.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:02.088+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:59:02.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:02.088+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:59:02.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:02 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:02.088+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:35.752848+0000 front 2024-04-21T08:58:35.752879+0000 (oldest deadline 2024-04-21T08:58:59.153523+0000) 2024-04-21T08:59:02.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:59:02.184+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.216507+0000 front 2024-04-21T08:58:20.216600+0000 (oldest deadline 2024-04-21T08:58:44.316664+0000) 2024-04-21T08:59:02.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:59:02.184+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:27.817843+0000 front 2024-04-21T08:58:27.817750+0000 (oldest deadline 2024-04-21T08:58:51.317800+0000) 2024-04-21T08:59:02.476 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5[43459]: 2024-04-21T08:59:02.184+0000 7fe70435e700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:34.218966+0000 front 2024-04-21T08:58:34.218988+0000 (oldest deadline 2024-04-21T08:58:59.519035+0000) 2024-04-21T08:59:02.642 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:02.382+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:02.642 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:02.382+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:02.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:02.802+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:02.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:02.802+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:02.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:02.653+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:02.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:02 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:02.653+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:03.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:02.642+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:03.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:02.642+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:03.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:02 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:02.642+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:03.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:03 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:03.065+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:03.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:03 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:03.065+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:03.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:03 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:03.065+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:03.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:03 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:03.118+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:59:03.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:03 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:03.118+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:59:03.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:03 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:03.118+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:35.752848+0000 front 2024-04-21T08:58:35.752879+0000 (oldest deadline 2024-04-21T08:58:59.153523+0000) 2024-04-21T08:59:03.491 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:03 smithi134 podman[69068]: 2024-04-21 08:59:03.233428807 +0000 UTC m=+5.493209628 container died 3fae4aa887062e3235704876e967eeaba0c0960dd9dac18f64d33b81b0fb8cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-21T08:59:03.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:03 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:03.348+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:03.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:03 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:03.348+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:03.850 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:03 smithi134 podman[69068]: 2024-04-21 08:59:03.590271978 +0000 UTC m=+5.850052809 container cleanup 3fae4aa887062e3235704876e967eeaba0c0960dd9dac18f64d33b81b0fb8cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-21T08:59:03.851 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:03 smithi134 bash[69068]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5 2024-04-21T08:59:03.851 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:03 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:03.669+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:03.851 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:03 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:03.669+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:04.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:03 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:03.690+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:04.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:03 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:03.690+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:04.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:03 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:03.690+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:04.226 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:03 smithi134 podman[69085]: 2024-04-21 08:59:03.899955306 +0000 UTC m=+0.662573523 container remove 3fae4aa887062e3235704876e967eeaba0c0960dd9dac18f64d33b81b0fb8cb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-21T08:59:04.226 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:03 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:03.851+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:04.227 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:03 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:03.851+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:04.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:04.098+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:59:04.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:04.098+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:59:04.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:04.098+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:35.752848+0000 front 2024-04-21T08:58:35.752879+0000 (oldest deadline 2024-04-21T08:58:59.153523+0000) 2024-04-21T08:59:04.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:04.103+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:04.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:04.103+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:04.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:04.103+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:04.578 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:04 smithi134 podman[69175]: 2024-04-21 08:59:04.1654589 +0000 UTC m=+0.018210167 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:59:04.578 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:04 smithi134 podman[69175]: 2024-04-21 08:59:04.309177438 +0000 UTC m=+0.161928704 container create fd6e66f8872d5546e7abd8217d8a742a671dac34b6f61de987480f72f9b9bab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-21T08:59:04.662 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:04 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:04.387+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:04.662 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:04 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:04.387+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:04.878 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:04 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:04.820+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:04.878 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:04 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:04.820+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:04.878 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:04 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:04.700+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:04.879 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:04 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:04.700+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:04.879 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:04 smithi134 podman[69175]: 2024-04-21 08:59:04.578763822 +0000 UTC m=+0.431515092 container init fd6e66f8872d5546e7abd8217d8a742a671dac34b6f61de987480f72f9b9bab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-21T08:59:04.879 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:04 smithi134 podman[69175]: 2024-04-21 08:59:04.58189965 +0000 UTC m=+0.434650918 container start fd6e66f8872d5546e7abd8217d8a742a671dac34b6f61de987480f72f9b9bab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:59:04.879 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:04 smithi134 podman[69175]: 2024-04-21 08:59:04.6872098 +0000 UTC m=+0.539961079 container attach fd6e66f8872d5546e7abd8217d8a742a671dac34b6f61de987480f72f9b9bab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-21T08:59:05.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:04 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:04.662+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:05.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:04 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:04.662+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:05.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:04 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:04.662+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:05.226 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 21 08:59:04 smithi134 podman[69175]: 2024-04-21 08:59:04.878834821 +0000 UTC m=+0.731586096 container died fd6e66f8872d5546e7abd8217d8a742a671dac34b6f61de987480f72f9b9bab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-21T08:59:05.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:05 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:05.091+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:59:05.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:05 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:05.091+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:59:05.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:05 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:05.091+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:35.752848+0000 front 2024-04-21T08:58:35.752879+0000 (oldest deadline 2024-04-21T08:58:59.153523+0000) 2024-04-21T08:59:05.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:05 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:05.109+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:05.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:05 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:05.109+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:05.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:05 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:05.109+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:05.535 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.5.service' 2024-04-21T08:59:05.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:59:05.574 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-21T08:59:05.574 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-21T08:59:05.574 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.6 2024-04-21T08:59:05.837 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:05 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:05.710+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:05.837 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:05 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:05.710+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:05.858 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:05 smithi082 systemd[1]: Stopping Ceph osd.6 for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:59:05.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:05 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:05.427+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:05.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:05 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:05.427+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:05.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:05 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:05.644+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:05.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:05 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:05.644+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:05.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:05 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:05.644+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:06.226 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:05 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:05.836+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:06.226 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:05 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:05.836+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:06.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:06 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:06.155+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:06.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:06 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:06.155+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:06.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:06 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:06.155+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:06.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:05 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:05.910+0000 7f6b354e2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:59:06.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:05 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:05.910+0000 7f6b354e2700 -1 osd.6 107 *** Got signal Terminated *** 2024-04-21T08:59:06.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:05 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:05.910+0000 7f6b354e2700 -1 osd.6 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:59:06.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:06 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:06.077+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:59:06.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:06 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:06.077+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:59:06.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:06 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:06.077+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:35.752848+0000 front 2024-04-21T08:58:35.752879+0000 (oldest deadline 2024-04-21T08:58:59.153523+0000) 2024-04-21T08:59:06.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:06 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:06.460+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:06.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:06 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:06.460+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:06.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:06 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:06.604+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:06.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:06 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:06.604+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:06.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:06 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:06.604+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:06.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:06 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:06.865+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:06.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:06 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:06.865+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:06.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:06 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:06.675+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:06.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:06 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:06.675+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:07.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:07 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:07.034+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:59:07.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:07 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:07.034+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:59:07.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:07 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:07.034+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:35.752848+0000 front 2024-04-21T08:58:35.752879+0000 (oldest deadline 2024-04-21T08:58:59.153523+0000) 2024-04-21T08:59:07.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:07 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:07.151+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:07.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:07 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:07.151+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:07.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:07 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:07.151+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:07.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:07 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:07.468+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:07.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:07 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:07.468+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:07.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:07 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:07.597+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:07.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:07 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:07.597+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:07.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:07 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:07.597+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:07.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:07 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:07.835+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:07.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:07 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:07.835+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:07.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:07 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:07.661+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:07.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:07 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:07.661+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:08.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:08 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:08.131+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:08.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:08 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:08.131+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:08.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:08 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:08.131+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:08.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:08 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:08.058+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:59:08.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:08 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:08.058+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:59:08.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:08 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:08.058+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:35.752848+0000 front 2024-04-21T08:58:35.752879+0000 (oldest deadline 2024-04-21T08:58:59.153523+0000) 2024-04-21T08:59:08.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:08 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:08.474+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:08.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:08 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:08.474+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:08.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:08 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:08.569+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:08.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:08 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:08.569+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:08.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:08 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:08.569+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:08.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:08 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:08.875+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:08.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:08 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:08.875+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:08.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:08 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:08.681+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:08.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:08 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:08.681+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:08.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:08 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:08.681+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:09.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:09 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:09.029+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:59:09.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:09 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:09.029+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:59:09.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:09 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:09.029+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:35.752848+0000 front 2024-04-21T08:58:35.752879+0000 (oldest deadline 2024-04-21T08:58:59.153523+0000) 2024-04-21T08:59:09.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:09 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:09.162+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:09.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:09 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:09.162+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:09.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:09 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:09.162+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:09.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:09 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:09.162+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:09.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:09 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:09.462+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:09.879 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:09 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:09.462+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:09.879 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:09 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:09.462+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.852071+0000 front 2024-04-21T08:58:45.852055+0000 (oldest deadline 2024-04-21T08:59:09.351894+0000) 2024-04-21T08:59:09.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:09 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:09.581+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:09.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:09 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:09.581+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:09.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:09 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:09.581+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:09.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:09 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:09.832+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:09.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:09 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:09.832+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:09.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:09 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:09.642+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:09.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:09 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:09.642+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:09.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:09 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:09.642+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:10.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:10 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:10.053+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.450027+0000 front 2024-04-21T08:58:19.450162+0000 (oldest deadline 2024-04-21T08:58:45.350250+0000) 2024-04-21T08:59:10.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:10 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:10.053+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:25.351042+0000 front 2024-04-21T08:58:25.350969+0000 (oldest deadline 2024-04-21T08:58:51.250920+0000) 2024-04-21T08:59:10.359 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:10 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6[56379]: 2024-04-21T08:59:10.053+0000 7f6b31ad6700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:35.752848+0000 front 2024-04-21T08:58:35.752879+0000 (oldest deadline 2024-04-21T08:58:59.153523+0000) 2024-04-21T08:59:10.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:10 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:10.171+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:10.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:10 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:10.171+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:10.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:10 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:10.171+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:10.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:10 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:10.171+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:10.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:10 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:10.434+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:10.879 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:10 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:10.434+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:10.879 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:10 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:10.434+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.852071+0000 front 2024-04-21T08:58:45.852055+0000 (oldest deadline 2024-04-21T08:59:09.351894+0000) 2024-04-21T08:59:10.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:10 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:10.534+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:10.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:10 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:10.534+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:10.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:10 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:10.534+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:10.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:10 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:10.534+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:10.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:10 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:10.799+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:10.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:10 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:10.799+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:10.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:10 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:10.799+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:10.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:10 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:10.690+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:10.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:10 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:10.690+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:10.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:10 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:10.690+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:11.608 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:11 smithi082 podman[104118]: 2024-04-21 08:59:11.393992479 +0000 UTC m=+5.625479607 container died 46384a50a837edb5421f34bd7eb85e0d2e8e8659777e7f22a7eef7486817b5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-21T08:59:11.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:11 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:11.149+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:11.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:11 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:11.149+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:11.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:11 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:11.149+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:11.609 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:11 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:11.149+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:11.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:11 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:11.467+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:11.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:11 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:11.467+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:11.879 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:11 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:11.467+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.852071+0000 front 2024-04-21T08:58:45.852055+0000 (oldest deadline 2024-04-21T08:59:09.351894+0000) 2024-04-21T08:59:11.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:11 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:11.575+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:11.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:11 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:11.575+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:11.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:11 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:11.575+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:11.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:11 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:11.575+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:11.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:11 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:11.830+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:11.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:11 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:11.830+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:11.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:11 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:11.830+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:11.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:11 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:11.718+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:11.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:11 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:11.718+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:11.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:11 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:11.718+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:12.069 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:11 smithi082 podman[104118]: 2024-04-21 08:59:11.785796899 +0000 UTC m=+6.017284056 container cleanup 46384a50a837edb5421f34bd7eb85e0d2e8e8659777e7f22a7eef7486817b5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.vendor=CentOS) 2024-04-21T08:59:12.069 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:11 smithi082 bash[104118]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6 2024-04-21T08:59:12.358 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:12 smithi082 podman[104135]: 2024-04-21 08:59:12.070326502 +0000 UTC m=+0.675541909 container remove 46384a50a837edb5421f34bd7eb85e0d2e8e8659777e7f22a7eef7486817b5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:59:12.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:12 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:12.103+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:12.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:12 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:12.103+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:12.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:12 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:12.103+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:12.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:12 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:12.103+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:12.655 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:12 smithi082 podman[104225]: 2024-04-21 08:59:12.299970318 +0000 UTC m=+0.018050630 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:59:12.656 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:12 smithi082 podman[104225]: 2024-04-21 08:59:12.404679652 +0000 UTC m=+0.122759990 container create 461b592218515dc6495ef1126ee703cd25b0594f07afe9e1ed25675c8ccf54c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-21T08:59:12.656 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:12 smithi082 podman[104225]: 2024-04-21 08:59:12.656761364 +0000 UTC m=+0.374841660 container init 461b592218515dc6495ef1126ee703cd25b0594f07afe9e1ed25675c8ccf54c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-21T08:59:12.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:12 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:12.418+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:12.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:12 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:12.418+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:12.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:12 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:12.418+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.852071+0000 front 2024-04-21T08:58:45.852055+0000 (oldest deadline 2024-04-21T08:59:09.351894+0000) 2024-04-21T08:59:12.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:12 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:12.608+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:12.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:12 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:12.608+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:12.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:12 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:12.608+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:12.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:12 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:12.608+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:12.956 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:12 smithi082 podman[104225]: 2024-04-21 08:59:12.659911086 +0000 UTC m=+0.377991382 container start 461b592218515dc6495ef1126ee703cd25b0594f07afe9e1ed25675c8ccf54c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:59:12.956 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:12 smithi082 podman[104225]: 2024-04-21 08:59:12.732012637 +0000 UTC m=+0.450092939 container attach 461b592218515dc6495ef1126ee703cd25b0594f07afe9e1ed25675c8ccf54c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6-deactivate, ceph=True, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-21T08:59:12.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:12 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:12.836+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:12.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:12 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:12.836+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:12.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:12 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:12.836+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:12.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:12 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:12.692+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:12.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:12 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:12.692+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:12.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:12 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:12.692+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:13.283 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 21 08:59:12 smithi082 podman[104225]: 2024-04-21 08:59:12.957547088 +0000 UTC m=+0.675627387 container died 461b592218515dc6495ef1126ee703cd25b0594f07afe9e1ed25675c8ccf54c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:59:13.284 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:13 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:13.094+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:13.284 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:13 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:13.094+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:13.284 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:13 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:13.094+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:13.284 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:13 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:13.094+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:13.627 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.6.service' 2024-04-21T08:59:13.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:59:13.664 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-21T08:59:13.664 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-21T08:59:13.664 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.7 2024-04-21T08:59:13.690 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:13 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:13.599+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:13.690 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:13 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:13.599+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:13.690 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:13 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:13.599+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:13.690 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:13 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:13.599+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:13.690 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:13 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:13.398+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:13.690 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:13 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:13.398+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:13.690 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:13 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:13.398+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.852071+0000 front 2024-04-21T08:58:45.852055+0000 (oldest deadline 2024-04-21T08:59:09.351894+0000) 2024-04-21T08:59:13.986 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:13 smithi106 systemd[1]: Stopping Ceph osd.7 for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:59:14.080 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:13 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:13.856+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:14.080 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:13 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:13.856+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:14.080 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:13 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:13.856+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:14.080 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:13 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:13.739+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:14.080 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:13 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:13.739+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:14.080 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:13 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:13.739+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:14.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:14 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:14.049+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:14.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:14 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:14.049+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:14.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:14 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:14.049+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:14.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:14 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:14.049+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:14.378 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:13 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:13.986+0000 7f8935350700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:59:14.378 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:13 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:13.986+0000 7f8935350700 -1 osd.7 107 *** Got signal Terminated *** 2024-04-21T08:59:14.378 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:13 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:13.986+0000 7f8935350700 -1 osd.7 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:59:14.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:14 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:14.404+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:14.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:14 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:14.404+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:14.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:14 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:14.404+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.852071+0000 front 2024-04-21T08:58:45.852055+0000 (oldest deadline 2024-04-21T08:59:09.351894+0000) 2024-04-21T08:59:14.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:14 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:14.601+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:14.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:14 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:14.601+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:14.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:14 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:14.601+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:14.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:14 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:14.601+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:14.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:14 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:14.709+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:14.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:14 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:14.709+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:14.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:14 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:14.709+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:14.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:14 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:14.850+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:14.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:14 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:14.850+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:14.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:14 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:14.850+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:15.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:15 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:15.039+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:15.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:15 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:15.039+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:15.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:15 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:15.039+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:15.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:15 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:15.039+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:15.879 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:15 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:15.416+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:15.879 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:15 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:15.416+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:15.879 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:15 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:15.416+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.852071+0000 front 2024-04-21T08:58:45.852055+0000 (oldest deadline 2024-04-21T08:59:09.351894+0000) 2024-04-21T08:59:15.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:15 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:15.604+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:15.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:15 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:15.604+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:15.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:15 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:15.604+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:15.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:15 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:15.604+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:15.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:15 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:15.663+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:15.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:15 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:15.663+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:15.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:15 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:15.663+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:15.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:15 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:15.833+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:15.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:15 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:15.833+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:15.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:15 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:15.833+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:16.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:16 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:16.005+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:16.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:16 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:16.005+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:16.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:16 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:16.005+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:16.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:16 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:16.005+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:16.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:16 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:16.466+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:16.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:16 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:16.466+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:16.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:16 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:16.466+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.852071+0000 front 2024-04-21T08:58:45.852055+0000 (oldest deadline 2024-04-21T08:59:09.351894+0000) 2024-04-21T08:59:16.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:16 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:16.611+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:16.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:16 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:16.611+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:16.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:16 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:16.611+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:16.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:16 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:16.611+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:16.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:16 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:16.691+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:16.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:16 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:16.691+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:16.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:16 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:16.691+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:16.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:16 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:16.788+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:16.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:16 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:16.788+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:16.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:16 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:16.788+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:17.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:17 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:17.006+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:17.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:17 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:17.006+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:17.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:17 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:17.006+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:17.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:17 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:17.006+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:17.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:17 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:17.006+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:17.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:17 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:17.488+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:17.879 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:17 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:17.488+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:17.879 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:17 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:17.488+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.852071+0000 front 2024-04-21T08:58:45.852055+0000 (oldest deadline 2024-04-21T08:59:09.351894+0000) 2024-04-21T08:59:17.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:17 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:17.651+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:17.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:17 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:17.651+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:17.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:17 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:17.651+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:17.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:17 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:17.651+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:17.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:17 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:17.695+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:17.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:17 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:17.695+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:17.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:17 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:17.695+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:17.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:17 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:17.765+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:17.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:17 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:17.765+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:17.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:17 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:17.765+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:17.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:17 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:17.765+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.753566+0000 front 2024-04-21T08:58:54.753555+0000 (oldest deadline 2024-04-21T08:59:17.553922+0000) 2024-04-21T08:59:18.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:18 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:18.052+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:18.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:18 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:18.052+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:18.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:18 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:18.052+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:18.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:18 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:18.052+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:18.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:18 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:18.052+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:18.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:18 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:18.448+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.448318+0000 front 2024-04-21T08:58:28.448350+0000 (oldest deadline 2024-04-21T08:58:53.148592+0000) 2024-04-21T08:59:18.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:18 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:18.448+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.649625+0000 front 2024-04-21T08:58:33.649598+0000 (oldest deadline 2024-04-21T08:59:00.050259+0000) 2024-04-21T08:59:18.878 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:18 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7[48880]: 2024-04-21T08:59:18.448+0000 7f8931944700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.852071+0000 front 2024-04-21T08:58:45.852055+0000 (oldest deadline 2024-04-21T08:59:09.351894+0000) 2024-04-21T08:59:18.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:18 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:18.687+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:18.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:18 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:18.687+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:18.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:18 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:18.687+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:18.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:18 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:18.687+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:18.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:18 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:18.687+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:18.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:18 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:18.685+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:18.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:18 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:18.685+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:18.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:18 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:18.685+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:18.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:18 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:18.685+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:18.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:18 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:18.779+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:18.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:18 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:18.779+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:18.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:18 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:18.779+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:18.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:18 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:18.779+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.753566+0000 front 2024-04-21T08:58:54.753555+0000 (oldest deadline 2024-04-21T08:59:17.553922+0000) 2024-04-21T08:59:19.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:19 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:19.071+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:19.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:19 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:19.071+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:19.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:19 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:19.071+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:19.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:19 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:19.071+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:19.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:19 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:19.071+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:19.650 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:19 smithi106 podman[67254]: 2024-04-21 08:59:19.396455244 +0000 UTC m=+5.513798162 container died 607674195e2803057ccc4c998f86faa4646c2be508aaab5c15391d0e04a91263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-21T08:59:19.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:19 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:19.668+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:19.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:19 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:19.668+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:19.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:19 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:19.668+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:19.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:19 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:19.668+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:19.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:19 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:19.785+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:19.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:19 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:19.785+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:19.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:19 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:19.785+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:19.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:19 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:19.785+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.753566+0000 front 2024-04-21T08:58:54.753555+0000 (oldest deadline 2024-04-21T08:59:17.553922+0000) 2024-04-21T08:59:20.028 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:19 smithi106 podman[67254]: 2024-04-21 08:59:19.778291716 +0000 UTC m=+5.895634620 container cleanup 607674195e2803057ccc4c998f86faa4646c2be508aaab5c15391d0e04a91263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2) 2024-04-21T08:59:20.028 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:19 smithi106 bash[67254]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7 2024-04-21T08:59:20.029 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:19 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:19.650+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:20.029 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:19 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:19.650+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:20.029 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:19 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:19.650+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:20.029 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:19 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:19.650+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:20.029 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:19 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:19.650+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:20.340 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:20 smithi106 podman[67280]: 2024-04-21 08:59:20.029535256 +0000 UTC m=+0.627833164 container remove 607674195e2803057ccc4c998f86faa4646c2be508aaab5c15391d0e04a91263 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-21T08:59:20.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:20 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:20.038+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:20.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:20 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:20.038+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:20.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:20 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:20.038+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:20.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:20 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:20.038+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:20.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:20 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:20.038+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:20.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:20 smithi106 podman[67374]: 2024-04-21 08:59:20.244804592 +0000 UTC m=+0.018013649 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:59:20.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:20 smithi106 podman[67374]: 2024-04-21 08:59:20.355287661 +0000 UTC m=+0.128496715 container create cfb045cfc063284d12beaf771d728eeb7031e9dad79166533c064024926cfdee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:59:20.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:20 smithi106 podman[67374]: 2024-04-21 08:59:20.574437204 +0000 UTC m=+0.347646245 container init cfb045cfc063284d12beaf771d728eeb7031e9dad79166533c064024926cfdee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:59:20.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:20 smithi106 podman[67374]: 2024-04-21 08:59:20.577401106 +0000 UTC m=+0.350610148 container start cfb045cfc063284d12beaf771d728eeb7031e9dad79166533c064024926cfdee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-21T08:59:20.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:20 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:20.697+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:20.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:20 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:20.697+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:20.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:20 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:20.697+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:20.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:20 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:20.697+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:20.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:20 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:20.798+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:20.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:20 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:20.798+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:20.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:20 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:20.798+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:20.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:20 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:20.798+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.753566+0000 front 2024-04-21T08:58:54.753555+0000 (oldest deadline 2024-04-21T08:59:17.553922+0000) 2024-04-21T08:59:21.129 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:20 smithi106 podman[67374]: 2024-04-21 08:59:20.692002402 +0000 UTC m=+0.465211458 container attach cfb045cfc063284d12beaf771d728eeb7031e9dad79166533c064024926cfdee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7-deactivate, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-21T08:59:21.129 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 21 08:59:20 smithi106 podman[67374]: 2024-04-21 08:59:20.87756169 +0000 UTC m=+0.650770723 container died cfb045cfc063284d12beaf771d728eeb7031e9dad79166533c064024926cfdee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-21T08:59:21.129 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:20 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:20.667+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:21.129 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:20 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:20.667+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:21.129 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:20 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:20.667+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:21.129 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:20 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:20.667+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:21.129 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:20 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:20.667+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:21.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:21 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:21.003+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:21.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:21 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:21.003+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:21.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:21 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:21.003+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:21.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:21 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:21.003+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:21.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:21 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:21.003+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:21.515 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.7.service' 2024-04-21T08:59:21.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:59:21.593 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-21T08:59:21.594 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-21T08:59:21.594 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.8 2024-04-21T08:59:21.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:21 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:21.674+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:21.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:21 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:21.674+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:21.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:21 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:21.674+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:21.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:21 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:21.674+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:21.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:21 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:21.674+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:21.928 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:21 smithi134 systemd[1]: Stopping Ceph osd.8 for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:59:21.929 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:21 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:21.769+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:21.929 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:21 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:21.769+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:21.929 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:21 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:21.769+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:21.929 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:21 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:21.769+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.753566+0000 front 2024-04-21T08:58:54.753555+0000 (oldest deadline 2024-04-21T08:59:17.553922+0000) 2024-04-21T08:59:21.929 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:21 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:21.712+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:21.929 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:21 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:21.712+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:21.929 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:21 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:21.712+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:21.929 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:21 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:21.712+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:22.226 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:21 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:21.928+0000 7f90fdad8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:59:22.226 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:21 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:21.928+0000 7f90fdad8700 -1 osd.8 107 *** Got signal Terminated *** 2024-04-21T08:59:22.226 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:21 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:21.928+0000 7f90fdad8700 -1 osd.8 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:59:22.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:22 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:22.048+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:22.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:22 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:22.048+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:22.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:22 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:22.048+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:22.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:22 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:22.048+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:22.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:22 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:22.048+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:22.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:22 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:22.689+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:22.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:22 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:22.689+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:22.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:22 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:22.689+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:22.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:22 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:22.689+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:22.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:22 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:22.723+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:22.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:22 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:22.723+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:22.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:22 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:22.723+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:22.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:22 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:22.723+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.753566+0000 front 2024-04-21T08:58:54.753555+0000 (oldest deadline 2024-04-21T08:59:17.553922+0000) 2024-04-21T08:59:23.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:22 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:22.642+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:23.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:22 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:22.642+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:23.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:22 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:22.642+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:23.129 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:22 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:22.642+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:23.129 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:22 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:22.642+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:23.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:23 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:22.999+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:23.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:23 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:22.999+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:23.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:23 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:22.999+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:23.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:23 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:22.999+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:23.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:23 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:22.999+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:23.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:23 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:23.620+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:23.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:23 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:23.620+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:23.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:23 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:23.620+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:23.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:23 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:23.620+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:23.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:23 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:23.620+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:23.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:23 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:23.720+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:23.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:23 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:23.720+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:23.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:23 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:23.720+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:23.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:23 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:23.720+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:23.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:23 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:23.675+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:23.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:23 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:23.675+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:23.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:23 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:23.675+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:23.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:23 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:23.675+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.753566+0000 front 2024-04-21T08:58:54.753555+0000 (oldest deadline 2024-04-21T08:59:17.553922+0000) 2024-04-21T08:59:23.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:23 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:23.675+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.054851+0000 front 2024-04-21T08:58:58.054862+0000 (oldest deadline 2024-04-21T08:59:23.354878+0000) 2024-04-21T08:59:24.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:24.042+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:24.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:24.042+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:24.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:24.042+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:24.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:24.042+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:24.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:24.042+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:24.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:24 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:24.581+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:24.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:24 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:24.581+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:24.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:24 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:24.581+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:24.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:24 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:24.581+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:24.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:24 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:24.581+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:24.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:24 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:24.581+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:24.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:24 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:24.696+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:24.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:24 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:24.696+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:24.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:24 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:24.696+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:24.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:24 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:24.696+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:24.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:24 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:24.657+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:24.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:24 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:24.657+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:24.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:24 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:24.657+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:24.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:24 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:24.657+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.753566+0000 front 2024-04-21T08:58:54.753555+0000 (oldest deadline 2024-04-21T08:59:17.553922+0000) 2024-04-21T08:59:24.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:24 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:24.657+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.054851+0000 front 2024-04-21T08:58:58.054862+0000 (oldest deadline 2024-04-21T08:59:23.354878+0000) 2024-04-21T08:59:25.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:24.994+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:25.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:24.994+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:25.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:24.994+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:25.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:24.994+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:25.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:24 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:24.994+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:25.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:25 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:25.536+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:25.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:25 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:25.536+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:25.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:25 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:25.536+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:25.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:25 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:25.536+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:25.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:25 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:25.536+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:25.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:25 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:25.536+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:25.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:25 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:25.640+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:25.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:25 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:25.640+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:25.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:25 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:25.640+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:25.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:25 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:25.640+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.753566+0000 front 2024-04-21T08:58:54.753555+0000 (oldest deadline 2024-04-21T08:59:17.553922+0000) 2024-04-21T08:59:25.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:25 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:25.640+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.054851+0000 front 2024-04-21T08:58:58.054862+0000 (oldest deadline 2024-04-21T08:59:23.354878+0000) 2024-04-21T08:59:25.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:25 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:25.684+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:25.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:25 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:25.684+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:25.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:25 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:25.684+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:25.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:25 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:25.684+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:25.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:25 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:25.684+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:26.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:26 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:26.008+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:26.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:26 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:26.008+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:26.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:26 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:26.008+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:26.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:26 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:26.008+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:26.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:26 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:26.008+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:26.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:26 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:26.008+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:01.747312+0000 front 2024-04-21T08:59:01.747390+0000 (oldest deadline 2024-04-21T08:59:25.247154+0000) 2024-04-21T08:59:26.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:26 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:26.529+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:26.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:26 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:26.529+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:26.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:26 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:26.529+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:26.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:26 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:26.529+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:26.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:26 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:26.529+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:26.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:26 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:26.529+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:26.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:26 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:26.630+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:29.149250+0000 front 2024-04-21T08:58:29.149117+0000 (oldest deadline 2024-04-21T08:58:50.249350+0000) 2024-04-21T08:59:26.976 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:26 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:26.630+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.851032+0000 front 2024-04-21T08:58:37.850834+0000 (oldest deadline 2024-04-21T08:58:59.550998+0000) 2024-04-21T08:59:26.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:26 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:26.630+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.752545+0000 front 2024-04-21T08:58:44.752528+0000 (oldest deadline 2024-04-21T08:59:10.052380+0000) 2024-04-21T08:59:26.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:26 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:26.630+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.753566+0000 front 2024-04-21T08:58:54.753555+0000 (oldest deadline 2024-04-21T08:59:17.553922+0000) 2024-04-21T08:59:26.977 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:26 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8[50480]: 2024-04-21T08:59:26.630+0000 7f90fa0cc700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.054851+0000 front 2024-04-21T08:58:58.054862+0000 (oldest deadline 2024-04-21T08:59:23.354878+0000) 2024-04-21T08:59:26.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:26 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:26.697+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:26.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:26 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:26.697+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:26.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:26 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:26.697+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:26.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:26 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:26.697+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:26.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:26 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:26.697+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:27.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:27 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:27.009+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:27.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:27 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:27.009+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:27.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:27 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:27.009+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:27.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:27 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:27.009+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:27.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:27 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:27.009+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:27.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:27 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:27.009+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:01.747312+0000 front 2024-04-21T08:59:01.747390+0000 (oldest deadline 2024-04-21T08:59:25.247154+0000) 2024-04-21T08:59:27.804 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:27 smithi134 podman[69293]: 2024-04-21 08:59:27.502656268 +0000 UTC m=+5.703519977 container died 7343d3fb26aa5124bfad8af6d06d5adcc12eb466d60c18d314eeb22d22b0c262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-21T08:59:27.804 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:27 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:27.676+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:27.804 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:27 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:27.676+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:27.804 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:27 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:27.676+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:27.804 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:27 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:27.676+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:27.804 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:27 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:27.676+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:27.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:27 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:27.529+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:27.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:27 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:27.529+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:27.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:27 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:27.529+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:27.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:27 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:27.529+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:27.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:27 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:27.529+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:27.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:27 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:27.529+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:28.163 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:27 smithi134 podman[69293]: 2024-04-21 08:59:27.912135461 +0000 UTC m=+6.112999182 container cleanup 7343d3fb26aa5124bfad8af6d06d5adcc12eb466d60c18d314eeb22d22b0c262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, ceph=True) 2024-04-21T08:59:28.163 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:27 smithi134 bash[69293]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8 2024-04-21T08:59:28.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:28 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:28.005+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:28.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:28 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:28.005+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:28.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:28 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:28.005+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:28.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:28 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:28.005+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:28.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:28 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:28.005+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:28.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:28 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:28.005+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:01.747312+0000 front 2024-04-21T08:59:01.747390+0000 (oldest deadline 2024-04-21T08:59:25.247154+0000) 2024-04-21T08:59:28.471 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:28 smithi134 podman[69312]: 2024-04-21 08:59:28.163655671 +0000 UTC m=+0.657116075 container remove 7343d3fb26aa5124bfad8af6d06d5adcc12eb466d60c18d314eeb22d22b0c262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-21T08:59:28.726 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:28 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:28.672+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:28.726 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:28 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:28.672+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:28.726 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:28 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:28.672+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:28.726 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:28 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:28.672+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:28.726 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:28 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:28.672+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:28.727 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:28 smithi134 podman[69402]: 2024-04-21 08:59:28.472799022 +0000 UTC m=+0.102627027 container create 300e950844e25058b96a3584e0f00f9a3e8798e0674e1ae3b8a13b109cff81de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-21T08:59:28.727 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:28 smithi134 podman[69402]: 2024-04-21 08:59:28.388622911 +0000 UTC m=+0.018450933 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:59:28.727 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:28 smithi134 podman[69402]: 2024-04-21 08:59:28.716842138 +0000 UTC m=+0.346670143 container init 300e950844e25058b96a3584e0f00f9a3e8798e0674e1ae3b8a13b109cff81de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-21T08:59:28.727 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:28 smithi134 podman[69402]: 2024-04-21 08:59:28.719867908 +0000 UTC m=+0.349695917 container start 300e950844e25058b96a3584e0f00f9a3e8798e0674e1ae3b8a13b109cff81de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:59:28.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:28 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:28.575+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:28.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:28 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:28.575+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:28.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:28 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:28.575+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:28.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:28 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:28.575+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:28.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:28 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:28.575+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:28.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:28 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:28.575+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:29.010 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:28 smithi134 podman[69402]: 2024-04-21 08:59:28.825587934 +0000 UTC m=+0.455415940 container attach 300e950844e25058b96a3584e0f00f9a3e8798e0674e1ae3b8a13b109cff81de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, ceph=True) 2024-04-21T08:59:29.310 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 21 08:59:29 smithi134 podman[69402]: 2024-04-21 08:59:29.011263247 +0000 UTC m=+0.641091246 container died 300e950844e25058b96a3584e0f00f9a3e8798e0674e1ae3b8a13b109cff81de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-21T08:59:29.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:29 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:29.030+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:29.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:29 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:29.030+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:29.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:29 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:29.030+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:29.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:29 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:29.030+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:29.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:29 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:29.030+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:29.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:29 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:29.030+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:01.747312+0000 front 2024-04-21T08:59:01.747390+0000 (oldest deadline 2024-04-21T08:59:25.247154+0000) 2024-04-21T08:59:29.643 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.8.service' 2024-04-21T08:59:29.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:59:29.681 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-21T08:59:29.681 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-21T08:59:29.681 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.9 2024-04-21T08:59:29.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:29 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:29.594+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:29.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:29 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:29.594+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:29.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:29 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:29.594+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:29.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:29 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:29.594+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:29.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:29 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:29.594+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:29.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:29 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:29.594+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:29.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:29 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:29.657+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:29.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:29 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:29.657+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:29.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:29 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:29.657+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:29.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:29 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:29.657+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:29.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:29 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:29.657+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:29.985 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:29 smithi082 systemd[1]: Stopping Ceph osd.9 for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:59:30.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:29 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:29.985+0000 7f923daa9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:59:30.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:29 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:29.986+0000 7f923daa9700 -1 osd.9 107 *** Got signal Terminated *** 2024-04-21T08:59:30.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:29 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:29.986+0000 7f923daa9700 -1 osd.9 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:59:30.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:30 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:30.016+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:30.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:30 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:30.016+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:30.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:30 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:30.016+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:30.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:30 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:30.016+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:30.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:30 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:30.016+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:30.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:30 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:30.016+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:01.747312+0000 front 2024-04-21T08:59:01.747390+0000 (oldest deadline 2024-04-21T08:59:25.247154+0000) 2024-04-21T08:59:30.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:30 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:30.621+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:30.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:30 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:30.621+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:30.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:30 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:30.621+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:30.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:30 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:30.621+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:30.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:30 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:30.621+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:30.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:30 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:30.621+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:30.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:30 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:30.686+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:30.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:30 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:30.686+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:30.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:30 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:30.686+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:30.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:30 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:30.686+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:30.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:30 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:30.686+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:31.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:31 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:31.063+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:31.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:31 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:31.063+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:31.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:31 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:31.063+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:31.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:31 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:31.063+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:31.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:31 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:31.063+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:31.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:31 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:31.063+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:01.747312+0000 front 2024-04-21T08:59:01.747390+0000 (oldest deadline 2024-04-21T08:59:25.247154+0000) 2024-04-21T08:59:31.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:31 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:31.576+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:31.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:31 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:31.576+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:31.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:31 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:31.576+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:31.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:31 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:31.576+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:31.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:31 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:31.576+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:31.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:31 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:31.576+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:31.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:31 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:31.710+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:31.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:31 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:31.710+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:31.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:31 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:31.710+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:31.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:31 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:31.710+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:31.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:31 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:31.710+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:32.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:32 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:32.012+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:32.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:32 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:32.012+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:32.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:32 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:32.012+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:32.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:32 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:32.012+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:32.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:32 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:32.012+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:32.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:32 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:32.012+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:01.747312+0000 front 2024-04-21T08:59:01.747390+0000 (oldest deadline 2024-04-21T08:59:25.247154+0000) 2024-04-21T08:59:32.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:32 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:32.574+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:32.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:32 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:32.574+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:32.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:32 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:32.574+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:32.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:32 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:32.574+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:32.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:32 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:32.574+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:32.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:32 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:32.574+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:32.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:32 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:32.709+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:32.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:32 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:32.709+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:32.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:32 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:32.709+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:32.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:32 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:32.709+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:32.977 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:32 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:32.709+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:33.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:33 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:33.050+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:33.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:33 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:33.050+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:33.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:33 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:33.050+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:33.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:33 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:33.050+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:33.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:33 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:33.050+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:33.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:33 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:33.050+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:01.747312+0000 front 2024-04-21T08:59:01.747390+0000 (oldest deadline 2024-04-21T08:59:25.247154+0000) 2024-04-21T08:59:33.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:33 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:33.535+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:33.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:33 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:33.535+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:33.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:33 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:33.535+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:33.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:33 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:33.535+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:33.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:33 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:33.535+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:33.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:33 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:33.535+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:33.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:33 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:33.716+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:33.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:33 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:33.716+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:33.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:33 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:33.716+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:33.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:33 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:33.716+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:33.976 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:33 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:33.716+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:34.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:34 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:34.036+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.141431+0000 front 2024-04-21T08:58:20.141563+0000 (oldest deadline 2024-04-21T08:58:46.041612+0000) 2024-04-21T08:59:34.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:34 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:34.036+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:26.042270+0000 front 2024-04-21T08:58:26.042037+0000 (oldest deadline 2024-04-21T08:58:51.942165+0000) 2024-04-21T08:59:34.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:34 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:34.036+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.243286+0000 front 2024-04-21T08:58:37.243388+0000 (oldest deadline 2024-04-21T08:59:01.943345+0000) 2024-04-21T08:59:34.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:34 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:34.036+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:43.644334+0000 front 2024-04-21T08:58:43.644378+0000 (oldest deadline 2024-04-21T08:59:08.944448+0000) 2024-04-21T08:59:34.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:34 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:34.036+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.746110+0000 front 2024-04-21T08:58:54.746229+0000 (oldest deadline 2024-04-21T08:59:16.446034+0000) 2024-04-21T08:59:34.359 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:34 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9[63431]: 2024-04-21T08:59:34.036+0000 7f923a09d700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:01.747312+0000 front 2024-04-21T08:59:01.747390+0000 (oldest deadline 2024-04-21T08:59:25.247154+0000) 2024-04-21T08:59:34.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:34 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:34.547+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:34.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:34 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:34.547+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:34.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:34 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:34.547+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:34.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:34 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:34.547+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:34.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:34 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:34.547+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:34.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:34 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:34.547+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:35.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:34 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:34.749+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:35.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:34 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:34.749+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:35.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:34 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:34.749+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:35.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:34 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:34.749+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:35.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:34 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:34.749+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:35.802 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:35 smithi082 podman[104341]: 2024-04-21 08:59:35.503503654 +0000 UTC m=+5.628427725 container died 707e1c0f2d35e0c907a17bc9ef7c39fe791056b9b5f37f370d32f7e3a4c7e8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-21T08:59:35.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:35 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:35.500+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:35.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:35 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:35.500+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:35.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:35 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:35.500+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:35.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:35 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:35.500+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:35.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:35 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:35.500+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:35.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:35 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:35.500+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:35.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:35 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:35.500+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:10.686935+0000 front 2024-04-21T08:59:10.686957+0000 (oldest deadline 2024-04-21T08:59:35.386916+0000) 2024-04-21T08:59:36.077 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:35 smithi082 podman[104341]: 2024-04-21 08:59:35.902346757 +0000 UTC m=+6.027270827 container cleanup 707e1c0f2d35e0c907a17bc9ef7c39fe791056b9b5f37f370d32f7e3a4c7e8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-21T08:59:36.077 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:35 smithi082 bash[104341]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9 2024-04-21T08:59:36.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:35 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:35.799+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:36.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:35 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:35.799+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:36.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:35 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:35.799+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:36.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:35 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:35.799+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:36.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:35 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:35.799+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:36.358 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:36 smithi082 podman[104358]: 2024-04-21 08:59:36.078301597 +0000 UTC m=+0.568599206 container remove 707e1c0f2d35e0c907a17bc9ef7c39fe791056b9b5f37f370d32f7e3a4c7e8dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-21T08:59:36.789 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:36 smithi082 podman[104449]: 2024-04-21 08:59:36.417395089 +0000 UTC m=+0.017646960 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:59:36.789 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:36 smithi082 podman[104449]: 2024-04-21 08:59:36.520849655 +0000 UTC m=+0.121101528 container create b1b0256f1809118d34f03889a6c98f1e72357c072ee538bf4d68afef90645cd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:59:36.789 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:36 smithi082 podman[104449]: 2024-04-21 08:59:36.7896985 +0000 UTC m=+0.389950371 container init b1b0256f1809118d34f03889a6c98f1e72357c072ee538bf4d68afef90645cd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:59:36.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:36 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:36.480+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:36.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:36 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:36.480+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:36.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:36 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:36.480+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:36.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:36 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:36.480+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:36.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:36 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:36.480+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:36.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:36 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:36.480+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:36.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:36 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:36.480+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:10.686935+0000 front 2024-04-21T08:59:10.686957+0000 (oldest deadline 2024-04-21T08:59:35.386916+0000) 2024-04-21T08:59:37.081 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:36 smithi082 podman[104449]: 2024-04-21 08:59:36.792980387 +0000 UTC m=+0.393232259 container start b1b0256f1809118d34f03889a6c98f1e72357c072ee538bf4d68afef90645cd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:59:37.081 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:36 smithi082 podman[104449]: 2024-04-21 08:59:36.873030161 +0000 UTC m=+0.473282032 container attach b1b0256f1809118d34f03889a6c98f1e72357c072ee538bf4d68afef90645cd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:59:37.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:36 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:36.765+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:37.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:36 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:36.765+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:37.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:36 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:36.765+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:37.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:36 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:36.765+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:37.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:36 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:36.765+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:37.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:36 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:36.765+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:37.350 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 21 08:59:37 smithi082 podman[104449]: 2024-04-21 08:59:37.082126453 +0000 UTC m=+0.682378317 container died b1b0256f1809118d34f03889a6c98f1e72357c072ee538bf4d68afef90645cd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:59:37.761 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.9.service' 2024-04-21T08:59:37.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:59:37.797 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-21T08:59:37.797 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-21T08:59:37.798 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.10 2024-04-21T08:59:37.823 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:37 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:37.462+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:37.824 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:37 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:37.462+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:37.824 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:37 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:37.462+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:37.824 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:37 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:37.462+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:37.824 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:37 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:37.462+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:37.824 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:37 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:37.462+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:37.824 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:37 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:37.462+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:10.686935+0000 front 2024-04-21T08:59:10.686957+0000 (oldest deadline 2024-04-21T08:59:35.386916+0000) 2024-04-21T08:59:38.073 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:37 smithi106 systemd[1]: Stopping Ceph osd.10 for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:59:38.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:37 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:37.790+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:38.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:37 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:37.790+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:38.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:37 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:37.790+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:38.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:37 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:37.790+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:38.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:37 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:37.790+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:38.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:37 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:37.790+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:38.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:38 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:38.073+0000 7ff8247f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:59:38.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:38 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:38.073+0000 7ff8247f7700 -1 osd.10 107 *** Got signal Terminated *** 2024-04-21T08:59:38.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:38 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:38.073+0000 7ff8247f7700 -1 osd.10 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:59:38.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:38 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:38.432+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:38.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:38 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:38.432+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:38.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:38 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:38.432+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:38.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:38 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:38.432+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:38.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:38 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:38.432+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:38.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:38 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:38.432+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:38.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:38 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:38.432+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:10.686935+0000 front 2024-04-21T08:59:10.686957+0000 (oldest deadline 2024-04-21T08:59:35.386916+0000) 2024-04-21T08:59:39.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:38 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:38.840+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:39.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:38 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:38.840+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:39.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:38 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:38.840+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:39.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:38 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:38.840+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:39.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:38 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:38.840+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:39.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:38 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:38.840+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:39.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:39.410+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:39.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:39.410+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:39.880 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:39.410+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:39.880 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:39.410+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:39.880 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:39.410+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:39.880 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:39.410+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:39.880 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:39 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:39.410+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:10.686935+0000 front 2024-04-21T08:59:10.686957+0000 (oldest deadline 2024-04-21T08:59:35.386916+0000) 2024-04-21T08:59:40.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:39 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:39.821+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:40.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:39 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:39.821+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:40.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:39 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:39.821+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:40.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:39 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:39.821+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:40.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:39 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:39.821+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:40.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:39 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:39.821+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:40.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:40 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:40.436+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:40.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:40 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:40.436+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:40.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:40 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:40.436+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:40.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:40 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:40.436+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:40.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:40 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:40.436+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:40.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:40 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:40.436+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:40.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:40 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:40.436+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:10.686935+0000 front 2024-04-21T08:59:10.686957+0000 (oldest deadline 2024-04-21T08:59:35.386916+0000) 2024-04-21T08:59:41.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:40 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:40.824+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:41.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:40 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:40.824+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:41.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:40 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:40.824+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:41.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:40 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:40.824+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:41.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:40 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:40.824+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:41.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:40 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:40.824+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:41.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:41.427+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:41.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:41.427+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:41.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:41.427+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:41.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:41.427+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:41.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:41.427+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:41.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:41.427+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:41.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:41 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:41.427+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:10.686935+0000 front 2024-04-21T08:59:10.686957+0000 (oldest deadline 2024-04-21T08:59:35.386916+0000) 2024-04-21T08:59:42.128 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:41 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:41.827+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:42.128 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:41 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:41.827+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:42.128 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:41 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:41.827+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:42.128 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:41 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:41.827+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:42.128 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:41 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:41.827+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:42.128 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:41 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:41.827+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:42.128 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:41 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:41.827+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-21T08:59:16.275937+0000 front 2024-04-21T08:59:16.275999+0000 (oldest deadline 2024-04-21T08:59:41.575877+0000) 2024-04-21T08:59:42.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:42 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:42.447+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:20.977768+0000 front 2024-04-21T08:58:20.977736+0000 (oldest deadline 2024-04-21T08:58:43.877809+0000) 2024-04-21T08:59:42.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:42 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:42.447+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-21T08:58:28.479498+0000 front 2024-04-21T08:58:28.479541+0000 (oldest deadline 2024-04-21T08:58:51.979697+0000) 2024-04-21T08:59:42.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:42 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:42.447+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:33.680537+0000 front 2024-04-21T08:58:33.680477+0000 (oldest deadline 2024-04-21T08:59:00.081039+0000) 2024-04-21T08:59:42.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:42 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:42.447+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:45.683317+0000 front 2024-04-21T08:58:45.683396+0000 (oldest deadline 2024-04-21T08:59:09.783129+0000) 2024-04-21T08:59:42.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:42 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:42.447+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:49.783421+0000 front 2024-04-21T08:58:49.783439+0000 (oldest deadline 2024-04-21T08:59:18.584163+0000) 2024-04-21T08:59:42.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:42 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:42.447+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:58:58.584625+0000 front 2024-04-21T08:58:58.584711+0000 (oldest deadline 2024-04-21T08:59:23.884539+0000) 2024-04-21T08:59:42.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:42 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10[55622]: 2024-04-21T08:59:42.447+0000 7ff820deb700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:10.686935+0000 front 2024-04-21T08:59:10.686957+0000 (oldest deadline 2024-04-21T08:59:35.386916+0000) 2024-04-21T08:59:43.145 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:42 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:42.853+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:43.145 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:42 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:42.853+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:43.145 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:42 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:42.853+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:43.145 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:42 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:42.853+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:43.145 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:42 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:42.853+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:43.145 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:42 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:42.853+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:43.145 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:42 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:42.853+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-21T08:59:16.275937+0000 front 2024-04-21T08:59:16.275999+0000 (oldest deadline 2024-04-21T08:59:41.575877+0000) 2024-04-21T08:59:43.715 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:43 smithi106 podman[67480]: 2024-04-21 08:59:43.438253024 +0000 UTC m=+5.451856061 container died 892ee78c46f68fbddb8a4421ba838881a6654d18b6628b3558f013de4c85175c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-21T08:59:43.715 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:43 smithi106 podman[67480]: 2024-04-21 08:59:43.715935477 +0000 UTC m=+5.729538502 container cleanup 892ee78c46f68fbddb8a4421ba838881a6654d18b6628b3558f013de4c85175c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-04-21T08:59:44.131 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:43 smithi106 bash[67480]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10 2024-04-21T08:59:44.131 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:43 smithi106 podman[67496]: 2024-04-21 08:59:43.959148073 +0000 UTC m=+0.517179830 container remove 892ee78c46f68fbddb8a4421ba838881a6654d18b6628b3558f013de4c85175c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-21T08:59:44.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:43 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:43.882+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:44.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:43 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:43.882+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:44.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:43 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:43.882+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:44.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:43 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:43.882+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:44.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:43 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:43.882+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:44.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:43 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:43.882+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:44.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:43 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:43.882+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-21T08:59:16.275937+0000 front 2024-04-21T08:59:16.275999+0000 (oldest deadline 2024-04-21T08:59:41.575877+0000) 2024-04-21T08:59:44.595 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:44 smithi106 podman[67584]: 2024-04-21 08:59:44.18323872 +0000 UTC m=+0.018165669 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:59:44.595 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:44 smithi106 podman[67584]: 2024-04-21 08:59:44.301727016 +0000 UTC m=+0.136653993 container create 7139014574c589da0b4b3a31c1c1139bcb8e502570fd9ea08e1fd97e392e6d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:59:44.595 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:44 smithi106 podman[67584]: 2024-04-21 08:59:44.52075573 +0000 UTC m=+0.355682666 container init 7139014574c589da0b4b3a31c1c1139bcb8e502570fd9ea08e1fd97e392e6d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2) 2024-04-21T08:59:44.596 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:44 smithi106 podman[67584]: 2024-04-21 08:59:44.523748188 +0000 UTC m=+0.358675132 container start 7139014574c589da0b4b3a31c1c1139bcb8e502570fd9ea08e1fd97e392e6d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=) 2024-04-21T08:59:44.596 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:44 smithi106 podman[67584]: 2024-04-21 08:59:44.595495443 +0000 UTC m=+0.430422402 container attach 7139014574c589da0b4b3a31c1c1139bcb8e502570fd9ea08e1fd97e392e6d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-21T08:59:44.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 21 08:59:44 smithi106 podman[67584]: 2024-04-21 08:59:44.811272186 +0000 UTC m=+0.646199122 container died 7139014574c589da0b4b3a31c1c1139bcb8e502570fd9ea08e1fd97e392e6d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-21T08:59:45.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:44 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:44.851+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:45.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:44 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:44.851+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:45.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:44 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:44.851+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:45.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:44 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:44.851+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:45.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:44 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:44.851+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:45.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:44 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:44.851+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:45.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:44 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:44.851+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-21T08:59:16.275937+0000 front 2024-04-21T08:59:16.275999+0000 (oldest deadline 2024-04-21T08:59:41.575877+0000) 2024-04-21T08:59:45.384 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.10.service' 2024-04-21T08:59:45.461 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:59:45.461 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-21T08:59:45.461 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-21T08:59:45.461 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.11 2024-04-21T08:59:45.764 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:45 smithi134 systemd[1]: Stopping Ceph osd.11 for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:59:46.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:45 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:45.763+0000 7fcafbe03700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:59:46.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:45 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:45.763+0000 7fcafbe03700 -1 osd.11 107 *** Got signal Terminated *** 2024-04-21T08:59:46.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:45 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:45.763+0000 7fcafbe03700 -1 osd.11 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:59:46.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:45 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:45.894+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:46.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:45 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:45.894+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:46.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:45 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:45.894+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:46.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:45 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:45.894+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:46.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:45 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:45.894+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:46.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:45 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:45.894+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:46.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:45 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:45.894+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-21T08:59:16.275937+0000 front 2024-04-21T08:59:16.275999+0000 (oldest deadline 2024-04-21T08:59:41.575877+0000) 2024-04-21T08:59:47.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:46 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:46.913+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:47.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:46 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:46.913+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:47.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:46 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:46.913+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:47.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:46 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:46.913+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:47.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:46 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:46.913+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:47.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:46 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:46.913+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:47.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:46 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:46.913+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-21T08:59:16.275937+0000 front 2024-04-21T08:59:16.275999+0000 (oldest deadline 2024-04-21T08:59:41.575877+0000) 2024-04-21T08:59:48.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:47 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:47.902+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:48.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:47 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:47.902+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:48.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:47 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:47.902+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:48.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:47 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:47.902+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:48.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:47 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:47.902+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:48.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:47 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:47.902+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:48.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:47 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:47.902+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-21T08:59:16.275937+0000 front 2024-04-21T08:59:16.275999+0000 (oldest deadline 2024-04-21T08:59:41.575877+0000) 2024-04-21T08:59:49.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:48 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:48.858+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:49.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:48 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:48.858+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:49.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:48 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:48.858+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:49.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:48 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:48.858+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:49.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:48 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:48.858+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:49.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:48 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:48.858+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:49.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:48 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:48.858+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-21T08:59:16.275937+0000 front 2024-04-21T08:59:16.275999+0000 (oldest deadline 2024-04-21T08:59:41.575877+0000) 2024-04-21T08:59:50.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:49 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:49.844+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-21T08:58:19.666777+0000 front 2024-04-21T08:58:19.666840+0000 (oldest deadline 2024-04-21T08:58:44.966842+0000) 2024-04-21T08:59:50.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:49 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:49.844+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-21T08:58:37.670439+0000 front 2024-04-21T08:58:37.670541+0000 (oldest deadline 2024-04-21T08:59:01.770489+0000) 2024-04-21T08:59:50.226 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:49 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:49.844+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-21T08:58:44.671674+0000 front 2024-04-21T08:58:44.671689+0000 (oldest deadline 2024-04-21T08:59:08.171455+0000) 2024-04-21T08:59:50.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:49 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:49.844+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-21T08:58:54.072487+0000 front 2024-04-21T08:58:54.072474+0000 (oldest deadline 2024-04-21T08:59:18.673171+0000) 2024-04-21T08:59:50.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:49 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:49.844+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-21T08:59:00.374283+0000 front 2024-04-21T08:59:00.374187+0000 (oldest deadline 2024-04-21T08:59:25.674354+0000) 2024-04-21T08:59:50.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:49 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:49.844+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-21T08:59:05.674953+0000 front 2024-04-21T08:59:05.674968+0000 (oldest deadline 2024-04-21T08:59:36.275256+0000) 2024-04-21T08:59:50.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:49 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:49.844+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-21T08:59:16.275937+0000 front 2024-04-21T08:59:16.275999+0000 (oldest deadline 2024-04-21T08:59:41.575877+0000) 2024-04-21T08:59:50.227 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:49 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11[57100]: 2024-04-21T08:59:49.844+0000 7fcaf43ef700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.134:6820 osd.8 since back 2024-04-21T08:59:25.076649+0000 front 2024-04-21T08:59:25.076681+0000 (oldest deadline 2024-04-21T08:59:49.776749+0000) 2024-04-21T08:59:51.472 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:51 smithi134 podman[69519]: 2024-04-21 08:59:51.217485455 +0000 UTC m=+5.555825155 container died d9f9e1cdc8c7698197f9ea7c09e3063f8bfbe8fe49c4cb2ebe087e27e23222c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-21T08:59:51.726 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:51 smithi134 podman[69519]: 2024-04-21 08:59:51.563670912 +0000 UTC m=+5.902010612 container cleanup d9f9e1cdc8c7698197f9ea7c09e3063f8bfbe8fe49c4cb2ebe087e27e23222c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-21T08:59:51.726 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:51 smithi134 bash[69519]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11 2024-04-21T08:59:52.181 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:51 smithi134 podman[69538]: 2024-04-21 08:59:51.756479135 +0000 UTC m=+0.534257328 container remove d9f9e1cdc8c7698197f9ea7c09e3063f8bfbe8fe49c4cb2ebe087e27e23222c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, ceph=True) 2024-04-21T08:59:52.458 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:52 smithi134 podman[69626]: 2024-04-21 08:59:52.182030311 +0000 UTC m=+0.108345676 container create d7f6be42db963bbe5f8e4f8cf15670f3957b6c53fee2e102a9a60c326ec932f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-21T08:59:52.459 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:52 smithi134 podman[69626]: 2024-04-21 08:59:52.091472507 +0000 UTC m=+0.017787885 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:59:52.459 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:52 smithi134 podman[69626]: 2024-04-21 08:59:52.384307237 +0000 UTC m=+0.310622602 container init d7f6be42db963bbe5f8e4f8cf15670f3957b6c53fee2e102a9a60c326ec932f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-21T08:59:52.459 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:52 smithi134 podman[69626]: 2024-04-21 08:59:52.387326392 +0000 UTC m=+0.313641758 container start d7f6be42db963bbe5f8e4f8cf15670f3957b6c53fee2e102a9a60c326ec932f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-21T08:59:52.459 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:52 smithi134 podman[69626]: 2024-04-21 08:59:52.459462767 +0000 UTC m=+0.385778152 container attach d7f6be42db963bbe5f8e4f8cf15670f3957b6c53fee2e102a9a60c326ec932f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:59:52.726 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:52 smithi134 podman[69626]: 2024-04-21 08:59:52.683494017 +0000 UTC m=+0.609809382 container died d7f6be42db963bbe5f8e4f8cf15670f3957b6c53fee2e102a9a60c326ec932f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:59:53.204 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 21 08:59:53 smithi134 podman[69652]: 2024-04-21 08:59:53.180769183 +0000 UTC m=+0.491673520 container remove d7f6be42db963bbe5f8e4f8cf15670f3957b6c53fee2e102a9a60c326ec932f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:59:53.208 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@osd.11.service' 2024-04-21T08:59:53.246 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:59:53.246 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-21T08:59:53.246 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-21T08:59:53.246 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.a 2024-04-21T08:59:53.578 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:59:53 smithi082 systemd[1]: Stopping Ceph mds.a for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:59:53.858 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:59:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-a[70246]: 2024-04-21T08:59:53.577+0000 7fe946cdd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:59:53.859 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:59:53 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-a[70246]: 2024-04-21T08:59:53.577+0000 7fe946cdd700 -1 mds.a *** got signal Terminated *** 2024-04-21T08:59:54.859 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:59:54 smithi082 podman[104567]: 2024-04-21 08:59:54.695287537 +0000 UTC m=+1.242433202 container died 54b972facf11646326bc478b5486664c7527f68404d28b9af29865023ceb0edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-21T08:59:55.352 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:59:55 smithi082 podman[104567]: 2024-04-21 08:59:55.068656682 +0000 UTC m=+1.615802355 container cleanup 54b972facf11646326bc478b5486664c7527f68404d28b9af29865023ceb0edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-04-21T08:59:55.352 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 21 08:59:55 smithi082 bash[104567]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-a 2024-04-21T08:59:55.501 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.a.service' 2024-04-21T08:59:55.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:59:55.536 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-21T08:59:55.536 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-21T08:59:55.536 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.d 2024-04-21T08:59:56.109 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:59:55 smithi082 systemd[1]: Stopping Ceph mds.d for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:59:56.109 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:59:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-d[69290]: 2024-04-21T08:59:55.846+0000 7fd6fe5e4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:59:56.109 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:59:55 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-d[69290]: 2024-04-21T08:59:55.846+0000 7fd6fe5e4700 -1 mds.d *** got signal Terminated *** 2024-04-21T08:59:57.888 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:59:57 smithi082 podman[104700]: 2024-04-21 08:59:57.625215828 +0000 UTC m=+1.873714137 container died 01b15b9008c67a1bcd886cc11099a025f7ee882ba804fd89a4f1d08ba7cae350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-21T08:59:58.247 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:59:57 smithi082 podman[104700]: 2024-04-21 08:59:57.988591735 +0000 UTC m=+2.237090050 container cleanup 01b15b9008c67a1bcd886cc11099a025f7ee882ba804fd89a4f1d08ba7cae350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-21T08:59:58.247 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 21 08:59:57 smithi082 bash[104700]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-d 2024-04-21T08:59:58.409 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.d.service' 2024-04-21T08:59:58.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:59:58.441 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-21T08:59:58.441 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-21T08:59:58.441 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.g 2024-04-21T08:59:58.783 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:59:58 smithi082 systemd[1]: Stopping Ceph mds.g for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T08:59:59.108 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:59:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-g[69899]: 2024-04-21T08:59:58.783+0000 7f9eb84ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:59:59.109 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 08:59:58 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-g[69899]: 2024-04-21T08:59:58.783+0000 7f9eb84ae700 -1 mds.g *** got signal Terminated *** 2024-04-21T09:00:03.108 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 09:00:02 smithi082 podman[104822]: 2024-04-21 09:00:02.704580795 +0000 UTC m=+4.042874651 container died 81d7a4d8f12c2ff7ff9ef2f04854687bea804a1d523b0c4b56864d4f9abe19d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-21T09:00:03.534 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 09:00:03 smithi082 podman[104822]: 2024-04-21 09:00:03.316954796 +0000 UTC m=+4.655248650 container cleanup 81d7a4d8f12c2ff7ff9ef2f04854687bea804a1d523b0c4b56864d4f9abe19d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-21T09:00:03.534 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 21 09:00:03 smithi082 bash[104822]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-g 2024-04-21T09:00:03.748 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.g.service' 2024-04-21T09:00:03.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:00:03.780 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-21T09:00:03.781 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-21T09:00:03.781 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.j 2024-04-21T09:00:04.103 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 21 09:00:03 smithi082 systemd[1]: Stopping Ceph mds.j for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T09:00:04.358 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 21 09:00:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-j[68940]: 2024-04-21T09:00:04.103+0000 7eff6a892700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:00:04.358 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 21 09:00:04 smithi082 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-j[68940]: 2024-04-21T09:00:04.103+0000 7eff6a892700 -1 mds.j *** got signal Terminated *** 2024-04-21T09:00:07.028 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 21 09:00:06 smithi082 podman[104944]: 2024-04-21 09:00:06.754713135 +0000 UTC m=+2.760485596 container died 9ea64cc0decbdfd0afceb90af110ca3325649689b83e30288386ceff15b80a87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS) 2024-04-21T09:00:07.358 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 21 09:00:07 smithi082 podman[104944]: 2024-04-21 09:00:07.111758612 +0000 UTC m=+3.117531071 container cleanup 9ea64cc0decbdfd0afceb90af110ca3325649689b83e30288386ceff15b80a87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-21T09:00:07.359 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 21 09:00:07 smithi082 bash[104944]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-j 2024-04-21T09:00:07.541 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.j.service' 2024-04-21T09:00:07.572 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:00:07.572 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-21T09:00:07.572 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-21T09:00:07.573 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.b 2024-04-21T09:00:07.868 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 09:00:07 smithi106 systemd[1]: Stopping Ceph mds.b for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T09:00:08.128 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 09:00:07 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b[60834]: 2024-04-21T09:00:07.867+0000 7f68b05a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:00:08.128 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 09:00:07 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b[60834]: 2024-04-21T09:00:07.867+0000 7f68b05a3700 -1 mds.b *** got signal Terminated *** 2024-04-21T09:00:11.018 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 09:00:10 smithi106 podman[67707]: 2024-04-21 09:00:10.739135373 +0000 UTC m=+2.976184375 container died 6327164844ccc216ff5ab8fa5f075bf1d48bca5a93cf23834a6a0f9ab610c2f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T09:00:11.293 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 09:00:11 smithi106 podman[67707]: 2024-04-21 09:00:11.092826957 +0000 UTC m=+3.329876012 container cleanup 6327164844ccc216ff5ab8fa5f075bf1d48bca5a93cf23834a6a0f9ab610c2f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-21T09:00:11.293 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 09:00:11 smithi106 bash[67707]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b 2024-04-21T09:00:11.526 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.b.service' 2024-04-21T09:00:11.547 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 09:00:11 smithi106 podman[67721]: 2024-04-21 09:00:11.294091538 +0000 UTC m=+0.536401863 container remove 6327164844ccc216ff5ab8fa5f075bf1d48bca5a93cf23834a6a0f9ab610c2f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-21T09:00:11.547 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 09:00:11 smithi106 systemd[1]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.b.service: Deactivated successfully. 2024-04-21T09:00:11.547 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 09:00:11 smithi106 systemd[1]: Stopped Ceph mds.b for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T09:00:11.547 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 21 09:00:11 smithi106 systemd[1]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.b.service: Consumed 48.126s CPU time. 2024-04-21T09:00:11.560 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:00:11.560 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-21T09:00:11.560 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-21T09:00:11.560 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.e 2024-04-21T09:00:11.878 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 09:00:11 smithi106 systemd[1]: Stopping Ceph mds.e for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T09:00:12.378 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 09:00:11 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-e[60220]: 2024-04-21T09:00:11.888+0000 7f44c06f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:00:12.378 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 09:00:11 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-e[60220]: 2024-04-21T09:00:11.888+0000 7f44c06f8700 -1 mds.e *** got signal Terminated *** 2024-04-21T09:00:15.020 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 09:00:14 smithi106 podman[67828]: 2024-04-21 09:00:14.736684573 +0000 UTC m=+2.957067472 container died 251b53e53e7370bdf6db1b62ebac839f0b8fbc68ba6035a46ef7b8f6272f4a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-21T09:00:15.281 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 09:00:15 smithi106 podman[67828]: 2024-04-21 09:00:15.021521792 +0000 UTC m=+3.241904701 container cleanup 251b53e53e7370bdf6db1b62ebac839f0b8fbc68ba6035a46ef7b8f6272f4a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-21T09:00:15.281 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 21 09:00:15 smithi106 bash[67828]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-e 2024-04-21T09:00:15.408 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.e.service' 2024-04-21T09:00:15.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:00:15.440 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-21T09:00:15.440 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-21T09:00:15.440 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.h 2024-04-21T09:00:15.628 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 21 09:00:15 smithi106 systemd[1]: Stopping Ceph mds.h for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T09:00:16.128 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 21 09:00:15 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-h[60529]: 2024-04-21T09:00:15.766+0000 7fde25efe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:00:16.128 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 21 09:00:15 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-h[60529]: 2024-04-21T09:00:15.766+0000 7fde25efe700 -1 mds.h *** got signal Terminated *** 2024-04-21T09:00:19.000 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 21 09:00:18 smithi106 podman[67950]: 2024-04-21 09:00:18.736676372 +0000 UTC m=+3.072381244 container died 28df946b28d71a43e064b999014677292252992232ba0abbe8e2192dbfc671f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-21T09:00:19.351 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 21 09:00:19 smithi106 podman[67950]: 2024-04-21 09:00:19.092271459 +0000 UTC m=+3.427976313 container cleanup 28df946b28d71a43e064b999014677292252992232ba0abbe8e2192dbfc671f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-21T09:00:19.351 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 21 09:00:19 smithi106 bash[67950]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-h 2024-04-21T09:00:19.470 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.h.service' 2024-04-21T09:00:19.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:00:19.503 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-21T09:00:19.503 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-21T09:00:19.503 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.k 2024-04-21T09:00:19.628 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 09:00:19 smithi106 systemd[1]: Stopping Ceph mds.k for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T09:00:20.128 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 09:00:19 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-k[59900]: 2024-04-21T09:00:19.812+0000 7fbb14d7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:00:20.129 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 09:00:19 smithi106 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-k[59900]: 2024-04-21T09:00:19.812+0000 7fbb14d7f700 -1 mds.k *** got signal Terminated *** 2024-04-21T09:00:23.021 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 09:00:22 smithi106 podman[68082]: 2024-04-21 09:00:22.747660779 +0000 UTC m=+3.021614598 container died 723e3ca6f5c51798d7c6cb5b13c19edaa9ed2f79f79bd04393dc719a47c1fe06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-21T09:00:23.363 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 09:00:23 smithi106 podman[68082]: 2024-04-21 09:00:23.112948264 +0000 UTC m=+3.386902092 container cleanup 723e3ca6f5c51798d7c6cb5b13c19edaa9ed2f79f79bd04393dc719a47c1fe06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2) 2024-04-21T09:00:23.363 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 21 09:00:23 smithi106 bash[68082]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-k 2024-04-21T09:00:23.489 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.k.service' 2024-04-21T09:00:23.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:00:23.520 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-21T09:00:23.520 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-21T09:00:23.521 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.c 2024-04-21T09:00:23.829 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 09:00:23 smithi134 systemd[1]: Stopping Ceph mds.c for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T09:00:24.226 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 09:00:23 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c[62554]: 2024-04-21T09:00:23.828+0000 7fc8205dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:00:24.226 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 09:00:23 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c[62554]: 2024-04-21T09:00:23.828+0000 7fc8205dd700 -1 mds.c *** got signal Terminated *** 2024-04-21T09:00:26.976 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 09:00:26 smithi134 podman[69735]: 2024-04-21 09:00:26.71995526 +0000 UTC m=+2.985951226 container died 3b91812a31b32032dfd74397be3dc46cf38c82b70d6ff7d151531a662dd1f482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-21T09:00:27.476 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 09:00:27 smithi134 podman[69735]: 2024-04-21 09:00:27.161999197 +0000 UTC m=+3.427995173 container cleanup 3b91812a31b32032dfd74397be3dc46cf38c82b70d6ff7d151531a662dd1f482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-21T09:00:27.476 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 09:00:27 smithi134 bash[69735]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c 2024-04-21T09:00:27.476 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 21 09:00:27 smithi134 podman[69761]: 2024-04-21 09:00:27.338171639 +0000 UTC m=+0.620502069 container remove 3b91812a31b32032dfd74397be3dc46cf38c82b70d6ff7d151531a662dd1f482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0) 2024-04-21T09:00:27.590 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.c.service' 2024-04-21T09:00:27.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:00:27.625 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-21T09:00:27.626 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-21T09:00:27.626 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.f 2024-04-21T09:00:27.940 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 21 09:00:27 smithi134 systemd[1]: Stopping Ceph mds.f for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T09:00:28.226 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 21 09:00:27 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-f[62239]: 2024-04-21T09:00:27.940+0000 7f93fb2b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:00:28.226 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 21 09:00:27 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-f[62239]: 2024-04-21T09:00:27.940+0000 7f93fb2b0700 -1 mds.f *** got signal Terminated *** 2024-04-21T09:00:30.976 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 21 09:00:30 smithi134 podman[69869]: 2024-04-21 09:00:30.706139261 +0000 UTC m=+2.862054482 container died 7a919ece608ee4d638212cb4978075f4a29e94127c2bfe1e4b13efe98f0e2cd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-21T09:00:31.349 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 21 09:00:31 smithi134 podman[69869]: 2024-04-21 09:00:31.065616144 +0000 UTC m=+3.221531321 container cleanup 7a919ece608ee4d638212cb4978075f4a29e94127c2bfe1e4b13efe98f0e2cd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-21T09:00:31.349 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 21 09:00:31 smithi134 bash[69869]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-f 2024-04-21T09:00:31.496 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.f.service' 2024-04-21T09:00:31.528 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:00:31.528 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-21T09:00:31.528 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-21T09:00:31.528 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.i 2024-04-21T09:00:31.976 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 09:00:31 smithi134 systemd[1]: Stopping Ceph mds.i for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T09:00:31.976 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 09:00:31 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-i[62856]: 2024-04-21T09:00:31.842+0000 7fe60ba19700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:00:31.976 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 09:00:31 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-i[62856]: 2024-04-21T09:00:31.842+0000 7fe60ba19700 -1 mds.i *** got signal Terminated *** 2024-04-21T09:00:34.976 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 09:00:34 smithi134 podman[69991]: 2024-04-21 09:00:34.721974714 +0000 UTC m=+2.979135736 container died 51575d3f162f0b56a52f4faa1aedef3d2667a16cb425309c3319fcd92c801074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-21T09:00:35.328 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 09:00:35 smithi134 podman[69991]: 2024-04-21 09:00:35.069086453 +0000 UTC m=+3.326247486 container cleanup 51575d3f162f0b56a52f4faa1aedef3d2667a16cb425309c3319fcd92c801074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-21T09:00:35.328 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 21 09:00:35 smithi134 bash[69991]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-i 2024-04-21T09:00:35.488 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.i.service' 2024-04-21T09:00:35.521 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:00:35.522 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-21T09:00:35.522 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-21T09:00:35.522 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.l 2024-04-21T09:00:35.839 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 09:00:35 smithi134 systemd[1]: Stopping Ceph mds.l for 24d05dee-ffba-11ee-bc93-c7b262605968... 2024-04-21T09:00:36.226 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 09:00:35 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-l[61925]: 2024-04-21T09:00:35.838+0000 7f03afc21700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T09:00:36.226 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 09:00:35 smithi134 ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-l[61925]: 2024-04-21T09:00:35.838+0000 7f03afc21700 -1 mds.l *** got signal Terminated *** 2024-04-21T09:00:38.976 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 09:00:38 smithi134 podman[70112]: 2024-04-21 09:00:38.707524069 +0000 UTC m=+2.961999075 container died c8fd270304aacba838a25333ca58c975eafe6888f7f1fc3617340a063d7c8f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-l, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-21T09:00:39.239 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 09:00:39 smithi134 podman[70112]: 2024-04-21 09:00:39.056005633 +0000 UTC m=+3.310480628 container cleanup c8fd270304aacba838a25333ca58c975eafe6888f7f1fc3617340a063d7c8f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T09:00:39.239 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 09:00:39 smithi134 bash[70112]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-l 2024-04-21T09:00:39.474 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.l.service' 2024-04-21T09:00:39.494 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 09:00:39 smithi134 podman[70126]: 2024-04-21 09:00:39.240001251 +0000 UTC m=+0.525606859 container remove c8fd270304aacba838a25333ca58c975eafe6888f7f1fc3617340a063d7c8f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-24d05dee-ffba-11ee-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-21T09:00:39.494 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 09:00:39 smithi134 systemd[1]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.l.service: Deactivated successfully. 2024-04-21T09:00:39.495 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 09:00:39 smithi134 systemd[1]: Stopped Ceph mds.l for 24d05dee-ffba-11ee-bc93-c7b262605968. 2024-04-21T09:00:39.495 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 21 09:00:39 smithi134 systemd[1]: ceph-24d05dee-ffba-11ee-bc93-c7b262605968@mds.l.service: Consumed 32.596s CPU time. 2024-04-21T09:00:39.504 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T09:00:39.504 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-21T09:00:39.505 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T09:00:39.694 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T09:00:42.128 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T09:00:42.316 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T09:00:44.828 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T09:00:45.017 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T09:00:47.320 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T09:00:47.351 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T09:00:47.381 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T09:00:47.407 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-21T09:00:47.409 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526/remote/smithi082/crash 2024-04-21T09:00:47.410 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/crash -- . 2024-04-21T09:00:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T09:00:47.437 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now 2024-04-21T09:00:47.439 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526/remote/smithi106/crash 2024-04-21T09:00:47.440 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/crash -- . 2024-04-21T09:00:47.477 INFO:teuthology.orchestra.run.smithi106.stderr:tar: /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T09:00:47.477 INFO:teuthology.orchestra.run.smithi106.stderr:tar: Error is not recoverable: exiting now 2024-04-21T09:00:47.479 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526/remote/smithi134/crash 2024-04-21T09:00:47.480 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/crash -- . 2024-04-21T09:00:47.516 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T09:00:47.516 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2024-04-21T09:00:47.517 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-21T09:00:47.517 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-21T09:00:47.551 INFO:tasks.cephadm:Compressing logs... 2024-04-21T09:00:47.551 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T09:00:47.593 DEBUG:teuthology.orchestra.run.smithi106:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T09:00:47.595 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T09:00:47.615 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.log 2024-04-21T09:00:47.615 INFO:teuthology.orchestra.run.smithi082.stderr:: No such file or directory 2024-04-21T09:00:47.615 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T09:00:47.615 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.g.log 2024-04-21T09:00:47.616 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.log: /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.j.log 2024-04-21T09:00:47.617 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.3.log 2024-04-21T09:00:47.617 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.0.log 2024-04-21T09:00:47.618 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mgr.x.log 2024-04-21T09:00:47.619 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mon.a.log 2024-04-21T09:00:47.619 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.6.log 2024-04-21T09:00:47.620 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mon.a.log: 90.2% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T09:00:47.620 INFO:teuthology.orchestra.run.smithi082.stderr: 86.9% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T09:00:47.621 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.log 2024-04-21T09:00:47.621 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-04-21T09:00:47.621 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.2.log 2024-04-21T09:00:47.621 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.log: -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.f.log 2024-04-21T09:00:47.621 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T09:00:47.621 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.5.log 2024-04-21T09:00:47.622 INFO:teuthology.orchestra.run.smithi082.stderr: 94.6% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-21T09:00:47.624 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.c.log 2024-04-21T09:00:47.624 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mon.c.log 2024-04-21T09:00:47.624 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mgr.z.log 2024-04-21T09:00:47.624 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.i.log 2024-04-21T09:00:47.624 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mgr.z.log: 86.9% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T09:00:47.624 INFO:teuthology.orchestra.run.smithi134.stderr: 90.3% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T09:00:47.630 INFO:teuthology.orchestra.run.smithi106.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-21T09:00:47.634 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T09:00:47.634 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.a.log 2024-04-21T09:00:47.634 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T09:00:47.634 INFO:teuthology.orchestra.run.smithi082.stderr: 84.2% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T09:00:47.635 INFO:teuthology.orchestra.run.smithi134.stderr: 92.0% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-21T09:00:47.635 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T09:00:47.635 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.11.log 2024-04-21T09:00:47.635 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.8.log 2024-04-21T09:00:47.635 INFO:teuthology.orchestra.run.smithi134.stderr: 83.8% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T09:00:47.636 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.log 2024-04-21T09:00:47.636 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T09:00:47.636 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.7.log 2024-04-21T09:00:47.636 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.b.log 2024-04-21T09:00:47.639 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.4.log 2024-04-21T09:00:47.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.1.log 2024-04-21T09:00:47.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mgr.y.log 2024-04-21T09:00:47.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.h.log 2024-04-21T09:00:47.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.10.log 2024-04-21T09:00:47.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T09:00:47.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.e.log 2024-04-21T09:00:47.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T09:00:47.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mon.b.log 2024-04-21T09:00:47.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.k.log 2024-04-21T09:00:47.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T09:00:47.646 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.d.log 2024-04-21T09:00:47.646 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T09:00:47.657 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.l.log 2024-04-21T09:00:47.659 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T09:00:47.661 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.9.log 2024-04-21T09:00:47.665 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-21T09:00:47.674 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T09:00:47.679 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T09:00:47.687 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T09:00:47.694 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-21T09:00:47.700 INFO:teuthology.orchestra.run.smithi134.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T09:00:47.712 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T09:00:47.725 INFO:teuthology.orchestra.run.smithi082.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-21T09:00:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-21T09:00:47.850 INFO:teuthology.orchestra.run.smithi106.stderr: 91.0% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T09:00:47.989 INFO:teuthology.orchestra.run.smithi106.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T09:00:47.992 INFO:teuthology.orchestra.run.smithi082.stderr: 90.2% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-21T09:00:48.026 INFO:teuthology.orchestra.run.smithi106.stderr: 83.9% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T09:00:48.028 INFO:teuthology.orchestra.run.smithi106.stderr: 86.9% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T09:00:48.093 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-21T09:00:48.216 INFO:teuthology.orchestra.run.smithi106.stderr: 90.3% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T09:00:48.227 INFO:teuthology.orchestra.run.smithi134.stderr: 94.9% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-21T09:00:48.233 INFO:teuthology.orchestra.run.smithi134.stderr: 94.6% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-21T09:00:48.289 INFO:teuthology.orchestra.run.smithi082.stderr: 94.9% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-21T09:00:48.504 INFO:teuthology.orchestra.run.smithi082.stderr: 95.0% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-21T09:00:48.779 INFO:teuthology.orchestra.run.smithi082.stderr: 90.6% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-21T09:00:54.339 INFO:teuthology.orchestra.run.smithi106.stderr: 92.1% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-21T09:00:59.619 INFO:teuthology.orchestra.run.smithi134.stderr: 94.4% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-21T09:01:01.228 INFO:teuthology.orchestra.run.smithi134.stderr: 94.5% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-21T09:01:10.033 INFO:teuthology.orchestra.run.smithi082.stderr: 91.4% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-21T09:01:11.682 INFO:teuthology.orchestra.run.smithi082.stderr: 91.5% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-21T09:01:12.295 INFO:teuthology.orchestra.run.smithi134.stderr: 91.6% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-21T09:01:12.387 INFO:teuthology.orchestra.run.smithi134.stderr: 91.7% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-21T09:01:14.329 INFO:teuthology.orchestra.run.smithi082.stderr: 91.4% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-21T09:01:17.538 INFO:teuthology.orchestra.run.smithi134.stderr: 91.6% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-21T09:01:19.207 INFO:teuthology.orchestra.run.smithi082.stderr: 91.4% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-21T09:01:19.209 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-21T09:01:19.210 INFO:teuthology.orchestra.run.smithi082.stderr:real 0m31.603s 2024-04-21T09:01:19.210 INFO:teuthology.orchestra.run.smithi082.stderr:user 1m44.233s 2024-04-21T09:01:19.210 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m3.342s 2024-04-21T09:01:20.125 INFO:teuthology.orchestra.run.smithi134.stderr: 91.4% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-21T09:01:20.127 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-04-21T09:01:20.127 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m32.518s 2024-04-21T09:01:20.127 INFO:teuthology.orchestra.run.smithi134.stderr:user 2m14.197s 2024-04-21T09:01:20.128 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m4.118s 2024-04-21T09:02:24.441 INFO:teuthology.orchestra.run.smithi106.stderr: 91.8% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-21T09:02:58.334 INFO:teuthology.orchestra.run.smithi106.stderr: 94.8% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-21T09:03:04.446 INFO:teuthology.orchestra.run.smithi106.stderr: 91.6% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-21T09:03:08.517 INFO:teuthology.orchestra.run.smithi106.stderr: 91.5% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-21T09:03:17.145 INFO:teuthology.orchestra.run.smithi106.stderr: 94.4% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-21T09:03:17.907 INFO:teuthology.orchestra.run.smithi106.stderr: 94.5% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-21T09:04:18.368 INFO:teuthology.orchestra.run.smithi106.stderr: 91.7% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-21T09:06:51.074 INFO:teuthology.orchestra.run.smithi106.stderr: 93.0% -- replaced with /var/log/ceph/24d05dee-ffba-11ee-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-21T09:06:51.076 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-21T09:06:51.076 INFO:teuthology.orchestra.run.smithi106.stderr:real 6m3.465s 2024-04-21T09:06:51.076 INFO:teuthology.orchestra.run.smithi106.stderr:user 4m52.203s 2024-04-21T09:06:51.076 INFO:teuthology.orchestra.run.smithi106.stderr:sys 0m10.785s 2024-04-21T09:06:51.077 INFO:tasks.cephadm:Archiving logs... 2024-04-21T09:06:51.077 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526/remote/smithi082/log 2024-04-21T09:06:51.078 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T09:07:09.672 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526/remote/smithi106/log 2024-04-21T09:07:09.673 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T09:07:57.876 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526/remote/smithi134/log 2024-04-21T09:07:57.877 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T09:08:17.888 INFO:tasks.cephadm:Removing cluster... 2024-04-21T09:08:17.889 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 --force 2024-04-21T09:08:18.079 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T09:08:18.502 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 --force 2024-04-21T09:08:18.697 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T09:08:19.289 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 24d05dee-ffba-11ee-bc93-c7b262605968 --force 2024-04-21T09:08:19.480 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 24d05dee-ffba-11ee-bc93-c7b262605968 2024-04-21T09:08:20.037 INFO:tasks.cephadm:Removing cephadm ... 2024-04-21T09:08:20.038 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T09:08:20.056 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T09:08:20.073 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T09:08:20.088 INFO:tasks.cephadm:Teardown complete 2024-04-21T09:08:20.089 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-21T09:08:20.100 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-21T09:08:20.101 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T09:08:20.103 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T09:08:20.115 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T09:08:20.221 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-21T09:08:20.235 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-21T09:08:20.235 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T09:08:20.238 DEBUG:teuthology.orchestra.run.smithi106:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T09:08:20.240 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T09:08:20.252 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-04-21T09:08:20.252 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-21T09:08:20.254 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-04-21T09:08:20.257 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T09:08:20.257 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-04-21T09:08:20.257 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 -126us[ -93us] +/- 89ms 2024-04-21T09:08:20.257 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 -397us[ -363us] +/- 54ms 2024-04-21T09:08:20.257 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +637us[ +637us] +/- 63ms 2024-04-21T09:08:20.257 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T09:08:20.259 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T09:08:20.259 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-04-21T09:08:20.259 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 -133us[ -105us] +/- 89ms 2024-04-21T09:08:20.259 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -410us[ -382us] +/- 54ms 2024-04-21T09:08:20.259 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +628us[ +656us] +/- 62ms 2024-04-21T09:08:20.259 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T09:08:20.272 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T09:08:20.272 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-21T09:08:20.272 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 37 -146us[ -164us] +/- 89ms 2024-04-21T09:08:20.272 INFO:teuthology.orchestra.run.smithi106.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -437us[ -455us] +/- 54ms 2024-04-21T09:08:20.272 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +616us[ +598us] +/- 62ms 2024-04-21T09:08:20.272 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T09:08:20.273 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-21T09:08:20.284 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-21T09:08:20.284 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-21T09:08:20.295 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T09:08:20.382 DEBUG:teuthology.orchestra.run.smithi106:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T09:08:20.473 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T09:08:20.537 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T09:08:20.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:08:20.566 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T09:08:20.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:08:20.594 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T09:08:20.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:08:20.620 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-21T09:08:20.631 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-21T09:08:20.642 INFO:teuthology.task.internal:Duration was 2337.721215 seconds 2024-04-21T09:08:20.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-21T09:08:20.653 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-21T09:08:20.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T09:08:20.656 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T09:08:20.658 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T09:08:20.690 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T09:08:20.695 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T09:08:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T09:08:21.177 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-21T09:08:21.177 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-04-21T09:08:21.177 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T09:08:21.198 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi106.front.sepia.ceph.com 2024-04-21T09:08:21.198 DEBUG:teuthology.orchestra.run.smithi106:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T09:08:21.218 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-04-21T09:08:21.218 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T09:08:21.243 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-21T09:08:21.243 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T09:08:21.246 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T09:08:21.262 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T09:08:21.310 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-21T09:08:21.310 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T09:08:21.313 DEBUG:teuthology.orchestra.run.smithi106:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T09:08:21.328 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T09:08:21.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-21T09:08:21.911 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-21T09:08:21.911 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T09:08:21.936 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T09:08:21.961 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T09:08:21.986 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-21T09:08:22.126 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T09:08:22.129 DEBUG:teuthology.orchestra.run.smithi106:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T09:08:22.131 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T09:08:22.149 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-04-21T09:08:22.154 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-04-21T09:08:22.160 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = core 2024-04-21T09:08:22.185 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T09:08:22.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:08:22.216 DEBUG:teuthology.orchestra.run.smithi106:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T09:08:22.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:08:22.232 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T09:08:22.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T09:08:22.246 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-21T09:08:22.258 INFO:teuthology.task.internal:Transferring archived files... 2024-04-21T09:08:22.259 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526/remote/smithi082 2024-04-21T09:08:22.259 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T09:08:22.300 DEBUG:teuthology.misc:Transferring archived files from smithi106:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526/remote/smithi106 2024-04-21T09:08:22.300 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T09:08:22.335 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526/remote/smithi134 2024-04-21T09:08:22.335 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T09:08:22.370 INFO:teuthology.task.internal:Removing archive directory... 2024-04-21T09:08:22.370 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T09:08:22.372 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T09:08:22.375 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T09:08:22.420 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-21T09:08:22.434 INFO:teuthology.task.internal:Not uploading archives. 2024-04-21T09:08:22.434 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-21T09:08:22.446 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-21T09:08:22.446 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T09:08:22.449 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T09:08:22.451 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T09:08:22.461 INFO:teuthology.orchestra.run.smithi082.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 09:08 /home/ubuntu/cephtest 2024-04-21T09:08:22.463 INFO:teuthology.orchestra.run.smithi106.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 09:08 /home/ubuntu/cephtest 2024-04-21T09:08:22.474 INFO:teuthology.orchestra.run.smithi134.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 09:08 /home/ubuntu/cephtest 2024-04-21T09:08:22.475 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-21T09:08:22.488 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-21T09:08:22.582 INFO:teuthology.nuke:Checking targets against current locks 2024-04-21T09:08:22.607 DEBUG:teuthology.nuke:shortname: smithi082 2024-04-21T09:08:22.607 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T09:08:22.629 DEBUG:teuthology.nuke:shortname: smithi106 2024-04-21T09:08:22.629 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T09:08:22.633 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:23:30.286312', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCpZN/cVekf/8bChBUe7YM06GyrOTAkrKgTpxP870P/PGnZY5GZVM43heKzMP6yZroQpndv8oCerPEUCSuDaMqU='} 2024-04-21T09:08:22.652 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:23:30.288418', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKfndWoCu9JAh2dDQweXrGZUyUspjAOLuZmnJ2dw6UQXmkAokkcP+U9f4I6USO/+i9x6Fy9D1tn3JbnDwZWJ4a4='} 2024-04-21T09:08:22.656 DEBUG:teuthology.nuke:shortname: smithi134 2024-04-21T09:08:22.657 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T09:08:22.676 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666526', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:23:30.287465', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLlnjZa4dD40YjtP5Xwt4EoGQpoz546iQ37BU1BCF3FLyZ22Wn56euhAG87DiXfrt5ZpJrmZgcosN6A8ZRUw1hg='} 2024-04-21T09:08:22.686 INFO:teuthology.orchestra.console.smithi082:Power off 2024-04-21T09:08:22.686 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T09:08:22.712 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2024-04-21T09:08:22.713 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:08:22.728 INFO:teuthology.orchestra.console.smithi106:Power off 2024-04-21T09:08:22.728 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T09:08:22.742 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-04-21T09:08:22.751 DEBUG:teuthology.orchestra.console.smithi106:power off output: Chassis Power Control: Down/Off 2024-04-21T09:08:22.752 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:08:22.768 INFO:teuthology.orchestra.console.smithi134:Power off 2024-04-21T09:08:22.768 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T09:08:22.777 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-04-21T09:08:22.791 DEBUG:teuthology.orchestra.console.smithi134:power off output: Chassis Power Control: Down/Off 2024-04-21T09:08:22.791 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:08:22.813 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-04-21T09:08:26.743 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:08:26.778 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:08:26.814 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:08:26.854 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-04-21T09:08:26.888 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-04-21T09:08:26.926 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-04-21T09:08:30.857 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:08:30.889 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:08:30.927 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:08:30.967 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-04-21T09:08:30.997 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is off 2024-04-21T09:08:31.035 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is off 2024-04-21T09:08:31.098 INFO:teuthology.orchestra.console.smithi106:Power off completed 2024-04-21T09:08:31.136 INFO:teuthology.orchestra.console.smithi134:Power off completed 2024-04-21T09:08:34.969 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T09:08:35.080 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-04-21T09:08:35.181 INFO:teuthology.orchestra.console.smithi082:Power off completed 2024-04-21T09:08:35.285 INFO:teuthology.run:Summary data: description: fs/workload/{0-ubuntu_22.04 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 2337.7212154865265 failure_reason: 'error during scrub thrashing: Error releasing root: -124 (Unknown)' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-04-21T09:08:35.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-21T09:08:35.364 INFO:teuthology.run:FAIL